使用长字符串的protobuf中的奇怪行为 [英] Strange behaviour in protobuf with long strings

问题描述

我尝试将数据从客户端发送到服务器。两个应用程序都是用java编写的。但是他们使用通过SWIG封装器在c ++中实现的tls层。 tls层期望来自客户端的字符串，将其传输到服务器端并通知java服务器应用程序（并传递字符串）。但是，此字符串应包含序列化数据。不知怎的，我很难使用protobuf来序列化数据。我想使用一个名为 ToDoListMessage的Java protobuf类。 protobuf看起来像这样：

  message ToDoListMessage {
 optional string user = 1; 
可选字符串token = 2;但是生成的java类无法解析之前序列化的数据：       / p> 
 
 
 
 
 com.google.protobuf.InvalidProtocolBufferException：协议消息
标签的电线类型无效。
 
 
我目前没有将数据发送到服务器。只需测试客户端的序列化和解析部分：
  ToDoListMessageProto msg = ToDoListMessageProto.newBuilder（）。setUser（test）。 setToken（38632735722755）。build（）; 
 
 byte b [] = msg.toByteArray（）; 
 String sMsg = Arrays.toString（b）; 
 System.out.println（send message =+ sMsg）; 
 ToDoListMessageProto outputmessage; 
 outputmessage = ToDoListMessageProto.parseFrom（sMsg.getBytes（））; 
  
邮件内容如下：
  [10，4，116，101，115，116，18，14，51，56，54，51，50，55，51，53，55，50，50，55，53 ，53] 
  
 我尝试过的：
 
 
 
 1）我发现的所有解决方案都说这个问题可以通过使用 CodedOutputStream 来解决。但tls层期待一个字符串，而不是一个流。但我也试图如下：
  ByteArrayOutputStream bos = new ByteArrayOutputStream 
 CodedOutputStream cos = CodedOutputStream.newInstance（bos）; 
 msg.writeTo（cos）; 
 cos.flush（）; 
 byte b [] = msg.toByteArray（）; 
 String sMsg = Arrays.toString（b）; 
  
但我得到了与上面的解析相同的错误：
  CodedInputStream cis = CodedInputStream.newInstance（sMsg.getBytes（））; 
 ToDoListMessageProto message = ToDoListMessageProto.parseFrom（cis）; 
  
 2）我也尝试使用UTF8编码的字符串而不是类似数组： p> 
 
 
  String sMsg = new String（b）; 
  
在这种情况下，应用程序的行为更奇怪。对于短的令牌（例如小于129位），解析工作，但对于长令牌失败：
 
 
 com.google.protobuf。 InvalidProtocolBufferException：解析
协议消息时，输入意外地在
字段中间结束。这可能意味着输入被截断或
嵌入的消息错误地报告了自己的长度。
 
 
我真的可以不知道为什么。目前该令牌只包含数字。
 
 
 
有没有人知道一个解决方案如何从protobuf获得一个可以正确解析的序列化字符串？
 
 
 
 再次：在此测试中没有涉及tls传输。   
 
 
   p>因为我直接从Protobuf消息获取字节数组，所以不可能传递编码。我发现还有一个 toByteString 方法的消息，但使用 toStringUtf8 在这个ByteString似乎不工作：
  String sMsg = msg.toByteString（）。toStringUtf8（）; 
 System.out.println（send message =+ sMsg）; 
 ToDoListMessageProto outputmessage; 
 outputmessage = ToDoListMessageProto.parseFrom（sMsg.getBytes（））; 
  
我得到相同的错误消息（如果我使用一个长或短的令牌， ）
解决方案
我无法解决原始问题。但是我最终最终做的是生成Java Protobuf类，并使用它们将数据转换为 byte [] 。之后，我将 byte [] 传递给C ++。在服务器端，我通过JNI将 byte [] 从C ++ TLS层发送到Java服务器应用程序。 Java服务器应用程序本身再次使用Java Protobuf类来解析 byte [] 到一个对象。在我的Java源代码中涉及到 String 。这工作，但我仍然好奇，如果有办法解决原始问题。
 
I'm trying to send data from a client to a server. both application's are written in java. But they use a tls layer implemented in c++ over SWIG Wrappers. The tls layer expects a string from the client, transmit it to the server-side and notifies the java server application (and passes the string). However this string should contain serialized data. Somehow I struggle to use protobuf for serializing the data. I would like to use a java protobuf class named ToDoListMessage. The protobuf looks like this:
message ToDoListMessage{  
    optional string user = 1;  
    optional string token = 2;
}
But the generated java class fails to parse the data which it serialized before:

  
com.google.protobuf.InvalidProtocolBufferException: Protocol message
  tag had invalid wire type.
I'm currently not sending the data to the server. Just testing the serialize and parse part on clientside:
ToDoListMessageProto msg = ToDoListMessageProto.newBuilder().setUser("test").setToken("38632735722755").build();        

byte b [] = msg.toByteArray();  
String sMsg = Arrays.toString(b);   
System.out.println("send message = " + sMsg);
ToDoListMessageProto outputmessage;         
outputmessage = ToDoListMessageProto.parseFrom(sMsg.getBytes());
The message looks like:
[10, 4, 116, 101, 115, 116, 18, 14, 51, 56, 54, 51, 50, 55, 51, 53, 55, 50, 50, 55, 53, 53]
What I tried:

1) All solutions I found so far say this problem might be solved by using a CodedOutputStream. But the tls layer is expecting a string, not a stream. However I also tried to following:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
CodedOutputStream cos = CodedOutputStream.newInstance(bos);
msg.writeTo(cos);   
cos.flush();
byte b [] = msg.toByteArray();              
String sMsg = Arrays.toString(b);   
But I get the same error as above for this parsing:
CodedInputStream cis = CodedInputStream.newInstance(sMsg.getBytes());
ToDoListMessageProto message = ToDoListMessageProto.parseFrom(cis);
2) I also tried to use a UTF8 encoded string instead of the array-like one:
String sMsg = new String(b);
In this case the app behaves even more strange. For short "tokens" (e.g. less than 129 bit) the parsing works, but fails for long tokens:

  
com.google.protobuf.InvalidProtocolBufferException: While parsing a
  protocol message, the input ended unexpectedly in the middle of a
  field.  This could mean either than the input has been truncated or
  that an embedded message misreported its own length.
I really can't tell why. Currently the token does only contain numbers.

Does anyone know a solution how I can get a serialized string from protobuf which can be parsed correctly?

Again: there is no tls transmission involved in this testing. Currently everything is done on the client side.

Update:

Because I fetch the byte array directly from the Protobuf Message it is not possible to pass an encoding. I found there is also a toByteString Method for the message but using toStringUtf8 on this ByteString doesn't seem to work neither:
String sMsg = msg.toByteString().toStringUtf8();
System.out.println("send message = " + sMsg);
ToDoListMessageProto outputmessage;         
outputmessage = ToDoListMessageProto.parseFrom(sMsg.getBytes());
I get the same error messages (which differ if I use a long or a short token, see above)
解决方案
I was not able to resolve the original problem. But what I finally ended up doing was to generate Java Protobuf classes and used them for converting the data to byte[]. Afterwards I passed the byte[] to C++. On server side I send the byte[] from the C++ TLS layer to the Java server application through JNI. The Java server application itself uses the Java Protobuf classes again to parse the byte[] to an object. No String is involved in my Java source code. This works, but I'm still curious, if there is a way to solve the original issue.

                        
这篇关于使用长字符串的protobuf中的奇怪行为的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！