Protobuf子消息的延迟解码 [英] Protobuf lazy decoding of sub message

问题描述

我在项目中使用proto 3(java).我有一些嵌入较小消息的巨大protobuf.有没有一种方法可以使我只想查看的少数嵌套子消息实现部分解码.我当前遇到的问题是我需要将这个基于大型原型的记录数据与另一个记录连接起来，但是我的联接基于很小的子消息，所以我不想解码整个大型原型并且只能解码嵌套的消息(字符串ID)加入，然后仅解码整个protobuf以获取加入的数据.

I am using proto 3 (java) in my projects . I have some huge protobufs embedded with smaller messages . Is there a way I can acheive partial decoding of only few nested sub messages that I want to look at. The current issue I am having is I need to join this huge proto based record data with another records ,but my join are based on very small sub messages ,so I don't want to decode the entire huge protobuf and be able to only decode the nested message (string id) to join and then only decode the entire protobuf for the joined data.

我尝试使用 [lazy = true] 标记方法，但是在生成的代码中看不到任何区别，我还尝试了在有和没有懒键工作的情况下对反序列化时间进行基准测试，似乎丝毫没有影响.默认情况下，是否为所有字段都启用此功能?还是有可能吗?我确实看到在protobuf-github中没有几个类LazyFields.java和测试用例，因此我假设此功能已实现.

I tried using the [lazy=true] tagging method , but I don't see any difference in generated code , also I tried benchmarking the deserialization time with and without the lazy key work and it didn't seem to affect at all . Is this feature by default on for all fields? Or is this even possible? I do see there are few classes LazyFields.java and test cases in the protobuf-github so I assume this feature has been implemented.

推荐答案

对于以后碰巧看到此对话且难以理解的人，这是Marc所说的:

For those that happen to look at this conversation later and finding it hard to understand, here's what Marc's talking about:

如果您的对象是类似的东西

If your object is something like

message MyBigMessage{
  string id = 1;
  int sourceType = 2 ;
  And many other fields here, that would be expensive to parse .......

}

您将获得一个必须解析的字节块.但是，您只想解析来自特定来源的消息，并且可能匹配特定的id范围.您可以先将这些字节与另一条消息一起解析为:

And you get a block of bytes that you have to parse. But you want to only parse messages from a certain source and maybe match a certain id range. You could first parse those bytes with another message as:

message MyFilterMessage{
  string id = 1; //has to be 1 to match
  int sourceType = 2 ; //has to be 1 to match
  And NOTHING ELSE here.......
}

然后，您可以查看sourceType和id.如果它们与您要过滤的内容匹配，则可以再次解析字节，但是这次使用MyBigMessage解析整个内容.

And then, you could look at sourceType and id. If they match whatever you are filtering for, then, you could go and parse the bytes again, but this time, using MyBigMessage to parse the whole thing.

要知道的另一件事:仅供参考:根据这篇文章，自2017年起，Java(消息集除外)已禁用了惰性解析: https://github.com/protocolbuffers/protobuf/issues/3601#issuecomment-341516826 我不知道目前的状态.懒得试图找出答案！:-)

One other thing to know: FYI: As of 2017, lazy parsing was disabled in Java (except MessageSet) according to this post: https://github.com/protocolbuffers/protobuf/issues/3601#issuecomment-341516826 I dont know the current status. Too lazy to try to find out ! :-)

这篇关于Protobuf子消息的延迟解码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！