Java 中的协议缓冲区分隔 I/O 函数是否有 C++ 等效项? [英] Are there C++ equivalents for the Protocol Buffers delimited I/O functions in Java?

问题描述

我正在尝试使用 C++ 和 Java 从文件中读取/写入多个协议缓冲区消息.Google 建议在消息之前编写长度前缀，但默认情况下无法这样做(我可以看到).

I'm trying to read / write multiple Protocol Buffers messages from files, in both C++ and Java. Google suggests writing length prefixes before the messages, but there's no way to do that by default (that I could see).

但是，2.1.0 版的 Java API 收到了一组Delimited"显然可以完成这项工作的 I/O 函数:

However, the Java API in version 2.1.0 received a set of "Delimited" I/O functions which apparently do that job:

parseDelimitedFrom
mergeDelimitedFrom
writeDelimitedTo

是否有 C++ 等价物?如果没有，Java API 附加的大小前缀的有线格式是什么，以便我可以在 C++ 中解析这些消息?

Are there C++ equivalents? And if not, what's the wire format for the size prefixes the Java API attaches, so I can parse those messages in C++?

这些现在存在于 google/protobuf/util/delimited_message_util.h 自 v3.3.0.

These now exist in google/protobuf/util/delimited_message_util.h as of v3.3.0.

推荐答案

我在这里参加聚会有点晚了，但以下实现包括其他答案中缺少的一些优化，并且在输入 64MB 后不会失败(尽管它仍然强制执行 64MB 限制在每个单独的消息上，而不是在整个流上).

I'm a bit late to the party here, but the below implementations include some optimizations missing from the other answers and will not fail after 64MB of input (though it still enforces the 64MB limit on each individual message, just not on the whole stream).

(我是 C++ 和 Java protobuf 库的作者，但我不再为 Google 工作.抱歉，这段代码从未进入官方库.这就是它的样子.)

(I am the author of the C++ and Java protobuf libraries, but I no longer work for Google. Sorry that this code never made it into the official lib. This is what it would look like if it had.)

bool writeDelimitedTo(
    const google::protobuf::MessageLite& message,
    google::protobuf::io::ZeroCopyOutputStream* rawOutput) {
  // We create a new coded stream for each message.  Don't worry, this is fast.
  google::protobuf::io::CodedOutputStream output(rawOutput);

  // Write the size.
  const int size = message.ByteSize();
  output.WriteVarint32(size);

  uint8_t* buffer = output.GetDirectBufferForNBytesAndAdvance(size);
  if (buffer != NULL) {
    // Optimization:  The message fits in one buffer, so use the faster
    // direct-to-array serialization path.
    message.SerializeWithCachedSizesToArray(buffer);
  } else {
    // Slightly-slower path when the message is multiple buffers.
    message.SerializeWithCachedSizes(&output);
    if (output.HadError()) return false;
  }

  return true;
}

bool readDelimitedFrom(
    google::protobuf::io::ZeroCopyInputStream* rawInput,
    google::protobuf::MessageLite* message) {
  // We create a new coded stream for each message.  Don't worry, this is fast,
  // and it makes sure the 64MB total size limit is imposed per-message rather
  // than on the whole stream.  (See the CodedInputStream interface for more
  // info on this limit.)
  google::protobuf::io::CodedInputStream input(rawInput);

  // Read the size.
  uint32_t size;
  if (!input.ReadVarint32(&size)) return false;

  // Tell the stream not to read beyond that size.
  google::protobuf::io::CodedInputStream::Limit limit =
      input.PushLimit(size);

  // Parse the message.
  if (!message->MergeFromCodedStream(&input)) return false;
  if (!input.ConsumedEntireMessage()) return false;

  // Release the limit.
  input.PopLimit(limit);

  return true;
}

这篇关于Java 中的协议缓冲区分隔 I/O 函数是否有 C++ 等效项?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！