协议缓冲区:如何将多个消息序列化和反序列化到一个文件中(C++)? [英] protocol buffers: how to serialize and deserialize multiple messages into a file (c++)?

问题描述

我是 Protocol Buffers 和 C++ 的新手，但我的任务要求我使用这两者.我想多次将数据(消息)结构写入单个文件并能够读取数据.我可以读写一条消息，但事实证明多条消息更难.我已经寻找了几个小时的答案，但我似乎无法将数据作为结构读取.任何示例代码或指针都会非常有帮助.

I am new to Protocol Buffers and c++ but my task requires me to use the two. I want to write a structure of data ( message) into a single file multiple times and be able to read the data. i can read and write a single message but multiple messages is proving harder. I have looked for answers for hours but i can't seem to be able to read the data as a structure. Any example code or pointers will be very helpful.

这是我的结构格式:

    typedef struct Entry
       {
          char  name[ NAME_MAX];
          int   id;
          int   age;
          char  DoB[32]; 
       } entry;

这是我一直用来写入文件的内容:

This is what i've been using to write into a file:

    Person_File file;
    fstream output("file", ios::out | ios::trunc | ios::binary);
    file.SerializeToOstream(&output);

我尝试将文件编辑选项更改为追加而不是截断，但这并不能让我以我想要的格式读取数据.

I have tried to change the file editing options to append instead of truncate but that does not let me read the data in the format that I want.

这是我用来阅读的:

    fstream input("file", ios::in | ios::binary);
    file.ParseFromIstream(&input);

这些是我的 .proto 文件的内容:

These are the contents of my .proto file:

    message Person {


message File {
    required string pname =1;
    required int32 id =2;
    required int32 age =3;
    required string DoB =4;
}

repeated File file =1;

}

从我所做的所有搜索来看，CodedInputStream/CodedOutputStream 似乎是我最好的选择，但我找不到任何详细的示例或解释来帮助我理解.我知道协议缓冲区不是自定界的，这可能是我无法以原始格式读回消息的原因.任何帮助，将不胜感激.谢谢

From all the searching i've done it seems like CodedInputStream/CodedOutputStream are my best options but I haven't been able to find any detailed examples or explanation to help me understand. I understand that Protocol Buffers are not self delimiting and this is probably the reason why I can't read my messages back in the original format. Any help would be appreciated. Thanks

我尝试根据收到的消息使用 CodedOutputStream，但它似乎没有将任何内容写入文件.

I have tried to use CodedOutputStream based on the messages I've recieved but it doesn't seem to write anything into the file.

    int fd = open("file.txt",O_WRONLY | O_APPEND | O_CREAT);

    FileOutputStream* file_ostream_ = new FileOutputStream(fd);
    CodedOutputStream* ostream_ = new CodedOutputStream(file_ostream_);


ostream_->WriteLittleEndian32(file.ByteSize());
    file.SerializeToCodedStream(ostream_);

使用此代码后，文件显示为空白.我哪里出错了?

After using this code the file comes out blank. Where am I going wrong ?

推荐答案

我尝试根据收到的消息使用 CodedOutputStream，但它似乎没有向文件中写入任何内容.

I have tried to use CodedOutputStream based on the messages I've recieved but it doesn't seem to write anything into the file.

我怀疑您的问题是您没有删除 CodedOutputStream 或 FileOutputStream.这些对象缓冲输出并在它们的析构函数中刷新缓冲区，所以如果你从不销毁它们，它们就不会写入最后一个缓冲区，在这种情况下，它是唯一的缓冲区.

I suspect your problem here is that you aren't deleting the CodedOutputStream nor the FileOutputStream. These objects buffer output and flush the buffer in their destructors, so if you never destroy them, they won't write the last buffer, which in this case is the only buffer.

我建议在堆栈上分配这些对象(作为局部变量).那么你不可能忘记摧毁它们.

I recommend allocating these objects on the stack (as local variables). Then you can't possibly forget to destroy them.

除此之外，这里的代码使用 CodedInputStream 和 CodedOutputStream 来读取/写入分隔消息.每条消息都以一个表示大小的 varint 为前缀，它与 Java Protobuf 库的 writeDelimitedTo()/parseDelimitedFrom() 格式相同(该功能从未成为C++ 库).

With that out of the way, here's code that uses CodedInputStream and CodedOutputStream to read/write delimited messages. Each message is prefixed with a varint indicating the size, which is the same format as the Java Protobuf library's writeDelimitedTo()/parseDelimitedFrom() (a feature that never made it into the C++ library).

bool writeDelimitedTo(
    const google::protobuf::MessageLite& message,
    google::protobuf::io::ZeroCopyOutputStream* rawOutput) {
  google::protobuf::io::CodedOutputStream output(rawOutput);

  // Write the size.
  const int size = message.ByteSize();
  output.WriteVarint32(size);

  uint8_t* buffer = output.GetDirectBufferForNBytesAndAdvance(size);
  if (buffer != NULL) {
    // Optimization:  The message fits in one buffer, so use the faster
    // direct-to-array serialization path.
    message.SerializeWithCachedSizesToArray(buffer);
  } else {
    // Slightly-slower path when the message is multiple buffers.
    message.SerializeWithCachedSizes(&output);
    if (output.HadError()) return false;
  }

  return true;
}

bool readDelimitedFrom(
    google::protobuf::io::ZeroCopyInputStream* rawInput,
    google::protobuf::MessageLite* message) {
  google::protobuf::io::CodedInputStream input(rawInput);

  // Read the size.
  uint32_t size;
  if (!input.ReadVarint32(&size)) return false;

  // Tell the stream not to read beyond that size.
  auto limit = input.PushLimit(size);

  // Parse the message.
  if (!message->MergePartialFromCodedStream(&input)) return false;
  if (!input.ConsumedEntireMessage()) return false;

  // Release the limit.
  input.PopLimit(limit);

  return true;
}

这篇关于协议缓冲区:如何将多个消息序列化和反序列化到一个文件中(C++)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！