更新 List<T> 中的现有字段并重新保存到同一个 protobuf.net 文件中 [英] Update existing field in List&lt;T&gt; and re-save into same protobuf.net file

问题描述

获得具有 100 多个字段的 POCO(使用 DataMember(Order=X) 属性将它们标记为 ProtoBuf.Net 序列化).

Got POCO with 100+ fields (using DataMember(Order=X) attribute to mark them for ProtoBuf.Net serialisation).

我们需要对文件进行一些后期处理，其中一个字段的值需要计算(我们只能在阅读一次后才能进行)更新并再次保存.Atm 我们正在重新保存到新文件中，这显然有一些 IO/文件开销.

We need to do some post processing of the file where one field's value needs to calced (which we can only do once we have read through it once) be updated and saved down again. Atm we are resaving into new file and this obviously has some IO/file overhead.

有什么方法可以更新现有文件，其中只需要更新和保存字段X"的值.希望这应该比从头生成新文件更快.

Is there any way I can update existing file where only Field "X"'s value needs to be updated and saved down. Hopefully this should be quicker than generating new file from scratch.

示例 POCO 数据结构(每个文件将有大约 300 万个这些项目)

Sample POCO Data Structure (and each file will have ~3mln of those items)

[DataContract]
public class DataItem
{
    [DataMember(Order=1)]
    public string ProfitCentre {get; set;}

    [DataMember(Order=2)]
    public string BusinessFunction {get; set;}

    //this is the value that needs to be updated once file is written
    [DataMember(Order=3)]
    public double AdjustedAmount {get; set; } 
}

推荐答案

这在一定程度上取决于模型的结构.protobuf 格式允许追加更新，这意味着:可以在文件的末尾写入一个值，然后将其应用于代码模型.但是，有一个限制:它不适用于 repeated 元素(包括 map<...> 元素).它可以应用于根对象或任意数量的optional/required 子对象，但不能应用于repeated.另一个限制是您需要一个 API 来写入该值并且仅该值.有多种方法可以实现这一点，例如，如果我们有:

That depends a bit on the structure of the model. The protobuf format allows for append-as-update, meaning: a value can be written at the end of the file that is then applied to the code model. There is, however, a limitation: it doesn't work with repeated elements (which includes map<...> elements). It can apply to the root object or any number of optional / required sub-objects, but not repeated. The other limitation is that you need an API that writes that value and only that value. There are various ways of achieving this, for example if we have:

Foo
 - Bar (field 3 of Foo)
   - Blap (field 7 of Bar)
     - X (string, field 4 of Blap)

然后这样做的许多方法之一是使用第二个模型仅用于此目的来更新 X 字段，如:

then one of many ways of doing that would be to have a second model just for this purpose that updates the X field, as:

[ProtoContract]
class FooUpdate {
    [ProtoMember(3)]
    public BarUpdate Bar {get;set;} = new BarUpdate();
}
[ProtoContract]
class BarUpdate {
    [ProtoMember(7)]
    public BlapUpdate Blap {get;set;} = new BlapUpdate();
}
[ProtoContract]
class BlapUpdate {
    [ProtoMember(4)]
    public string X {get;set;}
}

现在；假设我们之前写了一个 regular Foo 到一个文件，我们可以构造:

now; assuming we previously wrote a regular Foo to a file, we can construct:

var obj = new FooUpdate { Bar = { Blap = { X = "abcdef" } } };
Serializer.Serialize(existingFile, obj);

附加仅此数据(我在这里假设 existingFile 位于末尾，用于附加目的).

which appends just this data (I'm assuming here that existingFile is positioned at the end, for append purposes).

请注意，使用条件序列化"也可以实现类似的功能，但是当我们可以......没有任何东西可以关闭时，关闭一切似乎需要做很多工作.

Note that something similar can also be achieved using "conditional serialization", but it seems a lot of work to turn everything off when we can just... not have anything to turn off.

如果您要写入的数据位于 repeated/map<...> 字段中，但是:您只需要重写整个文件.在这种情况下， append 会导致额外的列表元素，或者地图元素中数据的丢失.

If the data you want to write is in a repeated / map<...> field, however: you'll just have to rewrite the entire file. In that context, append would result in additional list elements, or the loss of data from map elements.

这篇关于更新 List<T> 中的现有字段并重新保存到同一个 protobuf.net 文件中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！