通过protobuf替换binaryformatter [英] Replace binaryformatter by protobuf

问题描述

我需要替换一个库来存储文件中的数据（serialize / deserialize）
这个库目前使用BinaryFormatter，但是对于大型列表来说它很慢。
stackoverflow上的很多帖子表明protobuf是真正的性能，所以我试图使用它。

I need to replace a library that store data in files (serialize/deserialize) This library do it currently using BinaryFormatter, but it's slow for large lists. Many posts here on stackoverflow show that protobuf is really performant, so I'm trying to use it.

为了使替换不重写很多我的要求是：

In order to make a replacement without rewriting a lot of code, my requirements are:

• 必须序列化通用数据，因为我必须连接 Store< T> ;（T data）（大部分时间T被标记为DataContract和/或Serializable属性）


• 我不能修改所有不同的objets类添加protobuf属性，所以我需要一个不使用这些属性的泛型方法


• 我不需要向后兼容性：如果T对象模式更改（即新属性），所有生成的文件已过时且将被删除/重新创建

• Must serialize generic data, as I've to interface with Store<T>(T data) (most of time T is marked with DataContract and/or Serializable attribute)
• I can't modify all different objets classes to add protobuf attributes, so I need a generic method that doesn't use these attributes
• I don't need backward compatibility: if the T object schema change (i.e. new property), all generated files are obsolete and will be deleted / recreated

我的第一个和初始实现如下：

My first and naive implementation looks like this:

public bool Store<T>(string key, T data)
{
    var formatter = Serializer.CreateFormatter<T>();
    using (var fileStream = new FileStream(this.GetFilePath(key), FileMode.Create))
    {
        formatter.Serialize(fileStream, data);
        return true;
    }
}

但是我遇到了这个异常：

But then I got this exception:

不希望类型，并且不能推断合同：
My.Application.Namespace.ShortcutData

Type is not expected, and no contract can be inferred: My.Application.Namespace.ShortcutData

目前我有点卡住了，我没有找到有关如何使用protobuf-net的好教程。

Currently I'm a bit stuck, I didn't find a good tutorial on how to use protobuf-net.

是否可以使用protobuf达到这些要求？

Is it possible to reach these requirements with protobuf ? Do you have good tutorial on how to do this ?

编辑：

这个问题确实是我需要的告诉protobuf如何（de）序列化数据。
以下是我现在的状态：

The problem is indeed that I need to tell protobuf how to (de)serialize data. Here is what I have now:

public bool Store<T>(string key, T data)
{
    this.Register<T>();
    var formatter = Serializer.CreateFormatter<T>();
    using (var fileStream = new FileStream(this.GetFilePath(key), FileMode.Create))
    {
        formatter.Serialize(fileStream, data);
        fileStream.Close();
        return true;
    }
}

主代码位于Register方法中： p>

The main code is in the Register method:

protected void Register(Type type)
{
    if (type.IsGenericType)
    {
        var arguments = type.GetGenericArguments();
        foreach (var argument in arguments)
            this.Register(argument);
    }

    if (!this._registeredTypes.Contains(type) && !type.IsValueType)
    {
        this._registeredTypes.Add(type);
        var properties = type.GetProperties();
        foreach (var property in properties)
        {
            this.Register(property.PropertyType);
        }

        try
        {
            ProtoBuf.Meta.RuntimeTypeModel.Default
                .Add(type, false)
                .Add(properties
                    .Where( p => p.CanWrite)
                    .OrderBy(x => x.Name)
                    .Select(x => x.Name)
                    .ToArray());
        }
        catch
        {
            // I've a problem here: I sometime have an error for an already registered type (??)
        }
    }
}

我知道这不是一个干净的代码，但这只是一个

I know this is not a clean code, but this is just an in-replacement of existing code and a complete rewrite with protobuf in mind will follow in a second step.

推荐答案

我只有刚才的代码开始玩Protobuf网..所以请不要以为我的答案是在任何方式的正确的方式做事情。 Marc Gravell无疑会在某个时候给你一个答案，你一定要注意。

I have only just started playing with Protobuf-net.. so please don't assume my answer is in any way the "proper" way to do things. Marc Gravell will no doubt provide you an answer at some point that you should definitely take note of.

..但是，我的概念证明，我写的其他天需要我得到一堆类序列化和快速。为此，我想出了这一点：

..however, my proof-of-concept that I wrote the other day required that I get a heap of classes serializing and quickly. To that end.. I came up with this:

var assembly = Assembly.Load("Assembly.Name.Here");

foreach (var type in assembly.GetTypes().Where(x => typeof (IInterfaceEachClassImplements).IsAssignableFrom(x))) {
    RuntimeTypeModel
        .Default
        .Add(type, false)
        .Add(type.GetProperties()
            .Select(x => x.Name)
            .ToArray());
}

基本上，它会根据接口将每个类加载到protobuf-net类型模型他们都实现了..不属于他们的属性。

Basically, it loads each class into the protobuf-net typemodel based on an interface they all implement.. not attributes applied to them.

希望这会指向正确的方向。

Hopefully this points you in the right direction anyway.

这篇关于通过protobuf替换binaryformatter的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！