使用动态发射波苏斯快速序列化和反序列化 [英] Fast serialization and deserialization using dynamically emitted POCOs
问题描述
我目前SQL序列表行成有效的存储二进制格式。序列化/反序列化的二进制数据转化成名单，其中，对象＆gt; 行。我想升级此使用波苏斯，将动态生成（发出），每列一个字段。
I am currently serializing SQL table rows into a binary format for efficient storage. I serialize/deserialize the binary data into a List<object> per row. I'm trying to upgrade this to use POCOs, that will be dynamically generated (emitted) with one Field per column.
我一直在网上搜索了几个小时后，像EF，T4，ExpandoObject奥姆斯/框架已经迷迷糊糊的，但所有这些既可以使用一个动态对象（属性可以增加/在飞行中删除），或简单地生成一个POCO前编制。我不能使用模板，因为表的架构是未知在编译时，使用动态对象将是矫枉过正（和慢），因为我知道属性及其类型的确切集合。我需要生成每个表中的一个POCO，对应于柱的油田，并与相应设置的数据类型（INT - > INT，TEXT - >字符串）。
I've been searching online for hours and have stumbled upon ORMs/frameworks like EF, T4, ExpandoObject, but all of these either use a dynamic object (properties can be added/removed on the fly) or simply generate a POCO before compiling. I cannot use templating because the schema of the tables is unknown at compile time, and using dynamic objects would be overkill (and slow) since I know the exact set of properties and their types. I need to generate one POCO per table, with Fields corresponding to columns, and with the data types set accordingly (INT -> int, TEXT -> string).
生成POCO后，我会继续使用发射CIL获取/设置属性，就像什么<一href="http://jamesheppinstall.word$p$pss.com/2012/05/18/dynamic-method-invocation-in-c-with-petapoco-part-2/"相对=nofollow> PetaPoco不为静态编译波苏斯。我希望这一切繁琐程序会比使用无类型列表更快，并给我高保真波苏斯被强类型，并且可以由CLR加速。我是正确的假设呢？并能对你产生波苏斯在运行时启动我？而将使用波苏斯快得多或更多的内存效率比使用名单，其中，对象＆gt; ？基本上，这将是值得的麻烦？我已经知道如何加快获取/使用发射CIL设置栏目。
After generating the POCO, I'll proceed to get/set properties using emitted CIL, much like what PetaPoco does for statically compiled POCOs. I'm hoping all of this rigmarole will be faster than using untyped Lists, and give me high-fidelity POCOs that are strongly-typed and can be accelerated by the CLR. Am I correct to assume this? and can you start me off on generating POCOs at runtime? And will using POCOs be much faster or much more memory-efficient than using a List<object>? Basically, will it be worth the trouble? I already know how to accelerate getting/setting Fields using emitted CIL.
推荐答案
从评论和聊天，似乎这方面的一个关键部分仍是创造一个动态类型;好了，下面是一个完整的例子，显示了一个完全可序列化（由任何普通的序列化）类型。当然，你可以添加更多的类型 - 也许索引器获取由编号或名称属性， INotifyPropertyChanged的等
From comments and chat, it seems that a key part of this is still creating a dynamic type; ok, here's a full example that shows a fully serializable (by any common serializer) type. You could of course add more to the type - maybe indexers to get properties by number or by name, INotifyPropertyChanged, etc.
另外 - 关键点：您必须缓存和重新使用生成的键入实例。做的不可以保持再生这个东西......你会出血的记忆。
Also - critical point: you must cache and re-use the generated Type instances. Do not keep regenerating this stuff... you will hemorrhage memory.
using Newtonsoft.Json;
using ProtoBuf;
using System;
using System.IO;
using System.Reflection;
using System.Reflection.Emit;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;
using System.Xml.Serialization;
public interface IBasicRecord
{
object this[int field] { get; set; }
}
class Program
{
static void Main()
{
object o = 1;
int foo = (int)o;
string[] names = { "Id", "Name", "Size", "When" };
Type[] types = { typeof(int), typeof(string), typeof(float), typeof(DateTime?) };
var asm = AppDomain.CurrentDomain.DefineDynamicAssembly(
new AssemblyName("DynamicStuff"),
AssemblyBuilderAccess.Run);
var module = asm.DefineDynamicModule("DynamicStuff");
var tb = module.DefineType("MyType", TypeAttributes.Public | TypeAttributes.Serializable);
tb.SetCustomAttribute(new CustomAttributeBuilder(
typeof(DataContractAttribute).GetConstructor(Type.EmptyTypes), new object[0]));
tb.AddInterfaceImplementation(typeof(IBasicRecord));
FieldBuilder[] fields = new FieldBuilder[names.Length];
var dataMemberCtor = typeof(DataMemberAttribute).GetConstructor(Type.EmptyTypes);
var dataMemberProps = new[] { typeof(DataMemberAttribute).GetProperty("Order") };
for (int i = 0; i < fields.Length; i++)
{
var field = fields[i] = tb.DefineField("_" + names[i],
types[i], FieldAttributes.Private);
var prop = tb.DefineProperty(names[i], PropertyAttributes.None,
types[i], Type.EmptyTypes);
var getter = tb.DefineMethod("get_" + names[i],
MethodAttributes.Public | MethodAttributes.HideBySig, types[i], Type.EmptyTypes);
prop.SetGetMethod(getter);
var il = getter.GetILGenerator();
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldfld, field); // .Foo
il.Emit(OpCodes.Ret); // return
var setter = tb.DefineMethod("set_" + names[i],
MethodAttributes.Public | MethodAttributes.HideBySig, typeof(void), new Type[] { types[i] });
prop.SetSetMethod(setter);
il = setter.GetILGenerator();
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldarg_1); // value
il.Emit(OpCodes.Stfld, field); // .Foo =
il.Emit(OpCodes.Ret);
prop.SetCustomAttribute(new CustomAttributeBuilder(
dataMemberCtor, new object[0],
dataMemberProps, new object[1] { i + 1 }));
}
foreach (var prop in typeof(IBasicRecord).GetProperties())
{
var accessor = prop.GetGetMethod();
if (accessor != null)
{
var args = accessor.GetParameters();
var argTypes = Array.ConvertAll(args, a => a.ParameterType);
var method = tb.DefineMethod(accessor.Name,
accessor.Attributes & ~MethodAttributes.Abstract,
accessor.CallingConvention, accessor.ReturnType, argTypes);
tb.DefineMethodOverride(method, accessor);
var il = method.GetILGenerator();
if (args.Length == 1 && argTypes[0] == typeof(int))
{
var branches = new Label[fields.Length];
for (int i = 0; i < fields.Length; i++)
{
branches[i] = il.DefineLabel();
}
il.Emit(OpCodes.Ldarg_1); // key
il.Emit(OpCodes.Switch, branches); // switch
// default:
il.ThrowException(typeof(ArgumentOutOfRangeException));
for (int i = 0; i < fields.Length; i++)
{
il.MarkLabel(branches[i]);
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldfld, fields[i]); // .Foo
if (types[i].IsValueType)
{
il.Emit(OpCodes.Box, types[i]); // (object)
}
il.Emit(OpCodes.Ret); // return
}
}
else
{
il.ThrowException(typeof(NotImplementedException));
}
}
accessor = prop.GetSetMethod();
if (accessor != null)
{
var args = accessor.GetParameters();
var argTypes = Array.ConvertAll(args, a => a.ParameterType);
var method = tb.DefineMethod(accessor.Name,
accessor.Attributes & ~MethodAttributes.Abstract,
accessor.CallingConvention, accessor.ReturnType, argTypes);
tb.DefineMethodOverride(method, accessor);
var il = method.GetILGenerator();
if (args.Length == 2 && argTypes[0] == typeof(int) && argTypes[1] == typeof(object))
{
var branches = new Label[fields.Length];
for (int i = 0; i < fields.Length; i++)
{
branches[i] = il.DefineLabel();
}
il.Emit(OpCodes.Ldarg_1); // key
il.Emit(OpCodes.Switch, branches); // switch
// default:
il.ThrowException(typeof(ArgumentOutOfRangeException));
for (int i = 0; i < fields.Length; i++)
{
il.MarkLabel(branches[i]);
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldarg_2); // value
il.Emit(types[i].IsValueType ? OpCodes.Unbox_Any : OpCodes.Castclass, types[i]); // (SomeType)
il.Emit(OpCodes.Stfld, fields[i]); // .Foo =
il.Emit(OpCodes.Ret); // return
}
}
else
{
il.ThrowException(typeof(NotImplementedException));
}
}
}
var type = tb.CreateType();
var obj = Activator.CreateInstance(type);
// we'll use the index (via a known interface) to set the values
IBasicRecord rec = (IBasicRecord)obj;
rec[0] = 123;
rec[1] = "abc";
rec[2] = 12F;
rec[3] = DateTime.Now;
for (int i = 0; i < 4; i++)
{
Console.WriteLine("{0} = {1}", i, rec[i]);
}
using (var ms = new MemoryStream())
{
var ser = new XmlSerializer(type);
ser.Serialize(ms, obj);
Console.WriteLine("XmlSerializer: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
using (var writer = new StreamWriter(ms, Encoding.UTF8, 1024, true))
{
var ser = new JsonSerializer();
ser.Serialize(writer, obj);
}
Console.WriteLine("Json.NET: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
var ser = new DataContractSerializer(type);
ser.WriteObject(ms, obj);
Console.WriteLine("DataContractSerializer: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
Serializer.NonGeneric.Serialize(ms, obj);
Console.WriteLine("protobuf-net: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
// note: NEVER do this unless you have a custom Binder; your
// assembly WILL NOT deserialize in the next AppDomain (i.e.
// the next time you load your app, you won't be able to load)
// - shown only for illustration
var bf = new BinaryFormatter();
bf.Serialize(ms, obj);
Console.WriteLine("BinaryFormatter: {0} bytes", ms.Length);
}
}
}
输出：
XmlSerializer: 246 bytes
Json.NET: 81 bytes
DataContractSerializer: 207 bytes
protobuf-net: 25 bytes
BinaryFormatter: 182 bytes
这篇关于使用动态发射波苏斯快速序列化和反序列化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
