C＃中有效的小字节数组 [英] Efficient small byte-arrays in C#

问题描述

我有一大堆非常小的物件。为了确保数据的存储非常紧凑，我重写了类，以便将字节数组中的所有信息存储为可变字节编码。这些数百万个对象的大多数实例只需要3到7个字节来存储所有数据。

在内存分析之后，我发现这些字节数组总是占用32个字节。

有没有办法将信息更紧密地存储到位一个字节[]？指向非托管数组会更好吗？

  class MyClass 
 {
 byte []压缩; 
 
 public MyClass（IEnumerable< int> data）
 {
压缩=压缩（数据）; 
} 
 
私有字节[] compress（IEnumerable< int>数据）
 {
 // ... 
} 
 
 private IEnumerable< int>解压缩（byte [] compressedData）
 {
 // ... 
} 
 
 public IEnumerable< int> Data {get {return decompress（compressed）; }} 
} 
  

解决方案

你所面临的问题会消耗记忆力。一个是对象开销，另一个是对齐到32或64位边界的对象（取决于你的构建）。你目前的做法有两个问题。以下资料更详细地描述了这一点：

• 内存和字符串


• C＃字符串占用多少内存

当我在。

简单的解决方案是简单地创建一个具有长值的单个成员的结构。它的方法将处理打包和打包字节进出这个长时间，使用shift和掩码位fiddling。

另一个想法将是一个类，通过ID提供对象，并将实际的字节存储在单个后台 List< byte> 中。但这会变得复杂和凌乱。我认为结构思想更加简单明了。

I have a huge collection of very small objects. To ensure the data is stored very compactly I rewrote the class to store all information within a byte-array with variable-byte encoding. Most instances of these millions of objects need only 3 to 7 bytes to store all the data.

After memory-profiling I found out that these byte-arrays always take at least 32 bytes.

Is there a way to store the information more compactly than bit-fiddled into a byte[]? Would it be better to point to an unmanaged array?

class MyClass
{
    byte[] compressed;

    public MyClass(IEnumerable<int> data)
    {
        compressed = compress(data);
    }

    private byte[] compress(IEnumerable<int> data)
    {
        // ...
    }

    private IEnumerable<int> decompress(byte[] compressedData)
    {
        // ...
    }

    public IEnumerable<int> Data { get { return decompress(compressed); } }
}

解决方案

There are a couple problems you're facing that eat up memory. One is object overhead, and the other is objects aligning to 32 or 64 bit boundaries (depending on your build). Your current approach suffers from both issues. The following sources describe this in more detail:

• Of Memory and Strings
• How much memory does a C# string take up

I played around with this when I was fiddling with benchmarking sizes.

A solution that is simple would be to simply create a struct that has a single member that is a long value. Its methods would handle packing and unpacking bytes into and out of that long, using shift and mask bit fiddling.

Another idea would be a class that served up objects by ID, and stored the actual bytes in a single backing List<byte>. But this would get complicated and messy. I think the struct idea is much more straightforward.

这篇关于C＃中有效的小字节数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！