最快的读写二进制文件的方法 [英] Fastest way of reading and writing binary

问题描述

我目前正在优化应用程序，经常执行的操作之一是读写二进制文件.我需要两种功能:

I'm currently optimizing an application, one of the operations that is done very often is reading and writing binary. I need 2 types of functions:

Set(byte[] target, int index, int value);

int Get(byte[] source, int index);

这些函数对于按大小无序排列的有符号和无符号short，int和long是必需的.

These functions are needed for signed and unsigned short, int and long in big and little endian order.

这里是一些示例，但是我需要对优点和缺点进行评估:

Here are some examples i've made, but i need a evaluation about the advantages and disadvantages:

第一种方法是使用Marshal将值写入byte []的内存中，第二种方法是使用普通指针来完成此操作，第三种方法是使用BitConverter和BlockCopy来完成此操作

first method is using Marshal to write the value into the memory of the byte[], the second is using plain pointers to accomplish this and the third uses BitConverter and BlockCopy to do this

unsafe void Set(byte[] target, int index, int value)
{
    fixed (byte* p = &target[0])
    {
        Marshal.WriteInt32(new IntPtr(p), index, value);
    }
}

unsafe void Set(byte[] target, int index, int value)
{
    int* p = &value;
    for (int i = 0; i < 4; i++)
    {
        target[offset + i] = *((byte*)p + i);
    }
}

void Set(byte[] target, int index, int value)
{
    byte[] data = BitConverter.GetBytes(value);
    Buffer.BlockCopy(data, 0, target, index, data.Length);
}

这是Read/Get方法:

And here are the Read/Get methods:

第一个使用Marshal从byte []中读取值，第二个使用普通指针，第三个使用BitConverter:

the first is using Marshal to read the value from the byte[], the second is using plain pointers and the third is using BitConverter again:

unsafe int Get(byte[] source, int index)
{
    fixed (byte* p = &source[0])
    {
        return Marshal.ReadInt32(new IntPtr(p), index);
    }
}

unsafe int Get(byte[] source, int index)
{
    fixed (byte* p = &source[0])
    {
        return *(int*)(p + index);
    }
}

unsafe int Get(byte[] source, int index)
{
    return BitConverter.ToInt32(source, index);
}

边界检查需要完成，但这还不是我的问题...

boundary checking needs to be done but isn't part of my question yet...

如果有人能说出在这种情况下最好和最快的方法，或者给我一些其他解决方案，我会很高兴.通用的解决方案会更好

I would be pleased if someone can tell what would be the best and fastest way in this case or give me some other solutions to work on. A generic solution would be preferable

我刚刚做了一些性能测试，结果如下:

I Just did some performance testing, here are the results:

设置元帅:45毫秒，设置指针:48毫秒，设置位转换器:71毫秒 获取元帅:45毫秒，获取指针:26毫秒，获取BitConverter:30毫秒

Set Marshal: 45 ms, Set Pointer: 48 ms, Set BitConverter: 71 ms Get Marshal: 45 ms, Get Pointer: 26 ms, Get BitConverter: 30 ms

似乎使用指针是最快的方法，但是我认为Marshal和BitConverter进行了一些内部检查...有人可以验证这一点吗?

it seems that using pointers is the fast way, but i think Marshal and BitConverter do some internal checking... can someone verify this?

推荐答案

重要提示:如果只需要一个字节序，请参阅wj32/dtb的指针魔术

Important: if you only need the one endian, see the pointer magic by wj32 / dtb

就我个人而言，我将直接写入Stream(可能带有一些缓冲)，然后重新使用通常可以认为是干净的共享缓冲区.然后，您可以做一些捷径，并假设索引为0/1/2/3.

Personally, I would be writing directly to a Stream (perhaps with some buffering), and re-using a shared buffer that I can generally assume is clean. Then you can make some shortcuts and assume index 0/1/2/3.

当然不要使用BitConverter，因为这不能同时用于您需要的小端/大端.我也倾向于只使用移位而不是不安全等.基于以下情况，它实际上是最快的(因此，我很高兴这就是我已经在我的代码中做到的方式

Certainly don't use BitConverter, as that can't be used for both little/big-endian, which you require. I would also be inclined to just use bit-shifting rather than unsafe etc. It is actally the fastest, based on the following (so I'm glad that this is how I already do it my code here, look for EncodeInt32Fixed):

Set1: 371ms
Set2: 171ms
Set3: 993ms
Set4: 91ms <==== bit-shifting ;-p

代码:

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
static class Program
{
    static void Main()
    {
        const int LOOP = 10000000, INDEX = 100, VALUE = 512;
        byte[] buffer = new byte[1024];
        Stopwatch watch;

        watch = Stopwatch.StartNew();
        for (int i = 0; i < LOOP; i++)
        {
            Set1(buffer, INDEX, VALUE);
        }
        watch.Stop();
        Console.WriteLine("Set1: " + watch.ElapsedMilliseconds + "ms");

        watch = Stopwatch.StartNew();
        for (int i = 0; i < LOOP; i++)
        {
            Set2(buffer, INDEX, VALUE);
        }
        watch.Stop();
        Console.WriteLine("Set2: " + watch.ElapsedMilliseconds + "ms");

        watch = Stopwatch.StartNew();
        for (int i = 0; i < LOOP; i++)
        {
            Set3(buffer, INDEX, VALUE);
        }
        watch.Stop();
        Console.WriteLine("Set3: " + watch.ElapsedMilliseconds + "ms");

        watch = Stopwatch.StartNew();
        for (int i = 0; i < LOOP; i++)
        {
            Set4(buffer, INDEX, VALUE);
        }
        watch.Stop();
        Console.WriteLine("Set4: " + watch.ElapsedMilliseconds + "ms");

        Console.WriteLine("done");
        Console.ReadLine();
    }
    unsafe static void Set1(byte[] target, int index, int value)
    {
        fixed (byte* p = &target[0])
        {
            Marshal.WriteInt32(new IntPtr(p), index, value);
        }
    }

    unsafe static void Set2(byte[] target, int index, int value)
    {
        int* p = &value;
        for (int i = 0; i < 4; i++)
        {
            target[index + i] = *((byte*)p + i);
        }
    }

    static void Set3(byte[] target, int index, int value)
    {
        byte[] data = BitConverter.GetBytes(value);
        Buffer.BlockCopy(data, 0, target, index, data.Length);
    }
    static void Set4(byte[] target, int index, int value)
    {
        target[index++] = (byte)value;
        target[index++] = (byte)(value >> 8);
        target[index++] = (byte)(value >> 16);
        target[index] = (byte)(value >> 24);
    }
}

这篇关于最快的读写二进制文件的方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！