我怎样才能让一个二进制文件的反向扫描速度更快？ [英] How can I make reverse scanning of a binary file faster?

问题描述

我有一个二进制文件规范，它描述了分组数据结构。每个数据分组有一个两字节的同步模式，所以扫描一个包的开始是可能的，使用 BinaryReader在和的FileStream 组合：

I have a binary file specification that describes a packetized data structure. Each data packet has a two-byte sync pattern, so scanning for the beginning of a packet is possible, using a BinaryReader and FileStream combination:

while(!reader.EndOfFile)
{
    // Check for sync pattern.
    if (reader.ReadUInt16() != 0xEB25)
    {
        // Move to next byte.
        reader.BaseStream.Seek(-1, SeekOrigin.Current);
        continue;
    }

    // If we got here, a sync pattern was found.
}

这个过程中工作完全正常的前进方向，但在反方向类似code扫描幅度至少有两个数量级慢：

This process works perfectly fine in the forward direction, but similar code scanning in the reverse direction is at least two orders of magnitude slower:

while(!reader.BeginningOfFile)
{
    // Check for sync pattern.
    if (reader.ReadUInt16() != 0xEB25)
    {
        // Move to previous byte.
        reader.BaseStream.Seek(-3, SeekOrigin.Current);
        continue;
    }

    // If we got here, a sync pattern was found.
}

我已经尝试了一些解决方法，如移动回用任意数量（目前为1兆字节），并向前扫描，但它越来越清楚的是我真正需要的是一个 BinaryReader在或的FileStream 被修改，以在正向读取时获得足够的性能特点和反向的。

I've tried a few workarounds, like moving back by an arbitrary amount (currently 1 megabyte) and scanning forward, but it's becoming clear that what I really need is a BinaryReader or FileStream that is modified to have adequate performance characteristics when reading in both forward and reverse directions.

我已经有一个 FastFileStream 其通过继承一个普通的改善着读取性能的FileStream 和缓存位置和长度特性（它也提供了 BeginningOfFile 和 EndOfFile 属性）。这就是驱动读在code以上变量。

I already have a FastFileStream which improves forward read performance by subclassing an ordinary FileStream and caching the Position and Length properties (it also provides the BeginningOfFile and EndOfFile properties). That's what drives the reader variable in the code above.

有没有类似的东西我可以做些什么来改善反向读取性能，或许通过将的MemoryStream 作为缓冲？

Is there something similar I could do to improve reverse reading performance, perhaps by incorporating a MemoryStream as a buffer?

推荐答案

LB在注释中使用内存映射文件所提到的，你可能会pssed的表现IM $ P $。

L.B mentioned in a comment to use a Memory Mapped file, you may be impressed with the performance.

请尝试是这样的：

var memoryMapName = Path.GetFileName(fileToRead);

using (var mapStream = new FileStream(fileToRead, FileMode.Open))
{
    using (var myMap = MemoryMappedFile.CreateFromFile(
                            mapStream, 
                            memoryMapName, mapStream.Length,
                            MemoryMappedFileAccess.Read, null, 
                            HandleInheritability.None, false))
    {                    
        long leftToRead = mapStream.Length;
        long mapSize = Math.Min(1024 * 1024, mapStream.Length);
        long bytesRead = 0;
        long mapOffset = Math.Max(mapStream.Length - mapSize, 0);

        while (leftToRead > 1)
        {
            using (var FileMap = myMap.CreateViewAccessor(mapOffset, 
                                 mapSize, MemoryMappedFileAccess.Read))
            {
                long readAt = mapSize - 2;
                while (readAt > -1)
                {
                    var int16Read = FileMap.ReadUInt16(readAt);
                    //0xEB25  <--check int16Read here                            
                    bytesRead += 1;
                    readAt -= 1;
                }
            }

            leftToRead = mapStream.Length- bytesRead;
            mapOffset = Math.Max(mapOffset - mapSize, 0);
            mapSize = Math.Min(mapSize, leftToRead);
        }
    }
}

这篇关于我怎样才能让一个二进制文件的反向扫描速度更快？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！