内存映射文件读取文件的结尾？ [英] Memory Mapped File to Read End of File?

问题描述

我有一个6GB的文件，最后20行是坏的。我想在.NET 4中使用内存映射文件来读取最后几行，并将其显示在console.writelines中，然后转到最后20行，并将其替换为String.Empty。什么是一个很酷的方式来做到这一点，使用内存映射文件/流与C＃示例？

谢谢。

无论如何，这里是一个C＃实现，不使用内存映射文件，而是一个普通的FileStream。它基于 ReverseStreamReader 实现（代码也包含在内）。我会很好奇，看到它在性能和内存消耗方面与其他MMF解决方案相比。

  public static void OverwriteEndLines（string filePath，int linesToStrip）
 {
 if（filePath == null）
 throw new ArgumentNullException（filePath）; 
 
 if（linesToStrip< = 0）
 return; 
 
 using（FileStream file = new FileStream（filePath，FileMode.Open，FileAccess.ReadWrite））
 {
 using（ReverseStreamReader reader = new ReverseStreamReader（file））
 {
 int count = 0; 
 do 
 {
 string line = reader.ReadLine（）; 
 if（line == null）//文件结尾
 break; 
 
 count ++; 
 if（count == linesToStrip）
 {
 // write CR LF 
 for（int i = 0; i< linesToStrip; i ++）
 {
 file.WriteByte（（byte）'\r'）; 
 file.WriteByte（（byte）'\\\
'）; 
} 
 
 //截断文件到当前流位置
 file.SetLength（file.Position）; 
 break; 
} 
} 
 while（true）; 
 
 
 
 $ b //注意：我们还没有实现所有的ReadXXX方法
 public class ReverseStreamReader：StreamReader 
 {
 private bool _returnEmptyLine; 
 
 public ReverseStreamReader（Stream stream）
：base（stream）
 {
 BaseStream.Seek（0，SeekOrigin.End）; 
 
 $ b $ public override int Read（）
 {
 if（BaseStream.Position == 0）
 return -1; 
 
 BaseStream.Seek（-1，SeekOrigin.Current）; 
 int i = BaseStream.ReadByte（）; 
 BaseStream.Seek（-1，SeekOrigin.Current）; 
返回i; 
 
 $ b public override string ReadLine（）
 {
 if（BaseStream.Position == 0）
 {
 if（_returnEmptyLine） 
 {
 _returnEmptyLine = false; 
返回string.Empty; 
} 
返回null; 
} 
 
 int read; 
 StringBuilder sb = new StringBuilder（）; （（read = Read（））> = 0）
 if（read =='\\\
'）
 {
 read = Read （）; 
 //支持windows& （（读> 0）&&（read！='\r'））
 {
 BaseStream.Position ++; 
 
 if（BaseStream.Position == 0）
 {
 //处理特殊的第一行空格
 _returnEmptyLine = true; 
} 
 break; 
} 
 sb.Append（（char）read）; 
} 
 
 //反向字符串。注意这是可选的，如果我们不需要字符串内容
 if（sb.Length> 1）
 {
 char [] array = new char [sb.Length]; 
 sb.CopyTo（0，array，0，array.Length）; 
 Array.Reverse（array）; 
返回新的字符串（数组）; 
} 
 return sb.ToString（）; 
} 
} 
  

I have a 6gb file and the last 20 lines are bad. I would like to use a memory-mapped file with .NET 4 to read the last few lines and display them in console.writelines, and later go the last 20 lines and replace them with String.Empty. What is a cool way to do that using a memory-mapped file/stream with a C# example?

Thanks.

解决方案

Memory Mapped Files can be a problem for big files (typically files that are of a size equivalent or bigger than the RAM), in case you eventually map the whole file. If you map only the end, that should not be a real issue.

Anyway, here is a C# implementation that does not use Memory Mapped File, but a regular FileStream. It is based on a ReverseStreamReader implementation (code also included). I would be curious to see it compared to other MMF solutions in terms of performance and memory consumption.

public static void OverwriteEndLines(string filePath, int linesToStrip)
{
    if (filePath == null)
        throw new ArgumentNullException("filePath");

    if (linesToStrip <= 0)
        return;

    using (FileStream file = new FileStream(filePath, FileMode.Open, FileAccess.ReadWrite))
    {
        using (ReverseStreamReader reader = new ReverseStreamReader(file))
        {
            int count = 0;
            do
            {
                string line = reader.ReadLine();
                if (line == null) // end of file
                    break;

                count++;
                if (count == linesToStrip)
                {
                    // write CR LF
                    for (int i = 0; i < linesToStrip; i++)
                    {
                        file.WriteByte((byte)'\r');
                        file.WriteByte((byte)'\n');
                    }

                    // truncate file to current stream position
                    file.SetLength(file.Position);
                    break;
                }
            }
            while (true);
        }
    }
}

// NOTE: we have not implemented all ReadXXX methods
public class ReverseStreamReader : StreamReader
{
    private bool _returnEmptyLine;

    public ReverseStreamReader(Stream stream)
        : base(stream)
    {
        BaseStream.Seek(0, SeekOrigin.End);
    }

    public override int Read()
    {
        if (BaseStream.Position == 0)
            return -1;

        BaseStream.Seek(-1, SeekOrigin.Current);
        int i = BaseStream.ReadByte();
        BaseStream.Seek(-1, SeekOrigin.Current);
        return i;
    }

    public override string ReadLine()
    {
        if (BaseStream.Position == 0)
        {
            if (_returnEmptyLine)
            {
                _returnEmptyLine = false;
                return string.Empty;
            }
            return null;
        }

        int read;
        StringBuilder sb = new StringBuilder();
        while((read = Read()) >= 0)
        {
            if (read == '\n')
            {
                read = Read();
                // supports windows & unix format
                if ((read > 0) && (read != '\r'))
                {
                    BaseStream.Position++;
                }
                else if (BaseStream.Position == 0)
                {
                   // handle the special empty first line case
                    _returnEmptyLine = true;
                }
                break;
            }
            sb.Append((char)read);
        }

        // reverse string. Note this is optional if we don't really need string content
        if (sb.Length > 1)
        {
            char[] array = new char[sb.Length];
            sb.CopyTo(0, array, 0, array.Length);
            Array.Reverse(array);
            return new string(array);
        }
        return sb.ToString();
    }
}

这篇关于内存映射文件读取文件的结尾？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！