无法使用MemoryMappedViewStream读取大型日志文件 [英] Unable to read large log file with MemoryMappedViewStream
问题描述
问题:是否可以使用MemoryMappedViewStream读取超过2GB的文件?
Question: Is there a way to read files in excess of 2GB using MemoryMappedViewStream?
我正在尝试读取1到12 GB范围内的日志文件. 1GB的文件显示为OK，但是读取4GB的文件时出现以下错误:
I am attempting to read log files that are anywhere from 1 to 12 GB. The 1GB files read OK, but I am receiving the following error when reading a 4GB file:
System.IO.IOException HResult = 0x80070008消息=不够 存储空间可用于处理此命令.
System.IO.IOException HResult=0x80070008 Message=Not enough storage is available to process this command.
Source = System.Core StackTrace:在
System.IO .__ Error.WinIOError(Int32 errorCode，字符串mayFullPath)
在
System.IO.MemoryMappedFiles.MemoryMappedView.CreateView(SafeMemoryMappedFileHandle
memMappedFileHandle，MemoryMappedFileAccess访问，Int64偏移量，
int64大小)
System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewStream(Int64
偏移量，Int64大小，MemoryMappedFileAccess访问)
System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewStream()在
ExchIISParserLib.LogParser.ParseLogs(Int32 daysago)在
...
Source=System.Core StackTrace: at
System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at
System.IO.MemoryMappedFiles.MemoryMappedView.CreateView(SafeMemoryMappedFileHandle
memMappedFileHandle, MemoryMappedFileAccess access, Int64 offset,
Int64 size) at
System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewStream(Int64
offset, Int64 size, MemoryMappedFileAccess access) at
System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewStream() at
ExchIISParserLib.LogParser.ParseLogs(Int32 daysago) in
...
我的系统有足够的磁盘和内存空间可用于读取4GB文件.有问题的代码行是:
My system has plenty of disk and memory space available to read the 4GB file. The line of code in question is:
MemoryMappedViewStream memoryMappedViewStream = MemoryMappedFile.CreateFromFile(log, FileMode.Open).CreateViewStream();
在研究工作中，我发现当文件超过2GB时，MemoryMappedViewStream似乎是有问题的.
In my research efforts, I have found that MemoryMappedViewStream seems to be problematic when files exceed 2GB.
https://stackoverflow.com/a/49738895/4630376
我已经查看了CreateViewStream()方法的offset和size参数.但是，这些文件似乎只是在指定文件上创建一个静态窗口，而该窗口不会读取整个文件.
I have looked at the offset and size parameters for the CreateViewStream() method. But those appear to just create a static window over the specified file, which does not read the entire file.
推荐答案
内存映射视图流是内存映射视图上的流.它不会在整个文件上提供流，而仅提供您映射的部分.您仍然需要将文件分块映射以读取整个内容.除非您确实需要共享内存，否则最好分块读取文件.
A memory mapped view stream is a stream over a memory mapped view. It does not provide a stream on the whole file, but just the part that you map. You will still need to map the file in chunks to read the whole thing. Unless you really need shared memory you are probably just better off with reading the file in chunks.
这篇关于无法使用MemoryMappedViewStream读取大型日志文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
