处理和显示非常大的文本文件的内容 [英] Processing and displying the content of a very large text file

问题描述

嗨
有人对处理和显示非常大的文本文件的内容有任何想法吗?

当前，当文件大小为数十兆字节时，我的尝试失败. (StringBuilder构造函数因内存不足异常而死亡.)

我需要:
*显示完整日志
*显示完整日志，不包含包含某些关键字的行
*仅显示包含某些关键词的行.这些必须按关键字列出
*随意显示或隐藏补充信息
*轻松跟踪关键词行回到完整日志.

=======================================
这是故事的较长版本:
我有一个第三方(即我无法控制该规范)软件，它吐出了无数日志，因此我需要编写一个工具来对其进行分析.

日志是一个简单的文本文件，以"\ r \ n"分隔.

常见大小约为10MB.

在这种尺寸下，记事本无法应付，但是还有其他查看器可以显示内容，例如写字板.

但是，显示是一回事，理解是另一回事.

因此，我开发了一种工具，可以根据一组预定的关键字在一段时间内提取一条日志，并在每个关键字的列表框中列出它们.

因此，一个列表框列出了所有与打开文件"相关的日志，另一个列表框列出了与开始分析"相关的日志，依此类推.

另外，某些命令需要很长的输入字符串.例如，开始分析"接受如何执行分析的XML输入.对于这些日志，可以按照用户的指示隐藏或显示详细信息.

列表框中的每个项目都可以追溯到完整的日志，双击该日志将在富文本框中显示.

这正是我所需要的.我可以在完整的日志中交叉引用关键字，并在任何给定时间仅查看我需要的信息...

除了...

当文件大小为数十兆字节时，它会炸毁.

我要处理的最大文件超过200MB.

如何扩展我的小型日志分析器以处理更大的日志文件?
有没有更好的方法来实现我的意思?

任何建议都将不胜感激.

Hi
Anyone got any idea on processing and displaying the content of a very large text file?

Currently, my attempt fails when the file size is in tens of megabytes. (StringBuilder constructor dies with Out of Memory Exception.)

I need to:
*Display the full log
*Display the full log without lines containing certain key words
*Display only the lines containing certain key words. These must be listed per key word
*Display or hide supplementary information at will
*Track a key word line back to the full log with ease.

=========================================
Here is the longer version of the story:
I''ve got a 3rd party (i.e. of which I have no control over the spec) software that spits out endless amounts of log, and I need to write a tool to analyze it.

The log is a simple text file delimited by "\r\n".

A common size is around 10MB.

At this size, notepad can''t cope, but there are other viewers that can display the content, e.g. wordpad.

However, displaying is one thing, making sense of it is quite another.

So I made a tool that pulls out a line of log a time according to a set of predetermined key words and lists them in list boxes per key word.

So one list box lists all the "Open file" related log, another lists a "Start analysis" related log and so on.

Also, some commands take long input strings. For example, "Start analysis" takes an XML input of how to perform the analysis. For these logs, the details can be hidden or shown as instructed by the user.

Each item in the list boxes can be traced back to the full log which is displayed in a rich text box with double click.

It''s exactly what I needed. I can cross reference the key words with the full log and view just the information I need at any given time...

except...

it blows up when the file size is in the tens of megabytes.

The biggest file I have to hand is over 200MB.

How can I extend my little log analyzer to cope with much bigger log files?
Is there a better way of implementing what I mean to do?

Any suggestions greatly appreciated.

推荐答案

您能想到一个替代方法:代替家庭烘焙的日志记录，您可以登录系统事件日志(可通过applet查看: eventvwr.msc /s).使用System.Diagnostics.EventLog， http://msdn.microsoft.com/en-us/library /system.diagnostics.eventlog.aspx [ ^ ].

将这些问题留给操作系统！

另请参阅我过去的解决方案中的一些实用建议:
如何在文件夹下创建事件日志 [ ^ ]，
将MsBuild OutPut即时发送到TextBox Windows应用程序中 [ ^ ].

—SA

Can you think of an alternative: instead of you home-baked logging, you can log in System event log (viewable via the applet: eventvwr.msc /s). Use System.Diagnostics.EventLog, http://msdn.microsoft.com/en-us/library/system.diagnostics.eventlog.aspx[^].

Leave those problems to the OS!

See also some practical advice in my past solutions:
How to create event log under a folder[^],
MsBuild OutPut to the TextBox on the fly in Windows Application[^].

—SA

这篇关于处理和显示非常大的文本文件的内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！