保持UI响应在分析一个非常大的日志文件 [英] Keeping the UI responsive while parsing a very large logfile

问题描述

我正在写解析一个非常大的日志文件，使用户可以在一个TreeView格式查看内容的应用程序。我用一个BackgroundWorker读取该文件，并在解析每封邮件中，我使用的BeginInvoke来获取GUI线程将节点添加到我的树视图。不幸的是，有两个问题：

I'm writing an app that parses a very large logfile, so that the user can see the contents in a treeview format. I've used a BackGroundWorker to read the file, and as it parses each message, I use a BeginInvoke to get the GUI thread to add a node to my treeview. Unfortunately, there's two issues:

• 树形视图是反应迟钝点击或滚动，而该文件正在解析。我希望用户能够检查（即扩张），而该文件的解析，让他们不必等待整个文件来完成解析节点。


• 树形视图每次添加新节点时闪烁

下面的表格中的代码：

private void btnChangeDir_Click(object sender, EventArgs e)
{
    OpenFileDialog browser = new OpenFileDialog();

    if (browser.ShowDialog() == DialogResult.OK)
    {
        tbSearchDir.Text = browser.FileName;
        BackgroundWorker bgw = new BackgroundWorker();
        bgw.DoWork += (ob, evArgs) => ParseFile(tbSearchDir.Text);
        bgw.RunWorkerAsync();
    }
}

private void ParseFile(string inputfile)
{
    FileStream logFileStream = new FileStream(inputfile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
    StreamReader LogsFile = new StreamReader(logFileStream);

    while (!LogsFile.EndOfStream)
    {
        string Msgtxt = LogsFile.ReadLine();
        Message msg = new Message(Msgtxt.Substring(26)); //Reads the text into a class with appropriate members
        AddTreeViewNode(msg);
    }
}

private void AddTreeViewNode(Message msg)
{
    TreeNode newNode = new TreeNode(msg.SeqNum);

    BeginInvoke(new Action(() =>
                               {
                                   treeView1.BeginUpdate();
                                   treeView1.Nodes.Add(newNode);
                                   treeView1.EndUpdate();
                                   Refresh();
                               }
                    )); 
}

什么需要改变？

What needs to be changed?

修改

New code, to replace the last function above:
        List<TreeNode> nodeQueue = new List<TreeNode>(1000);

        private void AddTreeViewNode(Message msg)
        {
            TreeNode newNode = new TreeNode(msg.SeqNum);

            nodeQueue.Add(newNode);

            if (nodeQueue.Count == 1000)
            {
                var buffer = nodeQueue.ToArray();
                nodeQueue.Clear();
                BeginInvoke(new Action(() =>
                                           {
                                               treeView1.BeginUpdate();
                                               treeView1.Nodes.AddRange(buffer);
                                               treeView1.EndUpdate();
                                               Refresh();
                                               Application.DoEvents();
                                           }
                                ));
            }
        }

不知道为什么，我离开了刷新，并在的DoEvents那里。测试其它评论有点...

Not sure why I left the Refresh and the DoEvents in there. A bit of testing other comments...

推荐答案

调用/的BeginInvoke 使用 PostMessage的内部编组从任意线程到UI线程的请求。 的BeginInvoke 将充斥你的windows消息队列，需要由UI线程处理的消息，伴随的事实，要禁用树添加的每个节点重绘可能是影响你的它加载，而与树的互动能力。

Invoke/BeginInvoke uses PostMessage internally to marshal a request from an arbitrary thread to the UI thread. BeginInvoke will be flooding your windows message queue with messages which need to be processed by the UI thread, that accompanied by the fact that you are disabling the tree redraw with every node you add is probably impacting your ability to interact with the tree while it is loading.

一个选择是批处理多项更新，然后发送更新分批树。所以解析文件和更新每100树或一些数量的节点，而不是1在时间

One option would be to batch a number of updates together and then send them update the tree in batches. So parse the file and update the tree with every 100 or some number of nodes rather than 1 at a time.

更新：在您的编辑分批添加节点我会建议如下。

Update: After your edit to add nodes in batches I would suggest the following.

1而使用调用比的BeginInvoke ，否则队列满了，而树正被更新，那么一旦树更新下一个千年的节点已经准备好要插入可让您立即回到原来的地方在哪里。

1- Rather use Invoke than BeginInvoke, otherwise the queue fills up while the tree is being updated and then once the tree is updated the next thousand nodes are ready to be inserted which puts you right back where you where.

2-休眠插入每批使得存在一个时期，用户界面可以响应后数100毫秒。您可以使用此玩，这将平衡性能与用户体验。再也睡不着了会觉得用户反应更灵敏，但最终将需要更长的时间加载的所有数据。

2- Sleep a few 100 milliseconds after inserting each batch so that there is a period that UI can respond. You can play with this, this will balance performance vs. user experience. Longer sleep will feel more responsive for the user, but will ultimately take longer to get all the data loaded.

3注意，目前的解决方案，分批将错过最后几节点如果总数不是1000

3- Note that your current batching solution will miss the last few nodes if the total count is not a multiple of 1000

    private void AddTreeViewNode(Message msg) 
    { 
        TreeNode newNode = new TreeNode(msg.SeqNum); 

        nodeQueue.Add(newNode); 

        if (nodeQueue.Count == 1000) 
        { 
            var buffer = nodeQueue.ToArray(); 
            nodeQueue.Clear(); 
            Invoke(new Action(() => 
                { 
                    treeView1.BeginUpdate(); 
                    treeView1.Nodes.AddRange(buffer); 
                    treeView1.EndUpdate(); 
                }));
            System.Threading.Thread.Sleep(500); 
        } 
    }

这篇关于保持UI响应在分析一个非常大的日志文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！