使用FileSystemWatcher和C#处理具有多个文件的文件夹 [英] Processing Folder With Multiple Files Using FileSystemWatcher and C#

问题描述

我创建了一个相对简单的Windows应用程序，该应用程序监视文件夹中的文件.当在文件夹中创建一个新文件时，该应用程序(通过FileSystemWatcher)将打开该文件并处理内容.长话短说，内容与Selenium一起用于通过IE11自动化网页.每个文件大约需要20秒的处理时间.

I have created a relatively simple windows app that watches a folder for files. When a new file is created into the folder the app (via FileSystemWatcher) will open the file and process the contents. Long story short, the contents are used with Selenium to automate a web page via IE11. This processing takes about 20 seconds per file.

问题是，如果大约在同一时间在文件夹中创建了多个文件，或者在应用程序正在处理文件时，FileSystemWatcher onCreated看不到下一个文件.因此，当第一个文件上的处理完成时，应用程序将停止.同时，文件夹中有一个未处理的文件.如果在onCreated处理完成后添加文件，则可以正常工作并处理下一个文件.

The problem is if more than one file is created into the folder at roughly the same time or when the app is processing a file, FileSystemWatcher onCreated does not see the next file. So when processing completes on the first file the app just stops. Meanwhile there is a file in the folder that does not get processed. If a file is added after the onCreated processing is finished it works fine and processes that next file.

有人可以指导我解决该问题吗?过多的细节是非常欢迎的.

Can someone please guide me towards what I should be looking at to solve this? Excessive detail is very welcome.

推荐答案

FileSystemWatcher(您已经注意到)不可靠，对于丢失的文件，您将始终必须添加自定义"/手动逻辑(另外，请注意，同一文件可能会看到多个事件)

FileSystemWatcher (as you've already noticed) is not reliable ,you will always have to add a "custom"/manual logic for missing files (also,note that you might see more than one event for the same file)

在下面，您可以看到一个简单的示例，其中包含对未处理文件的背景"检查.
您可以使用并发集合来避免锁定，例如 BlockingCollection
您还可以选择并行处理文件
我正在根据计时器处理文件，但是您可以使用自己的策略.
如果您不想实时处理文件，可能甚至不需要FileSystemWatcher

Below you can see a simple example with a "background" check for unprocessed file.
You could avoid the locks by using concurrent collections e.g BlockingCollection
You could also chose to process your files in parallel
I'm processing the file based on a timer but you could use your own strategy.
If you don't want to process the file ,in real time ,probably you don't even need the FileSystemWatcher

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;

namespace ConsoleAppDemo
{
    class Program
    {
        private static object lockIbj = new object();
        private static List<string> _proccessedFiles = new List<string>();
        private static readonly List<string> toProccessFiles = new List<string>();
        private static List<string> _proccessingFiles = new List<string>();
        private const string directory = @"C:\Path";
        private const string extension = @"*.txt";
        static void Main(string[] args)
        {
            FileSystemWatcher f = new FileSystemWatcher();
            f.Path = directory;
            f.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
                             | NotifyFilters.FileName | NotifyFilters.DirectoryName;
            f.Filter = extension ;
            f.Created += F_Created;
            f.EnableRaisingEvents = true;

            Timer manualWatcher = new Timer(ManuallWatcherCallback, null, 0, 3000);

            Timer manualTaskRunner = new Timer(ManuallRunnerCallback, null, 0, 10000);

            Console.ReadLine();
        }

        private static void F_Created(object sender, FileSystemEventArgs e)
        {
            lock (lockIbj)
            {
                toProccessFiles.Add(e.FullPath);
                Console.WriteLine("Adding new File from watcher: " + e.FullPath);
            }

        }

        private static void ManuallWatcherCallback(Object o)
        {
            var files = Directory.GetFiles(directory, extension);
            lock (lockIbj)
            {
                foreach (var file in files)
                {
                    if (!_proccessedFiles.Contains(file) && !toProccessFiles.Contains(file) && !_proccessingFiles.Contains(file))
                    {
                        toProccessFiles.Add(file);
                        Console.WriteLine("Adding new File from manuall timer: " + file);
                    }
                }

            }
        }

        private static bool processing;
        private static void ManuallRunnerCallback(Object o)
        {
            if (processing)
                return;

            while (true)
            {
                //you could proccess file in parallel
                string fileToProcces = null;

                lock (lockIbj)
                {
                    fileToProcces = toProccessFiles.FirstOrDefault();
                    if (fileToProcces != null)
                    {
                        processing = true;
                        toProccessFiles.Remove(fileToProcces);
                        _proccessingFiles.Add(fileToProcces);
                    }
                    else
                    {
                        processing = false;
                        break;


                    }
                }

                if (fileToProcces == null)
                    return;

                //Must add error handling
                ProccessFile(fileToProcces);
            }
        }

        private static void ProccessFile(string fileToProcces)
        {
            Console.WriteLine("Processing:" + fileToProcces);
            lock (lockIbj)
            {
                _proccessingFiles.Remove(fileToProcces);
                _proccessedFiles.Add(fileToProcces);
            }
        }
    }
}

这篇关于使用FileSystemWatcher和C#处理具有多个文件的文件夹的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！