使用 Windows 服务编写同步应用程序以并行处理文件 [英] Writing a sync application using Windows Service to process files in Parallel

问题描述

我的 Windows 服务器上有一个文件夹，人们将在其中上传 CSV 文件到 C:\Uploads.

I have a folder on my windows server, where people will be uploading CSV files to, C:\Uploads.

我想编写一个简单的 Windows 服务应用程序，它将扫描此上传文件夹(每 5 秒)并收集文件并并行处理它们(线程/每个文件?).但是主扫描过程不能重叠，即需要加锁.

I want to write a simple windows service application that will scan this uploads folder (every 5 seconds) and collect the files in and process them in parallel (Thread /per File?). However, the main scanning process should not overlap, i.e. locking is required.

所以，我是这样试验的:我知道这不是 Windows 服务代码，它是一个测试想法的控制台应用程序...

So, I was experimenting with it like this: I am aware this is not windows service code, it's a console app to test ideas...

更新代码，基于 dcastro 的回复

class Program
{
    static Timer _InternalTimer;
    static Object _SyncLock = new Object();

    static void Main(string[] args)
    {
        _InternalTimer = new Timer(InitProcess, null, 0, 5000); // Sync cycle is every 5 sec

        Console.ReadKey();
    }

    private static void InitProcess(Object state)
    {
        ConsoleLog("Starting Process");
        StartProcess();
    }

    static void StartProcess()
    {
        bool lockTaken = false;
        try
        {
            Monitor.TryEnter(_SyncLock, ref lockTaken);
            if (lockTaken)
            {
                ConsoleLog("Lock Acquired. Doing some dummy work...");

                List<string> fileList = new List<string>()
                {
                    "fileA.csv",
                    "fileB.csv"
                };

                Parallel.ForEach(fileList, (string fileName) =>
                {
                    ConsoleLog("Processing File: " + fileName);
                    Thread.Sleep(10000); // 10 sec to process each file
                });

                GC.Collect();
            }
            else
                ConsoleLog("Sync Is Busy, Skipping Cycle");
        }
        finally
        {
            if (lockTaken)
                Monitor.Exit(_SyncLock);
        }
    }

    static void ConsoleLog(String Message)
    {
        Console.WriteLine("[{0}]: {1}",
            DateTime.UtcNow.ToString("HH:mm:ss tt"),
            Message);
    }
}

运行时是这样的:

这看起来对吗?任何有关改进此问题的帮助/提示将不胜感激.

Does this look right? Any help/tips on improving this will be much appreciated.

推荐答案

对我来说似乎很好，除了您不需要使用 Task.Factory.StartNew 启动任务.System.Threading.Timer 已经在 ThreadPool 上执行您的回调，因此无需再启动另一个也将在线程池上运行的任务.

It seems fine to me, apart from the fact that you don't need to start a task with Task.Factory.StartNew. The System.Threading.Timer already executes your callback on the ThreadPool, so there's no need to launch yet another task that will also be run on the thread pool.

此外，如果您的计时器每 5 秒计时一次，并且您预计需要大约 10 秒来处理文件，那么您的线程将开始排队等待释放锁.这发生在您发布的示例中.

Also, if your timer ticks every 5 seconds, and you expect it to take about 10 secs to process the files, then your threads will begin to queue up waiting for the lock to be released. That happened on the example you posted.

如果是这种情况，我要么将计时器的周期增加到 10 秒以上，要么使用 Monitor.TryEnter 而不是常规的 lock.TryEnter 会尝试获取锁，不管锁是否被获取，都会立即返回.如果该锁当前被另一个线程占用，您只需完全跳过此勾.

If this is the case, I would either increase the timer's period to more than 10 secs, or use Monitor.TryEnter instead of a regular lock. TryEnter will try to acquire the lock, and return immediately regardless of whether or not the lock was taken. If the lock is currently taken by another thread, you just skip this tick entirely.

这篇关于使用 Windows 服务编写同步应用程序以并行处理文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！