System.IO.FileSystemWatcher监视网络服务器文件夹 - 性能注意事项 [英] System.IO.FileSystemWatcher to monitor a network-server folder - Performance considerations

问题描述

我要一台网络服务器的变化上观看一个文件夹树。这些文件都具有特定的扩展。有树约200个文件夹和1200的文件，我看的扩展。

我不能写入在服务器上运行服务（禁区！）那么该溶液必须是本地的客户机。时效性不是特别重要的。我可以忍受的通知一分钟或更长时间的延迟。我看的创建，删除，重命名和更改。

会使用.NET System.IO.fileSystemWatcher创建许多服务器上的负载的？

怎么样10个独立观察家削减的文件夹/文件的数量被监视？ （下降到200从700文件夹，1200从5500文件计）更多的网络流量，而不是少了？我的想法是服务器把观看文件在1棵树上一次洗牌。我不一定有这个选项，因此观察家的团队。

我想其他的解决办法是定期检查，如果FSW在服务器上创建一个过度的负荷，或者如果它不为一大堆的系统管理员类型的理由工作。

有没有更好的方式来做到这一点？

解决方案

从一个服务器的负载点，使用IO.FileSystemWatcher为您所描述的情况下远程更改通知可能是可能是最有效的方法。它使用 FindFirstChangeNotification 和的 ReadDirectoryChangesW 的Win32 API内部函数，从而以优化的方式与网络重定向通信（假定标准Windows网络：如果第三方重定向时，它不支持所需要的功能，事情不会在所有的工作）。在.NET包装还使用异步I / O和一切，进一步确保最高效率。

这个解决方案的唯一问题是，它不是很可靠。除了不必处理网络连接要走了暂时（这是不是太大的问题，因为IO.FileSystemWatcher将触发错误事件在这种情况下，你可以处理），其作用机制具有一定的基本限制。从MSDN文档的Win32 API函数：

• ReadDirectoryChangesW失败，ERROR_INVALID_PARAMETER当缓冲区长度大于64 KB和应用程序监控的目录在网络上。这是由于一个数据包大小限制与下面的文件共享协议

• 要求FindFirstChangeNotification的远程文件系统时，通知可能无法返回

在换言之：在高负载下（当你需要一个大的缓冲区），更有甚者，在随机不确定的情况下，你可能不会得到你所期望的通知。这是连一个问题与本地文件系统观察，但它更在网络上的问题。 这里的另一个问题，关于SO 的细节所固有的可靠性问题，在更详细一点的API。

在使用文件系统观察，你的应用程序应该能够处理这些限制。例如：

• 如果你正在寻找的文件有序列号，存储你有通知了最近的序列号，所以你可以看看在未来的通知差距和处理的文件，你没有得到通知;

• 在接收通知，总是做完整的目录扫描。这听起来可能非常糟糕，但由于扫描是事件驱动的，它仍然比哑投票更有效。此外，只要你保持的文件在一个目录的总数，以及目录进行扫描的次数，在千元左右，性能这个操作的影响应该是pretty的最小的反正。

设置多个监听器是你应该避免尽可能多的：如果有的话，这会让事情变得少可靠...

总之，如果你绝对有无使用文件系统观察，事情可以工作得不错，只要你意识到的限制，不要指望1：1的通知修改的每个文件/创建。

因此​​，如果您还有其他的选择（从本质上讲，具有过程写文件，在非基于文件系统的方式通知您：任何常规RPC方法是一种进步...），这是绝对值得研究的来看一个可靠点。

I want to watch a folder tree on a network server for changes. The files all have a specific extension. There are about 200 folders in the tree and about 1200 files with the extension I am watching.

I can't write a service to run on the server (off-limits!) so the solution has to be local to the client. Timeliness is not particularly important. I can live with a minute or more delay in notifications. I am watching for Create, Delete, Rename and Changes.

Would using the .NET System.IO.fileSystemWatcher create much of a load on the server?

How about 10 separate watchers to cut down the number of folders/files being watched? (down to 200 from 700 folders, 1200 from 5500 files in total) More network traffic instead of less? My thoughts are a reshuffle on the server to put the watched files under 1 tree. I may not always have this option hence the team of watchers.

I suppose the other solution is a periodic check if the FSW creates an undue load on the server, or if it doesn't work for a whole bunch of SysAdmin type reasons.

Is there a better way to do this?

解决方案

From a server load point of view, using the IO.FileSystemWatcher for remote change notifications in the scenario you describe is probably the most efficient method possible. It uses the FindFirstChangeNotification and ReadDirectoryChangesW Win32 API functions internally, which in turn communicate with the network redirector in an optimized way (assuming standard Windows networking: if a third-party redirector is used, and it doesn't support the required functionality, things won't work at all). The .NET wrapper also uses async I/O and everything, further assuring maximum efficiency.

The only problem with this solution is that it's not very reliable. Other than having to deal with network connections going away temporarily (which isn't too much of a problem, since IO.FileSystemWatcher will fire an error event in this case which you can handle), the underlying mechanism has certain fundamental limitations. From the MSDN documentation for the Win32 API functions:

• ReadDirectoryChangesW fails with ERROR_INVALID_PARAMETER when the buffer length is greater than 64 KB and the application is monitoring a directory over the network. This is due to a packet size limitation with the underlying file sharing protocols

• Notifications may not be returned when calling FindFirstChangeNotification for a remote file system

In other words: under high load (when you would need a large buffer) or, worse, under random unspecified circumstances, you may not get the notifications you expect. This is even an issue with local file system watchers, but it's much more of a problem over the network. Another question here on SO details the inherent reliability problems with the API in a bit more detail.

When using file system watchers, your application should be able to deal with these limitations. For example:

• If the files you're looking for have sequence numbers, store the last sequence number you got notified about, so you can look for 'gaps' on future notifications and process the files for which you didn't get notified;

• On receiving a notification, always do a full directory scan. This may sound really bad, but since the scan is event-driven, it's still much more efficient than dumb polling. Also, as long as you keep the total number of files in a single directory, as well as the number of directories to scan, under a thousand or so, the impact of this operation on performance should be pretty minimal anyway.

Setting up multiple listeners is something you should avoid as much as possible: if anything, this will make things even less reliable...

Anyway, if you absolutely have to use file system watchers, things can work OK as long as you're aware of the limitations, and don't expect 1:1 notification for every file modified/created.

So, if you have other options (essentially, having the process writing the files notify you in a non-file system based way: any regular RPC method will be an improvement...), those are definitely worth looking into from a reliability point of view.

这篇关于System.IO.FileSystemWatcher监视网络服务器文件夹 - 性能注意事项的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！