vb.net控制台应用程序允许multhithreading进程复制文件 [英] vb.net console application to allow multhithreading process to copy files

问题描述

您好，



我想知道是否有可能让多线程应用程序从源文件夹执行大约20GB文件的复制过程到目标文件夹。目前我有一个复制过程，一次复制1个文件，需要我的过程更快。任何人都可以通过一些代码来探索这是如何完成的并进行说明？如果可以向我展示，我真的很感激。

Hi There,

I would like to know if it is possible to have a multithread application that does a copy process of about 20GB of files from a source folder to a destination folder. At the moment I have a copy process that copies 1 file at a time and need my process to be faster. Can anyone explian how this is done and illustrate with some code? I would really be thankfull if this could be showed to me.

推荐答案

我想知道是否可以有一个执行复制过程的多线程应用程序从源文件夹到目标文件夹的大约20GB文件



- 是的。我实际上花时间写一个...调整它，以便人们可以使用它来创建批处理作业...调整每个复制操作将使用多少线程...然后微软的人通过包括使我的解决方案毫无价值在Windows 8中天真地使用此功能。您还可以使用 Robocopy ，这是一个用于复制大量文件的多线程命令行工具，可在早期版本的Windows中使用。



- Pete

"I would like to know if it is possible to have a multithread application that does a copy process of about 20GB of files from a source folder to a destination folder"

- Yes. I actually took the time to write one... tweak it so that people could use it to create batch jobs... adjust how many threads each copy operation would use... and then the guys at Microsoft made my solution worthless by including this functionality naively in Windows 8. You can also use Robocopy, which is a multithreaded command line tool for copying large amounts of files, and is available in earlier versions of windows.

- Pete

这里没有提到的是多线程复制操作可能完全浪费时间，具体取决于您的磁盘配置。



如果你正在运行一个磁盘阵列然后复制操作可能会加速一点（不要指望数量级的性能提高！）因为阵列中的各个驱动器可以同时读/写。



但是，如果这是一个磁盘，则驱动器一次只能读取或写入单个文件的一个部分。让多个线程同时尝试读取驱动器的不同部分只会在n-1个线程中等待1个线程完成其操作。



您的问题是I / O绑定，而不是计算绑定。在I / O操作中投入更多线程几乎不能提高性能。



性能更高的驱动器可以获得更大的性能提升。

What noone here has mentions is that multi-threading your copy operation may be a complete waste of time depending on your disk configuration.

If you're running a disk array then the copy operation might speed up a bit (don't expect orders of magnitude performance increases!) because individual drives in the array can be read/written at the same time.

But, if this is a single disk, the drive can only read or write one section of a single file at a time. Having multiple threads trying to read different parts of the drive at the same time will only reasult in n-1 of those threads waiting for 1 thread to complete its operation.

Your problem is I/O bound, not compute bound. Throwing more threads at an I/O operation does little to improve performance.

A bigger jump in performance is gained by higher-performance drives.

你可以采取各种方法来做到这一点，



第一个是实现Parallel.For或Parallel.ForEach将使用.Net的并行类来分发所需操作的方法。



这个链接给出了一个Parallel.ForEach的例子

Parallel.ForEach示例 [ ^ ]



还有一个并行的例子。仅查看此处 [ ^ ]



如果这不能完成您的任务，那么您将需要使用线程查看您自己的实现，您可以在其中创建线程（最多可控制的限制），并且每个线程都负责复制文件。



现在，我对此有了自己的关注，那就是你进行的复制/粘贴过程越多，它可能只会越快。这一点是磁盘I / O速度或内存空间（如在RAM中）开始瓶颈的点。



当你在Windows和你的时候开始单独复制大量文件，但同时，您应该注意到所有副本显示都会变慢。需要注意的事项。

There are various approaches you could take to do this,

The first would be to implement the "Parallel.For" or "Parallel.ForEach" approach which will use .Net's parallel class to distribute the required operation.

This link gives an example of the Parallel.ForEach
Parallel.ForEach Example[^]

There is also an example of the Parallel.For look just here[^]

If that does not do your task, then you will want to look at your own implementation using threading, where you create threads (up to a controlled limit), and each one is responsible for copying a file.

Now, I have my own concern with this, and that is that the more copy/paste processes you have going on the faster it might be only to a point. This point is the point at which the Disk I/O speed or the Memory space (as in RAM) start to bottle neck.

When you are in windows and you start copying lots of files individually but at the same time, you should notice that all the copies "appear" to slow down. Just something to be aware of.

这篇关于vb.net控制台应用程序允许multhithreading进程复制文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！