File.Copy与手动FileStream.Write对于复制文件 [英] File.Copy vs. Manual FileStream.Write For Copying File

问题描述

我的问题是关于中复制文件的性能。我们有需要大量的文件系统到不同的位置，包括在同一网络上访问Windows共享，FTP站点，AmazonS3等围绕移动文件时，我们都是一个Windows网络上，我们可以使用逃脱媒体管理系统System.IO.File.Copy（源，目的地）复制文件。因为很多时候我们只有一个输入流（如一个MemoryStream），我们试图抽象的复制操作采取一种输入流和输出流，但我们看到一个巨大的性能下降。下面是一些code复制文件作为一个讨论点使用。

My problem is in regards file copying performance. We have a media management system that requires a lot of moving files around on the file system to different locations including windows shares on the same network, FTP sites, AmazonS3, etc. When we were all on one windows network we could get away with using System.IO.File.Copy(source, destination) to copy a file. Since many times all we have is an input Stream (like a MemoryStream), we tried abstracting the Copy operation to take an input Stream and an output Stream but we are seeing a massive performance decrease. Below is some code for copying a file to use as a discussion point.

public void Copy(System.IO.Stream inStream, string outputFilePath)
{
    int bufferSize = 1024 * 64;

    using (FileStream fileStream = new FileStream(outputFilePath, FileMode.OpenOrCreate, FileAccess.Write))
    {

        int bytesRead = -1;
        byte[] bytes = new byte[bufferSize];

        while ((bytesRead = inStream.Read(bytes, 0, bufferSize)) > 0)
        {
            fileStream.Write(bytes, 0, bytesRead);
            fileStream.Flush();
        }
    }
}

有谁知道为什么这不是File.Copy进行这么多慢？有什么我可以做，以提高性能？我只是将不得不在把特殊的逻辑来看看，如果我从一个窗户的位置复制到另一个 - 在这种情况下，我只想用File.Copy而在其他情况下，我将使用流

Does anyone know why this performs so much slower than File.Copy? Is there anything I can do to improve performance? Am I just going to have to put special logic in to see if I'm copying from one windows location to another--in which case I would just use File.Copy and in the other cases I'll use the streams?

请让我知道你在想什么，你是否需要额外的信息。我曾尝试不同的缓冲区大小，它看起来像一个64K缓冲区大小是最适合我们的小文件和256K +是我们的大的文件提供更好的缓冲区大小 - 但在这两种情况下，它的性能比File.Copy差远了（ ）。提前致谢！

Please let me know what you think and whether you need additional information. I have tried different buffer sizes and it seems like a 64k buffer size is optimal for our "small" files and 256k+ is a better buffer size for our "large" files--but in either case it performs much worse than File.Copy(). Thanks in advance!

推荐答案

File.Copy是围绕的 CopyFile Win32函数和此功能需要大量的注意力从MS船员（记得慢复制性能这个Vista的相关的线程）。

File.Copy was build around CopyFile Win32 function and this function takes lot of attention from MS crew (remember this Vista-related threads about slow copy performance).

若干线索，以提高你的方法的性能：

Several clues to improve performance of your method:

1. 像许多刚才说从循环中除去Flush方法。你不需要它。


2. 增加缓冲可能会有帮助，但仅限于文件到文件的操作，网络共享，或这将代替减缓的FTP服务器。 60 * 1024是理想的网络股，至少Vista之前。对于FTP 32K将足以在大多数情况下。


3. 提供您的缓存策略（在你的情况下，连续读取和写入）帮助操作系统，使用的FileStream构造控装置，带的 FileOptions 参数（SequentalScan）。


4. 您可以通过使用异步模式（网络到文件的情况下特别有用）加快复制，但不使用线程对于这一点，而不是使用重叠IO（的BeginRead，EndRead，BeginWrite，EndWrite在.NET），以及不要忘记的FileStream构造函数中设置异步选项（参见 FileOptions ）

1. Like many said earlier remove Flush method from your cycle. You do not need it at all.
2. Increasing buffer may help, but only on file-to-file operations, for network shares, or ftp servers this will slow down instead. 60 * 1024 is ideal for network shares, at least before vista. for ftp 32k will be enough in most cases.
3. Help os by providing your caching strategy (in your case sequential reading and writing), use FileStream constructor override with FileOptions parameter (SequentalScan).
4. You can speed up copying by using asynchronous pattern (especially useful for network-to-file cases), but do not use threads for this, instead use overlapped io (BeginRead, EndRead, BeginWrite, EndWrite in .net), and do not forget set Asynchronous option in FileStream constructor (see FileOptions)

异步复制图案的例子：

int Readed = 0;
IAsyncResult ReadResult;
IAsyncResult WriteResult;

ReadResult = sourceStream.BeginRead(ActiveBuffer, 0, ActiveBuffer.Length, null, null);
do
{
    Readed = sourceStream.EndRead(ReadResult);

    WriteResult = destStream.BeginWrite(ActiveBuffer, 0, Readed, null, null);
    WriteBuffer = ActiveBuffer;

    if (Readed > 0)
    {
      ReadResult = sourceStream.BeginRead(BackBuffer, 0, BackBuffer.Length, null, null);
      BackBuffer = Interlocked.Exchange(ref ActiveBuffer, BackBuffer);
    }

    destStream.EndWrite(WriteResult);
  }
  while (Readed > 0);

这篇关于File.Copy与手动FileStream.Write对于复制文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！