提供要使用HttpListener下载的文件时性能不佳 [英] Bad performance when offering files to download with HttpListener

查看:102
本文介绍了提供要使用HttpListener下载的文件时性能不佳的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在尝试使用 C#中的 HttpListener 创建一个简单的Web服务器,并提供要下载的文件.我看到传输率真的很差,尤其是与从共享中复制相同文件相比.这是 HttpListener 已知的,可以做些什么来改进它?

I'm trying to create a simple web server using HttpListener in C# and offer files to download. I'm seeing really bad transfer rates especially compared to copying that same file from a share. Is this known to HttpListener and what can be done to improve it?

以下是有关我已完成的有关该问题的研究的一些其他信息.在本地连接时,下载速率提高了很多,但是在这种情况下,复制文件几乎是立即完成的,因此很难测量差异率.但是,在远程连接( LAN 环境,机器彼此相邻)时,传输时间大约是从共享中进行简单文件复制的时间的25倍.可用的网络带宽似乎并未用于加快速度.

Here's some additional info about research I have done about the problem. Download rates improve a lot when connecting locally but copying the file is done almost instantly in this case, so it's hard to measure a difference ratio. When connecting remotely (LAN environment, machines right next to each other) however, the transfer time is roughly 25x the time of a simple file copy from a share. The available network bandwidth doesn't seem to be used to speed this up.

我发现了有关 HttpListener 的其他问题和讨论,这些问题和讨论似乎表明存在类似的问题,请参见此处:

I've found some other questions and discussions about HttpListener that seem to indicate similar issues, see here:

HttpListener与本机性能

HttpListener性能优化(但是,这与下载无关)

HttpListener Performance Optimization (this is not regarding downloads however)

MSDN文档还指出 HttpListener 基于 http.sys ,它可以限制带宽.可能是这里发生了一些不必要的带宽限制,还是我的代码有问题?在我测试过的计算机(Windows 7和Windows 2008 R2)上,没有IIS.

MSDN docs also state that HttpListener is based on http.sys which allows for bandwidth throttling. Could it be that some unwanted bandwidth throttling is going on here or is there something wrong with my code? On the machines I've tested with (Windows 7 and Windows 2008 R2), no IIS was present.

在我的示例中,我正在像这样启动 HttpListener :

In my sample, I'm starting an HttpListener like so:

  HttpListener listener = new HttpListener();
  listener.Prefixes.Add("http://*:80/");
  listener.Start();

这是我简单文件下载的代码:

Here's the code for my simple file download:

  HttpListenerResponse response = null;
  try {
      HttpListenerContext context = listener.GetContext();

      response = context.Response;

      using( FileStream fs = File.OpenRead( @"c:\downloadsample\testfile.pdf" ) ) {

          byte[] buffer = new byte[ 32768 ];
          int read;
          while( ( read = fs.Read( buffer, 0, buffer.Length ) ) > 0 ) {
              response.OutputStream.Write( buffer, 0, read );
          }
      }

  } finally {
      if( response != null )
          response.Close();
  }

(修复了一些链接...)

(edit: fixed some links...)

推荐答案

总体

运行的两个测试(C#HttpListener提供文件和smb文件复制测试)包含太多变量,无法得出有关HttpListener与本机代码性能的任何有用结论.

Overall

The two tests that were run (C# HttpListener serving a file and an smb file copy test) contain too many variables to draw any useful conclusions about the performance of HttpListener vs. native code.

在这种情况下,所有其他代码都应怀疑会导致性能问题,并应将其从测试用例中删除.

All other code in such cases should be suspect of causing the performance problem and should be removed from the test case.

不幸的是,该问题为文件提供的实现不是最佳的,因为它从文件中将一个块读取到一个托管字节数组中,然后在调用时阻塞以将该块写入内核.它会将文件的字节复制到托管数组中,然后再从托管数组中退出(在此过程中不添加任何值).使用.Net 4.5,您可能已经在文件流和输出流之间调用了CopyToAsync,这将使您不必搞清楚如何并行执行此操作.

Unfortunately, the question's implementation of serving a file is non-optimal as it reads a chunk from the file into a managed byte array, then blocks on a call to write that block to the kernel. Its copying the bytes of the file into a managed array and back out of a managed array (adding no value in the process). With .Net 4.5 you could have called CopyToAsync between the file stream and the output stream, which would get you out of the business of figuring out how to do that in parallel.

下面的测试显示HttpListener返回字节的速度与IIS Express 8.0返回文件一样快.在这篇文章中,我在一个低俗的802.11n网络上对此服务器进行了测试,该服务器在VM上,并且通过HttpListener和IIS Express仍然达到了100+ Mbps.

The test below shows that HttpListener is just as fast at sending back bytes as IIS Express 8.0 returning a file. For this post I tested this on a cheesy 802.11n network with the servers on a VM and still reached 100+ Mbps with both HttpListener and IIS Express.

原始帖子中唯一需要更改的是它如何读取文件以将其中继回客户端.

The only thing that needs to be changed in the original post is how it reads the file to relay it back to the client.

如果要通过HTTP提供文件,则可能应该使用既处理HTTP内容又处理文件打开/缓存/中继的现有Web服务器.您会发现很难击败现有的Web服务器,特别是当您在图片中添加gzipping动态响应时(在这种情况下,幼稚的方法偶然在发送任何响应之前gzip压缩了整个响应,这浪费了本来可以浪费时间的时间)发送字节).

If you want to serve files via HTTP you should probably use an existing web server that handles both the HTTP side of things and the file opening/caching/relaying. You will find it hard to beat existing web servers, particularly when you bring gzipping dynamic responses into the picture (in that case the naive approach gzips the entire response, accidentally, before sending any of it, which wastes time that could have been used for sending bytes).

我创建了一个测试,该测试返回一个10 MB的整数字符串(在启动时生成一次),以测试HttpListener预先分配了整个块时可以返回数据的速度(类似于在执行时,它可以执行的操作)您使用CopyToAsync).

I created a test that returns a 10 MB string of integers (generated once on startup) to allow testing the speed at which HttpListener can return data when it's given the whole block up front (which is similar to what it could do when you use CopyToAsync).

客户端计算机:2013年中的MacBook Air,1.7 GHz Core i7服务器计算机:iMac 2011年中,3.4 GHz Core i7-VMWare Fusion 6.0中托管的Windows 8.1,桥接网络网络:通过Airport Extreme(距离8英尺)的802.11n下载客户端:在Mac OS X上为curl

Client Computer: MacBook Air mid-2013, 1.7 GHz Core i7 Server Computer: iMac mid-2011, 3.4 GHz Core i7 - Windows 8.1 Hosted in VMWare Fusion 6.0, Bridged Networking Network: 802.11n via Airport Extreme (located 8 feet away) Download Client: curl on Mac OS X

IIS Express 8.0配置为提供18 MB的文件,并且HttpListenerSpeed程序设置为返回10 MB和100 MB的响应.测试结果基本相同.

IIS Express 8.0 was configured to serve an 18 MB file and the HttpListenerSpeed program was setup to return 10 MB and 100 MB responses. The test results were essentially identical.

Harolds-MacBook-Air:~ huntharo$ curl -L http://iMacWin81.local:8000/TortoiseSVN-1.8.2.24708-x64-svn-1.8.3.msi > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18.4M  100 18.4M    0     0  13.1M      0  0:00:01  0:00:01 --:--:-- 13.1M

Harolds-MacBook-Air:~ huntharo$ curl -L http://iMacWin81.local:8000/TortoiseSVN-1.8.2.24708-x64-svn-1.8.3.msi > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18.4M  100 18.4M    0     0  13.0M      0  0:00:01  0:00:01 --:--:-- 13.1M

Harolds-MacBook-Air:~ huntharo$ curl -L http://iMacWin81.local:8000/TortoiseSVN-1.8.2.24708-x64-svn-1.8.3.msi > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18.4M  100 18.4M    0     0  9688k      0  0:00:01  0:00:01 --:--:-- 9737k

HttpListenerSpeed结果

Harolds-MacBook-Air:~ huntharo$ curl -L http://iMacWin81.local:8080/garbage > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18.4M  100 18.4M    0     0  12.6M      0  0:00:01  0:00:01 --:--:-- 13.1M

Harolds-MacBook-Air:~ huntharo$ curl -L http://iMacWin81.local:8080/garbage > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18.4M  100 18.4M    0     0  13.1M      0  0:00:01  0:00:01 --:--:-- 13.1M

Harolds-MacBook-Air:~ huntharo$ curl -L http://iMacWin81.local:8080/garbage > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 18.4M  100 18.4M    0     0  13.2M      0  0:00:01  0:00:01 --:--:-- 13.2M

HttpListenerSpeed代码

using System;
using System.Threading.Tasks;
using System.Net;
using System.Threading;

namespace HttpListenerSpeed
{
    class Program
    {
        static void Main(string[] args)
        {
            var listener = new Listener();

            Console.WriteLine("Press Enter to exit");
            Console.ReadLine();

            listener.Shutdown();
        }
    }

    internal class Listener
    {
        private const int RequestDispatchThreadCount = 4;
        private readonly HttpListener _httpListener = new HttpListener();
        private readonly Thread[] _requestThreads;
        private readonly byte[] _garbage;

        internal Listener()
        {
            _garbage = CreateGarbage();

            _httpListener.Prefixes.Add("http://*:8080/");
            _httpListener.Start();
            _requestThreads = new Thread[RequestDispatchThreadCount];
            for (int i = 0; i < _requestThreads.Length; i++)
            {
                _requestThreads[i] = new Thread(RequestDispatchThread);
                _requestThreads[i].Start();
            }
        }

        private static byte[] CreateGarbage()
        {
            int[] numbers = new int[2150000];

            for (int i = 0; i < numbers.Length; i++)
            {
                numbers[i] = 1000000 + i;
            }

            Shuffle(numbers);

            return System.Text.Encoding.UTF8.GetBytes(string.Join<int>(", ", numbers));
        }

        private static void Shuffle<T>(T[] array)
        {
            Random random = new Random();
            for (int i = array.Length; i > 1; i--)
            {
                // Pick random element to swap.
                int j = random.Next(i); // 0 <= j <= i-1
                // Swap.
                T tmp = array[j];
                array[j] = array[i - 1];
                array[i - 1] = tmp;
            }
        }

        private void RequestDispatchThread()
        {
            while (_httpListener.IsListening)
            {
                string url = string.Empty;

                try
                {
                    // Yeah, this blocks, but that's the whole point of this thread
                    // Note: the number of threads that are dispatching requets in no way limits the number of "open" requests that we can have
                    var context = _httpListener.GetContext();

                    // For this demo we only support GET
                    if (context.Request.HttpMethod != "GET")
                    {
                        context.Response.StatusCode = (int)HttpStatusCode.NotFound;
                        context.Response.Close();
                    }

                    // Don't care what the URL is... you're getting a bunch of garbage, and you better like it!
                    context.Response.StatusCode = (int)HttpStatusCode.OK;
                    context.Response.ContentLength64 = _garbage.Length;
                    context.Response.OutputStream.BeginWrite(_garbage, 0, _garbage.Length, result =>
                    {
                        context.Response.OutputStream.EndWrite(result);
                        context.Response.Close();
                    }, context);
                }
                catch (System.Net.HttpListenerException e)
                {
                    // Bail out - this happens on shutdown
                    return;
                }
                catch (Exception e)
                {
                    Console.WriteLine("Unexpected exception: {0}", e.Message);
                }
            }
        }

        internal void Shutdown()
        {
            if (!_httpListener.IsListening)
            {
                return;
            }

            // Stop the listener
            _httpListener.Stop();

            //  Wait for all the request threads to stop
            for (int i = 0; i < _requestThreads.Length; i++)
            {
                var thread = _requestThreads[i];
                if (thread != null) thread.Join();
                _requestThreads[i] = null;
            }
        }
    }
}

这篇关于提供要使用HttpListener下载的文件时性能不佳的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆