WCF服务中传输大量数据 [英] Transfer large amount of data in WCF service

问题描述

我在 WCF 中创建了一个 Web 服务，它返回超过 54000 个数据行，每行 10 个数据.我使用 wsHttpBinding 进行通信.该服务适用于较少的数据(即 2000 行)，但在尝试发送具有 50000 多行(~2MB)的大型记录集时会爆炸.异常信息是这样的

I have created a web service in WCF which returns more than 54000 data-rows with 10 data in each row. I have used the wsHttpBinding for communication. The service works well with less data (i.e. 2000 rows) but it bombs out when attempting to send large record set with 50000+ rows(~2MB). The exception message is like this

接收对 http://localhost:9002/MyService.svc 的 HTTP 响应时出错.这可能是由于服务端点绑定未使用 HTTP 协议.这也可能是由于服务器中止了 HTTP 请求上下文(可能是由于服务关闭).有关更多详细信息，请参阅服务器日志.

An error occurred while receiving the HTTP response to http://localhost:9002/MyService.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.

请不要告诉我在客户端使用分页 - 我知道它会解决问题.但我需要客户端的全部数据.

Please don't tell me to use pagination on the client side - I know that it will solve the problem. But I need the whole chunk of data in the client-end.

我在服务器上的服务配置是这样的

My service configuration on server is as

<system.serviceModel>
  <bindings>
    <wsHttpBinding>
      <binding name="MyWsHttpBinding" />
    </wsHttpBinding>
  </bindings>
  <services>
    <service name="AdminService">
      <endpoint address="AdminSrv"
                binding="wsHttpBinding"
                contract="IAdminService"/>
      <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
      <host>
        <baseAddresses>
          <add baseAddress="/Bus/IRfotoWCF" />
        </baseAddresses>
      </host>
    </service>
  </services>
  <behaviors>
    <serviceBehaviors>
      <behavior>
        <!-- To avoid disclosing metadata information, 
                  set the value below to false and remove the metadata endpoint above before deployment -->
        <serviceMetadata httpGetEnabled="True"/>
        <!-- To receive exception details in faults for debugging purposes, 
                  set the value below to true.  Set to false before deployment 
                  to avoid disclosing exception information -->
        <serviceDebug includeExceptionDetailInFaults="True" />
      </behavior>
    </serviceBehaviors>
  </behaviors>
  <serviceHostingEnvironment multipleSiteBindingsEnabled="true"></serviceHostingEnvironment>
</system.serviceModel>

我的客户端配置是这样的

My client configuration is as

<system.serviceModel>
  <bindings>
    <basicHttpBinding>
      <binding name="BasicHttpBinding_IAdminService" closeTimeout="00:01:00"
               openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
               allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
               maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
               messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered" 
               useDefaultWebProxy="true">
        <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384" 
                      maxBytesPerRead="4096" maxNameTableCharCount="16384" />
        <security mode="None">
          <transport clientCredentialType="None" proxyCredentialType="None" realm="" />
          <message clientCredentialType="UserName" algorithmSuite="Default" />
        </security>
      </binding>
    </basicHttpBinding>
  </bindings>
  <client>
    <endpoint address="http://localhost/TestService/AdminService.svc" 
              binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IAdminService"
              contract="IAdminService" name="BasicHttpBinding_IAdminService" />
  </client>
</system.serviceModel>

有人能帮我在客户端和服务器端进行精确配置吗?即使我需要将绑定从 wsHttpBinding 更改为 netTcpBinding - 我这样做也没有问题.提前致谢.

Would someone help me out with excact configuration both in the client and server side. Even if i need to change binding from wsHttpBinding to netTcpBinding - i have no problem doing that. Thanks in advance.

推荐答案

经过大量调查最终我得到了解决方案.实际上有很多事情需要改变.

After a lot of investigation finnally i got the solution. Actually a number of things need to be changed.

需要在服务器端进行以下更改.

首先我必须在我的 httpRuntime 元素中将 maxRequestLength 设置为更大的值，以在更长的时间内运行请求.

First I had to set a maxRequestLength to a larger value in my httpRuntime element to run the request for longer period.

<system.web>    
<httpRuntime maxRequestLength="102400" />
</system.web>

第二我介绍了 netTcpBinding 绑定与对 maxBufferSize、maxBufferPoolSize、maxReceivedMessageSize 的自定义更改，其值很大 2147483647.

Second i introduced netTcpBinding binnding with custom changes on maxBufferSize, maxBufferPoolSize, maxReceivedMessageSize with a large value of 2147483647.

<binding name="myNetTcpBinding" 
maxBufferPoolSize="2147483647" 
maxBufferSize="524288" 
maxReceivedMessageSize="2147483647">

Third 在 serviceBehaviors 和 endpointBehaviors 中添加 maxItemsInObjectGraph 如下所示(不要忘记提及行为service 和 endpoint 节点中的名称)

Third add maxItemsInObjectGraph in both of the serviceBehaviors and endpointBehaviors like bellow (dont forget to mention the behaviour names in the service and endpoint node)

    <behaviors>
      <serviceBehaviors>        
        <behavior name="myNetTcpBehaviour">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name="myNetTcpEndPointBehaviour">
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </endpointBehaviors>
    </behaviors>

最后我的服务器配置如下

<system.web>    
    <httpRuntime maxRequestLength="102400" />
</system.web>


  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="MyWsHttpBinding" />
      </wsHttpBinding>
      <netTcpBinding>
        <binding name="myNetTcpBinding"
                 closeTimeout="00:01:00"
                 openTimeout="00:01:00"
                 receiveTimeout="00:10:00"
                 sendTimeout="00:01:00"
                 transactionFlow="false"
                 transferMode="Buffered"
                 transactionProtocol="OleTransactions"
                 hostNameComparisonMode="StrongWildcard"
                 listenBacklog="10"
                 maxBufferPoolSize="2147483647"
                 maxBufferSize="524288"
                 maxConnections="10"
                 maxReceivedMessageSize="2147483647">
          <readerQuotas maxDepth="32"
                        maxStringContentLength="8192"
                        maxArrayLength="16384"
                        maxBytesPerRead="4096"
                        maxNameTableCharCount="16384" />
          <reliableSession ordered="true"
                           inactivityTimeout="00:10:00"
                           enabled="false" />
          <security mode="Transport">
            <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
          </security>
        </binding>
      </netTcpBinding>
    </bindings>
    <services>
      <service name="AdminService" behaviorConfiguration="myNetTcpBehaviour">
        <endpoint address="AdminSrv" 
                  binding="netTcpBinding" 
                  bindingConfiguration="myNetTcpBinding"
                  contract="IAdminService"
                  behaviorConfiguration="myNetTcpEndPointBehaviour"/>

        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="/Bus/IRfotoWCF" />
          </baseAddresses>
        </host>
      </service>
    <behaviors>
      <serviceBehaviors>        
        <behavior name="myNetTcpBehaviour">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name="myNetTcpEndPointBehaviour">
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </endpointBehaviors>
    </behaviors>
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true"></serviceHostingEnvironment>
  </system.serviceModel>

现在在客户端配置中，您需要更改maxBufferSize="2147483647" maxBufferPoolSize="524288" maxReceivedMessageSize="2147483647"

Now on the client-side configuratioin you need to change the maxBufferSize="2147483647" maxBufferPoolSize="524288" maxReceivedMessageSize="2147483647"

并且您还需要在端点行为配置中添加maxItemsInObjectGraph="2147483647".

and also you need to add maxItemsInObjectGraph="2147483647" in endpoint behaviour configuration.

        <endpointBehaviors>
            <behavior name="myEndPointBehavior">
                <dataContractSerializer maxItemsInObjectGraph="2147483647" />
            </behavior>
        </endpointBehaviors>

现在我可以在 5.30 分钟内传输 30000 行，其中查询执行了 10 秒，所以传输时间是 5.20 分钟 - 仍然很多>.

Now i can transmit 30000 rows within 5.30 min where the query executed for 10 sec so the transmission time is 5.20 min - still a lot.

请随时发表评论和任何改进建议.

Feel free to comment and any suggestion for improvement.

这篇关于WCF服务中传输大量数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！