WCF 在自托管服务上流式传输大数据 (500MB/1GB) [英] WCF Streaming large data (500MB / 1GB) on a self hosted service
问题描述
我目前在尝试使用 WCF 自托管服务(无 IIS)发送大数据时遇到问题.使用流结果传输 500MB，我的服务因 System.OutOfMemoryException 崩溃.有没有可能传输这么大的数据量?
I'm currently experiencing an issue trying to send large data using WCF self hosted service (no IIS). Transferring 500MB using streaming results with my service crashing with System.OutOfMemoryException. Is it possible at all to transfer such amount of data?
这是我的 WCF 配置:
Here is my WCF configuration:
<system.serviceModel>
<services>
<service name="CIService" behaviorConfiguration="CIBehavior">
<host>
<baseAddresses>
<add baseAddress="net.tcp://localhost:6547/CIService/CIService.svc" />
</baseAddresses>
</host>
<endpoint binding="netTcpBinding"
bindingConfiguration="netTcpBindingConfig"
behaviorConfiguration="CIBehavior.EndpointBehavior"
contract="CIService.ICreatable" />
<endpoint address="mex"
binding="mexHttpBinding"
name="mexTcpBinding"
contract="IMetadataExchange" />
</service>
</services>
<serviceHostingEnvironment multippleSiteBindingEnabled="True" />
<bindings>
<netTcpBinding>
<binding name="netTcpBindingConfig" closeTimeout="00:01:00" openTimeout="00:01:00"
receiveTimeout="01:00:00" sendTimeout="00:10:00"
hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxConnections="10"
maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
transferMode="Streamed">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647"
maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
</binding>
</netTcpBinding>
</bindings>
<behaviors>
<serviceBehaviors>
<behavior name="CIBehavior">
<serviceMetadata httpGetEnabled="False" />
<serviceDebug includeExceptionDetailInFaults="true" />
<serviceThrottling maxConcurrentCalls="200" maxConcurrentInstances="2147483647" maxConcurrentSessions="100" />
<dataContractSerializer maxItemsInObjectGraph="2147483647"/>
</behavior>
</serviceBehaviors>
<endpointBehavior>
<behavior name="CIBehavior.EndpointBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehavior>
</behaviors>
</system.serviceModel>
我的客户端配置:
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_ICreatable"
closeTimeout="00:01:00" openTimeout="00:01:00"
receiveTimeout="01:00:00" sendTimeout="00:10:00"
transactionFlow="false"
transferMode="Streamed"
transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard"
listenBacklog="10"
maxBufferPoolSize="2147483647"
maxBufferSize="2147483647"
maxConnections="10"
maxReceivedMessageSize ="2147483647">
<readerQuotas
maxDepth="2147483647"
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647" />
<reliableSession ordered="true" inactivityTimeout="00:10:00" anabled="false" />
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint name="NetTcpBinding_ICreatable"
address="net.tcp://localhost:6547/CIService/CIService.svc"
binding="netTcpBinding"
bindingConfiguration="NetTcpBinding_ICreatable"
behaviorConfiguration="CIBehavior.EndpointBehavior"
contract="ICreatable" />
</client>
<behaviors>
<endpointBehavior>
<behavior name="CIBehavior.EndpointBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehavior>
</behaviors>
</system.serviceModel>
推荐答案
您不需要将 maxBufferSize 或 maxBufferPoolSize 设置得太高，这些可能会导致您的内存不足异常.默认值应该没问题.
You dont need maxBufferSize or maxBufferPoolSize set so high these are possibly causing your out of memory exception. The defaults should be fine.
查看大数据和流媒体 在 MSDN 上，特别是大数据的特殊安全注意事项这一段很重要
Check out Large Data and Streaming on MSDN, specifically the section Special Security Considerations for Large Data this piece of the text is important
需要 MaxBufferSize 属性来约束内存WCF 缓冲区.将其设置为安全值(或保留它在默认值)流式传输时.例如，假设您的服务必须接收最大 4 GB 的文件并将它们存储在本地磁盘.还假设你的记忆力受到这样的限制一次只能缓冲 64 KB 的数据.然后你会设置MaxReceivedMessageSize 为 4 GB，MaxBufferSize 为 64 KB.还，在您的服务实现中，您必须确保您只读从 64 KB 块的传入流中读取下一个前一个之前的块已写入磁盘并被丢弃凭记忆.
The MaxBufferSize property is required to constrain the memory that WCF buffers. It is important to set this to a safe value (or keep it at the default value) when streaming. For example, suppose your service must receive files up to 4 GB in size and store them on the local disk. Suppose also that your memory is constrained in such a way that you can only buffer 64 KB of data at a time. Then you would set the MaxReceivedMessageSize to 4 GB and MaxBufferSize to 64 KB. Also, in your service implementation, you must ensure that you read only from the incoming stream in 64-KB chunks and do not read the next chunk before the previous one has been written to disk and discarded from memory.
我整理了一个非常简单的示例，将数据从自托管服务流式传输到控制台客户端.为了保持帖子简短，我只添加了服务器代码和部分客户端.
I put together a very simple example of streaming data from a self hosted service to a console client. To keep the post short I only added the server code and part of the client.
服务合同
using System.IO;
using System.ServiceModel;
namespace Service
{
[ServiceContract]
public interface IStream
{
[OperationContract]
Stream GetLargeObject();
}
}
服务实现
using System;
using System.IO;
using System.ServiceModel;
namespace Service
{
[ServiceBehavior]
public class StreamService : IStream
{
public Stream GetLargeObject()
{
// Add path to a big file, this one is 2.5 gb
string filePath = Path.Combine(Environment.CurrentDirectory, "C:\Temp\BFBC2_PC_Client_R11_795745_Patch.exe");
try
{
FileStream imageFile = File.OpenRead(filePath);
return imageFile;
}
catch (IOException ex)
{
Console.WriteLine(String.Format("An exception was thrown while trying to open file {0}", filePath));
Console.WriteLine("Exception is: ");
Console.WriteLine(ex.ToString());
throw;
}
}
}
}
服务主体
using System;
using System.ServiceModel;
namespace Service
{
class Program
{
static void Main(string[] args)
{
try
{
using (var serviceHost = new ServiceHost(typeof(StreamService)))
{
serviceHost.Open();
Console.WriteLine("Press Any Key to end");
Console.ReadKey();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
}
服务 app.config
The service app.config
<?xml version="1.0"?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="StreamServiceBehavior">
<serviceMetadata httpGetEnabled="True" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="NewBinding0" transferMode="Streamed"/>
</netTcpBinding>
</bindings>
<services>
<service behaviorConfiguration="StreamServiceBehavior" name="Service.StreamService">
<endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
bindingConfiguration="NewBinding0" bindingName="" contract="Service.IStream" />
<endpoint address="mex" binding="mexHttpBinding"
contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://localhost:8080/StreamService" />
</baseAddresses>
</host>
</service>
</services>
</system.serviceModel>
</configuration>
启动服务，可能需要在admin账户下运行才能打开socket.创建客户端控制台应用程序并使用 url http://localhost:8080/StreamService 添加服务引用，使用 Service 作为生成客户端的命名空间.
Launch the service, may need to run under admin account to open the socket. Create a client console application and add a service reference using the url http:// localhost:8080 / StreamService, using Service as the namespace for the generated client.
客户端主
using System;
using System.IO;
using Client.Service;
namespace Client
{
class Program
{
static void Main(string[] args)
{
try
{
using (StreamClient streamClient = new StreamClient())
{
streamClient.Open();
using (FileStream fileStream = new FileStream("c:\temp\bigfile.exe",FileMode.Create))
{
streamClient.GetLargeObject().CopyTo(fileStream);
}
}
Console.WriteLine("Press any key to end");
Console.ReadKey();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
}
生成的客户端配置文件需要稍作修改，增加receiveTimeout并设置maxReceivedMessageSize="4294967295"
The generated client config file will need to be modified slightly, increase receiveTimeout and set maxReceivedMessageSize="4294967295"
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_IStream" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:01:00"
transactionFlow="false" transferMode="Streamed" transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard" listenBacklog="10"
maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="10"
maxReceivedMessageSize="4294967295">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
<message clientCredentialType="Windows" />
</security>
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
bindingConfiguration="NetTcpBinding_IStream" contract="Service.IStream"
name="NetTcpBinding_IStream">
</endpoint>
</client>
</system.serviceModel>
启动服务，然后启动客户端.它可以毫无问题地流式传输大文件.
Launch the service then the client. It will stream a large file without issue.
这篇关于WCF 在自托管服务上流式传输大数据 (500MB/1GB)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
