数据的异步处理 [英] Asynchronous Processing of Data

问题描述

在分钟我试图组建一个异步的TCP服务器来接收数据，然后我想处理，提取值和插入到SQL Server。

一旦数据被接收并确认为整个邮件的基本概念，我认为将是最好的，应该被传递到某种收集的信息，等待处理的先进先出的基础上，将解析值和它们插入到SQL Server。我想这是什么被称为生产者/消费者模式。

我一直在做一些寻找到这样做的最好的收集/方式迄今见过的BlockingCollection，ConcurrentCollection和BufferBlock使用异步/等待，我认为这可能是要走的路，但说实话我不确定

最好的例子，我发现是斯蒂芬·克利的博客特别是这篇文章中， http://blog.stephencleary.com/2012/11/async -producerconsumer队列，using.html

我的主要保留意见是，我绝不想放慢或中断这对我会建议使用它可以看到在上面的链接的多个生产者/消费者例子消息的接收，但我想知道的是;

• 我是正确的这个假设还是有我的情况下这样做的更合适的方式。
• 如果即时通讯在我的假设任何人都可以提出执行这一考虑我的使用情况下，最好的方法是正确的。

任何及所有的帮助是非常AP preciated。

解决方案

  

在分钟我试图组建一个异步的TCP服务器来接收数据，然后我想处理，提取值和插入到SQL Server。

有一个常见的​​陷阱有这种情形的。它通常是错误的报告成功返回给客户端时的工作还有待完成。大部分时间我已经看到了这个设计的时候，这是因为效率要求自我施加的开发，而不是由客户端或因技术原因。因此，首先，退后一步，使的绝对保证的，你想返回一个成功完成消息给客户端时，该操作尚未实际完成呢。

如果你确定这是你想做的事，然后有另外一个问题，你必须问：这是可以接受的损失的请求？也就是说，你告诉客户端，操作成功完成后，将系统仍然稳定，如果操作实际上并没有完全过？

在这个问题的答案通常是不。在这一点上，最常见的架构解决方案是有一个的外的过程的可靠队列（如Azure的队列或MSMQ），具有独立的后端（如Azure的工人角色或Win32的服务），且处理队列中的消息。这肯定是复杂的体系结构，但它是一个的需要的并发症，如果系统的必须的返回结束的消息早的和的决不能丢失消息。

在另一方面，如果丢失的消息是可以接受的，那么你可以让他们在内存中。只有在这种情况下，你可以使用我的博客中提到的内存生产者/消费者类型之一。这是一个非常罕见的情况，但它确实时有发生的时间。

At the minute I am trying to put together an asynchronous tcp server to receive data which I then want to process, extracting values and inserting to sql server.

The basic concept I thought would be best is once the data is received and confirmed as the entire message, the message should then be passed of to some sort of collection to await processing on a FIFO basis, which will parse the values and insert them to sql server. I suppose this is whats known as the consumer/producer pattern.

I have been doing some looking into the best collection / way of doing this and have so far seen the BlockingCollection,ConcurrentCollection and BufferBlock using async/await and i think this may be the way to go but to be honest im not sure.

The best example i have found is on Stephen Cleary's blog in particular this article, http://blog.stephencleary.com/2012/11/async-producerconsumer-queue-using.html

My main reservations are that I in no way want to slow down or interrupt the receiving of messages which to me would suggest using the multiple producer/consumer example which can be seen at the above link, but what i want to know is;

• Am i correct in this assumption or is there a more suitable way of doing this in my scenario.
• And if im correct in my assumption could anyone suggest the best way of implementing this taking into consideration my use case.

Any and all help is much appreciated.

解决方案

At the minute I am trying to put together an asynchronous tcp server to receive data which I then want to process, extracting values and inserting to sql server.

There's a common pitfall with this kind of scenario. It is usually wrong to report success back to the client when the work has yet to be done. Most of the time I've seen this design, it's because of an efficiency "requirement" self-imposed by the developer, not by the client or for technical reasons. So first, take a step back and make absolutely sure that you do want to return a "successful completion" message to the client when the operation has not actually completed yet.

If you are sure that's what you want to do, then there's another question you must ask: is it acceptable to lose requests? That is, after you tell the client that the operation successfully completed, will the system still be stable if the operation does not actually ever complete?

The answer to that question is usually "no." At that point, the most common architectural solution is to have an out-of-process reliable queue (such as an Azure queue or MSMQ), with an independent backend (such as an Azure worker role or Win32 service) that processes the queue messages. This definitely complicates the architecture, but it is a necessary complication if the system must return completion messages early and must not lose messages.

On the other hand, if losing messages is acceptable, then you can keep them in-memory. It is only in this case that you can use one of the in-memory producer/consumer types mentioned on my blog. This is a very rare situation, but it does happen from time to time.

这篇关于数据的异步处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！