在WCF服务的数据处理保证 [英] Guaranteed processing of data in WCF service

问题描述

我有一个从SAP进程记录数以万计的进给WCF服务。服务调用需要一个的XElement作为其主要参数和处理XML在我们的数据库中更新记录。目前的意图是让WCF服务异步调用，并有服务调用与状态进行处理的每个记录发送回调用者相同的文档。

I have a WCF service that processes a feed of tens of thousands of records from SAP. The service call takes an XElement as its main parameter and processes the XML to update records in our database. The current intent is to have the WCF service be called asynchronously, and to have the service call send back to the caller the same document with statuses for each record processed.

我M也想办法解决多线程的数据处理，虽然这可能不是最终会买我什么。

I'm also looking into ways to multithread the processing of the data, though this may not end up buying me anything.

由于这可能需要一段时间，我很担心有关在WCF服务的去世会发生什么事，被重新启动等。我需要知道哪些记录我处理，我却没有，并能够完成剩余的记录处理。

Because this could take a while, I'm concerned about what will happen if the WCF services dies, gets restarted, etc. I need to know which records I've processed, and which I haven't, and be able to complete processing on the remaining records.

我已经能够拿出与状态更新每个节点最好的（我不得不这样做，反正，送返回给调用者），并保存此文件硬盘驱动器。但是，保存文件的潜在大10万次并没有真正似乎是可行的。

The best I've been able to come up with is to update each node with a status (I have to do this, anyway, to send back to the caller), and save this file to the hard drive. But saving a file that large potentially 100,000 times doesn't really seem feasible.

我可以用其他什么战略，我处理它们来追踪这些记录？

What other strategies could I use to track these records as I process them?

TIA！

詹姆斯

TIA!
James

推荐答案

也许你可以把记录（从你的XML）在数据库中第一次，也许在一个特殊的记录进行处理表。每行也可能被标记为某种方式将它们与一个特定请求相关。处理来自数据库的行。如在处理每一个，更新状态字段（对应于节点状态，你将有上的XmlElement更新）。当你完成，你既可以回去和更新XML（如果你还没有在此期间坠毁），或者你可以生成新的XML（可能是有问题的，如果你不能来回转换XML的>数据库 - > XML

Maybe you could put the records (from your XML) in your database first, maybe in a special "records to be processed" table. Each row might also be tagged with some way to correlate them with a specific request. Process the rows from the database. As you process each one, update the status field (corresponding to the node status that you would have updated on the XmlElement). When you are finished, you could either go back and update the XML (if you haven't crashed in the meantime) or you could generate new XML (could be problematic if you can't round trip the conversion XML->database->XML.

如果服务停止，它应该是相对简单的检查数据库以查找尚未处理的记录和完成处理它们。

If the service dies, it should be relatively simple to examine the database to find the records that have not been processed and finish processing them.

另外，可以编写XML文件保存到磁盘一次，保持一个表中只保存了状态字段（和一个或多个键，让你找到相应的记录数据库再次XML文件），过程的记录，当您去。完成后，通过读取状态表中的状态更新一举的XML文件中的状态字段更新数据库身份表中。

Alternatively, could write the XML file to disk once, keep a table in the database that holds ONLY the "status" field (and one or more keys to allow you to find the corresponding record in the XML file again), process the records, update the database "status" table as you go. When finished, update the status fields in the XML file in one fell swoop by reading the status from the "status" table.

此外，如果服务停止，它应该是足够简单检查状态表，看看哪些行已被处理，哪些还没有。

Again, if the service dies, it should be simple enough to examine the "status" table to see which rows have been processed and which have not.

祝你好运！

这篇关于在WCF服务的数据处理保证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！