使用国内长途，一个人如何实现批量处理？ [英] Using DDD, How Does One Implement Batch Processing?

问题描述

我有逻辑由从一个系统中选择大量的记录，执行多个转换（基于业务规则），并将它们插入到另一个系统的

I have logic consisting of selecting a large number of records from one system, performing multiple transformations (based on business rules) and inserting them into another system.

好像打到每个实例这些记录为对象，对其执行转换，然后将所有这些对象到另一个系统中的高性能（和内存）。

It seems like a high performance (and memory) hit to instantiate each of these records as an object, perform transformations on them and then insert all of these object into the other system.

是实现这一目标的DDD跳过类/对象的最佳途径，做到直通SQL，也许一个存储过程？

Is the best way to achieve this in DDD to skip the classes/objects and do it straight through SQL, maybe a stored procedure?

有没有使用DDD实现这一目标更好的办法？

Is there a better way using DDD to achieve this goal?

注：系统采用SQL数据库，此刻对象存储喜欢的CouchDB是不是一种选择

Note: The systems use SQL databases, at the moment object stores like CouchDB are not an option.

推荐答案

很多建在DDD的分布式系统使用的是事件驱动的架构，在那里，而不是等待执行在一个批次所有转换，因为每个实体经历状态的变化，将导致其由系统进行改造，实体提出了一些发布到某种类型的消息总线的事件（如骡子用于Java，MassTransit为.NET）。您的转化体系将认购该事件，并为每个消息在您的系统到达时，它会在邮件中标识的实体进行转换，然后发布另一个消息到目标系统。

A lot of distributed systems built on DDD are using an Event-Driven Architecture, where rather than waiting to perform all the transformations in one batch, as each entity undergoes the state change that would cause it to be transformed by your system, the entity raises an event that gets published to a message bus of some kind (e.g. Mule for Java, MassTransit for .NET). Your transformation system will subscribe to this events, and as each message arrives in your system, it will perform the transformation on the entity identified in the message and then publish another message to the destination system.

这种涓流处理，它可以连续运行，终日不把你的系统上的那种负荷，将必要的工作正在盘后运行。如果你关心性能，这种体系结构可能会导致有COB 5分钟后改变了最后一个记录，其中一个批处理作业可能甚至不能够，直到凌晨3点运行（毕竟其他批处理作业有一个系统已完成）。

This kind of "trickle processing" can run continuously, all day long without putting the kind of load on your system that would necessitate the job being run after-hours. If you're concerned about performance, this kind of architecture might result in a system that has the last record transformed 5 minutes after COB, where a batch job might not even be able to run until 3 am (after all the other batch jobs have finished).

如果你真的不想在目标系统进行更新，直到午夜，例如，只是队列中的消息，直到午夜，然后将它们发布到目标系统的终结点。

If you truly don't want the target system to be updated until midnight, e.g., just queue the messages up until midnight, and then publish them to the destination system's endpoint.

Greg Young的博客已经和$ P $对这种架构的广泛psented。看看他在InfoQ中文站的工作。

Greg Young has blogged and presented extensively on this kind of architecture. Check out his work on InfoQ.

这篇关于使用国内长途，一个人如何实现批量处理？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！