Mule ESB-设计文件相互依赖时的多文件处理流程 [英] Mule ESB - design a multi file processing flow when files are dependent on each other

问题描述

Okie ...我知道File inbound元素中的Datamapper，Batch和Streaming支持.我想知道的是以下情况下的集成设计模式:

Okie...I am aware of Datamappers, Batch and streaming support in File inbound elements. What i want to know is the design pattern for integration when:

• 您有多个文件(csv或xml)要处理，例如:1个名为products.csv的文件包含有关产品的所有详细信息.另一个文件images.csv具有指向products.csv文件中列出的每个产品的图像的URL.另一个文件说prices.csv包含每个产品的价格详细信息.
• 所有文件都使用PK类型(例如产品SKU或产品ID)相互链接.因此，products.csv中的每一行在images.csv中都有一个链接，而prices.csv中有一行.
• 您需要处理所有文件并将其保存在DB中或合并为单个XML或JSON.我的意思是说，要制造一个VO或一个实体，使产品具有图像列表并有价格.所有具有"关系都可以从产品对象/实体中导航.

人们建议如何使用Mule ESB设计它.我了解单个CSV的设计.使用批处理流程，您可以使用流式文件连接器读取文件，然后使用流式数据映射器提取数据，然后将数据转换为VO并放入DB中.这是直截了当的.在数据库插入级别或整个安装过程中在此处添加批处理提交也可以提高性能.但是，如我在方案中所述，当您有多个文件时该怎么办?

How do you folks propose to design this using Mule ESB. I know about designs for a single CSV. Using the batch flows, you read the file in using a streaming file connector, then use a streaming data mapper to extract data and then transform the data into VOs and put in DB. This is straight forward. Adding batch-commit here at DB insert level or the whole setup improves performance also. But what to do when you have multiple files as i said in my scenario?

推荐答案

已多次在StackOverflow上用不同的措词询问此问题.通常，答案是让文件入站端点从众多文件中选择一个，然后与请求者一起在流中挑选其他文件.

This has been asked on StackOverflow several times, with different wording. Usually the answer is to have a file inbound-endpoint pick one of the many files and then pick the other files down in the flow with a requester.

请参阅: https://github.com/mulesoft/mule-module-requester

在您的情况下，主文件将作为输入流可用，而图像和价格查找文件将被加载到内存中(在Maps for ex中)，以便您在处理主流时可以快速访问它们.

In your case, the main file would be available as an input stream while the images and prices lookup files would be loaded in memory (in Maps for ex) so you can access them quickly when processing the main stream.

这篇关于Mule ESB-设计文件相互依赖时的多文件处理流程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！