在talend中运行并行作业 [英] Running parallel jobs in talend

问题描述

我遇到的情况是，我需要并行运行五个不同的子作业.问题是，在我的选择查询中，我将获得五个不同的ID，然后对于每个特定的ID，我需要运行五个不同的作业. tparrallelize组件的问题在于，它不允许我将上下文变量传递给每个子作业，即在这种情况下为id.

I have a situation where i need to run five different child jobs in talend in parallel. Problem is that, in my select query i would be getting five different ID's and then for each particular id , i need to run five different jobs. Problem with tparrallelize component is that , it does not allow me to pass context variables to each sub job, i.e id in this particular case.

从表格限制5中选择ID； ---->同一作业的五个不同实例，其ID为不同的参数

select id from table limit 5; ----> five different instance of same job with different id as parameter

任何帮助将不胜感激

谢谢

推荐答案

我不确定我是否正确理解了您在这里做什么，但是您是否要分解每个ID并将它们存储为5个单独的上下文变量，然后每个作业都可以访问自己的上下文变量，并为其存储正确的ID，然后使用该变量.

I'm not sure if I properly understand what you're doing here but if you were to break out each of those IDs and store them as 5 separate context variables then each job could access their own context variable with the right ID stored for each of them and use that.

因此，我将从数据库输入组件开始(只需选择所需的ID)，然后将其输入tFlowToIterate.通过迭代流将其连接到tFixedFlowInput组件，并在架构中创建2个字段键"和值".使用内联表指定键"应为((Integer)globalMap.get("tFlowToIterate_1_CURRENT_ITERATION"))，值"应为((String)globalMap.get("row1.SupplierPartNumber")).

So I would start with your database input component (just select the IDs you want) and feed that into a tFlowToIterate. Connect this via an iterate flow into a tFixedFlowInput component and create 2 fields in your schema, "key" and "value". Use the inline table to specify that "key" should be ((Integer)globalMap.get("tFlowToIterate_1_CURRENT_ITERATION")) and "value" should be ((String)globalMap.get("row1.SupplierPartNumber")).

然后将其放入tMap组件中，在其中将"ContextNumber" + row2.key放入映射的键列中，以使其比您的上下文中的迭代编号更明显，然后将其直接输入到tContextLoad.

I'd then throw this into a tMap component where I'd put "ContextNumber" + row2.key into the mapped key column just to make it a bit more obvious than the iteration number as your context and then feed that directly into a tContextLoad.

从那里，您可以将OnSubjobOK连接到tParallelize组件，并将所有作业链接在一起.在每个作业中，将作业配置为使用适当的上下文变量.

From there you can OnSubjobOK to your tParallelize component and link all your jobs together. In each job configure the jobs to use the appropriate context variable.

这篇关于在talend中运行并行作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！