芹菜任务链和访问** [英] Celery Task Chain and Accessing **kwargs

问题描述

我的情况类似于此处所述的情况，除了而不是链接具有多个参数的任务，我想链接返回具有多个条目的字典的任务。

I have a situation similar to the one outlined here, except that instead of chaining tasks with multiple arguments, I want to chain tasks that return a dictionary with multiple entries.

这是-非常宽松和抽象---我正在尝试做的事情：

This is -- very loosely and abstractly --- what I'm trying to do:

任务.py

@task()
def task1(item1=None, item2=None):
  item3 = #do some stuff with item1 and item2 to yield item3
  return_object = dict(item1=item1, item2=item2, item3=item3)
  return return_object

def task2(item1=None, item2=None, item3=None):
  item4 = #do something with item1, item2, item3 to yield item4
  return_object = dict(item1=item1, item2=item2, item3=item3, item4=item4)
  return return_object

在ipython上工作，我可以调用task1单独和异步，没有问题。

Working from ipython, I am able to call task1 individually and asynchronously, with no problems.

我还可以单独调用task2，并将task1返回的结果作为双星参数：

I can ALSO call task2 individually with the result returned by task1 as a double-star argument:

>>res1 = task1.s(item1=something, item2=something_else).apply_async()
>>res1.status
'SUCCESS'
>>res2 = task2.s(**res1.result).apply_async()
>>res2.status
'SUCCESS

但是，我最终想要达到的最终结果与上述相同，但是通过一条链，在这里，我不知道如何拥有用task1返回的（位置）参数实例化task2，但是使用task1.result作为** kwargs实例化。

However, what I ultimately want achieve is the same end result as above, but via a chain, and here, I can't figure out how to have task2 instantiated not with (positional) arguments returned by task1, but with task1.result as **kwargs:

chain_result = (task1.s(item1=something, item2=something_else) | task2.s()).apply_async()  #THIS DOESN'T WORK!

我怀疑我可以返回并重写我的任务，以便它们返回位置参数而不是字典，这也许可以解决问题，但是在我看来，应该有某种方法可以使用具有** double star功能的功能访问task2中task1的返回对象。我还怀疑我在这里缺少有关Celery子任务实现或* args vs。** kwargs的明显信息。

I suspect that I can go back and rewrite my tasks so that they return positional arguments instead of a dictionary, and this may clear things up, but it seems to me that there should be some way to access task1's return object in task2 with the equivalent functionality of the **double star. I also suspect that I'm missing something here fairly obvious about either Celery subtask implementation or *args vs. **kwargs.

希望这是有道理的。并先感谢您提供任何提示。

Hope this makes sense. And thanks in advance for any tips.

推荐答案

链和其他canvas基本元素属于
功能实用程序家族，例如 map 和 reduce 。

chain and the other canvas primitives are in the family of functional utilities like map and reduce.

例如其中 map（target，items）调用列表中每个项目的 target（item），
Python有一个很少使用的地图版本，称为 itertools.starmap ，
则称为 target（* item） 。

E.g. where map(target, items) calls target(item) for every item in the list, Python has a rarely used version of map called itertools.starmap, which instead calls target(*item).

虽然我们可以添加 starchain 甚至 kwstarchain 到工具箱，这些
将会非常专业化，并且可能不会经常使用。

While we could add starchain and even kwstarchain to the toolbox, these would be very specialized and probably not used as often.

有趣的是，Python已使列表和生成器表达式$ b变得不必要了$ b，以便将地图替换为 [项目中项目的目标（item）] ，将星图替换为 [项目中项目的目标（* item） ] 。

Interestingly Python has made these unnecessary with the list and generator expressions, so that map is replaced with [target(item) for item in item] and starmap with [target(*item) for item in item].

因此，我认为我们不应该为每个原语实现几种替代方法，而应该把重点放在寻找更灵活的支持方式上这个，例如就像用芹菜驱动的发电机表达式（如果可能的话，或者如果没有类似的强大功能）

So instead of implementing several alternatives for each primitive I think we should focus on finding a more flexible way of supporting this, e.g. like having celery powered generator expressions (if possible, and if not something similarly powerful)

这篇关于芹菜任务链和访问**的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！