Django/Python中的Celery任务最佳做法 [英] Celery task best practices in Django/Python

问题描述

考虑一个需要繁重工作才能异步完成的功能.呼叫客户端可以立即获取缓存的版本，也可以获取数字正在处理的响应(对客户端有效的响应).

Consider a function that requires some heavy lifting to be done asynchronously. Calling clients can either recive a cached version immediately, or recive a response that the numbers are being crunched (a valid response for the clients).

以下代码段是否是该模式的合理实现?

Is the following snippet a sound implementation for this pattern?

from django.core import cache
from proj.celery import app

class SomeModel(models.Model):
    # ...
    def get_crunched_numbers(self):
        cache_key = 'foo:{}'.format(self.id)
        res = cache.get(cache_key)
        if not res:
            @app.task
            def gen_crunched_numbers():
                res = do_heavy_lifting()
                cache.set(cache_key, res)
                return res
            gen_crunched_numbers.delay()
            return 'crunching... come back later'
        else:
            return res

在将所有逻辑包含在一个代码段中的同时，还有更好的替代方法来运行Celery任务吗?

Are there better alternatives to running Celery tasks like so, while containing all the logic in a single piece of code?

编辑:如注释中所述，该代码甚至无法正常工作.因此，关于好的模式的任何建议都是必须的.

Edit: as mentioned in the comments, this code doesn't even work. So any suggestions on a nice pattern are much obliged.

推荐答案

您的代码看起来很混乱.为什么不在课堂之外定义celery任务函数，并这样调用它:

Your code looks very confusing. Why don't you define celery task function outside of your class and call it like this:

from django.core import cache
from proj.celery import app

class SomeModel(models.Model):
    # ...
    def get_crunched_numbers(self):
        cache_key = 'foo:{}'.format(self.id)
        res = cache.get(cache_key)
        if not res:
            gen_crunched_numbers.delay(cache_key)
            return 'crunching... come back later'
        else:
            return res

@app.task
def gen_crunched_numbers(cache_key):
    res = do_heavy_lifting()
    cache.set(cache_key, res)
    return res

我通常也更喜欢使用 bind = True 创建任务:

Also I usually prefer to create tasks with bind=True:

@app.task(bind=True)
def gen_crunched_numbers(self, cache_key):
    res = do_heavy_lifting()
    cache.set(cache_key, res)
    return res

我可以访问任务上下文通过 self.request .例如，根据是通过celery还是直接调用函数来更改行为:

Which give me access to task context through self.request. For example to change behavior depending if function is called through celery or directly:

@app.task(bind=True)
def gen_crunched_numbers(self, cache_key):
    res = do_heavy_lifting()
    cache.set(cache_key, res)
    if self.request.called_directly:
        return res
    else:
        return { 'result': res, 'cache': cache_key }

这篇关于Django/Python中的Celery任务最佳做法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！