芹菜-一秒钟完成一项任务 [英] Celery - one task in one second

问题描述

我使用Celery向服务器发出请求(在任务中).我有硬限制-一秒钟内只有1个请求(来自一个ip).我阅读了此，所以它是我想要的-1/s.在 celeryconfig.py 中，我有:

I use Celery to make requests to the server (in tasks). I have hard limit - only 1 request in one second (from one ip). I read this, so its what I want - 1/s. In celeryconfig.py I have:

CELERY_DISABLE_RATE_LIMITS = False

CELERY_DEFAULT_RATE_LIMIT = "1/s"

但是我收到消息，每秒请求太多.

But I have the messages, that I have too many requests per second.

在 call.py 中，我使用我认为 rate_limits 不起作用，因为我在 celeryconfig.py 中犯了一个错误.

I think, rate_limits does not work, because I have a mistake in celeryconfig.py.

该如何解决?谢谢！

推荐答案

使用

celery -A your_app worker -l info

默认并发性等于您的计算机具有的内核数.因此，即使您将速率限制设置为"1/s"，它也会尝试同时处理多个任务.

the default concurrency is equal to the number of the cores your machine has. So,eventhough you set a rate limit of '1/s', it is trying to process multiple tasks concurrently.

还要在celery_config中设置 rate_limit 是一个坏主意.现在您只有一项任务，如果您向应用程序添加新任务，则速率限制将相互影响.

Also setting a rate_limit in celery_config is a bad idea. Now you have only one task, if you add new tasks to your app, the rate limits will affect each other.

这是每秒完成一项任务的一种简单方法.

A simple way to achieve your one task per one second is this.

tasks.py

import time

from  celery  import  Celery
app = Celery('tasks', backend='amqp', broker='amqp://guest@localhost//')

@app.task()
def task1():
    time.sleep(1)
    return('task1')

现在以并发数

celery -A my_taks.py worker -l info -c 1

这将每秒仅执行一个任务.这是上面代码的日志.

This will execute only one task per second. Here is my log with the above code.

[2014-10-13 19:27:41,158: INFO/MainProcess] Received task: task1[209008d6-bb9d-4ce0-80d4-9b6c068b770e]
[2014-10-13 19:27:41,161: INFO/MainProcess] Received task: task1[83dc18e0-22ec-4b2d-940a-8b62006e31cd]
[2014-10-13 19:27:41,168: INFO/MainProcess] Received task: task1[e1b25558-0bb2-405a-8009-a7b58bbfa4e1]
[2014-10-13 19:27:41,171: INFO/MainProcess] Received task: task1[2d864be0-c969-4c52-8a57-31dbd11eb2d8]
[2014-10-13 19:27:42,335: INFO/MainProcess] Task task1[209008d6-bb9d-4ce0-80d4-9b6c068b770e] succeeded in 1.170940883s: 'task1'
[2014-10-13 19:27:43,457: INFO/MainProcess] Task task1[83dc18e0-22ec-4b2d-940a-8b62006e31cd] succeeded in 1.119711205s: 'task1'
[2014-10-13 19:27:44,605: INFO/MainProcess] Task task1[e1b25558-0bb2-405a-8009-a7b58bbfa4e1] succeeded in 1.1454614s: 'task1'
[2014-10-13 19:27:45,726: INFO/MainProcess] Task task1[2d864be0-c969-4c52-8a57-31dbd11eb2d8] succeeded in 1.119111023s: 'task1'

这篇关于芹菜-一秒钟完成一项任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！