芹菜立即超过了Heroku上的内存 [英] Celery immediately exceeds memory on Heroku

问题描述

我正在将Celery进程部署到Heroku，并且每次启动时，它都会立即开始占用内存并在超过最大值后崩溃。

I'm deploying a Celery process to Heroku and every time it starts, it immediately starts to rack up memory usage and crash after it exceeds the maximum.

I只有一个名为 test_task的任务每分钟打印一次。

I only have one task called "test_task" that prints once per minute.

这是使用Celery的Django应用程序，其Redis后端托管在Heroku上。

This is Django app using Celery with a Redis backend hosted on Heroku.

Proc文件：

web: daphne chatbot.asgi:channel_layer --port $PORT --bind 0.0.0.0 --verbosity 1
chatworker: python manage.py runworker --verbosity 1
celeryworker: celery -A chatbot worker -l info

Heroku日志：

app[celeryworker.1]: [INFO] 2018-09-11 23:06:36,710 : Scheduling celery jobs...
app[celeryworker.1]: [INFO] 2018-09-11 23:06:36,880 : adding minute task
app[celeryworker.1]: 
app[celeryworker.1]: -------------- celery@f46a12fb-1666-449a-b287-2ce90f95cf2c v4.2.0 (windowlicker)
app[celeryworker.1]: --- * ***  * -- Linux-4.4.0-1027-aws-x86_64-with-debian-jessie-sid 2018-09-11 23:06:37
app[celeryworker.1]: -- * - **** ---
app[celeryworker.1]: ---- **** -----
app[celeryworker.1]: - ** ---------- [config]
app[celeryworker.1]: - ** ---------- .> app:         chatbot:0x7f1c2a5ad5d0
app[celeryworker.1]: - ** ---------- .> transport:   redis://east-1-4.ec2.cloud.redislabs.com:13275//
app[celeryworker.1]: - ** ---------- .> results:     disabled://
app[celeryworker.1]: - *** --- * --- .> concurrency: 8 (prefork)
app[celeryworker.1]: -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
app[celeryworker.1]: --- ***** -----
app[celeryworker.1]: -------------- [queues]
app[celeryworker.1]: .> celery           exchange=celery(direct) key=celery
app[celeryworker.1]: 
app[celeryworker.1]: 
app[celeryworker.1]: [tasks]
app[celeryworker.1]: . companies.tasks.test_task
app[celeryworker.1]: 
heroku[celeryworker.1]: Process running mem=616M(120.4%)
heroku[celeryworker.1]: Error R14 (Memory quota exceeded)
app[api]: Build succeeded
app[celeryworker.1]: [2018-09-11 23:06:55,033: INFO/MainProcess] Connected to redis://rediscloud:**@redis-13275.c12.us-east-1-4.ec2.cloud.redislabs.com:13275//
app[celeryworker.1]: [2018-09-11 23:06:55,201: INFO/MainProcess] mingle: searching for neighbors
app[celeryworker.1]: [2018-09-11 23:06:56,462: INFO/MainProcess] mingle: all alone
heroku[celeryworker.1]: Process running mem=1063M(207.8%)
heroku[celeryworker.1]: Error R15 (Memory quota vastly exceeded)
heroku[celeryworker.1]: Stopping process with SIGKILL
heroku[celeryworker.1]: State changed from up to crashed
heroku[celeryworker.1]: State changed from crashed to starting
heroku[celeryworker.1]: Process exited with status 137

是否有东西显然我做错了吗？

Is there something obvious I'm doing wrong?

推荐答案

原来将celery worker进程中的--concurrency参数更改为一个较小的有效值

Turns out changing the --concurrency argument in the celery worker process to a lower number worked.

我将其设置为 celery-一个聊天机器人工作者-l info --concurrency 2 工作！

这篇关于芹菜立即超过了Heroku上的内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！