celeryd进程的数量取决于--concurrency设置？ [英] Does the number of celeryd processes depend on the --concurrency setting?

问题描述

我们正在管理Celery，并以

开头。

  celeryd --events --loglevel = INFO --concurrency = 2 
  

但是，这会创建一个最多三层的过程图， celeryd进程（Supervisor产生一个celeryd，它产生几个其他，它再次产生进程）。我们的机器有两个CPU内核。

所有这些进程是否都在处理任务？也许有些只是工人池？

解决方案

你不应该有7个进程，如果 - concurrency 为2。

开始的实际进程是：

• 主要消费者流程

  
  
  

  代表工作到工作者集区




• 工作者池（这是 - 并发决定的数字）

这是3个并发两个进程。

此外，启动一个用于清理信号量的非常轻量级的进程
如果force_execv被启用（默认情况下，你使用一些其他运输
比redis或rabbitmq）。

一些情况下，进程列表也包括线程。
如果使用除了rabbitmq / redis之外的传输，worker可以启动多个线程，
包括一个Mediator线程，除非启用 CELERY_DISABLE_RATE_LIMITS / p>

We are running Celery behind Supervisor and start it with

celeryd --events --loglevel=INFO --concurrency=2

This, however, creates a process graph that is up to three layers deep and contains up to 7 celeryd processes (Supervisor spawns one celeryd, which spawns several others, which again spawn processes). Our machine has two CPU cores.

Are all of these processes working on tasks? Are maybe some of them just worker pools? How is the --concurrency setting connected to the number of processes actually spawned?

解决方案

You shouldn't have 7 processes if --concurrency is 2.

The actual processes started is:

• The main consumer process

  Delegates work to the worker pool

• The worker pool (this is the number that --concurrency decides)

So that is 3 processes with a concurrency of two.

In addition a very lightweight process used to clean up semaphores is started if force_execv is enabled (which it is by default i you're using some other transport than redis or rabbitmq).

NOTE that in some cases process listings also include threads. the worker may start several threads if using transports other than rabbitmq/redis, including one Mediator thread that is always started unless CELERY_DISABLE_RATE_LIMITS is enabled.

这篇关于celeryd进程的数量取决于--concurrency设置？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！