在celery中将线程动态添加/删除到工作池 [英] Dynamically add/remove threads to the worker pool in celery

问题描述

如何从任务中向当前的多处理池添加更多线程(并删除线程)(即celeryd是使用CELERYD_CONCURRENCY = 10运行的，但我想即时将其更改为CELERYD_CONCURRENCY = 15)?

How do I add more threads (and remove threads) to the current multiprocessing pool, from within a task (i.e. celeryd was run with CELERYD_CONCURRENCY = 10 but I want to change it on-the-fly to CELERYD_CONCURRENCY = 15)?

有一个名为

There is a function called celery.concurrency.processes.TaskPool.Pool.grow but I have no idea how to call that from a running task or whether it is the correct function to do that.

推荐答案

阅读源代码:

https://github.com/ask/celery/blob/master/celery/concurrency/processes/__init__.py

既有 grow()又有 shrink()，尽管后者似乎有点腥.

there's both grow() and shrink(), although the latter seems a tad fishy.

您需要在某个位置保留对池的引用，如果只有一个池，则将其保持全局.

you'd need to keep a reference to the pool somewhere, if you have only one pool, keep it global.

腔室张贴者:如果 multiprocessing 实际上意味着要运行多个单独的过程，那么当您尝试缩小或增长时，您可能已经处于子过程中，这显然不会工作.

caveat poster: if multiprocessing actually means running multiple separate processes, you might already be in a child process when you try to shrink or grow, and obviously that won't work.

这篇关于在celery中将线程动态添加/删除到工作池的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！