Celery任务可以重新启动吗？ [英] Can Celery tasks survive restart?

问题描述

我需要构建一个处理两种类型任务的系统。一种类型可以创建更多本身的任务或另一种类型的任务。工人人数很少（2-3），只有一名主人。最重要的要求是系统应妥善处理重新启动：即重新启动时，正在进行的任务应从头开始，工作人员应选择在重新启动之前排队的任务。
看着芹菜似乎适合这种用例。但是，我有两个问题：

I need to build a system which handles two types of tasks. One type can create more tasks of itself or of the other type. There will be very few workers (2-3) and only one host. The most important requirement is that the system should handle restarts gracefully: i.e. on restart, tasks that were in progress should start from scratch and the workers should pick up tasks which were queued prior to restart. Looking at Celery it appears to be suitable for this use case. However, I have a couple of questions:

1）Celery是否能够按所述方式处理整个系统的重启？ （Celery + Worker + Broker，一切重新启动，完全断电。）

1) Is Celery able to handle restarts of the whole system as described? (Celery + workers + broker, everything restarts, complete power failure).

2）在这种情况下，最好使用哪种经纪人？我更喜欢SQLAlchemy + SQLite，因为它更独立，但是如果RabbitMQ是正确的选择，考虑到优雅地处理完全重启的要求，那么我可以使用它。

2) What is the best broker to use for this scenario? I would prefer SQLAlchemy + SQLite as it is more "self contained", but if RabbitMQ is the right choice, given the requirement to handle full restarts gracefully, then I can go with it.

推荐答案

1）Celery是否能够按所述方式处理整个系统的重启？ （Celery + Worker + Broker，一切重新启动，完全断电。）

1) Is Celery able to handle restarts of the whole system as described? (Celery + workers + broker, everything restarts, complete power failure).

是的，当您在RabbitMQ上创建自己的队列时（永久性
队列），即使服务器重新启动后，任务也会恢复为
，并由Celery执行。

Yes, When you create your own queue on RabbitMQ ( Persistent queue ) even after the restart of your server the task will be resumed and will be carried out by Celery.

2）在这种情况下最好使用什么经纪人？我宁愿使用SQLAlchemy + SQLite，因为它更独立，但是如果RabbitMQ是正确的选择，考虑到优雅地处理完全重启的要求，那么我可以使用它。

2) What is the best broker to use for this scenario? I would prefer SQLAlchemy + SQLite as it is more "self contained", but if RabbitMQ is the right choice, given the requirement to handle full restarts gracefully, then I can go with it.

使用RabbitMQ。我们正在生产中使用RabbitMQ + Celery。因此，我会建议
建议您使用RabbitMQ作为代理。

USe RabbitMQ. we are using RabbitMQ + Celery in production. So i would suggest you to use RabbitMQ as a broker.

芹菜+ RabbitMQ最佳做法

消息可靠性

注意：

1. 使用更多队列（即不仅是默认队列）


2. 使用优先级工作者


3. 使用Celery's错误处理机制

这篇关于Celery任务可以重新启动吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！