带烧瓶的后台工作人员 [英] Background Worker with Flask

问题描述

我有一个基于 python/Flask 的 web 应用程序，它有一个相应的后台作业，它持续运行，定期为每个注册用户轮询数据.

I have a webapp that's built on python/Flask and it has a corresponding background job that runs continuously, periodically polling for data for each registered user.

我希望这个后台作业在系统启动时启动并一直运行直到它关闭.我没有设置/etc/rc.d 脚本，而是在应用程序启动时让 Flask 应用程序生成一个新进程(使用多处理模块).

I would like this background job to start when the system starts and keep running til it shuts down. Instead of setting up /etc/rc.d scripts, I just had the flask app spawn a new process (using the multiprocessing module) when the app starts up.

因此，通过此设置，我只需部署 Flask 应用程序，这也将使后台工作程序运行.

So with this setup, I only have to deploy the Flask app and that will get the background worker running as well.

这有什么缺点?这是一个完整而彻底的 hack，在某种程度上是脆弱的，还是设置具有相应后台任务的 web 应用程序的好方法?

What are the downsides of this? Is this a complete and utter hack that is fragile in some way or a nice way to set up a webapp with corresponding background task?

推荐答案

您的方法的缺点是它可能会以多种方式失败，尤其是在停止和重新启动 Flask 应用程序方面.

The downside of your approach is that there are many ways it could fail especially around stopping and restarting your flask application.

• 您将不得不处理正常关机的问题，让您的工作人员有机会完成当前的任务.
• 有时您的工作人员不会按时停止，并且可能会在您重新启动 Flask 应用程序时启动另一个工作人员时逗留.

以下是我建议的一些方法，具体取决于您的限制条件:

Here are some approches I would suggest depending on your constraints:

你只需要编写一个脚本来完成你想要的任何任务，cron 会每隔几分钟为你运行一次.优点:cron 会定期为你运行，并在系统启动时启动.缺点:如果任务耗时太长，您可能会同时运行多个脚本实例.您可以为这个问题找到一些解决方案 这里.

You only have to write a script that does whatever task you want and cron will take care of running it for you every few minutes. Advantages: cron will run it for you periodically and will start when the system starts. Disadvantages: if the task takes too long, you might have multiple instances of your script running at the same time. You can find some solutions for this problem here.

supervisord 是一种处理不同守护进程的巧妙方法.您可以将其设置为运行您的应用程序、后台脚本或 两者 和让他们从服务器开始.唯一的缺点是你必须安装 supervisord 并确保它的守护进程在服务器启动时正在运行.

supervisord is a neat way to deal with different daemons. You can set it to run your app, your background script or both and have them start with the server. Only downside is that you have to install supervisord and make sure its daemon is running when the server starts.

uwsgi 是部署flask 应用程序的一种非常常见的方式.它几乎没有功能，可用于管理后台工作人员.

uwsgi is a very common way for deploying flask applications. It has few features that might come in handy for managing background workers.

Celery 是一个基于分布式消息传递的异步任务队列/作业队列.它专注于实时操作，但也支持调度.我认为这是为烧瓶应用程序或任何其他基于 python 的应用程序安排后台任务的最佳解决方案.但是使用它会带来一些额外的负担.您将至少介绍以下流程:- 经纪人(rabbitmq 或 redis)- 一位工人- 调度器

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. I think this is the best solution for scheduling background tasks for a flask application or any other python based application. But using it comes with some extra bulk. You will be introducing at least the following processes: - a broker (rabbitmq or redis) - a worker - a scheduler

您还可以让 supervisord 管理上述所有进程，并在服务器启动时让它们启动.

You can also get supervisord to manage all of the processes above and get them to start when the server starts.

在您寻求减少进程数量的过程中，我强烈建议您使用基于 crontab 的解决方案，因为它可以让您走得更远.但请确保您的后台脚本留下执行跟踪或某种类型的日志.

In your quest of reducing the number of processes, I would highly suggest the crontab based solution as it can get you a long way. But please make sure your background script leaves an execution trace or logs of some sort.

这篇关于带烧瓶的后台工作人员的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！