如何运行长期(无限)Python 进程? [英] How do I run long term (infinite) Python processes?

问题描述

我最近开始尝试使用 Python 进行 Web 开发.到目前为止，我已经在使用带有 mod_wsgi 的 Apache 和 Python 2.7 的 Django Web 框架方面取得了一些成功.但是，我遇到了一些问题，比如让进程不断运行、更新信息等.

I've recently started experimenting with using Python for web development. So far I've had some success using Apache with mod_wsgi and the Django web framework for Python 2.7. However I have run into some issues with having processes constantly running, updating information and such.

我编写了一个名为daemonManager.py"的脚本，它可以启动和停止所有或单个 python 更新循环(我应该称它们为守护进程吗?).它通过分叉来实现，然后为它应该运行的特定功能加载模块并开始无限循环.它在 /var/run 中保存一个 PID 文件以跟踪进程.到现在为止还挺好.我遇到的问题是:

I have written a script I call "daemonManager.py" that can start and stop all or individual python update loops (Should I call them Daemons?). It does that by forking, then loading the module for the specific functions it should run and starting an infinite loop. It saves a PID file in /var/run to keep track of the process. So far so good. The problems I've encountered are:

• 有时其中一个进程会退出.我早上检查了 ps，过程就结束了.没有记录错误(我正在使用 logging 模块)，并且我涵盖了我能想到的所有异常并记录它们.此外，我认为这些退出进程与我的代码没有任何关系，因为我所有的进程都运行完全不同的代码并以非常相似的间隔退出.我当然可能是错的.Python 进程在运行数天/数周后就死掉是正常的吗?我应该如何解决这个问题?我是否应该编写另一个守护进程来定期检查其他守护进程是否仍在运行?如果该守护进程停止怎么办?我不知道如何处理这个问题.

• Now and then one of the processes will just quit. I check ps in the morning and the process is just gone. No errors were logged (I'm using the logging module), and I'm covering every exception I can think of and logging them. Also I don't think these quitting processes has anything to do with my code, because all my processes run completely different code and exit at pretty similar intervals. I could be wrong of course. Is it normal for Python processes to just die after they've run for days/weeks? How should I tackle this problem? Should I write another daemon that periodically checks if the other daemons are still running? What if that daemon stops? I'm at a loss on how to handle this.

如何以编程方式知道进程是否仍在运行?我将 PID 文件保存在 /var/run 中并检查 PID 文件是否存在以确定进程是否正在运行.但是，如果该进程因意外原因而终止，则 PID 文件将保留.因此，每次进程崩溃(每周几次)时，我都必须删除这些文件，这有点违背了目的.我想我可以检查一个进程是否在文件中的 PID 处运行，但是如果另一个进程已经启动并被分配了死进程的 PID 呢?我的守护进程会认为该进程运行良好，即使它已经死了很久.我又一次不知该如何处理这个问题.

How can I programmatically know if a process is still running or not? I'm saving the PID files in /var/run and checking if the PID file is there to determine whether or not the process is running. But if the process just dies of unexpected causes, the PID file will remain. I therefore have to delete these files every time a process crashes (a couple of times per week), which sort of defeats the purpose. I guess I could check if a process is running at the PID in the file, but what if another process has started and was assigned the PID of the dead process? My daemon would think that the process is running fine even if it's long dead. Again I'm at a loss just how to deal with this.

任何关于如何最好运行无限 Python 进程的有用答案，希望也能说明上述问题，我会接受

Any useful answer on how to best run infinite Python processes, hopefully also shedding some light on the above problems, I will accept

我在 Ubuntu 机器上使用 Apache 2.2.14.
我的 Python 版本是 2.7.2

I'm using Apache 2.2.14 on an Ubuntu machine.
My Python version is 2.7.2

推荐答案

我将首先声明这是管理长时间运行的进程 (LRP) 的一种方法——实际上并非由任何拉伸.

I'll open by stating that this is one way to manage a long running process (LRP) -- not de facto by any stretch.

根据我的经验，最好的产品来自于专注于您正在处理的特定问题，同时将支持技术委托给其他图书馆.在这种情况下，我指的是后台进程的行为(双叉的艺术)、监控和日志重定向.

In my experience, the best possible product comes from concentrating on the specific problem you're dealing with, while delegating supporting tech to other libraries. In this case, I'm referring to the act of backgrounding processes (the art of the double fork), monitoring, and log redirection.

我最喜欢的解决方案是http://supervisord.org/

使用像 supervisord 这样的系统，您基本上可以编写一个传统的 Python 脚本，该脚本在陷入无限"循环时执行任务.

Using a system like supervisord, you basically write a conventional python script that performs a task while stuck in an "infinite" loop.

#!/usr/bin/python

import sys
import time

def main_loop():
    while 1:
        # do your stuff...
        time.sleep(0.1)

if __name__ == '__main__':
    try:
        main_loop()
    except KeyboardInterrupt:
        print >> sys.stderr, '\nExiting by user request.\n'
        sys.exit(0)

以这种方式编写脚本使开发和调试变得简单方便(您可以轻松地在终端中启动/停止它，随着事件的展开观察日志输出).当需要投入生产时，您只需定义一个调用脚本的主管配置(这是定义程序"的完整示例，其中大部分是可选的:http://supervisord.org/configuration.html#program-x-section-example).

Writing your script this way makes it simple and convenient to develop and debug (you can easily start/stop it in a terminal, watching the log output as events unfold). When it comes time to throw into production, you simply define a supervisor config that calls your script (here's the full example for defining a "program", much of which is optional: http://supervisord.org/configuration.html#program-x-section-example).

Supervisor有一堆的配置选项，我就不一一列举了，但我会说它专门解决了你描述的问题:

Supervisor has a bunch of configuration options so I won't enumerate them, but I will say that it specifically solves the problems you describe:

• 背景/守护
• PID 跟踪(可以配置为在进程意外终止时重新启动进程)
• 在您的脚本中正常登录(如果使用日志记录模块而不是打印，则为流处理程序)，但让主管为您重定向到一个文件.

这篇关于如何运行长期(无限)Python 进程?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！