可以一个python脚本知道同一个脚本的另一个实例正在运行...然后与它谈话？ [英] can a python script know that another instance of the same script is running... and then talk to it?

问题描述

我想要防止同一个长时间运行的python命令行脚本的多个实例同时运行，我希望新实例能够在新的实例之前将数据发送到原始实例实例自杀。我如何以跨平台的方式做这个？

I'd like to prevent multiple instances of the same long-running python command-line script from running at the same time, and I'd like the new instance to be able to send data to the original instance before the new instance commits suicide. How can I do this in a cross-platform way?

具体来说，我想启用以下行为：

Specifically, I'd like to enable the following behavior:

1. foo.py 从命令行启动，并且将保持运行很长时间 - 几天或几周


2. 每隔几分钟相同的脚本重新启动，但使用不同的命令行参数




4. 如果其他实例正在运行，则实例＃2应将其命令行参数发送到实例＃1，然后实例＃2应该退出。


5. 实例＃1，如果从另一个脚本接收命令行参数，应该启动一个新的线程，并使用在步骤中发送的命令行参数上面）开始执行＃2实例的工作。

1. "foo.py" is launched from the command line, and it will stay running for a long time-- days or weeks until the machine is rebooted or the parent process kills it.
2. every few minutes the same script is launched again, but with different command-line parameters
3. when launched, the script should see if any other instances are running.
4. if other instances are running, then instance #2 should send its command-line parameters to instance #1, and then instance #2 should exit.
5. instance #1, if it receives command-line parameters from another script, should spin up a new thread and (using the command-line parameters sent in the step above) start performing the work that instance #2 was going to perform.

所以我在寻找两件事： python程序知道自己的另一个实例正在运行，然后一个python命令行程序如何与另一个程序通信？

So I'm looking for two things: how can a python program know another instance of itself is running, and then how can one python command-line program communicate with another?

使这更复杂，同一个脚本需要运行在Windows和Linux上，所以理想情况下解决方案将只使用Python标准库，而不是任何特定于操作系统的调用。虽然如果我需要一个Windows编码路径和一个* nix编码路径（和一个大的 if 语句在我的代码中选择一个或另一个）代码解决方案是不可能的。

Making this more complicated, the same script needs to run on both Windows and Linux, so ideally the solution would use only the Python standard library and not any OS-specific calls. Although if I need to have a Windows codepath and an *nix codepath (and a big if statement in my code to choose one or the other), that's OK if a "same code" solution isn't possible.

我意识到我可能制定出一个基于文件的方法（例如实例＃1监视一个目录的变化，文件到该目录，当它想要做的工作），但我有点关心在非正常的机器关闭后清理这些文件。我理想地能够使用内存中的解决方案。但是我又很灵活，如果基于持久性文件的方法是唯一的方法，我可以选择。

I realize I could probably work out a file-based approach (e.g. instance #1 watches a directory for changes and each instance drops a file into that directory when it wants to do work) but I'm a little concerned about cleaning up those files after a non-graceful machine shutdown. I'd ideally be able to use an in-memory solution. But again I'm flexible, if a persistent-file-based approach is the only way to do it, I'm open to that option.

更多详细信息：I因为我们的服务器正在使用一个监视工具，该工具支持运行python脚本来收集监视数据（例如数据库查询或Web服务调用的结果），然后监视工具索引供以后使用。其中一些脚本启动非常昂贵，但在启动后运行起来很便宜（例如进行数据库连接而不是运行查询）。因此，我们选择让它们在无限循环中运行，直到父进程杀死它们。

More details: I'm trying to do this because our servers are using a monitoring tool which supports running python scripts to collect monitoring data (e.g. results of a database query or web service call) which the monitoring tool then indexes for later use. Some of these scripts are very expensive to start up but cheap to run after startup (e.g. making a DB connection vs. running a query). So we've chosen to keep them running in an infinite loop until the parent process kills them.

这样做效果很好，但是在较大的服务器上，同一脚本的100个实例可能正在运行，即使它们每20分钟只收集一次数据。这会对RAM，DB连接限制等造成严重破坏。我们希望从100个线程切换到具有100个线程的一个进程，每个线程执行之前一个脚本正在执行的工作。

This works great, but on larger servers 100 instances of the same script may be running, even if they're only gathering data every 20 minutes each. This wreaks havoc with RAM, DB connection limits, etc. We want to switch from 100 processes with 1 thread to one process with 100 threads, each executing the work that, previously, one script was doing.

但是改变监视工具调用脚本的方式是不可能的。我们需要保持调用相同（启动具有不同命令行参数的进程），但是更改脚本以识别另一个是活动的，并且使新脚本发送其工作指令（从命令行参数）到旧脚本。

But changing how the scripts are invoked by the monitoring tool is not possible. We need to keep invocation the same (launch a process with different command-line parameters) but but change the scripts to recognize that another one is active, and have the "new" script send its work instructions (from the command line params) over to the "old" script.

BTW，这不是我想在一个脚本的基础上做的。相反，我想将此行为封装到许多脚本作者可以利用的库中，我的目标是使脚本作者能够编写不知道多实例问题的简单的单线程脚本，并处理多线程

BTW, this is not something I want to do on a one-script basis. Instead, I want to package this behavior into a library which many script authors can leverage-- my goal is to enable script authors to write simple, single-threaded scripts which are unaware of multi-instance issues, and to handle the multi-threading and single-instancing under the covers.

推荐答案

Alex Martelli建立沟通渠道的方法是适当的。我将使用multiprocessing.connection.Listener创建一个监听器，在你选择。文档：
http://docs.python.org /library/multiprocessing.html#multiprocessing-listeners-clients

The Alex Martelli approach of setting up a communications channel is the appropriate one. I would use a multiprocessing.connection.Listener to create a listener, in your choice. Documentation at: http://docs.python.org/library/multiprocessing.html#multiprocessing-listeners-clients

您可以选择使用AF_UNIX for Linux和AF_PIPE for Windows，而不使用AF_INET（套接字） 。希望一个小的如果不会伤害。

Rather than using AF_INET (sockets) you may elect to use AF_UNIX for Linux and AF_PIPE for Windows. Hopefully a small "if" wouldn't hurt.

编辑：我想一个例子不会伤害。这是一个基本的。

Edit: I guess an example wouldn't hurt. It is a basic one, though.

#!/usr/bin/env python

from multiprocessing.connection import Listener, Client
import socket
from array import array
from sys import argv

def myloop(address):
    try:
        listener = Listener(*address)
        conn = listener.accept()
        serve(conn)
    except socket.error, e:
        conn = Client(*address)
        conn.send('this is a client')
        conn.send('close')

def serve(conn):
    while True:
        msg = conn.recv()
        if msg.upper() == 'CLOSE':
            break
        print msg
    conn.close()

if __name__ == '__main__':
    address = ('/tmp/testipc', 'AF_UNIX')
    myloop(address)

这适用于OS X，所以它需要测试与Linux和（在替换正确的地址后）Windows。从安全点存在很多注意事项，主要的是conn.recv取消它的数据，所以你几乎总是更好地与recv_bytes。

This works on OS X, so it needs testing with both Linux and (after substituting the right address) Windows. A lot of caveats exists from a security point, the main one being that conn.recv unpickles its data, so you are almost always better of with recv_bytes.

这篇关于可以一个python脚本知道同一个脚本的另一个实例正在运行...然后与它谈话？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！