两个独立的 Python 引擎之间的通信 [英] Communication between two separate Python engines

问题描述

问题陈述如下:

我正在使用 Abaqus，这是一个用于分析机械问题的程序.它基本上是一个独立的 Python 解释器，有自己的对象等.在这个程序中，我运行一个 python 脚本来设置我的分析(所以这个脚本可以修改).它还包含一个必须在接收到外部信号时执行的方法.这些信号来自我在自己的 Python 引擎中运行的主脚本.

I am working with Abaqus, a program for analyzing mechanical problems. It is basically a standalone Python interpreter with its own objects etc. Within this program, I run a python script to set up my analysis (so this script can be modified). It also contains a method which has to be executed when an external signal is received. These signals come from the main script that I am running in my own Python engine.

目前，我有以下工作流程:当 Abaqus 脚本必须执行特定函数时，主脚本将布尔值设置为 True，并将此布尔值腌制到文件中.Abaqus 脚本会定期检查此文件以查看布尔值是否已设置为 true.如果是这样，它会进行分析并腌制输出，以便主脚本可以读取此输出并对其进行操作.

For now, I have the following workflow: The main script sets a boolean to True when the Abaqus script has to execute a specific function, and pickles this boolean into a file. The Abaqus script regularly checks this file to see whether the boolean has been set to true. If so, it does an analysis and pickles the output, so that the main script can read this output and act on it.

我正在寻找一种更有效的方法来通知另一个进程开始分析，因为有很多不必要的检查正在进行中.通过 pickle 交换数据对我来说不是问题，但当然欢迎更有效的解决方案.

I am looking for a more efficient way to signal the other process to start the analysis, since there is a lot of unnecessary checking going on right know. Data exchange via pickle is not an issue for me, but a more efficient solution is certainly welcome.

搜索结果总是给我提供子进程或类似的解决方案，这是针对在同一个解释器中启动的两个进程.我也看过ZeroMQ，因为这应该可以实现这样的事情，但我认为这是矫枉过正，希望在python中找到解决方案.两个解释器都运行 python 2.7(虽然版本不同)

Search results always give me solutions with subprocess or the like, which is for two processes started within the same interpreter. I have also looked at ZeroMQ since this is supposed to achieve things like this, but I think this is overkill and would like a solution in python. Both interpreters are running python 2.7 (although different versions)

推荐答案

像@MattP 一样，我将添加我的理解声明:

Like @MattP, I'll add this statement of my understanding:

背景

我相信您正在运行一种名为 abaqus 的产品.abaqus 产品包括一个链接的 Python 解释器，您可以以某种方式访问​​它(可能通过在命令行上运行 abaqus python foo.py).

I believe that you are running a product called abaqus. The abaqus product includes a linked-in python interpreter that you can access somehow (possibly by running abaqus python foo.py on the command line).

您在同一台机器上也有单独的 python 安装.您正在开发代码，可能包括 numpy/scipy，以在该 python 安装上运行.

You also have a separate python installation, on the same machine. You are developing code, possibly including numpy/scipy, to run on that python installation.

这两个安装是不同的:它们有不同的二进制解释器、不同的库、不同的安装路径等.但它们位于同一台物理主机上.

These two installations are different: they have different binary interpreters, different libraries, different install paths, etc. But they live on the same physical host.

您的目标是使您编写的普通 python"程序能够与在Abaqus python"环境中运行的一个或多个脚本进行通信，以便这些脚本可以在 Abaqus 系统内执行工作并返回结果.

Your objective is to enable the "plain python" programs, written by you, to communicate with one or more scripts running in the "Abaqus python" environment, so that those scripts can perform work inside the Abaqus system, and return results.

解决方案

这是一个基于套接字的解决方案.有两个部分，abqlistener.py 和 abqclient.py.这种方法的优点是它使用定义良好的等待工作"机制.不轮询文件等.它是一个硬"API.您可以从同一台机器上的进程连接到侦听器进程，运行相同版本的 python，或者从不同的机器，或者从不同版本的 python，或者从 ruby​​ 或 C 或 perl 甚至 COBOL.它允许您在系统中放置一个真正的气隙"，因此您可以以最小的耦合开发这两个部分.

Here is a socket based solution. There are two parts, abqlistener.py and abqclient.py. This approach has the advantage that it uses a well-defined mechanism for "waiting for work." No polling of files, etc. And it is a "hard" API. You can connect to a listener process from a process on the same machine, running the same version of python, or from a different machine, or from a different version of python, or from ruby or C or perl or even COBOL. It allows you to put a real "air gap" into your system, so you can develop the two parts with minimal coupling.

服务器部分是abqlistener.目的是将其中一些代码复制到 Abaqus 脚本中.然后 abq 进程将成为服务器，侦听特定端口号上的连接，并进行响应工作.发回回复，或者不.等等.

The server part is abqlistener. The intent is that you would copy some of this code into your Abaqus script. The abq process would then become a server, listening for connections on a specific port number, and doing work in response. Sending back a reply, or not. Et cetera.

我不确定您是否需要为每项作业进行设置工作.如果是这样，那将必须是连接的一部分.这只会启动 ABQ，监听端口(永远)，并处理请求.任何特定于工作的设置都必须是工作流程的一部分.(也许发送一个参数字符串，或者一个配置文件的名称，或者其他什么.)

I am not sure if you need to do setup work for each job. If so, that would have to be part of the connection. This would just start ABQ, listen on a port (forever), and deal with requests. Any job-specific setup would have to be part of the work process. (Maybe send in a parameter string, or the name of a config file, or whatever.)

客户端部分是abqclient.这可以移动到一个模块中，或者只是复制/粘贴到您现有的非 ABQ 程序代码中.基本上，您打开到正确的主机:端口组合的连接，并且您正在与服务器通话.发送一些数据，取回一些数据等

The client part is abqclient. This could be moved into a module, or just copy/pasted into your existing non-ABQ program code. Basically, you open a connection to the right host:port combination, and you're talking to the server. Send in some data, get some data back, etc.

这些东西主要是从在线示例代码中抓取的.因此，如果您开始深入研究，它应该看起来非常熟悉.

This stuff is mostly scraped from example code on-line. So it should look real familiar if you start digging into anything.

这是 abqlistener.py:

Here's abqlistener.py:

# The below usage example is completely bogus. I don't have abaqus, so
# I'm just running python2.7 abqlistener.py [options]
usage = """
abacus python abqlistener.py [--host 127.0.0.1 | --host mypc.example.com ] \\
        [ --port 2525 ]

Sets up a socket listener on the host interface specified (default: all
interfaces), on the given port number (default: 2525). When a connection
is made to the socket, begins processing data.
"""



import argparse

parser = argparse.ArgumentParser(description='Abacus listener',
    add_help=True,
    usage=usage)

parser.add_argument('-H', '--host', metavar='INTERFACE', default='',
                    help='Interface IP address or name, or (default: empty string)')
parser.add_argument('-P', '--port', metavar='PORTNUM', type=int, default=2525,
                    help='port number of listener (default: 2525)')

args = parser.parse_args()

import SocketServer
import json

class AbqRequestHandler(SocketServer.BaseRequestHandler):
    """Request handler for our socket server.

    This class is instantiated whenever a new connection is made, and
    must override `handle(self)` in order to handle communicating with
    the client.
    """

    def do_work(self, data):
        "Do some work here. Call abaqus, whatever."
        print "DO_WORK: Doing work with data!"
        print data
        return { 'desc': 'low-precision natural constants','pi': 3, 'e': 3 }

    def handle(self):
        # Allow the client to send a 1kb message (file path?)
        self.data = self.request.recv(1024).strip()
        print "SERVER: {} wrote:".format(self.client_address[0])
        print self.data
        result = self.do_work(self.data)
        self.response = json.dumps(result)
        print "SERVER: response to {}:".format(self.client_address[0])
        print self.response
        self.request.sendall(self.response)


if __name__ == '__main__':
    print args
    server = SocketServer.TCPServer((args.host, args.port), AbqRequestHandler)
    print "Server starting. Press Ctrl+C to interrupt..."
    server.serve_forever()

这里是abqclient.py:

usage = """
python2.7 abqclient.py [--host HOST] [--port PORT]

Connect to abqlistener on HOST:PORT, send a message, wait for reply.
"""

import argparse

parser = argparse.ArgumentParser(description='Abacus listener',
    add_help=True,
    usage=usage)

parser.add_argument('-H', '--host', metavar='INTERFACE', default='',
                    help='Interface IP address or name, or (default: empty string)')
parser.add_argument('-P', '--port', metavar='PORTNUM', type=int, default=2525,
                    help='port number of listener (default: 2525)')

args = parser.parse_args()

import json
import socket

message = "I get all the best code from stackoverflow!"

print "CLIENT: Creating socket..."
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

print "CLIENT: Connecting to {}:{}.".format(args.host, args.port)
s.connect((args.host, args.port))

print "CLIENT: Sending message:", message
s.send(message)

print "CLIENT: Waiting for reply..."
data = s.recv(1024)

print "CLIENT: Got response:"
print json.loads(data)

print "CLIENT: Closing socket..."
s.close()

当我一起运行它们时，它们会打印出来:

And here's what they print when I run them together:

$ python2.7 abqlistener.py --port 3434 &
[2] 44088
$ Namespace(host='', port=3434)
Server starting. Press Ctrl+C to interrupt...

$ python2.7 abqclient.py --port 3434
CLIENT: Creating socket...
CLIENT: Connecting to :3434.
CLIENT: Sending message: I get all the best code from stackoverflow!
CLIENT: Waiting for reply...
SERVER: 127.0.0.1 wrote:
I get all the best code from stackoverflow!
DO_WORK: Doing work with data!
I get all the best code from stackoverflow!
SERVER: response to 127.0.0.1:
{"pi": 3, "e": 3, "desc": "low-precision natural constants"}
CLIENT: Got response:
{u'pi': 3, u'e': 3, u'desc': u'low-precision natural constants'}
CLIENT: Closing socket...

参考:

argparse, SocketServer, json, socket 都是标准"Python图书馆.

argparse, SocketServer, json, socket are all "standard" Python libraries.

这篇关于两个独立的 Python 引擎之间的通信的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！