在多线程程序中同步嵌入式 Python [英] Synchronizing embedded Python in multi-threaded program

问题描述

以下是在多线程程序中使用 Python 解释器的示例:

Here is the example of using Python interpreter in multi-threaded program:

#include <python.h>
#include <boost/thread.hpp>

void f(const char* code)
{
    static volatile auto counter = 0;
    for(; counter < 20; ++counter)
    {
        auto state = PyGILState_Ensure();
        PyRun_SimpleString(code);
        PyGILState_Release(state);

        boost::this_thread::yield();
    }
}

int main()
{
    PyEval_InitThreads();
    Py_Initialize();
    PyRun_SimpleString("x = 0\n");
    auto mainstate = PyEval_SaveThread();

    auto thread1 = boost::thread(f, "print('thread #1, x =', x)\nx += 1\n");
    auto thread2 = boost::thread(f, "print('thread #2, x =', x)\nx += 1\n");
    thread1.join();
    thread2.join();

    PyEval_RestoreThread(mainstate);
    Py_Finalize();
}

看起来不错，但没有同步.Python 解释器在 PyRun_SimpleString 期间多次释放和重新获取 GIL(参见 文档，p.#2).

It looks fine, but it isn't synchronized. Python interpreter releases and reacquires GIL multiple times during PyRun_SimpleString (see docs, p.#2).

我们可以使用我们自己的同步对象序列化 PyRun_SimpleString 调用，但这是错误的方式.

We can serialize PyRun_SimpleString call by using our own synchronization object, but it's a wrong way.

Python 有自己的同步模块 - _thread 和 threading.但它们在这段代码中不起作用:

Python has its own synchronization modules - _thread and threading. But they don't work in this code:

Py_Initialize();
PyRun_SimpleString(R"(
import _thread
sync = _thread.allocate_lock()

x = 0
)");

auto mainstate = PyEval_SaveThread();

auto thread1 = boost::thread(f, R"(
with sync:
    print('thread #1, x =', x)
    x += 1
)");

• 它产生一个错误 File "", line 3, in NameError: name '_[1]' 未定义 和死锁.

如何同步嵌入式python代码最有效的方式?

How to synchronize embedded python code most efficient way?

推荐答案

with 声明有 问题在 Python 3.1 中，但在 Python 3.2 和 Python 2.7 中已修复.

with statement has issue in Python 3.1, but it was fixed in Python 3.2 and Python 2.7.

所以正确的解决方案是使用threading模块进行同步.

So the right solution is to use the threading module for synchronization.

为避免此类问题，不应使用在全局字典中使用临时变量的多线程代码，或为每个线程使用不同的全局字典.

To avoid such issues, one shouldn't use multi-threaded code which uses temporary variables in globals dictionary, or use different globals dictionaries for each thread.

这篇关于在多线程程序中同步嵌入式 Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！