嵌入式Python:多个子解释器不起作用 [英] Embedded Python: Multiple Sub-Interpreters not working

问题描述

我正试图了解子口译员和GIL.但是我的实验经常失败(相同的代码很少起作用).

I'm trying to understand sub-interpreters and GIL. But my experiment is failing often(The same code rarely works).

从SO问题和一些站点收集信息，我有以下代码产生了2个非python线程.这些线程中的每一个都被赋予了一个python子解释器.我想在这些线程中释放GIL并在C ++中调用DLL函数(尽管此示例未详细说明.在这里，我只是写到stdout).基本上，我想在执行中看到并发性(非Python DLL调用).

Gathering info from SO questions and few sites, I have the following code which spawns 2 non-python threads. Each of these threads are given a python sub-interpreter. I want to release GIL inside these threads and call a DLL function in C++(though this example does not detail that. Here I just write to stdout). Basically I want to see concurrency in the execution(Non-Python DLL invocation).

#include <iostream>
#include <thread>
#include <Python.h>

void worker(PyInterpreterState* interp, int n) 
{
    PyThreadState* ts;
    ts = PyThreadState_New(interp);

    PyThreadState_Swap(ts);

    PyThreadState* _save;
    _save = PyEval_SaveThread();

    std::cout << n << std::endl;  // Non-Python execution. My Focus.

    PyEval_RestoreThread(_save);

    PyThreadState_Swap(ts);

    PyThreadState_Clear(ts);
    PyThreadState_DeleteCurrent();

    return;
}


int main()
{
    try 
    {
        Py_Initialize();

        PyEval_InitThreads();

        PyThreadState* _main = PyThreadState_Get();

        PyThreadState* i1 = Py_NewInterpreter();
        PyThreadState* i2 = Py_NewInterpreter();

        std::thread t1(worker, i1->interp, 1);
        std::thread t2(worker, i2->interp, 2);

        t1.join();
        t2.join();

        PyThreadState_Swap(i1);
        PyThreadState_Clear(i1);
        Py_EndInterpreter(i1);
        PyThreadState_Swap(i2);
        PyThreadState_Clear(i2);
        Py_EndInterpreter(i2);

        PyThreadState_Swap(_main);

        Py_Finalize();
        return 0;
    }
    catch(std::exception& e)
    {
        std::cout << "Exception:" << std::endl << e.what() << std::endl;
    }
}

始终运行单个线程.如图所示，当我运行2个线程时，我得到以下任何输出.

Running a single thread works all the time. When I run 2 threads as shown, I get any of the following outputs.

1. 在PyEval_SaveThread()中，

2
Fatal Python error: drop_gil: GIL is not locked
Python runtime state: initialized

Current thread 0x00002d08 (most recent call first):
<no Python frame>

1. 在PyEval_SaveThread()中，

1
Fatal Python error: PyEval_SaveThread: NULL tstate
Python runtime state: initialized

Current thread 0x00003eb8 (most recent call first):
<no Python frame>

其中一个线程成功，另一个线程失败.

Either of the thread succeeds, the other one fails.

1. 很少有作品.相同的代码.

1
2

有人可以阐明这一点吗?需要帮忙.谢谢.

Can someone shed some light on this? Need help. Thanks.

推荐答案

1. 在您的worker()函数中，调用PyThreadState_Swap(). Python文档说:

1. In your worker() function, you call PyThreadState_Swap(). The Python docs say that:

全局解释器锁必须保持并且不能释放.

The global interpreter lock must be held and is not released.

您必须在调用PyThreadState_Swap()之前获取GIL，并在退出worker()之前释放它.

You must acquire the GIL before calling PyThreadState_Swap() and release it before exiting worker().

1. 在主线程中，您等待线程终止，同时按住GIL.这是一个僵局，因为线程将需要获取GIL才能在Python中完成任何有用的工作.

有关以下内容的详细工作说明，请访问 https://stackoverflow.com/a/26570708/99279 查看我的回答.如何做到这一点，甚至链接到示例代码.

Please see my answer at https://stackoverflow.com/a/26570708/99279 for detailed working instructions on how to do that, and even a link to sample code.

有了GIL和子解释器的线程状态后，现在可以安全地调用常规Python API，只需添加一个PyEval_SaveThread()/PyEval_RestoreThread()对.

Once you have the GIL, and the thread state for for the sub interpreter, so that it is now safe to call the regular Python API, you can just add a PyEval_SaveThread()/PyEval_RestoreThread() pair.

这篇关于嵌入式Python:多个子解释器不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！