Python，Threads，GIL和C ++ [英] Python, Threads, the GIL, and C++

问题描述

有什么方法使boost :: python控制Python GIL与python的每次交互？

Is there some way to make boost::python control the Python GIL for every interaction with python?

我正在使用boost :: python编写一个项目。我试图为外部库写一个C ++包装，并用python脚本控制C ++库。我不能改变外部库，只有我的包装程序。 （我正在为所述外部库编写一个功能测试应用程序）

I am writing a project with boost::python. I am trying to write a C++ wrapper for an external library, and control the C++ library with python scripts. I cannot change the external library, only my wrapper program. (I am writing a functional testing application for said external library)

外部库用C编写，并使用函数指针和回调来做很多繁重的工作。它是一个消息系统，所以当一个消息进来时，一个回调函数被调用，例如。

The external library is written in C and uses function pointers and callbacks to do a lot of heavy lifting. Its a messaging system, so when a message comes in, a callback function gets called, for example.

我在我的库中实现了一个观察者模式，到一个回调。

I implemented an observer pattern in my library so that multiple objects could listen to one callback. I have all the major players exported properly and I can control things very well up to a certain point.

外部库创建了处理消息，发送消息，处理的线程，这些回调可能会从不同的进程中调用，我最近发现python不是线程安全的。

The external library creates threads to handle messages, send messages, processing, etc. Some of these callbacks might be called from different processes, and I recently found out that python is not thread safe.

这些观察者可以在python中定义，所以我需要能够调用python和python需要调用到我的程序在任何时候。

These observers can be defined in python, so I need to be able to call into python and python needs to call into my program at any point.

我设置对象和观察者像这样

I setup the object and observer like so

class TestObserver( MyLib.ConnectionObserver ):
    def receivedMsg( self, msg ):
        print("Received a message!")

ob = TestObserver()
cnx = MyLib.Conection()
cnx.attachObserver( ob )

然后我创建一个源发送到连接，并调用receivedMsg函数。

Then I create a source to send to the connection and the receivedMsg function is called.

send（'msg'）将进入我的C ++应用程序，去C库，它将发送消息，连接将获得它，然后调用回调，它回到我的C ++库和连接尝试通知所有观察者，这里是python类，所以它调用该方法。

So a regular source.send('msg') will go into my C++ app, go to the C library, which will send the message, the connection will get it, then call the callback, which goes back into my C++ library and the connection tries to notify all observers, which at this point is the python class here, so it calls that method.

当然，回调从连接线程调用，而不是主应用程序线程。

And of course the callback is called from the connection thread, not the main application thread.

昨天一切都崩溃，我无法发送1条消息。然后在Cplusplus-sig档案中挖掘，我学习了GIL和一些漂亮的函数来锁定东西。

Yesterday everything was crashing, I could not send 1 message. Then after digging around in the Cplusplus-sig archives I learned about the GIL and a couple of nifty functions to lock things up.

所以我的C ++ python包装为我的观察者类看起来像这样

So my C++ python wrapper for my observer class looks like this now

struct IConnectionObserver_wrapper : Observers::IConnectionObserver, wrapper<Observers::IConnectionObserver>
{
    void receivedMsg( const Message* msg )
    {
        PyGILState_STATE gstate = PyGILState_Ensure();
        if( override receivedMsg_func = this->get_override( "receivedMsg" ) )
            receivedMsg_func( msg );
        Observers::IConnectionObserver::receivedMsg( msg );
        PyGILState_Release( gstate );
    }
}

然而，当我尝试发送超过250条消息，例如

And that WORKS, however, when I try to send over 250 messages like so

for i in range(250)
    source.send('msg")

它会再次崩溃。与之前相同的消息和症状，

it crashes again. With the same message and symptoms that it has before,

PyThreadState_Get: no current thread

所以我认为这一次我有一个问题调用到我的C ++应用程序，而不是调用python。

so I am thinking that this time I have a problem calling into my C++ app, rather then calling into python.

我的问题是，有一些方法来提高： ：python处理GIL本身每次与python交互我​​不能在代码中找到任何东西，真的很难找到source.send调用进入boost_python：（

My question is, is there some way to make boost::python handle the GIL itself for every interaction with python? I can not find anything in the code, and its really hard trying to find where the source.send call enters boost_python :(

推荐答案

我发现一个非常模糊的邮件列表上的帖子说，使用
PyEval_InitThreads（）;
在BOOST_PYTHON_MODULE
，停止崩溃。

I found a really obscure post on the mailing list that said to use PyEval_InitThreads(); in BOOST_PYTHON_MODULE and that actually seemed to stop the crashes.

它仍然是一个垃圾拍摄是否程序报告所有的消息，它得到与否。如果我发送2000，大部分时间它说它有2000，但有时它报告显着减少。

Its still a crap shoot whether it the program reports all the messages it got or not. If i send 2000, most of the time it says it got 2000, but sometimes it reports significantly less.

我怀疑这可能是由于线程访问我的计数器同时，我回答这个问题，因为这是一个不同的问题。

I suspect this might be due to the threads accessing my counter at the same time, so I am answering this question because that is a different problem.

要修复

BOOST_PYTHON_MODULE(MyLib)
{
    PyEval_InitThreads();
    class_ stuff

这篇关于Python，Threads，GIL和C ++的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！