蟒蛇，线程，GIL和C ++ [英] Python, Threads, the GIL, and C++

问题描述

有没有一些方法，使升压::蟒蛇控制Python的GIL与蟒蛇的每一次互动？

Is there some way to make boost::python control the Python GIL for every interaction with python?

我写与升压::蟒蛇的项目。我想写一个外部库C ++包装，并使用Python脚本控制的C ++库。我不能改变外部库，只是我的包装程序。 （我写说外部库功能测试应用程序）

I am writing a project with boost::python. I am trying to write a C++ wrapper for an external library, and control the C++ library with python scripts. I cannot change the external library, only my wrapper program. (I am writing a functional testing application for said external library)

外部库是用C语言编写，并使用函数指针和回调做很多繁重的工作。它是一种信息系统，因此当消息进来，回调函数被调用，例如

The external library is written in C and uses function pointers and callbacks to do a lot of heavy lifting. Its a messaging system, so when a message comes in, a callback function gets called, for example.

我实现我的库中的观察者模式，使多个对象可以监听一个回调。我把所有的主要参与者正确导出，我可以控制的东西很好地达到某一个点。

I implemented an observer pattern in my library so that multiple objects could listen to one callback. I have all the major players exported properly and I can control things very well up to a certain point.

外部库创建线程来处理邮件，发送邮件，处理等一些回调的可能会从不同的进程调用，我最近发现蟒蛇不是线程安全的。

The external library creates threads to handle messages, send messages, processing, etc. Some of these callbacks might be called from different processes, and I recently found out that python is not thread safe.

这些观察者可以在Python中定义的，所以我需要能够调用到Python和python的需求在任何时候打电话到我的程序。

These observers can be defined in python, so I need to be able to call into python and python needs to call into my program at any point.

我设定的目标和观察员像这样

I setup the object and observer like so

class TestObserver( MyLib.ConnectionObserver ):
    def receivedMsg( self, msg ):
        print("Received a message!")

ob = TestObserver()
cnx = MyLib.Conection()
cnx.attachObserver( ob )

然后，我创建一个源发送到连接和receivedMsg函数被调用。

Then I create a source to send to the connection and the receivedMsg function is called.

所以经常source.send（'味精'）将进入我的C ++应用程序，进入C库，它会发送消息，连接将得到它，然后调用回调函数，它会回到我的C ++库和连接试图通知所有观察者，在这个点是Python类在这里，所以它调用该方法。

So a regular source.send('msg') will go into my C++ app, go to the C library, which will send the message, the connection will get it, then call the callback, which goes back into my C++ library and the connection tries to notify all observers, which at this point is the python class here, so it calls that method.

当然，回调是从连接线程调用，而不是主应用程序线程。

And of course the callback is called from the connection thread, not the main application thread.

昨天一切都崩溃，我不能发送1消息。然后在CPLUSPLUS-SIG档案周围挖后，我了解了GIL和几个漂亮的功能，锁定的事情了。

Yesterday everything was crashing, I could not send 1 message. Then after digging around in the Cplusplus-sig archives I learned about the GIL and a couple of nifty functions to lock things up.

所以，我观察我的类C ++ Python包装看起来现在这​​个样子

So my C++ python wrapper for my observer class looks like this now

struct IConnectionObserver_wrapper : Observers::IConnectionObserver, wrapper<Observers::IConnectionObserver>
{
    void receivedMsg( const Message* msg )
    {
        PyGILState_STATE gstate = PyGILState_Ensure();
        if( override receivedMsg_func = this->get_override( "receivedMsg" ) )
            receivedMsg_func( msg );
        Observers::IConnectionObserver::receivedMsg( msg );
        PyGILState_Release( gstate );
    }
}

和这样的作品，但是，当我尝试发送超过250的消息，像这样

And that WORKS, however, when I try to send over 250 messages like so

for i in range(250)
    source.send('msg")

它再次崩溃。具有相同消息和症状，它有前，

it crashes again. With the same message and symptoms that it has before,

PyThreadState_Get: no current thread

所以我想，这个时候，我有一个问题打电话到我的C ++应用程序，而不是调用到蟒蛇。

so I am thinking that this time I have a problem calling into my C++ app, rather then calling into python.

我的问题是，有没有一些方法，使升压::蟒蛇处理GIL本身与蟒蛇的每一次互动？我找不到在code任何东西，它真的很难试图找到其中source.send呼叫进入boost_python：（

My question is, is there some way to make boost::python handle the GIL itself for every interaction with python? I can not find anything in the code, and its really hard trying to find where the source.send call enters boost_python :(

推荐答案

我发现的邮件列表说要用上一个真正的默默无闻后
PyEval_InitThreads（）;
在BOOST_PYTHON_MODULE
而实际上似乎停止崩溃。

I found a really obscure post on the mailing list that said to use PyEval_InitThreads(); in BOOST_PYTHON_MODULE and that actually seemed to stop the crashes.

它仍然是一个废话拍摄程序是否它报告它得到与否的所有消息。如果我送2000，其中大部分是说，它得到了2000，但有时报道显著少。时间

Its still a crap shoot whether it the program reports all the messages it got or not. If i send 2000, most of the time it says it got 2000, but sometimes it reports significantly less.

我怀疑这可能是由于在同一时间访问我的柜台线程，所以我回答这个问题，因为这是一个不同的问题。

I suspect this might be due to the threads accessing my counter at the same time, so I am answering this question because that is a different problem.

要解决只是做

BOOST_PYTHON_MODULE(MyLib)
{
    PyEval_InitThreads();
    class_ stuff

这篇关于蟒蛇，线程，GIL和C ++的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！