如何重新加载Python3 C扩展模块? [英] How to Reload a Python3 C extension module?

问题描述

我为Python 3.2写了一个C扩展名(mycext.c).该扩展依赖于存储在C头(myconst.h)中的常量数据.头文件是由Python脚本生成的.在同一脚本中，我利用了最近编译的模块. Python3 myscript(未完整显示)中的工作流程如下:

I wrote a C extension (mycext.c) for Python 3.2. The extension relies on constant data stored in a C header (myconst.h). The header file is generated by a Python script. In the same script, I make use of the recently compiled module. The workflow in the Python3 myscript (not shown completely) is as follows:

configure_C_header_constants() 
write_constants_to_C_header() # write myconst.h
os.system('python3 setup.py install --user') # compile mycext
import mycext
mycext.do_stuff()

这在第一次Python会话中效果很好.如果我在同一会话中重复该过程(例如，在单元测试的两个不同测试用例中)，则始终会(重新)加载mycext的第一个编译版本.

This works perfectly fine the in a Python session for the first time. If I repeat the procedure in the same session (for example, in two different testcases of a unittest), the first compiled version of mycext is always (re)loaded.

如何有效地使用最新的编译版本重新加载扩展模块?

How do I effectively reload a extension module with the latest compiled version?

推荐答案

您可以使用 Python的导入机制永远不会dlclose()共享库.加载后，该库将保留直到该过程终止.

Python's import mechanism will never dlclose() a shared library. Once loaded, the library will stay until the process terminates.

您的选择(按使用率递减排序):

Your options (sorted by decreasing usefulness):

1. 将模块导入到子进程中，然后在重新编译后再次调用该子进程，即，您有一个简单执行的Python脚本do_stuff.py

import mycext
mycext.do_stuff()

您使用以下命令调用该脚本

and you call this script using

subprocess.call([sys.executable, "do_stuff.py"])

将标头中的编译时常量转换为可以从Python更改的变量，从而无需重新加载模块.

Turn the compile-time constants in your header into variables that can be changed from Python, eliminating the need to reload the module.

在删除对该模块的所有引用后手动dlclose()该库(这有点脆弱，因为您自己没有保存所有引用).

Manually dlclose() the library after deleting all references to the module (a bit fragile since you don't hold all the references yourself).

滚动您自己的导入机制.

Roll your own import mechanism.

这是一个如何完成此操作的示例.我写了一个最小的Python C扩展mini.so，只导出了一个名为version的整数.

Here is an example how this can be done. I wrote a minimal Python C extension mini.so, only exporting an integer called version.

>>> import ctypes
>>> libdl = ctypes.CDLL("libdl.so")
>>> libdl.dlclose.argtypes = [ctypes.c_void_p]
>>> so = ctypes.PyDLL("./mini.so")
>>> so.PyInit_mini.argtypes = []
>>> so.PyInit_mini.restype = ctypes.py_object 
>>> mini = so.PyInit_mini()
>>> mini.version
1
>>> del mini
>>> libdl.dlclose(so._handle)
0
>>> del so

这时，我增加了mini.c中的版本号并重新编译.

At this point, I incremented the version number in mini.c and recompiled.

>>> so = ctypes.PyDLL("./mini.so")
>>> so.PyInit_mini.argtypes = []
>>> so.PyInit_mini.restype = ctypes.py_object 
>>> mini = so.PyInit_mini()
>>> mini.version
2

您会看到使用了新版本的模块.

You can see that the new version of the module is used.

供参考和实验，这里是mini.c:

For reference and experimenting, here's mini.c:

#include <Python.h>

static struct PyModuleDef minimodule = {
   PyModuleDef_HEAD_INIT, "mini", NULL, -1, NULL
};

PyMODINIT_FUNC
PyInit_mini()
{
    PyObject *m = PyModule_Create(&minimodule);
    PyModule_AddObject(m, "version", PyLong_FromLong(1));
    return m;
}

这篇关于如何重新加载Python3 C扩展模块?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！