如何扩展Python并制作C包? [英] How to extend Python and make a C-package?
问题描述
前一段时间，我在C应用程序中嵌入并扩展了Python 2.7.在火车上晚些时候，我将其带到Python 3，并且模块注册的许多初始化对我来说也发生了变化.
I embedded and extended Python 2.7 in my C application a while ago. Late on the train I am bringing it to Python 3, and a lot of initializations for the module registration changed for me.
在我使用 PyModule_Create 创建模块并添加成员之后，甚至添加了子模块，这样我就可以执行:
Before I used PyModule_Create to create the module and added the members afterwards, even sub-modules so I could execute:
from foo.bar import bas
我将顶层"模块添加/添加到了 PyEval_GetBuiltins()中，这在Py 2中可能是错误的，但它确实有效.现在在Py 3中，我在上面的代码中收到此异常:
I added/appended the 'top-level' module to PyEval_GetBuiltins(), which might have been wrong in Py 2, but it worked. Now in Py 3 I receive this exception on the code above:
Traceback (most recent call last):
File "foo.py", line 1, in <module>
ModuleNotFoundError: No module named 'foo.bar'; 'foo' is not a package
在查找文档时，我现在找到了一个带有 PyImport_ExtendInittab 的示例.我对此有两个问题:
Looking up the docs, I found now an example with PyImport_ExtendInittab. I have two questions regarding this:
1) Inittab 应该是什么意思?医生说了什么意思，但是这种命名有点恼人.什么是 Inittab ?我不应该将它称为 PyImport_ExtendBuiltins .
1) What is Inittab supposed to mean? The doc says what it means, but this naming is slighly irritating. What is an Inittab? Shouldn't it be called PyImport_ExtendBuiltins, that I would understand.
2)我只能找到添加了普通模块的示例.使用 PyImport_ExtendInittab 也可以创建带有子模块的软件包吗?
2) I can only find examples where plain modules get added. Is creating a package with sub-modules possible with PyImport_ExtendInittab too?
非常感谢！
推荐答案
无论如何推荐的方式，我都不知道您要拉到这里的内容(嵌套扩展模块)是否是 OK 通过 [Python 3.Docs]:模块-包.
但是，我作为个人练习来做(重现问题，解决问题).
I don't know if what you're trying to pull here (nested extension modules) is OK, anyway the recommended way for structuring code is via [Python 3.Docs]: Modules - Packages.
However, I did this (reproducing the problem, fixing it) as a personal exercise.
列出2个相关页面:
- [Python 3.Docs]: Module Objects
- [Python 2.Docs]: Module Objects
环境:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q061692747]> tree /a /f
Folder PATH listing for volume SSD0-WORK
Volume serial number is AE9E-72AC
E:.
| test00.py
|
+---py2
| mod.c
|
\---py3
helper.c
mod.c
虚拟模块试图重现问题中提到的行为.
Dummy module attempting to reproduce the behavior mentioned in the question.
mod.c :
#include <stdio.h>
#include <Python.h>
#define MOD_NAME "mod"
#define SUBMOD_NAME "submod"
static PyObject *pMod = NULL;
static PyObject *pSubMod = NULL;
static PyMethodDef modMethods[] = {
{NULL}
};
PyMODINIT_FUNC initmod() {
if (!pMod) {
pMod = Py_InitModule(MOD_NAME, modMethods);
if (pMod) {
PyModule_AddIntConstant(pMod, "i", -69);
pSubMod = Py_InitModule(MOD_NAME "." SUBMOD_NAME, modMethods);
if (pSubMod) {
PyModule_AddStringConstant(pSubMod, "s", "dummy");
if (PyModule_AddObject(pMod, SUBMOD_NAME, pSubMod) < 0) {
Py_XDECREF(pMod);
Py_XDECREF(pSubMod);
return;
}
}
}
}
}
输出:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q061692747\py2]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***
[prompt]> "f:\Install\pc032\Microsoft\VisualCForPython2\2008\Microsoft\Visual C++ for Python\9.0\vcvarsall.bat" x64
Setting environment for using Microsoft Visual Studio 2008 x64 tools.
[prompt]> dir /b
mod.c
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\02.07.17\include" mod.c /link /NOLOGO /DLL /OUT:mod.pyd /LIBPATH:"c:\Install\pc064\Python\Python\02.07.17\libs"
mod.c
Creating library mod.lib and object mod.exp
[prompt]> dir /b
mod.c
mod.exp
mod.lib
mod.obj
mod.pyd
mod.pyd.manifest
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_02.07.17_test0\Scripts\python.exe"
Python 2.7.17 (v2.7.17:c2f86d86e6, Oct 19 2019, 21:01:17) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>>
>>> [item for item in sys.modules if "mod" in item]
[]
>>> import mod
>>>
>>> [item for item in sys.modules if "mod" in item] # !!! NOTICE the contents !!!
['mod.submod', 'mod']
>>>
>>> mod
<module 'mod' from 'mod.pyd'>
>>> mod.i
-69
>>> mod.submod
<module 'mod.submod' (built-in)>
>>> mod.submod.s
'dummy'
>>>
>>> from mod.submod import s
>>> s
'dummy'
>>>
如所见，导入带有子模块的模块，将子模块添加到 sys.path 中(没有看，但我 99.99％确定这是由 Py_InitModule )
As seen, importing the module with submodules, adds the submodules in sys.path (didn't look, but I am 99.99% sure this is performed by Py_InitModule)
转换为 Python 3 .由于这是第一步，所以将2条注释行视为不存在.
Conversion to Python 3. Since this is the 1st step, treat the 2 commented lines as they were not there.
mod.c :
#include <stdio.h>
#include <Python.h>
//#include "helper.c"
#define MOD_NAME "mod"
#define SUBMOD_NAME "submod"
static PyObject *pMod = NULL;
static PyObject *pSubMod = NULL;
static PyMethodDef modMethods[] = {
{NULL}
};
static struct PyModuleDef modDef = {
PyModuleDef_HEAD_INIT, MOD_NAME, NULL, -1, modMethods,
};
static struct PyModuleDef subModDef = {
PyModuleDef_HEAD_INIT, MOD_NAME "." SUBMOD_NAME, NULL, -1, modMethods,
};
PyMODINIT_FUNC PyInit_mod() {
if (!pMod) {
pMod = PyModule_Create(&modDef);
if (pMod) {
PyModule_AddIntConstant(pMod, "i", -69);
pSubMod = PyModule_Create(&subModDef);
if (pSubMod) {
PyModule_AddStringConstant(pSubMod, "s", "dummy");
if (PyModule_AddObject(pMod, SUBMOD_NAME, pSubMod) < 0) {
Py_XDECREF(pMod);
Py_XDECREF(pSubMod);
return NULL;
}
//addToSysModules(MOD_NAME "." SUBMOD_NAME, pSubMod);
}
}
}
return pMod;
}
输出:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q061692747\py3]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***
[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2017\VC\Auxiliary\Build\vcvarsall.bat" x64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.23
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
[prompt]> dir /b
helper.c
mod.c
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.07.06\include" mod.c /link /NOLOGO /DLL /OUT:mod.pyd /LIBPATH:"c:\Install\pc064\Python\Python\03.07.06\libs"
mod.c
Creating library mod.lib and object mod.exp
[prompt]> dir /b
helper.c
mod.c
mod.exp
mod.lib
mod.obj
mod.pyd
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe"
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>>
>>> [item for item in sys.modules if "mod" in item]
[]
>>> import mod
>>>
>>> [item for item in sys.modules if "mod" in item] # !!! NOTICE the contents !!!
['mod']
>>>
>>> mod
<module 'mod' from 'e:\\Work\\Dev\\StackOverflow\\q061692747\\py3\\mod.pyd'>
>>> mod.i
-69
>>> mod.submod
<module 'mod.submod'>
>>> mod.submod.s
'dummy'
>>>
>>> from mod.submod import s
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'mod.submod'; 'mod' is not a package
>>> ^Z
[prompt]>
如图所示，嵌套导入是不可能的.这是因为 sys.modules 中不存在 mod.submod .一般而言，不再将嵌套"扩展子模块通过包含初始化功能的模块导入.唯一的选择是手动导入它们.
请注意:我认为 Python 3 的限制是有原因的，所以下面的内容就像玩火一样.
As seen, nested import is not possible. That is because mod.submod is not present in sys.modules. As a generalization, "nested" extension submodules are no longer made importable through the module that contains them initialization function. the only option is to import them manually.
As a note: I think this Python 3 restriction is there for a reason, so what comes below is like playing with fire.
分解 mod.c 中的两行.
Decomment the 2 lines from mod.c.
helper.c :
int addToSysModules(const char *pName, PyObject *pMod) {
PyObject *pSysModules = PySys_GetObject("modules");
if (!PyDict_Check(pSysModules)) {
return -1;
}
PyObject *pKey = PyUnicode_FromString(pName);
if (!pKey) {
return -2;
}
if (PyDict_Contains(pSysModules, pKey)) {
Py_XDECREF(pKey);
return -3;
}
Py_XDECREF(pKey);
if (PyDict_SetItemString(pSysModules, pName, pMod) == -1)
{
return -4;
}
return 0;
}
输出:
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.07.06\include" mod.c /link /NOLOGO /DLL /OUT:mod.pyd /LIBPATH:"c:\Install\pc064\Python\Python\03.07.06\libs"
mod.c
Creating library mod.lib and object mod.exp
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe"
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import sys
>>>
>>> [item for item in sys.modules if "mod" in item]
[]
>>> import mod
>>>
>>> [item for item in sys.modules if "mod" in item] # !!! NOTICE the contents :) !!!
['mod.submod', 'mod']
>>>
>>> from mod.submod import s
>>> s
'dummy'
>>>
如上所述，这似乎更像是一种工作方法.较干净的解决方案是通过软件包更好地组织模块.
As I stated above, this seems more like a workaraound. A cleaner solution would be to better organize the modules via packages.
因为这是出于演示目的，并且为了使代码尽可能简单，所以我并不总是检查 Python C API 函数返回的代码.这可能导致难以发现错误(甚至崩溃)，并且绝不应该这样做(尤其是在生产代码中).
Since this is for demo purposes, and to keep the code as simple as possible, I didn't always check Python C API functions return codes. This can lead to hard to find errors (even crashes) and should never be done (especially in production code).
我不太确定 PyImport_ExtendInittab 的效果到底是什么，因为我没有玩过，但是
I am not very sure what PyImport_ExtendInittab effect really is as I didn't play with it, but [Python 3.Docs]: Importing Modules - int PyImport_ExtendInittab(struct _inittab *newtab) states (emphasis is mine):
应在 Py_Initialize()之前调用 .
This should be called before Py_Initialize().
因此，在我们的上下文中调用它是不可能的.
So, calling it in our context, is out of the question.
还提到了这个(旧的)讨论(不确定它是否包含相关信息，但是仍然)
Also mentioning this (old) discussion (not sure whether it contains relevant information, but still) [Python.Mail]: [Python-Dev] nested extension modules?.
这篇关于如何扩展Python并制作C包?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
