使用cython从多个pyx文件制作可执行文件 [英] Make executable file from multiple pyx files using cython

问题描述

我正试图从我的python源文件中制作一个unix可执行文件。

I am trying to make one unix executable file from my python source files.

我有两个文件， p1.py 和 p2.py

p1.py：-

from p2 import test_func 
print (test_func())

p2.py：-

def test_func():
    return ('Test')

现在，我们可以看到 p1.py 取决于 p2.py 。我想通过将两个文件组合在一起来制作一个可执行文件。我正在使用cython。

Now, as we can see p1.py is dependent on p2.py . I want to make an executable file by combining two files together. I am using cython.

我将文件名更改为 p1.pyx 和 p2.pyx

I changed the file names to p1.pyx and p2.pyx respectively.

现在，我可以使用cython使文件可执行，

Now, I can make file executable by using cython,

cython p1.pyx --embed

它将生成一个名为C的源文件 p1.c 。接下来，我们可以使用gcc使其可执行，

It will generate a C source file called p1.c . Next we can use gcc to make it executable,

gcc -Os -I /usr/include/python3.5m -o test p1.c -lpython3.5m -lpthread -lm -lutil -ldl 

但是如何合并两个文件放入一个可执行文件中？

But how to combine two files into one executable ?

推荐答案

您必须跳过一些循环才能使其工作。

There are some loops you have to jump through to make it work.

首先，您必须知道生成的可执行文件是一个非常薄的层，它仅将整个工作委托给 pythonX.Ym.so 。调用

First, you must be aware that the resulting executable is a very slim layer which just delegates the whole work to (i.e. calls functions from) pythonX.Ym.so. You can see this dependency when calling

ldd test
...
libpythonX.Ym.so.1.0 => not found
...

因此，要运行该程序，您需要 LD_LIBRARY_PATH 显示到 libpythonX.Ym.so 的位置，或使用-构建exe -rpath 选项，否则在 test 启动时，动态加载程序将引发类似于

So, to run the program you either need to have the LD_LIBRARY_PATH showing to the location of the libpythonX.Ym.so or build the exe with --rpath option, otherwise at the start-up of test dynamic loader will throw an error similar to

/ test：加载共享库时出错：libpythonX.Ym.so.1.0：无法打开共享对象文件：没有这样的文件或目录

/test: error while loading shared libraries: libpythonX.Ym.so.1.0: cannot open shared object file: No such file or directory

通用构建命令如下所示：

The generic build command would look like following:

gcc -fPIC <other flags> -o test p1.c -I<path_python_include> -L<path_python_lib> -Wl,-rpath=<path_python_lib> -lpython3.6m <other_needed_libs>

还可以针对静态版本的python库进行构建，从而消除了对运行时的依赖libpythonX.Ym，请参见例如 SO-post 。

It is also possible to build against static version of the python-library, thus eliminating run time dependency on the libpythonX.Ym, see for example this SO-post.

生成的可执行文件 test 的行为与作为python解释器的行为完全相同。这意味着现在 test 将失败，因为它找不到模块 p2 。

The resulting executable test behaves exactly the same as if it were a python-interpreter. This means that now, test will fail because it will not find the module p2.

一个简单的解决方案是就地对p2模块进行cythonize（ cythonize p2.pyx -i ）：您将获得所需的行为-但是，您将不得不分发结果共享对象 p2.so 以及 test 。

One simple solution were to cythonize the p2-module inplace (cythonize p2.pyx -i): you would get the desired behavior - however, you would have to distribute the resulting shared-object p2.so along with test.

很容易将两个扩展名捆绑到一个可执行文件中-只需将两个被cythonized的c文件传递给gcc：

It is easy to bundle both extension into one executable - just pass both cythonized c-files to gcc:

# creates p1.c:
cython --empbed p1.pyx
# creates p2.c:  
cython p2.pyx
gcc ... -o test p1.c p2.c ...

但是现在出现了一个新的（或旧的）问题： test -executable无法再次找到模块 p2 ，因为没有 p2.py 并且在python路径上没有 p2.so 。

But now a new (or old) problem arises: the resulting test-executable cannot once again find the module p2, because there is no p2.py and no p2.so on the python-path.

有两个类似的地方所以对此问题有疑问，此处和此处。在您的情况下，建议的解决方案有些过头了，在将p2模块导入 p1.pyx 文件中之前，足以初始化p2模块使其起作用：

There are two similar SO questions about this problem, here and here. In your case the proposed solutions are kind of overkill, here it is enough to initialize the p2 module before it gets imported in the p1.pyx-file to make it work:

# making init-function from other modules accessible:
cdef extern  object PyInit_p2();

#init/load p2-module manually
PyInit_p2()  #Cython handles error, i.e. if NULL returned

# actually using already cached imported module
#          no search in python path needed
from p2 import test_func
print(test_func())

在模块导入前调用模块的init函数（实际上，模块不会真正第二次导入，仅在高速缓存中查找），如果模块之间存在循环依赖性，则也可以使用。例如，如果模块 p2 导入模块 p3 ，该模块导入 p2

Calling the init-function of a module prior to importing it (actually the module will not be really imported a second time, only looked up in the cache) works also if there are cyclic dependencies between modules. For example if module p2 imports module p3, which imports p2in its turn.

警告：自Cython 0.29起，Cython使用多阶段初始化默认为Python> = 3.5，因此调用 PyInit_p2 是不够的（请参见例如这样的帖子）。要关闭此多阶段初始化，应将 -DCYTHON_PEP489_MULTI_PHASE_INIT = 0 传递给gcc或类似的其他编译器。

Warning: Since Cython 0.29, Cython uses multi-phase initialization per default for Python>=3.5, thus calling PyInit_p2 is not enough (see e.g. this SO-post). To switch off this multi-phase initialization -DCYTHON_PEP489_MULTI_PHASE_INIT=0should be passed to gcc or similar to other compilers.

注意：但是，即使完成上述所有操作，嵌入式解释器也将需要其标准库（例如，请参见 SO-post ）-要使其真正独立，还有很多工作要做！因此，也许应该留意 @DavidW的建议：

Note: However, even after all of the above, the embedded interpreter will need its standard libraries (see for example this SO-post) - there is much more work to do to make it truly standalone! So maybe one should heed @DavidW's advice:

不要这样做可能是绝大多数
人的最佳解决方案。

"don't do this" is probably the best solution for the vast majority of people.

---

警告：如果我们将 PyInit_p2（）声明为

from cpython cimport PyObject
cdef extern  PyObject *PyInit_p2();

PyInit_p2(); # TODO: error handling if NULL is returned

Cython将不再处理错误及其责任。代替

Cython will no longer handle the errors and its our responsibility. Instead of

PyObject *__pyx_t_1 = NULL;
__pyx_t_1 = PyInit_p2(); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error)
__Pyx_GOTREF(__pyx_t_1);
__Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;

代码变为公正的

(void)(PyInit_p2());

即

另一方面使用

cdef extern from *:
    """
    PyObject *PyInit_p2(void);
    """
    object PyInit_p2()

不适用于g ++-必须在声明中添加 extern C 。

will not work with g++ - one has to add extern C to declaration.

这篇关于使用cython从多个pyx文件制作可执行文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！