使用 setuptools 和 swig 构建模块 [英] Building a module with setuptools and swig

问题描述

我有一堆 SWIG 接口(foo.i、bar.i 等).我想使用 setuptools 将它们构建到我的平台(MS Windows)的 Python(3.6.4)模块中.该模块应包括 SWIG 生成的 Python 文件 (*.py)、二进制扩展 (*.pyd) 和编译的缓存 (*.pyc).我的 setup.py 基本上是这样的:

I have a bunch of SWIG interfaces (foo.i, bar.i, etc.). I want to build them into a Python (3.6.4) module for my platform (MS Windows), with setuptools. The module should include the SWIG-generated Python files (*.py), the binary extensions (*.pyd) and the compiled caches (*.pyc). My setup.py is essentially like this:

from setuptools import setup, Extension
from pathlib import Path
paths=list(Path('.').glob('*.i'))
py=[path.stem for path in paths]
ext=[Extension('_' + path.stem, [str(path)]) for path in paths]
setup(py_modules=py, ext_modules=ext)

现在我按照以下步骤构建它:

Now I build it with the following steps:

python setup.py build_ext -I..\include --swig-opts="-I..\include -c++" -b pkg
python setup.py build_py -c -d pkg
echo. > pkg\__init__.py

使用这些步骤，我在 pkg 目录下得到了我想要的东西.

Using these steps, I get exactly what I want, under the pkg directory.

我的问题是:有没有办法使用 setup.py 的单个调用来获得这种效果，例如，setup.py build?我认为构建应该调用 build_ext，但是我看不到如何通过，例如 swig-opts 选项.

My question is: Is there no way to get this effect using a single invocation of setup.py, e.g., setup.py build? I think that build is supposed to call build_ext, but then I cannot see how to pass, e.g., the swig-opts option.

传递 SWIG 选项已解决(h/t @hoefling).解决方案如下:

Passing the SWIG options is solved (h/t @hoefling). The solution looks like this:

ext=[Extension(name='_' + path.stem,
            sources=[str(path)],
            swig_opts=['-I../include', '-c++'],
            include_dirs=['../include'])
            for path in paths]

然而，剥掉那层洋葱后，我现在可以看到下面的层，它是这样的:setup.py build 作为单个调用，想要首先运行 build_py，然后运行 ​​build_ext.你能明白为什么这会失败吗?我没有 Python 资源.我的模块中的 Python 脚本将由 SWIG 生成，但 SWIG 在 build_ext 步骤之前不会运行.因此，我最终得到了同样的问题，即如何在单个调用中构建模块.

However, with that layer of the onion peeled off, I can now see the layer below, which is this: setup.py build as a single invocation, wants to run the build_py first, and then build_ext afterward. Can you see why this fails? I have no Python sources. The Python scripts in my module are to be generated by SWIG, but SWIG doesn't get run until the build_ext step. Thus, I end up with the same question, which is how to build the module in a single invocation.

C:\some\path> python setup.py build
running build
running build_py
file foo.py (for module foo) not found
file foo.py (for module foo) not found
running build_ext
building '_foo' extension
swigging foo.i to foo_wrap.cpp
swig.exe -python -I../include -c++ -o foo_wrap.cpp foo.i
creating build
creating build\temp.win-amd64-3.6
creating build\temp.win-amd64-3.6\Release
cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I../include /EHsc /Tpfoo_wrap.cpp /Fobuild\temp.win-amd64-3.6\Release\foo_wrap.obj foo_wrap.cpp
Creating build\wib.win-amd64-3.6
link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /EXPORT:PyInit__foo build\temp.win-amd64-3.6\Release\foo_wrap.obj /IMPLIB:build\temp.win-amd64-3.6\Release\_foo.cp36-win_amd64.lib
Creating library build\temp.win-amd64-3.6\Release\_foo.cp36-win_amd64.lib and object build\temp.win-amd64-3.6\Release\_foo.cp36-win_amd64.exp
Generating code
Finished generating code

完成后，foo.py 存在于当前工作目录 foo.i 旁边，_foo.cp36-win_amd64.pyd 存在于 build\lib.win-amd64-3.6 下.(仅供参考，为了保护专有信息，此记录略有编辑，例如，我没有显示所有命令回显的完整路径.而且我没有在来源中包含 bar.i.)

When complete, foo.py exists in the current working directory next to foo.i, and _foo.cp36-win_amd64.pyd exists under build\lib.win-amd64-3.6. (FYI, this transcript is edited slightly to protect proprietary information, e.g., I don't show all the full paths to all the echos of commands. And I did not include bar.i in the sources.)

澄清:

1. 错误是这个:File foo.py (for module foo) not found

尽管输出中存在所有必需的文件，除了 foo.cp36-win_amd64.pyc(我可能没有它)，我必须手动将 foo.py 和 _foo.cp36-win_amd64.pyd 复制到一些新的目录来创建一个干净的 Python 模块.

Though all the required files exist in the output, except foo.cp36-win_amd64.pyc which I could possibly live without, I have to manually copy foo.py and _foo.cp36-win_amd64.pyd into some new directory to make a clean Python module.

推荐答案

问题是一旦你声明了 py_modules，distutils 包含 build_py作为 build 中的子命令，并且由于 build 类中 sub_commands 列表的顺序，它总是首先执行.它只是不知道 SWIG 稍后会在 build_ext 中生成 python 模块包装器.我想您可以将其视为一个错误，因为人们会期望 distutils 能够处理生成的模块，但是我不知道.

The problem is that once you declare py_modules, distutils includes build_py as subcommand in build and it is always being executed first because of the ordering of sub_commands list in build class. It simply doesn't know that SWIG will generate python module wrappers later in build_ext. I guess you can see this as a bug since one would expect distutils to be able to cope with generated modules, but meh.

由于您只有 SWIG 接口并且您的所有 python 模块都是生成的包装器(因此在您的情况下 build_ext 不依赖于 build_py)，您可以更改顺序build 中的子命令，所以 build_ext 首先执行，然后是其余的.这将确保在执行 build_py 之前生成 python 模块.

Since you only have SWIG interfaces and all of your python modules are generated wrappers (so in your case build_ext is not dependent on build_py), you can change the ordering of subcommands in build so build_ext is executed first, then the rest. This will ensure that the python modules are generated before build_py is executed.

基本上，您需要覆盖 build 命令类并更改 sub_commands 列表中元素的顺序.在python中有很多方法可以重新排序列表；下面是我的提议.为了重新排序列表，我使用了 itertools 中的列表条件分区配方 食谱:

Basically, you need override the build command class and change the ordering of elements in sub_commands list. There are lots of ways to reorder a list in python; below is my proposal. For reordering the list I used the list conditional partitioning recipe from itertools recipes:

import itertools
from setuptools import setup, Extension
from distutils.command.build import build as build_orig


def partition(pred, iterable):
    t1, t2 = itertools.tee(iterable)
    return itertools.filterfalse(pred, t1), filter(pred, t2)


class build(build_orig):

    def finalize_options(self):
        super().finalize_options()
        condition = lambda el: el[0] == 'build_ext'
        rest, sub_build_ext = partition(condition, self.sub_commands)
        self.sub_commands[:] = list(sub_build_ext) + list(rest)


setup(
    ...,
    cmdclass={'build': build},
)

这篇关于使用 setuptools 和 swig 构建模块的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！