Distutils 显然因(工作)SWIG 扩展而失败 [英] Distutils appharently fails with a (working) SWIG extension

问题描述

我通过 SWIG 将 C 库包装在一个 python 模块中，这里称为myExample".如果我编译:

$swig -python myExample.i$gcc -c myExample_wrap.c -I/usr/lib/python2.7 -fPIC -std=c99$ld -shared myExample_wrap.so -llapacke -o _myExample.so

我获得了一个完整的工作模块(我使用的某些功能需要 liblapacke).现在我想通过pip install"安装这个模块.

根据 distutils 部分(https://docs.python.org/2.7/distutils/setupscript.html)，我写了我的 setup.py 文件:

from distutils.core 导入设置，扩展设置(名称='我的例子'，版本='0.1'，ext_modules=[Extension('_myExample',['myExample.i'], libraries= ['lapacke'])] )

并以某种方式编辑 MANIFEST.in 以保留来源并避免出现类似本网站上类似问题的问题(即仅包含 myExample.h 和 myExample.c).然后我跑:

$python setup.py sdist

并通过pip install"获得可安装的包.它似乎完成了(没有错误，没有警告)，但是...它不起作用.在这个可安装的模块中(_myExample.so" - 注意下划线，它似乎是 distutils 所要求的 [也许它隐藏了答案?])有些方法不同，有些方法缺失等等......因此我决定每次一步.通过编译:

$python setup.py build_ext

我已经遇到了同样的问题:最终的模块与通过开头解释的通常编译获得的模块不同.

总结:给定一个 SWIG 接口，传统编译它或通过 distutils 编译它会产生不同的结果.怎么可能?我的 setup.py 错了吗?是否有另一种方法可以在不依赖 distutils 或 setuptools(产生相同问题)的情况下获取 pip 可安装模块?

Ps:包装代码很长，因此很遗憾我无法提供详细列表，但我完全可以在需要时添加更多内容.例如，手动编译的接口成功包含AdaptiveInterpolation"(工作正常)，而distutil 生产的接口有AdaptiveInterpolation_set"、AdaptiveInterpolation_get"，或者有很多以new_"开头的方法(在我的原版中没有)代码).

解决方案

实际上，分发此类包有两种选择:bdist_wheel 和 sdist.

让我们以 distutils.

example.h

int fact(int n);

example.c

#include "example.h"事实上(int n){如果(n == 0){返回 1；}别的 {返回 n * 事实(n - 1)；}}

example.i

%module 示例%{#define SWIG_FILE_WITH_INIT#include "example.h"%}事实上(int n)；

让我们构建一个 SWIG 接口，它默认生成 example_wrap.c.

swig -python example.i

轮子

Python Wheels 是一种现代分发格式，它简化了预构建包的分发(即包的用户不需要编译器、开发头文件等来安装它).要使用它，我们需要 setuptools 和 wheel(可以使用 pip 或从操作系统存储库安装，sudo apt-get installpython-wheel python-setuptools).

setup.py

from setuptools import setup, Extension设置(名称 = '例子',版本 = '0.1',作者 = 'SWIG 文档',description = '来自文档的简单 swig 示例',ext_modules = [扩展名('_example', sources = ['example_wrap.c', 'example.c'])],py_modules = ['示例'],package_data = {'': ['example.h']} # sdist 需要)

您可以使用以下方法构建轮子:

python setup.py bdist_wheel

在我的机器上它产生 example-0.1-cp27-cp27mu-linux_x86_64.whl 我可以用 pip 安装并像 python -c 'import 一样测试例子;打印(example.fact(5))'.注意文件名.它对兼容的 Python 版本、ABI 和平台进行编码.这是内容列表(unzip -l ...).

存档:example-0.1-cp27-cp27mu-linux_x86_64.whl长度日期时间名称--------- ---------- ----- ----2609 2018-02-24 11:06 示例.py70968 2018-02-24 13:31 _example.so10 2018-02-24 14:58 example-0.1.dist-info/DESCRIPTION.rst290 2018-02-24 14:58 示例-0.1.dist-info/metadata.json17 2018-02-24 14:58 示例-0.1.dist-info/top_level.txt105 2018-02-24 14:58 示例-0.1.dist-info/WHEEL193 2018-02-24 14:58 示例-0.1.dist-info/METADATA617 2018-02-24 14:58 示例-0.1.dist-info/RECORD------- -------74809 8 个文件

要构建兼容性更好的轮子，例如，您可以查看 manylinux.>

来源

sdist 代表源代码分发，这意味着您的用户需要有一个编译器和相关的开发依赖项.源代码分发构建于:

python setup.py sdist

生成的 example-0.1.tar.gz 包含 (tar -tf ...):

example-0.1/示例-0.1/example.c示例-0.1/PKG-INFO示例-0.1/example.egg-info/示例-0.1/example.egg-info/PKG-INFOexample-0.1/example.egg-info/dependency_links.txt示例-0.1/example.egg-info/top_level.txt示例-0.1/example.egg-info/SOURCES.txt示例-0.1/setup.cfg示例-0.1/example.py示例-0.1/setup.py示例-0.1/example_wrap.c示例-0.1/example.h

I am wrapping via SWIG a C library in a python module, here called "myExample". If I compile:

$swig -python myExample.i
$gcc -c myExample_wrap.c -I /usr/lib/python2.7 -fPIC -std=c99
$ld -shared myExample_wrap.so -llapacke -o _myExample.so

I obtain a full working module (liblapacke is necessary for some functions I used). Now I'd like to make this module installable via "pip install".

According to the distutils section (https://docs.python.org/2.7/distutils/setupscript.html), I wrote my setup.py file:

from distutils.core import setup, Extension
setup(name='myExample',
version='0.1',
ext_modules=[Extension('_myExample',['myExample.i'], libraries= ['lapacke'])] )

and edited MANIFEST.in in a way to preserve the sources and avoid problems like in similar questions on this website (i.e. just including myExample.h and myExample.c). Then I run:

$python setup.py sdist

and obtained the package installable via "pip install". It seemed done (no errors, no warnings), but...it does not work. In this installable module ("_myExample.so" - notice the underscore, it seems to be required by distutils [maybe does it hide an answer?]) some methods are different, some are missing, etc... Consequently I decided to to one step per time. By just compiling:

$python setup.py build_ext

I already obtained the same problem: the final module is different from the one obtained via the usual compilation explained at the beginning.

Summing up: given a SWIG interface, compiling it traditionally or compiling it via distutils produces a different result. How could it be possible? Is my setup.py wrong? Is there maybe an alternative way for obtaining a pip-installable module without relying on distutils or setuptools(which produces the same problems)?

Ps: the wrap code is very long and so I cannot unfortunately give a detailed list, but I am fully available in adding more in case of need. For instance, the manually compiled interface successfully contains the "AdaptiveInterpolation" (working fine), while the distutil-produced one have "AdaptiveInterpolation_set", "AdaptiveInterpolation_get", or there are a lot of methods starting with "new_" (absent in my original code).

解决方案

Practically, there are two options for distribution of such package: bdist_wheel and sdist.

Let's take SWIG's example from the docs for distutils.

example.h

int fact(int n);

example.c

#include "example.h"

int fact(int n) {
    if (n == 0) {
        return 1;
    }
    else {
        return n * fact(n - 1);
    }
}

example.i

%module example

%{
#define SWIG_FILE_WITH_INIT
#include "example.h"
%}

int fact(int n);

Let's build a SWIG interface, which by default produces example_wrap.c.

swig -python example.i 

Wheel

Python Wheels is a modern distribution format which simplifies distribution of pre-built packages (i.e. users of the package will not need a compiler, development headers and the like to install it). To use it we'll need setuptools and wheel (can be installed with pip or from OS repositories, sudo apt-get install python-wheel python-setuptools).

setup.py

from setuptools import setup, Extension


setup(
  name = 'example',
  version = '0.1',
  author = 'SWIG Docs',
  description = 'Simple swig example from docs',
  ext_modules = [
    Extension('_example', sources = ['example_wrap.c', 'example.c'])],
  py_modules = ['example'],
  package_data = {'': ['example.h']}  # needed for sdist
)

You can build the wheel with:

python setup.py bdist_wheel

On my machine it produces example-0.1-cp27-cp27mu-linux_x86_64.whl which I can install with pip and test like python -c 'import example; print(example.fact(5))'. Note the filename. It encodes compatible Python version, ABI and platform. Here's listing of the contents (unzip -l ...).

Archive:  example-0.1-cp27-cp27mu-linux_x86_64.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
     2609  2018-02-24 11:06   example.py
    70968  2018-02-24 13:31   _example.so
       10  2018-02-24 14:58   example-0.1.dist-info/DESCRIPTION.rst
      290  2018-02-24 14:58   example-0.1.dist-info/metadata.json
       17  2018-02-24 14:58   example-0.1.dist-info/top_level.txt
      105  2018-02-24 14:58   example-0.1.dist-info/WHEEL
      193  2018-02-24 14:58   example-0.1.dist-info/METADATA
      617  2018-02-24 14:58   example-0.1.dist-info/RECORD
---------                     -------
    74809                     8 files

To build wheels with better compatibility, you can, for instance, look at manylinux.

Source

sdist stands for source distribution, which means your users will need to have a compiler and relevant development dependencies. Source distribution is built with:

python setup.py sdist

Produced example-0.1.tar.gz contains (tar -tf ...):

example-0.1/
example-0.1/example.c
example-0.1/PKG-INFO
example-0.1/example.egg-info/
example-0.1/example.egg-info/PKG-INFO
example-0.1/example.egg-info/dependency_links.txt
example-0.1/example.egg-info/top_level.txt
example-0.1/example.egg-info/SOURCES.txt
example-0.1/setup.cfg
example-0.1/example.py
example-0.1/setup.py
example-0.1/example_wrap.c
example-0.1/example.h

这篇关于Distutils 显然因(工作)SWIG 扩展而失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！