尝试使用distutils为mingw32交叉编译SWIG Python扩展时出错 [英] Error when trying to cross-compile SWIG Python extension for mingw32 using distutils

问题描述

我正在尝试使用distutils模块在Linux for Linux（mingw32）上交叉编译一个简单的SWIG Python扩展。

最终目标是为某个库编译一个Python包装器并能够在Windows上使用它。很明显，我从最基本的例子开始，不幸的是它失败了。

以下是我正在使用的文件：

example.c

  / *档案：example.c * / 
 
 / *全局变量* / 
 double Foo = 3.0; 
 
 / *计算正整数的最大公约数* / 
 int gcd（int x，int y）{
 int g; 
 g = y; 
 while（x> 0）{
 g = x; 
 x = y％x; 
 y = g; 
} 
 return g; 
} 
  

example.i - SWIG接口文件

  / *文件：example.i * / 
％模块示例
 
％inline％{ 
 extern int gcd（int x，int y）; 
 extern double Foo; 
％} 
  

setup.py

 ＃setup.py 
从distutils.core导入distutils 
导入设置，扩展
 
设置（name =SWIG example，
 version =1.0，
 ext_modules = [Extension（_ example，[example.i，example.c]）]）
  

为了使用本机（Linux）gcc编译器进行编译，我在调用：

  python setup.py build 
  

一切都像一个魅力！不幸的是，当试图指定Windows目标时：

  python setup.py build --compiler = mingw32 
  

我得到gcc无法识别-mdll开关的错误：

 运行构建
运行build_ext 
构建'_example'扩展
 swigging example.i转换为example_wrap.c 
 swig -python -o example_wrap .c example.i 
创建构建
创建构建/ temp.linux-x86_64-2.7 
 gcc -mdll -O -Wall -I / home / jojek / anaconda / include / python2.7 -c example_wrap.c -o build / temp.linux-x86_64-2.7 / example_wrap.o 
 gcc：错误：无法识别的命令行选项'-mdll'
错误：命令'gcc'失败，退出状态1 
  

非常合理，因为工具链是无效的。我确信我的机器上安装了 mingw32 。通过调用 dpkg -L mingw32 我知道编译器位于 / usr / bin / i586-mingw32msvc-gcc 中。

我的下一步是用我的编译器的实际路径覆盖CC环境变量。当我尝试再次编译它时，然后我得到以下错误，缺少 sys / select.h 头文件：

 运行构建
运行build_ext 
构建'_example'扩展
 swigging example.i转换为example_wrap.c 
 swig -python -o example_wrap.c example.i 
创建构建
创建构建/ temp.linux-x86_64-2.7 
 / usr / bin / i586-mingw32msvc-gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I / home / jojek / anaconda / include / python2.7 -c example_wrap.c -o build / temp.linux-x86_64-2.7 / example_wrap.o 
 example_wrap.c：1：警告：-fPIC忽略目标（所有代码都是位置独立的）
从/home/jojek/anaconda/include/python2.7/Python.h:58包含的文件中，来自example_wrap.c的
：125：
 /home/jojek/anaconda/include/python2.7/pyport.h:351:24：error：sys / select.h：No such file or directory 
错误：命令'/ usr / bin / i586-mingw32msvc-gcc'失败，退出st atus 1 
  

有人知道如何管理这项任务吗？

解决方案

使用distutils编译Python模块时，幕后会有相当多的事情发生。在你的问题中，每一次尝试都会越来越近，但是你现在遇到的问题是你正在使用Windows（交叉）编译器的Linux头文件。 （ sys / select.h isn不支持mingw32 ，但cygwin可能会是一个不同的故事）。实际上，缺少配置头文件会导致您的交叉编译尝试使用POSIX接口而不是Win32选项。

我的回答倒退了几个步骤并开始在Linux上使用mingw32简单地构建模块，然后当我们证明我们拥有了所有必需的时候，我们将使用distutils。

m还假设您没有Windows构建框（甚至虚拟机）可用于在Windows上简单地构建您的扩展，因为这比交叉编译简单得多。如果您正在阅读本文，并且可以选择使用Windows框来构建Windows Python扩展，那么请执行此操作并节省时间和精力。这表示可以仅使用Linux机器来构建Windows Python模块。

从mingw32开始安装并在您的Linux机器上工作（例如，使用Debian / Ubuntu软件包）第一步是获取Windows头文件（或更具体的配置）。我假设你将目标定位于大多数人在搜索引擎中键入python windows时获得的内部版本，因​​此我从 python.org ，并从那里提取它们。

我们想从两个方面获得Python发行版：

1. python27.dll（通常放在c：\ windows \ system32或c：\ windows \ 'include'目录（通常放在c：\python27\include）中

在Linux下，您可以通过几种不同的方式来提取它。您可以使用Wine来安装MSI文件。我使用cabextract和7z在我的测试中取得了成功，例如使用cabextract：

  cabextract /tmp/python-2.7。 10.msi -F'* .h'
 cabextract /tmp/python-2.7.10.msi -F'python27.dll'
  

（注意：如果你使用7z，你会在第二个名为'python'的内部存档中找到你真正想要的文件。）

此时，您还可以提取通常位于c：\python27\libs\中的文件libpython27.a，但该文件不足以使用mingw32进行链接。 >

考虑到头文件，我们现在已经有足够的资源来编译我们的扩展，尽管如上所述，为了让mingw32与python27.dll链接，我们需要先做更多的工作。我们将需要一个名为pexports的工具来列出Python DLL中的所有导出符号，并让dlltool为mingw32生成一个存根库以链接到它。我直接下载了 pexports ，然后将其解压缩为：

  tar xvf〜/ Downloads / pexports-0.47-mingw32-bin.tar.xz 
  

解压后，我们就得到一个单独的Windows可执行文件。我在这里使用Wine来直接运行它;或者，您可以提取源代码，并将其构建为在Linux主机上本地运行的工具：

  tar xvf〜/ Downloads / pexports-0.47-mingw32-src.tar.xz 
（cd pexports-0.47&& ./configure&& make）
  

或者您可以使用Python模块复制工具的功能 pefile （它运行良好的跨平台），以提取我们关心的出口，如果您希望避免使用Wine。

无论如何，pexports您可以生成一个.def文件，其中包含我们需要的dlltool信息：

  wine bin / pexports.exe -v python27。 dll> python27.def 
  

或者（如果您已经将pexports构建为本地工具），简单地说：

  ./ pexports-0.47 / pexports -v python27.dll> python27.def 
  

其中python27.dll是我们之前从.msi文件中提取的。

（这是我的 pexports参考资料）

获得.def文件后，您可以使用mingw32 dlltool生成一个.a文件，稍后我们将使用它来连接我们的Python模块：

  i586-mingw32msvc-dlltool -A --dllname python27.dll --def python27.def --output-lib libpython27.a 
  

现在我们已经达到了可以考虑运行SWIG本身来生成我们编译的代码。我进一步简化了您的示例界面，仅仅是：

 ％模块测试
 
％inline％{ 
 int gcd（int x，int y）{
 int g; 
 g = y; 
 while（x> 0）{
 g = x; 
 x = y％x; 
 y = g; 
} 
 return g; 
 
％} 
  

然后在我的Linux机器上运行SWIG：

  swig -Wall -python test.i 
  

这是我编译时生成的test_wrap.c文件：

  i586-mingw32msvc-gcc test_wrap.c -I ../ include -Wall -Wextra -shared -o _test.pyd ./libpython27.a 
  

我们有一个使用Linux构建的Windows Python模块。

为了检查它是否真正运行，我将test.py和_test.pyd复制到Windows框中然后执行：

  Python 2.7.10（默认，2015年5月23日，09:40:32）[MSC v.1500 32位（Intel）]在win32 
上输入help，copyright，credits或license以获取更多信息。 
>>>导入测试
>>> test.gcd（1024,512）
 512 
>>> 
  

现在剩下的就是确保distutils可以找到正确的包含文件和库链接操纵它的路径。

I am trying to cross-compile a simple SWIG Python extension on Linux for Windows (mingw32), using the distutils module.

The ultimate goal is to compile a Python wrapper for some library and being able to use it on Windows. Obviously I started with the most basic example and unfortunately it fails.

Here are the files I am using:

example.c

/* File : example.c */

/* A global variable */
double Foo = 3.0;

/* Compute the greatest common divisor of positive integers */
int gcd(int x, int y) {
  int g;
  g = y;
  while (x > 0) {
    g = x;
    x = y % x;
    y = g;
  }
  return g;
}

example.i - SWIG interface file

/* File : example.i */
%module example

%inline %{
extern int    gcd(int x, int y);
extern double Foo;
%}

setup.py

# setup.py
import distutils
from distutils.core import setup, Extension

setup(name = "SWIG example",
      version = "1.0",
      ext_modules = [Extension("_example", ["example.i","example.c"])])

In order to compile using the native (Linux) gcc compiler, I am invoking:

python setup.py build

Everything works like a charm! Unfortunately when trying to specify the Windows target:

python setup.py build --compiler=mingw32

I get the error saying that gcc can't recognize -mdll switch:

running build
running build_ext
building '_example' extension
swigging example.i to example_wrap.c
swig -python -o example_wrap.c example.i
creating build
creating build/temp.linux-x86_64-2.7
gcc -mdll -O -Wall -I/home/jojek/anaconda/include/python2.7 -c example_wrap.c -o build/temp.linux-x86_64-2.7/example_wrap.o
gcc: error: unrecognized command line option ‘-mdll’
error: command 'gcc' failed with exit status 1

Fair enough, it makes perfect sense, since toolchain is not valid. I made sure that mingw32 is installed on my machine. By calling dpkg -L mingw32 I know that compiler is located in /usr/bin/i586-mingw32msvc-gcc.

My next step was to override the CC environmental variable with the actual path to my compiler. When I try to compile it again, then I am getting the following error with missing sys/select.h header file:

running build
running build_ext
building '_example' extension
swigging example.i to example_wrap.c
swig -python -o example_wrap.c example.i
creating build
creating build/temp.linux-x86_64-2.7
/usr/bin/i586-mingw32msvc-gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/jojek/anaconda/include/python2.7 -c example_wrap.c -o build/temp.linux-x86_64-2.7/example_wrap.o
example_wrap.c:1: warning: -fPIC ignored for target (all code is position independent)
In file included from /home/jojek/anaconda/include/python2.7/Python.h:58,
                 from example_wrap.c:125:
/home/jojek/anaconda/include/python2.7/pyport.h:351:24: error: sys/select.h: No such file or directory
error: command '/usr/bin/i586-mingw32msvc-gcc' failed with exit status 1

Does anyone have an idea how to manage that task?

解决方案

There's quite a bit going on behind the scenes when you compile Python modules using distutils. You're getting closer with each try in your question, however the problem you've now encountered is that you're using Linux header files with a Windows (cross) compiler. (sys/select.h isn't supported with mingw32, cygwin might be a different story though). In reality it's the lack of the configuration header file that's causing your cross compile to try and use the POSIX interfaces instead of Win32 alternatives.

My answer rewinds a few steps and starts out simply building the module by hand, using mingw32 on Linux and then we'll look at using distutils once we've proven that we have all that's required.

I'm also assuming that you don't have a Windows build box (or even a VM) available to simply build your extension on Windows natively since that's far simpler than cross compiling. If you're reading this and have the option to use a Windows box to build your Windows Python extensions, do that instead and save time and effort. That said it is possible to build Windows Python modules using only a Linux box.

Starting with mingw32 already installed and working on your Linux box (e.g. using the Debian/Ubuntu packages) the first step is to get the Windows header files (or configuration to be more specific). I'm assuming you're targeting the build that most people get when they type "python windows" into a search engine so I downloaded the Windows MSI installers from python.org and extracted them from there.

There are two things we want to get from the Python distribution:

1. python27.dll (Usually gets placed in c:\windows\system32 or c:\windows\syswow64)
2. The 'include' directory (Usually gets placed in c:\python27\include)

Under Linux there are a few different ways you can extract this. You could use Wine to install the MSI file. I used both cabextract and 7z with success in my testing though, for example with cabextract:

cabextract /tmp/python-2.7.10.msi -F '*.h'
cabextract /tmp/python-2.7.10.msi -F 'python27.dll'

(Note: if you use 7z you'll find the files you really want inside a second, inner archive named 'python').

At this point you could also extract the file 'libpython27.a' which usually lives inside c:\python27\libs\ however this file isn't sufficient or even useful for linking using mingw32.

Given the header files we've now got enough to compile our extension, although as noted above to get mingw32 to link against python27.dll we need to do a bit more work first. We're going to need a tool called pexports to list all the exported symbols in the Python DLL and let dlltool generate a stub library for mingw32 to link against. I downloaded pexports directly and then extracted it with:

tar xvf ~/Downloads/pexports-0.47-mingw32-bin.tar.xz

Once that's extracted we get a single Windows executable. I used Wine in my example here to run it directly; alternatively, you could extract the source, and build it as a tool to run natively on the Linux host:

tar xvf ~/Downloads/pexports-0.47-mingw32-src.tar.xz
(cd pexports-0.47 && ./configure && make)

or you could have duplicated the functionality of the tool using the Python module pefile (which runs fine cross platform) to extract the exports that we care about if you were looking to avoid using Wine as well.

Anyway with pexports you can generate a .def file that contains the information we need for dlltool:

wine bin/pexports.exe -v python27.dll > python27.def

or, (if you've built pexports as a native tool), simply:

./pexports-0.47/pexports -v python27.dll > python27.def

where python27.dll is what we extracted from the .msi file earlier.

(This was my pexports reference)

Once you've got the .def file you can use the mingw32 dlltool to generate a .a file that we'll use later to link our Python module against:

i586-mingw32msvc-dlltool -A --dllname python27.dll --def python27.def --output-lib libpython27.a

Now we've reached a point where we can think about running SWIG itself to generate the code for us to compile. I simplified your example interface even further to be just:

%module test

%inline %{
int gcd(int x, int y) {
  int g;
  g = y;
  while (x > 0) {
    g = x;
    x = y % x;
    y = g;
  }
  return g;
}
%}

And then ran SWIG on my Linux box as:

swig -Wall -python test.i

This generated test_wrap.c which I compiled with:

i586-mingw32msvc-gcc test_wrap.c -I../include -Wall -Wextra -shared -o _test.pyd ./libpython27.a

And there we have a Windows Python module built using just Linux.

To check it really runs I copied test.py and _test.pyd to a Windows box and then did:

Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import test
>>> test.gcd(1024, 512)
512
>>>

Now all that remains is to make sure distutils can find the right include files and libraries to link against by manipulating its paths.

这篇关于尝试使用distutils为mingw32交叉编译SWIG Python扩展时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！