使用 setuptools 编译扩展模块的困难 [英] difficulties using setuptools to compile extension module
问题描述
我正在开发一个设计为 pip 安装的包(模块?).该包包含一个 C++ 扩展，使用 pybind11 包装，它链接到一些其他库.
我能够手动编译和链接扩展库而没有问题，但是我无法正确配置我的setup.py文件，以便扩展模块在调用 pip3 install my_package.
我一直在关注此代码(链接到通过 pybind11 文档)来配置我的 setup.py 文件，该文件包含在下面.
问题描述
为了完整起见，我的包目录如下所示:
my_package|-setup.py|-my_package|-__init__.py|-some_code.py|-src|my_extension.cpp|some_header.h我调用 pip3 install 如下:
pip3 install --user --upgrade --disable-pip-version-check .-vvv
(版本检查被禁用，因为机器无法访问互联网).
我可以从输出到控制台看到 gcc 似乎被调用了，但我对输出的了解不够，无法理解它是否被正确调用.my_extension 似乎构建没有任何错误:
构建my_extension"扩展创建 build/temp.linux-x86_64-3.7创建 build/temp.linux-x86_64-3.7/my_package创建 build/temp.linux-x86_64-3.7/my_package/srcgcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstackprotector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/include/python3.7m -I/home//.local/include/python3.7m -I/usr/include/eigen3 -I/usr/include/python3.7m -c my_package/src/my_extension.cpp -o build/temp.linux-x86_64-3.7/my_package/src/my_extension.o -O3 -fPIC -march=native -DVERSION_INFO="0.0.1" -std=c++14 -fvisibility=hidden 但是，无论出于何种原因，python 解释器都找不到 my_extension.定位 src 子模块:
我可以看到它是空的:
<预><代码>>>>ls/home/也就是说，这里似乎没有 my_extension.so 文件.
问题
下面的
setup.py是否存在任何明显问题?我很欣赏这是一个有点复杂的例子.如果没有，我该如何进一步解决此问题?我完全不知道如何进行.
我的 setup.py
from setuptools import setup, Extension从 setuptools.command.build_ext 导入 build_ext导入系统导入设置工具__version__ = '0.0.1'类 get_pybind_include(object):def __init__(self, user=False):self.user = 用户def __str__(self):导入pybind11返回 pybind11.get_include(self.user)ext_modules = [延期('my_extension',['my_package/src/my_extension.cpp'],include_dirs=[get_pybind_include(),get_pybind_include(user=True),'/usr/include/eigen3'],语言='c++'),]def has_flag(compiler, flagname):导入临时文件使用 tempfile.NamedTemporaryFile('w', suffix='.cpp') 作为 f:f.write('int main (int argc, char **argv) { return 0; }')尝试:compiler.compile([f.name], extra_postargs=[flagname])除了 setuptools.distutils.errors.CompileError:返回错误返回真def cpp_flag(编译器):if has_flag(compiler, '-std=c++14'):返回 '-std=c++14'elif has_flag(编译器，'-std=c++11'):返回 '-std=c++11'别的:raise RuntimeError('不支持的编译器——至少需要 C++11 支持！')类 BuildExt(build_ext):"""用于添加编译器特定选项的自定义构建扩展."""c_opts = {'msvc': ['/EHsc'],'unix': [],}如果 sys.platform == '达尔文':c_opts['unix'] += ['-stdlib=libc++', '-mmacosx-version-min=10.7']def build_extensions(self):ct = self.compiler.compiler_typeopts = self.c_opts.get(ct, [])# 额外的编译器选项选项 += ['-O3','-fPIC','-三月=本地']如果 ct == 'unix':opts.append('-DVERSION_INFO="%s"' % self.distribution.get_version())opts.append(cpp_flag(self.compiler))如果 has_flag(self.compiler, '-fvisibility=hidden'):opts.append('-fvisibility=hidden')elif ct == 'msvc':opts.append('/DVERSION_INFO=\\"%s\\"' % self.distribution.get_version())对于 self.extensions 中的 ext:ext.extra_compile_args = optsbuild_ext.build_extensions(self)设置(name='my_package',版本=__版本__，作者='',author_email='',网址='',说明='',long_description='',包 = setuptools.find_packages(),ext_modules=ext_modules,install_requires=['pybind11>=2.2'],cmdclass={'build_ext': BuildExt},zip_safe=假，)Extension('my_extension', ...) 创建一个可以导入的顶级扩展
import my_extension要使其成为包的子模块，请在包名前加上:
Extension('my_package.my_extension', ...)I am developing a package (module?) designed to be pip-installed. The package contains a C++ extension, wrapped using pybind11, which links to some other libraries.
I am able to compile and link the extension libraries manually without issue, however I am unable to configure my setup.py file properly such that the extension module installs when invoking pip3 install my_package.
I have been following this code (which is linked to by the pybind11 documentation) to configure my setup.py file, which is included below.
Description of the Problem
For completeness, my package directory looks something like this:
my_package
|-setup.py
|-my_package
|-__init__.py
|-some_code.py
|-src
| my_extension.cpp
| some_header.h
I invoke pip3 install as follows:
pip3 install --user --upgrade --disable-pip-version-check . -vvv
(the version check is disabled as the machine does not have access to the internet).
I can see from the output to the console that gcc appears to be invoked, but I do not know enough about the output to understand if it is invoked correctly. my_extension appears to be built without any errors:
building 'my_extension' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/my_package
creating build/temp.linux-x86_64-3.7/my_package/src
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/include/python3.7m -I/home/<my_user>/.local/include/python3.7m -I/usr/include/eigen3 -I/usr/include/python3.7m -c my_package/src/my_extension.cpp -o build/temp.linux-x86_64-3.7/my_package/src/my_extension.o -O3 -fPIC -march=native -DVERSION_INFO="0.0.1" -std=c++14 -fvisibility=hidden
However, for whatever reason, my_extension cannot be found by the python interpreter. Locating the src submodule:
>>> from my_package import src
>>> src.__path__
['/home/<my_user>/.local/lib/python3.7/site-packages/my_package/src']
I can see that it is empty:
>>> ls /home/<my_user>/.local/lib/python3.7/site-packages/my_package/src
__init__.py __pycache__
that is, there does not appear to be a my_extension.so file here.
Questions
Are there any obvious problems with the
setup.pybelow? I appreciate this is a somewhat complicated example.If not, how might I go about troubleshooting this issue further? I am completely stuck with how to procede.
My setup.py
from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext
import sys
import setuptools
__version__ = '0.0.1'
class get_pybind_include(object):
def __init__(self, user=False):
self.user = user
def __str__(self):
import pybind11
return pybind11.get_include(self.user)
ext_modules = [
Extension(
'my_extension',
['my_package/src/my_extension.cpp'],
include_dirs=[
get_pybind_include(),
get_pybind_include(user=True),
'/usr/include/eigen3'
],
language='c++'
),
]
def has_flag(compiler, flagname):
import tempfile
with tempfile.NamedTemporaryFile('w', suffix='.cpp') as f:
f.write('int main (int argc, char **argv) { return 0; }')
try:
compiler.compile([f.name], extra_postargs=[flagname])
except setuptools.distutils.errors.CompileError:
return False
return True
def cpp_flag(compiler):
if has_flag(compiler, '-std=c++14'):
return '-std=c++14'
elif has_flag(compiler, '-std=c++11'):
return '-std=c++11'
else:
raise RuntimeError('Unsupported compiler -- at least C++11 support is needed!')
class BuildExt(build_ext):
"""A custom build extension for adding compiler-specific options."""
c_opts = {
'msvc': ['/EHsc'],
'unix': [],
}
if sys.platform == 'darwin':
c_opts['unix'] += ['-stdlib=libc++', '-mmacosx-version-min=10.7']
def build_extensions(self):
ct = self.compiler.compiler_type
opts = self.c_opts.get(ct, [])
# extra compiler options
opts += [
'-O3',
'-fPIC',
'-march=native'
]
if ct == 'unix':
opts.append('-DVERSION_INFO="%s"' % self.distribution.get_version())
opts.append(cpp_flag(self.compiler))
if has_flag(self.compiler, '-fvisibility=hidden'):
opts.append('-fvisibility=hidden')
elif ct == 'msvc':
opts.append('/DVERSION_INFO=\\"%s\\"' % self.distribution.get_version())
for ext in self.extensions:
ext.extra_compile_args = opts
build_ext.build_extensions(self)
setup(
name='my_package',
version=__version__,
author='',
author_email='',
url='',
description='',
long_description='',
packages = setuptools.find_packages(),
ext_modules=ext_modules,
install_requires=['pybind11>=2.2'],
cmdclass={'build_ext': BuildExt},
zip_safe=False,
)
Extension('my_extension', …) creates a top-level extension that you can import with
import my_extension
To make it a submodule of your package prepend the name of the package:
Extension('my_package.my_extension', …)
这篇关于使用 setuptools 编译扩展模块的困难的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
