使用 Cython 和 C++ 进行项目组织 [英] Project organization with Cython and C++

问题描述

我想为我的 C++ 项目提供 Python 接口.从技术上讲，我决定使用 Cython 来包装 C++ 代码.随着时间的推移，整个项目旨在成为一个 Python 扩展模块，但起初，这是高度实验性的.渐渐地，C++ 类需要暴露给 Python.

I want to provide my C++ project with a Python interface. Technically, I have decided to use Cython for wrapping the C++ code. Over time, the entire project is meant to become a Python extension module, but at first, this is highly experimental. Gradually, C++ classes need to be exposed to Python.

我的问题是如何最好地组织文件和构建配置，以便 Cython 生成的和人工编写的 C++ 代码不会混合在一起，并且 Python 扩展模块与其他目标完全分离.

My question is how to best organize files and build configurations so that Cython-generated and human-written C++ code do not get mixed and the Python extension module is cleanly built seperate from the other targets.

我想像这样的目录结构用于源文件，以及一些用于 Cython 的构建目录.

I imagine a directory structure like this for the source files, and some build directory for Cython.

Project/
    src/
        *.h
        *.cpp
    cython/
        Project.pyx
        setup.py

推荐答案

基本上我有 3 个文件夹:

Basically I have 3 folders :

1. CPROJECT，C++ 库:生成一个 libcproject.so 共享对象
2. CYPPROJECT，cythonized Python 扩展:使用 Cython 生成 cyproject.so
3. DEPENDENCIES，依赖项:我复制两个项目的外部需求

1. CPROJECT, The C++ library : producing a libcproject.so shared object
2. CYPROJECT, The cythonized Python extension : producing the cyproject.so using Cython
3. DEPENDENCIES, The dependencies : where I copy external requirements for both projects

在 1. 我构建了 C++ 扩展(使用 gcc 编译 - -shared, -fPIC 编译选项)将暴露给python 并且 CYPPROJECT 依赖于向 Python 公开功能.作为后处理命令，生成的 .so 被复制到 DEPENDENCIES/libcproject/(以及 include 文件).这样，库当然也可以在纯 C++ 项目中独立使用.

In 1. I build the C++ extension (compiled with gcc - -shared, -fPIC compile options) that will be exposed to python and that the CYPROJECT relies on to expose features to Python. As a post processing command, the resulting .so is copied into DEPENDENCIES/libcproject/ (as well as the include files). This way the library is, of course, usable independently in a pure C++ project as well.

在 2. 我使用了 3 个子文件夹:

In 2. I make use of 3 sub-folders :

• adapters :主要包含 C++ 附加类(通常是从 libcproject.so 提供的类派生的类).这些通常是使用特定于 Cython 要求的功能增强的类(例如存储目标 Python 版本的 PyObject * C 版本 - 从 object 继承 - 给定类以及引用计数管理，通过 Py_XINCREF 和 Py_DECREF，...).
• pyext : 所有 Cython 手写 .pyx 文件的存储位置.
• setup :包含 setup.sh 脚本(用于设置依赖路径和调用 python setup.py build_ext --inplace用于生成最终的 cyproject.so(要添加到 PYTHONPATH)和 cyproject.pyx.

• adapters : which mainly contains C++ additional classes (often classes derived from the ones provided by libcproject.so). Those are usually classes that are enhanced with functionalities specific to Cython requirements (such as storing the PyObject * C version of a targeted Python version - inherited from object - of a given class and the reference counting management, via Py_XINCREF and Py_DECREF, ...).
• pyext : where are stored all the Cython hand written .pyx files.
• setup : containing the setup.sh script (for setting up the dependencies paths and calling the python setup.py build_ext --inplace for generating the final cyproject.so (to be added to the PYTHONPATH) and cyproject.pyx.

那么 setup 子文件夹中有什么?

So what's in the setup sub-folder ?

这是setup.sh的示例代码:

export PYTHONPATH=$PYTHONPATH:../../../DEPENDENCIES/Cython-0.18
export PATH=$PATH:../../../DEPENDENCIES/libcproject:../../../DEPENDENCIES/Cython-0.18/bin

# Note the `../../../DEPENDENCIES/libcproject`...

CC="gcc"   
CXX="g++"   
    python setup.py build_ext --inplace

这里是一个setup.py的例子(主要是为了演示额外的适配器是如何编译的):

And here an example of setup.py (mainly to demonstrate how the additional adapters are compiled):

import sys
import os
import shutil

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

# Cleaning
for root, dirs, files in os.walk(".", topdown=False):
    for name in files:
        if (name.startswith("cyproject") and not(name.endswith(".pyx"))):
            os.remove(os.path.join(root, name))
    for name in dirs:
        if (name == "build"):
            shutil.rmtree(name)

# Building
setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = [
    Extension("cyproject", 
              sources=["cyproject.pyx", 
                       "adapter/ALabSimulatorBase.cpp", 
                       "adapter/ALabSimulatorTime.cpp", 
                       "adapter/ALabNetBinding.cpp", 
                       "adapter/AValueArg.cpp", 
                       "adapter/ALabSiteSetsManager.cpp", 
                       "adapter/ALabSite.cpp", 
                       ],
              libraries=["cproject"],
              language="c++",
              extra_compile_args=["-I../inc", "-I../../../DEPENDENCIES/python2.7/inc", "-I../../../DEPENDENCIES/gsl-1.8/include"], 
              extra_link_args=["-L../lib"]
              extra_compile_args=["-fopenmp", "-O3"],
              extra_link_args=[]
              )
    ]
)                   

最后是主要的 .pyx，它将 cython 部分的所有手写 .pyx 链接在一起 [cyproject.pyx] :

And finally, the main .pyx, that links all the hand written .pyxs of the cython part together [cyproject.pyx] :

include "pyext/Utils.pyx" 
include "pyext/TCLAP.pyx" 
include "pyext/LabSimulatorBase.pyx"
include "pyext/LabBinding.pyx"
include "pyext/LabSimulatorTime.pyx"
...

注意:Cython 生成的所有文件都保留在这个 setup 文件夹中，与手写的东西(adapters 和 pyext)分开，正如预期的那样.

Note : All the files generated by Cython remains in this setup folder, well separated from the hand written stuffs (adapters and pyext), as expected.

在 3. 中使用单独的 DEPENDENCIES 文件夹可以将事情分开(以防我移动 CYPPROJECT - 及其依赖项- 在其他一些环境中).

In 3. Using a separated DEPENDENCIES folder allows to keep things well separated (in case I would move the CYPROJECT - and its dependencies - in some other environment).

所有这些都是为了让您对如何组织此类项目有一个概述(我希望是相关的).

All of this to give you an overview (a pertinent one, I hope) on how one can organize that sort of project.

这篇关于使用 Cython 和 C++ 进行项目组织的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！