使用LAPACK分发基于Cython的扩展 [英] Distributing Cython based extensions using LAPACK
问题描述
我正在编写一个包含Cython扩展并使用 LAPACK （和 BLAS ）的Python模块。我愿意使用 clapack 或 lapacke 或某种 f2c 或 f2py 解决方案。重要的是，我能够在紧密的循环中从Cython调用 lapack 和 blas 例程，而没有Python调用开销。
I am writing a Python module that includes Cython extensions and uses LAPACK (and BLAS). I am open to using either clapack or lapacke, or some kind of f2c or f2py solution if necessary. What is important is that I am able to call lapack and blas routines from Cython in tight loops without Python call overhead.
我找到了一个示例此处。但是，该示例取决于SAGE。我希望我的模块无需安装SAGE就可以安装，因为我的用户不太可能需要SAGE或其他任何东西。我的用户可能安装了numpy，scipy，pandas和scikit Learn之类的程序包，因此这些程序包是合理的依赖项。什么是要使用的接口的最佳组合？最小的setup.py文件看起来像什么，可以从中获取编译所需的信息（例如numpy，scipy等）？
I've found one example here. However, that example depends on SAGE. I want my module to be installable without installing SAGE, since my users are not likely to want or need SAGE for anything else. My users are likely to have packages like numpy, scipy, pandas, and scikit learn installed, so those would be reasonable dependencies. What is the best combination of interfaces to use, and what would the minimal setup.py file look like that could fetch the necessary information (from numpy, scipy, etc.) for compilation?
编辑：
这是我最后要做的。它可以在我的Macbook上使用，但我不知道它的便携性。当然，有更好的方法。
Here is what I ended up doing. It works on my macbook, but I have no idea how portable it is. Surely there's a better way.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
from Cython.Build import cythonize
from numpy.distutils.system_info import get_info
# TODO: This cannot be the right way
blas_include = get_info('blas_opt')['extra_compile_args'][1][2:]
includes = [blas_include,numpy.get_include()]
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = cythonize([Extension("cylapack", ["cylapack.pyx"],
include_dirs = includes,
libraries=['blas','lapack'])
])
)
之所以可行，是因为在我的Macbook上， clapack.h 头文件与 cblas.h 处于同一目录中。然后，我可以在pyx文件中执行此操作：
This works because, on my macbook, the clapack.h header file is in the same directory as cblas.h. I can then do this in my pyx file:
ctypedef np.int32_t integer
cdef extern from "cblas.h":
double cblas_dnrm2(int N,double *X, int incX)
cdef extern from "clapack.h":
integer dgelsy_(integer *m, integer *n, integer *nrhs,
double *a, integer *lda, double *b, integer *ldb, integer *
jpvt, double *rcond, integer *rank, double *work, integer *
lwork, integer *info)
推荐答案
这个问题正确无误，您可以将SciPy的Cython包装器用于BLAS和LAPACK例程。这些包装器记录在这里：
If I have understood the question correctly, you could make use of SciPy's Cython wrappers for BLAS and LAPACK routines. These wrappers are documented here:
- BLAS
- LAPACK
在文档说明中，您有责任检查传递给这些函数的所有数组是否与Fortran例程正确对齐。您可以根据需要在.pyx文件中简单地导入和使用这些功能。例如：
As the documentation states, you are responsible for checking that any arrays that you pass to these functions are aligned correctly for the Fortran routines. You can simply import and use these functions as needed in your .pyx file. For instance:
from scipy.linalg.cython_blas cimport dnrm2
from scipy.linalg.cython_lapack cimport dgelsy
鉴于这是经过良好测试的，广泛使用的代码，可在不同平台上运行，我认为它是可靠分发直接调用BLAS和LAPACK例程的Cython扩展的理想选择。
Given that this is well-tested, widely-used code that runs on different platforms, I'd argue that it is a good candidate for reliably distributing Cython extensions that directly call BLAS and LAPACK routines.
如果您不想使用代码要完全依赖SciPy，可以在SciPy的 linalg 目录此处。有用的参考是这几行setup.py 列出了源文件和头文件。请注意，需要使用Fortran编译器！
If you do not want your code to have a dependency on the entirety of SciPy, you can find many of the relevant files for these wrapper functions in SciPy's linalg directory here. A useful reference is these lines of setup.py which list the source and header files. Note that a Fortran compiler is required!
在理论中，应该可以仅在此处隔离编译BLAS所需的源文件。和LAPACK Cython包装器，然后将它们捆绑为模块的独立扩展。
In theory it should be possible to isolate only the source files here that are needed to compile the BLAS and LAPACK Cython wrappers and then bundle them as an independent extension with your module.
在实践中，这很奇怪。 linalg子模块的构建过程需要一些Python函数来帮助在不同平台上进行编译（例如，来自此处）。构建还依赖于其他C和Fortran源文件（此处 ），其路径被硬编码到这些Python函数中。
In practice this is very fiddly to do. The build process for the linalg submodule requires some Python functions to aid the compilation on different platforms (e.g. from here). Building also relies upon other C and Fortran source files (here), the paths of which are hard-coded into these Python functions.
显然，要确保SciPy在不同的操作系统和体系结构上正确编译，已经做了很多工作。
Clearly a lot of work has gone into making sure that SciPy compiles properly on different operating systems and architectures.
我确信这是可行的，但是在整理文件并调整路径之后，我还没有找到独立于SciPy其余部分来构建linalg子模块这一部分的正确方法。如果我找到正确的方法，我将确保更新此答案。
I'm sure it is possible to do, but after shuffling files about and tweaking paths, I have not yet found the right way to build this part of the linalg submodule independently from the rest of SciPy. Should I find the correct way, I'll be sure to update this answer.
这篇关于使用LAPACK分发基于Cython的扩展的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
