如何使用像pybrain这样的外部python库在cython中编译我的python代码 [英] How to compile my python code in cython with external python libs like pybrain

问题描述

我需要更多性能来运行我的神经网络，所以我认为用cython构建它是一个好主意。我正在像这样构建我的代码：

I need more perfomance running my neural network, so I thinked that building it with cython will be good idea. I am building my code like this:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("my_code.pyx")
)

但是它将建立我使用的外部python文件吗？像pybrain，skimage和PIL一样。
如果不是，如何强制cython来构建它们。

But will it build external python files that I use? Like pybrain, skimage and PIL in my case. If not, how to force cython to build them.

推荐答案

不，外部python文件不会被cythonized并且除非您将它们专门作为扩展名添加到 setup.py 中。据我所知，没有简单的方法可以做到这一点。

No, external python files will not be cythonized and compiled unless you specifically add them to your setup.py as an extension. As far as I know there is no trivial way to do this.

这意味着所有对外部文件的调用都将在 Python空间中处理，因此可以不能充分利用Cython的潜力。例如，将对所有对外部文件的调用进行类型检查，这会浪费大量时间。如果使用 cython -a yourfile.pyx 来对文件进行cythonize，然后查看创建的C代码，就会看到此信息。黄色越多，则代码中的 pythony 就越多。

This means that all calls to the external files will be handled in 'Python-space' and hence can not use the full potential of Cython. For example all calls to an external file will be type checked, which wastes a lot of time. You can see this if you cythonize a file using cython -a yourfile.pyx and take a look at the created C code. The more yellow there is the more pythony your code is.

您有以下选择：

1. 查找提供Cython或C级访问的库/软件包。不幸的是，使用Cython可能会发现好的（或根本没有）好的机会，而为C库构建包装器的工作很多。注意，他们自己用C实现的包（例如numpy）已经相当快了。我不知道这与您所讨论的软件包的行为如何。乍一看，pybrains似乎是纯python。


2. 获取要使用的软件包的源代码，并使用Cython自己进行编译。


3. 使用诸如 lineprofiler / kernprof （优化时首先应始终始终）并尝试仅对运行时瓶颈进行cython。

1. Find libraries / packages that offer Cython or C-level access. Unfortunately chances are low that you will find good ones (or any at all) using Cython and building a wrapper for a C library is a lot of work. Note that packages that themselve are implemented in C (like numpy for example) already are reasonably fast. I do not know how this behaves with your packages in question. pybrains seems to be pure python from what I saw at first glance.
2. Get the source code of the packages you want to use and compile them yourself with Cython. This might be an awful lot of work and not worth the time.
3. Find the bottlenecks using a profiler like lineprofiler / kernprof (this should always be the first step when optimizing) and try to cythonize only the runtime bottlenecks.

我个人会选择第三个选项，因为第一个选项和第二个选项可能都需要大量工作才能取得可疑的结果。

I personally would go with option three, as options one and two both might require a lot of work on your side with questionable outcome.

这篇关于如何使用像pybrain这样的外部python库在cython中编译我的python代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！