将ATLAS/MKL链接到已安装的Numpy [英] Link ATLAS/MKL to an installed Numpy

问题描述

TL; DR 如何在不重建的情况下将ATLAS/MKL链接到现有的Numpy.

TL;DR how to link ATLAS/MKL to existing Numpy without rebuilding.

我使用Numpy来计算大型矩阵，但发现它非常慢，因为Numpy仅使用1个核来进行计算.经过大量搜索后，我发现我的Numpy没有链接到某些优化的库，例如ATLAS/MKL.这是我的numpy配置:

I have used Numpy to calculate with the large matrix and I found that it is very slow because Numpy only use 1 core to do calculation. After doing a lot of search I figure that my Numpy does not link to some optimized library like ATLAS/MKL. Here is my config of numpy:

>>>import numpy as np
>>>np.__config__.show()
blas_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
lapack_info:
    libraries = ['lapack']
    library_dirs = ['/usr/lib']
    language = f77
atlas_threads_info:
    NOT AVAILABLE
blas_opt_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

基于这个原因，我想将ATLAS/MKL链接到Numpy.但是，我的Numpy是从PIP安装的，所以我不想手动安装，因为我想使用最新版本.我已经进行了一些搜索，但它们仅用于从头开始构建.因此，我的问题是:

For this reason, I want to link ATLAS/MKL to Numpy. However, my Numpy is installed from PIP so I don't want to install manually because I want to use the latest version. I have done some search but they are only for building from scratch. For this reason, my question are:

• 有什么方法可以将ATLAS/MKL链接到Numpy，而无需重新构建?
• 我发现配置信息保存在已安装的Numpy文件夹的 _ config _.py 中.那么修改它可以解决我的问题吗?如果是，请您告诉我如何?

推荐答案

假设您正在运行某种形式的linux，这是您可以执行的一种方法:

Assuming you're running some flavour of linux, here's one way you could do it:

1. 找出当前使用ldd链接到的BLAS库numpy.

1. Find out what BLAS library numpy is currently linked against using ldd.

• 对于numpy低于v1.10的版本:

$ ldd /<path_to_site-packages>/numpy/core/_dotblas.so

例如，如果我通过apt-get安装numpy，它将链接到

For example, if I install numpy via apt-get, it links to

...
libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fed81de8000)
...

如果_dotblas.so不存在，则可能意味着numpy在最初安装时未检测到任何BLAS库，在这种情况下，它根本不构建任何BLAS-相关组件.如果使用pip安装numpy而不手动指定BLAS库，则经常会发生这种情况(请参见下文).如果您要链接到外部BLAS库，恐怕别无选择，只能重建numpy.

If _dotblas.so doesn't exist, this probably means that numpy failed to detect any BLAS libraries when it was originally installed, in which case it simply doesn't build any of the BLAS-dependent components. This often happens if you install numpy using pip without manually specifying a BLAS library (see below). I'm afraid you'll have no option but to rebuild numpy if you want to link against an external BLAS library.

• 对于numpy v1.10及更高版本:

_dotblas.so已被删除 ，但是您应该可以检查multiarray.so的依赖项:

_dotblas.so has been removed from recent versions of numpy, but you should be able to check the dependencies of multiarray.so instead:

$ ldd /<path_to_site-packages>/numpy/core/multiarray.so

如果尚未安装ATLAS/MKL/OpenBLAS，请安装.顺便说一句，我绝对会推荐使用ATLAS上的OpenBLAS-看看此答案(尽管基准数据现在可能有点过时了.

Install ATLAS/MKL/OpenBLAS if you haven't already. By the way, I would definitely recommend OpenBLAS over ATLAS - take a look at this answer (although the benchmarking data is now probably a bit out of date).

使用update-alternatives创建指向您选择的新BLAS库的符号链接.例如，如果在/opt/OpenBLAS/lib中安装了libopenblas.so，则可以这样做:

Use update-alternatives to create a symlink to the new BLAS library of your choice. For example, if you installed libopenblas.so into /opt/OpenBLAS/lib, you would do:

$ sudo update-alternatives --install /usr/lib/libblas.so.3 \
                                     libblas.so.3 \
                                     /opt/OpenBLAS/lib/libopenblas.so \
                                     50

您可以为单个目标库配置多个符号链接，从而可以在多个已安装的BLAS库之间手动切换.

You can have multiple symlinks configured for a single target library, allowing you to manually switch between multiple installed BLAS libraries.

例如，当我调用$ sudo update-alternatives --config libblas.so.3时，我可以在3个库之一中进行选择:

For example, when I call $ sudo update-alternatives --config libblas.so.3, I can choose between one of 3 libraries:

  Selection    Path                                    Priority   Status
------------------------------------------------------------
  0            /opt/OpenBLAS/lib/libopenblas.so         40        auto mode
  1            /opt/OpenBLAS/lib/libopenblas.so         40        manual mode
  2            /usr/lib/atlas-base/atlas/libblas.so.3   35        manual mode
* 3            /usr/lib/libblas/libblas.so.3            10        manual mode

如果您真的想要numpy的最新"版本，则还可以查看我关于使用以下代码从源代码编译numpy的答案OpenBLAS集成.

If you really want the "newest" version of numpy, you could also take a look at my answer on compiling numpy from source with OpenBLAS integration.

正如@tndoan在评论中提到的，可以通过在~/.numpy-site.cfg中放置一个配置文件来使pip尊重numpy的特定配置-请参见

As @tndoan mentioned in the comments, it's possible to make pip respect a particular configuration for numpy by placing a config file in ~/.numpy-site.cfg - see this answer for more details.

我个人的喜好是手动配置和构建numpy.这并不是特别困难，它可以使您更好地控制numpy的配置.

My personal preference is to configure and build numpy by hand. It's not particularly difficult, and it gives you better control over numpy's configuration.

这篇关于将ATLAS/MKL链接到已安装的Numpy的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！