将 ATLAS/MKL 链接到已安装的 Numpy [英] Link ATLAS/MKL to an installed Numpy

问题描述

TL;DR 如何在不重建的情况下将 ATLAS/MKL 链接到现有的 Numpy.

TL;DR how to link ATLAS/MKL to existing Numpy without rebuilding.

我用Numpy计算过大矩阵，我发现它很慢，因为Numpy只使用1个核来做计算.经过大量搜索后，我发现我的 Numpy 没有链接到一些优化的库，如 ATLAS/MKL.这是我的 numpy 配置:

I have used Numpy to calculate with the large matrix and I found that it is very slow because Numpy only use 1 core to do calculation. After doing a lot of search I figure that my Numpy does not link to some optimized library like ATLAS/MKL. Here is my config of numpy:

>>>import numpy as np
>>>np.__config__.show()
blas_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
lapack_info:
    libraries = ['lapack']
    library_dirs = ['/usr/lib']
    language = f77
atlas_threads_info:
    NOT AVAILABLE
blas_opt_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

出于这个原因，我想将 ATLAS/MKL 链接到 Numpy.但是，我的 Numpy 是从 PIP 安装的，所以我不想手动安装，因为我想使用最新版本.我进行了一些搜索，但它们仅用于从头开始构建.因此，我的问题是:

For this reason, I want to link ATLAS/MKL to Numpy. However, my Numpy is installed from PIP so I don't want to install manually because I want to use the latest version. I have done some search but they are only for building from scratch. For this reason, my question are:

• 有没有办法将 ATLAS/MKL 链接到 Numpy 而无需再次重建?
• 我发现配置信息保存在 Numpy 安装文件夹中的 _config_.py 中.那么修改它会解决我的问题吗?如果是，请告诉我如何操作?

推荐答案

假设您正在运行某种风格的 linux，那么您可以采用以下一种方法:

Assuming you're running some flavour of linux, here's one way you could do it:

1. 使用 ldd 找出当前链接的 BLAS 库 numpy.

1. Find out what BLAS library numpy is currently linked against using ldd.

• 对于早于 v1.10 的 numpy 版本:

$ ldd /<path_to_site-packages>/numpy/core/_dotblas.so

例如，如果我通过 apt-get 安装 numpy，它会链接到

For example, if I install numpy via apt-get, it links to

...
libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fed81de8000)
...

如果 _dotblas.so 不存在，这可能意味着 numpy 在最初安装时未能检测到任何 BLAS 库，在这种情况下它根本就没有t 构建任何依赖于 BLAS 的组件.如果您使用 pip 安装 numpy 而没有手动指定 BLAS 库(见下文)，则经常会发生这种情况.如果您想链接到外部 BLAS 库，恐怕您别无选择，只能重建 numpy.

If _dotblas.so doesn't exist, this probably means that numpy failed to detect any BLAS libraries when it was originally installed, in which case it simply doesn't build any of the BLAS-dependent components. This often happens if you install numpy using pip without manually specifying a BLAS library (see below). I'm afraid you'll have no option but to rebuild numpy if you want to link against an external BLAS library.

• 对于 numpy v1.10 及更新版本:

_dotblas.so 已从 numpy 的最新版本中删除，但您应该可以改为检查 multiarray.so 的依赖项:

_dotblas.so has been removed from recent versions of numpy, but you should be able to check the dependencies of multiarray.so instead:

$ ldd /<path_to_site-packages>/numpy/core/multiarray.so

如果您还没有安装 ATLAS/MKL/OpenBLAS.顺便说一句，我肯定会推荐 OpenBLAS 而不是 ATLAS - 看看这个答案(尽管基准数据现在可能是有点过时了).

Install ATLAS/MKL/OpenBLAS if you haven't already. By the way, I would definitely recommend OpenBLAS over ATLAS - take a look at this answer (although the benchmarking data is now probably a bit out of date).

使用 update-alternatives 创建指向您选择的新 BLAS 库的符号链接.例如，如果您将 libopenblas.so 安装到 /opt/OpenBLAS/lib 中，您将:

Use update-alternatives to create a symlink to the new BLAS library of your choice. For example, if you installed libopenblas.so into /opt/OpenBLAS/lib, you would do:

$ sudo update-alternatives --install /usr/lib/libblas.so.3 
                                     libblas.so.3 
                                     /opt/OpenBLAS/lib/libopenblas.so 
                                     50

您可以为单个目标库配置多个符号链接，从而允许您在多个已安装的 BLAS 库之间手动切换.

You can have multiple symlinks configured for a single target library, allowing you to manually switch between multiple installed BLAS libraries.

例如，当我调用 $ sudo update-alternatives --config libblas.so.3 时，我可以选择 3 个库之一:

For example, when I call $ sudo update-alternatives --config libblas.so.3, I can choose between one of 3 libraries:

  Selection    Path                                    Priority   Status
------------------------------------------------------------
  0            /opt/OpenBLAS/lib/libopenblas.so         40        auto mode
  1            /opt/OpenBLAS/lib/libopenblas.so         40        manual mode
  2            /usr/lib/atlas-base/atlas/libblas.so.3   35        manual mode
* 3            /usr/lib/libblas/libblas.so.3            10        manual mode

如果你真的想要最新"版本的 numpy，你也可以看看 我关于从源代码编译 numpy 的回答OpenBLAS 集成.

If you really want the "newest" version of numpy, you could also take a look at my answer on compiling numpy from source with OpenBLAS integration.

正如@tndoan 在评论中提到的，通过在 ~/.numpy-site.cfg 中放置一个配置文件，可以使 pip 尊重 numpy 的特定配置- 有关详细信息，请参阅此答案.

As @tndoan mentioned in the comments, it's possible to make pip respect a particular configuration for numpy by placing a config file in ~/.numpy-site.cfg - see this answer for more details.

我个人的偏好是手动配置和构建 numpy.这不是特别困难，它可以让你更好地控制 numpy 的配置.

My personal preference is to configure and build numpy by hand. It's not particularly difficult, and it gives you better control over numpy's configuration.

这篇关于将 ATLAS/MKL 链接到已安装的 Numpy的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！