在将ubuntu从12.04升级到14.04之后，为什么使用python的numpy进行矩阵乘法会变得如此缓慢? [英] Why did matrix multiplication using python's numpy become so slow after upgrading ubuntu from 12.04 to 14.04?
问题描述
我曾经有过Ubuntu 12.04，最近又做了一个全新的Ubuntu 14.04安装.我正在研究的东西涉及大型矩阵的乘法运算(〜2000 X 2000)，为此我使用了numpy.我遇到的问题是，现在计算时间要长10到15倍.
I used to have Ubuntu 12.04 and recently did a fresh installation of Ubuntu 14.04. The stuff I'm working on involves multiplications of big matrices (~2000 X 2000), for which I'm using numpy. The problem I'm having is that now the calculations are taking 10-15 times longer.
从Ubuntu 12.04到14.04意味着从Python 2.7.3到2.7.6以及从numpy 1.6.1到1.8.1.但是，我认为这个问题可能与numpy链接到的线性代数库有关.我只能找到 libblas.so.3 和 liblapack，而不是 libblas.so.3gf 和 liblapack.so.3gf .如此.3.
Going from Ubuntu 12.04 to 14.04 implied going from Python 2.7.3 to 2.7.6 and from numpy 1.6.1 to 1.8.1. However, I think that the issue might have to do with the linear algebra libraries that numpy is linked to. Instead of libblas.so.3gf and liblapack.so.3gf, I can only find libblas.so.3 and liblapack.so.3.
我还安装了libopenblas和libatlas:
I also installed libopenblas and libatlas:
$ sudo apt-get install libopenblas-base libatlas3-base
并尝试了它们，但速度没有改变.所以，我的问题是:
and tried them, but the slowdown doesn't change. So, my questions are:
- 带有和不带有"gf"的软件包之间有什么区别?
- 这可能会导致矩阵乘法减慢吗?
- 如果是，我该如何返回libblas.so.3gf和liblapack.so.3gf?它们似乎在Ubuntu 14.04中不再使用.
非常感谢！
推荐答案
wim是正确的，因为问题可能是由numpy链接到较慢的BLAS库(例如，参考CBLAS库而不是ATLAS)引起的.
wim is correct, in that the problem is probably caused by numpy linking to a slower BLAS library (e.g. the reference CBLAS library rather than ATLAS).
您可以通过在numpy的已编译共享库之一上调用ldd实用程序来检查在运行时链接了哪个BLAS库.
You can check which BLAS library is being linked at runtime by calling the ldd utility on one of numpy's compiled shared libraries.
例如，如果您使用apt-get在标准位置安装了numpy:
For example, if you installed numpy in the standard location using apt-get:
~$ ldd /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so
...
libblas.so.3 => /usr/lib/libblas.so.3 (0x00007f01f0188000)
...
此输出告诉我numpy与/usr/lib/libblas.so.3链接.通常，这是到参考CBLAS库的符号链接，这很慢.
This output tells me that numpy is linked against /usr/lib/libblas.so.3. This is usually a symlink to the reference CBLAS library, which is pretty slow.
您可以按照wim的建议，删除通过apt-get安装的numpy版本，然后使用pip或直接下载源代码自行构建.但是，我会强烈劝阻您不要使用sudo pip install ...在系统范围内安装Python模块.这是个坏习惯，因为您可能会在系统范围的Python环境中破坏依赖关系.
You could, as wim suggests, remove the version of numpy installed via apt-get and build it yourself, either using pip or by downloading the source directly. However, I would strongly discourage you from using sudo pip install ... to install Python modules system-wide. This is a bad habit to get into, since you run the risk of breaking dependencies in your system-wide Python environment.
使用pip install --user ...安装到~/.local/目录中更为安全，甚至更好，将其安装到完全独立的
It is much safer to either install into your ~/.local/ directory using pip install --user ... or even better, to install into a completely self-contained virtualenv.
另一种选择是使用 update-alternatives 来强制系统范围内的numpy链接到另一个BLAS库.我在这里写了一个上一个答案，其中显示了如何执行此操作.
Another option would be to use update-alternatives to force your system-wide numpy to link against a different BLAS library. I've written a previous answer here that shows how to do this.
这篇关于在将ubuntu从12.04升级到14.04之后，为什么使用python的numpy进行矩阵乘法会变得如此缓慢?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
