使用Python进行平行精确矩阵对角化 [英] Parallel exact matrix diagonalization with Python

问题描述

有人知道并行精确矩阵对角化的实现版本(也许使用scipy/numpy)吗(等效地，找到本征系统)?如果有帮助，我的矩阵是对称且稀疏的.我不想花一天时间重新发明轮子.

Is anyone aware of an implemented version (perhaps using scipy/numpy) of parallel exact matrix diagonalization (equivalently, finding the eigensystem)? If it helps, my matrices are symmetric and sparse. I would hate to spend a day reinventing the wheel.

我的矩阵至少为10,000x10,000(但最好至少大20倍).目前，我只能使用4核Intel机器(具有超线程功能，因此每个内核2个进程)，每个机器约3.0Ghz，并具有12GB的RAM.以后我可能会访问具有256GB RAM的128核节点〜3.6Ghz/核，因此单机/多核应该这样做(对于我的其他并行任务，我一直在使用multiprocessing).我希望算法能够很好地扩展.

My matrices are at least 10,000x10,000 (but, preferably, at least 20 times larger). For now, I only have access to a 4-core Intel machine (with hyperthreading, so 2 processes per core), ~3.0Ghz each with 12GB of RAM. I may later have access to a 128-core node ~3.6Ghz/core with 256GB of RAM, so single machine/multiple cores should do it (for my other parallel tasks, I have been using multiprocessing). I would prefer for the algorithms to scale well.

我确实需要精确的对角线化，所以scipy.sparse例程对我来说并不好(尝试过，效果不佳).我一直在使用numpy.linalg.eigh(我看到只有一个内核可以完成所有计算).

I do need exact diagonalization, so scipy.sparse routines are not be good for me (tried, didn't work well). I have been using numpy.linalg.eigh (I see only single core doing all the computations).

(对于原始问题):是否存在一个在线资源，我可以在其中找到有关编译SciPy的更多信息，从而确保并行执行?

Alternatively (to the original question): is there an online resource where I can find out more about compiling SciPy so as to insure parallel execution?

推荐答案

对于对称稀疏矩阵特征值/特征向量查找，可以使用scipy.sparse.linalg.eigsh.它在后台使用ARPACK，并且有并行的ARPACK实现.如果您的scipy安装使用串行版本，则AFAIK，SciPy可以用一个进行编译.

For symmetric sparse matrix eigenvalue/eigenvector finding, you may use scipy.sparse.linalg.eigsh. It uses ARPACK behind the scenes, and there are parallel ARPACK implementations. AFAIK, SciPy can be compiled with one if your scipy installation uses the serial version.

但是，如果矩阵需要所有特征值和特征向量，这不是一个好答案，因为稀疏版本使用Lanczos算法.

However, this is not a good answer, if you need all eigenvalues and eigenvectors for the matrix, as the sparse version uses the Lanczos algorithm.

如果矩阵不是很大，则只需使用numpy.linalg.eigh.它使用LAPACK或BLAS，并且可能在内部使用并行代码.

If your matrix is not overwhelmingly large, then just use numpy.linalg.eigh. It uses LAPACK or BLAS and may use parallel code internally.

如果最终自己动手，请注意，SciPy/NumPy使用不同的高度优化的线性代数程序包(而不是纯Python)完成所有繁重的工作.因此，并行性的性能和程度在很大程度上取决于编译SciPy/NumPy安装所使用的库.

If you end up rolling your own, please note that SciPy/NumPy does all the heavy lifting with different highly optimized linear algebra packages, not in pure Python. Due to this the performance and degree of parallelism depends heavily on the libraries your SciPy/NumPy installation is compiled with.

(您的问题并没有揭示您是否只是想在多个处理器或几台计算机上运行并行代码.而且，矩阵的大小对最佳方法有很大影响.因此，此答案可能是完全正确的标记).

(Your question does not reveal if you just want to have parallel code running on several processors, or on several computers. Also, the size of your matrix has a big impact on the best method. So, this answer may be completely off-the-mark.)

这篇关于使用Python进行平行精确矩阵对角化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！