的Python：对于套1的，-1的，和0的内存有效的矩阵创建由SciPy的最小二乘法进行优化 [英] Python: Memory-efficient matrix creation for sets of 1's, -1's, and 0s to be optimized by scipy least squares

问题描述

我通过遍历字符串列表并将其转换为1的，-1的，和0的阵列。例如 - 我可以具有以下列表：

I'm iterating through a list of strings and translating them into arrays of 1's, -1's, and 0's. For example - I may have the following list:

A,B,-C
A,-D
B,C,-D

这将成为一个biglist等于：

Which will become a "biglist" equal to:

[
 [1  1 -1  0],
 [1  0  0 -1],
 [0  1  1 -1]
]

目前，我只是通过串的每一行循环，1或-1，如果它是独一无二的，归零不存在的那些（例如字符串，D不是$分配的值在第一线p $ psent，所以它是0）。我做上面的傻方式基本上是：

At the moment, I'm simply looping through every line of strings, assigning values of 1 or -1 to the string if it is unique, and zeroing out the ones that do not exist (for example, D is not present in the first line, so it is 0). The silly way I'm doing the above is basically:

for line_of_strings in all_strings:
    for the_string in line_of_strings:
        entry[string_index] = (1 or -1)

    biglist.append(entry)

最后，我有一个很好的一套名单上，我运行：

Eventually, I have a nice set of lists on which I run:

scipy.optimize.nnls(biglist)

这工作，但缠占用的内存和时间一卡车。有没有更有效的方式去了解呢？使用numpy的或SciPy的阵列/矩阵，也许？

This works, but winds up taking up a truckload of memory and time. Is there a more efficient way to go about this? Using numpy or scipy arrays/matrices, perhaps?

推荐答案

使用，而不是列出numpy的阵列似乎使相当多的差别时间上，至少在一个简单的例子：

Using numpy arrays instead of lists seems to make quite a bit of difference timewise, at least in a trivial example:

$ python -mtimeit -s"from scipy.optimize import nnls; m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]; b=[1, 2, 3]" "nnls(m, b)"
10000 loops, best of 3: 38.5 usec per loop

$ python -mtimeit -s"import numpy as np; from scipy.optimize import nnls; m = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]); b=[1, 2, 3]" "nnls(m, b)"
100000 loops, best of 3: 20 usec per loop

$ python -mtimeit -s"import numpy as np; from scipy.optimize import nnls; m = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]); b=np.array([1, 2, 3])" "nnls(m, b)"
100000 loops, best of 3: 11.4 usec per loop

我期待numpy的阵列将拥有更小的内存占用为好。如果输入是相当稀疏，如果性能仍不令人满意，这可能是值得研究，如果 NNLS 接受稀疏矩阵。

这篇关于的Python：对于套1的，-1的，和0的内存有效的矩阵创建由SciPy的最小二乘法进行优化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！