向量化pythonic方法来获取大于当前元素的元素数 [英] Vectorized pythonic way to get count of elements greater than current element

问题描述

我想将数组b中的每个条目与其相应的列进行比较，以找出(从该列中)有多少个条目大于引用.我的代码似乎令人尴尬地慢，我怀疑这是由于for循环而不是矢量运算引起的.

I'd like to compare every entry in array b with its respective column to find how many entries (from that column) are larger than the reference. My code seems embarrassingly slow and I suspect it is due to for loops rather than vector operations.

我们可以将以下代码矢量化"吗?

Can we 'vectorize' the following code?

import numpy as np

n = 1000
m = 200
b = np.random.rand(n,m)
p = np.zeros((n,m))

for i in range(0,n): #rows
   for j in range(0,m):  # cols
     r = b[i,j]  
     p[i,j] = ( ( b[:,j] > r).sum() ) / (n) 

经过深思熟虑，我认为对每一列进行排序将使以后的比较更快，从而改善整体运行时间.

After some more thought, I think sorting each column would improve overall runtime by making the later comparisons much faster.

经过更多搜索后，我相信我希望按列显示percentileofscore(

After some more searching I believe I want column-wise percentileofscore (http://lagrange.univ-lyon1.fr/docs/scipy/0.17.1/generated/scipy.stats.percentileofscore.html)

推荐答案

只需要更深入的研究就可以确定我们可以简单地在每列中使用argsort()索引来获得大于当前元素的计数在每次迭代中.

It just needed a bit of deeper study to figure out that we could simply use argsort() indices along each column to get the count of greater than the current element at each iteration.

方法1

牢记这一理论，一种解决方案是简单地使用两个argsort -ing来获取计数-

With that theory in mind, one solution would be simply using two argsort-ing to get the counts -

p = len(b)-1-b.argsort(0).argsort(0)

方法2

鉴于argsort索引是唯一数字，我们可以进一步优化它.因此，第二步argsort可以使用一些数组分配+ advanced-indexing，就像这样-

We could optimize it further, given the fact that the argsort indices are unique numbers. So, the second argsort step could use some array-assignment + advanced-indexing, like so -

def count_occ(b):
    n,m = b.shape     
    out = np.zeros((n,m),dtype=int)
    idx = b.argsort(0)
    out[idx, np.arange(m)] = n-1-np.arange(n)[:,None]
    return out

最后，对于两种方法，均按问题中的说明除以n.

Finally, divide by n as noted in the question for both the approaches.

确定到目前为止发布的所有方法-

Timing all the approaches posted thus far -

In [174]: np.random.seed(0)
     ...: n = 1000
     ...: m = 200
     ...: b = np.random.rand(n,m)

In [175]: %timeit (len(b)-1-b.argsort(0).argsort(0))/float(n)
100 loops, best of 3: 17.6 ms per loop

In [176]: %timeit count_occ(b)/float(n)
100 loops, best of 3: 12.1 ms per loop

# @Brad Solomon's soln
In [177]: %timeit np.sum(b > b[:, None], axis=-2) / float(n)
1 loop, best of 3: 349 ms per loop

# @marco romelli's loopy soln
In [178]: %timeit loopy_soln(b)/float(n)
10 loops, best of 3: 139 ms per loop

这篇关于向量化pythonic方法来获取大于当前元素的元素数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！