Python/Numpy-掩码数组非常慢 [英] Python/Numpy - Masked Arrays are Very Slow

问题描述

我可以做些什么来加快numpy中的蒙版数组吗?我有一个非常低效的函数，我改写为使用掩码数组(在这里，我可以只掩码行，而不是像我所做的那样进行复制和删除行).但是，我震惊地发现被屏蔽的功能要慢10倍，因为被屏蔽的数组要慢得多.

Is there anything I can do to speed up masked arrays in numpy? I had a terribly inefficient function that I re-wrote to use masked arrays (where I could just mask rows instead of make copies and delete rows as I was doing). However, I was shocked to find that the masked function was 10x slower because the masked arrays are so much slower.

例如，采取以下措施(对我而言，被遮罩的速度要慢6倍以上):

As an example, take the following (masked is more then 6 times slower for me):

import timeit
import numpy as np
import numpy.ma as ma

def test(row):
   return row[0] + row[1]

a = np.arange(1000).reshape(500, 2)
t = timeit.Timer('np.apply_along_axis(test, 1, a)','from __main__ import test, a, np')
print round(t.timeit(100), 6)

b = ma.array(a)
t = timeit.Timer('ma.apply_along_axis(test, 1, b)','from __main__ import test, b, ma')
print round(t.timeit(100), 6)

推荐答案

我不知道为什么蒙版数组函数如此缓慢地移动，但是由于听起来您正在使用蒙版来选择行(而不是单个值) )，则可以从被屏蔽的行创建一个常规数组，并改用np函数:

I have no idea why the masked array functions are moving so slowly, but since it sounds like you are using the mask to select rows (as opposed to individual values), you can create a regular array from the masked rows and use the np function instead:

b.mask = np.zeros(500)
b.mask[498] = True
t = timeit.Timer('c=b.view(np.ndarray)[~b.mask[:,0]]; np.apply_along_axis(test, 1, c)','from __main__ import test, b, ma, np')
print round(t.timeit(100), 6)

更好的是，根本不使用掩码数组；只需将您的数据和一维掩码数组作为单独的变量进行维护:

Better yet, don't use masked arrays at all; just maintain your data and a 1D mask array as separate variables:

a = np.arange(1000).reshape(500, 2)
mask = np.ones(a.shape[0], dtype=bool)
mask[498] = False
out = np.apply_along_axis(test, 1, a[mask])

这篇关于Python/Numpy-掩码数组非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！