根据字典有效地替换数组中的元素-NumPy/Python [英] Efficiently replace elements in array based on dictionary - NumPy / Python

问题描述

首先，如果在其他地方回答过我，我深表歉意.我所能找到的只是有关替换给定值的元素的问题，而不是有关多个值的元素的问题.

First, of all, my apologies if this has been answered elsewhere. All I could find were questions about replacing elements of a given value, not elements of multiple values.

我有数千个大型np.array，就像这样:

I have several thousand large np.arrays, like so:

# generate dummy data
input_array = np.zeros((100,100))
input_array[0:10,0:10] = 1
input_array[20:56, 21:43] = 5
input_array[34:43, 70:89] = 8

在这些数组中，我想根据字典替换值:

In those arrays, I want to replace values, based on a dictionary:

mapping = {1:2, 5:3, 8:6}

方法

这时，我正在使用一个简单的循环，并结合了华丽的索引编制:

approach

At this time, I am using a simple loop, combined with fancy indexing:

output_array = np.zeros_like(input_array)

for key in mapping:
    output_array[input_array==key] = mapping[key]

问题

我的数组的尺寸是2000到2000年，字典有大约1000个条目，因此，这些循环要花很长时间.

problem

My arrays have dimensions of 2000 by 2000, the dictionaries have around 1000 entries, so, these loops take forever.

是否有一个函数，该函数仅采用字典(或类似形式)形式的数组和映射，并输出更改后的值?

is there a function, that simply takes an array and a mapping in the form of a dictionary (or similar), and outputs the changed values?

非常感谢您的帮助！

我使用

%%timeit -r 10 -n 10

import numpy as np
np.random.seed(123)

sources = range(100)
outs = [a for a in range(100)]
np.random.shuffle(outs)
mapping = {sources[a]:outs[a] for a in(range(len(sources)))}

对于每种解决方案:

np.random.seed(123)
input_array = np.random.randint(0,100, (1000,1000))

divakar，方法3:

%%timeit -r 10 -n 10
k = np.array(list(mapping.keys()))
v = np.array(list(mapping.values()))

mapping_ar = np.zeros(k.max()+1,dtype=v.dtype) #k,v from approach #1
mapping_ar[k] = v
out = mapping_ar[input_array]

5.01 ms ± 641 µs per loop (mean ± std. dev. of 10 runs, 10 loops each)

divakar，方法2:

%%timeit -r 10 -n 10
k = np.array(list(mapping.keys()))
v = np.array(list(mapping.values()))

sidx = k.argsort() #k,v from approach #1

k = k[sidx]
v = v[sidx]

idx = np.searchsorted(k,input_array.ravel()).reshape(input_array.shape)
idx[idx==len(k)] = 0
mask = k[idx] == input_array
out = np.where(mask, v[idx], 0)

56.9 ms ± 609 µs per loop (mean ± std. dev. of 10 runs, 10 loops each)

divakar，方法1:

%%timeit -r 10 -n 10

k = np.array(list(mapping.keys()))
v = np.array(list(mapping.values()))

out = np.zeros_like(input_array)
for key,val in zip(k,v):
    out[input_array==key] = val

113 ms ± 6.2 ms per loop (mean ± std. dev. of 10 runs, 10 loops each)

eelco:

%%timeit -r 10 -n 10
output_array = npi.remap(input_array.flatten(), list(mapping.keys()), list(mapping.values())).reshape(input_array.shape)

143 ms ± 4.47 ms per loop (mean ± std. dev. of 10 runs, 10 loops each)

yatu

%%timeit -r 10 -n 10

keys, choices = list(zip(*mapping.items()))
# [(1, 5, 8), (2, 3, 6)]
conds = np.array(keys)[:,None,None]  == input_array
np.select(conds, choices)

157 ms ± 5 ms per loop (mean ± std. dev. of 10 runs, 10 loops each)

原始的循环方法:

%%timeit -r 10 -n 10
output_array = np.zeros_like(input_array)

for key in mapping:
    output_array[input_array==key] = mapping[key]

187 ms ± 6.44 ms per loop (mean ± std. dev. of 10 runs, 10 loops each)

感谢超级帮助！

推荐答案

方法1:使用数组数据循环1

一种方法是提取数组中的键和值，然后使用类似的循环-

One approach would be extracting the keys and values in arrays and then use a similar loop -

k = np.array(list(mapping.keys()))
v = np.array(list(mapping.values()))

out = np.zeros_like(input_array)
for key,val in zip(k,v):
    out[input_array==key] = val

与原始数据相比，此数据的优势在于用于高效数据提取的数组数据的空间局部性，该变量在迭代中使用.

Benefit with this one over the original one is the spatial-locality of the array data for efficient data-fetching, which is used in the iterations.

此外，由于您提到了thousand large np.arrays.因此，如果mapping词典保持不变，那么获取阵列版本的步骤-k和v将是一次性设置过程.

Also, since you mentioned thousand large np.arrays. So, if the mapping dictionary stays the same, that step to get the array versions - k and v would be a one-time setup process.

方法2:使用searchsorted

Approach #2 : Vectorized one with searchsorted

可以使用 np.searchsorted -

sidx = k.argsort() #k,v from approach #1

k = k[sidx]
v = v[sidx]

idx = np.searchsorted(k,input_array.ravel()).reshape(input_array.shape)
idx[idx==len(k)] = 0
mask = k[idx] == input_array
out = np.where(mask, v[idx], 0)

方法3:使用映射数组对整数键进行矢量化处理的一种方法

使用整数数组的映射数组可以建议使用矢量化的数组，当它由输入数组索引时，将直接导致最终输出-

A vectorized one could be suggested using a mapping array for integer keys, which when indexed by the input array would lead us directly to the final output -

mapping_ar = np.zeros(k.max()+1,dtype=v.dtype) #k,v from approach #1
mapping_ar[k] = v
out = mapping_ar[input_array]

这篇关于根据字典有效地替换数组中的元素-NumPy/Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！