使用numpy在for循环中进行多核处理 [英] multi core processing in for loop using numpy

问题描述

我使用numpy计算了向量. 如何使用多核和numpy计算向量?

I calculated vector using numpy. How can I calculate vector using multicore and numpy?

import numpy as np

num_row, num_col = 6000, 13572

ss = np.ones((num_row, num_col), dtype=np.complex128)
ph = np.random.standard_normal(num_row)
fre = np.random.standard_normal(num_row)
tau = np.random.standard_normal(num_col)

for idx in range(num_row):
    ss[idx, :] *= np.exp(1j*(ph[idx] + fre[idx]*tau))

推荐答案

我们可以利用

We could leverage broadcasting to have a NumPy based solution -

ss = np.exp(1j*(ph[:,None] + fre[:,None]*tau))

将此内容移植到 numexpr 进行利用快速 transcendental操作以及多核心能力-

Porting this over to numexpr to leverage fast transcendental operations alongwith multi-core capability -

import numexpr as ne

def numexpr_soln(ph, fre):
    ph2D = ph[:,None]
    fre2D = fre[:,None]
    return ne.evaluate('exp(1j*(ph2D + fre2D*tau))')

时间-

In [23]: num_row, num_col = 6000, 13572
    ...: ss = np.ones((num_row, num_col), dtype=np.complex128)
    ...: ph = np.random.standard_normal(num_row)
    ...: fre = np.random.standard_normal(num_row)
    ...: tau = np.random.standard_normal(num_col)

# Original soln
In [25]: %%timeit
    ...: for idx in range(num_row):
    ...:     ss[idx, :] *= np.exp(1j*(ph[idx] + fre[idx]*tau))
1 loop, best of 3: 4.46 s per loop

# Native NumPy broadcasting soln
In [26]: %timeit np.exp(1j*(ph[:,None] + fre[:,None]*tau))
1 loop, best of 3: 4.58 s per loop

对于具有不同数量的内核/线程的Numexpr解决方案-

For Numexpr solution with varying number of cores/threads -

# Numexpr solution with # of threads = 2
In [51]: ne.set_num_threads(nthreads=2)
Out[51]: 2

In [52]: %timeit numexpr_soln(ph, fre)
1 loop, best of 3: 2.18 s per loop

# Numexpr solution with # of threads = 4
In [45]: ne.set_num_threads(nthreads=4)
Out[45]: 4

In [46]: %timeit numexpr_soln(ph, fre)
1 loop, best of 3: 1.62 s per loop

# Numexpr solution with # of threads = 8
In [48]: ne.set_num_threads(nthreads=8)
Out[48]: 8

In [49]: %timeit numexpr_soln(ph, fre)
1 loop, best of 3: 898 ms per loop

这篇关于使用numpy在for循环中进行多核处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！