在Numpy中执行零功能 [英] Performance of zeros function in Numpy
问题描述
我刚刚注意到numpy
的zeros
函数具有奇怪的行为:
I just noticed that the zeros
function of numpy
has a strange behavior :
%timeit np.zeros((1000, 1000))
1.06 ms ± 29.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit np.zeros((5000, 5000))
4 µs ± 66 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
另一方面,ones
似乎具有正常行为.
有人知道为什么用zeros
函数初始化一个小的numpy数组比花一个大的数组要花更多的时间吗?
On the other hand, ones
seems to have a normal behavior.
Is anybody know why initializing a small numpy array with the zeros
function takes more time than for a large array ?
(Python 3.5,numpy 1.11)
(Python 3.5, numpy 1.11)
推荐答案
这看起来像calloc
达到了一个阈值,在该阈值下,OS要求内存清零,而无需手动对其进行初始化.查看源代码,最终numpy.zeros
委托给calloc
以获得清零的内存块,如果与numpy.empty
进行比较,则不执行初始化:
This looks like calloc
hitting a threshold where it makes an OS request for zeroed memory and doesn't need to initialize it manually. Looking through the source code, numpy.zeros
eventually delegates to calloc
to acquire a zeroed memory block, and if you compare to numpy.empty
, which doesn't perform initialization:
In [15]: %timeit np.zeros((5000, 5000))
The slowest run took 12.65 times longer than the fastest. This could mean that a
n intermediate result is being cached.
100000 loops, best of 3: 10 µs per loop
In [16]: %timeit np.empty((5000, 5000))
The slowest run took 5.05 times longer than the fastest. This could mean that an
intermediate result is being cached.
100000 loops, best of 3: 10.3 µs per loop
您会看到np.zeros
对于5000x5000阵列没有初始化开销.
you can see that np.zeros
has no initialization overhead for the 5000x5000 array.
实际上,在您尝试访问该内存之前,该操作系统甚至没有真正"分配该内存.在无数TB可用空间的机器上,对TB级阵列的请求成功完成:
In fact, the OS isn't even "really" allocating that memory until you try to access it. A request for terabytes of array succeeds on a machine without terabytes to spare:
In [23]: x = np.zeros(2**40) # No MemoryError!
这篇关于在Numpy中执行零功能的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!