`内置方法 numpy.core._multiarray_umath.implement_array_function` 是性能瓶颈吗? [英] Is `built-in method numpy.core._multiarray_umath.implement_array_function` a performance bottleneck?
问题描述
我在一些模拟中使用 numpy v1.18.2，并使用内置函数，例如 np.unique、np.diff 和np.interp.我在标准对象上使用这些函数，即列表或 numpy 数组.
I'm using numpy v1.18.2 in some simulations, and using inbuilt functions such as np.unique, np.diff and np.interp. I'm using these functions on standard objects, i.e lists or numpy arrays.
当我检查 cProfile 时，我看到这些函数调用了一个内置方法 numpy.core._multiarray_umath.implement_array_function 并且这个方法解释了32.5% 我的运行时间！据我所知，这是一个包装器，它执行一些检查以确保传递给函数的参数与函数兼容.
When I checked with cProfile, I saw that these functions make a call to an built-in method numpy.core._multiarray_umath.implement_array_function and that this method accounts for 32.5% of my runtime! To my understanding this is a wrapper that performs some checks to make sure the that the arguments passed to the function are compatible with the function.
我有两个问题:
- 这个函数 (
implement_array_function) 是否真的占用了太多时间，或者它实际上是我正在做的操作 (np.unique,np.diff,np.interp) 实际上一直在占用?也就是说，我是否误解了 cProfile 输出?我对snakeviz的分层输出感到困惑.请参阅此处的snakeviz输出以及此处. - 有什么办法可以禁用/绕过它，因为不需要每次都检查输入，因为我传递给这些 numpy 函数的参数已经在我的代码中控制了?我希望这能让我提高性能.
- Is this function (
implement_array_function) actually taking up so much time or is it actually the operations I'm doing (np.unique,np.diff,np.interp) that is actually taking up all this time? That is, am I misinterpreting the cProfile output? I was confused by the hierarchical output of snakeviz. Please see snakeviz output here and details for the function here. - Is there any way to disable it/bypass it, since the inputs need not be checked each time as the arguments I pass to these numpy functions are already controlled in my code? I am hoping that this will give me a performance improvement.
我已经看到了这个问题(什么是 numpy.core._multiarray_umath.implement_array_function 以及为什么它会花费大量时间?)，但我无法理解该功能究竟是什么或做什么.我也试图理解 NEP 18，但不能弄清楚如何准确解决问题.请填补我的知识空白并纠正任何误解.另外，如果有人能像我 5 岁(r/explainlikeimfive/)那样向我解释这一点，而不是假设具有开发人员级别的 Python 知识，我将不胜感激.
I already saw this question (what is numpy.core._multiarray_umath.implement_array_function and why it costs lots of time?), but I was not able to understand what exactly the function is or does. I also tried to understand NEP 18, but couldn't make out how to exactly solve the issue. Please fill in any gaps in my knowledge and correct any misunderstandings. Also I'd appreciate if someone can explain this to me like I'm 5 (r/explainlikeimfive/) instead of assuming developer level knowledge of python.
推荐答案
以下所有信息均来自 NEP 18.
All the information below is taken from NEP 18.
这个函数 (implement_array_function) 是否真的占用了太多时间，或者它实际上是我正在做的操作 (np.unique, np.diff, np.interp) 实际上一直在占用?
Is this function (
implement_array_function) actually taking up so much time or is it actually the operations I'm doing (np.unique,np.diff,np.interp) that is actually taking up all this time?
正如@hpaulj 在评论中正确提到的，调度程序的开销为每个 numpy 函数调用增加了 2-3 微秒.一旦在 C 中实现，这可能会缩短到 0.5-1 微秒.参见 此处.
As @hpaulj correctly mentioned in the comment, the overhead of the dispatcher adds 2-3 microseconds to each numpy function call. This will probably be shorten to 0.5-1 microseconds once it is implemented in C. See here.
有什么办法可以禁用/绕过它
Is there any way to disable it/bypass it
是的，从 NumPy 1.17 开始，您可以将环境变量 NUMPY_EXPERIMENTAL_ARRAY_FUNCTION 设置为 0(在导入 numpy 之前)，这将禁用 implement_array_function代码>(请参阅此处).类似的东西
Yes, from NumPy 1.17, you can set the environment variable NUMPY_EXPERIMENTAL_ARRAY_FUNCTION to 0 (before importing numpy) and this will disable the use of implement_array_function (See here). Something like
import os
os.environ['NUMPY_EXPERIMENTAL_ARRAY_FUNCTION'] = '0'
import numpy as np
然而，禁用它可能不会给你带来任何显着的性能提升，因为它的开销只有几微秒，这也将是以后 numpy 版本的默认设置.
However, disabling it probably would not give you any notable performance improvement as the overhead of it is just few microseconds, and this will be the default in later numpy version too.
这篇关于`内置方法 numpy.core._multiarray_umath.implement_array_function` 是性能瓶颈吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
