为什么__getitem __(key)和get(key)比[key]慢得多? [英] Why are __getitem__(key) and get(key) significantly slower than [key]?

问题描述

据我了解，方括号不过是__getitem__的包装.这是我对此进行基准测试的方式:

It was my understanding that brackets were nothing more than a wrapper for __getitem__. Here is how I benchmarked this:

首先，我生成了一个半大字典.

First, I generated a semi-large dictionary.

items = {}
for i in range(1000000):
    items[i] = 1

然后，我使用cProfile测试以下三个功能:

Then, I used cProfile to test the following three functions:

def get2(items):
    for k in items.iterkeys():
        items.get(k)

def magic3(items):
    for k in items.iterkeys():
        items.__getitem__(k)

def brackets1(items):
    for k in items.iterkeys():
        items[k]

结果看起来像这样:

         1000004 function calls in 3.779 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    3.779    3.779 <string>:1(<module>)
        1    2.135    2.135    3.778    3.778 dict_get_items.py:15(get2)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  1000000    1.644    0.000    1.644    0.000 {method 'get' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'iterkeys' of 'dict' objects}


         1000004 function calls in 3.679 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    3.679    3.679 <string>:1(<module>)
        1    2.083    2.083    3.679    3.679 dict_get_items.py:19(magic3)
  1000000    1.596    0.000    1.596    0.000 {method '__getitem__' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'iterkeys' of 'dict' objects}


         4 function calls in 0.136 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.136    0.136 <string>:1(<module>)
        1    0.136    0.136    0.136    0.136 dict_get_items.py:11(brackets1)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'iterkeys' of 'dict' objects}

基准测试的问题所在吗?我尝试用简单的通过"替换括号访问，以确保实际访问数据，并发现通过"运行得更快.我对此的解释是，确实确实正在访问数据.我还尝试将其追加到新列表中，从而得到了类似的结果.

Is the issue in the way I am benchmarking? I tried replacing the bracket access with a simple "pass" to ensure that the data was actually being accessed, and found that "pass" was running much quicker. My interpretation of this was that the data was indeed being accessed. I've also tried appending to a new list, which gave similar results.

推荐答案

首先，由Not_a_Golfer发布的反汇编:

First, the disassembly posted by Not_a_Golfer:

>>> d = {1:2}
>>> dis.dis(lambda: d[1])
  1           0 LOAD_GLOBAL              0 (d)
              3 LOAD_CONST               1 (1)
              6 BINARY_SUBSCR       
              7 RETURN_VALUE   

>>> dis.dis(lambda: d.get(1))
  1           0 LOAD_GLOBAL              0 (d)
              3 LOAD_ATTR                1 (get)
              6 LOAD_CONST               1 (1)
              9 CALL_FUNCTION            1
             12 RETURN_VALUE  

>>> dis.dis(lambda: d.__getitem__(1))
  1           0 LOAD_GLOBAL              0 (d)
              3 LOAD_ATTR                1 (__getitem__)
              6 LOAD_CONST               1 (1)
              9 CALL_FUNCTION            1
             12 RETURN_VALUE

现在，正确地进行基准测试对于将任何内容读入结果显然很重要，而我对此所知不多.但是，假设确实存在差异(这对我来说很有意义)，这是我对为什么存在差异的猜测:

Now, getting the benchmarking right is obviously important to read anything into the results, and I don't know enough to help much there. But assuming there really is a difference (which makes sense to me), here's my guesses about why there is:

1. dict.get只是做更多"；它必须检查该键是否存在，如果不存在，则返回其第二个参数(默认为None).这意味着存在某种形式的条件捕获或异常捕获，因此，与检索与键关联的值的更基本的操作相比，它具有不同的时序特性，对此我完全不感到惊讶.

1. dict.get simply "does more"; it has to check if the key is present, and if not return its second argument (which defaults to None). This means there's some form of conditional or exception-catching, so I am completely unsurprised that this would have different timing characteristics to the more basic operation of retrieving the value associated with a key.

Python具有用于预订"操作的特定字节码(如反汇编所示).内置类型，包括dict，主要是用C语言实现的，它们的实现不一定要遵循普通的Python规则(仅需要它们的接口，并且即使在这里也有很多极端的情况).因此，我的猜测是BINARY_SUBSCR操作码的实现或多或少直接 到支持此操作的内置类型的基础C实现.对于这些类型，我希望实际上是__getitem__作为包装C实现的Python级别方法而存在，而不是括号语法调用Python级别的方法.

Python has a specific bytecode for the "subscription" operation (as demonstrated in the disassembly). The builtin types, including dict, are implemented primarily in C and their implementations do not necessarily play by the normal Python rules (only their interfaces are required to, and there are plenty of corner cases even there). So my guess would be that the implementation of the BINARY_SUBSCR opcode goes more-or-less directly to the underlying C implementations of builtin types that support this operation. For these types, I expect that it is actually __getitem__ that exists as a Python-level method to wrap the C implementation, rather than that the bracket syntax invokes the Python-level method.

对于实现__getitem__的自定义类的实例，以thing[key]为基准对thing.__getitem__(key)进行基准测试可能会很有趣；您实际上可能会看到相反的结果，因为BINARY_SUBSCR操作码将不得不在内部退回去做等同的工作来查找和调用该方法.

It might be interesting to benchmark thing.__getitem__(key) against thing[key] for an instance of a custom class that implements __getitem__; you might actually see the opposite results there as the BINARY_SUBSCR op-code would internally have to fall back to doing equivalent work to looking up the method and invoking it.

这篇关于为什么__getitem __(key)和get(key)比[key]慢得多?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！