PEP 412 是否使 __slots__ 变得多余? [英] Does PEP 412 make __slots__ redundant?

问题描述

PEP 412，在 Python 3.3 中实现，引入改进的属性字典处理，有效减少类实例的内存占用.__slots__ 也是为了同样的目的而设计的，那么再使用 __slots__ 有什么意义吗?

为了自己找出答案，我运行了以下测试，但结果没有多大意义:

class Slots(object):__slots__ = ['a', 'b', 'c', 'd', 'e']def __init__(self):自我.a = 1自我.b = 1自我.c = 1自我.d = 1自我.e = 1类 NoSlots(对象):def __init__(self):自我.a = 1自我.b = 1自我.c = 1自我.d = 1自我.e = 1

Python 3.3 结果:

<预><代码>>>>sys.getsizeof([Slots() for i in range(1000)])出[1]:9024>>>sys.getsizeof([NoSlots() for i in range(1000)])出[1]:9024

Python 2.7 结果:

<预><代码>>>>sys.getsizeof([Slots() for i in range(1000)])出[1]:4516>>>sys.getsizeof([NoSlots() for i in range(1000)])出[1]:4516

我原以为 Python 2.7 的大小会有所不同，所以我认为测试有问题.

解决方案

不，PEP 412不会使 __slots__ 变得多余.

<小时>

首先，Armin Rigo 是对的，您没有正确测量它.您需要测量的是对象的大小，加上值，加上 __dict__ 本身(仅适用于 NoSlots)和键(适用于 NoSlots代码>仅).

或者你可以按照他的建议去做:

cls = Slots if len(sys.argv) >1 其他 NoSlots定义 f():tracemalloc.start()objs = [cls() for _ in range(100000)]打印(tracemalloc.get_traced_memory())F()

当我在 OS X 上的 64 位 CPython 3.4 上运行它时，Slots 得到 8824968，NoSlots 得到 25624872.因此，看起来 NoSlots 实例占用 88 个字节，而 Slots 实例占用 256 个字节.

<小时>

这怎么可能?

因为 __slots__ 和 key-split __dict__ 之间还有两个区别.

首先，字典使用的哈希表保持低于 2/3 满，并且它们呈指数增长并且具有最小大小，因此您将有一些额外的空间.通过查看注释很好的 source:您将有 8 个哈希桶，而不是 5 个插槽指针.

其次，字典本身不是免费的；它有一个标准的对象头、一个计数和两个指针.这听起来可能不是很多，但是当您谈论一个只有几个属性的对象时(请注意，大多数 对象只有几个属性……)，dict 标头可以生成尽可能多的属性与哈希表的区别.

当然，在你的例子中，值，所以这里涉及的唯一成本是对象本身，加上 5 个槽或 8 个哈希桶和 dict 标头，所以差异非常显着.在现实生活中，__slots__ 很少有那么大的好处.

<小时>

最后，请注意 PEP 412 仅声明:

<块引用>

基准测试表明面向对象程序的内存使用减少了 10% 到 20%

想想你在哪里使用__slots__.要么节省如此之大以至于不使用 __slots__ 会很荒谬，要么你真的需要挤出最后 15%.或者，您正在构建一个 ABC 或其他类，您希望这些类由谁知道是什么子类，而子类可能需要节省.无论如何，在这些情况下，如果没有 __slots__，您获得一半的收益，甚至三分之二的收益，仍然很少会足够；您仍然需要使用 __slots__.

真正的胜利是在不值得使用__slots__的情况下；您将免费获得一笔小额福利.

(另外，肯定有一些程序员过度使用__slots__，也许这种变化可以说服他们中的一些人将精力投入到微优化其他不太相关的东西上，如果你很幸运.)

PEP 412, implemented in Python 3.3, introduces improved handling of attribute dictionaries, effectively reducing the memory footprint of class instances. __slots__ was designed for the same purpose, so is there any point in using __slots__ any more?

In an attempt to find out the answer myself, I run the following test, but the results don't make much sense:

class Slots(object):
    __slots__ = ['a', 'b', 'c', 'd', 'e']
    def __init__(self):
        self.a = 1
        self.b = 1
        self.c = 1
        self.d = 1
        self.e = 1  

class NoSlots(object):
    def __init__(self):
        self.a = 1
        self.b = 1
        self.c = 1
        self.d = 1
        self.e = 1

Python 3.3 Results:

>>> sys.getsizeof([Slots() for i in range(1000)])
Out[1]: 9024
>>> sys.getsizeof([NoSlots() for i in range(1000)])
Out[1]: 9024

Python 2.7 Results:

>>> sys.getsizeof([Slots() for i in range(1000)])
Out[1]: 4516
>>> sys.getsizeof([NoSlots() for i in range(1000)])
Out[1]: 4516

I would have expected the size to differ at least for Python 2.7, so I assume there is something wrong with the test.

解决方案

No, PEP 412 does not make __slots__ redundant.

---

First, Armin Rigo is right that you're not measuring it properly. What you need to measure is the size of the object, plus the values, plus the __dict__ itself (for NoSlots only) and the keys (for NoSlots only).

Or you could do what he suggests:

cls = Slots if len(sys.argv) > 1 else NoSlots
def f():
    tracemalloc.start()
    objs = [cls() for _ in range(100000)]
    print(tracemalloc.get_traced_memory())
f()

When I run this on 64-bit CPython 3.4 on OS X, I get 8824968 for Slots and 25624872 for NoSlots. So, it looks like a NoSlots instance takes 88 bytes, while a Slots instance takes 256 bytes.

---

How is this possible?

Because there are still two differences between __slots__ and a key-split __dict__.

First, the hash tables used by dictionaries are kept below 2/3rds full, and they grow exponentially and have a minimum size, so you're going to have some extra space. And it's not hard to work out how much space by looking at the nicely-commented source: you're going to have 8 hash buckets instead of 5 slots pointers.

Second, the dictionary itself isn't free; it has a standard object header, a count, and two pointers. That might not sound like a lot, but when you're talking about an object that's only got a few attributes (note that most objects only have a few attributes…), the dict header can make as much difference as the hash table.

And of course in your example, the values, so the only cost involved here is the object itself, plus the the 5 slots or 8 hash buckets and dict header, so the difference is pretty dramatic. In real life, __slots__ will rarely be that much of a benefit.

---

Finally, notice that PEP 412 only claims:

Benchmarking shows that memory use is reduced by 10% to 20% for object-oriented programs

Think about where you use __slots__. Either the savings are so huge that not using __slots__ would be ridiculous, or you really need to squeeze out that last 15%. Or you're building an ABC or other class that you expect to be subclassed by who-knows-what and the subclasses might need the savings. At any rate, in those cases, the fact that you get half the benefit without __slots__, or even two thirds the benefit, is still rarely going to be enough; you'll still need to use __slots__.

The real win is in the cases where it isn't worth using __slots__; you'll get a small benefit for free.

(Also, there are definitely some programmers who overuse the hell out of __slots__, and maybe this change can convince some of them to put their energy into micro optimizing something else not quite as irrelevant, if you're lucky.)

这篇关于PEP 412 是否使 __slots__ 变得多余?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！