multiprocessing.Manager().dict().setdefault()是否损坏? [英] Is multiprocessing.Manager().dict().setdefault() broken?

问题描述

其最迟而且可能很愚蠢的部门提出:

The its-late-and-im-probably-stupid department presents:

>>> import multiprocessing
>>> mgr = multiprocessing.Manager()
>>> d = mgr.dict()
>>> d.setdefault('foo', []).append({'bar': 'baz'})
>>> print d.items()
[('foo', [])]         <-- Where did the dict go?

位置:

>>> e = mgr.dict()
>>> e['foo'] = [{'bar': 'baz'}]
>>> print e.items()
[('foo', [{'bar': 'baz'}])]

版本:

>>> sys.version
'2.7.2+ (default, Jan 20 2012, 23:05:38) \n[GCC 4.6.2]'

臭虫还是臭虫?

更多相同的内容，在python 3.2上:

More of the same, on python 3.2:

>>> sys.version
'3.2.2rc1 (default, Aug 14 2011, 21:09:07) \n[GCC 4.6.1]'

>>> e['foo'] = [{'bar': 'baz'}]
>>> print(e.items())
[('foo', [{'bar': 'baz'}])]

>>> id(type(e['foo']))
137341152
>>> id(type([]))
137341152

>>> e['foo'].append({'asdf': 'fdsa'})
>>> print(e.items())
[('foo', [{'bar': 'baz'}])]

字典代理中的列表如何不包含其他元素?

How can the list in the dict proxy not contain the additional element?

推荐答案

这是一些非常有趣的行为，我不确定它是如何工作的，但是我会弄清楚为什么是这样的行为.

This is some pretty interesting behavior, I am not exactly sure how it works but I'll take a crack at why the behavior is the way it is.

首先，请注意multiprocessing.Manager().dict()不是dict，它是DictProxy对象:

First, note that multiprocessing.Manager().dict() is not a dict, it is a DictProxy object:

>>> d = multiprocessing.Manager().dict()
>>> d
<DictProxy object, typeid 'dict' at 0x7fa2bbe8ea50>

DictProxy类的目的是为您提供一个可以在进程之间安全共享的dict，这意味着它必须在常规dict函数之上实现一些锁定.

The purpose of the DictProxy class is to give you a dict that is safe to share across processes, which means that it must implement some locking on top of the normal dict functions.

显然，此处实现的一部分是不允许您直接访问嵌套在DictProxy内部的可变对象，因为如果允许，您将能够以绕过所有锁定的方式修改共享对象使得DictProxy可以安全使用.

Apparently part of the implementation here is to not allow you to directly access mutable objects nested inside of a DictProxy, because if that was allowed you would be able to modify your shared object in a way that bypasses all of the locking that makes DictProxy safe to use.

有些证据表明您无法访问可变对象，这与setdefault()所发生的情况类似:

Here is some evidence that you can't access mutable objects, which is similar to what is going on with setdefault():

>>> d['foo'] = []
>>> foo = d['foo']
>>> id(d['foo'])
140336914055536
>>> id(foo)
140336914056184

使用普通词典，您会期望d['foo']和foo指向相同的列表对象，而对其中一个的修改将对另一个列表进行修改.如您所见，由于多处理模块对过程的安全性提出了更高的要求，因此DictProxy类并非如此.

With a normal dictionary you would expect d['foo'] and foo to point to the same list object, and modifications to one would modify the other. As you have seen, this is not the case for the DictProxy class because of the additional process safety requirement imposed by the multiprocessing module.

编辑:来自

edit: The following note from the multiprocessing documentation clarifies what I was trying to say above:

注意:对dict和list代理中的可变值或项的修改不会通过管理器传播，因为代理无法知道何时修改其值或项.要修改此类项目，您可以将修改后的对象重新分配给容器代理:

Note: Modifications to mutable values or items in dict and list proxies will not be propagated through the manager, because the proxy has no way of knowing when its values or items are modified. To modify such an item, you can re-assign the modified object to the container proxy:

# create a list proxy and append a mutable object (a dictionary)
lproxy = manager.list()
lproxy.append({})
# now mutate the dictionary
d = lproxy[0]
d['a'] = 1
d['b'] = 2
# at this point, the changes to d are not yet synced, but by
# reassigning the dictionary, the proxy is notified of the change
lproxy[0] = d

---

根据上述信息，以下是您如何重写原始代码以与DictProxy一起使用的方法:

---

Based on the above information, here is how you could rewrite your original code to work with a DictProxy:

# d.setdefault('foo', []).append({'bar': 'baz'})
d['foo'] = d.get('foo', []) + [{'bar': 'baz'}]

正如爱德华·洛珀(Edward Loper)在评论中建议的那样，对上述代码进行了编辑以使用 get() 而不是 setdefault().

这篇关于multiprocessing.Manager().dict().setdefault()是否损坏?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！