python的货架模块有最大尺寸吗? [英] Is there a maximum size for python's shelve module?

问题描述

当我尝试打开超过一定大小的搁置的持久文件时，出现了这个异常，实际上很小(<1MB)，但是我不确定确切的数字在哪里.现在，我知道pickle有点像python的混蛋，虽然搁置并没有被认为是一个特别健壮的解决方案，但是它恰好很好地解决了我的问题(理论上)，而且我一直无法找到原因这种例外.

I'm getting this exception when trying to open shelve persisted files over a certain size which is actually pretty small (< 1MB) but I'm not sure where the exactly number is. Now, I know pickle is sort of the bastard child of python and shelve isn't thought of as a particularly robust solution, but it happens to solve my problem wonderfully (in theory) and I haven't been able to find a reason for this exception.

Traceback (most recent call last):
  File "test_shelve.py", line 27, in <module>
    print len(f.keys())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shelve.py", line 101, in keys
    return self.dict.keys()
SystemError: Negative size passed to PyString_FromStringAndSize

我可以始终如一地重现它，但在Google上找不到很多.这是一个将重现的脚本.

I can reproduce it consistently, but I haven't found much on google. Here's a script that will reproduce.

import shelve
import random
import string
import pprint

f = shelve.open('test')
# f = {}

def rand_list(list_size=20, str_size=40):
    return [''.join([random.choice(string.ascii_uppercase + string.digits) for j in range(str_size)]) for i in range(list_size)]

def recursive_dict(depth=3):
    if depth==0:
        return rand_list()
    else:
        d = {}
        for k in rand_list():
            d[k] = recursive_dict(depth-1)
        return d

for k,v in recursive_dict(2).iteritems():
    f[k] = v

f.close()

f = shelve.open('test')
print len(f.keys())

推荐答案

关于错误本身:

网上流传的想法是数据大小超过最大 该机器上可能的整数(最大32位(有符号)整数 是2147483647)，被Python解释为负数.

The idea circulating on the web is the data size exceeded the largest integer possible on that machine (the largest 32 bit (signed) integer is 2 147 483 647), interpreted as a negative size by Python.

您的代码与2.7.3一起运行，因此可能是固定的错误.

Your code is running with 2.7.3, so may be a fixed bug.

这篇关于python的货架模块有最大尺寸吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！