迭代到大范围时出现Python内存错误 [英] Python Memory Error while iterating to a big range

问题描述

total = 0x = 2**32对于范围(x)中的我:总计 = 总计 + i；打印(总计)

我在循环到 2**32 范围时遇到 MemoryError.有没有办法在不耗尽内存的情况下进行迭代?

解决方案

当您尝试创建一个包含前 2³² 个非负整数的列表时会发生这种情况(我正在使用Windows 10 系统上的 Python 2.7.11):

<预><代码>>>>对于我在范围内(2**32):通过...回溯(最近一次调用最后一次):文件<stdin>"，第 1 行，位于 <module>溢出错误:范围()结果有太多项目

人们可能会期望，如果问题是在内存中同时拥有如此大量的项目，那么解决方案可能是通过生成器一次处理一个项目……但事实并非如此:

<预><代码>>>>for i in xrange(2**32): pass...回溯(最近一次调用最后一次):文件<stdin>"，第 1 行，位于 <module>溢出错误:Python int 太大而无法转换为 C long

xrange() 内置函数的文档说明为什么会出现这个错误:

<块引用>

CPython 实现细节:xrange() 旨在简单快速.实现可能会施加限制来实现这一点.Python 的 C 实现将所有参数限制为原生 C long(短"Python 整数)，并且还要求元素数量适合原生 C long.

问题是 2³² 不能作为输入参数传递给 xrange，因为该数字大于 Python 中的最大短"整数.试试这个来说服自己:

<预><代码>>>>导入系统>>>sys.maxint # 2**31 - 12147483647>>>sys.maxint + 1 # 2**31 自动转换为long"int2147483648L>>>2**312147483648L

如果您需要重复执行计算超过 2³¹ 次(以下示例中为 2³⁴ 次)，您可以使用嵌套 for 循环:

<预><代码>>>>循环数 = 0>>>对于 xrange(2**4) 中的 i:...对于 xrange(2**30) 中的 j:... # 做东西... 循环 += 1...>>>循环17179869184L>>>2**3417179869184L

上面的代码是一种相当幼稚的解决方法.while 循环似乎是一个更合适的解决方案:

<预><代码>>>>循环数 = 0>>>while 循环 <2**34:... # 做东西... 循环 += 1...>>>循环17179869184L

total = 0
x = 2**32
for i in range(x):
    total = total + i;
print(total) 

I am getting a MemoryError while looping to a range of 2**32. Is there any way to do the iteration without running out of memory?

解决方案

This is what happens when you try to create a list that contains the first 2³² non-negative integers (I'm using Python 2.7.11 on a Windows 10 system):

>>> for i in range(2**32): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: range() result has too many items

One might expect that if the problem is to have such a large number of items simultaneously in memory, the solution could be that of handling one item at a time through a generator... but it is not:

>>> for i in xrange(2**32): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long

The documentation of xrange() built-in function explains why this error occurs:

CPython implementation detail: xrange() is intended to be simple and fast. Implementations may impose restrictions to achieve this. The C implementation of Python restricts all arguments to native C longs ("short" Python integers), and also requires that the number of elements fit in a native C long.

The issue is that 2³² cannot be passed as an input argument to xrange because that number is greater than the maximum "short" integer in Python. Try this to convince yourself:

>>> import sys
>>> sys.maxint       # 2**31 - 1
2147483647
>>> sys.maxint + 1   # 2**31 is automatically converted to "long" int
2147483648L
>>> 2**31
2147483648L

You could use nested for loops if you need to repeatedly perform a computation more than 2³¹ times (2³⁴ times in the following example):

>>> loops = 0
>>> for i in xrange(2**4):
...    for j in xrange(2**30):
...        # do stuff
...        loops += 1
...
>>> loops
17179869184L
>>> 2**34
17179869184L

The code above is a rather naïve workaround. A while loop seems to be a much more adequate solution:

>>> loops = 0
>>> while loops < 2**34:
...    # do stuff
...    loops += 1
...
>>> loops
17179869184L

这篇关于迭代到大范围时出现Python内存错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！