为什么Python会“抢先"试图计算一个很大的数字时会挂起? [英] Why does Python "preemptively" hang when trying to calculate a very large number?

问题描述

我问过

I've asked this question before about killing a process that uses too much memory, and I've got most of a solution worked out.

但是，这里存在一个问题:我尝试使用的方法似乎无法计算大量数字.下面的这段代码旨在为该进程设置10秒的CPU时间限制.

However, there is one problem: calculating massive numbers seems to be untouched by the method I'm trying to use. This code below is intended to put a 10 second CPU time limit on the process.

import resource
import os
import signal

def timeRanOut(n, stack):
    raise SystemExit('ran out of time!')
signal.signal(signal.SIGXCPU, timeRanOut)

soft,hard = resource.getrlimit(resource.RLIMIT_CPU)
print(soft,hard)
resource.setrlimit(resource.RLIMIT_CPU, (10, 100))

y = 10**(10**10)

期望在运行此脚本(在Unix计算机上)时看到的内容是:

What I expect to see when I run this script (on a Unix machine) is this:

-1 -1
ran out of time!

相反，我没有输出.我获得输出的唯一方法是使用 Ctrl + C ，如果我在 Ctrl + C 之后输入10秒:

Instead, I get no output. The only way I get output is with Ctrl + C, and I get this if I Ctrl + C after 10 seconds:

^C-1 -1
ran out of time!
CPU time limit exceeded

如果我在10秒钟之前 Ctrl + C ，那么我必须做两次，控制台输出看起来像这样:

If I Ctrl + C before 10 seconds, then I have to do it twice, and the console output looks like this:

^C-1 -1
^CTraceback (most recent call last):
  File "procLimitTest.py", line 18, in <module>
    y = 10**(10**10)
KeyboardInterrupt

在尝试解决这个问题的过程中，我还将time.sleep(2)放在打印和大数计算之间.它似乎没有任何作用.如果将y = 10**(10**10)更改为y = 10**10，则print和sleep语句将按预期工作.在打印语句中添加flush=True或在打印语句后添加sys.stdout.flush()也不起作用.

In the course of experimenting and trying to figure this out, I've also put time.sleep(2) between the print and large number calculation. It doesn't seem to have any effect. If I change y = 10**(10**10) to y = 10**10, then the print and sleep statements work as expected. Adding flush=True to the print statement or sys.stdout.flush() after the print statement don't work either.

为什么不能限制CPU时间来计算大量数字?如何解决或至少减轻这种情况?

Why can I not limit CPU time for the calculation of a very large number? How can I fix or at least mitigate this?

其他信息:

Python版本:3.3.5 (default, Jul 22 2014, 18:16:02) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]

Python version: 3.3.5 (default, Jul 22 2014, 18:16:02) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]

Linux信息:Linux web455.webfaction.com 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

推荐答案

TLDR: Python会预先计算代码中的常量.如果通过至少一个中间步骤计算出非常大的数字，则进程 将受到CPU时间限制.

TLDR: Python precomputes constants in the code. If any very large number is calculated with at least one intermediate step, the process will be CPU time limited.

花了很多时间进行搜索，但是我发现了证据，证明Python 3 确实可以对在计算任何东西之前在代码中找到的常量文字进行预计算.其中一个是此网页: 窥视孔适用于Python的优化器 .我在下面引用了其中的一些内容.

It took quite a bit of searching, but I have discovered evidence that Python 3 does precompute constant literals that it finds in the code before evaluating anything. One of them is this webpage: A Peephole Optimizer for Python. I've quoted some of it below.

ConstantExpressionEvaluator

此类会预先计算一些常量表达式，并将它们存储在函数的常量列表中，其中包括显而易见的二进制和一元运算以及仅由常量组成的元组.特别值得注意的是，编译器不会将复杂的文字表示为常量，而是将其表示为表达式，因此2 + 3j显示为

ConstantExpressionEvaluator

This class precomputes a number of constant expressions and stores them in the function's constants list, including obvious binary and unary operations and tuples consisting of just constants. Of particular note is the fact that complex literals are not represented by the compiler as constants but as expressions, so 2+3j appears as

LOAD_CONST n (2) LOAD_CONST m (3j) BINARY_ADD

此类将这些内容转换为

LOAD_CONST q (2+3j)

对于使用复杂常量的代码，可以大大提高性能.

which can result in a fairly large performance boost for code that uses complex constants.

使用2+3j作为示例的事实非常有力地表明，不仅小常数已被预先计算和缓存，而且代码中的任何常量常量也被预先计算和缓存.我还发现了此评论在另一个Stack  Overflow问题( > Python中是否缓存了常量计算? ):

The fact that 2+3j is used as an example very strongly suggests that not only small constants are being precomputed and cached, but also any constant literals in the code. I also found this comment on another Stack Overflow question (Are constant computations cached in Python?):

请注意，对于Python 3，窥孔优化器确实会预先计算1/3常量. (当然，特定于CPython.)– Mark Dickinson 10月7日在19:40

Note that for Python 3, the peephole optimizer does precompute the 1/3 constant. (CPython specific, of course.) – Mark Dickinson Oct 7 at 19:40

替换为这些事实提供了支持

These are supported by the fact that replacing

y = 10**(10**10)

与此 一起挂起，即使我从未调用过该函数！

with this also hangs, even though I never call the function!

def f():
    y = 10**(10**10)

---

好消息

幸运的是，我的代码中没有这么大的文字常量.此类常量的任何计算都将在以后发生，这可能会且受CPU时间限制的限制.我改变了

---

The good news

Luckily for me, I don't have any such giant literal constants in my code. Any computation of such constants will happen later, which can be and is limited by the CPU time limit. I changed

y = 10**(10**10)

对此

x = 10
print(x)
y = 10**x
print(y)
z = 10**y
print(z)

并根据需要获得此输出！

and got this output, as desired!

-1 -1
10
10000000000
ran out of time!

故事的寓意:如果在CPU中没有较大的文字常量，则通过CPU时间或内存消耗(或某些其他方法)限制进程将有效. Python尝试预先计算的代码.

The moral of the story: Limiting a process by CPU time or memory consumption (or some other method) will work if there is not a large literal constant in the code that Python tries to precompute.

这篇关于为什么Python会“抢先"试图计算一个很大的数字时会挂起?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！