PyPy显示不正确的基准测试结果? [英] PyPy displaying inaccurate benchmark results?

问题描述

我正在研究 Project Euler ，想知道是否可以使用PyPy加快解决方案的速度.但是，我发现结果令人失望，因为它花费了更多的时间进行计算.

I was working on Project Euler and wondered if I could speed up my solution using PyPy. However, I found results quite disappointing, as it took more time to compute.

d:\projeuler>pypy problem204.py
3462.08630405 mseconds

d:\projeuler>python problem204.py
1823.91602542 mseconds

因为毫秒的输出是使用python的time模块计算的，所以我再次使用内置的基准测试命令运行了它.

Since mseconds output were calculated using python's time modules, so I ran it again using builtin benchmark commands.

d:\projeuler>pypy -mtimeit -s "import problem204" "problem204._main()"
10 loops, best of 3: 465 msec per loop

d:\projeuler>python -mtimeit -s "import problem204" "problem204._main()"
10 loops, best of 3: 1.87 sec per loop

PyPy报告说，完成运行大约需要半秒钟.但是，我尝试运行pypy problem204几次，输出结果甚至都没有接近基准测试的.5秒.与pypy不同，python的mtimeit结果与输出一致. pypy是给我不准确的基准，还是我不理解一些魔术?

PyPy reports that it took about half second to finish running. However, I tried running pypy problem204 several times and outputs were never even close to benchmarked .5 seconds. unlike pypy, python's mtimeit results are consistent with outputs. Is pypy giving me inaccurate benchmarks, or is there some magic I don't understand?

推荐答案

请注意timeit

1. 多次运行该语句(在您的情况下为10)，并且
2. 进行几次(默认为3次)，并给出最小值，原因是文档中概述的.

这取决于您的代码，但是完全有可能将JIT编译器归咎于这种令人困惑的结果.每次启动新的pypy进程时都会发生JIT预热开销，但在timeit基准测试期间仅发生一次(因为在同一进程中多次运行_main).此外，如果您的代码的某些部分经常运行，以至于_main仅运行一次，而不是仅编译一次(例如，运行三次)，则不会编译该文件，则后续运行也将更快，从而进一步消除了第一次运行的最佳结果.一个(即一次运行pypy problem204.py的那个).

It depends on your code, but it's entirely possible that the JIT compiler is to blame for this confusing result. The JIT warmup overhead is incurred every time you launched a new pypy process, but only once during the timeit benchmark (because that one runs _main several times in the same process). Moreover, if some part of your code is run so often that it's not compiled when _main runs once, but only when it runs, say, three times, subsequent runs will also be faster, which further removes the best result from the first one (i.e. the one for running pypy problem204.py once).

timeit结果是正确的，因为它(大致)与在最佳情况下代码的速度相匹配-预热的JIT编译器，很少将CPU丢失给其他程序，等等. 您的问题是您想知道一些不同的东西-包括JIT预热在内的时间.

The timeit result is correct in that it (roughly) matches how fast the code will be in the best case - warmed-up JIT compiler, rarely losing the CPU to other programs, etc. Your problem is that you want to know something different - the time including JIT warmup.

这篇关于PyPy显示不正确的基准测试结果?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！