针对PyPy进行优化 [英] Optimizing for PyPy

问题描述

(这是 PyPy的统计探查器的后续步骤)

我正在PyPy下运行一些Python代码，并希望对其进行优化.

I'm running some Python code under PyPy and would like to optimize it.

在Python中，我将使用statprof或lineprofiler来了解导致减速的确切行，并尝试解决它们.不过，在PyPy中，这两种工具均未真正报告合理的结果，因为PyPy可能会优化某些行.我也不想使用cProfile，因为我很难发现所报告功能的哪一部分是瓶颈.

In Python, I would use statprof or lineprofiler to know which exact lines are causing the slowdown and try to work around them. In PyPy though, both of the tools don't really report sensible results as PyPy might optimize away some lines. I would also prefer not to use cProfile as I find it very difficult to distil which part of the reported function is the bottleneck.

有人对如何进行有一些提示吗?也许另一个分析器在PyPy下能很好地工作?一般来说，如何为PyPy优化Python代码?

Does anyone have some tips on how to proceed? Perhaps another profiler which works nicely under PyPy? In general, how does one go about optimizing Python code for PyPy?

推荐答案

如果您了解PyPy架构的工作方式，您将意识到尝试查明各个代码行并没有真正的成效.您从以RPython编写的Python解释器开始，然后通过跟踪JIT运行该JIT，该JIT生成流程图，然后转换这些图以优化RPython解释器.这意味着被JIT编辑的RPython解释器所运行的Python代码的布局可能与优化的汇编程序实际运行的结构完全不同.此外，请记住，JIT始终在循环或函数上运行，因此获取逐行统计信息并不那么有意义.因此，我认为cProfile可能确实是一个不错的选择，因为它将使您了解优化的重点.一旦知道了哪些功能是瓶颈，就可以将精力花在那些较慢的功能上，而不用试图修复一行Python代码.

If you understand the way the PyPy architecture works, you'll realize that trying to pinpoint individual lines of code isn't really productive. You start with a Python interpreter written in RPython, which then gets run through a tracing JIT which generates flow graphs and then transforms those graphs to optimize the RPython interpreter. What this means is the layout of your Python code being run by the RPython interpreter being JIT'ed may have very different structure than the optimized assembler actually be run. Furthermore, keep in mind that the JIT always works on a loop or a function, so getting line-by-line stats is not as meaningful. Consequently, I think cProfile may really be a good option for you since it will give you an idea of where to concentrate your optimization. Once you know which functions are your bottlenecks, you can spend your optimization efforts targeting those slower functions, rather than trying to fix a single line of Python code.

在执行此操作时请记住，PyPy与cPython具有非常不同的性能特征.始终尝试以尽可能简单的方式编写代码(顺便说一句，这并不意味着要减少行数).还有其他一些启发式方法，例如使用专用列表，当您使用少量的大多数为常量的键时在dict上使用对象，避免使用C Python API进行C扩展等.

Keep in mind as you do this that PyPy has very different performance characteristics than cPython. Always try to write code in as simple a way as possible (that doesn't mean as few lines as possible btw). There are a few other heuristics that help such as using specialized lists, using objects over dicts when you have a small number of mostly constant keys, avoiding C extensions using the C Python API, etc.

如果真的，真的要坚持在生产线水平上进行优化.有一些选择.一个叫做JitViewer( https://bitbucket.org/pypy/jitviewer )，它可以让您拥有JIT对您的代码所做的工作的一个非常底层的视图.例如，您甚至可以看到与Python循环相对应的汇编器指令.使用该工具，您可以真正了解PyPy在代码的某些部分运行的速度，因为您现在可以做一些愚蠢的事情，例如计算循环或其他内容使用的汇编指令的数量.

If you really, really insist on trying to optimize at the line level. There are a few options. One is called JitViewer (https://bitbucket.org/pypy/jitviewer), which will let you have a very low level view of what the JIT is doing to your code. For instance, you can even see the assembler instructions which correspond to a Python loop. Using that tool, you can really get a sense for just how fast PyPy will behave with certain parts of the code, as you can now do silly things like count the number of assembler instructions used for your loop or something.

这篇关于针对PyPy进行优化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！