逐步跟踪Python表达式求值 [英] Tracing Python expression evaluation step by step

问题描述

我正在尝试编写Python表达式评估可视化工具，它将展示如何逐步评估Python表达式（出于教育目的）。 Philip Guo的Python Tutor很棒，但是它逐行评估Python程序，我发现学生有时不理解像这样的单行表达式如何排序（[4，2，3，1] + [ 5，6]）[1] == 2 被评估，我想将这个过程可视化。 （似乎还没有人做过-至少我什么都没发现。）理想的解决方案将创建这样的字符串序列：

I'm trying to write Python expression evaluation visualizer, that will show how Python expressions are evaluated step by step (for education purposes). Philip Guo's Python Tutor is great, but it evaluates Python program line by line, and I found that students sometimes do not understand how one-line expressions like sorted([4, 2, 3, 1] + [5, 6])[1] == 2 are evaluated and I'd like to visualize this process. (It seems that nobody did it yet — at least I found nothing.) The ideal solution will create a sequence of strings like this:

sorted([4, 2, 3, 1] + [5, 6])[1] == 2
sorted( >> [4, 2, 3, 1] + [5, 6] << )[1] == 2
>> sorted([4, 2, 3, 1, 5, 6]) << [1] == 2
>> [1 2 3 4 5 6][1] << == 2
>> 2 == 2 <<
True

此处>> 和<< 用于突出显示在当前步骤中求值的表达式的一部分，然后替换为其值。 （也许以后我会尝试将此序列转换为某种动画。）

Here >> and << are used to highlight a part of the expression that is evaluated on the current step and then replaced by its value. (Maybe, I will try to convert this sequence to some kind of animation later.)

我目前的策略是使用 ast.parse（） 将该字符串解析为AST，然后找到将首先进行评估的节点，并使用 eval（compile（node，``，'eval'）））（我绝对不希望重新实现整个Python :)），将评估结果转换为AST Node（使用 repr 然后 ast.parse（）？）并将当前节点替换为结果节点，然后使用 codegen.to_source 从中生成修改后的代码字符串（已修改）AST，然后继续相同的过程，直到树中只有一个文字为止。

My current strategy is to use ast.parse() to parse the string into AST, then find a node that will be evaluated first, evaluate it with eval(compile(node, '', 'eval')) (I'm definitely do not want to reimplement the whole Python :)), convert the result of evaluation into AST Node (with repr and then ast.parse()?) and substitute current node with the result node, then use codegen.to_source to produce modified code string from (modified) AST and continue the same process until I have only one literal in the tree.

我的问题是：如何找到一个首先被评估的节点？看来我可以使用子类 ast.NodeVisitor 遍历树的深度优先，但是我不确定如何检测到到达所需的节点以及如何检测

My question is: how can I find a node that will be evaluated first? It seems that I can traverse the tree depth-first with subclassing ast.NodeVisitor, but I'm not sure how can I detect that I reached the desired node and how can I stop traversing after it?

编辑。

我最初对树进行转换的方法可能不可行。实际上，没有必要对Python表达式进行评估的基本步骤是将某些子表达式替换为更简单的子表达式（例如在算术中）。例如，列表推导提供了一个更为复杂的行为，无法用用该事物替换该事物，然后递归重复来表达。因此，我重申了一个问题。我需要一些方法来以编程方式显示如何逐步评估Python表达式。例如，@ jasonharper提到的MacroPy的跟踪功能目前是可以接受的解决方案。不幸的是，MacroPy似乎已被抛弃，无法在Python 3中使用。是否有任何想法可以在不移植完整MacroPy的情况下类似于Python 3中的这种跟踪行为？

It is possible that my initial approach with transformation of the tree is not feasible. In fact, an elementary step of evaluation of Python expression is not neccessary have to be a replacement of some sub-expression to a simpler one (like in arithmetic). For example, list comprehensions provide a much more complicated behaviour that cannot be expressed in terms replace this thing by that thing, then repeat recursively. So I restate a the question a little bit. I need some way to programmaticaly show how Python expressions are evaluated step by step. For example, MacroPy's tracing feature, mentioned by @jasonharper, is acceptable solution at this stage. Unfortunately, MacroPy seem to be abandoned and doesn't work with Python 3. Are there any ideas how to resemble this tracing behaviour in Python 3 without porting full MacroPy?

编辑2。

就在我授予这个赏金之后，我发现了类似问题和调试器具有非常接近的功能。但是，由于该问题没有最终答案，而且我不需要完整的调试器，因此我仍在寻找可以在例如Jupyter环境中使用的答案。

Just after I awarded this bounty, I found similar question and a debugger with very close features. However, as there's no ultimate answer for that question, and I do not need the full debugger, I'm still looking for an answer that can be used for example in Jupyter environment.

推荐答案

在 Thonny IDE 中实现了表达步进。

它使用AST工具，其中每个（子）表达式 e 都转换为 after（before（<位置信息>），e）。函数之前和之后是伪函数，用于在Python的跟踪系统中引起额外的 call 事件。这些额外的调用通知（子）表达式评估何时开始或刚刚结束。 （添加了类似的伪函数以检测每个语句的开始和结束。）

It uses AST instrumentation, where each (sub)expression e is transformed into after(before(<location info>), e). Functions before and after are dummy functions for causing extra call-events in Python's tracing system. These extra calls notify when (sub)expression evaluation is about to start or has just ended. (Similar dummy functions are added to detect start and end of each statement.)

AST规范和这些新事件的解释在 thonny.backend.FancyTracer 。

AST instrumentation and interpretation of these new events is done in thonny.backend.FancyTracer.

Python的AST节点包含相应文本范围的开始位置，但有时不正确。终端位置完全丢失。 thonny.ast_utils.mark_text_ranges 试图解决这个问题（但目前解决方案还不完整）。

Python's AST nodes contain start position of corresponding text ranges, but they are sometimes incorrect. End positions are completely missing. thonny.ast_utils.mark_text_ranges tries to take care of this (but the solution is incomplete at the moment).

如果有人将Thonny的相关功能提取到更通用的软件包中。甚至两个软件包-一个用于计算Python AST的位置信息，另一个用于详细跟踪Python代码。如果有人带头，我愿意提供帮助。

这篇关于逐步跟踪Python表达式求值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！