一步一步跟踪 Python 表达式求值 [英] Tracing Python expression evaluation step by step

问题描述

我正在尝试编写 Python 表达式评估可视化工具，它将展示如何逐步评估 Python 表达式(用于教育目的).Philip Guo 的 Python Tutor 很棒，但是它是逐行评估 Python 程序的，我发现学生有时不明白像 sorted([4, 2, 3, 1] + [5, 6])[1] == 2 被评估，我想形象化这个过程.(似乎还没有人这样做——至少我什么也没发现.)理想的解决方案是创建一个这样的字符串序列:

I'm trying to write Python expression evaluation visualizer, that will show how Python expressions are evaluated step by step (for education purposes). Philip Guo's Python Tutor is great, but it evaluates Python program line by line, and I found that students sometimes do not understand how one-line expressions like sorted([4, 2, 3, 1] + [5, 6])[1] == 2 are evaluated and I'd like to visualize this process. (It seems that nobody did it yet — at least I found nothing.) The ideal solution will create a sequence of strings like this:

sorted([4, 2, 3, 1] + [5, 6])[1] == 2
sorted( >> [4, 2, 3, 1] + [5, 6] << )[1] == 2
>> sorted([4, 2, 3, 1, 5, 6]) << [1] == 2
>> [1 2 3 4 5 6][1] << == 2
>> 2 == 2 <<
True

这里的 >> 和 << 用于突出显示在当前步骤上计算的表达式的一部分，然后由其值替换.(也许，我稍后会尝试将此序列转换为某种动画.)

Here >> and << are used to highlight a part of the expression that is evaluated on the current step and then replaced by its value. (Maybe, I will try to convert this sequence to some kind of animation later.)

我目前的策略是使用ast.parse()将字符串解析成AST，然后找一个先求值的节点，用eval(compile(node,'', 'eval')) (我绝对不想重新实现整个 Python :))，将计算结果转换为 AST 节点(使用 repr 然后 ast.parse()?) 并将当前节点替换为结果节点，然后使用 codegen.to_source 从(修改后的)AST 生成修改后的代码字符串并继续相同的过程，直到我在树中只有一个文字.

My current strategy is to use ast.parse() to parse the string into AST, then find a node that will be evaluated first, evaluate it with eval(compile(node, '', 'eval')) (I'm definitely do not want to reimplement the whole Python :)), convert the result of evaluation into AST Node (with repr and then ast.parse()?) and substitute current node with the result node, then use codegen.to_source to produce modified code string from (modified) AST and continue the same process until I have only one literal in the tree.

我的问题是:如何找到将首先评估的节点?似乎我可以使用子类化 ast.NodeVisitor 遍历树深度优先，但我不确定如何检测到我到达了所需的节点，以及如何在它之后停止遍历?

My question is: how can I find a node that will be evaluated first? It seems that I can traverse the tree depth-first with subclassing ast.NodeVisitor, but I'm not sure how can I detect that I reached the desired node and how can I stop traversing after it?

编辑.

我最初的树变换方法可能不可行.事实上，Python 表达式求值的基本步骤不是必须将某些子表达式替换为更简单的子表达式(如算术).例如，列表推导式提供了一种更复杂的行为，不能用用那个东西替换这个东西，然后递归地重复来表达.所以我稍微重申一下这个问题.我需要一些方法来以编程方式显示 Python 表达式是如何逐步计算的.例如，@jasonharper 提到的 MacroPy 的 tracing 功能是现阶段可接受的解决方案.不幸的是，MacroPy 似乎已被放弃并且不适用于 Python 3.是否有任何想法如何在不移植完整的 MacroPy 的情况下在 Python 3 中模拟这种跟踪行为?

It is possible that my initial approach with transformation of the tree is not feasible. In fact, an elementary step of evaluation of Python expression is not neccessary have to be a replacement of some sub-expression to a simpler one (like in arithmetic). For example, list comprehensions provide a much more complicated behaviour that cannot be expressed in terms replace this thing by that thing, then repeat recursively. So I restate a the question a little bit. I need some way to programmaticaly show how Python expressions are evaluated step by step. For example, MacroPy's tracing feature, mentioned by @jasonharper, is acceptable solution at this stage. Unfortunately, MacroPy seem to be abandoned and doesn't work with Python 3. Are there any ideas how to resemble this tracing behaviour in Python 3 without porting full MacroPy?

EDIT2.

就在我获得这个赏金之后，我发现了类似的问题和具有非常接近功能的调试器.但是，由于该问题没有最终答案，而且我不需要完整的调试器，因此我仍在寻找可以在 Jupyter 环境中使用的答案.

Just after I awarded this bounty, I found similar question and a debugger with very close features. However, as there's no ultimate answer for that question, and I do not need the full debugger, I'm still looking for an answer that can be used for example in Jupyter environment.

推荐答案

Expression stepping 在 Thonny IDE 中实现.

Expression stepping is implemented in Thonny IDE.

它使用 AST 检测，其中每个(子)表达式 e 被转换为 after(before(), e).函数 before 和 after 是用于在 Python 的跟踪系统中引起额外的 call 事件的伪函数.这些额外的调用通知(子)表达式评估即将开始或刚刚结束.(添加了类似的伪函数来检测每个语句的开始和结束.)

It uses AST instrumentation, where each (sub)expression e is transformed into after(before(<location info>), e). Functions before and after are dummy functions for causing extra call-events in Python's tracing system. These extra calls notify when (sub)expression evaluation is about to start or has just ended. (Similar dummy functions are added to detect start and end of each statement.)

这些新事件的 AST 检测和解释在 thonny.backend.FancyTracer.

AST instrumentation and interpretation of these new events is done in thonny.backend.FancyTracer.

Python 的 AST 节点包含相应文本范围的起始位置，但有时不正确.结束位置完全丢失.thonny.ast_utils.mark_text_ranges 试图解决这个问题(但目前解决方案不完整).

Python's AST nodes contain start position of corresponding text ranges, but they are sometimes incorrect. End positions are completely missing. thonny.ast_utils.mark_text_ranges tries to take care of this (but the solution is incomplete at the moment).

如果有人将相关功能从 Thonny 中提取到更通用的包中，那就太好了.甚至可能有两个包——一个用于计算 Python AST 的位置信息，另一个用于详细跟踪 Python 代码.如果有人带头，我愿意提供帮助.

这篇关于一步一步跟踪 Python 表达式求值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！