如何获得“声明结束”的行号？在Python ast中 [英] How to get lineno of "end-of-statement" in Python ast

问题描述

我正在尝试处理在Python中操作另一个脚本的脚本，要修改的脚本具有以下结构：

I am trying to work on a script that manipulates another script in Python, the script to be modified has structure like:

class SomethingRecord(Record):
    description = 'This records something'
    author = 'john smith'

我使用 ast 来找到 description 行号，并使用一些代码进行更改原始文件，其新的描述字符串基于行号。

I use ast to locate the description line number, and I use some code to change the original file with new description string base on the line number. So far so good.

现在唯一的问题是 description 偶尔是多行字符串，例如

Now the only issue is description occasionally is a multi-line string, e.g.

    description = ('line 1'
                   'line 2'
                   'line 3')

或

    description = 'line 1' \
        'line 2' \
        'line 3'

并且我只有第一行的行号，而没有后面的行。所以我的单行替换器会做

and I only have the line number of the first line, not the following lines. So my one-line replacer would do

    description = 'new value'
        'line 2' \
        'line 3'

并且代码被破坏了。我发现，如果我同时知道描述赋值的开始和结束/行数，则可以修复我的代码来处理这种情况。如何使用Python标准库获取此类信息？

and the code is broken. I figured that if I know both the lineno of start and end/number of lines of description assignment I could repair my code to handle such situation. How do I get such information with Python standard library?

推荐答案

我看着其他答案；当您真正的问题是修改代码之一时，似乎人们在进行后空翻以避开计算行数的问题。这表明基线机制并不能帮助您真正需要的方式。

I looked at the other answers; it appears people are doing backflips to get around the problems of computing line numbers, when your real problem is one of modifying the code. That suggests the baseline machinery is not helping you the way you really need.

如果您使用程序转换系统（PTS），您可以避免很多这种废话。

If you use a program transformation system (PTS), you could avoid a lot of this nonsense.

好的PTS会将您的源代码解析为AST ，然后让您应用源代码级重写规则来修改AST，最后将修改后的AST转换回源文本。通常，PTS基本上接受这种形式的转换规则：

A good PTS will parse your source code to an AST, and then let you apply source-level rewrite rules to modify the AST, and will finally convert the modified AST back into source text. Generically PTSes accept transformation rules of essentially this form:

   if you see *this*, replace it by *that*

[构建AST的解析器不是PTS。他们不允许这样的规则；您可以编写临时代码来破解树，但这通常很尴尬。他们不是通过AST来重新生成文本。]

[A parser that builds an AST is NOT a PTS. They don't allow rules like this; you can write ad hoc code to hack at the tree, but that's usually pretty awkward. Not do they do the AST to source text regeneration.]

（我的PTS，请参见bio，称为）DMS是可以实现此目的的PTS。通过使用以下重写规则，可以轻松实现OP的特定示例：

(My PTS, see bio, called) DMS is a PTS that could accomplish this. OP's specific example would be accomplished easily by using the following rewrite rule:

 source domain Python; -- tell DMS the syntax of pattern left hand sides
 target domain Python; -- tell DMS the syntax of pattern right hand sides

 rule replace_description(e: expression): statement -> statement =
     " description = \e "
  ->
     " description = ('line 1'
                      'line 2'
                      'line 3')";

一个转换规则的名称为 replace_description ，以区别于所有我们可能会定义的另一条规则。规则参数（e：表达式）指示模式将允许源语言定义的任意表达式。 statement-> statement 表示规则将源语言的语句映射到目标语言的语句；我们可以使用DMS提供的Python语法中的任何其他语法类别。这里使用的 是 metaquote ，用于区分规则语言的语法和主题语言的语法。第二个-> 将源模式 this 与目标模式 that 分开。

The one transformation rule is given an name replace_description to distinguish it from all the other rule we might define. The rule parameters (e: expression) indicate the pattern will allow an arbitrary expression as defined by the source language. statement->statement means the rule maps a statement in the source language, to a statement in the target language; we could use any other syntax category from the Python grammar provided to DMS. The " used here is a metaquote, used to distinguish the syntax of the rule language form the syntax of the subject language. The second -> separates the source pattern this from the target pattern that.

您会发现不需要PTS通过使用与解析源文件相同的解析器实际解析模式，将规则表面语法转换为相应的AST，用于模式生成/生成的AST用于实现模式匹配/替换。从AST来看，原始代码的实际布局（行距，换行符，注释）不会影响DMS的匹配或替换能力，注释不是问题，因为注释是附加到树节点而不是树节点；它们DMS确实捕获了所有树元素的行和精确列信息；只是不需要实现t变形。使用该行/列信息，DMS也将代码布局保留在输出中。

You'll notice that there is no need to mention line numbers. The PTS converts the rule surface syntax into corresponding ASTs by actually parsing the patterns with the same parser used to parse the source file. The ASTs produced for the patterns are used to effect the pattern match/replacement. Because this is driven from ASTs, the actual layout of the orginal code (spacing, linebreaks, comments) don't affect DMS's ability to match or replace. Comments aren't a problem for matching because they are attached to tree nodes rather than being tree nodes; they are preserved in the transformed program. DMS does capture line and precise column information for all tree elements; just not needed to implement transformations. Code layout is also preserved in the output by DMS, using that line/column information.

其他PTS通常提供类似的功能。

Other PTSes offer generally similar capabilities.

这篇关于如何获得“声明结束”的行号？在Python ast中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！