将python源代码转换为带有完整注释的AST的简单方法 [英] Easy way to convert python source code to an AST with comments intact

查看:37
本文介绍了将python源代码转换为带有完整注释的AST的简单方法的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我已经对如何在保留注释的情况下捕获 python AST 进行了大量搜索.建议的方法包括使用 asttokenize 库来完成工作.

我在按照我的要求使用这些库方面取得了相当大的成功,但我觉得必须有更好的方法.

这个想法源于lib2to3 将python2代码转换为python3代码并带有注释保存.此外,该过程被称为 Source-Code-in-Python2 -> AST -> Source-Code-in-Python3(以简化方式提出).

我的问题是如何捕获中间 AST?我查看了 python-docs 但没有命令行标志来获取 AST.

只是为您提供上下文:我正在尝试将 python 源代码转换为 XML 文件(保留注释)以进行进一步处理

解决方案

只是为您提供上下文:我正在尝试将 python 源代码转换为 XML 文件(保留注释)以供进一步处理

em>

一种简单"的方法是使用已经可以做到这一点的工具,而不是重新发明它,尤其是在您时间紧迫的情况下.

我们的 DMS 软件再工程工具包可以解析 Python(和许多其他语言)、构建 AST 和捕获注释,并将结果树吐出为 XML.请参阅下面的示例.

备注:XML 最初看起来不错,但它是表示/分析/转换代码的笨拙方式.像 DMS 这样的工具存在的原因是提供所有必要的机制来以比 XML 转换更有效的方式操作解析的 AST,并且扩展性更好(例如,数百万行代码):最终,节省工程时间和运行时间.

即使您决定使用 XML,您将从哪里获得处理它的好工具?(XSLT 不是正确的答案).最后,如果您打算修改程序,并且更改了 XML,您打算如何取回源代码?DMS 可以修改 AST 并重新生成有效的源程序文本(包括注释).

因此,虽然 DMS 会以 XML 格式导出 AST(因为像您这样的人似乎坚持这样做),但此功能在实践中很少使用.典型的用例是解析、分析、修改 AST,然后漂亮地打印修改后的 AST,所有这些都以集成的方式使用 DMS.

对于这个python程序:

# 标题中的注释导入系统代币布兰克斯=1类 MyClassNameTranslator:# get_name 查找名称def get_name(self, name):"""获取真实姓名的翻译"""返回 self.realnames[name]

DMS 生成其 AST 的以下 XML 版本,并包含捕获的评论:

C:[snip]PythonToolsParser>run ..domainparser ++XML C:[snip]tiny.pyPython~v3_0 域解析器 2.5.15 版版权所有 (C) 1996-2013 Semantic Designs, Inc;版权所有;标清机密由 DMS (R) 软件再造工具包提供支持树中有 165 个树节点.<?xml version="1.1" encoding="ISO-8859-1"?><!-- 使用 DMS PrintASTasXML (v.1.03) --><!-- XML 生成于 2014/03/01 12:14:49 --><DMSForest><tree node="Python" type="1" domain="1" id="yk0x" parents="0" line="2" column="1" file="1"><tree node="file_input" type="2" domain="1" id="yk0w" line="2" column="1" file="1"><tree node="file_input_element_list" type="4" domain="1" id="yk0v" line="2" column="1" file="1"><tree node="file_input_element_list" type="4" domain="1" id="yjww" line="2" column="1" file="1"><tree node="file_input_element_list" type="4" domain="1" id="yjvc" line="2" column="1" file="1"><tree node="file_input_element_list" type="4" domain="1" id="yjus" line="2" column="1" file="1"><tree node="file_input_element_list" type="3" domain="1" id="ydby" line="2" column="1" file="1"/><tree node="file_input_element" type="5" domain="1" id="yjuq" line="2" column="1" file="1"><tree node="NEWLINE" type="282" domain="1" id="ydbn"literal="0" line="2" column="1" file="1"><precomment child="0" index="1">#标题中的注释</precomment><tree node="file_input_element" type="6" domain="1" id="yjvb" line="3" column="1" file="1"><tree node="stmt" type="7" domain="1" id="yjva" line="3" column="1" file="1"><tree node="simple_stmt" type="9" domain="1" id="yjv9" line="3" column="1" file="1"><tree node="small_stmt_list" type="11" domain="1" id="yjv3" line="3" column="1" file="1"><tree node="small_stmt" type="45" domain="1" id="yjv1" line="3" column="1" file="1"><tree node="'import'" type="305" domain="1" id="yjup"literal="0" line="3" column="1" file="1"/><tree node="dotted_as_name_list" type="53" domain="1" id="yjuz" line="3" column="8" file="1"><tree node="dotted_as_name" type="60" domain="1" id="yjuy" line="3" column="8" file="1"><tree node="dotted_name" type="61" domain="1" id="yjux" line="3" column="8" file="1"><tree node="NAME" type="310" domain="1" id="yjuv" line="3" column="8" file="1"><literal>sys</literal><tree node="NEWLINE" type="282" domain="1" id="yjuw"literal="0" line="3" column="11" file="1"/><tree node="file_input_element" type="6" domain="1" id="yjwv" line="5" column="1" file="1"><tree node="stmt" type="7" domain="1" id="yjwu" line="5" column="1" file="1"><tree node="simple_stmt" type="9" domain="1" id="yjwt" line="5" column="1" file="1"><tree node="small_stmt_list" type="11" domain="1" id="yjwo" line="5" column="1" file="1"><tree node="small_stmt" type="14" domain="1" id="yjwl" line="5" column="1" file="1"><tree node="assign_stmt" type="15" domain="1" id="yjwj" line="5" column="1" file="1"><tree node="target_list" type="215" domain="1" id="yjvg" line="5" column="1" file="1"><tree node="targets" type="217" domain="1" id="yjvf" line="5" column="1" file="1"><tree node="target" type="221" domain="1" id="yjve" line="5" column="1" file="1"><tree node="NAME" type="310" domain="1" id="yjv8" line="5" column="1" file="1"><literal>TOKENBLANKS</literal><tree node="'='" type="284" domain="1" id="yjvd"literal="0" line="5" column="12" file="1"/><tree node="assign_rhs" type="29" domain="1" id="yjwh" line="5" column="13" file="1"><tree node="test_list" type="30" domain="1" id="yjwf" line="5" column="13" file="1"><tree node="tests" type="32" domain="1" id="yjwc" line="5" column="13" file="1"><tree node="test" type="151" domain="1" id="yjwa" line="5" column="13" file="1"><tree node="or_test" type="152" domain="1" id="yjw8" line="5" column="13" file="1"><tree node="and_test" type="154" domain="1" id="yjw4" line="5" column="13" file="1"><tree node="not_test" type="157" domain="1" id="yjw1" line="5" column="13" file="1"><tree node="comparison" type="158" domain="1" id="yjvz" line="5" column="13" file="1"><tree node="expr" type="170" domain="1" id="yjvx" line="5" column="13" file="1"><tree node="xor_expr" type="172" domain="1" id="yjvv" line="5" column="13" file="1"><tree node="and_expr" type="174" domain="1" id="yjvs" line="5" column="13" file="1"><tree node="shift_expr" type="176" domain="1" id="yjvq" line="5" column="13" file="1"><tree node="arith_expr" type="179" domain="1" id="yjvo" line="5" column="13" file="1"><tree node="term" type="182" domain="1" id="yjvn" line="5" column="13" file="1"><tree node="factor" type="187" domain="1" id="yjvm" line="5" column="13" file="1"><tree node="power" type="191" domain="1" id="yjvl" line="5" column="13" file="1"><tree node="value" type="194" domain="1" id="yjvk" line="5" column="13" file="1"><tree node="constant" type="197" domain="1" id="yjvj" line="5" column="13" file="1"><tree node="INTEGER" type="355" domain="1" id="yjvh"literal="1" line="5" column="13" file="1"/><tree node="NEWLINE" type="282" domain="1" id="yjvi"literal="0" line="5" column="14" file="1"/><tree node="file_input_element" type="6" domain="1" id="yk0u" line="7" column="1" file="1"><tree node="stmt" type="8" domain="1" id="yk0p" line="7" column="1" file="1"><tree node="compound_stmt" type="143" domain="1" id="yk0s" line="7" column="1" file="1"><tree node="decorators" type="144" domain="1" id="yjwx" line="7" column="1" file="1"/><tree node="'class'" type="330" domain="1" id="yjws"literal="0" line="7" column="1" file="1"/><tree node="NAME" type="310" domain="1" id="yjwy" line="7" column="7" file="1"><literal>MyClassNameTranslator</literal><tree node="':'" type="314" domain="1" id="yjwz"literal="0" line="7" column="28" file="1"/><tree node="block" type="115" domain="1" id="yk0q" line="7" column="29" file="1"><tree node="NEWLINE" type="282" domain="1" id="yjx0"literal="0" line="7" column="29" file="1"/><tree node="INDENT" type="324" domain="1" id="yjx1"literal="0" line="10" column="1" file="1"><precomment child="0" index="1"># get_name 查找名称</precomment><tree node="stmt_list" type="116" domain="1" id="yk0o" line="10" column="5" file="1"><tree node="stmt" type="8" domain="1" id="yk0j" line="10" column="5" file="1"><tree node="compound_stmt" type="119" domain="1" id="yk0m" line="10" column="5" file="1"><tree node="decorators" type="144" domain="1" id="yjx5" line="10" column="5" file="1"/><tree node="'def'" type="326" domain="1" id="yjx4"literal="0" line="10" column="5" file="1"/><tree node="NAME" type="310" domain="1" id="yjx6" line="10" column="9" file="1"><literal>get_name</literal><tree node="parameters" type="121" domain="1" id="yjxe" line="10" column="17" file="1"><tree node="'('" type="327" domain="1" id="yjx7"literal="0" line="10" column="17" file="1"/><tree node="optional_varargslist" type="123" domain="1" id="yjxd" line="10" column="18" file="1"><tree node="varargslist" type="126" domain="1" id="yjw6" line="10" column="18" file="1"><tree node="fpdef_test_list_prefix" type="131" domain="1" id="yjxk" line="10" column="18" file="1"><tree node="fpdef_test_list_prefix" type="130" domain="1" id="yjx9" line="10" column="18" file="1"/><tree node="fpdef_test_comma" type="132" domain="1" id="yjxh" line="10" column="18" file="1"><tree node="fpdef_test" type="133" domain="1" id="yjxc" line="10" column="18" file="1"><tree node="fpdef" type="135" domain="1" id="yjxb" line="10" column="18" file="1"><tree node="NAME" type="310" domain="1" id="yjx8" line="10" column="18" file="1"><文字>自我</文字><tree node="','" type="297" domain="1" id="yjxa"literal="0" line="10" column="22" file="1"/><tree node="fpdef_test" type="133" domain="1" id="yjw3" line="10" column="24" file="1"><tree node="fpdef" type="135" domain="1" id="yjvw" line="10" column="24" file="1"><tree node="NAME" type="310" domain="1" id="yjxg" line="10" column="24" file="1"><文字>名称</文字><tree node="')'" type="328" domain="1" id="yjvt"literal="0" line="10" column="28" file="1"/><tree node="':'" type="314" domain="1" id="yjxl"literal="0" line="10" column="29" file="1"/><tree node="block" type="115" domain="1" id="yk0k" line="10" column="30" file="1"><tree node="NEWLINE" type="282" domain="1" id="yjxf"literal="0" line="10" column="30" file="1"/><tree node="INDENT" type="324" domain="1" id="yjxm"literal="0" line="11" column="1" file="1"/><tree node="stmt_list" type="117" domain="1" id="yk0h" line="11" column="9" file="1"><tree node="stmt_list" type="116" domain="1" id="yjyq" line="11" column="9" file="1"><tree node="stmt" type="7" domain="1" id="yjyp" line="11" column="9" file="1"><tree node="simple_stmt" type="9" domain="1" id="yjyo" line="11" column="9" file="1"><tree node="small_stmt_list" type="11" domain="1" id="yjyk" line="11" column="9" file="1"><tree node="small_stmt" type="13" domain="1" id="yjyh" line="11" column="9" file="1"><tree node="testlist" type="255" domain="1" id="yjyf" line="11" column="9" file="1"><tree node="test_plus" type="256" domain="1" id="yjyd" line="11" column="9" file="1"><tree node="test" type="151" domain="1" id="yjya" line="11" column="9" file="1"><tree node="or_test" type="152" domain="1" id="yjy7" line="11" column="9" file="1"><tree node="and_test" type="154" domain="1" id="yjy5" line="11" column="9" file="1"><tree node="not_test" type="157" domain="1" id="yjy3" line="11" column="9" file="1"><tree node="comparison" type="158" domain="1" id="yjy1" line="11" column="9" file="1"><tree node="expr" type="170" domain="1" id="yjxy" line="11" column="9" file="1"><tree node="xor_expr" type="172" domain="1" id="yjxw" line="11" column="9" file="1"><tree node="and_expr" type="174" domain="1" id="yjxv" line="11" column="9" file="1"><tree node="shift_expr" type="176" domain="1" id="yjxu" line="11" column="9" file="1"><tree node="arith_expr" type="179" domain="1" id="yjxt" line="11" column="9" file="1"><tree node="term" type="182" domain="1" id="yjxs" line="11" column="9" file="1"><tree node="factor" type="187" domain="1" id="yjxr" line="11" column="9" file="1"><tree node="power" type="191" domain="1" id="yjxq" line="11" column="9" file="1"><tree node="value" type="194" domain="1" id="yjxp" line="11" column="9" file="1"><tree node="constant" type="200" domain="1" id="yjxo" line="11" column="9" file="1"><tree node="string_sequence" type="208" domain="1" id="yjxj" line="11" column="9" file="1"><tree node="STRING" type="362" domain="1" id="yjxn" line="11" column="9" file="1"><literal>获取真实姓名的翻译</literal><tree node="NEWLINE" type="282" domain="1" id="yjxi"literal="0" line="11" column="48" file="1"/><tree node="stmt" type="7" domain="1" id="yk0g" line="12" column="9" file="1"><tree node="simple_stmt" type="9" domain="1" id="yk0f" line="12" column="9" file="1"><tree node="small_stmt_list" type="11" domain="1" id="yk09" line="12" column="9" file="1"><tree node="small_stmt" type="39" domain="1" id="yk06" line="12" column="9" file="1"><tree node="'return'" type="302" domain="1" id="yjyn"literal="0" line="12" column="9" file="1"/><tree node="testlist" type="255" domain="1" id="yk03" line="12" column="16" file="1"><tree node="test_plus" type="256" domain="1" id="yk02" line="12" column="16" file="1"><tree node="test" type="151" domain="1" id="yk01" line="12" column="16" file="1"><tree node="or_test" type="152" domain="1" id="yk00" line="12" column="16" file="1"><tree node="and_test" type="154" domain="1" id="yjzz" line="12" column="16" file="1"><tree node="not_test" type="157" domain="1" id="yjzy" line="12" column="16" file="1"><tree node="comparison" type="158" domain="1" id="yjzx" line="12" column="16" file="1"><tree node="expr" type="170" domain="1" id="yjzw" line="12" column="16" file="1"><tree node="xor_expr" type="172" domain="1" id="yjzv" line="12" column="16" file="1"><tree node="and_expr" type="174" domain="1" id="yjzu" line="12" column="16" file="1"><tree node="shift_expr" type="176" domain="1" id="yjzt" line="12" column="16" file="1"><tree node="arith_expr" type="179" domain="1" id="yjzs" line="12" column="16" file="1"><tree node="term" type="182" domain="1" id="yjzr" line="12" column="16" file="1"><tree node="factor" type="187" domain="1" id="yjzq" line="12" column="16" file="1"><tree node="power" type="191" domain="1" id="yjzp" line="12" column="16" file="1"><tree node="value" type="195" domain="1" id="yjzo" line="12" column="16" file="1"><tree node="value" type="195" domain="1" id="yjz0" line="12" column="16" file="1"><tree node="value" type="193" domain="1" id="yjyu" line="12" column="16" file="1"><tree node="atom" type="207" domain="1" id="yjyt" line="12" column="16" file="1"><tree node="NAME" type="310" domain="1" id="yjyr" line="12" column="16" file="1"><文字>自我</文字><tree node="trailer" type="228" domain="1" id="yjyz" line="12" column="20" file="1"><tree node="'.'" type="312" domain="1" id="yjys"literal="0" line="12" column="20" file="1"/><tree node="NAME" type="310" domain="1" id="yjyv" line="12" column="21" file="1"><literal>realnames</literal><tree node="trailer" type="227" domain="1" id="yjzn" line="12" column="30" file="1"><tree node="index" type="230" domain="1" id="yjzm" line="12" column="30" file="1"><tree node="'['" type="358" domain="1" id="yjyy"literal="0" line="12" column="30" file="1"/><tree node="subscript_list" type="234" domain="1" id="yjzj" line="12" column="31" file="1"><tree node="subscript" type="249" domain="1" id="yjzi" line="12" column="31" file="1"><tree node="test" type="151" domain="1" id="yjzh" line="12" column="31" file="1"><tree node="or_test" type="152" domain="1" id="yjzg" line="12" column="31" file="1"><tree node="and_test" type="154" domain="1" id="yjzf" line="12" column="31" file="1"><tree node="not_test" type="157" domain="1" id="yjze" line="12" column="31" file="1"><tree node="comparison" type="158" domain="1" id="yjzd" line="12" column="31" file="1"><tree node="expr" type="170" domain="1" id="yjzc" line="12" column="31" file="1"><tree node="xor_expr" type="172" domain="1" id="yjzb" line="12" column="31" file="1"><tree node="and_expr" type="174" domain="1" id="yjza" line="12" column="31" file="1"><tree node="shift_expr" type="176" domain="1" id="yjz9" line="12" column="31" file="1"><tree node="arith_expr" type="179" domain="1" id="yjz8" line="12" column="31" file="1"><tree node="term" type="182" domain="1" id="yjz7" line="12" column="31" file="1"><tree node="factor" type="187" domain="1" id="yjz6" line="12" column="31" file="1"><tree node="power" type="191" domain="1" id="yjz5" line="12" column="31" file="1"><tree node="value" type="193" domain="1" id="yjz4" line="12" column="31" file="1"><tree node="atom" type="207" domain="1" id="yjz3" line="12" column="31" file="1"><tree node="NAME" type="310" domain="1" id="yjz1" line="12" column="31" file="1"><文字>名称</文字><tree node="']'" type="359" domain="1" id="yjz2"literal="0" line="12" column="35" file="1"/><tree node="NEWLINE" type="282" domain="1" id="yjzl"literal="0" line="12" column="36" file="1"/><tree node="DEDENT" type="325" domain="1" id="yk0e"literal="0" line="14" column="1" file="1"/><tree node="DEDENT" type="325" domain="1" id="yk0i"literal="0" line="14" column="1" file="1"/><文件索引><File index="1">C:/DMS/Domains/Python/v2_6/Examples/tiny.py</File></文件索引><域索引><域索引="1">Python~v3_0</域></域索引></DMSForest>

I have done fair bit of searching around how to capture python ASTs with comments preserved. The suggested way includes using ast and tokenize libraries to get the job done.

I have had fair bit of success in utilizing these libraries as per my requirement but I feel there has to be a better way.

This thought stems from the fact that lib2to3 converts python2 code to python3 code with comments preserved. Also the process is stated to be Source-Code-in-Python2 -> AST -> Source-Code-in-Python3 (to be put forth in a simplified manner).

My question is how do I capture the in-between AST? I have looked at python-docs but there is no command line flag to get hold of the AST.

Just to provide you the context: I am trying to convert python source code to an XML file (with comments preserved) for some further processing

解决方案

Just to provide you the context: I am trying to convert python source code to an XML file (with comments preserved) for some further processing

An "easy" way is to use a tool that already does this, rather than reinventing it, especially if you are short on time.

Our DMS Software Reengineering Toolkit can parse Python (and many other languages), build ASTs, and capture comments, and spit out the resulting tree as XML. See example below.

A remark: XML initially seems nice, but is a clumsy way to represent/analyze/transform code. The reason that tools like DMS exist, is to provide all the machinery necessary to manipulate the parsed ASTs in ways that are more effective than XML transformation, and scale much better (e.g., to millions of lines of code): ultimately, to save engineering time and runtime.

Even if you decide on XML, where are you going to get good tools to process it? (XSLT isn't the right answer). Finally, if you intend to modify the program, and you change the XML, how do you intend to get the source code back? DMS can modify ASTs and regenerate valid source program text (including the comments).

So while DMS will export ASTs in XML (because people like you seem to insist on it), this feature is rarely used in practice. The typical use case is parse, analyze, modify the AST, then prettyprint the modified AST, all using DMS in an integrated way.

For this python program:

# A comment in the header

import sys

TOKENBLANKS=1

class MyClassNameTranslator:

    # get_name looks up name
    def get_name(self, name):
        """Get a translation for a real name"""
        return self.realnames[name]

DMS generates the follow XML version of its AST, complete with captured comments:

C:[snip]PythonToolsParser>run ..domainparser ++XML C:[snip]tiny.py

Python~v3_0 Domain Parser Version 2.5.15
Copyright (C) 1996-2013 Semantic Designs, Inc; All Rights Reserved; SD Confidential
Powered by DMS (R) Software Reengineering Toolkit
165 tree nodes in tree.
<?xml version="1.1" encoding="ISO-8859-1"?>
<!-- Using DMS PrintASTasXML (v.1.03) -->
<!-- XML generated on 2014/03/01 12:14:49 -->
<DMSForest>
  <tree node="Python" type="1" domain="1" id="yk0x" parents="0" line="2" column="1" file="1">
<tree node="file_input" type="2" domain="1" id="yk0w" line="2" column="1" file="1">
  <tree node="file_input_element_list" type="4" domain="1" id="yk0v" line="2" column="1" file="1">
    <tree node="file_input_element_list" type="4" domain="1" id="yjww" line="2" column="1" file="1">
      <tree node="file_input_element_list" type="4" domain="1" id="yjvc" line="2" column="1" file="1">
    <tree node="file_input_element_list" type="4" domain="1" id="yjus" line="2" column="1" file="1">
      <tree node="file_input_element_list" type="3" domain="1" id="ydby" line="2" column="1" file="1"/>
      <tree node="file_input_element" type="5" domain="1" id="yjuq" line="2" column="1" file="1">
        <tree node="NEWLINE" type="282" domain="1" id="ydbn" literal="0" line="2" column="1" file="1">
          <precomment child="0" index="1"># A comment in the header</precomment>
        </tree>
      </tree>
    </tree>
    <tree node="file_input_element" type="6" domain="1" id="yjvb" line="3" column="1" file="1">
      <tree node="stmt" type="7" domain="1" id="yjva" line="3" column="1" file="1">
        <tree node="simple_stmt" type="9" domain="1" id="yjv9" line="3" column="1" file="1">
          <tree node="small_stmt_list" type="11" domain="1" id="yjv3" line="3" column="1" file="1">
        <tree node="small_stmt" type="45" domain="1" id="yjv1" line="3" column="1" file="1">
          <tree node="'import'" type="305" domain="1" id="yjup" literal="0" line="3" column="1" file="1"/>
          <tree node="dotted_as_name_list" type="53" domain="1" id="yjuz" line="3" column="8" file="1">
            <tree node="dotted_as_name" type="60" domain="1" id="yjuy" line="3" column="8" file="1">
              <tree node="dotted_name" type="61" domain="1" id="yjux" line="3" column="8" file="1">
            <tree node="NAME" type="310" domain="1" id="yjuv" line="3" column="8" file="1">
              <literal>sys</literal>
            </tree>
              </tree>
            </tree>
          </tree>
        </tree>
          </tree>
          <tree node="NEWLINE" type="282" domain="1" id="yjuw" literal="0" line="3" column="11" file="1"/>
        </tree>
      </tree>
    </tree>
      </tree>
      <tree node="file_input_element" type="6" domain="1" id="yjwv" line="5" column="1" file="1">
    <tree node="stmt" type="7" domain="1" id="yjwu" line="5" column="1" file="1">
      <tree node="simple_stmt" type="9" domain="1" id="yjwt" line="5" column="1" file="1">
        <tree node="small_stmt_list" type="11" domain="1" id="yjwo" line="5" column="1" file="1">
          <tree node="small_stmt" type="14" domain="1" id="yjwl" line="5" column="1" file="1">
        <tree node="assign_stmt" type="15" domain="1" id="yjwj" line="5" column="1" file="1">
          <tree node="target_list" type="215" domain="1" id="yjvg" line="5" column="1" file="1">
            <tree node="targets" type="217" domain="1" id="yjvf" line="5" column="1" file="1">
              <tree node="target" type="221" domain="1" id="yjve" line="5" column="1" file="1">
            <tree node="NAME" type="310" domain="1" id="yjv8" line="5" column="1" file="1">
              <literal>TOKENBLANKS</literal>
            </tree>
              </tree>
            </tree>
          </tree>
          <tree node="'='" type="284" domain="1" id="yjvd" literal="0" line="5" column="12" file="1"/>
          <tree node="assign_rhs" type="29" domain="1" id="yjwh" line="5" column="13" file="1">
            <tree node="test_list" type="30" domain="1" id="yjwf" line="5" column="13" file="1">
              <tree node="tests" type="32" domain="1" id="yjwc" line="5" column="13" file="1">
            <tree node="test" type="151" domain="1" id="yjwa" line="5" column="13" file="1">
              <tree node="or_test" type="152" domain="1" id="yjw8" line="5" column="13" file="1">
                <tree node="and_test" type="154" domain="1" id="yjw4" line="5" column="13" file="1">
                  <tree node="not_test" type="157" domain="1" id="yjw1" line="5" column="13" file="1">
                <tree node="comparison" type="158" domain="1" id="yjvz" line="5" column="13" file="1">
                  <tree node="expr" type="170" domain="1" id="yjvx" line="5" column="13" file="1">
                    <tree node="xor_expr" type="172" domain="1" id="yjvv" line="5" column="13" file="1">
                      <tree node="and_expr" type="174" domain="1" id="yjvs" line="5" column="13" file="1">
                    <tree node="shift_expr" type="176" domain="1" id="yjvq" line="5" column="13" file="1">
                      <tree node="arith_expr" type="179" domain="1" id="yjvo" line="5" column="13" file="1">
                        <tree node="term" type="182" domain="1" id="yjvn" line="5" column="13" file="1">
                          <tree node="factor" type="187" domain="1" id="yjvm" line="5" column="13" file="1">
                        <tree node="power" type="191" domain="1" id="yjvl" line="5" column="13" file="1">
                          <tree node="value" type="194" domain="1" id="yjvk" line="5" column="13" file="1">
                            <tree node="constant" type="197" domain="1" id="yjvj" line="5" column="13" file="1">
                              <tree node="INTEGER" type="355" domain="1" id="yjvh" literal="1" line="5" column="13" file="1"/>
                            </tree>
                          </tree>
                        </tree>
                          </tree>
                        </tree>
                      </tree>
                    </tree>
                      </tree>
                    </tree>
                  </tree>
                </tree>
                  </tree>
                </tree>
              </tree>
            </tree>
              </tree>
            </tree>
          </tree>
        </tree>
          </tree>
        </tree>
        <tree node="NEWLINE" type="282" domain="1" id="yjvi" literal="0" line="5" column="14" file="1"/>
      </tree>
    </tree>
      </tree>
    </tree>
    <tree node="file_input_element" type="6" domain="1" id="yk0u" line="7" column="1" file="1">
      <tree node="stmt" type="8" domain="1" id="yk0p" line="7" column="1" file="1">
    <tree node="compound_stmt" type="143" domain="1" id="yk0s" line="7" column="1" file="1">
      <tree node="decorators" type="144" domain="1" id="yjwx" line="7" column="1" file="1"/>
      <tree node="'class'" type="330" domain="1" id="yjws" literal="0" line="7" column="1" file="1"/>
      <tree node="NAME" type="310" domain="1" id="yjwy" line="7" column="7" file="1">
        <literal>MyClassNameTranslator</literal>
      </tree>
      <tree node="':'" type="314" domain="1" id="yjwz" literal="0" line="7" column="28" file="1"/>
      <tree node="block" type="115" domain="1" id="yk0q" line="7" column="29" file="1">
        <tree node="NEWLINE" type="282" domain="1" id="yjx0" literal="0" line="7" column="29" file="1"/>
        <tree node="INDENT" type="324" domain="1" id="yjx1" literal="0" line="10" column="1" file="1">
          <precomment child="0" index="1"># get_name looks up name</precomment>
        </tree>
        <tree node="stmt_list" type="116" domain="1" id="yk0o" line="10" column="5" file="1">
          <tree node="stmt" type="8" domain="1" id="yk0j" line="10" column="5" file="1">
        <tree node="compound_stmt" type="119" domain="1" id="yk0m" line="10" column="5" file="1">
          <tree node="decorators" type="144" domain="1" id="yjx5" line="10" column="5" file="1"/>
          <tree node="'def'" type="326" domain="1" id="yjx4" literal="0" line="10" column="5" file="1"/>
          <tree node="NAME" type="310" domain="1" id="yjx6" line="10" column="9" file="1">
            <literal>get_name</literal>
          </tree>
          <tree node="parameters" type="121" domain="1" id="yjxe" line="10" column="17" file="1">
            <tree node="'('" type="327" domain="1" id="yjx7" literal="0" line="10" column="17" file="1"/>
            <tree node="optional_varargslist" type="123" domain="1" id="yjxd" line="10" column="18" file="1">
              <tree node="varargslist" type="126" domain="1" id="yjw6" line="10" column="18" file="1">
            <tree node="fpdef_test_list_prefix" type="131" domain="1" id="yjxk" line="10" column="18" file="1">
              <tree node="fpdef_test_list_prefix" type="130" domain="1" id="yjx9" line="10" column="18" file="1"/>
              <tree node="fpdef_test_comma" type="132" domain="1" id="yjxh" line="10" column="18" file="1">
                <tree node="fpdef_test" type="133" domain="1" id="yjxc" line="10" column="18" file="1">
                  <tree node="fpdef" type="135" domain="1" id="yjxb" line="10" column="18" file="1">
                <tree node="NAME" type="310" domain="1" id="yjx8" line="10" column="18" file="1">
                  <literal>self</literal>
                </tree>
                  </tree>
                </tree>
                <tree node="','" type="297" domain="1" id="yjxa" literal="0" line="10" column="22" file="1"/>
              </tree>
            </tree>
            <tree node="fpdef_test" type="133" domain="1" id="yjw3" line="10" column="24" file="1">
              <tree node="fpdef" type="135" domain="1" id="yjvw" line="10" column="24" file="1">
                <tree node="NAME" type="310" domain="1" id="yjxg" line="10" column="24" file="1">
                  <literal>name</literal>
                </tree>
              </tree>
            </tree>
              </tree>
            </tree>
            <tree node="')'" type="328" domain="1" id="yjvt" literal="0" line="10" column="28" file="1"/>
          </tree>
          <tree node="':'" type="314" domain="1" id="yjxl" literal="0" line="10" column="29" file="1"/>
          <tree node="block" type="115" domain="1" id="yk0k" line="10" column="30" file="1">
            <tree node="NEWLINE" type="282" domain="1" id="yjxf" literal="0" line="10" column="30" file="1"/>
            <tree node="INDENT" type="324" domain="1" id="yjxm" literal="0" line="11" column="1" file="1"/>
            <tree node="stmt_list" type="117" domain="1" id="yk0h" line="11" column="9" file="1">
              <tree node="stmt_list" type="116" domain="1" id="yjyq" line="11" column="9" file="1">
            <tree node="stmt" type="7" domain="1" id="yjyp" line="11" column="9" file="1">
              <tree node="simple_stmt" type="9" domain="1" id="yjyo" line="11" column="9" file="1">
                <tree node="small_stmt_list" type="11" domain="1" id="yjyk" line="11" column="9" file="1">
                  <tree node="small_stmt" type="13" domain="1" id="yjyh" line="11" column="9" file="1">
                <tree node="testlist" type="255" domain="1" id="yjyf" line="11" column="9" file="1">
                  <tree node="test_plus" type="256" domain="1" id="yjyd" line="11" column="9" file="1">
                    <tree node="test" type="151" domain="1" id="yjya" line="11" column="9" file="1">
                      <tree node="or_test" type="152" domain="1" id="yjy7" line="11" column="9" file="1">
                    <tree node="and_test" type="154" domain="1" id="yjy5" line="11" column="9" file="1">
                      <tree node="not_test" type="157" domain="1" id="yjy3" line="11" column="9" file="1">
                        <tree node="comparison" type="158" domain="1" id="yjy1" line="11" column="9" file="1">
                          <tree node="expr" type="170" domain="1" id="yjxy" line="11" column="9" file="1">
                        <tree node="xor_expr" type="172" domain="1" id="yjxw" line="11" column="9" file="1">
                          <tree node="and_expr" type="174" domain="1" id="yjxv" line="11" column="9" file="1">
                            <tree node="shift_expr" type="176" domain="1" id="yjxu" line="11" column="9" file="1">
                              <tree node="arith_expr" type="179" domain="1" id="yjxt" line="11" column="9" file="1">
                            <tree node="term" type="182" domain="1" id="yjxs" line="11" column="9" file="1">
                              <tree node="factor" type="187" domain="1" id="yjxr" line="11" column="9" file="1">
                                <tree node="power" type="191" domain="1" id="yjxq" line="11" column="9" file="1">
                                  <tree node="value" type="194" domain="1" id="yjxp" line="11" column="9" file="1">
                                <tree node="constant" type="200" domain="1" id="yjxo" line="11" column="9" file="1">
                                  <tree node="string_sequence" type="208" domain="1" id="yjxj" line="11" column="9" file="1">
                                    <tree node="STRING" type="362" domain="1" id="yjxn" line="11" column="9" file="1">
                                      <literal>Get a translation for a real name</literal>
                                    </tree>
                                  </tree>
                                </tree>
                                  </tree>
                                </tree>
                              </tree>
                            </tree>
                              </tree>
                            </tree>
                          </tree>
                        </tree>
                          </tree>
                        </tree>
                      </tree>
                    </tree>
                      </tree>
                    </tree>
                  </tree>
                </tree>
                  </tree>
                </tree>
                <tree node="NEWLINE" type="282" domain="1" id="yjxi" literal="0" line="11" column="48" file="1"/>
              </tree>
            </tree>
              </tree>
              <tree node="stmt" type="7" domain="1" id="yk0g" line="12" column="9" file="1">
            <tree node="simple_stmt" type="9" domain="1" id="yk0f" line="12" column="9" file="1">
              <tree node="small_stmt_list" type="11" domain="1" id="yk09" line="12" column="9" file="1">
                <tree node="small_stmt" type="39" domain="1" id="yk06" line="12" column="9" file="1">
                  <tree node="'return'" type="302" domain="1" id="yjyn" literal="0" line="12" column="9" file="1"/>
                  <tree node="testlist" type="255" domain="1" id="yk03" line="12" column="16" file="1">
                <tree node="test_plus" type="256" domain="1" id="yk02" line="12" column="16" file="1">
                  <tree node="test" type="151" domain="1" id="yk01" line="12" column="16" file="1">
                    <tree node="or_test" type="152" domain="1" id="yk00" line="12" column="16" file="1">
                      <tree node="and_test" type="154" domain="1" id="yjzz" line="12" column="16" file="1">
                    <tree node="not_test" type="157" domain="1" id="yjzy" line="12" column="16" file="1">
                      <tree node="comparison" type="158" domain="1" id="yjzx" line="12" column="16" file="1">
                        <tree node="expr" type="170" domain="1" id="yjzw" line="12" column="16" file="1">
                          <tree node="xor_expr" type="172" domain="1" id="yjzv" line="12" column="16" file="1">
                        <tree node="and_expr" type="174" domain="1" id="yjzu" line="12" column="16" file="1">
                          <tree node="shift_expr" type="176" domain="1" id="yjzt" line="12" column="16" file="1">
                            <tree node="arith_expr" type="179" domain="1" id="yjzs" line="12" column="16" file="1">
                              <tree node="term" type="182" domain="1" id="yjzr" line="12" column="16" file="1">
                            <tree node="factor" type="187" domain="1" id="yjzq" line="12" column="16" file="1">
                              <tree node="power" type="191" domain="1" id="yjzp" line="12" column="16" file="1">
                                <tree node="value" type="195" domain="1" id="yjzo" line="12" column="16" file="1">
                                  <tree node="value" type="195" domain="1" id="yjz0" line="12" column="16" file="1">
                                <tree node="value" type="193" domain="1" id="yjyu" line="12" column="16" file="1">
                                  <tree node="atom" type="207" domain="1" id="yjyt" line="12" column="16" file="1">
                                    <tree node="NAME" type="310" domain="1" id="yjyr" line="12" column="16" file="1">
                                      <literal>self</literal>
                                    </tree>
                                  </tree>
                                </tree>
                                <tree node="trailer" type="228" domain="1" id="yjyz" line="12" column="20" file="1">
                                  <tree node="'.'" type="312" domain="1" id="yjys" literal="0" line="12" column="20" file="1"/>
                                  <tree node="NAME" type="310" domain="1" id="yjyv" line="12" column="21" file="1">
                                    <literal>realnames</literal>
                                  </tree>
                                </tree>
                                  </tree>
                                  <tree node="trailer" type="227" domain="1" id="yjzn" line="12" column="30" file="1">
                                <tree node="index" type="230" domain="1" id="yjzm" line="12" column="30" file="1">
                                  <tree node="'['" type="358" domain="1" id="yjyy" literal="0" line="12" column="30" file="1"/>
                                  <tree node="subscript_list" type="234" domain="1" id="yjzj" line="12" column="31" file="1">
                                    <tree node="subscript" type="249" domain="1" id="yjzi" line="12" column="31" file="1">
                                      <tree node="test" type="151" domain="1" id="yjzh" line="12" column="31" file="1">
                                    <tree node="or_test" type="152" domain="1" id="yjzg" line="12" column="31" file="1">
                                      <tree node="and_test" type="154" domain="1" id="yjzf" line="12" column="31" file="1">
                                        <tree node="not_test" type="157" domain="1" id="yjze" line="12" column="31" file="1">
                                          <tree node="comparison" type="158" domain="1" id="yjzd" line="12" column="31" file="1">
                                        <tree node="expr" type="170" domain="1" id="yjzc" line="12" column="31" file="1">
                                          <tree node="xor_expr" type="172" domain="1" id="yjzb" line="12" column="31" file="1">
                                            <tree node="and_expr" type="174" domain="1" id="yjza" line="12" column="31" file="1">
                                              <tree node="shift_expr" type="176" domain="1" id="yjz9" line="12" column="31" file="1">
                                            <tree node="arith_expr" type="179" domain="1" id="yjz8" line="12" column="31" file="1">
                                              <tree node="term" type="182" domain="1" id="yjz7" line="12" column="31" file="1">
                                                <tree node="factor" type="187" domain="1" id="yjz6" line="12" column="31" file="1">
                                                  <tree node="power" type="191" domain="1" id="yjz5" line="12" column="31" file="1">
                                                <tree node="value" type="193" domain="1" id="yjz4" line="12" column="31" file="1">
                                                  <tree node="atom" type="207" domain="1" id="yjz3" line="12" column="31" file="1">
                                                    <tree node="NAME" type="310" domain="1" id="yjz1" line="12" column="31" file="1">
                                                      <literal>name</literal>
                                                    </tree>
                                                  </tree>
                                                </tree>
                                                  </tree>
                                                </tree>
                                              </tree>
                                            </tree>
                                              </tree>
                                            </tree>
                                          </tree>
                                        </tree>
                                          </tree>
                                        </tree>
                                      </tree>
                                    </tree>
                                      </tree>
                                    </tree>
                                  </tree>
                                  <tree node="']'" type="359" domain="1" id="yjz2" literal="0" line="12" column="35" file="1"/>
                                </tree>
                                  </tree>
                                </tree>
                              </tree>
                            </tree>
                              </tree>
                            </tree>
                          </tree>
                        </tree>
                          </tree>
                        </tree>
                      </tree>
                    </tree>
                      </tree>
                    </tree>
                  </tree>
                </tree>
                  </tree>
                </tree>
              </tree>
              <tree node="NEWLINE" type="282" domain="1" id="yjzl" literal="0" line="12" column="36" file="1"/>
            </tree>
              </tree>
            </tree>
            <tree node="DEDENT" type="325" domain="1" id="yk0e" literal="0" line="14" column="1" file="1"/>
          </tree>
        </tree>
          </tree>
        </tree>
        <tree node="DEDENT" type="325" domain="1" id="yk0i" literal="0" line="14" column="1" file="1"/>
      </tree>
    </tree>
      </tree>
    </tree>
  </tree>
</tree>
  </tree>
  <FileIndex>
<File index="1">C:/DMS/Domains/Python/v2_6/Examples/tiny.py</File>
  </FileIndex>
  <DomainIndex>
<Domain index="1">Python~v3_0</Domain>
  </DomainIndex>
</DMSForest>

这篇关于将python源代码转换为带有完整注释的AST的简单方法的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆