python如何构建其AST [英] How does python build its AST

问题描述

关于python如何将文件解析为已编译的
字节码的快速问题。它是否先将整个文件解析为AST，然后编译AST，或者它是否可以动态构建和编译AST，因为它读取了

表达式？ （如果是前一种情况，为什么在

定义之前不能调用函数？）


谢谢，

Jordan

解决方案

12月7日下午3:23，MonkeeSage< MonkeeS ... @ gmail.comwrote：

关于python如何将文件解析为已编译的
字节码的快速问题。它是否先将整个文件解析为AST，然后编译AST，或者它是否可以动态构建和编译AST，因为它读取了

表达式？ （如果是前一种情况，为什么在

定义之前不能调用函数？）


谢谢，

Jordan

Python使用高度优化的基于LL（1）解析器的表来创建一个

语法树。在Python 2.5中，它在编译之前将具体语法树

（CST）转换为AST。在此之前，它直接编译了

CST。我不确定你要求的是什么（在括号中）？

解析器操作或预处理树？后者肯定是

，你可以使用

解析器模块和编译功能构建自己的编译机器。


Kay

12月7日上午9:50，Kay Schluehr< kay.schlu ... @ gmx.netwrote：

12月7日下午3:23，MonkeeSage< MonkeeS ... @ gmail.comwrote：

关于python如何解析的快速问题一个文件编译成

字节码。它是否先将整个文件解析为AST，然后编译AST，或者它是否可以动态构建和编译AST，因为它读取了

表达式？ （如果是前一种情况，为什么在

定义之前不能调用函数？）

谢谢，<
Jordan

Python使用高度优化的基于LL（1）解析器的表来创建一个

语法树。在Python 2.5中，它在编译之前将具体语法树

（CST）转换为AST。在此之前，它直接编译了

CST。我不确定你要求的是什么（在括号中）？

解析器操作或预处理树？后者肯定是

，你可以使用

解析器模块和编译功能构建自己的编译机器。


Kay

感谢您的回复。你回答了我的主要问题。第二个

问题是为什么在源文件中声明之前尝试使用变量/函数

的一个NameError，因为python已经是

看到第一次建立CST / AST的声明？在

编译时，它不应该已经知道了吗？ （原谅我的

无知。）


问候，

Jordan

< blockquote> 12月7日上午9：03，MonkeeSage< MonkeeS ... @ gmail.comwrote：

12月7日上午9：50，Kay Schluehr < kay.schlu ... @ gmx.netwrote：

12月7日下午3:23，MonkeeSage< MonkeeS ... @ gmail.comwrote：

关于python如何将文件解析为已编译的
字节码的快速问题。它是否先将整个文件解析为AST，然后编译AST，或者它是否可以动态构建和编译AST，因为它读取了

表达式？ （如果是前一种情况，为什么在

定义之前不能调用函数？）

谢谢，

Jordan

Python使用高度优化的表基于LL（1）解析器创建一个

语法树。在Python 2.5中，它在编译之前将具体语法树

（CST）转换为AST。在此之前，它直接编译了

CST。我不确定你要求的是什么（在括号中）？

解析器操作或预处理树？后者肯定是可能的，你可以使用

解析器模块和编译函数构建自己的编译机器。

Kay

感谢您的回复。你回答了我的主要问题。第二个

问题是为什么在源文件中声明之前尝试使用变量/函数

的一个NameError，因为python已经是

看到第一次建立CST / AST的声明？在

编译时，它不应该已经知道了吗？ （原谅我的

无知。）


问候，

Jordan

请记住，Python是一种高度动态的语言。你不能

保证在实际执行之前可以访问名称

指向你试图访问它。例如：

>> Hello（）＃这应该怎么称呼？

Traceback（最近一次调用最后一次）：

文件"< stdin>"，第1行，在< module>

NameError：name''Hello''未定义

>>＃标准hello函数
def Hello（）：print''Hello！''

....

>> Hello（ ）

您好！

>>>
＃通过全局字典修改
def Vikings（）：打印''垃圾邮件，垃圾邮件，垃圾邮件！垃圾邮件！''

....

>> globals（）[''olleH''[:: - 1]] = Vikings
Hello（）

垃圾邮件，垃圾邮件，垃圾邮件！垃圾邮件！

>>>
＃另一个基于字典的修改
globals（）[''你好''] = lambda：''走开！''
你好（）

''走开！''

>>>
＃删除语法糖并直接创建一个函数对象
import new
import compiler
new_code = compiler.compile（''print" Die in a fire！"''， ''Hateful''，''single''）
Hello = new.function（new_code，{}，''Angry''）
Hello（）

死火！

>>>
＃可调用对象（不是函数） ！）
类AnnoyingNeighbor（对象）：

.... def __call __（self）：

....打印''Hi-diddly-ho，邻居！''

....

>> Hello = AnnoyingNeighbor（）
Hello（）

Hi-diddly-ho，邻居！

>>>

如果这是在一个文件中，那么第一次调用哪个版本的Hello应该转到

到？应该将''Hello''名称绑定到函数，可调用的

类实例，new.function的结果？如果另一个

声明怎么回事


问题是，在Python中，函数是一流的对象，

就像类一样定义和其他变量。 ''def''语句

是执行的东西。它将

函数的代码块编译成一个函数对象，然后将其分配给

函数的名称。


当Python遇到函数调用时，它必须在

当前名称字典中查找名称。在任何但最简单的情况下，名称

可能不会引用其先前的值：任何干预调用或

语句可能会将该名称反弹到另一个对象，

直接，通过副作用，甚至通过C接口。你的
在解析def语句之前不能调用函数

你可以在分配任何内容之前打印变量的值

it。


与C不同，Python下的函数并不特殊。与C ++不同，类

也不是特别特别。它们有语法糖来帮助

编码下降，但它们只是Python对象，并且与所有其他Python对象一样受限于

规则。


- 贾森

A quick question about how python parses a file into compiled
bytecode. Does it parse the whole file into AST first and then compile
the AST, or does it build and compile the AST on the fly as it reads
expressions? (If the former case, why can''t functions be called before
their definitions?)

Thanks,
Jordan

解决方案

On Dec 7, 3:23 pm, MonkeeSage <[e-mail]:

A quick question about how python parses a file into compiled
bytecode. Does it parse the whole file into AST first and then compile
the AST, or does it build and compile the AST on the fly as it reads
expressions? (If the former case, why can''t functions be called before
their definitions?)

Thanks,
Jordan

Python uses a highly optimized table based LL(1) parser to create a
syntax tree. In Python 2.5 it transforms the concrete syntax tree
( CST ) into an AST before compilation. Before that it compiled the
CST directly. I''m not sure what you are asking for ( in parentheses )?
Parser actions or preprocessing the tree? The latter is definitely
possible and you can build your own compilation machinery using the
parser module and the compile function.

Kay

On Dec 7, 9:50 am, Kay Schluehr <[e-mail]:

On Dec 7, 3:23 pm, MonkeeSage <[e-mail]:

A quick question about how python parses a file into compiled
bytecode. Does it parse the whole file into AST first and then compile
the AST, or does it build and compile the AST on the fly as it reads
expressions? (If the former case, why can''t functions be called before
their definitions?)

Thanks,
Jordan

Python uses a highly optimized table based LL(1) parser to create a
syntax tree. In Python 2.5 it transforms the concrete syntax tree
( CST ) into an AST before compilation. Before that it compiled the
CST directly. I''m not sure what you are asking for ( in parentheses )?
Parser actions or preprocessing the tree? The latter is definitely
possible and you can build your own compilation machinery using the
parser module and the compile function.

Kay

Thanks for your reply. You answered my main question. The secondary
question is why is it a NameError to try to use a variable/function
prior to the declaration in a source file, since python has already
seen the declaration on the first pass building the CST/AST? At
compile time, shouldn''t it already know about it? (Forgive my
ignorance.)

Regards,
Jordan

On Dec 7, 9:03 am, MonkeeSage <[e-mail]:

On Dec 7, 9:50 am, Kay Schluehr <[e-mail]:

On Dec 7, 3:23 pm, MonkeeSage <[e-mail]:

A quick question about how python parses a file into compiled
bytecode. Does it parse the whole file into AST first and then compile
the AST, or does it build and compile the AST on the fly as it reads
expressions? (If the former case, why can''t functions be called before
their definitions?)

Thanks,
Jordan

Python uses a highly optimized table based LL(1) parser to create a
syntax tree. In Python 2.5 it transforms the concrete syntax tree
( CST ) into an AST before compilation. Before that it compiled the
CST directly. I''m not sure what you are asking for ( in parentheses )?
Parser actions or preprocessing the tree? The latter is definitely
possible and you can build your own compilation machinery using the
parser module and the compile function.

Kay

Thanks for your reply. You answered my main question. The secondary
question is why is it a NameError to try to use a variable/function
prior to the declaration in a source file, since python has already
seen the declaration on the first pass building the CST/AST? At
compile time, shouldn''t it already know about it? (Forgive my
ignorance.)

Regards,
Jordan

Remember that Python is a highly dynamic language. You can''t
guarantee that a name will be accessible until the actual execution
point that you try to access it. For example:

>>Hello() # What should this call?

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name ''Hello'' is not defined

>># The standard hello function
def Hello(): print ''Hello!''

....

>>Hello()

Hello!

>>>
# Modification through the globals dictionary
def Vikings(): print ''Spam, spam, spam! Spam!''

....

>>globals()[ ''olleH''[::-1] ] = Vikings
Hello()

Spam, spam, spam! Spam!

>>>
# Another dictionary-based modification
globals()[''Hello''] = lambda: ''Go away!''
Hello()

''Go away!''

>>>
# Remove the syntactic sugar and make a function object directly
import new
import compiler
new_code = compiler.compile( ''print "Die in a fire!"'', ''Hateful'', ''single'')
Hello = new.function( new_code, {}, ''Angry'')
Hello()

Die in a fire!

>>>
# A callable object (not a function!)
class AnnoyingNeighbor(object):

.... def __call__(self):
.... print ''Hi-diddly-ho, neighbor!''
....

>>Hello = AnnoyingNeighbor()
Hello()

Hi-diddly-ho, neighbor!

>>>

If this was in a file, which version of Hello should the first call go
to? Should the ''Hello'' name be bound to the function, the callable
class instance, the result of new.function? What if another
statement

The problem is that, in Python, functions are first-class objects,
just like class definitions and other variables. The ''def'' statement
is something that gets executed. It compiles the code block for the
function into a function object, then assigns it to the name of the
function.

When Python hits a function call, it must look up the name in the
current name dictionary. In any but that most simple cases, the name
may not refer to its previous values: any intervening calls or
statements could have rebound that name to another object, either
directly, through side-effects, or even through the C interface. You
can''t call a function before its def statement is parsed any more than
you can print the value of a variable before you assign anything to
it.

Unlike C, functions are not special under Python. Unlike C++, classes
aren''t terribly special either. They have syntactic sugar to help the
coding go down, but they are only Python objects, and are bound to the
same rules as all other Python objects.

--Jason

这篇关于python如何构建其AST的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！