Python语法在内部如何使用? [英] How is the Python grammar used internally?
问题描述
I'm trying to get a deeper understanding of how Python works, and I've been looking at the grammar shown at http://docs.python.org/3.3/reference/grammar.html.
我注意到它说您也必须更改parsermodule.c，但说实话，我只是不关注这里发生的事情.
I notice it says you would have to change parsermodule.c also, but truthfully I'm just not following what's going on here.
我知道语法是如何阅读该语言的规范，但是...我什至不知道该写的是什么.它看起来几乎像Python，但实际上不是.
I understand that a grammar is a specification for how to read the language, but...I can't even tell what this is written in. It looks almost like Python but then it isn't.
我希望能更好地理解此规范以及Python在内部如何使用它来做某事.取决于什么(答案是一切，但我具体是指引擎"的哪个方面正在处理它)，使用它的方式以及它如何与编译/运行脚本联系在一起?
I'm looking to get a better understanding of this specification and how it is used internally by Python to....do things. What depends on it (the answer is everything, but I mean specifically which aspect of the "engine" is processing it), what uses it, how does it tie in to compiling/running a script?
很难相信整个语言可以归结为两页的说明...
It's hard to believe that the whole language comes down to a two page specification...
推荐答案
语法用于描述语言中所有可能的字符串.在指定解析器应如何解析语言时也很有用.
A grammar is used to describe all possible strings in a language. It is also useful in specifying how a parser should parse the language.
在此语法中，似乎他们使用的是自己的 EBNF ，其中非终止符是任何小写单词，而终止符则全部是大写字母或用引号引起来.例如，NEWLINE是终端，arith_expr是非终端，而'if'也是终端.任何非终结符都可以由其相应生产规则的冒号右侧的任何内容替换.例如，如果您查看第一个规则:
In this grammar it seems like they are using their own version of EBNF, where a non-terminal is any lowercase word and a terminal is all uppercase or surrounded by quotes. For example, NEWLINE is a terminal, arith_expr is a non-terminal and 'if' is also a terminal. Any non-terminal can be replaced by anything to the right of the colon of it's respective production rule. For example, if you look at the first rule:
单输入:NEWLINE | simple_stmt | compound_stmt NEWLINE
single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
我们可以用NEWLINE，simple_stmt或compound_stmt后跟NEWLINE之一替换single_input.假设我们将其替换为"compound_stmt NEWLINE"，那么我们将查找compound_stmt的生产规则:
We can replace single_input with one of either a NEWLINE, a simple_stmt or a compound_stmt followed by a NEWLINE. Suppose we replaced it with "compound_stmt NEWLINE", then we would look for the production rule for compound_stmt:
compound_stmt:if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef |装饰
compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated
并选择我们要使用的其中一个，并用它代替"compound_stmt"(将NEWLINE保留在该位置)
and choose which of these we want to use, and substitute it for "compound_stmt" (Keeping NEWLINE in it's place)
假设我们要生成有效的python程序:
Suppose we wanted to generate the valid python program:
if 5 < 2 + 3 or not 1 == 5:
raise
我们可以使用以下推导:
We could use the following derivation:
- 单输入
- compound_stmt NEWLINE
- if_stmt NEWLINE
- 如果"测试:"套件NEWLINE
- 'if'or_test':'NEWLINE INDENT stmt stmt DEDENT NEWLINE
- 'if'and_test'or'and_test':'NEWLINE INDENT simple_stmt DEDENT NEWLINE
- 'if'not_test'或'not_test':'NEWLINE INDENT small_stmt DEDENT NEWLINE
- 'if'比较'或''not'not_test':'NEWLINE INDENT flow_stmt DEDENT NEWLINE
- 'if'expr comp_op expr'或''not'比较':'NEWLINE INDENT raise_stmt DEDENT NEWLINE
- 'if'arith_expr'<' arith_expr'或''not'arith_expr comp_op arith_expr':'NEWLINE INDENT'raise'DEDENT NEWLINE
- 如果"一词<"术语'+'术语'或''非'arith_expr == arith_expr':'NEWLINE INDENT'raise'DEDENT NEWLINE
- '如果'NUMBER'<' NUMBER'+'NUMBER'or''not'NUMBER == NUMBER':'NEWLINE INDENT'raise'DEDENT NEWLINE
- single_input
- compound_stmt NEWLINE
- if_stmt NEWLINE
- 'if' test ':' suite NEWLINE
- 'if' or_test ':' NEWLINE INDENT stmt stmt DEDENT NEWLINE
- 'if' and_test 'or' and_test ':' NEWLINE INDENT simple_stmt DEDENT NEWLINE
- 'if' not_test 'or' not_test ':' NEWLINE INDENT small_stmt DEDENT NEWLINE
- 'if' comparison 'or' 'not' not_test ':' NEWLINE INDENT flow_stmt DEDENT NEWLINE
- 'if' expr comp_op expr 'or' 'not' comparison ':' NEWLINE INDENT raise_stmt DEDENT NEWLINE
- 'if' arith_expr '<' arith_expr 'or' 'not' arith_expr comp_op arith_expr ':' NEWLINE INDENT 'raise' DEDENT NEWLINE
- 'if' term '<' term '+' term 'or' 'not' arith_expr == arith_expr ':' NEWLINE INDENT 'raise' DEDENT NEWLINE
- 'if' NUMBER '<' NUMBER '+' NUMBER 'or' 'not' NUMBER == NUMBER ':' NEWLINE INDENT 'raise' DEDENT NEWLINE
在这里有两个注意事项，首先，我们必须从被列为起始非终端的非终端之一开始.在该页面中，他们将它们列出为single_input，file_input或eval_input.其次，一旦所有符号都终止了，派生就完成了(因此得名).第三，更常见的做法是每行进行一次替换，为简洁起见，我立即进行了所有可能的替换，并开始在结尾处跳过步骤.
A couple of notes here, firstly, we must start with one of the non-terminals which is listed as a starting non-terminal. In that page, they list them as single_input, file_input, or eval_input. Secondly, a derivation is finished once all the symbols are terminal (hence the name). Thirdly, it is more common to do one substitution per line, for the sake of brevity I did all possible substitutions at once and started skipping steps near the end.
给定一个语言字符串，我们如何找到它的派生?这是解析器的工作.解析器对生产序列进行逆向工程，以首先检查它是否确实是有效字符串，然后再检查如何从语法中得出该字符串.值得注意的是，许多语法可以描述一种语言.但是，对于给定的字符串，每个语法的推导当然会有所不同.因此，从技术上讲，我们为语法而不是语言编写解析器.一些语法更容易解析，一些语法更易于阅读/理解.这个属于前者.
Given a string in the language, how do we find it's derivation? This is the job of a parser. A parser reverse-engineers a production sequence to first check that it is indeed a valid string, and furthermore how it can be derived from the grammar. It's worth noting that many grammars can describe a single language. However, for a given string, it's derivation will of course be different for each grammar. So technically we write a parser for a grammar not a language. Some grammars are easier to parse, some grammars are easier to read/understand. This one belongs in the former.
这也不指定整个语言，而是它的外观.语法对语义一无所知.
Also this doesn't specify the entire language, just what it looks like. A grammar says nothing about semantics.
如果您对解析和语法有更多的兴趣，我建议 Grune，Jacobs-解析技术 .它是免费的，适合自学.
If you're interested in more about parsing and grammar I recommend Grune, Jacobs - Parsing Techniques. It's free and good for self-study.
这篇关于Python语法在内部如何使用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
