用于在 Python 中进行词法分析、标记化和解析的资源 [英] Resources for lexing, tokenising and parsing in python
问题描述
人们可以向我指出有关使用 Python 进行词法分析、解析和标记的资源吗?
Can people point me to resources on lexing, parsing and tokenising with Python?
我正在对一个开源项目 (hotwire) 进行一些黑客攻击，并且想做对词法, 解析和标记输入的命令.因为它是真正的工作代码，所以相当复杂，而且有点难以计算.
I'm doing a little hacking on an open source project (hotwire) and wanted to do a few changes to the code that lexes, parses and tokenises the commands entered into it. As it is real working code it is fairly complex and a bit hard to work out.
我之前没有研究过 lex/parse/tokenise 的代码，所以我认为一种方法是在这方面完成一两个教程.我希望能学到足够的知识来浏览我真正想要改变的代码.有什么适合的吗?(理想情况下可以在一个下午完成，而不必先购买和阅读龙书......)
I haven't worked on code to lex/parse/tokenise before, so I was thinking one approach would be to work through a tutorial or two on this aspect. I would hope to learn enough to navigate around the code I actually want to alter. Is there anything suitable out there? (Ideally it could be done in an afternoon without having to buy and read the dragon book first ...)
(2008 年 10 月 7 日)以下答案都不是我想要的.有了它们，我可以从头开始生成解析器，但我想学习如何从头开始编写我自己的基本解析器，而不是使用 lex 和 yacc 或类似工具.完成后，我可以更好地理解现有代码.
(7 Oct 2008) None of the below answers quite give what I want. With them I could generate parsers from scratch, but I want to learn how to write my own basic parser from scratch, not using lex and yacc or similar tools. Having done that I can then understand the existing code better.
那么有人可以指点我一个教程，在那里我可以只使用 python 从头开始构建一个基本的解析器吗?
So could someone point me to a tutorial where I can build a basic parser from scratch, using just python?
推荐答案
我很高兴使用 PLY.它是 Lex & 的纯 Python 实现.Yacc，具有许多小细节，使其非常 Pythonic 且易于使用.自 Lex &Yacc 是最流行的 lexing &解析工具，用于大多数项目，PLY具有站在巨人肩膀上的优势.Lex & 网上有很多知识.Yacc，你可以自由地将其应用到 PLY.
I'm a happy user of PLY. It is a pure-Python implementation of Lex & Yacc, with lots of small niceties that make it quite Pythonic and easy to use. Since Lex & Yacc are the most popular lexing & parsing tools and are used for the most projects, PLY has the advantage of standing on giants' shoulders. A lot of knowledge exists online on Lex & Yacc, and you can freely apply it to PLY.
PLY 也有一个很好的文档页面，其中包含一些简单的示例来帮助您入门.
PLY also has a good documentation page with some simple examples to get you started.
有关大量 Python 解析工具的列表，请参阅this.
For a listing of lots of Python parsing tools, see this.
这篇关于用于在 Python 中进行词法分析、标记化和解析的资源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
