在哪里可以学习编写词法分析器的基础知识? [英] Where can I learn the basics of writing a lexer?
问题描述
我想学习如何编写词法分析器.我的大学课程分配了一个任务，我们必须编写一个解析器(和一个词法分析器一起使用)，但这是在没有指示或反馈的情况下(超出标准)提供给我们的，所以我并没有从中学到太多.
I want to learn how to write a lexer. My university course had an assignment where we had to write a parser (and a lexer to go along with it) but this was given to us with no instruction or feedback (beyond the mark) so I didn't really learn much from it.
在搜索了该主题之后，我只能找到相当高级的文章，这些文章着重于我认为比我所处的位置要走几步的领域.我想讨论一种为非常简单的语言编写词法分析器的基础知识，我可以以此为基础来研究对更复杂的语言进行标记化.
After searching for this topic, I can only find fairly advanced write ups which focus on areas which I feel are a few steps ahead of where I am at. I want a discussion on the basics of writing a lexer for a very simple language which I can use as a basis for investigating tokenising more complex languages.
在这个阶段，我对最佳实践或优化技术并不真正感兴趣，而宁愿关注重点.有什么好的资源可以帮助我入门?
At this stage I'm not really interested in best practices or optimisation techniques but instead prefer a focus on the essentials. What are some good resources to get me started?
推荐答案
基本上有两种编写词法分析器的主要方法:
Basically there are two main approaches to writing a lexer:
- Creating a hand-written one in which case I recommend this small tutorial.
- Using some lexer generator tools such as lex. In this case, I recommend reading the tutorials to the particular tool of choice.
我也想推荐
Also I would like to recommend the Kaleidoscope tutorial from the LLVM documentation. It runs through the implementation of a simple language and in particular demonstrates how to write a small lexer. There is a C++ and an Objective Caml version of the tutorial.
关于该主题的经典教科书是编译器:原理，技术和工具也称为《龙书》.但是，这可能属于相当高级的写信"类别.
The classical textbook on the subject is Compilers: Principles, Techniques, and Tools also known as the Dragon Book. However this probably falls under the category of "fairly advanced write ups".
这篇关于在哪里可以学习编写词法分析器的基础知识?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
