Java中的词法分析器 [英] Lexical Analyzer In Java

问题描述

写词法分析阶段



查看更多：Java C＃3.5

为了简单起见，我们将仅从：

•一个变量类型（int）

•基本数学（+， - ，*，/）

•打印命令输出结果

（基本上它只不过是一个简单的计算器）。



必须加载到源文件中（作为命令行参数）并逐行处理，删除空格和注释，并将每个单词或符号分类为标记。令牌表示要解析的原子单元，并且通常被实现为源文件中的一个或一系列短字符，例如int，+或42。然后，您将输出（标准输出）文件的标记化版本，如下所述。

如果我们开发编译器，那么可能会使用Flex和Bison工具。这个项目只需要使用Flex，它将为我们处理词法分析，将一组与每种令牌类型相关的正则表达式作为输入。



这是示例输入文件的内容（为源文件创建所选语言的语法;仅对下面的4行代码）：



example1.java :( 4行代码如下）

隐藏复制代码

write the lexical analysis phase

See more: Java C#3.5
To keep it simple we will start with only:
• one variable type ﴾"int"﴿
• basic math (+, -, *, /)
• Print command to output results
(Basically it will be little more than a simple calculator).

must load in the source file ﴾as a command‐line argument﴿ and process it line-by-line, removing whitespace and comments and categorizing each word or symbol as a token. A token represents an atomic unit to be parsed, and is typically realized as one or a short series of characters in a source file such as "int", "+", or "42". You will then output ﴾to standard out﴿ a tokenized version of the file, as described in detail below.
If we get to develop our compiler then will maybe using the tools Flex and Bison. This project will only require the use of Flex, which will handle lexical analysis for us, taking as input a set of regular expressions associated with each token type.

Here is the contents of an example input file (Create syntax for selected language for source file; only for 4 lines of code below):

example1.java: (4 lines of code below)
Hide Copy Code

val test_num = 3 * (7.2 + 12.1); // This is my comment.
// The next line intentionally left blank!;

System.out.println(test_num);

Example2.java :(下面5行代码，不扫描bool应该处理未知令牌）

隐藏复制代码

Example2.java: (5 lines of code below, not scanning for bool should handle unknown token)
Hide Copy Code

val test_num = 3 * (7.2 + 12.1); // This is my comment.
// The next line intentionally left blank!;

System.out.println(test_num);
bool not_done = 0;

可执行文件应该产生：

隐藏复制代码

executable should produce:
Hide Copy Code

TYPE: val
ID: test_num
ASCII_CHAR: =
VAL_LITERAL: 3
ASCII_CHAR: *
ASCII_CHAR: (
VAL_LITERAL: 7.2
ASCII_CHAR: +
VAL_LITERAL: 12.1
ASCII_CHAR: )
ASCII_CHAR: ;

我知道我必须为ASCII字符和文字构建数组。 。等等。我的问题是如何让它从一个文本文件中逐行读取？

I know that I have to build arrays for the ASCII characters as well as the Literals...etc. My question is how do I get it to read from a text file, line by line?

推荐答案

快速谷歌java文件读取给出所有这些建议 [ ^ ]。

A quick Google for "java file read" gives all these suggestions[^].

这篇关于Java中的词法分析器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！