令牌有什么作用，为什么需要在C ++编程中创建令牌？ [英] What does Tokens do and why they need to be created in C++ programming?

问题描述

我正在读一本书（Bjarne Stroustrup的编程原理和实践）。

他在其中介绍了代币：

令牌是一系列字符，代表我们认为是单位的事物，例如数字或运算符。这就是C ++编译器处理其源代码的方式。实际上，大多数形式的文本分析都是通过某种形式的令牌化。

 类令牌{
公众：
 char类型； 
双重价值； 
}; 
  

我确实知道了它们的意思，但是他从没有详细解释，这让我很困惑。

解决方案

正如其他人所提到的，Bjrane指的是词法分析。

一般而言，标记||创建令牌是一个处理输入流并将其划分为块的过程，而无需担心空格等，@ StoryTeller之前对此进行了最详尽的描述。
或正如bjrane所说：是代表我们认为单位的字符序列。

令牌本身是C ++用户的一个示例，定义的类型'UDT'，例如int或char，因此可以使用token定义变量并保存值。

UDT既可以具有成员函数，也可以具有数据成员。在您的代码中定义了两个非常基本的成员函数。

1）种类2）值

 类令牌{
 public：
 char类型； 
双重价值； 
}; 
  

基于它，我们可以初始化或构造其对象。

 令牌token_kind_one {'+'}; 
  

使用其种类（运算符） +初始化token_kind_one。

 令牌token_kind_two {'8'，3.14}; 
  

和token_kind_two，其种类（整数/数字）为'8'，且值为3.14。 / p>

我们假设我们有一个十个字符的表达式1 + 2 * 3（5/4），可以转换为十个令牌。

代币：

  | ------------------ ---- | --------------------- | 
类型|’8’|’+’|’8’|’*’|’8’|’（’|’8’|’/’|‘8’|’）’|| 
 | ------------------------ | --------------------- | 
价值| 1 | | 2 | | 3 | | 5 | | 4 | | 
 | ------------------------ | --------------------- | 
  

C ++编译器将文件数据传输到令牌序列，从而跳过所有空格。为了使其易于理解。

I am reading a book (Programming Principles and Practice by Bjarne Stroustrup).

In which he introduce Tokens:

"A token is a sequence of characters that represents something we consider a unit, such as a number or an operator. That’s the way a C++ compiler deals with its source. Actually, "tokenizing" in some form or another is the way most analysis of text starts."

class Token {
public:
    char kind;
    double value;
};

I do get what they are but he never explains this in detail and its quite confusing to me.

解决方案

As mentioned by others Bjrane is referring to Lexical analysis.

In general terms tokenizing || creating tokens, is a process of processing input streams and dividing them into blocks, without worrying about whitespaces etc. best described earlier by @StoryTeller. "or as bjrane said: is a sequence of characters that represent something we consider a unit".

The token itself is an example of a C++ user-defined type'UDT' like int or char, so token can be used to define variables and hold values.

UDT can have member functions as well as data members. In your code you define two member functions which is very basic.

1)Kind, 2)Value

class Token {
public:
   char kind;
   double value;
};

Based on it we can initialize or construct its objects.

Token token_kind_one{'+'};

Initializing token_kind_one with its kind(operator) '+'.

Token token_kind_two{'8',3.14};

and token_kind_two with its kind(integer/number) '8' and with a value of 3.14.

Lets assume we have an expression of ten characters 1+2*3(5/4), which translates to ten tokens.

Tokens:

        |----------------------|---------------------|
Kind    |'8' |'+' |'8' |'*'|'8'|'('|'8' |'/'|'8' |')'|
        |----------------------|---------------------|
Value   | 1  |    | 2  |   | 3 |   | 5  |   | 4  |   |
        |----------------------|---------------------|

C++ compiler transfer file data to a token sequence skipping all whitespaces. To make it understandable to itself.

这篇关于令牌有什么作用，为什么需要在C ++编程中创建令牌？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！