如何在Flex规则中引用lex或解析参数? [英] How to reference lex or parse parameters in flex rules?

问题描述

我知道我可以在.y文件中声明%parse-param {struct my_st *arg}.因此yyparse()更改为yyparse(struct my_st *arg).但是我该如何在flex规则中引用该参数呢?例如:

I know I can declare %parse-param {struct my_st *arg} in a .y file. And so yyparse() is changed to be yyparse(struct my_st *arg). But how do I reference the argument in the flex rules? For example:

[0-9]+  { do_work(arg); return NUMBER; }

我想创建一个可重入的解析器，所以我需要这样做.请帮助我，谢谢！

I want to make a reentrant parser, so I need to do this. Please help me, thanks!

推荐答案

您需要将参数传递给yylex.这就需要修改bison解析器描述，以便解析器使用所需的参数调用yylex以及flex扫描器描述，以便扫描器生成具有所需参数的yylex.

You need to pass the argument through to yylex. That requires a modification of both the bison parser description, so that the parser calls yylex with the desired arguments, and the flex scanner description, so that the scanner generates yylex with the desired parameters.

Bison和flex不会相互通信，也不会看到彼此的源文件.但是，对于扫描仪来说，由bison生成的头文件是正常的，并且bison允许将代码直接插入该头文件中的可能性.这样就可以将整个配置放入野牛文件中.

Bison and flex do not communicate with each other and do not see each other's source files. However, it is normal for the scanner to #include the header file generated by bison, and bison allows the possibility of inserting code directly into this header file. That makes it possible to put the entire configuration into the bison file.

在野牛中，您可以使用yylex指定其他参数> %lex-param指令.但是您需要注意如果您

In bison, you specify additional parameters for yylex using the %lex-param directive. But you need to be aware of the additional argument(s) which would also be added to the call if you %define api.pure.

如果您使用的是野牛3.0或更高版本，则可以使用

If you use bison 3.0 or more recent, you can use

%param { struct my_st *arg }

缩写为

%lex-param { struct my_st *arg }
%parse-param { struct my_st *arg }

使用单个指令(如果您的野牛是最近的)是有意义的，因为无法将局部变量声明插入到yyparse函数中.因此，只能传递给yylex的变量是全局变量和yyparse的参数. [注1]

It makes sense to use a single directive (if your bison is sufficiently recent) because there is no way to insert a local variable declaration into the yyparse function. So the only variables which could be passed through to yylex are global variables and parameters to yyparse. [Note 1]

请记住，在野牛文件中声明yylex和yyerror是您的责任.即使您使用%lex-param，野牛也不会自动为yylex生成声明. [注2]

Remember that it is your responsibility to declare yylex and yyerror in the bison file. Even if you use %lex-param, bison will not automatically generate a declaration for yylex. [Note 2]

Flex通常会为yylex生成一个声明，因此您不能简单地将声明放入bison生成的头文件中，然后将#include放入扫描仪中.但是，如果定义了 YY_DECL宏，则flex -c生成的扫描程序将不会向前声明yylex，它将在yylex的定义中使用YY_DECL宏.您可以使用此功能将yylex声明放入野牛描述中，从而将其传递给Flex扫描仪.

Flex does normally generate a declaration for yylex, so you cannot simply put your declaration into the bison-generated header file and then #include it into the scanner. However, if the YY_DECL macro is defined, then the flex-generated scanner will not forward-declare yylex, and it will use the YY_DECL macro in the definition of yylex. You can use this feature to put the declaration of yylex into the bison description in a way that it will be passed through to the flex scanner.

请注意，这与%bison-bridge的使用不兼容，%bison-bridge的使用通常用于修改yylex的原型以与可重入解析器一起使用. %bison-bridge当然有用，但是不允许向yylex调用添加更多参数.如果要使用%bison-bridge，则唯一的选择是将其他参数添加到其他数据"； yyscan_t对象中的对象.

Note that this is not compatible with the use of %bison-bridge, which is often used to modify the prototype of yylex to work with a reentrant parser. %bison-bridge is certainly useful, but it does not allow adding more parameters to the yylex call. If you want to use %bison-bridge, your only option is to add the additional parameter(s) to the "extra data" object in the yyscan_t object.

在野牛中，您可以向已生成的标头中添加声明使用%code requires部分或%code provides部分.区别在于requires段在头文件中早于声明YYSTYPE和YYLTYPE的位置.如果使用纯解析器，则yylex原型通常将引用YYSTYPE(如果使用位置，则引用YYLTYPE)，因此它需要放在%code provides部分中.为了与flex进行优雅的接口，可以使用YY_DECL宏来生成yylex声明.

In bison, you can add declarations to the generated header using either %code requires sections or %code provides sections. The difference is that requires segments are earlier in the header file, before YYSTYPE and YYLTYPE have been declared. If you use a pure parser, the yylex prototype will usually refer to YYSTYPE (and YYLTYPE, if you use locations), so it needs to go in a %code provides section. In order to interface gracefully with flex, you can use the YY_DECL macro to generate the yylex declaration.

因此，您可能会得到如下所示的内容:[注3]

So you might end up with something like the following: [Note 3]

文件:mylanguage.y

%code requires {
  #include <stdio.h>

  typedef struct Context { ... } Context;

  /* structs used in the %union declaration would go here */
}

%define api.pure full
%locations
%parse-param { Context* context }
%lex-param { Context* context }

%code provides {
   #define YY_DECL \
       int yylex(YYSTYPE* yylvalp, YYLTYPE* yyllocp, Context* context)
   YY_DECL;

   int yyerror(YYLTYPE* yyllocp, Context* context, const char* message);
}

然后，您只需按照常规方式将生成的标头插入到flex文件中即可:

Then you would just insert the generated header into your flex file in the normal way:

文件:mylanguage.l

%{
   /* System library includes */
   #include "mylanguage.tab.h"
%}

bison中的%define api.pure full声明避免了对全局变量yylval和yylloc的需要.但是，flex生成的扫描器还使用了许多其他内部全局变量.为了使扫描仪真正可重入，您需要在弹性文件中添加%option reentrant.使用该选项时，预计yylex将包含参数yyscan_t yyscanner(以及flex定义的所有其他与词法分析器相关的功能).您需要管理yyscanner值，因此需要如上所述将其通过yyparse传递给yylex.您还需要初始化和销毁​​它，如 flex手册. (如果要使用YY_DECL生成yylex原型，则它在yylex中的名称必须恰好是yyscanner.)

The %define api.pure full declaration in bison avoids the need for the global variables yylval and yylloc. However, there are a number of other internal globals used by a flex-generated scanner; in order to make the scanner truly re-entrant, you need to add %option reentrant to your flex file. With that option, yylex is expected to include the parameter yyscan_t yyscanner (as are all the other lexer-related functions defined by flex). You need to manage the yyscanner value, so it will need to be passed through yyparse to yylex as above. You also need to initialize and destroy it, as described in the flex manual. (Its name in yylex must be precisely yyscanner, if you are generating the yylex prototype using YY_DECL.)

如果还希望传递自己的上下文对象，则可以在yyparse和yylex中添加两个参数，或者可以将上下文对象包含在yyscan_t对象中，如Flex手册有关额外数据的部分.

If you also want to pass your own context object, you could add two parameters to yyparse and yylex, or you could include your context object inside the yyscan_t object as described in the Flex manual section on Extra Data.

最后，如果您使用bison纯解析器API，则需要更改编写flex操作的方式.而不是分配给全局yylval(例如yylval.integer = atol(yytext);)，您需要通过作为参数传递的指针进行分配:yylvalp->integer = atol(yytext);. (参数的名称取决于您；我在这里使用的是我上面指定的参数.)

Finally, if you use the bison pure parser API, then you need to change the way you write flex actions. Instead of assigning to the global yylval (eg. yylval.integer = atol(yytext);), you need to assign through the pointer passed as an argument: yylvalp->integer = atol(yytext);. (The name of the argument is up to you; the one I use here is the one I specified above.)

1. 较旧的实现允许您通过定义宏YYLEX_PARAM来指定yylex的参数.从bison 3.0开始不再支持此功能，因此您不应该使用它.

1. Older implementations allowed you to specify the arguments to yylex by defining the macro YYLEX_PARAM. This is no longer supported as of bison 3.0, so you shouldn't use it.

如果使用%parse-param，则附加参数也将添加到yyerror.如果您%define api-pure full，yyerror也将收到一个定位对象.您的yyerror声明必须保持一致.

If you use %parse-param, the additional parameter will also be added to yyerror. And if you %define api-pure full, yyerror will also receive a location object. Your declaration of yyerror needs to be consistent.

%locations指令强制野牛生成存储每个令牌的位置信息的代码.我在这里使用它是因为它使原型可预测.如果没有它，则仅当您实际引用语义动作中某个位置的位置时，原型才会包含YYLTYPE参数.如果您不打算使用令牌位置，则可能希望删除%locations指令和所有YYLTYPE参数.但是通常位置信息很有用.

The %locations directive forces bison to generate code which stores location information for each token. I use it here because it makes the prototypes predictable. Without it, the prototypes would include YYLTYPE arguments only if you actually refer to a location somewhere in a semantic action. If you don't intend to use token locations, you might prefer to remove the %locations directive and all the YYLTYPE arguments. But usually the location information is useful.

这篇关于如何在Flex规则中引用lex或解析参数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！