如何在野牛中增加堆栈大小(并解决“内存耗尽") [英] How to increase stack size in bison (and solve "memory exhausted")

问题描述

我基于野牛的解析器开始扼杀我最近生成的一些中等大小的文件.

My bison-based parser started choking on some moderately sized files I've generated recently.

它引发了一个有关内存耗尽"的异常.

It throws an exception about "memory exhausted."

野牛手册页说，这很可能是由于使用了右手递归.在不尝试重写语法的情况下(我的期限很紧)，我想简单地增加堆栈以使解析器解析该文件.我试图遵循野牛手册页，并将YYMAXDEPTH定义为比默认值10000大的某个数字，但这没有用.当我查看bison的输出时，似乎仅在这种情况下，当定义YYSTACK_RELOCATE和YYSTACK_RELOCATE时，才有条件地使用YYMAXDEPTH:

The bison man page says that this is likely due to use of right-hand recursion. Without trying to rewrite the grammar (I am on a tight deadline), I would like to simply increase the stack to get the parser to parse this file. I tried to follow the bison man page and #define YYMAXDEPTH to some number larger than the default 10000, but that didn't work. When I look at the output from bison, it seems like YYMAXDEPTH is only conditionally used when YYSTACK_RELOCATE is defined and YYSTACK_RELOCATE is defined only in this case:

#if (! defined yyoverflow \
     && (! defined __cplusplus \
         || (defined YYLTYPE_IS_TRIVIAL && YYLTYPE_IS_TRIVIAL \
             && defined YYSTYPE_IS_TRIVIAL && YYSTYPE_IS_TRIVIAL)))

由于我使用C ++进行编译，因此上述#ifdef失败，因此未定义YYSTACK_RELOCATE.

Since I am compiling with C++, the above #ifdef fails and thus YYSTACK_RELOCATE is not defined.

这是错误还是功能?谁知道增加堆栈大小的正确方法是什么?

Is this a bug or a feature? Anybody knows what's the right way to increase the stack size?

顺便说一句，这是野牛生成的代码片段，其中堆栈溢出:

BTW, this is the snippet of the bison-generated code where the stack overflows:

....
....
if (yyss + yystacksize - 1 <= yyssp)
{
  /* Get the current used size of the three stacks, in elements.  */
  YYSIZE_T yysize = yyssp - yyss + 1;

#ifdef yyoverflow
  {
    /* Give user a chance to reallocate the stack.  Use copies of
       these so that the &'s don't force the real ones into
       memory.  */
    YYSTYPE *yyvs1 = yyvs;
    yytype_int16 *yyss1 = yyss;
    YYLTYPE *yyls1 = yyls;

    /* Each stack pointer address is followed by the size of the
       data in use in that stack, in bytes.  This used to be a
       conditional around just the two extra args, but that might
       be undefined if yyoverflow is a macro.  */
    yyoverflow (YY_("memory exhausted"),
                &yyss1, yysize * sizeof (*yyssp),
                &yyvs1, yysize * sizeof (*yyvsp),
                &yyls1, yysize * sizeof (*yylsp),
                &yystacksize);

    yyls = yyls1;
    yyss = yyss1;
    yyvs = yyvs1;
  }
#else /* no yyoverflow */
# ifndef YYSTACK_RELOCATE
  goto yyexhaustedlab;             // <====== Overflows and throws exception here
# else
  ......
  ......

推荐答案

通常，C ++对象(与C数据类型不同)不能由memcpy进行重定位.因此，bison拒绝重新定位其堆栈，除非它以某种方式知道堆栈对象是琐碎的"，并且如果这些对象是C ++对象，则假定它们不是.

In general, C++ objects -- unlike C datatypes -- cannot be relocated by memcpy. Consequently, bison refuses to relocate its stack unless it somehow knows that the stack objects are "trivial", and if the objects are C++ objects, it assumes that they are not.

YYMAXDEPTH是解析器将分配的最大堆栈，但是初始堆栈大小为YYINITDEPTH.由于C ++堆栈无法重定位，因此初始大小必须足以容纳任何输入，因此您需要增加YYINITDEPTH而不是YYMAXDEPTH.

YYMAXDEPTH is the maximum stack that the parser will allocate, but the initial stack size is YYINITDEPTH. Since C++ stacks cannot be relocated, the initial size must be large enough for any input, so you need to increase YYINITDEPTH, not YYMAXDEPTH.

或者，您可以弄清楚如何告诉bison C ++堆栈对象是可重定位的(如果可以)，或者您可以尝试更新版本的bison:据称，v3更愿意让您自己烧死自己，但是我自己还没有尝试过.最后，您可以定义yyoverflow以提供您自己的堆栈重定位机制:不幸的是，这没有文档说明，并且不必要地复杂.

Alternatively, you could figure out how to tell bison that the C++ stack objects are relocatable (if they are), or you could try a more recent version of bison: allegedly, v3 is more willing to let you burn yourself, but I haven't tried it myself. Finally, you can define yyoverflow to provide your own stack relocation mechanism: unfortunately, this is undocumented and unnecessarily complicated.

这篇关于如何在野牛中增加堆栈大小(并解决“内存耗尽")的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！