Bash脚本大小限制? [英] Bash script size limitation?

问题描述

我有一个bash脚本，当在RHEL或OS X上运行时，会出现以下错误:

第62484行:意外令牌'newline'附近的语法错误

第62484行:`-o_gz'

这是一个自动生成的脚本，用于解决我公司使用的Grid Engine计算集群引入的限制.它全部由一堆几乎相同的if/elif组成.我看不到错误来自哪一行的特殊信息.当我在错误行之前运行脚本的开头部分时，它可以正常工作.这使我认为可能存在一些bash脚本长度限制.我在网上找不到的唯一参考文献是 iAdjunct 的评论.

围绕错误的脚本部分看起来像这样(有一些简化):

.
.
.
.
elif [ $task_number -eq 2499 ]
then
    /some/tool/executable \
    -use_prephased_g \
    -m \  
    /some/text/file \
    -h \  
    /some/zipped/file \
    -l \  
    -int \
     45063854 \
     46063853 \
    -Ne \ 
     20000 \
    -o \  
    /some/output/file \
    -verbose \
    -o_gz #==============> ****THIS IS LINE 62484****
elif [ $task_number -eq 2500 ]
then
    /some/tool/executable \
    -use_prephased_g \
    -m \  
    /some/other/text/file \
    -h \  
    /some/other/zipped/file \
    -l \  
    -int \
     98232182 \
     99232182 \
    -Ne \ 
     20000 \
    -o \  
    /some/other/output/file \
    -verbose \
    -o_gz
elif [ $task_number -eq 2501 ] 
.
.
.
.

这会给任何人敲响钟声吗?

解决方案

是的，这是bash的限制.

这不是脚本大小的限制；相反，它是对解析器堆栈深度的限制，其作用是限制某些构造的复杂性.特别是，它将把if语句中elif子句的数量限制为大约2500.

在我对一个问题的回答中，针对不同的句法构造(迭代管道)对该问题进行了更长的分析在 Unix& Linux stackexchange 网站.

case语句没有此限制，您提供的示例肯定看起来像case语句的良好匹配.

(与case语句的区别在于if条件语句的语法与管道构造的语法一样，是右递归的，而case语句的语法是左递归的.之所以限制if语句与管道限制的不同之处在于，elif子句的语法构造具有一个以上的符号，因此每个重复使用四个堆栈插槽而不是三个.)

如果case语句对您不起作用-甚至即使它不起作用-您也可以尝试构建if语句的预编译二进制搜索树:

if (( task_number < 8 )); then
  if (( task_number < 4 )); then
    if (( task_number < 2 )); then
      if (( task_number < 1)); then
        # do task 0
      else
        # do task 1
      fi;
    elif (( task_number < 3 )); then
      # do task 2
    else
      # do task 3
    fi
  elif (( task_number < 6 )); then
    if (( task_number < 5 )); then
      # do task 4
    else
      # do task 5
    fi
  elif (( task_number < 7 )); then
    # do task 6
  else
    # do task 7
  fi
elif (( task_number < 12 )); then
  if (( task_number < 10 )); then
    if (( task_number < 9 )); then
      # do task 8
    else
      # do task 9
    fi
  elif (( task_number < 11 )); then
    # do task 10
  else
    # do task 11
  fi
elif (( task_number < 14 )); then
  if (( task_number < 13 )); then
    # do task 12
  else
    # do task 13
  fi
elif (( task_number < 15 )); then
  # do task 14
else
  # do task 15
fi

由于每个完整的if语句在被识别后仅占据单个堆栈节点，因此复杂度限制将取决于if语句的嵌套深度，而不是子句的数量.作为额外的好处，在平均情况下，它将执行的比较少得多.

如果除了顺序条件列表之外别无选择，则可以使用单独的if语句:

while :; do
  if condition1; then
    # do something
  break; fi; if condition2; then
    # do something
  break; fi; if condition3; then
    # do something
  break; fi; if condition4; then
    # do something
  break; fi
  # No alternative succeeded
  break
done

非常规缩进旨在说明简单的程序转换:只需将每个elif替换为break;fi;if，并用while包围整个内容(以提供break的目标.)

I have a bash script that, when run on RHEL or OS X, gives the following error:

line 62484: syntax error near unexpected token `newline'

line 62484: ` -o_gz'

This is an auto-generated script to work around a limitation introduced by the grid engine compute cluster used in my company. It is all composed of a bunch of almost-identical if/elif's. I can't see anything special with the line where the error comes from. When I run the beginning part of the script before the error line, it works without problems. This makes me think that there may be some bash script length limitation. The only reference I could find on the web was the comment by iAdjunct.

The part of the script around the error looks like this (with some simplifications):

.
.
.
.
elif [ $task_number -eq 2499 ]
then
    /some/tool/executable \
    -use_prephased_g \
    -m \  
    /some/text/file \
    -h \  
    /some/zipped/file \
    -l \  
    -int \
     45063854 \
     46063853 \
    -Ne \ 
     20000 \
    -o \  
    /some/output/file \
    -verbose \
    -o_gz #==============> ****THIS IS LINE 62484****
elif [ $task_number -eq 2500 ]
then
    /some/tool/executable \
    -use_prephased_g \
    -m \  
    /some/other/text/file \
    -h \  
    /some/other/zipped/file \
    -l \  
    -int \
     98232182 \
     99232182 \
    -Ne \ 
     20000 \
    -o \  
    /some/other/output/file \
    -verbose \
    -o_gz
elif [ $task_number -eq 2501 ] 
.
.
.
.

Does this ring any bells for anyone?

解决方案

Yes, this is a limitation with bash.

It's not a script size limit; rather it's a limit to the depth of the parser stack, which has the effect of restricting the complexity of certain constructs. In particular, it will restrict the number of elif clauses in an if statement to about 2500.

There is a longer analysis of this problem with respect to a different syntactic construct (iterated pipes) in my answer to a question on the Unix & Linux stackexchange site.

case statements don't have this limitation, and the sample you provide certainly looks like a good match for a case statement.

(The difference with case statements is that the grammar for if conditional statements, like that of pipe constructs, is right recursive, while the grammar for case statements is left recursive. The reason the limitation on if statements is different from the limitation on pipes is that the grammatical construct for an elif clause has one more symbol, so each repetition uses four stack slots rather than three.)

If the case statement doesn't work for you -- or even if it does -- you could try building a precompiled binary search tree of if statements:

if (( task_number < 8 )); then
  if (( task_number < 4 )); then
    if (( task_number < 2 )); then
      if (( task_number < 1)); then
        # do task 0
      else
        # do task 1
      fi;
    elif (( task_number < 3 )); then
      # do task 2
    else
      # do task 3
    fi
  elif (( task_number < 6 )); then
    if (( task_number < 5 )); then
      # do task 4
    else
      # do task 5
    fi
  elif (( task_number < 7 )); then
    # do task 6
  else
    # do task 7
  fi
elif (( task_number < 12 )); then
  if (( task_number < 10 )); then
    if (( task_number < 9 )); then
      # do task 8
    else
      # do task 9
    fi
  elif (( task_number < 11 )); then
    # do task 10
  else
    # do task 11
  fi
elif (( task_number < 14 )); then
  if (( task_number < 13 )); then
    # do task 12
  else
    # do task 13
  fi
elif (( task_number < 15 )); then
  # do task 14
else
  # do task 15
fi

Because each complete if statement only occupies a single stack node after it is recognized, the complexity limitation will be on the nesting depth of the if statements rather than the number of clauses. As an additional bonus, it will execute a lot fewer comparisons in the average case.

If you have no alternative other than a sequential list of conditions, you can use separate if statements:

while :; do
  if condition1; then
    # do something
  break; fi; if condition2; then
    # do something
  break; fi; if condition3; then
    # do something
  break; fi; if condition4; then
    # do something
  break; fi
  # No alternative succeeded
  break
done

The unconventional indent is intended to illustrate the simple program transformation: simply replace every elif with break;fi;if and surround the whole thing with a while (to provide the target for the breaks.)

这篇关于Bash脚本大小限制?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！