如何使用Gradle 2.10将ANTLR词法分析器语法导入另一个语法？ [英] How can I import an ANTLR lexer grammar into another grammar using Gradle 2.10?

问题描述

我一直使用Terence Parr的最终ANTLR 4参考学习ANTLR 4，我一直在使用Gradle 2.10及其内置的ANTLR插件追踪。然而，我从第4章第38-41页得到的一些代码在使用我的Gradle构建脚本时可能会遇到一些麻烦。 （之所以我使用Gradle而不是直接使用ANTLR，是因为我想最终将ANTLR集成到我为论文写作的Java Web应用程序中，并且我更愿意使用构建工具来实现这一目标我可以自动执行ANTLR到Java的代码生成过程，并轻松管理我的依赖关系。）

我创建了两个ANTLR 4语法文件（标准语法）和 src / main / antlr / org / jbduncan / CommonLexerRules.g4 （一个词法分析器语法），其中通过 import CommonLexerRules 语句依赖于CommonLexerRules.g4 。但是当我尝试在命令行中用 build.gradle 运行 gradlew generateGrammarSource 时（同时粘贴到本问题的末尾） gradle.properties ），我得到以下错误：

  12:45:21：执行外部任务'generateGrammarSource'... 
：generateGrammarSource 
错误（110）：org \jbduncan\Expr.g4：3：7：无法找到或加载语法CommonLexerRules 
 warning（ 125）：org\jbduncan\Expr.g4:12:11：分析器中令牌NEWLINE的隐式定义
 warning（125）：org\jbduncan\Expr.g4:13：6：隐式定义解析器中的标记ID 
警告（125）：org\jbduncan\Expr.g4:19：6：解析器中令牌INT的隐式定义
：generateGrammarSource失败
 
 FAILURE ：构建失败，出现异常。 
 
 *出错：
任务'：generateGrammarSource'的执行失败。 
>在语法生成期间有1个错误
 
 *尝试：
使用--stacktrace选项运行以获取堆栈跟踪。使用--info或--debug选项运行以获取更多日志输出。 
 
构建失败
 
总时间：3.166秒
语法生成过程中出现1个错误
 12:45:24：外部任务执行完成'generateGrammarSource 。 
  

看看这个错误信息，在我看来，Gradle的ANTLR插件能够找到 Expr.g4 但以某种方式无法找到。

我试图用一些外部的Gradle插件来解决这个错误（ https://github.com/melix/antlr4-gradle-plugin 和 https://github.com/Abnaxos/gradle-antlr4-plugin ），而不是内置的，但是当我尝试了他们每个人时，他们介绍了自己的问题，我

使用直接从ANTLR网站下载的ANTLR 4.5.2 jar 允许我编译我的两个文法没有问题，但正如我前面讨论过的，这对我来说是一个非常不理想的选择，因为我不得不手动编译我的语法，而我相信Gradle可以自动为我做。

我的问题是：如何解决上述错误并让Gradle将我的词法分析器语法导入到我的其他语法中？

build.gradle

  apply plugin：'java'
 apply plugin ：'application'
 apply plugin：'antlr'
 
 sourceCompatibility = 1.8 
 targetCompatibility = 1.8 
 
 [compileJava，compileTestJava] *。options *。 encoding ='UTF-8'
 
 group ='org.jbduncan'
 version ='1.0-SNAPSHOT'
 mainClassName ='org.jbduncan.Application'
 
存储库{
 jcenter（）
} 
 
依赖关系{
 antlrorg.antlr：antlr4：$ antlrVersion
 compilecom .google.guava：guava：$ guavaVersion
 testCompilecom.google.guava：guava-testlib：$ guavaVersion
 testCompilecom.google.truth：truth：$ truthVersion
 testCompilejunit：junit：$ junitVersion
} 
 
 //将所有生成的源代码发送到build /以外的目录，以解决
中的问题// IntelliJ IDEA它无法识别build /中的源文件。 
 def generatedJavaSourcesDir ='src / generated / java'
 
 generateGrammarSource {
 arguments + = ['-visitor'] 
 outputDirectory = file（generatedJavaSourcesDir）
} 
 
 sourceSets {
 main {
 java {
 srcDir generatedJavaSourcesDir 
} 
} 
} 
 
 clean {
 delete generatedJavaSourcesDir 
} 
 
任务包装（类型：Wrapper）{
 distributionUrl =http://services.gradle.org/distributions /gradle-$gradleVersion-bin.zip

  

gradle.properties

  gradleVersion = 2.10 
 
＃依赖版本
 antlrVersion = 4.5 
 guavaVersion = 19.0 
 junitVersion = 4.12 
 truthVersion = 0.28 
  

src / main / antlr / org / jbduncan / Expr.g4

 语法Expr; 
 
导入CommonLexerRules; //包含来自CommonLexerRules的所有规则。g4 
 
 @header {
 package org.jbduncan; 
} 
 
 / **起始规则;在这里开始解析。 * / 
 prog：stat +; 
 
 stat：expr NEWLINE＃printExpr 
 | ID'='expr NEWLINE＃assign 
 | NEWLINE＃blank 
; 
 
 expr：expr op =（'*'|'/'）expr＃MulDiv 
 | expr op =（'+'|' - '）expr＃AddSub 
 | INT＃int 
 | ID＃id 
 | '（'expr'）'＃parens 
; 
 
 MUL：'*'; //将令牌名称赋给'*'语法
 DIV：'/'; 
 ADD：'+'; 
 SUB：' - '; 
  

src / main / antlr / org / jbduncan / CommonLexerRules.g4

 词法分析器CommonLexerRules; //注意词法分析器语法
 
 ID：[a-zA-Z] +; //匹配标识符
 INT：[0-9] +; //匹配整数
 NEWLINE：'\r'？ '\\\
'; //将换行符返回给解析器（是结束语句信号）
 WS：[\t]  - >跳过//抛出空白
  

src / main / java / org / jbduncan / Application.java

  package org.jbduncan; 
 
 import org.antlr.v4.runtime.ANTLRInputStream; 
 import org.antlr.v4.runtime.CommonTokenStream; 
 import org.antlr.v4.runtime.tree.ParseTree; 
 
 import java.io.FileInputStream; 
 import java.io.IOException; 
 import java.io.InputStream; 
 
 public final class Application {
 public static void main（String [] args）throws IOException {
 String inputFile =（args.length> 0）？ args [0]：null; 
 
 InputStream inputStream =（inputFile == null）？ System.in：new FileInputStream（inputFile）; 
 ANTLRInputStream input = new ANTLRInputStream（inputStream）; 
 ExprLexer lexer = new ExprLexer（input）; 
 CommonTokenStream tokens = new CommonTokenStream（lexer）; 
 ExprParser parser = new ExprParser（tokens）; 
 ParseTree tree = parser.prog（）; //解析;从prog 
 
开始System.out.println（tree.toStringTree（parser））; //打印树为文本
} 
} 
  

解决方案 div>

在将它发布到Gradle论坛 https://discuss.gradle.org/t/using-gradle-2-10s-antlr -plugin-to-import-an-antlr-4-lexer-grammar-into-another-grammar / 14970 。

I've been learning about ANTLR 4 with Terence Parr's The Definitive ANTLR 4 Reference, which I've been following so far using Gradle 2.10 and its built-in ANTLR plugin. However I'm having some trouble getting some code which I adapted from Chapter 4, pp. 38-41 to work properly with my Gradle build script. (The reason I'm using Gradle, rather than ANTLR directly, is because I want to eventually integrate ANTLR into a Java web application which I'm making for my dissertation, and I'd strongly prefer to use a build tool for this so I can automate the ANTLR-to-Java code generation process and easily manage my dependencies.)

I've created two ANTLR 4 grammars (which I've pasted towards the end of this question): src/main/antlr/org/jbduncan/Expr.g4 (a standard grammar) and src/main/antlr/org/jbduncan/CommonLexerRules.g4 (a lexer grammar), where Expr.g4 depends on CommonLexerRules.g4 via an import CommonLexerRules statement. But when I try to run gradlew generateGrammarSource at the command-line with my build.gradle (also pasted towards the end of this question along with my gradle.properties), I get the following error:

12:45:21: Executing external task 'generateGrammarSource'...
:generateGrammarSource
error(110): org\jbduncan\Expr.g4:3:7: can't find or load grammar CommonLexerRules
warning(125): org\jbduncan\Expr.g4:12:11: implicit definition of token NEWLINE in parser
warning(125): org\jbduncan\Expr.g4:13:6: implicit definition of token ID in parser
warning(125): org\jbduncan\Expr.g4:19:6: implicit definition of token INT in parser
:generateGrammarSource FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':generateGrammarSource'.
> There was 1 error during grammar generation

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 3.166 secs
There was 1 error during grammar generation
12:45:24: External task execution finished 'generateGrammarSource'.

Looking at this error message, it seems to me that Gradle's ANTLR plugin is able to find Expr.g4 but is somehow not able to find CommonLexerRules.g4.

I've made an attempt to resolve this error using a couple of external Gradle plugins (https://github.com/melix/antlr4-gradle-plugin and https://github.com/Abnaxos/gradle-antlr4-plugin) instead of the built-in one, but when I tried each of them, they introduced their own problems which I wasn't able to resolve.

Using the ANTLR 4.5.2 jar downloaded straight from the ANTLR website does let me to compile my two grammars without problems, but as I discussed earlier this is a really undesirable option for me, as I'd have to compile my grammars manually whereas I believe Gradle could do it automatically for me.

My question is this: How can I resolve the error above and get Gradle to import my lexer grammar into my other grammar?

---

build.gradle

apply plugin: 'java'
apply plugin: 'application'
apply plugin: 'antlr'

sourceCompatibility = 1.8
targetCompatibility = 1.8

[compileJava, compileTestJava]*.options*.encoding = 'UTF-8'

group = 'org.jbduncan'
version = '1.0-SNAPSHOT'
mainClassName = 'org.jbduncan.Application'

repositories {
    jcenter()
}

dependencies {
    antlr "org.antlr:antlr4:$antlrVersion"
    compile "com.google.guava:guava:$guavaVersion"
    testCompile "com.google.guava:guava-testlib:$guavaVersion"
    testCompile "com.google.truth:truth:$truthVersion"
    testCompile "junit:junit:$junitVersion"
}

// Send all generated source code to a directory other than build/, to workaround an issue in
// IntelliJ IDEA where it fails to recognise source files in build/.
def generatedJavaSourcesDir = 'src/generated/java'

generateGrammarSource {
    arguments += ['-visitor']
    outputDirectory = file(generatedJavaSourcesDir)
}

sourceSets {
    main {
        java {
            srcDir generatedJavaSourcesDir
        }
    }
}

clean {
    delete generatedJavaSourcesDir
}

task wrapper(type: Wrapper) {
    distributionUrl = "http://services.gradle.org/distributions/gradle-$gradleVersion-bin.zip"
}

gradle.properties

gradleVersion=2.10

# Dependency versions
antlrVersion=4.5
guavaVersion=19.0
junitVersion=4.12
truthVersion=0.28

src/main/antlr/org/jbduncan/Expr.g4

grammar Expr;

import CommonLexerRules; // includes all rules from CommonLexerRules.g4

@header {
    package org.jbduncan;
}

/** The start rule; begin parsing here. */
prog: stat+;

stat: expr NEWLINE # printExpr
    | ID '=' expr NEWLINE # assign
    | NEWLINE # blank
    ;

expr: expr op=('*'|'/') expr # MulDiv
    | expr op=('+'|'-') expr # AddSub
    | INT # int
    | ID # id
    | '(' expr ')' # parens
    ;

MUL: '*'; // assigns token name to '*' used above in grammar
DIV: '/';
ADD: '+';
SUB: '-';

src/main/antlr/org/jbduncan/CommonLexerRules.g4

lexer grammar CommonLexerRules; // note "lexer grammar"

ID: [a-zA-Z]+ ; // match identifiers
INT: [0-9]+ ; // match integers
NEWLINE: '\r'? '\n' ; // return newlines to parser (is end-statement signal)
WS: [ \t] -> skip ; // toss out whitespace

src/main/java/org/jbduncan/Application.java

package org.jbduncan;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;

public final class Application {
  public static void main(String[] args) throws IOException {
    String inputFile = (args.length > 0) ? args[0] : null;

    InputStream inputStream = (inputFile == null) ? System.in : new FileInputStream(inputFile);
    ANTLRInputStream input = new ANTLRInputStream(inputStream);
    ExprLexer lexer = new ExprLexer(input);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    ExprParser parser = new ExprParser(tokens);
    ParseTree tree = parser.prog(); // parse; start at prog

    System.out.println(tree.toStringTree(parser)); // print tree as text
  }
}

解决方案

I managed to find an answer to my question after posting it on the Gradle forums at https://discuss.gradle.org/t/using-gradle-2-10s-antlr-plugin-to-import-an-antlr-4-lexer-grammar-into-another-grammar/14970.

这篇关于如何使用Gradle 2.10将ANTLR词法分析器语法导入另一个语法？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！