使用 ANTLR 4 在 Python 3 中生成 Java 解析器

Question

从这里使用词法器和解析器：

https://raw.githubusercontent.com/antlr/grammars-v4/master/java/JavaLexer.g4

https://raw.githubusercontent.com/antlr/grammars-v4/master/java/JavaParser.g4

使用 antlr-4.6 生成 Python3 目标

java -jar ./antlr-4.6-complete.jar -Dlanguage=Python3 ./JavaLexer.g4

java -jar ./antlr-4.6-complete.jar -Dlanguage=Python3 ./JavaParser.g4

但是，im无法在生成的解析器上运行编译单元（）方法。它错误地说

ipdb> parser.compilationUnit（）

File "/home/sviyer/onmt-fresh/java/JavaParser.py", line 1063, in compilationUnit
    localctx = JavaParser.CompilationUnitContext(self, self._ctx, self.state)
  File "/home/sviyer/.conda/envs/allennlp/lib/python3.6/site-packages/antlr4/error/ErrorStrategy.py", line 223, in sync
    raise InputMismatchException(recognizer)
antlr4.error.Errors.InputMismatchException: None
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "TestAntlr.py", line 13, in <module>
    parser.compilationUnit()
  File "/home/sviyer/onmt-fresh/java/JavaParser.py", line 1063, in compilationUnit
    localctx = JavaParser.CompilationUnitContext(self, self._ctx, self.state)
  File "/home/sviyer/.conda/envs/allennlp/lib/python3.6/site-packages/antlr4/error/ErrorStrategy.py", line 126, in reportError
    self.reportInputMismatch(recognizer, e)
  File "/home/sviyer/.conda/envs/allennlp/lib/python3.6/site-packages/antlr4/error/ErrorStrategy.py", line 266, in reportInputMismatch
    + " expecting " + e.getExpectedTokens().toString(recognizer.literalNames, recognizer.symbolicNames)
  File "/home/sviyer/.conda/envs/allennlp/lib/python3.6/site-packages/antlr4/error/ErrorStrategy.py", line 522, in getTokenErrorDisplay
    s = t.text
AttributeError: 'int' object has no attribute 'text'

不过，词法分析器工作正常，解析器会解析它。我的代码是：

流 = ANTLR4。输入流（代码）

词法分析器 = JavaLexer（stream）

Toks = antlr4。CommonTokenStream（lexer）

parser = JavaParser（stream）

Answer 1

您的代码不正确。试试这个：

code = open('sample.java', 'r').read()
codeStream = InputStream(code)
lexer = JavaLexer(codeStream)
# First lexing way
tokensStream = CommonTokenStream(lexer)
parser = JavaParser(tokensStream)
# Second lexing way
'''tokens = lexer.getAllTokens()
tokensSource = ListTokenSource(tokens)
tokensStream = CommonTokenStream(tokensSource)
parser = JavaParser(tokensStream)'''
tree = parser.compilationUnit()
print "Tree " + tree.toStringTree(recog=parser);

此外，请使用最新的稳定 ANTLR 版本 （4.7）。