Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to emit a token in ANTLR4?

Tags:

java

antlr

antlr4

This is my grammar.g4:

FOO: 'x';
BAR: 'y'
  { /* here! */ };

I want tokens BAR and FOO to be emitted when the input contains y. What should I write in the here! part? Please, don't suggest to rewrite the grammar, this is just a test sample, my real case is much more complex.

like image 851
yegor256 Avatar asked Dec 06 '25 18:12

yegor256


1 Answers

Here's a quick demo:

grammar T;

@lexer::members {
  private java.util.LinkedList<Token> tokens = new java.util.LinkedList<>();

  @Override
  public Token nextToken() {
    return tokens.isEmpty() ? super.nextToken() : tokens.poll();
  }
}

parse
 : foo bar foo EOF
 ;

foo : FOO;
bar : BAR;

FOO : 'x';
BAR : 'y' {this.tokens.offer(new CommonToken(TLexer.FOO, "x"));};

SPACES : [ \t\r\n] -> skip;

Test with:

TLexer lexer = new TLexer(CharStreams.fromString("x y"));
TParser parser = new TParser(new CommonTokenStream(lexer));

System.out.println(parser.parse().toStringTree(parser));

which will print:

(parse (foo x) (bar y) (foo x) <EOF>)
like image 144
Bart Kiers Avatar answered Dec 08 '25 08:12

Bart Kiers



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!