Really slow compilation of a big grammar [closed]

Question

Reproducing

Sorry, I couldn't come up with an MRE. My code is on glot.io. It isn't perfect, because it's autogenerated. Here are steps to reproduce:

Step 1

Compile the code: time raku --stagestats ECMA-grammar.rakumod. It takes 7 seconds on my machine. OK...

Stage start      :   0.000
Stage parse      :   6.814
Stage syntaxcheck:   0.000
Stage ast        :   0.000
Stage optimize   :   0.068
Stage mast       :   0.208
Stage mbc        :   0.014
Stage moar       :   0.000

real    0m7.210s
user    0m7.208s
sys     0m0.413s

Step 2

Uncomment token EqualityExpression (lines 193..196) and compile the code. On my machine it now takes 19 seconds to compile, that's already too long.

Step 3

For the experiment, I created an optimized version of this token. If I comment out lines 193..196 and uncomment 197..200 ones, it takes only 11 seconds to compile on my machine. It's interesting, but still much longer than 7 seconds.

Other steps

If you try to uncomment more tokens, the compilation time grows exponentially: 1 minute, 2, 5 and much more. I was never able to compile the whole file.

Questions

1. So, what exactly causes such a long compilation time? In other words, what should I avoid so that compilation takes normal time?
2. Should I create an issue on github?

Answer 1

Sadly, this is a known issue (although I haven't been able to find a specific issue for it, so maybe make one with the code that lives on glot.io).

For comparison though, the compilation of the core.c setting (a 3.3MB file with 99K lines) takes ~47 seconds on an M1. Your grammar appears to be much smaller, so maybe it's tickling some recursion issue.

In any case, this is why we pre-compile stuff in Rakudo. If you put that grammar in a module, and load that module, it should be nearly instantaneous: loading the core c/d setting takes .12 seconds (compiling takes about 48 seconds): that's a 400x speed increase :-)