Pyparsing: Grammar cannot parse whitespace while parsing arithmetic expression with exponential and logarithm functions

Question

Intention:

To parse arithmetic expressions with support for logarithms and exponentials. Any of the following expressions are valid;

x + y
exp(x) + y
exp(x+y)
exp(log(x)+exp(z))+exp(y)
x+log(exp(y))
x + 2

Source Code:

import pyparsing as pp
arith_expr = pp.Forward()
op = pp.oneOf("^ / * % + -")
exp_funcs = pp.Regex(r"(log|exp)(2|10|e)?")
operand = pp.Word(pp.alphas, pp.alphanums + "_") | pp.Regex(r"[+-]?\d+(:?\.\d*)?(:?[eE][+-]?\d+)?")
func_atom = operand ^ (pp.Optional(exp_funcs) + "(" + arith_expr + ")")

comp_expr = pp.Combine(func_atom + pp.ZeroOrMore(op + func_atom))

arith_expr << comp_expr
print arith_expr.parseString("exp(datasize+ 2) +3")

Observation

The grammar is able to parse every such arithmetic expressions but sadly fails to parse when whitespace appears around either operand or operator. The grammar is unable to parse the following expressions;

exp(x+ 2)
exp( x + 2 )
x + 2

I have tried debugging the grammar with setDebug( ) and found the following error on every such failure;

Expected ")"

I reckon Pyparsing is whitespace insensitive after going through it's documentation and my day to day use. I have tried debugging with every possible change that i could. None of them solved the issue. I appreciate your valuable suggestions. :)

Answer 1

The problem is with your use of Combine, which squashes all the tokens together into a single token. Whitespace between tokens is ignored in pyparsing, but whitespace within a token is not.

To fix this, get rid of Combine and then pass the result to ''.join to get it back into one string.