Escaping reserved characters

Jul 12, 2014 at 8:40 PM
Hi guys,

I have written a grammar where I have defined the following escape sequence because I use parentheses to group expressions, but want to handle that string values can contain parentheses also:
value.Escapes.Add(@"))", ")");
But when I read the value of the parsed token, it still contains the two parentheses. The parser hasn't replaced the escape sequence.

Is it supposed to, or should I do that myself? I was expecting it to.

Jul 12, 2014 at 9:18 PM
Token has Text and Value properties - Text is 'as is', and Value should be escaped text. Are you checking the Value?
Jul 13, 2014 at 3:06 PM
I am actually using the ValueString property. I need a string.
Jul 13, 2014 at 3:16 PM
In the debugger I see that Text, Value and ValueString properties are all the same.

This is how value is defined:
var value = new FreeTextLiteral("value", FreeTextOptions.AllowEof, "$", ")");
Jul 14, 2014 at 6:05 PM
FreeTextLiteral?! I'm not sure I'm not even sure I thought about this scenario. And using ")" as end char seems strange. I think you should re-evaluate your options and your language constructs. As for finding why it does not work - just step thru the code, see if escaping is even enabled for free text literal.
Jul 14, 2014 at 6:18 PM
It might very well be that I am doing this wrong.

I am implementing a query api, where a single line of text is parsed to apply a filter.

The string could be:
And using parentheses is supported as well to force affinity. eg.
I have an identifier and a value separated by an operator, and possibly surrounded by parentheses.
var identifier = TerminalFactory.CreateCSharpIdentifier("identifier");
var value = new FreeTextLiteral("value", FreeTextOptions.AllowEof, "$", ")");
The value can be any type, and contain spaces if need be. This all works, but as explained the escaping of the end parentheses somehow doesn't. It will throw an exception if mathing start and end parentheses don't exist, and haven't been properly escaped. But it won't remove escaping from the string.

As I understood it, I would have to tell the freetext literal that when it encountered either a $ or a ) then a new token started.
Jul 16, 2014 at 6:42 AM
you made up quite unusual rules. So you want literals to be of either type, ints or strings, but strings without any quotes. And you want to catch the end of literal by certain 'special' chars. In this case, I would suggest to drop the escaping of 'double-parenth'; for one thing, you might have double parenth symbol as parenth, closing two nested parenth expressions. Secondly, think about readability of the expression by a human (it is humans who write these, right?). This escaping rule might be really confusing and hard to analyse the expression structure - even if your parser is capable of doing this using some formal rules.