Terminal with tabs

Apr 16, 2012 at 9:40 PM
Edited Apr 16, 2012 at 9:52 PM

Hello,

I want to be able to parse this :

? toto

    - titi

This is the code i have currently :

IdentifierTerminal IDENTIFIER = new IdentifierTerminal("IDENTIFIER");
var TEMPLATE = new FreeTextLiteral("TEMPLATE", FreeTextOptions.AllowEof, "\n", "\r\n");
var PATTERNS = new NonTerminal("PATTERNS");
var PATTERN = new NonTerminal("PATTERN");
PATTERNS.Rule = MakeStarRule(PATTERNS, PATTERN);
PATTERN.Rule = ToTerm("?") + IDENTIFIER + NewLine + ToTerm("    -") | "\t" + TEMPLATE;

 But whitespaces are not analyzed, and tabulations are transformed into 4 whitespaces.
Is there a way to have a keyterm that havbe whitespaces in it?
Thanks

 

EDIT : I'm trying with space but i'd prefer to work with tabs "\t"

Coordinator
Apr 18, 2012 at 6:52 AM

that's really non-standard task for Irony, and alike parser - to mess with structures with no constant delimiters like spaces. You see, each time you write grammar rule like "a + b", parser assumes that there maybe 1 or more delimiters between a and b tokens. 

It might work after all, but you have to probably override Grammar.SkipWhitespace method - and see the effects.

Apr 18, 2012 at 8:15 AM

Now i have overrided the SkipWhiteSpaces and never eat them.

All is going well, but i've got one more problem.

My language can have multiple empty lines, how can i build a rule with empty lines (can be just a newline or a new line full of spaces)

Thanks

Coordinator
Apr 19, 2012 at 5:52 PM

Well, try adding NewLinePlus terminal to your rules

Apr 19, 2012 at 8:56 PM
Edited Apr 19, 2012 at 8:56 PM

Thanks for the reply

It did the tricks using NewLinePlus and this rule :

var _spacePlus = new NonTerminal("empty_line");            
var space = ToTerm(" ");
_spacePlus.Rule = space | _spacePlus + PreferShiftHere() + space;

 The problem is now i've got lots of empty_line statements in the parse tree. How can i define that i don't want this statement in the parse tree

Thanks

Coordinator
Apr 19, 2012 at 8:58 PM

use MarkTransient method