Can't filter MakeStarRule NonTerminal from parse tree if subrule is an "OR" rule

Oct 3, 2011 at 10:08 PM

Say I have a couple of rules for nonterminals in a grammar such as:

    // a statement list consists of any number of statements
    statementList.Rule = MakeStarRule(statementList, null, statement);
    statement.Rule = ifStatement | switchStatement | foreachStatement | withStatement | breakStatement;
If I do not want the statement list node to appear in the parse tree, I set:

Using MarkTransient to filter out unwanted nodes in the parse tree works great, except in cases as above where the NonTerminal has a MakeStarRule AND the and child NonTerminal Rule is an OR of various other NonTerminals.  Is this a bug, and/or is there a workaround for allowing these types of nodes to be filtered?



Oct 3, 2011 at 11:23 PM

As far as I get it, you want StatementList to be there if you have 2 or more statements, but to disappear if there's just one, so that this single statement appears directly under list's parent. That's not how it supposed to work. These Parse tree adjustments using Transient flag are supposed to be "stable", not depending on particular source input. So that if a non-terminal is marked transient, it NEVER appears in the output tree. It does not work in the manner "sometimes yes, sometimes no". That's by design. You should filter out the statementList node at the time of AST construction I think. I do this sometimes in my projects for optimization purposes: adjusting AST tree, not Parse tree.

Oct 3, 2011 at 11:52 PM

Thanks Roman,

My desire is for the ParseTree to NEVER have nodes such as statement lists, but they ALWAYS appear.  I'm not desiring it to be conditional based on source input.

The behavior I'm seeing is if a NonTerminal has a MakeStar rule, and the subrule is an "OR" rule, then that NonTerminal cannot be filtered from the ParseTree by marking as transient.  Other NonTerminals that use MakeStar or have an "OR" rule can be filtered out just fine (the statement above can be filtered out fine, for example).

Is there something else that can cause nodes marked transient to appear in the ParseTree?


Oct 4, 2011 at 12:01 AM

Ok, sorry - misunderstood you.

All I can say is that it's not a bug, I never intended lists to be transient. So the situation you're describing was never "had in mind". Therefore I would advise against using this, even in the cases when it does work. As for why it does not work in one particular case - my guess is that star-list is in fact more than a simple list. Look at MakeSTarRule, the method creates "plus" list first then wraps it into star list after that, the reasons are explained there in comments. So my guess is that this multi-level structure somehow does not work with transient flag, the way you expect it to work. Look at the code and try to step thru with debugger if you're curious.

Anyway, advising not to use this combination (transient lists).


Oct 4, 2011 at 12:04 AM

by the way, there's some diagnostics in parser builder for transient nodes, it verifies that transient node is properly constructed. An error would show up as grammar error (like shift reduce conflicts). Do you see any errors on Grammar Errors page in Gr explorer?

Oct 4, 2011 at 12:14 AM
Edited Oct 4, 2011 at 12:17 AM

OK, I'll keep the extra hierarchy for the lists.

Thanks, Dave

Oct 4, 2011 at 12:47 AM

Yes, looking at Parser.Language.Errors, I get explicit messages to not mark lists as transient :-). "List non-terminals cannot be marked transient; list: (statementList)"

I also see some "Reduce-reduce" conflicts and a "Shift-reduce" conflict I'll have to explore...