unable to split into two general sections

Apr 12, 2013 at 5:46 PM
I'm trying to separate my parsing into two general sections separated by "on cancel:". However, "on cancel:" never shows up as a valid terminal. When I include it I get errors saying that such text is invalid. Here is my main grammar creation method.
        public ProcedureGrammar(IEnumerable<TelemetryDefinitionModel> telemetryDefinitions)
        {
            // in BNF:
            // program := row*
            // connector = at, +=, = 
            // row := "set" [actionName] connector actionValue actionUnit? "until" [telmetryName] operator telemetryValue telmetryUnit?

            var program = new NonTerminal("Program");

            var telemetryTriples = new NonTerminal("TelemetryTriples");
            var actionSetTriples = new NonTerminal("ActionSetTriples");
            var actionDecTriples = new NonTerminal("ActionDecTriples");
            var actionIncTriples = new NonTerminal("ActionIncTriples");
            var actionMulTriples = new NonTerminal("ActionMulTriples");

            var actions = telemetryDefinitions.Where(td => !td.IsReadonly).ToList();
            if (actions.Count <= 0)
                throw new ArgumentException("Expected at least one non-readonly telemetry item definition.", "telemetryDefinitions");

            telemetryTriples.Rule = CreateTerminal(telemetryDefinitions, "until");
            actionSetTriples.Rule = CreateTerminal(actions, "set", "at");
            actionDecTriples.Rule = CreateTerminal(actions, "dec", "by");
            actionIncTriples.Rule = CreateTerminal(actions, "inc", "by");
            actionMulTriples.Rule = CreateTerminal(actions, "mul", "by");

            var comment = new CommentTerminal("Comment", "#", "\r", "\n", "\u2085", "\u2028", "\u2029");
            NonGrammarTerminals.Add(comment);

            var startSection = new NonTerminal("StartSection");
            var cancelSection = new NonTerminal("CancelSection");

            var actionTriple = new NonTerminal("ActionTriplet");
            actionTriple.Rule = actionSetTriples | actionIncTriples | actionDecTriples | actionMulTriples;

            var startSectionRow = new NonTerminal("StartSectionRow");
            startSectionRow.Rule = actionTriple;
            startSectionRow.Rule |= actionTriple + telemetryTriples; // combo of the above two
            startSectionRow.Rule |= telemetryTriples;
            startSectionRow.Rule |= ToTerm("run cancel");
            startSection.Rule = MakeStarRule(startSection, NewLinePlus, startSectionRow); // apparently Irony automatically includes EOF as a valid terminal

            var cancelSectionRow = new NonTerminal("CancelSectionRow");
            cancelSectionRow.Rule = actionTriple;
            var cancel = ToTerm("on cancel:");
            cancelSection.Rule = cancel + NewLinePlus + MakeStarRule(cancelSection, NewLinePlus, cancelSectionRow);
            
            program.Rule = startSection + cancelSection.Q();

            MarkPunctuation("[", "]", "until", "at", "by");
            MarkTransient(actionTriple, telemetryTriples, actionSetTriples, actionDecTriples, actionMulTriples, actionIncTriples);
            RegisterBracePair("[", "]");
            Root = program;
        }
Apr 12, 2013 at 9:59 PM
After looking at the GwBasic grammar, I decided to try the approach of adding a NewLine to my two sections and leaving that out of the star rule creation. That has given me much better results. I have this grammar (below) that successfully parses my data now. I would still like the ability to make the "on cancel:" be transient. It seems, though, that if I replace it with a non-terminal, I can't make that non-terminal transient; it always stays. I would like it to work like this: program.Rule = startSection + cancelSection;
        public ProcedureGrammar(IEnumerable<TelemetryDefinitionModel> telemetryDefinitions)
        {
            // in BNF:
            // program := row*
            // connector = at, +=, = 
            // row := "set" [actionName] connector actionValue actionUnit? "until" [telmetryName] operator telemetryValue telmetryUnit?

            var program = new NonTerminal("Program");

            var telemetryTriples = new NonTerminal("TelemetryTriples");
            var actionSetTriples = new NonTerminal("ActionSetTriples");
            var actionDecTriples = new NonTerminal("ActionDecTriples");
            var actionIncTriples = new NonTerminal("ActionIncTriples");
            var actionMulTriples = new NonTerminal("ActionMulTriples");

            var actions = telemetryDefinitions.Where(td => !td.IsReadonly).ToList();
            if (actions.Count <= 0)
                throw new ArgumentException("Expected at least one non-readonly telemetry item definition.", "telemetryDefinitions");

            telemetryTriples.Rule = CreateTerminal(telemetryDefinitions, "until");
            actionSetTriples.Rule = CreateTerminal(actions, "set", "at");
            actionDecTriples.Rule = CreateTerminal(actions, "dec", "by");
            actionIncTriples.Rule = CreateTerminal(actions, "inc", "by");
            actionMulTriples.Rule = CreateTerminal(actions, "mul", "by");

            var comment = new CommentTerminal("Comment", "#", "\r", "\n", "\u2085", "\u2028", "\u2029");
            NonGrammarTerminals.Add(comment);

            var startSection = new NonTerminal("StartSection");
            var cancelSection = new NonTerminal("CancelSection");

            var actionTriple = new NonTerminal("ActionTriplet");
            actionTriple.Rule = actionSetTriples | actionIncTriples | actionDecTriples | actionMulTriples;

            var startSectionRow = new NonTerminal("StartSectionRow");
            startSectionRow.Rule = NewLine;
            startSectionRow.Rule |= actionTriple + NewLine;
            startSectionRow.Rule |= actionTriple + telemetryTriples + NewLine;
            startSectionRow.Rule |= telemetryTriples + NewLine;
            startSectionRow.Rule |= ToTerm("run cancel") + NewLine;
            startSection.Rule = MakeStarRule(startSection, startSectionRow); // apparently Irony automatically includes EOF as a valid terminal

            var cancelSectionRow = new NonTerminal("CancelSectionRow");
            cancelSectionRow.Rule = NewLine;
            cancelSectionRow.Rule = actionTriple + NewLine;
            cancelSection.Rule = MakeStarRule(cancelSection, cancelSectionRow);

            var cancel = ToTerm("on cancel:") + NewLinePlus;
            cancel.Precedence = 1;
            MarkPunctuation(cancel);

            program.Rule = startSection + ((cancel + cancelSection) | Empty);

            MarkPunctuation("[", "]", "until", "at", "by");
            MarkTransient(actionTriple, telemetryTriples, actionSetTriples, actionDecTriples, actionMulTriples, actionIncTriples);
            RegisterBracePair("[", "]");
            LanguageFlags |= LanguageFlags.NewLineBeforeEOF;
            Root = program;
        }
Coordinator
Apr 13, 2013 at 4:14 PM
Transient means not 'remove this from output' but 'Replace it with the single child node it has'. Like BinaryOp terminal might be transient, so binary expression contains the operation token (like "+") directly in middle position, instead of having BinaryOp node with "+" token inside it.