This project has moved and is read-only. For the latest updates, please go here.

Non greedy FreeTextLiteral

Dec 21, 2016 at 7:37 AM
I'm new to Irony (and parsing/grammars in general) so please bear with me.

I'm trying to parse strings of the form:
asdf asdf asdf key = fdsa
I'm trying to break this up into "asdf asdf asdf" and "key = fdsa". Basically this is supposed to be composed of 2 types of elements, freetext and then an identifier=freetext

the identifiers are keywords that would match a IdentifierTerminal("identifier", IdOptions.None).

Is there a way to parse this? I got showstopped pretty quickly with a freeText string being greedy and grabbing everything. I guess in an ideal world I could specify a terminator rule of the form (identifier + "=") but I suspect this is not possible. I suspect most likely what I'm trying to do is not possible for reasons/concepts I don't fully understand (ambiguous grammar? lookahead?)

My rather amateurish attempt:
            var identifier = new IdentifierTerminal("identifier", IdOptions.None);
            var freeText = new FreeTextLiteral("freeText", FreeTextOptions.AllowEof);

            NonTerminal args = new NonTerminal("args");
            NonTerminal arg = new NonTerminal("arg");

            arg.Rule = (identifier + "=" + freeText) | freeText;
            args.Rule = MakeStarRule(args, arg);
            this.Root = args;
Dec 25, 2016 at 8:45 AM
I don't think it's possible to break your sample expr into (freetext + (key = fdsa)) - this is just LALR parser limitations
you will have to parse it as:
(wordList + "=" + word)
wordList.Rule = MakeStartRule(workdList, word)

all terms before = are parsed as one combined list
Dec 28, 2016 at 10:32 AM
Edited Dec 28, 2016 at 10:33 AM
Hi rivantsov,

Thanks for the advice. I have gotten a basic system that does what I want, although given the rewrite feel like I am using Irony more as a lexer than an actual parser.

I was wandering through the examples and code and wondering if it would be theoretically possible to use the CustomActionHere directives to control the shift/reduce behaviour to develop a more meaningful parse tree?

And is there any code that explains the CustomParserAction in more detail? I've tried reading the CSharpGrammar ResolveLessThanConflict but I struggle a bit to map it to my (limited) knowledge of shift-reduce parser theory.

From what I understand a shift simply pulls the next unread token into the parse stack. But why does the CustomParserAction have a list of possible ShiftActions (and why do some appear to be duplicated)? From videos I've seen the notion of a 'shift' is a shift has no options - it just depends on where you are in the tree. I guess the ShiftActions are related somehow to jumping between the internal parser states that Irony constructs (and on this note, is there a way to get a more meaningful dump of what the states represent?

A reduce then replaces subtrees in the parse stack with production rules. That makes sense, although I have not been able to effectively use this anywhere - when I try to call a reduce it does not appear to do anything.

My rather amateurish code that might maybe help explain my level of misunderstanding is below. I was hoping to simply duplicate the parse behaviour of uncommenting C0 (and commenting the line below it) and then go from there, but I can't even achieve that.

    public partial class Attempt7 : Grammar
        public Attempt7() : base(false)
            var word = new IdentifierTerminal("word");

//            var freeText = new NonTerminal("freeText");
//            freeText.Rule = freeWord | ;

            var element = new NonTerminal("element");
//            element.Rule = word; // C0 works
            element.Rule = CustomActionHere(ExecuteCustomAction) + word;
//            element.Rule = ExecuteCustomAction(Resolve, Preview) + word; // C2 does not work

            var elementList = new NonTerminal("elementList");
            elementList.Rule = elementList + element | element | Empty;

            this.Root = elementList;

        private void Preview(CustomParserAction action)

        private void ExecuteCustomAction(ParsingContext context, CustomParserAction customAction)
            Console.WriteLine("Resolve fire at " + context.CurrentToken + " " + context.CurrentToken.Location);

            ParserAction action;


            if (customAction.ReduceActions.Count > 0)
                foreach (var ra in customAction.ReduceActions)
                    Console.WriteLine("\t REDUCTION ACTION : {0} {1} {2} {3}", ra, ra.Production, ra.Production.LValue, ra.Production.RValues);

                foreach (var sa in customAction.ShiftActions)
                    Console.WriteLine("\t SHIFT ACTION : {0} {1} {2}", sa, sa.Term, sa.Term.Name);

            if (customAction.ReduceActions.Count > 0)
                action = customAction.ReduceActions.First();

                Console.WriteLine("Execute action: " + action);
//                action = customAction.ShiftActions.First(x => x.Term == context.CurrentToken.Terminal); // C3 implodes with KeyNotFoundException
//                action.Execute(context);