CreateTokenFilters and Empty documents

Aug 28, 2012 at 2:54 PM
Edited Aug 28, 2012 at 2:54 PM


I've build a language parse using Irony, based on several samples, including MiniPyhton.

I'm encountering a problem caused (apparently) by CreateTokenFilters override. With this override, Empty statement is not recognize on the base statement Rule.

For exemple, Root = Stmt; Stmt.Rule = something|Empty;

In that case, parser will always says : expecting something.

Is there any workaround to do this ?




public override void CreateTokenFilters(LanguageData language, TokenFilterList filters)
            var outlineFilter = new CodeOutlineFilter(language.GrammarData,
              OutlineOptions.ProduceIndents | OutlineOptions.CheckBraces, ToTerm(@"\")); // "\" is continuation symbol

 PS : Same problem in MiniPython sample

Aug 29, 2012 at 5:51 PM

This is in fact a bug in miniPython grammar. Add the Empty term to Stmt definition:

      Stmt.Rule = AssignmentStmt | Expr | ReturnStmt | Empty;

Note that considering the other definitions:

      ExtStmt.Rule = Stmt + Eos | FunctionDef;
      Block.Rule = Indent + StmtList + Dedent;
      StmtList.Rule = MakePlusRule(StmtList, ExtStmt);
      this.Root = StmtList;       // Set grammar root

.. we define empty line as Empty followed by Eos (end-of-statement) symbol.

Make similar change in your grammar.