irony Forum Rss Feed

New Post: System.NullReferenceException in BuildAst

jarin — Sat, 18 May 2013 17:52:28 GMT

Hi Roman, thanks for your reply.

I've started by stripping my grammar to bare minimum (only numbers and operators) but had the same issue. During debugging I found that DefaultLiteralNodeType was null and found this discussion that explained what was going on. Overriding BuildAst() did the trick and I got the basic grammar working. I'll start adding more stuff to my grammar and see what happens. At least I've got slightly better understanding of how it all works now.

Thanks!

New Post: System.NullReferenceException in BuildAst

rivantsov — Sat, 18 May 2013 04:11:13 GMT

at first look, you are missing AST node type for colname terminal. Stop in debugger on exception and look around on data involved (which non-terminal/terminal is there). But this is my guess - colname. You have to provide AstNode implementation - a class that knows how to interpret the colname at runtime.
Let me know if you're stuck, I may be able to investigate in details later
Roman

New Post: System.NullReferenceException in BuildAst

jarin — Fri, 17 May 2013 13:24:11 GMT

I've got very simple grammar adapted from "Writing a calculator in C# using Irony" sample (see below) and parsing works fine, however when I add

LanguageFlags = LanguageFlags.CreateAst;

to generate AST I get following error in Parser Output tab:

AstNodeType or AstNodeCreator is not set on non-terminals: Irony.Parsing.BnfTermList. Either set Term.AstConfig.NodeType, or provide default values in AstContext.

and an exception in Irony Grammar Explorer:

System.NullReferenceException: Object reference not set to an instance of an object.
   at Irony.Ast.AstBuilder.BuildAst(ParseTreeNode parseNode) in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony\Ast\AstBuilder.cs:line 97
   at Irony.Ast.AstBuilder.BuildAst(ParseTreeNode parseNode) in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony\Ast\AstBuilder.cs:line 86
   at Irony.Ast.AstBuilder.BuildAst(ParseTree parseTree) in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony\Ast\AstBuilder.cs:line 38
   at Irony.Parsing.Grammar.BuildAst(LanguageData language, ParseTree parseTree) in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony\Parsing\Grammar\Grammar.cs:line 499
   at Irony.Parsing.Parser.Parse(String sourceText, String fileName) in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony\Parsing\Parser\Parser.cs:line 88
   at Irony.GrammarExplorer.fmGrammarExplorer.ParseSample() in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony.GrammarExplorer\fmGrammarExplorer.cs:line 359
   at Irony.GrammarExplorer.fmGrammarExplorer.btnParse_Click(Object sender, EventArgs e) in C:\Programming\Irony_2013_03_10\Irony_2013_03_10\Irony.GrammarExplorer\fmGrammarExplorer.cs:line 507
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

My grammar is:

public class TestGrammar1 : Grammar
    {
        public TestGrammar1 () : base(false)
        {
            var number = new NumberLiteral("number", NumberOptions.AllowSign | NumberOptions.AllowStartEndDot);
            var fnname = new IdentifierTerminal("fnname");
            
            var colname = new StringLiteral("colname");
            colname.AddStartEnd("[", "]", StringOptions.None);

            var expression = new NonTerminal("expression");
            var binexpr = new NonTerminal("binexpr", typeof(BinaryOperationNode));
            var parexpr = new NonTerminal("parexpr");
            var fncall = new NonTerminal("fncall", typeof(FunctionCallNode));
            var binop = new NonTerminal("binop", "operator");

            expression.Rule = parexpr | binexpr | number | colname | fncall;
            parexpr.Rule = "(" + expression + ")";
            binexpr.Rule = expression + binop + expression;
            binop.Rule = ToTerm("+") | "-" | "/" | "*";
            fncall.Rule = fnname + "(" + expression + ")";
            this.Root = expression;

            MarkPunctuation("(",")");

            RegisterOperators(1, "+", "-");
            RegisterOperators(2, "*", "/");
            AddOperatorReportGroup("operator");
            
            MarkTransient(parexpr, expression, binop);
            LanguageFlags = LanguageFlags.CreateAst;
        }
    }

My test expression:

sum([a])

Can somebody please help me figure out what's wrong with my grammar?
Thanks!

New Post: Fatal error in code colorizer.Colorizing has been disabled

rivantsov — Tue, 14 May 2013 21:24:21 GMT

hi
sorry, do not have much time right now, it would be a few days
Roman

New Post: Fatal error in code colorizer.Colorizing has been disabled

nadvi — Sun, 12 May 2013 15:10:37 GMT

As-Salam-wa-alicum

Sir,

Thank you very much for your effort but the way you advice need so many work at comiletime to determine the actual thing because of its generality
and it is possible to make it a little more specific.I did it but when I try to refactor emofAttributeDiclaration,functionSignatureBody and ArrayType so
many conflict come.Please help me here.My grammar file and the test file is here By the way do you have the original grammar
file on which you do work to fix conflict?

Thank you very much again for reading my mail by spending your valuable time and working on it.

With regards
Nadvi

New Post: "Reduce-reduce" errors eat my lunch; we really need some context on them

brantheman — Fri, 26 Apr 2013 21:45:58 GMT

Thanks for the clue! I've got it all figured out. Here's a little documentation for anyone else dealing with Reduce-Reduce errors:

You can only have one of each RValue (right-hand side of the arror value) in the "Reduce Items" section. If you have more than one it can't decide which non-terminal to use in a given scenario. (It likely doesn't matter which one gets used; I guess Irony doesn't want to take any chances.)

This won't work as all child trees support an Empty:

foreach(var group in groups)
{
   ....
   var myRule = ....;
   myRule.Rule = MakeStarRule(myRule, eachRowOfMyRule);

   if(program.Rule == null)
       program.Rule = myRule;
   else program.Rule |= myRule;
}

You have to do it like this instead:

foreach(var group in groups)
{
   ....
   var myRule = ....;
   if(program.Rule == null)
      myRule.Rule = MakeStarRule(myRule, eachRowOfMyRule);
   else
      myRule.Rule = MakePlusRule(myRule, eachRowOfMyRule);

   if(program.Rule == null)
       program.Rule = myRule;
   else program.Rule |= myRule;
}

New Post: "Reduce-reduce" errors eat my lunch; we really need some context on them

rivantsov — Fri, 26 Apr 2013 20:55:00 GMT

my guess is that you have several optional sections, optionally (!) included, and parser does not know how to interpret an empty file. To say more, I need to see the grammar

New Post: "Reduce-reduce" errors eat my lunch; we really need some context on them

brantheman — Fri, 26 Apr 2013 20:33:40 GMT

Here is the output from Grammar Explorer. I assume that I look in the "Reduce Items" section, but I don't know what to look for there.

State S0 (Inadequate)
  Reduce-reduce conflicts on inputs: EOF LF
  Shift items:
    Program' -> ·Program EOF 
    Program -> ·StartSection.grp1 
    StartSection.grp1 -> ·StartSectionRow.grp1+ 
    StartSectionRow.grp1+ -> ·StartSectionRow.grp1+ StartSectionRow.grp1 
    StartSectionRow.grp1+ -> ·StartSectionRow.grp1 
    StartSectionRow.grp1 -> ·OptionalTelemetryTriples.grp1 LF 
    OptionalTelemetryTriples.grp1 -> ·TelemetryTriples.grp1 
    TelemetryTriples.grp1 -> ·Telemetry 
    Telemetry -> ·until Unnamed0 Telemetry.Operator Telemetry.Value OptionalMetric 
    TelemetryTriples.grp1 -> ·Telemetry 
    Telemetry -> ·until Unnamed1 Telemetry.Operator Telemetry.Value OptionalMetric 
    StartSectionRow.grp1 -> ·ActionTriplet.grp1 OptionalTelemetryTriples.grp1 LF 
    ActionTriplet.grp1 -> ·ActionSetTriples.grp1 
    ActionSetTriples.grp1 -> ·Action 
    Action -> ·set Unnamed2 at Action.Value OptionalMetric 
    ActionSetTriples.grp1 -> ·Action 
    Action -> ·set Unnamed3 at Action.Value OptionalMetric 
    ActionTriplet.grp1 -> ·ActionIncTriples.grp1 
    ActionIncTriples.grp1 -> ·Action 
    Action -> ·inc Unnamed4 by Action.Value OptionalMetric 
    ActionIncTriples.grp1 -> ·Action 
    Action -> ·inc Unnamed5 by Action.Value OptionalMetric 
    ActionTriplet.grp1 -> ·ActionDecTriples.grp1 
    ActionDecTriples.grp1 -> ·Action 
    Action -> ·dec Unnamed6 by Action.Value OptionalMetric 
    ActionDecTriples.grp1 -> ·Action 
    Action -> ·dec Unnamed7 by Action.Value OptionalMetric 
    ActionTriplet.grp1 -> ·ActionMulTriples.grp1 
    ActionMulTriples.grp1 -> ·Action 
    Action -> ·mul Unnamed8 by Action.Value OptionalMetric 
    ActionMulTriples.grp1 -> ·Action 
    Action -> ·mul Unnamed9 by Action.Value OptionalMetric 
    Program -> ·StartSection.grp2 
    StartSection.grp2 -> ·StartSectionRow.grp2+ 
    StartSectionRow.grp2+ -> ·StartSectionRow.grp2+ StartSectionRow.grp2 
    StartSectionRow.grp2+ -> ·StartSectionRow.grp2 
    StartSectionRow.grp2 -> ·OptionalTelemetryTriples.grp2 LF 
    OptionalTelemetryTriples.grp2 -> ·TelemetryTriples.grp2 
    TelemetryTriples.grp2 -> ·Telemetry 
    Telemetry -> ·until Unnamed10 Telemetry.Operator Tail OptionalMetric 
    TelemetryTriples.grp2 -> ·Telemetry 
    Telemetry -> ·until Unnamed11 Telemetry.Operator Telemetry.Value OptionalMetric 
    StartSectionRow.grp2 -> ·ActionTriplet.grp2 OptionalTelemetryTriples.grp2 LF 
    ActionTriplet.grp2 -> ·ActionSetTriples.grp2 
    ActionSetTriples.grp2 -> ·Action 
    Action -> ·set Unnamed12 at Action.Value OptionalMetric 
    ActionTriplet.grp2 -> ·ActionIncTriples.grp2 
    ActionIncTriples.grp2 -> ·Action 
    Action -> ·inc Unnamed13 by Action.Value OptionalMetric 
    ActionTriplet.grp2 -> ·ActionDecTriples.grp2 
    ActionDecTriples.grp2 -> ·Action 
    Action -> ·dec Unnamed14 by Action.Value OptionalMetric 
    ActionTriplet.grp2 -> ·ActionMulTriples.grp2 
    ActionMulTriples.grp2 -> ·Action 
    Action -> ·mul Unnamed15 by Action.Value OptionalMetric 
  Reduce items:
    StartSection.grp1 -> · [EOF]
    OptionalTelemetryTriples.grp1 -> · [LF]
    StartSection.grp2 -> · [EOF]
    OptionalTelemetryTriples.grp2 -> · [LF]
  Transitions: Program->S1, StartSection.grp1->S2, StartSectionRow.grp1+->S3, StartSectionRow.grp1->S4, OptionalTelemetryTriples.grp1->S5, TelemetryTriples.grp1->S6, Telemetry->S7, until->S8, Telemetry->S9, ActionTriplet.grp1->S10, ActionSetTriples.grp1->S11, Action->S12, set->S13, Action->S14, ActionIncTriples.grp1->S15, Action->S16, inc->S17, Action->S18, ActionDecTriples.grp1->S19, Action->S20, dec->S21, Action->S22, ActionMulTriples.grp1->S23, Action->S24, mul->S25, Action->S26, StartSection.grp2->S27, StartSectionRow.grp2+->S28, StartSectionRow.grp2->S29, OptionalTelemetryTriples.grp2->S30, TelemetryTriples.grp2->S31, Telemetry->S32, Telemetry->S33, ActionTriplet.grp2->S34, ActionSetTriples.grp2->S35, Action->S36, ActionIncTriples.grp2->S37, Action->S38, ActionDecTriples.grp2->S39, Action->S40, ActionMulTriples.grp2->S41, Action->S42

New Post: Fatal error in code colorizer.Colorizing has been disabled

rivantsov — Fri, 26 Apr 2013 17:16:58 GMT

Man, it's time to take over the thing, rollup your sleeves and start pushing it. I've fixed all the conflicts (the most serious ones), so now you can move in small increments. Just start modifying this parameter definition when it is a 'lambda' to bring back the old code I removed - as far as I understand this is the problem now
Try it, read something about LALR parsing and about conflicts, try it again. Let me know if you're stuck
Roman

New Post: "Reduce-reduce" errors eat my lunch; we really need some context on them

rivantsov — Fri, 26 Apr 2013 17:13:11 GMT

well sorry for your lunch, but - don't quite get what's the problem with the conflict? did you try to double-click on conflict message in Grammar Explorer? It will bring you to the state printout, which show you all.
About NewLineBeforeEOF - this has no impact on parser construction. It is at runtime, when parsing, the scanner injects NewLine before sending EOF to parser, just to adjust for source files that have no final line-break, but the language is line-based.

New Post: "Reduce-reduce" errors eat my lunch; we really need some context on them

brantheman — Thu, 25 Apr 2013 21:31:33 GMT

"Reduce-reduce conflict. State S0, lookaheads: EOF LF. Selected reduce on first production in conflict set. (S0)"

My grammar file is only 170 lines long, but I still have absolutely no idea where to even start with an error like this. Can we put the nearest non-terminal name in the error somewhere?

Also, I thought Irony treated EOF as an alternate for LF if I used this: LanguageFlags |= LanguageFlags.NewLineBeforeEOF. Is that not true? Why would it have a conflict between the two?

New Post: "Reduce-reduce" errors eat my lunch; we really need some context on them

brantheman — Thu, 25 Apr 2013 21:31:33 GMT

New Post: Fatal error in code colorizer.Colorizing has been disabled

nadvi — Thu, 25 Apr 2013 02:51:15 GMT

As-Salam-wa-alicum

Sir,

Thank you very much . You are working hard for this. In my thesis I will acknowledge you.Actually you removed the iterateExp that's why these lines are not parsed.I have add the code to functionCallArgs but stile it gives parser error.Invalid character ';' but in immediate previous grammar it was parse able.Please help me here.From this grammar it seems to me that after parsing a lot of work has to do. To avoid the work I explicitly define the expressions .

I uploaded the grammar at SLAng grammar

Thank you very much again for reading my mail by spending your valuable time and working on it.

With regards
Nadvi

New Post: Fatal error in code colorizer.Colorizing has been disabled

rivantsov — Wed, 24 Apr 2013 20:46:13 GMT

Now some comments

The biggest problem is compound identifier lists - identifiers or function calls connected with dots: 'foo.Bar(x).prop' More problems come from other similar lists with delimiters "::" and "->" instead of dot. This is a kinda known problem (same happens in c#), attempt to define exact definitions of each list separately brings a lot of conflicts. So the solution is to define one generic list that covers all variations, let it parse, and then verify after parsing that the parsed list is valid for the place in the source. I defined such list as chainedSequenceExt - it is a sequence of identifiers or function calls, optionally starting with "::", delimited by "::", "." or "->", and ending with array spec (like [number]). Here is the definition:

      var sequenceDelimiter = new NonTerminal("sequenceDelimiter"); //dot, ::,  ->
      var chainedSequence = new NonTerminal("chainedSequence");
      var chainedSequenceExt = new NonTerminal("chainedSequenceExt"); 
      var sequenceMember = new NonTerminal("sequenceMember");
      var arrayDim = new NonTerminal("arrayDim");
      var arraySpecOpt = new NonTerminal("arraySpecOpt");

      sequenceMember.Rule = identifier | functionCall;
      sequenceDelimiter.Rule = dot | scopeIn | propertyIs; // . :: ->
      chainedSequence.Rule = MakePlusRule(chainedSequence, sequenceDelimiter, sequenceMember); 
      arrayDim.Rule = "*" | identifier | number | number + comma + number | number + comma + "*";
      arraySpecOpt.Rule = Empty | "[" + arrayDim + "]";
      // Note: defining scopeInOpt (with rule: Empty|scopeIn) - does not work. 
      chainedSequenceExt.Rule = scopeIn + chainedSequence + arraySpecOpt | chainedSequence + arraySpecOpt;

Use it in all places instead of objectType, customType, etc. I did not remove these, just reassigned them (like 'var customType = chainedSequenceExt;') - cleanup all these. There are many terms and nonTerminals now declared but not used anymore - clean this up.
And remember - you need to add after-parse verification of the chainedSequenceExt lists.

Note that in most places (like in expression argument), a single identifier is in fact a primitive case of chainedSequenceExt; so if you have already included chainedSequenceExt as an alternative of some rule, do not include identifier - it will bring conflict, because parser does not know which to use to interpret a single identifier.

Your other trouble comes from the attempt to define concrete function names in grammar (like 'collect', 'insertAt'), and to define rules for using them explicitly. This messes up things completely. Do not do this, they are just function calls, so let them be parsed as this. Recognize them as special, built-in methods when analysing the parse tree. I removed/disabled all these special function definitions
After this all conflicts are gone. The grammar now 'defines' a broader language, so it must be supplemented with after grammar checks - do it.
Had to bring dot as operator back. It appears you allow dot to be applied to expressions, like:
(t - startDate.inMs()).round()

so added dot to binaryOperator and set to highest prec value
Now have a parse error, unexpected ";" symbol here, right after 'Schedule'

  schedule->iterate(s : Schedule ; 
                    next : Schedule = schedule->any(true) |
                    if next.nextStartDate(t) = -1

I did not find ";" symbol defined anywhere in grammar, so this is something missing or not implemented yet.

New Post: Fatal error in code colorizer.Colorizing has been disabled

rivantsov — Wed, 24 Apr 2013 20:43:06 GMT

here's fixed version:
http://ge.tt/94592ye/v/0?c

New Post: Fatal error in code colorizer.Colorizing has been disabled

rivantsov — Wed, 24 Apr 2013 16:52:50 GMT

no, nothing is missing, what I meant is that I'm working on it, expect response soon

New Post: Fatal error in code colorizer.Colorizing has been disabled

nadvi — Wed, 24 Apr 2013 08:02:50 GMT

As-Salam-wa-alicum

Sir,

May be the link is missing.

Thank you for response.

With regards
Nadvi

New Post: Fatal error in code colorizer.Colorizing has been disabled

rivantsov — Wed, 24 Apr 2013 07:42:44 GMT

looking into this

New Post: Fatal error in code colorizer.Colorizing has been disabled

nadvi — Tue, 23 Apr 2013 01:35:05 GMT

As-Salam-wa-alicum

Sir,

If I remove dot from operator registration I get 8 conflict please help me here .

l uploaded the grammar file at SLAng Grammar

Thank you very much again for reading my mail by spending your valuable time.

With regards
Nadvi

New Post: Fatal error in code colorizer.Colorizing has been disabled

nadvi — Mon, 22 Apr 2013 23:34:29 GMT

As-Salam-wa-alicum

Sir,

Thank you very much for fixing the error .Would you like to advice me on how can I clarify the grammar because after parsing when I process the parse tree or abstract syntax tree it hep me to do by traversing .

Thank you very much again for reading my mail by spending your valuable time and fixing the error quickly .

With regards
Nadvi