Syntax error on missing optional statement

May 27, 2010 at 9:12 AM
Edited May 27, 2010 at 9:19 AM

Hello.

I need to parse the folowing example:

attribute x = 10 // optional statement - can be removed
attribute y = 15 // optional statement - can be removed
attribute width = 20 // mandatory statement
attribute height = 30 // mandatory statement

Order of attributes is important and should be (x, y, width, height), but 2 attributes (x, y) are optional. If I comment out the first attribute, I get syntax error "expected: x". How do I have to modify my grammar to handle this case?

Source grammar:

            var int_number = new NumberLiteral("int_number");
            var comment = new CommentTerminal("comment", "//", "\n", "\r");

            this.NonGrammarTerminals.Add(comment);

            var ATTRIBUTE = ToTerm("attribute");

            var attrX = new NonTerminal("attr-X");
            var attrY = new NonTerminal("attr-Y");
            var attrW = new NonTerminal("attr-Width");
            var attrH = new NonTerminal("attr-Height");

            this.Root = new NonTerminal("root");
            this.Root.Rule = attrX + attrY + attrW + attrH;

            attrX.Rule = Empty | ATTRIBUTE + "x" + "=" + int_number;
            attrY.Rule = Empty | ATTRIBUTE + "y" + "=" + int_number;
            attrW.Rule = ATTRIBUTE + "width" + "=" + int_number;
            attrH.Rule = ATTRIBUTE + "height" + "=" + int_number;

Coordinator
May 27, 2010 at 4:13 PM

You are obviously missing the statement delimiter (NewLine) from root's formula. By the way, you create root non-terminal twice.

As more general note, I would suggest to analyze these things - which attrs are present, and the order of attributes - in after-parse analysis of the parse tree. So define "root" in grammar as a generic sequence of attributes without specifying particular names or sequence, and then detect irregularities later, by going thru generated tree. 

May 28, 2010 at 9:21 AM
Edited May 28, 2010 at 11:17 AM

Attributes may have different type depending on the name. So I wanted to create concrete AST node type for each one of them and generate strongly typed DOM by traversing that tree.
.

A more complete example should be:

//attribute x = 10
attribute y = 10
attribute width = 10
attribute height = 10
attribute alignment = top
// string value can be complicated, and I would like to use different grammars for parsing it's content, depending on the attribute's name
attribute labels = "LABEL_A(bold, italic), LABEL_B(bold), LABEL_C(italic)"

 

 


So I changed the grammar as you advised:

 

            var int_number = new NumberLiteral("int_number");
            var quoted_string = new StringLiteral("quoted_string", "\"");
            var identifier = new IdentifierTerminal("identifier");
            var comment = new CommentTerminal("comment", "//", "\n", "\r");
            this.NonGrammarTerminals.Add(comment);
            var ATTRIBUTE = ToTerm("attribute");
            var attr_list = new NonTerminal("attr_list", typeof(AttrListNode));
            var attr = new NonTerminal("attr", typeof(AttrNode));
            var value = new NonTerminal("value");
            value.Rule = identifier | int_number | quoted_string;
            attr.Rule = ATTRIBUTE + identifier + "=" + value;
            attr_list.Rule = MakePlusRule(attr_list, attr);
            this.Root = attr_list;
            this.MarkTransient(value);
            this.LanguageFlags = LanguageFlags.CreateAst;

 

 


    public class AttrListNode : AstNode

 

    {
        public override void Init(ParsingContext context, ParseTreeNode treeNode)
        {
            base.Init(context, treeNode);
            foreach (var item in treeNode.ChildNodes)
            {
                var node = (AttrNode)item.AstNode;
                this.ChildNodes.Add(node);
            }
        }
    }

 

 


    public class AttrNode : AstNode

 

    {
        public override void Init(ParsingContext context, ParseTreeNode treeNode)
        {
            base.Init(context, treeNode);
            var identNode = treeNode.ChildNodes[1];
            var valueNode = treeNode.ChildNodes[3];
            this.ChildNodes.Add(identNode.AstNode as AstNode);
            this.ChildNodes.Add(valueNode.AstNode as AstNode);
            this.Name = (string)identNode.Token.Value;
            this.Value = valueNode.Token.Value;
        }
        public string Name { get; private set; }
        public object Value { get; private set; }
    }

 


After-parse analysis of the parse tree should be in AST creation phase? I don't underastand how to implement syntax and semantic errors' system in that case. 

Is there any example of after-parse analysis?

 

 

Jun 1, 2010 at 3:25 PM
Edited Jun 1, 2010 at 3:39 PM

Syntax errors will be detected by the parser so you shoudn't need to do anything special for it.  On the other hand, for semantic errors, you will need to do the post-parse analysis.

You can always just traverse the parse tree (pre-order works well) and do your checking there.

For example, for the attribute list, you could verify things like: (I don't know if your specifications require these -- they are just examples)
1) The identifiers of the child nodes are all different

foreach(var attributeNode in curNode.ChildNodes)
{
  // Verify attributeNode.ChildNodes[1].FindTokenAndGetText() is unique
}

2) "Mandatory" attributes are present

3) The child node's identifier belongs to a certain list of valid identifiers
You could also handle this case by defining a NonTerminal in the grammar with the list of all possible valid identifiers (instead of using an IdentiferTerminal).  Another solution could be to verify (or ignore completely!) that it is a valid identifier when you actually read the data into the object.

As for the role of the AST, it's mainly used to evaluate the input.  If you want to evaluate your input (e.g., you want to create some object and initialize it's "attribute"s to the values provided), then I would suggest doing the post-parsing checks during AST creation (the calls to AstNode::Init()).  I suspect that this is the case.

For example:

public class AttrListNode : AstNode
{
  public override void Init(ParsingContext context, ParseTreeNode treeNode)
  {
    base.Init(context, treeNode);

    HashSet<string> identifiers = new HashSet<string>();
    foreach(var childNode in treeNode.ChildNodes)
    {
      bool isUnique = identifiers.Add(childNode.FindTokenAndGetText());
      if (!isUnique)
      {
        throw new Exception("Identical attributes are not allowed!");
      }
      AddChild("Attr", childNode);
    }
  }
}

Brian