FreeTextLiteral

Oct 1, 2009 at 5:41 PM
Edited Oct 1, 2009 at 5:41 PM

Digging through Irony source, I found FreeTextLiteral, which seems what I need for the following case:

element type 1000 some description here
{
}
var number = new NumberLiteral("number");
var freeTextLiteral = new FreeTextLiteral("freeTextLiteral", "\n", "\r", Environment.NewLine);
var header.Rule = symbol("element") + symbol("type") + number + freeTextLiteral;

I'd like to know if that's the correct way to obtain the following tokens

element
type
1000
some description here

Of course I've already tried the above, but from time to time I get a "freeTextLiteral expected" compile error, especially if I add more rules to the grammar. 

Thanks in advance for any insight.

Oct 1, 2009 at 9:50 PM

Correct me if I'm wrong, but I think FreeTextLiteral is a literal that collects characters until it hits any of the provided terminating strings.  So, I would be very careful when using this literal as you have to consider all possible scenarios. 

For example, is the following allowed - no description text?

element type 1000
{
}

If so, you could get the error you described because the absence of text isn't specified in your grammar.  If this scenario is allowed then your grammar may look like this:

var number = new NumberLiteral("number");
var freeTextLiteral = new FreeTextLiteral("freeTextLiteral", "\n", "\r", Environment.NewLine);

var description = new NonTerminal("description");
description.Rule = Empty | freeTextLiteral;

var header.Rule = symbol("element") + symbol("type") + number + description;

Lastly, if the following is allowed, then you may have to consider another solution then the FreeTextLiteral.

element type 1000 some description here {
}

 

-mindcore

Coordinator
Oct 2, 2009 at 6:57 AM

I would agree with MindCore that "no-description" is a suspicioujs case - is it the case when it happens? In this case you should make description optional, just like Mindcore describes.

In this case it looks like scanner passes "number", then - before starting scanning description - it eats all whitespace chars, which include \n char. So when it is ready to scan description it is positioned at "{" char - therefore the error.  

Oct 2, 2009 at 9:00 AM
Edited Oct 2, 2009 at 9:02 AM

Hello, and thanks for your valuable insights.

I have no control over the original format; I am fairly sure it hasn't changed in years and will not produce edge cases such as the one MindCore described.

Therefore, there will always be a "description" before a newline.

 

Blocks, defined by

{ }

symbols, may or may not have a header as described above.

Cases are:

element type 1000 some description here (newline) { }
element (newline) { }
{ }

I think I already handle other cases in the grammar. It's just the "description" part before the newline.

In this case it looks like scanner passes "number", then - before starting scanning description - it eats all whitespace chars, which include \n char. So when it is ready to scan description it is positioned at "{" char - therefore the error.  

I guess this is what happens. You might have suggested a solution in your answers but bear with me, I'm a curious Irony user willing to learn fast. :)

I think I'll ask about subgrammars in another thread.

Thank you again, your help is much appreciated.

 

 

Coordinator
Oct 2, 2009 at 9:51 PM

So, if this info does not help to solve the problem then I'm afraid I need more detailed info. Probably the best way would be to get the entire grammar file and some failing samples (if you can share this with me at least). Contact me directly through this site, I'll reply and then you'll send the files

Oct 5, 2009 at 9:11 PM

Thanks a lot for your help, Roman. I use a very iterative approach while building the grammar as I don't have a set of BNF rules for the language I'm trying to parse, so the grammar itself is not quite complete. This is going to be a very busy period so I'm not sure when I'll be able to contact you, but I'll be sure to do so as soon as I have some spare time in the near future.