ConstantTerminal matching

Aug 11, 2010 at 8:03 AM


I have been using ConstantTerminal for a new grammar and I have just noticed an undesired behavior. See the following example.

If the constant table contains two contants "Number" and "NumberAlpha" in that specific order, whenever NumberAlpha appears in the code being parsed, it will be tokenized as Number, making the "Alpha" part of the code to cause a parsing error.

The only workaround I have found for this so far is creating the constant table with sorted strings from longer to shorter in length.

If there is any other way around this, please contribute to this discussion.



Aug 13, 2010 at 4:07 PM
this looks like an overlook, I should make sure this auto-sort happens automatically at scanner-construction time. You have a workaround for now, so no urgent matter. An alternative is to set the Term flag AllowAlphaNumAfter (or smth like this) that prohibits such partial scanning - it would require that recognized term is not followed by a number of letter, so it's not part of bigger term. I will have a look at this