irony Work Item Rss Feed

Commented Issue: Custom & FreeTextLiteral Terminals are not sorted by priority [6291]

rivantsov — Wed, 18 Nov 2009 16:22:24 GMT

Hey Roman,

First of all, I must say, great work! I've been using Irony for a little over a month with awesome results, but I finally ran into this bug.

I have two terminals, a CustomTerminal and a FreeTextLiteral. It seems that the terminals are being sorted by token name rather than priority. (They are stored in alphanumerically ascending order in Scanner::Data.FallbackTerminals). During the Scanner::SelectTerminals(char) function call, the terminals are just copied from Data.FallbackTerminals into termList (line #202), and no other sorting takes place in this function.

In my grammar, the terminals are declared as:

var endOfLineText = new CustomTerminal("EndOfLineText_Custom", EndOfLineTextMatcher);
var messageText = new FreeTextLiteral("MessageText", "{");
messageText.Priority = Terminal.HighestPriority;

I have tried reodering the declarations, adjusting the priorities on both, but with no luck. The current work-around that I'm using is to name them as "A_MessageText" and "Z_EndOfLineText_Custom" in order to get them to "sort" correctly (alphanumerically where "A_***" ends up with a higher 'priority' than "Z_***".

If you could get it so that it takes the priorities into consideration, I would greatly appreciate it. Thanks!

Created Issue: Custom & FreeTextLiteral Terminals are not sorted by priority [6291]

bdaugherty — Tue, 17 Nov 2009 14:56:46 GMT

Closed Issue: Polish charset [6174]

rivantsov — Mon, 09 Nov 2009 22:56:23 GMT

Hi!

I am using IdentifierTerminal
var identifier = new IdentifierTerminal("Term", "!@#$%^*_'.?", "!@#$%^*_'.?0123456789")

I have to add polish chars like ąĄćĆęĘłŁóÓńŃżŻźŹ so i changed declaration to:
var identifier = new IdentifierTerminal("Term", "!@#$%^*_'.?ąĄćĆęĘłŁóÓńŃżŻźŹ", "!@#$%^*_'.?0123456789ąĄćĆęĘłŁóÓńŃżŻźŹ")

Irony still not works with my polish chars. Maybe its problem with unicodes?

Kamil

Commented Issue: Polish charset [6174]

beny — Mon, 09 Nov 2009 19:14:05 GMT

Created Issue: Polish charset [6174]

beny — Mon, 09 Nov 2009 12:01:24 GMT

Closed Issue: Unary operator evaluation [4901]

rivantsov — Mon, 09 Nov 2009 07:05:34 GMT

When evaluating unary operators the second argument in the context should be set to null.

I declared a call dispatcher for a unary operator "not" just for booleans and I kept getting that it was not defined for System.String because there was a reminder string in the second argument from the context from the previous evaluation.

Closed Issue: Shadow Copy Assemblies when loading into explorer [5874]

rivantsov — Mon, 09 Nov 2009 07:04:25 GMT

I liked using the Grammar Explorer during development but found myself constantly closing and re-opening it to be able to compile new assemblies. the NUnit GUI runner uses shadow copies to avoid interfering with new builds and will automatically re-load new assemblies. I think this is a great idea and most of the implementation can probably be borrowed from NUnit.

Commented Issue: Shadow Copy Assemblies when loading into explorer [5874]

rivantsov — Wed, 21 Oct 2009 04:55:35 GMT

Thanks for your suggestion. Yes, that would be nice to have GrExpl running all the time, rebuild the grammar assembly when needed and make it reload automatically. But... correct me if I'm wrong, but from what I know/remember about how NUnit does this (and any other app) is by loading tested assembly into a separate app domain and executing tests there. After the tests are run the domain is unloaded, shadow or any other copies on disk are unlocked and you can overwrite them. It is a limitation of .NET fwk that once assembly is loaded, it cannot be unloaded, ever - unless it is loaded into separate domain, and the entire domain is unloaded. So having shadow copy is not enough - once you loaded assembly from it, it is locked for writing, and you cannot overwrite it with newer version.
As for loading grammar into a separate domain - hmm.. maybe, but certain compilcations may arise, and performance may suffer for sure. It will require more careful analysis of what's involved. Wanna try it? I cannot get to it now, have a long to do list already. So far, it is inconvenience, annoyance at worst, but not that serious I hope...
thanks again
Roman

Created Issue: Shadow Copy Assemblies when loading into explorer [5874]

shiften — Tue, 20 Oct 2009 03:44:27 GMT

Closed Issue: Grammar.CurrentGrammar can lead to nasty bugs with more than a single grammar derived object [5709]

rivantsov — Tue, 13 Oct 2009 22:38:53 GMT

Using the latest Irony code drop (33075).

I am developing a grammar and a simple compiler for a subset of the C preprocessor language.
Initially I had the compiler class create the grammar on each call to my Compile method.
But I decided to change that and to have the grammar object as a singleton to be created once. After I did that my unit tests started to fail in what looked like totally random places.

The reason was, that some of my unit tests also created the grammar object directly, in order to check the correctness of the parse tree for simple grammars.
Since my test runner (xUnit) runs the unit tests in a random order, this changed the Grammar.CurrentGrammar static property because a new Grammar object was created (randomly). However, the old one was still alive and was used in later tests.

This caused the following bug:
In GrammarDataBuilder.CreateProduction(), the following condition:
if (operand == Grammar.CurrentGrammar.Empty)
failed even when operand is the Empty keyword, because the Empty in operand was created in a different Grammar object than the one in Grammar.CurrentGrammar. So Empty terminals were not skipped, and this stage produced a grammar with incorrect productions.

IMHO, using the static CurrentGrammar field is more trouble than it’s worth.
In this specific method, there is no reason to use it, since GrammarDataBuilder has a _grammar field which points back to the correct grammar we’re working on.

Possible workarounds (in the current release, without changing Irony code):
- Always create a new Grammar object before using it
- If using a Grammar singleton, then before using it, check if (Grammar.CurrentGrammar != grammarSingleton) and if so, some other Grammar has “stolen” the CurrentGrammar reference, and we need to restore it by recreating the Grammar object.
Comments:

Fixed in latest drop

Commented Issue: Grammar.CurrentGrammar can lead to nasty bugs with more than a single grammar derived object [5709]

orwen — Fri, 09 Oct 2009 16:15:57 GMT

Hi, thanks for your quick response.
I looked again into the usage of CurrentGrammar in the helper methods you mentioned, and now I agree that your usage of it there, it does solve the problem, allowing easy usage of the custom operators when defining a custom grammar (which is one of the great things about Irony).

Sorry if I’ve been harsh, the bug wasn’t that difficult to find.
I think Irony is a great parser library, quite easy to understand and work with. I appreciate all the work you put into it!

Commented Issue: Grammar.CurrentGrammar can lead to nasty bugs with more than a single grammar derived object [5709]

rivantsov — Mon, 05 Oct 2009 15:35:22 GMT

You are right, it is a nasty bug. But the problem is not this static property by itself, just the use of it in this place. It was created to be used exclusively in helper methods of Grammar class which are called from custom grammar constructor (op_Plus, op_Pipe, Star, etc). There was a long search for solution for these methods, and thread-static CurrentGrammar property is a perfect solution. Using it in parser builder is a bug, don't know how it got there. I will fix it.
Thanks for finding it and sorry for your troubles
Roman

Created Issue: Grammar.CurrentGrammar can lead to nasty bugs with more than a single grammar derived object [5709]

orwen — Sat, 03 Oct 2009 22:36:56 GMT

Commented Issue: Assembly generation failed -- Referenced assembly 'Irony' does not have a strong name

rivantsov — Sun, 02 Aug 2009 04:27:33 GMT

Hello...I was getting this error when trying to use Irony inside a solution that had a key generated for another project. Apparently, if you try to mix key-signed with not signed assemblies you get this.

I generated a key for Irony on my local copy and it was fixed but you might want to generate a key for Irony and check it into the subversion.
Comments: ** Comment from web user: rivantsov **

Added assembly key

Commented Issue: Assembly generation failed -- Referenced assembly 'Irony' does not have a strong name

rivantsov — Wed, 22 Jul 2009 20:10:44 GMT

thanks for pointing to this, will give it a thought

Created Issue: Assembly generation failed -- Referenced assembly 'Irony' does not have a strong name

budmandave — Wed, 22 Jul 2009 20:01:30 GMT

Commented Issue: Unary operator evaluation

rivantsov — Thu, 09 Jul 2009 23:14:20 GMT

Will look into this. thanks for catching and reporting. Will fix it when I come to reimplementing runtime in latest source version.

Commented Issue: Unary operator evaluation

MichelangeloPM — Thu, 09 Jul 2009 15:15:25 GMT

*remainder

Created Issue: Unary operator evaluation

MichelangeloPM — Thu, 09 Jul 2009 15:14:19 GMT

Closed Issue: Possible null reference in Scanner.Parse

rivantsov — Tue, 02 Jun 2009 17:26:36 GMT

I realized (due to weird type resolvement) that the result = stack[0].Node in the begining of the parse method in Scanner kan result in result being null causing a nullReferenceException in the line below.
This is basically not an Irony error but the error reporting could be better. As it is you'd probably need to debug the Irony source to figure out what caused the null referance. In my case when run out side VS development hive I got an instance of Irony.Grammar instead of my own causing every token to be a syntax error. The weird part was when run in the hive or the Irony explorer everything worked fine. The weirdnees made it very hard to track down which was why I came to think of handling the error cracefully
Comments: