P32: Semi-automatic conflict resolution

Developer
Jul 31, 2011 at 6:18 PM
Edited Jul 31, 2011 at 6:21 PM

Hi, Roman!

I've committed custom grammar hints to my fork. Conflict resolution now basically works as described here.

Below is a GrammarExplorer screenshot (the sample is parsed using your ConflictResolutionTestGrammar):

Grammar hints look a tiny bit different from those you've suggested.

Here is an example:

// conflict resolution hints added to NonTerminals
fieldModifierList.ReduceIf(";").ComesBefore("(", "{");
propModifierList.ReduceIf("{").ComesBefore(";", "(");

// inline hint
fieldModifier.Rule = ToTerm("public") + ReduceIf(";").ComesBefore("{") | "readonly";

// hints should work with terminals as well
fieldModifierList.ReduceIf(Semicolon).ComesBefore(Parenthesis, Brace);

Hope this way hints look a bit more readable than with space-delimited list of symbols.

I've added unit tests for hint-based conflict resolver. Everything was tested on VS2010 and VS2008.

P.S. I didn't even start to think about the second part of the task (i.e., automatic advices, etc).
I'm not sure I'll have enough time to figure it out in the nearby future.

Coordinator
Aug 1, 2011 at 8:28 AM

Great!

Merged, pushed out to depot with my changes - new Interpreter.

thanks!

Roman

Developer
Aug 1, 2011 at 3:23 PM

Thanks, Roman!

> NOW 5 TIMES FASTER!

This is amazing! :)

Can't wait to check out how Refal behaves with a brand-new interpreter engine.

You've done really sophisticated optimizations regarding local variables.
I'd like to point out couple of things, though. The following lock statement:

protected void Resize(int newSize) {
  lock (this) {
    if (Values.Length >= newSize) return; 
    object[] tmp = Interlocked.Exchange(ref Values, null);
    Array.Resize(ref tmp, newSize);
    Interlocked.Exchange(ref Values, tmp);
  }
}

behaves exactly the same as this declarative specification:

[MethodImpl(MethodImplOptions.Synchronized)]
protected void Resize(int newSize) {
  if (Values.Length >= newSize) return; 
  object[] tmp = Interlocked.Exchange(ref Values, null);
  Array.Resize(ref tmp, newSize);
  Interlocked.Exchange(ref Values, tmp);
}

Locking 'this' instance, however, is not recommended, at least for public classes,
as the object can also be locked from outside of your code producing unnecessary collisions.
I'm sure you are aware of this fact, I've seen private LockObject declarations in AST node classes.

Also, lock statement is itself not very efficient: it relies on relatively heavy Monitor class.
.NET 3.5 and above provides cool lightweight lock primitives such as ReaderWriterLockSlim,
which allows, for instance, to lock the object for read and write operations separately.
On the other hand, you use locks so rarely, that lock optimization wouldn't probably make a noticeable difference.

I'm very excited about the new interpreter engine!
Could you please provide some info on when do you expect it to become production-ready? 

Regards, yallie.

Coordinator
Aug 1, 2011 at 6:02 PM

thanks for the tip! Yes, locks are on "rare" paths in interpreter, so we won't expect any impact from changes like this. On the other hand, I prefer to follow the recommended practice if it does not make a difference for the code in any other aspect. So I'll change this to use declarative style.

Slim lock - one of my other disappointments :( I rushed to try it when I found out about it, but... same stuff - it maybe helps with high contention on data, but lock acquisition/release cost is high and the same as old-style lock. So this class wont' help in Interpreter.

Making Interpreter production-ready. I hope to round it up in a few weeks. You're right, for now it's just a skeleton, with missing pieces in many places. Here's what on my do-first list:

1. The Binding functionality (LanguageRuntime_Binding.cs) should be completed and extended to support all scenarios.  

2. Importing .NET classes/functions into interpreter - for ex, to make Math class's functions directly available in expression evaluator

3. Implement MemberAccess AST node, which implements access to properties/methods of an object like obj.DoStuff(). This is important for scenarios when external object is provided from "outside" and put into Globals (it might be a .NET object or data row), and script computes a value using columns in a row. 

4. I plan to move ExpressionEvaluator to the main Irony assembly to make it available "out-of-the-box" - I realized there are many applications that need a simple expression evaluator with good external integration facilities and extensibility.

I'm rushing to finish all these initial items - expect more soon. Don't start fixing Refal yet, please, there might be changes in interpreter core code. Just investigate and be prepared

Roman

 

Aug 1, 2011 at 10:57 PM

Yallie:

Looking good.  I hope to test some of this later this week.  This should fix some of my issues and it will clean up one my grammars where I had to work around conflicts. Thank you.

Slim lock: I was similarly dissapointed because they dont scale up.  I also hit memory and processor performance issues due, amongst other things, to a hidden array of integers in the background .....

The declarative style is also equivalent to "lock( this )" so I am not sure if you gain anything at all by doing that?  I understood that recommended practice was to not lock a publicly available object? ("this" is public by definition, so the declarative style is also public?).

 

Developer
Aug 4, 2011 at 10:58 PM

Hi Roman!

>Slim lock - one of my other disappointments :(
>...lock acquisition/release cost is high and the same as old-style lock. 

Didn't know that. Should have benchmarked it myself before posting.

>Don't start fixing Refal yet, please, there might be changes in interpreter core code.

Sure, no problem. Seems like it's not yet possible, so I'll better wait for the next code update.

>Just investigate and be prepared

Ok :)

Developer
Aug 4, 2011 at 10:59 PM

>Looking good.  I hope to test some of this later this week.

Thanks Will! It would be great :)

>Slim lock: I was similarly dissapointed because they dont scale up.

That's sad, I didn't know that.

>The declarative style is also equivalent to "lock( this )"
>so I am not sure if you gain anything at all by doing that?  

You're right, it's no better than lock (this).

Perhaps it only makes a difference for code analysis tools: it's a lot easier
to analyze method attributes than to disassemble IL code and follow execution paths.

>I understood that recommended practice was to not lock a publicly available object?
>("this" is public by definition, so the declarative style is also public?).

Yes, declarative lock is also public, so I usually avoid both methods.
Locking a private object is always preferable.

Regards, yallie.