Dynamically Compiled Lambdas vs. Pure Reflection

by justin 1. September 2009 00:45

I’ve spent a little time messing around with expression trees in .net 4 over the last couple of weeks and so I decided to put together a test comparing the performance of compiling a lambda expression on the fly vs. simply using basic reflection to invoke a method. You will probably see some of this in a patch to CSLA for Silverlight in the future, since it is currently calling pure reflection. Regular CSLA uses DynamicMethod and uses the ILGenerator to build a similar call to a method. And actually, looking in reflector, that is exactly what the LambdaExpression.Compile() method is doing also, its is simply easier to build up the code you are trying to compile using the Expression tree objects.

Setup

MethodInfo barMethod = typeof(Foo).GetMethod("Bar");
MethodInfo bazMethod = typeof(Foo).GetMethod("Baz");

int iterations = 100000;

Foo foo = new Foo();
object[] p1 = new object[] { 100, "blah" };
object[] p2 = new object[] { 100, "blah", Guid.Empty };
public class Foo
{
    public static int staticCount = 0;
    public int count = 0;

    public void Bar(int x, string y)
    {
        count++;
    }

    public static void Baz(int x, string y, Guid z)
    {
        Foo.staticCount++;
    }
}

Rather than doing all of this in the body of each test I wanted them both to have everything setup and even, in both cases we are getting the method to call with basic reflection. The Foo class is the class we will be running the test on and the fields are there simply to give the method body something to do so the compiler doesn’t optimize it out.

Invoke via Reflection

DateTime start = DateTime.Now;
for (int x = 0; x < iterations; x++)
{
    barMethod.Invoke(foo, p1);
    bazMethod.Invoke(null, p2);
}
DateTime end = DateTime.Now;
Console.WriteLine("Time with reflection: {0}.", (end - start).TotalMilliseconds);

Here we’re very simply calling the Invoke member of MethodInfo. The first method is a member method the second is static.

Invoke via Delegate

start = DateTime.Now;
Action<object, object[]> call = null;
Action<object, object[]> staticcall = null;
for (int x = 0; x < iterations; x++)
{
    if (call == null)
    {
        call = CreateCaller(barMethod);
        staticcall = CreateCaller(bazMethod);
    }

    call(foo, p1);
    staticcall(null, p2);
}
end = DateTime.Now;
Console.WriteLine("Time with dynamic lambdas: {0}.", (end - start).TotalMilliseconds);

Here we are building two delegates on the first iteration and calling it directly for subsequent iterations. I’m using Action<object, object[]> to allow me to very generically call any method. The more you know about the method you are trying to call the nicer the delegate signature could be (and probably a little faster) but this is also designed to be extremely general.

Also, note that I am using a null check for caching but in reality you would probably have something more complex like a dictionary. This would add a little extra overhead onto the execution time.

Create Caller via Expression Builder

Here is where the real interesting part takes place, here we are using the System.Linq.Expressions namespace to build a lambda expression dynamically based on the MethodInfo object passed in. The build lambda is compiled into a Delegate designed specifically to call that method. There are two types of methods built, one for member methods and one for static methods.

using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
private static Action<object, object[]> CreateCaller(MethodInfo method)
{
    int index = 0;
    var p1 = Expression.Parameter(typeof(object), "instance");
    var p2 = Expression.Parameter(typeof(object[]), "parameters");
    var parameters = from p in method.GetParameters()
                     select Expression.Convert(
                            Expression.ArrayAccess(
                            p2,
                            Expression.Constant(index++)),
                            p.ParameterType);

    Expression instanceCheck = null;
    Expression call = null;
    if (method.IsStatic)
    {
        instanceCheck = Expression.IfThen(
            Expression.NotEqual(
                p1,
                Expression.Constant(null)),
            Expression.Throw(
                Expression.New(
                    typeof(ArgumentException).GetConstructor(
                        new Type[] { 
                                typeof(string),
                                typeof(string) }),
                    Expression.Constant(
                        "Argument must be null for a static method call."),
                    Expression.Constant("instance"))));

        call = Expression.Call(
            method,
            parameters);
    }
    else
    {
        instanceCheck = Expression.IfThen(
            Expression.Equal(
                p1,
                Expression.Constant(null)),
            Expression.Throw(
                Expression.New(
                    typeof(ArgumentNullException).GetConstructor(
                        new Type[] { typeof(string) }),
                    Expression.Constant("instance"))));
        call = Expression.Call(
                    Expression.Convert(p1, method.DeclaringType),
                    method,
                    parameters);
    }

    var lambda = Expression.Lambda<Action<object, object[]>>(
            Expression.Block(
                instanceCheck,
                call),
            p1,
            p2);

    return lambda.Compile();
}

Results

When running the test application here are the results:

Time with reflection: 391.0391.
Time with dynamic lambdas: 15.0015.

Not surprisingly the dynamic lambdas vastly outperform standard reflection. But please note that this is in Milliseconds and is after 100,000 iterations. This sort of optimization is really only needed on areas where reflection is used heavily and frequently.

Action vs. Delegate

One of the interesting things I learned from this test was the difference between compiling a strongly typed delegate (such as System.Action) and a System.Delegate. Simply changing my lambda test code to this:

for (int x = 0; x < iterations; x++)
{
    if (call == null)
    {
        call = CreateCaller(barMethod);
        staticcall = CreateCaller(bazMethod);
    }

    call.DynamicInvoke(foo, p1);
    staticcall.DynamicInvoke(null, p2);
}

(notice the .DynamicInvoke() instead of direct invocation)

Changes the tests to yield the results:

Time with reflection: 393.0393.
Time with dynamic lambdas: 925.0925.

Which is dramatically worse than standard reflection! So the trick is that you really need to get a strongly typed delegate for it to be worthwhile at all to call. My above experiment attempts to solve the problem by using a general purpose Action<object, object[]> (which should probably be a Func<object, object, object[]> now that I think about it) and uses casting to resolve all of the problems with calling the method incorrectly but the fact of the matter is that the more you know about the method signature the better you will be able at optimizing it. For example, if you are able to know that there will never be any return values, that is a big optimization, or if you know a common base class for the instance method or if there will never be a static method then you can dramatically tweak this even further. The general solution is much more complex however.

Tags: , , ,

dynamic | C# | .NET | CSLA

Comments

8/29/2009 8:14:34 AM #

Jeff Klawiter

This is close to a post Elijah Manor did last week about ASP.NET MVC 2.0. They were using dynamically compiled lambdas for the new strongly typed helpers and he found their performance were bad compared to reflection.

elijahmanor.com/.../...pedUIHelperPerformance.aspx

This is good things to do know since I'm doing some similar things in my VS2010 extension. Now I know what to look out for if I want to keep it performant.  

Jeff Klawiter |

8/30/2009 7:42:17 AM #

Jeff,

Wait...what? I thought Justin said this:

"Not surprisingly the dynamic lambdas vastly outperform standard reflection"

But that sounds contradictory to what you're saying Manor found, which is that dynamic lambdas are less performant than using Reflection.

I've also done work with performance timings on method invocation via Reflection, DynamicMethod, and expressions. Reflection is always the work, with the other two being very similar.

Jason Bock |

8/30/2009 9:28:11 AM #

Close in topic perhaps but different in results. I'm not sure how he came to those conclusions. The only think I can think of is that he is using his expressions to get values out of properties instead of methods and he appears to be compiling the lambda everytime rather than caching the delegate. If you had an object with a bunch of properties and you wanted to create a delegate for each property I could see how that would be slower I guess.

justin |

11/22/2009 12:26:20 AM #

Oh another thing I found out on some parallel research is that if you are able to cast your delegate into a strong type, such as Action<T> or Func<T> then you get the results I was seeing here, but if you use it as a Delegate and call DynamicInvoke then it ends up being incredibly slow, slower even then reflection. So that's an important factor.

justin |

Comments are closed

About Me

sweetest hat ever

I'm a software developer from Minnesota and this blog largely focuses on various technical concepts I am thinking about at the moment. I currently work for Microsoft in the St. Paul office of the Expression product group.

RecentPosts