Previously in MetaSharp I had a custom Templating engine for textual transformations. It was a custom grammar and was pretty simple. It worked well and all was good but it was becoming a maintenance burden for me and, I think, turned people off to MetaSharp more than attracted them. So I decided to spend a little time and try to integrate T4 into my templating library as a replacement.

It was actually reasonably simple to accomplish. The hardest part, of course, was working with AppDomains. T4 is a dynamic tool living in a static world and as such it requires an AppDomain to do its magic so it can all be unloaded without a trace. Because of this T4 isn’t very amenable to input. Most examples I found online start with firing up your template then beginning by reading a file or creating a class that connects to a database. In my case I have already have my metadata and would like to pass it into a T4 template but it’s in memory only. The first thing you have to do is make your model Serializable.

Next you need to know about the “inherits” and the “hostspecific” attributes. You add something like this to the top of your template:

<#@ template language="C#" inherits="SongMetaTemplate" hostspecific="true" debug="true" #>

This is for my Song sample application. I then create a SongMetaTemplate class that knows how to cast the model into a Song enumeration.

public abstract class SongMetaTemplate : MetaTemplate
{
    public IEnumerable<Song> Songs
    {
        get { return ((IEnumerable<INode>)this.Model).Cast<Song>(); }
    }
}

Adding a hostspecific attribute to the template causes the template class that inherits from the SongMetaTemplate to also have a Host property which gets loaded up with the Model provided by my pipeline. The MetaTemplate knows how to fetch that host model for me.

public abstract class MetaTemplate : TextTransformation
{
    private object model = null;

    protected object Model
    {
        get
        {
            if (model == null)
            {
                dynamic metame = this;
                var metaHost = metame.Host as MetaSharpTextTemplatingHost;
                if (metaHost != null)
                    this.model = metaHost.Model;
            }

            return model;
        }
    }

    public override string TransformText()
    {
        return this.GenerationEnvironment.ToString();
    }
}

And that’s it! My template can now get a hold of the Song AST parsed from the DSL earlier in the pipeline. Here is an example template in its entirety:

<#@ template language="C#" inherits="SongMetaTemplate" hostspecific="true" debug="true" #>
<#@ assembly name="DynamicSongBuilder" #>
<#@ import namespace="DynamicSongBuilder" #>

<# foreach(var song in this.Songs) { #>
    <# foreach(var b in song.Bars) { #>
        <# foreach(var n in b.Notes) { #>
            play(<#=song.Duration #>, "<#=n.Key #>", <#=n.Octave #>);
        <# }
    }
} #>

The template code is C# but the code generated is actually for my own custom language. In this case it’s parsed into Linq expressions and compiled into a delegate using a context object with a play method. It’s then executed dynamically and also collected by the GC when dereferenced. I plan to replace the custom language also, but its proving difficult without a reasonable parser to replace it.

Here is an example of the types of tests I’ve been writing recently, see if you can figure out what it’s doing:

[Test]
public void TypeNameParameterizedRuleSuccess()
{
    string expected = "a.b.c";
    var stream = new TextStream(expected);

    var list = new Func<PatternMatch, PatternMatch, PatternMatch>(
        (exp, separator) =>
        {
            return Pattern.And(
                exp,
                Pattern.ZeroOrMore(
                    Pattern.And(
                        separator,
                        exp)));
        });

    var pattern = Pattern.Projection(
        list(
            Pattern.Range('a', 'z'),
            Pattern.Value('.')),
        match => match.AsString());

    var actual = pattern(stream);
    Assert.AreEqual(expected, ((ValueNode)actual).Value);
}

beautiful!

I have created a small routine that allows you to use the IronPython runtime to create dynamic Types without requiring you to write any Python code.

The normal use case for IronPython in C# requires you to provide the IronPython runtime with some source code files, which will essentially eval and return to you a context which you can interact with dynamically. With some help on the forums I was able to find a way to provide an IronPython AST to the runtime instead of source code. This will be very useful for MetaSharp, since I already have a lot of mechanisms for creating an AST I will only need to simply create a transformer that will change an existing AST into IronPython AST and you can suddenly compile any DSL into arbitrarily complex object hierarchies at runtime.

One of the reasons for this is that I want to create a language workbench tool, where you can experiment with your transformations dynamically without having to recompile in the traditional sense. Also, it just gives you more options at runtime. This should be very handy.

Here is an example of what I’m talking about. My example code is very simple, create a class that implements a simple .net interface then create an instance of that class and call it’s method. Here is the actual python version:

#sample.py
from ConsoleApplication4 import IExample

class Example(IExample):
    def Do(self):
        return "hello python"

Executing and calling this at runtime in C# is quite easy to do out of the box actually.

var runtime = Python.CreateRuntime();
runtime.LoadAssembly(typeof(IExample).Assembly);
dynamic python = runtime.UseFile("sample.py");
            
var example = (IExample)python.Example();
Console.WriteLine(example.Do());

The rub here though is that you need that python source file laying around somewhere. Or you could generate code, put it in a temp file and use that instead but nonetheless, the creation of the dynamic object is done by creating python code only. My problem was to figure out how to bypass most of the parsing semantics and to provide an AST directly to the runtime instead of code files. The AST could be created by a MetaSharp transform step.

So instead you can now do this:

var runtime = Python.CreateRuntime();
runtime.LoadAssembly(typeof(IExample).Assembly);

var iexampleImport = typeof(IExample).Import();

var classDefinition = Dlr.Class(
    "Example",
    iexampleImport.ToEnumerable(),
    Dlr.Function(
        "Do",
        Dlr.Self().ToEnumerable(),
        "hello python!".AsConstant().Return()));

dynamic python = runtime.Compile(
    iexampleImport,
    classDefinition);

var example = (IExample)python.Example();
Console.WriteLine(example.Do());

I won’t go into all of the extension methods I created (see link to source at the end) but suffice it to say that the Dlr.Class method is creating a ClassDefinition object. Passing these into the runtime.Compile method is where all of the magic happens.

Figuring out what to do in the Compile extension method was the real hard part. It’s not well documented and some of the side effects are downright bizarre, clearly the API was not intended to be (mis)used in this way.

Fortunately for you, all you have to do is download the file below and you’re off creating dynamic Types using the DLR!