I am currently working on the new parser for CodeSmith Generator 6.0. When I wrote the original template parser back in 2004, I really had no idea what I was doing. :-) I had no idea what an AST was or what an LL(*) parser was, etc. That being said, I think I did an OK job and it has certainly served it’s purpose, but it can’t be easily changed and things are more complicated than they need to be.
I have been working on our new parser for a few weeks now and I have a pretty good start. Here is a list of things that we want to get out of the new parser:
I decided to make use of the parser framework that Actipro includes with their SyntaxEditor control. It is an LL(*) parser framework that lets you define the grammar right in C#. Here is what a sample grammar production looks like:
codeExpression.Production = @expressionStart + @expressionText["expr"] + @expressionEnd > Ast<ExpressionNode>() .SetProperty(n => n.Value, AstFrom("expr"));
This code is defining the grammar rule that makes up a code expression <%= SomeValue %> in a template. It says that there will be an expression start token followed by some expression text and then an expression end token. It is also constructing an AST node for the expression. The parser framework has really helped get our new parser of the ground quickly and I highly recommend it.
The cool thing about defining a grammar is that if there are errors in the template, you will get much more intuitive error information about what the parser saw and what it was expecting to see. Previously, we pretty much just parsed the template, generated code from it and tried to compile it. Here is an example of how we will be providing much better errors in Generator 6.0. Here is a template that has a missing % at the end of the first line:
<%@ CodeTemplate Language="C#" > <%= DateTime.Now %>
Here is the error that Generator 5.3 gives: error CodeSmith0104: You must specify a valid template language.
And here is the error message that the new parser gives: Line 2, Char 3: %> expected.
In my next post I will talk about the AST and how we are using the visitor pattern to do transformations on the parsed template content.
P.S. If you haven’t figured it out by now, I’m really bad about writing to my blog. Rather than break out the random excuse generator, I’m just going to start posting. Happy New Year!
You've been kicked (a good thing) - Trackback from DotNetKicks.com
Thank you for submitting this cool story - Trackback from DotNetShoutout
Sounds like how Irony works (http://irony.codeplex.com/), except Irony works directly with the Visual Studio editor, which is really where you should be taking this product if your going to be doing an extensive re-write. The one thing I dislike about CodeSmith is having to go into another IDE to develop templates. There isn't a reason for it.
Yes, we are going to be bringing it inside of VS. This is just a step required along the way. I actually looked at using Irony, but it seems like development has slowed down and we liked the parser included with Actipro's product so we decided to go that route.
In this post I’m going to talk about how the new template parser in CodeSmith Generator 6.0 creates an
WOW – thanks a lot for creating this. I will surely take advantage of this and tell others about it.
It has been Spring now for a few weeks, and while the weather in my home state of Texas still seems to
Oh, my this is brilliant. Thank you so much for sharing. I have extended its reach by also sharing on Twitter ,hope you don’t mind lol