Welcome to the CodeSmith Community!

CodeSmith Generator 6.0 Template Parser Progress Report - Part 1

CodeSmith Community

A description has not yet been added to this group.

CodeSmith Generator 6.0 Template Parser Progress Report - Part 1

Rate This
  • Comments 9

I am currently working on the new parser for CodeSmith Generator 6.0.  When I wrote the original template parser back in 2004, I really had no idea what I was doing. :-)  I had no idea what an AST was or what an LL(*) parser was, etc.  That being said, I think I did an OK job and it has certainly served it’s purpose, but it can’t be easily changed and things are more complicated than they need to be.

I have been working on our new parser for a few weeks now and I have a pretty good start.  Here is a list of things that we want to get out of the new parser:

  • Structure - Generate an AST as an intermediate step so that we can do transformations using a Visitor pattern and even LINQ queries.
  • Grammar - Create a grammar for the syntax so that we can give grammar errors.
  • Testability - Make it easy to unit test.
  • Maintainability - Make it easy to implement changes and add new features.

I decided to make use of the parser framework that Actipro includes with their SyntaxEditor control.  It is an LL(*) parser framework that lets you define the grammar right in C#.  Here is what a sample grammar production looks like:

codeExpression.Production = @expressionStart + @expressionText["expr"] + @expressionEnd
     > Ast<ExpressionNode>()
           .SetProperty(n => n.Value, AstFrom("expr"));

This code is defining the grammar rule that makes up a code expression <%= SomeValue %> in a template.  It says that there will be an expression start token followed by some expression text and then an expression end token.  It is also constructing an AST node for the expression.  The parser framework has really helped get our new parser of the ground quickly and I highly recommend it.

The cool thing about defining a grammar is that if there are errors in the template, you will get much more intuitive error information about what the parser saw and what it was expecting to see.  Previously, we pretty much just parsed the template, generated code from it and tried to compile it.  Here is an example of how we will be providing much better errors in Generator 6.0.  Here is a template that has a missing % at the end of the first line:

<%@ CodeTemplate Language="C#" >
<%= DateTime.Now %>

Here is the error that Generator 5.3 gives:
error CodeSmith0104: You must specify a valid template language.

And here is the error message that the new parser gives:
Line 2, Char 3: %> expected.

In my next post I will talk about the AST and how we are using the visitor pattern to do transformations on the parsed template content.

P.S. If you haven’t figured it out by now, I’m really bad about writing to my blog.  Rather than break out the random excuse generator, I’m just going to start posting.  Happy New Year!

kick it on DotNetKicks.com

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • * Please enter your name
  • * Please enter a comment
  • Post
  • You've been kicked (a good thing) - Trackback from DotNetKicks.com

  • Thank you for submitting this cool story - Trackback from DotNetShoutout

  • Sounds like how Irony works (http://irony.codeplex.com/), except Irony works directly with the Visual Studio editor, which is really where you should be taking this product if your going to be doing an extensive re-write.   The one thing I dislike about CodeSmith is having to go into another IDE to develop templates.  There isn't a reason for it.

  • Yes, we are going to be bringing it inside of VS.  This is just a step required along the way.  I actually looked at using Irony, but it seems like development has slowed down and we liked the parser included with Actipro's product so we decided to go that route.

  • In this post I’m going to talk about how the new template parser in CodeSmith Generator 6.0 creates an

  • In this post I’m going to talk about how the new template parser in CodeSmith Generator 6.0 creates an

  • WOW – thanks a lot for creating this. I will surely take advantage of this and tell others about it.

  • It has been Spring now for a few weeks, and while the weather in my home state of Texas still seems to

  • Oh, my this is brilliant. Thank you so much for sharing.  I have extended its reach by also sharing on Twitter ,hope you don’t mind  lol

Page 1 of 1 (9 items)
Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • * Please enter your name
  • * Please enter a comment
  • Post