TFS Version Control + netTiers + Codesmith

Sorry about the long post, but I really would like some help/feedback.

I am trying to use TFS, netTiers and codesmith, but really am having alot of difficulty.  I am wondering if anyone out there has already tackled this and if so, how?

Using these three applications together is generally not a problem, until you start wanting to branch and merge. 

Developers Environment:

• Visual Studio 2008 Team Suite
• SQL Server Express

We have a dedicated "codesmith" machine.  The repository is as follows:

Project1
   - branches
      -branch1
         - Configuration
            - Codesmith file (.csp)
         - Other Project Files
         - Project1.sln
   - trunk
      - Configuration
         - Codesmith file (.csp)
      - Other Project Files
      - Project1.sln

Here you can see that there is a .csp file in the source control.  The idea behind this is that developer A is working on branch1, which requires new DB tables and several DB changes.  He can make the changes on his local DB (which is in source control too).  Check those changes into his branch.  The codesmith monkey then logs onto the codesmith box and load up his branch.  Make nessecary changes to the codesmith config (.csp) and run codesmith against his branch.  Normal merging between the trunk and the branches can happen as usual.  When branch1 is finished, it can be merged back into the trunk.  This merge will include any new tables in the (.csp) etc.

This all should work very well.  In practice, it does not.

Alot of the problems come from the random order in which codesmith seems to want to do things.  The following files are problems (not a complete list):

• Project.Data - DataRepository.cs
• Project.Data.Bases - NetTiersProviderBase.cs
• Project.Data.SqlClient - Procedures.xml
• All VS Project Files (.csproj)

All these files generate large lists of essentially tables, but they are generated *seemingly* randomly every time.  I have managed to fix most of these, but changing the nettiers templates and sorting the tables before these lists are generated.  This then gives a consistent order for the merge tool to use.

The next problem is that the codesmith configuration file changes alot for each save too.  This causes so many problems that i simply no longer use codesmith to edit it.  I open it in notepad and add tables as necessary.

EVen with my various fixes, I am still having alot of problems.

Has anyone acheived what I am trying to do?  It seems to me that this is fairly fundamental to programming in teams.

Any ideas?

thanks,

We use TFS in our environment, although these issues your talking about would deal with any version control software and merging.

For Generated code we use this logic: (Only generated code within the branch)

1.  When merging, when there are conflicts, we always take the Source branch.  When the conflicts dialog opens, we just select all generated and choose to take the source branch (I believe this is override local changes).

2.  We don't merge nettiers config files.  We just manually change the nettiers config.  Again, we would have just overrode the target branch.

3.  Always regenerate.  After the merge, we just assume the merge didn't work for generated code and regen.

Does that make since?

jeff

SuperJeffe,

Thanks for the response.  Very helpful.

Can you briefly step me through your development procedure.  Do all your developers work on branches?  Do you use one single DB or does each developer have their own?  Do you codesmith on all the branches?  Do you codesmith on the trunk?  How to do handle the project files?

My fixes to the nettiers templates actually resolve 90% of the problems meaning that we are able to merge the generated files successfully (although I think I might not bother now).  Re-Gen after merge sounds useful.

Do you do the standard procedure of:

• Create Branch
• Merge Trunk -> Branch daily
• Merge Branch -> Trunk

I really appreciate you helping me on this.  Thanks for all your help.

Greg

Well, we have gone through several iterations.  We just had a consultant in who advised us on a good approach...I will try and explain both.

Here is our Team Project today:

• TeamProject
  • MainLine(Trunk) - This is what is in Production
  • Branches
    • Maintenance (SHort term fixes)
    • Development (long term development)
    • Other Misc Branches...(Major Development, Test Branches, etc)

Ok, so our Mainline is what is in Production.  Nothing is ever merged unless it is ready to be released.  That way, we always have what is in production.  We have no need to keep what was previously released.  We do this with Labels.

Here is sort of some things we do

1. All developers gen code.  We try not to have more than one person doing this at a time.  Avoid merging when checking in.  We always ask if someone is generating before we generate.  If someone else is, we make sure they have the Schema changes I introduced, or wait till they check-in.
2. We have the Nettiers project file in Source Control, but most everyone just has a writable copy they maintain.  Rarely get a new version from Source Control.  We try and generate all tables and views to make it easier for Dev's to know what to choose.
3. We have a version of the templates in each branch!  This is a must when updating templates.  You should update templates and test.  If you needed a production fix, you have to be able to generate of the templates it was released with.

The consultant has recommended removing the Maintenance branch, and just have 1 development line.  We are moving to SCRUM, so we should have shorted release cycles.  So we are in the middle of changing our lifecycle around some.

Now, we have shared DB's.  I am not a fan of developers having there own DB's.  Especially when generating code.  Things could get really messed up if Schema's aren't maintained.  So we have a DB environment for each branch (Mainline hits our ProdStage DB, Development hits our Dev DB).

hope that helps some.

jeff

Thanks Jeffe.

I will need some time for that to sink in.  Thanks for all your help.  I'm sure i'll have a few more questions for you later.  I think the key difference for us is that we dont gen all tables.  The product is 5 years old and has over 1300 tables.  We gen the tables as and when it is converted from classic ASP to .NET.  This just adds a layer of complication.

What happens in the following scenario:

• Code released with Gen of Table A
• Table A undergoes a facelift in Development DB.
• Table A is Generated in Dev Branch
• New Bug in Table A code found in production.

How do you go about fixing that bug?  The only thing I can think of is to branch the production line ... fix it then merge into production.  (only branch if its a big bug, otherwise simply develop on the production line).

Thanks again.

Yea, we used to Keep a branch for Versions, but to uncomplicate things, we do exactly that....if the bug is big enough, we would branch, merge back to production, then merge to Development.  The Developer would do all of this.  This ensures all the merges are done correctly.  Testing at each step...hopefully a unit test in a perfect world.

Again, as long as someone compiles everything before they check-in a gen, if they missed a table, they should not it before they check-in.  So I still don't see it as a big deal to have seperate config files.  If it ever became a huge issue, create a "Gen" machine, that developers remote into to Gen.  Then you could keep 1 config file.....just a thought.  Although, we rarely have issues with the config files.  The hardest thing between developers is making sure the paths are changed and the db connection strings all match.  Little querks with Codesmith.

Feel free to ask questions, these are the fun questions.

jeff

Thanks for all your help Jeff...  I think we need a consultant to come in!!   I dont think there are many in the UK.

I guess the biggest hurdle for us is the tables that we are generating.  One solution would be to simply generate all of them... 

The down-sides to this is are:

• There would be approx 600 - 700 generated tables that are never used in code
• Generated DLLs are already 30MB total, this would go up alot.
• Generating already takes ~4mins.  I'm assuming this will be more like 15 mins.

What are your thoughts on this?

A few more questions if that's ok.  How do you maintain the project files?  I guess you always take the source and then re-gen?  Because we dont codesmith all the tables, this is a much large problem for us.  One branch may have 5 new tables in, so when merging, extra care must be taken not to lose these.  Its not a case of just taking the source branch, because when you merge trunk -> branch both project files may have new entries.

The more I think about it, the more I think we need to gen all our tables, regardless of whether we are going to use them.

I also think that maybe we are a little "branch happy".  Essentialy, each developer has their own branch.  I think we will try and package them up, more like yours.

Do you have your DBs in version control?  How do you pass changes between them?  We are planning to use Team System for Database developers.  Seems to be exactly what we need.  Have you got any thoughts on this?

Thanks for all your help jeff,  you really are helping me out.

Greg

I wouldn't choose all tables.  If you have that many tables, just fight through it...it won't be that bad.  Like I said, if a dev forgets to check a table, he just has to regen.  Not that big of deal.

We do store our Stored Procs and Functions in version control.  We just manually check them out and modify them in Management studio.....this isn't the greatest way of doing it, but have been doing it that way forever.  We are now moving to Team Database since they just changed there licensing.  We will be using this as fast as I can move everyone over.

For pushing out changes, we used Red Gate SqlCompare(very nice tool!).  But since Team Database does the same thing, will probably just use it.  This will give us "merge" capabilities for database code.  Should be the bomb!

Next version of Nettiers will hopefully trim the size down some....alot of duplicate code in the data layers.  Looking ahead :)

jeff