Last month I'm focusing almost only on Continuous Integration. Number of hours spent on that subject allows me to pretend be advanced. So far all my CI experience was limited to CruiseControl.NET (95%) and a bit of Team Foundation Server. A problem with setting up CI using TFS is well known and there is no need to talk about once more. That's makes me very frustrated as I'm pure Micfosoft-fobic (is that correct word?) and I feel dirty when I have to use something non Microsoft in my development. I exaggerate actually.
Anyway, when I've read about changes in TFS that are coming with Orcas, my first thought was "at least". Unlucky too many duties on my head doesn't allow me to try those new features till last month. Finally, when part of my daily job became setting up CI, I had time to sit down and see what's going on in Orcas TFS. After few hours of downloading Virtual Machine image and unpacking I've end up with 14 (yes, fourteen) gig virtual disk (and few gigs of base disk) and I was ready to run.
What is coming out of the box? I do presume that the same components will installed when one will do clean install. However because I'm lazy I'm using ready to use VM. First of all I didn't notice any big differences versus previous TFS version. I was expecting something shiny new that will be just "wow", not at all. I don't have previous TFS to make deeper compare at the moment, maybe will do that later. We can find new service, Team Build Service which is likely to be responsible for continuous builds, nice. We can also find a lot of stuff in SQL. Nine (9) databases in total are working as a data store for version control, build reports and who know what else. Of course we have also TFS itself.
After that brief check what's on board I was ready to play with. I took one of existing test projects, set up new project on TFS and added it to source control. Setting up build, agent and general TFS configuration to work as CI server was a piece of cake. Orcas provide us nice GUI which is something I definitely like in. Setting up build requires filling only a few fields including name, workspace we want to use, project file and of course trigger. What is a bit strange, we cannot select project file we want. Well, actually we can, it can be any file as long it is TFSBuild.proj. As you can see on the image somewhere around, there is a few options to set up trigger. However it's impossible to set up delay between commit and build. If one want to force a new build has to add it to the queue.
After first builds has come I went to project portal and problems has started. All build data collected during build processes are stored in a separated database or databases. Reports are using TFSWarehouse database and were not working at all. From the very beginning I've got security error from SQL Reporting Server. Resolving that takes me few hours. Probably one who is Reporting Server guru would be able to do that in 5 minutes, I'm not. When finally I've got reporting working, there was nothing on report. Since that moment I've made a few builds but none appear on report. Like I said, I'm not Reporting Server guru; moreover I've never used that, so forgive me ignorance here. Finally some builds appears on the report, but not all of them. Finally I was able to see all my builds on report after few hours. I was trying to find where this delay come from. I've checked all settings I could with no result. Today, when I was writing this post, I've started another build. Screen bellow shows comparison information provided by GUI from Orcas and report from project portal.
As you can see, there are some differences on that picture. Some builds are just missing. Latest one "On Commit_20070612.1" will probably appear on report tomorrow, if at all.
At this moment I gave up. I had some work to do and after two days of playing with TFS I've got nice but not working CI. In our projects we are using NUnit, NCover etc. With regret I had no idea how to integrate those with TFS. Of course one can say that we can migrate to orcas and start using VSTS testing and coverage but that's not the point. I also want to integrate other stuff I have in my CI such as Symian or FxCop. I do believe it can be done with a bit of work, writing some tasks, adding data to correct database and so on, but that's not what it is about. I was expecting solution that will allow me to use all the best from TFS, source control, work item integration, project portal and management with flexibility and expandability of CC.NET. So at this moment I was really disappointed.
As a summary. I think Microsoft done nice job with TFS, but I have feeling that it's just first part of project. For me seems to be some response for criticism about difficulties with setting up CI on TFS. So now it's available but not fully. Like something that has been added just to add something. I do believe that further release will be much more flexible and will allow setting up CI we all want to do. Then one can think about migration from other solutions to TFS.
I gave up at the moment, however I know I will return to that topic and I know I will do anything I can to prove that TFS can be CI I want to be. One day.
06-12-2007 9:30 AM