Dave Donaldson

Critical thinking in software development

Search

Advertisement

Subscribe

Technorati

View blog authority
View blog reactions

My Tweets

  • Blogged: How To Move from Subtext to Graffiti http://tinyurl.com/4fhs5w
  • I forgot how good "Crackerman" by Stone Temple Pilots was.
  • My wife has discovered the nudge feature in Windows Live Messenger. The next time I'm in Redmond I shall hunt that person down.

Using TFS for CI: Gotchas

Monday, December 11 2006

I really, really want to use TFS for Continuous Integration, but the more I get into it (it meaning TFS) the more I realize that TFS isn't going to cut it for CI, at least not for our purposes. It's apparent that the TeamBuild portion of TFS was almost an afterthought, which is a shame because it's such an obvious thing to have as part of the overall TFS solution. At least it's actively being worked on, so I expect full CI functionality in the Orcas release.

Even though we won't be using TFS for CI on this project, that doesn't mean you can't do CI in TFS at all. There are plenty of examples and work out there that explain how to set up TFS for CI, and there are already some tools on the market (i.e. TeamCI, Automaton) that are starting to address the issue, but so far they all fall short.

Here are the main reasons why we won't be using TFS for CI:
  • TFS cannot perform multiple builds at the same time
    • This is the biggest issue for us. With a few different builds setup to run all day long, not being able to perform multiple builds at the same time is a killer.
  • TFS builds are very, very slow, especially when compared to CC.NET
    • This is a bit surprising because both TFS and CC.NET are configured to use MSBuild, yet the TFS builds are consistently taking 10 times as long. Both are running on the same machine, running the same tests and code coverage, and performing the same static code analysis. Why the huge gap?
  • TFS doesn't show code coverage for classes within assemblies; just the assemblies themselves
    • Showing the overall code coverage for an assembly is good, but I need to drill down into it. For instance, if an assembly has 20 classes, I want to know which ones are below 90% coverage. There's no reason I shouldn't be able to see that in a report. I *can* do this with CC.NET, but that's another post soon to come.
It'll be nice once the TeamBuild part of TFS matures, but for now, we're going to be using TFS as the data and reporting store and will use CC.NET to drive all of our builds.

Similar Posts

  1. Decreasing Developer Ramp Up Time
  2. Yes, You Can Do Agile With VSTS
  3. TechEd 2005 Recap

4 comment(s) so far

jennifer wrote on January 23, 2008

this is fascinating

Steve Harman wrote on January 23, 2008

Speaking of Code Coverage... have you ever looked into using NCoverExplorer (along with TestDriven.NET) to easily integrate running your unit test suite and viewing the coverage results? TD.NET is a seamless plug-in for Visual Studio and makes running your test suite and viewing the results extremely easy.

Just be careful or you'll end up like Phil [http://haacked.com/archive/2006/12/14/Indulging_My_OCD_Using_TestDriven.NET_With_NCoverExplorer.aspx].

Steve Harman wrote on January 23, 2008

Shit... I forgot that you run on Subtext and I can post actual hyperlinks... so here is the link to Phil's post about TD.NET and NCoverExplorer.

<OfftopicRant>
Speaking of Subtext, when are you going to send me this skin... or upload it to SubtextSkins.com?
</OfftopicRant>

Dave wrote on January 23, 2008

Steve - Yep, I know all about TD.NET and NCoverExplorer. However, I use MSTest from CC.NET to run all my unit tests and code coverage, which works very well. A post on that is forthcoming.

As for this skin, I forgot about sending it to you, so I'm doing that now :-)

Post your comment

Comment