Build Server Hardware Makes a Difference

In my last post I showed how we currently have 24 builds running at any given time, with more builds certainly to come in the future. With that many builds our build server can get pretty taxed at times and depending on what gets checked in and where, we might have 4-5 builds running at the same time. So not only is CruiseControl.NET constantly running, but so is MSBuild, BitDiffer, FxCop, PowerShell, Selenium, and Demeanor (and soon NUnit and NCover).

Up to a couple days ago, our build server (which is a virtual machine) had 1 CPU with 2 GB of memory. That’s not beefy by any means, and constantly running all of those tools and processes kept the server pegged for most of the day. That’s not to say it was bad, but you want your builds to run as fast as possible, we we knew it was time for an upgrade.

So I opened a ticket with our netops team to double the processing power of the build server, requesting another CPU and an additional 2 GB of memory. With our build server being a virtual machine, the netops guys were able to perform the hardware upgrade in only a few minutes, and here’s a sample of the result for the main Community Server build:

Running Time Before Upgrade
pre-hardware-upgrade-CS

Running Time After Upgrade
post-hardware-upgrade-CS

The running time was basically cut in half, which is a big improvement. Understand that this isn’t the case across all of our builds nor does it hold true all the time between all of our build runs, but a cursory look through several before and after runs does show an overall decrease in running times for most of our builds.

This is pretty important in a Continuous Integration environment. The goal is to be notified ASAP if something breaks, and your builds are the first step in that process. But the longer your builds take, the longer your feedback cycle is. You don’t want your builds to be the bottleneck in your feedback loop.

I know many times teams use whatever machine they can get their hands on to act as the build server, and in many cases that machine is by no means a high-powered box. I’ve been there myself and will always go that route when something better isn’t available. But don’t overlook the difference that a bigger, faster, beefier build machine can make. The time saved from having faster builds can add up quickly.