Dave Donaldson

Unit Testing and Code Coverage Guidelines

Friday, March 03 2006

These thoughts below are based on my experiences with unit testing and code coverage. I don't claim that these apply to everyone and/or every software project, but I think overall they are pretty good for most projects (if I didn't think it would be helpful I wouldn't have bothered blogging it). I'd love to get your feedback on this, so please leave a comment if something catches your fancy.

Test Data

To ensure quality unit tests are written and always passable (that's a key point), most test data should be created at the start of a unit test and deleted once the test is completed. This allows for each unit test to setup and tear down its own test data independent of other unit tests, which allows for an extremely repeatable process and greater number of passable unit tests.

However, sometimes there are exceptions where a unit test will require data to already exist in the database. If you have that case, try to script the data so it's at least automated in some form. This helps alleviate manual data entry of test data.

Write Unit Tests For Each Layer

More often than not applications are broken into several layers that include the UI, business logic, data access, service agents, etc. Each layer should have its own standalone set of unit tests so that the layer can be properly tested independent of any other layer. Meaning, layers should not depend on getting its test data from another layer. The unit tests for each layer must maintain their own sets of test data.

There will be instances when a method is nothing more than a pass-through from one layer to another. In this situation the pass-through method should still have its own set of unit tests. Often times this leads to duplicate unit tests in different layers. Some people cringe at this, but I think it's perfectly acceptable. Afterall, duplicate tests are not the same thing as duplicate implementation code. And who can argue with more tests?

Run Unit Tests For Each Layer by Themselves

This is important because it effects your code coverage metrics. Meaning, if you only ever run unit tests end-to-end with layers passing test data to other layers, it’s highly possible your code coverage statistics will be misleading (i.e. higher than it really is). To ensure your code coverage is at acceptable levels, you must run unit tests for each layer by themselves. This forces you to write unit tests for each layer, which in turn will require test data for each layer.

Run Unit Tests With All Layers Together

This is the end-to-end scenario mentioned above, which is important because it allows you to see how your unit tests perform when all layers are running together. If your code coverage is good when running each layer by itself, then it should also be good when running all unit tests from all layers together.

At Least 1 Unit Test Per Public Method

How do you know if you’re off to a good start with regards to code coverage? Answer: If you have at least one unit test per public method. This is how all test-first development should start because when writing unit tests before implementation code you always end up with at least one unit test per public method. Put another way, when practicing test-first it’s impossible to NOT start this way.

However, this begs the question: What is the first unit test a developer should write for a given use case? Answer: Usually the happy path (good inputs, good outputs); however, sometimes it’s easy to see which exceptions need handled so you might write an exception test first. But by and large the first unit test a developer will write for a use case will be for the happy path.

Write Unit Tests For Each Alternate Path

Writing unit tests for the happy path is the easy part. The trickier part is how to test all those other scenarios in the use case, typically referred to as alternate paths. Alternate paths are actually quite easy to recognize in code. They are the If…Else statements and the Switch…Case statements. Each check in an If…Else statement or Switch…Case statement requires a corresponding unit test.

The importance here is again code coverage. Having unit tests that hit the alternate paths of a use case will greatly increase your code coverage.

Write Unit Tests For Each Exception

This is the single most overlooked part of writing unit tests. All applications throw exceptions and hopefully you’ve written code that handles them appropriately. But how can you be sure you’ve handled them appropriately if you haven’t tested them? The unit testing frameworks (Junit, Nunit, VSTS, etc) include attributes that make it extremely easy to test your exceptions. And writing a unit test for an exception is actually the easiest of all unit tests to write.

And again, having unit tests for exceptions further raises your code covergage.

Code Coverage Levels

When I talk to developers about code coverage, I always get the question “What is an acceptable percentage for my application?“ I usually give the prototypical answer “It depends“, but then go on to say that I believe a good standard is 90% and higher. It is very difficult to achieve 100% code coverage in any software application, but my experience has shown that at least 90% code coverage is the sign of a well-tested application. If any layer in the your application is below 90% code coverage, more unit tests should be identified and written to make up the difference.