Embed Doc
Copy Link
Readcast
Collections

CommentGo Back

Download

Benefits of Unit Testing

Introduction

Writing software is very difficult. Writing bug-free software all the time is virtuallyimpossible. There are many reasons why this is so; anyone who has written anuntrivial piece of software knows how easily a human can miss a tiny detail, andhow the slightest of omissions can cause a most spectacular crash, most likely atthe most awkward moment.Unfortunately, the task of writing software promises to be no easier in the future.While software technologies do advance, so does the complexity of the softwarewe write. There is no magical trick in sight in the foreseeable future that wouldmake writing bug-free software simple and straightforward. Still, not all hope islost - there are some things we can do to substantially improve the quality of software we write.

Ad-hoc Testing

The development of new features to a software product goes in phases. Nomatter what the methodology, the developer usually starts with the requirements,proceeds to design, then writes the code, debugs it, and finally checks the codeinto the source code version control system. At this point, the feature isconsidered done. But how does the developer know that the code actually works?In many smaller companies and companies that are not really software housesbut still develop software for their own use, the developer spends a day or sotesting the feature, simply running the application and trying out the functionalitywith different inputs while observing that the outputs come out as designed. Afterhaving completed this testing, the developer has some confidence in that thecode works the way it is supposed to. But every developer who has ever writtencode to an actual product knows the nagging feeling of uncertainty. How good acoverage did the testing have? Did I test every single feature in the application –otherwise how can I be sure that the new code did not accidentally break someother, apparently unrelated, piece of functionality? What if new code written bysomebody else two weeks later will adversely affect this feature?Using developer time for manual ad hoc testing is not a cost effective way of ensuring software quality. Developer time is expensive and is better spent onwriting software rather than test-running it ad nauseum. This kind of testing findsout whether the new code affected some other piece of functionality only if thedeveloper tests every single feature in the application. Further, it gives noguarantee whatsoever that the feature still works two months down the road asmore features have been added to the system.

Quality Assurance

Some companies have a more or less formal quality assurance process with ateam dedicated for testing. The team may have test case scripts that the teamgoes through and manually tests the software. This activity is carried out afterevery internal release of the software. Customers will see a new version of thesoftware only after the tests pass and the QA team leader has signed it off.

This kind of testing is better than ad-hoc testing but it is far from optimal. Humantesters can and will make mistakes. While hired tester time is cheaper thandeveloper time, the cheapness is offset by the fact that the problems areuncovered relatively late in the development cycle. Every time QA acceptancetests fail, the developers get notified; they fix the problems and give a newversion of the software to QA. Time and energy is wasted on communication whilethe software bounces back and forth between the developers and the QA team. Inaddition to wasted time and energy, this trashing can have side effects such ascreating a ‘ghetto’ mentality in QA personnel.Quality assurance teams may also have automated testing systems. This is moreeffective than manual testing, but still is not the optimal situation. Test scriptingsystems tend to be expensive. Writing good test scripts and maintaining them issomething that a tester who was hired for a summer job is not likely able to do.Hiring someone who can write and maintain tests will likely cost as much ashiring a developer in the first place.

Automated Unit Tests

I am not going to describe how to write unit tests in this article. There are alreadyarticles on this site that can get you started. Suffice it to say that writing costeffective unit tests is not trivial. It is just like writing any other type of software:it takes time, effort and skill. For now, I’m just going to take the easy way outand list some very basic guidelines for writing and running unit tests.A unit test is a piece of code that exercises another piece of code. There areessentially two flavors of unit tests: point unit tests exercise a small piece of functionality, e.g., just one method or methods of a class. End-to-end testsexercise one feature across many, sometimes all, layers of an application. Anideal end-to-end test works like an end user story: "the user logs in, invokes themonthly revenue report, sets the date range of the report from July 1st to July31st, runs the report and observes that all the numbers come up as expected andadd up". This is a fairly liberal definition of a unit test; for the purposes of thisarticle anything that you would write e.g., as a NUnit test case is a unit test.When implementing a feature, a developer should write the code together with aunit test that ensures that the feature actually works. In practice this meanswriting code that e.g., calls an entry point method of the feature and then verifiesthat the code did what it was supposed to. For example, if you have a methodwhich transfers money from one account to another, the unit test would call thatmethod with accounts with known balances and then verify that the balancescome out the way they are expected to.When the feature and the unit tests are complete, the developer checks in thesource code - including the unit test code - to the source version control system.It is vital that the unit test code is checked in to the version control too. Thisallows integrating the unit tests with the build. It also allows other developers tocheck out the unit tests on their local machines and run them at will.As a rule, before a developer checks in any code into the main source controlrepository, she should get the latest versions of the unit tests from the versioncontrol on her development machine and run all the tests locally. Only after allunit tests pass should the developer check in new code. This ensures that thesource code in the version control system remains healthy at all times and can beused to rebuild the system at any time.

In order to get the full benefit from unit tests, the test suite should be run as partof the build process – this is what I mean by ‘automated’ unit tests. The buildalong with the unit tests should be scheduled to run automatically once or twice aday. If you have unit tests but do not run them as part of a scheduled build, thenyou are not getting the full benefit from the tests. By running the tests as part of a scheduled build, you test early and often. This again ensures that the sourcecode in the main repository remains healthy. In real life, developers mayaccidentally check in code that does not work. The daily build should be yourmain line of defense against these bugs. There are some wonderful and cheaptools for implementing build and test systems, e.g., if you are developing for.NET, you can use NAnt and NUnit frameworks. How to implement a build processis again a fairly large topic and resources on this site and elsewhere on the webcan get you started.Writing unit tests is an extra effort if compared to not writing tests at all, and thisshould not be ignored. In my experience writing unit tests adds roughly 10-30%to the time it takes to complete a feature. End-to-end tests tend to be the mosttime consuming type of tests to write. Then again, when writing end-to-end testsone usually very quickly creates a small framework as the common code in thetests is factored out. Thus writing end-to-end tests comes easier after the firstfew test cases have been written.Both types of tests - end-to-end and point tests – are needed for best coverage.End-to-end tests find different kinds of problems as they exercise a number of components that participate in the implementation of a feature. Point unit testsare also needed for the most important components because they can verify thefunctionality of a component more thoroughly.

Benefits of an Automated Unit Test Suite

An automated unit test suite brings along a number of important, tangibleadvantages as compared to other testing strategies.First,

unit tests find problems early in the development cycle

. The commonwisdom in software development is that the earlier problems are found, thecheaper it is to fix them. An automated unit test suite finds problems effectivelyas early as possible, long before the software reaches a customer, and evenbefore it reaches the QA team. Most of the problems in new code are alreadyuncovered before the developer checks the code into source control.Second,

an automated unit test suite watches over your code in twodimensions: time and space

. It watches over your code in the time dimensionbecause once you’ve written a unit test, it guarantees that the code you wroteworks now and in the future. It watches over your code in space dimensionbecause unit tests written for other features guarantee that your new code didnot break them; likewise it guarantees that code written for other features doesnot adversely affect the code you wrote for this feature.Third,

developers will be less afraid to change existing code

. Over time,software systems become more and more change resistant because developersare reluctant to change old code. This is natural because when changing old code,there is always the risk of breaking it or some other part of the system through aside-effect.