Unit testing seems to be one of those topics that will get people to arguing. What is a unit test? What value do they bring? How much time do they add to a project? You can’t write unit tests to test our code. And on and on… I think these questions continue to be asked because they are generally not answered well. To end this we need to find hard and fast answers to these questions. It is hard to argue for something if people can’t even agree on what it is.
u’nit test (noun) : An automated test, exercising a class, verifying that the class fulfills its responsibilities.
This is the absolute minimum needed for a definition of unit test. From here there could be many discussions about nice to haves with this definition (i.e. readability of the tests, naming of the test). Let’s examine this definition.
A unit test is not a unit test unless it is automated. If it is not automated then you must be using a debugger to step through a class to verify that it fulfills its responsibilities. This is a poor practice. It increases the viscosity of your environment; people will be less likely to perform this manual test every time it is needed. With automation thousands of unit tests can be run very quickly while thousands of manual tests will take a very long time. With out automation unit testing is next to worthless. The affect of unit tests is greater than their sum. To gain their full value all must be executed.
A unit test exercises just one class. What does this mean? Let’s say I have class Responder that needs to be tested. Responder has little public interface to speak of. The real meat is accessed by sending Responder messages through the file system. Verifying that Responder meets its responsibilities is also performed by receiving messages through the file system. Responder does not read and write messages to the files system itself, it uses a class called Tunnel. The temptation is to write a test that creates a Responder and rights and reads files to complete the test. Two thirds of what could go wrong is in the tunnels. There is more than one class being exercised. With a little redesign we can add some abstraction in to allow us to exercise just Responder. Exercising more than one class is an integration test.
A unit test will verify the subject class fulfills its responsibilities. As with exercising just one class a unit test is responsible to verify the responsibilities of just the subject class. When you are tempted to assert that an obligation of a class other than the subject is satisfied you are probably exercising more than the subject class. There should be unit tests for those other classes where their responsibilities are tested.
There are other levels of testing (integration, system…) where interactions are tested. Unit tests are mainly about testing the responsibilities of a class.
Naturally people feel exposed if the only tests on a system have tested the individual parts (other levels of testing should be exercised). What value is there in testing each part separately? It eliminates some of what you have to suspect when there is a bug. Unit tests have an affect on the code they test. The obvious affect is more reliable code. Some of the less obvious affects are loose coupling and ability to deal with change. This translates to the stakeholder as the project is more responsive to there needs as they figure them out. We all know that what is asked for at the beginning of the project and what is asked for by the end of the project can be very different things. Unit testing provides a means to balance these opposing forces: refactoring. It facilitates stepped, fine grained, or low level refactoring. With out true unit tests refactoing is a tricky business. In fact if your customer is asking for a change and you have good unit test coverage of your product the tests can be used to illustrate the risk involved with the change (i.e. before the change is applied X of Z tests are passing, after the change is applied Y of Z are passing).
If you are diligent there are other benefits to unit tests as well. Unit Tests can be, when written well, used as developer documentation. They can serve as executable design artifacts (similar to FiT tests). As a coach or architect they can be invaluable for identifying tangles, clutter, and bad or missing design.
When I here we can’t write unit tests for our code I think you don’t know how to use the tools of OOP. I have yet to come across a design problem that can not be solved with OOP to allow for simple and easy testing. It gets difficult when the subject code can not or will not be changed. Most times in these cases there is no simple easy way to create unit tests. Depending on the situation change may be necessary to create unit tests. Many times it is simply taking the time to do the right thing (i.e. creating an Object Mother). I think something that would go a long way to answering this question cold would be a set of patterns for common testing problems.
I think the most difficult question I have encountered is “How much time do they add to a project?”. I don’t think this question can be answered well in a general way. I think that for it to be affective it must address the specifics of the project(s) the questioner is thinking of. There are many variables and possible responses to the affects that unit testing will have on a project. Some developers get it right away, others take longer to understand how to practice TDD. It will reduce the amount of time spent debugging, and debugging is a laborious task. It will reduce the amount of bugs and recursion reaching the test team. The rapid feedback given to the developers should reduce the time needed to produce a feature or fix a bug. They no longer need to wait for the test team to perform verification to gain confidence that they have completed the task (and with the added bonus of not breaking anything else). The short simple (and some would say lame) answer is “Over the course of the project they will not add time but reduce the time needed”. The trick in the answer is “over the course of the project”. I would respond to that answer with “When will I first see the time savings?”. I am not sure I can win this line of questioning yet. It will invariably head to asking for a quantifiable or measurable difference: something tangible. The only way to kill this one is to find a way to make this question moot.
One of the interesting things about this last question is how it tempts me (and would guess others) to bring in other agile practices to the conversation. I am not sure if this is a good or a bad thing. There is defiantly a strong relation to other agile practices: refactoring, Continuous Integration, Continuous Design… At first I feel that their inclusion will bolster the case for unit tests and provide insight into their value. By including them in the discussion their value now seems to depend much on these other practices.