Writing unit tests is not just a formality. Unit tests are as important to an application as the code that runs it. Without them, code is fragile. A twentieth parameter is added to a common function, with yet another conditional added to the end, as backwards compatibility must be maintained and regressions must be avoided. The same bug is fixed for the fifth time this month. The codebase devolves into a house of cards, where even senior developers can’t make significant changes with confidence. To avoid this dystopian future, start writing unit tests for your code with diligence. In this post, we will outline a basic set of guidelines for writing them that will allow you to improve the stability, reliability, and maintainability of your application.
When to write them
Knowing when to write unit tests is the first step to getting better at writing them. There are a lot of differing opinions about this in the industry, but I don’t think the details matter too much, as long as you meet the following criteria.
All new code is committed with unit tests
The best start to have a well-tested application is to enforce that all new code comes with unit tests, whether you are working alone or as a part of a team. This is also the easiest time to write tests, as the code is fresh in your mind, and can also be written to be easier to test.
Bug fixes start with a new, failing unit test
Bugs often indicate a gap in your testing. This is the ideal time to write a unit test (or tests) to cover this case, so that you never have to worry about it again. You do this by writing it before the fix, ensuring that it fails, applying the fix, and ensuring that it passes.
Refactored code has passing unit tests
Before refactoring code, make sure it is well-tested. Ugly, working code is better than pretty, broken code.
How to write them
Writing thorough unit tests can be difficult. It can seem like a brain puzzle, trying to think of as many edge cases as you can. However, there is a more methodical approach. If you follow these guidelines, you will end up with a good set of tests.
Test only one thing
This is a tenet of unit testing. Your tests should test one thing only, and it should be clear (via proper naming and/or comments) exactly what that is. That does not mean you can’t have more than one assert statement. You may, for example, be testing an HTTP response. In that case, you may need to assert that the status code is correct and that the response body matches what you expect. On the other hand, testing this on several endpoints in the same test is an obvious violation.
Write code with testing in mind
The easier your code is to test, the better your tests will be. At a high level, this means breaking your code into small units that do one thing and keeping coupling to a minimum. Simply thinking about how you will test code as you write it will result in a major improvement. We covered this from a Django perspective in Tips for Testing Django Views.
Test the interface (black-box testing)
Black-box testing is writing tests without any regard for how the code is actually written. Essentially, this means you write tests with only the interface in mind; not the implementation. You focus on the inputs and outputs of the code; not what is in between. This is beneficial because the alternative (white-box testing) can result in tests that are limited in scope. If you struggle to write tests in this manner, you can try Test-Driven Development (TDD), where you write tests before you write the code.
Testing the interface essentially involves testing different types of input your code should (or should not) accept and verifying the correct result occurs. The extent to which you should test the interface depends on what kind of function it is.
Test valid input
Test valid input to ensure your code produces the correct result. There may be several different types of valid input that could or should be expected, so make sure you hit each case. For example, a function may accept a number as an integer or a string. Both cases are valid, so make sure they are both covered.
Test invalid input
Testing invalid input can depend on what type of function you are testing.
Does it accept user input or is it part of a public API? Then invalid input should be tested thoroughly, as there is really no limit to how wrong (or malicious) the input can be.
Is it strictly a private, internally-used function? Then some classes of invalid input probably don’t even need to be handled by it, as the caller is expected to pass something that at least resembles valid input (e.g. the correct type). If the caller violates that expectation, that would ideally be caught by its unit tests.
Something to keep in mind when testing any kind of input is to test thresholds. This means testing invalid input that is as close as possible to being valid, and testing valid input that is as close as possible to being invalid. Imagine you have a function that accepts only positive numbers (excluding zero). To test valid input, one of your tests should use the value
1, as it is the number right on the threshold of being negative (or zero). Likewise, for testing invalid input, you should use the value
-1, as it is the first negative number. Since
0 is generally treated as a special value, you should write a test for it as well.
This is beneficial because, combined with the next guideline, we can test an entire range of input without having to test each individual value.
As a counterpart to testing thresholds, we should also be testing extremes. Extreme input is the maximum/minimum value that should be considered valid. The benefit of this is twofold. One of the most common pitfalls of writing code that handles input is to not handle extreme values properly. Secondly, combined with testing thresholds, we can be relatively certain that all values in a range are covered without testing each value.
Test the code (white-box testing)
The counterpart to black-box testing is white-box testing, where you write tests based on how the code is written. This is beneficial, because you are able to more precisely identify edge cases, and you can ensure that each execution path is being tested.
Hit edge cases
By looking at your code, you should be able to more easily identify potential edge cases. For example, if a function is performing mathematical operations, does it handle integer overflow? This is your opportunity to think about the technical details and limitations of your code and put them to the test.
Hit every conditional
Every conditional in your code causes a branching of execution paths. It is important to make sure each of these are covered. To aid you in keeping track of this, I suggest using a code coverage measurement tool as you write tests. Aim for 100% coverage on new code.
If your application uses any external libraries or makes external API calls, it can seem a tall order to write tests for code that uses these. On the contrary, it is actually extraordinarily simple. Every major programming language has a mocking library, which allows you to replace objects or functions with mocks that have customized behavior.
Mocks are powerful. At its simplest, you can simply check to see if a mock function was called. For more advanced testing, you can check what arguments it was called with, check how many times it was called, make it produce a side effect (e.g. an exception), make it return a certain value, and much more. With all of these tools at your disposal, you should have no problem testing any function.
Don’t write too many
You may think there is no such thing as too many tests, but think again. An increasing number of tests per unit of code has a diminishing return, in that it creates additional overhead for any future code changes. This overhead comes in the form of tedious maintainability and long execution times. Following the guidelines above, you should be able to come up with a reasonable number of tests that have broad coverage. If I had to come up with a number, I would say around 10-15 tests per function should be the upper limit. Beyond that, you should either look at breaking up the function into smaller ones, or use the guidelines above to test more with fewer tests.
By being mindful of these guidelines, you can write better, more comprehensive unit tests. Anyone can make code changes with more confidence, regressions can be permanently eliminated, and you can be much more confident that your application is stable and accurate.