- Have a continuous integration solution in place. Really. If you don’t, you just burn money by writing tests. I would go so far and say, if you don’t have continuous integration, you should stop writing unit tests and do click testing. Let your CI system generate API docs, high level docs, code coverage report, testdox and every statical analysis info you generate.
- The definition of “tests pass” is “tests pass on the continous integration system”. “Works for me” has neither a place in the bugtracker nor everywhere else.
- If you can’t test it, the architecture is most likely wrong (exceptions are sessions and caching related code which is generally hard to test). Testability should be your main concern when writing code. What’s the use of fast or wonderful looking code, if you can’t repeatable prove it is working?
- Prefer method calls over annotations. A typo in
setExpectedExceptionwill trigger a transparent error, while a typo inexpectedExceptionwill lead to Obscure Test, and most likely a Mistery Guest. - Run the whole test harness twice. This will hellp to identify setup/teardown bugs. Create a random test suite to identify the hard to track mistakes.
- Run your testsuite really often. We run it with 15 seconds delay every minute and I’m pretty happy with it.
- Use good test names that describe the behavior of the unit. The behavior is not the unit you test itself, that’s what I see in the code, it is something like “calling register changes the status of the user to foobar” so the good test name would be “testRegisterChangesTheStatus …”.
- Aim for 100% code coverage. 95% is nothing to be proud about, I can guarantee, the missing 5% will be the hardest part.
6
Comments
Show comments linear or threaded
Jonathan Street says:
published on 2008|09|25, 16:19hIt looks like you have some issues with apostrophes. Spam also seems to be an issue.
How do you handle pavatars, gravatars and all the rest of them? Is there a plugin that is available for s9y or is it custom coded?
I’m not clear on what you mean by, "Create a random test suite to identify the hard to track mistakes." I’d appreciate it if you could expand on what you think should go into that.
Lars Strojny returns:
published on 2008|09|26, 13:35hWhen I got lazy, I sometimes reused complex setups. I did the setups in the suite, not in the test and so on. Really, really bad behavior but … humans. One way to find that out (and finding that out means, breaking the tests) is to run the tests in random order. If the developer knows they are run in a random order he just can’t do invalid shared setups anymore because his tests will break.
Daniel O'Connor answers:
published on 2008|09|25, 17:21hphpunit —testdox is awesome as well for capturing what you are actually testing.
Bill Karwin says:
published on 2008|09|25, 19:43hThis is a great list!
I would also add that a great way to get tests written is to generate coverage reports from the CI tool and keep them updated. Show classes with low coverage in alarming colors. Encourage a spirit friendly competition among the team of developers to see who can keep their code coverage up throughout the development cycle.
I agree that 100% should be the ideal goal, and if you can’t test some line of code, then you should probably rearchitect until you can. But there are nevertheless some rare edge cases that cannot be tested easily. Is it really worth it to spend days or weeks to get that last 2.5%?
The real goal is to test the functionality adequately. Code coverage of 100% isn’t proof of adequate testing, any more than 95% code coverage is proof of inadequate testing.
I wouldn’t want people to design code architecture for the sake of test coverage, instead of for the sake of quality. There’s a difference!
Lars Strojny states:
published on 2008|09|26, 13:33hAs I sad, there are exceptions like caching, sessions and maybe others. But generally, it is mostly doable to achieve a really good code coverage without refactoring for the sake of code coverage. Refactoring for the sake of testability is a another topic, and often a good thing.
Lars Strojny reckons:
published on 2008|09|26, 13:39hAdded your point about code coverage. It would deserve it’s own point but I don’t want to change the title ;)
Add comment