Larry O'Brien recently wrote a a post about relative sizes of tests vs. the code they exercise wherein he mentions an interesting concept: tests as quality diodes. For those of you who've forgotten how diodes worked, or never learned in the first place (lucky you), here's a decent introduction to diodes.
The immediate point of the metaphor was obvious - code that passes tests is quality, code that fails tests is not. But once upon a time, I was expected to know about the inner workings of diodes (thanks, ECE 3040), so I've taken the metaphor and run with it. To start with, I'll be comparing entire test suites to a single diode.
Warning: The following paragraphs contain scenes of graphic metaphor-stretching. A metaphor may have been harmed in the writing of this post.
One of the distinguishing characteristics of diodes is known as peak inverse voltage. If the voltage across a diode is going the wrong direction (a situation called reverse bias) and exceeds the diode's peak inverse voltage, then you get an unpleasant effect known as avalanching. This is pretty similar to what happens when you try to slap a test suite on a lot of traditional applications. Test suites are traditionally expected to be passing most of the time, and the code they exercise is supposed to be more good than bad (or it wouldn't be passing the tests most of the time). If instead you have more bad code than good, you've reverse-biased your quality diode. If there is pressure to get this dev-task done and get back to work—after all, it's not like you're adding a feature to the product—then you may have just exceeded your test suite's peak inverse voltage. The resulting avalanching tends to gut your test suite, either because you stop running it (you already know it will have failures) or because you throw it out entirely (why maintain code that isn't doing anything for us).
What should we do about a situation like this? In the world of electronics, one solution is the Zener diode. Instead of avalanching, it breaks down in a more controlled fashion, though it usually does so at much lower inverse voltages. We'd like our test suite to exhibit this same resistance to avalanching—a Zener test suite, so to speak.
One way to build a Zener test suite would be to reduce your test coverage. Sure, the first casualty will likely be the code that needs the most help, but some test coverage is better than none, and none is what you wind up with if your test suite starts avalanching. Once you have your Zener test suite in place, you can work on transforming it into a more traditional test suite by slowly increasing the coverage. That's not entirely accurate, though: if you do it right, your test suite will grow to have the benefits of the classic test suite while retaining the resistance to avalanching of the Zener test suite, and that's our real goal.
Back to flipping out...
