Testing is a necessary evil to many programmers, but it doesn't have to be all that evil.
The title of this article is a variation on a theme from Extreme Programming [1], or XP for short. XP is a code-centric discipline for getting software done right, on time, within budget, while having fun along the way. Quit laughing [2]. The XP approach is to take the best software practices to the extreme.
For example, if code reviews are good, then code should be reviewed constantly, even as it's written. Hence the XP practice of Pair Programming, where all code is written by two developers sharing a single workstation. One programmer pilots the keyboard while the other watches to catch mistakes and give strategic guidance. Then they switch roles as needed. The next day they may pair up with other folks.
Likewise, if testing is good, then all tests should be automated and run many times per day. An ever growing suite of unit tests should be executed whenever you create or modify any function, to ensure that the system is still stable. Furthermore, developers should integrate code into the complete, evolving system and run functional tests often (at least daily).
You've probably seen the old cartoon with the caption, "You guys start coding while I go find out what they want." I spent a number of years as a developer wondering why users couldn't figure out what they wanted before I started coding. I found it very frustrating to attend a weekly status meeting only to discover that what I completed the week before wasn't quite going to fit the bill because the analysts changed their mind. It's hard to reshape concrete while it's drying. Only once in my career have I had the luxury of a "finished spec" to code from [3].
Over the years, however, I've discovered that it is unreasonable to expect mere humans to be able to articulate software requirements in detail without sampling an evolving, working system. It's much better to specify a little, design a little, code a little, test a little. Then, after evaluating the outcome, do it all over again. The ability to develop from soup to nuts in such an iterative fashion is one of the great advances of this object-oriented era in software history. But it requires nimble programmers who can craft resilient (i.e., slow-drying) code. Change is hard.
Ironically, another kind of change that good programmers want desperately to perform has always been opposed by management: improving the physical design of working code. What maintenance programmer hasn't had occasion to curse the aging, flagship company product as a convoluted patchwork of spaghetti, wholly resistant to modification? The fear of tampering with a functioning system, while not totally unfounded, robs code of the resilience it needs to endure. "If it ain't broke, don't fix it" eventually gives way to "We can't fix it — rewrite it." Change is necessary.
Fortunately, we are now seeing the rise of the new discipline of Refactoring, the art of internally restructuring code to improve its design, without changing the functionality visible to the user [4]. Such tweaks include extracting a new function from another, or its inverse, combining methods; replacing a method with an object; parameterizing a method or class; or replacing conditionals with polymorphism. Now that this process of improving a program's internal structure has a name and the support of industry luminaries, we will likely be seeing more of it in the workplace.
But whether the force for change comes from analysts or programmers, there is still the risk that changes today will break what worked yesterday. What we're all after is a way to build code that withstands the winds of change and actually improves over time.
Many practices purport to support this quick-on-your-feet motif, of which XP is only one. In this article I explore what I think is the key to making incremental development work: a ridiculously easy-to-use automated unit test framework, which I have implemented in C++, C, and Java.
Unit tests are what developers write to gain the confidence to say the two most important things that any developer can say:
1. I understand the requirements.
2. My code meets those requirements.
I can't think of a better way ensure that you know what the code you're about to write should do than to write the unit tests first. This simple exercise helps focus the mind on the task ahead, and will likely lead to working code faster than just jumping into coding. Or, to express it in XP terms, Testing + Programming is faster than just Programming. Writing tests first also puts you on guard up front against boundary conditions that might cause your code to break, so your code is more robust right out of the chute.
Once you have code that passes all your tests, you then have the peace of mind that if the system you contribute to isn't working, it's not your fault. The statement, "All my tests pass" is a powerful trump card in the workplace that cuts through any amount of politics and hand waving.
Writing good unit tests is so important that I'm amazed I didn't discover its value earlier in my career. Let me rephrase that. I'm not really amazed, just disappointed. I still remember what turned me off to formal testing at my first job right out of school. The testing manager (yes, we had one in 1978!) asked me to write a unit-test plan, whatever that was. Being an impatient youth I thought it was silly to waste time writing a plan — why not just write the test? That encounter soured me on the idea of formal test plans for years thereafter.
Automated Testing
I think that most developers, like myself, would rather write code than write about code. But what does a unit test look like? Quite often developers just verify that some well behaved input produces the expected output, which they inspect visually. Two dangers exist in this approach. First, programs don't always receive just well behaved input. We all know that we should test the boundaries of program input, but it's hard to think about it when you're trying to just get things working. If you write the test for a function first before you start coding, you can wear your QA hat and ask yourself, "What could possibly make this break?" Code up a test that will prove the function you'll write isn't broken, then put on your developer hat and make it happen. You'll write better code than if you hadn't written the test first.
The second danger is inspecting output visually to see if things work. It's fine for toy programs, but production software is too complex for that kind of activity. It is tedious and error prone to visually inspect program output to see if a test passed. Most any such thing a human can do a computer can do, but without error. It's better to formulate tests as collections of Boolean expressions and have the test program report any failures.
As an example, suppose you need to build a Date class in C++ that has the following properties:
- A date can be initialized with a string (YYYY-MM-DD), three integers (Y, M, D), or nothing (today's date).
- A date object can yield its year, month, and day or a string of the form "YYYY-MM-DD".
- All relational comparisons are available, as well as computing the duration between two dates (in years, months, and days), and adding or subtracting a duration.
- Dates need to span an arbitrary number of centuries (e.g., 1600-2200)
Your class could store three integers representing the year, month, and day. (Just be sure the year is 16 bits or more to satisfy the last bullet above.) The interface for your Date class might look like this:
// date.h #include <string> #include "duration.h" // a three-int struct class Date { public: Date(); Date(int year, int month, int day); Date(const std::string&); int getYear() const; int getMonth() const; int getDay() const; string toString() const; friend operator<(const Date&, const Date&); friend operator<=(const Date&, const Date&); friend operator>(const Date&, const Date&); friend operator>=(const Date&, const Date&); friend operator==(const Date&, const Date&); friend operator!=(const Date&, const Date&); friend Duration duration(const Date&, const Date&); void addDuration(const Duration&); };You can now write tests for the functions you want to implement first, something like the following:
int main() { Date mybday(1951, 10, 1); test(mybday.getYear() == 1951); test(mybday.getMonth() == 10); test(mybday.getDay() == 1); cout << "Passed: " << nPass << ", Failed: " << nFail << endl; } /* Output: Passed: 3, Failed: 0 */In this case you can assume that the function test maintains the global variables nPass and nFail. The only visual inspection you do is to read the final score. If a test failed, then test would print out an appropriate message. The framework described below has such a test function, among other things.
As you continue in the test-and-code cycle you'll want to build a suite of tests that are always available to keep all your related classes in good shape through any future maintenance. As requirements change, you add or modify tests accordingly.
The TestSuite Framework
As you learn more about XP you'll discover that there are some automated unit test tools available for download, such as JUnit for Java and CppUnit for C++. These are brilliantly designed and implemented, but I want something even simpler. I want something that I can not only easily use but also understand internally and even tweak if necessary. And I can live without a GUI. So, in the spirit of TheSimplestThingThatCouldPossibleWork, I present the TestSuite Framework, as I call it, which consists of two classes: Test and Suite. Test is an abstract class you derive from to override the run method, which should in turn call _test [5] for each Boolean test condition you define. For the Date class above you could do something like the following:
// DateTest.h: Use the test class #include "test.h" #include "date.h" class DateTest : public Test { Date mybday; public: DateTest() : mybday(1951,10,1) {} void run() { // The tests to run: testConstructors(); } void testConstructors() { _test(mybday.getYear() == 1951); _test(mybday.getMonth() == 10); _test(mybday.getDay() == 1); } };You can now run the test very easily, like this:
#include "DateTest.h" int main() { DateTest t; t.setStream(&cout); // must precede run() t.run(); t.report(); } /* Output: Test "class DateTest": Passed: 3 Failed: 0 */As development continues on the Date class, you'll add other tests called from DateTest::run, and then execute the main program to see if they all pass.
The Test class uses RTTI to get the name of your class (e.g., DateTest) for the report [6]. The setStream method lets you specify where the output will go, and report sends output to that stream. See Listings 1 and 2 for the definition and implementation of Test. No rocket science here. Test just keeps track of the number of successes and failures as well as the stream where you want Test::report to print the results. _test and _fail are macros so that they can include filename and line number information available from the preprocessor (which is not available in the Java version, of course).
In addition to _test, the framework includes the functions _succeed and _fail, for cases where a Boolean test won't do. For example, a simple stack class template might have the following specification:
// Stack.h ... template<typename T> class Stack { public: Stack(size_t) throw(StackError, bad_alloc); ~Stack(); void push(const T&) throw(StackError); T pop() throw(StackError); T top() const throw(StackError); size_t size() const; ... };Before giving any thought at all to implementation, it's easy to come up with general categories of tests for this class:
class StackTest : public Test { enum {SIZE = 5}; Stack<int> stk; public: StackTest() : stk(SIZE) {} void run() { testUnderflow(); testPopulate(); testOverflow(); testPop(); testBadSize(); } ...But to test whether exceptions are working correctly, you have to generate an exception and call _succeed or _fail explicitly, as StackTest::testBadSize class illustrates:
void testBadSize() { try { Stack<int> s(0); _fail("Bad Size"); } catch (StackError&) { _succeed(); } }Since a stack of size zero is prohibited, "success" in this case means that a StackError exception was caught, so I have to call _succeed explicitly. The implementation of the Stack class template is in Listing 3. StackTest and its results appear in Listing 4.
Test Suites
Real projects usually contain many classes, so there needs to be a way to group tests together so you can just push a single button to test the entire project. The Suite class allows you to collect tests into a functional unit. You add a derived Test object to a Suite with the addTest method, or you can swallow an entire existing Suite with addSuite. To illustrate, I have four modules that work together to provide real-world date support. JulianDate is what I call an API of C functions that implement the basics of Julian Date arithmetic [7]. The JulianTime module uses JulianDate but adds support for hours, minutes, and seconds. Date uses JulianDate and adds various niceties for C++ users. The same relationship exists between Time and JulianTime. Since these classes are all interrelated and are packaged as a single library, I want to test them together. After defining each test class (derived from Test, of course), I group them into a suite entitled "Date and Time Tests." Here's an actual test run:
// test Suite for the Date projects #include <iostream> #include "suite.h" #include "JulianDateTest.h" #include "JulianTimeTest.h" #include "DateTest.h" #include "TimeTest.h" using namespace std; int main() { Suite s("Date and Time Tests", &cout); s.addTest(new JulianDateTest); s.addTest(new JulianTimeTest); s.addTest(new DateTest); s.addTest(new TimeTest); s.run(); long nFail = s.report(); s.free(); cout << "\nTotal failures: " << nFail << endl; return nFail; } /* Output: Suite "Date and Time Tests" =========================== Test "class MonthInfoTest": Passed: 18 Failed: 0 Test "class JulianDate": Passed: 36 Failed: 0 Test "class JulianTime": Passed: 29 Failed: 0 Test "class Date": Passed: 57 Failed: 0 Test "class Time": Passed: 84 Failed: 0 =========================== Total failures: 0 */Suite::run calls Test::run for each of its contained tests. Much the same thing happens for Suite::report. Individual test results can be written to separate streams, if desired. If the test passed to addSuite has a stream pointer assigned already, it keeps it. Otherwise it gets its stream from the Suite object. The code for Suite is in Listings 5 and 6. As you can see, Suite just holds a vector of pointers to Test. When it's time to run each test, it just loops through the tests in the vector calling their run method.
No C++? No Problem!
After I showed TestSuite to the developers where I used to work, a number of programmers were a little chagrined that they couldn't use it, because they were developing strictly in C. It took all of one afternoon to change all the classes to structs and rename things accordingly to come up with a C version. A simple test example is in Listing 7. The Java version was even easier, of course. You can get all the code on the CUJ web site (www.cuj.com/code).
It takes some discipline to write unit tests before you code, but if you have an automated tool, it makes it a lot easier. I just add a project in my IDE for a test suite for each project, and switch back and forth between the test and the real code as needed. There's no conceptual baggage, no extra test scripting language to learn, no worries — just point, click, and test!
Footnotes
[2] "Stop laughing," are Kent's own words. See ibid, p. xvi.
Chuck Allison is the owner of Fresh Sources, a company specializing in object-oriented software development, training, and mentoring. He has been a contributing member of J16, the C++ Standards Committee, since 1991, and is the author of C and C++ Code Capsules: A Guide for Practitioners, Prentice-Hall, 1998. You can email Chuck at chuck@freshsources.com.