Automated Unit Testing On-The-Cheap: Part 1

C++ Code Capsules

When Extreme Programming was all the rage in the late 1990s, automated unit testing was still a relatively fresh idea for many working software developers. JUnit and CppUnit emerged as the leading test frameworks for Java and C++, but I wanted something even simpler—code that students could understand and that was easy to use.

I thought I’d achieved it in the well-received article I wrote for the C/C++ Users journal in September 2000, The Simplest Automated Unit Test Framework That Could Possibly Work.

But I was wrong.

A few years later, while teaching and developing examples at Utah Valley University, I found myself wanting something even smaller. How small could I go? Did I really need inheritance? My freshman C++ students didn’t learn object-oriented programming right away anyway, but they needed to learn the importance of crafting robust code from the get-go, regardless of paradigm.

The result was a simple header file: test.h.

Minimal Requirements

Individual tests need to be boolean expressions, of course, so successes will be noted and failures should be flagged with their location and the expression that failed. Code that generates exceptions should also be validated. Capturing the source text of an expression calls for the preprocessor’s stringizing operator (see #cond below). (For a deep dive into the macro preprocessor see The Preprocessor.) Here is the basic test function macro:

#define test_(cond) do_test(#cond, cond, __FILE__, __LINE__)

If the source line is

test_(stk.top() == 1);

then the preprocessor replaces it with the following text in the compilation stream:

do_test("stk.top() == 1", stk.top() == 1, "tstack.cpp", 17);

indicating that the name of the file is tstack.cpp and the call occurred on line 17 of that file. The function do_test tests the condition and responds accordingly:

inline void do_test(const char* condText, bool cond, const char* fileName, long lineNumber) {
    if (!cond) 
        do_fail(condText, fileName, lineNumber);
    else
        ++nPass;
}

inline void do_fail(const char* text, const char* fileName, long lineNumber) {
    std::cout << "FAILURE: " << text << " in file " << fileName
              << " on line " << lineNumber << std::endl;
    ++nFail;
}

If the condition failed, the following would be reported:

FAILURE: stk.top() == 1 in file tstack.cpp on line 17

Handling Exceptions

Code that expects an exception to be thrown under certain conditions can be verified by the following function macro:

#define throw_(expr,T)                      \
    try {                                   \
        expr;                               \
        std::cout << "THROW ";              \
        do_fail(#expr,__FILE__,__LINE__);   \
    } catch (const T&) {                    \
        ++nPass;                            \
    } catch (...) {                         \
        std::cout << "THROW ";              \
        do_fail(#expr,__FILE__,__LINE__);   \
    }

The Stack class I am using for this example wraps std::stack with member functions that throw std::underflow_error when calling top or pop on an empty stack (std::stack does not throw). The following code tests proper exception handling:

Stack<int> stk;

test_(stk.size() == 0);

// Test exceptions (top and pop are invalid on empty stack)
throw_(stk.top(), std::underflow_error);
throw_(stk.pop(), std::underflow_error);
nothrow_(stk.size());

The Code

For completeness there are also succeed_, fail_, and nothrow_ macros. There is also a report_ function. Here is the complete header file:

#ifndef TEST_H
#define TEST_H
#include <cstddef>
#include <iostream>
using std::size_t; 

// Unit Test Scaffolding: Users call test_, fail_, succeed_, throw_, nothrow_, and report_
// AUTHOR: Chuck Allison (Creative Commons License, 2001 - 2017)

namespace {
    size_t nPass{0};
    size_t nFail{0};
    inline void do_fail(const char* text, const char* fileName, long lineNumber) {
        std::cout << "FAILURE: " << text << " in file " << fileName
                  << " on line " << lineNumber << std::endl;
        ++nFail;
    }
    inline void do_test(const char* condText, bool cond, const char* fileName, long lineNumber) {
        if (!cond) 
            do_fail(condText, fileName, lineNumber);
        else
            ++nPass;
    }
    inline void succeed_() noexcept {
        ++nPass;
    }
    inline void report_() {
        std::cout << "\nTest Report:\n\n";
        std::cout << "\tNumber of Passes = " << nPass << std::endl;
        std::cout << "\tNumber of Failures = " << nFail << std::endl;
    }
}

#define test_(cond) do_test(#cond, cond, __FILE__, __LINE__)
#define fail_(expr) do_fail(expr, __FILE__, __LINE__)
#define throw_(expr,T)                      \
    try {                                   \
        expr;                               \
        std::cout << "THROW ";              \
        do_fail(#expr,__FILE__,__LINE__);   \
    } catch (const T&) {                    \
        ++nPass;                            \
    } catch (...) {                         \
        std::cout << "THROW ";              \
        do_fail(#expr,__FILE__,__LINE__);   \
    }

#define nothrow_(expr)                      \
    try {                                   \
        expr;                               \
        ++nPass;                            \
    } catch (...) {                         \
        std::cout << "NOTHROW ";            \
        do_fail(#expr,__FILE__,__LINE__);   \
    }        
#endif

Here is the driver for the Stack class:

// tstack.cpp: Test driver for Stack<T>
#include "test.h"
#include "stack.h"

int main() {
    Stack<int> stk;

    test_(stk.size() == 0);

    // Test exceptions (top and pop are invalid on empty stack)
    throw_(stk.top(), std::underflow_error);
    throw_(stk.pop(), std::underflow_error);
    nothrow_(stk.size());

    // Test push and top
    stk.push(1);
    test_(stk.top() == 1);
    test_(stk.size() == 1);
    stk.push(2);
    test_(stk.top() == 2);
    test_(stk.size() == 2);

    // Test pop
    stk.pop();
    test_(stk.top() == 1);
    test_(stk.size() == 1);
    stk.pop();
    test_(stk.size() == 0);
    throw_(stk.top(), std::underflow_error);
    throw_(stk.pop(), std::underflow_error);

    report_();
}

When executed the report_ function prints:

Test Report:

    Number of Passes = 13
    Number of Failures = 0

Issues

The code resides in the anonymous namespace, which is fine when only testing code in a single file. That was sufficient for my purposes those many years ago, but multifile projects will end up with separate counters for success and failures. That is easily fixed by using a named namespace with inline variables, but I wrote this pre-C++17.

There is also the issue of code bloat due to including code in multiple files. That’s what modules are for.

In Part 2 I will fix these problems, but the need for stringizing remains to capture the text of the expression, calling for a hybrid preprocessor/module approach.