TacticalTestingFramework

A tactical testing framework is a framework which supports what I am calling TacticalTesting. In my opinion, this is not revolutionary stuff, it's evolutionary from existing TestingFrameworks. There are still ideas from TacticalTesting that haven't found a home. BTW, I haven't made a TacticalTestingFramework, this is my ideas on what I would want from a testing framework.

The testing framework is an environment which supports UnitTesting for a piece of code called a CodeUnit. This piece of code has been written to support the testing framework, so it has ClassInvariant testing coded in, and uses a BugReportingFramework. The tripartite inner box on the left is all the code under test, though the BugReportingFramework is a standard module which has been linked in.

The CodeUnit is not a trivial nor deterministic piece of code - it has an interface to an ExternalSystem which may be a CORBA server, database, real-time data feed, I dunno. The CodeUnit also has a public interface, which is the interface through which users will typically use the system. (I am trying to figure out whether the external interface and the public interface have any important differences - are we testing just the public interface or both of them?)

[an image may be missing here]

The pink arrows denote data flow through the system. The testing framework is responsible for the ends of any arrows which nobody else owns. Because no arrows leave the outer box, the testing framework has complete control of the environment of the code unit under test.

The testing framework is responsible for:

• faking the behaviour of the external interface
• running the tests which exercise the public interface
• helping to check the output of the public interface
• invoking the class invariant to check consistency
• showing bugs found in the tests, and also those reported to the bug reporting system as side effects of the tests

The code unit must:

• implement a code invariant (optional, really)
• reporting bugs it detects to the bug reporting system, whether they affect behaviour or not
• accepting fake implementations of external interfaces

The tactical testing techniques used in this framework are:

• CodeClassInvariants
• ReportBugsSilently - the bug reporting system is the place to which you report bugs so they don't get sent to /dev/null when the user redirects standard output.
• KeepErrorInformation - the bug reporting system supports RichBugReports, which contain all information known to be relevant to a particular problem.
• CheckOutputAutomatically - the tactical testing framework supports checking of output for many sorts of output.
• CallAndCheckResult - standard behaviour for a testing framework.

-- JohnFarrell

---

CategoryTesting