When not all tests can be automated, then there are
ManualTests. This is an
AntiPattern because all tests can be automated and it is usually a waste of time to perform
ManualTests.
The other reason to perform
ManualTests is that you need to test the software as if you were the customer, for example to reproduce bugs that are assigned to you in the
TaskDatabase (or
BugDatabase). This is a
GoodThing.
It is usually the case that you build the software every day (
DailyBuilds) or several times a day (
ContinuousIntegration). Every day a successful build should be put on
StagingServer for
ManualTests. All bugs found in
StagingServer should be either entered into the
TaskDatabase (or
BugDatabase) or either do the
ExtremeProgramming practice of
CaptureBugsWithTests.
...all tests can be automated...
This may be true in some or even most programming domains, but I am currently developing J2ME games for cell phones and I must disagree most strongly. Testing games is already difficult; how do you write a test for "the graphics appear correct"? Add to that, on the phone you can't see stdout, and don't have nearly enough physical resources available to accommodate a test harness - your barely have what you need for your actual app. Even the emulators will model the severely limited heap size of the phones even if they don't restrict the jar size.
Of course, you should keep track of the
ManualTests that you perform anyway, so that you can run through the list (playing the game in an emulator)
as if the tests were automated.
See
AutomatedTest,
NoBugDatabase.