The original message on
http://groups.yahoo.com/group/TestFirstUserInterfaces/:
The TestFirstUserInterfaces Principles
GUIs and GUI testing provide a roadbump to those learning
TestFirstProgramming. Given a difficult combination of experience, tools, and
goals, folks meet with common problems and mistakes. This essay
addresses the problems and issues GUIs generate, and constructs a
strategy that ports to any GUI situation.
What is TestFirstProgramming?
Briefly, write failing tests, write code to pass them, and then
refactor the code to improve its design. After the fewest possible
edits - say ten at the most - run all the tests, and predict their
results. This technique makes the odds of bugs and the odds of excess
refactoring very low.
Why Test FIRST?
Why wash your hands before performing surgery?
TestFirst is a good habit
that trades long hours debugging for short minutes writing new test
cases. Prevention is better than a cure. If tests unexpectedly fail,
passing tests should only be an Undo away.
Why is TFP for GUIs naturally hard?
The target situation is this: We should be able to write new test cases
that force
predictable changes in a GUI's appearance and response.
All other libraries submit to program control, making prediction easy.
But GUIs have a side that only users can see or touch.
We will address this problem by enabling test cases that temporarily
present GUIs, under test, for inspection and interaction. This, in
turn, allows us to grow extra test code -
TestFixtures - to increase
the odds that we can predict GUI changes.
Why is TFP for GUIs artificially hard?
GuiToolkits have steep learning curves. This inspires toolkit vendors
to compete to bundle suites of helper applications - wizards, form
painters, debuggers, etc. - with their GUIs. A good GUI tool can
flatten a GUI Toolkit's learning curve.
So the
GuiLayer, within a large project, may suffer neglect from much
of a team's total experience. Because the backend of a project - the
LogicLayer - is mysterious, complex, and valuable, it often receives
more senior attention, and more careful development practices. When
leaders think that GUI development is "easy", they might not pay so
much attention to the growth of its design.
To put it bluntly, senior developers carefully write
LogicLayers, and
associate developers paint and debug GUIs.
GUI Toolkit vendors reinforce this culture. Their marketeers claim,
"Anyone can write simple forms with our system!" The fun starts when a
team attempts to scale these simple forms up to complex and
well-balanced applications. While most of the Logic Layers in our
industry suffer from the bad effects of
WaterFall, GUI development
labors in the salt mines of
CodeAndFix. Then GUI Toolkit vendors
reinforce the situation by providing elaborate and tempting wizards
and debuggers, to help you write code you don't understand, and then
to help you debug it until it seems to work.
How to avoid Test-Firsting a GUI?
The first line of defense, in any Agile project, is to avoid fixing a
potential problem until it can be successfully forced to the surface.
Engineers try to cause the next problem they intend to solve. If they
can't cause the problem, they are finished.
For GUIs, this strategy leads to the idea that as much of a GUI's
logic as possible should be written in a separate module, decoupled
from the GUI. We will observe this decoupling by following a very
simple rule, with a name. The part of an application that is shaped
like the GUI, but is decoupled from it, is the "
RepresentationLayer".
This term corresponds to terms in other papers such as "
LogicalUserInterface" or "
PresentationLayer". We use our
RepresentationLayer to
convert one format to another - the Logic Layer's format to the format
the user experiences. However, we can easily distinguish our
Representation Layer by applying a very simple and important rule:
A Representation Layer may not use any identifier from the GUI Toolkit.
If your GUI Toolkit provides, say, "Label", "Form", and "Pushbutton"
identifiers, your
RepresentationLayer must not use any of them. This
technique exposes your Representation Layer to the full benefit of
logical design techniques, such as TDD, without the encumbrance of GUI
testing.
Anything the user can do to a GUI, a programmer can do, the same way,
to a Representation Layer. If the user can scroll a list box and
select a name, then a programmer could write statements that fetch a
list of names with a method, and could select the index of one name
with another method. The Logic Layer may or may not keep any names in
any list; the Representation Layer re-arranges data to satisfy user
needs.
Anything the GUI can do to a user, the Representation Layer must do to
a programmer. If selecting an item from ListBoxA repopulates ListBoxB,
then the Representation Layer provides a callback or an Observer
Pattern event to trigger this repopulation.
When the
GuiLayer is very thin, it becomes humble (per
TheHumbleDialogBox). It only delegates
all responsibilities directly to the next layer.
Why bother Test-Firsting the GUI?
Why do we insist on doing the impossible? No matter how thin you think
your GUI Layer is, and how much logic you push out into the
Representation Layer, the simple fact is that bugs spontaneously
generate in code without tests. TFP will illustrate this effect by
pushing the defect rate down, allowing the testless parts of your
program to stick out like sore thumbs.
Many GUIs, including mine, will grow and live happy lives without
tests. We could easily follow a policy of "Ain't broke don't fix it."
We could decide not to write tests in for some of our modules, until
we need to. Unlike a
LogicLayer, faults in a GUI are trivially hard to find and fix.
Except if your GUI is advanced enough you must make custom controls for it.
We could decide to "
CaptureBugsWithTests", and only
write tests after we detect bugs. So by attempting to follow the Agile
aphorism "
YouArentGonnaNeedIt", we might defer labor hoping to
permanently avoid it.
That strategy has one major flaw, called "
ActivationEnergy". If a
module already has tests, new tests are very easy to write. The
activation energy is low. However, modules without tests make new ones
very hard to write. (Hence books on the subject, such as
WorkingEffectivelyWithLegacyCode by
MichaelFeathers.) The activation energy
is much higher.
When you receive a defect report, the energy required to "just fix it"
is usually lower than the energy required to retrofit tests, capture
the bug with test(s), and
then kill it. So each time you "just fix
it", and put off the labor, the next time gets easier. And more
likely.
Tests are the only aspect of
AgileDevelopment that you
are going to need.
All new modules, including GUIs, must start with tests. That's the
only way to ensure that test rigs grow, with their modules, and remain
ready to assist in all advanced development.
How to TestFirst a GUI?
Wizards, form painters, and debuggers are useful tools. We start TFUI
by making sure our lowly test cases can compete with them. We need to
spend more time among our test cases than among our vendor's tools'
GUIs, or our target application's GUI.
That requires giving our test cases a simple but useful GUI of their own.
To get there efficiently, without spawning a new corporate division,
we will grow this GUI only by responding to real problems in our
project. This system works by giving names to solutions for those
problems, and ensuring the solutions reinforce each other.
If you use an off-the-shelf GUI test system, you can easily lead it
towards this system. However, even the most limited GUI Toolkits can
grow this system from their primitive components.
OneButtonTesting
To test, keep your hands on the keyboard or mouse, and tap one
function key or button. The button invokes a test rig to save all
changed files, run selected tests, and report the results back, as
fast as possible, and with no further interaction.
This principle generates all the other principles. They are all
reactions to common reasons we might need to perform any other
interaction at test time.
Just Another Library
When you learn a GUI Toolkit by learning to use its form painter, you
may begin to think in terms of "GUI on the outside, source code on the
inside". To learn to take control of windows and forms
as objects,
we set a very simple goal:
During a test run, permit the fewest possible obnoxious side effects.
Run as many test cases as possible without displaying windows.
That's just a goal; it's not a rule. Trying to eliminate window display
generates many useful side effects. Most windows can react to display
commands without displaying their results on your desktop. When you
harness that effect, your tests will run quickly and quietly. And that, in
turn, leads to less hesitation before hitting the One Test Button.
RegulateTheEventQueue
To take control of a window, under test, you must learn how its event
queue dispatches messages. Some messages you must throttle -
especially the paint messages. And some messages you must let through,
such as a test case's messages that simulate user input.
Control over these aspects provides enough tools for us to start
giving our test cases a GUI. Write a function called 'reveal()', and
temporarily call it from test cases.
TemporaryVisualInspections
Turn on the Event Queue, during a test case, to visually inspect its
window populated by test data. The test run shall block until the
window closes, then subsequent tests shall run.
TemporaryInteractiveTests
While the GUI displays, use it to manually research
GuiToolkit behavior.
Those two practices provide our test case's GUI. The reveal() fixture
displays the window under test, populated with test data. That allows us
to spot-check our test case, and adjust the test-side support for the other practices.
Then we comment out the reveal() call, and ensure the test code works without visual
inspections.
And the ability to temporarily reveal() leads to a convenient platform
to record images of windows under test.
BroadbandFeedback
GUIs can record animations of activities during tests. Acceptance Test
Frameworks can command test cases to record a gallery of results.
The previous 6 Principles force the review of GUI appearance and
behavior into the tests, away from the wizards, form painters, and
finished application. They allow us to take control of the situation,
and prevent the need to excessively manually test.
Now we leverage those Principles to develop test fixtures (test-side
reusable methods) that enable predictable changes.
QueryVisualAppearance
GUI Toolkits provide the ability to query what they would look like if
you could see them. All such queries are simulations. Tests must learn
how to query that a GUI correctly obeyed the commands from its GUI
Layer code.
- Tests on Controls may Get details which Production code Set.
- Tests on Scripts parse the script, looking for details.
- Tests on Paint() events use Mock Graphics objects to generate a LogString.
SimulateUserInput
Test cases simulate events and check responses.
- Loose User Simulations: Call the same method as an event handler would have called.
- Firm User Simulations: Find the handler bound to an event and call it.
- Strict User Simulations: Push a raw input event into the event queue.
FaultNavigation
After a test run, the engineer can tell if a compile fails, or a test
fails, or the code faults, without pressing any other button besides
the One Testing Button. After a failure, the engineer optionally
automatically navigates to the failing line.
Flow
During tests, the editor remains available, activated, and unblocked.
GUI development depends on focusing on problems early, and
finding solutions for the intersection of your GUI Toolkit and your
target application.
Many of these ideas may seem obvious. However, for every GUI project that
spontaneously solved them, there is another that missed a few. The
idea that your editor should remain unblocked during a test run might
seem impossible to programmers familiar with, say,
VisualBasic 6.
Many of these ideas may seem
too obvious. Your
GuiToolkit might spontaneously solve one of these problems for you. For example, the
TkCanvas makes
QueryVisualState very easy, by treating graphic commands as objects. You should still relate the easy principles to the others. The
TkCanvas can still, occasionally, Set a value different than it can Get.
TemporaryVisualInspections will rapidly and easily catch these issues.
Put together, the strategy seems to resemble a paradox: To avoid manually reviewing GUIs, you must write a reveal() method that manually, temporarily reveals a GUI, under test. This is not a paradox; it is taking control of the situation. Wizards, form painters, and debuggers no longer control us. We control when and how to manually review our GUIs. When test cases temporarily present GUIs in their exact tested state, differences between those GUIs and the test assertions' opinions become easy to fix.
--
PhlIp
So,
PhlIp, how do you feel about
WaTir?