Adding A Little Ambiguity To Your Plan

I was just talking with coworkers about the pros and cons of test plans with expected results. I tend to organize my testing with a series of questions to investigate, and or a rough map (often in .xls format) of areas to explore. Here we have some very well thought out test cases, including clear steps to follow and clear expected results.

Cem Kaner refers to having a test script with expected results as inattentional blindness by design. I agree wholeheartedly. In discussing this, an analogy occured to me to a 2004 Wired article about Hans Monderman, a Dutch traffic planner who has succeeded in making roads safer by making what to do more ambiguous. Here they discuss his solution to a frantic intersection that was the site of many accidents:

It’s the confluence of two busy two-lane roads that handle 20,000 cars a day, plus thousands of bicyclists and pedestrians. Several years ago, Monderman ripped out all the traditional instruments used by traffic engineers to influence driver behavior – traffic lights, road markings, and some pedestrian crossings – and in their place created a roundabout, or traffic circle. The circle is remarkable for what it doesn’t contain: signs or signals telling drivers how fast to go, who has the right-of-way, or how to behave. There are no lane markers or curbs separating street and sidewalk, so it’s unclear exactly where the car zone ends and the pedestrian zone begins. To an approaching driver, the intersection is utterly ambiguous – and that’s the point.

The general goal is to keep the driver’s brain engaged. If they enter the intersection thinking “I know just what needs to be done here”, they risk running on autopilot. If they enter and think “Whoa. This is a bit confusing, I better pay attention!” there are a lot less accidents.

I think this has clear parallels to testing. If we go to a section of the application and think “Ahh. I know just what needs to be checked here. I’ve followed this script before.” we are much more likely to miss the elephant in the middle of the room.

To quote Monderman,

“The trouble with traffic engineers is that when there’s a problem with a road, they always try to add something,” Monderman says. “To my mind, it’s much better to remove things.”

In many cases, I couldn’t agree more.

[Edit, 2007, June 1]

I originally titled this “Why You Should Make Your Test Plans Less Clear”, which at the moment seemed like an interesting teaser/title. I just changed it to “Adding A Little Ambiguity To Your Plan”. Why? “…You Should…” was meant tongue-in-cheek, but I read it again today and it irked me. Who is this yahoo telling me what I should do? Oh yeah, it’s me. And so I changed the title.

Entering the Metaweb

I am two weeks into a new position as Senior QA Engineer at Metaweb. The current plan is for me to lead testing of Freebase, our client application (currently in an invite-only Alpha). It was a bit surreal of a decision to make. It meant leaving the most fun work environment I’ve ever had – and after only 6 months, something I’ve never done before – but the opportunity to be a part of what Metaweb is trying to do is too rich an opportunity for me to pass up.

And what are we doing that’s so exciting? In short, we aim to organize the world’s knowledge – in ways that are more meaningful to machines than pages of text. We hope that Freebase will compliment the fantastic ways that Google and the Wikipedia already help us to access the information we need – solving certain problems that are challenging using current options.

You can read more about what we’re doing at the NY Times, at Tim O’Reilly of O’Reilly Books’ blog here and here, or listen to an in depth interview of our “Minister of Information”.

I am only a few weeks into the job. One of my first impressions (besides the usual excited blur as I learn a new application and explore a new domain) is that it’s very refreshing to come to an environment where there are already experienced testers. The last two places I’ve worked, I’ve enjoyed getting to create the test processes from scratch. Now I’m enjoying learning from what others have established…and providing thoughts and ideas from the perspective of an outsider coming in to an existing organization, rather than building from scratch.

Limeuristics

We’ve read James’, Mike’s and Jonathan’s compelling arguments for using mnemonics to keep useful testing heuristics at the tip of our tongue. But where, I ask you, can one find heuristics that rhyme?

In anticipation of National Limerick Day (May 12th), I humbly suggest using Limeuristics.

I’ve written a handful to get us started:

There are those who say testing’s a bore
And the moment they start it they snore.
Now my secret I’ll tell:
If you learn to test well
You’ll discover a thirst to explore.

A tester I know (who loves bisquits) –
Her peers thought her gifted by mystics.
“You’ve got it all wrong,”
She’d explain to the throng.
“It’s just that I use good heuristics”

Certifications are always a-fruitin’
To make resumes highfalutin’.
If they don’t certify
What they try to imply –
Our craft, I’m afraid they’re pollutin’

Every tester at times gets contusions
When their findings leave egos with bruisin’s.
But don’t say we’re unkind
For the things that we find
We must test to dispell our illusions

…and finally, with thanks to Chris McMahon and “the man from Japan” for the inspiration:

A limerick ’bout testing sublime
Should have meter and rhythm and rhyme
But I might bend a rule
While a bug I pursue
So don’t be surprised if I decide to add a boundary test into the last line.

Have a very happy National Limerick Day. If you have a Limeuristic you’d like me to add to the list, I’d happily add yours as well.

Cheers!