The Logic of Failure, Part 1

I’ve been reading Dietrich Dorner’s The Logic of Failure recently. I am not quite done and will post more soon, but there are a few quotes that I keep coming back to. Here’s one:

An individual’s reality model can be right or wrong, complete or incomplete. As a rule it will be both incomplete and wrong, and one would do well to keep that probability in mind. But this is easier said than done. People are most inclined to insist that they are right when they are wrong and when they are beset by uncertainty. (It even happens that people prefer their incorrect hypotheses to correct ones and will fight tooth and nail rather than abandon an idea that is demonstrably false.) The ability to admit ignorance or mistaken assumptions is indeed a sign of wisdom, and most individuals in the thick of complex situations are not, or not yet, wise. (p. 42)

Dorner gives a fascinating analysis of the reasonable decisions which led to the Chernobyl disaster. It’s interesting to note that there was no glaring mistake here – no one who fell asleep on the job or did something ‘insane’. There were a number of places where folks acted in violation of safety regulations. Why? Most of these was an experienced operator who knew that that rule was a bit more conservative than it needed to be in this case, for an operator of his/her skill, and each operator had broken that saftey rule before with positive results. (e.g. they were able to avoid overtime without causing any problems). In fact, Dorner points out that the nature of safety regulations is that we tend to be rewarded for violating them most of the time. If I choose not to wear my bike helmet, the main effects are that I can enjoy the wind in my hair and don’t look quite as silly as usual…most of the time. If we break internal before releasing a product, we may get similarly positive results…most of the time. Happily, on the projects I’ve been a part of the potential negative consequences of releasing buggy software tend to be less frightening than a nuclear meltdown, or a bike accident without a helmet. There are plenty of bugs in the world though that have cost lives, and many more that have cost jobs and money.

I am a big believer in having a group of informed stakeholders assess the relative risks of releasing v. waiting. I believe that my job as a tester is to make sure that that is as informed of a discussion as it can be, but not to try to assert control of blocking the release. Dorner is a humbling reminder for me that while I attempt to shed light on the current state of the application under test, my model can be assumed to be incomplete and incorrect.

I believe that one of the major challenges a tester faces is how to communicate the perceived quality of the software, including a map of what is not known, and what may be incomplete and incorrect.