Definitions of Testing

Elisabeth Hendrickson has just posted a great blog entry about definitions of testing. She lists several older definitions of testing, and then gives one she and Dale Emery crafted:

Dale Emery and I discussed this at length and decided that we agree that: Testing is a process of gathering information by making observations and comparing them to expectations.

I added two others that work well for me, from James Bach and Cem Kaner respectively, taken from this post on James Bach’s blog:

Testing: questioning a product in order to evaluate it (Bach version); technical investigation of a product, on behalf of stakeholders, with the objective of exposing quality-related information of the kind they seek (Kaner version).

I am a big fan of these three definitions, and think that both their similarities and their differences are quite interesting. I like that they emphasize testing as fundamentally a learning / questioning / investigative process. I like the Hendrickson/Emery emphasis on “…comparing them to expectations.” The open-endedness of ‘expectations’ versus the narrower ‘requirements’. This phrase reminds me that there are many documented and undocumented ways any software or system is expected to work, and harkens back to Jerry Weinberg’s definition of quality as “Value to some person”. This in turn can reminds me that we need to think about whose expectations matter to us, and what might be missing from our current map of the (typically vague, often conflicting) expectations that matter to our stakeholders.

Kaner approaches a very similar notion with a very different flavor when he says (my emphasis) “on behalf of stakeholders, with the objective of exposing quality-related information of the kind they seek.” Kaner is much more explicit that there is a certain set of stakeholders who determine what kind of information we are trying to discover. The set of expectations that we want to compare against are the set of expectations that matter to the stakeholders. These might be the stakeholders’ expectations…or they might be the expectations of others, e.g. valued customers, who matter to the stakeholders.

The origin of Elisabeth’s post was a question from a group of testers who’d been discussing the definitions of a test and testing. In the end, I appreciate knowing several definitions, precisely because of the conversation and reflection that the set as a whole generates.