Testing includes:
- Building a useful mental model of the application under test, including what value it should provide and what risks could threaten that value,
- Designing powerful tests, using that model to investigate important questions about the application’s quality,
- Test execution (which might be automated entirely, partly, or not at all),
- Analyzing the results to determine if there’s a problem, how severe it might be, or otherwise answer stakeholder questions, and
- Test reporting that clearly communicates the impact of the bug, and how to quickly reproduce it.
We often talk about testing as if it’s only test execution, yet often the most interesting, challenging, skill-intensive aspects of testing are in creating a mental model that helps us understand the problem space, designing tests to quickly and effectively answer key questions, analyzing what specifically the problem is, and communicating it effectively.
August 26, 2011 at 10:47 pm |
Thanks for sharing this post. I am looking for Software QA Testing companies with expertise in Functional Testing Services and localization testing services.
October 2, 2011 at 3:58 am |
this is why it’s good to have an end-to-end test management tool that covers the entire process, and not just spreadsheets or bug-tracking software.
April 29, 2012 at 5:50 am |
Actually the execution is maybe the least important. The design of the tests using a well thought out model and then the analysis and reporting are perhaps the most important. If we do no analysis afterwards, how can we figure out if what we did worked? and how can we improve without good analysis? Improving should be one of our key objectives, not only in the quality of the sw itself, but also in our testing processes and test effectiveness. Otherwise, why test at all?
October 11, 2012 at 11:40 am |
You are exaclty correct, in fact the test execution part is the smallest component of overall testing. Without proper planning, and designing of great test cases, you will not find the critical defects that will affect your customers.
October 17, 2012 at 7:56 am |
I’m right there with you! Nice post! I want to add:
it might be depending on which types of tests or testlevels you are working on.
- Example: short module tests: model and test cases are easier to create
- other example: integration testing – workflow testing: test model and creation of testcases is more complex. You have to consider for examples some interfaces between the single modules too – that means: more effort.
When boundary testing or equivalence class building is the task: Then it also will be usefull to create a short model. General in black box testing, when it is more then single module/function testing, the creation of a model and after that the creation of the test cases will be usefull and I think the right way!
KR, Ralf
January 2, 2013 at 7:19 am |
Jeff,
I love this post. I’ve attempted to deliver similar messages myself earlier but was not as elegant and concise. From here on out, I’m just going to quote you on this topic.
- Justin
January 20, 2013 at 11:18 pm |
[...] Testing != test execution by Jeff Fry – “We often talk about testing as if it’s only test execution, yet often the most interesting, challenging, skill-intensive aspects of testing are in creating a mental model that helps us understand the problem space, designing tests to quickly and effectively answer key questions, analyzing what specifically the problem is, and communicating it effectively.” [...]
April 5, 2013 at 2:45 am |
thanks for sharing this good points about testing with us..
i really liked & note down them in my note book…..