Testing != test execution

Testing includes:

1. Building a useful mental model of the application under test, including what value it should provide and what risks could threaten that value,
2. Designing powerful tests, using that model to investigate important questions about the application’s quality,
3. Test execution (which might be automated entirely, partly, or not at all),
4. Analyzing the results to determine if there’s a problem, how severe it might be, or otherwise answer stakeholder questions, and
5. Test reporting that clearly communicates the impact of the bug, and how to quickly reproduce it.

We often talk about testing as if it’s only test execution, yet often the most interesting, challenging, skill-intensive aspects of testing are in creating a mental model that helps us understand the problem space, designing tests to quickly and effectively answer key questions, analyzing what specifically the problem is, and communicating it effectively.