Introducing Exploratory Testing: Definition, How-To, Pros and Cons

Vlad

March 1, 2021
5 min read
Agile Testing Exploratory Testing Software Testing Test Charter

What is Exploratory Testing?

Exploratory testing is an approach that focuses on testing the software on the go instead of creating predefined test cases before test execution. It is a form of testing that is generally used across all agile development projects. Often, this type of testing is combined with experience-based testing techniques since the agile tester should have solid technical and business domain knowledge.

In exploratory testing, test design and test execution happen at the same time and are guided by a test charter. Test charter is a document that includes test goals and objectives as well as ideas to perform the testing, in other words, it’s a perfect fit for having a “guided” exploratory testing session. The output from one test in exploratory testing represents a guide for the upcoming test.

Why do you need Exploratory Testing?

Exploratory testing is important especially in Agile-oriented projects where there is a limited time available for test analysis as well as limited details for a particular user story once it’s being picked up for analysis. It’s not random testing, but it is a form of ad-hoc testing with the objective of discovering defects in the system early. Since it is not structured and not an actual technique, rather it is an approach that can be followed to learn more about the system and to document defects, ideas, risks, etc.

When it comes to documentation after the testing is complete, we need to make sure we capture the most important information that will be helpful for the whole team in general. Things that can be documented after an exploratory testing session are:

How much we covered with testing and what remains to be tested
The behavior of the system that needs to be captured and evaluated
Any unexpected actions or concerns about the system
Test notes, where all the info like defects found, ideas for testing edge cases are included

When it comes to statistics, Exploratory testing is proven to detect 14% more defects than scripted tests because you are not limited to how many different inputs you will use and what combinations to test. In other words, you are free to experiment with the system by combining different inputs and approaches.

>> If you didn’t know, regression testing is also beneficial for discovering defects early and effectively. Check out this post about what is Regression Testing and why it’s important for any organization?

How to perform Exploratory Test?

To perform Exploratory testing, a method called session-based testing management is used. This is an uninterpreted period of testing that usually lasts from 60 to 120 minutes. A test session allows the tester to have a high degree of flexibility and freedom while performing the test. During that session, several activities can be achieved like learning the system functionalities, evaluation of functional and non-functional aspects of the system as well as covering scenarios and edge cases.

In order to take full advantage of exploratory testing, the tester needs to consider these questions before the test execution:

What is the most important aspect that we need to know regarding the system?
What should happen if I try to input the specified set of values?
Is the system capable of building (installing) on different platforms?
Are requirements satisfied?

A form of guidance or session-based management can be defined before performing exploratory tests:

Classify common types of errors and define ideas to test the system
Create the test charter which can include:
- An actor who performs the testing
- Purpose of the test
- Setup that needs to be done for test execution
- Data that must be used for testing
- Different variations to perform while testing
- Take notes of any defects or unexpected behaviors
- The priority of each performed test based on the requirements
Test session (time-box session) where testers are testing for an uninterrupted time
Generate results and evaluate them by comparing them with the expected results from the test charter
Decide if there is anything left to be tested

Some of the familiar exploratory testing tools out there are:

Pros and Cons

Pros

More effective at finding bugs due to the high degree of flexibility
Perfect when the requirements are not completely defined
Based on the “thinking” activity rather than following a predefined script
Learn about the system’s core functionalities and perform edge case scenarios
New test ideas are being defined based on the output from previous test sessions

Cons

The tester needs to have a good technical knowledge
The tester needs to understand how the system works initially including the business domain
Documentation of all actions and defect can be time-consuming
Which test cases defined as the most important ones can be difficult to achieve especially where large integrated systems are being tested
Tester need to find a way how to report any defect found in a comprehensive way for other team members to easily grasp what is wrong with the system

>> Want to know more about Acceptance Testing maybe? Check out this awesome blog post that explains what it is, why is important, what are the test types, and how to perform.

The Bottom Line

Exploratory testing can be exceptionally beneficial for agile projects where there are short iterations and tight deadlines. Often, the customer cannot complete the requirement specification at requirements review meetings so you need to spend time clarifying all the things together with him. Exploratory testing can help because once completing the test session against the early developed system, the tester can give feedback on any potential defects, ideas for improvement, or changes to the existing requirements which in return can result in having a better product overall.

Share This Post