Many organizations today are having trouble correctly adopting a test automation strategy within their scope. Some are having trouble introducing it into their daily work and as part of their development, others are giving up right after they thought they successfully incorporated automation within their work.
But they struggle to realize that introducing automation from scratch is not a small thing to do whether at the beginning of a project or in the middle. You cannot just start automating test cases when you haven’t done the necessary planning, analysis, risk assessment, effort, resource allocation, etc. That’s why companies either have trouble implementing test automation or don’t know how to or maybe even they are not aware of what tools they need to acquire and what skills they need to gain. They need a test automation strategy.
What is a test automation strategy?
A test automation strategy helps you to determine what kind of automation to apply to your organization and how to apply it to your STLC. It comes down to defining what test cases to automate, the test environment, the test approach, the creation of the scripts, their execution, and the way of reporting the results. All of these parts need a detailed explanation of what they are, why they are needed, and how they can be of huge benefit to an organization. Let’s look at each one by one.
What should one test automation strategy consist of?
The main goal of every organization is successful automation integrated into their continuous development and testing, so a proper way of introducing a test automation strategy must be in place. The 6 factors that are a must when thinking about introducing automation are:
What to automate
Here an analysis needs to be performed where the team needs to decide which test cases to automate as well as select the right automation tool for both the API and the UI of the software. The scope of what needs to be covered can be immense so a good analysis beforehand is a must.
One general rule that you can follow is the Test Pyramid rule which says that you need to have as many tests at the unit level since those are simply short and fast to execute. As we go up in the pyramid the number of tests should be smaller than those at the unit level since the complexity here is bigger and the tests are generally slower. Another rule is basically automating tests by the risk associated with the product. The bigger the risk the better candidate for automation.
As far as tooling goes, deciding on it can be a challenge. You need to take the requirements, budget, licensing, training, and support into consideration when planning for such a tool.
What should be our test approach
The tests approach itself can be called a test strategy for a particular team and usually, it has two important techniques:
- Reactive, where the testing is not started until the requirements gathering, analysis, design, and coding is done which is not a preferred approach nowadays
- Proactive, where testing starts as early as possible in order to discover bugs in the system at an early stage so that it can be cheaper to fix rather than later in the development stages
The majority of organizations tend to follow the proactive approach and incorporating automation here is the end goal. Testing starts early and with the help of automation, you’ll have tests that are constantly executing, testing the limits of the development code and functionality, and reporting results and feedback faster than ever.
Risk Assessment
The risk document is an important part not only for test automation but for testing in general. Assessment of what the risk is, what will happen if the risk occurs and how we will be affected by it, what is the probability of the risk happening in the first place, what can be done to mitigate the risk, and the effort and cost of doing that. All these topics need to be addressed first and then a risk-based testing strategy can be carried out.
Environment configuration
Discussing how to configure the environment is really important and plays a big role in creating the test automation strategy. Various considerations need to take place when building an environment like:
- What is the infrastructure behind the environment?
- When is the input data that will be used in the environment?
- Gathering all necessary tools and licenses for the environment to be complete
- How we can access it and where the data is stored after and during execution?
- What would be the workload for that environment, e.g. how many will be testing on that environment?
- Is that environment a copy of a production environment or a full development focused?
- Will the environment be used for testing new features or will the purpose be to stabilize a current software version as part of a release?
All these topics need additional analysis upon discussion and then a stable and known environment will be created as a result.
Automation & execution
The core of the test automation strategy is the actual automation and execution. Here things like how to automate a test case and how to execute it are being discussed and a concept is being created.
Usually, this can be in a form of a TDD or a BDD approach where tests are written, automated, and executed during the development phase. The tests are executed again and again until they satisfy the requirement for a particular feature. A set of best practices on how to automate needs to be established, which automation design patterns should you use (a popular one would be the POM design pattern) as well as how to integrate those tests and their execution into a CI/CD process.
Report review
A report for a particular execution can be a great knowledge base for future improvements and refactoring. A report inspection or review plan needs to be set in place before automation. Here, several questions need some answers:
- How to descriptively show a failed test in the report and how to handle failed tests?
- What information should be included in eh report? Maybe software version, input data, OS version, browser version, third-party libraries version, percentage of passed/failed tests, etc.
- How to automatically create a bug if a test is failed due to a functionality error or a design flaw
- Will the report be sent to the responsible ones who will take a look at it and decide whether the build is ready to be released or not
Wrapping up
A good and robust test automation strategy will give you great results in the end. So, it is important to define it as early as possible to fully experience the benefits that will come up when the project is started because when you have a strategy in place, the confidence within the team and organization will be increased when you’re getting ready for a release to production.
