Data-Driven Testing
Definition
A test automation technique, that is used for creating test cases that have the same steps but different input variations. This technique helps in developing and maintaining test case efforts and it is one of the main focuses when it comes to testing automation strategy especially in Agile-oriented projects. When we follow the data-driven approach, we actually test scripts that accept different inputs to run with different sets of data. With that, we increase the test coverage and in parallel, we reduce the amount of maintaining tests.
The basic concept behind this technique is that we want to separate the set of data and the test logic. Having said that, the data-driven testing technique mainly focuses on having one test script but an array of data as an input to that test case.
Benefits
When we are in a situation where have a large quantity of data that needs to be tested properly but the test logic should remain the same. Instead of writing the same test logic for different sets of data, we have one test and multiple sets of data. After that, we end up having one test instead of many, to maintain and update in the future which requires less effort.
If a new test should be added but the logic should remain the same, we just need to add the correct data in the array and we’re done. As a result, the QA engineers can focus on doing automation or exploratory testing on new features.
Some of the benefits following this approach in Agile testing teams are:
- Increasing test coverage from sprint to sprint should b a minimal effort by adding just a new set of data to the test script
- Having a separate test data can be easaily understandable by different memeber of the team
- Reduce the maintenance of the data-driven tests and increases teh efforts on testing new feature per sprint
- Early feedback from all members including non-technical individuals since the test data is already separated and easy to understand by everyone
Examples
To illustrate how data-driven testing looks like I’m going to show you a Selenium C# example. I’ll try to fetch the text for each of the menu options shown in the screenshot below and will assert them with the values from the dataset.
We will store the data in a .csv file type and we’ll pick the values from there for assertion purposes.
[TestMethod]
[DataSource(
"Microsoft.VisualStudio.TestTools.DataSource.CSV",
@"C:\Downloads\CSVData.csv",
"CSVData#csv",
DataAccessMethod.Sequential)]
public void DataDrivenTest()
{
By locator = By.Id(TestContext.DataRow["TabMenu"].ToString());
IWebElement element = GenericHelper.GetElement(locator);
string elementText = element.Text;
AreEqual(elementText, TestContext.DataRow["MenuText"].ToString());
}I have a test called DataDrivenTest() which fetches every element from the menu bar and its text and compares the values with the data from the .csv file. This test will execute a couple of times depending on the number of data in the .csv file.
In order for this to work, we need to specify the method to act as a test by using [TestMethod] special keyword from the MsTest framework family. If you’re not familiar with MsTest, I suggest reading the How to write automated tests using MsTest framework blog.
In the data source special keyword, we specify the type of file we are going to store the data, the path where that file is located, and the access method for fetching the data which should be sequential.
In order for the fetching from file to work, we need to have headers in the file. In this case, TabMenu is a header that includes the locators from each and every menu item. And the MenuText header is the actual text of that element.
First, we fetch the locator by its Id attribute and inside we provide the TextContext special abstract class for storing and accessing execution information. We provide the DataRow[“TabMenu”] and converting it to a string. This means that we actually passed the menu-item-366 string in the id locator and so on. Then we fetch the element by that locator and we’re getting its text value. With an assert function, we are comparing the actual element that we fetched using a locator from the .csv file’s TabMenu column one by one with each of the values from the MenuText.
In this case, the test will execute 5 times for those two sets of data and by that, we just reduced the amount of test/code we need to write to cover 5 test scenarios.
Keyword-Driven Testing
Definition
This automation testing technique is basically applying keywords in the test script so that the test itself can become more readable and understandable. The concept behind it is that we need to define a set of keywords based on the product requirements and convert them into scripts or functions.
It reads the keyword from a .xlsx file as input data. Then the script will perform the action using that keyword and will return some results which are then stored back in the excel file.
Benefits
Some of the benefits of introducing keyword-driven technique in agile teams are:
- By creating keywords, we actually create a dictionary of keywords so that even non-technical person can understand what the keyoards actually do
- Gatehring a quality feedback from teh customer due to more udnerstandable test structure
- Be able to maintain a type of a living documentation because
- Reducing test maintenance
- Great chance to add abstraction in multiple elvel of testing suych as unit, integaration, system tests, etc.
Examples
Let’s take a similar example to the previous one.
Keywords | Description
Login 1. Log in to www.qamind.com
2. Fetch the menu item text
3. Assert it
private void fetchExcelValueAndPerformAction(string keyword,string locator, string expectedText)
{
switch (keyword)
{
case "Login":
LoginHelper.Login(GetElementLocator(locator));
break;
case "GetText":
string actualText = TextBoxHelper.GeElementText(GetElementLocator(locator));
AssertionHelper.AssertElementValue(actualText, expectedText);
break;
default:
throw new NoSuchKeywordFoundException("Keyword Not Found : " + keyword);
}
}public void LoginAndAssertMenuItem(string xlPath, string sheetName)
{
int totalRows = ExcelReaderHelper.GetTotalRows(xlPath, sheetName);
for (int i = 1; i < totalRows; i++)
{
var locator = ExcelReaderHelper.GetCellData(xlPath, sheetName, i, 1).ToString();
var keyword = ExcelReaderHelper.GetCellData(xlPath, sheetName, i).ToString();
fetchExcelValue(keyword, locator);
}
}[TestMethod]
public void KeywordDrivenTest()
{
KeywordDrivenClass.LoginAndAssertMenuItem(@"C:\Downloads\Data.xlsx", Sheet01);
}We fetch the cell from excel and performing an action which is login, fetching the values, and asserting them, and then in the next method, we loop through each and every row in the excel file. (in the future there may be more keywords to be added). Then we just call that method in a test by passing the excel path and the sheet name and we have successfully implemented a test following the keyword-driven test automation approach.
Differences between Data-Driven and Keyword-Driven Testing
| Data-Driven |
| Clear separation of test data from test logic but not that easy to maintain |
| Planning for separation of data and logic is fairly easy |
| It’s well structured and easy to comprehend but not in great detail |
| Keyword-Driven |
| Easy to maintain due to having multiple layers like data, script, logic, keywords, etc. |
| More planning is required for this type of approach |
| More understandable to non-technical persons |
Share This Post
Latest Posts
- Crucial Aspects Of Superior Test Code Quality
- Streamline Your Cucumber BDD Tests with Gherkin Lint: A Comprehensive Guide
- How to Test Authentication and Authorization with Confidence + Best Tips
- Exploring the Three Scrum Artifacts: Grasping Their Essence and Effective Utilization
- How do Scrum teams work effectively to deliver quality products to their stakeholders?
