Whenever new software is released, the need to test new functionality is obvious. However, it’s equally important to re-run old tests that the application previously passed. That way we can be sure that the new software does not re-introduce old defects or create new ones in the software. We call this type of testing as regression testing. Throughout this article, we will explore regression testing in detail. If you are new to software testing, be sure to also read the Beginners’ Guide for Software Testing.
What is Regression Testing?
“Testing of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, as a result of the changes made is called Regression Testing.”
A regression test is a system-wide test whose main purpose is to ensure that a small change in one part of the system does not break existing functionality elsewhere in the system. If you consider regression as unintended change, then this type of testing is the process of hunting for those changes. In simple terms, it is all about making sure that old bugs don’t come back to haunt you. Let’s take a look at a fictitious example that illustrates the concept.
When adding a new payment type to a shopping website, re-run old tests to ensure that the new code hasn’t created new defects or re-introduced old ones. Regression testing is important because, without it, it’s quite possible to introduce intended fixes into a system that create more problems than they solve.
Benefits of Regression Testing
Conducting regression tests benefits companies in a number of ways such as:
- It increases the chance of detecting bugs caused by changes to software and application
- It can help catch defects early and thus reduce the cost to resolve them
- Helps in researching unwanted side effects that might have been occurred due to a new operating environment
- Ensures better performing software due to early identification of bugs and errors
- Most importantly, it verifies that code changes do not re-introduce old defects
Regression testing ensures the correctness of the software so that the best version of the product is released to the market. However, in the real world, designing and maintaining a near-infinite set of regression tests is just not feasible. So you should know when to apply regression testing.
When to apply Regression Testing?
It is recommended to perform regression testing on the occurrence of the following events:
Next part of this article is about different types of regression testing.
What are the types of Regression Testing?
Regression testing is done through several phases of testing. It is for this reason, that there are several types of regression testing. Some of them are as follows:
Unit Testing: In unit testing when coding changes are made for a single unit, a tester, usually the developer responsible for the code – re-runs all previously-passed unit tests. In continuous development environments, automated unit tests are built into the code, making unit testing very efficient in comparison to other types of testing.
Progressive Testing: This type of testing works effectively when there are changes done in the software/application specifications as well as new test cases are designed.
Selective Testing: In selective testing testers use a subset of the current test cases to cut down the retesting cost and effort. A test unit must be rerun if and only if any of the program entities it covers have been changed.
Retest-All Testing: This type of testing strategy involves the testing of all aspects of a particular application as well as reusing all test cases even where the changes have not been made. It is time-consuming and is not much use when any small modification or change is done to the application.
Complete Testing: This testing is very useful when multiple changes have been done in the existing code. Performing this testing is highly valuable to identify unexpected bugs. Once this testing is completed, the final system can be made available to the user.
It is very important to know which type of testing suits your requirement. Next up, we will discuss how regression testing is implemented.
How is Regression Testing Implemented?
The procedure to implement regression testing is like the one you apply for any other testing process. Every time the software undergoes a change and a new release comes up, the developer carries out these steps as part of the testing process:
- First of all, he executes unit-level regression tests to validate code that they have modified, along with any new tests they have written to cover new or changed functionality
- Then the changed code is merged and integrated to create a new build of the application under test(AUT)
- Next, smoke tests are executed for assurance that the build is good before any additional testing is performed
- Once the build is declared good, integration tests are performed to verify the interaction between units of the application with each other and with back-end services such as databases
- Depending on the size and scope of the released code, either a partial or a full regression are scheduled
- Defects are then reported back to the development team
- Additional rounds of regression tests are performed if needed
That’s how regression testing is incorporated into a typical software testing process. The image below clearly depicts how regression testing performed.
Whenever some changes are made to the source code, the program execution fails for obvious reasons. After the failure, the source code is debugged in order to identify the bugs in the program. Appropriate modifications are made. Then the appropriate test cases are selected from the already existing test suite which covers all the modified and affected parts of the source code. New test cases are added if required. In the end, testing is performed using the selected test cases. Now you might be wondering which test cases to select.
Effective Regression Tests can be done by selecting the following test cases:
- Test cases which have frequent defects
- Complex test cases
- Integration test cases
- Test cases which cover the core functionality of a product
- Functionalities which are frequently used
- Test vases which frequently fail
- Boundary value test cases
With the regression testing process out of the way let’s check out various techniques.
Regression Testing Techniques
Regression testing simply confirms that modified software hasn’t unintentionally changed and it is typically performed using any combination of the following techniques:
Retest-All: This method simply re-tests the entire software suite, from top to bottom. In many cases, the majority of these tests are performed by automated tools. Certain times automation is not necessary. This technique is expensive as it requires more time and resources when compared to the other techniques.
Test Selection: Instead of choosing all the test cases, this method allows the team to choose a set of tests that will approximate full testing of the test suite. The primary advantage of this practice is that it requires far less time and effort to perform. Usually done by developers who will typically have better insight into the nuances of test edge-cases and unexpected behaviors.
Test Case Prioritization: The goal of this technique is to prioritize a limited set of test cases by considering more potential test cases ahead of less important ones. Test cases which could impact both current and future builds of the software are chosen.
These are the three major techniques. At times based on testing requirements these techniques are combined.
As useful as regression testing can be, it is not without its negative points. You need to understand the challenges that you might face when implementing it.
Challenges of Regression Testing
- Time Consuming: Techniques like retest-all need lot of time to test the entire suite of test cases
- Expensive: Costly because of resources and manpower that you need to test again and again, something which has already been developed, tested and deployed at early stages
- Complex: As the product expands, testers are often overwhelmed by the huge amount of test cases and fall victim to losing the track of test cases, overlooking the important test cases
Despite these negative points, regression testing is very useful in the software testing process. With regression testing, companies can prevent projects from going over budget, keep their team on track, and, most importantly, prevent unexpected bugs from damaging their products. With this, we have reached the end of the blog. Hope the things that you have learned here today will help you as you head out on your software testing journey.
If you found this article relevant, check out the live-online Selenium Certification Training by Edureka, a trusted online learning company with a network of more than 250,000 satisfied learners spread across the globe.
Got a question for us? Please mention it in the comments section of this ‘What is Regression Testing?’ article and we will get back to you.