You need to test this critical fix for the old version. Now!
Imagine your manager is telling you that. How long does it take you to find the original results of the old version, setup the old test environments, install the fixed version, run the tests, and compare them to the original results? If it is more than 7 minutes, read on.
This post will explain:
- What is traceability
- How to use traceability to re-create environments
- How traceability saves you time
- What additional benefits you get from complete traceability
Traceability is the most extensive form of tracking
If you do not know what traceability means, no problem.
Imagine that you can have a reference from every single executed step in a test case to the corresponding test plan, the corresponding software version, the corresponding test environment and how to set this environment up.
In general, the idea is that the available documentation is generated in a way that a new team could take over your project and could reproduce all your results.
Tracking allows you fast regression tests on any version at any point in time
Reproducing the results. This idea brings us to the main benefit of traceability. If your test automation system allows you to re-run tests the same way, they were originally executed you can easily do regression testing on old software versions.
It is a simple scenario: Version 1 and 2 of your software are in the field. R&D and testing is working extensively to get version 3 finished. As version 2 is the one that is current you still have everything ready to run regression tests on it. Even if it is for comparison only.
But here we go, a critical bug was found by an important customer who is still on version 1. If you have full traceability in your automation system you are up and running with your tests in a matter of minutes.
Select the old test plans, select the old environment, upload the fixed version 1 release, add a new testcase for the bug and re-run all tests as regression. Within less than one day you have a fully tested critical release. A happy manager, a happy customer and perhaps a cool gratification.
Having results, environments, SW versions, testcases & executions available for re-run saves time
Select the old test plans, select the old environment, upload the fixed version 1 release, add a new testcase for the bug and re-run all tests as regression. Within less than one day you have a fully tested critical release. A happy manager, a happy customer and perhaps a cool gratification.
The above paragraph has demonstrated how tracking everything for reproducibility can save you a lot of time. But what if the above scenario never happens. Perfect for you, no unexpected work! The additional tracing does not add any overhead to your testing as it is done completely transparent in the background.
The system should automatically save (with revisions):
- Testplans
- Testresults (Execution results)
- Test Models
- Software versions
- Test Environment versions
- Variables used for executions
If you have multiple system to track all this information today, think about how long it would take you to run a regression test on the oldest but still supported software version and create a set of documentation for your testing activities. Compare this to the 7 minutes it takes you in TestResults.io.
You are ready for audits
There is one more benefit. If you have full traceability you are always ready for audits. If you are working in any kind of regulated environment e.g., banks, medical, clinical, pharmaceutical, aerospace, you know how troublesome it can be to create all the cover documentation for your tests and your automated tests.
A complete test automation solution provides you all the required documentation together with your test results, for any version in any environment at any point in time.
Leave a Reply