Probably every test manager dreams of a world where test scripts work the first time, where tests and test scripts are maintained automatically, and where our test team spends their time on finding faults and going deeper in the system rather than redoing the same tests again and again because of changes in the specification. A dream where every test execution gives you a test report and provides immediate feedback on the application to be tested. Yes, well, it may be a dream, but now a new way of testing can help to make this dream become reality.
Let’s take a look at what people in embedded systems design do, and how we could “maybe” learn from them in testing. First, the development team gets requirements (the same requirements that we get in our testing team). Based on that, they start “coding” by specifying design models. The design model can be used and/or refined based on changes in the specification, and can be used to “automatically” generate code for the target platform. This design technique is used frequently in automotive and also in other application domains. This “automatic code generation” based on design models really helps to save time in a world where time to market is often the difference between success and failure.
Now let’s take a look at testing today. Most companies are still manually creating tests (even in embedded systems development – even in automotive) from specifications. But the difficult part here is how to manage all the tests once you get changes in the specification. How often does your specification change? For every change, testers will need to spend some time to identify the places in specific tests impacted by this change, to do the maintenance. But, testers are human, and when specifications change frequently and when there are a lot of tests to review, then clearly mistakes can happen. When all changes are done, then it’s time to update the automation. A big and long process starts where testers write a first test script, try executing it, modify it again and again, and at the end they have something automated. If we apply embedded system design techniques here, why don’t we also use a model in testing, and generate from it automatically test documentation and test scripts? This idea actually exists and is called Model Based Testing or MBT.
To differentiate an MBT model from a design model, a design model contains details and interactions with internal components, and considers architectural requirements that are not relevant from a black-box testing point of view. Models for testing generally have lower complexity than design models for the same system.
What I suggest you do is create a model independently from the specification, purely based on externally controllable and observable functional requirements. The interfaces for exchanging data with your product or interactions (like set, send, click, drag or get, check, and verify) can be automatically imported into the tool. All you need to do is to describe the functionality to be tested based on the requirements and data you have available. You can look at this MBT model like a picture (functionality) of a landscape (your product). When your MBT model is done, it’s time to generate your test suite, based on algorithms that come with the tool. Then, from the picture of this landscape, the tool will describe what it sees as “tests” required to cover the given functionality.
In addition to generation of tests, the tool will at the same time generate test scripts for your current test automation tool or test framework. With some test automation tools, it’s even possible to enable reuse of the data library for stimulation. In fact, in your model, you can use these library functions directly as keywords in actions (inputs and outputs)! The tool will then generate for you test scripts that are ready for automatic test execution. But let’s be frank here, there are natural limitations on the extent that we can automate test execution in general. But model based testing can surely help you increase your level of test automation, enable automatic test generation, and make your life easier. But, it’s still a tool.
To conclude, in testing some lessons can be learned from embedded system design techniques. These techniques are not research ideas – they have been successfully applied in industry for many years. So why not also apply these techniques in testing? You can save time and let the test team focus on exploring functionality rather than creating tests manually, converting these tests into test scripts manually, and updating tests manually when the specification changes. Why not use model based testing and automate everything?
Alexis Despeyroux is a Conformiq technical field application engineering specialist.