I recently had an opportunity to present in ETSI’s UCAAT conference a way to use component-enabled MBT to overcome some of the shortcomings with current interoperability and end-to-end testing approaches. Because this raised quite a lot of interest in the conference, I wanted to write a short blog post about the subject.
In the industry-standard approach, interoperability testing (which is closely related to end-to-end testing) is used to validate interoperability between system components, for example between handsets and network elements. It is widely recognized issue that interoperability problems limit and slow down new technology and service launches and, for example, ETSI also takes a strong stand on this in its standardization activities pointing out that “Interoperability is not something that will somehow get fixed at the end of the process.” (http://www.etsi.org/index.php/about/how-we-work/testing-and-interoperability).
Traditionally, interoperability testing can be only applied very late in the testing process, causing escalating costs and friction to resolve the interoperability issues.
The component-enabled model-based testing approach changes this, but let’s first see how this all works out. (The reason why I’m saying “component-enabled MBT” is that your models need to be compositional, that is the models need to be system or SUT models, not test models, and you need appropriate tooling for doing this kind of exercise, something like Conformiq Designer). Everything starts by creating system models of test objects. To continue the example from the Telco domain, one creates two models: one for the handset and another for the network element. We then apply a component-enabled MBT tool to generate tests for individual objects thus deriving separate tests for handsets and separate tests for network elements.
In the above figure we see this setup; on the left hand side we have a terminal device that is tested with a test suite generated using a model that captures the functional specification of the terminal. On the right, the network SUT.
Ok, so we have now “local component tests” for both parts of the overall system, but we have really not said anything about the interoperability of these two parts yet. We have only been able to evaluate the correctness of these components with respect to their functional specification. However, what we want to know whether these two components “interact” properly. As mentioned earlier in this post, here lies the problem with traditional testing approaches; one can only apply interoperability testing when both of these implementations are ready, pushing interoperability testing really late (too late if you ask me) in the process. This is where component-enabled MBT comes in!
Now that we have these two model components, we take these models and link them together across vendor boundaries to verify that the models interoperate. Because the models are actually formalizations of the functional specifications and the same models are used to test the individual components, they will exhibit the same interoperability problems as the real implementations. So here’s the catch: the real implementations are not needed for testing and all we need is to “glue” the models together in order to see whether they interoperate. This makes it possible to push testing earlier, much earlier, in the process helping us to spot out interoperability problems much earlier than implementation based interoperability testing would be possible. This all by simply gluing the models together!
The above figure demonstrates this setup. By gluing the two model components together we can use component-enabled MBT tool to verify the interoperability of the two components before the actual implementations even exist. That’s why we have greyed out these in the illustration.
Eventually, if one desires, the same models can be used to derive automatically orchestrated end-to-end tests with automatic verification of intra-system traffic. This is illustrated in the picture below
So in a nut shell, as summarized in my UCAAT session, component-enabled MBT enables us to:
- generate tests for individual objects,
- verify interoperability on model level, and
- generate automated interoperability tests for system-wide end-to-end testing.
Which again enables us to:
- move interoperability issue detection earlier in the process,
- verify interoperability locally before actual interoperability testing events, and
- cost-efficiently automate both component and end-to-end level test design.