Creating a system model – a model that directly describes the intended system behavior – is “easier” than creating tester models – models that describe the testing strategies themselves. This position has been demonstrated both within practical and theoretical frameworks. This correlates well with day-to-day observations of test managers: defining, designing, and expressing testing strategies is difficult regardless of whether they are expressed in a form of a computer readable model or not. Ultimately, this is because creating a system model is actually a translation problem, while modeling an explicit tester model is a true combinatorial and computational challenge. Thinking that a tester model is easier stems from a generally agreed upon insight that testers create or possess a mental model of the complete system under test operation based on understanding that they form from implicit or explicit system specifications and project experience.
This reasoning also reveals the other side of the coin, that tools for generating test cases from tester models are easier to implement, hence there are more of them available (MaTelo, GraphWalker, ModelJUnit, TPT, TestOptimal, Agile Designer, and so on) than tools for generating tests from system models (Conformiq Designer and Creator, MS SpecExplorer, Smartesting CertifyIt), which are at the top of the spectrum in terms of algorithmic complexity.
However, system modeling is often perceived as being more difficult than creating a tester model. But why is that?
Most often, this perception is simply because when modeling the intended system behavior, the tester really needs to understand how the system is supposed to operate. As you create a model of the intended system behavior, it is essential to clearly understand the intended operational characteristics of the system, which typically raises a lot of questions regarding the requirements. The mere act of modeling the intended system behavior often improves the quality of the requirements and many defects can already be seen in the specifications and requirements before even writing a single line of code. By creating a tester model, you may get away with “job half done” by not fully understanding what the system is supposed to do and then resorting to suboptimal testing efforts. The support from the available tools for modeling explicit testing strategies often does not enforce meaningful metrics for measuring the completeness and quality of testing efforts, resorting to heavily misused metrics, such as the number of test cases, source code statement coverage, and so on.
Another thing is that test engineers may feel alienated modeling the system behavior as that does not involve the same thinking process as you typically have when doing testing. You don’t really think about testing when you are modeling, so in a sense, the role of the tester moves a bit closer to developer or designer role.
Yet another practical issue is that system modeling requires different skill set than manual test design and test modeling. System models are abstract small programs, therefore the test engineer must be able to abstract and design programs and this requires programming skills — at least to some extent. On the other hand, quite often specifications and requirements are written in an informal notation that can be quite naturally translated in to program or model code – take business rules as an example. They are quite often described in pseudo code, decision tables or trees. Or take a protocol specification – you see a lot of state charts and pseudo code fragments, all which can be quite easily translated in to “code”.
As with any disruptive new technology, there are some obstacles on the way that hinders deployments of system model driven MBT. These obstacles, luckily enough, can be overcome by training and experience. The main business argument for switching to system model driven model based testing is that it makes business sense. If, however, a company’s practices do not allow it to measure the difference between mediocre, good, and very good testing, it is impossible to put through the methodical and cultural changes that system model driven MBT requires, including having to evolve engineers’ skills and responsibility in understanding and capturing the system operations.
