Every now and then Conformiq Designer and Creator are compared against other testing tools that employ one type of test design heuristic, for example a combinatorial testing tool for generating optimized pair-wise test data. The capability of generating such combinatorial data combinations is one testing method that in practice has been observed to be an efficient heuristic for finding certain types of issues. That being said, pair-wise testing (and combinatorial testing overall) is just one testing method and in and of itself has only limited value and use in testing.
Conformiq Designer and Creator support a large variety of different testing methods and heuristics that can be enabled and combined based through user preferences based on the testing needed. The test designer no longer needs to worry about how complicated the test design will become when using different methods because the Conformiq engine will perform that work automatically. The bottom line is that the Conformiq platform is a solution for automated test design where you describe the behavior of your system using a behavioral system model as opposed to describing a model of testing strategies (this is what we call a test model or a usage model) while at the same time allows the user to describe certain explicit testing strategies like combinatorial testing.
When you combine these two worlds (pure system and explicit test strategy modeling), you need to take into account the requirements and constraints from both of the worlds. This is where people may go off by thinking that Designer and Creator are just simpleminded tools for working with a given testing method or strategy. For example, even if combinatorial testing tools create an optimized set of test inputs for stimulating the system under test by applying a pair-wise or other combinatorial testing method, we also in testing need to be able to understand how the system should react to each such an input. This is the key part of the “test oracle problem” which is where pretty much all the testing tools excluding Conformiq simply stops and leaves this really important and difficult part as an exercise to the user. Manually or even semi automatically determining the correct response in a complicated test case is a huge endeavor, takes considerable time, and is a very error prone process. This is why manually created test cases are typically simple, which leads to many more test cases, which leads to much larger regression suites, which leads to longer test runs or guessing which test cases should be ignored.
The whole philosophy and core of Conformiq test generation platform is to fully automate the test oracle problem, not just implement some specific testing method which typically, like in the case of combinatorial testing, is quite trivial to do. In practice this means that Conformiq will automatically design and create test cases with test input used to stimulate the system under test combined with an exact expected response from the SUT all with full data content and with exact timings by applying a user defined set of testing methods and strategies and do all of this fully automatically.