Raincode’s solution to migrate PACBASE systems has been tested more than any other in the field, both in terms of number of sites it has been applied to, and in the total volume of code it has processed.
Still, it is not carved in stone. The migration toolset still evolves, to recognize COBOL quirks (some dare call them extensions) we never encountered before, or to support new features, new patterns, new transformations required by our customers.
But then comes the question, how do we validate the changes? How do we ensure that the migrated code still works as the original one, that functional equivalence remains guaranteed release after release? The migration machinery is a sophisticated piece of software, and it requires more than superficial testing here and there. We would not want to go to our prospects and customers, telling them that we are going to change their source code if we are not entirely confident that the functional equivalence is guaranteed.
Our quality control procedures aim at providing us with this confidence. It is based on a common strategy when it comes to software engineering, namely divide and conquer.
But before we proceed, we must first explain more about how the migration toolset is structured.
It is divided in two main components:
- The Raincode engine is our COBOL parsing and analysis facility. It deals with everything that has to do with recognizing COBOL programs, and supports transformations and queries on these programs. It recognizes numerous COBOL dialects, including SQL, CICS and DL/I extensions. It is a generic tool we use for a number of purposes, out of which PACBASE migration is just an example.
- On top of the Raincode engine, the COBOL restructuring facility actually transforms the code to make it more maintainable, more readable etc.
This clear separation in two layers is the cornerstone of our testing strategy, based on a simple premise: we never change both components simultaneously. We always change one of the components, and test the result extensively before considering changing the other.
How does this work in practice?
We keep a large portfolio made of several thousands of PACBASE-generated COBOL programs, together with the resulting code after restructuring. When changing only the engine, since the logic of the restructuring is not altered, we can ensure that the new engine produces the same restructured COBOL program logic to the character level. Failing that, the new version of the Raincode engine has an obvious change of behaviour.
When the restructuring facility evolves, we restructure a number of thoroughly crafted test programs that exercise the various transformations supported by the migration facility, run them and ensure that their execution remains consistent with the previous release of the restructuring facility.
Differences in the resulting COBOL programs are also inspected manually, to ensure that they are in line with the changes that have just been made to the restructuring facility.
This layered approach to regression testing has yielded excellent results. Differences of behaviour are detected as early as possible in the process, rather than relying on customer testing to ensure that the restructured programs do indeed behave as they did before.
After all, this is a volume business, and requiring extensive ad hoc testing would defeat the purpose altogether.
As always, comments are very welcome!
That’s all for now…