- Conformance Certification
- Passive Testing Approach Benefits
- Exactpro’s Test Harness for Passive Testing
- Using the Exactpro Test Harness for Conformance Procedures and Automation
- Conformance Testing Workflow with Exactpro Test Harness for Passive Testing
- Using th2 Framework Modules to Optimise the Conformance Automation Solution: Reference Architecture and Goals
- Conclusion
This case study describes Exactpro’s passive-testing-based solution for streamlining customer conformance testing automation for trading, broker and post-trade venues.
Conformance Certification
Conformance certification (also known as conformance testing) is a mandatory step in ensuring that customer systems comply with the officially declared exchange/broker certification rules. Conformance certification is conducted in order to prevent compatibility issues between the trading platform and the systems of the trading participants connecting to it.
For the purposes of this case study, we will use the term ‘trading venue’ to imply any parties conducting conformance certification. The list of these includes but is not limited to exchanges, alternative trading systems (ATSs), multilateral trading facilities (MTFs), Swaps Execution Facilities (SEFs), broker and post-trade systems.
Some of the typical challenges faced during conformance certification are:
- world time zones: time differences cause difficulties with planning and coordination of testing by teams located in various parts of the globe;
- performance: certification can be required by several customers concurrently due to business or technical events, such as the deployment of new software, regulatory or technical requirements;
- competency: specialists conducting certification must have outstanding technical and business knowledge;
- coverage: certification tests must be based on typical scenarios specific to the system and produce meaningful results.
One of the ways to cope with these challenges is creating a scalable automated technology solution for conformance testing and the analysis of test results.
Exchange platforms typically apply the following three approaches to conformance certification and its automation:
- Customer testing against an exchange simulator: provision of a specialised simulator to customers to conduct certification. The customers conduct testing using the tool and provide certification logs;
- Customer testing against a replica of the exchange production system:
- provision of access to a specialised test environment which represents a smaller replica of a production system,
- documenting the procedures,
- conducting joint certification with the customer or customer self-certification,
- using passive testing methods – such as validation of the traffic capture of network connections or data from log files – to analyse the success of the certification procedure.
- Customer testing in the production system during dress rehearsal periods (before platform’s go-live date or during planned weekends):
- provision of access to the production (or soon-to-be production) environment,
- defining specific time windows for the access,
- defining specific market/segment or set of instruments for customer testing in the production environment,
- documenting the procedure and dress-rehearsal steps in order to have smooth weekend transitions,
- validation and analysis of the traffic capture of the network connections or data from log files.
This case study is focused on the second approach. However, trading venues’ certification teams can easily reap the benefits of passive testing automation for all the conformance approaches listed above.