The commitment of the Johannesburg Stock Exchange (JSE) to offering the best services on world class systems was behind its decision to launch the Integrated Trading and Clearing (ITaC) multi-year programme focused on the implementation of world‐class, multi‐product solutions to enhance the JSE’s trading and clearing functions. On the trading side, that meant migrating its Equity, Equity Derivatives and FX Derivatives markets to the Millennium Exchange™ and Millennium Surveillance™ platforms provided by LSEG Technology. For clearing, it involved the migration of certain markets onto the new Cinnober (now part of Nasdaq) Real‐Time Clearing (RTC) solution.
Exactpro's expertise spans a number of financial systems for trading, post-trade, risk management, securities data distribution, market surveillance systems, and all financial transactions processes from trading to clearing, settlement, trade and transaction reporting.
In this case study, we will focus on the JSE - Exactpro collaboration to test the Millennium Exchange™ trading platform.
In 2015, after a successful Proof of Concept project of Exactpro’s bespoke suite of test automation tools, the JSE selected Exactpro as its software Quality Assurance service provider. Exactpro and the JSE have since collaborated on several successful test automation projects in the trading space including:
- JSE’s LSEG Technology Equities Test Library Enhancement and Test Automation;
- JSE’s LSEG Technology Derivatives Test Library Enhancement & Test Automation;
- JSE’s LSEG Technology Equity Market Enhancement (EME) Test Library Automation;
- JSE Order Generation Tool development for JSE’s LSEG Technology Derivatives testing.
JSE Project Requirements
The JSE requirements were as follows:
- To perform a review of manual test cases provided by the JSE and a gap analysis of the test coverage achieved through the manual test cases. Based on the results of joint analysis and the corresponding recommendations, Exactpro had to automate a number of test cases;
- To design, automate and execute test scenarios created by Exactpro for new market functionalities on different code drop versions of the trading system;
- To develop an order generation tool for Equity and Derivative platforms testing activities.
Effective bespoke test automation solutions with wide coverage and focus on maintainability and sustainability is one of Exactpro’s differentiators as a software Quality Assurance service company. The motto of Exactpro is “Build Software to Test Software”.
Exactpro Test Automation Approach
To emulate the JSE’s client activity and the system responses, Exactpro used its bespoke test tools:
Sailfish is an active real-time tool used for exchange, MTF and broker systems testing. It can also be used as an exchange simulator for testing post-trade systems or smart derivatives. This is a web-based application written in Java which supports the simultaneous connections of multiple users for both incoming and outgoing streams. Features of the tool include:
- Sending messages, capturing and storing the response in a database and generating a report with the expected vs. the actual result comparison;
- Support of concurrent automated overnight runs of multiple scenarios to ensure that reports can be checked in the morning;
- Versatile and independent scripts which are adaptable to different systems;
- Flexibility to expand or customize functionality depending on clients' needs.
Shsha is a post-transactional passive testing tool which allows testing the backend of trading platforms, market data, post-trade and market surveillance systems without interacting with them. Shsha ensures compliance with audit and regulatory requirements. Features of the tool include:
- Support of various industry-standard and proprietary protocols;
- Capability to capture network traffic, parse it and store the messages in a database;
- Capability to process large volumes of data from various clients;
- Capability to run SQL queries;
- Capability to verify test coverage;
- Support of summarized reports.
Load Injector is a powerful load generator built to stress test scalable high-load trading infrastructures. The tool supports FIX (all versions), ITCH, LSE Native, SOLA SAIL & HSVF, HTTP, SOAP, and various binary trading-system protocols. Its architecture is flexible thus enabling Load Injector to be configured to support other protocols. It supports both model and measurement approaches of performance testing. Features of the tool include:
- Simulation of multiple client connections with a specified load shape for each connection or a group of connections;
- Generates up to 75K messages per second from a single CPU core;
- Measures latencies in microseconds;
- Produces graphical performance test reports.
Case Study - Functional and Non-Functional Testing for the Millennium Exchange™ trading platform
Based on the documentation provided by the JSE team (technical specification of the interface, functional requirements for the trading system) the Exactpro development team adapted the above test tools for the JSE. The tools enabled the testing of the entire message flow between the client and the trading system, including positive (production-like) and negative input data testing. The tools also made it possible to automate and execute complex test scenarios covering a complete end-to-end cycle:
- order/quote submission, amendment and cancellation for all order/quote types;
- market data flow validation for a life cycle of an order/quote (including recovery/replay services);
- trade reporting and publication;
- basic (for Equities) and advanced (for Derivatives) matching logic checks;
- price monitoring (Price Bands and Circuit Breakers) validation;
- carry forward orders (Good Till Date, Good Till Cancel) overnight checks;
- request for quotes (RFQ) submission, execution and cancellation;
- implied order generation and execution;
- interfaces connectivity, authentication and session management;
- trading phases and instrument statuses publication throughout the day.
For certain scenarios, when a particular type of messages is populated without client intervention (e.g. early at day/market start, during the day all day or late at day/market end), the passive testing tool Shsha was used.
TCP/UDP dumps were captured independently on the backend, and were processed and parsed to human readable messages by Shsha and inserted into the database. This meant that we were able to check the specific scenarios offline at any time and audit the output of the trading system during the day (e.g. coverage analysis):
Exactpro successfully automated and tested the following functionalities of the trading system:
- Equity and Derivative Order/Quote Management;
- Order Matching and Price Monitoring;
- Order Book Reconstruction and Trade Publication;
- Request for Quotes (RFQ) Management;
- Trade Reporting;
- Daily Life Cycle;
- Client Connectivity and Authentication;
- Reference Data Management.
As part of the JSE ITaC programme Exactpro also successfully delivered a high frequency Order Generation Tool. The JSE business analysts needed the ability to generate a versatile and flexible load of messages and had specific requirements for trades/order ratio, order/quote types, instruments, markets and rates. To meet these requirements, the Load Injector tool was enhanced with the following features:
- support of message templates and functions in templates to generate all required types of messages with specific values;
- support of different rates for different markets (Equities and Derivatives);
- support of multiple order types for different types of instruments;
- support of multiple order/trade ratio across different segments;
- the option to shuffle the generated messages and send them into the system according to a predefined load profile.
The Order Generation Tool allowed us to create a complex load for the trading system and allow the execution of a comprehensive set of non-functional tests.
The test automation tools (Sailfish, Shsha, Load Injector – for test automation of trading, and ClearTH – for test automation of clearing) delivered to the JSE by Exactpro as part of our collaboration in the trading test automation area have been successfully used by the JSE team and helped with the successful implementation of the multi-year ITaC programme into production on 29 April 2019. The introduction of new, internationally recognized systems means the JSE will strengthen its position as a global market player providing more stable and efficient trading and clearing services to its clients.
"I have been receiving very positive feedback from my JSE teams about their work with Exactpro. The automated testing solutions allow us to perform in-depth testing to ensure software quality before deploying into live service,” says Hendrik Kotze, Chief Information Officer, JSE. “We will continue our collaboration with Exactpro who help us with quality deliverables to ensure that the JSE continues offering reliable, secure and efficient capital markets across a diverse range of instruments, supported by cost-effective services”.
Exactpro’s clients innovate. With testing services provided by us, they can adopt innovative technology with confidence and improve the resiliency, performance and quality of their systems and platforms. Our company achieves this through deep testing performed by our teams using our innovative tools developed in accordance with our test automation principles. We are empowered by experience gained over the years and embrace a culture of relentless learning and dedication to our clients.