Compatibility Testing of Clients’ Protocol Connectivity to Exchange and Broker Systems

Abstract - The software development life cycle for exchanges and brokers, in addition to verification of functional and technical system characteristics, includes a mandatory Integration Testing stage with the Client Certification part. The goal of this stage is to ensure the compatibility of automated trading systems connected to an exchange or broker via financial industry protocols (such as FIX/FAST, ITCH or specialized binary access interfaces). This paper presents a unique tool developed for verifying the compatibility of trading systems. One distinctive feature of the tool is a unified means of supporting multiple protocols. The case study provides examples of the tool usage by trading participants for self-certification and during large migrations of trading platforms. Keywords: financial protocols, FIX-protocol, compatibility testing, self-certification, trading broker, exchange.

Introduction

One of the key business processes at any exchange or broker organisation is adapting its new clients to a newly developed trading platform and their support when new software is introduced [1; 2; 3]. One of the key parts of the process is testing conducted in order to detect problems pertaining to compatibility of the trading platform with systems of trading participants that connect to the exchange or broker, i.e. client certification [4].

Client certification is mandatory for any operator of an exchange or broker platform providing an opportunity to perform financial transactions. Client certification has become a common practice in the financial industry [4]. The official web-sites of pertinent organizations contain links to client certification rules and related documentation [6; 7; 8; 9].

Software that has successfully passed internal integration testing then undergoes a final test – integration with clients (normally, legal entities). It is a common practice that the operator of a trading platform and its clients define a time period for active testing and each of them assesses its results on either end.

Due to the fact that representatives of different organisations are involved in certification testing, the process requires a substantial amount of coordination and team work. It is of utmost importance from both financial and reputational standpoints. Any software defect discovered during certification testing stage is expensive to fix as all SDLC cycles have been conducted [10] and any delays with detecting compatibility problems introduce significant additional costs.

Some of the typical challenges faced during client certification are:

  • the existence of world time zones and hence difficulties with planning and coordination of testing by teams of professionals physically located in various parts of the globe;
  • performance: certification can be required by several clients concurrently due to business or technical events, such as the deployment of new software, regulatory or technical requirements;
  • competency: Quality Assurance specialist conducting certification must have the adequate level of technical and business knowledge;
  • coverage: certification tests must be based on typical scenarios and produce meaningful results.

The financial industry community is starting to form the understanding of the fact that one of the ways to cope with these challenges may be the creation of more optimal automated technological solutions for certification testing and the analysis of test results [11]. Despite the importance of automation of exchange/broker client certification process, it seems that this topic has not been discussed at length in Russian or foreign scientific literature.

There are currently several commercial test tools built for faster client certification available on the market:

  • FIX Conductor by Lasalletech [12];
  • FACTS or B2BITS by EPAM Systems [13];
  • CeritFIX by Greenline [14];
  • Catalys Studio by Cameron [15];
  • Ignition by FIXFlyer [16];
  • VegaFABS by Pravega Financial Technologies [17];
  • Certpoint by Tradepoint Systems [18];
  • Certification Platform by FixSpec.com [19].

Exchange platforms currently utilize the following two main approaches to certification:

  • Provision of a specialized simulator to conduct certification to clients [20; 21; 22]. The clients conduct testing using the tool and provide certification logs;
  • Provision of access to a specialized test environment which represents a smaller replica of a production system; documenting the procedures; conducting joint certification with the client or client self-certification [23]; usage of passive methods to analyze the success of certification procedure, such as traffic capture of network connections or data from log files.

Both of these approaches have been implemented to one degree or another in the currently available commercial solutions.

The main advantage of the method based on simulators usage is the reduction of pressure borne by the certifying organisation and its clients thanks to the possibility of conducting certification at any time irrespective of trading session hours. A problem that is typical for all simulators is the lack of guarantee that the actual messages sent with their help are identical to messages sent by software used in production systems [24].

In our opinion, the process of client certification can benefit from the inclusion of passive testing in it. The overall concept of passive testing has been described in [25], various algorithms of passive testing are offered in [26; 27]. The main advantage of passive testing is the fact that the test tool does not affect the system under test and does not create additional message streams. On the other hand, this advantage can also be viewed as a a disadvantage: in order to successfully conduct passive testing the needed message streams have to be created, hence the need to involve representatives of another organisation to achieve that.

This practice is inevitable during conducting the initial level of certification. However, to minimize the time spent by third-party organizations it is necessary to utilise more effective methods, an example of which would be an approach described in [28]. By using the results of the initial level of certification for automated creation of new test scenarios, and provided that the access protocol specification remains the same, the trading platform operator receives an opportunity to conduct the second round of client certification on its own.

Client certification tools currently available on the market have a number of limitations, including:

  • exclusive for one protocol only, e.g. FIX [29];
  • interactive tools for data analysis and creation of new scenarios can hardly be used for processing massive amounts of heterogeneous client connections data.

The authors are currently involved in the development of a software solution by Exactpro Systems LLC targeted at overcoming the limitations mentioned above [30; 31], and using it in practice. The second part of this paper describes the solution. The third part contains a case study of utilizing the solution for trading exchange participants’ self-certification. The fourth part provides information regarding the usage of the developed solution during large-scale migrations of broker systems.

A Multiprotocol Solution for Passive Testing of Trading Systems

A Java- and MySQL data base management system-based tool for passive multiprotocol testing of network connections to trading systems has been developed. The approach is based on a unified description of protocol messages structure and a dictionary system. A dictionary is created for every protocol. Special modules transforming data streams into a unified internal format based on the dictionaries may be required for a group of protocols. The concept is alike the logic of Complex Event Processing [32]. A dictionary template used in the QuickFIX/J system [33] was taken as the basis and modified to achieve a higher level of abstraction.

The developed tool can receive incoming data on network traffic captured by using tcpdump [34] or various proxy-servers [35]. The user has the opportunity to load log-files containing large amounts of messages in an adjustable format.

The main table in the data base holds information about every captured packet, including re-sent TCP-data. If the trading system is under load the network packet can contain several messages, or a message can be split between several TCP-packets. Having extorted the messages from the network packets, the tool attempts to convert them into the unified format based on the dictionaries. The information about the network packets and messages that has not been checked by using the dictionaries is put into a separate table, which in essence contains the list of problems pertaining to the initial level of certification.

Details of captured and successfully processed messages can be stored in a relational data base by using one of the following main methods:

  • use of the general table containing message IDs, parameter name and its value;
  • use of a separate table for every message type where the message parameters are stored in the corresponding columns with names derived from parameter names.

The advantage of the first method is a high degree of abstraction. The second method is beneficial in terms of the ease of generating SQL queries in the data base. The developed tool uses a combination of both methods: every type of message has a corresponding table, and the data from repeating groups is stored in the general table. Picture 1 below shows how the developed tool is used for capturing and storing network messages.

Compatibility Testing of Clients’ Protocol Connectivity to Exchange and Broker Systems - Message processing in the developed certification testing tool

Pic. 1. Message processing in the developed certification testing tool.

The captured messages are decoded and placed in the tool’s data base. All certification reports are stored in the form of SQL queries. There is no need to amend the tool’s code if a new type of report or a new requirement is added.

By running a SQL query, the user has an opportunity of finding a list of messages and their status and highlight them with a certain colour via the tool’s GUI for better visibility.

SQL queries can be used for finding the needed messages-related statistics, the number of logons to applications, unparsed messages, etc.

Via the Graphical User Interface of the developed tool, the analyst responsible for Certification or Quality Assurance of the trading platform can view messages and events in real time, including network packet details, the results of verification by using the dictionary system, the values in individual fields, and the source binary data.

Certification tests and data reconciliation may be performed by using ordinary SQL queries. The tool allows highlighting and analysis of messages captured as a result of executing various queries. Picture 2 shows an example of a GUI window of the developed tool.

Compatibility Testing of Clients’ Protocol Connectivity to Exchange and Broker Systems - Graphical User Interface of the developed certification tool

Pic.2. Graphical User Interface of the developed certification tool.

Self-Certification of Exchange Trading Participants

Regulatory bodies mandate that the exchange operators provide equivalent access for all trading participants [36]. If the technology platform of an exchange undergoes changes, its operator must conduct certification of all connected trading systems within a short period of time. This requirement creates a significant work load for exchange operator’s departments responsible for client support. In order to reduce the work load, exchange organisations use the method of self-certification.

The clients are granted access to a test environment and provided with a scenario of performing certification tests. Having gone through the steps of the scenario the client sends the pertinent logs to the certifying organisation where the logs undergo additional verification. The process of trading participant certification can be significantly simplified if passive testing is part of it. The trading participants can just connect to the test environment and send the orders listed in the provided test scenario. They are therefore relieved of the duty to perform any tasks (such as the installation of additional software, log collection, etc.) other than those that are needed to connect to the exchange’s technology platform.

The developed tool captures all messages sent to the exchange by the client or vice versa, parses each message and puts the data into a relational data base. By using SQL queries, the certifying organisation’s analyst can receive the statistics about the attempts to perform the steps in the test scenario and their success for every trading participant. Network packets data also allows finding additional problems, such as connectivity breaks or message buffering issues.

Let’s review an example of a certification SQL scenario verifying the success of placing an aggressing market order which, as a result of its execution against orders placed on the opposite side of the book, leads to two or more trades:

insert into t_native_testcases
(user,sourceip,sourceport,testcase,timestamp,clordid,orderid,otherid)
select distinct n.user, n.sourceip, n.sourceport, 'MEx-012.2 Agg. MO' as testcase, n.timestamp, n.clordid, e.orderid, '' from t_lsenative_neworder n
, t_lsenative_executionreport e
, t_lsenative_executionreport e2
where n.user=e.user
and n.sourceip=e.destinationip and
n.sourceport=e.destinationport
and n.clordid=e.clordid and n.user=e2.user and
n.sourceip=e2.destinationip
and n.sourceport=e2.destinationport and n.clordid=e2.clordid
and n.ordertype=1 and e.ordstatus=1 and
e.tradeliquidityindicator='R' and e2.typeoftrade='2' and e2.ordstatus in (1,2)
and e2.tradeliquidityindicator='R' and e2.typeoftrade='2'
and e.execid <> e2.execid
order by user, clordid, orderid;

Groups of SQL scenarios covering all client connectivity certification requirements issued by the trading coordinator for a number of exchange platforms have been created [9].

Migrating a Broker Platform with Multiple Heterogeneous Client Connections

In contrast to exchange platforms, broker systems normally allow a much wider range of possibilities for tuning the protocol for interaction with client systems: for example, by using such systems as UL Bridge [37]. One client can concurrently use several brokers for accessing an exchange. Brokers often try to reduce the number of changes in communication protocols which a particular client must implement. This approach simplifies client interaction and leads to the existence of a large number of heterogeneous configurations on the broker platform end. If the internal changes of the broker platform take place, a need in a regression test occurs to verify compatibility with the client systems.

The developed tool allows processing of existing data from test and production environments and creating the needed library of active testing scenarios. Having performed these test scenarios in the test environment without involving the end clients, the tool launches SQL scenarios to compare the responses of the broker platform before and after the changes. The unique feature of the developed tool is the possibility to run this process concurrently for a large number of clients, connections, markets and types of messages. The flexibility of the method of creating SQL queries allows meeting the requirements for processing more complex differences, in addition to just excluding fields that are known to contain differing values (e.g. sequence numbers, timestamps, etc.) from comparison.

The approach has proven its reliability and efficiency when used during the migration of a trading platform of a major international broker organisation providing access to derivatives trading. The sequence of steps is shown on Picture 3.

Compatibility Testing of Clients’ Protocol Connectivity to Exchange and Broker Systems - A schematic example of a certification testing tool implementation

Pic.3. A schematic example of a certification testing tool implementation.

Production logs are loaded into the certification testing tool’s data base. The script generator is using the data base for creating a sequence of test scenarios performed by an active test tool for functional testing. Logs generated when test scenarios are executed are loaded into the data base to compare them with production logs.

Conclusions

As automated electronic trading volumes grow, the stability of the financial markets will more and more depend on the correctness of concurrent functioning of platforms supporting market infrastructure and automated trading systems connected to them. The cost of connecting new clients, their certification and ensuring their compatibility when changes are introduced on a regular basis has a significant impact on the overall cost of keeping exchanges and broker systems operational. The tool described in this paper is used in order to raise the efficiency and lower the cost of the processes. The tool has been successfully used at the final stages of deploying large scale systems for trading various asset classes of financial instruments into production, which Exactpro Systems LLC has been part of.

The paper describes the methods of client connectivity certification currently used in the industry and provides conclusions as to the advantages of an approach based on using a multiprotocol passive test tool which allows retaining the data in a relational DBMS. The paper also showcases a method of using the tool for creating active tests and reducing the cost of large scale migrations of systems with multiple heterogeneous client connections.

Further research will be focused on optimising the data base structure: in particular, expanding the developed methods to trading protocols based on program access interfaces (API) rather than on network interactions.

References

  1. Markit Magazine Issue 10 - Focus: Client Onboarding – Markit.com // [Electronic source]. – Access regime http://www.markit.com/assets/en/docs/markit-magazine/issue- 10/mm10_focus5-onboardingroundtable.pdf
  2. Fenergo 2013, Client Onboarding: Solving the Challenges, Maximizing the Opportunities. // [Electronic source]. – Access regime fenergo.com/industryknowledge/whitepapers/client-onboarding-solving-challenges-maximizingopportunities.html
  3. Axel Pierron, Anshuman Jaswal: Institutional Client On-Boarding in the Financial Industry Time to Move to the Industrialization Phase. // Celent Report, October 2012 // [Electronic source]. – Access regime celent.com/reports/institutional-client-boardingfinancial-industry
  4. Challenges and Solutions to Onboarding Trading Clients. // [Electronic source]. – Access regime http://www.trdpnt.com/blog/bid/294222/Challenges-and-Solutions-to-Onboarding- Trading-Clients
  5. FIA Market Access Risk Management Recommendations April 2010. // [Electronic source]. – Access regime http://www.futuresindustry.org/downloads/Market_Access-6.pdf
  6. Moscow Exchange. // [Electronic source]. – Access regime http://fs.moex.com/files/4531
  7. BOX Options Exchange. // [Electronic source]. – Access regime boxexchange.com/what-you-need-to-know/software-certification/
  8. Eris Exchange. // [Electronic source]. – Access regime http://www.erisfutures.com/sites/default/files/Electronic_Trading_Certi…
  9. TQ 601 - Technical Specification Turquoise Equities Guide to Certification. // [Electronic source]. – Access regime http://www.lseg.com/sites/default/files/content/documents/TQ601%20- %20Guide%20To%20Application%20Certifcation.pdf
  10. Forrest Shull, Vic Basili, Barry Boehm, A. Winsor Brown, Patricia Costa, Mikael Lindvall,Dan Port, Ioana Rus, Roseanne Tesoriero, Marvin Zelkowitz: What We Have Learned About Fighting Defects // METRICS '02 Proceedings of the 8th International Symposium on Software Metrics, IEEE Computer Society Washington, DC, USA 2002
  11. Candyce Edelen: Making the Case for Automated FIX Certification. // [Electronic source]. – Access regime advancedtrading.com/high-frequency/making-the-case-forautomated-fix-certif/240155554
  12. FIX Conductor™ from Lasalletech. // [Electronic source]. – Access regime lasalletech.com/products/fix-automated-onboarding
  13. FACTS от B2BITS, EPAM Systems. // [Electronic source]. – Access regime http://www.b2bits.com/trading_solutions/fix_testing_facts.html
  14. CertiFIX® from Greenline. // [Electronic source]. – Access regime greenlinetech.com/products/certifix.php
  15. Catalys Studio from Cameron. // [Electronic source]. – Access regime http://www.camerontec.com/products/catalys/catalys-studio/
  16. Ignition from FIXFlyer. // [Electronic source]. – Access regime http://fixflyer.com/materials/software/Ki/FlyerIgnition.pdf
  17. VegaFABS from Pravega® Financial Technologies. // [Electronic source]. – Access regime pravegatech.com/index.php?option=com_content&view=article&id=64&Itemid=65
  18. Certpoint from Tradepoint Systems. // [Electronic source]. – Access regime trdpnt.com/certpoint/
  19. Certification Platform from FixSpec.com // [Electronic source]. – Access regime http://forexmagnates.com/fixspec-launches-innovative-multi-venue-certification-utility-tostreamline- connectivity/
  20. The Stock Exchange of Hong Kong Limited. // [Electronic source]. – Access regime http://www.hkex.com.hk/eng/market/partcir/sehk/2012/Documents/CTMO06612E.pdf
  21. MyCTC (Certification and Testing Center) from BM&FBOVESPA. // [Electronic source]. – Access regime http://www.bmfbovespa.com.br/en-us/services/download/MyCTC-User- Guide.pdf
  22. CME Group. // [Electronic source]. – Access regime http://www.cmegroup.com/confluence/display/EPICSANDBOX/Client+System+Certification
  23. The London Stock Exchange. // [Electronic source]. – Access regime lseg.com/areas-expertise/technology/market-connectivity/customercertification- and-testing-services
  24. Zverev A., Bulda A.: Exchange Simulators for SOR/Algo Testing: Advantages vs. Shortcomings. / ExTENT conference. // [Electronic source]. – Access regime https://speakerdeck.com/exactpro/algo-testing-advantages-vs-shortcomings
  25. K. M. Brzezinski: Towards the Methodological Harmonisation of Passive Testing Across ICT Communities // Engineering the Computer Science and IT, October 2009
  26. David Lee, Dongluo Chen, Ruibing Hao, Raymond E. Miller, Jianping Wu and Xia Yin: Network Protocol System Monitoring – A Formal Approach With Passive Testing // IEEE/ACM Transactions on Networking, Vol. 14, No. 2, April 2006
  27. Ana Cavalli, Stephane Maag, Edgardo Montes de Oca: A passive conformance testing approach for a MANET routing protocol // Proceedings of the 2009 ACM symposium on Applied Computing March 2009, SAC '09
  28. James Newsome, David Brumley, Jason Franklin, Dawn Song: Replayer: Automatic Protocol Replay by Binary Analysis // CCS '06: Proceedings of the 13th ACM conference on Computer and communications security, October 2006
  29. FIX Protocol. // [Electronic source]. – Access regime http://fixprotocol.org
  30. Zverev A., Moskaleva O., Kudryavtseva M., Doronichev D., Bulda A.: Four Houses – Test Tools presentation. / ExTENT conference // [Electronic source]. – Access regime http://www.slideshare.net/extentconf/extent3-exactpro-fourhousestesttools2012-1
  31. Zaitseva N., Popovchuk N.: Next Step in Reconciliation Testing. / ExTENT conference // [Electronic source]. – Access regime http://www.slideshare.net/extentconf/extent3-exactpro-thenextstepinreconciliationtesting
  32. G. Cugola and A. Margara: Processing flows of information: From data stream to complex event processing // ACM Computing Surveys, 2011.
  33. QuickFIX/J . // [Electronic source]. – Access regime http://quickfixj.org
  34. Felix Fuentes, Dulal C. Kar: Ethereal vs. Tcpdump: a comparative study on packet sniffing tools for educational purpose // Journal of Computing Sciences in Colleges , Volume 20 Issue 4, April 2005
  35. The Grinder TCP Proxy. // [Electronic source]. – Access regime http://grinder.sourceforge.net/g3/tcpproxy.html
  36. Investment Services Directive – Markets in Financial Instruments Directive (MiFID)
  37. UL Bridge. // [Electronic source]. – Access regime ullink.com/fixsolutions.php