Sailfish - Thorough Test Automation for Trading Systems
- Sends a message, captures the response and produces a report with expected vs. actual result comparison
- Allows concurrent automated runs of multiple scenarios overnight so that reports can be checked in the morning
- Uses versatile and independent scripts which are adaptable to different systems
- Has flexibility to expand or customize functionality depending on the clients' needs
- An open-source test tool (GitHub download here)
Sailfish is an active real-time tool used for exchange, MTF and broker systems testing. It can also be used as an exchange simulator for testing post-trade systems. Sailfish is a web-based application which supports simultaneous connections of multiple users for both incoming and outgoing streams. It is written in Java.
- Tests Order Entry, Market Data and Post-Trade connections in one test scenario
- Executes test scripts in any order
- Simulates multiple user connections
- Simulates servers
- Stores all messages in a database
- Generates test reports
Can Sailfish Be Customized?
You are welcome to make the best of the modular structure of Sailfish: build on the shared framework with the plug-ins that you need, or have us make a bespoke plug-in for your system requirements. Every extension is used independently for each protocol version. You can go with the industry-standard protocols, such as FIX or SWIFT, or proprietary ones.
Sailfish is a simple keyword-driven test tool. The test libraries we developed for it include tests for various business contexts that range from Regulated Markets to MTFs, from Dark Pools to Clearing Houses and Brokerage Systems. They have been successfully implemented in a wide range of technical and middleware infrastructures.
Being a back-end tool, Sailfish is connected to gateways/APIs capturing trading and/or market data traffic.
Sailfish makes it possible to achieve fully autonomous scheduled test execution that does not require ongoing monitoring.
Sailfish Test Script
Sailfish test scripts are Comma Separated Values (.CSV) files, where each column corresponds with a separate message field.
Different test scripts are independent from each other and can be run in any sequence.
Each line in the spreadsheet contains an action that Sailfish performs in the system (e.g. Send, Receive, Verify, Execution Report)
The Test Execution Report
The test execution report reflects the difference between the expected outcome and actual behavior. If there are no discrepancies in the actions, the test is passed. If at least one action is failed, the whole test-script is failed. The report is transparent : you can easily identify which actions are failed and why.
Characteristics of Sailfish
- Capacity & Precision
- Throughput 40 transactions per second with validations / 800 in performance mode. Time precision ~25-50 ms
- Testing Type
- Active Real-Time
- Target SUT
- Trading Platforms, Market Data Delivery and Post-Trade Systems
- SUT Interface
- Back-end (typically connected to message gateways / APIs, and DBs); GUI Testing Capabilities supported via plug-ins to other tools (e.g., Selenium)
- SUT Interaction Method
- Message injection and capture for testing of real-time low-latency bi-directional message lows; DB queries for data verification
- Extant plug-ins for Industry-standard (FIX and dialects, FAST, SWIFT, ITCH, HTTP, SOAP, etc.) and Proprietary (MIT, SAIL, HSVF, RTF, RV, Reuters, Fidessa OA, Quant House, etc.) protocols. New plug-ins for additional protocols developed by request (codecs are shared between Sailfish and Shsha)
- Test Scripts
- Human-readable CSV files; scripts generated manually by test analysts or automatically by test script generator using results of passive testing performed by another tool (e.g., Shsha)
- Test Management, Execution and Reporting
- Integrated (Web front-end), allows for multiple simultaneous heterogeneous connections, consecutive execution of multiple planned scripts, test results summary and detailed test reports. REST API supports remote control of Sailfish instances. Optional Big Button framework is supported.
- Platform requirements
- Low footprint cross-platform application, MySQL or other RDBMS.