Software Testing for DLT Platforms
Software testing for financial technology platforms built with DLT has multiple aspects and is, therefore, associated with a whole set of requirements to the test harness required to support it. Test tool requirements:
- Open interface and technology stack diversity: to provide intelligent interaction with many widely adopted network protocols as well as API, UI, DLT and cloud endpoints;
- Configuration agility: to build complex test instruments capable of executing sophisticated test algorithms, including model-based testing approaches, to address complex fintech use cases;
- Data governance: to enable better access to test evidence and smart analytics for governance and regulatory compliance;
- End-to-end testing capabilities: to support extensive test libraries covering complex business life cycles and perform system verification at the confluence of functional and non-functional requirements;
- High-volume automated testing: designed to perform autonomous test execution of more functional tests under load to improve test coverage, system quality and resilience.
Functional Testing for DLT Platforms
Functional testing of DLT platforms suggests a number of test activities complementing functional checks typically performed to assess the correctness of a test system’s behaviour.
DLT-specific test harness capabilities:
- Nodes deployment with an ability to change the configuration
- CorDapps deployment
- Nodes administration
- Nodes registration in the Network
- Trigger transactions generation, verification and distribution across the nodes
- Database verification and monitoring (low-level ledger update validation)
Additional capabilities:
- JVM statistics gathering
- Log monitoring and analysis
Covering business flows on top of DLT networks
Interbank payments: Position update business flow
3 steps on the ledger side:
1st Initialisation of positions
2nd Trade creation and participant notification
3rd Positions update
At each of the steps, the test framework provides different methods to validate the endpoints, inputs and outputs.
The actual business flow steps and validation check items are transformed into the test script. Upon its execution, the test tool’s Report UI provides a detailed report showing deviations from the expected behaviour.
IRS & CDS Swaps Lifecycle Module Based on ISDA CDM
Business flow:
The test harness simulates a participant who submits a trade. The script sets the ledger time to the next fixing date and simulates floating rate fixings by the market data provider. Then it “time travels” to the next reset date (payment date) and generates derived events.
Test capabilities:
- External system/participant simulator
- In-built schedule to prompt interactions with DLT network
- Testing model for test input and output generation
- Test script containing a template with a swap contract info (swap type and structure) and lifecycle specifics (e.g. segregation of trades or multiple contracts by one member, event schedules, etc.).
Non-Functional Testing for DLT Platforms
Non-functional testing of any platform is complex: it requires comprehensive configuration and, most often, preliminary development activities.
To run automated non-functional tests, one needs to monitor various metrics of the platform under test and its hardware usage. Having the option to start/stop/adjust load injection or other actions based on the current system state is also a must. The Exactpro framework for automation in testing allows to initiate flows across large-scale networks of nodes of a distributed system. To gather metrics and visualise the results, our test framework uses its monitoring capabilities built using open source third-party tools.
Load & Stress Testing
To determine the maximum throughput processing on the reference infrastructure with no chain history and one state per transaction, the PerfTest CorDapp and a specific custom flow were used for a specific use case. This minimised the number of variables and allowed us to report the values in a meaningful way. In a DLT network, transaction throughput (measured in TPS) is a function of many dimensions (attachments, chain length, number of input/output states).
Failover
To ensure there is no loss of service in transaction notarisation with a variety of test CorDapps, the tool forces node failures within the network and checks the system’s behaviour.
Performance Testing
For test execution, latency characteristics are determined at certain quanta of the maximum transaction processing rate with a no-chain history.
Upon test run, the test tool generates a report on latency values with pre-determined input rates (usually, 25, 50, 100, 150% of the maximum value). For each input rate, the report specifies:
- average latency per transaction processing (mean and 99th percentile)
- average compute time (mean and 99th percentile)
- latency and compute time distribution charts
Soak Testing
Soak tests aim to evaluate the ability of a system under test to sustain load for prolonged periods of time.
In the example below, we set up a DLT environment on Microsoft Azure, with a multi-threaded RPC injector distributed across multiple global locations to initiate flows across large-scale networks of nodes against a remote network outside of the framework. Docker images are configured to run specific Corda enterprise versions, CorDapps and RPC clients for instantiation of test flows. System monitoring is implemented via open-source tools, such as DataDog, Graphite (for JMX stats), and bespoke scripts for additional memory monitoring.
For each node with a variety of test DApps, the test tool determines memory usage. In our example, two DApps are used to drive transactions through the network over an extended period of time, with a pre-determined input rate approaching its maximum value. Memory usage of a node JVM is monitored over
time while it is processing tens of millions of transactions (e.g. whether it grows, if memory is released, whether the node fails and/or restarts, etc.).
Exactpro Awarded Innovation Accolade for Blockchain Project with R3
London, 12th May 2020
Exactpro, a leading software testing provider for financial market infrastructures (FMIs), is pleased to announce the receipt of QA Financial’s “Automation Technology Project of the Year: Exactpro and R3” award, in recognition of its work with R3’s Corda Enterprise Distributed Ledger Technology (‘DLT’). Exactpro delivers testing services underpinned by tools and proven methodologies as a result of a significant R&D investment. The project saw the extension of comprehensive functional and non-functional capabilities to meet the new and exacting testing requirements of the latest Corda Enterprise DLT release, itself forming the basis of a few significant new FMI-driven platforms.
"There are many challenges in testing distributed ledger platforms, including the interoperability of nodes in the network and the near infinite permutations of connections between them. It's a challenge that expands into hundreds of millions of tests, and automation is the only viable solution," added James Carlyle, CIO, Head of Production, at R3.
Read more:
https://exactpro.com/news/exactpro/exactpro-awarded-innovation-accolade-blockchain-project-r3
Exactpro’s ClearTH Wins Best Tool Demo Award
Exactpro’s ClearTH test automation framework for DLT-based Post-Trade Systems research based on R3 Corda technology wins Best Tool Demo award at the 12th IEEE International Conference on Software Testing, Verification and Validation in 2019.
The demo can be viewed at https://www.youtube.com/watch?v=eKFc9yKGfuI
Read more
Distributed ledger technology (DLT) innovation is an important trend shaping the financial services industry. It’s no surprise that under the pressure of ongoing overall digital transformation, many financial institutions are keen to adopt DLT in development of their platforms. However, when the innovation is introduced on a scale of a major capital market participant, its potential impact is likely to be shared by a larger financial services community. To facilitate the smooth transition towards the emerging technologies, the regulatory bodies require systemically important financial market infrastructures to ensure that while innovating, they also stay operationally resilient. Read more