A Case Study for Testing Distributed Ledger Technology (DLT) Platforms and Infrastructures

Introduction

This case study discusses the development of new techniques for the functional and non-functional testing of Distributed Ledger Technologies (DLT), in particular, for their application to mission-critical Financial Market Infrastructures.

Exactpro expertise encompasses:

• testing enterprise DLT platforms – we have extended our comprehensive functional and non-functional test automation capabilities to meet the new and exacting testing requirements of enterprise delivered infrastructure, whether it is delivered on-prem or in the cloud, or in multi-cloud deployments;
• building applications on DLT – building and testing workflow solutions based on the latest DLT technologies;
• network testing of notaries – applying existing tools and expertise to test the latency and throughput of a complex multi-geography, multi-node network.

There are many challenges in testing distributed ledger platforms, including the interoperability of nodes in the network and the large number of connection permutations between them, understanding the differences between varying DLT permissioning logic. For testing, this leads to hundreds of millions of tests – a problem only solved by automation.
Furthermore, the technology is new and evolving quickly, and projects tend to be driven by two main factors: taking POC software into production and ensuring that platforms meet existing regulatory standards.

Functional and Non-Functional Testing for Distributed Ledger Technology Platforms: Challenges

We have encountered the following challenges when testing DLT-based platforms:

• Complexity – DLT-based infrastructures are complex and vary in their architecture. They can be extremely heterogeneous, i.e. it can be possible to deploy different versions of an application on different security devices, different databases, different Java machines and operating systems. This creates a combinatorial explosion when one tries to test them.
• Interoperability – A number of issues arise at the confluence between a distributed ledger and its connectivity to a non-DLT platform.
• High transaction throughput – such platforms are expected to be performant. So, it is necessary to do quite sophisticated performance tests.
• DLT-based infrastructures can be highly distributed and multi-threaded; that requires security, they need to be consistent (e.g. it becomes consistent after a certain delay).
• DLT platforms evolve because there are continuous upgrades to nodes, applications, networks.
• High availability (HA) is a must. The instruments ensuring there is no double-spend need to be highly resilient.

Test Automation Framework for DLT projects

ClearTH – an innovative way to test Clearing, Settlement and Back-Office systems. It is a unique Exactpro tool able to simultaneously execute multiple End-to-End test scenarios in batches. ClearTH easily detects abnormal behavior in the system under test and effectively predicts potential issues. It offers many built-in actions to cover the majority of activities in post-trade systems.

• Verifies each stage of the DLC
• Has an integrated schedule
• Automatically runs test scripts
• Creates multiple-day test scenarios
• Performs multiple concurrent tests
• Has integrated simulators
• Supports SWIFT ISO protocol

Woodpecker – a ClearTH extension acting as a platform for daemons combining functional and non-functional test approaches.

• Highly customizable (data source, data types, load, time, interface)
• Easy to transfer to different environments due to its ability to extract data right from the system under test
• Automatically generated test scenarios cover a vast majority of diverse conditions/data combinations
• A productive intersection of functional and non-functional testing approaches

ClearTH Test Automation Framework Adapted to DLT Projects

• The same complexity of participants/processes/asset classes
• Hybrid nature: conventional functionality + new business logic involving interactions within a blockchain network
• High diversity of underlying DLT frameworks
• New form of business logic execution – smart contracts
• Ability to support messaging interaction
• Ability to emulate separate nodes
• Triggering workflow via REST API/RPC calls
• Protocol-level interactions simulation
• Ability to capture the data flow and perform ledger verification checks

 

Case study 1: Functional Testing for Distributed Ledger Technology Platform – Corda

Read the full Case Study

ClearTH allows for:

• Nodes deployment with an ability to change the configuration
• CorDapps deployment
• Nodes Administration
• Nodes registration in the Network
• Trigger transactions generation, verification and distribution across the nodes
• JVM statistic gathering
• Log monitoring and analysis
• DB verification and monitoring (low-level ledger update validation)

Case study 2: Non-Functional Testing for DLT Platform – Corda

Non-Functional Testing of any platform is complex and requires robust configuration and may require preliminary development activities.
To run automated non-functional tests, it is required to monitor various metrics of the platform under test and its hardware usage. Having the option to start/stop/adjust load injection or other actions based on the current system state is also a must. The Exactpro automation framework allows to initiate flows across large-scale networks of nodes against a remote network. We use open-source tools to gather metrics and visualize the results.

Case Study 3: Distributed Ledger Technology in Position Update Business Flow

3 steps on the ledger side:

1st  Initialisation of positions

2nd Trade creation and participant notification

3rd  Positions update

Across each and every step, the test framework provides different methods to validate the endpoints, inputs and outputs.
The actual business flow steps and validation check items are transformed into the ClearTH matrix, after the execution of which the deviations from the expected behaviour can be analysed.

Case Study 4: PoC for IRS & CDS Swaps Lifecycle Module Based on CDM

ClearTH simulates a participant who submits a trade. The script sets the ledger time to the next fixing date and simulates floating rate fixings by the market data provider. Then it “time travels” to the next reset date (payment date) and generates derived events.

Model for test automation:

1. A template with the swap contract info (swap type and structure) and lifecycle specifics (e.g. segregation of trades or multiple contracts by one member, event schedules, etc.).
2. Spreadsheets with Input data: a Config file with different combinations of parameters values; reference and market data (e.g. floating rate fixings).
3. Macros generating multiple scenarios in the form of csv matrices – scripts executed by ClearTH.

Read more:
https://exactpro.com/news/exactpro/exactpro-enables-clearth-test-automation-framework-daml-built-applications
https://github.com/digital-asset/ex-cdm-swaps

ClearTH for DAML-based dApps Working on Top of Different DLT Frameworks

• DAML – a smart contract language with the capacity to run on top of multiple blockchains.
• ClearTH triggers different life cycle events (e.g. cash flow generation, trade execution, novation, termination, etc.).
• Simulates potential actions of different parties involved in a business process (e.g. initiating transactions or accepting trades).
• The ClearTH framework performs checks of the transaction results (account balance totals, interest rates calculations, the status of the trade – proposed, accepted or applied).

Exactpro Awarded Innovation Accolade for Blockchain Project with R3

"There are many challenges in testing distributed ledger platforms, including the interoperability of nodes in the network and the near infinite permutations of connections between them. It's a challenge that expands into hundreds of millions of tests, and automation is the only viable solution," added James Carlyle, CIO, Head of Production, at R3.

Read more:
https://exactpro.com/news/exactpro/exactpro-awarded-innovation-accolade-blockchain-project-r3

Exactpro’s ClearTH Wins Best Tool Demo Award