Black-Box Testing of Financial Virtual Assistants

Authors: Iosif Itkin, Elena Treshcheva, Luba Konnova, Pavel Braslavski, Rostislav Yavorskiy


Abstract:  We propose a hybrid technique of black-box testing of virtual assistants (VAs) in the financial sector. The specifics of the highly regulated industry imposes numerous limitations on the testing process: GDPR and other data protection requirements, the absence of interaction logs with real users, restricted access to internal data, etc. These limitations also decrease the applicability of a few VA testing methods that are widely described in the research literature. The approach suggested in this paper consists of semi-controlled interaction logging from the trained testers and subsequent augmenting the collected data for automated testing.


The paper was presented at the 2020 IEEE 20th International Conference on Software Quality, Reliability and Security in December 2020 in Macau, China. Full text is available for download via the link.