About Workshops
In our workshop series we will go over several hands-on exercises which will help you understand practical basics of AI and teach you some methods and tools used by the AI specialists, data scientists and AI testers during their work with AI-based systems. By the end of the course, you will be able to work on real-life datasets, build and test ML models using the skills you have acquired. You will learn to implement fundamental AI concepts using Python on the Google Colab platform, which is widely favored by AI and ML practitioners.
Workshop 1
Part of Guided plan
100 minsOverfit, Underfit and Good Models
What is included:
- ML model building
- Model evaluation
- Regression and prediction with polynomial and decision tree algorithms
Workshop 2
Part of Guided plan
Part of Accelerated plan
60 minsData Preparation for ML
What is included:
- Data preparation for ML
- Exploratory data analysis
Workshop 3
Part of Guided plan
Part of Accelerated plan
60 minsML Model Generation & Test
What is included:
- Dataset splitting using Scikit-learn library
- Building ML model using multinomial Naive Bayes classifier
Workshop 4
Part of Guided plan
Part of Accelerated plan
60 minsML Model Evaluation
What is included:
- Model evaluation methods
- Confusion matrix analysis using Scikit-learn and Matplotlib libraries
- Functional performance metrics
Workshop 5
Part of Guided plan
60 minsSimple Perceptron
What is included:
- Introduction to neural networks
- Supervised learning and training a simple perceptron model
Workshop 6
Part of Guided plan
Part of Accelerated plan
60 minsModel Explainability
What is included:
- Model explainability with the lime tool examples
Workshop 7
Part of Guided plan
60 minsPairwise Testing
What is included:
- Building support vector classification (SVC) model
- Pairwise testing with AllPairs tool
- Automated testing with Pytest framework
Workshop 8
Part of Guided plan
60 minsMetamorphic Testing
What is included:
- Metamorphic testing
- Plotting a correlation heatmap and parameter distributions using Seaborn library
- Test case generation
- Multiple test run with Pytest framework
Workshop 9
Part of Guided plan
60 minsExploratory Testing and EDA
What is included:
- Exploratory data analysis
- Getting information about data using the Pandas library
- Data visualisation using Matplotlib and Seaborn libraries
Workshop 10
Part of Guided plan
30 minsThe Use of AI in Testing
What is included:
- Available and impractical AI-based testing activities
Workshop 11
Part of Guided plan
60 minsBuild a Defect Prediction System
What is included:
- Defect prediction using Random Forest algorithm
- Feature engineering
Workshop 12
120 minsTest Scenario Generation and
Leveraging Code Coverage Data
What is included:
- Synthetic data generation
- Code coverage
- Imitation of high-frequency trading bot interactions with the stock exchange
- Using AI to optimise testing strategies
Overfit, Underfit and Good Models
100 minsPrerequisites:
ISTQB® CT-AI Syllabus Chapter 3
Basic Python programming skills
You will start with the fundamental principles of building an ML model based on available data using Python. You will explore the concepts of underfitting and overfitting, learning how to distinguish between them on different real-life models using various techniques, as well as visualising ML models. Additionally, you will learn how to build an ML regression model and make predictions using a polynomial model and a decision tree algorithm. You will also gain practical experience in evaluating ML models by calculating different ML functional performance metrics.
Data Preparation for ML
60 minsPrerequisites:
ISTQB® CT-AI Syllabus Chapter 4
You will practice data preparation activities for ML models, such as data acquisition (identification, gathering, and labelling), data pre-processing (cleaning, transformation, augmentation, and sampling), and feature engineering (feature selection and extraction). Additionally, you will perform exploratory data analysis to support the overall data preparation process and identify potential data issues.
ML Model Generation & Test
60 minsPrerequisites:
ISTQB® CI-AI Chapter 4
Workshop 2
You will learn about the importance and difference between training, validation and test datasets. You will practice splitting the initial data into these three parts using the Scikit-learn library, and build the classification model based on the multinomial Naive Bayes classifier. You will also learn how the volume of training data affects the accuracy of the ML model. You will explore techniques for tuning the model and effectively testing its performance.
ML Model Evaluation
60 minsPrerequisites:
ISTQB® CI-AI Chapter 5
Workshop 2&3
You will explore various methods for evaluating ML models. This will include practising both manual and automated calculations of different functional performance metrics and discussing the criteria for selecting appropriate metrics based on real-life scenarios. You will calculate and visualise the confusion matrix for a multilabel classification model using Scikit-learn and Matplotlib libraries and practice in different possible ways of metrics calculation based on this confusion matrix.
Simple Perceptron
60 minsPrerequisites:
ISTQB® CI-AI Chapter 6
You will be introduced to neural networks, beginning with the implementation of a simple perceptron model. You will understand that a neural network can be used for the supervised learning tasks and also as a binary linear classifier, and practice training your perceptron model with basic logical functions such as AND and OR.
Model Explainability
60 minsPrerequisites:
ISTQB® CI-AI Chapter 8
Workshop 2
Workshop 3
Workshop 4
You will learn how to make your ML model transparent in its decision-making process and in arriving at specific results. You will be introduced to various methods that enhance the explainability of AI-based systems. Using the open-source Local Interpretable Model-agnostic Explanations (LIME) tool, you will provide explainability for the classification model and visualise the resulting explanations.
Pairwise Testing
60 minsPrerequisites:
ISTQB® CI-AI Chapter 9
You will practise a pairwise testing technique for testing AI-based systems. Using the open-source AllPairs pairwise testing tool, you will generate a reduced set of pairwise combinations of parameters for a support vector classification (SVC) model designed to address a real-world problem and create test scenarios providing good test coverage while reducing the effort. Additionally, you will use the free, open-source pytest Python testing framework to execute multiple tests while making predictions for all pairwise combinations of the model parameters.
Pairwise Testing
60 minsPrerequisites:
ISTQB® CI-AI Chapter 9
You will derive metamorphic relations for the ML classification model based on the analysis of the model parameters, their values and relations. You will use the Seaborn library to plot the correlation heatmap and different parameter distributions. You will generate follow-up test cases based on the source test cases and consider two possible types of metamorphic relations: when the expected result is the same and not the same as the original expected result of the source test case. You will use the pytest testing framework for Python to run multiple tests.
Exploratory Testing and EDA
60 minsPrerequisites:
ISTQB® CI-AI Chapter 9
You will perform the experience-based testing by applying different exploratory testing and exploratory data analysis techniques. You will learn how to gain a quick understanding of the structure and content of a dataset, and how to obtain its general statistical information. You will apply different data visualisation techniques using Matplotlib and Seaborn, two popular Python libraries.
The Use of AI in Testing
30 minsPrerequisites:
ISTQB® CI-AI Chapter 11
You will learn how AI can be used in testing, focusing on how it can help optimise testing and which testing activities are currently impractical to be implemented using AI.
Build a Defect Prediction System
60 minsPrerequisites:
ISTQB® CI-AI Chapter 11
You will build a defect prediction system based on the source code metrics analysis using the Random Forest algorithm. You will practice feature engineering and learn that the selection of features has a significant impact on the prediction results.
Test Scenario Generation and
Leveraging Code Coverage Data
120 minsPrerequisites:
ISTQB® CI-AI Chapter 11
This interactive 2-hour workshop introduces practitioners to synthetic data generation as a way to enhance software testing workflows. Participants will explore how to work with a library of test scenarios as a dataset, produce additional data based on code coverage information, and how to leverage machine learning to optimise their testing strategies. The tools and resources demonstrated during the workshop are open-source materials, links to which will be provided during pre-workshop preparation. The simulation will mimic interaction of high-frequency trading bots with the stock exchange.
The workshop combines theory with hands-on practice, ensuring that participants not only understand the underlying concepts but also gain practical experience. By the end of the session, attendees will be equipped with the knowledge and inspiration to integrate massive data processing into their testing workflows and build innovative solutions for their real-world challenges.
Technology stack
You will implement several ML algorithms:
- Polynomial Regression
- Support Vector Machine (SVM)
- Multinomial Naive Bayes (MNB)
- Decision Tree
- Random Forest
- Simple perceptron