Workshops

Overfit, Underfit and Good Models

100 mins

Prerequisites:
ISTQB® CT-AI Syllabus Chapter 3
Basic Python programming skills

You will start with the fundamental principles of building an ML model based on available data using Python. You will explore the concepts of underfitting and overfitting, learning how to distinguish between them on different real-life models using various techniques, as well as visualising ML models. Additionally, you will learn how to build an ML regression model and make predictions using a polynomial model and a decision tree algorithm. You will also gain practical experience in evaluating ML models by calculating different ML functional performance metrics.

ML Model Development

100 mins

Prerequisites:
ISTQB® CT-AI Syllabus Chapter 4
Basic Python programming skills

Data Preparation for ML

You will practice data preparation activities for ML models, such as data acquisition (identification, gathering, and labelling), data pre-processing (cleaning, transformation, augmentation, and sampling), and feature engineering (feature selection and extraction). Additionally, you will perform exploratory data analysis to support the overall data preparation process and identify potential data issues.

Identify Training and Test Data and Create an ML Model

You will learn about the importance and difference between training, validation and test datasets. You will practice splitting the initial data into these three parts using the Scikit-learn library, and build the classification model based on the multinomial Naive Bayes classifier. You will also learn how the volume of training data affects the accuracy of the ML model. You will also explore techniques for tuning the model and effectively testing its performance.

ML Model Evaluation
Simple Perceptron

80 mins

Prerequisites:
ISTQB® CT-AI Syllabus Chapter 5&6
Workshop 2
Basic Python programming skills

Evaluate the Created ML Model

You will explore various methods for evaluating ML models. This will include practising both manual and automated calculations of different functional performance metrics and discussing the criteria for selecting appropriate metrics based on real-life scenarios.You will calculate and visualise the confusion matrix for a multilabel classification model using Scikit-learn and Matplotlib libraries and practice in different possible ways of metrics calculation based on this confusion matrix.

Implement a Simple Perceptron

You will be introduced to neural networks, beginning with the implementation of a simple perceptron model. You will understand that a neural network can be used for the supervised learning tasks and also as a binary linear classifier, and practice training your perceptron model with basic logical functions such as AND and OR.

Model Explainability
Pairwise Testing

80 mins

Prerequisites:
ISTQB® CT-AI Syllabus Chapter 8&9
Workshop 2&3
Basic Python programming skills

Model Explainability

You will learn how to make your ML model transparent in its decision-making process and in arriving at specific results. You will be introduced to various methods that enhance the explainability of AI-based systems. Using the open-source Local Interpretable Model-agnostic Explanations (LIME) tool, you will provide explainability for the classification model and visualise the resulting explanations.

Pairwise Testing

You will practise various methods and techniques for testing AI-based systems, beginning with pairwise testing. Using the open-source AllPairs pairwise testing tool, you will generate a reduced set of pairwise combinations of parameters for a support vector classification (SVC) model designed to address a real-world problem and create test scenarios providing good test coverage while reducing the effort. Additionally, you will use the free, open-source pytest Python testing framework to execute multiple tests while making predictions for all pairwise combinations of the model parameters.

Metamorphic Testing
Exploratory Testing and EDA

80 mins

Prerequisites:
ISTQB® CT-AI Syllabus Chapter 9
Basic Python programming skills

Metamorphic Testing

You will derive metamorphic relations for the ML classification model based on the analysis of the model parameters, their values and relations. You will use the Seaborn library to plot the correlation heatmap and different parameter distributions. You will generate follow-up test cases based on the source test cases and consider two possible types of metamorphic relations: when the expected result is the same and not the same as the original expected result of the source test case. You will use the pytest testing framework for Python to run multiple tests.

Exploratory Testing and Exploratory Data Analysis (EDA)

You will perform the experience-based testing by applying different exploratory testing and exploratory data analysis techniques. You will learn how to gain a quick understanding of the structure and content of a dataset, and how to obtain its general statistical information. You will apply different data visualisation techniques using Matplotlib and Seaborn, two popular Python libraries.

The Use of AI in Testing
Build a Defect Prediction System

60 mins

Prerequisites:
ISTQB® CT-AI Syllabus Chapter 11
Basic Python programming skills

The Use of AI in Testing

You will learn how AI can be used in testing, focusing on how it can help optimise testing and which testing activities are currently impractical to be implemented using AI.

Build a Defect Prediction System

You will build a defect prediction system based on the source code metrics analysis using the Random Forest algorithm. You will practice feature engineering and learn that the selection of features has a significant impact on the prediction results.