Data, Statistics & Classical ML Notes
Subject Overview
The second half of the Foundations year: take raw data, prepare it properly, and train traditional ML models that solve real problems. Six modules cover statistics for ML, data wrangling, supervised and unsupervised learning, and the workflow and experiment tracking that keep results reproducible. Students leave Year 1 able to clean data, train models, and reason about results.
Subject at a glance
8
Units
37
Topics
15
Hours
8
Infographics
Unit-wise Syllabus
8 units — click WhatsApp below to get the full notes for each
Unit 1: Statistics for ML (3 hours)
- Descriptive versus inferential statistics
- hypothesis testing
- p-values
- confidence intervals
- common pitfalls
- 1State H0 and H1
- 2Choose significance level
Usually α = 0.05
- 3Compute the test statistic
- 4Find the p-value
- 5Decide
Reject H0 if p < α
Study guide
This module builds statistical thinking for ML: descriptive versus inferential statistics, hypothesis testing, p-values, confidence intervals and common pitfalls. Learn what a p-value does not tell you, since misreading it is a classic mistake.
Unit 2: Data Wrangling (3 hours)
- Cleaning
- missing values
- outliers
- feature engineering
- scaling
- encoding categorical variables
- 1
Inspect
Shape, types, summary stats
- 2
Handle missing values
Drop or impute
- 3
Treat outliers
Cap, transform or remove
- 4
Encode categories
One-hot, ordinal, target
- 5
Scale features
Standardise or normalise
- 6
Engineer new features
Study guide
Real data is messy. You learn cleaning, missing values, outliers, feature engineering, scaling and encoding categorical variables. Expect to spend more time here than on modelling in real projects.
Unit 3: Supervised Learning I (4 hours)
- Linear regression
- logistic regression
- decision trees
- train/test splits
- cross-validation
Linear regression
A number
Simple, interpretable
Logistic regression
A class probability
Fast baseline for classification
Decision tree
Number or class
Easy to explain; can overfit
Study guide
Supervised learning begins with linear and logistic regression, decision trees, train-test splits and cross-validation. Always evaluate on data the model has not seen.
Unit 4: Supervised Learning II (4 hours)
- Random forests
- gradient boosting (XGBoost, LightGBM)
- SVMs
- model evaluation metrics
Accuracy
(TP + TN) / total
Precision
TP / (TP + FP)
Of predicted positives, how many are right
Recall
TP / (TP + FN)
Of actual positives, how many were found
F1 score
2 × P × R / (P + R)
ROC-AUC
Area under the ROC curve: 1.0 is perfect
Study guide
You move to random forests, gradient boosting (XGBoost, LightGBM), SVMs and model evaluation metrics. Boosted trees are often the strongest baseline for tabular data.
Unit 5: Unsupervised Learning (3 hours)
- K-means
- hierarchical clustering
- PCA
- t-SNE
- UMAP — when and why to use each
K-means
Groups into k round clusters
You know roughly how many groups
Hierarchical
Builds a tree of clusters
You want to see nested groups
PCA
Linear dimension reduction
Compress features, speed up models
t-SNE / UMAP
Non-linear 2D maps
Visualising high-dimensional data
Study guide
Unsupervised learning covers K-means, hierarchical clustering, PCA, t-SNE and UMAP and, importantly, when and why to use each. Use PCA for compression and t-SNE or UMAP for visualisation.
Unit 6: ML Workflow & Experiment Tracking (3 hours)
- scikit-learn pipelines
- hyperparameter tuning
- MLflow or Weights & Biases for tracking
- 1
Split train, validation and test
- 2
Build a pipeline
Preprocessing + model
- 3
Cross-validate a baseline
- 4
Tune hyperparameters
Grid, random or Bayesian search
- 5
Log runs
MLflow or Weights & Biases
- 6
Evaluate once on the test set
Study guide
You learn the workflow around models: scikit-learn pipelines, hyperparameter tuning and experiment tracking with MLflow or Weights & Biases. Tracking experiments keeps your results reproducible.
Unit 7: Tools introduced
- scikit-learn
- XGBoost
- LightGBM
- MLflow / Weights & Biases for experiment tracking
- Kaggle for datasets and competitions
- Claude / ChatGPT as a debugging and code review partner
scikit-learn
Models, pipelines, metrics
XGBoost and LightGBM
Gradient-boosted trees
MLflow / W&B
Experiment tracking
Kaggle
Datasets and competitions
AI assistants
Debugging and code review
Study guide
The tools introduced include scikit-learn, XGBoost, LightGBM, MLflow and Weights & Biases, Kaggle for datasets and competitions and AI assistants such as Claude or ChatGPT for debugging and code review. Join a Kaggle competition to practise on real problems.
Unit 8: Mini-projects
- Titanic survival prediction with a full ML pipeline
- customer segmentation on real e-commerce data
- house price predictor deployed as a simple Streamlit app
- 1
Explore the data
Who survived?
- 2
Clean and engineer features
Title, family size
- 3
Train baseline models
- 4
Tune the best model
- 5
Submit and compare on Kaggle
- 6
Deploy a predictor in Streamlit
Study guide
The mini-projects are Titanic survival prediction with a full pipeline, customer segmentation on real e-commerce data and a house price predictor deployed as a Streamlit app. Deploying a model, even simply, teaches more than any notebook.
Why Data, Statistics & Classical ML Matters
Classical machine learning remains the workhorse of industry for tabular data, and the statistical habits you build here prevent expensive mistakes later. Mastering pipelines, evaluation and tracking makes you job-ready for data scientist and ML engineer roles.
How to Prepare for Data, Statistics & Classical ML
- Build every project with a proper train, validation and test split.
- Enter a Kaggle competition and read top solutions.
- Track experiments with MLflow or Weights & Biases from the start.
Data, Statistics & Classical ML: Frequently Asked Questions
What is the difference between supervised and unsupervised learning?
Supervised learning learns from labelled data to predict outcomes, while unsupervised learning finds structure in unlabelled data.
Why use cross-validation?
Cross-validation gives a more reliable estimate of how a model will perform on new data by testing it on several different splits.