Machine Learning Notes
Subject Overview
The third Elective-II option, covering machine learning fundamentals and evaluation metrics, regression and classification (linear regression, KNN, logistic regression, Naive Bayes), ensemble methods and clustering (bagging, boosting, decision trees, random forests, k-means, SVM, PCA), and an introduction to time series and neural networks. A 4-credit elective theory paper.
Subject at a glance
4
Units
28
Topics
4
Credits
4
Infographics
Unit-wise Syllabus
4 units — open any unit for detailed topic-wise notes, diagrams and important exam questions
- What is machine learning
- problems/data/tools
- types of learning
- performance evaluation measures (accuracy, precision, recall, F-measure)
- error metrics
- data visualization
- linear regression
- gradient descent
- closed-form/normal equations
- features
- overfitting
- training/validation/test data
Hypothesis
ŷ = w₀ + w₁x₁ + … + wₙxₙ
Cost (MSE)
J = (1/2m) Σ (ŷ − y)²
Gradient descent
w ← w − α ∂J/∂w
Normal equation
w = (XᵀX)⁻¹ Xᵀ y
Study guide
This unit covers what machine learning is, problems, data and tools, types of learning, evaluation measures (accuracy, precision, recall, F-measure), error metrics, visualisation, linear regression, gradient descent, normal equations, features, overfitting and training, validation and test data. Implement linear regression with gradient descent to understand it.
- Classification problems and decision boundaries
- K-Nearest Neighbor
- logistic regression
- probability and classification
- Bayes optimal decisions
- Naive Bayes and Gaussian class-conditional distribution
Sigmoid
σ(z) = 1 / (1 + e^(−z))
Decision boundary
Predict 1 if σ(z) ≥ 0.5
Naive Bayes
P(c given x) ∝ P(c) Π P(xi given c)
k-NN
Majority class among the k nearest points
Study guide
You study classification problems and decision boundaries, k-NN, logistic regression, probability and classification, Bayes optimal decisions and Naive Bayes with Gaussian distributions. Compare k-NN, logistic regression and Naive Bayes on one data set.
- Bagging
- decision trees and random forests
- boosting
- clustering (k-means, hierarchical agglomeration)
- Support Vector Machines
- Principal Component Analysis
Training
Models in parallel on bootstrap samples
Models in sequence, fixing earlier errors
Reduces
Variance
Bias
Example
Random forest
AdaBoost, gradient boosting
Overfitting risk
Lower
Higher if over-trained
Study guide
You study bagging, decision trees and random forests, boosting, clustering (k-means, hierarchical), SVMs and PCA. Understand why ensembles often beat single models.
- Time series analysis
- Markov models
- autoregressive models
- introduction to neural networks and deep learning
AR(p)
yₜ = c + Σ φᵢ yₜ₋ᵢ + εₜ
Moving average smoothing
Mean of the last k values
Markov property
Next state depends only on the current state
Neuron
y = f(Σ wᵢ xᵢ + b)
Study guide
You study time series analysis, Markov models, autoregressive models and an introduction to neural networks and deep learning. Forecast a simple time series.
Why Machine Learning Matters
Machine learning is the most in-demand technical skill area in the world, and M.Sc IT graduates with solid ML and project experience have strong career options.
How to Prepare for Machine Learning
- Build projects on real data sets and share them on GitHub.
- Always evaluate on held-out data.
- Learn scikit-learn and read its documentation.
Machine Learning: Frequently Asked Questions
What is overfitting?
Overfitting is when a model fits the training data too closely, including noise, and performs poorly on new data.