Unit 4: Time series and neural networks
Machine Learning notes · PTU syllabus (PGCA1945)
On this page
Unit summary
Much real data arrives over time, and neural networks power modern AI. This unit covers time series analysis, Markov models, autoregressive models, and an introduction to neural networks and deep learning.
After this unit you can
- Describe the components of a time series
- Explain Markov models
- Fit autoregressive models
- Explain neural networks and deep learning
PTU syllabus topics
- Time series analysis
- Markov models
- autoregressive models
- introduction to neural networks and deep learning
AR(p)
yₜ = c + Σ φᵢ yₜ₋ᵢ + εₜ
Moving average smoothing
Mean of the last k values
Markov property
Next state depends only on the current state
Neuron
y = f(Σ wᵢ xᵢ + b)
Topic 1
Time series analysis
- Trend
- Long-term direction
- Seasonality
- Regular repeating pattern (monthly sales)
- Cyclic
- Longer, irregular economic cycles
- Noise
- Random variation
- Stationarity: constant mean and variance; achieved by differencing. Validate forecasts on the latest period (no shuffling).
Topic 2
Markov models
- Markov property: the next state depends only on the current state.
P(Xt+1 given Xt, …, X1) = P(Xt+1 given Xt)
Markov property
State vector at t+1 = state vector at t × P
Transition matrix P
Example
Weather: P(sunny→sunny) = 0.8, P(sunny→rainy) = 0.2, P(rainy→sunny) = 0.4, P(rainy→rainy) = 0.6. If today is sunny, tomorrow sunny with 0.8; the day after: 0.8 × 0.8 + 0.2 × 0.4 = 0.72.
- Hidden Markov models have unobserved states emitting observations (speech recognition, POS tagging).
Topic 3
Autoregressive models
AR(p): yt = c + φ1yt−1 + … + φpyt−p + εt
Regression on past values
MA(q)
Regression on past errors
ARIMA(p, d, q)
AR + differencing + MA
Example
AR(1) with c = 10, φ = 0.5 and yt = 40 → forecast yt+1 = 10 + 0.5 × 40 = 30.
Topic 4
Neural networks and deep learning
An artificial neuron computes a weighted sum of inputs plus a bias and applies an activation function. The perceptron learns with the rule w ← w + η (t − y) x.
- A single-layer perceptron can learn linearly separable functions like AND and OR, but not XOR — which needs a multi-layer network.
- A support vector machine (SVM) finds the separating line (hyperplane) with the maximum margin between classes; the closest points are support vectors.
- 1Forward pass
- 2Compute loss
- 3Back-propagate gradients
- 4Update weights (SGD, Adam)
- 5Repeat over epochs
CNN
Images — convolution and pooling
Medical image diagnosis
RNN and LSTM
Sequences and time series
Stock and sales forecasting
Transformer
Language with attention
Large language models
- Activation functions: sigmoid, tanh, ReLU; dropout reduces overfitting.
Key terms
- Stationarity
- Constant statistical properties over time
- Markov property
- Future depends only on the present state
- AR(p)
- Autoregressive model of order p
- ReLU
- max(0, x) activation function
- Deep learning
- Neural networks with many layers
Quick revision
- Trend, seasonality, cycle, noise; stationarity.
- Markov chains, transition matrix; HMM.
- AR, MA, ARIMA.
- Perceptron, back-propagation, CNN, RNN, transformer.
Important exam questions
Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).
Short-answer questions
- Q1.Name the components of a time series.
- Q2.What is the Markov property?
- Q3.Write the AR(1) model.
- Q4.What does d mean in ARIMA?
- Q5.What is an activation function?
- Q6.What is a CNN used for?
Long-answer questions
- Q1.Explain time series components and autoregressive models.
- Q2.Explain Markov models with an example.
- Q3.Explain neural networks and deep learning.
Stuck on this unit?
Message SBS on WhatsApp for help with Machine Learning, or to ask about studying M.Sc IT at Synetic.