Unit 3: Classification and association techniques
Data Mining for Business Decisions notes · PTU syllabus (MBA 941-18)
On this page
- Unit summary
- Regression and correlation in data mining
- Decision trees
- Bayesian classification
- k-nearest neighbour and case-based reasoning
- Neural networks
- Support vector machines
- Market basket analysis and association rules
- Genetic algorithms and link analysis
- The fuzzy set approach
- Key terms
- Quick revision
- Important questions
Unit summary
Classification and association techniques turn data into decisions — who will buy, default or churn, and which products go together. This unit covers regression and correlation, decision trees, clustering, neural networks, market basket analysis and association rules, genetic algorithms and link analysis, support vector machines, Bayesian classification, k-nearest neighbour, case-based reasoning and the fuzzy set approach.
After this unit you can
- Apply decision trees, Bayesian classification and k-NN
- Explain neural networks, SVM, genetic algorithms and case-based reasoning
- Apply market basket analysis and association rules
- Explain link analysis and the fuzzy set approach
PTU syllabus topics
- Regression and correlation
- decision trees
- clustering
- neural networks
- market basket analysis and association rules
- genetic algorithms and link analysis
- Support Vector Machine
- Bayesian classification
- k-Nearest Neighbour
- case-based reasoning
- fuzzy set approach
Topic 1
Regression and correlation in data mining
- Correlation measures the strength of linear association; regression models a target as a function of predictors — used for numeric prediction and to identify drivers.
- Logistic regression is a classification technique for binary targets (default or not).
Example
Predicting sales: a regression model for a chain of stores gives Sales = 20 + 3.5 × (Advertising spend in ₹ lakh). If next month's advertising budget is ₹4 lakh, predicted sales = 20 + 14 = ₹34 lakh.
Exam tip
Correlation shows association; regression is used for prediction. Neither proves causation by itself.
Topic 2
Decision trees
Each internal node tests an attribute; leaves give the class. Splits maximise information gain or reduce Gini impurity
- A decision tree splits data on the attribute that best separates the classes; leaves give the predicted class.
Entropy
−Σ p log2 p
Information gain (ID3)
Entropy before split − weighted entropy after
Gain ratio (C4.5)
Information gain ÷ split information
Gini index (CART)
1 − Σ p²
- Pruning removes branches that fit noise, reducing overfitting.
Example
A node has 10 customers: 5 buy, 5 do not; entropy = 1. Splitting on "income high" gives two pure groups; information gain = 1 − 0 = 1, the best possible split.
Topic 3
Bayesian classification
Naive Bayes classifies a record into the class with the highest posterior probability, assuming the attributes are independent given the class. It is simple, fast and works well with text, such as spam filtering.
Posterior
P(C given X) = P(X given C) × P(C) ÷ P(X)
Naive Bayes
Assumes attributes are independent given the class: P(X given C) = product of P(xi given C)
- Use: spam filtering, text classification, quick baseline models; fast and works well with many attributes even if independence is not exact.
Example
Spam filtering: 30% of emails are spam. The word "offer" appears in 60% of spam and 10% of normal emails. P(spam | "offer") = (0.6 × 0.3) ÷ (0.6 × 0.3 + 0.1 × 0.7) = 0.18 ÷ 0.25 = 0.72. An email containing "offer" is classed as spam.
Exam tip
Show the Bayes formula and one numerical calculation.
Topic 4
k-nearest neighbour and case-based reasoning
- k-NN: classify a new case by the majority class of its k nearest neighbours (Euclidean distance on standardised data); a "lazy learner" — no model is built in advance.
- Case-based reasoning (CBR): solve new problems by retrieving similar past cases, reusing and adapting their solutions, and retaining the new case (retrieve, reuse, revise, retain) — used in help desks and diagnosis.
Example
k-NN for loan approval: a new applicant is compared with past applicants on income and credit score. Of the 5 most similar past applicants, 4 repaid their loans, so the new applicant is classified as low risk.
Exam tip
Choose k carefully — too small is sensitive to noise; too large blurs the classes. Scale the variables first.
Topic 5
Neural networks
- Artificial neural network: layers of interconnected nodes (input, hidden, output) with weights adjusted by backpropagation to minimise error.
- Strengths: capture complex non-linear patterns — image, speech, fraud detection; weaknesses: need large data, black box, risk of overfitting. Deep learning uses many hidden layers.
Example
A neural network for fraud: a bank trains a network on millions of past transactions labelled fraud or genuine. The network learns complex patterns — unusual time, location and amount together — and flags suspicious new transactions in real time.
Exam tip
Strength: high accuracy on complex patterns. Weakness: a 'black box' that is hard to explain.
Topic 6
Support vector machines
- SVM finds the hyperplane that separates classes with the maximum margin; the closest points are the support vectors.
- Kernel trick (polynomial, radial basis function) handles non-linear boundaries by mapping data to higher dimensions.
- Use: text classification, credit scoring, image recognition; accurate but less interpretable.
Example
SVM for classification: an SVM separates satisfied from unsatisfied customers using survey scores. Where the groups overlap, a kernel function maps the data into a higher dimension where a clear separating boundary can be found.
Exam tip
Key terms: hyperplane, margin, support vectors and kernel trick.
Topic 7
Market basket analysis and association rules
Support
Transactions containing A and B ÷ total transactions
Confidence
Transactions containing A and B ÷ transactions containing A
Lift
Confidence ÷ support of B; above 1 means positive association
Example
In 1,000 bills, 200 contain bread, 150 contain butter and 100 contain both. Support = 10%, confidence (bread ⇒ butter) = 100 ÷ 200 = 50%, lift = 0.50 ÷ 0.15 = 3.33 — bread buyers are 3.3 times as likely to buy butter.
- Apriori algorithm: every subset of a frequent itemset must be frequent — generate candidates level by level and prune. FP-growth avoids candidate generation.
- Uses: store layout, cross-selling, bundles, recommendation ("frequently bought together").
Topic 8
Genetic algorithms and link analysis
- Genetic algorithms: optimisation inspired by natural selection — a population of candidate solutions evolves through selection, crossover and mutation; used for scheduling, feature selection, portfolio optimisation.
- Link analysis: examines relationships between entities in a network — PageRank, social network analysis, fraud rings in banking and insurance, telecom call graphs.
Example
Link analysis: a bank studies the network of money transfers between accounts and finds a ring of accounts repeatedly passing money among themselves — a pattern typical of money laundering.
Exam tip
Genetic algorithm steps: initial population → fitness evaluation → selection → crossover → mutation → repeat.
Topic 9
The fuzzy set approach
- Fuzzy logic allows partial membership between 0 and 1 instead of crisp categories — a customer can be 0.7 "high income" and 0.3 "medium income".
- Use: classification rules with linguistic terms ("if income is high and age is young then risk is low"), control systems, credit evaluation.
Example
Fuzzy customer value: instead of labelling customers strictly as 'high value' if they spend over ₹50,000, a fuzzy approach gives a customer who spends ₹48,000 a membership of 0.9 in the high-value group — avoiding an unfair hard cut-off.
Exam tip
Fuzzy logic is useful when categories have unclear boundaries, such as 'young', 'high income' or 'risky'.
Key terms
- Information gain
- Reduction in entropy from a split
- Naive Bayes
- Bayesian classifier assuming independent attributes
- Support vector
- Data point closest to the separating hyperplane
- Lift
- Confidence of a rule divided by support of the consequent
- Fuzzy set
- Set allowing partial membership
Quick revision
- Regression and logistic regression for prediction and classification.
- Decision trees: entropy, information gain, Gini; pruning.
- Naive Bayes; k-NN; CBR (retrieve, reuse, revise, retain).
- Neural networks (backpropagation); SVM (max margin, kernels).
- Association rules: support, confidence, lift; Apriori; genetic algorithms; link analysis; fuzzy sets.
Important exam questions
Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).
Short-answer questions
- Q1.What is information gain?
- Q2.State Bayes' theorem.
- Q3.Why is k-NN called a lazy learner?
- Q4.What is the kernel trick in SVM?
- Q5.Define support, confidence and lift.
- Q6.What is a genetic algorithm?
Long-answer questions
- Q1.Explain decision tree induction with a suitable example.
- Q2.Explain Bayesian classification and the k-nearest neighbour method.
- Q3.Explain market basket analysis and the Apriori algorithm with an example.
- Q4.Discuss neural networks, SVM and genetic algorithms in data mining.
Stuck on this unit?
Message SBS on WhatsApp for help with Data Mining for Business Decisions, or to ask about studying MBA at Synetic.
