Unit 3 of 4 · MBA Sem 3

Unit 3: Classification and association techniques

Data Mining for Business Decisions notes · PTU syllabus (MBA 941-18)

4 min read9 topics10 exam questions
On this page
  1. Unit summary
  2. Regression and correlation in data mining
  3. Decision trees
  4. Bayesian classification
  5. k-nearest neighbour and case-based reasoning
  6. Neural networks
  7. Support vector machines
  8. Market basket analysis and association rules
  9. Genetic algorithms and link analysis
  10. The fuzzy set approach
  11. Key terms
  12. Quick revision
  13. Important questions

Unit summary

Classification and association techniques turn data into decisions — who will buy, default or churn, and which products go together. This unit covers regression and correlation, decision trees, clustering, neural networks, market basket analysis and association rules, genetic algorithms and link analysis, support vector machines, Bayesian classification, k-nearest neighbour, case-based reasoning and the fuzzy set approach.

After this unit you can

  • Apply decision trees, Bayesian classification and k-NN
  • Explain neural networks, SVM, genetic algorithms and case-based reasoning
  • Apply market basket analysis and association rules
  • Explain link analysis and the fuzzy set approach

PTU syllabus topics

  • Regression and correlation
  • decision trees
  • clustering
  • neural networks
  • market basket analysis and association rules
  • genetic algorithms and link analysis
  • Support Vector Machine
  • Bayesian classification
  • k-Nearest Neighbour
  • case-based reasoning
  • fuzzy set approach
1

Topic 1

Regression and correlation in data mining

  • Correlation measures the strength of linear association; regression models a target as a function of predictors — used for numeric prediction and to identify drivers.
  • Logistic regression is a classification technique for binary targets (default or not).

Example

Predicting sales: a regression model for a chain of stores gives Sales = 20 + 3.5 × (Advertising spend in ₹ lakh). If next month's advertising budget is ₹4 lakh, predicted sales = 20 + 14 = ₹34 lakh.

Exam tip

Correlation shows association; regression is used for prediction. Neither proves causation by itself.

2

Topic 2

Decision trees

GraphDecision tree for loan approval
Income > 5L?Credit score >700?RejectApproveRejectYesNo

Each internal node tests an attribute; leaves give the class. Splits maximise information gain or reduce Gini impurity

  • A decision tree splits data on the attribute that best separates the classes; leaves give the predicted class.
Key formulasSplitting criteria
  • Entropy

    −Σ p log2 p

  • Information gain (ID3)

    Entropy before split − weighted entropy after

  • Gain ratio (C4.5)

    Information gain ÷ split information

  • Gini index (CART)

    1 − Σ p²

  • Pruning removes branches that fit noise, reducing overfitting.

Example

A node has 10 customers: 5 buy, 5 do not; entropy = 1. Splitting on "income high" gives two pure groups; information gain = 1 − 0 = 1, the best possible split.

3

Topic 3

Bayesian classification

Naive Bayes classifies a record into the class with the highest posterior probability, assuming the attributes are independent given the class. It is simple, fast and works well with text, such as spam filtering.

Key formulasBayes' theorem
  • Posterior

    P(C given X) = P(X given C) × P(C) ÷ P(X)

  • Naive Bayes

    Assumes attributes are independent given the class: P(X given C) = product of P(xi given C)

  • Use: spam filtering, text classification, quick baseline models; fast and works well with many attributes even if independence is not exact.

Example

Spam filtering: 30% of emails are spam. The word "offer" appears in 60% of spam and 10% of normal emails. P(spam | "offer") = (0.6 × 0.3) ÷ (0.6 × 0.3 + 0.1 × 0.7) = 0.18 ÷ 0.25 = 0.72. An email containing "offer" is classed as spam.

Exam tip

Show the Bayes formula and one numerical calculation.

4

Topic 4

k-nearest neighbour and case-based reasoning

  • k-NN: classify a new case by the majority class of its k nearest neighbours (Euclidean distance on standardised data); a "lazy learner" — no model is built in advance.
  • Case-based reasoning (CBR): solve new problems by retrieving similar past cases, reusing and adapting their solutions, and retaining the new case (retrieve, reuse, revise, retain) — used in help desks and diagnosis.

Example

k-NN for loan approval: a new applicant is compared with past applicants on income and credit score. Of the 5 most similar past applicants, 4 repaid their loans, so the new applicant is classified as low risk.

Exam tip

Choose k carefully — too small is sensitive to noise; too large blurs the classes. Scale the variables first.

5

Topic 5

Neural networks

  • Artificial neural network: layers of interconnected nodes (input, hidden, output) with weights adjusted by backpropagation to minimise error.
  • Strengths: capture complex non-linear patterns — image, speech, fraud detection; weaknesses: need large data, black box, risk of overfitting. Deep learning uses many hidden layers.

Example

A neural network for fraud: a bank trains a network on millions of past transactions labelled fraud or genuine. The network learns complex patterns — unusual time, location and amount together — and flags suspicious new transactions in real time.

Exam tip

Strength: high accuracy on complex patterns. Weakness: a 'black box' that is hard to explain.

6

Topic 6

Support vector machines

  • SVM finds the hyperplane that separates classes with the maximum margin; the closest points are the support vectors.
  • Kernel trick (polynomial, radial basis function) handles non-linear boundaries by mapping data to higher dimensions.
  • Use: text classification, credit scoring, image recognition; accurate but less interpretable.

Example

SVM for classification: an SVM separates satisfied from unsatisfied customers using survey scores. Where the groups overlap, a kernel function maps the data into a higher dimension where a clear separating boundary can be found.

Exam tip

Key terms: hyperplane, margin, support vectors and kernel trick.

7

Topic 7

Market basket analysis and association rules

Key formulasAssociation rule measures (rule A ⇒ B)
  • Support

    Transactions containing A and B ÷ total transactions

  • Confidence

    Transactions containing A and B ÷ transactions containing A

  • Lift

    Confidence ÷ support of B; above 1 means positive association

Example

In 1,000 bills, 200 contain bread, 150 contain butter and 100 contain both. Support = 10%, confidence (bread ⇒ butter) = 100 ÷ 200 = 50%, lift = 0.50 ÷ 0.15 = 3.33 — bread buyers are 3.3 times as likely to buy butter.

  • Apriori algorithm: every subset of a frequent itemset must be frequent — generate candidates level by level and prune. FP-growth avoids candidate generation.
  • Uses: store layout, cross-selling, bundles, recommendation ("frequently bought together").
8

Topic 8

Genetic algorithms and link analysis

  • Genetic algorithms: optimisation inspired by natural selection — a population of candidate solutions evolves through selection, crossover and mutation; used for scheduling, feature selection, portfolio optimisation.
  • Link analysis: examines relationships between entities in a network — PageRank, social network analysis, fraud rings in banking and insurance, telecom call graphs.

Example

Link analysis: a bank studies the network of money transfers between accounts and finds a ring of accounts repeatedly passing money among themselves — a pattern typical of money laundering.

Exam tip

Genetic algorithm steps: initial population → fitness evaluation → selection → crossover → mutation → repeat.

9

Topic 9

The fuzzy set approach

  • Fuzzy logic allows partial membership between 0 and 1 instead of crisp categories — a customer can be 0.7 "high income" and 0.3 "medium income".
  • Use: classification rules with linguistic terms ("if income is high and age is young then risk is low"), control systems, credit evaluation.

Example

Fuzzy customer value: instead of labelling customers strictly as 'high value' if they spend over ₹50,000, a fuzzy approach gives a customer who spends ₹48,000 a membership of 0.9 in the high-value group — avoiding an unfair hard cut-off.

Exam tip

Fuzzy logic is useful when categories have unclear boundaries, such as 'young', 'high income' or 'risky'.

Key terms

Information gain
Reduction in entropy from a split
Naive Bayes
Bayesian classifier assuming independent attributes
Support vector
Data point closest to the separating hyperplane
Lift
Confidence of a rule divided by support of the consequent
Fuzzy set
Set allowing partial membership

Quick revision

  • Regression and logistic regression for prediction and classification.
  • Decision trees: entropy, information gain, Gini; pruning.
  • Naive Bayes; k-NN; CBR (retrieve, reuse, revise, retain).
  • Neural networks (backpropagation); SVM (max margin, kernels).
  • Association rules: support, confidence, lift; Apriori; genetic algorithms; link analysis; fuzzy sets.

Important exam questions

Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).

Short-answer questions

  1. Q1.What is information gain?
  2. Q2.State Bayes' theorem.
  3. Q3.Why is k-NN called a lazy learner?
  4. Q4.What is the kernel trick in SVM?
  5. Q5.Define support, confidence and lift.
  6. Q6.What is a genetic algorithm?

Long-answer questions

  1. Q1.Explain decision tree induction with a suitable example.
  2. Q2.Explain Bayesian classification and the k-nearest neighbour method.
  3. Q3.Explain market basket analysis and the Apriori algorithm with an example.
  4. Q4.Discuss neural networks, SVM and genetic algorithms in data mining.

Stuck on this unit?

Message SBS on WhatsApp for help with Data Mining for Business Decisions, or to ask about studying MBA at Synetic.

Planning your MBA in Ludhiana? See the MBA at SBS →

WhatsApp us