Unit 2 of 2 · BCA Sem 3

Unit 2: Feature engineering techniques

Feature Engineering notes · PTU syllabus (UGDSE201)

3 min read5 topics10 exam questions
On this page
  1. Unit summary
  2. Binning and discretisation
  3. Polynomial and interaction features
  4. Encoding categorical data
  5. Feature extraction vs feature selection
  6. Principal Component Analysis (PCA)
  7. Key terms
  8. Quick revision
  9. Important questions

Unit summary

Once data is clean, new and better features can be built and unhelpful ones removed. This unit covers binning, polynomial and interaction features, encoding categorical data, the difference between feature extraction and selection, filter, wrapper and hybrid selection methods, and dimensionality reduction with PCA.

After this unit you can

  • Apply binning and create polynomial and interaction features
  • Encode categorical variables with one-hot and label encoding
  • Distinguish feature extraction from feature selection
  • Apply filter, wrapper and hybrid selection and reduce dimensions with PCA

PTU syllabus topics

  • Binning and discretization
  • polynomial and interaction features
  • one-hot and label encoding
  • feature extraction vs feature selection
  • filter/wrapper/hybrid selection methods
  • feature reduction via Principal Component Analysis
1

Topic 1

Binning and discretisation

Binning converts a continuous feature into categories (bins).

  • Equal-width binning: bins of the same range (0–20, 20–40 …).
  • Equal-frequency (quantile) binning: each bin has the same number of records.
  • Custom bins: based on domain knowledge (age groups: child, teen, adult, senior).

Binning reduces noise and the effect of outliers, but can lose detail.

Example

Binning ages: 0–12 child, 13–19 teen, 20–59 adult, 60+ senior. Equal-width binning of 0 to 100 into 4 bins gives widths of 25. Equal-frequency binning puts the same number of records in each bin.

2

Topic 2

Polynomial and interaction features

  • Polynomial features add powers of a feature (x², x³) so a linear model can fit curves.
  • Interaction features combine features (x₁ × x₂), capturing effects that appear only together — for example area × location rating for house prices.

Example

From features a and b, degree-2 polynomial features are 1, a, b, a², ab, b².

3

Topic 3

Encoding categorical data

Machine learning models work with numbers, so categories such as city or colour must be converted. Label encoding gives each category an integer. One-hot encoding creates a separate 0/1 column for each category.

GraphOne-hot encoding of a colour feature
ColourRedGreenBlueis_Redis_Greenis_Blue100010001

Use one-hot for nominal categories, label or ordinal encoding when the categories have a natural order

ComparisonOne-hot vs label encoding
One-hot encoding
Label encoding

How

One binary column per category

One integer per category

Example

City: Delhi → [1,0,0], Mumbai → [0,1,0]

Size: S → 0, M → 1, L → 2

Best for

Nominal data (no order)

Ordinal data (ordered)

Drawback

Many columns for many categories

Implies an order that may not exist

Example

Encoding a Colour column with Red, Green, Blue: label encoding gives Red = 0, Green = 1, Blue = 2, which wrongly suggests Blue is greater than Red. One-hot encoding creates three columns: Red → [1, 0, 0], Green → [0, 1, 0], Blue → [0, 0, 1], with no false order.

Exam tip

Use label (ordinal) encoding only when the categories have a real order, such as Low, Medium, High. Use one-hot encoding for names without order.

4

Topic 4

Feature extraction vs feature selection

  • Feature extraction creates new, fewer features from the original ones (PCA, TF-IDF from text).
  • Feature selection keeps a subset of the original features and discards the rest.
ClassificationFeature selection methods
Selection
  • Filter

    Rank by statistics independent of the model: correlation, chi-square, variance threshold

  • Wrapper

    Try subsets with a model: forward selection, backward elimination, recursive feature elimination

  • Embedded

    Selection during training: Lasso, tree feature importance

  • Hybrid

    Filter first to shortlist, then wrapper to finalise

Exam tip

Filter methods are fast but ignore the model; wrapper methods are accurate but slow. Hybrid methods balance both.

Example

The difference: from 10 features, selection keeps the best 4 as they are (for example by correlation with the target). Extraction (PCA) combines all 10 into 4 new features that are mixtures of the originals.

5

Topic 5

Principal Component Analysis (PCA)

GraphPCA finds the direction of maximum variance
Feature 2Feature 1OPC1 (most variance)PC2

Project the points onto PC1 to reduce two features to one while keeping most of the information

PCA reduces many correlated features into a few uncorrelated principal components that keep most of the variance (information).

ProcessPCA step by step
  1. 1

    Standardise the features

  2. 2

    Compute the covariance matrix

  3. 3

    Find eigenvalues and eigenvectors

  4. 4

    Sort components by eigenvalue

    Largest variance first

  5. 5

    Keep the top k components

    e.g. those explaining 95% of variance

  6. 6

    Project the data onto them

Benefits: fewer features, faster training, less overfitting and easier 2-D/3-D visualisation. Drawback: components are harder to interpret than original features.

Example

Steps of PCA: standardise the data; compute the covariance matrix; find its eigenvalues and eigenvectors; sort by eigenvalue; keep the top k eigenvectors; project the data onto them. If the eigenvalues are 2.5, 1.0 and 0.5, the first component explains 2.5 ÷ 4 = 62.5% of the variance.

Key terms

Binning
Grouping continuous values into intervals
Interaction feature
A feature formed by combining two or more features
One-hot encoding
Representing categories as binary columns
Feature selection
Choosing a useful subset of existing features
PCA
A technique that reduces features into principal components

Quick revision

  • Equal-width vs equal-frequency binning.
  • Polynomial: x²; interaction: x₁ × x₂.
  • One-hot for nominal, label for ordinal.
  • Filter (fast), wrapper (accurate), embedded, hybrid.
  • PCA keeps components with the largest variance.

Important exam questions

Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).

Short-answer questions

  1. Q1.What is binning?
  2. Q2.What are interaction features?
  3. Q3.When should one-hot encoding be used?
  4. Q4.Differentiate between feature extraction and feature selection.
  5. Q5.What is a wrapper method?
  6. Q6.What does PCA do?

Long-answer questions

  1. Q1.Explain binning, polynomial and interaction features with examples.
  2. Q2.Compare one-hot and label encoding.
  3. Q3.Explain filter, wrapper and hybrid feature selection methods.
  4. Q4.Explain Principal Component Analysis step by step.

Stuck on this unit?

Message SBS on WhatsApp for help with Feature Engineering, or to ask about studying BCA at Synetic.

WhatsApp us