Unit 2: Feature engineering techniques
Feature Engineering notes · PTU syllabus (UGDSE201)
On this page
Unit summary
Once data is clean, new and better features can be built and unhelpful ones removed. This unit covers binning, polynomial and interaction features, encoding categorical data, the difference between feature extraction and selection, filter, wrapper and hybrid selection methods, and dimensionality reduction with PCA.
After this unit you can
- Apply binning and create polynomial and interaction features
- Encode categorical variables with one-hot and label encoding
- Distinguish feature extraction from feature selection
- Apply filter, wrapper and hybrid selection and reduce dimensions with PCA
PTU syllabus topics
- Binning and discretization
- polynomial and interaction features
- one-hot and label encoding
- feature extraction vs feature selection
- filter/wrapper/hybrid selection methods
- feature reduction via Principal Component Analysis
How
A new 0/1 column per category
Each category gets an integer
Implies order
No
Yes
Best for
Nominal data such as colour
Ordinal data such as size
Drawback
Many columns for many categories
Models may read false order into nominal data
Topic 1
Binning and discretisation
Binning converts a continuous feature into categories (bins).
- Equal-width binning: bins of the same range (0–20, 20–40 …).
- Equal-frequency (quantile) binning: each bin has the same number of records.
- Custom bins: based on domain knowledge (age groups: child, teen, adult, senior).
Binning reduces noise and the effect of outliers, but can lose detail.
Topic 2
Polynomial and interaction features
- Polynomial features add powers of a feature (x², x³) so a linear model can fit curves.
- Interaction features combine features (x₁ × x₂), capturing effects that appear only together — for example area × location rating for house prices.
Example
From features a and b, degree-2 polynomial features are 1, a, b, a², ab, b².
Topic 3
Encoding categorical data
How
One binary column per category
One integer per category
Example
City: Delhi → [1,0,0], Mumbai → [0,1,0]
Size: S → 0, M → 1, L → 2
Best for
Nominal data (no order)
Ordinal data (ordered)
Drawback
Many columns for many categories
Implies an order that may not exist
Topic 4
Feature extraction vs feature selection
- Feature extraction creates new, fewer features from the original ones (PCA, TF-IDF from text).
- Feature selection keeps a subset of the original features and discards the rest.
Filter
Rank by statistics independent of the model: correlation, chi-square, variance threshold
Wrapper
Try subsets with a model: forward selection, backward elimination, recursive feature elimination
Embedded
Selection during training: Lasso, tree feature importance
Hybrid
Filter first to shortlist, then wrapper to finalise
Exam tip
Filter methods are fast but ignore the model; wrapper methods are accurate but slow. Hybrid methods balance both.
Topic 5
Principal Component Analysis (PCA)
PCA reduces many correlated features into a few uncorrelated principal components that keep most of the variance (information).
- 1
Standardise the features
- 2
Compute the covariance matrix
- 3
Find eigenvalues and eigenvectors
- 4
Sort components by eigenvalue
Largest variance first
- 5
Keep the top k components
e.g. those explaining 95% of variance
- 6
Project the data onto them
Benefits: fewer features, faster training, less overfitting and easier 2-D/3-D visualisation. Drawback: components are harder to interpret than original features.
Key terms
- Binning
- Grouping continuous values into intervals
- Interaction feature
- A feature formed by combining two or more features
- One-hot encoding
- Representing categories as binary columns
- Feature selection
- Choosing a useful subset of existing features
- PCA
- A technique that reduces features into principal components
Quick revision
- Equal-width vs equal-frequency binning.
- Polynomial: x²; interaction: x₁ × x₂.
- One-hot for nominal, label for ordinal.
- Filter (fast), wrapper (accurate), embedded, hybrid.
- PCA keeps components with the largest variance.
Important exam questions
Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).
Short-answer questions
- Q1.What is binning?
- Q2.What are interaction features?
- Q3.When should one-hot encoding be used?
- Q4.Differentiate between feature extraction and feature selection.
- Q5.What is a wrapper method?
- Q6.What does PCA do?
Long-answer questions
- Q1.Explain binning, polynomial and interaction features with examples.
- Q2.Compare one-hot and label encoding.
- Q3.Explain filter, wrapper and hybrid feature selection methods.
- Q4.Explain Principal Component Analysis step by step.
Stuck on this unit?
Message SBS on WhatsApp for help with Feature Engineering, or to ask about studying BCA at Synetic.