Unit 2: Feature engineering techniques
Feature Engineering notes · PTU syllabus (UGDSE201)
On this page
Unit summary
Once data is clean, new and better features can be built and unhelpful ones removed. This unit covers binning, polynomial and interaction features, encoding categorical data, the difference between feature extraction and selection, filter, wrapper and hybrid selection methods, and dimensionality reduction with PCA.
After this unit you can
- Apply binning and create polynomial and interaction features
- Encode categorical variables with one-hot and label encoding
- Distinguish feature extraction from feature selection
- Apply filter, wrapper and hybrid selection and reduce dimensions with PCA
PTU syllabus topics
- Binning and discretization
- polynomial and interaction features
- one-hot and label encoding
- feature extraction vs feature selection
- filter/wrapper/hybrid selection methods
- feature reduction via Principal Component Analysis
Topic 1
Binning and discretisation
Binning converts a continuous feature into categories (bins).
- Equal-width binning: bins of the same range (0–20, 20–40 …).
- Equal-frequency (quantile) binning: each bin has the same number of records.
- Custom bins: based on domain knowledge (age groups: child, teen, adult, senior).
Binning reduces noise and the effect of outliers, but can lose detail.
Example
Binning ages: 0–12 child, 13–19 teen, 20–59 adult, 60+ senior. Equal-width binning of 0 to 100 into 4 bins gives widths of 25. Equal-frequency binning puts the same number of records in each bin.
Topic 2
Polynomial and interaction features
- Polynomial features add powers of a feature (x², x³) so a linear model can fit curves.
- Interaction features combine features (x₁ × x₂), capturing effects that appear only together — for example area × location rating for house prices.
Example
From features a and b, degree-2 polynomial features are 1, a, b, a², ab, b².
Topic 3
Encoding categorical data
Machine learning models work with numbers, so categories such as city or colour must be converted. Label encoding gives each category an integer. One-hot encoding creates a separate 0/1 column for each category.
Use one-hot for nominal categories, label or ordinal encoding when the categories have a natural order
How
One binary column per category
One integer per category
Example
City: Delhi → [1,0,0], Mumbai → [0,1,0]
Size: S → 0, M → 1, L → 2
Best for
Nominal data (no order)
Ordinal data (ordered)
Drawback
Many columns for many categories
Implies an order that may not exist
Example
Encoding a Colour column with Red, Green, Blue: label encoding gives Red = 0, Green = 1, Blue = 2, which wrongly suggests Blue is greater than Red. One-hot encoding creates three columns: Red → [1, 0, 0], Green → [0, 1, 0], Blue → [0, 0, 1], with no false order.
Exam tip
Use label (ordinal) encoding only when the categories have a real order, such as Low, Medium, High. Use one-hot encoding for names without order.
Topic 4
Feature extraction vs feature selection
- Feature extraction creates new, fewer features from the original ones (PCA, TF-IDF from text).
- Feature selection keeps a subset of the original features and discards the rest.
Filter
Rank by statistics independent of the model: correlation, chi-square, variance threshold
Wrapper
Try subsets with a model: forward selection, backward elimination, recursive feature elimination
Embedded
Selection during training: Lasso, tree feature importance
Hybrid
Filter first to shortlist, then wrapper to finalise
Exam tip
Filter methods are fast but ignore the model; wrapper methods are accurate but slow. Hybrid methods balance both.
Example
The difference: from 10 features, selection keeps the best 4 as they are (for example by correlation with the target). Extraction (PCA) combines all 10 into 4 new features that are mixtures of the originals.
Topic 5
Principal Component Analysis (PCA)
Project the points onto PC1 to reduce two features to one while keeping most of the information
PCA reduces many correlated features into a few uncorrelated principal components that keep most of the variance (information).
- 1
Standardise the features
- 2
Compute the covariance matrix
- 3
Find eigenvalues and eigenvectors
- 4
Sort components by eigenvalue
Largest variance first
- 5
Keep the top k components
e.g. those explaining 95% of variance
- 6
Project the data onto them
Benefits: fewer features, faster training, less overfitting and easier 2-D/3-D visualisation. Drawback: components are harder to interpret than original features.
Example
Steps of PCA: standardise the data; compute the covariance matrix; find its eigenvalues and eigenvectors; sort by eigenvalue; keep the top k eigenvectors; project the data onto them. If the eigenvalues are 2.5, 1.0 and 0.5, the first component explains 2.5 ÷ 4 = 62.5% of the variance.
Key terms
- Binning
- Grouping continuous values into intervals
- Interaction feature
- A feature formed by combining two or more features
- One-hot encoding
- Representing categories as binary columns
- Feature selection
- Choosing a useful subset of existing features
- PCA
- A technique that reduces features into principal components
Quick revision
- Equal-width vs equal-frequency binning.
- Polynomial: x²; interaction: x₁ × x₂.
- One-hot for nominal, label for ordinal.
- Filter (fast), wrapper (accurate), embedded, hybrid.
- PCA keeps components with the largest variance.
Important exam questions
Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).
Short-answer questions
- Q1.What is binning?
- Q2.What are interaction features?
- Q3.When should one-hot encoding be used?
- Q4.Differentiate between feature extraction and feature selection.
- Q5.What is a wrapper method?
- Q6.What does PCA do?
Long-answer questions
- Q1.Explain binning, polynomial and interaction features with examples.
- Q2.Compare one-hot and label encoding.
- Q3.Explain filter, wrapper and hybrid feature selection methods.
- Q4.Explain Principal Component Analysis step by step.
Stuck on this unit?
Message SBS on WhatsApp for help with Feature Engineering, or to ask about studying BCA at Synetic.
