Unit 3: Correlation & regression analysis
Business Statistics notes · PTU syllabus (BBA 201-18)
On this page
Unit summary
Business variables move together — sales with advertising, price with demand. Correlation measures how strongly two variables are related and regression predicts one from the other. This unit covers simple, multiple and partial correlation, linear and non-linear correlation, scatter diagrams, Pearson's correlation coefficient, rank correlation, the principle of least squares, and regression coefficients with their properties.
After this unit you can
- Identify the types of correlation from a scatter diagram
- Compute Pearson's coefficient of correlation and Spearman's rank correlation
- Explain multiple and partial correlation
- Fit regression lines by the method of least squares and use them for prediction
- State the properties of regression coefficients and the relation between correlation and regression
PTU syllabus topics
- Simple
- multiple and partial correlation
- linear and non-linear correlation
- scatter diagrams
- Pearson's correlation coefficient
- Rank correlation
- principle of least squares
- regression coefficient properties
Topic 1
Correlation and its types
Correlation is the statistical measure of the degree and direction of the linear relationship between two variables. It does not prove cause and effect.
Positive
Both variables move in the same direction
Income and expenditure
Negative
Variables move in opposite directions
Price and demand
Zero
No relationship
Shoe size and marks
Simple
Two variables
Advertising and sales
Multiple
One variable with two or more others together
Yield and rainfall plus fertiliser
Partial
Relation of two variables, others held constant
Yield and rainfall, fertiliser fixed
Linear
Constant change — a straight line
Cost and quantity
Non-linear (curvilinear)
Unequal change — a curve
Fertiliser and yield beyond a point
Scatter diagram
Points cluster tightly along a rising line
Points cluster along a falling line
Points are scattered with no pattern
Reading a scatter diagram
| Meaning | |
|---|---|
| Rising pattern | Positive correlation |
| Falling pattern | Negative correlation |
| Tight band | High degree of correlation |
| Wide spread | Low degree |
| Curve | Non-linear relationship |
| No pattern | Zero correlation |
Degree of correlation
| Value of r | |
|---|---|
| +1 or −1 | Perfect positive or negative |
| 0.75 to 0.99 | Very high |
| 0.50 to 0.74 | Moderate to high |
| 0.25 to 0.49 | Low |
| below 0.25 | Negligible |
| 0 | No linear correlation |
Topic 2
Karl Pearson's coefficient of correlation
Covariance form
r = Cov(x, y) ÷ (σx σy), where Cov(x, y) = Σ(x − x̄)(y − ȳ) ÷ n
Raw-score form
r = [nΣxy − ΣxΣy] ÷ square root of {[nΣx² − (Σx)²][nΣy² − (Σy)²]}
Deviation form
r = Σdxdy ÷ square root of (Σdx² × Σdy²), dx = x − x̄, dy = y − ȳ
Range
−1 ≤ r ≤ +1
Coefficient of determination
r² — share of variation in y explained by x
Probable error
PE = 0.6745 (1 − r²) ÷ √n
- Limits
- Between −1 and +1
- Independent of change of origin and scale
- Adding or multiplying both variables by constants (same sign) leaves r unchanged
- Symmetric
- r(x, y) = r(y, x)
- Geometric mean
- r is the GM of the two regression coefficients
- Independent variables
- r = 0 (but r = 0 does not prove independence)
- Only linear
- Measures linear association
| x | y | xy | x² | y² |
|---|---|---|---|---|
| 1 | 2 | 2 | 1 | 4 |
| 2 | 4 | 8 | 4 | 16 |
| 3 | 5 | 15 | 9 | 25 |
| 4 | 4 | 16 | 16 | 16 |
| 5 | 5 | 25 | 25 | 25 |
| Σ = 15 | Σ = 20 | Σ = 66 | Σ = 55 | Σ = 86 |
Example
n = 5. r = (5 × 66 − 15 × 20) ÷ square root of {(5 × 55 − 15²)(5 × 86 − 20²)} = 30 ÷ square root of (50 × 30) = 30 ÷ 38.73 = +0.775 — high positive correlation.
Exam tip
Steps for r: tabulate x, y, xy, x², y²; take the totals; substitute in the raw-score formula; interpret the sign and size.
Topic 3
Spearman's rank correlation
Used when data are ranks or cannot be measured exactly (beauty, honesty) or when the relationship is not linear.
Without ties
ρ = 1 − [6 Σd²] ÷ [n (n² − 1)], d = difference of ranks
With tied ranks
Add the correction m(m² − 1) ÷ 12 to Σd² for each group of m tied items
Range
−1 to +1
| Student | Rank in Maths R₁ | Rank in Stats R₂ | d = R₁ − R₂ | d² |
|---|---|---|---|---|
| A | 1 | 2 | −1 | 1 |
| B | 2 | 1 | 1 | 1 |
| C | 3 | 4 | −1 | 1 |
| D | 4 | 3 | 1 | 1 |
| E | 5 | 5 | 0 | 0 |
| Total | 0 | 4 |
Example
ρ = 1 − (6 × 4) ÷ [5 × (25 − 1)] = 1 − 24 ÷ 120 = 0.8 — strong agreement between the two rankings.
- 1Arrange values and give ranks 1, 2, 3 …
- 2Tied items get the average of the ranks they occupy — two values tied at ranks 3 and 4 each get 3.5
- 3Compute d and d²
- 4Add the correction factor m(m² − 1) ÷ 12 for every tie group
- 5Apply the formula
Data
Measured values
Ranks (or values converted to ranks)
Assumption
Linear relation, roughly normal data
Distribution-free
Sensitive to outliers
Yes
Less
Computation
Longer
Quick for small samples
Multiple and partial correlation
Partial correlation
r₁₂.₃ = (r₁₂ − r₁₃ r₂₃) ÷ square root of [(1 − r₁₃²)(1 − r₂₃²)]
Multiple correlation
R₁.₂₃ = square root of [(r₁₂² + r₁₃² − 2 r₁₂ r₁₃ r₂₃) ÷ (1 − r₂₃²)]
Range
Partial: −1 to +1; multiple: 0 to 1
Meaning
r₁₂.₃ = relation of 1 and 2 with 3 held constant
Example
If r₁₂ = 0.8, r₁₃ = 0.6 and r₂₃ = 0.5: r₁₂.₃ = (0.8 − 0.3) ÷ square root of (0.64 × 0.75) = 0.5 ÷ 0.693 = 0.72.
Topic 4
Regression analysis
Regression is the statistical technique of estimating the average value of one variable from the known value of another. The line of best fit is obtained by the principle of least squares: it minimises the sum of the squares of the vertical differences between actual and estimated y.
The dashed vertical gaps are the errors; the line is placed so that the sum of their squares is least
Regression of y on x
y − ȳ = byx (x − x̄), with byx = r σy ÷ σx = Cov(x, y) ÷ σx²
Regression of x on y
x − x̄ = bxy (y − ȳ), with bxy = r σx ÷ σy
Normal equations (y = a + bx)
Σy = na + bΣx ; Σxy = aΣx + bΣx²
Direct formula
b = [nΣxy − ΣxΣy] ÷ [nΣx² − (Σx)²] ; a = ȳ − b x̄
Example
From the earlier data: byx = 30 ÷ 50 = 0.6 and bxy = 30 ÷ 30 = 1. Regression of y on x: y − 4 = 0.6 (x − 3), i.e. y = 2.2 + 0.6x. For x = 6, y = 5.8.
- Geometric mean
- r = ± square root of (byx × bxy)
- Same sign
- byx, bxy and r always have the same sign
- Not both greater than 1
- If one coefficient exceeds 1, the other is less than 1
- Arithmetic mean
- (byx + bxy) ÷ 2 ≥ r
- Origin and scale
- Regression coefficients are independent of change of origin but not of scale
- Intersection
- The two regression lines meet at (x̄, ȳ)
- Angle
- Lines coincide if r = ±1 and are perpendicular if r = 0
Purpose
Strength and direction of the relationship
Estimating one variable from the other
Variables
Symmetric — no dependent variable
Dependent and independent
Result
Single coefficient r
Equation (line)
Limits
−1 to +1
Coefficients can be any size
Cause and effect
Not implied
Assumed direction
The smaller the angle between the lines, the higher the correlation
- 1Draw a scatter diagram to check the relationship is linear
- 2Compute x̄, ȳ, byx and bxy
- 3Write the regression equation
- 4Substitute the known x to estimate y
- 5Predict only within the observed range of x
Key terms
- Correlation
- Degree and direction of linear relationship between two variables
- Scatter diagram
- Plot of paired values showing the pattern of relationship
- Pearson's r
- Covariance divided by the product of the standard deviations
- Rank correlation
- Correlation computed on ranks
- Partial correlation
- Correlation of two variables with others held constant
- Regression
- Estimating the average value of one variable from another
- Least squares
- Fitting a line by minimising the sum of squared errors
- Regression coefficient
- Slope of a regression line (byx or bxy)
Quick revision
- r lies between −1 and +1 and is independent of change of origin and scale.
- r = [nΣxy − ΣxΣy] ÷ √{[nΣx² − (Σx)²][nΣy² − (Σy)²]}.
- Spearman ρ = 1 − 6Σd² ÷ n(n² − 1).
- byx = r σy/σx; bxy = r σx/σy; r² = byx × bxy; lines meet at (x̄, ȳ).
- Correlation shows association, not causation; regression gives the prediction equation.
Important exam questions
Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).
Short-answer questions
- Q1.Define correlation. Name its types.
- Q2.What does a scatter diagram show?
- Q3.State the limits and two properties of Pearson's r.
- Q4.Write Spearman's rank correlation formula.
- Q5.Differentiate between simple, multiple and partial correlation.
- Q6.State the principle of least squares.
- Q7.State three properties of regression coefficients.
- Q8.Differentiate between correlation and regression.
- Q9.If byx = 0.8 and bxy = 0.45, find r.
Long-answer questions
- Q1.Explain the types of correlation with scatter diagrams.
- Q2.Calculate Pearson's coefficient of correlation for the given data and interpret it.
- Q3.Calculate the rank correlation coefficient, including a case with tied ranks.
- Q4.Explain multiple and partial correlation with formulas.
- Q5.Derive the two regression lines and state the properties of regression coefficients.
- Q6.Fit a regression line of y on x by least squares and estimate y for a given x.
Stuck on this unit?
Message SBS on WhatsApp for help with Business Statistics, or to ask about studying BBA at Synetic.
