Simple, Partial and Multiple

Correlation
Scatter Plot Examples
Strong relationships Weak relationships

y y

x x

y y

x Muhammad Usman x
Scatter Plot Examples
(continued)

No relationship

x
Muhammad Usman
Examples of Approximate “r” values
y y y

x x x
r = -1 r = -0.6 r=0

y
y

x x
r = +0.3 Muhammad Usman
r = +1
Correlation Coefficient
• Measure of the degree of linear association between two variables.

• The population correlation coefficient ρ (rho)

• The sample correlation coefficient r is an estimate of ρ

Muhammad Usman
Features of ρ and r
• Range between -1 and 1
• It is symmetrical w.r.t variables i.e., rxy = ryx
• Unit free
• “r” is independent of change in origin and scale

• The closer to -1, the stronger the negative linear relationship

• The closer to 1, the stronger the positive linear relationship
• The closer to 0, the weaker the linear relationship

Muhammad Usman
• The value of r ranges between ( -1) and ( +1)
• The value of r denotes the strength of the association as
illustrated by the following diagram.

strong intermediate weak weak intermediate strong

-1 -0.75 -0.25 0 0.25 0.75 1

indirect Direct
perfect perfect
correlation correlation
no relation
Muhammad Usman
Example
• The following data represent the Study hours and Study hours Marks
the Marks of 7 students in the subject of X Y
Mathematics. Find the Correlation Coefficient ‘r’ 5 50
and interpret it. 3 35
5 45
𝑥𝑦 3 26
𝑟= 4 30
𝑥2 𝑦2 3 35
4 40
• Coefficient of Correlation r = 0.804

Muhammad Usman
Example
Wing length Tail length (Y)
(X)
10.4 7.4
• The following data represent the 10.8 7.6
wing length and tail length of 11.1 7.9
sparrows 10.2 7.2
10.3 7.4
𝑥𝑦 10.2 7.1
𝑟= 10.7 7.4
𝑥2 𝑦2 10.5 7.2
10.8 7.8
11.2 7.7
10.6 7.8
11.4 8.3
Muhammad Usman
128.2 90.8
Hypothesis Testing about Correlation Coefficient
Case-I:- Population correlation co-efficient ρ is equal to zero

• Step-I: Formulation of hypothesis 𝐻0 : 𝜌 = 0, 𝜌 ≥ 0, 𝜌≤0

𝐻1 : 𝜌 ≠ 0, 𝜌 < 0, 𝜌>0
• Step-II: Level of Significance: α=0.05
• Step-III: Test Statistics 𝑟−𝜌 1 − 𝑟2
𝑡= 𝑤ℎ𝑒𝑟𝑒, 𝑆𝐸 𝑟 =
𝑆𝐸 𝑟 𝑛−2
• Step-IV: Calculations: Hypothesis Decision Rules

• Step-V: Decision Rule: 𝐻1 : 𝜌 ≠ 0 𝑡𝑐𝑎𝑙 ≤ −𝑡𝛼 , 𝑛−2 OR

2
𝑡𝑐𝑎𝑙 ≥ 𝑡𝛼 , 𝑛−2
2
𝐻1 : 𝜌 > 0 𝑡𝑐𝑎𝑙 ≥ 𝑡𝛼 ,(𝑛−2)
• Step-VI: Conclusion
𝐻1 : 𝜌 < 0 𝑡𝑐𝑎𝑙 ≤ −𝑡𝛼 ,(𝑛−2)
Muhammad Usman
Hypothesis Testing about Correlation Coefficient
Case-2:- Population correlation co-efficient ρ is equal to ρ0 where ρ0 not equal to zero.
𝐻0 : 𝜌 = 𝜌0 , 𝜌 ≥ 𝜌0 , 𝜌 ≤ 𝜌0
• Step-I: Formulation of hypothesis
𝐻1 : 𝜌 ≠ 𝜌0 , 𝜌 < 𝜌0 , 𝜌 > 𝜌0
• Step-II: Level of Significance: α=0.05
• Step-III: Test Statistics
1 1+𝑟 1 1+𝜌
𝑍𝑓 = 𝑙𝑛 , 𝜇𝑧 = 𝑙𝑛
𝑍𝑓 − 𝜇𝑧 2 1−𝑟 2 1−𝜌
𝑍= , 𝑤ℎ𝑒𝑟𝑒 1
𝑆𝐸 𝑍𝑓 𝑆𝐸 𝑍𝑓 =
𝑛−3
Hypothesis Decision Rules
• Step-IV: Calculations:
𝐻1 : 𝜌 ≠ 𝜌0 𝑍𝑐𝑎𝑙 ≤ −𝑍𝛼 OR
• Step-V: Decision Rule: 2
𝑍𝑐𝑎𝑙 ≥ 𝑍𝛼
2
𝐻1 : 𝜌 > 𝜌0 𝑍𝑐𝑎𝑙𝑐 ≥ 𝑍𝛼
• Step-VI: Conclusion
Muhammad Usman 𝐻1 : 𝜌 < 𝜌0 𝑍𝑐𝑎𝑙𝑐 ≤ −𝑍𝛼
Example
A random sample of size 28 pairs from a bivariate normal population
showed a correlation coefficient of 0.7. Is this value consistent with the
assumption that the correlation coefficient in the population is 0.5?
• Step 1: Hypothesis
• Step 2: Choose α
• Step-3: Test Statistics
Z = 1.6
• Step-4: Calculations
• Step-5: Decision Rule
• Step-6: Conclusion

Muhammad Usman
Partial Correlation
• The relationship between two variables may be affected by other variables
which either strengthen or weakens the relationship.
• Partial correlation is a measure of the strength of a relationship between two
variables while controlling for the effect of one or more other variables.
• Monthly Income and Education Level of an individual is affected by the
Experience of the individual. To get the REAL relationship between two
variables other extraneous factors which are suspected to affect the
relationship are controlled or partial out by the use of partial correlation
coefficients.
• Similarly, you might want to see if there is a correlation between caloric intake
and blood pressure, while controlling for weight or amount of exercise.
Muhammad Usman
Partial Correlation Coefficient
• If we have three variables X1, X2, and X3 then the population partial correlation
coefficient between X1 and X2, keeping the effect of X3 constant is denoted by
ρ12.3(read as rho one two dot three) and can be calculated in terms of simple
correlation coefficients as follows.
𝑟12 − 𝑟13 𝑟23
𝑟12.3 =
2 2
1 − 𝑟13 ∗ 1 − 𝑟23
• r12 is the simple correlation coefficient between X1 and X2 r12 = r21
• r13 is the simple correlation coefficient between X1 and X3
r13 = r31
r23 = r32
• r23 is the simple correlation coefficient between X2 and X3

Muhammad Usman
r13.2 and r23.1
• Similarly we can compute
𝑟13 − 𝑟12 𝑟23
𝑟13.2 =
2 2
1 − 𝑟12 ∗ 1 − 𝑟23

𝑟23 − 𝑟12 𝑟13

𝑟23.1 =
2 2
1 − 𝑟12 ∗ 1 − 𝑟13

• r12.3 r13.2 and r23.1 are known as first order partial correlation.

Muhammad Usman
Testing of hypothesis for Partial Correlation
The procedure of testing of hypothesis for Partial Correlation is similar to the
Simple Correlation
Case-I:- Population correlation co-efficient ρ12.3 is equal to zero
Case-2:- Population correlation co-efficient ρ12.3 is equal to some value other
than zero

Muhammad Usman
Testing of hypothesis for Partial Correlation
Case-I:- Population correlation co-efficient ρ12.3 is equal to zero
• Step-I: Formulation of hypothesis 𝐻0 : 𝜌12.3 = 0, 𝜌12.3 ≥ 0, 𝜌12.3 ≤ 0
𝐻1 : 𝜌12.3 ≠ 0, 𝜌12.3 < 0, 𝜌12.3 > 0
• Step-II: Level of Significance: α=0.05
• Step-III: Test Statistics
𝑟12.3 − 𝜌12.3 1 − 𝑟12.3 2
𝑡= 𝑤ℎ𝑒𝑟𝑒, 𝑆𝐸 𝑟12.3 = , q is the number of variable kept constant
𝑆𝐸 𝑟12.3 𝑛−q−2
• Step-IV: Calculations: Hypothesis Decision Rules
𝐻1 : 𝜌12.3 ≠ 0 𝑡𝑐𝑎𝑙 ≤ −𝑡𝛼 , 𝑛−𝑞−2 OR
• Step-V: Decision Rule: 2
𝑡𝑐𝑎𝑙 ≥ 𝑡𝛼 , 𝑛−𝑞−2
2
𝐻1 : 𝜌12.3 > 0 𝑡𝑐𝑎𝑙𝑐 ≥ 𝑡𝛼,(𝑛−𝑞−2)
• Step-VI: Conclusion
𝐻1 : 𝜌12.3 < 0 𝑡𝑐𝑎𝑙𝑐 ≤ −𝑡𝛼 ,(𝑛−𝑞−2)
Muhammad Usman
Testing of hypothesis for Partial Correlation
Case-II: Population correlation co-efficient ρ12.3 is equal to some value other than zero

• Step-I: Formulation of hypothesis 𝐻0 : 𝜌12.3 = 𝜌0 , 𝜌12.3 ≥ 𝜌0 , 𝜌12.3 ≤ 𝜌0

𝐻1 : 𝜌12.3 ≠ 𝜌0 , 𝜌12.3 < 𝜌0 , 𝜌12.3 > 𝜌0
• Step-II: Level of Significance: α=0.05
• Step-III: Test Statistics 𝑍𝑓 − 𝜇𝑧
𝑍= , 𝑤ℎ𝑒𝑟𝑒
𝑆𝐸 𝑍𝑓
1 1 + 𝑟12.3 1 1 + 𝜌12.3
𝑍𝑓 = 𝑙𝑛 , 𝜇𝑧 = 𝑙𝑛
2 1 − 𝑟12.3 2 1 − 𝜌12.3
1
• Step-IV: Calculations: 𝑆𝐸 𝑍𝑓 = Hypothesis Decision Rules
𝑛−𝑞−3 𝐻1 : 𝜌12.3 ≠ 𝜌0 𝑍𝑐𝑎𝑙 ≤ −𝑍𝛼 OR
• Step-V: Decision Rule: 𝑍𝑐𝑎𝑙 ≥ 𝑍𝛼
2

2
𝐻1 : 𝜌12.3 > 𝜌0 𝑍𝑐𝑎𝑙𝑐 ≥ 𝑍𝛼
• Step-VI: Conclusion
Muhammad Usman
𝐻1 : 𝜌12.3 < 𝜌0 𝑍𝑐𝑎𝑙𝑐 ≤ −𝑍𝛼
Example
• Find simple correlation coefficients r12, r23, r13 and X1 X2 X3
interpret the results 3 16 90
• Calculate partial correlation coefficients r13.2, r12.3, r23.1 5 10 72
and interpret the results, Also test the hypothesis that 6 7 54
ρ12.3= 0.70 8 4 42
12 3 30
14 2 12

Results:
n= 6, r12= -0.891, r13= -0.969, r23= 0.961
Muhammad Usman
Multiple Correlation
• Multiple correlation is a measure of the linear relationship between a single
dependent variable and a set of explanatory variables
• If X1, X2, and X3 are three variables and we want to measure the combined
effect of X2 and X3 on X1, then the Population correlation coefficient is denoted
by ρ1.23 (read as rho one dot two three) and can be calculated as
2 2
𝑟12 + 𝑟13 − 2𝑟12 𝑟13 𝑟23
𝑅1.23 = 2
1 − 𝑟23
• Its value is always between zero and 1. The R2 1.23 is the same quantity as is the
coefficient of multiple determination, calculated in a multiple regression taking
X1 response and X2, X3 as explanatory variables.

Muhammad Usman
R 2.13 and R 3.12
2 2
𝑟12 + 𝑟23 − 2𝑟12 𝑟13 𝑟23
𝑅2.13 = 2
1 − 𝑟13

2 2
𝑟13 + 𝑟23 − 2𝑟12 𝑟13 𝑟23
𝑅3.12 = 2
1 − 𝑟12

• Find multiple correlation coefficients R1.32, R2.13 and R3.12 and interpret the
results. Test the hypothesis that ρ3.12= 0

Muhammad Usman
Testing of hypothesis for Multiple Correlation
To test whether the multiple correlation coefficient ρ12.3 is equal to ZERO or not
• Step-I: Formulation of hypothesis 𝐻0 : 𝜌1.23 = 0
• Step-II: Level of Significance: α=0.05 𝐻1 : 𝜌1.23 ≠ 0
• Step-III: Test Statistics
2
𝑛 − 𝑞 − 1 𝑅1.23
𝐹= 2
𝑞 1 − 𝑅1.23
where q is the no. of variables whose combined effect is being seen on a response variable i. e. , in 𝑅1.23 , q = 2

• Step-IV: Calculations:
• Step-V: Decision Rule:
F𝐶𝑎𝑙𝑐 > 𝐹𝛼 ; (𝑞 , 𝑛 − 𝑞 − 1)
• Step-VI: Conclusion
Muhammad Usman
Example
The marks in Statistics (X1) are expressed as a function of marks in
Mathematics (X2), Economics (X3) and intelligence tests (X4). For a random
sample of 50 students, the Multiple Correlation Co-efficient R1.234 was found
to be 0.582. Test the hypothesis that the Multiple Correlation Co-efficient in
the Population is zero at α=0.05.
Solution:
Fcalc=7.87
Fα(q, n-q-1)=F0.05(3, 46)=2.81
Conclusion: Since the calculated value of F falls in the critical region, so we
reject the Null Hypothesis and may conclude that the Multiple Correlation
Co-efficient in the Population differs from zero Significantly.
Muhammad Usman