21MAB204T Covariance Correlation Regression

21MAB204T - Probability and Queueing Theory
Unit - III (Part -2)
Dr. E. Suresh & Dr. Krishnodas Goswami ,

Assistant Professor, Department of Mathematics,
SRM Institute of Science and Technology,
Kattankulathur - 603203.
March 10, 2024

.
Covariance
Covariance is a measure of the relationship between two random variables
and to what extent, they change together. Or we can say, in other words,
it defines the changes between the two variables, such that change in one
variable is equal to change in another variable.
Let X and Y be two dimensional random variables then the co-variance

between X and Y is given by
nh ih io
Cov(X , Y ) = E X − E (X ) Y − E (Y )
Note:
nh ih io
E X − E (X ) Y − E (Y )
h i
= E XY − XE (Y ) − YE (X ) + E (X )E (Y )
= E (XY ) − E (X )E (Y ) − E (Y )E (X ) + E (X )E (Y )
= E (XY ) − E [X ] · E [Y ].
Dr. E. Suresh & Dr. Krishnodas Goswami - Dept. of Maths 21MAB204T - Probability and Queueing Theory
Covariance
Properties
1 Cov(X , Y ) = E [XY ] − E [X ] · E [Y ].
h i2
2 Cov(X , X ) = E [XX ] − E [X ] · E [X ] = E X 2 − E (X ) .
3 Cov(X + a, Y + b) = Cov(X , Y ).
4 Cov(aX + c, bY + d) = abCov(X , Y ).
5 Var(X + Y ) = Var(X ) + Var(Y ) + 2Cov(X , Y ).
6 Var(X − Y ) = Var(X ) + Var(Y ) − 2Cov(X , Y ).
7 If X and Y are independent, then Cov(x, y ) = 0.
8 Var(X ± Y ) = Var(X ) ± Var(Y ).
Correlation
Sir Francis Galton (1822 - 1911) is

commonly regarded as the founder of the
statistical techniques of correlation and
linear regression. Galton produced over
340 papers and books. He also created
the statistical concept of correlation and
widely promoted regression toward the
mean. He was the first to apply
statistical methods to the study of human
differences and inheritance of intelligence,
and introduced the use of questionnaires
and surveys for collecting data on human
communities, which he needed for
genealogical and biographical works and
for his anthropometric studies.
Correlation
Correlation
The two variables vary such that the change in one

variable affects the change in another variable. Then
the variables are said to be correlated. Correlation
analysis is a statistical tool used to measure the
degree of relationship between any two variables.
Definitions
Correlation
Correlation is a statistical tool which studies the

nature of relationship between two variables.
It measures the degree ( or extent) of association

(or relationship) between two or more variables.
Definitions
Correlation
1
The relationship between two variables such that
a change in one variable results in a positive or
negative change in the other variable.
2
Also a greater change in one variable results in
corresponding greater or smaller change in
the other variable is known as correlation
COEFFICIENT OF CORRELATION
The degree to which the two variables are interrelated is measured by a

coefficient which is called the coefficient of correlation.
The coefficient of correlation between the two variables x, y is generally
denoted by
r or rxy or r (x, y )
ρ(x, y ) or ρ
Properties.
Let X and Y be two variables
1 Correlation coefficient lies between −1 to 1
−1 ≤ r ≤ 1.

1 If r = 1 then there is a perfect positive correlation.
2 If r = −1 then there is a perfect negative correlation.
3 If correlation coefficient is positive then
X and Y have the same direction.
4 If correlation coefficient is negative then
X and Y have opposite direction.
Properties.
1
If r = 0 then X and Y an uncorrelated (or) no
relation between X and Y .
2
If r = 0, then X and Y are independent.

The correlation coefficient is a pure number and
is not affected by
change of origin.
scale in magnitude.
Scatter Diagram Method
Karl Pearson’s Coefficient Method
1
It is also known as product moment correlation
coefficient.
2
It is suitable for Quantitative Measures.
(variables)
3
It is suitable for Linear Relationship only.
Formula
The coefficient of correlation between two random variables X and Y is

defined by
Cov(X , Y )
r (X , Y ) =
σx σy
σx 6= 0, σy 6= 0
Cov(X , Y )
r (X , Y ) = p p
V (X ) V (Y )
Linear relationship
If given Linear relationship ax + by + c = 0. Rewrite
to y = mx + c
m is slope.
c is Intercept.
if m > 0 then r = 1.
if m < 0 then r = −1.
if m = 0 then r = 0.
Example 1
The joint probability mass function of X and Y is
Y -1 1
X
0 1/8 3/8
1 2/8 2/8
Find the correlation co-efficient of X and Y .

Solution: The joint probability mass function of X and Y is
P
Y -1 1 p (x, y )
x
X
0 1/8 3/8 4/8
1 2/8 2/8 4/8
P
p (x, y ) 3/8 5/8 1
y
X
E [X ] = xP(x)

1 1 1
=0 +1 =
2 2 2
X
2 2
E [X ] = x P(x)

2 1 2 1 1
=0 + (1) =
2 2 2
X
E [Y ] = yP(y )
1
= (−1) (3/8) + 1 (5/8) =
X 4
2 2
E [Y ] = y P(y )
= (−1)2 (3/8) + (1)2 (5/8) = 1
XX
E [XY ] = xy p(x, y )
= x0 y0 p(x0 , y0 ) + x0 y1 p(x0 , y1 ) + x1 y0 p(x1 , y0 ) + x1 y1 p(x1 , y1 )
= 0(−1) (1/8) + 0(1) (3/8) + 1(−1) (2/8) + (1)(1) (2/8)
=0

1 1 −1
Cov(X , Y ) = E XY − E [X ]E [Y ] = 0 − =
2 4 8
σx2 = Var[X ]
2
2 1 2 1 1
= E [X ] − (E [X ]) = − =
2 2 4
1
σx =
2
Correlation co-efficient is
2
1 15
σy2= Var[Y ] = E [Y ] − (E [Y ]) = 1 − 2
= 2
4 16
√
15
σy =
4
Cov(x, y ) −1/8
r (X , Y ) = = √ ! = −0.2582
σx σy 1 15
2 4
r(X,Y) = -0.2582
Example 2
Two random variables X and Y are related as Y = 4X + 9. Find the
correlation coefficient between X and Y .
Solution: Given the relation between X and Y is
Y = 4X + 9 (1)
W.K.T., the correlation coefficient is
Cov(x, y )
r (X , Y ) =
σx σy
E (XY ) − E (X )E (Y )
= (2)
σx σy
For this, we have to find the following E (X ), E (Y ), E (XY ), σx , σy
E (Y ) = E (4X + 9)
= 4E (X ) + 9
E (XY ) = E [X (4X + 9)]
= E [4X 2 + 9X ]
= 4E [X 2 ] + 9E [X ]
Cov(X , Y ) = E [XY ] − E [X ] · E [Y ]
= 4E [X 2 ] + 9E [X ] − E [X ](4E [X ] + 9)
= 4E [X 2 ] + 9E [X ] − 4(E [X ])2 − 9E [X ]
= 4[E [X 2 ] − (E [X ])2 ]
Cov(X , Y ) = 4σx2
σY2 = Var[Y ] = Var[4X + 9]
= 42 Var[X ]
= 16Var[X ]
σy2 = 16σx2
σy = 4σx
Correlation co-efficient is
Cov(x, y ) 4σx2
r (X , Y ) = =
σx σy σx 4σx
r (X , Y ) = 1
which is a perfect positive correlation.

Example 3
If the independent random variables X and Y have the variances 36 and
16 respectively. Find the correlation coefficient between X + Y and
X −Y.
Solution:
Given X and Y are independent, which implies
E (XY ) = E (X )E (Y ) (1)
Given Var(X ) = 36
Var(Y ) = 16
Let U = X + Y
Let V = X − Y
W.K.T., the correlation coefficient between U and Y is
Cov(U, V )
r (U, V ) =
σU σV
E (UV ) − E (U)E (V )
= (2)
σU σV
Cov(U, V ) = E (UV ) − E (U)E (V )
= E [(X + Y )(X − Y )] − E [(X + Y )]E [(X − Y )]
= E X 2 − Y 2 − [E (X ) + E (Y )][E (X ) − E (Y )]

= E X 2 − [E (X )]2 − E Y 2 − [E (Y )]2

= Var(X ) − Var(Y )
= 36 − 16
Cov(U, V ) = 20
σU2 = E U 2 − [E (U)]2

= E (X + Y )2 − [E (X + Y )]2

= E X 2 + Y 2 + 2XY − [E (X ) + E (Y )]2

= E X 2 + E Y 2 + 2E (XY )

− [E (X )]2 − [E (Y )]2 − 2E (X )E (Y )
= E (X 2 ) − [E (X )]2 + E (Y 2 ) − [E (Y )]2

(Since X and Y are independent E (XY ) = E (X )E (Y ))

= Var(X ) + Var(Y )
= 36 + 12
σU2= 52
√
σU = 52
σV2 = E V 2 − [E (V )]2

= E (X − Y )2 − [E (X − Y )]2

= E X 2 − Y 2 + 2XY − [E (X ) + E (Y )]2

= E X 2 + E Y 2 − 2E (XY )

− [E (X )]2 − [E (Y )]2 + 2E (X )E (Y )
= E (X 2 ) − [E (X )]2 + E Y 2 − [E (Y )]2

(Since X and Y are independent E (XY ) = E (X )E (Y ))

= Var(X ) + Var(Y )
= 36 + 16
σV2 = 52
√
σV = 52
Correlation co-efficient,
Cov(U, V ) 20
r (U, V ) = =√ √
σu σv 52 52
r (X + Y , X − Y ) = 0.3846
Example 4
Let (X , Y & Z ) be the uncorrelated random variable with zero mean and
standard deviations 5,12 & 9 respectively. If U = X + Y , V = Y + Z ,
find the correlation coefficient between U and V .
Solution: Given
X , Y and Z are random random variables with mean is zero. i.e.,
E (X ) = 0
E (Y ) = 0
E (Z ) = 0
Given standard deviations of X , Y and Z are 5,12 and 9 i.e.,
σX = 5 (1)
σY = 12
σZ = 9
Given X and Y are uncorrelated, i.e.,
r (X , Y ) = 0
Cov(X , Y ) = 0
E (XY ) − E (X )E (Y ) = 0
E (XY ) − (0)(0) = 0
∴ E (XY ) = 0
Similarly
E (YZ ) = 0
E (ZX ) = 0
Now,
(1) ⇒ σX2 = E X 2 − [E (X )]2 = 25

E X 2 − [0]2 = 25

∴ E X 2 = 25

Similarly, E Y 2 = 144

E Z 2 = 81

Now
Cov(U, V ) = E (UV ) − E (U)E (V )

= E [(X + Y )(Y + Z )] − E (X + Y )E (Y + Z )
= E XY + XZ + Y 2 + YZ − [E (X ) + E (Y )][E (Y ) + E (Z )]

= E (XY ) + E (XZ ) + E Y 2 + E (YZ ) − E (X )E (Y )

− E (X )E (Z ) − [E (Y )]2 − E (Y )E (Z )
= 0 + 0 + 144 + 0 − (0)(0) − (0)(0) − 0 − (0)(0)
Cov(U, V ) = 144
σU2 = E U 2 − [E (U)]2

= E (X + Y )2 − [E (X + Y )]2

= E X 2 + Y 2 + 2XY − [E (X ) + E (Y )]2

= E X 2 + E Y 2 + 2E (XY ) − [E (X )]2 − [E (Y )]2 − 2E (X )E (X )

= 25 + 144 + 2(0) − 02 − 02 − 2(0)(0)

= 25 + 144
∴ σU2 = 169
σU = 13
σV2 = E V 2 − [E (V )]2

= E (Y + Z )2 − [E (Y + Z )]2

= E X 2 + 2YZ + Z 2 − [E (Y ) + E (Z )]2

= E Y 2 + E Z 2 + 2E (YZ ) − [E (Y )]2 − [E (Z )]2 − 2E (Y )E (Z )

= 144 + 81 + 2(0) − 02 − 02 − 2(0)(0)

= 144 + 81
∴ σV2 = 225
σV = 15
∴ Correlation co-efficient is
Cov(U, V ) 144
r (U, V ) = =
σU σV (13)(15)
r (U, V ) = 0.7385
Regression Analysis
Regression
Mathematical Relationship
Estimation
Regression analysis is a set of statistical processes for

estimating the relationships between a dependent
variable (often called the ’outcome’ or ’response’) and one
or more independent variables (often called ’predictors’,
’covariates’, ’explanatory variables’ or ’features’).
Regression Lines
The best average value of one variable associated

with the given value of the other variable may also
be estimated or predicted by means of an equation
and the equation is known as Regression equation.
Regression Line Y on X is
Y = a + bX
1 X is indepedent variable
2 Y is depedent variable
3 a is intercept
4 b is slope
5 b is regression coefficient of Y on X . i.e., byx
Regression line Y on X is derived by minimizing vertical

distance
Regression Line Y on X
(i) Regression equation of Y on X is

σy
Y-Y =r X −X
σx
where
X and Y are means of X and Y series.
σy
r is known as the regression coefficient of Y on X .
σx
σy
It is denoted by denoted by byx , i.e byx = r .
σx

Y - Y = byx X − X
Regression Line X on Y is
X = a + bY
1 Y is indepedent variable
2 X is depedent variable
3 a is intercept
4 b is slope
5 b is regression coefficient of X on Y . i.e., bxy
Regression line X on Y is derived by minimizing horizontal

distance
Regression Line X on Y
(ii) Regression equation of X on Y is

σx
X-X =r Y −Y
σy
X and Y are means of X and Y series.

σx
r is known as the regression coefficient of X on Y .
σy
σx
It is denoted by denoted by bxy , i.e bxy = r .
σy

X - X = bxy Y − Y
Properties
1 Either all byx , bxy , r are Positive or
all byx , bxy , r are Negative.
2 It is never possible that

byx is positive and bxy is negative and versa.
3 Both bxy and byx cannot be greater than one.
Properties
1 The correlation coefficient between X and Y is the

geometric mean of the regression coefficients
p
r = ± bxy × byx
p
2 If bxy , byx are positive then r = + bxy × byx
p
3 If bxy , byx are negative then r = − bxy × byx
Explanations.
Regression coefficients are

independent of change of origin.
but not for change of scale.
1
The regression lines become identical if r = ± 1.
Angle between them are 0 degree
2
If r = 0, then these lines are perpendicular to
each other.
Angle between them are 90 degree
(i) Regression lines intersect at point (were passes through
the point) (x, y )
(ii) Angle between regression lines :
1 − r2

σx σy
tan θ = ∗
r σx2 + σy2
Note:
π
(i) Suppose r = 0 then tan θ = ∞ → θ = = 90
2
The two regression lines are perpendicular to each other
(ii) If r = ±1 then tan θ = 0 → θ = 0 (or ) π
Here the lines of regression coincide.
Example 1
In a partially destroyed laboratory record of an analysis of correlation
data, following result only legible. Variance of X = 9. The regression
equations are 8X − 10Y + 66 = 0 and
40X − 18Y = 214. What are
(i) The means value of X and Y
(ii) the correlation coefficient bet X and Y
(iii) the S.D of Y .
Solution: (i) To find the means value of X and Y

The equations of the regression lines are
8X − 10Y = −66 (1)
and
40X − 18Y = 214 (2)

Since the regression lines were passes through the point X , Y . Hence
equations (1) and (2) becomes
(1) ⇒ 8X − 10Y = −66 (3)
(2) ⇒ 40X − 18Y = 214 (4)

Solve the equations (3) and (4), we get
X = 13, Y = 17
(ii) To find the correlation coefficient bet X and Y
Let us consider equation (1)
(1) ⇒ 8X − 10Y = −66
⇒ 10y = 8x + 66
8 66
⇒y = x+
10 10
y = 0.8x + 6.6
The regression coefficient of y on x is
8
byx = = 0.8
10
(2) can be written as
(2) ⇒ 40X − 18Y = 214
⇒ 40x = 18y + 214
18 214
⇒x = y+
40 40
x = 0.45x + 5.35
The regression coefficient of x on y is
18
byx = = 0.45
40
The correlation coefficient between X and Y is

p √
r = + byx × bxy = 0.8 × 0.45 = 0.60.
Note:
(1) ⇒8x − 10y + 66 = 0 (1) ⇒8x − 10y + 66 = 0
⇒ 8x = 10y − 66 ⇒ 10y = 8x + 666
10 66 8 66
⇒x = y− ⇒y = x+
8 8 10 10
10 8
∴ bxy = ∴ byx =
8 10
(2) ⇒40x − 18y − 214 = 0 (2) ⇒40x − 18y − 214 = 0
⇒40x − 214 = 18y ⇒ 40x = 18y + 214
40 214 18 214
⇒y = y− ⇒x = y+
18 18 40 40
40 18
∴ byx = ∴ bxy =
18p 40p
∴ r = ± bxy · byx ∴ r = ± bxy · byx
r r
10 40 18 8
r =± · = ±2.777 r =± · = ±0.6
8 18 40 10
p
W.K.T. If bxy , byx are positive then r = + bxy × byx ∴ r= 0.6
(iii) To find the S.D of Y .
We know that
σy
byx = r
σx
8 r σy 8
byx = ⇒ =
10 σx 10
r σy 8 0.6 × σy
⇒ = ⇒ = 0.8
σx 10 3
0.8 × 3
⇒ σy = = 4 ⇒ σy = 4
0.6
Example 2
The joint p.d.f of a two dimensional random variable is given by
1
f (x, y ) = (x + y ), 0 ≤ x ≤ 1, 0 ≤ y ≤ 2.
3
(i) Find the correlation coefficient.

(ii) Find the equations of the two lines of regression.
Solution
Z∞ Z∞
fX (x) = f (x, y )dy fY (y ) = f (x, y )dx
−∞ −∞
Z2 Z1
1 1
= (x + y )dy = (x + y )dy
3 3
0 0
2 1
1 x2

y2

1 = + xy
= xy + 3 2
3 2 0
0
1 1 1
= [2x + 2] = +y
3 3 2
2 1
fX (x) = [x + 1], 0 ≤ x ≤ 1 fY (y ) = [1 + 2y ], 0 ≤ y ≤ 2
3 6
Z∞ Z∞
E [X ] = xf (x)dx E [Y ] = yf (y )dy
−∞ −∞
Z1 Z2
2 1
= x (x + 1) dx = y (1 + 2y ) dy
3 6
0 0
1 2
2 x3 x2 1 y2 y3

= + =+2
3 3 2 6 2 3
0 0
2 1 1 10 1 16
= + = = 2+
3 3 2 18 6 3
5 11
X = E [X ] = Y = E [Y ] =
9 9
Z∞ Z∞
E X2 = 2
E Y2 = y 2 f (y )dy

x f (x)dx
−∞ −∞
Z1 Z1
2 2 1
= x (x + 1) dx = y2 (1 + 2y ) dy
3 6
0 0
Z1 Z1
2 3 2 1
y 2 + 2y 3 dy

= x +x dx =
3 6
0 0
1 2
2 x4 x3 1 y3 y4

= + = +2
3 4 3 0 6 3 4
0
2 1 1 1 8
= + − (0 + 0) = + 8 − (0 + 0)
3 4 3 6 3
7 16
= =
18 9
σX2 = Var(X ) = E [X 2 ] − (E [X ])2 σY2 = Var(Y ) = E [Y 2 ] − (E [Y ])2
2 2
7 5 16 11
= − = −
18 9 9 9
7 25 16 121
= − = −
18 81
r r9 81
13 23
σX = σY =
162 81
1 2
1 1 2 2
Z Z Z Z
E [XY ] = xyf (x, y )dydx = (x y + xy 2 )dydx
0 0 3 0 0
2
1 1 x 2y 2 xy 3 1 1
Z Z
8
= + dx = 2x 2 + x dx
3 0 2 3 0 3 0 3
2 1
3
1 2x 8x 1 2 4 1 6 2
= + = + = =
3 3 3 2 0 3 3 3 3 3 3
2 5 11 2 55 1
Cov [X , Y ] = E [XY ] − E [X ]E [Y ] = − × = − =−
3 9 9 3 81 81
−1 r
Cov (X , Y ) 81 2
r (X , Y ) = p p =r r =−
Var (X ) Var (Y ) 13 23 299
×
162 81
(ii) The lines of regression are
σx
[X − E [X ]] = r [Y − E [Y ]]
σy
(or)
σx
[X − X ] = r
[Y − Y ]
σy
r
r 13
5 2 162 11
X− =− × r Y−
9 299 23 9
81

5 1 11
X− =− Y−
9 23 9
1 14
X= - Y−
23 23
r σy
[Y − E [Y ]] = [X − E [X ]]
σx
(or)
σy
[Y − Y ] = r
[X − X ]
σx
r
r 23
11 2 81 5
Y− =− ×r X−
9 299 13 9
162

11 2 5
Y− =− X−
9 13 9
2X 17
Y= - −
13 13

21MAB204T Covariance Correlation Regression

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

21MAB204T Covariance Correlation Regression

Uploaded by

Copyright:

Available Formats

21MAB204T - Probability and Queueing Theory

Unit - III (Part -2)

Dr. E. Suresh & Dr. Krishnodas Goswami ,

March 10, 2024

Let X and Y be two dimensional random variables then the co-variance

Sir Francis Galton (1822 - 1911) is

The two variables vary such that the change in one

Correlation is a statistical tool which studies the

It measures the degree ( or extent) of association

The degree to which the two variables are interrelated is measured by a

The coefficient of correlation between the two variables x, y is generally

Let X and Y be two variables

Let X and Y be two variables

The coefficient of correlation between two random variables X and Y is

If given Linear relationship ax + by + c = 0. Rewrite

Find the correlation co-efficient of X and Y .

W.K.T., the correlation coefficient is

For this, we have to find the following E (X ), E (Y ), E (XY ), σx , σy

which is a perfect positive correlation.

(Since X and Y are independent E (XY ) = E (X )E (Y ))

(Since X and Y are independent E (XY ) = E (X )E (Y ))

Given standard deviations of X , Y and Z are 5,12 and 9 i.e.,

Cov(U, V ) = E (UV ) − E (U)E (V )

= E (XY ) + E (XZ ) + E Y 2 + E (YZ ) − E (X )E (Y )

= E X 2 + E Y 2 + 2E (XY ) − [E (X )]2 − [E (Y )]2 − 2E (X )E (X )

= 25 + 144 + 2(0) − 02 − 02 − 2(0)(0)

= E Y 2 + E Z 2 + 2E (YZ ) − [E (Y )]2 − [E (Z )]2 − 2E (Y )E (Z )

= 144 + 81 + 2(0) − 02 − 02 − 2(0)(0)

Regression analysis is a set of statistical processes for

The best average value of one variable associated

Regression line Y on X is derived by minimizing vertical

(i) Regression equation of Y on X is

Regression line X on Y is derived by minimizing horizontal

(ii) Regression equation of X on Y is

X and Y are means of X and Y series.

2 It is never possible that

1 The correlation coefficient between X and Y is the

Regression coefficients are

Solution: (i) To find the means value of X and Y

8X − 10Y = −66 (1)

(1) ⇒ 8X − 10Y = −66 (3)

(2) ⇒ 40X − 18Y = 214 (4)

(1) ⇒ 8X − 10Y = −66

The regression coefficient of y on x is

(2) ⇒ 40X − 18Y = 214

⇒ 40x = 18y + 214

The regression coefficient of x on y is

The correlation coefficient between X and Y is

(i) Find the correlation coefficient.

You might also like