University of zakho

Faculty of science
Department of mathematics

The Covariance of Two Random Variables

Under supervision of Dr: shilan

Reported by
• Zozan Salih
• Viyan Mahdi
• Wasan Khlil

3rd stage
Mathematics statistics

|Page1
Contents:

1. Introduction
2. Definition
3. Properties of covariance
4. Sums and integrals for computing covariance
5. Theorem
6. Example

|Page2
Introduction
Intuitively, we think of the dependence of two random
variables Y1 and Y2 as implying that one variable—say, Y1—
either increases or decreases as Y2 changes. We will
confine our attention to two measures of dependence: the
covariance between two random variables and their
correlation coefficient. In Figure 1(a) and (b), we give
plots of the observed values of two variables, 𝑌1 and 𝑌2, for
samples of 𝑛 = 10 experimental units drawn from each of
two populations. If all the points fall along a straight line, as
indicated in Figure 1(a), Y1 and Y2 are obviously dependent.
In contrast, Figure 1 (b) indicates little or no dependence
between Y1 and Y2. Suppose that we knew the values of E
(𝑌1) = µ1 and 𝐸(𝑌2) = µ2 and located this point on the
graph in Figure 1 . Now locate a plotted point, (𝑦1, 𝑦2), on
Figure 5.8(a) and measure the deviations (𝑦1 − µ1) and
(𝑦2 − µ2). Both deviations assume the same algebraic sign
for any point, (𝑦1, 𝑦2), and their product (𝑦1 − µ1) (𝑦2 −
µ2) is
positive. Points to the right of µ1 yield pairs of positive
deviations; points to the left produce pairs of negative
deviations; and the average of the product of the deviations
(𝑦1 − µ1)(𝑦2 − µ2) is large and positive. If the linear
relation indicated in Figure 1 (a) had sloped downward to
the right, all corresponding pairs of deviations would have
been of the opposite sign, and the average value of (𝑦1 −
µ1) (𝑦2 − µ2) would have been a large negative number.

|Page3
 FIGURE 1
Dependent and
independent
Observations for (𝑦1, 𝑦2)

The situation just described does not occur for Figure 1(b),
where little depen-dence exists between 𝑌1 and 𝑌2. Their
corresponding deviations(𝑦1 − µ1) 𝑎𝑛𝑑 (𝑦2 − µ2) will assume
the same algebraic sign for some points and opposite signs for
others.Thus, the product (𝑦1 − µ1)(𝑦2 − µ2) will be positive
for some points, negative for others, and will average to some
value near zero. Clearly, the average value of
(𝑌1 − µ1) (𝑌2 − µ2) provides a measure of the linear
dependence between Y1 and Y2. This quantity, 𝐸 [(𝑌1 −
µ1)(𝑌2 − µ2)], is called the covariance of 𝑌1 and 𝑌2.

DEFINITION
If Y1 and Y2 are random variables with means µ1 and µ2,
respectively, the covariance of Y1 and Y2 𝑖𝑠 𝐶𝑜𝑣(𝑌1, 𝑌2) =
𝐸 [(𝑌1 − µ1)(𝑌2 − µ2)] .The larger the absolute value of
the covariance of Y1 and Y2, the greater the linear dependence
between 𝑌1 and 𝑌2. Positive values indicate that 𝑌1 increases
as 𝑌2 increases; negative values indicate that 𝑌1 decreases
|Page4
as 𝑌2 increases. A zero value of the covarianindicates that the
variables are uncorrelated and that there is no linear
dependence between 𝑌1 and 𝑌2.
Unfortunately, it is difficult to employ the covariance as an
absolute measure of dependence because its value depends
upon the scale of measurement. As a result, it is difficult to
determine at first glance whether a particular covariance is
large or small. This problem can be eliminated by standardizing
its value and using the correlation coefficient, ρ, a quantity
related to the covariance and defined as

𝐶𝑂𝑉(𝑌1 ,𝑌2 )
𝜌= 𝜎1 𝜎2

where 𝜎1 and 𝜎2 are the standard deviations of 𝑌1 𝑎𝑛𝑑 𝑌2,

respectively. Supplemental discussions of the correlation
coefficient may be found in Hogg, Craig, and McKean (2005)
and Myers (2000).A proof that the correlation coefficient ρ
satisfies the inequality −1 ≤ 𝜌 ≤ 1
The sign of the correlation coefficient is the same as the sign of
the covariance.Thus, 𝜌 > 0 indicates that 𝑌2 increases as 𝑌1
increases, and 𝜌 = +1 implies perfect correlation, with all
points falling on a straight line with positive slope. A value of ρ
= 0 implies zero covariance and no correlation. A negative
coefficient of correlation implies a decrease in
𝑌2 𝑎𝑠 𝑌1 increases, and 𝜌 = −1 implies perfect correlation,
with all points falling on a straight line with negative slope. A
convenient computational formula for the covariance is
contained in the next theorem.

|Page5
Properties of covariance
1. 𝐶𝑜𝑣 (𝑎𝑋 + 𝑏, 𝑐𝑌 + 𝑑) = 𝑎𝑐𝐶𝑜𝑣 (𝑋, 𝑌) for constants
𝑎, 𝑏, 𝑐, 𝑑.
2. 𝐶𝑜𝑣(𝑋1 + 𝑋2, 𝑌) = 𝐶𝑜𝑣(𝑋1, 𝑌) + 𝐶𝑜𝑣 (𝑋2, 𝑌).
3. 𝐶𝑜𝑣(𝑋, 𝑋) = 𝑉𝑎𝑟(𝑋)
4. 𝐶𝑜𝑣(𝑋, 𝑌 ) = 𝐸(𝑋𝑌 ) − µ𝑋µ𝑌 .
5. 𝑉𝑎𝑟(𝑋 + 𝑌 ) = 𝑉𝑎𝑟(𝑋) + 𝑉𝑎𝑟(𝑌 ) +
2𝐶𝑜𝑣(𝑋, 𝑌 ) 𝑓𝑜𝑟 𝑎𝑛𝑦 𝑋 𝑎𝑛𝑑 𝑌 .
6. If 𝑋 and 𝑌 are independent then 𝐶𝑜𝑣(𝑋, 𝑌 ) = 0.
Warning: The converse is false: zero covariance does not
always imply independence.

Notes. 1. Property 4 is like the similar property for variance.

Indeed, if X = Y it is exactly that property: 𝑉𝑎𝑟(𝑋) = 𝐸(𝑋2) −
µ2𝑋. By Property 5, the formula in Property 6 reduces to the
earlier formula 𝑉𝑎𝑟(𝑋 + 𝑌 ) = 𝑉𝑎𝑟(𝑋) + 𝑉𝑎𝑟(𝑌 ) when
𝑋 and 𝑌 are independent. We give the proofs below.
However,understanding and using these properties is more
important than memorizing their proofs.

|Page6
Sums and integrals for computing covariance
Since covariance is defined as an expected value we compute it
in the usual way as a sum or integral.
Discrete case: If X and Y have joint pmf 𝑝(𝑥𝑖, 𝑦𝑗 ) then
𝑛 𝑚

𝐶𝑜𝑣(𝑋, 𝑌) = ∑ ∑ 𝑝 (𝑥𝑖, 𝑦𝑗) (𝑥𝑖 − µ𝑥) (𝑦𝑗 µ𝑌 )

𝑖=1 𝑗=1
𝑛 𝑚

= (∑ ∑ 𝑝(𝑥𝑖 𝑦𝑗 )𝑥𝑖 𝑦𝑗 ) _ µ 𝑥 µ𝑦 .
𝑖=1 𝑗=1

Continuous case: If 𝑋 and 𝑌 have joint pdf 𝑓(𝑥, 𝑦) over range

[𝑎, 𝑏] × [𝑐, 𝑑] then
𝑑 𝑏

𝐶𝑜𝑣(𝑋, 𝑌) = ∫ ∫(𝑥 − µ𝑥) (𝑦 − µ𝑦)𝑓(𝑥, 𝑦) 𝑑𝑥 𝑑𝑦

𝑐 𝑎
𝑑 𝑏

= (∫ ∫ 𝑥𝑦𝑓(𝑥, 𝑦)𝑑𝑥𝑑𝑦) − 𝜇𝑥 𝜇𝑦 .
𝑐 𝑎

THEOREM 1
If 𝑌1 𝑎𝑛𝑑 𝑌2 are random variables with means µ1 𝑎𝑛𝑑 µ2,
respectively, then
𝐶𝑜𝑣 (𝑌1, 𝑌2) = 𝐸 [(𝑌1 − µ1) (𝑌2 − µ2)]
= 𝐸(𝑌 1𝑌2) – 𝐸 (𝑌1)𝐸(𝑌2).

|Page7
Proof
𝐶𝑜𝑣 (𝑦1, 𝑦2) = 𝐸 [(𝑦1 − 𝑢2) (𝑦2 − 𝑢2)]
= 𝐸 (𝑦1 𝑦2 − 𝑢 1𝑦2 − 𝑢 2𝑦1 + 𝑢 1𝑢2).
the expected value of a sum is equal to the sum of the
expected values and , the expected value of a constant times a
function of random variables is the constant times the
expected value. Thus,
𝐶𝑜𝑣(𝑌1, 𝑌2) = 𝐸(𝑌 1𝑌2) − µ 1𝐸(𝑌2) −
µ 2𝐸(𝑌1) + µ 1µ2.
𝐵𝑒𝑐𝑎𝑢𝑠𝑒. 𝐸(𝑌1) = µ1 𝑎𝑛𝑑 𝐸(𝑌2)
= µ2, 𝑖𝑡 𝑓𝑜𝑙𝑙𝑜𝑤𝑠 𝑡ℎ𝑎𝑡
𝐶𝑜𝑣(𝑌1, 𝑌2) = 𝐸(𝑌 1𝑌2) − 𝐸(𝑌1)𝐸(𝑌2)
= 𝐸(𝑌 1𝑌2) − µ 1µ2.

EXAMPLE 1

Find the covariance between the amount in stock Y1 and

amount of sales Y2.
Solution Recall that Y1 and Y2 have joint density function given
by
3𝑦1 , 0 ≤ 𝑦2 ≤ 𝑦1 ≤ 1
𝑓 (𝑦1, 𝑦2) = { }
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒,
Thus,
1 𝑦 1 𝑦2
𝐸(𝑌 1𝑌2) = ∫0 ∫0 2 𝑦1 𝑦2 (3𝑦1 )𝑑𝑦2 𝑑𝑦1 = ∫0 3𝑦12 ( 22 ])
13 3 𝑦5 3
𝑑𝑦1=∫0 𝑦14 dy1 = ( 1 ]) = .
2 2 5 10

|Page8
we know that 𝐸(𝑌1) = 3/4 𝑎𝑛𝑑 𝐸(𝑌2) = 3/8. Thus,
𝐶𝑜𝑣(𝑌1, 𝑌2) = 𝐸(𝑌 1𝑌2) − 𝐸(𝑌1)𝐸(𝑌2)
= (3/10) − (3/4)(3/8) = .30 − .28 = .02.
In this example, large values of Y2 can occur only with large
values of Y1 and the density, 𝑓 (𝑦1, 𝑦2), is larger for larger
values of Y1 (see Figure 1). Thus, it is intuitive that the
covariance between Y1 and Y2 should be positive.

EXAMPLE 2
Let Y1 and Y2 have joint density given by
2𝑦1 0 ≤ 𝑦1 ≤ 1,0 ≤ 𝑦2 ≤ 1,
𝑓 (𝑦1, 𝑦2)={ }
0 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒.
Find the covariance of Y1 and Y2.
Solution
We have 𝐸(𝑌 1𝑌2) = 1/3. 𝐴𝑙𝑠𝑜, µ1 = 𝐸 (𝑌1) =
2/3 𝑎𝑛𝑑 µ2 = 𝐸 (𝑌2) = 1/2, 𝑠𝑜
𝐶𝑜𝑣 (𝑌1, 𝑌2) = 𝐸(𝑌 1𝑌2) − µ 1µ2
= (1/3) − (2/3) (1/2) = 0.

THEOREM 2
If Y1 and Y2 are independent random variables, then
𝐶𝑜𝑣 (𝑌1, 𝑌2) = 0.
Thus, independent random variables must be uncorrelated.

|Page9
Proof
Theorem 1 establishes that
𝐶𝑜𝑣 (𝑌1, 𝑌2) = 𝐸(𝑌 1𝑌2) − µ 1µ2.
Because Y1 and Y2 are independent, then
𝐸(𝑌 1𝑌2) = 𝐸(𝑌1)𝐸(𝑌2) = µ 1µ2,
and the desired result follows immediately.

Notice that the random variables Y1 and Y2 of Example 2 are

independent; hence, by Theorem 2, their covariance must be
zero. The converse of Theorem 2 is not true, as will be
illustrated in the following example.

EXAMPLE 3
Let 𝑌1 𝑎𝑛𝑑 𝑌2 be discrete random variables with joint
probability distribution as shown in Table . Show that Y1 and Y2
are dependent but have zero covariance.

Solution Calculation of marginal probabilities

𝑦𝑖𝑒𝑙𝑑𝑠 𝑝1 (−1) = 𝑝1(1) = 5/16 = 𝑝2(−1) =
𝑝2(1), 𝑎𝑛𝑑 𝑝1(0) = 6/16 = 𝑝2(0). The value 𝑝 (0, 0) = 0
in the center cell stands

| P a g e 10
Table : Joint probability distribution,
Y1
-1 0 1
Y2
-1 1/16 3/16 1/16
0 3/16 0 3/16
1 1/16 3/16 1/16

out. Obviously,
𝑝 (0, 0) ≠ 𝑝1(0)𝑝2(0),
and this is sufficient to show that Y1 and Y2 are dependent.
Again looking at the marginal probabilities, we see that
𝐸 (𝑌1) = 𝐸(𝑌2) = 0. 𝐴𝑙𝑠𝑜,

𝐸(𝑌 1𝑌2) = ∑ ∑ 𝑦1 𝑦2 𝑝 (𝑦1, 𝑦2)

𝑎𝑙𝑙𝑦1 𝑎𝑙𝑙𝑦2

= (−1)(−1)(1/16) + (−1)(0)(3/16)
+ (−1)(1)(1/16)
+ (0)(−1)(3/16) + (0)(0)(0)
+ (0)(1)(3/16)
+ (1)(−1)(1/16) + (1)(0)(3/16)
+ (1)(1)(1/16)
= (1/16) − (1/16) − (1/16) + (1/16)
= 0.

| P a g e 11
Thus,
𝐶𝑜𝑣 (𝑌1, 𝑌2) = 𝐸(𝑌1 𝑌2) − 𝐸(𝑌1)𝐸(𝑌2) = 0 −
0(0) = 0.
This example shows that the converse of Theorem 2 is not true.
If the covariance of two random variables is zero, the variables
need not be independent.

| P a g e 12
References
1.Mathematical Statistics with Applications٫ Dennis D. Wackerly
University of Florida .William Mendenhall III ٫University of
Florida, Emeritus Richard L. Scheaffer University of Florida,
Emeritus
2. Rice, John (2007). Mathematical Statistics and Data Analysis.
Belmont, CA: Brooks/Cole Cengage Learning. p. 138. ISBN 978-
0534-39942-9.
3.Papoulis (1991). Probability, Random Variables and Stochastic
Processes. McGraw-Hill.

| P a g e 13