273 - Lecture 8

Sariel Har-Peled
February 18, 2004

1 A question
You are given a fair coin. I give you a probability p, and you should output “1” with
probability p, and “0” with probability 1 − p. How to do that?

2 Linearity of expectation
Lemma 2.1 For a constant a, we have E[aX] = a E[X].
X X
Proof: E[aX] = aX(s) Pr[{s}] = a X(s) Pr[{s}] = a E[X].
s∈S s∈S

Lemma 2.2 For any two random variables X and Y , we have E[X + Y ] = E[X] + E[Y ].
X X X
Proof: E[X + Y ] = (X(s) + Y (s)) Pr[{s}] = X(s) Pr[{s}] + Y (s) Pr[{s}] =
s∈S s∈S s∈S
E[X] + E[Y ].
Note, that the two random variables need not be independent.

Theorem 2.3 (Linearity of Expectation.) For any a1 , . . . , an real numbers, and X1 , . . . , Xn

random variables, we have

E[a1 X1 + a2 X2 + · · · + an Xn ] = a1 E[X1 ] + a2 E[X2 ] + · · · + an E[Xn ] .

The proof is by induction, and is left as exercise.

Example 2.4 Roll two dice. What is the expected total value?
Sample space elements (i, j) - results of both dice throws where i, j ∈ {1, . . . , 6}.
X1 ((i, j)) = i - first throw result.
X2 ((i, j)) = j - second throw result.
Y = X1 + X2 .
We are interested in E[Y ]. By linearity of expectations, we have

E[Y ] = E[X1 + X2 ] = E[X1 ] + E[X2 ] = 3.5 + 3.5 = 7.

1
2.1 Indicator Variables
Definition 2.5 (Indicator Variable) Let E ⊆ S be an event in the probability space.
The indicator variable of E is the mapping IE : S → {0, 1}, such that: for s ∈ S, we have
(
1 s∈E
IE (s) =
0 s∈/ E.
Observation 2.6 E[IE ] = 0 · Pr[s ∈
/ E] + 1 · Pr[s ∈ E] = Pr[s ∈ E].
Together with linearity of expectation, we now have a powerful tool in our hands.

Example 2.7 We toss a coin 10 times. What is the expected number of times we get heads?
Let Xi be an indicator variable which is one if the ith coin cam up heads. Formally:
(
0 ith coin is T
Xi =
1 ith coin is H.
Let Y be the random variableP which is thePnumber of times wet got heads. Clearly, Y =
X1 + · · · + X10 and E[Y ] = E 10
i=1 Xi =
10
i=1 E[Xi ] (by linearity of expectations). But

1
E[Xi ] = Pr[Xi = 1] = .
2
Thus, E[Y ] = 10/2 = 5.

Example 2.8 220 sailors, we order them in a random order, and sent them to the cabins
in this order. Of course, every sailor was allocated a specific cabin. What is the expected
number of sailors that stay in their allocated cabin?
Namely, here |S| = 220!.
Here is a direct computation: Let X be the random variable which is the number of
sailors in the right cabins. Clearly
220
X
E[X] = i · Pr[X = i] .
i=0

However, it is completely unclear how compute Pr[X = i], and in fact seems hard.
Instead, let define the indicator random variable Xi , which is 1 if the ith sailor got to
the right cabin, and zero otherwise. Clearly,
" 220 # 220
X X
E[X] = E Xi = E[Xi ] .
i=1 i=1

However,
1
E[Xi ] = 1 · Pr[ith sailor in right cabin] + 0 · Pr[whatever] = ,
220
by symmetry. Thus,
220 220
X X 1
E[X] = E[Xi ] = = 1.
i=1 i=1
220

2
Example 2.9 Same question as above, but here every sailor randomly pick its cabin. Here
|S| = 220220 . Using similar argumentations, we have:
1
E[Xi ] = 1 · Pr[ith sailor in right cabin] + 0 · Pr[whatever] = ,
220
by symmetry. Thus,
220 220
X X 1
E[X] = E[Xi ] = = 1.
i=1 i=1
220
Note, that unlike the previous case, here we can compute this quantity directly, and you
should try and do it directly.

In this case, what is the expected number of sailors sleeping alone in its own cabin? In
this case, let Yi be a variable which is one if the ith sailor sleeps alone in its own cabin, and
zero otherwise. Clearly,

219219
E[Yi ] = Pr[ith sailor sleeps alone in her own cabin] = .
220220
Thus, the total expected number of such sailors is
" 220 # 219 219
219219
 
X 219 1 220 1
E Yi = 220 220
= == 1 − ≈ · = 0.36 · · ·
i=1
220 220 220 219 e

(Here I used the [imprecise] estimate (1 − 1/n)n ≈ 1/e.)

Example 2.10 What is the number of sailors sleeping alone? Not necessarily in their right
cabin?
Well, let Xi be an indicator variable if the ith sailor sleeps alone. We have:

220 · 219219
E[Xi ] = Pr[Xi = 1] = .
220220
Thus, let Y be the number sailors sleeping alone, we have:
220
X 220 · 219219
E[Y ] = E[Xi ] = 220 · = 81.11.
i=1
220220

2.2 Variance
Definition 2.11 (Variance) For a random variable X, its variance is
X
V[X] = (X(s) − E[X])2 Pr[{s}] .
s∈S
p
Its standard deviation is V[X]

3
3 Markov’s Inequality
Theorem 3.1 [Markov Inequality] For a non-negative variable X, and t > 0, we have:
h i
EX
Pr[X ≥ t] ≤ .
t
E[X]
Proof: Assume that this is false, and there exists t0 > 0 such that Pr[X ≥ t0 ] > t0
.
However,
X
E[X] = x · Pr[X = x]
x
X X
= x · Pr[X = x] + x · Pr[X = x]
x<t0 x≥t0
≥ 0 + t0 · Pr[X ≥ t0 ]
E[X]
> 0 + t0 · = E[X],
t0
a contradiction.

Example 3.2 As before, 220 sailors, every one chooses a cabin at random. What is the
probability that 3 sailors would sleep in their own cabins?
Well, let X be the number of sailors in their own cabins. We know that E[X] = 1. Thus,

E[X] 1
Pr[X ≥ 3] ≤ = ,
3 3
by the Markov inequality. Thus, we know something about this probability without working
too hard.