Discrete random variables

2
Example
A coin is thrown twice. The sample space is S = {(H, H); (H, T ); (T, H); (T, T )}
We may want to consider how many times it was a Tail. Let’s assign a number X to each
event. Then the number X will be the number of Tails.
Complete the following:

8
>
> (H, H) 7! . . .
>
<(H, T ) 7! . . .
X:
>
> (T, H) 7! . . .
>
:
(T, T ) 7! . . .

Let Y be 1 if both coins match and 0 otherwise.

Complete the following:

8
>
> (H, H) 7! . . .
>
<(H, T ) 7! . . .
Y :
>
> (T, H) 7! . . .
>
:
(T, T ) 7! . . .

X can take the values 0,1 or 2 and Y the values 0 or 1. We use the notation [X = 2] for
the event that Tails appears twice. Similarly, [X > 1] is the event that Tails appears at
least once, and [X 2 {0, 2}] is the event that there are an even number of Tails.

Reminder: an event is a collection of outcomes satisfying a given condition.

13
2.1. DEFINITIONS CHAPTER 2. DISCRETE RV

2.1 Definitions

Definition 13 (Random variable)

A random variable over a given sample space is a function that maps every outcome
to a real number.

X:S !R
where the domain S is the sample space and R is the codomain of the function, the
set of real numbers that may possibly come out.
The set of values taken on by X, i.e. the set of values that actually come out is called
the range of the function and is denoted by ⌦X . In other words,

⌦X = {X(x) | x 2 S}

Remarks

Random variable is a misnomer: it is actually a function.

Random variable (functions) are often called X (upper case X) so the values it
takes would be X(x). So a random variable is neither a variable nor random (it is
a function about random events) but it uses capital X which must not be confused
with a variable x.

In the previous examples, ⌦X = {0; 1; 2} and ⌦Y = {0; 1}.

Definition 14 (Discrete random variables)

A random variable that takes on finite or countably infinite number of values is called a
discrete random variable.

Let S be the sample space associated with a random event and X be a discrete random
variable. If for each value of X we associate the corresponding probability, then we obtain
the probability density function of the random variable X.

Definition 15 (Probability density function)

Let X be a discrete random variable. Then its probability density function fX (x) is
given by (
P (X = x), x 2 ⌦X
fX (x) =
0, otherwise

A. Lemmon 14 Ma1
CHAPTER 2. DISCRETE RV 2.1. DEFINITIONS

Example
Let’s take our first example, where the number X represents the number of Tails. We
have:
S = {(H, H); (H, T ); (T, H); (T, T )}: it’s the domain of X;
⌦X = {0; 1; 2}: it’s the range of X, a subset of R.
1
P (X = 0) =
4
1
P (X = 1) =
2
1
P (X = 2) =
4

Remark
We will call pi the probability P (X = xi ), for each xi 2 ⌦X . Then, if ⌦X has n elements,
we have
X n
pi = 1
i=1

We can display this information in a table called the probability distribution table:

3
X
xi 0 1 2
i=1

1 2 1
pi 1
4 4 4

1 3
P (X 6 xi ) 1
4 4

The graph of a probability density function can be represented by a bar graph. On the
horizontal axis are the values of xi 2 ⌦X and on the vertical axis are the corresponding
probabilities (0 6 pi 6 1).

1 pi
0.75
0.5
0.25

0 1 2 xi

A. Lemmon 15 Ma1
2.2. EXPECTED VALUE CHAPTER 2. DISCRETE RV

2.2 Expected value

Intuitively, the expected value informs about what to expect in an experiment “in the
long run ”, after many trials. In most cases, there could be such value in the sample
space. Concretely, the expected value of a discrete random variable is the average of
all possible values of a random variable, where each value is weighted according to the
probability that it will appear.

Definition 16 (Expected value)

Let S be the sample space and X a discrete random variable, where ⌦X = {x1 , x2 , . . . , xn }.
Then the expected value E(X) is given by
n
X
E(X) = xi · pi
i=1

The expected value is sometimes also called the expectation, the average or the mean
value.

Examples
Let’s consider two dice, a fair one and a loaded one. Each dice is thrown once. Let X
and Y be the random variables indicating the number shown by the fair and loaded dice
respectively. Calculate their respective expectations knowing that we have the following
distributions:

(1) Fair dice

6
X
xi 1 2 3 4 5 6
i=1

1 1 1 1 1 1
pi 1
6 6 6 6 6 6

1 2 3 4 5 6 21
xi · pi = 3.5
6 6 6 6 6 6 6

1 1 1 1 1 1 21
E(X) = 1 · +2· +3· +4· +5· +6· = = 3.5
6 6 6 6 6 6 6

(2) Loaded dice

A. Lemmon 16 Ma1
CHAPTER 2. DISCRETE RV 2.2. EXPECTED VALUE

6
X
yi 1 2 3 4 5 6
i=i

2 2 3 3 4 4
pi 1
18 18 18 18 18 18

2 4 9 12 20 24 71
yi · pi ⇡ 3.94
18 18 18 18 18 18 18

1 1 1 1 2 2 71
E(Y ) = 1 · +2· +3· +4· +5· +6· = ⇡ 3.94
9 9 6 6 9 9 18

Remark
If all the probabilities are equal, then the expected value
x1 + x2 + · · · + xn
E(x) =
n
is called the arithmetic mean, or simply the mean of x1 , x2 , . . . , xn . Such distribution is
called a uniform distribution.
Properties

(1) E(X + k) = E(X) + k, k 2 R

(2) E(k · X) = k · E(X), k 2 R

Remarks

(1) The last two properties equivalent to E(a · X + b) = a · E(X) + b

(2) Usually, the expected value doesn’t represent the most likely outcome, it’s the long-
run average value of repetitions of the experiment it represents.
(3) In some game, X can represent the player’s earnings, or benefits. In this case, we
will say that the game is:
profitable if E(X) > 0 (it’s worth playing the game, you would expect to win
money)
fair if E(X) = 0 (you break even)
unprofitable if E(X) < 0 (it’s not worth playing the game, you would expect
to lose money)
(4) In the case of discrete random variables, E(X) may not be a value of X.

A. Lemmon 17 Ma1
2.3. VARIANCE CHAPTER 2. DISCRETE RV

2.3 Variance and standard deviation

The expected value of a random variable gives no information about how the distribution
is spread out. This information is given by the deviation, also known as the dispersion.
Let’s take a look at the following example to understand what is meant by that.

Examples

6
X
xi 1 2 3 4 5 6
i=1

1 1 1 1 1 1
pi 1
6 6 6 6 6 6

1 2 3 4 5 6 21
xi · pi = 3.5
6 6 6 6 6 6 6

6
X
y1 1 2 3 4 5 6
i=1

5 4 1 1 4 5
pi 1
20 20 20 20 20 20

5 8 3 4 20 30 70
yi · pi = 3.5
20 20 20 20 20 20 20

A. Lemmon 18 Ma1
CHAPTER 2. DISCRETE RV 2.3. VARIANCE

6
X
zi 1 2 3 4 5 6
i=1

1 4 5 5 4 1
pi 1
20 20 20 20 20 20

1 8 15 20 20 6 70
z i · pi = 3.5
20 20 20 20 20 20 20

Note that all three random variables have the same expected value, but you can see that
the distributions are quite di↵erent. Variable Z is the one most centered around the
expected value. Indeed:

1 1 2
P (3 6 X 6 4) = P (X = 3) + P (X = 4) = + = = 33.3%
| {z } | {z } 6 6 6
=p3 =p4

1 1 2
P (3 6 Y 6 4) = P (Y = 3) + P (Y = 4) = + = = 10%
| {z } | {z } 20 20 20
=p3 =p4

5 5 10
P (3 6 Z 6 4) = P (Z = 3) + P (Z = 4) = + = = 50%
| {z } | {z } 20 20 20
=p3 =p4

Intuitively, the mean (absolute) deviation is the expected value of the di↵erence between
the variable’s mean (µ = E(X)) and the variable’s realization, that is, the average of the
di↵erence between the variable’s mean and each possible value, weighted according to the
probability that the value will occur. This translates to

n
X
|xi µ| · pi
i=1

Concretely, this formula is not practical. This is why people introduced the variance,
which is the expected value of the squared di↵erence between the variable’s realization
and the variable’s mean, and managed to get the standard deviation by taking the
square root of the variance. Mean deviation and standard deviation do not give exactly
the same result, but are close. The standard deviation is more a↵ected by larger values
and is more commonly used, thanks to the variance theorem. Larger variance reflects a
greater spread in the probability distribution.

A. Lemmon 19 Ma1
2.3. VARIANCE CHAPTER 2. DISCRETE RV

Definition 17 (Variance)
Let X be a discrete random variable, where ⌦X = {x1 , x2 , . . . , xn }, and let µ = E(X).
Then the variance V (X) is given by
n
X
V (X) = (xi µ)2 · pi = E[(X µ)2 ]
i=1

That is, the variance is the expected value of the squared di↵erence between the vari-
able’s realization and the variable’s mean (expected value).

Definition 18 (Standard deviation)

The standard deviation of a discrete random variable X, denoted (X), is given by
p
(X) = V (X)

Example
We roll a dice once. Let X be the random variable indicating the number obtained, as
1
summarized in the previous example. Then P (X = xi ) = and µ = 3.5. Applying the
6
definition of the variance, we obtain
1 1 1
V (X) = (1 3.5)2 · + (2 3.5)2 · + (3 3.5)2 ·
6 6 6
1 1 1
+ (4 3.5)2 · + (5 3.5)2 · + (6 3.5)2 ·
6 6 6
1
= [( 2.5)2 + ( 1.5)2 + ( 0.5)2 + 0.52 + 1.52 + 2.52 ] ·
6
35 1
= ·
2 6
35
=
12
Hence, the standard deviation is
r
35 ⇠
(X) = = 1.7
12
The variance doesn’t have the same unit as the random variable X (because of the square),
but the standard deviation does. Here, the standard deviation tells us that when we roll
a dice, we are, in some sort of average, 1.7 away from the expected value being 3.5.
(Remember that the variance is more a↵ected by larger values, this is why we don’t get
1.5 as expected by the mean deviation).

Theorem 12 (Variance)
Let X be a discrete random variable, where ⌦X = {x1 , x2 , . . . , xn }, and let µ = E(X).
Then the variance
n
X
V (X) = x2i · pi µ2 = E(X 2 ) [E(X)]2
i=1

A. Lemmon 20 Ma1
CHAPTER 2. DISCRETE RV 2.3. VARIANCE

Example
Let’s calculate the variance and the standard deviation for the random variable X given
above:

6
X
xi 1 2 3 4 5 6
i=1

1 1 1 1 1 1
pi 1
6 6 6 6 6 6

1 2 3 4 5 6 21 7
xi · pi =
6 6 6 6 6 6 6 2

1 1 1 1 1 1 91
x2i · pi 12 · 22 · 32 · 42 · 52 · 62 ·
6 6 6 6 6 6 6

✓ ◆2
2 91 2 7 35
V (X) = E(X ) [E(X)] = =
6 2 12
and r
35 ⇠
(X) = = 1.7
12

Example
Calculate the variance and the standard deviation for the random variable Z given above.

Remark
The unit of the standard deviation is equal to the unit of its random variable, whereas
the unit of the variance is the squared of the the unit of its random variable. Hence the
standard deviation is more meaningful.

Properties

(1) V (X + k) = V (X), 8k 2 R
(2) V (k · X) = k 2 · V (X), 8k 2 R

Remark
The last two properties are equivalent to V (a · X + b) = a2 · V (X)

A. Lemmon 21 Ma1
2.4. BINOMIAL DISTRIBUTION CHAPTER 2. DISCRETE RV

2.4 Binomial distribution

This section introduces the binomial distribution, a discrete distribution, which has
found wide applications in many scientific areas. It is applicable to experiments that
employ repeated binary trials.

Example
A coin is tossed ten times. What is the probability of obtaining exactly seven tails?

Definition 19 (Binomial distribution)

A random variable X is said to follow a binomial distribution if
✓ ◆
n
P (X = k) = B(k; n; p) = · pk · q n k
k

where

n = number of trials

k = number of successes out of n trials (0 6 k 6 n)

p = probability of getting a success (S) per trial

q = 1 p = probability of getting a failure (F) per trial

✓ ◆
n n!
= Ckn =
k k!(n k)!

Remarks

n ✓ ◆
X n
(1) · pk · q n k
= (p + q)n = 1n = 1 since q = 1 p
k
k=0

(2) When an experiment is repeated n times, we are in the context of ✓ permutations,

◆
n
that is, the order of each outcome matters. The binomial coefficient gives the
k
number of outcomes satisfying the given condition.

A. Lemmon 22 Ma1
CHAPTER 2. DISCRETE RV 2.4. BINOMIAL DISTRIBUTION

Expected value, variance and standard deviation

Example
Let X be the random variable counting how many times we obtain 6 dots when rolling a
fair dice 6 times. How many time would we expect to obtain 6 dots?
Applying the definition of the expected value, we have:
✓ ◆ ✓ ◆0 ✓ ◆ 6
6 1 5
P (X = 0) = · ·
0 6 6
✓ ◆ ✓ ◆1 ✓ ◆ 5
6 1 5
P (X = 1) = · ·
1 6 6
✓ ◆ ✓ ◆2 ✓ ◆ 4
6 1 1
P (X = 2) = · ·
2 6 6
✓ ◆ ✓ ◆3 ✓ ◆ 3
6 1 5
P (X = 3) = · ·
3 6 6
✓ ◆ ✓ ◆4 ✓ ◆ 2
6 1 5
P (X = 4) = · ·
4 6 6
✓ ◆ ✓ ◆5 ✓ ◆ 1
6 1 5
P (X = 5) = · ·
5 6 6
✓ ◆ ✓ ◆6 ✓ ◆ 0
6 1 5
P (X = 6) = · ·
6 6 6

E(X) = 1 (surprised?)

Theorem 13 (binomial distribution)

Let X be a random variable following a binomial distribution. Then

(1) E(X) = np

(2) V (X) = npq = np(1 p)

p p
(3) (X) = npq = np(1 p)

Using the formula with our previous example, we get:

r
1 1 5 5 5⇠
E(X) = 6 · = 1 V (X) = 6 · · = and (X) = = 0.91
6 6 6 6 6

A. Lemmon 23 Ma1
2.4. BINOMIAL DISTRIBUTION CHAPTER 2. DISCRETE RV

A. Lemmon 24 Ma1