Lecture 3

Random variables

CH2010 Engineering Statistics AY2023 v6 PL 1

Intended learning outcomes
• Concepts:
• Random variables
• Probability distribution
• Probability mass function
• Probability density function
• Joint probability mass/density function
• Skills
• Identify PMFs and PDFs
• Convert between cumulative probability distributions and PMFs/PDFs
• Calculate probability from PMFs and PDFs.

CH2010 Engineering Statistics AY2023 v6 PL 2

So far, we have been looking at…
Sample space for rolling 1 die: {1, 2, 3, 4, 5, 6}

Sample space for tossing 1 coin: {H, T}

Sample space for drawing 1 card from a deck of

52 cards: {the 52 cards within the deck}

CH2010 Engineering Statistics AY2023 v6 PL 3

3.1 Random variables
As engineers, we tend to characterise the entire sample space using a single
parameter, X, the random variable.
As such, the event can also be described algebraically.
• Drawing 1 card, X = 1 if A, X = 2 if 2,…, X = 13 if K.
• Queuing time at the noodle stall in Canteen A.
• Tossing a coin, X = 1 if H, X = 0 if T.
• If random variable only takes 0 and 1, it is a Bernoulli random variable

CH2010 Engineering Statistics AY2023 v6 PL 4

Discrete random variables

If the sample points are whole numbers or can be assigned to whole numbers, i.e. X
takes integers, then the sample space is a discrete sample space. X in this case is a
discrete random variable.
• Drawing 1 card from a complete deck of playing cards.

CH2010 Engineering Statistics AY2023 v6 PL 5

Continuous random variables

If the sample space contains an infinite number of possibilities, i.e. X takes any
number within a certain region, then the sample space is a continuous sample space.
X in this case is a continuous random variable.
• The instantaneous speed of a taxi on its way from Changi Airport to NTU.

CH2010 Engineering Statistics AY2023 v6 PL 6

3.2 Discrete probability distributions
Tossing a fair coin twice. Each head is 1 point, each tail is 0 point. X is the random
variable describing the total number of points, x. The probability of X taking the value of x
is f(x). The set of (x,f(x)) is the probability function, probability mass function or
probability distribution of the discrete random variable X.
1st toss 2nd toss Overall Probability Points x f(x)
H HH 0.25 2
2 0.25
H
T HT 0.25 1
1 0.50

T H TH 0.25 1
0 0.25
T TT 0.25 0

CH2010 Engineering Statistics AY2023 v6 PL 7

Upper case X vs. lower case x
In this course, the upper-case variables, e.g. X, Y, Z denote the names of the
random variables.
The lower-case variables, e.g. x, y, z denote specific values that the random
variables X, Y, Z take.
When we manipulate the algebra or derive expressions, we tend to use X to
represent a generic case without committing to any specific value.

If you find it confusing, you can use the two cases interchangeably.

CH2010 Engineering Statistics AY2023 v6 PL 8

Definition 3.1

CH2010 Engineering Statistics AY2023 v6 PL 9

Cumulative distribution function
What is the probability of discrete random variable X being a particular value x?
• We can answer this using the probability mass function, f(x).

What is the probability of X being smaller than or equal to a particular value x?

• We can answer this question using the cumulative probability distribution
function, F(x).

CH2010 Engineering Statistics AY2023 v6 PL 10

 Definition 3.2

1st toss 2nd toss Overall Probability Points x f(x) F(x)

1
H HH 0.25 2
2 1.0 0.8
0.25
H
0.6

Probability
T HT 0.25 1
1 0.50 0.75 0.4

0.2
T H TH 0.25 1
0 0.25 0.25 0
0 1 2
T TT 0.25 0 x
CH2010 Engineering Statistics AY2023 v6 PL 11
 Example 3.1
Toss a fair coin 4 times, each head is 1 point, each tail is 0 point. X is the total
number of points.
Size of sample space = 24 = 16
æ 4ö 1 1
çç ÷÷ = 1 f (0) = F (0) =
Possible ways of getting 0 points (0H4T)= è0ø 16 16
æ 4ö 4 5
Possible ways of getting 1 point (1H3T) = çç ÷÷ = 4
è1ø
f (1) =
16
F (1) =
16
æ 4ö 6 11
Possible ways of getting 2 points (2H2T) = çç ÷÷ = 6
è 2ø
f (2) =
16
F (2) =
16
æ 4ö 4 15
Possible ways of getting 3 points (3H1T) = çç ÷÷ = 4 f (3) = F (3) =
16
è 3ø 16
æ 4ö 1 16
Possible ways of getting 4 points (4H0T) = çç ÷÷ = 1 f (4) = F (4) =
è 4ø 16 16

CH2010 Engineering Statistics AY2023 v6 PL 12

 Example 3.1 continued

Probability mass function plot Discrete cumulative distribution function plot

CH2010 Engineering Statistics AY2023 v6 PL 13

3.3 Continuous probability distributions

• There are infinite number of sample points in a

continuous sample space.
• Therefore, the probability of the continuous random
variable, X being a particular value x is → 0.
• Continuous probability distribution cannot be given
in tabular form.
• Here, we are interested in the probability of X
meeting certain criteria.
P(a < X < b) or (X ≥ c);
also,
P(a < X ≤ b) = P(a < X < b) + P(X = b) = P(a < X < b).
CH2010 Engineering Statistics AY2023 v6 PL 14
Probability density function

The continuous probability distribution can be stated by a formula, f(x), the

probability density function or simply the density function, used as such:
b
P(a < X < b) = ò f ( x)dx
a

CH2010 Engineering Statistics AY2023 v6 PL 15

Definition 3.3

CH2010 Engineering Statistics AY2023 v6 PL 16

Example 3.2

• A temperature controller is installed on a chemical reactor.

• The error in the temperature controller, can be described by a random variable
X with probability density function:

𝑥!
𝑓 𝑥 = $ 3 , −1 < 𝑥 < 2
0, elsewhere
(a) Verify that f(x) is indeed a probability density function.
(b) Find P(0 < x ≤ 1),

CH2010 Engineering Statistics AY2023 v6 PL 17

Example 3.2 continued

(a) Use Definition 3.3

Condition 1: obviously, f(x) ≥ 0.
Condition 2:
2 3 2
¥ 2 x x 8 1
ò
-¥
f ( x)dx = ò
-1 3
dx =
9
= + =1
9 9
-1

(b) Using formula 3 in Definition 3.3, we obtain

2 3 1
1 x x 1
P(0 < X £ 1) = ò dx = =
0 3 9 0
9
CH2010 Engineering Statistics AY2023 v6 PL 18
 Definition 3.4

It immediately follow that:

P(a < X < b) = F (b) - F (a )

dF ( x)
f ( x) = , if the derivative exists
dx
CH2010 Engineering Statistics AY2023 v6 PL 19
Shapes of density functions

CH2010 Engineering Statistics AY2023 v6 PL 20

Summary
• Random variable, X.
• Probability distribution of discrete/continuous random variables can be described using
probability mass functions (PMFs)/density functions (PDFs).
• Represent discrete probability mass function in table form or histograms.
• Represent continuous probability density function using an algebraic function or a PDF
plot.
• Probability mass functions give probabilities.
• Integral of probability density functions give probabilities
• Integrating PDFs from the smallest value of x gives cumulative probability function,
F(x) from PDFs.

CH2010 Engineering Statistics AY2023 v6 PL 21

3.4 Joint probability distributions
• So far, we have been looking at the probability (viz. probability distribution and
probability density of a single event, described by random variable X.
• There are many practical examples where one needs to know the probability of complex
events.
• If the complex events can be described by a combination of simple events, the outcome of
each is described by random variables X, Y, and so on, then the probability distribution
is described by the joint probability distribution:

f ( x, y, ) = P( X = x, Y = y, )

CH2010 Engineering Statistics AY2023 v6 PL 22

Example 3.4
• Roll two dice simultaneously. The outcome of the first die is X, the outcome of
the second die is Y.
• The sample space of (X, Y) can be tabulated.
• By inspection, f(x, y) = 1/36 for each combination of (x, y)

X\Y 1 2 3 4 5 6
1 (1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
2 (2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
3 (3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
4 (4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
5 (5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
6 (6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

CH2010 Engineering Statistics AY2023 v6 PL 23

Definition 3.5

CH2010 Engineering Statistics AY2023 v6 PL 24

Example 3.5
• A box contains 3 blue pens, 2 red pens and 3 green pens.
• 2 pens are selected at random.
• X = No. of blue pens selected
• Y = No. of red pens selected
(a) Find the joint probability function f(x,y)
(b) P( x + y ≤ 1)

Note the difference between f and P for discrete random variables

f is the probability of any particular combination of random variables
P can be the probability of all combinations that meets certain criteria, such as in
Example 3.5
CH2010 Engineering Statistics AY2023 v6 PL 25
Example 3.5 continued
Answer: the possible values of (x, y) are:
(0, 0), (0, 1), (1, 0), (1, 1), (0, 2) and (2, 0).

2 green pens 1 blue pen, 1 green pen 2 blue pens

What is f(0, 0)?

It is the probability of choosing 2 green pens out of total of 8 pens.
The probability can be calculated by counting sample space.
First, total sample space size = number of ways to choose any 2 pens from 8 pens
æ8ö
çç ÷÷ = 28
è 2ø
CH2010 Engineering Statistics AY2023 v6 PL 26
Example 3.5 continued
Second, how many ways is there to choose 2 green pens out of a total 3?
æ 3ö
çç ÷÷ = 3
è 2ø
Therefore,
3
f (0,0) =
28
Similarly, how many ways is there to choose 1 green pen out of a total 3 and 1 red
pen out of a total of 2?
æ 3 öæ 2 ö
çç ÷÷çç ÷÷ = 6
è 1 øè 1 ø
Therefore,
6
f (0,1) =
28
CH2010 Engineering Statistics AY2023 v6 PL 27
Example 3.5 continued
Following the argument, it would become clear that the joint probability
distribution function can be expressed:
Number of ways to Number of ways to Number of ways to
choose x blue pens choose y red pens from 2 choose 2 – x – y green
from 3 pens from 3

æ 3 öæ 2 öæ 3 ö
çç ÷÷çç ÷÷çç ÷÷
è x øè y øè 2 - x - y ø
f ( x, y ) =
æ8ö
çç ÷÷
è 2ø

Number of ways to choose any 2 pens from total of 8

CH2010 Engineering Statistics AY2023 v6 PL 28

Example 3.5 continued
Calculating the probability for all possible combinations of (x,y), we can tabulate
the joint probability distribution:

(b) Identify the region where x + y ≤ 1,

we get:
3 3 9 9
P( x + y £ 1) = + + =
28 14 28 14
CH2010 Engineering Statistics AY2023 v6 PL 29
Joint density function

The joint probability distribution tabulated in Example 3.5 may be plotted

on the x-y plane as:

f (x, y)

2
x
0 1 2
Source: Math StackExchange

Thus, it is foreseeable that for 2 continuous random variables, the joint probability
distribution, i.e. joint density function, will look like a curved surface lying above
the x-y plane, where the height is the joint probability density.
CH2010 Engineering Statistics AY2023 v6 PL 30
Definition 3.6

CH2010 Engineering Statistics AY2023 v6 PL 31

Example 3.6
A private owned business operates a drive-in facility with usage X and a walk-in facility
with usage Y on any given day.
Usage = proportional of time the facility is in use.
The joint density function of random variables X and Y is given by:
ì2
ï (2 x + 3 y ), 0 £ x £ 1,0 £ y £ 1
f ( x, y ) = í 5
ïî 0, elsewhere

(a) Verify condition 2 of Definition 3.6

(b) Find
é 1 1 1ù
P ê0 < x < , < y < ú
ë 2 4 2û
CH2010 Engineering Statistics AY2023 v6 PL 32
Example 3.6 continued
Answer:
(a) From Definition 3.6, we integrate f(x, y) over the whole region:

¥ ¥ 1 1 2
=ò
ò ò 0 ò0 5
f ( x, y )dxdy (2 x + 3 y )dxdy
-¥ -¥
x =1
1 é 2x 6 xy ù
2
=ò ê + ú dy
0
ë 5 5 û x =0
1
æ 2 6y ö
1 é 2 y 3y ù 2
=ò ç + ÷dy = ê + ú
0 5 5 ø
è ë 5 5 û0
2 3
= + =1
5 5
CH2010 Engineering Statistics AY2023 v6 PL 33
Example 3.6 continued
Answer:
(b) To calculate the probability, we use
é 1 1 1ù 1/ 2 1/ 2 2
P ê0 < x < , < y < ú = ò ò (2 x + 3 y )dxdy
ë 2 4 2û 1 / 4 0 5 "
" $% "
! 2𝑥 ! 6𝑥𝑦 ! ! 1 3𝑦
2 + 𝑑𝑦 = 2 + 𝑑𝑦
" 5 5 $%& " 10 5
# #
1/ 2
é y 3y2 ù
=ê + ú
ë10 10 û1 / 4
1 éæ 1 3 ö æ 1 3 öù 13
= êç - ÷ - ç + ÷ú =
10 ëè 2 4 ø è 4 16 øû 160
CH2010 Engineering Statistics AY2023 v6 PL 34
3.5 Marginal distribution
Integrating a joint probability distribution function or
joint probability density function f(x,y) with respect to Y
h(y) g(x) or X gives us g(x) or h(y), respectively.
g(x) and h(y) are the marginal distributions of X and Y,
respectively.
f(x, y)
Marginal distributions can be thought of similar to (but
not exactly) 2-D projections of the 3-D joint PDF on the
x-z plane or the y-z plane, where the z-direction
represents the value of f(x, y).

CH2010 Engineering Statistics AY2023 v6 PL 35

Definition 3.7

CH2010 Engineering Statistics AY2023 v6 PL 36

Example 3.7
• For Example 3.5 give the marginal distribution of X and Y.

h(0)
h(1)
h(2)

g(0) g(1) g(2)

CH2010 Engineering Statistics AY2023 v6 PL 37

Example 3.8
Find g(x) and h(y) for the joint density function in Example 3.6.
ì2
ï (2 x + 3 y ), 0 £ x £ 1,0 £ y £ 1
f ( x, y ) = í 5
ïî 0, elsewhere
Solution:
y =1
¥ 1 2 é 4 xy 6 y 2 ù 4x + 3
g ( x) = ò f ( x, y )dy = ò (2 x + 3 y )dy = ê + ú =
-¥ 0 5 5 10 5
ë û y =0
for 0 ≤ x ≤ 1, and g(x) = 0 elsewhere
¥ 1 2 2(1 + 3 y )
h( y ) = ò f ( x, y )dx = ò (2 x + 3 y )dx =
-¥ 0 5 5
for 0 ≤ y ≤ 1, and h(y) = 0 elsewhere

CH2010 Engineering Statistics AY2023 v6 PL 38

3.6 Conditional probability distribution
From Lecture 2, we recall that P( A Ç B)
P( B | A) =
P( A)
If A corresponds to the event that X = x; B corresponds to the event that Y
= y, then
P ( X = x, Y = y ) f ( x, y )
P(Y = y | X = x) = =
P( X = x) g ( x)

The resulting expression, f(x, y)/g(x) is a function of y at a given x and is a

PDF itself, which is called a conditional probability distribution.

CH2010 Engineering Statistics AY2023 v6 PL 39

Definition 3.8

Definition 3.8 can be extended to describe probability that the value of X falls
between a and b given Y = y for discrete and continuous random variables.
P(a < X < b | Y = y ) = å f ( x | y)
a < x <b

CH2010 Engineering Statistics AY2023 v6 PL 40

Example 3.9
Referring to Example 3.5, find the conditional distribution of X, given that Y = 1, and use it to
determine P(X = 0|Y = 1).
Solution: we need to find f(x|1). First, we find that
2
3 3 3
h(1) = å f ( x,1) = + +0=
x =0 14 14 7

Now f ( x,1) æ 7 ö
f ( x | 1) = = ç ÷ f ( x,1), x = 0,1,2.
h(1) è3ø
Therefore,
æ7ö æ 7 öæ 3 ö 1 æ7ö æ 7 öæ 3 ö 1
f (0 | 1) = ç ÷ f (0,1) = ç ÷ç ÷ = f (1 | 1) = ç ÷ f (1,1) = ç ÷ç ÷ =
è3ø è 3 øè 14 ø 2 è3ø è 3 øè 14 ø 2
æ7ö æ7ö
f (2 | 1) = ç ÷ f (2,1) = ç ÷(0 ) = 0
è3ø è3ø
CH2010 Engineering Statistics AY2023 v6 PL 41
Example 3.9 (continued)
Therefore, the conditional distribution of X, given Y = 1 is

x 0 1 2
f(x|1) ½ ½ 0

Finally,
1
P( X = 0 | Y = 1) = f (0 | 1) =
2

Recall from Example 3.5 that X = no. of blue pens, Y = no. of red pens.
X = 0 → no blue pen, Y = 1 → one red pen.
Therefore, we know that if 1 and only 1 of the 2 pens selected is red, there is a 0.5
probability that the other one is green. There is also 0.5 probability that the other
one is blue.
CH2010 Engineering Statistics AY2023 v6 PL 42
Statistical independence
Re-writing the second point in Definition 3.8 in the form of product rule, we have:

f ( x, y ) = f ( x | y ) h ( y )
Substituting this into Definition 3.7 (marginal distribution), we have:
¥ ¥
g ( x) = ò f ( x, y )dy = ò f ( x | y )h( y )dy
-¥ -¥
If f(x|y) does not depend on y,
¥
g ( x) = f ( x | y ) ò h( y )dy
-¥
By definition of marginal distribution:
¥
ò
-¥
h( y )dy = 1

CH2010 Engineering Statistics AY2023 v6 PL 43

Definition 3.12
Therefore:
g ( x) = f ( x | y )
Substituting back into the first line:
f ( x, y ) = g ( x ) h ( y )

CH2010 Engineering Statistics AY2023 v6 PL 44

Definition 3.13
Definition 3.12 can be extended to n random variables.

CH2010 Engineering Statistics AY2023 v6 PL 45

Summary
• Joint probability distribution
• Marginal probability distribution
• Compute marginal probability distribution from joint probability distribution
• Conditional probability distribution
• Compute conditional probability distribution from joint probability distribution
• Determine the statistical dependence/independence of random variables in a joint
probability distribution.

CH2010 Engineering Statistics AY2023 v6 PL 46