Chapter Four

Probability

1
Why study Probability?
• The objective of this course is to study inferential
statistics.
• In inferential statistics we use sample statistics to
reach a conclusion about the population parameters.
• Sample statistics are not constants they are random
variables (uncertain).
• Probability is used to understand the behaviour of a
random variable .
• To introduce probability, we first need to define
some important terms.

2
 Random Experiment
• A random experiment is an action or process
that leads to one of several possible
outcomes. For example:
Experiment Outcomes
Flip a coin Heads, Tails
Record exam marks Numbers: 0, 1, 2, ..., 100
Measure assembly time t > 0 seconds
Record course grades A, B, C, D, and F
Record a voter’s choice Candidate A, candidate B,…
3
 Sample Space
• The first step in assigning probabilities is to produce a
list of the outcomes.
• A sample space of a random experiment is a list of all
possible outcomes (simple events)of the experiment.
• A sample space is usually denoted by S, while the
outcomes are denoted by O1, O2, …, Ok.
• Using notation from set theory, we can represent the
sample space and its outcomes as:

S = {O1, O2, …, Ok}

4
Examples for a Sample Space
1- If we drop a coin once, what is the sample space?
S={H,T}
2- If we drop a coin twice, what is the sample space?
S={(H,H), (H,T), (T,H), (T,T)}
3- If we roll a die once, what is the sample space?
S={1, 2, 3, 4, 5, 6}
• Methods for describing a sample space
– Listing (such as in the above two examples)
– tree diagram
– Venn diagram

5
Sample Space Example: Tree Diagram for
flipping a coin 2 times
H {H,H}

T {H,T}

{T,H}
H

T {T,T}

6
 Event
An event is a collection or set of one or more
simple events…subset of S.
e.g.: Getting an odd number when rolling a
die. B={1,3,5}

7
 Probability of an Event
• The probability of an event is the sum of the
probabilities of the simple events that constitute the
event.

• E.g. (assuming a fair die) S = {1, 2, 3, 4, 5, 6} and

P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6

Then:
P(EVEN) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 3/6 = 1/2

8
 Requirements of Probabilities
• Given a sample space S = {O1, O2, …, Ok}, the
probabilities assigned to the outcome must satisfy these
requirements:

(1) The probability of any outcome is between 0 and 1

• i.e. 0 ≤ P(Oi) ≤ 1 for each i, and

(2) The sum of the probabilities of all the outcomes equals 1

• i.e. P(O1) + P(O2) + … + P(Ok) = 1
P(Oi) represents the probability of outcome i
9
 Approaches to Assigning Probabilities
There are three ways to assign a probability, P(O i), to an
outcome, Oi, namely:

1. Classical approach: make certain assumptions (such as

equally likely) about a situation.

2. Relative frequency: assigning probabilities based on

experimentation or historical data.

3. Subjective approach: Assigning probabilities based on

the assignor’s judgment, intuition or reasoning.
11
 1. Classical Probability
• Probability of an event is the number of outcomes leading
to the event divided by the total number of outcomes.
• If an experiment has n possible outcomes [all equally
likely to occur], this method would assign a probability of
1/n to each outcome.
• Example:
Experiment: Rolling a die
Sample Space: S = {1, 2, 3, 4, 5, 6}
Probabilities: Each sample point has a 1/6 chance of
occurring.
• What is the probability of obtaining a head when you
drop a coin once?
11
 2. Relative Frequency Probability
• This method is based on historical data.
• It is computed after performing the experiment.
• The probability of an event is the Number of times an
event occurred divided by the number of trials

P(E )  n e
N
Where :
N  total number of trials
n e
 number of outcomes
producing E
12
 Examples of Relative Frequency Probability
• Example 1: In a hospital, a surgical operation has succeeded 120 times
when performed to 150 patients in the past. If a new patient will
perform this operation, what is the probability of surgical success
Solution: N=150 and ne=120, so P(Success)=0.80
• Example 2: The following are grades of 30 students on a Statistics
course. If a new student joins the course, what is the probability she
will get a B?
Statistics Grade
• The probability of an event #isofthe
students Probability
Number of times an event occurred
divided by theF number of trials 1 1/30 = .03 =P(F)
D 2 2/30 = .07 = P(D)
C 10 10/30 = .33 = P(C)
B 12 12/30 = .40 = P(B)
A 5 5/30 = .17 = P(A)
Total 30 ∑ = 1.00
13
 Four Important Types of Probability

We have discussed different types of events, and

different ways for assigning probabilities to events. Now,
we are going to study different types of probabilities
resulting from combining events in various ways. These
are:
• Marginal Probability
• Union Probability
• Joint Probability
• Conditional Probability
14
 Four Types of Probability

Marginal Union Joint Conditional

P( X ) P( X  Y ) P( X  Y ) P( X| Y )
The probability The probability The probability The probability
of X occurring of X or Y of X and Y of X occurring
occurring occurring given that Y
has occurred

X X Y X Y
Y

15
1. Marginal Probability
• Marginal probability of an event X is denoted by P(X).
• A marginal probability is usually computed by dividing
some subtotal by the whole (total).
• E.g.: The probability of a person wearing glasses in a
company.
This is obtained by dividing the number of people wearing
glasses in the company by the total number of people.
• In a cross-tabulation, a marginal probability is obtained
from the margins of the table, by adding across rows or
columns, and dividing by the total.

16
 Demonstration Example
The following is a contingency table for 155 employees
working in a company according to their gender and type
of position they hold. What is the probability that a randomly
selected employee holds a technical position?
Type of Gender
Position Male Female
Managerial 8 3
Professional 31 13
Technical 52 17
Other 9 22

There are 52+17 = 69 technical employees.

The marginal probability is thus P(T) = 69/155 17
2. Joint Probability
• The intersection of events X and Y is the event that both X and Y
occur and is denoted by X AND Y (X ∩Y).
• The probability of the intersection is called the joint probability.
• E.g.: The probability of a person wearing glasses and is left-
handed in a company.
This is obtained by dividing the number of people who are
wearing glasses and are left-handed by the total number of
people in the company. Being left-handed only or wearing
glasses only is not sufficient.
• In a cross-tabulation, a joint probability is obtained from the
cells of the table by dividing the cell frequency by the grand
total of the table.

18
 Demonstration Example
Continuing with the previous example…What is the probability
that a randomly selected employee is a female and
holds a technical position?

Type of Gender
Position Male Female
Managerial 8 3
Professional 31 13
Technical 52 17
Other 9 22

This is a joint probability as it represents the intersection of two events

(being a female and holding a technical position). There are 17 employees
satisfying both conditions. The probability is thus 17/155 = 0.11
19
 Probability Matrix
• A probability matrix displays the joint and marginal
probabilities. It can be useful in solving problems.
• In the demo example, the probability matrix is:
“what’s the probability an employee “what’s the probability an employee is professional”?
is both professional and male”? Marginal probability
Joint
probability Male B1 Female B2 P(Ai)
Managerial A1 .05 .02 .07

Professional A2 .20 .08 .28

Technical A3 .34 .11 .45

Other A4 .06 .14 .20

P(Bj) .65 .35 1.00

“what’s the probability an employee is male”? 20
Marginal probability
3. Union Probability – Addition Rule
• The union of events X and Y is the event that either X or Y
or both occur and is denoted by X OR Y (XUY).
• Union Probability is the sum of marginal probabilities
minus the joint probability (to avoid repetition of intersection).
• It is calculated using the General Rule for Addition:
P( X  Y )  P( X )  P( Y )  P( X  Y )

X Y

21
General Law of Addition - Example

P(T  F)  P(T )  P(F)  P(T  F)

T F P ( T )  . 45
P ( F )  . 35
.11 P ( T  F )  . 11
.45 .35
P ( T  F )  . 45  . 35  . 11
 0 . 69

22
In the demo example…
• Using the probability matrix
Male B1 Female B2 P(Ai)
Managerial A1 .05 .02 .07

Professional A2 .20 .08 .28

Technical A3 .34 .11 .45

Other A4 .06 .14 .20

P(Bj) .65 .35 1.00

P(T  F )  P (T )  P ( F )  P (T  F )
 .45  .35  .11
 .69 23
Special Case for Addition Rule
If
X and
Yare
mutually
exclusi
P(X 
Y)0

P(XY)P(X)P(
Y )

X Y

24
Demonstration Example
Type of Gender
Position Male Female Total
Managerial 8 3 11
Professional 31 13 44
Technical 52 17 69
Other 9 22 31
Total 100 55 155

P(TO)P (T)P(O) Since Technical and

69 31 Other are mutually
  exclusive and
155155
.645 P(T  O)  0
25
 Venn Diagram of the X or Y
but not Both Case

X Y

P( X  Y )  P( X  Y )
E.g.: Probability an employee is Female or Technical but not both?
26
 The Neither/Nor Region

Not in X, not in Y X Y

P( X  Y )  1  P( X  Y )
c

E.g.: Probability an employee is neither Female nor Technical?

27
 4. Conditional Probability
• Conditional probability is used to determine how two
events are related; that is, we can determine the probability
of one event given the occurrence of another related event.
• Experiment: randomly select one student in Cairo University
P(randomly selected student is male) =…
P(randomly selected student is male given that the student
is from FEPS)=… as if the population is now FEPS only.
• Conditional probabilities are written as P(X | Y) and read as
“the probability of X given Y” and is calculated as the joint
probability of X and Y divided by the marginal probability of
Y.
P( X Y )
P( X | Y ) 
P(Y ) 28
 Conditional Probability…
• Again, the probability of an event given that another event
has occurred is called a conditional probability…

• P( A and B) = P(A)*P(B/A) = P(B)*P(A/B) both are true

• Keep this in mind…

29
Conditional Probability Example
Example : 2 events F and T such that
P ( F )  . 35
P ( T )  0 . 45
P ( F  T )  . 11
P (T  F ) . 11
P (T | F )    . 31
P (F ) . 35
P (T  F ) . 11
P (F |T )    . 24
P (T ) . 45

T F
.11 .35

30
In the demo example…
• What’s the probability that a randomly
selected employee is Technical given that she
is a Female?
– Recall:
– T = employee is Technical
– F = employee is Female
• Thus, we want to know “what is P(T | F) ?”

31
• Using the probability matrix

Male B1 Female B2 P(Ai)

Managerial A1 .05 .02 .07

Professional A2 .20 .08 .28

Technical A3 .34 .11 .45

Other A4 .06 .14 .20

P(Bj) .65 .35 1.00

P (T  F ) .11
P (T F )    .31
P( F ) .35
Thus, there is a 31% chance that a randomly selected
employee is Technical given that the employee is Female.

32
• What is the probability of Technical given
Male? What can you conclude?
P (T  F ) .34
P(T M )    .52
P( M ) .65

• Can you find the probability of selecting a

female given she is technical?
P(T  F ) .11
P( F T )    .24
P(T ) .45

33
 Independent Events
• Occurrence of one event does not affect the
occurrence or nonoccurrence of the other event
• The conditional probability of X given Y is equal to
the marginal probability of X.
• The conditional probability of Y given X is equal to
the marginal probability of Y.
If X and Y are independen t events,
P ( X | Y )  P ( X ) , and
P (Y | X )  P (Y ).

34
Examples on Independent Events

1. In our demo example are the events of being

technical and being female independent?
2. In the following example, are events A and D
independent?
D E
8
A 8 12 20 P( A| D)  .2353
34
B 20 30 50 20
P( A)  .2353
85
C 6 9 15
P( A| D)  P( A)  0.2353
34 51 85

35
Example
• A company has 100 employees, 55 of them are married and 33
are supervisors. Also 22 employees are married and supervisors.
If one employee is selected randomly, find the following:
1) The probability matrix
2) The probability that the employee is married
3) The probability that the employee is married subject to that
he/she is supervisor
4) The probability that the employee is supervisor subject to that
he/she is married
5) Use the multiplication rule to find the joint probability that the
employee is married and supervisor.
6) What is the method you used to evaluate the above probabilities

36
Answer
Probability Matrix P(S)0.33
of Employees
P(M)0.55
Married 0.22
P(M| S) 0.67
Supervisor Yes No Total 0.33
Yes .22 .11 .33
No .33 .34 .67
P
(
MS
)
P(
S
)P
(M|
S )
Total .55 .45 1.00

(
0.
33)(
0.
67
)0.
2

The method is relative frequency

37
 Special Law of Multiplication
for Independent Events
• General Rule

P( X  Y )  P( X)  P(Y| X)  P(Y )  P( X| Y )
• Special Rule
If events X and Y are independent,
P( X )  P( X | Y ), and P(Y )  P(Y | X ).
Consequently,
P( X  Y )  P( X )  P(Y )

38