Elementary Probability Theory

Probability theory plays an important role in the theories and applications of statistics. Probability
theory is concerned with the study of random experiments. Most examples of probability are
based on the following experiments:

1. Experiments Dice: In these experiments, the dice consists of six faces, each face taking a
number from one to six.

2. Experiments Coin: In these experiments, the coin consists of two faces, Head (H) or Tail (T) .

3. Ball box experiments: It contains different balls.

4. Playing card experiments: It consists of 52 cards and is divided into four groups, as follows:

a. group of Head b. group of spade c. group of club d. group of diamond .

An experiment: An experiment is some process that we do.

Populations: A population is the largest group of people or things in which we are interested at a
particular time.

Sample : From the population, we select some elements on which we collect our information, this
part of population on which we collect the data is called the sample. Denoted by  .

• φ⊆ Ω 𝑖𝑠 𝑎𝑛 𝑒𝑣𝑒𝑛𝑡 (impossible event)

• Ω⊆ Ω is an event (Sure event).

Probability of an event = (# of ways it can happen) / (total number of outcomes)

P(E) = (no. of ways E can happen) / (Total number of outcomes denoted by  )

As example when you throw a coin, the probability of getting heads is

1 1
P( H )  and the probability of getting a tail is P(T ) 
2 2
The probability that this event will not occur (its failure) is represented by P( Ei )  1  P( Ei )

Example: A box containing white 3balls and 5 black balls. If a ball is drawn from the box at
random, what is the probability that it will be black?

5
P( B) 
8
1
Example: You tossed a coin 3 times, what is the probability of getting tails twice?

The number of possible states is 8 because

2  2  2  8 ( HHH , HHT , HTH , HTT ,THH ,THT ,TTH ,TTT )

3
Therefore, the possible cases are 3 namely P( H ) 
8
Sample size: the number of elements in the sample is called the sample size, and is denoted by n.

The sample space: Is a set of points representing all possible results of an experiment, where each
result is represented by a point or element in the sample space.

Throw a coin once Throw a coin twice

H T HH HT TH TT

A random experiment: Is an experiment whose outcome cannot be known because it is subject

to the laws of probability. (Throw a coin is a random experiment)

Independent events: Are events that, if one of them occurs, does not prevent or affect the
occurrence of other events. (When throwing two coins, getting heads on the first coin, for
example, does not affect the outcome of the second coin)

Non independent events: It is a result of what happened, one of which affects the events of other
events. (For example: If we have a box containing white and black balls, then when two balls are
pulls up sequentially so that the first ball is not returned, the result of the second pulls up is
affected by the result of the first draw)

Variables: The characteristics to be measured on the elements of the population or sample are
called variables.

Event: Is a point or several points in the sample space and is symbolized by: Ei

Mutually (exclusive) events: We say that two events E1 , E2 are mutually exclusive or unlikely if
it is impossible for them to occur together. As throw a coin.

Types of Variables: Variables are classified as:

2
1. Quantitative variables: the values of quantitative variables are numbers indicating how much or
how many of something.

Example: family size, Numbers of patients, weight..

There are two types of quantitative variables:

a. Discrete variables: A discrete variable can have countable number of values.

Example: Family size¸ number of patients

b. continuous variables: A variables can have any value within a certain interval of values.

Example: Blood sugar level

2. Qualitative variables: the values of a qualitative variable are words indicating to which an
element of the population belong.

Example: sex, educational level, blood type.

Type of population

Qualitative Quantitative

Discrete Continuous

Example1: Suppose that we measure whether or not one regulary takes a vitamin for a sample of
50 pregnant Saudi women.

The variable: whether or not one reglarly takes a vitamin.

Population: all pregnant.

Sample size: 50

Type of variale: Qualitative.

Example 2: Suppose that we measure the hemoglobin level in g/ dl for a sample of 75 people who
have a certain disease.

The variable: hemoglobin level.

3
Population: All people who have a certain disease sample size 75

Type of variable: continuous quantitative.

Example 3: Number on children in family

3 5 2 4 0 1 3 4
5 2 3 2 3 3 2 1
Variable: number of children.

Sample size: 16

Simple Frequency Distribution

Example 3: Number on children in family.

3 5 2 4 0 1 3 4
5 2 3 2 3 3 2 1
Variable: no. of children

Sample size: 16

Number of children frequency Relative freq.=freq./n Percentage freq.=

(variable) R.F.*100%
0 1 0.0625 6.25%
1 2 0.125 12.5%
2 4 0.25 25%
3 5 0.3125 31.25%
4 2 0.125 12.5%
5 2 0.125 12.5%
Total n=16 1.00 100%

Graphical Representation:
(No. of women)

5
Frequency

4
3
2
1

0 1 2 3 4 5
4
 X=Number of children

Example 2 : Dice experiment

There are six different outcomes {1, 2, 3, 4, 5, 6}
1
What’s the probability of rolling a one? P(1) 
6
2 1
What’s the probability of rolling a one or six? P(1or 6)  
6 3
What’s the probability of rolling an even number (i.e., rolling a two, four or a six)?

Remark: The probability of an event can only be between 0 and 1 and can also be written as a
percentage.
Example 3: Selected a ball from a box containing 6 balls numbered 1,2,3,4,5 and 6
• Consider the following events:
• E1= getting an even number = {2,4,6}
• E2 = getting a number less than 4= {1,2,3}
• E3 = getting 1 or 3 = {1,3}
• E4 = getting an odd number = {1,3,5}
• E5 = getting a negative number = {} =φ
• E6 = getting a number less than 10 = {1,2,3,4,5,6} = Ω

Solution:
  1,2,3,4,5,6  n     6
E1  2,4,6  n  E1   3
E2  1,2,3  n  E2   3
E3  1,3  n  E3   2
Then
3 1 3 1 2
P( E1 )   , P( E2 )   , P( E3 ) 
6 2 6 2 6

Properties of Probability.

Let A and B be any two events defined on the sample space Ω.

1. The probability of an event E is defined as P(E) = [Number of favourable outcomes of E]/[ total
number of possible outcomes of E].

5
2. The probability of a sure event or certain event is 1.

3. The probability of an impossible event is 0.

4. The probability of an event E is a number P(E) such that 0 ≤ P (E) ≤ 1. Probability is always a
positive number.

5. If A and B are 2 events that are mutually exclusive, then P(A⋃B) = P(A) + P(B).

6. An elementary event is an event having only one outcome. The sum of the probabilities of such
events of an experiment is 1.

7. The sum of probabilities of an event and its complementary event is 1. P(A) + P(A’) = 1.

8. P(A⋃B) = P(A) + P(B) – P(A⋂B).

A A⋂B B A B
B
A B

A B A B
Intersection, Union and Comp of Sets

The intersection of two (or three, or four, or more) sets contains the elements common to all of
the sets. The intersection of sets is a set, itself.

The intersection of sets A and B (denoted as A∩B) is the set containing the elements that are in
both A and B.

The union of a collection of sets is a set containing all the elements of individual sets. The union
of two sets A and B is a new set denoted by A  B , which contains all the elements of the set A
and all the elements of set B (without repetition).
Complement of an event: Ac is the complement of A.
c
A consists of all outcomes of Ω but are not in A.
A

6
9. P(A⋂B) = P(A) + P(B) – P(A⋃B) .

10. If A1, A2, A3 ,………, An are mutually exclusive events, then P(A1 ⋃ A2 ⋃ A3… ⋃ An) = P(A1)
+ P(A2 ) + ………. + P(An)

Example 4:

A carton consists of 200 watches of which 190 are good, 7 have minor defects and 3 have major
defects. A merchant Reena will only accept the watches which are good, but another merchant
Seema will only reject the watches which have major defects. One watch is drawn at random from
the carton. What is the probability that it is acceptable to Reena? Also, find the probability that it
is acceptable to Seema?

Solution:

A watch is drawn at random from 200 watches. So there are 200 equally likely outcomes.

No. of outcomes acceptable to Reena = 190/200 = 0.95

No. of outcomes acceptable to Seema = (190+7 )/200 = 197/200 = 0.985

Example:

Experiment: selecting a ball from a box containing 6 balls numbered 1,2,3,4,5, and 6 randomly.
• Define the following events:
• E1 = {2,4,6}= getting an even number
• E2= {1,2,3}= getting a number less than 4.
• E3 = {1,3} = getting 1 or 3.
• E4= {1,3,5} = getting an odd number.
• (1)E1𝖴 𝐸2 = 1,2,3,4,6, …. 𝑛 Ω 6
• P(E1𝖴 𝐸2) = = 5/6
6
E1  E4  1,2,3,4,5,6  P  E1  E4   1
6
1
E1  E2  2  P  E1  E2  
6

7
Counting with Permutations and Combinations
Combinatorics is a branch of mathematics that focuses on the study of finite or countable discrete
structures. Combinatorics has applications in optimization, computer science and statistical
physics. There are times that it makes sense to count the number of ways an event could occur by
looking at each possible outcome. However, when a large number of outcomes exist, this method
becomes inefficient. If someone asked you how many possible regular license plates there are for
the state of Orange, it would not be feasible to count each and every one. instead, you could use
the fact that on the typical Oregon license plate, there are 4 numbers and 3 letters. Using this
information, about how many license plates could there be?

The Counting Principle : Consider choice A with 3 options  A1 , A2 , A3  and choice B with 2
options  B1 , B2  . If you had to choose an option from A and then an option from B , the overall
total number of options would be 3  2  6 . The options are

 A1B1 , A1B2 , A2 B1 , A2 B2 , A3 B1 , A3 B2  . You can see where the 6 comes from by making a
decision tree and using the Fundamental Counting Principle. A decision tree is a graph that
models the options possible at each stage of an experiment. To make a decision tree, you 1st need
to determine how many decisions you are making. Here, there are only two decisions to make: 1)
choose an option from A , and 2) choose an option from B , so you will have two "slots" in your
decision tree. Next, think about how many possibilities there are for the 1st choice (in this case
there are 3), and how many possibilities there are for the 2nd choice (in this case there are 2). The
Fundamental Counting Principle says that you can multiply those numbers together to get the total
number of outcomes”

A1

A A2

Decision Tree A3

B B1

B2

Combinations and Permutations: Another type of counting question is when you have a given
number of objects, you want to choose some (or all) of them, and you want to know how many
ways there are to do this. For example, a teacher with a class of 30 students wants 5 of them to do

8
a presentation, and she wants to know how many ways this could happen. These types of
questions have to do with combinations and permutations. The difference between combinations
and permutations is whether or not the order you are choosing the objects matters.
 A teacher choosing a group to make a presentation is a combination problem, because
order does not matter.
 A teacher choosing 1st-, 2nd-, and 3rd-place winners in a science fair is a permutation
problem, because the order does matter. (1st place and 2nd place are different outcomes.)
The factorial symbol, !, means to multiply every natural number up to and including that
whole number together. For example, 5!  5  4  3  2 1 . The factorial symbol is used in
the formulas for permutations and combinations.”

Factorial: Denoted by n!, n!  n  n  1 n  2 ......1.

n!
Note: a. 0!  1, b. 1!  1, c.  (n  1)!
n
Combination Formula The number of ways to choose k objects from a group of n objects where
order does not matter is
n n!
C  k  
  k !(n  k )!
n k

Example:
7 7! 7  6  5  4  3  2 1
C (7,3)  7 C3       105
 
3 3!(7  3)! 3  2  1  4  2  1

Permutation Formula It is the number of ordered choice methods that can be formed from
several things by taking all or some of them. It is denoted by n Pk , i.e. permutations of k from n
and its law is
n n! n!
P  k ! k   k! 
k !(n  k )! (n  k )!
n k
 
Example
7! 7  6  5  4  3  2 1
P(7,3)    210
(7  3)! 4  3  2 1

Podiums Notice that in both combination and permutation problems, you are not allowed to repeat
your choices. Any time you are allowed to repeat and order does not matter, you can use the
Fundamental Counting Principle. (Problems with repetition where order does not matter are more
complex and not discussed in this section.) Whenever you do a counting problem, the 1st thing

9
you should decide is whether the problem is a Fundamental Counting Principle problem, a
permutation problem, or a combination problem. You'll find that permutation problems can also
be solved with the Fundamental Counting Principle, but the opposite is not true. There are many
Fundamental Counting Principle problems (ones where you are allowed to repeat choices) that
cannot be solved with the permutation formula.
Examples:
Example 1 You are going on a road trip with 4 friends in a car that fits 5 people. How many
different ways can everyone sit if you have to drive the whole way?
Solution:
The Fundamental Counting Principle is a great way of thinking about this problem. You have to
sit in the driver's seat. There are 4 options for the 1st passenger seat. Once that person is seated,
there are 3 options for the next passenger seat. This goes on until there is one person left with 1
seat: 1.4.3.2.1  24 .

Example 2 How many different ways can the gold, silver, and bronze medals be awarded in an
Olympic event with 12 athletes competing?

Solution:
Since the order does matter with the 3 medals, this is a permutation problem. You will start with
12 athletes and then choose and arrange 3 different winners.

12! 12! 12 11 10  9!

12 P3     1320
(12  3)! 9! 9!

Example 3: You are deciding which awards you are going to display in your room. You have 8
awards, but you only have room to display 4 awards. Right now you are not worrying about how
to arrange the awards, so the order does not matter. How many ways could you choose the 4
awards to display?
Solution:
Since order does not matter, this is a combination problem. You start with 8 awards and then
choose 4
8  8! 8  7  6  5  4!
C   4    70
    
8 4
  4!(8 4)! 4 3 2 1 4!

Note that if you try to use the Fundamental Counting Principle with this question, you will need to
do an extra step of reasoning. There are 8 options you could choose 1st, then 7 left, then 6, and
lastly 5.
8  7  6  5  1680

10
This number is so big because it takes order into account, which you don't care about. It is the
same result you would get if you used the permutation formula instead of the combination
formula. To get the right answer, you need to divide this number by the number of ways 4 objects
can be arranged, which is 4!  24 . This has to do with the connection between the combination
formula and the permutation formula
Example 4: Recall the problem from the Introduction: How many Oregon license plates could be
created with 3 letters and 4 numbers? Solution: A license plate that has 3 letters and 4 numbers
can be represented by the Fundamental Counting Principle with 7 spaces. You can use the
Fundamental Counting Principle because order definitely does matter with license plates. The 1st
spot is a number, the next three spots are letters, and the last three spots are numbers. Note that
when choosing a license plate, repetition is allowed.
10  26  26  26 10 10 10  104  263  175760000 .

Example 5: There are 20 hockey players on a pro NHL team, 2 of whom are goalies. How many
sets of 5 skaters and 1 goalie can be on the ice at the same time?
Solution: The question asks for how many on the ice, implying that order does not matter. This is
combination problem with 2 combinations. You need to choose 1 goalie out of a possible of 2, and
choose 5 skaters out of a possible 18.

 2 18  18!
1  5   2   17136
   5! 13!

Example 6: How many different ways could you score a 70% on a 10-question test, where each
question is weighted equally and is either right or wrong?
Solution: The order of the questions you got right does not matter, so this is a combination
problem
10  10!
7    120
  7! 3!

Example 7: How many different 4-digit ATM passwords are there? Assume you can repeat digits.

Solution: Order does matter. There are 10 digits, and repetition is allowed. You can use the
Fundamental Counting Principle for each of the 4 options.

10 10 10 10  10000

11
 H.W

1. Simplify each of the following expressions so that do not have a factorial symbol:

7! 110! 52!
a. , b. , c.
3! 105!5! 49!
2. If we have four letters A, B, C, D and two letters are chosen from them. How many ways can
these two letters be chosen? (ans 12)

3. If the numbers from 1 to 9 are written on cards and placed in a box, then 5 cards are drawn from
it. (One after the other) How many pentagonal numbers (with different numbers) can be formed?

(ans: 15120)

4. How many arrangements can be formed from the letters of a word “statistics”? (ans: 50400)

(note: all letters 10, s=3,t=3,a=1,i=2,c=1)

5. How many selection methods can be obtained to select a committee of 5 people from a total of
9 people? (ans: 126)

6. It is known that the number of playing cards is 52 cards, that a card Spade can be dealt in 13
ways, and that a heart card can be dealt in 13 ways as well. When a random card is drawn from
the playing cards, the number of possible ways to choose is? ……..

7. If two cards are drawn from a deck of playing cards such that one is one the Spade and the other
is, then there are (…………….) ways to do this.

8. A box contains 6 red balls, 4 black balls, and 3 white balls. In how many ways can 5 balls be
chosen so that 3 of them are red and 2 are black? (ans: 120)

9. A coach has a team of 20 players. He wanted to choose a team consisting of 11 players,

including 1 goalkeeper, 2 for defense, 3 for midfield, and 5 for attack. If 2 players can be
goalkeepers, 5 can be defense, 6 can be attack, and 7 are midfield. How many teams can be
formed? (and: 8400)

12
13