Foundations FOR Inferential

Statistics-1
Probability –Elementary Concepts

DEVENDRA LAL KARANJIT

 Probability ! What is this?
• Everyday We talk about chances of happening of
different events, don’t we?
• Have you ever expressed chance in number form?
• Put simply, Probability is the ‘chance expressed
numerically’
• It gives the likelihood of happening of some event.
• Probability values take values between 0 and 1.
• Larger the probability value, more is the likelihood
of happening of that event.
 Probability? Why should we study
this?
• To understand the further topics of Inferential
Statistics in the probabilistic way
• To understand that ‘Whole Statistics is based on
notion of probability’
• To interpret the decisions in Hypothesis testing
• To mathematically model ‘Processes’
• …………………………..
 Probability Keywords and their
Meaning
• When we talk of Probability, we must specify the
EVENT we are talking about.
• Events are denoted by Uppercase letters, A,B,C etc.
• P(A) is read as Probability of happening of A
• P(Ac) is read as ‘Probability of happening of A
Complement’ ( not happening of A) =1-P(A)
• Events are outcomes of a TRIAL
Sample Space: The set of all possible outcomes of an experiment is
called the sample space for that experiment. The list of all possible
outcomes of a random experiment is called ‘Collectively Exhaustive
List’.

• Examples:
• 1.Sample space of a coin tossing experiment is S= {Head,
Tail}
• 2.If we consider the experiment of throwing a die the sample
space would be
• S={1, 2, 3, 4, 5, 6}.
• 3.If a card is drawn from a pack of 52 cards, the sample space
has 52 members:
• S={A, K,Q,……..2 ,A, K, Q……..2,A, K,
Q,….2,A, K, Q,……2}
• 4. A box contains 4 balls, 2 balls are drawn then the sample
space is S={B1B2, B1B3, B1B4, B2B3, B2B4, B3B4}
 Note:
• It may not be possible( or practicable) to list
all the outcome in every experiment, e. g. Can
you list all the possible cases of drawing three
cards from a pack of 52 cards?
• In those situation you must be able to ‘count’
the number of possible cases by using
Permutations and/or combinations.
 Trial: Roll a die !
• The outcome is not sure but it could be one of
{1,2,3,4,5,6}
• These are exhaustive events and this set is
called ‘Sample space’
• Suppose, We are interested in ‘event of seeing
an ace (one dot)’
• Can you give some more examples of events?
 Probability Keywords
• Favorable events for an event: all events
that would lead us to happening of an
event under consideration.
• Mutually Exclusive Events: events that
can not happen together
• Independent Events: Events are said to
be independent if happening or non
happening do not effect and not affected
by happening of others.
 Probability: Approaches
1. Classical Approach (‘a priori’ approach)
• Uses ‘mathematical’ definition
• Outcomes are assumed to be equally likely and mutually
exclusive
• Called ‘a priori’ because it can be obtained without having
to do the experiment
• What is Probability of not happening of A, or P(Ac)?

Number of cases favorable to A

P( A) 
Total number of cases
 Remarks and Examples
• Suitable for Tossing Coins, Drawing Balls/Cards,
Rolling Dice (unbiased)
• Not suitable in real life conditions, because in real life
conditions, equally likely events are not possible
• If someone jumps from the ‘Dharahara’ tower what is
the probability of his/her surviving?
• If you bid for a contract, what is the probability of
getting it?
• What is the probability of Federer winning this year’s
US open?
Odds: Other ways of expressing
Probability
• The probability of an event is sometimes expressed in terms of
odds (especially in game situations) by using either the phrase
'odds in favour of an event A' or the phrase 'odds against A".
• The 'odds in favour of A' is the ratio P(A)/P(Ac), and the 'odds
against A' is the ratio P(Ac)/ P(A), provided that the
denominator is not zero. Odds are usually expressed in the
terms of ratio p:q of two positive integers.
• The statements P(A)=m/N or odds in favor of A is m:(N-m) or
odds against A is (N-m):m conveys same meaning.
• For example, odds for Federer to win a match against Nadal is
4:5 means that P(Federer’s winning)=4/9 and odds against him
is 7:11 means P(Nadal’s winning )=11/18
 Probability: Approaches
2. Empirical Approach
• Also called Statistical approach
• Based on outcomes of a number of trials
• More realistic than classical approach
• But probability based on different number of trial often
fluctuate
• Large number of trails are necessary to stabilize the
probability i.e. m/N becomes stable as N becomes
sufficiently large

number of times an event happen

P( A) 
number of trials
m
P( A )  lim
N  N
Remarks on Empirical Approach
• The relative frequency table you calculated in previous sessions gives
the probability of a randomly selected observation falling into
different classes.
• In this approach, we look for the identical past events before assigning
the probability. More past event we consider, more accurate is the
probability.
Think!
• How would you get the probability of dying of a person 60 years old,
within one year. Do you think this probability would remain constant
year after year.
• Suppose you are going to get married on the February 14, 2010. How
would you make the statement of the probability of raining on that day.
• The old people of Kathmandu say that on the day of "Bhoto Jatra of
Macchindranath", it will rain. Keeping aside the religious belief, do
you think that there is any empirical reason.
 Probability: Approaches
2. Subjective Approach
• Based on ‘expertise’ in subject matter
• Result of past events, future anticipations, present
scenarios
• Result of Analysis + Intuition+ Guess
• What is the probability that ‘Queen of Hearts’ was set
as ‘Joker’ in this game of ‘Marriage’?
• What is the probability that Nadal will win the next
year’s French Open?
• What is the probability that one of the Williams
Sisters will not reach the final of next year’s French
Open?
 Example: Suppose there is going to
be a football match between two
teams A and B.
• Suppose you are going to bet for one of them. The
classical probability simply says that the probability of
their winning is 0.5 each. In relative frequency
approach, one would go for the past matches between
those teams in identical conditions. In subjective
approach, one may analyze about the performance of the
teams in last season, fitness of players, goal-saving
record of the keeper( in case of a penalty shoot-out),
ground conditions, audience support, records of strikers
etc. before making probability assessments.
Exercises 4.3
 Probability Types:

• Marginal Probability-happening of only one

event
• Joint Probability-happening of two or more
events simultaneously.
• Conditional probability- probability of
happening of one event given the other events’
record.
 Composite Events
• Combination of events
• In assigning probability, the events may be of simple
or composite kind.
• To compute the probabilities of composite events, the
usual way of listing of favourable cases may not be
feasible, so we require the rules of assigning
probabilities for different composite events, such as
The composite event where one or another happen
The composite event where both of events happen
 Probability Rules
• Rule of Addition (to get probability of happening
A or B in different conditions of mutual
exclusiveness of A and B)
• Rule of Multiplication (to get simultaneous
happening of A and B)
• Total Rule of Probability (Mixture of above two
rules, to get probability of an event, which is being
affected by many other events)
• Bayes Rule (Special rule that deals with Posterior
events)
 Independent Events
• Two or more events are said to be independent
if happening or non-happening of one of them
does not effect and not affected by happening
or non happening of others.
• In real life, we generally tag events as
independent when we do not have reasons to
believe that they are dependent.
• Understand the ball drawing schemes.
 Ball drawing Schemes
• One by One Without Replacement(WOR)- The
draws are dependent.
• One by One With Replacement(WR)-The
draws are independent.
• Drawing at Random-same as WOR,
hypergeometric rule.
 Conditional Probability
• The conditional probability of event B, known
(or given) that another event A has already
happened is denoted by P(B|A), it is the
probability of B, conditioned with the
occurrence of A. If A and B are independent
event then P(A|B)=P(A), P(B|A)=P(B). Here
the event that comes after the vertical line is
called the conditioning event.
 Examples:
1. Suppose from a bag containing 2 red balls and 3 black balls, a ball is drawn and
not replaced before making the second draw. Let A= event of getting a red ball in
the first draw and B= event of getting a red ball in the second draw. Then
•P(B|A) = P(red ball in second draw| red ball in first draw)=1/4
•P(B|Ac )= P(red ball in second draw|black in first)=2/4
•However if the balls were replaced before the second draw then
•P(B|A)=2/5=P(B). This example shows that the ‘Sample with Replacement’
gives rise to Independent Events and ‘Sampling without Replacement’ gives rise
to Dependent Events.

2. Suppose there are 3 boxes B1, B2 and B3 which contain balls of different
colours as shown below,
•B1: 5 red and 3 green, B2: 4 black and 2 green, B3: 5 red and 4 black
•Suppose a box is selected at random, and a ball is drawn from the selected box.
Let A= event of getting a black ball, then the event A is dependent on event of
selecting different boxes, as such, we give the conditional probabilities as follows,
P(A|B1) = 0, P(A|B2)= 4/6, and P(A|B3) = 4/9
 Examples: (Contd.)
Very Moderately A little Very Total
Satisfied Satisfied Dissatisfied Dissatisfied

Male 108 100 19 6 233

Female 77 71 8 11 167
Total 185 171 27 17 400

•What is the probability that a randomly selected employee is

very satisfied and is male?
•What is the probability that a randomly selected male is
'moderately satisfied‘?
•What is the probability that a 'Very Dissatisfied' employee is a
female?
 Problem Solving Steps
• Read the question carefully.
• Identify the given events in the problem.
• Give proper symbols to the events like,
A,B,A|B AB AUB etc.
• Identify the asked probability-whether it is a
marginal, joint or conditional probability.
• Apply appropriate rule and get answer.
 Rule of Addition of Probability:
• Let A and B be events, then AB gives the event at A or B
happens(at least one of them occurs) and AB(or simply AB)
gives the event that both happens simultaneously( A and B
happens). We state the following rule in addition:
1. When A and B are not mutually exclusive,
• P(AB)=P(A) + P(B) –P(AB)
2. when A and B are mutually exclusive, they can not happen
simultaneously i.e P(AB)=0 then P(AB)=P(A) + P(B)
• For 3 events A, B, and C we have,
• P(ABC)=P(A)+P(B)+P(C)-P(AB) –P(BC)-
P(AC)+P(ABC) and
• when A, B and C are mutually exclusive
P(ABC)=P(A)+P(B)+P(C)
 Do Exercises 4.4
i) Suppose A=event of getting a spade in a draw of a
card from a deck of cards
• B= event of getting a face card
• Then P(A)=13/52, P(B)= 12/52, P(AB)=3/52
• So P(AB)=P(spade or face card)=22/52
• This is the case of non-mutually exclusive events.
ii)suppose A= event of getting a sum of 6 on roll of two
fair dice
• B= event of getting a sum of 7
• then A and B are mutually exclusive event.
• We know that P(A)=5/36 and P(B)=6/36 so P(A or
B)= P(getting 6 or 7)=11/36
 Multiplication Rule
• The multiplication rule of probability gives the probability of
simultaneous happening of two or more events, P(AB) gives
the probability of simultaneous happening of events A and B,
and this probability is called the joint probability of occurrence
of A and B.
• This probability has different formulas under the conditions of
dependence and independence.
• Case 1: when the events A and B are independent P(AB) =
P(A)P(B)
• Case 2: When the events A and B are dependent
P(AB)=P(A).P(B|A) =P(B)P(A|B)
• For three or more events?
 Examples:
• Consider the experiment of rolling a pair of fair dice and tossing a
coin. Let A= event of getting a head and B= event of getting 4. Then A
and B are independent and P(AB)=P(head and 4) =
P(A)xP(B)=1/2X1/6 = 1/24
• Consider the experiment of drawing two balls from a box containing 2
red and 4 white balls successively without replacement. Let A= event
of getting a red ball on the first draw and B= event of getting a red ball
in second draw then we know that P(A) = P(red ball in first draw)= 2/5
and P(B|A)=1/4, then the joint probability of happening of A and B,
i.e. probability of getting red ball in both draws is given by P(AB)=
P(A) P(B|A)=2/20
• Missing Card Example: Suppose in a pack of cards, 2 cards are
missing. Suppose A1 = event that both missing cards are red, A2=
Both missing cards are Black, A3= event that missing cards are one of
each color. Further suppose that one card is drawn from this pack and
let X= event that a black card turns out. Then we say that event X is
dependent on A1, A2 and A3. The conditional probabilities are given
by, P(X|A1)= 26/50, P(X|A2)= 24/50 and P(X|A3)= 25/50
Literal Definition and Symbols
Literal Expressions Symbolic Expressions
All A, B and C happen

A and B happen but C does not

At least one of A, B, C happens

At most one of A, B, C happens

None happen

Two of three events happens

At least two happens

One and only one happens

Exactly one happens

 Class work
• Finding out Joint, Marginal and Conditional
Probabilities from a Two-way table (Cross tables).
Many problems in your textbook may be quickly
solved using a cross table.
• Tree Diagrams –See pages 178-179
• Suppose a box contains 3 red and 4 white balls, 3
balls are successively drawn from it without
replacement. Draw a tree diagram and hence find the
probability that exactly two are red. Does this answer
match with the previous way (hyper geometric
model)
• Exercises 4.5
Probabilities under Dependence
• Joint and Conditional Probabilities under
dependence can be obtained by transposing the
formula of multiplication rule.
• The marginal probability of an event A under
dependence is the sum of all joint probabilities
which lead to happening of A. This is called
‘Rule of Total Probability’.
 Rule of Total Probability
• Let A be any event, which can happen only in
conjunction with one and only one of the mutually
exclusive events E1, E2, ……En ( that is A may
happen only after the happening of one of these
events). Then the marginal probability of happening
of A is given by,
• Note that this formula is the mixture of "Rule of
Addition " and "Rule of Multiplication" of
probabilities. Further note that the events (AEj),
which give the probability that A will happens in
conjunction with Ej are mutually exclusive.
 Examples:
i) Suppose there are 3 boxes B1, B2 and B3 which contain balls
of different colours as shown below,
B1: 5 red and 3 green, B2: 4 black and 2 green, B3: 5 red and 4
black
Suppose a box is selected at random, and a ball is drawn from a
selected ball, what is the probability that it is a green ball ?
• Hint: Here the event A of getting a green ball happens only
after one of the events of selecting a box. Use rule of total
probability.
ii) In the Missing Card Example what is the probability of
drawing a black card?
 Bayes’ Rule for posterior
probability
• Definition: Let A be any event, which can happen only in conjunction with
one and only one of the mutually exclusive events E1, E2, ……En ( that is
A may happen only after the happening of one of these events). Then the
marginal probability of happening of A is given by,
n n
P( A)   P( A  E j )   P( E j ) P( A | E j )
j 1 j 1
further if the event A actually happens, then we are concerned
about the probability of the event (Ej|A ) that the event A was
due to Ej, this probability is given by Bayes' theorem:

P( A  E j ) P( A  E j )
P( E j | A )   n

 P( E
P( A )
j )P( A | E j )
j 1
 Remarks and Examples
• Class Demonstration: How to have an idea about the Missing Cards?
• The contents of urn I, II and III as follows.
Urn I: 1 white, 2 black, 3 red balls,
Urn II: 2 white, 1 black, 1 red ball,
Urn III: 4 white, 5 black, and 3 red balls.
An urn is chosen at random and two balls are drawn. They happen to be white
and red. What is the probability that they come from Urn I, II or III?

In 2009, there were 5 candidates for the post of Principal of a college. Their
probabilities of appointment are in the ratio 1:4:5:2:3. All the candidates had
promised to buy a Snooker Board in the college but their respective
probabilities of buying are 0.5, 0.8, 0.1, 0.6 and 0.7.
• What is the probability that there would be a Snooker board in the college.
• Someone was appointed and bought a Snooker Board, find the posterior
probabilities of different candidates selection.

Do Exercises-4.7
Home work: Review and Applications
 Concluding Remarks
• Hints and solutions to difficult problems are
available, BUT solve examples with
UNDERSTANDING. Don’t memorize.
• These concepts are necessary for chapter 5
Remember-
• WRONG Understanding is dangerous than
NOT understanding.
• ‘HORN PLEASE’ doesn’t mean ‘SEE YOU
Again’.