Probability Theory

Dr. H. Gladius Jennifer

Associate Professor
SPH, SRMIST
 Introduction - Probability
• Probability many times each day..
• A patient has a 50-50 chance of surviving..
• He is 90% chance that has a particular disease..
 Definition - Probability
• Probability is the chance that some event will happen.

• Its a ratio of the number of ways a certain event can occur to

the number of possible outcomes

Probability (p) = number of favorable outcomes

number of possible outcomes
• The probability of a particular event is the fraction of outcomes in which
the event occurs. The probability of event A is denoted by P(A).
• Probability values are between 0 (the event never occurs) and 1 (the
event always occurs).
• The sum of probabilities of all mutually exclusive events in a process
is 1. For example, if there are n possible mutually exclusive outcomes,
then
n
 P (i )  1
i 1
 Terms used in Probability
• Outcome : One possible result of a probability.
• Sample Space : List of possible outcomes for a probability event.
(S = HH,HT, TH, TT)
• Random : Outcomes that occur at random if each outcome
is equally likely to occur.
• Event : The possible outcomes of a stochastic process.
A deterministic process has only one possible outcome
A subset of possible outcomes

Ex: A={HH}, B={HT, TH}

• Simple Event : specific outcome or type of outcome.

• Complementary Events:
The events of one outcome happening and that outcomes not happening
are complimentary
The sum of the probabilities of complementary events is 1.

• Events can be mutually exclusive or independent event

• Events may or may not be mutually exclusive.
• Events that are not mutually exclusive are called independent events.
 Probability of an Event
• The probability of a particular event is the fraction of outcomes
in which the event occurs. The probability of event A is denoted
by P(A).

• P = 0 (the event never occurs)

• P = 1 (the event always occurs)
Probability of an event…

If an event can occur in N mutually exclusive and

equally likely ways, and if m of these possess a trait, E, the
probability of the occurrence of E is read as
P(E) = m/N
 Types of Probability

• Subjective probability
• Classical probability
• Relative Frequency
• Marginal probability
• Conditional probability
• Joint Probability
 Relative Frequency

• Assuming that an experiment can be repeated many times and

assuming that there are one or more outcomes that can result
from each repetition.
• Then, the probability of a given outcome is the number of times
that outcome occurs divided by the total number of repetitions.
• Marginal Probability:
They appear on the “margins” of a probability table.
It is probability of single outcome

• Conditional probability:
It is the probability of an event on condition that certain criteria
is satisfied.

• Joint probability:
It is the probability of occurrence of two or more events
together
 Properties of Probability

• The probability ranges between 0 and 1

• If an outcome cannot occur, its probability is 0
• If an outcome is sure, it has a probability of 1
• The sum of probabilities of mutually exclusive outcomes is equal to 1
P(A) + P(A) = 1
 Probability of Flip a coin
What is the probability of flipping a tail?
P (T) = favorable outcomes / possible outcomes
= 1 out of 2
(or) 0.5
(or) 50%

P(H) = 50%
 Probability of Roll a Dice
• The probability of roll a dice - 2

S = 1,2,3,4,5,6

P (2) = favorable outcomes / possible outcomes

=1/6
 Probability of Deck of Cards
The probability of picking a heart…
P (Heart) = favorable outcomes / possible outcomes
= 13 / 52 = 1/4

The probability of picking a heart = 0.25 (or) 25%

P (non heart) = 39/52 = 3/4

The probability of picking a no heart = 0.75 (or) 75%
Probability Theory- Laws
 Additive Law
• If A and B are mutually exclusive events, then the probability of either A or B to
occur is the union
P(A  B)  P(A)  P(B)
• Example: The probability of a hat being red is ¼, the probability of the hat
being green is ¼, and the probability of the hat being black is ½. Then, the
probability of a hat being red OR black is ¾.

B be two events in a sample space S. The probability of the union of

• Let A and
A and B is

P( A  B)  P( A)  P( B)  P( A  B).
 Multiplicative Law
• If A and B are independent events, then the probability that both
A and B occur is the intersection

P(A  B)  P(A) P(B)

• Intersection: Two events, A and B, are said to be independent

if the occurrence or nonoccurrence of one of the events does not
 change the probability of the occurrence of the other event.
 Conditional Probability
• What is the probability of event A to occur given than event B
did occur. The conditional probability of A given B is

P(A  B)
P(A | B) 
P(A)


When two events A and B are
mutually exclusive, P(AB) = 0
and P(AB) = P(A) + P(B).
A: male - India India Australia
P(A) = 20/120 Male 20 40
B: female - India Female 30 30
P(B) = 30/120
A and B are mutually P(AB)
P(AB) == P(A)
P(A) ++ P(B)
P(B)
==20/120
20/120++30/120
30/120
exclusive, so that
==50/120
50/120
 Example: Complementary Event

A: Male India Australia

P(A) = 60/120 Male 20 40
B: Female Female 30 30
P(B) = ?

A and B are P(B)

P(B) == 1-
1- P(A)
P(A)
complementary ==1-
1-60/120
60/120==60/120
60/120
 Bayes Theory
• Bayesian Methods Bayesian methods are named in honor of the Reverend
• Thomas Bayes (1702–1761), an English clergyman who had an interest in mathematics.
• Bayesian methods are an example of subjective probability, since it takes into
consideration the degree of belief that one has in the chance that an event will occur.
• While probabilities based on classical or relative frequency concepts are designed to
allow for decisions to be made solely on the basis of collected data, Bayesian methods
make use of what are known as prior probabilities and posterior probabilities.
• The prior probability of an event is a probability based on prior knowledge, prior experience, or results
derived from prior data collection activity.
• The posterior probability of an event is a probability obtained by using new information to update or revise
a prior probability.
• As more data are gathered, the more is likely to be known about the “true” probability of the event under
consideration.
• Although the idea of updating probabilities based on new information is in direct contrast to the philosophy
behind frequency-of-occurrence probability, Bayesian concepts are widely used.
• For example, Bayesian techniques have found recent application in the construction of e-mail spam filters.
• Typically, the application of Bayesian concepts makes use of a mathematical formula called Bayes’ theorem.
 The Central Limit Theorem

• Given a population of any non normal functional form with a mean m and
finite variance s2, the sampling distribution of x, computed from samples of
size n from this population, will have mean m and variance s2/n and will be
approximately normally distributed when the sample size is large.

• mathematical formulation of the central limit theorem is that the distribution of

Z = x- / n
approaches a normal distribution with mean 0 and variance 1 as n -> ∞.
• Note that the central limit theorem allows us to sample from non normally distributed
populations with a guarantee of approximately the same results as would be obtained if
the populations were normally distributed provided that we take a large sample.
• The importance of this will become evident later when we learn that a normally
distributed sampling distribution is a powerful tool in statistical inference.
• In the case of the sample mean, we are assured of at least an approximately normally
distributed sampling distribution under three conditions:
• (1) when sampling is from a normally distributed population;
• (2) when sampling is from a nonnormally distributed population and our sample is large;
and
• (3) when sampling is from a population whose functional form is unknown to us as long
as our sample size is large.
Thank You!