ENG 3120

Statistics

Dr. Deniz Karataş

deniz.karatas@cbu.edu.tr
 COURSE OUTLINE

• COURSE4- CONDITIONAL PROBABILITY,

TOTAL PROBABILITY THEOREM, BAYES'
THEOREM
 DEFINITIONS
• Probability Experiment • Equally Likely Events
• Process which leads to well-defined results call • Events which have the same probability of occurring.
outcomes
• Complement of an Event
• Outcome
• All the events in the sample space except the given
• The result of a single trial of a probability experiment events.
• Sample Space • Empirical Probability
• Set of all possible outcomes of a probability
• Uses a frequency distribution to determine the
experiment
numerical probability. An empirical probability is a
• Event relative frequency.
• One or more outcomes of a probability experiment • Subjective Probability
• Classical Probability • Uses probability values based on an educated guess
• Uses the sample space to determine the numerical or estimate. It employs opinions and inexact
probability that an event will happen. Also called information.
theoretical probability.
 DEFINITIONS
• Mutually Exclusive Events
• Two events which cannot happen at
the same time.
• Disjoint Events
• Another name for mutually exclusive
events.
• Independent Events
• Two events are independent if the
occurrence of one does not affect the
probability of the other occurring.
• Dependent Events
• Two events are dependent if the first event
affects the outcome or occurrence of the
second event in a way the probability is
changed.
• Conditional Probability
• The probability of an event occurring given that
another event has already occurred.
• Bayes' Theorem
• A formula which allows one to find the
probability that an event occurred as the result
of a particular previous event.
 INTRODUCTION TO PROBABILITY
• Sample Spaces
Second Die
• A sample space is the set of all possible outcomes. First
However, some sample spaces are better than others. Die
1 2 3 4 5 6
• Consider the experiment of flipping two coins. It is
possible to get 0 heads, 1 head, or 2 heads. Thus, the
sample space could be {0, 1, 2}. Another way to look at it
1 2 3 4 5 6 7
is flip { HH, HT, TH, TT }. The second way is better
because each event is as equally likely to occur as any 2 3 4 5 6 7 8
other.
• When writing the sample space, it is highly desirable to 3 4 5 6 7 8 9
have events which are equally likely.
• Another example is rolling two dice. The sums are { 2, 3, 4 5 6 7 8 9 10
4, 5, 6, 7, 8, 9, 10, 11, 12 }. However, each of these aren't
equally likely. The only way to get a sum 2 is to roll a 1
on both dice, but you can get a sum of 4 by rolling a 1-3,
5 6 7 8 9 10 11
2-2, or 3-1. The following table illustrates a better
sample space for the sum obtain when rolling two dice. 6 7 8 9 10 11 12
 INTRODUCTION TO PROBABILITY
• Classical Probability • If just the first and last columns were written,
• The above table lends itself to describing data we would have a probability distribution. The
another way -- using a probability distribution. relative frequency of a frequency distribution
Let's consider the frequency distribution for the
above sums. is the probability of the event occurring. This
is only true, however, if the events are equally
Sum Frequency Relative Frequency likely.
2 1 1/36
• This gives us the formula for classical
3 2 2/36
4 3 3/36
probability. The probability of an event
5 4 4/36
occurring is the number in the event divided
6 5 5/36
by the number in the sample space. Again,
7 6 6/36
this is only true when the events are equally
8 5 5/36
likely. A classical probability is the relative
9 4 4/36 frequency of each event in the sample space
10 3 3/36 when each event is equally likely.
11 2 2/36
12 1 1/36
• P(E) = n(E) / n(S)
 INTRODUCTION TO PROBABILITY
• Empirical Probability
• Empirical probability is based on observation.
The empirical probability of an event is the
relative frequency of a frequency distribution
based upon observation.
• P(E) = f / n
• Probability Rules
• There are two rules which are very important.
• All probabilities are between 0 and 1 inclusive
• 0 <= P(E) <= 1
• The sum of all the probabilities in the sample
space is 1
• There are some other rules which are also
important.
• The probability of an event which cannot
occur is 0.
• The probability of any event which is not in
the sample space is zero.
• The probability of an event which must
occur is 1.
• The probability of the sample space is 1.
• The probability of an event not occurring is
one minus the probability of it occurring.
• P(E') = 1 - P(E)
 PROBABILITY RULES
• "OR" or Unions • Example 1:
• Mutually Exclusive Events • Given: P(A) = 0.20, P(B) = 0.70, A and B are disjoint.
• Two events are mutually exclusive if they cannot • I like to use what's called a joint probability
occur at the same time. Another word that means distribution. (Since disjoint means nothing in
mutually exclusive is disjoint. common, joint is what they have in common -- so
• If two events are disjoint, then the probability of the values that go on the inside portion of the
them both occurring at the same time is 0. table are the intersections or "and"s of each pair of
events). "Marginal" is another word for totals -- it's
• Disjoint: P(A and B) = 0 called marginal because they appear in the
• If two events are mutually exclusive, then the margins.
probability of either occurring is the sum of the B B' Marginal
probabilities of each occurring. A 0.0 0.2 0.2
• Specific Addition Rule A' 0.7 0.1 0.8
Marginal 0.7 0.3 1.0
• Only valid when the events are mutually exclusive.
• The values in red are given in the problem.
• P(A or B) = P(A) + P(B) The grand total is always 1.00. The rest of the
values are obtained by addition and subtraction.
 PROBABILITY RULES
• Non-Mutually Exclusive Events • Interpreting the table
• In events which aren't mutually exclusive, there is some
overlap. When P(A) and P(B) are added, the probability of • Certain things can be determined from the joint
the intersection (and) is added twice. To compensate for probability distribution. Mutually exclusive events
that double addition, the intersection needs to be will have a probability of zero. All inclusive events
subtracted.
will have a zero opposite the intersection. All
• General Addition Rule inclusive means that there is nothing outside of
• Always valid. those two events: P(A or B) = 1.
• P(A or B) = P(A) + P(B) - P(A and B) B B' Marginal
A A and B are
• Example 2: Mutually
• Given P(A) = 0.20, P(B) = 0.70, P(A and B) = 0.15 Exclusive if
this value is 0
A' A and B are
B B' Marginal All Inclusive if
A 0.15 0.05 0.2 this value is 0
A' 0.55 0.25 0.8
Marginal 0.70 0.3 1.0
Marginal 1
 PROBABILITY RULES
• "AND" or Intersections • Example 3:
• Independent Events • P(A) = 0.20, P(B) = 0.70, A and B are independent.
• Two events are independent if the occurrence
of one does not change the probability of the B B' Marginal
other occurring.
• An example would be rolling a 2 on a die and A 0.14 0.06 0.2
flipping a head on a coin. Rolling the 2 does
not affect the probability of flipping the head.
• If events are independent, then the A' 0.56 0.24 0.8
probability of them both occurring is the
product of the probabilities of each occurring. Marginal 0.70 0.3 1.0
• Specific Multiplication Rule
• Only valid for independent events
• The 0.14 is because the probability of A and B is
• P(A and B) = P(A) * P(B) the probability of A times the probability of B or
0.20 * 0.70 = 0.14.
 PROBABILITY RULES
• Dependent Events B B' Marginal
A 0.08 0.12 0.2
• If the occurrence of one event does affect the probability A' 0.62 0.18 0.8
of the other occurring, then the events are dependent.
Marginal 0.7 0.3 1.0
• Conditional Probability
• Independence Revisited
• The probability of event B occurring that event A has
already occurred is read "the probability of B given A" • The following four statements are equivalent
and is written: P(B|A) • A and B are independent events
• General Multiplication Rule • P(A and B) = P(A) * P(B)
• Always works. • P(A|B) = P(A)
• P(A and B) = P(B|A) * P(A) • P(B|A) = P(B)
• Example 4: • The last two are because if two events are independent,
• P(A) = 0.20, P(B) = 0.70, P(B|A) = 0.40 the occurrence of one doesn't change the probability of
the occurrence of the other. This means that the
• A good way to think of P(B|A) is that 40% of A is B. 40% of probability of B occurring, whether A has happened or
the 20% which was in event A is 8%, thus the intersection not, is simply the probability of B occurring.
is 0.08.
 PROBABILITY RULES
• Conditional Probability
• Recall that the probability of an event
occurring given that another event
has already occurred is called a
conditional probability.
• The probability that event B occurs,
given that event A has already
occurred is
• P(B|A) = P(A and B) / P(A)
• This formula comes from the general
multiplication principle and a little bit
of algebra.
• Since we are given that event A has occurred,
we have a reduced sample space.
• Instead of the entire sample space S, we now
have a sample space of A since we know A
has occurred.
• So the old rule about being the number in the
event divided by the number in the sample
space still applies.
• It is the number in A and B (must be in A since
A has occurred) divided by the number in A.
• If you then divided numerator and
denominator of the right hand side by the
number in the sample space S, then you have
the probability of A and B divided by the
probability of A.
 PROBABILITY RULES
• Example 1: • What is the probability of a randomly
• The question, "Do you smoke?" was asked of selected individual being a male? This is the
100 people. Results are shown in the table. total for male divided by the total = 60/100 =
0.60. Since no mention is made of smoking or
not smoking, it includes all the cases.
Yes No Total • What is the probability of a randomly
Male 19 41 60 selected individual smoking? Again, since no
Female 12 28 40 mention is made of gender, this is a marginal
probability, the total who smoke divided by
Total 31 69 100 the total = 31/100 = 0.31.
• What is the probability of a randomly • What is the probability of a randomly
selected individual being a male who smokes? selected male smoking? This time, you're told
This is just a joint probability. The number of that you have a male - think of stratified
"Male and Smoke" divided by the total = sampling. What is the probability that the
19/100 = 0.19 male smokes? Well, 19 males smoke out of 60
males, so 19/60 = 0.31666...
 PROBABILITY RULES
• Example 1:
• The question, "Do you smoke?" was asked of 100 people. Results are shown in
the table.
Yes No Total
Male 19 41 60
Female 12 28 40
Total 31 69 100

• What is the probability that a randomly selected smoker is male? This time,
you're told that you have a smoker and asked to find the probability that the
smoker is also male. There are 19 male smokers out of 31 total smokers, so 19/31
= 0.6129 (approx)
 PROBABILITY RULES
• Example 2: Company Good Defective Total
Aberations 0.50-0.025 = 0.05(0.50) = 0.025 0.50
• There are three major manufacturing 0.475
Brochmailians 0.30-0.021 = 0.07(0.30) = 0.021 0.30
companies that make a product: Aberations, 0.279
Brochmailians, and Chompielians. Aberations Chompieliens 0.20-0.020 = 0.10(0.20) = 0.020 0.20
has a 50% market share, and Brochmailians 0.180
Total 0.934 0.066 1.00
has a 30% market share. 5% of Aberations'
product is defective, 7% of Brochmailians'
product is defective, and 10% of
Chompieliens' product is defective. • Notice that the 5%, 7%, and 10% defective rates
don't go into the table directly. This is because they
• Solution: This information can be placed into are conditional probabilities and the table is a joint
a joint probability distribution. probability table. These defective probabilities are
conditional upon which company was given. That
• The percent of the market share for is, the 7% is not P(Defective), but P(Defective|
Chompieliens wasn't given, but since the Brochmailians). The joint probability P(Defective
marginals must add to be 1.00, they have a and Brochmailians) = P(Defective|Brochmailians) *
20% market share. P(Brochmailians).
 PROBABILITY RULES
Company Good Defective Total
• What is the probability that a defective
Aberations 0.50-0.025 = 0.05(0.50) = 0.025 0.50
0.475 product came from Brochmailians?
P(Brochmailian|Defective) =
Brochmailians 0.30-0.021 = 0.07(0.30) = 0.021 0.30
0.279 P(Brochmailian and Defective) /
P(Defective) = 0.021/0.066 = 7/22 = 0.318
Chompieliens 0.20-0.020 = 0.10(0.20) = 0.020 0.20
0.180 (approx).
Total 0.934 0.066 1.00 • Are these events independent? No. If
they were, then P(Brochmailians|
• The "good" probabilities can be found by Defective)=0.318 would have to equal the
subtraction as shown above, or by multiplication P(Brochmailians)=0.30, but it doesn't.
using conditional probabilities. If 7% of Also, the P(Aberations and
Brochmailians' product is defective, then 93% is
good. 0.93(0.30)=0.279.
Defective)=0.025 would have to be
P(Aberations)*P(Defective) =
• What is the probability a randomly selected 0.50*0.066=0.033, and it doesn't.
product is defective? P(Defective) = 0.066
 BAYES’ THEOREM
• Bayes' theorem is a mathematical formula
used to determine the conditional
probability named by the 18th century
English mathematician Thomas Bayes.
• After the fact that there is an A event,
Bayes rule is also used to calculate the
conditional probability of one of the factors
that have an effect on the occurrence of
this event. • P (A | B) = Probability of event A when
event B occurs.
• It is the method used to determine the
effect (conditional probability) of any • P (A) = Probability of event A.
symptom on occurrence of A in a wide set
of symptoms. • P (B | A) = The probability of event B
happening when event A occurs.
• It is the most popular and widely used
theory among probability theories. • P (B) = Probability of event B.
 BAYES’ THEOREM
• Conditional probability has a wide
application in statistics.
• The total probability and bayes theorem
developed based on this concept gains
importance especially in decision making.
• When the probability of a B event cannot
be calculated directly, the probability of a
B event can be found based on these two
concepts.
• In cases where various reasons can give
the same result, it may not be known for
what reason this occurs, although
sometimes the result is known.
• Bayes' theorem is used when it is desired
to investigate with which probability and
why.
• In other words, Bayes' theorem provides
the opportunity to analyze backwards
while the result is clear.
• Bayes rule is used in calculating
conditional probabilities.
• If more than two events (factor, option,
cause) have an effect on the occurrence
of an A event, when the A event occurs,
the probability of observation of one of
the factors is calculated according to the
Bayesian rule.
 BAYES THEOREM
• Example1: X gene was detected in 70% of people with A
desease, which is seen in 5% of the population.
• However, just 20% of non-sick people were found to have the
X gene.
• In the light of these data, what is the probability that people
carrying the X gene become ill?

• Solution:
 BAYES THEOREM
• Example2: According to a study,
1 out of every 43 children get a
certain disease that occurs in
adulthood, and although it is not
completely reliable, the test of
an ill child gives 80% positive
and a healthy child's test 10%
positive.
• According to this information,
what is the probability that a
child with a positive test result
will actually become ill?
• Solution:
• P (A): The probability that the child
will become sick = 1/43
• P (B): Test probability of positive =
1/43 * 0.80 + 42/43 * 0.10 = 5/43
• P (A | B): Possibility of positive test
disease (this is asked)
• P (B | A): The probability that the test
of the child with the disease will be
positive = 0.80
• P (A | B) = P (B | A) * P (A) / P (B) =>
(0.80 * 1/43) / (5/43) = 0.16 = 16%.
 BAYES THEOREM
• Example3: Ali says it is itchy.
• There is a test for cat allergy, but this
test is not always true: For people
who are really allergic, it is 80% that
the test gives a "Yes" result.
• For people without allergies, the test
gives a "Yes" rate of 10% ("false
positive").
• If there is an allergy in 1% of the
population and the test shows "Yes",
what is the probability that Ali is really
allergic?
• Solution:
• A: Allergy, B: Test ‘Yes’
• P (A): Allergy probability = 0.01
• P (B): The probability that the test will be yes =?
(we will have to calculate)
• P (A | B): Possibility of allergies if the test is yes =?
(desired result)
• P (B | A): In case of an allergy, the probability of
the test going out yes = 0.80
• Let's find P (B) first. We will calculate the
probability of the test going out for those with and
without allergies, that is, for the entire population.
• We found P (B) = 0.01 * 0.80 + 0.99 * 0.10 = 0.107.
• P (A | B) = P (A) * P (B | A) / P (B) => 0.01 * 0.80 /
0.107 = 0.075 => ~ 7.5%.