Discrete Probability

CSE 215 - Foundations in Computer Science

Dennis Wong
Basic Probability
An experiment is a procedure that yields one of a given set of possible outcomes.

The sample space is the set of possible outcomes.

An event is a subset of the sample space.

The probability of an event E, which is a subset of a finite sample space S where

each outcome is equally likely is given by:

p(E) = |E| / |S|

Example: What is the probability of rolling a 5 with 2 dice?

Solution: There are 36 outcomes for rolling 2 dice: {(1, 1), (1, 2), …, (6, 6)}

There are 4 ways to roll a 5: {(1, 4), (2, 3), (3, 2), (4, 1)}. Thus the probability that a
single roll produces a 5 is 4/36.
Basic Probability
Question: In the last example, why didn’t we say there were 11
different possible outcomes corresponding to the sum of the dice: {2,
3, . . . , 12}, thus probability of getting a 5 is 5 / 11?

Example 2: What is the probability of rolling a 5 with 3 dice?




Solution: There are 63 possible outcomes. There are 6 ways to roll a 5:

{(1, 1, 3), (1, 3, 1), (3, 1, 1), (1, 2, 2), (2, 1, 2), (2, 2, 1)}

Thus the probability that a single roll produces a 5 is 6 / 216.





Basic Probability
Example 3: What is the probability of obtaining a 4 of a kind in a 5 card poker hand?

Solution: We have two counting problems: we must count the number of different 5
card hands (the sample space) and the number of hands with 4 of a kind (the event).

|S| = C(52, 5) = 2598960


|E| = C(13, 1) C(48, 1) = 624

Thus the probability of getting 4 of a kind is 624 / 2598960 ≈ 0.00024.

Example 4: What is the probability of obtaining a full house in a 5 card poker hand?



Solution: A full house consists of a 3 of a kind and 2 of a kind:

|S| = C(52, 5) = 2598960


|E| = C(13, 1) C(4, 3) C(12, 1) C(4, 2) = 3744

Thus the probability of getting 4 of a kind is 3744 / 2598960 ≈ 0.0014.

Sampling with and without replacement
Example: What is the probability of selecting the numbered balls 18,
22, 21, 1, 49 in this order from a bin containing 50 balls?

No replacement: The number of possible outcomes is

P(50, 5) = 50 ∙ 49 ∙ 48 ∙ 47 ∙ 46 = 254251200.

Thus the probability of such an event is 1 / 254251200.


With replacement: The number of possible outcomes is 505 =

312500000. Thus the probability of such an event is 1 / 312500000.

Probability of the complement of an event
Theorem: Let E be an event from a sample space S. The probability of the

event E, the complement of E, is given by:

p(E) = 1 - p(E)

Example: A sequence of 10 bits is generated randomly. What is the probability

that at least two of the bits are 0?

Let E be the subset of strings with at least two 0 bits.

We could count the number of length 10 binary strings that have at least 2
zeros, but it is easier to count those that do not have at least 2 zeros.

So |E| = C(10, 0) + C(10, 1) = 11. |S| is given by the number of length 10 binary
10
strings: 2 = 1024.

Thus p(E) = 11 / 1024. which means p(E) = 1 - p(E) = 1013 / 1024.

Probability Theory
In many experiments, not all outcomes are equally likely. When
different outcomes occur with different probabilities we will need a
more general model.

Theorem: Let S be a finite sample space with n possible outcomes:

x1, x2, …, xn. We can assign probabilities to each event such that
the following two conditions are satisfied:

(1): 0 ≤ p(xi) ≤ 1

(2): p(x1) + p(x2) + … + p(xn) = 1

The function p from the set of all outcomes of the sample space S

is called a probability distribution.

Probability Theory
Suppose S has n elements. The uniform distribution assigns the probability 1 / n

to each element of S. Each outcome is equally likely in this distribution.

Theorem: The probability of an event E is the sum of the probabilities of the

outcomes of E:

p(E) = ∑ p(s)

s∈E

Example: A single die is loaded so that the 3 appears twice as often as each of the
other numbers. The five other outcomes are equally as likely. What is the

probability that an odd number appears when we roll this die?

Solution: First we must determine the probability distribution for this experiment.
Then we want to find p(E) where E = {1, 3, 5}.

p(1) = p(2) = p(4) = p(5) = p(6) = 1/7 and p(3) = 2/7.

Thus p(E) = 1/7 + 1/7 + 2/7 = 4/7.

Conditional Probability
Theorem: Let E and F be events with p(F) > 0. The conditional probability
of event E given event F denoted p(E | F) is defined as

p(E | F) = p(E ∩ F) / p(F)

Example: You flip a fair coin 3 times. What is the probability that there are
an odd number of tails? What if we know the event F - that the first flip is
tails?

The probability the first flip is tails is 1/2. The number of outcomes with an
odd number of tails where the first one is tails is 2: TTT, THH. There are 8
possible outcomes, so p(E ∩ F) = 1/4. Thus

p(E | F) = (1/4) / (1/2) = 1/2.

Conditional Probability
Example: You are at the casino playing blackjack. In a standard deck of 52
cards, what is the probability that you get a blackjack given that your first
card is a King?

Solution: The event E is you get a blackjack. The event F is that your first
card is a King. We want to compute the conditional probability p(E | F). The
probability of getting a king first is given by p(F) = 1/13. What is p(E | F)?

The total number of outcomes is 52 · 51. To get a blackjack with the first
card a king, we require the second card to be an ace. There are 4 ways to
get a king on the first card, and 4 ways to get an ace with the second card.
Thus p(E ∩ F) =16 / (52 ∙ 51). Thus

p(E | F) = p(E ∩ F) / p(F) = 16 / (52 ∙ 51) / (1/13) = 4/51.

The Monty Hall 3 door puzzle

Monty Hall problem: At the end of a game show, you are asked to select one of three doors.
Behind one the doors is a large prize, and nothing behind the others. After you select a
door, one of the other two doors is opened to reveal nothing. You now have the choice to
change doors. What should you do?

Solution: We want to compute the probabilities of winning the large prize if we keep the
same door or if we switch doors.

Initially, the probability of selecting the prize is 1/3. Even when a door is opened to reveal
nothing, this does not change the probability that your initial choice was correct. NOTE: the
probability of winning does not increase to 1/2 just because a door has been opened (since
a door with a prize is never revealed).

Now given that the door you chose has 1/3 chance of winning, this means that there is a
2/3 chance that the prize is behind the other two doors. This means that after a door has
been opened to reveal nothing, the other door must have a 2/3 chance of holding the prize.

Another way to think about it: if you picked wrong initially which is 2 out of 3 times, then if
you switch doors you will win.
 The Monty Hall 3 door puzzle

Behind door 1 Behind door 2 Behind door 3 Stay with door #1 Switch door
Car Goat Goat Win Lose
Goat Car Goat Lose Win
Goat Goat Car Lose Win
Independence
Suppose you flip a coin twice and you know the outcome of the first flip is
tails (event F). Does this outcome have any effect on the probability that the
second flip is also a tails (event E)? If not, then we say that the events are E

and F are independent.

Theorem: The events E and F are independent if and only if

p(E ∩ F) = p(E) p(F).

So let’s verify that the coin flips above are independent. The probability that
the first flip is tails: p(F) = 2/4. The probability that the second flip is tails:
p(F) = 2/4. The probability that both are tails:

p(E ∩ F) = 1/4 = 2/4 · 2/4.

Bernoulli Trials
Suppose that an experiment can have only two possible outcomes - like flipping
a coin. Each performance of such an experiment is called a Bernoulli trial,
named after James Bernoulli.

In general, the two possible outcomes are success and failure, where p denotes

the probability of success and q denotes the probability of failure. Thus p + q = 1.

Theorem: The probability of exactly k success in n independent Bernoulli trials is

k n-k
C(n, k) p q .

Example: Consider a bit generator that produces a 1 with probability 0.6, and a 0
with probability 0.4. What is the probability that the generator will produce a
length 8 bitstring with exactly two 1’s?

2 6
Solution: C(8, 2) 0.6 0.4