2015 Fall

Engineering Mathematics 2

Ch 1. Probability

Kyungchun Lee
Dept. of EIE, SeoulTech

[1] Schaum's Outline of Probability, Random Variables, and Random Processes, H. Hsu.
[2] Probability and Statistics for Engineers and Scientists, 4/E, Anthony J. Hayter.
[3] Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer
Engineers, 2/E, R. D. Yates and D. J. Goodman.
[4] An Introduction to Analog and Digital Communications, Simon Haykin, Michael Moher.
 Monty Hall Problem

Numb3rs.S01,E13
2
Monty Hall problem
 Suppose you're on a game show, and you're given the choice
of three doors:
– Behind one door is a car.
– Behind the others are goats.
– You pick a door, say No. 1, and the host, who knows what's behind the
doors, opens another door, say No. 3, which has a goat.
– He then says to you, "Do you want to pick door No. 2?" Is it to your
advantage to switch your choice?

http://en.wikipedia.org/wiki/Monty_Hall_problem 3
Monty Hall Problem
 The result of switching or staying after initially picking Door 1
in each case Shown by the Host
Door 1 result if result if
Case Door 2 Door 3
(Initial pick) switching staying
1 Car Goat Goat Goat Car
2 Goat Car Goat Car Goat
3 Goat Goat Car Car Goat

 Aids to understanding
– Considering the same problem with 1,000 doors instead of just three.
– There are 999 doors with goats behind them and one door with a
prize.
– The player picks a door. The game host then opens 998 of the other
doors revealing 998 goats.
– To switch, or not to switch?

4
Probability in Movies

5
Overview
Ch. 1 Probability - Signal
- Basic Concepts: Sample Space and Events Processing
- Definition of Probability
- Conditional Probability - Communication
- Independent Events System

- Artificial
Ch. 2 Random Variables Intelligence

- Financial Market,
Gambling Science,
Ch. 3 Multiple Random Variables Reliability
Analysis,
Government,
Regulation, and
Ch. 4 Functions of Random Variables, Expectation, Everyday Life
Limit Theorems

6
Outline
 1.2 Sample Space and Events
 1.3 Algebra of Sets
 1.4 Probability Space
 1.6 Conditional Probability
 1.7 Total Probability
 1.8 Independent Events

7
1.2 Sample Space and Events
 Experiment
– Any process of observation
 Outcome
– Observation of the experiment
 Random Experiment
– An experiment with unpredictable outcomes
– Examples
• Roll of a die: Outcomes {1,2,3,4,5,6}
• Toss of a coin: Outcomes {Head, Tail}
• Drawing a card from a desk
• How long do you wait for the arrival of a bus at a bus stop?
• How many students are in the class room?
• Selecting a message signal for transmission from several messages

8
Sample Space
 Sample Space
– The set of all possible outcomes of a random experiment

 Example 1.1: The sample space for the experiment of

(a) Tossing a coin once
S={H, T}
(b) Tossing a coin twice
S={HH, HT, TH, TT}

 Example 1.2: The sample space for the experiment of tossing

a coin repeatedly and of counting the number of tosses
required until the first head appears.
S={1, 2, 3, …} : An infinite number of outcomes
9
Sample Space
 Example: Sample space for rolling two dice

 Example 1.3: Sample space for the experiment of measuring

(in hours) the lifetime of a transistor.

the life of a transistor in hours 10

Sample Space
 Discrete & Continuous
– Discrete sample space
• A finite number of sample points or countably infinite sample points
(one-to-one correspondence with the positive integers).
– Continuous sample space
• The sample points constitute a continuum.

11
Events
 Notations
: is an element of S

: is not an element of S

: A set A is a subset of B

 Event
– Any subset of the sample space S
 Elementary event
– A sample point of S
 Certain event
– The sample space S itself

12
Events
 Example 1.4: Consider the experiment of Example 1.2.
– A: The event that the number of tosses required until the first head
appears is even.
– B: The event that the number of tosses required until the first head
appears is odd.
– C: The event that the number of tosses required until the first head
appears is less than 5

13
Events
 The terminology of set theory and probability.

14
1.3 Algebra of Sets
Set Operations
 Set Operations
– Equality
Venn Diagrams
– The complement of A

– The union of A and B

– The intersection of A and B

15
Set Operations
– Generalizations of the union and intersection

16
Set Operations
– The difference of A and B

• Note

– The symmetrical difference of A and B

• Note

– Null Set
• The set containing no element

17
Set Operations
– Disjoint Sets
• Two sets A and B are called disjoint or mutually exclusive if

– A collection of sets is collectively exhaustive if and only if

18
Set Operations
 From the definition of the event

19
Set Operations
 Partition
– The collection is said to form a partition of S if
(i) for

(ii)

 Size (or cardinality) of set A

– The number of elements contained in A
– Properties when sets have a finite number of elements

20
Set Operations
 Product (or Cartesian product) of sets A and B
– The set of ordered pairs of elements from A and B

– Properties

 Example 1.5

21
Identities
 Basic laws

22
Identities
 Commutative Laws

 Associative Laws

 Distributive Laws

– Extensions

23
Identities
 De Morgan's Laws

– Extensions

 n 
n
  Ai   A i
 i 1  i 1

 n  n
  Ai   A i
 i 1  i 1

24
1.4 Probability Space
Event Space
 Event space F

 Example 1.6: Consider the experiment of tossing a coin once

in Example 1.1.
– Examples of the event space

– is not an event space. Why?

25
Probability Space
 Probability space
– The probability of the event AF is denoted by P(A)
– The triplet (S, F, P) : probability space.

26
Probability Measure
 Classical Definition of Probability
– Consider an experiment with equally likely finite outcomes.
– Then,

 Properties
i) If A and B are disjoint

ii)

iii)

27
Probability Measure
 Example 1.7
– Consider an experiment of rolling a die.
– The outcomes

– Events

– Probabilities of Events

28
Probability Measure
 Relative Frequency Definition of Probability
– Suppose that the random experiment is repeated n times.
– If event A occurs n(A) times, then the probability of event A is defined
as

Relative Frequency

 Properties
i)

ii) If A and B are mutually exclusive events, then

29
Probability Measure
 Example: Rolling two dice
– The event A: The sum is equal to 6.
• P(A) = 5 x (1/36) = 5/36

30
Probability Measure
– The event B: At least one 6 recorded
• P(B) = 11 x (1/36) = 11/36

31
Probability Measure
 Axioms of Probability
– Axiom 1

– Axiom 2

– Axiom 3
• For any countable collection A1, A2,... of mutually exclusive events

32
Elementary Properties of Probability
 By using the axioms, we get

33
Elementary Properties of Probability

which implies

34
Elementary Properties of Probability

35
Elementary Properties of Probability

36
Elementary Properties of Probability

which implies

37
1.6 Conditional Probability
 Conditional probability of an event A given event B
– The probability that event A occurs when it is known that event B
occurs
Joint probability of A and B

 Similarly,

 Then, we have

: Bayes' rule

38
Conditional Probability
 Example)

39
Conditional Probability
 Example)
– A blood test is 95% effective in detecting the viral infection when it is,
in fact, present.
– However, the test also yields false positive result for 1% of the healthy
persons tested.
– 0.5% of the population has the infection.
– If a person is tested to be positive, would you decide that he has the
infection?
P[positive|I]  0.95

P[positive |no I]  0.01 I: Infection

P[I]  0.005

40
Conditional Probability
 The probability that a person has the infection, given that his
test result is positive:

P[I positive]
P[I|positive] 
P[positive]
P[positive|I]  P[I]

P[positive|I]  P[I]  P[positive|no I]  P  no I 
(0.95)  (0.005)

(0.95)  (0.005)  (0.01)  (0.995)

 0.323  0.5
What would be your decision?

41
1.7 Total Probability
 The events are called mutually exclusive and
exhaustive if

42
Total Probability
 Then, : total probability
of event B

 From the Bayes' rule,  P[ Ai B]

: Bayes' theorem

 P[ B ]
conditioned on B conditioned on events Ai 43
1.8 Independent Events
 Two events A and B are said to be (statistically) independent if
and only if

which is equivalent to

44
Independent Events
 Example

– A and B are independent?

• Yes, because

45
Independent Events
 If two events A and B are independent, then it can be shown
that A and 𝐵 are also independent.

(why?)

46
Independent Events
 Three events A, B, C are said to be independent if and only if

– Isn’t the first condition sufficient?

 See the Example in the next page.

47
Independent Events
 Example:

– Events

– The first condition is satisfied:

– But, A1 and A2 are not independent because

48
Independent Events
 Extension of independence
– are independent if and only if
for every subset

49
Independent Events
 Mutually exclusive events

 Independent events

50
Summary
 Sample space, event, and outcome are probability terms for
the set theory concepts of universal set, set, and element.
 A probability measure P(A) is a function that assigns a
number between 0 and 1 to every event A in a sample space.
The assigned probabilities conform to the three axioms.
 A conditional probability P(A|B) describes the likelihood of A
given that B has occurred.
 A and B are independent events if and only if P(AB)=P(A)P(B).

51