MEL761: Statistics for Decision Making

Probability Classical view Experimental view Various rules Examples

Dr S G Deshmukh
Mechanical Department Indian Institute of Technology
1

Learning Objectives
Comprehend the different ways of assigning probability. Understand and apply marginal, union, joint, and conditional probabilities. Select the appropriate law of probability to use in solving problems. Solve problems using the laws of probability including the laws of addition, multiplication and conditional probability Revise probabilities using Bayes rule.
2

Methods of Assigning Probabilities

Classical method of assigning probability (rules and laws) Relative frequency of occurrence (cumulated historical data) Subjective Probability (personal intuition or reasoning)

Classical Probability
Number of outcomes leading to the event divided by the n total number of outcomes P( E ) = e N possible Where: Each outcome is equally N = total number of outcomes likely Determined a priori -- before ne = number of outcomes in E performing the experiment Applicable to games of chance Objective -- everyone correctly using the method assigns an identical 4 probability

Relative Frequency Probability

Based on historical data Computed after performing the experiment Number of times an event occurred divided by the number of trials Objective -- everyone correctly using the method assigns an identical probability
P( E) = ne N W ere : h N = to n m er o trials tal u b f

= n m er o o tco es u b f u m

p d cin E ro u g

Subjective Probability
Comes from a persons intuition or reasoning Subjective -- different individuals may (correctly) assign different numeric probabilities to the same event Degree of belief Useful for unique (single-trial) experiments
New product introduction Initial public offering of common stock Site selection decisions Sporting events

Structure of Probability
Experiment Event Elementary Events Sample Space Unions and Intersections Mutually Exclusive Events Independent Events Collectively Exhaustive Events Complementary Events

Experiment
Experiment: a process that produces outcomes More than one possible outcome Only one outcome per trial Trial: one repetition of the process Elementary Event: cannot be decomposed or broken down into other events Event: an outcome of an experiment may be an elementary event, or may be an aggregate of elementary events usually represented by an uppercase letter, e.g., A, E1

An Example Experiment
Experiment: randomly select, without replacement, two families from the residents of Tiny Town
x Elementary Event: the sample includes families A and C x Event: each family in the sample has children in the household x Event: the sample families own a total of four automobiles Tiny Town Population
Family Children in Household Number of Automobiles

A B C D

Yes Yes No Yes

3 2 1 2
9

Sample Space
The set of all elementary events for an experiment Methods for describing a sample space
roster or listing tree diagram set builder notation Venn diagram

10

Sample Space: Example

Experiment: randomly select, without replacement, two families from the residents of Tiny Town Each ordered pair in the sample space is an elementary event, for example -- (D,C)
Family Children in Household Number of Automobiles Listing of Sample Space

A B C D

Yes Yes No Yes

3 2 1 2

(A,B), (A,C), (A,D), (B,A), (B,C), (B,D), (C,A), (C,B), (C,D), (D,A), (D,B), (D,C)
11

Sample Space: Tree Diagram for Random Sample of Two Families B

A C D A B C D A B D A B C
12

Sample Space: Set Notation for Random Sample of Two Families

S = {(x,y) | x is the family selected on the first draw, and y is the family selected on the second draw} Concise description of large sample spaces

13

Sample Space
Useful for discussion of general principles and concepts
Listing of Sample Space

(A,B), (A,C), (A,D), (B,A), (B,C), (B,D), (C,A), (C,B), (C,D), (D,A), (D,B), (D,C)

Venn Diagram

14

Union of Sets
The union of two sets contains an instance of each element of the two sets.
Y = { 2,3,4,5,6} X = { 1,4,7,9}
X Y

X Y = { 1,2,3,4,5,6,7,9}

C = { IBM , DEC , Apple }

C F = { IBM , DEC , Apple , Grape , Lime}

F = { Apple , Grape , Lime}

15

Intersection of Sets
The intersection of two sets contains only those element common to the two sets.
Y = { 2,3,4,5,6} X Y = { 4} X = { 1,4,7,9}

C = { IBM , DEC , Apple } C F = { Apple }

F = { Apple , Grape , Lime}

16

Mutually Exclusive Events

Events with no common outcomes Occurrence of one event precludes the occurrence of the other event
C = { IBM , DEC , Apple } C F ={ F = { Grape , Lime }

Y = { 2,3,4,5,6} X Y = {

X = { 1,7,9}

P( X Y ) = 0
17

Independent Events
Occurrence of one event does not affect the occurrence or nonoccurrence of the other event The conditional probability of X given Y is equal to the marginal probability of X. The conditional probability of Y given X is equal to the marginal probability of Y.
P ( X | Y ) = P ( X ) a n dP ( Y | X ) = P ( Y )
18

Collectively Exhaustive Events

Contains all elementary events for an experiment

E1

E2

E3

Sample Space with three collectively exhaustive events

19

Complementary Events
All elementary events not in the event A are in its complementary event.
Sample Space

P ( S a m p leS p a ce = 1 )
P ( A ) = 1 P ( A)

20

Counting the Possibilities

mn Rule Sampling from a Population with Replacement Combinations: Sampling from a Population without Replacement

21

mn Rule
If an operation can be done m ways and a second operation can be done n ways, then there are mn ways for the two operations to occur in order. A cafeteria offers 5 salads, 4 meats, 8 vegetables, 3 breads, 4 desserts, and 3 drinks. A meal is two servings of vegetables, which may be identical, and one serving each of the other items. How many meals are available?

22

Sampling from a Population with Replacement

A tray contains 1,000 individual tax returns. If 3 returns are randomly selected with replacement from the tray, how many possible samples are there? (N)n = (1,000)3 = 1,000,000,000

23

Combinations
A tray contains 1,000 individual tax returns. If 3 returns are randomly selected without replacement from the tray, how many possible samples are there?

N! 1000! N = = 166,167,00 0 = n n!( N n)! 3!(1000 3)!

24

Four Types of Probability

Marginal Probability Union Probability Joint Probability Conditional Probability

25

Four Types of Probability

Marginal Union Joint Conditional

P( X )
The probability of X occurring

P( X Y )
The probability of X or Y occurring

P( X Y )
The probability of X and Y occurring

P( X | Y )
The probability of X occurring given that Y has occurred

X Y

X Y Y
26

General Law of Addition

P ( X Y ) = P ( X ) + P (Y ) P ( X Y )
X Y

27

General Law of Addition -Example

P( N S ) = P( N ) + P( S ) P( N S )
N
.70
.56

S
.67

P( N ) P( S ) P ( N S ) P ( N S )

=70 . =67 . =56 . =70 +67 56 . . . = 0.81

28

Office Design Problem Probability Matrix

Increase Storage Space Yes No .14 .56 .19 .11 .33 .67

Noise Reduction

Yes No Total

Total .70 .30 1.00

29

Office Design Problem Probability Matrix

Increase Storage Space Yes No .14 .56 .19 .11 .33 .67 Total .70 .30 1.00

Noise Reduction

Yes No Total

P( N S ) = P( N ) + P( S ) P( N S ) =.70 +.67 .56 =.81

30

Office Design Problem Probability Matrix

Increase Storage Space Yes No .14 .56 .19 .11 .33 .67 Total .70 .30 1.00

Noise Reduction

Yes No Total

P( N S ) =.56+.14 +.11 =.81

31

Venn Diagram of the X or Y but not Both Case

X Y

32

The Neither/Nor Region

P( X Y ) = 1 P( X Y )
33

The Neither/Nor Region

P( N S ) = 1 P( N S ) = 1.81 =.19
34

Special Law of Addition

If X and Y are mutua exclusive, lly P( X Y ) = P( X ) + P(Y )

Y X
35

Example..
Type of

Position Managerial Professional Technical Clerical Total

Gender Male Female 8 3 31 13 52 17 9 22 100 55

Total 11 44 69 31 155

P(T C ) = P(T ) + P(C ) 69 31 = + 155 155 =.645

36

Example..
Type of

Position Managerial Professional Technical Clerical Total

Gender Male Female 8 3 31 13 52 17 9 22 100 55

Total 11 44 69 31 155

P ( P C ) = P ( P ) + P (C ) 44 31 = + 155 155 =.484

37

Law of Multiplication
P ( X Y ) = P ( X ) P (Y | X ) = P (Y ) P ( X | Y )
80 P( M ) = = 0. 5714 140 P( S| M ) = 0. 20 P ( M S ) = P ( M ) P ( S| M ) = ( 0. 5714)( 0. 20) = 0.1143

38

Law of Multiplication Example..

Probability Matrix of Employees
Supervisor Yes No Total Married Yes No Total .1143 .1000 .2143 .4571 .3286 .7857 .5714 .4286 1.00

30 = 0.2143 140 80 P(M ) = = 0.5714 140 P( S | M ) = 0.20 P(S ) =

P(M S) = P( M) P(S| M) = (0. 5 1 )(0. 2 ) = 0.1 4 74 0 13

P(MS) = P(M) P(MS) P(S) =1 P(S) =1 0. 24 = 0. 75 13 87 = 0.51 0.14 = 0. 47 74 13 51 P(M S) = P(S) P(MS) = 0.24 0.14 = 0.10 13 13 00 P(M) =1 P(M) =1 0.51 = 0. 48 74 26
39

P(M S) = P(S) P(MS) = 0. 75 0. 47 = 0.38 87 51 26

Special Law of Multiplication for Independent Events

General Law

P ( X Y ) = P ( X ) P (Y | X ) = P (Y ) P ( X | Y ) Special Law
If events X and Y are independent, P( X ) = P( X | Y ), and P(Y ) = P(Y | X ). Consequently, P( X Y ) = P( X ) P(Y )

40

Law of Conditional Probability

The conditional probability of X given Y is the joint probability of X and Y divided by the marginal probability of Y.

P ( X Y ) P (Y | X ) P ( X ) P( X |Y ) = = P (Y ) P (Y )

41

Law of Conditional Probability

S
.56

N
.70

P ( N ) =70 . P ( N S ) =56 . P ( N S ) P( S | N ) = P( N ) .56 = .70 =80 .

42

Office Design Problem

Increase Storage Space Yes No .14 .56 .19 .11 .33 .67 Total .70 .30 1.00

Noise Reduction

Yes No Total

Reduced Sample Space for Increase Storage Space = Yes

P ( N S ) .11 P( N | S ) = = P(S ) .67 = .164

43

Independent Events
If X and Y are independent events, the occurrence of Y does not affect the probability of X occurring. If X and Y are independent events, the occurrence of X does not affect the probability of Y occurring.
If X and Y are independent events , P( X | Y ) = P( X ), and P(Y | X ) = P(Y ).
44

Independent Events Demonstration Problem

Geographic Location Northeast Southeast D E Finance A Manufacturing B Communications C .12 .15 .14 .41 .05 .03 .09 .17 Midwest F .04 .11 .06 .21 West G .07 .06 .08 .28 .35 .37

.21 1.00

P ( A G ) 0.07 P( A| G ) = = = 0.33 P(G ) 0.21 P( A| G ) = 0.33 P( A) = 0.28

P( A) = 0.28

45

Independent Events Demonstration Problem

D A B C 8 20 6 34

E 12 30 9 51 20 50 15 85

8 P( A| D) = =.2353 34 20 P( A) = =.2353 85 P( A| D) = P( A) = 0.2353

46

Revision of Probabilities: Bayes Rule

An extension to the conditional law of probabilities Enables revision of original probabilities with new information
P( Xi| Y ) = P(Y | Xi ) P( Xi ) P(Y | X1) P( X1) + P(Y | X 2 ) P( X 2 ) + P(Y | Xn ) P( Xn )

47

Revision of Probabilities with Bayes' Rule: Example

P( A) = 0.65 P( S ) = 0.35 P(d | A) = 0.08 P(d | S ) = 0.12 P( A | d ) = P(d | A) P( A) P(d | A) P( A) + P(d | S ) P( S ) (0.08)(0.65) = = 0.553 (0.08)(0.65) + (0.12)(0.35) P(d | S ) P( S ) P( S | d ) = P(d | A) P( A) + P(d | S ) P( S ) (0.12)(0.35) = = 0.447 (0.08)(0.65) + (0.12)(0.35)

48

Revision of Probabilities with Bayes Rule:

Prior Probability Event Conditional Probability Joint Probability Revised Probability

P( Ei )
0.65

P(d| Ei )
0.08

P(Ei d) P( Ei| d )
0.052 0.052 0.094 =0.553

A Alamo

S South Jersey

0.35

0.12

0.042 0.094

0.042 0.094 =0.447

49

Revision of Probabilities with Bayes' Rule:Example

A 0.65 Defective 0.08 0.052

Acceptable 0.92 Defective 0.12 Acceptable 0.88 0.042

0.094

S 0.35

50

Probability for a Sequence of Independent Trials

25 percent of a banks customers are commercial (C) and 75 percent are retail (R). Experiment: Record the category (C or R) for each of the next three customers arriving at the bank. Sequences with 1 commercial and 2 retail customers. R2 R3 C1 R1 R C2 R R3 C
51

Probability for a Sequence of Independent Trials

Probability of specific sequences containing 1 commercial and 2 retail customers, assuming the events C and R are independent
1 3 3 9 P(C1 R 2 R 3) = P(C ) P( R) P( R) = = 4 4 4 64 3 1 3 9 P( R1 C 2 R 3) = P( R) P(C ) P( R) = = 4 4 4 64 3 3 1 9 P( R1 R 2 C 3) = P( R) P( R) P(C ) = = 4 4 4 64

52

Probability for a Sequence of Independent Trials

Probability of observing a sequence containing 1 commercial and 2 retail customers, assuming the events C and R are independent
P( (C1 R 2 R 3) ( R1 C 2 R 3) ( R1 R 2 C 3)) = P(C1 R 2 R 3) + P( R1 C 2 R 3) + P( R1 R 2 C 3) 9 9 9 27 = + + = 64 64 64 64
53

Probability for a Sequence of Independent Trials

Probability of a specific sequence with 1 commercial and 2 retail customers, assuming the events C and R are independent
9 P( C R R ) = P ( C ) P ( R ) P ( R ) = 64

Number of sequences containing 1 commercial and 2 retail customers n n! 3! nCr = = = =3 r r !( n r ) ! 1!( 3 1) ! Probability of a sequence containing 1 commercial and 2 retail customers
9 27 ( 3) = 64 64
54

Probability for a Sequence of Dependent Trials

Twenty percent of a batch of 40 tax returns contain errors. Experiment: Randomly select 4 of the 40 tax returns and record whether each return contains an error (E) or not (N). Outcomes with exactly 2 erroneous tax returns
E1 E2 N3 N4 E1 N2 E3 N4 E1 N2 N3 E4 N1 E2 E3 N4 N1 E2 N3 E4 N1 N2 E3 E4

55

Probability for a Sequence of Dependent Trials

Probability of specific sequences containing 2 erroneous tax returns (three of the six sequences)
P( E 1 E 2 N 3 N 4) = P( E 1) P( E 2| E 1) P( N 3| E 1 E 2) P( N 4| E 1 E 2 N 3) 55,552 8 7 32 31 = = 0.01 50 49 48 47 5,527 ,200 P( E 1 N 2 E 3 N 4) = P( E 1) P( N 2| E 1) P( E 3| E 1 N 2 ) P ( N 4| E 1 N 2 E 3) 55,552 8 32 7 31 = = 0.01 50 49 48 47 5,527 ,200 P( E 1 N 2 N 3 E 4) = P( E 1) P( N 2| E 1) P( N 3| E 1 N 2 ) P ( E 4| E 1 N 2 N 3) 55,552 8 32 31 7 = = 0.01 50 49 48 47 5,527 ,200
56

Probability for a Sequence of Independent Trials

Probability of observing a sequence containing exactly 2 erroneous tax returns
P(( E1 E 2 N 3 N 4 ) ( E1 N 2 E 3 N 4 ) ( E1 N 2 N 3 E 4 ) ( N1 E 2 E 3 N 4 ) ( N1 E 2 N 3 E 4 ) ( N1 N 2 E 3 E 4 )) = P( E1 E 2 N 3 N 4 ) + P( E1 N 2 E 3 N 4 ) + P( E1 N 2 N 3 E 4 ) + P( N1 E 2 E 3 N 4 ) + P( N1 E 2 N 3 E 4 ) + P( N1 N 2 E 3 E 4 ) 55, 552 55, 552 55, 552 55, 552 55, 552 55, 552 = + + + + + 5, 527 , 200 5, 527 , 200 5, 527 , 200 5, 527 , 200 5, 527 , 200 5, 527 , 200 0. 06
57

Probability for a Sequence of Dependent Trials

Probability of a specific sequence with exactly 2 erroneous tax returns
55 ,552 8 7 32 31 P ( E 1 E 2 N 3 N 4 ) = = 0.01 50 49 48 47 5,527 ,200

Number of sequences containing exactly 2 erroneous tax returns n n! 4! n nCr = = C = = =6 r r !( n r ) ! 2 !( 4 2) ! r Probability of a sequence containing exactly 2 erroneous tax returns
55,552 ( 6) 0.06 5,527 ,200
58

Examples on Probability
A bag contains 5 white balls & 4 black balls. One ball is drawn at random. What is the probability of drawing alternative white and black ball? Ans:
=

5 4 4 3 3 2 2 1 1 9 8 7 6 5 4 3 2 1 126
59

Examples on Probability
If on an average rain falls on 12 days in every 30 day. Find probability 1) that first 3 of a given week will be fine and remainder wet. 2) that rain will fall on just 3 days of a given week. Ans. Here p= 12/30 =0.40 1) (0.40) (0.40) (0.40) (0.60)4 = (0.40)3 (0.60)4 = 0.0038 2) 7c3 (0.40)3 (0.60)4 = 0.2903

60

Chebyshevs Inequality
1 p [ x k ] k = ( x ) f ( x)
2 2 2

= ( x ) f ( x) + ( x ) f ( x) + ( x ) f ( x)
2 2 2 R1 R2 R3

R1 = x ( k ) R 2 = ( k ) < x < ( + k ) R 3 = x ( + k )
61

Chebyshevs Inequality (Cont)

R ( x ) f ( x ) + ( x ) f ( x )
2 2 2 R1 R3

But ( x ) k ...inR1

( x ) k ...inR3
or x k ...inR1or 3

k
2 2 2

[ f ( x ) + f ( x )]
R1 R3

1 [ f ( x ) + f ( x )] k
62