Chapter 1.

Probability
Introduction to probability
1. Set definitions
2. Set operations
3. Probability introduced through sets and relative
frequency
4. Joint and conditional probability
5. Independent events
6. Combined experiments
7. Bernoulli trials
8. Summary
1
Chapter 1. Probability
 Introduction

Probabilistic Method In Electrical Engineering

Probability is the study of random or non-deterministic experiments.
Probability applications arises in most science and engineering
People sometimes applies probability without knowing it

2
 Introduction

Signals and systems of interest to engineers are

not deterministic.
 You cannot predict behavior precisely.
 Deterministic analysis and design methods usually don’t work!

Signals Like: x(t )  a cos( t   ), y (t )  a u (t ) e t

N 1 k n 2
j
z[n]   ak e N
, w[n]   [n]  4 [n  3]
k 0

are rare
3
 Introduction

There are many applications for probability:

• Weather forecast ( Temperature, Humidity, Pressure )
we hear it every day in the news.
• Stock prices change randomly.
If you know how the prices changes exactly please let me know.

It has applications in Science, Engineering, Finance…

Even sometimes we apply probability without knowing it.

4
 1.1 Set Definitions
Set A
element a  A a A

Tabular method Rule method

A  {2, 4,6,8,10,} A  {a  N a is even}

N  the set of natural numbers, N={1,2,3,...}

Z  the set of integers Q  the set of rational numbers

R  the set of real numbers

5
Chapter 1. Probability
 1.1 Set Definitions
Def: Set A is finite, if A has a finite number of elements.
not finite = infinite
Def: Set A is countable, if there is an onto function f : N  A.
not countable = uncountable
finite  countable A  {2, 4,6} is countable.

N , Z , & Q are countable.

R  Q ( the set of irrational numbers) is uncountable.
R is uncountable.

6
Chapter 1. Probability
 1.1 Set Definitions
Def: Set A is a subset of set B, if a  A  a  B.
A B
If at least one element exists in B which is not in A ,
 
B

A B A

 = null set A B
A  B    A & B are disjoint (mutually exclusive).
A & B have no common elements B A

Example Let the sets A and B be given as

A = { 1, 3, 5, 7 } and B = { 2, 4, 6, 8, 10, 12, 14 }  
7
Chapter 1. Probability
 1.1 Set Definitions
Ex 1.1-1: A  {1,3,5,7}, B  {1, 2,3,}, C  {0.5  c  8.5}
D  {0.0}, E  {2, 4,6,8,10,12,14}, F  {5.0  f  12.0}

finite -- infinite --
countable -- uncountable --
A B A E  D 

Universal  set  = 𝑆  Contains all outcomes

rolling a die  S  {1, 2,3, 4,5,6}
toss two coins  S  {HH , HT , TH , TT }
8
Chapter 1. Probability
 1.2 Set Operations
Venn Diagram

9
Chapter 1. Probability
 1.2 Set Operations
A B & B A  AB  Equality

A  B  {x : x  A & x  B}  A  B  Difference

B SB C
S  A A
N N
A1  A2    AN   An A1  A2    AN   An  Multiple Sets
n 1 n 1

A B  B  A A B  B  A 

10
Chapter 1. Probability
 1.2 Set Operations
( A  B)  C  A  ( B  C )  A  B  C

( A  B)  C  A  ( B  C )  A  B  C

A  ( B  C )  ( A  B)  ( A  C )

A  ( B  C )  ( A  B)  ( A  C )

( A  B)  A  B

( A  B)  B  A

Duality    Example1:
  S Example2:
11
Chapter 1. Probability
 1.3 Probability Introduced Through Sets and
Relative Frequency
Two Definitions of Probability: - set theory and axioms
- relative frequency

Experiment: Ex am p le:  Expermints  of  

A random activity with
Rolling a die.
- sample space S S  {1, 2,3, 4,5,6}
A  {1,3,5} B  {2, 4,6}
- event
C  {1, 2,3, 4}
2
- probability P P ( A)  P ( B )  0.5 P (C ) 
3
12
Chapter 1. Probability
 1.3 Probability Introduced Through Sets and
Relative Frequency
Sample  space    𝑺
set of all possible outcomes in the experiment

Rolling a die  S  {1, 2,3, 4,5,6}

Toss a coin  S  {H , T }
Toss 2 coins  S  {HH , HT , TH , TT }
Choose a number in [0,1]  S  {s : 0  s  1}
Choose a number in N  S  N  {1, 2,3,}
discrete & continuous
finite & infinite
countable & uncountable
13
Chapter 1. Probability
 1.3 Probability Introduced Through Sets and
Relative Frequency
Event = a subset of S
  the set of all events

Toss a coin  S  {H , T }    { ,{H },{T }, S}

Toss 2 coins  S  {HH , HT , TH , TT }

   { ,{HH },{HT },,{HH , HT , TH }, S}
Choose a number in [0,1]  S  {s : 0  s  1}
A  {a : 0  a  0.5}
Choose a number in N  S  N  {1, 2,3,}
A  {1,3,5,}

14
Chapter 1. Probability
 1.3 Probability Introduced Through Sets and
Relative Frequency
Probability P = a function from  to [0,1]
Probability is a function that maps events to the real line (Number)
Toss a coin  S  {H , T }
 P ( )  0, P({H })  0.6, P({T })  0.4, P( S )  1
To assign these probabilities or numbers, it must satisfies three axioms:
Probability    Axioms:
1. P ( A)  0
2. P ( S )  1
3. A  B    P ( A  B )  P ( A)  P ( B )
 
3'. mutually exclusive {An : n  1, 2,}  P (An )   P( An )
n 1 n 1
15
Chapter 1. Probability
 1.3 Probability Introduced Through Sets and
Relative Frequency

Example:
Experiment: S  {s : 0  s  1}
Event: A  {x : a  x  b}, where 0  a  b  1
Probability of event A: P( A)  P({x : a  x  b})  b  a
P ({0.5})  0, P((0.3,0.7])  0.4

Exercise: Show that:

1. P ( A)  1  P( A)
2. P ( A  B )  P ( A)  P ( B)  P ( A  B )
16
Chapter 1. Probability
 1.3 Probability Introduced Through Sets and
Relative Frequency

Probability as a relative frequency

coin toss nH
P ({H })  lim
n n

17
Chapter 1. Probability
 1.4 Joint and Conditional Probability
Joint   probability   P( A  B)  Probability (prob.) that
Events A and B occur simultaneously
Example: S={1,2,3,4,5,6},

2/6

Conditional   probability  of  event   𝐴, given  event   𝐵

P ( A  B)  Prob. that events A occur
P( B)  0  P( A B) 
P( B) given that B occurred
Properties:
1. P( A B )  0 2. P( S B)  1
Axioms of Prob.
3. A  C    P ( A  C B)  P( A B)  P (C B )
4. A  B    P( A B)  0
18
Chapter 1. Probability
 1.4 Joint and Conditional Probability
Ex 1.4-1: A Box contain 100 resistors having resistance and
tolerance as shown:
Total 10% 5%
Define three events 24 14 10 W 22
A draw a 47 W 44 16 28 W 47
B draw a resistor with 5% tolerance 32 8 24 W 100
C draw a 100 W 100 38 62 Total
44 62 32
P ( A)  , P( B)  , P (C ) 
100 100 100
28 P ( A  B ) 0.28 28
P( A  B)  P( A B)   
100 P( B) 0.62 62
P( A  C ) 0
P( A  C )  0 P( A C )   0
P (C ) 0.32
19
Chapter 1. Probability
 1.4 Joint and Conditional Probability
Total   P robability

( )
𝑁
𝐴= 𝐴 ∩ 𝑆= 𝐴 ∩ ∪ 𝐵𝑛
𝑁 𝑛=1

¿ ∪ ( 𝐴 ∩ 𝐵𝑛 )
𝑛=1

mutually exclusive 
N N
P ( A)  P[( A  Bn )]   P( A  Bn )
n 1 n 1

20
Chapter 1. Probability
 1.4 Joint and Conditional Probability
Bayes ′  Theorem P ( Bn A) 
P ( Bn  A) P ( A Bn ) P ( Bn )

P ( A) P ( A)
P ( A Bn ) P ( Bn )
P ( Bn A) 
P ( A B1 ) P ( B1 )    P ( A BN ) P ( BN )
Ex 1.4-2:

B1 =1 before the channel

B2 =0 before the channel

A1 =1 after the channel

A2 =0 after the channel
21
Chapter 1. Probability
 1.4 Joint and Conditional Probability
P ( B1 A1 )  ?
P ( A1 )  P ( A1 B1 ) P( B1 )  P ( A1 B2 ) P ( B2 )  0.9  0.6  0.1 0.4  0.58
P ( B1  A1 ) P ( A1 B1 ) P ( B1 ) 0.9  0.6 54
P ( B1 A1 )    
P ( A1 ) P ( A1 ) 0.58 58
P ( B2 A1 )  ?  1  P ( B1 A1 )

P ( B1 A2 )  ? P ( B2 A2 )  ?

Q. Which statement is true?

or

22
Chapter 1. Probability
 1.5 Independent Events
Def: Two events A & B are (statistically) independent if
P ( A  B )  P ( A) P( B ), P ( A)  0, P ( B )  0
P ( B  A) P ( B ) P ( A)
A & B independent  P ( B A)    P( B)
P ( A) P ( A)
If A & B are independent then A & B independent
Def: 3 events A1 , A2 , & A3 independent 
P ( Ai )  0
P ( A1  A2 )  P ( A1 ) P ( A2 )
P ( A1  A3 )  P ( A1 ) P ( A3 )
P ( A2  A3 )  P ( A2 ) P( A3 )
P ( A1  A2  A3 )  P ( A1 ) P ( A2 ) P( A3 )
23
Chapter 1. Probability
 1.5 Independent Events

Ex: S  {1, 2,3, 4} A1  {1, 2}, A2  {2,3}, A3  {1,3}

P ( Ai  Aj )  P ( Ai ) P ( Aj ), i  j A1 , A2 , & A3 pairwise independent

P ( A1  A2  A3 )  P ( A1 ) P ( A2 ) P ( A3 ) A1 , A2 , & A3 NOT independent

Fact: A1 , A2 , & A3 independent  A1 & (A2  A3 ) independent

Fact: A1 , A2 , & A3 independent  A1 & (A2  A3 ) independent

24
Chapter 1. Probability
 1.6 Combined Experiments
2 independent experiments ( S1 , 1 , P1 ) & ( S2 , 2 , P2 )

Can define a combined probability space ( S , , P)

S  S1  S2  {( s1 , s2 ) : s1  S1 , s2  S2 }

  1   2  { A1  A2 : A1  1 , A2  2 }

P ( A1  A2 )  P1 ( A1 ) P2 ( A2 ), A1  1 , A2  2
P ( A1  S 2 )  P1 ( A1 ) P2 ( S2 )  P1 ( A1 )
25
Chapter 1. Probability
 1.6 Combined Experiments
3 independent experiments ( Si , i , Pi ), i  1, 2,3

Can define a combined probability space ( S , , P)

S  S1  S 2  S3   1   2  3

P ( A1  A2  A3 )  P1 ( A1 ) P2 ( A2 ) P3 ( A3 ), Ai  i

n n!
Permutation P  n(n  1) (n  r  1) 
r
 n  r !
n n!
Combination  r   r! n  r !
    n
 n  r n r
binomial expansion ( x  y )    r  x y
n

r 0  
26
Chapter 1. Probability
 1.7 Bernoulli Trials
Basic experiment - 2 possible outcomes (A or A)
P ( A)  p P ( A)  1  p
Bernoulli Trials - repeat the basic experiment N times
(Assume that elementary events are independent for every trial.)

N k
P ({ A occurs exactly k times})    p (1  p) N k
k 
Ex 1.7-1: P ( A)  0.4 N 3
3 2
P (2 hits)    0.4 (1  0.4)1  0.288
 2
27
Chapter 1. Probability
 1.7 Bernoulli Trials
 3 3
P (3 hits)    0.4 (1  0.4)0  0.064
 3
P ({carrier sunk})  P(2 hits)  P(3 hits)  0.352

Ex 1.7-3: P ( A)  0.4 N  120

 120  50 70 120!  ?
P (50 hits)    0.4 (1  0.4) ?
 50 
large N  De Moivre-Laplace approximation
Poisson approximation

28
Chapter 1. Probability