CHAPTER 1
Probability Models in Electrical and
Computer Engineering
INHA University INHA University http://multinet.inha.ac.kr http://multinet.inha.ac.kr
Prof. Sang Prof. Sang Prof. Sang Prof. Sang  Jo Yoo Jo Yoo Jo Yoo Jo Yoo
sjyoo@inha.ac.kr sjyoo@inha.ac.kr sjyoo@inha.ac.kr sjyoo@inha.ac.kr
http://multinet.inha.ac.kr http://multinet.inha.ac.kr http://multinet.inha.ac.kr http://multinet.inha.ac.kr
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Course Info.
Course objective
To learn basics of probability and random processes
Text Book
A. LeonGarcia, Probability, Statistics, and Random Processes for Electrical
Engineering, 3
rd
Ed.
Instructor
INHA University INHA University
2
Instructor
Prof. SangJo Yoo (Office: HiTech 316, Tel: 8608304, Email: sjyoo@inha.ac.kr)
Office hours: by appointment (Telephone or Email)
Course Home Page:
http://multinet.inha.ac.kr/
Homework:
Assignment are issued weekly and due at the beginning of class one week after
they are assigned
Late/copied assignments will never be accepted.
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Course Info.
Exams:
2 exams1 midterm and 1 final exam
Closed book
No makeups for missed exams (Any request for exceptions should be made in
advance)
Grading:
Attendant: 10%
INHA University INHA University
3
Homework: 10%
Midterm: 40%
Final exam: 40%
eClass:
Lecture notes and homework will be posted on eClass
Encourage the use of Q & A board on eClass web site
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Course Objectives
In this course we will study basics of probability and random
processes:
Probability models in electrical engineering
Basic concepts of probability theory
Random variables
INHA University INHA University
4
Multiple random variables
Random processes
Analysis and processing of random signals
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Models in Analysis and Design
Need to build systems that work in a chaotic environment
Communication systems that must provide continuous and errorfree
communication.
Computer system that must satisfy the diverse processing demands.
Models as tools in analysis and design
Help the marking of choices from various feasible alternatives.
INHA University INHA University
5
Help the marking of choices from various feasible alternatives.
Actual implementation and experimental evaluation of the alternatives are
too costly.
Models can be used to make decisions.
Probability models enable the designers to successfully build systems that
are efficient, reliable, and costeffective.
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Models in Analysis and Design
INHA University INHA University
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Mathematical Models
Model
An approximate representation of a physical system.
Attempts to explain observed behavior using a set of simple and
understandable rules.
Mathematical models
Used when the observational phenomenon has measurable properties.
Consists of a set of assumption about how a system works.
INHA University INHA University
7
Consists of a set of assumption about how a system works.
Stated in the form of mathematical relations involving parameters and
variable of the system.
How we can validate the mathematical model many real observations?
Computer simulation model
Consists of computer program that simulates the system.
A means of checking the predictions made by a mathematical model.
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Types of Models
Deterministic model
The conditions under which an experiment is carried out determine the
exact outcome of the experiment.
The solution of a set of mathematical equations specifies the exact
outcome of the experiment (e.g., circuit theory).
Probability model
INHA University INHA University
8
Probability model
For systems involving unpredictable variation and randomness.
Random experiment: an experiment in which the outcome varies in an
unpredictable fashion when the experiment is repeated under the same
conditions.
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Example: Electrical Network
1. Represent the given circuit using
systems of linear equations
2. Find the solution
1 I 3 I
2 I
INHA University INHA University
9
V RI
I
I
I
I I I I I I I I
I I I I I
I I I I I
=
(
(
(
=
(
(
(
(
(
(
= + = + +
= = +
= = + +
0
6
6
21 3 6
3 12 0
6 0 10
0 21 3 6 0 ) ( 3 12 ) ( 6
6 3 12 0 6 ) ( 3 9
6 6 10 0 6 ) ( 6 4
3
2
1
3 2 1 2 3 3 1 3
3 2 3 2 2
3 1 3 1 1
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Probability Model
Random experiment
An experiment where the outcome varies in an unpredictable fashion
when the experiment is repeated under the same conditions.
(i) It is performed according to some set of rules,
(ii) It can be repeated arbitrarily often,
(iii) The result of each performance depends on chance and cannot be
INHA University INHA University
10
(iii) The result of each performance depends on chance and cannot be
predicted uniquely.
Sequential random experiments performing a sequence of simple
random subexperiments
E.g., First toss a coin, then throw a dice.
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Probability Model
Example of a random experiment
A ball is selected from an urn containing three identical balls,
labeled 0, 1, and 2
Sample space S: The set of all possible outcomes
In this example, S ={0, 1, 2}
The outcome of the experiment is random.
4
INHA University INHA University
11
0 10 20 30 40 50 60 70 80 90 100
2
1
0
1
2
3
4
Trial number
O
u
t
c
o
m
e
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Statistical Regularity
A model must enable us to make predictions about the future
behavior of a system.
Statistical regularity
Average obtained in long sequences of repetitions of random
experiments yield approximately the same value.
INHA University INHA University
12
Example
Suppose the urn experiment is repeated n times; k=0,1,2
Relative frequency
Probability of the outcome k
n
n N
n f
k
k
) (
) ( =
k k
n
P n f =
) (
lim
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Example: Relative Frequency
INHA University INHA University
13
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Properties of Relative Frequency
The number of occurrences of an outcome
The relative frequencies
0 ( ) 1 k f n
0 ( )
k
N n n for 1, 2,..., k K =
for 1, 2,..., k K =
INHA University INHA University
14
The sum of all the relative frequencies
Event C = A or B occurs (cannot occur simultaneously)

\

= =
= =
K K
1 1
) ( 1 ) (
k
k
k
k n n N n f Q
( ) ( ) ( )
c a b
f n f n f n = +
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Axioms of Probability
Relative frequency: problems
Not clear when and it what mathematical sense the limit exist.
We cannot perform an experiment an infinite number of times.
Probability of an event
1.
2.
0 ( ) 1 P A
[ ] 1 P S =
INHA University INHA University
2.
3. If A and B are events that cannot occur simultaneously,
then
[ ] 1 P S =
[ or ] [ ] [ ] P A B P A P B = +
S
A B
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Building A Probability Model
Procedures for building a probability model
Define the random experiment inherent in the application
Specify the sample space S of all possible outcomes and events of
interest.
Specify a probability assignment from which the probability of all events
of interest can be computed
INHA University INHA University
Example
Test a telephone conversation to determine whether a speaker is
currently speaking or not.
On the average, P
active
=1/3, P
silent
=2/3
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Example: Telephone Conversation
On the average, P
active
=1/3, P
silent
=2/3
Physical situation
What is the probability that more than 24 speakers out of 48 independent speakers
are active at the same time?
Can we use a probability model?
Assumptions
All speakers are the same
All languages have the same silenceactivity behavior
INHA University INHA University
All languages have the same silenceactivity behavior
Build a probability model
Use an urn experiment
Active period black ball, silent period white ball
Select a ball from an urn containing two white balls and one black ball
(P
active
=1/3, P
silent
=2/3 )
What is the probability that more than 24 black balls are selected in 48
independent repetitions of the urn experiment?
http://multinet.inha.ac.kr
Multimedia Network Lab Multimedia Network Lab. .. .. .. .
Example: Probability Model
Binary communication channel
Every T seconds, the transmitter accepts a binary input (0 or 1) and
transmits a signal
The receiver makes a decision for the receiver signal
Transmitter errors may occur with probability
INHA University INHA University