Stochastic Systems

Lecture 𝟏𝒂
Dr. Hafsa Iqbal
Department of Electrical Engineering,
School of Electrical Engineering and Computer Science,
National University of Sciences and Technology,
Pakistan
About Me
• Academics:
• BSc in Electrical Engineering, UET Taxila

• MS in Electrical Engineering from CEME, NUST

• PhD in Autonomous Systems from Carlo III University of Madrid, Spain

and University of Genoa, Italy

• Research Areas
• Self-driving vehicle

• Dynamic Bayesian Networks

• Autonomous Systems
2
About Me
• Major Achievements:
• Gold medal in MS

• Awarded with the title of Excellent research degree (PhD)

• Scholarship for BS and MS

• EU project funding for the PhD studies

• Reviewer of the ‘IEEE Transaction of Intelligent Transportation

System’

• Reviewer of the flagship conferences of ITS society

3
About Me
• Experience:
• Research Assistant at CARE, Islamabad

• Lab Assistant at University of Genoa, Italy

• Assistant Professor at SEECS, NUST

4
Ice Breaking!!!

• Introduce Yourself
• Name
• Institute name of your BSc

5
Code of Ethics
• Attendance timings
• Late arrival in class
• Cell phone ringing
• Respect peer, faculty and staff through actions and
speech
• Students should not sleep during class
• Should not disturb your fellows
• Assignment submissions
• We will follow the NUST policy on plagiarism

6
Course Material

Enrollment Code: 𝟒𝟖𝟏𝟑𝟕𝟔𝟎𝟓𝟐

• Textbook
• Probability and Random Processes for Electrical Engineering, by

Alberto Leon-Garcia.

• Probabilistic Graphical Models: Principle and Techniques by

Daphne Koller and Nir Friedman

7
Course Material
• Reference book:
• Probability and Stochastic Processes – A Friendly Introduction
for Electrical and Computer Engineers, by Roy D. Yates and
David J. Goodman
• Stanford University’s Lectures on Coursera ‘Probabilistic
Graphical Models’
• MIT OCW ‘Probabilistic Systems Analysis and Applied
Probability’.

• Lecture Notes

8
Course Assessment
Exam: 2 OHTs and 1 Final
Home work: 2 Assignments (min)
Design reports: 1 Design report based on Semester Project
Quizzes: 3 Quizzes (min)
Quizzes: 7.5%
Assignments: 7.5%
2 One Hour Tests 30%
Grading:
Final Exam: 45%
Semester Project: 10%

9
Course Outlines

• Stochastic systems

• Probabilistic Graphical Model

• Representation
• Inference
• Learning

10
What is Stochastic System???

11
Stochastic Systems

• Stochastic means Pertaining to Chance

• Stochastic system: Anything random that evolves in time

• Stochastic Systems is an area of systems theory that deals

with dynamic as well as static systems, which can be
characterized by stochastic processes (random processes)

• Now, What is Random Process??

12
Stochastic Systems
• Random Process
• Indexed (time or space) family of random variables
• Collection of random variables

• Now, Random Variable?

• Whose possible values are numerical outcomes of a random
phenomenon
• Which can take on a set of possible different values (similarly
to other mathematical variables), each with an associated
probability

13
Mathematical Models
• A Model is an approximate representation of a physical
situation
• Used instead of experiments
• Mathematical Models are used when the observational
phenomenon has measurable properties
• It is a description of a system using mathematical
concepts and language (equations)
• Based on certain assumptions
14
Deterministic Models
• Provides Exact outcome of the experiment

• Solution of a set of mathematical equations gives the

exact outcome of the experiment

• Consistency in the outcomes under similar conditions

15
Probability (Stochastic) Models
• Outcome of the experiment is Unpredictable under
similar conditions
• Deterministic models cannot be used for random
experiments
• Terminologies
• Experiment
• Outcome
• Sample Space

16
Probability (Stochastic) Models
• Terminologies
• Events
• Probability
N𝑘 (𝑛)
• Relative Frequency 𝑓𝑘 𝑛 =
n

• Statistical Regularity
• Probability for equiprobable outcomes

17
Probability (Stochastic) Models

• Relative frequencies are a number b/w ZERO and ONE

0 ≤ 𝑓𝑘 (𝑛) ≤ 1

• Sum of all the relative frequencies equals ONE

𝑓𝑘 (𝑛) = 1
• Occurrence of any outcome in n trials is a number b/w
ZERO and n
0 ≤ N𝑘 (𝑛) ≤ 𝑛

18
Probability (Stochastic) Models
• Developing a Probability model uses axioms
• Define the random experiment inherent in the
application(experiment)
• Specify the set S of all possible outcomes and the
events of interest(sample space and events)
• Specify a probability assignment from which the
probabilities of all events of interest can be computed
• Assumptions required

19
Axioms of Probability
• Probability law is a function that assigns a number to
events
• A Probability law for the experiment E is a rule that
assigns to each event A, a number P[A], called Probability
of A, satisfying
• 0≤𝑃 𝐴 ≤1
• 𝑃 𝑆 =1
• If 𝐴 𝐵 = ∅, then 𝑃 𝐴 𝐵 =𝑃 𝐴 +𝑃 𝐵

20
Corollary

• 𝑃 𝐴𝑐 = 1 − 𝑃 𝐴

•0≤𝑃 𝐴 ≤1

•𝑃 ∅ =0

• 𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵

• 𝐴 ⊂ 𝐵 then 𝑃 𝐴 ≤ 𝑃 𝐵

21
Random Experiment
• An experiment in which the outcome varies in an
unpredictable manner
• It is specified by stating an experimental procedure and a
set of one or more measurements or observations
• Select a ball from an urn containing balls numbered 1 to 50.
Note the number of the ball
• Toss a coin three times and note the number of heads
• Pick two numbers at random between zero and one.

22
Sample Space
• Types of Sample Space (SS)
• Finite (Discrete)
• Countably infinite (Discrete)
• Uncountably infinite (Continuous)
• SS can be multi-dimensional
• SS can be written as Cartesian product of other sets
• Event is a Subset of SS

23
Sample Space
• Event mostly contains multiple outcomes
• An event with single outcome is called Elementary Event
• Certain Event: Always occurs
• Null Event: Never occurs
• Set Theory Terminologies:
• Union (or) /Intersection (and)
• Mutually Exclusive (when intersection is null event)
• Implies (Subset)
• DeMorgan’s Rules 𝑨 ∪ 𝑩 ′ = 𝑨′ ∩ 𝑩′ and 𝑨 ∩ 𝑩 ′ = 𝑨′ ∪ 𝑩′
24
Discrete Sample Space
• Probability is specified by giving the probabilities of the
elementary events
• Equally likely outcomes (1/n)

• Example: Urn with 10 identical balls numbered 0 to 9

• Example: Three times coin toss

• Example: Rolling dice

25
Continuous Sample Space
• Probability law in experiments with continuous sample
spaces specify a rule for assigning numbers to intervals of
the real line and rectangular regions in the plane
• Example: Picking a random number inside an interval
• Example: Lifetime of a computer memory chip
• How can an elementary event have zero probability?

26
Conditional Probability
• Probability of event A given that event B has occurred is
termed as Conditional Probability P[A|B]
𝑃[𝐴 𝐵]
𝑃[𝐴|𝐵] =
𝑃[𝐵]
• SS is reduced to event B
• Normalization of probability of events that occur jointly
with B, i.e. 𝑃[𝐴 𝐵]
• Satisfies axioms of probability
27
Conditional Probability
• Example: A ball is selected from an urn containing two
black balls, numbered 1 and 2, and two white balls,
numbered 3 and 4. The number and color of the ball is
noted, so the sample space is
{(1, 𝑏), (2, 𝑏), (3, 𝑤), (4, 𝑤)} Assuming that the four
outcomes are equally likely, find 𝑃(𝐴|𝐵) and
𝑃(𝐴|𝐶) where A, B, and C are the following events:
• 𝐴 = {(1, 𝑏), (2, 𝑏)} “black ball selected,”
• 𝐵 = {(2, 𝑏), (4, 𝑤)} “even-numbered ball selected,” and
• 𝐶 = {(3, 𝑤), (4, 𝑤)} “number of ball is greater than 2.”

28
Tree Diagram Concept
• Conditional Probability is useful for finding the probability
in sequential experiments

29
Tree Diagram Concept
• Example: An urn contains two black balls and three white
balls. Two balls are selected at random from the urn
without replacement and the sequence of colors is noted.
Find the probability that both balls are black.

• Example: Localization of a robot

30
 Conditional Probability
• Theorem on Total Probability: Probability of an event A
can be defined in terms of the intersection of A with the
mutually exclusive events that form the complete SS

𝑃 𝐴 = 𝑃 𝐴 𝐵1 𝑃 𝐵1 + 𝑃 𝐴 𝐵2 𝑃 𝐵2 + ⋯ + 𝑃 𝐴 𝐵𝑛 𝑃 𝐵𝑛

• Useful for sequential random experiment (tree diagram)

31
Bayes’ Rule
• Let 𝐵1 , 𝐵2 , … , 𝐵𝑛 be mutually exclusive events that form the
complete SS, Given an event A is occurred then

𝑃[𝐴 𝐵𝑗 ] 𝑃 𝐴 𝐵𝑗 𝑃[𝐵𝑗 ]
𝑃[𝐵𝑗 |𝐴] = = 𝑛
𝑃[𝐴] 𝑘=1 𝑃 𝐴 𝐵𝑘 𝑃[𝐵𝑘 ]

is given by Bayes’ rule

• Useful for cases where ‘a-priori’ probabilities of events (that
form the SS) are known and we have to find the ‘a-posteriori’
probabilities of these event(s) after the occurrence of event A

32
Bayes’ Rule
• Example: what is the probability of two girls given at
least one girl?

P (at least 1 girl | 2girls) P(2girls)

P(2 girls | at least 1 girl) =
P (at least 1 girl)
Solution:

33
Independence of Events
• Events A and B are said to be independent, if the knowledge
of occurrence of A doesn’t alter the probability of event B
• In this case, P[A] = P[A|B]
• Mutually exclusive vs independent?
• A General Rule: If two events have non-zero probabilities
and are mutually exclusive, they cannot be independent
• Example: Selecting two numbers from a square region
(example 2.32)

34
Independence of Events
• Example: A ball is selected from an urn containing two
black balls, numbered 1 and 2, and two white balls,
numbered 3 and 4. The number and color of the ball is
noted, so the sample space is
{(1, 𝑏), (2, 𝑏), (3, 𝑤), (4, 𝑤)} . Are events A and B
independent? Are events A and C independent?
where A, B, and C are the following events:
• 𝐴 = {(1, 𝑏), (2, 𝑏)} “black ball selected,”
• 𝐵 = {(2, 𝑏), (4, 𝑤)} “even-numbered ball selected,” and
• 𝐶 = {(3, 𝑤), (4, 𝑤)} “number of ball is greater than 2.”
1
we already knows that 𝑃 𝐴 𝐵 = , 𝑃 𝐴 𝐶 = 0
2
35