MATH 3800

MATHEMATICAL MODELS AND NUMERICAL METHODS

DISCRETE PROBABILISTIC MODELING

© 2020 Gary Bazdell (excluding images)

EXAMPLE
Consider a town where the residents only purchase two brands of
cereals:
𝐶 − Cinnamon Toast Crunch
𝐹 − Froot Loops
0.2

0.8 𝐶 𝐹 0.7

0.3

Discrete Probabilistic Modeling - Introduction 2

INTRODUCTION
A Markov chain is a process consisting of a sequence of events with the
following properties:
1. An event has a finite number of outcomes, called states. The
process is always in one of these states.
2. At each stage or period of the process, a particular outcome can
transition from its present state to any other state or itself.
3. The probability of going from one state to another in a single stage
is represented by a transition matrix for which entries in each row
are between 0 and 1; each column sums to 1.

Discrete Probabilistic Modeling - Introduction 3

EXAMPLE
We can represent this example by the transition matrix
𝐶 𝐹
0.8 0.3 𝐶
𝑇=
0.2 0.7 𝐹

Terminology
• A vector where the entries are non-negative and sum to 1 is called a
probability vector.
• A square matrix whose columns are probability vectors is called a
stochastic matrix.

Discrete Probabilistic Modeling - Introduction 4

MARKOV CHAIN
A sequence of probability vectors 𝑥0 , 𝑥1 , 𝑥2 , … together with a
stochastic matrix 𝑇 such that

𝑥𝑗+1 = 𝑇𝑥𝑗

(𝑗 ≥ 0) is called a Markov Chain.

Note: In a Markov chain, the future depends only upon the present:
Not upon the past.

Discrete Probabilistic Modeling - Markov Chain 5

EXAMPLE
Let’s say that 65% of cereal eaters eat Cinnamon Toast Crunch and
35% eat Froot Loops. We can represent this by a state vector

0.65
𝑥0 =
0.35

What will the percentages be next year? And the year after?

Discrete Probabilistic Modeling - Introduction 6

STEADY STATE (EQUILIBRIUM)
If
lim 𝑥𝑛 = 𝑞
𝑛→∞

𝑇𝑞 = 𝑞

Then the steady state or equilibrium 𝑞 is achieved and the system

stays there.

How do we find 𝑞?

Discrete Probabilistic Modeling - Markov Chain 7

EIGENVECTOR AND EIGENVALUES
Since 𝑇𝑞 = 𝑞, this means that 𝑞 is an eigenvector of 𝑇 corresponding
to the eigenvalue of 1.

𝑇𝑞 = 𝑞
𝑇𝑞 − 𝑞 = 0
𝑇−𝐼 𝑞 =0

NOTE There are infinitely many solutions (since any multiple of and
eigenvector is still an eigenvector) but only 1 of them is a probability
vector.

Discrete Probabilistic Modeling - Markov Chain 8

IMPORTANT FACTS
Does 𝑞 depend on the initial stat 𝑥0 ?

1. no, 𝑞 is independent of 𝑥0 , 𝑞 is determined by 𝑇.

2. What does depend on 𝑥0 is how long it takes 𝑞 to be reached.
1. If 𝑥0 is close to 𝑞, 𝑞 is reached quickly.
2. If 𝑥0 is far from 𝑞, it will take longer to reach 𝑞.

Discrete Probabilistic Modeling - Markov Chain 9

EXERCISE 1
Find the transition matrix and equilibrium solution of the following
0.15

0.10
0.75 𝐴 𝐵 0.80

0.10 0.10

0.20 0.10

0.70
Discrete Probabilistic Modeling - Markov Chain 10
 MATH 3800
MATHEMATICAL MODELS AND NUMERICAL METHODS

MODELING COMPONENT AND SYSTEM RELIABILITY

© 2020 Gary Bazdell (excluding images)

MODELING COMPONENT AND SYSTEM RELIABILITY
The reliability of a component or system is the probability that it will
not fail over a specific time period 𝑛.

We define 𝑓 𝑡 to be the failure rate of an item, component, or system

over time 𝑡, so 𝑓 𝑡 is a probability distribution.

We define the reliability of the item, component, or system by

𝑅 𝑡 =1−𝑓 𝑡

Discrete Probabilistic Modeling - System Reliability 12

PROBABILITY RULES
𝑃 𝐴 or 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴 and 𝐵

where 𝑃(𝐴 and 𝐵) denotes the probability that 𝐴 and 𝐵 both occur at
the same time as an outcome in a trail of a procedure.

𝑃 𝐴 and 𝐵 = 𝑃 𝐴 𝑃(𝐵)

when 𝐴 and 𝐵 are independent by assumption.

Discrete Probabilistic Modeling - System Reliability 13

SERIES SYSTEMS
A series system is one that performs well as long as all the components
are fully functional.

𝑅1 𝑡 = 0.9 𝑅2 𝑡 = 0.95 𝑅3 𝑡 = 0.96

If the reliabilities of the three components are given, then the system
reliability is defined to be the product
𝑅𝑠 = 𝑅1 𝑡 𝑅2 𝑡 𝑅3 𝑡

Discrete Probabilistic Modeling - System Reliability 14

PARALLEL SYSTEMS
A parallel systems is one that performs as long as a single one of its
components remains operational.
𝑅1 𝑡 = 0.95

𝑅2 𝑡 = 0.96

If the components are independent, then we define the system

reliability to be
𝑅𝑠 𝑡 = 𝑅1 𝑡 + 𝑅2 𝑡 − 𝑅1 𝑡 𝑅2 𝑡

Discrete Probabilistic Modeling - System Reliability 15

EXERCISE 2
Find the reliability of the following system.

0.92 0.99

0.94 0.95

0.95 0.97

Discrete Probabilistic Modeling - System Reliability 16