FAKULTAS TEKNIK SIPIL DAN LINGKUNGAN

SI-5101
Engineering Analysis

Markov Chains
an Introduction
BIEMO W. SOEMARDI
b.soemardi@itb.ac.id

13/09/2022
SI-5101 ANALISIS REKAYASA – Ir. Biemo W. Soemardi Ph.D 1
 What is Markov Models?
▪ Markov model is a special form of probabilistic model
▪ A stochastic model used to model randomly
continuous changing systems where it is assumed that
future states (of variables) depend only on the present
state and not on the consequence of events that
preceded it (Markov property). Andrey Andreyevich Markov
(1856–1922) - Russian mathematician

❖ Markov analysis assumes that the system starts in an initial state or

condition, and changing over times.
❖ Predicting future states involves of knowing the system’s likelihood or
probability of changing from one state to another.

SI-5101 ANALISIS REKAYASA – Ir. Biemo W. Soemardi Ph.D 2

 TODAY TOMORROW DAY AFTER

?
80% no rain tomorrow

20% rain tomorrow

not raining today

60% no rain tomorrow

40% rain tomorrow

raining today

SI-5101 ANALISIS REKAYASA 3

the quality or state of lacking a pattern; unpredictability:

SI-5101 ANALISIS REKAYASA 4

the quality or state of lacking a pattern; unpredictability:

SI-5101 ANALISIS REKAYASA 5

 There are a limited or finite number of states
A fixed set of states
(conditions)
Fixed transition probabilities and
The probability of changing states remains the same
possibility of getting from any state to
over time (memoryless)
another through a series of transition
- Future state can be predicted from previous state
and transition matrix
- What happens in the long run will be completely
Markov process converges to a unique
determined by the transition probabilities
distribution over states
- What happens in the long run will not depend on
where the process started or what happened along
the way
The size and makeup of the system do
not change during the analysis
SI-5101 ANALISIS REKAYASA 6
 0.4

TODAY TOMORROW rain

0.2
80% no rain tomorrow

0.6
20% rain tomorrow
not raining today no rain

60% no rain tomorrow

0.8
40% rain tomorrow no rain
rain
raining today
0.4 0.6 rain
P=
0.2 0.8 no rain

TRANSITION MATRIX
SI-5101 ANALISIS REKAYASA 7
Let Xi be the weather of day i, 1 < i < t
MARKOV property:
Xi+1, the state of the system at time ti+1 depends solely on the state of the system at time t

x1 x2 x3 x4

Probability of Xt+1
P[Xt+1 = xt+1 | X1 …. Xt = x1 … xt] P[Xt+1 = xt+1 | Xt = xt]

P[Xt+1 = b | Xt = a] = Pab Transition probabilities are independent of time (t)

(stationary)
SI-5101 ANALISIS REKAYASA 8
 0.1

0.9 MRT BUS 0.8

0.2

MRT BUS

0.9 0.1 MRT

P=
0.2 0.8 BUS

TRANSITION MATRIX
SI-5101 ANALISIS REKAYASA 9
 BUS ? MRT

0.2 * 0.9 0.2 * 0.8

0.18 + 0.16
P[BUS → ? → MRT] = 0.34

BUS
0.9 0.1 0.9 0.1 0.83 0.17
P2 MRT = =
0.2 0.8 0.2 0.8 𝟎. 𝟑𝟒 0.66
MRT

SI-5101 ANALISIS REKAYASA 10

 MRT ? ? BUS

0.9 0.1 0.83 0.17 0.781 𝟎. 𝟐𝟏𝟗

P3BUS = =
0.2 0.8 0.34 0.66 0.438 0.562

SI-5101 ANALISIS REKAYASA 11

 0.9 0.1 0.781 0.219
P= P3 =
0.2 0.8 0.438 0.562

P [X3 = MRT] = 60% * 0.781 + 40% * 0.438 = 0.6438

P [X2 = MRT] = 60% * 0.830 + 40% * 0.340 = 0.6438

Week-1 Week-2 Week-3

MRT 60% 63.4% 64%
Proportion t = Proportion initial * Pn BUS 40% 36.6% 36%

SI-5101 ANALISIS REKAYASA 12

 Proportion t = Proportion initial * Pn

Initial Stage x Transition Matrix = Initial State

0.67
0.1
0.66
0.9 MRT BUS 0.8
0.65
0.2
P[Xi = MRT]
0.90 0.10
𝟎. 𝟔𝟕 𝟎. 𝟑𝟑 = [0.67 0.33]
0.20 0.80

stationary distribution
steady state
0.6
week - i
SI-5101 ANALISIS REKAYASA 13
SI-5101 ANALISIS REKAYASA 14
Tugas #2
• Pada sebuah fasilitas produksi instrument pengukuran dikenal tiga kondisi produksi
(rendah, menengah dan tinggi). Kondisi tersebut berpengaruh kepada kondisi produksi
pada jadwal produksi berikutnya dengan ketentuan sebagai berikut:

Kebolehjadian (%)
Kondisi Kondisi Kapasitas (unit per-hari)
Rendah Sedang Tinggi
Rendah 60 30 10 Rendah 100 - 150
Sedang 30 45 25 Sedang 151 - 250
Tinggi 10 35 55 Tinggi > 250

• Jika saat ini jumlah produksi adalah sebesar 200 unit per-hari, berapa keboleh jadian
kapasitas produksi pada jadwal produksi ke 6 mencapai 300 unit per-hari?
• Pada kondisi langgeng berpa proporsi kebolehjadian ketiga kondisi produksi tersebut?
• Tugas dikumpulkan dengan nama SI-5101-T3-NIM dan dikirimkan via Ms Team dan
b.soemardi@itb.ac.id paling lambat Jumat, 16 September 2022 pkl 10:00 WIB
15
SI-5101 ANALISIS REKAYASA 16