2 views

Uploaded by Nazar Aziz

go head for upload

- Stochastics Tutorial
- Game Theory
- Cashflow Analysis Problems With Solution
- 5-Markoc Chain Monte Carlo Simulation
- UT Dallas Syllabus for cs3341.501 05f taught by Pankaj Choudhary (pkc022000)
- When do diffusion-limited trajectories become memoryless?
- 120
- UpdateJan08
- An Analysis of Telecommunication Queuing System and Making More Efficient With Empirical Distribution of Service Time and Inter-Arrival Time (F-I-F-O Service)
- A Study on Piezoelectric Elements and Its Utility in Designing of Electronic Scale
- FE_Ch01 Wiener Process
- TR134_Thrainsson.pdf
- Food Template
- StationaryStochasticProcesses.pdf
- c16 Queue Models (1)
- Lecture9_6up
- Stochastic Processes - 1 - - Unit 3 - Week 2
- syl_msc_stats.pdf
- emcshane
- n,es

You are on page 1of 51

Theorem

Assoc. Prof. Dr. Rashid A. Saeed

MSC Comp and Comm

Elex, FoE, SUST,

Agenda

Memoryless Information

Processes

Markov Processes and n-gram

Models

D

r.

R

a

s

hi

d

A

. 2

Description

Sometimes we are interested in how a random

variable changes over time.

The study of how a random variable evolves

over time includes stochastic processes.

An explanation of stochastic processes – in

particular, a type of stochastic process known

as a Markov chain is included.

We begin by defining the concept of a

stochastic process.

A continuous –time Markov Chain (CTMC)

A Discrete –time Markov Chain (DTMC)

3

Dr. Rashid A. Saeed

What is a Markov Chain?

One special type of discrete-time is called a Markov

Chain.

Definition: A discrete-time stochastic process is a

Markov chain if, for t = 0,1,2… and all states

P(Xt+1 = it+1|Xt = it, Xt-1=it-1,…,X1=i1, X0=i0)

=P(Xt+1=it+1|Xt = it)

Essentially this says that the probability distribution of

the state at time t+1 depends on the state at time t(it)

and does not depend on the states the chain passed

through on the way to it at time t.

4

Dr. Rashid A. Saeed

In our study of Markov chains, we make further

assumption that for all states i and j and all t,

P(Xt+1 = j|Xt = i) is independent of t.

This assumption allows us to write P(Xt+1 = j|Xt

= i) = pij where pij is the probability that given

the system is in state i at time t, it will be in a

state j at time t+1.

If the system moves from state i during one

period to state j during the next period, we call

5

that a transition from i to j has occurred.

Dr. Rashid A. Saeed

We call the vector q= [q1, q2,…qs] the initial probability

distribution for the Markov chain.

In most applications, the transition probabilities are displayed

as an s x s transition probability matrix P. The transition

probability matrix P may be written as

p p22 p2 s

P 21

ps1 ps 2 pss

6

Dr. Rashid A. Saeed

For each I

j s

p

j 1

ij 1

nonnegative.

Hence, all entries in the transition probability matrix are

nonnegative, and the entries in each row must sum to 1.

7

Dr. Rashid A. Saeed

Markov processes

ashid A. Saeed

8

The Gambler’s Ruin Problem

At time 0, I have $2. At times 1, 2, …, I play a game

in which I bet $1, with probabilities p, I win the

game, and with probability 1 – p, I lose the game.

My goal is to increase my capital to $4, and as soon

as I do, the game is over. The game is also over if my

capital is reduced to 0.

Let Xt represent my capital position after the time

t game (if any) is played

X0, X1, X2, …. May be viewed as a discrete-time

stochastic process

9

Dr. Rashid A. Saeed

The Gambler’s Ruin Problem

$0 $1 $2 $3 $4

1 0 0 0 0

1 p 0 p 0 0

P = 0 1 p 0 p 0

0 0 1 p 0 p

0 0 0 0 1

The Cola Example

Suppose the entire cola industry produces only two colas.

Given that a person last purchased cola 1, there is a 90%

chance that their next purchase will be cola 1.

Given that a person last purchased cola 2, there is an 80%

chance that their next purchase will be cola 2.

1. If a person is currently a cola 2 purchaser, what is the

probability that they will purchase cola 1 two purchases from

now?

2. If a person is currently a cola 1 a purchaser, what is the

probability that they will purchase cola 1 three purchases

from now?

11

Dr. Rashid A. Saeed

20%

2 1 90%

80%

The Cola Example

10%

the state at any given time being the type of cola the

person last purchased.

Hence, each person’s cola purchases may be represented

by a two-state Markov chain, where

State 1 = person has last purchased cola 1

State 2 = person has last purchased cola 2

If we define Xn to be the type of cola purchased by a

person on her nth future cola purchase, then X0, X1, …

may be described as the Markov chain with the following

transition matrix: X2 X1

X0

12 90%

Dr. Rashid A. Saeed

The Cola Example X2 X1

X0

90%

Cola1 Cola 2

Cola 1 .90 .10

P

Cola 2 .20 .80

1. We seek P(X2 = 1|X0 = 2) = P21(2) = element 21 of P2:

P

2

.20 .80 .20 .80 .34 .66

13

Dr. Rashid A. Saeed

Markov Processes

A Markov source consists of:

an alphabet A

a set of states ∑,

a set of transitions between states,

a set of labels for the transitions and

two sets of probabilities.

14

Diagrammatic representation of a

Markov source

15

16

Markov chain in Network queuing

Markov Chain for M/M/1 system

17

Birth-death chain

– Arrival rate of λ packets per second

– Over a small interval δ,

18

Markov model of a scalar passage of

music.

19

Find mth-order Markov model?

21

A Markov source equivalent to a

3-gram model.

22

Ch 05: Little’s Theorem and

M/M/1

ELEXM 621 Random Process and Queuing

Theorem

Assoc. Prof. Dr. Rashid A. Saeed

MSC Comp and Comm

Elex, FoE, SUST,

Agenda

Introduction

Queuing systems

Categories, Kendall notation

Markovian queuing systems

Little’s result

M/M/1

24

Dr. Rashid A. Saeed

What is queuing theory?

Queuing theory: performance evaluation of

resource sharing systems, specifically, for

teletraffic systems

25

Dr. Rashid A. Saeed

Performance of queuing systems

26

Dr. Rashid A. Saeed

Block diagram of a queuing system

27

Dr. Rashid A. Saeed

Description of queuing systems

28

Dr. Rashid A. Saeed

Examples in details

29

Dr. Rashid A. Saeed

Examples in details

30

Dr. Rashid A. Saeed

Performance measures

31

Dr. Rashid A. Saeed

Poisson process

32

Dr. Rashid A. Saeed

Poisson process

33

Dr. Rashid A. Saeed

Exponential distribution and

memoryless property

34

Dr. Rashid A. Saeed

Poisson process and exponential

distribution

35

Dr. Rashid A. Saeed

Kendall’s notation A/S/m/c/p/O

36

Dr. Rashid A. Saeed

Notation Example

M/M/3/20/1500/FCFS – single queue system with:

Exponentially distributed arrivals

Exponentially distributed service times

Three servers

Capacity 20 (queue size is 20 – 3 = 17)

Population is 1500 total

Service discipline is FCFS

Often, assume infinite queue and infinite population and

FCFS, so just M/M/3

37

Dr. Rashid A. Saeed

Markovian Queuing Systems

38

Dr. Rashid A. Saeed

System variables

39

Dr. Rashid A. Saeed

Little’s result

40

Dr. Rashid A. Saeed

Example

41

Dr. Rashid A. Saeed

42

Dr. Rashid A. Saeed

43

Dr. Rashid A. Saeed

44

Dr. Rashid A. Saeed

45

Dr. Rashid A. Saeed

46

Dr. Rashid A. Saeed

47

Dr. Rashid A. Saeed

48

Dr. Rashid A. Saeed

49

Dr. Rashid A. Saeed

M/M/1 queuing systems

50

Dr. Rashid A. Saeed

Thank You

51

Dr. Rashid A. Saeed

- Stochastics TutorialUploaded byBrofessor Paul Nguyen
- Game TheoryUploaded bySaddam Shan
- Cashflow Analysis Problems With SolutionUploaded bySampaioVeiga
- 5-Markoc Chain Monte Carlo SimulationUploaded byrosethou
- UT Dallas Syllabus for cs3341.501 05f taught by Pankaj Choudhary (pkc022000)Uploaded byUT Dallas Provost's Technology Group
- When do diffusion-limited trajectories become memoryless?Uploaded byMaciej Dobrzyński
- 120Uploaded byebenesarb
- UpdateJan08Uploaded byapi-27370939
- An Analysis of Telecommunication Queuing System and Making More Efficient With Empirical Distribution of Service Time and Inter-Arrival Time (F-I-F-O Service)Uploaded byIJSTE
- A Study on Piezoelectric Elements and Its Utility in Designing of Electronic ScaleUploaded byAlexander Decker
- FE_Ch01 Wiener ProcessUploaded byMario Zamora
- TR134_Thrainsson.pdfUploaded byAnonymous dkML2wzU
- Food TemplateUploaded byAubrey Holt
- StationaryStochasticProcesses.pdfUploaded byMike Hawk
- c16 Queue Models (1)Uploaded byMangal Gupta
- Lecture9_6upUploaded byharris
- Stochastic Processes - 1 - - Unit 3 - Week 2Uploaded byAnonymous oVmxT9Kzrb
- syl_msc_stats.pdfUploaded byAmit Kumar
- emcshaneUploaded byAl Azhary Masta
- n,esUploaded byPeishi Ong
- Stationary MrkovUploaded byShoba Devulapalli
- QUEUING THEORY.docxUploaded byRishita Jaggi
- Multi-agent Multi-objective Planning for Fair Distribution of ResourcesUploaded byAnna Kosiorek
- examDSF2011Uploaded bymuralimovies
- 2692221Uploaded byahmed22gouda22
- TIME-ABSTRACTING BISIMULATION FOR MARKOVIAN TIMED AUTOMATAUploaded byAnonymous rVWvjCRLG
- Analysis of Cognitive Radio Networks With ImperfectUploaded byss_18
- حلول بروبليم لاحتمال شابتر 4Uploaded byAhmad Y. Ghuson
- VAJ000547Uploaded bysanthitry
- part9Uploaded byElvin Digno Dulce

- Editable Digital Signal Processing Principles Algorithms and Applications Third EditionUploaded byAndrei Serghie
- Lab3Uploaded byNazar Aziz
- Cooperative Diversity in Wireless NetworksUploaded byNazar Aziz
- ZigZag Decoding, Combating Hidden Terminals in Wireless.pdfUploaded byNazar Aziz
- A Study of Selected Security IssuesUploaded byNazar Aziz
- Wireless Lan Security (Ieee 802.11b)Uploaded byAmadasun Bigyouth Osayi
- Decoupling Congestion Control and Bandwidth Allocation PolicyWith.pdfUploaded byNazar Aziz
- Boosting Rate Adaptation InUploaded byNazar Aziz
- (2016)System Design of Internet-Of-Things for ResidentialUploaded byNazar Aziz
- Real-Time Communications Over HybridUploaded byNazar Aziz
- A Software Architecture for Physical Layer Wireless Network EmulationUploaded byNazar Aziz
- Increasing the Robustness of Networked Systems.pdfUploaded byNazar Aziz
- Performance Issues in Mobile Wireless NetworksUploaded byNazar Aziz
- Goldsmith MSthesisUploaded byMirza Bilal
- A Wireless Protocol to Prevent Wormhole AttacksUploaded byNazar Aziz
- Adaptive Quality of Service for Wireless Ad-hoc NetworksUploaded byNazar Aziz
- A Virtualization Architecture ForUploaded byNazar Aziz
- Ch 05 - Markov ModelUploaded byNazar Aziz
- introductiontoradarsystems-merrillskolnikiiirded-140506103641-phpapp02.pdfUploaded byNazar Aziz
- ConductingLiteratureReview(3)Uploaded bytsagay
- Lecture 5 sinal space analysis.pptxUploaded byNazar Aziz
- Lecture 5 Sinal Space AnalysisUploaded byNazar Aziz
- Lecture4 Characterization of Communication Signals and SystemsUploaded byNazar Aziz

- IT Project Manager or Management or Mainframe Data SecurityUploaded byapi-78722083
- CCP603Uploaded byapi-3849444
- Cisco Router Configuration GuildUploaded byjerry666a
- Guia completa de NetappUploaded byYolanda Moreno Alberca
- 6314ijngn02 msc.pdfUploaded byKarir Cerahku
- SEL Compass FAQsUploaded byCristian Garcia
- Adafruit Arduino Lesson 1 BlinkUploaded byDSD
- xpp_A_docUploaded byNik20132013
- BCO2891-Taking VMware vSphere Storage APIs - Data Protection to the Limit Pushing the Backup Performance Envelope_Final_US.pdfUploaded bykinan_kazuki104
- World Recipe Management SystemUploaded bysujathalavi
- What Are Minimum IP Network Bandwidths Required for Watching 4K UHD IPTVUploaded by779482688
- Precision Dcd0Uploaded byLubna Batt
- Cisco BE4000 Partner GuideUploaded bysaishowaguu2
- EST FireShield PlusUploaded byJorge Mejia
- USB ProtocolsUploaded byPebo Green
- WIZ220IO User Manual V1 0 0Uploaded bydrwisum
- Docu of EsmsUploaded bysivakrishna
- The Active Directory StoryUploaded bySathish Kumar Marimuthu
- CLOUD COMPUTING LAB SETUP USING HADOOP & OPEN NEBULAUploaded byD.KESAVARAJA
- I T reportUploaded byajayi
- MadurodamUploaded byDarek Czarnecki
- rhcsa cheatsheetUploaded byStefan Preradovic
- Cyberoam_Anti_Virus_Implementation_Guide.pdfUploaded byDecio Ramires
- ESP8266 at Instruction Set en v0.30Uploaded byMordhwaj Patel
- knoppix-cheatcodesUploaded byVedasEverywhere
- Spring SecurityUploaded bySuresh Reddy
- Tixi_Data_HE400+600_EN_090222Uploaded byS M Naveed
- IVCSeriesSmallPLCProgrammingManual (1)Uploaded byTrần Nhật Vương
- Guia de Configuracao MS9520Uploaded byfranksatrian
- RAMCWC-2013Uploaded byAvinash Dubey