© All Rights Reserved

8 views

© All Rights Reserved

- Poisson Process
- Longevity Risks Modelling and Financial Engineering
- WARNING: Physics Envy May Be Hazardous To Your Wealth!
- qres_1
- Algorithms of Hidden Markov Model and a Prediction Method on Product Preferences
- 50 Questions to Help Your CMMS Search
- MTBF and Power Supply Reliability
- ShowFlow2 Step by Step Guide
- wavelet transform
- Normal in Vol Time
- ECE316-Yates6-10-Wong
- MC0074-Statiscal and Numerica Methods Using C Ans
- 10.1.1.58
- 2286398
- A 1
- Monte Carlo Simulations
- ch4.2
- UNcertainty Propagation and Sensitivity Analysis in Mechanical Models
- 04-A Probability Models
- Chapter 04

You are on page 1of 36

Science Applications: Chapter 6 on

Stochastic Processes

Kishor S. Trivedi

Visiting Professor

Dept. of Computer Science and Engineering

Indian Institute of Technology, Kanpur

What is a Stochastic Process?

Stochastic Process: is a family of random variables

{X(t) | t T} (T is an index set; it may be discrete or

continuous)

Values assumed by X(t) are called states.

State space (I): set of all possible states

Sometimes called a random process or a chance

process

Stochastic Process Characterization

At a fixed time t=t1, we have a random variable X(t1).

Similarly, we have X(t2), .., X(tk).

X(t1) can be characterized by its distribution function,

States X(t) (i.e. time t) may be discrete/continuous

Classification of Stochastic Processes

Four classes of stochastic processes:

discrete-time process stochastic sequence {Xn | n T}

(e.g., probing a system every 10 ms.)

Example: a Queuing System

Queue (waiting station)

Random arrivals

m

Inter arrival time Service time

servers

distribution fn. FY distribution fn. FS

Service times: S1, S2, (iid with a common cdf FS)

Notation for a queuing system: FY /FS/m

Some interarrival/service time distributions types are:

M: Memoryless (i.e., EXP)

D: Deterministic

G: General distribution

Discrete/Continuous Stochastic Processes

Nk: Number of jobs waiting in the system at the time of kth jobs

departure Stochastic process {Nk| k=1,2,}:

Discrete time, discrete state

Nk

k

Discrete

Continuous Time, Discrete Space

X(t): Number of jobs in the system at time t. {X(t) | t T} forms a

continuous-time, discrete-state stochastic process, with,

X(t)

Continuous

Discrete Time, Continuous Space

Wk: waiting time for the kth job. Then {Wk | k T} forms a Discrete-time,

Continuous-state stochastic process, where,

Wk

Discrete k

Continuous Time, Continuous Space

Y(t): total service time for all jobs in the system at time t. Y(t)

forms a continuous-time, continuous-state stochastic process,

Where,

Y(t)

t

Further Classification

n.

Further Classification (contd.)

Can the nth order distribution be simplified?

Yes. Under some simplifying assumptions:

Independence

Discrete time independent process {Xn | n=1,2,} (X1, X2, .. are iid,

non-negative rvs), e.g., repair/replacement after a failure.

Markov process introduces a limited form of dependence

Markov Process

Stochastic proc. {X(t) | t T} is Markov if for any t0 < t1< < tn<

t, the conditional distribution satisfies the Markov property:

Markov Process

We will only deal with discrete state Markov

processes i.e., Markov chains

In some situations, a Markov chain may also exhibit

time-homogeneity

current state, independent of how it reached this

particular state; but in a non homogeneous case,

current time can also determine the future.

For a homogeneous Markov chain current time is also

not needed to determine the future.

Let Y: time spent in a given state in a hom. CTMC

Homogeneous CTMC-Sojourn time

Since Y, the sojourn time, has the memoryless prop.

time Markov chain, sojourn time in a state follows

EXP( ) distribution (not true for non-hom CTMC)

Hom. DTMC sojourn time dist. Is geometric.

Semi-Markov process is one in which the sojourn

time in a state is generally distributed.

Bernoulli Process

{Yi} forms a Bernoulli Process, an example of a renewal

process.

Define another stochastic process , {Sn | n=1,2,3,..}, where

Sn = Y1 + Y2 ++ Yn (i.e. Sn :sequence of partial sums)

Sn = Sn-1+ Yn (recursive form)

P[Sn = k | Sn-1= k] = P[Yn = 0] = (1-p) and,

of a homogeneous DTMC

Renewal Counting Process

Renewal counting process: # of renewals

(repairs, replacements, arrivals) by time t: a

continuous time process:

If time interval between two renewals follows

EXP distribution, then Poisson Process

Note:

For a fixed t, N(t) is a random variable

(in this case a discrete random variable

known as the Poisson random variable)

The family {N(t), t 0} is a stochastic

process, in this case, the homogeneous

Poisson process

{N(t), t 0} is a homogeneous CTMC as

well

Poisson Process

A continuous time, discrete state process.

N(t): no. of events occurring in time (0, t]. Events may be,

1. # of packets arriving at a router port

2. # of incoming telephone calls at a switch

3. # of jobs arriving at file/compute server

4. Number of component failures

Events occurs successively and that intervals between these

successive events are iid rvs, each following EXP( )

2. : failure rate (1/ : average time between failures)

Poisson Process (contd.)

N(t) forms a Poisson process provided:

1. N(0) = 0

2. Events within non-overlapping intervals are independent

3. In a very small interval h, only one event may occur (prob.

p(h))

(memoryless) distribution.

E[N(t)] = Var[N(t)] = t ; What about E[N(t)/t], as t infinity?

Merged Multiple Poisson Process Streams

Consider the system,

Decomposing a Poisson Stream

Decompose a Poisson process using a prob. switch

Cond. pmf

Since,

The uncond. pmf

Generalizing the Poisson Process

Poisson Process

Non-Homogeneous Poisson

Process (NHPP)

Non-Homogeneous Poisson

Process (NHPP)

If the expected number of events per unit time, l, changes with

age (time), we have a non-homogeneous Poisson model. We

assume that:

1. If 0 t, the pmf of N(t) is given by:

PN t k mt / k!em t k 0, 1, 2, ...

k

period [0, t]

2. Counts of events in non-overlapping time periods are

mutually independent.

occurrence of events or time-dependent failure rate

m(t ) 0 l (x) dx

t

NHPP(cont.)

Generalizing Poisson Process

Poisson Process

Process (NHPP) Process

Renewal Counting Process

Poisson process EXP( ) distributed interarrival times.

What if the EXP( ) assumption is removed renewal proc.

Renewal proc. : {Xi | i=1,2,} (Xis are iid non-EXP rvs)

Xi : time gap between the occurrence of (i-1) st and ith event

time stochastic process. N(t) denotes no. of renewals in the

interval (0, t].

Renewal Counting Processes (contd.)

Sn t

For N(t), what is P(N(t) = n)?

tn

More arrivals possible

Let, m(t) = E[N(t)]. Then, m(t) = mean no.

of arrivals in time (0,t]. m(t) is called the

renewal function.

Renewal Density Function

Renewal density function:

then

d(t) = , t >= 0 and m(t) = t , t >= 0.

P[N(t)=n] = e t ( t)n/n! i.e Poisson pmf

Fn(t) will turn out to be n-stage Erlang

Alternating Renewal Process

1

I(t)

Operating Restoration

Time

Where:

Failure times T1, T2, are mutually independent with

a common distribution function W

Restoration times D1, D2, are mutually independent

with a common distribution function G

The sequences {Tn} and {Dn} are independent

Availability Analysis

Availability: is defined is the ability of a system to

provide the desired service.

If no repair/replacement,Availability(t)=Reliability(t)

If repairs are possible, then above is pessimistic.

MTBF

T1 D1 T2 D2 T3 D3 T4 D4 .

Availability Analysis (contd.)

renewal

Repair is completed with in this interval

x t

1. System does not fail before time t A(t)

= R(t)

2.System fails, but the repair is completed

before time t

Therefore, A(t) = sum of these two

probabilities

Availability Expression

dA(x) : Incremental availability

Repair is completed with in this interval

0 x x+dx t

Renewed life time >= (t-x)

that the renewal occurs in the interval (x,x+dx])

Availability Expression (contd.)

A(t) can also be expressed in the Laplace domain.

Lw(s)/s

However,

Availability, MTTF and MTTR

Steady state availability A is:

limit via LHospital rule and using the moment

generating property of the LT, we get the

required result for the steady-state

A=MTTF/(MTTF+MTTR)

Availability Example

Assuming EXP( ) density fn for g(t) and w(t)

Generalizing Poisson Process

Bernoulli Process

Poisson Process

Homogeneous

Compound Poisson Renewal Counting Continuous Time

Process Process Markov Chain

Process (NHPP) Discrete Time

Markov Chain

Non-Homogeneous

Continuous Time Semi-Markov

Markov Chain Process

Markov Regenerative

Process

- Poisson ProcessUploaded byJohniel ⎝⎲⎲⎠⎝⓿⏝⓿⎠ Babiera
- Longevity Risks Modelling and Financial EngineeringUploaded byDavid Dorr
- WARNING: Physics Envy May Be Hazardous To Your Wealth!Uploaded byCervino Institute
- qres_1Uploaded bykittu423
- Algorithms of Hidden Markov Model and a Prediction Method on Product PreferencesUploaded byJournal of Computing
- 50 Questions to Help Your CMMS SearchUploaded byAnonymous gg2Mi4
- MTBF and Power Supply ReliabilityUploaded byMazlan Mansor
- ShowFlow2 Step by Step GuideUploaded bywahaha06
- wavelet transformUploaded byrajbharaths1094
- Normal in Vol TimeUploaded byIgor
- ECE316-Yates6-10-WongUploaded byzorothemask
- MC0074-Statiscal and Numerica Methods Using C AnsUploaded by... Arnab
- 10.1.1.58Uploaded byBandi Babloo
- 2286398Uploaded byAja Nasrun
- A 1Uploaded byAndrew Olah
- Monte Carlo SimulationsUploaded byDaniel Casas-Orozco
- ch4.2Uploaded byjuntujuntu
- UNcertainty Propagation and Sensitivity Analysis in Mechanical ModelsUploaded byLuis Horacio Martínez Martínez
- 04-A Probability ModelsUploaded byanndayo
- Chapter 04Uploaded byNdivhuho Neosta
- RammnUploaded byPawan Gupta
- 1002.pdfUploaded byali.jahaya
- section1_HW3Uploaded byprachi
- AHigherOrderHiddenMarkovChainModulatedModelforAssetAllocationUploaded byParlog Ioana
- notes8Uploaded bysdsds-54
- Lecture 32Uploaded byEd Z
- Reliability Investigation of Steam Turbine Used in Thermal plant.pdfUploaded byAbhishek Kumar Singh
- lec14.pdfUploaded byVanitha Shah
- BrownianUploaded bySnigdha Sharma
- ReliableUploaded byRamesh Alhat

- RFID SystemUploaded byGagan Gupta
- 01F_Syscalls.pdfUploaded byGurjar Devyani
- ch20_1Uploaded byMazin el-Hag
- Gradiance Online Accelerated Learning SolUploaded byGurjar Devyani
- Virtual function in c++Uploaded byGurjar Devyani
- knapsackUploaded bySumit Bide
- Session Psqt DabgUploaded byGurjar Devyani
- Provisional Allotment Letter R3Uploaded byGurjar Devyani
- ComAct1.pdfUploaded byGurjar Devyani
- H2007_03_Week9PredicateUploaded byGurjar Devyani
- Tikona QuickPay MayUploaded byGurjar Devyani
- Vita Kishor s TrivediUploaded byGurjar Devyani
- 16 BrochureUploaded byGurjar Devyani
- MP M.tech 2016 Participating InstitutesUploaded byGurjar Devyani
- Allied Courses 2015Uploaded bykurnoy
- Core Java Assignments lab 3Uploaded byGurjar Devyani
- The DesignUploaded byGurjar Devyani
- Basic SyntaxUploaded byGurjar Devyani
- ArraysUploaded byGurjar Devyani
- StringsUploaded byGurjar Devyani

- Arch BridgeCodeUploaded byNitin Yadav
- TutorialUploaded byFusea Nicusor Sebastian
- A Smart Monitoring of Faults in Power Transformers and Maintenance Based on Wi-FiUploaded byInnovative Research Publications
- Stope Performance VillaescusaUploaded bykrainajacka
- Road DesignUploaded bycapitan_barboza
- etdUploaded byYizza Ornella Francesetti
- hw1-2017[2140]Uploaded byMohit Duseja
- InTech-Use of Induction Heating in Plastic Injection MoldingUploaded byJose Manuel Oliveira Pereira
- Preliminary ReportUploaded byLee Jones
- Cellular GlassUploaded byPaolo Ng
- THR_Quantum_Computers.03.pdfUploaded byldhutch
- Eng Mch-Prd-Ctlg-09Uploaded bymesayan
- Sig Figs, Math OpsUploaded byDon King Evangelista
- 1. Crystal ImperfectionsUploaded byHumaira Al-Junied
- A Comprehensive Evaluation of Intelligent Classifiers for Fault Identification in Three-phase Induction MotorsUploaded byIsrael Zamudio
- MotorsUploaded bylucky_indian
- YLP-HPUploaded byramjoce
- Em Practical Journal Paper i 13 - 15 Batch (1)Uploaded bySnehanshu Barapatre
- Lecture 1.1 Signals and SystemsUploaded byAmir Mustakim
- نظام التأريض - م.محمد سعد حامد_Electrical engineering community.pdfUploaded byMohamed Hassan
- TRANS LABUploaded bySheikh Nouman Mohsin Ramzi
- A New Development in Continuous Torque Monitoring CouplingsUploaded bymlouredocasado
- 799 Atomic StructureUploaded bySallar Qazi
- ams_7259Uploaded bychulhan
- RPDIR-L16.1 Fluoroscopy Systems WEBUploaded byFatimah Nisaulilmi
- Miscible Gas Drive Lab Presentation11Uploaded byAteeb Dutani
- Plain Concrete FootingUploaded byAnonymous mcHqIfbnV1
- appendix_b_page_530.pdfUploaded byLuana Pinheiro
- ejerciciosUploaded byCarlos Fernandez Alonso
- Developments on a Biomass GasifierUploaded byjrri16

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.