Quasi Birth Death Processes

© All Rights Reserved

2 views

Quasi Birth Death Processes

© All Rights Reserved

- solusi 4.18
- Ks Trivedi
- markov.ppt
- Cable Modelling in ATP
- An Analysis of the Information Spreading Delay in Heterogeneous Mobility DTNs
- THE FUNDAMENTAL THEOREM OF MARKOV CHAINS
- Mit Matlab Course
- Pugh Matrix Template
- 31
- Decentralized Control of Large Scale Storage Based Renewable Energy Systems
- 45_844
- Matrix Algebra
- markovchainproblems
- ParametricControlMechanical Wood Thesis1995
- syllabus
- Chong
- pe_byron
- CR_2013.pdf
- Matrices
- prp 2018

You are on page 1of 30

Publisher: Taylor & Francis

Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer

House, 37-41 Mortimer Street, London W1T 3JH, UK

Publication details, including instructions for authors and subscription information:

http://www.tandfonline.com/loi/lstm19

dependent quasi-birth-and-death processes

a

Australia, 5005 E-mail:

b

Australia, 5005 E-mail:

Published online: 21 Mar 2007.

To cite this article: L. Bright & P.G. Taylor (1995) Calculating the equilibrium distribution in level dependent quasi-birthand-death processes, Communications in Statistics. Stochastic Models, 11:3, 497-525, DOI: 10.1080/15326349508807357

To link to this article: http://dx.doi.org/10.1080/15326349508807357

Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) contained

in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no

representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of

the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,

and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied

upon and should be independently verified with primary sources of information. Taylor and Francis shall

not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other

liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or

arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic

reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any

form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://

www.tandfonline.com/page/terms-and-conditions

Dependent Quasi-Birt h-and-Deat h Processes

L. BRIGHT and P.G. TAYLOR

Department of Applied h/Iathematics

The University of Adelaide, South Austra!ia 5005, Australia

email: lbright@maths.adelaide.edu.au,ptaylor@maths.adelaide.edu.au

detail by many authors. This paper considers the calculation of equilibrium

distributions in level dependent QBD processes which are an extension of

the classical QBD process. In addition to the general case, we consider

a number of special cases of level dependent QBD processes, presenting

algorithms for computing the equilibrium distribution.

process, level, phase.

1. INTRODUCTION

Consider a regular continuous time two-dimensional Markov chain X ( t )

on the state space {(k, j) ; k 2 0 , l 5 j 5 M k ) . In the state (k, j ) , k is

referred to as the level of the state and j is referred to as the phase of the

state. Suppose the generator matrix for this Marltov chain is of the block

partitioned form

>

Mk x Mk, and Mk x Mk+1respectively. A Markov chain with this structure

is referred to as a level dependent quasi-birth-and-death process (hereby

abbreviated to LDQBD). We will assume that the process is irreducible.

I f w e p u t M k = M ~ k Z ~ a n d Q f ) = Qv o~ > o , Q ~ ) =VQk L~1 ,

They can be used to model, for example, queues in a random environment,

high speed communication systems, database systems and multiprogramming systems (see [6], [lo], [ll],[19], [20] and [9]). Since LDQBD's are a

generalisation of level independent processes, they enable us to model a

wider class of problems. For example, the P H I M I 1 queue (i.e the single

server queue with a phase-type arrival process) can be analysed using a

level independent QBD whereas the PH/M/oo can be analysed using a

LDQBD. Similarly, LDQBD's enable us to model infinite server queues in

a random environment rather than just single server queues. LDQBD's

also enable us to model more complex communication systems and multiprogramming systems.

Level independent QBD's have been dealt with widely in the literature.

Neuts [12] has developed methods for determining positive recurrence of

X ( t ) and for calculating the equilibrium distribution. Neuts also considered level independent QBD7s where there is some non-homogeneous

boundary behaviour but the process is eventually level independent. Recently Latouche and Ramaswami [7] have developed a logarithmic reduction algorithm that can be used to calculate the equilibrium distribution

in the level independent case. The computational complexity of this algorithm is significantly less than that for previously existing algorithms. The

methods we present for LDQBD's involve an extension of this logarithmic

reduction algorithm to the level dependent case.

The special case where X ( t ) has a finite state space has also received significant attention in the literature. This occurs when for some I?, QiK) = 0

and Q f ) = Q!') = Q r ) = 0 V k 2 l?

1. Hajeli [4] initially considered

level independent processes on a finite state space and Gaver, Jacobs and

Latouche [3] developed a general method for the level dependent case on

499

a finite state space. Ye and Li [18] have developed a computationally efficient algorithm for the case where the generator is piecewise homogeneous.

In the level dependent case on an infinite state space, as far as the authors

know, there is no general method for determining positive recurrence and

the problem of calculating the equilibrium distribution has not been addressed.

In order to calculate the ecpilibrium distribution for a LDQBD, it is

necessary to evaluate the family of matrices { R k k, 2 0) which are the

minimal non-negative solutions to the system of equations

is the expected sojourn time in the state (k + 1 , j ) per unit sojourn in the

state ( k , i) before returning to level k , given the process started in state

( k , i).

(1)

Define mo to be a positive left eigenvector of the matrix Q(P) RoQz

and define m k by m k = mo l$;j Re. Then it is easily seen that m =

(mo,

m l , . . .) satisfies mQ = 0. From Theorem 4.5 in Anderson [ I ] , X ( t )

is positive recurrent if and only if m e < oo where e is a column vector of

ones. In this case, the equilibrium distribution x = ( x o X, I , . . .) of X ( t ) is

given by xk = m k / m e .

show that the equation

[QjO)

+ R ~ Q ~= )o ]

This is not often the case since generally only numerical solutions can be

found for the Rk matrices. Hence we must calculate a truncated sum. If

we define ( x ~ ( I < * 0) )5~ ,k 5 Ii* to be the stationary probability that

500

<

i 5 k*,1 5 j 5 M i ) then it i s clear that x k ( I < * ) 0, 5 k 5 I(* is given by

of all states in and below level I<*, rather we are calculating an invariant

measure for the equilibrium distribution for all states in and below level

Ii* and then we are normalising this invariant measure over these states.

It is obvious that V I<* 2 0 , x k ( I < * )is an upper bound for xk. So we

have

and

xk = lim x k ( I < * ) V k 2 0

Downloaded by [UQ Library] at 10:20 24 April 2015

IC'iec

where the inequality and the limit hold elementwise. Provided we take I<*

large enough, we hope to have xk x x k ( K * ) . The question of finding a

sufficiently large value of Ii*will be considered in a later section.

The rest of the paper is as follows. In Section 2 we present a n algorithm

for evaluating x k ( I < * )given a value of I<*. In Section 3 we discuss how to

efficiently compute the matrices Rk and in Section 4 we give methods for

choosing a value for K * . Section 5 presents some numerical examples and

we summarise our results in Section 6.

2. CALCULATING xlc(I<*)FOR A GIVEN VALUE OF K*

to know R k , 0 5 k 5 IiV*- 1. It is possible to obtain a closed from

expression for R k by developing a continuous time analogue of Theorem

2.2 in Ramaswami and Taylor [14]. To do this it is necessary to use the

jump chain of X ( t ) . The jump chain of X ( t ) is a discrete time LDQBD

obtained by observing X ( t ) at time points immediately after a transition

has occurred.

Lemma 1

If X ( t ) is positive recurrent then Rk is given by

501

where Uf and D! are Adk-l x Mk-1+2tand Mk-1 x Mk-l-2c matrices respectively and are defined recursively by

Proof

LDQBD's. In discrete time LDQBD's the generator matrix Q in (1.1) is

replaced'by a stochastic matrix P which has the same block-partitioned

form as Q. Suppose for the jump chain of X ( t ) this matrix P consists

of the sub matrices ALk), AIk) and AF). In discrete time LDQBD7s it is

necessary to calculate the family of matrices {Rk, 0 3 k) which are the

minimal non-negative solutions of the equations

jump chain has an invariant measure is

uf

where

and D: are defined recursively by a set of equations similar to

equations (2.2) to (2.5). Since S ( t ) is positive recurrent, from results

in [I] we know that the jump chain must be recurrent and so have an

invariant measure. Ramaswami and Taylor showed that Rk can be written

as Rk = Qk~kJQ;l,where RkJ is the matrix Rk for the jump chain and Qk is

defined by Qk = -diag[(Q(,k))ii]. Using this expression together with (2.7j,

it is a simple matter to derive equation (2.1).

In order to use Lemma 1 to calculate Rk, we need to truncate the infinite

sum in (2.1). A simple way to do this is to define Rk(n) as the sum of

the first n 1 terms in (2.1) and then to take Rk = Rk(L) such that

(Rk(L) - R k ( L - I)),,,, < 6 for some tolerance E where (M),,, is the

maximum entry of the matrix Ad.

repeatedly. By re-arranging equation (1.2), we have

<

k 5 I<* - 1) is to use equation (2.1) to calculate Rrc*-l and then use

502

to be stable we require that the inverse contain non-negative elements. We

prove this is the case and that the inverse does exist in the appendix. Note

that the truncation rule given above does not guarantee that ( R k ( n )Rk(n- I)),,,

< E holds V n > L which is what we want. An algorithm for

calculating {xn.(I<*),

0 5 k 5 I<*}for a given value of I(* is as follows.

05k

Calculate R I ~ - 1

Recursively calculate Rip-2, RI(*-3,

..., RO

) o s.t. x o ( K * ) e= I

for k = 1 to I<* do

xk(I<*)

=x~-~(I<*)R~-~

Normalise xo(I<*),

xl(I<*),

...,xk(I<*)s.t.

c&=

x,(IC*)e

~

=1

3. EFFICIENTLY COMPUTING Rk

the sum (2.1) and we tool; Rk = R k ( L )where L was chosen such that

( R k ( L) Rk(L- I)),,,

< E . An efficient method for determining the value

of L is expressed in the following algorithm.

Algorithm 2 : Computing Rk

because both quantities involve the same inverse and so it is sensible to

compute them together to avoid calculating the same inverse twice. To see

this we restate equations (2.2) to (2.5) for U: and D:+~(+~.

in Algorithm 2 but do not use it. However,

it is no extra effort to compute this quantity because we already have the

necessary inverse from calculating U:

We refer to U i and D:+2t+l as the UD-pair UD(C, k). At the end of Algorithm 2 we have calculated the UD-pairs UD(e, k) for for C = 0, l , . . . ,L

and fixed k. To do this we need to calculate other UD-pairs UD(n, m) for

n < e. We consider a way to compute these quantities so that UD-pairs are

not computed more than once. By storing various UD-pairs it is possible

to develop an efficient method for calculating UD(e, k), l = 0 , . . . , L. To

determine which UD-pairs to calculate and which to store, it is useful to

represent the recursive relationships expressed by equations (3.3) and (3.4)

by a diagram. Figure 1 shows that to calculate UD(e, k), we need to know

the UD-pairs UD(C- 1, k), UD(e- 1, k+2e-1), and UD(C- 1, k +2.2e-1).

1-1.

k

k+2e-1

I~+2.2~-l

need first to calculate UD(1, k + m ) for in = 0 , 2 , 4 and UD(0, k + m) for

m = 0 , 1 , . . . , 6 . Similarly, to calculate UD(3, k) we need first to calculate

UD(2, k + m) for m = 0,4,8, UD(1, k + rn) for m = 0 , 2 , 4 , . . . ,12 and

UD(0, k + m) for in = 0 , 1 , 2 , . . . ,14. Figure 2 represents the last two

statements. To calculate UD(1, k) we need to calculate all the UD-pairs

in the second smallest triangle in Figure 2. To calculate UD(2, k) we need

to calculate all the UD-pairs in the third smallest triangle and to calculate

UD(3, k) we need to calculate all the UD-pairs in the largest triangle.

Figure 2 can be extended to any arbitrary value of !. From Figure 2 we

see that if we calculate UD(l, k ) for e = 0 , 1 , 2 , 3 by directly implementing

equations (3.3) and (3.4) then we will be computing some UD-pairs over

and over again. However, it is possible to calculate UD-pairs in an order

that will result in not computing any UD-pair more than once. The UDpairs needed for Algorithm 2 can be calculated (and used) in the following

order.

A l g o r i t h m 3 : Evaluating UD-pairs for A l g o r i t h m 2

do

for i = 0 to !

for j = k

j =k

+ (2e-i+1- 1 ) 2 ~to

remove all UD-pairs from storage except the pairs

UD(j, k

+ (2e-j - 1)2j+l),j

= 0,1,. . .,

When Algorithm 2 stops, we have calculated the UD-pairs UD(-t, k)

for -t = 0 , 1 , . . . , L and fised I;. If these UD-pairs are computed according

to Algorithm 3 then a total of 4.2L - L - 3 inverses must be calculated.

If the UD-pairs were calculated by directly implementing the recursive

equations (3.3) and (3.4) then a total of approximately i.5L inverses must

be calculated. Note that with Algorithm 3 the number of inverses still

increases exponentially with L. However, for many practical examples

only a small value of L (say four or five) is needed. To see this, we give a

physical interpretation of equation (2.1).

Recall that (Rk)i,jis interpreted as the expected sojourn time in the

state (k + 1,j) per unit sojourn in the state (k, i) before returning to level

k, given the process started in state (k, i). In order to calculate Rk,we

need to consider all sample paths that start and finish a t level k and don't

visit any levels below level k 1 in between. For each of these sample

paths we have to calculate the expected sojourn times in each of the states

506

in level k 1. Adding together the sojourn times for all the sample paths

will give us Rk. Ramaswami and Taylor [14] observed that the 1th term

in (2.1) calculates the expected sojourn times in level k 1 for sample

paths that go at least as high as level k - 1 2e and no higher than level

k - 1 2'+'. So when we truncate the sum in (2.1) to L 1 terms, we only

consider the sample paths that remain below level k - 1 2L+2. Thus even

for relatively small L (say four or five) we consider paths which move up 64

or 128 levels before returning. For most physical processes this accounts

for most of the probability mass.

+

+

requires the storage of many UD-pairs. However, it is not difficult to alter

the algorithm to decrease the amount of storage space required. In fact it

is possible to alter Algorithm 3 so that at any time only 1 1 UD-pairs

need to be in storage.

we want to find a level I(* such

that the equilibrium probability of being in a state in or above level K* is

approximately zero. In some special cases it is possible to calculate K* a

priori. In this section we firstly discuss a general method for calculating

I<* and then we look at some special cases.

For a LDQBD we must study the particular process and devise some

way of showing that the process is or is not positive recurrent. If X ( t ) is

positive recurrent then xt(I<*), k 2 0 is an upper bound for the equilibrium distribution and from equation (1.8) it can be seen that by choosing

I<* very large, xk(I<*)will give a close approximation to the equilibrium

distribution. However, as I<*increases, so does the computational effort

required to calculate xk(K1). To see this consider Algorithm 1. The main

computational overhead is in computing the inverses in the first two steps.

To calculate RIc.-l with equation (2.1) requires the calculation of approximately 4.2L inverses and to calculate each of Rk for I<*- 1 > k 2 0 requires

one inverse.

In some applications L is large for small values of k and smaller for

large values of k. Suppose that for some value of k we need a very large

value of L. In this case, the more efficient method would be to calculate

Rk for a larger value of k (one which requires only L = 4 or 5 say) and then

calculate the Rkmatrices for lower values of k using the recursion in (2.8).

For example, suppose we required L = 7 to calculate Rk,, for some value

507

kl. This requires approximately 4.27 = 512 inverses. Now suppose in order

to calculate

we required only L = 4. This requires approximately

4.24 = 64 inverses. If we use the recursion in (2.8) to calculate Rk,,

we

need to calculate a further 40 inverses giving a total of 104 inverses. The

second method is obviously the more efficient.

values of I<* until we obtain one that is sufficient. This method is not

guaranteed to work. It is certainly possible to do better for some LDQBD's

(discussed in Sections 4.2 and 4.3 below). However, for most processes of

physical interest, the following algorithm will halt at a suitable value of

I<*.

Algorithm 4 : A general method for calculating

Set I<& = 0

do

Choose I<;,, such that I<:,,

> I&

RIpw-l

else repeat above two steps

Recursively calculate

RIp,-2,

RI<;,,-3, . . . ,R I ~ , ~

Solve X ~ ( I L ~ , , , , ) ( Q \ ~R) ~ Q ~

= )0 )subject to xo(K~,,)e= 1.

for k = I<:/,

+ 1 t o I<;,,

do

x k ( I i i e w )= Z ~ - ~ ( I < ~ , , ) R ~ - ~

Normalise xo(I<iew),x l ( K i e W )., . . , x k ( I < i e w )

s.t.

~ k xm(I<iew)e

= ~ =1

until ( x ~ ; , , ( I ( ~ ~<~ E) )e

Set I<* =

We hope that by choosing I<*

in this fashion that we will find a value

of I<*such that xk x o V k > K*. It is certainly possible to construct

LDQBD's for which this is not the case. However, for most LDQBD's

that arise in practice, the matrices Q f ) ,

and Q r ) can be expressed

in some parametric form. Indeed this is necessary if a process is to be

analysed using a computer. It would be very difficult, if not impossible,

even to input into a com uter the parameters of a process involving a large

number of unrelated Qlif matrices. By considering the parametric form of

found by

the matrices QIk),it should be possible to tell if the value of I<*

the above method is accurate. For the above reasons, careful consideration

must be given to the structure of the LDQBD when using this method to

find I(*.

~(1")

If Q r ) = Q2, Q ( ~=~ Q1

) and Q f ) = Qo, V k 2 K, then the process is

essentially a level independent QBD where there is some level dependent

behaviour in the levels 0 5 k. .<? I We refer to the states in these levels

as the boundary states.

Such processes have also been considered in Neuts 1121. The approach

there also uses xk = X ~ ( R ~ ) ' " -to" find xk, k > I? which is what we

obtain from equation (1.5). To find xk, 0 5 k 5 I?, Neuts solved

This does not highlight the matris analytic structure of xk for k 5 I?. A

nice property of our method is that it shows that xa has a matrix analytic

form for all 0 5 k 5 I?.

In this particular case, it is possible to determine if X ( t ) is positive

recurrent and to determine the equilibrium distribution exactly. Neuts

[12] has shown that X ( t ) is positive recurrent if and only if x Q o e < x Q z e

where Q = Q1+ Q o + Q 2 and rr is the solution to rrQ = o such that x e = 1.

Because of the physical interpretation of R k , it is clear that Rk = RI;.for

k E . If X ( t ) is positive recurrent then

>

(1.5). This is one of the few cases where this is possible. Since X ( t ) is

a level independent QBD for levels I? and above, when we calculate R K ,

we are in fact dealing with a level independent QBD. In this case, the

expression in (2.1) simplifies to the case in Latouche and Ramaswami [7].

An algorithm for this case is given below.

Algorithm 5 : Calculating the equilibrium distribution

for a large number of boundary states

Calculate R R

Recursively calculate RRWl,RRv2,...,RO

xo [E!=~

Calculate xl: k

e=1

Condition 1 : V k, i 3 j s.t (

>

~ f ) ) i , ~

0.

a state in the level below according to some positive rate. Note that this

class of processes is a non-trivial class. The P H / M / c o queue modelled

as a LDQBD has this property. When condition 1 holds, it is possible to

construct a second LDQBD that stochastically dominates the first process.

This second process also has the property that its marginal distribution

over the levels is very easy to calculate. It is then possible to calculate a

value of I{*using this marginal distribution. Before we present our results,

we define what it means for a process X'(t) on a partially ordered state

space S , to stochastically dominate S ( t ) .

510

ordered set S. Then TJ stocl~asticallydominates y (written y 5 , jj) if for

all monotone functions f (.) : S + R

) state space S and density function

~ ( t stochastically

)

dominates the process X ( t ) with state space S and

density function x ( t ) if

~ ( 0 5s

) a:(O)

2 ( t ) for all t .

+ x ( t ) 5,

(4.2)

so we can take the limit as t -+ cm of both sides of the inequality and

obtain

x <$ a:

(4.3)

The following theorem describes how to construct a process X ( t ) that

stochastically dominates X ( t ) . The proof of the theorem is left until the

appendix.

Theorem 1

Let X ( t ) be a continuous time LDQBD which satisfies condition 1.

Moreover, let X'(t) be a continuous time LDQBD on the same state space

as X ( t ) and with generator

where

511

Q

= ( Q Y ) ) ~ , j~ ,# i , k 2 o

(4.9)

and (v),~,and (v),,, are the minimum and maximum entries of the vector

v respectively. Then ~ ( tstochastically

)

dominates X ( t ) .

We define the dominating process of X ( t ) (when it exists) to be the

process X ( t ) given by the above theorem. X ( t ) can be given a physical

interpretation as follows. The vectors Qf-l'e consist of the sums of the

rates from a state in level k - 1 to a,ll the states in level k and the vectors

&f-l)e consist of the sums of the rates from a state in level k - 1 to all

the states in level k - 2. Call these rates the total up and down rates from

level k - 1 respectively. The rate at which X ( t ) moves from phase i in level

k 2 1 to phase j in level k 1 is given by the greater of the maximum

total up rate from level k - 1 divided by the number of states in level k 1

and the rate at which X ( t ) moves from state ( k ,i ) to state ( k 1, j ) . ~ ( t )

moves from states in level 0 to states in level 1 in the same way that X ( t )

moves from states in level 0 to states in level 1. The rate at which X ( t )

moves from phase i in level k 2 2 to phase j in level k - 1 is given by the

lesser of the minimum total down rate from level k divided by the number

of states in level k - 1 and the rat,e at which X ( t ) moves from state (k, i )

to state ( k - 1,j ) . X ' ( t ) cannot move from level 1 to level 0. X ( t ) between

states on level k in the same way that X ( t ) moves between states on level

k. It is easily seen that , y ( t ) is an irreducible Markov chain on the state

space { ( k ,j ) : k 2 1 , l 5 j 5 Adk}.

>

1 ) be the marginal distributions over

the levels for X ( t ) and - y ( t ) respectively under their stationary distributions. Define m as an invariant measure for X ( t ) and partition m as

r7z = (ml,f i t z ,ms,.. .). If we define Ln = m , e then provided G-I =

Cr=l Ln < cc then an equilibrium distribution for X ( t ) exists and jn

is then given by .& = GI,,. Now in the case where for all k we have

j~r-l'e),~~~

~i:-~)e)~,~~

5 ( Q $ ~ ) )W,

~ ,j ~then {l,,n 2 1 )

2

i , j ~ 2 i . and

i l . f ~+

t 1

is the equilibrium distribution of t,he standard birth-and-death process on

state space { i 2 1 ) with transition rates q ( i ,j ) given by

(QP))

r1(0,1) = 0 ,

q ( i ,i 1) = ( ~ [ - ' ) e ) , ~i >

~ 0,

g(1,O) = 0.

q(i,i - 1 )

So {j,,, n

= (Q;-"e),,,,,,

> 0.

> 1 ) is given by

+

< 1 V i 2 I for some I since then the terms in the infinite sum will be

CY

512

>

1 ) exists, 5 must exist

and therefore x must exist since ~ ( tstochastically

)

dominates X ( t ) .

Now we want to find I<* such that CTZIc.en < E . It can be shown from

equation (4.3) that Cjm=,C j 5 C&, Fj V n 1 SO it is sufficient to find K*

such that

03

C jn<~.

(4.11)

>

n=K*

Often it is the case that we have an analytic form for the matrices Qi(k)

and hence an analytic

form for the rates q ( i ,j ) . In this case it is possible

to compute C:&. Cn analytically, using (4.10) and hence we can find K*

from (4.11).

When the rates q ( i , j ) are not l i n o ~ l analytically,

l

we can still compute

the quantity { & ( N ) ,1 n 5 N ) given by

<

>

>

we have & ( N )

& V 1 n N and &(m) = jn. Hence we have

f n ( ~ 2

) Cn V l 5 n 5 N , N 21.

< <

We can then choose Ii* such that TIi.(lr'*) < E. The reasoning behind

this choice of Ii* is as follows. Firstly, &.(I<*) < E tells us that lip < E so

the equilibrium probability that S ( t ) is in level I<' is small (according to

some tolerance E ) . We want I<* such that the equilibrium probability that

X ( t ) is in level I<* or above is small. So when we choose I<* as above, we

assume that if the equilibrium probability that X ( t ) is in level I<* is small

then the equilibrium probability that S ( t )is in level I<* or above is also

small. We hope that this will be the case but there will be situations where

this will fail to hold. If it can be shown that q ( k , I."+ l ) / q ( k + 1, k ) cr < 1

holds V k 2 I<* then we have

<

and hence

So in this case we have an upper bound on

- en and provided cr is

small, the bound will be small. Here we can say xk(I<*)is approximately

equal to the equilibrium distribution of being in level k. In other cases more

detailed analysis of X ( t ) may be necessary to determine if this choice of

513

I(* is sufficiently large for xk(I<*)to approximate xk. At the very least,

xk(I<*)can always be interpreted as an upper bound for the equilibrium

distribution.

We now give an algorithm for determining I<* when condition 1 holds.

A l g o r i t h m 6 : Calculating { x k ( K * )0, 5 k 5 I<*)

w h e n c o n d i t i o n 1 holds

Set I<* = 1

Set ~ ( I C =

* )1

while ( j ( K * ) > E ) d o

I<* =I<*

+1

Proceed with Algorithm 1 using the current value of I<*

As mentioned before, if condition 1 holds, then from any state, X ( t )

moves down to a state in the next highest level according to some positive

rate. In many cases condition 1 will not hold for all states but if it holds

for all states in and above level I?- then it is still possible to construct a

dominating process. This weaker form of condition 1 is as follows.

Condition 2 : V k 2

I? and V i 3 j

s.t ( & P 1 ) i j

> 0.

with generator matrix Q of the same form as Q in (4.4) with

~ ( 1 ~ )

and

given by

QP)

QF),

5 14

stochastically dominates X ( t ) provided condition 2 holds for X ( t ) . Note

that ~ ( tis) irreducible only on the levels I(,I?

1 , k 2 , . . .. In the

same way that we defined in(^) we can define { j , ( ~ )I?, 5 n N) and

this will have a similar form to & defined in (4.12). Then we choose I(*

such that ircw(I<*)

< E . SO when condition 2 holds, we have the following

algorithm.

<

5 k 5 I<*)

A l g o r i t h m 7 : Calculating {xk(I<*),O

w h e n condition 2 holds

Set I<* = I?

Set j ( K 3 ) = 1

Ir" = I<* + 1

Calculate

i~p

(I<*)

where there are only two phases at each level. We then consider a case

where there are 20 phases at each level. Consider an M / M / o o queue in

a random environment. This is an infinite server queue whose behaviour

is determined by the state of a in-state irreducible Marltov process Q. If

Q is in state i and there are k customers in the queue then the arrival

and service rates for the queue are A i and kpi respectively. If Q changes

from state i to state j , the arrival and service rates change to Xj and k p j

instantaneously. Such a queue can be modelled as a LDQBD where the

state (k, j ) represents k customers in the queue and Q being in state j.

The matrices Q f ) , ~ ( l k )and

are given by

QP)

515

elements of the vector v.

compute the equilibrium distribution. It is easy to show that the dominating process is positive recurrent for all values of X and p so the original

process is always positive recurrent. Consider the case where Q is a 2-stage

Markov process and Q is given by

2 1, X1 = 40 and X2 = 5. For this

and the other examples below, we took Rk = Rk(n) s.t ( R k ( n )- Rk(nI)),,, < 10-12. In Algorithm 6 we took 6 = 10-lo which gave a value of 88

for I<*.In Table 1 we give the marginal probabilities l k .In order to see if

the value I<*

= 88 was sufficiently large, we used Algorithm 1 to calculate

the equilibrium distribution with I<*= 188. We found that the absolute

value of the differences of the two results was less than 10-l4 in all cases.

Next we altered the above example by setting p:, = 0. This corresponds to

a situation where the servers are all idle when Q is in state 2. Here both

conditions 1 and 2 fail to hold so we must, use Algorithm 4 to compute the

equilibrium distribution.

In order to malie an initial guess for Ii*in Algorithm 4, we used the

results for the previous example. Since we have set pz = 0, we expect

that we will require a value of I<*at least as high as in the previous

example. For this reason we guessed an initial value of I<* = 88. We also

= 88 proved to be

set E = 10-lo and we found that our initial guess of I<*

a sufficiently large choice. The marginal probabilities lk are presented in

Table 2. For this example we haven't proved that the process is actually

positive recurrent but the values in Table 2 certainly suggest that it is.

Obviously when analysing processes it is desirable to be able to determine

positive recurrence before we attempt to compute any of the Rk matrices.

How this is done in general is a topic for further research.

Now consider the PHIMloo queue. This is the infinite server queue

with a Marliovian service time distribution and a phase-type interarrival

time distribution. For more details on phase-type distributions, see 112,

Chapter 21. We considered the phase-type distribution

0.05841196Elo(1.168239125) 0.94158804Elo(18.83176088) which is the

mixture of 2 ten stage Erlang distributions considered by Ramaswami and

Latouche [13]. This queue can also be modelled as a LDQBD where the

state (k, j ) represents k customers and the interarrival process being in

phase j . Since we have a mixture of 2 ten stage Erlang distributions, there

are 20 phases at each level. If we let each customer be served a t rate p then

Table 1

QP)

we have

= k p I where I is the identity matrix. The matrices ~ fand

)

are obtained by considering the phase-type arrival process. Condition

1 holds for this process so we used Algorithm 6 (with the same value of E

as before) to calculate the equilibrium distribution. We set p = 0.5 and

found I(* = 84. The marginal probabilities Ct are presented in Table 3.

QY)

In all three of the above examples the estimated values of IC* are larger

than necessary. That is, the algorithms tend to over estimate the value of

I<*. For example, for the PH/A4/oo queue the algorithm gave IC* = 84

yet from Table 3, the equilibrium probability of being in any state above

level 18 is negligible.

Latouche and Ramaswami [7] have observed in the level independent

case that it is possible to construct processes such that the algorithm re-

Table 2

Table 3

is of course also relevant. However, given the physical interpretation discussed in Section 3, most physically meaningful processes should only require a relatively small number of iterations. Another potential problem is

the risk of overflow or underflow errors when recursively calculating the Rk

matrices. This problem was also observed in Gaver, Jacobs and Latouche's

[3] computation of the matrices ck.These problems are more likely to arise

when either Qf) or Qf) are very large or very small compared to the other

matrices in Q. For some cases where this problem arises, it is possible that

it could be eliminated via a numerical scaling tcchnique.

6. CONCLUSIONS

The algorithms presented in this paper allow the computation of the equilibrium distribution for LDQBD's. The basic form of the algorithms is to

choose a level I<*,calculate an invariant measure on the levels 0 2 k 2 I<*,

and then normalise this invariant measure. To calculate the equilibrium

distribution we must first calculate the matrices R k , k 2 0. The method

used to calculate these matrices has the advantage of having a nice physical

interpretation. The critical step in the algorithms is finding a sufficiently

large value of I<*. Currently more work is being done on more elaborate

methods for determining I{*as well as methods for determining positive

recurrence.

APPENDIX

Proof that ( - Q ~ " + ~ ) - R ~ + ~ Q ~exists

+ ~ ) ) -and

' contains non-negative

elements

time LDQBD. The jump chain is obtained from X ( t ) by observing X ( t )

only at time points when there is a change of state. The jump chain has

transition matrix

interest are {R;, k 2 0) which are the minimal non-negative solutions to

the equations

k 2 0.

process visits the state (k + 1,j ) before the process first returns to level k

given the process started in state (k,i). Define P[{!);(~+~,~)

as the taboo

probability that given that the jump chain starts in state (k, i), it is in state

(k 1,j) at time n and has not visited any state in level k in between.

Given the interpretation of R i , a simple probabilistic argument gives

for R:, u/,:+') is interpreted as the taboo probability that, starting from

state (k + 1,i) at time t = 0, the jump chain visits the state (k + 1,j) at

Let

some time t

520

is strictly s ~ ~ b s t o c h a ~ tNOW

i c . define an absorbing Markov

matrix u(~+')

chain by considering the following. Observe the jump chain at all time

points where it is in level 1; 1 and before it has visited level k for the

first time. Then take the jump chain visiting level k for the first time as

absorption occuring. Since the jump chain is irreducible, we know that

there is a non-zero probability of absorption. Given the interpretation of

U(k+l),we have an absorbing Markov chain with transition matrix

in Kemeny and Snell [5], it can be shown that the inverse (I - u(~+'))-'

exists. Furthermore, [5] also gives us that (I - U("+'))-' contains only

non-negative elements. From Ramaswami and Taylor [14] we have the

relationship Rk =

and by using this and substituting in the

values of A f ) , A\" and d f ) ,into ( I - u("+'))-',it is seen that the inverse

(-Q(:+') - R ~ + ~ Q & ' "exists

+ ~ ) )and

- ~ contains only non-negative elements.

@k~i@iil

P r o o f of Theorem 1

Before we prove Theorem 1 we present some theoretical results. For

more details on these results, see, for example, [17], [15] and [16].

Let

indicate a partial order and 3 indicate a quasi-order. For a set

S, 5 is a partial order on S if

(i) s 5 s Vs E S;

(ii) s 5 t and t

5 s imply s = t ;

(iii) s 5 t and t 5 u imply s 5 u,

and 4 is a quasi-order on S if

(i) s -: s is false Vs E S;

Ross [15] states that given a quasi-order 4 , a partial order 5 can be defined

by

x 5 y if and only if x 4 y or x = y.

(0.2)

If S is the state space of a stochastic process endowed with a partial order,

then for any subset r of S, define

52 1

due to Massey [S].

Lemma A

Let X l ( t ) and Xz(t) be uniform Markov processes on the state space

S with transition rates ql(i,j ) and qz(i,j) respectively (i.e for m = 1,2,

C j qm(i,j) is bounded V j ) . If 5 is a partial order on S then Xl(t) stochastically dominates X2(t) if and only if for all x 5 y in S and all increasing

sets r, the following hold

and

Brandt and Last [2] extended Lemma A by removing the requirement that

Xl(t) and &(t) be uniform. We now use this extended form of Lemma

A to prove Theorem 1. First define the relation 4 by (i, j ) 4 (k, 1) if and

only if i < k. It is easy to show that 4 is a quasi-order. Define the relation

4 according to (0.2).

Now according to this partial order, the increasing sets are of the form

where I {1,2,. . .Mi-l} is some non-empty set of indices. That is, the

increasing sets contain all the states in levels i and above and a t least one

state in level i - 1. Let @(i)= UI Q i S I and define O by

Thus @(i)is the set of all increasing sets r such that (e, j) E r V e 2 i , 1 5

j 5 M!, and (i - 1,j) E r for at least one 1 I j 5 Mi-1 and O is the set

= q5 V i # j and that the

of all increasing sets. Note that @(i)n @(j)

cardinality of O is infinite but the cardinality of @(i) is finite V i 2 1.

Tn order to prove Theorem 1, we need to show that conditions (i) and

(ii) hold in Lemma A with ql(x, z ) given by the matrices Q f ) ,

and

and q2(9,2 ) given by the matrices ~ f )Q\')

, and

of equations

(4.6)-(4.9).

~r)

QP)

QY)

522

y = (i,k ) . We have

> 1 and x

= ( i - 1, j ) ,

and

so it is clear that (i) holds. Note that with I? E @ ( i ) , i > 1, (i) holds

trivially for all other x 4 y E I?. Consider r E @ ( I ) and x = (0, j ) ,

y = (1,k). We have

and

of x that we must consider are x = ( i ,j ) and x = ( i - 1,j ) . For x = ( i ,j )

we have

~ l ( x2 ), = C ( ~ f ) ) j , t

f$r

ts I

(i-l,t)$r

and

and

523

from (4.9) and (4.7). For I' E @ ( I )we need only consider x = (1,j) and

x = (0, j ) . For x = (1, j) we have

and

and

The proof that (ii) holds for all x 5 y E S and all increasing sets

similar to the above. This completes the proof.

I? is

0

Acknowledgments

The authors would like to thank Guy Latouche, Nigel Bean and the

anonymous referees for suggestions that helped in the revision of this

manuscript. The first author aclmowledges The Australian Telecommunications and Electronics Research Board for supporting this research.

References

[I] Anderson, W. J., Continuous-Time Markov Chains, An ApplicationsOriented Approach, Springer-Verlag (1991).

[2] Brandt, A. and Last, G., On the pathwise comparison of jump processes driven by stochastic intensities, preprint, Humbolt-Universitat

zu Berlin (1993).

[3] Gaver, D. P., Jacobs, P. A, and Latouche, G., Finite birth-anddeath models in randomly changing environments, Adv. App. Prob.

524

[4] Hajek, B., Birth-death processes on integers with phases and general

boundaries, J. App. Prob. 19 (1982), 488-499.

[5] Kemeny, J. G., and Snell, J. L. Finite Markov Chqins, Springer-Verlag

(1976).

[6] Latouche, G., Algorithmic analysis of a multip~ogramming-multiprocessor computer system, J. Assoc. Comp. Mach,. 28 (1981), 662679.

[7] Latouche, G. and Ramaswami, V., A logarithmic reduction algorithm

for quasi-birth-and-death processes, J . App. Puob. 30 (1993), 650-674.

[8] Massey, W. A., Stochastic orderings for Marlcov processes on partially

ordered spaces, Mathematics of Operations Research 12 (1987), 350367.

[9] Nelson, R. D., A performance evaluation of a general parallel processing model, Performance Evaluation Review 18 (1990), 13-26.

[lo] Neuts, M. F., The M/M/l queue with randomly varying arrival and

service rates, Opsearch 15 (1987), 139-157.

[ll]Neuts, M. F., Further results on the M/M/1 queue with varying rates,

[12] Neuts, M. F., Matrix Geometric Solutions in Stochastic Models, Johns

Hopkins (1981).

[13] Ramaswami, V. and batouche, G., An experimental evaluation of the

matrix-geometric method for the GI/PH/l queue, Commun. Statist.

- Stochastic Models 5(4) (1989), 629-667.

[14] Ramaswami, V. and "paylor, P. G., Some properties of the rate matrices in level dependeqt quasi-birth-and-death processes with a countable number of phases, submitted for publication.

[15] Ross, I(. A . and Wright, C. R. B., Discrete Mathematics, Prentice

Hall, 2 edition (19881,

[16] Smeitink, E., Stochastic Models for Repairable Systems, PhD thesis,

Vrije University (1992).

and Other Stochastic

1171 Stoyan, D., Comparison Methods for Qz~ez~es

Models, Wiley series in probability and mathematical statistics (1983).

[18] Ye, J., and Li, S., Folding algorithm: A computational method for

finite QpP processes with level-dependent transitions, IEEE Trans.

Commun. 42 (1994), 625-639,

525

[19] Zulierman, M., Applications of matrix-geometric solutions for queueing performance evaluation of a hybrid switching system, J. Aust.

Math. Soc. Ser. B 31 (1989), 219-239.

[20] Zukerman, M. and Kirton, P., Applications of matrix-geometric solutions to the analysis of the bursty data queue in a B-ISDN switching

system, Proc. GLOBECOM '88 (1988), 1635-1639.

Received:

Revised:

Accepted:

8/21/1994

4/4/1995

41411995

- solusi 4.18Uploaded byMivTah Al Bughisiyyah
- Ks TrivediUploaded byMonzieAir
- markov.pptUploaded byMuhammad Akmal Hussain
- Cable Modelling in ATPUploaded byJuan Carlos Chacón Chavarin
- An Analysis of the Information Spreading Delay in Heterogeneous Mobility DTNsUploaded bymiladydesummer
- THE FUNDAMENTAL THEOREM OF MARKOV CHAINSUploaded byMichael Pearson
- Mit Matlab CourseUploaded bywistonjr
- Pugh Matrix TemplateUploaded byRudrani Sarkar
- 31Uploaded byShaheer Shaheen
- Decentralized Control of Large Scale Storage Based Renewable Energy SystemsUploaded byhom3646
- 45_844Uploaded byprabhjot singh1
- Matrix AlgebraUploaded byGaurav Shrivastava
- markovchainproblemsUploaded byalphakiller
- ParametricControlMechanical Wood Thesis1995Uploaded byyxiefacebook
- syllabusUploaded byabhinav ranjan
- ChongUploaded byEpic Win
- pe_byronUploaded byCHui Tupak Yupanky
- CR_2013.pdfUploaded byAndrea Young
- MatricesUploaded byRocket
- prp 2018Uploaded bySriramachandar
- Analyzing Electricity Consumption Behavior Of Households Using Big Data AnalyticsUploaded byAnonymous vQrJlEN
- 10.1.1.14Uploaded byHac Minh Phuc
- pnas.201019712SIUploaded byTridip Sardar
- Moore 1971Uploaded bymrjunoon
- Symoro_RoboticaUploaded byJasón De Elea
- DTState SpaceUploaded byراية أبو شاكر
- IIpuc Modelqp MathUploaded byRanjan Prasad
- Chapter 5 (5.2-Concept Selection)Uploaded bySyafiq Mazlan
- reUploaded bystathiss11
- mm dealer.pdfUploaded byLuca Pilotti

- Bayesian StatisticsUploaded byDaniel Bogiatzis Gibbons
- 03 Lent Lecture 2 Tvdvc2Uploaded byDaniel Bogiatzis Gibbons
- Economics of Infinity (1)Uploaded byDaniel Bogiatzis Gibbons
- 07_lent_Topic 2 - Generalized Method of Moments, Part II - The Linear Model_mw217Uploaded byDaniel Bogiatzis Gibbons
- Hawkes Processes and Applied FinanceUploaded byDaniel Bogiatzis Gibbons
- Econometrics NotesUploaded byDaniel Bogiatzis Gibbons
- Aspects of Bayesian InferenceUploaded byDaniel Bogiatzis Gibbons
- Lecture Series 1 Linear Random and Fixed Effect Models and Their (Less) Recent ExtensionsUploaded byDaniel Bogiatzis Gibbons
- Advances in Women's Rights in Saudi ArabiaUploaded byDaniel Bogiatzis Gibbons
- 200Uploaded byDaniel Bogiatzis Gibbons
- Lecture 15-3 Cross Section and Panel (Truncated Regression, Heckman Sample Selection)Uploaded byDaniel Bogiatzis Gibbons
- Takeshi Amemiya-Advanced Eco No MetricsUploaded bymalikaygusuz
- Asian Countries of InterestUploaded byDaniel Bogiatzis Gibbons

- KAUST Grad ProgramsUploaded byNorberto Ventura
- CalCurveComparisons_2013.pdfUploaded byBrian Antonio Perez Balarezo
- two marksUploaded byapi-324108650
- Many-Objective Evolutionary Algorithms a Survey - B. Li, J. Li, K. Tang, X. YaoUploaded byLee
- GestureRecognition C# ParallelUploaded byOdheta
- TiseanUploaded byDavidWilliam
- Deep-drone-object 2Uploaded bySamuel Brand
- GENETIC ALGORITHM FOR TESTING WEB APPLICATIONSUploaded byCS & IT
- Chef and CombinatoricsUploaded byfullmetal
- Comparing Simulated Annealing and Genetic Algorithm in Learning FCMUploaded byFabián Seguel
- Programming in c Data Structures 14pcd23Uploaded byISHITA BASU ROY
- 01_2algorithms.pdfUploaded bysudhakarm13
- Engg100 SoUploaded bytchanlatif
- Machine HintsUploaded byDenno Stein
- Computation of Similarity Measures Sequential Data Using Generalized Suffix TreesUploaded byInformatika Universitas Malikussaleh
- Week1 Programming ChallengesUploaded byjames
- Familias 3 - Extensions and New FunctionalityUploaded by(unknown)
- NCTM ArticleUploaded bytfchung83
- M. Tech II Semester Q.P Oct 2015 Day 4Uploaded byBhÿråvá Sûrÿà Sàñgí
- A Methodology for the Synthesis of Red-Black TreesUploaded bydiegotorrent
- Genetic Algorithm in Matlab (Exemple )Uploaded byAstrid Lovasz
- Fast Hash Table Lookup Using Extended Bloom Filter: An Aid to Network ProcessingUploaded byhenderson
- CEG TEUploaded byBryan Lee Jun Han
- A Generalized Concept for Fuzzy Rule InterpolationUploaded byLyna Choo
- A Genetic Algorithm for Resource-Constrained SchedulingUploaded byHenne Popenne
- Peterson.psUploaded bypostscript
- 3.Mathematical Analysis of Nonrecursive AlgorithmsUploaded byIK Surya Negara
- 8Ai Question BankUploaded byRigobert Ngeleja
- Cpds Lab ManualdasdasUploaded bySangeetha Sangu BC
- Data Mining Discretization Methods and PerformancesUploaded byAashish Khajuria