Models

Queuing

KEY POINTSOF THE CHAPTER

Prcducerhonsumer analysisshouldbe usedlo determrne theappropnate

buffersizesfor buffereddata when productioncan exceedconsumption
for shonbu$ts
: . Because of lheir inherentdigitalnature,discreteprobabilitymodelsare
bestusedwhenmodelingreal time systems
3 . An MM/l qDeue canbeusedlo modela simpleinteruptdrivenreaftime
system

Thischapterprovtdesa brief infioductionto queuingtheoryandits application in

rhe analysisof real-timesystems,and discusses application in the prcdiction
of
buffer sizes and in performancemeasutementThe reader is assumedto be
familiarwith the conceptsof expenment, event'samplespace,andprobabilityof
occurenceInterested readerswho wish to pursuetheslDdyof queumgtheoryin
greaterdetailshouldconsultthe treatments gilen in [28] and [82]'

10.1 PROBABILITYFUNCTIONS

The most importantconceptin the mathematical modelingof a sysem is the

rardom variable.A randomvariahle is a funclionthat maPselementsof the
samplespace,thesetof outcomes to someexPdment'into ihe real line Iftle
samplespaceis discrete,thentherandomvariableis sajdrobe a dtscrete roMom
sPace
rdri;ble. If thesample is continuous, thenthe random is
variable sardto be
 Chap.i0 r QueuingModels

a continuousrandomran,rble Wewill usetheappropriate discreteor continuous

randomvariablesand associated probabilitydrstnbutions
to descnbeprocesses
depending on thenafureof th€underlyingsamplespaceFor example,thearrival
timeofan interuptcanbemodeledby a continuous randomvariable,wher%sthe
numberof packetsreceivedby a cefiainprocessis modeledby a dtscrete
random
variable.

10.1,1Contlnuous
The continuousreal-valuedfunction/is called a prcbabilit\t densit,f nctnn or
probabiliry distribwion for t}le continuousnndom vaiable X lf the followine
conditlonshoid:

t . flr) > 0 for all -r e ill

2. flr) d, = |
3 . The prcbability that the randomvariableX takeson a value betweena and lr,
denotedp(a < X < b) is givenby
p(a<x<b) = 1 \ x d) , t
Note that for a confinuousprcbability drstriburion and conslant c, the value
p(,r =.)=0 Moreover,recall that the "averagevalue," or mea, ofthe density
functronf A denoted p whie ils rariance, a fieasure of how widely scafteredthe
values ofr are from the mean. is denoted.'2.
Some common continuousprobabilily dntnbutions include exponential,
Weibull, and Caussian.

I EXAMPLE IO.I
TheGaDsieor nomaldislnbudon
fo! a random
variable
X is givenby
I G-p),
hlx) = E=a ,c (10r)

with meanp andvaaanceo': An exampleoi sucha densiryfuncnonwirh F = 0 andd, = I is given

in Figure101 I

I EXAMPLE10.2
The exponentialdistribut'o.nfor a randod vanabler is given by

/d=L " (10.2)

The "average"valDefor rhisdisrriburion
can be shoM lo be i t821.The graph
disriburion b shown in Figrre l0 2 I
Sec 102 I DiscreE u3

P =0

Figur€ l0.l The Gaussanprobabllltyfunction{ith F = 0 and0: = |

disLribulion
Figure l0: The exponential

I EXAMPLE,1O.3
? is givenby
for a rdndofrvariable
TheWeibulldisLriburion
lil)= "BP-'--'/>o 00r)

densityfuncdonfor vdious values

whered > 0 md p > 0. The graphsof theWeibdllProbability
of rhepamete^ o andp aregivenin FLg!rel0 3 I

10.2DISCRETE
The real-valueddiscretefunction/ is a called a prcbabiliry distributianfunction
or probabilit! dislributio, for the discreterandomvariableXifthe folowing three
conditionshold:
 Chap l0 I QueuingModels

Figure 10.3 The weibull prcbability tunction for different p and o = l

1. /(i) > 0 for all lalues t € X.

2. >J€) = l. wherer rangesover values of X.
3 The probability that the random variable X tates on a value i, denoted
p(X = .r), is siven by /(i).

Somecommondiscrcteprcbabilitydistdbutionsincludea discreteform of
unifom distribution,ihe binomialdisttibutlon,and thePoissondistribulioL

I EXAMPLE10.4
A fEquendyuseddiscEleprobabity distibnlion,lbe Poissondishbutionis usedlo
phenon€nain which the occurence of individual eventsis independeniof my Previou
Fomally, it rs definedas
(r 0' --
P,(t =
k!
with t,/ > 0 Plr) de.otes the probabihty of t occuren@s in time r, atd I r ihe
@cuEencesper unit lime. Nore that the probrbility distribution of the inlemrivdl lime 'rL
betw@n two consecutiveevents)in a Poissonproce$ is given by $e ex?onential
disdbulionwith meani.

The Poissonprccessis a good model for many naturally occuring

phenomena.In a Poissonprocess,the prcbability distribution for lhe
occufiences in time I is Poisson, and the probability distribution of &
between occufences is exponential We will be ushg ihese lwo
distibutronsexclusively.
S€c.10.3t BasicBulTerSizeCalculat'on
u5

10.3 BASICBUFFEBSIZECALCULATICIN

Recallthata bufferis a sel of memo4/locationsthatprovidetemporarystorage

for dafathat arebeingpassedfrom one processto ailother,whenthe receiving

calleda burst perncl.

imaginean oil funrrelplacedoverthecrankcase
To visualizethis scenario,

beinglost?The next sectionsseekto examinethis problem

Figure10.4 Butreringsystem

10.3.1HandlingBurstsol Data
If both P(l) and C(t) arc constant,denoted P and C, respectively-itnd ii'r^'
consumption mte C is grealer than or equal to P, then no buffer is needcd since +lie

handle the ovedlow a burst of penod f, note that the total dala Produced is Pa
while the total data coosumedin that time is CZ Thus, there is ail excessol
246 Chap.l0 r QueuingModels

(P C)T ( 1 05 )
Thrsis how muchdatamustbe storedin thebuffer.Thus,lhe buffersizeis
B=(P C)T (106)
$here C is the consumptionrate, P is the production rate, and I is the burst
time

I EXAMPLE 10.5
is prcviding
A device datato a realtimecomputer
viaDMAat9600bytevsecond
in bustsof one'
seconddurdtronevcry20 secondsThe compdLer is clpableof processi.gihe data,t E00bytev
secondAssumrng thereis suliicientdne ro emprylhe bufferbeioreanotherbu6t occurswhat
shouldLheninidum buiiersizebe?Usingequationl0 6 yields
B = (9600 E00)I = 8800bytes

Handlingdala that occur in burstswilh equatlon106 is possibleonly if the

buffer can be emptledbeforeanofherbulst occurs.For example,in Exanple
105, emptyingthe bufferw l take11 seconds sufTicrent time beforefhe next
expected burst If bwstsoccurfoo ftequently,thenno stzebufferw l avail.In
thjs casethe systemis unstable,and eitherupgradrng the processoror down-
gradingthe productionprocessis necessary to solvethe problem

10.3.2 Variable Buffer Size Calculation

It is oftennot accuateto assumethatproductionandconsumption ratesaswell
as productionfimesare fixed; they wili ftequentlybe variable.
In thrs sectionwe examinea casem which prcductionand consumptron
rates are variable Although we will pedolm the analysis for continuous
variables,the results can be adaptedfor discretevanablesby replacmg
summaiionfor integrahonm the formulas.In addltion, they can be generalized
to handleboththediscrercandcontinuous casesby employilgRiomann-Stieljes
integration(see,for example, [163])
Suppose we havea producerprocessthatproducesdan at a rategivenby
the functionp(t) AIso supposethat anothertask consumesor usesthe data
producedby the first task at a late deteminedby the funchon.(l). The data
are producedduringa burstpedodf = t2 -,r, wheret1 and 12represent ihe
start and finish times of the burst- Then the buffer size neededat time I > 4
can be exDressed as
B(7) = tp(r) - c(t)) dt (r0.7)
Thus, we need to know when B(O is a maximum to detemine the rcquied
buffer size. Note that when p(l) and c0) are constant,then equation10.7
becomesequation10.6(seeExercises).
S€c 10I r BasicBufferSizeCalculation

I EXAMPLE10.6
dataal a rare(in bytesper second)Iharis d.refr,nedby thefuncrion
A taskproduces

wth r.oprcsenting
theblnt periodThe dalam c.nsutuedby a raskat a ratedeterninedby rhe

The eraphsfor pG) and.G) de depidedin FigureI0 5 The quest'onthenis: If thebuat periodis
knownto be 1.5seconds,whatis thenaessarybuffersize?In ibis caseit canbe seenby inspecling

FiguE 10.5 Producered consumd distribnlion fu.ctions

t h ea r e aui n d etrt l ec w e s f o r 4 r ) e d . ( 4 r f i a A
r 'drn m|B rp(r=d/,orr=1.6 Plueerng
dds inio equalion 10.?,
B(16) = Ip<t)- c<t)ldt
''
= 10000 ltt - tt4t)l.h + l {ffi 1 l\2 t) ttatldl
28 Cbap.10 r Queuin8Models

^ l r 1 1 6 - l r q
I00o0ll/8)d lo + l0il00(2)
' l, 10000(5/E)
r L

10000(3/8) - 2)- 10000(5/8X2

+ l0(X,0(3.2 56- l)

6000bytes

If the burstperiodis also variably detemined by function r.(r), then for a

bursloccuringat trmelt,
tu\tt) .
l lr(r\ - c(t\ldt 110.8)
I EXAMPLE10.7
In theprevjDus
exmple.if lhebusr sta$sat a ijmederemined
by theGausio distribution

-T
L(t) = == e-

tnenu(0) = 0.053991Inspecting
tJreproductiondd consDnprion
cbes againsbowsthar tte
ddinum butrersib is neededqben i = 0.053991RecatculaLion of rhe butrersize now
yields
B(os199l) = Iplt) - c(t)l dl

= l0m0 I$ - 1t4t)l dl

= loooo(3iE)
I i""""'
= l00(n(3/8)t0-0029151
= l lbyles I

Determiningwhenth€rmaximumbuffer sizeis neededis easilydoneb)

graphingthe consumerand producerfunction cu es,andtheninspectingthemro
determinewhen the differ,ancein the areasunder the cuavesis atest, Use
equation10.7to determine this differenceand thus the maximumbuffer size.

10.4CLASSICAL T HEORY
QUEUING

The classicalqueuingprol)lem involves one or more prcdueerprocessescalled

seners alLdone or mort) coDsumerprocessescalled calomer,r, A standard
notationfor a queuingsyslem is a three-ruple.The first componentdescdbesrbe
probability dislribution for ihe time betweenarrival of customers,the secondi
the probabilitydistlibution of tjme neededto serviceeachcustomer,andthe third
is the number of servers (see Figure 10.6). ln a real{ime system,the firsr
cohponentmight be tbe arrival time probabilily distributionfor a certainintem.F
rcqDest,the secondwould b e the time neededto servicethat interruptrequestad
the third componentwoulC be "1" for a single-processingsystem,morc for
 249
I CldlsrcalQueujn8Theory

of a queuing
Figure10.6 ComPonenLs

systems
multiprocessing of thismodelto predict
WecanuseknownProperties
system
timesfor tasksin a real_time
service

'10.4.1The M/M/l Queue

available.
Fufiher study of the
*" *iU presenf these results without denvation
heorelicaia\pects
'' canbe foundinl28lorlS2l FofexamplelelNbethenumbel
of cu\tomer5
oi *.tot*rs tr'" qu"ue t'ellinB p = I/F rhenlhe averagenumber
in the queue in such a system is

(10.9)

with variance

J - (10101
(I p)'?

The averagetime a customerspendsin the systemis

I /!L (10.11
l-p
has pmb€bilit\
The random variable y for the dme spent In the system
distribution
s()') = - (10.1:,
P(1 P)tP('-Pt

with ) > 0.
250 Chap 10 r QueuingModels

Fmally,1tcarlbeshown[82] thattheprobabilitythatat leasttr customenare

in the queueis
P[>ftjosysteor]=pr ( 1 0r .3 )
In the MA4/l model, the probabilily of exceedinga certain number ol
custome$ in the system decreasesgeometrically, If interrupt requests are
consideredcustomersin a ceflain system,then two suchrequestsm the systemat
the same time (a trme overloadedcondition) have a far greaterprobability of
occurrencethan thrce or more such requests,Thus, building systemsthat can
toleratea single time-overloadwill contrjbutesignificantly to systemreliabllity,
while worrying aboutmultrpletime-overloadconditionsis probablynot wofth lhe
bother Let us now show how the MA4/1 queue can be used in real-tlme
syslems,

'10.4.2Serviceand ProductionRales

Consider an MA4/l system in which ihe customer reprcsentsan interupt

requestof a certain tlTe ard the se er reprcsentsthe processingrequired for
that req est. ln this single-processormodel, waiters m the queuercpresenta
time-overloaded condition.Becauseof the naiureof the arrivaland processing
times, this condition could theoretically occur. Suppose,however, that you
could vary either the arrival or the processingtimes. Varying the arrival time,
which is r€presentedby the parameterI could tle accomplishedby changing
hardwareor altenng the processcausingtbe irterrupt. Changingthe processng
time, representedby lhe parameterp, corild be accomplishedby one of the
techniquesdiscussedin Chapter9. In any case,fixing one of theqetwo
paramete$,and selecting secondparameterin such a way as to reducethe
probability that more thaDone intefrupt will be in the systemsimultaneously,
will ensfie that time-overloadingcannot occur wilhin a specific confidenc€
inreflal.

I EXAMPLE10.8
SuppoieI ihe meo inteluval time belweenintempt rcqDests,is tued to be l0 millisecondsl-e
us find themq4 processingtme, l, necessaryto gueetee that the prcbability of tine-ovedoadint
(nore thd one inienpt requestjn Lhesy3ten) is lets than 1',6 We cd use eqration 10.13as

P r > 2 i t r s y s ,=e(n; lr= , "

6.or= . _
. ltootXtot to1'
1/;
 5d 104 I Cla$icalQueuingTheory 2Sl

I
= < 0001 second

Thus,lhe meo gocessingtime,i, shouldbe no hore than I millisecond

to Sueanle€ftat dne_
cannotoccurwith 99%coifidence-
ove.loading I

I EXAMPLE10.9
Supposewe fix l. {he mean seflice line, to be 5 mrlliteconds,md seekto find the avenge aEivar
tine, i, necesary to gua.anreelhat the !rcbability ofiime ovedoadinsh les thd l% We can use
equation10.13 ds follows

0' <001

j il> ro

1
=+->0.050second
A

Hence, the averageinlelarival time beNeen two inrenll reqDestsshould be at least 50

ro gueanteeonly a l% risk of lime'overlodding
milliseconds I

g 10.4.3MoreBufterCalculalions
tg A secondmodel using ihe l\4A/Vl queueportraysthe "customers"as data being
placedin a buffer The "service"lime is lhe time neededto pick up thedataby
someconsumer task.Hereour k[owledgeof MA4/1 queuesis usedto calculate
he the average buffer size neededto hold the datausingequa'tion10.9,and the
rly' averagetime a datumsPends in the system(its age)usingequation1011

I E X A M P L E1 0 . 1 0
a, and h
A prccessprodncesdatawith ,ntermival tihes given by the eJgonflriat dist ibution rk
5'.
consumedby,apr@e$ at a dle Srvenby theexponentidldislribution5d To calculale$e avease
rdmberof dal! 'ren! In $e butrer,lse equdnon109:

- =P 4 = /- = 54
N
l13as l p l 4 / 5

The aveEge"age" of I dala item co be calcul.led with equation10111

rltL \15
7=-=-=lsecond
l-p 1 415

A prcbability distributionfor the randomvariabledeterminhgtb€ ate of tu

datumcan be found by qsinge4ualion10.12.
 Cbtp l0 I QueuingModels

10.4.4irEDPvrrev " - Modelinq

Response-Time lhe
_average responsedme for a prccesshandling _o an inkrrupt requeslin
:;1#;'fxr*,n.t*l#::l
il:;:JT:tJ":H"ililH"I
Tte
dblenceof other compenng
'"""
proc<:ssr!q'r' - -
'" rure
.,,re.l if an MA/V! nodel
the averagetime "pent in
In thrscaseute equal'on,
i\ assumed 'ime\
ll-^-. - ,,-.,
(therespon:e
request
i;:l'r'L# o,;iJ;;i
I EXAMPLE10.1'!

I1 ro be
en i' \eenIrcm equatonl0
I = 15 tnlttite"onos
I
5
r-l
3

Tlil:'tr"il:il
t':t"3kffi
.".*L*"i:oili'*''il*i",";-H;lf
glves

2
s(i\ = - e-
I

Notethattheresponsetimewillbedeleteriouslyaffectediftheinterruptmle
greaterihan ihe meanservlceIarc'

10.4.5OtherOueuingModels
or:*::;""f"f"'Tf:ti"J;l-1:
ina.vanetv
used
;" --;;.;be
;il*.X:,*ti'::*ilfi Tff:l','
$"X*.i.':iT ;.'l'#
,ilJf.':::t:i:'r*t*liig*,,**'H:."
i:':.T:ff rormodering
berrxed
'#Js can
f,::il:"H1ii'1l"T;#,lt*1ll$ffi:'"" "[ff;
queues
realisticfrmte-siz'ed
TffI
il.'Ii:'Jf"L"":1.0:IL,:.:i'liT:H:ii'H.'ff
'".,.'i+:i* iffi
l',*"i:il*l'"*ffi
..".;,'ff :$iff;l :r*I"l'.y,fJ:
Til:'t$
^trT:.T
3:Tl::T"*',
:l'ffi1'il"1'1T"ff# F*1ii;.*tt*:T?liTi
d; :i:r:'-:::":::.':*1'3"T;1'"
' *'*
i'#,H::ff il'ff ;;ffi+:'i':::
:ffi'Tl $:i:I"15"$fortime-overroads
lillttf i"illllii,i"Jii"*i.iJ"'i-t'"',"ots
Sec.10.6r Edang\ Fomula

10.5 LITTLE'S LAW

An imponantresultin queuingtheory,litfu'r 1dlr,hassomeapplication

in real_
timesystemperfomanceprediction.Lirtle s la\ twhichappearidin 196l) slares
thal the avetagenumberof customers
in a queuingsystem,N,_ is equalto the
avemgearrivalmte of the customers
to tha!system,r"u,timesthe avetagetime
sPentin thatsystem,1av
( 10.14)
If ft serversare present,$en
s (10.1s)
wherer",,av is the averagearival ratefor customers to serer i and l,.rv is lhe
averageservicetime for serverL
Viewingeachprocessasa seflerandinterrDpt arrivalsascusfomers, Little-s
law canbe usedas a roughestimate for CpU utilizationin sporadic,
periodic,or
mixed interrupt systems.
For example,a systemis knownto havepeiodic interup$ occufiingat lu,
20 and 100 millisecondsalld a spomdic interrupt that is known to occur on
avemgeevery I second.The averageprocessingtime for theseinterruptsis 3, E,
25,and30 milliseconds. Thenby Litde'slaw theavemgenumberof customers in
rne queuels
N," = (1/10)3+ (1/20)E+ (1/100)25+ (1/1000)30
= 0.98
This resultis thesameone\re obtainedby usingequation9.7 in Chapter9
of the!ext.Remembet thatalthoughlhisis anestimate
ofaverageCpU urilizadon,
fanstent overloadsmight still be present,as we will see in the followinc
secuon.

10.6 ERLANG'SFORMULA

assumethat the avemgeservice(process)rime is p and the averagearival time

(mterruptmte) is I, then the fraction of time that all serversare busy (a rifte-
overloadedcondition)is given by

(.\lp\-/nt
(10.r6)
>rd'Q/t')rlkl
This resultdatesbackto 1917!
25lt Chap.10 r Queuing

ApplyingErlang'sFormulato thepreviousexamplegivesm = 4, )r =
F = 1 65 :
(380^6 14 /4
= 0.81
l + (380/16.5)+ (390/16.5)2t2
+ (380/16.5)38!
+ (380/l6.r1t4l
This means there is a potenhal for time-ovedoading 83.4% of the time. Based
theaverage
timeloadingfigureof987, andwhatwe learnedfrom
analysis (707, or less timeloading is needed to guaGntee no overloacls ir
pefect world), this seems reasonable.

10.7 EXERCTSES

l. In Exanple l0 5 whar would b€ the requiredburer size lor a Crventine / if rhe

were at 2400 baud,2 secondslong, md occuEedevery 5 seconds?
2, A prodDcergeneratesdata at 1 byre pe. ?00 naoseconds in bu6ls of 64 ,ilobE
consumerce Md the data in 32 bit words bDr only at a Ere of one *ord dw
nicros€condsCalculatetbeminimumiizebutre..equiredto avoidspillovea
thereis enoughtime betw@r burststo empty the buffer
3. Show that q,hentbe producersd consumerlarks haveconstdl rates,thenequarjo!
becomes equation106
4, A prcduer prcce\ is l$oqn ro be rble'o proces daracr a are $d i<
dishbuL€dwrth averagesenice tine of 3 millisecoddsper dalum wllat is the
allowableaverage datadle if theprobabilityofcollisionis (o be0 l%? A$ume H
datadive at iniedals Lhatde erponentially dsrributed
5. Discu$ losible inle.pretationsof equarion10.10if N ls the eipeclednunberof
i.ems in somebutrn