Lecture 25

What is Queuing Theory?

BE (CS)
Spring 2022 Semester ➢ Queuing theory
▪ Study that quantifies the phenomenon of waiting in
lines using representative measures of performance,
CS-438: Computer Systems Modeling
such as
o average queue length,
o average waiting time in queue, and
Lectures 25-27 o average service time.
Basic Structure Of Queuing Models ▪ It uses queuing models to represent the various types of
queuing systems that arise in practice.
▪ Advantages of queuing models
Dr Syed Zaffar Qasim o Formulas exist for each model
Assistant Professor (CIS)
o very helpful for operating a queuing system in the
1 most effective way. 2

The Basic Queuing Process

What is Queuing Theory?
▪ Customers requiring service are generated over time by
▪ The models enable finding an appropriate balance an input source.
between the cost of service and the amount of waiting.
o Providing too much service capacity to operate the
system involves excessive costs.
o But not providing enough service capacity results in
excessive waiting and all its unfortunate Fig 1: The basic queuing process
consequences.
▪ These customers enter the queuing system and join a
▪ The operating characteristics of queuing systems are
queue.
determined largely by two statistical properties,
namely, ▪ The required service is performed for the customer by
the service mechanism after which the customer leaves
o the probability distribution of inter-arrival times and
the queuing system.
o the probability distribution of service times.
▪ At certain times, a member of the queue is selected for
3
service by some rule known as the queue discipline.
4

CSM (CS-438) 1
 Input Source (Calling Population)
▪ Calling population: population from which
arrivals come.
▪ One characteristic of the input source is its size.
o total number of distinct potential customers
that might require service from time to time.
▪ The size may be assumed to be either infinite or
finite.
▪ An infinite source is forever abundant (e.g., calls
arriving at a telephone exchange).
▪ Because the calculations are far easier for the
infinite case, this assumption often is made even
when the actual size is some relatively large finite
number.
Input Source (Calling Population)
▪ The common assumption: Poisson process; i.e.,
o the number of customers generated in a given time
has a Poisson distribution.
o Arrivals to the queuing system occur randomly but
at a certain fixed mean rate, regardless of how many
customers already are there
o so the size of the input source is infinite.
▪ An equivalent assumption: the probability
distribution of the time between consecutive
arrivals is an exponential distribution.
CSM (CS-438)
Input Source (Calling Population)
▪ The finite case is more difficult analytically
because
o the number of users in the queuing system
affects the number of potential users outside
the system at any time.
▪ However, the finite assumption must be made if
o the rate at which the input source generates
new users is significantly affected by the
number of users in the queuing system.
▪ The statistical pattern by which customers are
generated over time must also be specified.
5 6
Queuing Behavior
▪ The queuing behavior of customers plays a role
in waiting-line analysis.
▪ Human customers may jockey from one queue
to another in the hope of reducing waiting
time.
▪ They may also balk from joining a queue
altogether because of anticipated long delay, or
▪ They may renege from a queue because they
have been waiting too long.
7
8
2
 Queue
▪ The queue is where customers wait before being
served.
▪ A queue is characterized by the maximum
permissible number of customers that it can
contain.
▪ Queues are called infinite or finite, according to
whether this number is infinite or finite.
▪ The assumption of an infinite queue is the standard
one for most queuing models.
▪ However, for queuing systems where this upper
bound is small enough that it actually would be
reached with some frequency, it becomes
necessary to assume a finite queue. 9 10
Service Mechanism
▪ The service mechanism consists of
o one or more service facilities,
o each of which contains one or more parallel service
channels, called servers.
▪ If there is more than one service facility, the customer
may receive service from a sequence of these (service
channels in series).
▪ At a given facility, the customer enters one of the parallel
service channels and is completely serviced by that
server.
▪ A queuing model must specify the arrangement of the
facilities and the number of servers (parallel channels) at
each one.
▪ Most elementary models assume one service facility with
either one server or a finite number of servers. 11
CSM (CS-438)
Queue Discipline
The order in which members are selected from a
queue
▪ An important factor in the analysis of queuing
models.
▪ First-come-first-served (FCFS) usually is assumed by
queuing models, unless stated otherwise.
▪ Other disciplines include Last Come, First Served
(LCFS) and Service In Random Order (SIRO).
▪ Customers may also be selected from the queue based
on some order of priority.
▪ For example, rush jobs at a shop are processed ahead
of regular jobs.
Service Mechanism
▪ Service time (or holding time): The time elapsed from
the commencement of service to its completion for a
customer at a service facility.
▪ The service-time distribution that is most frequently
assumed in practice
o (largely because it is far more tractable than any
other)
o is the exponential distribution.
▪ Other important service-time distributions are
o the degenerate distribution (constant service time)
and
o the Erlang (gamma) distribution.
12
3
 Cost-based queuing decision model
➢ Cost optimization model: we seek the minimization
of the sum of two costs:-
o the cost of offering the service and the
o cost of waiting.
▪ Fig 2 depicts a typical cost model (in Rs per unit time).
Fig 2
▪ The main obstacle: difficulty of obtaining reliable
estimates of the cost of waiting, particularly when
human behavior is involved. 13
An Elementary Queuing Process
▪ A server can be a human, a machine, a vehicle, an
electronic device, etc.
▪ By the same token, the customers in the waiting line can
be people, computer jobs in a queue.
▪ For example, they may be items waiting for a certain
operation by a given type of machine, or they may be
cars waiting in front of a tollbooth.
15
CSM (CS-438)
Lecture 26
An Elementary Queuing Process
▪ Queuing theory has been applied to many different types
of waiting-line situations.
▪ The most prevalent type is the following:-
▪ A single waiting line (which may be empty at times)
forms in front of a single service facility, within which
are stationed one or more servers (fig 3).
Fig 3: An elementary
queuing system
▪ Each customer generated by an input source is serviced
by one of the servers, perhaps after some waiting in the
queue (waiting line). 14
An Elementary Queuing Process
▪ It is not necessary that there actually be a physical
waiting line forming in front of a physical structure
that constitutes the service facility.
▪ The members of the queue may instead be scattered
throughout an area, waiting for a server to come to
them, e.g., machines waiting to be repaired.
▪ The server or group of servers assigned to a given
area constitutes the service facility for that area.
16
4
 An Elementary Queuing Process
▪ Many of the models further assume that
o all inter-arrival times are independent and identically
distributed and that
o all service times are independent and identically
distributed.
▪ Such models conventionally are labeled as follows:-
o where M = exponential distribution (Markovian), ,
o D = degenerate distribution (constant times),
o Ek = Erlang distribution (shape parameter = k),
o G = general distribution (any arbitrary distribution
allowed).
Terminology and Notation
▪ Unless otherwise noted, the following standard
terminology and notation will be used:-
1. State of system n(t) = number of jobs in queuing
system at time t (t ≥ 0).
2. Queue length = number of jobs waiting for
service to begin = state of system minus number
of jobs being served.
3. Pn(t) = probability of exactly n jobs in queuing
system at time t, given number at time 0.
4. s = number of servers (parallel service channels)
in queuing system.
CSM (CS-438)
An Elementary Queuing Process
▪ For example, the M/M/s model assumes that
o both inter-arrival times and service times have an
exponential distribution and that
o the number of servers is s (any positive integer).
▪ The M/G/1 model assumes that inter-arrival times
have an exponential distribution,
o but it places no restriction on what the
distribution of service times must be,
o whereas the number of servers is restricted to be
exactly 1.
17 18
Terminology and Notation
5. n = mean arrival rate (expected number of
arrivals per unit time) of new jobs when n jobs are
in system.
7. µn = mean service rate for overall system
(expected number of jobs completing service per
unit time) when n jobs are in system.
▪ When n is a constant for all n, this constant is
denoted by .
▪ When the mean service rate per busy server is a
constant for all n ≥ 1, this constant is denoted by µ.
19 20
5
 Terminology and Notation Lecture 27
Terminology and Notation
▪ (In this case, µn = s µ when n ≥ s, that is, when all s servers
are busy.)
▪ Certain notation also is required to describe steady-
▪ Under these circumstances, 1/ and 1/µ are the expected state results.
inter-arrival time and the expected service time, respectively.
▪ Also,  =/(sµ) is the utilization factor for the service ▪ When a queuing system has recently begun
facility, i.e., operation,
o the expected fraction of time the individual servers are o the state of the system (no. of jobs in the system)
busy, will be greatly affected
o because 1/(sµ) represents the fraction of the system’s ❖by the initial state and
service capacity (sµ) that is being utilized on the average
❖by the time that has since elapsed.
by some arriving customer.
▪ The system is said to be in a transient condition.

21 22

Terminology and Notation

Terminology and Notation
▪ The following notation assumes that the system is in a
▪ However, after sufficient time has elapsed, the steady-state condition:
state of the system becomes essentially
independent of the initial state and the elapsed
time (except under unusual circumstances).
▪ The system has now essentially reached a steady-
state condition, where the probability
distribution of the state of the system remains the
same (the steady-state or stationary distribution)
over time.
▪ Queuing theory has tended to focus largely on the
steady-state condition.

23 24

CSM (CS-438) 6
 Relationships between L, W, Lq, and Wq
▪ Assume that n is a constant  for all n.
▪ It has been proved that in a steady-state queuing
process,
L = W
▪ Because John D. C. Little provided the first rigorous
proof, this equation sometimes is referred to as Little’s
formula.
▪ Furthermore, the same proof also shows that
Lq = Wq
▪ If the n are not equal, then  can be replaced in these
equations by 𝝀,ത the average arrival rate over the long
run.
Relationships between L, W, Lq, and Wq
▪ Now assume that the mean service time is a constant,
1/µ for all n ≥ 1.
▪ It then follows that
▪ These relationships are extremely important because
o all four fundamental quantities: L, W, Lq, and Wq
can be immediately determined
o as soon as one is found analytically.
▪ This situation is fortunate because
o some of these quantities often are much easier to
find than others
o when a queuing model is solved from basic
principles.

25 26

Relationships between L, W, Lq, and Wq

Problem
Suppose that statistical analysis shows that average
arrival and service rates in a database server are 20
and 25 per secs. Also average number jobs in a
queue is estimated as 5. Compute the average
waiting time (in queue), average waiting time (in
system) and average number of jobs in the system.

27

CSM (CS-438) 7