Computer system

modeling and simulation

3. Queueing Model

Sosina M.
AAiT
Introduction
Queues occur whenever there is competition for limited resources
 The entities that queue for service are called customers/users/ jobs

 The limited resources are dispensed by servers

 waiting lines of requests form in front of these servers

Computer system modeling and simulation 2

Queuing system
A typical queuing system includes the following components
 Input population
 The source of units that become the customers to the system

 Arrivals
 The units from the population that enter the system seeking service

 Queue
 The line that houses the units that are awaiting their turn to be serviced
 Service facilities
 The place where the units are processed

 Departures
 The units that have completed their service and leave the system

Computer system modeling and simulation 3

Queuing system

Computer system modeling and simulation 4

Queueing systems
A single queue system
A system with multiple queues and servers

A system of interconnected queues

Computer system modeling and simulation 5

Queueing systems
Queueing models are models of systems providing service

Wide range of potential application areas

 Vehicular traffic
 Traffic signal, bottlenecks
 Banking
 Customer service
 Communication
 Transmission delay, medium access control, protocol evaluation
 Computer systems
 Parallel processing, client-server interaction , peer-to-peer
 and so on
Computer system modeling and simulation 6
Examples

Computer system modeling and simulation 7

Queuing systems
Two approaches to analyzing queuing systems
 Analytical modeling
 Abstraction of a system based on probability theory
 Represents the description of a formal system consisting of equations used to
estimate the performance of the system
 A better choice for a simple queuing system

 Simulation modeling
 Often used to analyze the complex queuing systems in which analytical methods
become intractable

Computer system modeling and simulation 8

Queueing models
Queueing models, in varying degrees of detail, are today the intrinsic
structures for studying many discrete-event systems

Queueing models are employed for designing and evaluating the

performance of queueing systems
 Server utilization, waiting line length, waiting time, etc.

Computer system modeling and simulation 9

Characteristics of queueing models
Key elements
 Customer – anything that arrives at a facility and requires service

 Server – any resource that provides the requested service

The calling population

 The population of potential customers

 Can be finite or infinite

 In an infinite population model

 The arrival rate is not affected by the number of customers being served and waiting
 In finite population mode
 The arrival rate to the queueing system depends on the number of customers being
served and waiting

Computer system modeling and simulation 10

Characteristics of queueing models
System capacity
 The number of customers that may be in the waiting line or system

The arrival process

Infinite-population models
 The arrival process usually characterized in terms of inter-arrival times of successive
customers
 Arrivals can be deterministic or random
 Random arrivals
 Interarrival times are usually characterized by a probability distribution
 Customers may arrive one at a time or in batches
 The batch may be of constant size or of random size
 The most common model- Poisson model or exponential inter-arrival time
Computer system modeling and simulation 11
Characteristics of queueing models
Infinite-population models
 Scheduled (deterministic) arrivals
 Interarrival times could be either constant or constant plus or minus a small
random amount

Finite population models

 The arrival process is characterized in a completely different fashion

 E.g., machine repair problem

 Pending customers- customers outside the queuing system
 Runtime – the length of time from departure from the queueing system until that
customer’s next arrival to the queue
 Runtime – exponential, Gamma, Weibull

Computer system modeling and simulation 12

Characteristics of queueing models
The machine repairman problem (finite population queueing model)

 There are M machines that are subject to breakdown and a single repairman to
fix them. Each machine operates independently of the other machines and its
time to failure is governed by an exponential distribution with parameter 𝜆.
Thus each machine fails at rate 𝜆 per unit time. The single repair man requires
1
𝜇 times units to repair a failed machine and again the repair process is
independent of the failure process and has an exponential distribution.

Computer system modeling and simulation 13

Queue behavior and queue discipline
Queue discipline
 Refers to scheduling algorithm to select the next customer from the queue

 Common algorithms
 First come first out (FIFO)
 Last in first out (LIFO)
 Service in random order (SIRO)
 Shortest processing time first (SPT)
 Service according to priority (PR)
 Can be either preemptive or non-preemptive

Computer system modeling and simulation 14

Queue behavior and queue discipline
Service times and the service mechanism
 Describes the number of servers and the distribution of the service time

 The service time may be constant or of random duration

 Service times of successive arrivals {S1, S2, S3, …} are usually characterized as a
sequence of IID random variables

 Distribution used - Exponential, Weibull, gamma, lognormal and truncated

normal

Computer system modeling and simulation 15

Queue behavior and queue discipline
Example 1 – a discount warehouse
 Customers may either serve themselves or wait for one of the three clerks

 Finally leave after paying a single cashier

Computer system modeling and simulation 16

Queue behavior and queue discipline
Example 2- a candy manufacturer
 a production line that consists of three machines

 The first machine makes and wraps, the second packs 50 pieces in box, the third
machines seals and wraps the box

Computer system modeling and simulation 17

Queueing notation
Different types of queueing systems
 Kendall ’ s notation A/B/c/N/K
 A represents the interarrival time distribution
 B represents the service time distribution
 c represents the number of parallel servers
 N represents the system capacity
 K represents the size of calling population
 The common symbols for A and B
 M (exponential or markov), D (constant or deterministic), Ek (Erlang of order k),
G(arbitrary or general)

 Example M/M/1/∞/∞ (in short M/M/1)

Computer system modeling and simulation 18

Performance of Queueing systems
The purpose of building a queuing model and running a simulation is to
obtain meaningful statistics

Long run measures of performance of queueing systems

 Long-run time average number of customers in the system (L) and in the queue
(LQ)

 The long run average time spent in system (w) and in the queue (wQ) per
customer

 Server utilization (portion of time that a server is busy) (ρ)

Computer system modeling and simulation 19

Time-average number in system L
Consider a queueing system over a period of time T
 Let L(t) denotes the number of customers in the system at time t

 The time-weighted average number in a system

1 ∞ 1 𝑇
𝐿= 𝑖=0 𝑖𝑇𝑖 = 𝐿 𝑡 𝑑𝑡
𝑇 𝑇 0

L- which is called the long run time average

number in the system
1 𝑇
𝐿= 0
𝐿 𝑡 𝑑𝑡 →L as T→ ∞
𝑇
Computer system modeling and simulation 20
Time-average number in system L

If simulation run length T is sufficiently long, the estimator 𝐿 becomes

arbitrarily close to L

The number of customers waiting in line

1 ∞ 𝑄 1 𝑇
𝐿𝑄 = 𝑖𝑇
𝑖=0 𝑖 = 𝐿 𝑡 𝑑𝑡
𝑇 𝑇 0 𝑄

Computer system modeling and simulation 21

Average time spent in system per customer w
If the queueing system is simulated for period of time T
 Record the time each customers spends in the system (W1, W2,…,WN)

 N = the number of arrivals during [0 T]

1 ∞
𝑊= 𝑖=0 𝑊𝑖
𝑁

1 ∞ 𝑄
𝑊𝑄 = 𝑊
𝑖=0 𝑖
𝑁

Computer system modeling and simulation 22

The conservation equation
Conservation equation
 λ =arrival rate

 𝑊= average waiting time

 Then, 𝐿 = λ 𝑊

Proof
𝑁 𝑇
 𝑖=1 𝑊𝑖 = 0
𝐿 𝑡 𝑑𝑡

1 𝑇 𝑁1
𝑖=1 𝑊𝑖 =λ𝑊
𝑁
 𝐿= 𝐿 𝑡 𝑑𝑡=
𝑇 0 𝑇𝑁

Computer system modeling and simulation 23

Server utilization
Server utilization – the portion of time that a server is busy

𝐵𝑢𝑠𝑦 𝑝𝑒𝑟𝑖𝑜𝑑 𝑛𝑜 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑎𝑟𝑟𝑖𝑣𝑒𝑑 𝑖𝑛 𝑇 ∗ 𝑠𝑒𝑟𝑣𝑖𝑐𝑒 𝑡𝑖𝑚𝑒 λ𝑇 ∗ 1/𝜇

𝜌= = =
𝑜𝑏𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛 𝑝𝑒𝑟𝑖𝑜𝑑 𝑇 𝑇 𝑇

Server utilization in G/G/1/∞/∞ queues

λ
𝜌=𝐸 𝑠 λ=
𝜇

Server utilization in G/G/c/∞/∞ queues

λ
𝜌=𝐸 𝑠 λ=
𝑐𝜇

Computer system modeling and simulation 24

Multi-server queues with Poisson arrivals
M/M/c/N/∞

If an arrival occurs when the system is full, that arrival is turned away and
doesn’t enter the system

The effective arrival rate (λ 𝑒 ) – the mean number of arrivals per time unit
who enter and remain in the system
 λ 𝑒< λ

 λ 𝑒 =λ 1 − 𝑃𝑁 1 − 𝑃𝑁 =the probability that a customer upon arrival will find space and be able to

enter the system

Computer system modeling and simulation 25

Example
Suppose your computer center currently consists of a single server of speed μ. Jobs
arrive according to a Poisson process with rate λ, and their service times are
Exponentially distributed. Suppose the current response time is considered
intolerable by the users. A second, faster server, running at speed αμ (for α >1), is
purchased and added to the system. Analysis the mean response time of the system.
 What is the response time if the original server is entirely disconnected (i.e., simply letting the
faster server run by itself).

Computer system modeling and simulation 26

Example

Poisson (λ) μ1
1. The original system

μ1
2. A second (faster) server is added to the system
Poisson (λ)
μ2

Poisson (λ) μ2 3. The original server is disconnected

Computer system modeling and simulation 27

Example

μ1
4- A separate queue for each server
Poisson (λ)
μ2

Computer system modeling and simulation 28

Network of queues
Queueing network models have been extensively applied to represent and
analyze resource sharing systems, such as production, communication and
computer systems.

Queuing networks are classified into types: open and closed

 Closed networks - the number of customers, i.e., the population, in the network is
constant

 Open networks – customers may arrive from (depart to) places outside the
network

Computer system modeling and simulation 29

Open queueing networks
a) A single queue system - a customer arrives at a single sever system, waits in the buffer
until the system becomes available, departs when the processing is complete. E.g., A
barber shop with one chair, client-server system with one server, etc.

b) A multi-server system

Computer system modeling and simulation 30

Open queueing networks
c) A customer requires k distinct types of service in series

d) Introduces the concept of quality control in which a tester determines whether or

not the work performed on items is satisfactory. When it is not, the item must be
reprocessed by the server.

Computer system modeling and simulation 31

Open queueing networks
e) A customer requires k distinct types of service in series

Computer system modeling and simulation 32

Closed queuing networks
 Customers/Jobs leaving the system immediately re-enter it

Computer system modeling and simulation 33

Network of queues
Many systems are naturally modeled as networks of single queues
 Customers departing from one queue may be routed to another
 Provided that no customers are created or destroyed in the queue, the departure rate out
of a queue is the same as the arrival rate into the queue, over long run
 Throughput = μ ∗ P the server is bus + 0 ∗ P the server is idle = μ ∗ ρ
 If customers arrive to queue i at rate λ 𝑖 , and a fraction 0 ≤ 𝑃𝑖𝑗 ≤ 1 of them routed to
queue j upon departure, then the arrival rate from queue I to queue j is 𝝀 𝒊 𝑷𝒊𝒋

 The overall arrival rate into queue j, λ 𝑗 , is the sum of the arrival rate from all sources
 𝛌 𝐣 = 𝐚𝐣 + 𝐚𝐥𝐥 𝐢 𝛌 𝐢 𝐏𝐢𝐣

 If queue j has cj parallel servers, each working at rate 𝜇𝑗 , the long run utilization of each
𝝀𝒋
server is 𝝆𝒋 =
𝒄𝒋 𝝁𝒋
Computer system modeling and simulation 34
Example
Considering that the input arrival processes from outside of the network are independent
and Poisson with mean rates shown in the Fig. The message transmission times are
exponentially distributed with mean rates for the different queues shown in Fig.
 Determine the mean delay experienced by a message from input to output of the network

Computer system modeling and simulation 35

Example
𝜌
 𝐿=λ𝑊 =
1−𝜌

Average number of customer in the network

𝑚 𝜌𝑖
𝐿= 𝑖=1 1−𝜌
𝑖

Average delay through the network

1 𝜌𝑖
𝑊 = 𝑚
where λ=the total average load on the network
λ 𝑖=1 1−𝜌
𝑖

Computer system modeling and simulation 36

Example 1
1) Consider a server farm with a central dispatcher and 3 homogeneous hosts (servers). Each
arriving job is immediately dispatched to one of the hosts with the fewest number of jobs for
processing. Jobs arrive from outside according to a Poisson process with rate λ = 3 job/s. Once
jobs are assigned to a host, they are processed there in FCFS (first come first served ) order. Job
sizes (service requirements) are Exponentially distributed with rate μ =2 s/job.
I. Assume that rather than a queue at each host, jobs are pooled at one central queue. When a host is
done working on a job, it grabs the first job in the central queue to work on.

II. A power saving scheme is implemented to reduce the power consumption of

the servers. The servers go to sleep mode immediately after finishing the
current job if there is no job waiting in the queue and wake up only when the
number of jobs in the queue is greater than 3. How is the system response
time affected by the power saving scheme?

Computer system modeling and simulation 37

Example 2
2) Consider a communication system with the following settings:
 There are two types of packets, high and low priority packets. For each packet type, there
is a separate queue and a FIFO queue discipline is applied. Packets in the low-priority
queue are served only if there is no packets in the high-priority queue.

 The packets are transmitted over a communication link with a capacity of 100Mb/s and
the packet length distribution follows an exponential distribution with a mean 25Mb.

 The packets arrive to the system according to a Poisson process at an average rate λ =
2 packet/s. The probability that an arriving packet belongs to a high priority class is 0.3.

Computer system modeling and simulation 38

Examples 3
3) Your system consists of a single CPU with finite buffer capacity. Jobs arrive according to a
Poisson process with rate λ jobs/sec. The job sizes are Exponentially distributed with mean
1/𝜇 seconds. Jobs are serviced in FCFS order. Let N − 1 denote the maximum number of
jobs that your system can hold in the queue. Thus, including the job serving, there are a
maximum of N jobs in the system at any one time. If a job arrives when there are already N
jobs in the system, then the arriving job is rejected.

I. How to reduce the loss probability in the system?

Computer system modeling and simulation 39

Example 4
4) Consider this common congestion management scheme: Jobs are served in an M/M/1
queue, provided that the number of jobs is no more than 𝑇ℎ𝑖𝑔ℎ . Once the number of jobs
hits 𝑇ℎ𝑖𝑔ℎ , a second server is immediately added, creating an M/M/2. The second server
continues to be utilized until the number of jobs drops to 𝑇𝑙𝑜𝑤 , at which point the second
server is removed, and we are back to an M/M/1, and the process repeats. Assume that jobs
arrive according to a Poisson process with rate 𝜆 and that job sizes are Exponentially
distributed with rate 𝜇.

Computer system modeling and simulation 40

Example 5
Consider a supercomputing center with k servers and no waiting room, where outside
arrivals occur according to a Poisson process with rate 𝜆. Assume that with probability pi an
arriving job is of “type i,” which means that the job requires i servers simultaneously. If there
are fewer than i servers free, the type i job is dropped. Otherwise the type i job grabs the i
servers that it needs and holds these for 𝐸𝑥𝑝(𝜇𝑖) time, after which it releases all i servers at
once.

Computer system modeling and simulation 41

Example 6
6) Messages arrive at a node of a telecommunication network to be transmitted on an
output line. From measurements, we know that the arrival process and the
service process are characterized as follows:

 Interarrival times υ are distributed so that E[υ2] =2E[υ]2.

 The message service time, τ, has a distribution so that E[τ2] =E[τ]2.

A suitable queuing model should be envisaged for this system in order to determine
the mean delay experienced by a message to cross the node.

Computer system modeling and simulation 42

Example 7
7) We consider a transmission system, where we have N input traffic flows
(each characterized by an independent Poisson arrival process of packets
with mean rate λ), which correspond to distinct buffers served by a shared
transmission line. Let τ denote the packet transmission time. The
transmission line serves the different buffers cyclically: it transmits a
packet from a buffer (if it is not empty) and then switches instantaneously
to service the next buffer according to a fixed service cycle.
I. We have to determine the mean delay experienced by a packet from its
arrival at the system to its departure

Computer system modeling and simulation 43

Example 8
8) Let us consider the telecommunication network in the Fig. below. This network is
composed of nodes interconnected by links. The network operates a form of connection
admission control on the arriving messages from outside in order to block the excess
traffic. We model this control by considering that an arriving message is refused (i.e., not
admitted, blocked) by the network with probability Pb. Knowing that the total input traffic
to the network has a mean rate λ and that the total mean number of messages in the whole
network is N, it is requested to evaluate the mean delay experienced by an accepted
message in order to cross the network.

Computer system modeling and simulation 44

Further reading
Sokolowski, J.A. and Banks, C.M. eds., 2011. Principles of modeling and
simulation: a multidisciplinary approach. John Wiley & Sons.

Banks, J., CARSON II, J.S. and Barry, L., 2005. Discrete-event system simulation,
fourth edition.

Lavenberg, S.S., 1989. A perspective on queueing models of computer

performance. Performance evaluation, 10(1), pp.53-76.

Harchol-Balter, M., 2013. Performance modeling and design of computer systems:

queueing theory in action. Cambridge University Press.

Computer system modeling and simulation 45