Number of servers

In simple queueing systems, servers are identical

the simplified notation in queue where:

• The size of the population is infinite
• The service discipline is FIFO
 The concept of customer classes

A queueing system can serve several classes of customers

characterized by:
• different arrival processes
• different service times
• service priority according to their class
 Some queue performances

l(T) m(T)
L(T)

W(T)
• A(T) : number of customers arrived
• D(T) : number of departures
• l(T) : average arrival rate
• m (T) : average departure rate
• L(T) : average number of customers
• Wk: witing time of k-th customer in the system
• 1  
AT
W T   
A T  k 1
Wk average witing time
 Stability of the queueing system

l(T) m(T)
Queueing
system

Definition : A queueing system is stable if the number of

customers in the system remains finite.
Implication of the stability: l < m
N(t) : number of customers in the system

l
Poisson
arrivals
Exponentially
distributed service time
 Little's law

For a stable queueing system,

L = l×W

where
• L : average number of customers in the system
• W : average response time
• TH : average throughput rate

Queueing system
l m
L

W
 Stability condition of M/M/1 queue
Poisson arrival, exponential service, 1 server

The M/M/1 queue is stable iff

l<m
or equivalently
r<1
where
• r  l/m is called the traffic ratio, utility.

The number of customers in the system is unlimited and hence

there is no steady state when the system is not stable.
 Markov chain of the M/M/1 queue

When the system is stable, stationary probability distribution exists.

Let : pi is probability of existence i item in queue
 Steady state distribution of M/M/1 queue

state 0 : 1m   0 l

state 0-1:  2 m   1l

Balance equations 
state 0-1-...-n:  n 1 m   n l



Normalization equations:   n  1
n=0

With r  l/m,
p0  1 - p
p n = rn p 0
 Performance measures of M/M/1 queue

Ls = Number of customers in the queue = r/1-r) = l/(m-l)

Ws = Sojourn time in the system = 1/1-r)m = 1/(m-l)
Lq = queue length = l2/(m-l)m  Ls - r
Wq = average waiting time in the queue = l/(m-l)m  Ws - 1/m
l = departure rate ,
Server utilization ratio = r
Server idle ratio = P0 = 1 - r
P{n > k} = Probability of more than k customers = rk+1
 M/M/C queue

Exponentially distributed
service tim

l
Poisson
N(t) : number of customers
arrivals
in the system

N(t) is a birth and death process with

• The birth rate l.
• The deadth rate is not constant and is equal to N(t)m if
N(t)  C and Cm if N(t) > C.
Stability condition : l< cm.
 Steady state distribution of M/M/C queue
Distribution :
r l/m
pn = rn/n! 0, " 0 < n  C
n
r
 n C     C , "n  0
C
-1
 C -1 r n rC 
0     
 n 0 n! C !1 - r C  
 

m m m m

0 1 2 3

l l l l

Markov chain of M/M/2 queue

 Performance mesures of M/M/C queue
Ls = Number of customers in the system = Lq + r

Ws = elapsed time in the system = Wq + 1/m

Lq = Average queue length

Wq = Average waiting time = Lq / l

 = Average number of busy server,  = r

w = Waiting probability
= C + C+1 + ...
= C/(1-r/C)
 M/M/C with impatient customers
Steady state distribution :
r  l/m
Pn = rn/n! P0, " 0 < n  C
-1
 C rn 
P0    
 n  0 n! 
 

Pourcentage of lost customers = PC

Server utilization ratio = (1 – PC) l/Cm

Pn depends on the distribution of service time T only through its

mean, i.e. with m = E[T]
 Closed Queuing Network
Example

p m2
m1
1-p m3

p = 0.5, m1 = 4, m2 = 1, m3 = 2
N= 2, 3, 4
 Printing task queue
job1 job2 job3 job4 job5 job6

^ ^
| |
| |_________
| |

jobs removed here (dequeue) new jobs

added here
(enqueue)
 Queues
• Collection with access only to the item that
has been present the longest
• Last in last out or first in first out
• enqueue, dequeue, front, rear
• priority queues and deques
Deletion Front Rear Insertion

Data1 Data2 Data3 Data4

 A Model of a Queue

Head:
Tail:
All items are
All new items
deleted from
are added on
this end
this end
 Operations on Queues
• Insert(item): (also called enqueue)
– It adds a new item to the tail of the queue
• Remove( ): (also called delete or dequeue)
– It deletes the head item of the queue, and returns to the caller. If the queue
is already empty, this operation returns NULL
• getHead( ):
– Returns the value in the head element of the queue
• getTail( ):
– Returns the value in the tail element of the queue
• isEmpty( )
– Returns true if the queue has no items
• size( )
– Returns the number of items in the queue
 Different types of queue
1. Circular queue
2. Double Ended Queue
3. Priority queue
 Applications of Queues
• Direct applications
– Waiting lines
– Access to shared resources (e.g., printer)
– Multiprogramming
• Indirect applications
– Auxiliary data structure for algorithms
– Component of other data structures
 Priority Queues
Some data structures you have (probably) seen so far:
- Linked lists
- Hash tables
- Trees
- Graphs
- Stacks
- Queues

A (regular) queue has first-in, first-out (FIFO) behavior, where

items must enter at the rear and leave from the front.

In a priority queue items must leave from the front, but they
enter on the basis of a priority (key) value. The queue is kept
sorted by this value.
Priority Queue Operation Design

Create: inherit from Queue Info class,

add a number of items attribute
Full: return whether the number of items
is equal to the maximum array size
Empty: return whether the number of
items is equal to zero
Remove: for you to design
Add: for you to design
• FIFO (First In First Out) structure
• Elements inserted at the tail (Enqueue)
• Elements removed from the head (Dequeue)
• Useful in many situations
– Print queues, message queues, etc.
• Can be implemented in several ways
– Statically (using array)
– Dynamically (using pointers)
– Using the Queue<T> class
 Static Queue
• Static (array-based) implementation
– Has limited (fixed) capacity
– Implement as a “circular array”
– Has head and tail indices, pointing to the
head and the tail of the cyclic queue
 Sequence – Solution with a Queue
int n = 3, p = 16;
Queue<int> queue = new Queue<int>();
queue.Enqueue(n);
int index = 0;
while (queue.Count > 0)
{
int current = queue.Dequeue();
index++;
if (current == p)
{
Console.WriteLine("Index = {0}", index);
return;
}
queue.Enqueue(current+1);
queue.Enqueue(2*current);
}
 Priority Queue Implementation
class PriorityQueue<T> where T:IComparable<T>
{
private OrderedBag<T> bag;
public int Count
{
get { return bag.Count; }
private set{ }
}
public PriorityQueue()
{
bag = new OrderedBag<T>();
}
public void Enqueue(T element)
{
bag.Add(element);
}
public T Dequeue()
{
var element = bag.GetFirst();
bag.RemoveFirst();
return element;
}
}
 Empty or Full?
• Empty queue
– back = front - 1
• Full queue?
– the same!
– Reason: n values to represent n+1 states
• Solutions
– Use a boolean variable to say explicitly whether the
queue is empty or not
– Make the array of size n+1 and only allow n elements to
be stored
– Use a counter of the number of elements in the queue
Queue Implementation of Linked List
class Queue {
public:
Queue(int size = 10); // constructor
~Queue() { delete [] values; }// destructo
bool IsEmpty(void);
bool IsFull(void);
bool Enqueue(double x);
bool Dequeue(double & x);
void DisplayQueue(void);
private:
int front; // front index
int rear; // rear index
int counter; // number of elements
int maxSize; // size of array queue
double* values;// element array
};
 Queue Class
• Attributes of Queue
– front/rear: front/rear index
– counter: number of elements in the queue
– maxSize: capacity of the queue
– values: point to an array which stores elements of the queue
• Operations of Queue
– IsEmpty: return true if queue is empty, return false otherwise
– IsFull: return true if queue is full, return false otherwise
– Enqueue: add an element to the rear of queue
– Dequeue: delete the element at the front of queue
– DisplayQueue: print all the data
 IsEmpty & IsFull
• Since we keep track of the number of elements
that are actually in the queue: counter, it is
easy to check if the queue is empty or full.
bool Queue::IsEmpty() {
if (counter) return false;
else return true;
}
bool Queue::IsFull() {
if (counter < maxSize) return false;
else return true;
}
 Enqueue
bool Queue::Enqueue(double x) {
if (IsFull()) {
cout << "Error: the queue is full." << endl;
return false;
}
else {
// calculate the new rear position (circular)
rear = (rear + 1) % maxSize;
// insert new item
values[rear] = x;
// update counter
counter++;
return true;
}
}
 Dequeue
bool Queue::Dequeue(double & x) {
if (IsEmpty()) {
cout << "Error: the queue is empty." << endl;
return false;
}
else {
// retrieve the front item
x = values[front];
// move front
front = (front + 1) % maxSize;
// update counter
counter--;
return true;
}
}
 Printing the elements

void Queue::DisplayQueue() {
cout << "front -->";
for (int i = 0; i < counter; i++) {
if (i == 0) cout << "\t";
else cout << "\t\t";
cout << values[(front + i) % maxSize];
if (i != counter - 1)
cout << endl;
else
cout << "\t<-- rear" << endl;
}
}
 Using Queue

int main(void) {
Queue queue(5);
cout << "Enqueue 5 items." << endl;
for (int x = 0; x < 5; x++)
queue.Enqueue(x);
cout << "Now attempting to enqueue again..." << endl;
queue.Enqueue(5);
queue.DisplayQueue();
double value;
queue.Dequeue(value);
cout << "Retrieved element = " << value << endl;
queue.DisplayQueue();
queue.Enqueue(7);
queue.DisplayQueue();
return 0;
}
 Enqueue
void Queue::Enqueue(double x) {
Node* newNode = new Node;
newNode->data = x;
newNode->next = NULL;
if (IsEmpty()) {
front = newNode;
rear = newNode;
} rear
else {
rear->next = newNode; 8 5
rear = newNode;
} rear
counter++;
8 5
}
newNode
 Dequeue
bool Queue::Dequeue(double & x) {
if (IsEmpty()) {
cout << "Error: the queue is empty." << endl;
return false;
}
else {
x = front->data;
Node* nextNode = front->next;
delete front;
front = nextNode;
counter--;
}
} front

3 8 5
front

8 5
 Printing all the elements

void Queue::DisplayQueue() {
cout << "front -->";
Node* currNode = front;
for (int i = 0; i < counter; i++) {
if (i == 0) cout << "\t";
else cout << "\t\t";
cout << currNode->data;
if (i != counter - 1)
cout << endl;
else
cout << "\t<-- rear" << endl;
currNode = currNode->next;
}
}
 Queues
• Collection with access only to the item that has
been present the longest
• Last in last out or first in first out
• enqueue, dequeue, front, rear
• priority queues and deques
Front Rear
Deletion
Insertion

Data1 Data2 Data3 Data4

 Implementation of Queues

• To Enqueue an element X, we increment Size and Rear,

then set Queue[Rear]=X.

• To Dequeue an element, we set the return value to

Queue[Front], decrement Size, and then increment Front.

• Whenever Front or Rear gets to the end of the array, it is

wrapped around to the beginning. This is known as a
circular array implementation.
 Initial 2 4
Front Rear
State
Enqueue (1) 1 2 4
Rear Front

Enqueue (3) 1 3 2 4
Rear Front

Dequeue, which 1 3 2 4
returns 2 Rear Front

Dequeue, which 1 3 2 4
returns 4 Front Rear

Dequeue, which
returns 1
1 3 2 4
Rear
Front

Dequeue, which
returns 3 and makes 1 3 2 4
Rear Front
the Queue empty
Implementation of Queues

Front Header

……

Rear /
 Linked List Implementation of Queues

Front
Empty Queue Reart /

Enqueue x Front
Reart x /

Enqueue y Front
Reart x y /

Dequeue x Front
Reart x y /
A node in a doubly linked list: A compound object that
stores a reference to an element and two references, called
next and prev, to the next and previous nodes, respectively.

Element

Node
Reference to an
element
Reference to next
next node
prev Reference to
previous node
For convenience, a doubly linked list has a header node and a
trailer node. They are also called sentinel nodes, indicating
both the ends of a list.

header trailer

Baltimore Rome Seattle

Difference from singly linked lists:

- each node contains two links.
- two extra nodes: header and trailer, which contain no
elements.
 Double-Ended Queues
Recall that a queue has a front and a rear. Elements can be
added at the rear and can be removed at the front.

Front Rear

Now we also want to add elements at the front and remove

them at the rear.
 The Deque Abstract Data Type
A deque D is an abstract data type that supports the following
four fundamental methods:

insertFirst(e): Insert a new element e at the beginning of D.

Input: Object; Output: None.
insertLast(e): Insert a new element e at the end of D.
Input: Object; Output: None.
removeFirst(): Remove and return the first element of D; an
error occurs if D is empty.
Input: None; Output: Object
removeLast(): Remove and return the last element of D; an
error occurs if D is empty.
Input: None; Output: Object
Other supporting methods:

first(): Return the first element of D; an error occurs if D is

empty
Input: None; Output: Object
last(): Return the last element of D; an error occurs if D is
empty
Input: None; Output: Object
size(): Return the number of elements of D.
Input: None; Output: integer
isEmpty(): Determine if D is empty.
Input: None; Output: Boolean
public interface Deque {
void insertFirst(Object e);
void insertLast(Object e);
Object removeFirst();
Object removeLast();
Object first();
Object last();
int size();
boolean isEmpty();
}
This table shows a series of operations and their effects. The deque is
empty initially.

Operation Output D
insertFirst(3) - (3)
insertFirst(5) - (5,3)
removeFirst() 5 (3)
insertLast(7) - (3,7)
removeFirst() 3 (7)
removeLast() 7 ()
removeFirst() “error” ()
isEmpty() true ()
 Class MyDeque
public class MyDeque implements Deque {
DLNode header, trailer;
int size;
public MyDeque() {
header = new DLNode();
trailer = new DLNode();
header.setNext( trailer );
trailer.setPrev( header );
size = 0; }

public Object first() throws DequeEmptyException {

if( isEmpty() )
throw new DequeEmptyException( "Deque is empty." );
header.getNext().getElement();

return }
public void insertFirst( Object o ) {
DLNode second = header.getNext();
DLNode first = new DLNode( o, header, second );
second.setPrev( first );
header.setNext( first );
size++;
}

public Object removeLast() {

if( isEmpty() )
throw new DequeEmptyException( "Deque is empty." );
DLNode last = trailer.getPrev();
Object o = last.getElement();
DLNode secondtolast = last.getPrev();
trailer.setPrev( secondtolast );
secondtolast.setNext( trailer );
size--;
return o; }
 Queue
• A queue is a first-in-first-out linear data structure.
• It supports two operations:
– void enQueue(Object d)
– Object deQueue()

Data Data
In Out
 scheduling
• Let set of tasks to be processed by 1 server, find
the average completion time, average service
time, average waiting time, using batch schedule.
• Task 1 2 3 4 5 6 7 8 9 10 11 12 13
• Time 25 22 12 20 2 50 30 10 2 20 40 10 5
 solution
• Using FCFS (first come first serviced)
• Task 1 2 3 4 5 6 7 8 9 10 11 12 13
Service
• time 25 22 12 20 2 50 30 10 2 20 40 10 5
Wait
• time 0 25 47 59 79 81 131 161 171 173 193 233 243
• Av= 122.8
Complet.
Time 25 47 59 79 81 131 161 171 173 193 233 243 248
Av.= 141.8
av. Completion time=av. Wait time + av. Service time
 Let set of tasks to be processed by 1 server, find the
average completion time, average service time, average
waiting time, using time sharing with slot time =10 time
units

av. Completion time<av. Wait time + av. Service time

Av. Completion time > case of batch
av. Wait time > case of batch
av. Service time is the same t batch
 tasks have the next priorities, find
completion time, average service time,
average waiting time
 For the next problem , tasks have the priorities that
shortest execution task is first serviced (SEFS),
find completion time, average service time, average
waiting time

av. Completion time= av. Wait time + av. Service time,

Av. Completion time < case of batch
av. Wait time < case of batch
 Using 2 parallel processors, in batch system,
find the average completion time, average
service time, average waiting time

av. Completion time= av. Wait time + av. Service time, as in case of batch
Av. Completion time < case of batch – for 1 processor
av. Wait time < case of batch – for 1 processor
Using 2 serial processors, find the av.
completion time, av. service time, av.
waiting time
 Gantt time diagram:

For server s2: idle times:(0-25)+ (27-47)+(117-131)+(163-171)

S1 end services after 258
av. Completion time for Si= av. Wait time Si+ av. Service time Si, i=1,2
Av. Completion time = av. Completion time for server S2
av. Wait time in system= av. Waiting time for S2
Idle time may be for either S1 or S2
 Using 3 parallel processors, find the average
completion time, average service time,
average waiting time

av. Completion time for 3 servers< av. Completion time for 2 servers < av. Completion time for 1 servers
av. Wait time for 3 servers< av. Wait time for 2 servers < av. Wait time for 1 servers
 exercise
• Write instructions to Delete and insert in
queue
• For an empty queue , what is its content, its
length:
Insert 4, del , insert 5, insert 7, insert 12, del,
del,