Queueing Theory

Chapter 17

Queueing Theory-1
 Basic Queueing Process

Arrivals Queue Service

Arrival time Capacity Number of servers
distribution (infinite or finite) (one or more)
Calling population Queueing Service time
(infinite or finite) discipline distribution

Queueing System
Queueing Theory-2
 Examples and Applications

Call centers (help desks, ordering goods)

Manufacturing
Banks
Telecommunication networks
Internet service
Intelligence gathering
Restaurants
Other examples.

Queueing Theory-3
 Labeling Convention (Kendall-Lee)

/ / / / /
Interarrival Service Number of Queueing System Calling
time time servers discipline capacity population
distribution distribution size

M Markovian (exponential FCFS first come, Notes:

interarrival times, first served
Poisson number of LCFS last come,
arrivals) first served
D Deterministic SIRO service in
Ek Erlang with shape random order
parameter k GD general
G General discipline

Queueing Theory-4
 Labeling Convention (Kendall-Lee)

Examples:
M/M/1
M/M/5
M/G/1
M/M/3/LCFS
Ek/G/2//10
M/M/1///100

Queueing Theory-5
 Terminology and Notation
State of the system
Number of customers in the queueing system (includes customers in
service)
Queue length
Number of customers waiting for service
= State of the system - number of customers being served

N(t) = State of the system at time t, t 0

Pn(t) = Probability that exactly n customers are in the queueing
system at time t

Queueing Theory-6
 Terminology and Notation
n = Mean arrival rate (expected # arrivals per unit time)
of new customers when n customers are in the system

s = Number of servers (parallel service channels)

n = Mean service rate for overall system
(expected # customers completing service per unit time)
when n customers are in the system

Note: n represents the combined rate at which all busy servers

(those serving customers) achieve service completion.

Queueing Theory-7
 Terminology and Notation
When arrival and service rates are constant for all n,
= mean arrival rate
(expected # arrivals per unit time)
= mean service rate for a busy server
1/ = expected interarrival time
1/ = expected service time
= /s
= utilization factor for the service facility
= expected fraction of time the systems service capacity (s)
is being utilized by arriving customers ()

Queueing Theory-8
 Terminology and Notation
Steady State

When the system is in steady state, then

Pn = probability that exactly n customers are in the queueing system
L = expected number of customers in queueing system
=

Lq = expected queue length (excludes customers being served)

=

Queueing Theory-9
 Terminology and Notation
Steady State

When the system is in steady state, then

= waiting time in system (includes service time)
for each individual customer
W = E[]

q = waiting time in queue (excludes service time)

for each individual customer
Wq = E[q]

Queueing Theory-10
 Littles Formula
Demonstrates the relationships between L, W, Lq, and Wq
Assume n= and n= Intuitive Explanation:
(arrival and service rates
constant for all n)
In a steady-state queue,

L W
Lq Wq

1
W Wq

Queueing Theory-11
 Littles Formula (continued)
This relationship also holds true for
(expected arrival rate)
when n are not equal.

L W

Lq Wq

1
W Wq


where n Pn
n 0

Recall, Pn is the steady state probability of having n customers in the system

Queueing Theory-12
 Heading toward M/M/s

The most widely studied queueing models are of the form

M/M/s (s=1,2,)
What kind of arrival and service distributions does this model
assume?
Reviewing the exponential distribution.
If T ~ exponential(), then

A picture of the distribution:

Queueing Theory-13
 Exponential Distribution Reviewed

If T ~ exponential(), then

fT (t ) e t
t 0
0 t 0
t

u t
FT (t ) P(T t ) e du 1 e
u 0

E[T] = ______ Var(T) = ______

Queueing Theory-14
 Property 1
Strictly Decreasing

The pdf of exponential, fT(t), is a strictly decreasing function

A picture of the pdf:

fT(t)
1
Area P T 0.393
2
1 1
Area P T 0.239
2
1
Area P T 0.368

e

t
1 1
2

Queueing Theory-15
 Property 2
Memoryless

The exponential distribution has lack of memory

i.e. P(T > t+s | T > s) = P(T > t) for all s, t 0.

Example:
P(T > 15 min | T > 5 min) = P(T > 10 min)

The probability distribution has no memory of what has already

occurred.

Queueing Theory-16
 Property 2
Memoryless
Prove the memoryless property

Is this assumption reasonable?

For interarrival times

For service times

Queueing Theory-17
 Property 3
Minimum of Exponentials
The minimum of several independent exponential random
variables has an exponential distribution

If T1, T2, , Tn are independent r.v.s, Ti ~ expon(i) and

U = min(T1, T2, , Tn),
n
U ~expon( i )
i 1

Example:
If there are n servers, each with exponential service times with mean ,
then U = time until next service completion ~ expon(____)

Queueing Theory-18
 Property 4
Poisson and Exponential

If the time between events, Xn ~ expon(), then

the number of events occurring by time t, N(t) ~ Poisson(t)

( t )n e t
P( N( t ) n ) for n 0,1,2, ...
n!
P( N( t ) 0) e t

Note:
E[X(t)] = t, thus the expected number of events per unit time is

Queueing Theory-19
 Property 5
Proportionality

For all positive values of t, and for small t,

P(T t+t | T > t) t

i.e. the probability of an event in interval t is proportional to the length

of that interval

Queueing Theory-20
 Property 6
Aggregation and Disaggregation

The process is unaffected by aggregation and disaggregation

Aggregation Disaggregation

N1 ~ Poisson(1) N1 ~ Poisson(p1)

N2 ~ Poisson(2) p1 N2 ~ Poisson(p2)
N ~ Poisson() N ~ Poisson() p2

= 1+2++k pk
Nk ~ Poisson(k) Nk ~ Poisson(pk)

Note: p1+p2++pk=1

Queueing Theory-21
 Back to Queueing
Remember that N(t), t 0, describes the state of the system:
The number of customers in the queueing system at time t

We wish to analyze the distribution of N(t) in steady state

Queueing Theory-22
 Birth-and-Death Processes
If the queueing system is M/M////, N(t) is a birth-and-death
process
A birth-and-death process either increases by 1 (birth), or
decreases by 1 (death)
General assumptions of birth-and-death processes:
1. Given N(t) = n, the probability distribution of the time remaining until the
next birth is exponential with parameter n
2. Given N(t) = n, the probability distribution of the time remaining until the
next death is exponential with parameter n
3. Only one birth or death can occur at a time

Queueing Theory-23
Rate Diagrams

Queueing Theory-24
Steady-State Balance Equations

Queueing Theory-25
 M/M/1 Queueing System
Simplest queueing system based on birth-and-death
We define
= mean arrival rate
= mean service rate
= / = utilization ratio
We require < , that is < 1 in order to have a steady state
Why?

Rate Diagram

0 1 2 3 4

Queueing Theory-26
 M/M/1 Queueing System
Steady-State Probabilities

Calculate Pn, n = 0, 1, 2,

Queueing Theory-27
 M/M/1 Queueing System
L, Lq, W, Wq

Calculate L, Lq, W, Wq

Queueing Theory-28
 M/M/1 Example: ER
Emergency cases arrive independently at random
Assume arrivals follow a Poisson input process (exponential
interarrival times) and that the time spent with the ER doctor is
exponentially distributed
Average arrival rate = 1 patient every hour
=

Average service time = 20 minutes to treat each patient

=

Utilization
=

Queueing Theory-29
 M/M/1 Example: ER
Questions
What is the
1. probability that the doctor is idle?
2. probability that there are n patients?
3. expected number of patients in the ER?
4. expected number of patients waiting for the doctor?
5. expected time in the ER?
6. expected waiting time?
7. probability that there are at least two patients waiting?
8. probability that a patient waits more than 30 minutes?

Queueing Theory-30
 Car Wash Example
Consider the following 3 car washes
Suppose cars arrive according to a Poisson input process and
service follows an exponential distribution
Fill in the following table

L Lq W Wq P0

Car Wash A 0.1 0.5

car/min car/min

Car Wash B 0.1 0.11

car/min car/min

Car Wash C 0.1 0.1

car/min car/min

What conclusions can you draw from your results?

Queueing Theory-31
 M/M/s Queueing System
We define
= mean arrival rate
= mean service rate
s = number of servers (s > 1)
= / s = utilization ratio
We require < s , that is < 1 in order to have a steady state

Rate Diagram

0 1 2 3 4

Queueing Theory-32
 M/M/s Queueing System
Steady-State Probabilities

1
P0 s 1
and Pn = CnP0

n 0
( / )n
n! ( / )s
s! ( 11/ s )

n

n 1, 2, ..., s
n!
n 1 n 2 ... 0
where n
C
n n 1...1 n


ns n s 1, s 2, ...
s! s Queueing Theory-33
 M/M/s Queueing System
L, Lq, W, Wq

P0 ( / )s
Lq
s! (1 )2 How to find L? W? Wq?

P0 s 1

(s 1)! s 1(s )2

P ( / ) s
1 e t ( s 1 / )
P( t ) e t 1 0
s!(1 ) s 1 /

s 1 s (1 )t
P(q t ) 1 Pn e
n 0

Queueing Theory-34
 M/M/s Example: A Better ER
As before, we have
Average arrival rate = 1 patient every hour
= 2 patients per hour
Average service time = 20 minutes to treat each patient
= 3 patients per hour
Now we have 2 doctors
s=
Utilization
=

Queueing Theory-35
 M/M/s Example: ER
Questions
What is the
1. probability that both doctors are idle?
a) probability that exactly one doctor is idle?
2. probability that there are n patients?
3. expected number of patients in the ER?
4. expected number of patients waiting for a doctor?
5. expected time in the ER?
6. expected waiting time?
7. probability that there are at least two patients waiting?
8. probability that a patient waits more than 30 minutes?

Queueing Theory-36
Performance s=1 s=2
Measurements
2/3 1/3

L 2 3/4

Lq 4/3 1/12

W 1 hr 3/8 hr

Wq 2/3 hr 1/24 hr

P(at least two patients waiting in 0.296 0.0185

queue)

P(a patient waits more than 30 0.404 0.022

minutes)

Queueing Theory-37
 Travel Agency Example
Suppose customers arrive at a travel agency according to a
Poisson input process and service times have an exponential
distribution
We are given
= .10/minute = 1 customer every 10 minutes
= .08/minute = 8 customers every 100 minutes
If there were only one server, what would happen?
How many servers would you recommend?

Queueing Theory-38
Queueing Theory-39
Queueing Theory-40
Queueing Theory-41
 Single Queue vs. Multiple Queues
Would you ever want to keep separate queues for separate
servers?

Single
queue

vs.

Multiple
queues

Queueing Theory-42
 Bank Example
Suppose we have two tellers at a bank
Compare the single server and multiple server models
Assume = 2, = 3
L Lq W Wq P0

Queueing Theory-43
 Bank Example
Continued
Suppose we now have 3 tellers
Again, compare the two models

Queueing Theory-44
 M/M/s//K Queueing Model
(Finite Queue Variation of M/M/s)

Now suppose the system has a maximum capacity, K

We will still consider s servers
Assuming s K, the maximum queue capacity is K s
List some applications for this model:

Draw the rate diagram for this problem:

Queueing Theory-45
 M/M/s//K Queueing Model
(Finite Queue Variation of M/M/s)
Rate Diagram

0 1 2 3 4

Balance equations: Rate In = Rate Out

Queueing Theory-46
 M/M/s//K Queueing Model
(Finite Queue Variation of M/M/s)
Solving the balance equations, we get the following steady state
probabilities:

n
P
n! n 0
for n 1, 2, ..., s

1 n
P0 Pn P

n s
s K
s s! n 0
for n s, s 1, ..., K
1 ( /n! ) ( /s! ) n s
n s

s
n 1 n s 1 0 nK

Verify that these equations match those given in the text for the single
server case (M/M/1//K)

Queueing Theory-47
 M/M/s//K Queueing Model
(Finite Queue Variation of M/M/s)

P0 ( / )s K s K s
Lq [1 ( K s ) (1 )], where / s
s! (1 ) 2

s 1
s 1
L nPn Lq s1 Pn
n 0 n 0

To find W and Wq:

Although L W and Lq Wq because n is not equal for all n,

L W and Lq Wq where n Pn (1 PK )
n 0

Queueing Theory-48
 M/M/s///N Queueing Model
(Finite Calling Population Variation of M/M/s)

Now suppose the calling population is finite

We will still consider s servers
Assuming s K, the maximum queue capacity is K s
List some applications for this model:

Draw the rate diagram for this problem:

Queueing Theory-49
 M/M/s///N Queueing Model
(Finite Calling Population Variation of M/M/s)
Rate Diagram

0 1 2 3 4

Balance equations: Rate In = Rate Out

Queueing Theory-50
 M/M/s///N Results
1
P0 s 1 n n

N

n 0
N! N!
(N n )! n! n s (N n )! s! s n s

N!
n

P0 for n 0,1,..., s
(N n )! n!

n
Pn N !
P0 for s nN
n s
(N n )! s! s


0 for nN

N s 1
s 1
Lq (n s )Pn L nPn Lq s1 Pn
n s n 0 n 0
Queueing Theory-51
 Queueing Models with Nonexponential Distributions

M/G/1 Model
Poisson input process, general service time distribution with mean 1/
and variance 2
Assume = / < 1
Results
P0 1
2 2 2
Lq
2(1 )
L Lq
Wq Lq /

W Wq 1/

Queueing Theory-52
 Queueing Models with Nonexponential Distributions

M/Ek/1 Model
Erlang: Sum of exponentials

fT (t )
k k 1 kt
k
t e for t 0
k 1!
Think it would be useful?
1
Can readily apply the formulae on previous slide where 2
k 2

Other models
M/D/1
Ek/M/1
etc

Queueing Theory-53
 Application of Queueing Theory

We can use the results for the queueing models when making
decisions on design and/or operations
Some decisions that we can address
Number of servers
Efficiency of the servers
Number of queues
Amount of waiting space in the queue
Queueing disciplines

Queueing Theory-54
 Number of Servers

Suppose we want to find the number of servers that minimizes the

expected total cost, E[TC]
Expected Total Cost = Expected Service Cost + Expected Waiting Cost
(E[TC]= E[SC] + E[WC])
How do these costs change as the number of servers change?
Expected cost

Number of servers Queueing Theory-55

 Repair Person Example

SimInc has 10 machines that break down frequently and 8 operators

The time between breakdowns ~ Exponential, mean 20 days
The time to repair a machine ~ Exponential, mean 2 days
Currently SimInc employs 1 repair person and is considering hiring
a second
Costs:
Each repair person costs $280/day
Lost profit due to less than 8 operating machines:
$400/day for each machine that is down
Objective: Minimize total cost
Should SimInc hire the additional repair person?

Queueing Theory-56
 Repair Person Example
Problem Parameters
What type of problem is this?
M/M/1
M/M/s
M/M/s/K
M/G/1
M/M/s finite calling population
M/Ek/1
M/D/1

What are the values of and ?

Queueing Theory-57
 Repair Person Example
Rate Diagrams
Draw the rate diagram for the single-server and two-server case

Single server 0 1 2 3 4 8 9 10

Two servers 0 1 2 3 4 8 9 10

Expected service cost (per day) = E[SC] =

Expected waiting cost (per day) = E[WC] =

Queueing Theory-58
 Repair Person Example
Steady-State Probabilities

Write the balance equations for each case

How to find E[WC] for s=1? s=2?

Queueing Theory-59
 Repair Person Example
E[WC] Calculations

s=1 s=2
N=n g(n)
Pn g(n) Pn Pn g(n) Pn
0 0 0.271 0 0.433 0
1 0 0.217 0 0.346 0
2 0 0.173 0 0.139 0
3 400 0.139 56 0.055 24
4 800 0.097 78 0.019 16
5 1200 0.058 70 0.006 8
6 1600 0.029 46 0.001 0
7 2000 0.012 24 0.0003 0
8 2400 0.003 7 0.00004 0
9 2800 0.0007 0 0.000004 0
10 3200 0.00007 0 0.0000002 0
E[WC] $281/day $48/day
Queueing Theory-60
 Repair Person Example
Results

We get the following results

s E[SC]: E[WC]: E[TC]:
1 $280/day $281/day $561/day
2 $560/day $48/day $608/day
3 $840/day $0/day $840/day

What should SimInc do?

Queueing Theory-61
 Supercomputer Example

Emerald University has plans to lease a supercomputer

They have two options
Mean number of jobs
Supercomputer Cost per day
per day
MBI 30 jobs/day $5,000/day
CRAB 25 jobs/day $3,750/day

Students and faculty jobs are submitted on average of 20 jobs/day,

distributed Poisson

i.e. Time between submissions ~ __________

Which computer should Emerald University lease?

Queueing Theory-62
 Supercomputer Example
Waiting Cost Function
Assume the waiting cost is not linear:

h() = 500 + 400 2 ( = waiting time in days)

What distribution do the waiting times follow?

What is the expected waiting cost, E[WC]?

Queueing Theory-63
 Supercomputer Example
Results
Next incorporate the leasing cost to determine the expected total
cost, E[TC]

Which computer should the university lease?

Queueing Theory-64