Queueing Theory

Ming Kin Lai Feb 2, 2007

Queueing theory
A branch of applied probability theory aka traffic theory, congestion theory, theory of mass service, theory of stochastic service systems Applications in - communication networks - computer systems - machine plants, etc

Service center
Consider a service center and a population of customers (jobs, tasks) that enters the service center to obtain service If a customer arrives and the service is exhausted, he enters a waiting line 3 elements of a service center
Population of customers Server(s) Waiting line

Simple examples of the use of queueing theory in networking

Dimensioning of buffers in routers or multiplexers Determining the number of trunks in a central office in POTS (plain old telephone service) Calculating end-to-end throughput in networks

Telephone service
Too expensive to maintain permanent path between 2 phone sets Facilities needed to establish and maintain a talking path between a pair of phone sets are
provided in a common pool, to be used by a call when required and returned to the pool when no longer needed

Possible that the system unable to set up a call on demand because of lack of equipment How much equipment must be provided so that the proportion of calls experiencing delays will be below a specified acceptable level?

Queueing theory answers questions like ..

Mean waiting time in the queue Mean system response time (waiting time in queue + service time) Mean utilization of the servers Distribution of the number of customers in queue Distribution of the number of customers in system Probability of the queue is full or empty

Basic model
Customer population infinite size nth customer arrives at time n Inter-arrival time between 2 customers: tn := n - n-1 tn independent and identically distributed random variable Service time xn - independent and identically distributed random variable

Queueing system
Distribution of inter-arrival time Distribution of service time Number of servers Size of waiting line (infinite or finite) Service discipline, queue discipline
FIFO LIFO Random Round robin Priority

3 characteristics of a queueing model

Input process: the sequence of request for service, often specified in terms of interarrival time Service mechanism: # of servers and service time Queue discipline: disposition of blocked customers (customers who find all servers busy)

Queue discipline
Blocked customers (customers who find all servers busy)
leave the system immediately (Blocked customers cleared ) wait for service in a queue (Blocked customers delayed): FIFO, LIFO, wait for service in a queue as long as time T (sojourn time); if he receives service before the expiration of T, he then holds the server for the remainder of the time T; he departs after T regardless (Blocked customers held)

Kendall notation
A/B/m/N S A distribution of inter-arrival time B distribution of service time m number of servers N max size of the waiting line ( if omitted) S queue discipline (FIFO if omitted)

Choices for inter-arrival time and service time

A and B can be:
M (Markov) exponential distribution D (Deterministic) constant value Ek (Erlang k) Erlangian distribution Hk (Hyper-k) hyper-exponential distribution General general distribution

Steady state & equilibrium

Steady state solutions Steady state: where the system after a long time running tends to reach a stable state, e.g. distribution of customers in the system does not change Transient solutions: short term system response

Statistical equilibrium
The statistical-equilibrium distribution is stationary The state probabilities do not vary with time Practical applications of queueing theory largely concerned with the statisticalequilibrium properties of a system Transient solution describes the system as a function of time

M/M/1 queues
Simplest queue system (M/M/1/-FIFO) Easy to calculate performance metrics due to the exponential distribution of A and B

Markovian systems
Distributions of A and B are exponential and thus exhibit the Markov (memory-less) property Exponential distribution Markov property Consequence of Markov property:
State of the system summarizable in a single variable, i.e. number of customers in the system Markovian systems directly mappable to continuous-time Markov chain which can then be solved

M/M/1
M - inter-arrival times has an exponential distribution (which has the Markov property) The next dozen slides explain things about the inter-arrival times distributed exponentially, or the input process being a Poisson process

Input process is Poisson process

3 characteristics of a queueing system
Input process Service mechanism Queue discipline

Exponential distribution of inter-arrival time = input process is a Poisson process

Poisson process is a stochastic process

Stochastic process: a sequence of random variables, {N(t), t0} {N(0), N(1), N(2), N(3), }

Random variable
A function defined on a sample space and about whose values a probability statement can be made Examples of N(t):
Number of phone calls occurring in an interval of length t Number of customers waiting in a queue or in service at time t Number of deaths in an epidemic occurred in (0, t) Population of a city at time t

Poisson process and Poisson distribution

A Poisson process is a stochastic process which is defined in terms of the occurrences of events. This counting process, given as a function of time N(t), represents the number of events since time t = 0 Also, the number of events between time a and time b is given as N(b) N(a) and has a Poisson distribution

Poisson distribution
Poisson distribution is a discrete probability distribution. It expresses the probability of a number of events occurring in a fixed period of time if these events
occur with a known average rate, and are independent of the time since the last event.

Poisson distribution
The probability that there are exactly k occurrences (k being a non-negative integer, k = 0, 1, 2, ...) is

e = 2.71828..., is a positive real number, equal to the expected number of occurrences that occur during the given interval. For instance, if the events occur on average every 4, and you are interested in the number of events occurring in a 10 minute interval, you would use as model a Poisson distribution with = 10/4 = 2.5.

Poisson distribution
Mean: Variance:

Exponential distribution
(negative) exponential distributions are a class of continuous probability distribution. used to model the time between independent events that happen at a constant average rate. probability density function
where > 0 is a parameter of the distribution, often called the rate parameter. The distribution is supported on the interval [0,).

cumulative distribution function

Exponential distribution
Mean: -2 Variance:
-1

Poisson distribution of number of events per unit time vs Exponential distribution of waiting time until first occurrence of an event
In that case where is taken to be the rate, i.e., the average number of occurrences per unit time, if Nt is the number of occurrences before time t then we have and the waiting time T until the first occurrence is a continuous random variable with an exponential distribution (with parameter ). This may be deduced from the fact that

Saying the same thing

In a Poisson process with rate , the number of points occurring in a fixed length t has the Poisson distribution, or equivalently, the lengths of the intervals separating successive points are independent and have identical, exponential distributions

Poisson process, Poisson distribution, exponential distribution

Number of events between time a and time b is given as N(b) N(a) and has a Poisson distribution
Inter-arrival time of event A and event B is given as B - A and has an exponential distribution

When and if the input process is a Poisson process

In M/M/1 queues, input process is a Poisson process {N(0), N(1), N(2), N(3), } where N(t) has the Poisson distribution with mean t where is the birth rate (arrival rate) The assumption of Poisson input leads to easier math

Markovian systems
Distributions of A and B are exponential and thus exhibit the Markov (memory-less) property Exponential distribution Markov property Consequence of Markov property:
State of the system summarizable in a single variable, i.e. number of customers in the system Markovian systems directly mappable to continuous-time Markov chain which can then be solved

Markov property
A continuous nonnegative random variable X is said to have the Markov property if for every t > 0 and every x > 0 P{X>t+x | X > t} = P{X>x} A random variable with an exponential distribution has the Markov property

M/M/1 performance measure

pk := steady state probability that the system is in state k Utilization (fraction of time the server is busy) := 1 p0 = Mean number of customers in system = / 1- Mean response time = 1 / - is the birth rate, is the death rate

M/M/m performance measure

Queueing networks
Queues can be chained to form queueing networks where the departures from one queue enter the next queue. Queueing networks can be:
open closed

Open queueing networks have an external input and an external final destination. Closed queueing networks are completely contained and the customers circulate continually never leaving the network.

Simulation of Queueing Systems

Ming Kin Lai Feb 16, 2007

Single-server Queue
Population of Customers

Queue discipline FIFO,

- inter-arrival rate inter-arrival time = 1/

Queue

Server

- service time

Kenall Notation A/B/c

A Distribution of the inter-arrival rates of customers B Distribution of the service times of customers C number of servers A and B can be:
M: Markov (exponential) D: Deterministic (constant) G: General (arbitrary) .

Performance measures
Mean time waiting in queue average time of a customer waiting in queue Sojourn time (aka system time, residence time) time waiting in queue + time being served Mean sojourn time average sojourn time of a customer

Hand simulation of a G/D/1 (FIFO) Queue

interarrival time

- servc time

Time of Arrival

Start Service

End Service

Time in Queue

1 2 3 4 5

5 3 16 5 1

4 4 4 4 4

5 8 24 29 30

5 9 24 29 33

9 13 28 33 37

0 1 0 0 3

6
7 8 9 10
Total

2
3 17 2 5

4
4 4 4 4 40

32
35 52 54 59

37
41 52 56 60

41
45 56 60 64

5
6 0 2 1 18

Performance measures
Mean time waiting in queue : 18/10 = 1.8 time units Mean sojourn time (40+18)/10 = 5.8 time units

Hand simulation of a G/D/2 (FIFO) Queue (without feedback)

interarrival time

-servc time

Time of Arrival

Server Assigned

Start Service

End Service

Time in Queue

1 2 3 4 5 6 7 8

5 3 16 5 1 2 3 17

4 4 4 4 4 4 4 4

5 8 24 29 30 32 35 52

1 2 1 1 2 1 2 1

5 8 24 29 30 33 35 52

9 12 28 33 34 37 39 56

0 0 0 0 0 1 0 0

9 10
Total

2 5

4 4
40

54 59

2 1

54 59

58 63

0 0
1

Performance measures
Mean time waiting in queue : 1/10 = 0.1 time units Mean sojourn time (40+1)/10 = 4.1 time units

Queue with Feedback

1-p p

Population of Customers

Hand simulation of a G/D/1 (FIFO) Queue (with feedback)

interarrival time

- svc
time

Time of Arrival

Start Service

End Service

Time in Queue

Back?

1 2 1

5 3

4 4 4

5 8 9

5 8 12

9 12 16

0 0 3

3
4 5

16
5 1

4
4 4

24
29 30

24
29 30

28
33 34

0
0 0

6
4 7

4
4

32
33 35

33
37 41

37
41 45

1
4 6

4
8 9 10 Total 17 2 5

4
4 4 4 40

41
52 54 59

45
52 54 59

49
56 58 63

4
0 0 0 18

Arrival in bulk

Hand simulation of a G/D/2 (FIFO) Queue arrival in bulk

interarrival time

- svc time

Time of Arrival

Server Assigned

Start Service

End Service

Time in Queue

1-1 1-2 1-3 2-1 2-2

4 4 4

5 5 5 29 29

1 2 1 1 2

5 5 9 29 29

9 9 13 33 33

0 0 4 0 0

24

4 4

3-1
4-1 4-2 4-3

3
3

4
4 4 4

32
35 35 35

1
2 1 2

33
35 37 39

37
39 41 43

1
0 2 4

4-4
Total

4
40

35

41

45

6
17

Performance measures
Mean time waiting in queue : 17/10 = 1.7 time units Mean sojourn time (40+17)/10 = 5.7 time units

(Open) Queue Network

p20
p22

p21

p23

Q1

Q2

Q3

Population of Customers

Existing
host
core core core

host
core

LNs

Design 1
host
core core core

host
core

LNs

Num of threads per core determined by thread pool?

Design 2
host
core core core

host
core

LNs

The programmer determines the num of threads?! This forces programmer to use more LNs than # cores, at least = # cores

12/2006 Design
host
core core core

host
core

LNs

 Maybe the good thing about adding one more layer (i.e. MESS) above OS is that programmer does not need to worry about how many threads to use