IE 611 Stochastic models

Class Project Questions

Following questions are for group project in IE 611. You can form a group of one, two or
three students for this Project.
Q1. Snakes and ladders game:
The game of snakes and ladders consists of a board having 100 squares labeled 1, 2, 3..100.
on board, there are ten snakes: (27,10), (55,16), (61,14), (69,50), (79,5), (81,44), (87,31),
(91,25), (95,49), and (97,59). There are also ten ladders: (6,23), (8,30), (13,47), (20,39),
(33,70), (37,75), (41,62), (57,83), (66,89), and (77,96). The player starts from square 1. In
his/her turn, he/she rolls a die. He/she is on square i and the number turned up is d, he/she
moves to square i+d. however, if square i+d =u is start of a snake or ladder (u,v), then he/she
has to move immediately to square v.
a) Formulate the game as a Markov chain. Write transition probability matrix.
b) How long does it typically take a single player to finish a game by reaching square 100?
c) For simplicity, assume that the board has only 20 squares labeled from 1 to 20.
i) Without any ladder or snake, find the probability that square i is occupied during a
game starting from square 1.
ii) Without any ladder or snake, what is expected number of tosses of a die a solo
player has to make before the game end?
iii) With ladder (3,10), what is expected number of tosses of a die a solo player has to
make before the game end?
[Adapted from: D.L. Minh. Applied probability models". Duxbury 2002.]
Q2. Buffer design of a communication channel:
Consider a communication system with a finite-capacity buffer and multiple transmission
channels. Messages arrive at the buffer according to a Poisson process with rate . The
messages are first stored in the buffer which has capacity for only N messages (excluding the
messages in transmission). Each message which finds, upon arrival, that the buffer is full is lost
and does not influence the system. At fixed clock times t=0,1,..messages are taken out from
the buffer and are synchronously transmitted. Each transmission channel can transmit only one
message at a time. The transmission is constant and equal equals one time unit for each
massage. There are c transmission channels and so at most c messages can synchronously be
transmitted. The transmission of a message can only start at a clock time so that a message
which finds, upon arrival, that a transmission line is idle has to wait until a subsequent clock
time
a) Model the system as a Markov chain.
b) How should we choose the buffer size so that the long-run fraction of messages lost does not
exceed a prespecified value?
[Adapted from: Henk C. Tijms. "Stochastic Models an algorithmic approch". John
wiley&sons.]

IE 611 Stochastic models

Class Project Questions

Q3) Consider an algorithm for generating a random spanning tree of an undirected, d-regular
connected graph G. The algorithm is as follows:
a) Choose a starting vertex uniformly at random
b) Follow the standard random walk over vertices
c) The first time a vertex is entered by the random walk, add the edge used to enter it to the
spanning tree (the starting vertex is \entered" at time 0 so receives no entering edge)
d) Stop when all vertices have been entered at least once
To analyze the algorithm, we need to examine a Markov chain that, on the surface, seems
unrelated. The Markov Chain is on rooted spanning trees of G, in other words, spanning trees
with a specific vertex selected as the root. Once a root r is selected, each vertex v r in the tree
has a unique parent: the first vertex on the (unique) path from v to r. Consider the following
random transition rule from tree T with root r:
1) Choose a random neighbor v of r to be the new root
2) Delete the parent edge of v from T
3) Add the edge (r; v) to the T (this is now r's parent edge)
Note that as the rooted-tree markov chain evolves, the root vertex is tracing out a standard
random walk on G.
(a) Prove that (once we add self loops) this Markov Chain has a unique stationary distribution.
(b) Prove that the stationary distribution is uniform over rooted spanning trees. Conclude that if
we ignore the root, then the tree of the stationary distribution is a (uniformly) random spanning
tree.
(c) Now we connect this Markov chain to the tree generation algorithm described initially.
Suppose that the markov chain is initiated in its stationary distribution infinitely far in the past,
and consider its state at time 0. Argue that to know the rooted tree at time 0, it is sufficient to
know, for each vertex v, the identity of the last vertex to replace v as root in the Markov chain
before time 0. In other words, that the state at time 0 is defined by the sequence of roots r......... r-1, r0 encountered before time 0, and that we only need a suffix of this sequence that
includes every vertex at least once.
Q4) In an urn experiment where P molecules are distributed in two containers A and B. At
each trial a molecule is chosen at random and moved from its container to another container.
Suppose there are exactly k molecules in the container A. Then the probability that the
k
Pk
molecule is chosen from urn A is P and that to from urn B is
P
A) Construct a markov chain and find its transition probability matrix
B) Let there initially be j molecule in the first container and let X(n) = 2k-a if at the nth step the
number of molecules in container A are k (so that X(n) is the difference of the number of

IE 611 Stochastic models

Class Project Questions

molecules in the 2 containers) let en = E(X(n)), prove that en+1 = [(a-2) en]/a, where en = (12/a)n(2j-a)
Q5) Neal's token instrument:
In single line operations on Indian Railways, some trains still run by the physical token system,
called the Neal's token instrument, as follows. For using the track between station A and B, let
trains going from A to B be called "down" trains and trains from B to A be called "up" trains.
Some number of metal balls called tokens (say K in number) are there which can be deposited
in an interlocked set of instruments at A and B. If a train arrives at A and there is a token
available in the instrument, it requests it. Till this token is physically deposited in the
instrument at B, no other train (up or down) can enter the A-B section. If trains requesting the
use of section A-B are either up or down trains with equal probability, what is the number of
tokens required so that trains do not have to wait for want of a token (track otherwise being
free)? You would have to make some assumptions about the arrival process, for example, you
can assume Poisson arrivals with the same rate (l) in each direction. In the Poisson case, you
may not be able to ensure that trains NEVER wait. See what you can say about this. Also try
some other sensible arrival distributions.
For photos of the Neal's token instrument on the Kalka Simla line, pl see
http://www.tribuneindia.com/2003/20031108/windows/main1.htm

Q6). Out of the d doors of my house, suppose that in the beginning k> 0 are unlocked and dk
are locked. Every day, I use exactly one door, and I am equally likely to pick any of the d
doors. At the end of the day, I leave the door I used that day locked.
(a) Show that the number of unlocked doors at the end of day n, Ln, evolves as the state in a
Markov process for n 1. Write down the transition probabilities pij.
(b) List transient and recurrent states.
(c) Is there an absorbing state? How does rij(n) behave as n ?
(d) Now, suppose that each day, if the door I pick in the morning is locked, I will leave it
unlocked at the end of the day, and if it is initially unlocked, I will leave it locked. Repeat parts
(a)-(c) for this strategy.
(e) My third strategy is to alternate between leaving the door I use locked one day and
unlocked the next day (regardless of the initial condition of the door.) In this case, does the
number of unlocked doors evolve as a Markov chain, why/why not?
Q7) Mr. Mean Variance has the only key which locks or unlocks the door to Building 59, the
Probability Building. He visits the door once each hour on the hour. When he arrives: If the
door is unlocked, he locks it with probability 0.3.If the door is locked, he unlocks it with
probability 0.8.

IE 611 Stochastic models

Class Project Questions

(a) After he has been on the job several months, is he more likely to lock the door or to unlock
it on a randomly selected visit?
(b) With the process in the steady state, Joe arrived at Building 59 two hours ahead of Harry.
What is the probability that each of them found the door in the same condition?
(c) Given the door was open at the time Mr. Variance was hired, determine the expected value
of the number of visits up to and including the one on which he unlocks the door himself for
the first time.
In the following, suppose that every time after he leaves the building, he comes back after T
hours, where T is equal to 1 with probability 1/2, and to 2 with probability 1/2.
Q8) Jogger problem
Each morning an individual leaves his house and goes for a run. He is equally likely to leave
either from the front or the back door. Upon leaving the house, he chooses a pair of running
shoes (or goes running barefoot if there are no shoes at the door from which he departed). On
his return he is equally likely to enter, and leave his running shoes, either by the front door or
the back door. If he owns a total of k pairs of running shoes, what proportion of the time does
he run barefoot?
[Adapted From: Sheldon M Ross: Introduction to Probability Models]
Q9) Umbrella problem
An individual has k umbrellas. Each morning when he leaves home, if it is raining, he takes
an umbrella with him to office. Similarly in the evening, if it is raining, he takes an umbrella
home with him. If it is not raining he does not carry an umbrella. Find the proportion of the
time that the man gets wet if on a given day it is equally likely to rain or not rain.
[Adapted from: Sheldon M Ross: Introduction to stochastic Models]

Q10) At the Probability Coffee house of IIT, there is only one cashier. Due to the limited
space, she allows only m customers to line before her at any time. If customer finds there is m
customer including the one being served by the cashier, he will leave the Coffee house
immediately. Every minute, exactly one of the following occurs:
a) One new customer arrives with probability p;
b) One existing customer leaves with probability kq, where k is the number of customers
in the House; or
c) No new customer arrives and no existing customer leaves with probability1-p-kq
if there is at least one customer in the House, and with probability 1-p otherwise.

IE 611 Stochastic models

Class Project Questions

1. This problem can be modeled as a birth-death process.

Define appropriate states and draw the transition probability graph.
2. After the House has been open for a long time, you walk into the House.
Calculate how many customers you expect to see in line.
Q11) Suppose the gamblers ruin problem that the probability of winning a bet depends on the
gamblers present fortune. Specifically, suppose that i is the probability that the gambler wins
a bet when his or her fortune is i. given that the gamblers initial fortune is i, let P(i) denote the
probability that the gambler reaches N before 0.
1. Derive a formula that relates P(i) to P(i-1) and P(i+1)
2. Using the same approach as in the gambler ruin problem, solve the equation of
part (1) for P(i).
[Adapted from: Sheldon M. Ross Introduction to Probability models eighth edition,
Academic press,(263)]
Q12) At all times an urn contains N balls-some white balls and some black balls. At each
stage, a coin having probability p, 0<p<1, of landing heads is flipped. If head appears, then a
ball is chosen at random from the urn and is replaced with white ball; if tails appears, then a
ball is chosen from the urn and is replaced by a black ball. Let Xn denote the number of white
ball in the urn after the nth stage.
1. Is [Xn, ,n0] a Markov chain? If so explain why.
2. What are its classes? What are the periods? Are they transient or recurrent?
3. Compute the transition probabilities Pij.
4. Let N=2. Find the proportion of time in each state.
5. If p=1, what is the expected time until there are only white balls in the urn if
initially there are i white and N-i black?
[Adapted from: Sheldon M. Ross Introduction to Probability models eighth edition,
Academic press,(264)]