ISyE 3232

Z. She , YL. Chang

Stochastic Manufacturing and Service Models

Fall 2015

Homework 7 - Solutions
October 5, 2015
1. Daily demand for paint brushes at a particular store follows the demand distribution:
d
P (D = d)

0
.5

1
.15

2
.3

3
.03

4
.02

The stock level is reviewed every evening and when warranted an order is placed at the central warehouse to augment stock. Orders arrive over night and are available to meet the demand on the morning
of the next day. The fixed cost for placing an order at the end of the day is $0.2 and the per unit order
cost is $0.05, the daily per unit holding cost is is $0.01 and each brush sells at $0.5 (unfulfilled orders
are lost).
(1) Calculate the stationary distribution.
(2) Compute the long run expected daily profit under the following inventory policies:
(a) If the number of brushes at the end of the day is 3 or less, the management orders enough brushes
to bring the on hand inventory to 6 brushes.
If we order when the stock level is less than or equal to 3, then transition matrix and stationary
distribution of Xn is as follows

0
0
0.02 0.03 0.3 0.15 0.5
0
0
0.02 0.03 0.3 0.15 0.5

0
0
0.02 0.03 0.3 0.15 0.5

0
0.02 0.03 0.3 0.15 0.5
P =
.
0

0.02 0.03 0.3 0.15 0.5

0
0

0
0.02 0.03 0.3 0.15 0.5
0
0
0
0.02 0.03 0.3 0.15 0.5
To find the stationary distribution, we can use the following R code:
install.packages("DTMCPack")
library(DTMCPack)
values <- c(0,0,0,0,0.02,0,0,0,0,0,0,0.03,0.02,0,
0.02,0.02,0.02,0.02,0.3,0.03,0.02,0.03,0.03,0.03,0.03,0.15,0.3,0.03,
0.3,0.3,0.3,0.3,0.5,0.15,0.3,0.15,0.15,0.15,0.15,0,0.5,0.15,
0.5,0.5,0.5,0.5,0,0,0.5);
mymatrix <- matrix(values,nrow=7,ncol=7);
statdistr(mymatrix)


= 0.0069 0.0134 0.1186 0.1123 0.3467 0.1508 0.2513 .
For each stock level, the cost will be as following table:
Stock level
(i)
0
1
2
3
4
5
6

Holding

Fixed

0 0.01
1 0.01
2 0.01
3 0.01
4 0.01
5 0.01
6 0.01

0.2
0.2
0.2
0.2
0
0
0

Ordering
Cost
6 0.05
5 0.05
4 0.05
3 0.05
0
0
0
1

E[Sales]

C(i)

0.5 E[6 D]
0.5 E[6 D]
0.5 E[6 D]
0.5 E[6 D]
0.5 E[4 D]
0.5 E[5 D]
0.5E[6 D]

-0.5 + 0.46=-0.04
-0.46 + 0.46=0
-0.42 + 0.46=0.04
-0.38 + 0.46=0.08
-0.04 + 0.46=0.42
-0.05 + 0.46=0.41
-0.06 + 0.46=0.40

E[6 D] = E[5 D] = E[4 D] = 0 0.5 + 1 0.15 + 2 0.3 + 3 0.03 + 4 0.02 = 0.92 Thus,
long-run expected profit is
6
X

C(i)i = 0.04 0.0069 + 0 0.0134 + 0.04 0.1186 + 0.08 0.1123

i=0

+ 0.42 0.3467 + 0.41 0.1508 + 0.40 0.2513 = 0.3214

(b) If the number of brushes at the end of the day is 4 or less, the management orders enough brushes
to bring the on hand inventory to 6 brushes.
If we order when the stock level is less than or equal to 4, then transition matrix and stationary
distribution of Xn is as follows

0
0
0.02 0.03 0.3 0.15 0.5
0
0
0.02 0.03 0.3 0.15 0.5

0
0
0.02
0.03 0.3 0.15 0.5

0
0.02 0.03 0.3 0.15 0.5
P =
.
0

0
0
0.02
0.03
0.3
0.15
0.5

0 0.02 0.03 0.3 0.15 0.5

0
0
0
0.02 0.03 0.3 0.15 0.5
To find the stationary distribution, we can use the following R code:
install.packages("DTMCPack")
library(DTMCPack)
values <- c(0,0,0,0,0,0,0,
0,0,0,0,0,0.02,0,
0.02,0.02,0.02,0.02,0.02,0.03,0.02,
0.03,0.03,0.03,0.03,0.03,0.3,0.03,
0.3,0.3,0.3,0.3,0.3,0.15,0.3,
0.15,0.15,0.15,0.15,0.15,0.5,0.15,
0.5,0.5,0.5,0.5,0.5,0,0.5);
mymatrix <- matrix(values,nrow=7,ncol=7);
mymatrix
statdistr(mymatrix)
=

0.0046

0.0223

0.0923

0.2654

0.2308

0.3846

For each stock level, the cost will be as following table:

Stock level
(i)
0
1
2
3
4
5
6

Holding

Fixed

0 0.01
1 0.01
2 0.01
3 0.01
4 0.01
5 0.01
6 0.01

0.2
0.2
0.2
0.2
0.2
0
0

Ordering
Cost
6 0.05
5 0.05
4 0.05
3 0.05
2 0.05
0
0

E[Sales]

C(i)

0.5 E[6 D]
0.5 E[6 D]
0.5 E[6 D]
0.5 E[6 D]
0.5 E[6 D]
0.5 E[5 D]
0.5E[6 D]

-0.5 + 0.46=-0.04
-0.46 + 0.46=0
-0.42 + 0.46=0.04
-0.38 + 0.46=0.08
-0.34 + 0.46=0.12
-0.05 + 0.46=0.41
-0.06 + 0.46=0.40

E[6 D] = E[5 D] = 0 0.5 + 1 0.15 + 2 0.3 + 3 0.03 + 4 0.02 = 0.92

Thus, long-run expected profit is
6
X

C(i)i = 0.04 0 + 0 0.0046 + 0.04 0.0223 + 0.08 0.0923

i=0

+ 0.12 0.2654 + 0.41 0.2308 + 0.40 0.3846 = 0.2886

2

(c) Which policy is better and why?

The 1st policy is better because it gives a higher long-run average profit per day.
2. A company has three machines. On any day, each working machine breaks down with probability 0.3,
independent of other machines. At the end of each day, the machines that have broken down are sent
to a repairman who can work on only one machine at a time. When the repairman has one or more
machines to repair at the beginning of a day, he repairs and returns exactly one at the end of that day.
Let Xn be the number of working machines at the beginning of day n
(a) Is {Xn : n 0} a Markov chain? Why?
{Xn : n 0} is a Markov chain with a state space {0, 1, 2, 3} and a transition matrix

0
1
0
0
0
0.3
0

0.7
P =
2
2
2
0
0.3

0.3

0.7
0.7
1




3
3
2
1
1
2
0.33
0.73
2 0.3 0.7
1 0.3 0.7
The process is Markovian because the probability of states in time n + 1 only depends on the
state in time n as shown in the transition matrix.
(b) Given that one machine is working today, what is the probability that no machine will be working
tomorrow?
P10 = 0
(c) Given that no machine is working today, what is the probability that no machine will be working
five days later?
Note that

0.00959531 0.169275 0.470173 0.350957

0.00947584 0.169024 0.470737 0.350763

P6 =
0.00946836 0.168766 0.470703 0.351063
0.00948808 0.168813 0.470589 0.35111
6
Thus P00
= 0.00959531.