Dynamic Economics: Quantitative Methods and Applications

This page intentionally left blank
Dynamic Economics
Dynamic Economics
Quantitative Methods and

Applications
Jerome Adda and

Russell Cooper
The MIT Press

Cambridge, Massachusetts
London, England
( 2003 Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form by any
electronic or mechanical means (including photocopying, recording, or information
storage and retrieval) without permission in writing from the publisher.
This book was set in Palatino on 3B2 by Asco Typesetters, Hong Kong, and was
printed and bound in the United States of America.
Library of Congress Cataloging-in-Publication Data
Adda, Jerome.
Dynamic economics : quantitative methods and applications / Jerome Adda and
Russell Cooper.
p. cm.
Includes bibliographical references and index.
ISBN 0-262-01201-4 (hc. : alk. paper)
1. Economics, Mathematical. 2. MacroeconomicsMathematical modes.
3. Econometric models. I. Cooper, Russell W., 1955 II. Title.
HB135.D935 2003
330 0 .01 0 5195dc21 2003042126
a` Lance Armstrong, notre maitre a` tous
Contents
1 Overview 1
I Theory
2 Theory of Dynamic Programming 7

2.1 Overview 7
2.2 Indirect Utility 7
2.2.1 Consumers 7
2.2.2 Firms 8
2.3 Dynamic Optimization: A Cake-Eating Example 9
2.3.1 Direct Attack 10
2.3.2 Dynamic Programming Approach 12
2.4 Some Extensions of the Cake-Eating Problem 16
2.4.1 Innite Horizon 16
2.4.2 Taste Shocks 20
2.4.3 Discrete Choice 22
2.5 General Formulation 24
2.5.1 Nonstochastic Case 24
2.5.2 Stochastic Dynamic Programming 29
2.6 Conclusion 31
3 Numerical Analysis 33
3.1 Overview 33
3.2 Stochastic Cake-Eating Problem 34
3.2.1 Value Function Iterations 34
3.2.2 Policy Function Iterations 40
3.2.3 Projection Methods 41
3.3 Stochastic Discrete Cake-Eating Problem 46
3.3.1 Value Function Iterations 47
viii Contents
3.4 Extensions and Conclusion 50

3.4.1 Larger State Spaces 50
3.5 Appendix: Additional Numerical Tools 52
3.5.1 Interpolation Methods 52
3.5.2 Numerical Integration 55
3.5.3 How to Simulate the Model 59
4 Econometrics 61
4.1 Overview 61
4.2 Some Illustrative Examples 61
4.2.1 Coin Flipping 61
4.2.2 Supply and Demand Revisited 74
4.3 Estimation Methods and Asymptotic Properties 79
4.3.1 Generalized Method of Moments 80
4.3.2 Maximum Likelihood 83
4.3.3 Simulation-Based Methods 85
4.4 Conclusion 97
II Applications
5 Stochastic Growth 103

5.1 Overview 103
5.2 Nonstochastic Growth Model 103
5.2.1 An Example 105
5.2.2 Numerical Analysis 107
5.3 Stochastic Growth Model 111
5.3.1 Environment 112
5.3.2 Bellmans Equation 113
5.3.3 Solution Methods 115
5.3.4 Decentralization 120
5.4 A Stochastic Growth Model with Endogenous Labor
Supply 122
5.4.1 Planners Dynamic Programming Problem 122
5.4.2 Numerical Analysis 124
5.5 Confronting the Data 125
5.5.1 Moments 126
5.5.2 GMM 128
5.5.3 Indirect Inference 130
5.5.4 Maximum Likelihood Estimation 131
Contents ix
5.6 Some Extensions 132

5.6.1 Technological Complementarities 133
5.6.2 Multiple Sectors 134
5.6.3 Taste Shocks 136
5.6.4 Taxes 136
5.7 Conclusion 138
6 Consumption 139
6.1 Overview and Motivation 139
6.2 Two-Period Problem 139
6.2.1 Basic Problem 140
6.2.2 Stochastic Income 143
6.2.3 Portfolio Choice 145
6.2.4 Borrowing Restrictions 146
6.3 Innite Horizon Formulation: Theory and Empirical
Evidence 147
6.3.1 Bellmans Equation for the Innite Horizon
Problem 147
6.3.2 Stochastic Income 148
6.3.3 Stochastic Returns: Portfolio Choice 150
6.3.4 Endogenous Labor Supply 153
6.3.5 Borrowing Constraints 156
6.3.6 Consumption over the Life Cycle 160
6.4 Conclusion 164
7 Durable Consumption 165

7.1 Motivation 165
7.2 Permanent Income Hypothesis Model of Durable
Expenditures 166
7.2.1 Theory 166
7.2.2 Estimation of a Quadratic Utility Specication 168
7.2.3 Quadratic Adjustment Costs 169
7.3 Nonconvex Adjustment Costs 171
7.3.1 General Setting 172
7.3.2 Irreversibility and Durable Purchases 173
7.3.3 A Dynamic Discrete Choice Model 175
8 Investment 187
8.2 General Problem 188
x Contents
8.3 No Adjustment Costs 189

8.4 Convex Adjustment Costs 191
8.4.1 Q Theory: Models 192
8.4.2 Q Theory: Evidence 193
8.4.3 Euler Equation Estimation 198
8.4.4 Borrowing Restrictions 201
8.5 Nonconvex Adjustment: Theory 202
8.5.1 Nonconvex Adjustment Costs 203
8.5.2 Irreversibility 208
8.6 Estimation of a Rich Model of Adjustment Costs 209
8.6.1 General Model 209
8.6.2 Maximum Likelihood Estimation 212
8.7 Conclusion 213
9 Dynamics of Employment Adjustment 215

9.1 Motivation 215
9.2 General Model of Dynamic Labor Demand 216
9.3 Quadratic Adjustment Costs 217
9.4 Richer Models of Adjustment 224
9.4.1 Piecewise Linear Adjustment Costs 224
9.4.2 Nonconvex Adjustment Costs 226
9.4.3 Asymmetries 228
9.5 The Gap Approach 229
9.5.1 Partial Adjustment Model 230
9.5.2 Measuring the Target and the Gap 231
9.6 Estimation of a Rich Model of Adjustment Costs 235
9.7 Conclusion 238
10 Future Developments 241

10.2 Price Setting 241
10.2.1 Optimization Problem 242
10.2.2 Evidence on Magazine Prices 244
10.2.3 Aggregate Implications 245
10.3 Optimal Inventory Policy 248
10.3.1 Inventories and the Production-Smoothing
Model 248
10.3.2 Prices and Inventory Adjustment 252
10.4 Capital and Labor 254
Contents xi
10.5 Technological Complementarities: Equilibrium

Analysis 255
10.6 Search Models 257
10.6.1 A Simple Labor Search Model 257
10.6.2 Estimation of the Labor Search Model 259
10.6.3 Extensions 260
10.7 Conclusion 263
Bibliography 265
Index 275
Dynamic Economics
1 Overview
In this book we study a rich set of applied problems in economics

that emphasize the dynamic aspects of economic decisions. Although
our ultimate goals are the applications, we provide some basic tech-
niques before tackling the details of specic dynamic optimization
problems. This way we are able to present and integrate key tools
such as dynamic programming, numerical techniques, and simula-
tion based econometric methods. We utilize these tools in a variety
of applications in both macroeconomics and microeconomics. Over-
all, this approach allows us to estimate structural parameters and to
analyze the effects of economic policy.
The approach we pursue to studying economic dynamics is struc-
tural. As researchers we have frequently found ourselves inferring
underlying parameters that represent tastes, technology, and other
primitives from observations of individual households and rms
as well as from economic aggregates. When such inferences are suc-
cessful, we can then test competing hypotheses about economic
behavior and evaluate the effects of policy experiments.
To appreciate the benets of this approach, consider the following
policy experiment. In recent years a number of European govern-
ments have instituted policies of subsidizing the scrapping of old
cars and the purchase of new cars. What are the expected effects of
these policies on the car industry and on government revenues?
At some level this question seems easy if a researcher knows the
demand function for cars. But of course that demand function is, at
best, elusive. Further the demand function estimated in one policy
regime is unlikely to be very informative for a novel policy experi-
ment, such as this example of car scrapping subsidies.
An alternative approach is to build and estimate a model of
household dynamic choice of car ownership. Once the parameters of
2 Chapter 1
this model are estimated, then various policy experiments can be

evaluated.1 This approach seems considerably more difcult than
just estimating a demand function, and of course this is the case. It
requires the specication and solution of a dynamic optimization
problem and then the estimation of the parameters. But, as we argue
in this book, this methodology is feasible and yields exciting results.
The integration of dynamic optimization with parameter estima-
tion is at the heart of our approach. We develop this idea by orga-
nizing the book in two parts.
Part I provides a review of the formal theory of dynamic opti-
mization. This is a tool used in many areas of economics, includ-
ing macroeconomics, industrial organization, labor economics, and
international economics. As in previous contributions to the study
of dynamic optimization, such as by Sargent (1987) and by Stokey
and Lucas (1989), our presentation starts with the formal theory of
dynamic programming. Because of the large number of other con-
tributions in this area, our presentation in chapter 2 relies on existing
theorems on the existence of solutions to a variety of dynamic pro-
gramming problems.
In chapter 3 we present the numerical tools necessary to conduct
a structural estimation of the theoretical dynamic models. These
numerical tools serve both to complement the theory in teaching
students about dynamic programming and to enable a researcher to
evaluate the quantitative implications of the theory. In our experi-
ence the process of writing computer code to solve dynamic pro-
gramming problems has proved to be a useful device for teaching
basic concepts of this approach.
The econometric techniques of chapter 4 provide the link between
the dynamic programming problem and data. The emphasis is on
the mapping from parameters of the dynamic programming prob-
lem to observations. For example, a vector of parameters is used to
numerically solve a dynamic programming problem that is used to
simulate moments. An optimization routine then selects a vector of
parameters to bring these simulated moments close to the moments
observed in the data.
Part II is devoted to the application of dynamic programming
to specic areas of economics such as the study of business cycles,
consumption, and investment behavior. The presentation of each
1. This exercise is described in some detail in the chapter on consumer durables in this
book.
Overview 3
application in chapters 5 through 10 contains four elements: pre-

sentation of the optimization problem as a dynamic programming
problem, characterization of the optimal policy functions, estimation
of the parameters, and policy evaluation using this model.
While the applications might be characterized as macroeconomics,
the methodology is valuable in other areas of economic research, in
terms of both the topics and the techniques. These applications uti-
lize material from many other parts of economics. For example, the
analysis of the stochastic growth model includes taxation and our
discussion of factor adjustment at the plant level is certainly relevant
to researchers in labor and industrial organization. Moreover we
envision these techniques to be useful in any problem where the
researcher is applying dynamic optimization to the data. The chap-
ters contain references to other applications of these techniques.
What is new about our presentation is the use of an integrated
approach to the empirical implementation of dynamic optimiza-
tion models. Previous texts have provided a mathematical basis
for dynamic programming, but those presentations generally do not
contain any quantitative applications. Other texts present the under-
lying econometric theory but generally without specic applications.
Our approach does both, and thus aims to link theory and applica-
tion as illustrated in the chapters of part II.
Our motivation for writing this book should be clear. From
the perspective of understanding dynamic programming, explicit
empirical applications complement the underlying theory of opti-
mization. From the perspective of applied macroeconomics, explicit
dynamic optimization problems, posed as dynamic programming
problems, provide needed structure for estimation and policy
evaluation.
Since the book is intended to teach empirical applications of
dynamic programming problems, we have created a Web site for the
presentation of computer codes (MATLAB and GAUSS) as well as
data sets useful for the applications. This material should appeal to
readers wishing to supplement the presentation in part II, and we
hope that Web site will become a forum for further development of
codes.
Our writing this book has beneted from joint work with Joao
Ejarque, John Haltiwanger, Alok Johri, and Jonathan Willis. We
thank these co-authors for their generous sharing of ideas and com-
puter code as well as their comments on the nal draft. Thanks also
4 Chapter 1
go to Victor Aguirregabiria, Yan Bai, Joyce Cooper, Dean Corbae,

Zvi Eckstein, Simon Gilchrist, Hang Kang, Peter Klenow, Sam Kor-
tum, Valerie Lechene, Nicola Pavoni, Aldo Rustichini, and Marcos
Vera for comments on various parts of the book. We also appreciate
the comments of outside reviewers and the editorial staff at The MIT
Press. Finally, we are grateful to our many masters and doctoral
students at Tel Aviv University, University of Texas at Austin, the
IDEI at the Universite de Toulouse, the NAKE PhD program in
Holland, the University of Haifa, the University of Minnesota, and
University College London for their numerous comments and sug-
gestions during the preparation of this book.
I Theory
2 Theory of Dynamic
Programming
2.1 Overview
The mathematical theory of dynamic programming as a means of

solving dynamic optimization problems dates to the early contribu-
tions of Bellman (1957) and Bertsekas (1976). For economists, the
contributions of Sargent (1987) and Stokey and Lucas (1989) provide
a valuable bridge to this literature.
2.2 Indirect Utility
Intuitively, the approach of dynamic programming can be under-

stood by recalling the theme of indirect utility from basic static con-
sumer theory or a reduced form prot function generated by the
optimization of a rm. These reduced form representations of pay-
offs summarize information about the optimized value of the choice
problems faced by households and rms. As we will see, the theory
of dynamic programming takes this insight to a dynamic context.
2.2.1 Consumers
Consumer choice theory focuses on households that solve

VI; p max uc
c
subject to pc I;
where c is a vector of consumption goods, p is a vector of prices and

I is income.1 The rst-order condition is given by
1. Assume that there are J commodities in this economy. This presentation assumes
that you understand the conditions under which this optimization problem has a
solution and when that solution can be characterized by rst-order conditions.
8 Chapter 2
uj c
l for j 1; 2; . . . ; J;
pj
where l is the multiplier on the budget constraint and uj c is the

marginal utility from good j.
Here VI; p is an indirect utility function. It is the maximized level
of utility from the current state I; p. Someone in this state can be
predicted to attain this level of utility. One does not need to know
what that person will do with his income; it is enough to know that
he will act optimally. This is very powerful logic and underlies the
idea behind the dynamic programming models studied below.
To illustrate, what happens if we give the consumer a bit more
income? Welfare goes up by VI I; p > 0. Can the researcher predict
what will happen with a little more income? Not really since the
optimizing consumer is indifferent with respect to how this is spent:
uj c
VI I; p for all j:
pj
It is in this sense that the indirect utility function summarizes the

value of the households optimization problem and allows us to
determine the marginal value of income without knowing more
about consumption functions.
Is this all we need to know about household behavior? No, this
theory is static. It ignores savings, spending on durable goods, and
uncertainty over the future. These are all important components in
the household optimization problem. We will return to these in later
chapters on the dynamic behavior of households. The point here was
simply to recall a key object from optimization theory: the indirect
utility function.
2.2.2 Firms
Suppose that a rm must choose how many workers to hire at a

wage of w given its stock of capital k and product price p. Thus the
rm must solve
Pw; p; k max pf l; k wl:
l
A labor demand function results that depends on w; p; k. As with

VI; p, Pw; p; k summarizes the value of the rm given factor
Theory of Dynamic Programming 9
prices, the product price p, and the stock of capital k. Both the exi-
ble and xed factors can be vectors.
Think of Pw; p; k as an indirect prot function. It completely
summarizes the value of the optimization problem of the rm given
w; p; k.
As with the households problem, given Pw; p; k, we can directly
compute the marginal value of allowing the rm some additional
capital as Pk w; p; k p fk l; k without knowing how the rm will
adjust its labor input in response to the additional capital.
But, is this all there is to know about the rms behavior? Surely
not, for we have not specied where k comes from. So the rms
problem is essentially dynamic, though the demand for some of
its inputs can be taken as a static optimization problem. These are
important themes in the theory of factor demand, and we will return
to them in our rm applications.
2.3 Dynamic Optimization: A Cake-Eating Example
Here we will look at a very simple dynamic optimization problem.

We begin with a nite horizon and then discuss extensions to the
innite horizon.2
Suppose that you are presented with a cake of size W1 . At each
point of time, t 1; 2; 3; . . . ; T, you can eat some of the cake but must
save the rest. Let ct be your consumption in period t, and let uct
represent the ow of utility from this consumption. The utility func-
tion is not indexed by time: preferences are stationary. We can
assume that u is real valued, differentiable, strictly increasing, and
strictly concave. Further we can assume lim c!0 u 0 c ! y. We could
represent your lifetime utility by
X
T
b t1 uct ;
t1
where 0 a b a 1 and b is called the discount factor.

For now, we assume that the cake does not depreciate (spoil) or
grow. Hence the evolution of the cake over time is governed by
Wt1 Wt ct 2:1
2. For a very complete treatment of the nite horizon problem with uncertainty, see
Bertsekas (1976).
10 Chapter 2
for t 1; 2; . . . ; T. How would you nd the optimal path of con-

sumption, fct g1T ?3
2.3.1 Direct Attack
One approach is to solve the constrained optimization problem

directly. This is called the sequence problem by Stokey and Lucas
(1989). Consider the problem of
X
T
max bt1 uct 2:2
fct g1T ; fWt g2T1 t1
subject to the transition equation (2.1), which holds for t 1; 2; 3; . . . ;

T. Also there are nonnegativity constraints on consuming the cake
given by ct b 0 and Wt b 0. For this problem, W1 is given.
Alternatively, the ow constraints imposed by (2.1) for each t
could be combined, yielding
X
T
ct WT1 W1 : 2:3
t1
The nonnegativity constraints are simpler: ct b 0 for t 1; 2; . . . ; T

and WT1 b 0. For now, we will work with the single resource con-
straint. This is a well-behaved problem as the objective is concave
and continuous and the constraint set is compact. So there is a solu-
tion to this problem.4
Letting l be the multiplier on (2.3), the rst-order conditions are
given by
b t1 u 0 ct l for t 1; 2; . . . ; T
and
l f;
where f is the multiplier on the nonnegativity constraint on WT1 .

The nonnegativity constraints on ct b 0 are ignored, as we can
assume that the marginal utility of consumption becomes innite
as consumption approaches zero within any period.
3. Throughout, the notation fxt g1T is used to dene the sequence x1 ; x2 ; . . . ; xT for
some variable x.
4. This comes from the Weierstrass theorem. See Bertsekas (1976, app. B) or Stokey
and Lucas (1989, ch. 3) for a discussion.
Combining the equations, we obtain an expression that links con-

sumption across any two periods:
u 0 ct bu 0 ct1 : 2:4
This is a necessary condition of optimality for any t: if it is violated,

the agent can do better by adjusting ct and ct1 . Frequently (2.4) is
referred to as a Euler equation.
To understand this condition, suppose that you have a proposed
(candidate) solution for this problem given by fct g1T , fWt g2T1 .
Essentially the Euler equation says that the marginal utility cost of
reducing consumption by e in period t equals the marginal utility
gain from consuming the extra e of cake in the next period, which
is discounted by b. If the Euler equation holds, then it is impossible
to increase utility by moving consumption across adjacent periods
given a candidate solution.
It should be clear though that this condition may not be sufcient:
it does not cover deviations that last more than one period. For
example, could utility be increased by reducing consumption by e
in period t saving the cake for two periods and then increasing
consumption in period t 2? Clearly, this is not covered by a single
Euler equation. However, by combining the Euler equation that hold
across period t and t 1 with that which holds for periods t 1 and
t 2, we can see that such a deviation will not increase utility. This is
simply because the combination of Euler equations implies that
u 0 ct b 2 u 0 ct2
so that the two-period deviation from the candidate solution will not
increase utility.
As long as the problem is nite, the fact that the Euler equation
holds across all adjacent periods implies that any nite deviations
from a candidate solution that satises the Euler equations will not
increase utility.
Is this enough? Not quite. Imagine a candidate solution that sat-
ises all of the Euler equations but has the property that WT > cT so
that there is cake left over. This is clearly an inefcient plan: satisfy-
ing the Euler equations is necessary but not sufcient. The optimal
solution will satisfy the Euler equation for each period until the
agent consumes the entire cake.
Formally, this involves showing that the nonnegativity constraint
on WT1 must bind. In fact, this constraint is binding in the solution
12 Chapter 2
above: l f > 0. This nonnegativity constraint serves two important

purposes. First, in the absence of a constraint that WT1 b 0, the
agent would clearly want to set WT1 y. This is clearly not fea-
sible. Second, the fact that the constraint is binding in the optimal
solution guarantees that cake does not remain after period T.
In effect the problem is pinned down by an initial condition
(W1 is given) and by a terminal condition (WT1 0). The set of
(T 1) Euler equations and (2.3) then determine the time path of
consumption.
Let the solution to this problem be denoted by VT W1 , where T
is the horizon of the problem and W1 is the initial size of the cake.
VT W1 represents the maximal utility ow from a T-period problem
given a size W1 cake. From now on, we call this a value function.
This is completely analogous to the indirect utility functions ex-
pressed for the household and the rm.
As in those problems, a slight increase in the size of the cake leads
to an increase in lifetime utility equal to the marginal utility in any
period. That is,
VT0 W1 l b t1 u 0 ct ; t 1; 2; . . . ; T:
It doesnt matter when the extra cake is eaten given that the con-
sumer is acting optimally. This is analogous to the point raised
above about the effect on utility of an increase in income in the con-
sumer choice problem with multiple goods.
2.3.2 Dynamic Programming Approach
Suppose that we change the problem slightly: we add a period 0

and give an initial cake of size W0 . One approach to determining
the optimal solution of this augmented problem is to go back to the
sequence problem and resolve it using this longer horizon and new
constraint. But, having done all of the hard work with the T period
problem, it would be nice not to have to do it again.
Finite Horizon Problem

The dynamic programming approach provides a means of doing
so. It essentially converts a (arbitrary) T-period problem into a two-
period problem with the appropriate rewriting of the objective func-
tion. This way it uses the value function obtained from solving a
shorter horizon problem.
By adding a period 0 to our original problem, we can take advan-

tage of the information provided in VT W1 , the solution of the T-
period problem given W1 from (2.2). Given W0 , consider the problem
of
max uc0 bVT W1 ; 2:5
c0
where
W1 W0 c 0 ; W0 given:
In this formulation the choice of consumption in period 0 determines
the size of the cake that will be available starting in period 1, W1 .
Now, instead of choosing a sequence of consumption levels, we just
nd c0 . Once c0 and thus W1 are determined, the value of the prob-
lem from then on is given by VT W1 . This function completely
summarizes optimal behavior from period 1 onward. For the pur-
poses of the dynamic programming problem, it does not matter how
the cake will be consumed after the initial period. All that is impor-
tant is that the agent will be acting optimally and thus generating
utility given by VT W1 . This is the principle of optimality, due to
Richard Bellman, at work. With this knowledge, an optimal decision
can be made regarding consumption in period 0.
Note that the rst-order condition (assuming that VT W1 is dif-
ferentiable) is given by
u 0 c0 bVT0 W1
so that the marginal gain from reducing consumption a little in
period 0 is summarized by the derivative of the value function. As
noted in the earlier discussion of the T-period sequence problem,
VT0 W1 u 0 c1 b t u 0 ct1
for t 1; 2; . . . ; T 1. Using these two conditions together yields

u 0 ct bu 0 ct1
for t 0; 1; 2; . . . ; T 1, a familiar necessary condition for an optimal

solution.
Since the Euler conditions for the other periods underlie the cre-
ation of the value function, one might suspect that the solution to the
T 1 problem using this dynamic programming approach is identi-
14 Chapter 2
cal to that of the sequence approach.5 This is clearly true for this
problem: the set of rst-order conditions for the two problems are
identical, and thus, given the strict concavity of the uc functions,
the solutions will be identical as well.
The apparent ease of this approach, however, may be misleading.
We were able to make the problem look simple by pretending that
we actually know VT W1 . Of course, the way we could solve for this
is by either tackling the sequence problem directly or building it
recursively, starting from an initial single-period problem.
On this recursive approach, we could start with the single-
period problem implying V1 W1 . We would then solve (2.5) to build
V2 W1 . Given this function, we could move to a solution of the
T 3 problem and proceed iteratively, using (2.5) to build VT W1
for any T.
Example
We illustrate the construction of the value function in a specic
example. Assume uc lnc. Suppose that T 1. Then V1 W1
lnW1 .
For T 2, the rst-order condition from (2.2) is
1 b
;
c1 c2
and the resource constraint is

W1 c1 c2 :
Working with these two conditions, we have
W1 bW1
c1 and c2 :
1b 1b
From this, we can solve for the value of the two-period problem:
V2 W1 lnc1 b lnc2 A2 B2 lnW1 ; 2:6
where A2 and B2 are constants associated with the two-period prob-
lem. These constants are given by

1 b
A2 ln b ln ; B2 1 b:
1b 1b
5. By the sequence approach, we mean solving the problem using the direct approach
outlined in the previous section.
Importantly, (2.6) does not include the max operator as we are sub-
stituting the optimal decisions in the construction of the value func-
tion, V2 W1 .
Using this function, the T 3 problem can then be written as
V3 W1 max lnW1 W2 bV2 W2 ;
W2
where the choice variable is the state in the subsequent period. The
rst-order condition is
1
bV20 W2 :
c1
Using (2.6) evaluated at a cake of size W2 , we can solve for V20 W2

implying:
1 B2 b
b :
c1 W2 c 2
Here c2 the consumption level in the second period of the three-

period problem and thus is the same as the level of consumption in
the rst period of the two-period problem. Further we know from
the two-period problem that
1 b
:
c2 c3
This plus the resource constraint allows us to construct the solution

of the three-period problem:
W1 bW1 b 2 W1
c1 ; c2 ; c3 :
1 b b2 1 b b2 1 b b2
Substituting into V3 W1 yields
V3 W1 A3 B3 lnW1 ;
where

1 b 2 b2
A3 ln b ln b ln ;
1 b b2 1 b b2 1 b b2
B3 1 b b 2 :
This solution can be veried from a direct attack on the three-period
problem using (2.2) and (2.3).
16 Chapter 2
2.4 Some Extensions of the Cake-Eating Problem
Here we go beyond the T-period problem to illustrate some ways

to use the dynamic programming framework. This is intended as an
overview, and the details of the assertions, and so forth, will be pro-
vided below.
2.4.1 Innite Horizon
Basic Structure
Suppose that for the cake-eating problem, we allow the horizon to
go to innity. As before, one can consider solving the innite hori-
zon sequence problem given by
X
y
max b t uct
fct g1 ; fWt gy
y
2 t1
along with the transition equation of
Wt1 Wt ct for t 1; 2; . . . :
In specifying this as a dynamic programming problem, we write
VW max uc bVW c for all W:

c A 0; W
Here uc is again the utility from consuming c units in the current

period. VW is the value of the innite horizon problem starting
with a cake of size W. So in the given period, the agent chooses cur-
rent consumption and thus reduces the size of the cake to W 0 W c,
as in the transition equation. We use variables with primes to denote
future values. The value of starting the next period with a cake of
that size is then given by VW c, which is discounted at rate b < 1.
For this problem, the state variable is the size of the cake (W)
given at the start of any period. The state completely summarizes all
information from the past that is needed for the forward-looking
optimization problem. The control variable is the variable that is
being chosen. In this case it is the level of consumption in the current
period c. Note that c lies in a compact set. The dependence of the
state tomorrow on the state today and the control today, given by
W 0 W c;
is called the transition equation.
Alternatively, we can specify the problem so that instead of

choosing todays consumption we choose tomorrows state:
VW max uW W 0 bVW 0 for all W: 2:7
W 0 A 0; W
Either specication yields the same result. But choosing tomorrows

state often makes the algebra a bit easier, so we will work with (2.7).
This expression is known as a functional equation, and it is often
called a Bellman equation after Richard Bellman, one of the origi-
nators of dynamic programming. Note that the unknown in the
Bellman equation is the value function itself: the idea is to nd a
function VW that satises this condition for all W. Unlike the nite
horizon problem, there is no terminal period to use to derive the
value function. In effect, the xed point restriction of having VW
on both sides of (2.7) will provide us with a means of solving the
functional equation.
Note too that time itself does not enter into Bellmans equation: we
can express all relations without an indication of time. This is the
essence of stationarity.6 In fact we will ultimately use the station-
arity of the problem to make arguments about the existence of a
value function satisfying the functional equation.
A nal very important property of this problem is that all infor-
mation about the past that bears on current and future decisions is
summarized by W, the size of the cake at the start of the period.
Whether the cake is of this size because we initially have a large cake
and can eat a lot of it or a small cake and are frugal eaters is not rel-
evant. All that matters is that we have a cake of a given size. This
property partly reects the fact that the preferences of the agent do
not depend on past consumption. If this were the case, we could
amend the problem to allow this possibility.
The next part of this chapter addresses the question of whether
there exists a value function that satises (2.7). For now we assume
that a solution exists so that we can explore its properties.
The rst-order condition for the optimization problem in (2.7) can
be written as
u 0 c bV 0 W 0 :
6. As you may already know, stationarity is vital in econometrics as well. Thus

making assumptions of stationarity in economic theory have a natural counterpart
in empirical studies. In some cases we will have to modify optimization problems to
ensure stationarity.
18 Chapter 2
This may look simple, but what is the derivative of the value func-
tion? It is particularly hard to answer this, since we do not know
VW. However, we can use the fact that VW satises (2.7) for all
W to calculate V 0 . Assuming that this value function is differentiable,
we have
V 0 W u 0 c;
a result we have seen before. Since this holds for all W, it will hold in
the following period, yielding
V 0 W 0 u 0 c 0 :
Substitution leads to the familar Euler equation:
u 0 c bu 0 c 0 :
The solution to the cake-eating problem will satisfy this necessary
condition for all W.
The link from the level of consumption and next periods cake (the
controls from the different formulations) to the size of the cake (the
state) is given by the policy function:
c fW; W 0 jW 1 W fW:
Substituting these values into the Euler equation reduces the prob-
lem to these policy functions alone:
u 0 fW bu 0 fW fW for all W:
The policy functions above are important in applied research,
for they provide the mapping from the state to actions. When ele-
ments of the state as well as the action are observable, these policy
functions will provide the means for estimating the underlying
parameters.
An Example
In general, it is not actually possible to nd closed form solutions
for the value function and the resulting policy functions. So we try to
characterize certain properties of the solution, and for some cases,
we solve these problems numerically.
Nevertheless, as indicated by our analysis of nite horizon prob-
lems, there are some specications of the utility function that allow
us to nd a closed form solution to the value function. Suppose, as
above, that uc lnc. From the results of the T-period problem,

we might conjecture that the solution to the functional equation takes
the form of
VW A B lnW for all W:

By this expression we have reduced the dimensionality of the un-
known function VW to two parameters, A and B. But can we
nd values for A and B such that VW will satisfy the functional
equation?
Let us suppose that we can. For these two values the functional
equation becomes
A B lnW max
0
lnW W 0 bA B lnW 0 for all W:
W
2:8
After some algebra, the rst-order condition becomes
bB
W 0 jW W:
1 bB
Using this in (2.8) results in

W bBW
A B lnW ln b A B ln for all W:
1 bB 1 bB
Collecting the terms that multiply lnW and using the requirement
that the functional equation holds for all W, we nd that
1
B
1b
is required for a solution. After this, the expression can also be used
to solve for A. Thus we have veried that our guess is a solution to
the functional equation. We know that because we can solve for
A; B such that the functional equation holds for all W using the
optimal consumption and savings decision rules.
With this solution, we know that
c W1 b; W 0 bW:
This tells us that the optimal policy is to save a constant fraction of

the cake and eat the remaining fraction.
The solution to B can be estimated from the solution to the T-
period horizon problems where
20 Chapter 2
X
T
BT b t1 :
t1
Clearly, B limT!y BT . We will be exploiting the idea of using the

value function to solve the innite horizon problem as it is related to
the limit of the nite solutions in much of our numerical analysis.
Below are some exercises that provide some further elements to
this basic structure. Both begin with nite horizon formulations and
then progress to the innite horizon problems.
exercise 2.1 Utility in period t is given by uct ; ct1 . Solve a T-

period problem using these preferences. Interpret the rst-order
conditions. How would you formulate the Bellman equation for the
innite horizon version of this problem?
exercise 2.2 The transition equation is modied so that

Wt1 rWt ct ;
where r > 0 represents a return from holding cake inventories.

Solve the T-period problem with this storage technology. Interpret
the rst-order conditions. How would you formulate the Bellman
equation for the innite horizon version of this problem? Does the
size of r matter in this discussion? Explain.
2.4.2 Taste Shocks
A convenient feature of the dynamic programming problem is the

ease with which uncertainty can be introduced.7 For the cake-eating
problem, the natural source of uncertainty has to do with the agents
appetite. In other settings we will focus on other sources of uncer-
tainty having to do with the productivity of labor or the endow-
ments of households.
To allow for variations of appetite, suppose that utility over con-
sumption is given by
euc;
where e is a random variable whose properties we will describe
below. The function uc is again assumed to be strictly increasing
7. To be careful, here we are adding shocks that take values in a nite and thus
countable set. See the discussion in Bertsekas (1976, sec. 2.1) for an introduction to the
complexities of the problem with more general statements of uncertainty.
and strictly concave. Otherwise, the problem is the original cake-

eating problem with an initial cake of size W.
In problems with stochastic elements, it is critical to be precise
about the timing of events. Does the optimizing agent know the
current shocks when making a decision? For this analysis, assume
that the agent knows the value of the taste shock when making cur-
rent decisions but does not know the future values of this shock.
Thus the agent must use expectations of future values of e when
deciding how much cake to eat today: it may be optimal to consume
less today (save more) in anticipation of a high realization of e in the
future.
For simplicity, assume that the taste shock takes on only two
values: e A feh ; el g with eh > el > 0. Further we can assume that the
taste shock follows a rst-order Markov process,8 which means that
the probability that a particular realization of e occurs in the cur-
rent period depends only the value of e attained in the previous
period.9 For notation, let pij denote the probability that the value of
e goes from state i in the current period to state j in the next period.
For example, plh is dened from
plh 1 Probe 0 eh j e el ;
where e 0 refers to the future value of e. Clearly, pih pil 1 for
i h; l. Let P be a 2 2 matrix with a typical element pij that
summarizes the information about the probability of moving across
states. This matrix is logically called a transition matrix.
With this notation and structure, we can turn again to the cake-
eating problem. We need to carefully dene the state of the system
for the optimizing agent. In the nonstochastic problem, the state was
simply the size of the cake. This provided all the information the
agent needed to make a choice. When taste shocks are introduced,
the agent needs to take this factor into account as well. We know
that the taste shocks provide information about current payoffs and,
through the P matrix, are informative about the future value of the
taste shock as well.10
8. For more details on Markov chains, we refer the reader to Ljungqvist and Sargent
(2000).
9. The evolution can also depend on the control of the previous period. Note too that
by appropriate rewriting of the state space, richer specications of uncertainty can be
encompassed.
10. This is a point that we return to below in our discussion of the capital accumula-
tion problem.
22 Chapter 2
Formally the Bellman equation is written
VW; e max
0
euW W 0 bEe 0 j e VW 0 ; e 0 for all W; e;
W
where W 0 W c as before. Note that the conditional expectation is

denoted here by Ee 0 j e VW 0 ; e 0 which, given P, is something we can
compute.11
The rst-order condition for this problem is given by
eu 0 W W 0 bEe 0 j e V1 W 0 ; e 0 for all W; e:

Using the functional equation to solve for the marginal value of cake,
we nd that
eu 0 W W 0 bEe 0 j e e 0 u 0 W 0 W 00 : 2:9
This, of course, is the stochastic Euler equation for this problem.
The optimal policy function is given by
W 0 jW; e:
The Euler equation can be rewritten in these terms as

eu 0 W jW; e bEe 0 j e e 0 u 0 jW; e jjW; e; e 0 :
The properties of the policy function can then be deduced from this
condition. Clearly, both e 0 and c 0 depend on the realized value of e
so that the expectation on the right side of (2.9) cannot be split into
two separate pieces.
2.4.3 Discrete Choice
To illustrate the exibility of the dynamic programming approach,

we build on this stochastic problem. Suppose that the cake must
be eaten in one period. Perhaps we should think of this as the wine-
drinking problem, recognizing that once a good bottle of wine is
opened, it must be consumed. Further we can modify the transition
equation to allow the cake to grow (depreciate) at rate r.
The cake consumption example becomes then a dynamic, sto-
chastic discrete choice problem. This is part of a family of problems
called optimal stopping problems.12 The common element in all of
11. Throughout we denote the conditional expectation of e 0 given e as Ee 0 j e .

12. Eckstein and Wolpin (1989) provide an extensive discussions of the formulation
and estimation of these problems in the context of labor applications.
these problems is the emphasis on the timing of a single event: when

to eat the cake, when to take a job, when to stop school, when to stop
revising a chapter, and so on. In fact, for many of these problems,
these choices are not once in a lifetime events, so we will be looking
at problems even richer than those of the optimal stopping variety.
Let V E W; e and V N W; e be the values of eating size W cake
now (E) and waiting (N), respectively, given the current taste shock
e A feh ; el g. Then
V E W; e euW
and
V N W bEe 0 j e VrW; e 0 ;
where
VW; e maxV E W; e; V N W; e for all W; e:

To understand this better, the term euW is the direct utility ow
from eating the cake. Once the cake is eaten, the problem has ended.
So V E W; e is just a one-period return. If the agent waits, then there
is no cake consumption in the current period, and in the next period
the cake is of size rW. As tastes are stochastic, the agent choosing
to wait must take expectations of the future taste shock, e 0 . The agent
has an option in the next period of eating the cake or waiting some
more. Hence the value of having the cake in any state is given by
VW; e, which is the value attained by maximizing over the two
options of eating or waiting. The cost of delaying the choice is
determined by the discount factor b while the gains to delay are
associated with the growth of the cake, parameterized by r. Further
the realized value of e will surely inuence the relative value of con-
suming the cake immediately.
If r a 1, then the cake doesnt grow. In this case there is no gain
from delay when e eh . If the agent delays, then utility in the next
period will have to be lower due to discounting, and with probabil-
ity phl , the taste shock will switch from low to high. So waiting to eat
the cake in the future will not be desirable. Hence
VW; eh V E W; eh eh uW for all W:

In the low e state, matters are more complex. If b and r are suf-
ciently close to 1, then there is not a large cost to delay. Further, if plh is
24 Chapter 2
sufciently close to 1, then it is likely that tastes will switch from low
to high. Thus it will be optimal not to eat the cake in state W; el .13
Here are some additional exercises.
exercise 2.3 Suppose that r 1. For a given b, show that there

exists a critical level of plh , denoted by plh such that if plh > plh , then
the optimal solution is for the agent to wait when e el and to eat
the cake when eh is realized.
exercise 2.4 When r > 1, the problem is more difcult. Suppose
that there are no variations in tastes: eh el 1. In this case there is
a trade-off between the value of waiting (as the cake grows) and the
cost of delay from discounting.
Suppose that r > 1 and uc c 1g =1 g. What is the solution
to the optimal stopping problem when br 1g < 1? What happens if
br 1g > 1? What happens when uncertainty is added?
2.5 General Formulation
Building on the intuition gained from the cake-eating problem, we

now consider a more formal abstract treatment of the dynamic pro-
gramming approach.14 We begin with a presentation of the non-
stochastic problem and then add uncertainty to the formulation.
2.5.1 Nonstochastic Case
Consider the innite horizon optimization problem of an agent with

a payoff function for period t given by s~st ; ct . The rst argument of
the payoff function is termed the state vector st . As noted above,
this represents a set of variables that inuences the agents return
within the period, but by assumption, these variables are outside
of the agents control within period t. The state variables evolve over
time in a manner that may be inuenced by the control vector ct ,
the second argument of the payoff function. The connection between
the state variables over time is given by the transition equation:
13. In the following chapter on the numerical approach to dynamic programming, we

study this case in considerable detail.
14. This section is intended to be self-contained and thus repeats some of the material
from the earlier examples. Our presentation is by design not as formal as say that
provided in Bertsekas (1976) or Stokey and Lucas (1989). The reader interested in more
mathematical rigor is urged to review those texts and their many references.
st1 tst ; ct :
So, given the current state and the current control, the state vector
for the subsequent period is determined.
Note that the state vector has a very important property: it
completely summarizes all of the information from the past that is
needed to make a forward-looking decision. While preferences and
the transition equation are certainly dependent on the past, this
dependence is represented by st : other variables from the past do
not affect current payoffs or constraints and thus cannot inuence
current decisions. This may seem restrictive but it is not: the vector
st may include many variables so that the dependence of current
choices on the past can be quite rich.
While the state vector is effectively determined by preferences and
the transition equation, the researcher has some latitude in choosing
the control vector. That is, there may be multiple ways of represent-
ing the same problem with alternative specications of the control
variables.
We assume that c A C and s A S. In some cases the control is
restricted to be in a subset of C that depends on the state vector:
c A Cs. Further we assume that s~s; c is bounded for s; c A S C.15
For the cake-eating problem described above, the state of the sys-
tem was the size of the current cake Wt and the control variable
was the level of consumption in period t, ct . The transition equation
describing the evolution of the cake was given by
Wt1 Wt ct :
Clearly, the evolution of the cake is governed by the amount of
current consumption. An equivalent representation, as expressed in
(2.7), is to consider the future size of the cake as the control variable
and then to simply write current consumption as Wt1 Wt .
There are two nal properties of the agents dynamic optimization
problem worth specifying: stationarity and discounting. Note that
neither the payoff nor the transition equations depend explicitly on
time. True the problem is dynamic, but time per se is not of the
essence. In a given state the optimal choice of the agent will be the
same regardless of when he optimizes. Stationarity is important
15. Ensuring that the problem is bounded is an issue in some economic applications,
such as the growth model. Often these problems are dealt with by bounding the sets C
and S.
26 Chapter 2
both for the analysis of the optimization problem and for empirical
implementation of innite horizon problems. In fact, because of sta-
tionarity, we can dispense with time subscripts as the problem is
completely summarized by the current values of the state variables.
The agents preferences are also dependent on the rate at which
the future is discounted. Let b denote the discount factor and assume
that 0 < b < 1. Then we can represent the agents payoffs over the
innite horizon as
X
ty
b t s~st ; ct : 2:10
t0
One approach to optimization is to maximize (2.10) through the

choice of fct g for t 0; 1; 2; . . . given s0 and subject to the transition
equation. Let Vs0 be the optimized value of this problem given the
initial state.
Alternatively, one can adopt the dynamic program approach and
consider the following equation, called Bellmans equation:
Vs max s~s; c bVs 0 for all s A S; 2:11

c A Cs
where s 0 ts; c. Here time subscripts are eliminated, reecting the

stationarity of the problem. Instead, current variables are unprimed
while future ones are denoted by a prime.
As in Stokey and Lucas (1989), the problem can be formulated as
Vs max
0
ss; s 0 bVs 0 for all s A S: 2:12
s A Gs
This is a more compact formulation, and we will use it for our pre-
sentation.16 Nonetheless, the presentations in Bertsekas (1976) and
Sargent (1987) follow (2.11). Assume that S is a convex subset of < k .
Let the policy function that determines the optimal value of the
control (the future state) given the state be given by s 0 fs. Our
interest is ultimately in the policy function, since we generally
observe the actions of agents rather than their levels of utility. Still,
to determine fs, we need to solve (2.12). That is, we need to nd
the value function that satises (2.12). It is important to realize that
while the payoff and transition equations are primitive objects that
16. Essentially this formulation inverts the transition equation and substitutes for c in
the objective function. This substitution is reected in the alternative notation for the
return function.
models specify a priori, the value function is derived as the solution

of the functional equation, (2.12).
There are many results in the lengthy literature on dynamic pro-
gramming problems on the existence of a solution to the functional
equation. Here we present one set of sufcient conditions. The reader
is referred to Bertsekas (1976), Sargent (1987), and Stokey and Lucas
(1989) for additional theorems under alternative assumptions about
the payoff and transition functions.17
theorem 1 Assume that ss; s 0 is real-valued, continuous, and

bounded, 0 < b < 1, and that the constraint set, Gs, is nonempty,
compact-valued, and continuous. Then there exists a unique value
function Vs that solves (2.12).
Proof See Stokey and Lucas (1989, thm. 4.6).

Instead of a formal proof, we give an intuitive sketch. The key
component in the analysis is the denition of an operator, commonly
denoted as T, dened by18
TWs max ss; s 0 bWs 0 for all s A S:
s 0 A Gs
So by this mapping we take a guess on the value function, and

working through the maximization for all s, we produce another
value function, TWs. Clearly, any Vs such that Vs TVs
for all s A S is a solution to (2.12). So we can reduce the analysis to
determining the xed points of TW.
The xed point argument proceeds by showing the TW is a con-
traction using a pair of sufcient conditions from Blackwell (1965).
These conditions are (1) monotonicity and (2) discounting of the
mapping TV. Monotonicity means that if Ws b Qs for all s A S,
then TWs b TQs for all s A S. This property can be directly
veried from the fact that TV is generated by a maximization
problem. That is, let fQ s be the policy function obtained from
max ss; s 0 bQs 0 for all s A S:

s 0 A Gs
When the proposed value function is Ws, then
17. Some of the applications explored in this book will not exactly t these conditions
either. In those cases we will alert the reader and discuss the conditions under which
there exists a solution to the functional equation.
18. The notation dates back at least to Bertsekas (1976).
28 Chapter 2
TWs max ss; s 0 bWs 0 b ss; fQ s bWfQ s

s 0 A Gs
b ss; fQ s bQfQ s 1 TQs for all s A S:
Discounting means that adding a constant to W leads TW to

increase by less than this constant. That is, for any constant k,
TW ks a TWs bk for all s A S where b A 0; 1. The term
discounting reects the fact that b must be less than 1. This prop-
erty is easy to verify in the dynamic programming problem:
TW k max ss; s 0 bWs 0 k TW bk for all s A S;
s 0 A Gs
since we assume that the discount factor is less than 1.

The fact that TW is a contraction allows us to take advantage
of the contraction mapping theorem.19 This theorem implies that (1)
there is a unique xed point and (2) this xed point can be reached
by an iteration process using an arbitrary initial condition. The rst
property is reected in the theorem given above.
The second property is used extensively as a means of nding
the solution to (2.12). To better understand this, let V0 s for all s A S
be an initial guess of the solution to (2.12). Consider V1 TV0 . If
V1 V0 for all s A S, then we have the solution. Or else, consider
V2 TV1 , and continue iterating until TV V so that the func-
tional equation is satised. Of course, in general, there is no reason
to think that this iterative process will converge. However, if TV
is a contraction, as it is for our dynamic programming framework,
then the Vs that satises (2.12) can be found from the iteration of
TV0 s for any initial guess, V0 s. This procedure is called value
function iteration, and it will be a valuable tool for applied analysis
of dynamic programming problems.
The value function that satises (2.12) may inherit some properties
from the more primitive functions that are the inputs to the dynamic
programming problem: the payoff and transition equations. As we
will see, the property of strict concavity is useful for various appli-
cations.20 The result is given formally by
theorem 2 Assume that ss; s 0 is real-valued, continuous, concave,
and bounded, 0 < b < 1, that S is a convex subset of < k , and that
the constraint set is nonempty, compact-valued, convex, and contin-
19. See Stokey and Lucas (1989) for a statement and proof of this theorem.
20. Dene ss; s 0 as concave if sls1 ; s10 1 ls2 ; s20 b lss1 ; s10 1 lss2 ; s20
for all 0 < l < 1 where the inequality is strict if s1 0 s2 .
uous. Then the unique solution to (2.12) is strictly concave. Further

fs is a continuous, single-valued function.
Proof See theorem 4.8 in Stokey and Lucas (1989).
The proof of the theorem relies on showing that strict concavity

is preserved by TV: namely, if Vs is strictly concave, then so is
TVs. Given that ss; c is concave, then we can let our initial guess
of the value function be the solution to the one-period problem:
V0 s 1 max ss; s 0 :
s 0 A Gs
V0 s is strictly concave. Since TV preserves this property, the

solution to (2.12) will be strictly concave.
As noted earlier, our interest is in the policy function. Note that by
this theorem, there is a stationary policy function that depends only
on the state vector. This result is important for econometric applica-
tion as stationarity is often assumed in characterizing the properties
of various estimators.
The cake-eating example relied on the Euler equation to determine
some properties of the optimal solution. However, the rst-order
condition from (2.12) combined with the strict concavity of the value
function is useful in determining properties of the policy function.
Beneveniste and Scheinkman (1979) provide conditions such that
Vs is differentiable (Stokey and Lucas 1989, thm. 4.11). In our dis-
cussion of applications, we will see arguments that use the concavity
of the value function to characterize the policy function.
2.5.2 Stochastic Dynamic Programming
While the nonstochastic problem may be a natural starting point,

in actual applications it is necessary to consider stochastic ele-
ments. The stochastic growth model, consumption/savings decisions
by households, factor demand by rms, pricing decisions by sellers,
search decisions, all involve the specication of dynamic stochastic
environments.
Also empirical applications rest upon shocks that are not observed
by the econometrician. In many applications the researcher appends
a shock to an equation prior to estimation without being explicit
about the source of the error term. This is not consistent with the
approach of stochastic dynamic programming: shocks are part of
the state vector of the agent. Of course, the researcher may not
30 Chapter 2
observe all of the variables that inuence the agent, and/or there
may be measurement error. Nonetheless, being explicit about the
source of error in empirical applications is part of the strength of this
approach.
While stochastic elements can be added in many ways to dynamic
programming problems, we consider the following formulation in
our applications. Let e represent the current value of a vector of
shocks, namely random variables that are partially determined by
nature. Let e A C be a nite set.21 Then, by the notation developed
above, the functional equation becomes
Vs; e max ss; s 0 ; e bEe 0 j e Vs 0 ; e 0 for all s; e: 2:13

s 0 A Gs; e
Further we have assumed that the stochastic process itself is

purely exogenous as the distribution of e 0 depends on e but is inde-
pendent of the current state and control. Note too that the distribu-
tion of e 0 depends on only the realized value of e: that is, e follows a
rst-order Markov process. This is not restrictive in the sense that if
values of shocks from previous periods were relevant for the distri-
bution of e 0 , then they could simply be added to the state vector.
Finally, note that the distribution of e 0 conditional on e, written as
0
e je, is time invariant. This is analogous to the stationarity properties
of the payoff and transition equations. In this case the conditional
probability of e 0 je are characterized by a transition matrix, P. The
element pij of this matrix is dened as
pij 1 Probe 0 ej je ei ;
which is just the likelihood that ej occurs in the next period, given
that ei occurs today. Thus this transition matrix is used to compute
the transition probabilities in (2.13). Throughout we assume that
P
pij A 0; 1 and j pij 1 for each i. By this structure, we have
theorem 3 If ss; s 0 ; e is real-valued, continuous, concave, and
bounded, 0 < b < 1, and the constraint set is compact and convex,
then
1. there exists a unique value function Vs; e that solves (2.13),

2. there exists a stationary policy function, fs; e.
21. As noted earlier, this structure is stronger than necessary but accords with the
approach we will take in our empirical implementation. The results reported in Bert-
sekas (1976) require that C is countable.
Proof As in the proof of theorem 2, this is a direct application of

Blackwells theorem. That is, with b < 1, discounting holds. Likewise
monotonicity is immediate as in the discussion above. (See also the
proof of proposition 2 in Bertsekas 1976, ch. 6.)
The rst-order condition for (2.13) is given by
ss 0 s; s 0 ; e bEe 0 j e Vs 0 s 0 ; e 0 0: 2:14
Using (2.13) to determine Vs 0 s 0 ; e 0 yields the Euler equation

ss 0 s; s 0 ; e bEe 0 j e ss 0 s 0 ; s 00 ; e 0 0: 2:15
This Euler equation has the usual interpretation: the expected sum of
the effects of a marginal variation in the control in the current period
s must be zero. So, if there is a marginal gain in the current period,
this gain, in the expectation, is offset by a marginal loss in the next
period. Put differently, if a policy is optimal, there should be no
variation in the value of the current control that will, in the expecta-
tion, make the agent better off. Of course, ex post (after the realiza-
tion of e 0 ), there could have been better decisions for the agent and,
from the vantage point of hindsight, evidence that mistakes were
made. That is to say,
ss 0 s; s 0 ; e bss 0 s 0 ; s 00 ; e 0 0 2:16
will surely not hold for all realizations of e 0 . Yet, from the ex ante
optimization, we know that these ex post errors were not predicable
given the information available to the agent. As we will see, this
powerful insight underlies the estimation of models based on a sto-
chastic Euler equation such as (2.15).
2.6 Conclusion
This chapter has introduced some of the insights in the vast litera-
ture on dynamic programming and some of the results that will be
useful in our applications. This chapter has provided a theoretical
structure for the dynamic optimization problems we will confront
throughout this book. Of course, other versions of the results hold in
more general circumstances. The reader is urged to study Bertsekas
(1976), Sargent (1987), and Stokey and Lucas (1989) for a more com-
plete treatment of this topic.
3 Numerical Analysis
3.1 Overview
This chapter reviews numerical methods used to solve dynamic

programming problems. This discussion provides a key link between
the basic theory of dynamic programming and the empirical analysis
of dynamic optimization problems. The need for numerical tools
arises from the fact that dynamic programming problems generally
do not have tractable closed form solutions. Hence techniques must
be used to approximate the solutions of these problems. We present
a variety of techniques in this chapter that are subsequently used in
the macroeconomic applications studied in part II of this book.
The presentation starts by solving a stochastic cake-eating prob-
lem using a procedure called value function iteration. The same
example is then used to illustrate alternative methods that operate
on the policy function rather than the value function. Finally, a ver-
sion of this problem is studied to illustrate the solution to dynamic
discrete choice problems.
The appendix and the Web page for this book contain the pro-
grams used in this chapter. The applied researcher may nd these
useful templates for solving other problems. In the appendix we
present several tools, such as numerical integration and interpolation
techniques, that are useful numerical methods.
A number of articles and books have been devoted to numerical
programing. For a more complete description, we refer the reader to
Judd (1998), Amman et al. (1996), Press et al. (1986), and Taylor and
Uhlig (1990).
34 Chapter 3
3.2 Stochastic Cake-Eating Problem
We start with the stochastic cake-eating problem dened by
VW; y max uc bEy 0 j y VW 0 ; y 0 for all W; y 3:1

0acaWy
with W 0 RW c y. Here there are two state variables: W, the

size of the cake brought into the current period, and y, the stochastic
endowment of additional cake. This is an example of a stochastic
dynamic programming problem from the framework in (2.5.2).
We begin by analyzing the simple case where the endowment
is iid: the shock today does not give any information on the shock
tomorrow. In this case the consumer only cares about the total
amount that can be potentially eaten, X W y, and not the partic-
ular origin of any piece of cake. In this problem there is only one
state variable X. We can rewrite the problem as
VX max uc bEy 0 VX 0 for all X 3:2
0acaX
with X 0 RX c y 0 .
If the endowment is serially correlated, then the agent has to keep
track of any variables that allow him to forecast future endowment.
The state space will include X but also current and maybe past real-
izations of endowments. We present such a case in section 3.3 where
we study a discrete cake eating problem. Chapter 6.1 also presents
the continuous cake-eating problem with serially correlated shocks.
The control variable is c, the level of current consumption. The size
of the cake evolves from one period to the next according to the
transition equation. The goal is to evaluate the value VX as well as
the policy function for consumption, cX.
3.2.1 Value Function Iterations
This method works from the Bellman equation to compute the value
function by backward iterations on an initial guess. While sometimes
slower than competing methods, it is trustworthy in that it reects
the result, stated in chapter 2, that (under certain conditions) the
solution of the Bellman equation can be reached by iterating the
value function starting from an arbitrary initial value. We illustrate
this approach here in solving (3.2).1
1. We present additional code for this approach in the context of the nonstochastic
growth model presented in chapter 5.
Numerical Analysis 35
In order to program value function iteration, there are several

important steps:
1. Choosing a functional form for the utility function.
2. Discretizing the state and control variable.
3. Building a computer code to perform value function iteration
4. Evaluating the value and the policy function.
We discuss each steps in turn. These steps are indicated in the code
for the stochastic cake-eating problem.
Functional Form and Parameterization

We need to specify the utility function. This is the only known
primitive function in (3.2): recall that the value function is what we
are solving for! The choice of this function depends on the problem
and the data. The consumption literature has often worked with a
constant relative risk aversion (CRRA) function:
c 1g
uc :
1g
The vector y will represent the parameters. For the cake-eating

problem g; b are both included in y. To solve for the value function,
we need to assign particular values to these parameters as well as
the exogenous return R. For now we assume that bR 1 so that the
growth in the cake is exactly offset by the consumers discounting of
the future. The specication of the functional form and its parame-
terization are given in part I of the accompanying Matlab code for
the cake-eating problem.
State and Control Space

We have to dene the space spanned by the state and the control
variables as well as the space for the endowment shocks. For each
problem, specication of the state space is important. The com-
puter cannot literally handle a continuous state space, so we have
to approximate this continuous space by a discrete one. While the
approximation is clearly better if the state space is very ne (i.e., has
many points), this can be costly in terms of computation time. Thus
there is a trade-off involved.
For the cake-eating problem, suppose that the cake endowment
can take two values, low yL and high yH . As the endowment is
36 Chapter 3
assumed to follow an iid process, denote the probability a shock yi

by pi for i L; H. The probability of transitions can be stacked in a
transition matrix:

pL pH
p with pL pH 1:
pL pH
In this discrete setting, the expectation in (3.2) is just a weighted
sum, so the Bellman equation can be simply rewritten as
X
VX max uc b pi VRX c yi for all X:
0acaX
iL; H
For this problem it turns out that the natural state space is given
by XL ; XH . This choice of the state space is based on the economics
of the problem, which will be understood more completely after
studying household consumption choices. Imagine, though, that
endowment is constant at a level yi for i L; H. Then, given the
assumption bR 1, the cake level of the household will (trust us)
eventually settle down to Xi for i L; H. Since the endowment is
stochastic and not constant, consumption and the size of the future
cake will vary with realizations of the state variable X, but it turns
out that X will never leave this interval.
The neness of the grid is simply a matter of choice too. In the
program let ns be the number of elements in the state space. The
program simply partitions the interval XL ; XH into ns elements. In
practice, the grid is usually uniform, with the distance between two
consecutive elements being constant.2
Call the state space CS , and let is be an index:
CS fX is gins 1
s
with X 1 XL ; X ns XH :
The control variable c takes values in XL ; XH . These are the extreme
levels of consumption given the state space for X. We discretize this
space into a grid of size nc , and call the control space CC fc ic ginc 1
c
.
Value Function Iteration and Policy Function

Here we must have a loop for the mapping TvX, dened as
X
TvX max uc b pi vj RX c yi : 3:3
c
iL; H
2. In some applications it can be useful to dene a grid that is not uniformally spaced;
see the discrete cake-eating problem in section 3.3.
In this expression vX represents a candidate value function that is a

proposed solution to (3.2). If TvX vX, then indeed vX is the
unique solution to (3.2). Thus the solution to the dynamic program-
ming problem is reduced to nding a xed point of the mapping
TvX.
Starting with an initial guess v0 X, we compute a sequence of
value functions vj X:
X
vj1 X Tvj X max uc b pi vj RX c yi :
c
iL; H
The iterations are stopped when jvj1 X vj Xj < e for all is , where
e is a small number. As T: is a contraction mapping (see chapter 2),
the initial guess v0 X does not inuence the convergence to the xed
point, so one can choose v0 X 0, for instance. However, nding
a good guess for v0 X helps to decrease the computing time. By the
contraction mapping property, we know that the convergence rate is
geometric, parameterized by the discount rate b.
Let us review in more detail how the iteration is done in practice.
At each iteration, the values vj X are stored in a ns 1 matrix:
2 3
vj X 1
6 7
6 .. 7
6 . 7
6 7
6 v X is 7
V6 j 7:
6 7
6 .. 7
6 . 7
4 5
ns
vj X
To compute vj1 , we start by choosing a particular size for the cake

at the start of the period, X is . We then search among all the points
in the control space CC for the point where uc bEvj X 0 is maxi-

mized. We will denote this point c ic . Finding next periods value

involves calculating vj RX is c ic yi , i L; H. With the assump-
tion of a nite state space, we look for the value vj : at the point

nearest to RX is c ic yi . Once we have calculated the new value
for vj1 X is , we can proceed to compute similarly the value vj1 :
for other sizes of the cake and other endowment at the start of the
period. These new values are then stacked in V. Figure 3.1 gives an
example of how this can be programmed on a computer. (Note that
the code is not written in a particular computer language, so one has
to adapt the code to the appropriate syntax. The code for the value
function iteration piece is part III of the Matlab code.)
38 Chapter 3
i_s=1
do until i_s>n_s * Loop over all sizes of the
total amount of cake X *
c_L=X_L * Min value for consumption *
c_H=X[i_s] * Max value for consumption *
i_c=1
do until i_c>n_c * Loop over all consumption
levels *
c=c_L+(c_H-c_L)/n_c*(i_c-1)
i_y=1
EnextV=0 * Initialize the next value
to zero *
do until i_y>n_y * Loop over all possible
realizations of the future
endowment *
nextX=R*(X[i_s]-c)+Y[i_y] * Next period amount of
cake *
nextV=V(nextX) * Here we use interpolation
to find the next value
function *
EnextV=EnextV+nextV*Pi[i_y] * Store the expected future

value using the transition
matrix *
i_y=i_y+1
endo * End of loop over
endowment *
aux[i_c]=u(c)+beta*EnextV * Stores the value of a given
consumption level *
i_c=i_c+1
endo * End of loop over
consumption *
newV[i_s,i_y]=max(aux) * Take the max over all
consumption levels *
i_s=i_s+1
endo * End of loop over size of
cake *
V=newV * Update the new value
function *
Figure 3.1
Stochastic cake-eating problem
Figure 3.2
Value function, stochastic cake-eating problem
Once the value function iteration piece of the program is com-

pleted, the value function can be used to nd the policy function,
c cX. This is done by collecting all the optimal consumption
values c ic for every value of X is . Here again, we only know the
function cX at the points of the grid. We can use interpolating
methods to evaluate the policy function at other points. The value
function and the policy function are displayed in gures 3.2 and 3.3
for particular values of the parameters.
As discussed above, approximating the value function and the
policy rules by a nite state space requires a large number of points
on this space (ns has to be big). These numerical calculations are
often extremely time-consuming. So we can reduce the number of
points on the grid, while keeping a satisfactory accuracy, by using
interpolations on this grid. When we have evaluated the function

vj RX is c ic yi , i L; H, we use the nearest value on the grid to

approximate RX is c ic yi . With a small number of points on the
grid, this can be a very crude approximation. The accuracy of the
computation can be increased by interpolating the function vj : (see
the appendix for more details). The interpolation is based on the
values in V.
40 Chapter 3
Figure 3.3
Policy function, stochastic cake-eating problem
3.2.2 Policy Function Iterations
The value function iteration method can be rather slow, as it con-

verges at a rate b. Researchers have devised other methods that can
be faster to compute the solution to the Bellman equation in an in-
nite horizon. The policy function iteration, also known as Howards
improvement algorithm, is one of these. We refer the reader to Judd
(1998) or Ljungqvist and Sargent (2000) for more details.
This method starts with a guess of the policy function, in our case
c0 X. This policy function is then used to evaluate the value of using
this rule forever:
X
V0 X uc0 X b pi V0 RX c0 X yi for all X:
iL; H
This policy evaluation step requires solving a system of linear

equations, given that we have approximated RX c 0 X yi by an
X on our grid. Next we do a policy improvement step to compute
c1 X:
" #
X
c1 X arg max uc b pi V0 RX c yi for all X:
c
iL; H
Given this new rule, the iterations are continued to nd V1 ;

c2 ; . . . ; cj1 until jcj1 X cj Xj is small enough. The conver-
gence rate is much faster than the value function iteration method.
However, solving the policy evaluation step can sometimes be
quite time-consuming, especially when the state space is large. Once
again, the computation time can be much reduced if the initial guess
c0 X is close to the true policy rule cX.
3.2.3 Projection Methods
These methods compute directly the policy function without calcu-

lating the value functions. They use the rst-order conditions (Euler
equation) to back out the policy rules. The continuous cake problem
satises the rst-order Euler equation
u 0 ct bREt u 0 ct1
if the desired consumption level is less than the total resources

X W y. If there is a corner solution, then the optimal consump-
tion level is cX X. Taking into account the corner solution, we
can rewrite the Euler equation as
u 0 ct maxu 0 Xt ; bREt u 0 ct1 :

We know that by the iid assumption, the problem has only
one state variable X, so the consumption function can be written
c cX. As we consider the stationary solution, we drop the sub-
script t in the next equation. The Euler equation can then be refor-
mulated as
u 0 cX maxu 0 X; bREy 0 u 0 cRX cX y 0 0 3:4
or
FcX 0: 3:5
The goal is to nd an approximation c^X of cX, for which (3.5) is

approximately satised. The problem is thus reduced to nd the zero
of F, where F is an operator over function spaces. This can be done
with a minimizing algorithm. There are two issues to resolve. First,
42 Chapter 3
we need to nd a good approximation of cX. Second, we have to

dene a metric to evaluate the t of the approximation.
Solving for the Policy Rule

Let fpi Xg be a base of the space of continuous functions, and let
C fci g be a set of parameters. We can approximate cX by
X
n
c^X; C ci pi X:
i1
There is an innite number of bases to chose from. A simple one is to

consider polynomials in X so that c^X; C c0 c1 X c2 X 2 :
Although this choice is intuitive, it is not usually the best choice. In
the function space this base is not an orthogonal base, which means
that some elements tend to be collinear.
Orthogonal bases will yield more efcient and precise results.3 The
chosen base should be computationally simple. Its elements should
look like the function to approximate, so that the function cX
can be approximated with a small number of base functions. Any
knowledge of the shape of the policy function will be to a great help.
If, for instance, this policy function has a kink, a method based only
on a series of polynomials will have a hard time tting it. It would
require a large number of powers of the state variable to come
somewhere close to the solution.
Having chosen a method to approximate the policy rule, we now
have to be more precise about what bringing Fc^X; C close to
zero means. To be more specic, we need to dene some operators
on the space of continuous functions. For any weighting function
gx, the inner product of two integrable functions f1 and f2 on a
space A is dened as

h f1 ; f2 i f1 x f2 xgx dx: 3:6
A
Two functions f1 and f2 are said to be orthogonal, conditional on a

weighting function gx, if h f1 ; f2 i 0. The weighting function indi-
cates where the researcher wants the approximation to be good. We
are using the operator h: ; :i and the weighting function to construct
a metric to evaluate how close F^ cX; C is to zero. This will be done
3. Popular orthogonal bases are Chebyshev, Legendre, or Hermite polynomials.

by solving for C such that
hFc^X; C; f Xi 0;
where f X is some known function. We next review three methods
that differ in their choice for this function f X.
First, a simple choice for f X is F^cX; C itself. This denes the
least square metric as
min hFc^X; C; Fc^X; Ci:

C
By the collocation method, detailed later in this section, we can

choose to nd C as
min hFc^X; C; dX Xi i; i 1; . . . ; n;
C
where dX Xi is the mass point function at point Xi , meaning that

dX 1 if X Xi and dX 0 elsewhere. Another possibility is to
dene
cX; C; pi Xi;
min hF^ i 1; . . . ; n;
C
where pi X is a base of the function space. This is called the Galer-

kin method. An application of this method can be seen below, where
the base is taken to be tent functions.
Figure 3.4 displays a segment of the computer code that calcu-
lates the residual function F^ cX; C when the consumption rule is
approximated by a second-order polynomial. This can then be used
in one of the proposed methods.
Collocation Methods
Judd (1992) presents in some detail this method applied to the
growth model. The function cX is approximated using Chebyshev
polynomials. These polynomials are dened on the interval 0; 1 and
take the form
pi X cosi arccosX; X A 0; 1; i 0; 1; 2; . . . :
For i 0, this polynomial is a constant. For i 1, the polynomial

is equal to X. As these polynomials are only dened on the 0; 1
interval, one can usually scale the state variables appropriately.4 The
4. The polynomials are also dened recursively by pi X 2Xpi1 X pi2 X, i b 2,

with p0 0 1 and pX; 1 X.
44 Chapter 3
procedure c(x) * Here we define an

cc=psi_0+psi_1*x+psi_2*x*x approximation for the
return(cc) consumption function based
endprocedure on a second-order
polynomial *
i_s=1
total amount of cake *
utoday=U 0 (c(X[i_s])) * Marginal utility of
consuming *
ucorner=U 0 (X[i_s]) * Marginal utility if corner
solution *
EnextU=0 * Initialize expected future
i_y=1 marginal utility *
do until i_y>n_y * Loop over all possible
realizations of the future
endowment *
nextX=R(X[i_s]- * Next amount of cake *
c(X[i_s]))+Y[i_y]
nextU=U 0 (c(nextX)) * Next marginal utility of
consumption *
EnextU=EnextU+nextU*Pi[i_y] * Here we compute the expected
future marginal utility of
consumption using the
transition matrix Pi *
i_y=i_y+1
endo * End of loop over endowment *
F[i_s]=utoday-
max(ucorner,beta*EnextU)
i_s=i_s+1
endo * End of loop over size of
cake *
Figure 3.4
Stochastic cake-eating problem, projection method
policy function can then be expressed as
X
n
c^X; C ci pi X:
i1
Next the method nds C, which minimizes

cX; C; dX Xi i;
hF^ i 1; . . . ; n;
where d is the mass point function. Hence the method requires

cX; C is zero at some particular points Xi and not over the
that F^
whole range XL ; XH . The method is more efcient if these points are
chosen to be the zeros of the base elements pi X, here Xi cosp=2i.
This method is referred to as an orthogonal collocation method. C is
Figure 3.5
Basis functions, nite element method
the solution to a system of nonlinear equations:

cXi ; C 0;
F^ i 1; . . . ; n:
This method is good at approximating policy functions that are rel-

atively smooth. A drawback is that the Chebyshev polynomials tend
to display oscillations at higher orders. The resulting policy function
cX will also tend to uctuate. There is no particular rule for choos-
ing n, the highest order of the Chebyshev polynomial. Obviously
the higher n is, the better will be the approximation, but this comes
at the cost of increased computation.
Finite Element Methods

McGrattan (1996) illustrates the nite element method with the sto-
chastic growth model (see also Reddy 1993 for an in-depth discus-
sion of nite elements).
To start, the state variable X is discretized over a grid fX is gins 1
s
.
The nite element method is based on the following functions:
8
>
> X X is 1
> i
> if X A X is 1 ; X is ;
>
> X s X is 1
<
pis X X is 1 X
>
>
> is 1 X is
if X A X is ; X is 1 ;
>
> X
>
:
0 elsewhere:
46 Chapter 3
The function pis X is a simple function in 0; 1, as illustrated in

gure 3.5. It is in fact a simple linear interpolation (and an order
two spline; see the appendix for more on these techniques). On the
interval X is ; X is 1 , the function c^X is equal to the weighted sum
of pis X and pis 1 X. Here the residual function satises
cX; C; pi Xi 0;
hF^ i 1; . . . ; n:
Equivalently, we could choose a constant weighting function:

X
pis XF^
cX dX 0; is 1; . . . ; ns :
0
This gives a system with ns equations and ns unknowns, fcis gins 1 s

.
This nonlinear system can be solved to nd the weights fcis g. To
solve the system, the integral can be computed numerically using
numerical techniques; see the appendix. As in the collocation
method, the choice of ns is the result of a trade-off between increased
precision and a higher computational burden.
3.3 Stochastic Discrete Cake-Eating Problem
We present here another example of a dynamic programming model.

It differs from the one presented in section 3.2 in two ways. First, the
decision of the agent is not continuous (how much to eat) but dis-
crete (eat or wait). Second, the problem has two state variables as the
exogenous shock is serially correlated.
The agent is endowed with a cake of size W. In each period the
agent has to decide whether or not to eat the entire cake. Even if
not eaten, the cake shrinks by a factor r each period. The agent also
experiences taste shocks, possibly serially correlated, and which fol-
low an autoregressive process of order one. The agent observes the
current taste shock at the beginning of the period, before the decision
to eat the cake is taken. However, the future shocks are unobserved
by the agent, introducing a stochastic element into the problem.
Although the cake is shrinking, the agent might decide to postpone
the consumption decision until a period with a better realization of
the taste shock. The program of the agent can be written in the form
VW; e maxeuW; bEe 0 j e VrW; e 0 ; 3:7
where VW; e is the intertemporal value of a cake of size W condi-

tional of the realization e of the taste shock. Here Ee 0 denotes the
expectation with respect to the future shock e, conditional on the

value of e. The policy function is a function dW; e that takes a value
of zero if the agent decides to wait or one if the cake is eaten. We can
also dene a threshold e W such that

dW; e 1 if e > e W;
dW; e 0 otherwise.
As in section 3.2 the problem can be solved by value function
iterations. However, the problem is discrete, so we cannot use the
projection technique as the decision rule is not a smooth function but
a step function.
3.3.1 Value Function Iterations
As before, we have to dene, rst, the functional form for the utility
function, and we need to discretize the state space. We will consider
r < 1, so the cake shrinks with time and W is naturally bounded
between W, the initial size and 0. In this case the size of the cake
takes only values equal to r t W, t b 0. Hence CS fr i Wg is a judi-
cious choice for the state space. Contrary to an equally spaced grid,
this choice ensures that we do not need to interpolate the value
function outside of the grid points.
Next, we need to discretize the second state variable, e. The shock
is supposed to come from a continuous distribution, and it follows
an autoregressive process of order one. We discretize e in I points
I
fei gi1 following a technique presented by Tauchen (1986) and sum-
marized in the appendix. In fact we approximate an autoregressive
process by a Markov chain. The method determines the optimal dis-
I
crete points fei gi1 and the transition matrix pij Probet ei jet1
ej such that the Markov chain mimics the AR(1) process. Of course,
the approximation is only good if I is big enough.
In the case where I 2, we have to determine two grid points eL
and eH . The probability that a shock eL is followed by a shock eH is
denoted by pLH . The probability of transitions can be stacked in a
transition matrix:

pLL pLH
p
pHL pHH
with the constraints that the probability of reaching either a low or a

high state next period is equal to one: pLL pLH 1 and pHL pHH
48 Chapter 3
i_s=2
cake *
i_e=1
do until i_e>2 * Loop over all possible
realizations of the taste shock
*
ueat=u(W[i_s],e[i_e]) * Utility of doing the eating now
*
nextV1=V[i_s-1,1] * Next period value if low taste
shock *
nextV2=V[i_s-1,2] * Next period value if high taste
shock *
EnextV=nextV1*p[i_e,1]+
nextV2*p[i_e,2]
newV[i_s,i_e]
=max(ueat,beta*EnextV)
* Take the max between eating now
or waiting *
i_e=i_e+1
endo * End of loop over taste shock *
i_s=i_s+1
endo * End of loop over size of cake *
V=newV * Update the new value function *
Figure 3.6
Discrete cake-eating problem
1. For a given size of the cake W is r is W and a given shock ej , j L

or H, it is easy to compute the rst term ej ur is W. To compute the
second term we need to calculate the expected value of tomorrows
cake. Given a guess for the value function of next period, v: ; :, the
expected value is
Ee 0 j ej vr is 1 W pjL vr is 1 W; eL pjH vr is 1 W; eH :
The recursion is started backward with an initial guess for V: ; :.

For a given state of the cake Wis and a given shock ej , the new value
function is calculated from equation (3.7). The iterations are stopped
when two successive value functions are close enough. In numerical
computing the value function is stored as a matrix V of size nW ne ,
where nW and ne are the number of points on the grid for W and e.
At each iteration the matrix is updated with the new guess for the
value function. Figure 3.6 gives an example of a computer code that
obtains the value function vj1 W; e given the value vj W; e.
The way we have computed the grid, the next period value is
simple to compute as it is given by Vis 1; :. This rule is valid if
Figure 3.7
Value function, discrete cake-eating problem
is > 1. Computing V1; : is more of a problem. One way is to use an

extrapolation method to approximate the values, given the knowl-
edge of Vis ; :, is > 1.
Figure 3.7 shows the value function for particular parameters. The
utility function is taken to be uc; e lnec, and lne is supposed to
follow an AR(1) process with mean zero, autocorrelation re 0:5 and
with an unconditional variance of 0.2. We have discretized e into
four grid points.
Figure 3.8 shows the decision rule, and the function e W. This
threshold was computed as the solution of:
uW; e W bEe 0 j e VrW; e 0 ;

which is the value of the taste shock that makes the agent indifferent
between waiting and eating, given the size of the cake W.
We return later in this book to examples of discrete choice models.
In particular, we refer the readers to the models presented in sections
8.5 and 7.3.3.
50 Chapter 3
Figure 3.8
Decision rule, discrete cake-eating problem
3.4 Extensions and Conclusion
In this chapter we reviewed the common techniques used to solve

the dynamic programming problems of chapter 2. We applied these
techniques to both deterministic and stochastic problems, to contin-
uous and discrete choice models. These methods can be applied as
well to more complicated problems.
3.4.1 Larger State Spaces
The two examples we have studied in sections 3.2 and 3.3 have small
state spaces. In empirical applications the state space often needs to
be much larger if the model is to confront real data. For instance, the
endowment shocks might be serially correlated or the interest rate R
might also be a stochastic and persistent process.
For the value function iteration method, this means that the suc-
cessive value functions have to be stacked in a multidimensional
matrix. Also the value function has to be interpolated in several
dimensions. The techniques in the appendix can be extended to deal
with this problem. However, the value function iteration method

quickly encounters the curse of dimensionality. If every state vari-
able is discretized into ns grid points, the value function has to be
evaluated by N ns points, where N is the number of state variables.
This demands an increasing amount of computer memory and so
slows down the computation. A solution to this problem is to eval-
uate the value function for a subset of the points in the state space
and then to interpolate the value function elsewhere. This solution
was implemented by Keane and Wolpin (1994).
Projection methods are better at handling larger state spaces.
Suppose that the problem is characterized by N state variables
fX1 ; . . . ; XN g. The approximated policy function can be written as
nj
N X
X j
c^X1 ; . . . ; XN cij pij Xj :
j1 ij 1
j
The problem is then characterized by auxiliary parameters fci g.
exercise 3.1 Suppose that uc c 1g =1 g. Construct the code to

solve for the stochastic cake-eating problem using the value function
iteration method. Plot the policy function as a function of the size of
the cake and the stochastic endowment for g f0:5; 1; 2g. Compare
the level and slope of the policy functions for different values of g.
How do you interpret the results?
exercise 3.2 Consider, again, the discrete cake-eating problem of

section 3.3. Construct the code to solve for this problem, with iid
taste shocks, using uc lnc, eL 0:8, eH 1:2, pL 0:3, and
pH 0:7. Map the decision rule as a function of the size of the cake.
exercise 3.3 Consider an extension of the discrete cake-eating
problem of section 3.3. The agent can now choose among three
actions: eat the cake, store it in fridge 1 or in fridge 2. In fridge 1, the
cake shrinks by a factor r: W 0 rW. In fridge 2, the cake diminish by
a xed amount: W 0 W k. The program of the agent is charac-
terized as
VW; e maxV Eat W; e; V Fridge 1 W; e; V Fridge 2 W; e
8
Eat
>
<V W; e euW;
with V Fridge 1 W; e bEe 0 VrW; e 0 ;
>
: Fridge 2
V W; e bEe 0 VW k; e 0 :
52 Chapter 3
Construct the code to solve for this problem, using uc lnc,

eL 0:8, eH 1:2, pL 0:5, and pH 0:5. When will the agent switch
from one fridge to the other?
exercise 3.4 Consider the stochastic cake-eating problem. Suppose

that the discount rate b is a function of the amount of cake con-
sumed: b Fb1 b2 c, where b 1 and b 2 are known parameters and
F is the normal cumulative distribution function. Construct the
code to solve for this new problem using value function iterations.
Suppose g 2, b1 1:65, pL pH 0:5, yL 0:8, yH 1:2, and
b2 1. Plot the policy rule c cX. Compare the result with that
of the case where the discount rate is independent of the quantity
consumed. How would you interpret the fact that the discount rate
depends on the amount of cake consumed?
3.5 Appendix: Additional Numerical Tools
In this appendix we provide some useful numerical tools that are

often used in solving dynamic problems. We present interpolation
methods, numerical integration methods, as well as a method to
approximate serially correlated processes by a Markov process. The
last subsection is devoted to simulations.
3.5.1 Interpolation Methods
We briey review three simple interpolation methods. For further

readings, see, for instance, Press et al. (1986) or Judd (1996).
When solving the value function or the policy function, we often
have to calculate the value of these functions outside of the points
of the grid. This requires one to be able to interpolate the function.
Using a good interpolation method can also save computer time and
space since fewer grid points are needed to approximate the func-
tions. Let us denote f x the function to approximate. We assume
that we know this function at a number of grid points x i , i 1; . . . ; I.
We denote by fi f x i the values of the function at these grid
points. We are interested in nding an approximate function f^x
such that f^x F f x, based on the observations fx i ; fi g. We present
three different methods and use as an example the function f x
x sinx. Figure 3.9 displays the results for all the methods.
Figure 3.9
Approximation methods
Least Squares Interpolation

A natural way to approximate f is to use an econometric tech-
nique, such as OLS, to estimate the function f^:. The rst step is to
assume a functional form for f^. For instance, we can approximate f
with a polynomial in x such as
f^x a0 a1 x aN x N ; N < I:
By regressing fi on x i , we can easily recover the parameters an . In

practice, this method is often not very good, unless the function f
is well behaved. Higher-order polynomials tend to uctuate and
can occasionally give an extremely poor t. This is particularly true
when the function is extrapolated outside of the grid points, when
x > xI or x < x1 . The least square method is a global approximation
method. As such, the t can be on average satisfactory but mediocre
almost everywhere. This can be seen in the example in gure 3.9.
Linear Interpolation
This method ts the function f with piecewise linear functions on
the intervals xi1 ; x i . For any value of x in xi1 ; x i , an approxima-
54 Chapter 3
tion f^x of f x can be found as
fi fi1
f^x fi1 x xi1 :
x i xi1
A ner grid will give a better approximation of f x. When x is

greater than xI , using this rule can lead to numerical problems as
the expression above may not be accurate. Note that the approxima-
tion function f^ is continuous but not differentiable at the grid points.
This can be an undesirable feature as this nondifferentiability can be
translated to the value function or the policy function.
The linear interpolation method can be extended for multivariate
functions. For instance, we can approximate the function f x; y
given data on fx i ; yj ; fij g. Denote dx x x i =xi1 x i and dy
y yi = yi1 yi . The approximation can be written as
f^x; y dx dy fi1; j1 1 dx dy fi; j1 dx 1 dy fi1; j

1 dx 1 dy fi; j :
The formula can be extended to higher dimensions as well.
Spline Methods
This method extends the linear interpolation by tting piecewise
polynomials while ensuring that the resulting approximate func-
tion f^ is both continuous and differentiable at the grid points x i . We
restrict ourself to cubic splines for simplicity, but the literature on
splines is very large (e.g., see De Boor 1978). The approximate func-
tion is expressed as
fî x fi ai x xi1 bi x xi1 2 ci x xi1 3 ; x A xi1 ; x i :

Here for each point on the grid, we have to determine three param-
eters fai ; bi ; ci g, so in total there is 3 I parameters to compute.
However, imposing the continuity of the function and of its deriva-
tive up to the second order reduces the number of coefcients:
fî x fî1 x;
fî0 x fî1
0
x;
fî00 x fî1
00
x:
It is also common practice to apply f^100 x1 fÎ00 xI 0. With these
constraints, the number of coefcients to compute is down to I. Some
algebra gives
8
>
> fi fi1
> ai
> bi x i xi1 ci x i xi1 2 ; i 1; . . . ; I;
>
> x i x i1
>
>
>
>
>
> b bi
>
<ci i1 ; i 1; . . . ; I 1;
3x i xi1
>
>
>
> bI
>
> cI ;
>
>
>
> 3xI xI1
>
>
>
:
ai 2bi x i xi1 3ci x i xi1 2 ai1 :
Solving this system of equation leads to expressions for the coef-

cients fai ; bi ; ci g. Figure 3.9 shows that the cubic spline is a very
good approximation to the function f .
3.5.2 Numerical Integration
Numerical integration is often required in dynamic programming

problems to solve for the expected value function or to integrate
out an unobserved state variable. For instance, solving the Bell-

man equation (3.3) requires one to calculate EvX 0 vX 0 dFX 0 ,
where F: is the cumulative density of the next period cash-on-hand
X. In econometric applications some important state variables might
not be observed. For this reason one may need to compute the deci-
sion rule unconditional of this state variable. For instance, in the
stochastic cake-eating problem of section 3.2, if X is not observed,

one could compute c cX dFX, which is the unconditional
mean of consumption, and match it with observed consumption. We
present three methods that can be used when numerical integration
is needed.
Quadrature Methods
There are a number of quadrature methods. We briey detail the
Gauss-Legendre method (more detailed information can be found in
Press et al. 1986). The integral of a function f is approximated as
1
f x dx F w1 f x1 wn f xn ; 3:8
1
where wi and x i are n weights and nodes to be determined. Integra-

tion over a different domain can be easily handled by operating a
56 Chapter 3
change of the integration variable. The weights and the nodes are
computed such that (3.8) is exactly satised for polynomials of
degree 2n 1 or less. For instance, if n 2, denote fi x x i1 , i
1; . . . ; 4. The weights and nodes satisfy
1
w1 f1 x1 w2 f1 x2 f1 x dx;
1
1
w1 f2 x1 w2 f2 x2 f2 x dx;
1
1
w1 f3 x1 w2 f3 x2 f3 x dx;
1
1
w1 f4 x1 w2 f4 x2 f4 x dx:
1
This is a system of four equations with four unknowns. The solu-

tions are w1 w2 1 and x2 x1 0:578. For larger values of n,
the computation is similar. By increasing the number of nodes n, the
precision increases. Notice that the nodes are not necessarily equally
spaced. The weights and the value of the nodes are published in the
literature for commonly used values of n.
Approximating an Autoregressive Process with a Markov Chain

In this discussion we follow Tauchen (1986) and Tauchen and Hus-
sey (1991) and show how to approximate an autoregressive process
of order one by a rst-order Markov process. This way we can sim-
plify the computation of expected values in the value function itera-
tion framework.
To return to the value function in the cake-eating problem, we
need to calculate the expected value given e:
VW; e maxeuW; Ee 0 j e VrW; e 0 :

The calculation of an integral at each iteration is cumbersome. So we
discretize the process et , into N points e i , i 1; . . . ; N. Now we can
replace the expected value by
2 3
X N
VW; e i max4euW; pi; j VrW; e j 5; i 1; . . . ; N:
j1
Figure 3.10
Example of discretization, N 3
As in the quadrature method, the method involves nding nodes

e j and weights pi; j . As we will see below, the e i and the pi; j can be
computed prior to the iterations.
Suppose that et follows an AR(1) process, with an unconditional
mean m and an autocorrelation r:
et m1 r ret1 ut ; 3:9
where ut is a normally distributed shock with variance s 2 . To dis-
cretize this process, we need to determine three different objects.
First, we need to discretize the process et into N intervals. Second,
we need to compute the conditional mean of et within each intervals,
which we denote by z i , i; . . . ; N. Third, we need to compute the
probability of transition between any of these intervals, pi; j . Figure
3.10 shows the plot of the distribution of e and the cut-off points e i as
well as the conditional means z i .
We start by discretizing the real line into N intervals, dened by
the limits e 1 ; . . . ; e N1 . As the process et is unbounded, e 1 y
and e N1 y. The intervals are constructed such that et has an
equal probability of 1=N of falling into them. Given the normality
N1
assumption, the cut-off points fe i gi1 are dened as
i1 i
e m e m 1
F F ; i 1; . . . ; N; 3:10
se se N
where F is the cumulative of p the normal

density and se is the
standard deviation of e equal to s= 1 r. Working recursively, we
get
58 Chapter 3

i i1
1
e se F m:
N
Now that we have dened the intervals, we want to nd the average

value of e within a given interval. We denote this value by z i , which
is computed as the mean of et conditional on et A e i ; e i1 :
fe i m=se fe i1 m=se
z i Eet jet A e i ; e i1 se m:
Fe i1 m=se Fe i m=se
From (3.10), we know that the expression simplies to

i i1
e m e m
z i Nse f f m:
se se
Next we dene the transition probability as
pi; j Pet A e j ; e j1 j et1 A e i ; e i1

e i1 j1
N um 2 =2se2 e m1 r ru
pi; j p e F
2pse e i s
j
e m1 r ru
F du:
s
The computation of pi; j requires the computation of a nontrivial

integral. This can be done numerically. Note that if r 0, meaning
e is an iid process, the expression above is simply
1
pi; j :
N
We can now dene a Markov process zt that will mimic an auto-

regressive process of order one, as dened in (3.9). zt takes its values
N
in fz i gi1 and the transition between period t and t 1 is dened as
Pzt z j jzt1 z i pi; j :
By increasing N, the discretization becomes ner and the Markov

process gets closer to the real autoregressive process.
Example For N = 3, r 0:5, m 0, and s 1, we have
1
z 1:26; z 2 0; z 3 1:26;
and
2 3
0:55 0:31 0:14
6 7
p 4 0:31 0:38 0:31 5:
0:14 0:31 0:55
3.5.3 How to Simulate the Model
Once the value function is computed, the estimation or the evalua-

tion of the model often requires the simulation of the behavior of
the agent through time. If the model is stochastic, the rst step is to
generate a series for the shocks, for t 1; . . . ; T. Then we go from
period to period and use the policy function to nd out the optimal
choice for this period. We also update the state variable and proceed
to next period.
How to Program a Markov Process

The Markov process is characterized by grid points, fz i g and by a
transition matrix p, with elements pij Prob yt z j =yt1 z i . We
start in period 1. The process zt is initialized at, say, z i . Next, we
have to assign a value for z2 . To this end, using the random genera-
tor of the computer, we draw a uniform variable u in 0; 1. The state
in period 2, j, is dened as
t=1
oldind=1 * Variable to keep track of
state in period t-1 *
y[t]=z[oldind] * Initialize first period *
do until t>T * Loop over all time periods *
u=uniform(0,1) * Generate a uniform random
variable *
sum=0 * Will contain the cumulative
sum of pi *
ind=1 * Index over all possible
values for process *
do until u<=sum * Loop to find out the state
in period t *
sum=sum+pi[oldind,ind] * Cumulative sum of pi *
ind=ind+1
endo
y[t]=z[ind] * State in period t *
oldind=ind * Keep track of lagged state *
t=t+1
endo
Figure 3.11
Simulation of a Markov process
60 Chapter 3
j
X j1
X
pi; l < u a pi; l ;
l1 l1
or j 1 if u < pi; 1 . The values for the periods ahead are constructed
in a similar way. Figure 3.11 presents a computer code that will
construct iteratively the values for T periods.
How to Simulate the Model

For the model we need to initialize all stochastic processes that are
the exogenous shock and the state variables. The state variables can
be initialized to their long-run values or to some other value. Often
the model is simulated over a long number of periods and the rst
periods are discarded to get rid of initial condition problems.
The value of the state variables and the shock in period 1 are
used to determine the choice variable in period 1. In the case of the
continuous stochastic cake-eating problem of section 3.2, we would
construct c1 cX1 . Next we can generate the values of the state
variable in period 2, X2 RX1 c1 y2 , where y2 is calculated
using the method described in section 3.5.3 above. This procedure
would be repeated over T periods to successively construct all the
values for the choice variables and the state variables.
4 Econometrics
4.1 Overview
In this chapter we review the techniques for estimating parameters

of models based on dynamic programming. This chapter is orga-
nized in two sections. In section 4.2 we present two simple examples
that illustrate the different estimation methodologies. We analyze a
simple coin-ipping experiment and the classic problem of supply
and demand. We review standard techniques such as maximum
likelihood and the method of moments as well as simulated estima-
tion techniques. The reader who is already familiar with econometric
techniques could go to section 4.3, which gives more details on these
techniques and studies the asymptotic properties of the estimators. A
more elaborate dynamic programming model of cake eating is used
to illustrate these different techniques.
4.2 Some Illustrative Examples
4.2.1 Coin Flipping
In this simple coin-ipping example the coin is not necessarily fair,

and the outcome of the draw is either heads with a probability P1
or tails with a probability P2 1 P1 , with fP1 ; P2 g A 0; 1 0; 1.
We are interested in estimating the probability of each outcome. We
observe a series of T draws from the coin. Denote the realization of
the tth draw by xt , which is equal either to 1 (if heads) or 2 (if tails).
The data set at hand is thus a series of observations fx1 ; x2 ; . . . ; xT g.
This section will describe a number of methods to uncover the
probabilities fP1 ; P2 g from observed data.
62 Chapter 4
The simple coin-ipping example can be extended in two direc-

tions. First, we can try to imagine a coin with more than two sides
(a dice). We are then able to consider more than two outcomes per
draw. In this case we denote P fP1 ; . . . ; PI g a vector with I ele-
ments where Pi Pxt i is the probability of outcome i. We are
interested in estimating the probabilities fPi gi1;...; I . For simplicity we
sometimes state results for the case where I 2, but the generaliza-
tion to a larger number of outcomes is straightforward.
Second, it may be possible that the draws are serially correlated.
The probability of obtaining a head might depend on the outcome
of the previous draw. In this case we want to estimate Pxt j j
xt1 i. We also consider the generalized example below.
Of course, a researcher might not be interested in these proba-
bilities alone but rather, as in many economic examples, in the
parameters that underlie P. To be more specic, suppose a model
parameterized by y A Y H R k that determines P. That is, associated
with each y is a vector of probabilities P. Denote by My the map-
ping from parameters to probabilities: M : Y ! 0; 1 I .
In the case where I 2, we could consider a fair coin, namely
y 1/2; 1/2 and P P1 ; P2 1/2; 1/2. Alternatively we could
consider a coin that is biased toward heads, with y 2/3; 1/3 and
P P1 ; P2 2/3; 1/3. In these examples the model M is the iden-
tity, My y. In practice, we would have to impose y A 0; 1 in
the estimation algorithm. Another way of specifying the model is to
chose a function M: that is naturally bounded between 0 and 1. In
this case we can let y to belong to R. For instance, the cumulative
distribution of the normal density, noted F: satises this condi-
tion. In the fair coin example we would have y 0; 0 and P
F0; F0 1/2; 1/2. With the biased coin we would have y
0:43; 0:43, as F0:43 2/3 and F0:43 1/3.
Maximum Likelihood
iid case We start with the case where the draws from the coin are
identically and independently distributed. The likelihood of observ-
ing the sample fx1 ; x2 ; . . . ; xT g is given by
Y
I
Lx; P PiNi ;
i1
Econometrics 63
where Ni is the number of observations for which event i occurs.

T
Thus L represents the probability of observing fxt gt1 given P. The
maximum likelihood estimator of P is given by
P arg max L: 4:1

After we derive the rst-order condition for a maximum of Lx; P,
the maximum likelihood estimate of Pi , i 1; 2; . . . ; I becomes
Ni
Pi P : 4:2
j Nj
In words, the maximum likelihood estimator of Pi is the fraction of

occurrences of event i.
Suppose that one has a model M: for the probabilities para-
meterized by y. So, indirectly, the likelihood of the sample depends
on this vector of parameters, denote it L ~x; y Lx; My. In that
case the maximum likelihood estimator of the parameter vector y
is given by
y arg max L
~x; y:
y
In effect, by a judicious choice of y, we choose the elements of P

to maximize the likelihood of observing the sample. In fact, by max-
imizing this function we would end up at the same set of rst-order
conditions, (4.2), that we obtained in solving (4.1).
Example 4.1 Suppose I 2 and that My Fy, where F: is
the cumulative distribution function of the standardized normal
density.1 In this case p1 Pxt 1 Fy and p2 1 Fy. The
parameter is estimated by maximizing the likelihood of observing
the data
y arg max Fy N1 1 Fy N2 ;
y
where N1 and N2 are the number of observations that fall into cate-
gories 1 and 2. Straightforward derivation gives

1 N1
y F :
N1 N2
1. This is in fact the structure of a probit model.

64 Chapter 4
markov structure Similar issues arise in a model that exhibits

more dynamics, as when the outcomes are serially correlated. Let Pij
denote the probability of observing event j in period t 1 condi-
tional on observing event i in period t:
Pij Probxt1 j j xt i:
P
These conditional probabilities satisfy: Pij A 0; 1 and j Pij 1 for
i 1; 2; . . . ; I. Intuitively, the former condition says that given the
current state is i, in period t 1 all j A I will occur with positive
probability and the latter condition requires that these probabilities
sum to one. The probability of observing the sample of data is
Y
T
Lx; P Px1 ; . . . ; xT Pxl j xl1 Px1 :
l2
Let Nij denote the number of observations in which state j oc-

curred in the period following state i. Then the likelihood function is
0 1
Y Nij
Lx; P @ Pij A Px1 :
ij
We can express the probability of the rst observation as a function

of the Pij probabilities:
X
I X
I
Px1 Px1 j x0 j Pj1 :
j1 j1
As before, the conditional probabilities and this initial probability

can, in principle, depend on y. Thus the maximum likelihood esti-
mator of y would be the one that maximizes Lx; P. Note that there
are now a large number of probabilities that are estimated through
maximum likelihood: II 1. Thus a richer set of parameters can be
estimated with this structure.
Method of Moments
Continuing with our examples, we consider an alternative way to
estimate the parameters. Consider again the iid case, and suppose
that there are only two possible outcomes, I 2, such that we have a
repeated Bernoulli trial. Given a sample of observations, let m denote
a moment computed from the data. For example, m might simply be
Econometrics 65
the fraction of times event i 1 occurred in the sample. In this case

m P1 .
Let my denote the same moment calculated from the model when
the data-generating process (the model M) is parameterized by y. For
now, assume that the number of parameters, k, is equal to one so
that the number of parameters is equal to the number of moments
(the problem is then said to be just identied). Consider the follow-
ing optimization problem:
minmy m 2 :
y
Here we are choosing the parameters to bring the moment from

the model as close as possible to that from the actual data. The y that
emerges from this optimization is a method of moments estimator,
and we denote this estimate by y^.
Example 4.2 Suppose that we chose as a moment the fraction of

times event i 1 occurs in the sample. From our model of coin ip-
ping, this fraction is equal to Fy. The parameter is estimated by
minimizing the distance between the fraction predicted by the model
and the observed one:
2
N1
y arg min Fy :
y N1 N2
Solving the minimization problem gives

N1
y F1 :
N1 N2
Hence, with this choice of moment, the method of moment estimator

is the same as that of the maximum likelihood seen in example 4.1.
In example 4.2 the particular moment we chose was the fraction
of heads in the sample. Often in a data set there is a large set of
moments to chose from. The method of moments does not guide
us in the choice of a particular moment. So which moment should
we consider? Econometric theory has not come out with a clear
indication of optimal moments. However, the moments should be
informative of the parameters to be estimated. This means that the
moments under consideration should depend on the parameters in
such a way that slight variations in their values results in different
values for the moments.
66 Chapter 4
With the choice of moment different from the one in example 4.2,
the method of moments estimator would be different from the max-
imum likelihood estimator. However, asymptotically, when the size
of the data set increases, both estimators converge to the true value.
More generally, let m be a m 1 column vector of moments from
the data. If k < m, the model is said to be over identied, as there are
more moments than parameters to estimate. If k m, the model is
said to be just identied, and if k > m, the model is under identied.
In the latter case estimation cannot be achieved as there are too
many unknown parameters.
So, if k a m, the estimator of y comes from
minmy m 0 W 1 my m:
y
In this quadratic form, W is a weighting matrix. As explained below,

the choice of W is important for obtaining an efcient estimator of y
when the model is overidentied.
Using Simulations
In many applications the procedures outlined above are difcult to
implement, either because the likelihood of observing the data or the
moments is difcult to compute analytically or because it involves
solving too many integrals. Put differently, the researcher does not
have an analytic representation of My. If this is the case, then esti-
mation can still be carried out numerically using simulations.
Consider again the iid case, where I 2. The simulation approach
proceeds in the following way: First, we x y, the parameter of
My. Second, using the random number generator of a computer,
we generate S draws fus g from a uniform distribution over 0; 1.
We classify each draw as heads (denoted i 1) if us < My or tails
(denoted i 2) otherwise. The fractions of the two events in the
simulated data are used to approximate PiS y by counting the
number of simulated observations that take value i, denoted by Si .
So, PiS y Si /S. The simulated maximum likelihood estimator is
dened as
Y
yS arg max PiS y Ni ;
y
i
where, as before, Ni refers to the fraction of observations in which i

occurs. The estimator is indexed by S, the number of simulations.
Econometrics 67
Figure 4.1
Log likelihood, true y0 0
Obviously, a larger number of simulation draws will yield more

precise estimates. Figure 4.1 shows the log-likelihood for the coin-
ipping example based on two series of simulation with respectively
50 and 5,000 draws. The observed data set is for a series of 100
draws. The log-likelihood has a maximum at the true value of the
parameter, although the likelihood is very at around the true value
when the number of simulations is small.
exercise 4.1 Build a computer program that computes the likeli-

hood function using simulations of a sample of T draws for the case
where I 3.
For the method of moments estimator, the procedure is the
same. Once an articial data set has been generated, we can com-
pute moments both on the articial data and on the observed data.
Denote by m S y a moment derived from the simulated data. For
instance, m and m S y could be the fraction of heads in the observed
sample and in the simulated one. The simulated method of mo-
ments estimator is dened as
yS arg minm S y m 0 W 1 m S y m:
y
68 Chapter 4
Figure 4.2
Objective function, simulated method of moments, true y0 0
Figure 4.2 shows the objective function for the simulated method
of moments. The function has a minimum at the true value of
the parameter. Once again, using more simulation draws obtains a
smoother function, which is easier to minimize.
exercise 4.2 Build a computer program that computes the objective

function using simulations of a sample of T draws for the case where
I 3.
In both methods the estimation requires two steps. First, given a
value of y, one needs to simulate articial data and compute either
a likelihood or a moment from this data set. Second, using these
objects, the likelihood or the objective function has to be evaluated
and a new value for the parameters, closer to the true one, found.
These two steps are repeated until convergence occurs to the true
value.
To compute the simulated data, we need to draw random shocks
using the random number generator of a computer. Note that the
random draws have to be computed once and for all at the start of
the estimation process. If the draws change between iterations, it
Econometrics 69
would be unclear whether the change in the criterion function comes

from a change in the parameter or from a change in the random
draws.
The ability to simulate data opens the way to yet another estima-
tion method: indirect inference. This method depends on an auxil-
iary model chosen by the researcher. The model should be easy to
estimate by standard techniques and should capture enough of the
interesting variation in the data. Let us denote it by M ~ c, where c is
a vector of auxiliary parameters describing this new model. Given a
guess for the vector of structural parameters y, the true (structural)
model can be simulated to create a new data set. The auxiliary model
is estimated both on the real data and on the simulated one, provid-
ing two sets of auxiliary parameters. The vector y is chosen such that
the two sets of auxiliary parameters are close to each other.
Note that the vector of auxiliary parameters c is of no particular
interest as it describes a misspecied model (M ~ ). Within the con-
text of the original model, it has no clear interpretation. However, it
serves as a mean to identify and estimate the structural parameters y.
Example 4.3 If, for instance, My Fy, the model has no closed-
form solution as the cumulative of the normal density has no ana-
lytical form. Instead of approximating it numerically, we can use the
indirect inference method to estimate parameters of interest without
computing this function. We might turn to an auxiliary model that
is easier to estimate. For instance, the logit model has closed forms
for the probabilities. Denote by c the auxiliary parameter of the logit
model. With such a model, the probability of observing xt 1 is
equal to
expc
Pxt 1 :
1 expc
Denote by N1 and N2 the number of cases that fall into categories 1

and 2. The log-likelihood of observing some data is
expc 1
L N1 ln N2 ln
1 expc 1 expc
N1 c N1 N2 ln1 expc:
Maximization of this log-likelihood and some rearranging gives a

simple formula for the ML estimator of the auxiliary parameter: c
70 Chapter 4
lnN1 /N2 . We can compute this estimator of the auxiliary parameter

both for our observed data and for the simulated data by observing
in each case the empirical frequencies. Denote the former by c^ and
the latter by c^ S y. The indirect inference estimator is then

^ S ^ 2 S1 y N1 2
yS arg minc y c arg min ln ln :
y y S2 y N2
In this example, as the probit model is difcult to estimate by maxi-

mum likelihood directly, we have replaced it with a logit model
that is easier to estimate. Although we are not interested in c, this
parameter is a means to estimate the parameter of importance y.
So far we have not discussed the size of the simulated data set.
Obviously one expects that the estimation will be more efcient if S
is large, as the moments, the likelihood, or the auxiliary model will
be pinned down with greater accuracy. Using simulations instead of
analytical forms introduces randomness into the estimation method.
For short samples this randomness can lead to biased estimates.
For instance, with the simulated maximum likelihood, we need the
number of simulation draws to go to innity to get rid of the bias.
This is not the case for the simulated method of moment or the indi-
rect inference, although the results are more precise for a large S. We
discuss this issue later on in this chapter.
Identication Issues
We conclude this section on coin ipping with an informal discus-
sion of identication issues. Up to here, we implicitly assumed that
the problem was identied, that the estimation method and the data
set allowed us to get a unique estimate of the true vector of parame-
ters y.
A key issue is the dimensionality of the parameter space k relative
to I, the dimensionality of P. First, suppose that k I 1 so that the
dimensionality of y is the same as the number of free elements of
P.2 Second, assume that My is one to one. This means that M is a
function and that for every P there exists only one value of y such
that P My. In this case we effectively estimate y from P by using
the inverse of the model: y M1 P .
This is the most favorable case of identication, and we would say
the parameters of the model are just identied. It is illustrated in
P
2. This is not I since we have the restriction i Pi 1.
Econometrics 71
Figure 4.3
Just identication
gure 4.3 for I 2 and k 1. There is a unique value of the param-

eter y for which the probability predicted by the model My is
equal to the true probability.
A number of problems can arise even for the special case of
k I 1. First, it might be that the model My is not invertible.
Thus, for a given maximum likelihood estimate of P , there could be
multiple values of y that generate this vector of probabilities. In this
case the model is not identied. This is shown in gure 4.4. Example
4.4 shows a method of moments estimation where a particular choice
of moment leads to nonidentication.
Example 4.4 Let us label heads 1 and tails 2. Suppose that instead of
focusing on the mean of the sample (i.e., the fraction of heads), we
chose the variance of the sample. The variance can be expressed as
Vx Ex 2 Ex 2
2
N1 N2 N1 N2
4 2
N1 N2 N1 N2 N1 N2 N1 N2

N1 N1
1 :
N1 N2 N1 N2
72 Chapter 4
Figure 4.4
Nonidentication
The theoretical and the empirical moments then are

my Fy1 Fy;

N1 N1
m 1 :
N1 N2 N1 N2
This might appear as a perfectly valid choice of moment, but in fact it

is not. The reason is that the function Fy1 Fy is not a mono-
tone function but a hump-shaped one and thus is not invertible. For
both low and high values of y, the function is close to 0. The variance
is maximal when the probability of obtaining a head is equal to that
of obtaining a tail. If either tails or heads are likely, the variance
is going to be low. So a low variance indicates that either heads
or tails are more frequent but does not tell us which occurrence is
more likely. Hence, in this case, the variance is not a valid moment
to consider for identication purposes.
Second, it might be that for a given value of P , there does not
exist a value of y such that My P . In this case the model is sim-
ply not rich enough to t the data. This is a situation of misspeci-
Econometrics 73
Figure 4.5
Zero likelihood
cation. Put differently, there is a zero-likelihood problem here as

the model, however parameterized, is unable to match the observa-
tions. This is illustrated in gure 4.5.
So, returning to the simple coin-ipping example, if there is a
single parameter characterizing the probability of a head occurring
and the mapping from this parameter to the likelihood of heads is
one to one, then this parameter can be directly estimated from the
fraction of heads. But it might be that there are multiple values of
this parameter that can generate the same fraction of heads in a
sample. In this case the researcher needs to bring additional infor-
mation to the problem. If there is no value of this parameter that
can generate the observed frequency of heads, the model needs to be
re-specied.
If, instead of k I 1, we have more dimensions to y than infor-
mation in P k > I 1, we have a situation where the model is again
underidentied. Given the maximum likelihood estimate of P ,
there are multiple combinations of the parameters that, through the
model, can generate P . Again, the researcher needs to bring addi-
tional information to the problem to overcome the indeterminacy of
74 Chapter 4
the parameters. So in the coin-ipping example a physical theory

that involves more than a single parameter cannot be used to esti-
mate from data that yield a single probability of heads.
Alternatively, if k < I 1, then the parameters are overidentied.
In this case there may not be any y that is consistent with all the
components of P. In many applications, such as those studied in
this book, this situation allows the researcher a more powerful test
of a model. If a model is just identied, then essentially there exists
a y such that P can be generated by the model. But when a model
is overidentied, matching the model to the data is a much more
demanding task. Thus a model that succeeds in matching the data
characterized by P when the parameters are overidentied is con-
sidered a more compelling model.
4.2.2 Supply and Demand Revisited
The classic problem of supply and demand will serve as an illustra-

tion of the previous estimation methods and to elucidate the prob-
lem of identication. Suppose that supply depends on prices p and
the weather z. Demand depends on prices and income y:
q S a p p a z z eS supply;
4:3
q D b p p b y y eD demand:
Both the demand and supply shocks are iid, normally distributed,
with mean zero, variances sS2 and sD2 , and covariance rSD . In total,
this model has seven parameters. We solve for the reduced form by
expressing the equilibrium variables as function of the exogenous
variables y and z:
by az eD eS
p y z A1 y A2 z U1 ;
a p bp a p bp aP b P
4:4

a p by az b p a p eD b p eS
q y z B1 y B2 z U2 ;
a p bp a p bp aP b P
where A1 , A2 , B1 , and B2 are the reduced-form parameters. These

parameters can be consistently estimated from regressions using the
reduced form. If the system is identied, we are able to recover all
the structural parameters from the reduced-form coefcients using
Econometrics 75
B1 B2
ap ; bp ;
A1 A2
4:5
B1 B2 B1 B2
b y A1 ; az A2 :
A1 A2 A1 A2
With these four parameters it is straightforward to back out the

variance of the demand and supply shocks. We can compute eS
q a p p az z and calculate the empirical variance. The same proce-
dure can be applied to recover eD .
The estimation in two steps is essentially an instrumental variable
estimation where y and z are used as instruments for the endoge-
nous variables p and q. Instead of two-step OLS, we can use a
number of alternative methods including the method of moments,
maximum likelihood, and indirect inference. We review these meth-
ods in turn.
Method of Moments
Denote by y the vector of parameters describing the model
y a p ; az ; bp ; b y :
For simplicity assume that sD , sS , and rSD are known to the
researcher. From the data compute a list of empirical moments, for
example, of the variance of prices and quantities and the covariances
between prices, quantities, income, and the weather. Denote m
fm1 ; m 2 ; m3 ; m4 g 0 a 4 1 vector of empirical moments with3
cov p; y covp; z
m1 ; m3 ;
Vy Vz
4:6
covq; y covq; z
m2 ; m4 :
V y Vz
These moments can be computed directly from the data. For

instance, m1 can be expressed as
PT
pt p yt y
m1 t1PT 2
:
t1 yt y
3. If we also want to estimate sD ; sS , and rSD , we can include additional moments

such as E p, Eq, V p, Vq, or cov p; q.
76 Chapter 4
From the model we can derive the theoretical counterpart of

these moments, expressed as functions of the structural parameters.
We denote these theoretical moments by my fm1 y; m 2 y; m3 y;
m4 yg. Starting with the expressions in (4.4), some straightforward
algebra gives
by az
m1 y ; m3 y ;
a p bp a p bp
4:7
a p by az b p
m 2 y ; m4 y :
a p bp a p bp
The basis of the method of moments estimation is that at the true

value of the vector of parameters,
Emi y mi 0; i f1; . . . ; 4g:

This is called an orthogonality condition. In practical terms, we
can bring the moments from the model as close as possible to the
empirical ones by solving
y arg min Ly arg minm my 0 Wm my: 4:8

y y
The ergodicity condition on the sample is the assumption used to

make the empirical and the theoretical moments the same as the
sample size goes to innity. Note that this assumption is easily
violated in many macroeconomic samples, as the data are nonsta-
tionary. In practice, most of the macro data are rst made stationary
by removing trends.
How do the results of (4.8) compare to the results in (4.5)? Note
that by our choice of moments, m1 y A1 , m 2 y B1 , m3 y A2 ,
and m4 y B2 . At the optimal value of the parameters we solve
the same problem as in (4.4). This leads to exactly the same values
for the parameters as in (4.5). The method of moments approach
collapses the two steps of the previous section into a single one. The
reduced form is estimated and the nonlinear system of equations
solved in a single procedure.
Could we chose other moments to estimate the structural parame-
ters? As in example 4.4, the answer is both yes and no. The moments
must be informative of the parameters of the model.
For instance, if we chose m1 Ez, the average value of weather,
this moment is independent of the parameterization of the model,
Econometrics 77
since z is an exogenous variable. Hence we are left to estimate four

parameters with only three identifying equations. Any moment
involving an endogenous variable ( p or q in our example) can be
used in the estimation and would asymptotically produce the same
results. With a nite number of observations, higher-order moments
cannot be precisely computed, so an estimation based on covp 4 ; y,
say, would not be very efcient.
Finally, note that in computing the moments of (4.7), we have not
used the assumption that the error terms eD and eS are normally dis-
tributed. Whatever their joint distribution, (4.8) would give a con-
sistent estimate of the four parameters of interest. The next section
presents the maximum likelihood estimation, which assumes the
normality of the residuals.
Maximum Likelihood
The likelihood of observing jointly a given price p and a quantity q
conditional on income and weather can be derived from the reduced
form (4.4) as f p A1 y A2 z; q B1 y B2 z, where f : ; : is the joint
density of the disturbances U1 and U2 and where A1 , A2 , B1 , B2 are
dened as in (4.4). The likelihood of the entire sample is thus
Y
T
Ly f pt A1 yt A2 zt ; qt B1 yt B2 zt : 4:9
t1
We assume here that eD and eS are normally distributed, so U1 and

U2 are also normally distributed with zero mean.4 The maximization
of the likelihood function with respect to the reduced form coef-
cients is a straightforward exercise. It will give asymptotically con-
sistent estimates of A1 , A2 , B1 , and B2 . Since there is a one-to-one
4. The variance of U1 and U2 are dened as

sD2 sS2 2rDS ap2 sD2 bp2 sS2 2ap bp rDS
s12 2
; s22 ;
ap b p ap b p 2
and the covariance between U1 and U2 is
ap sD2 b p sS2 rDS ap b p
r12 :
ap b p 2
The joint density of U1 and U2 can be expressed as
!
1 1 u12 u22
f u1 ; u2 p exp 2ru1 u2
2ps1 s2 1 r 2 21 r 2 s12 s22
with r r12 /s1 s2 .
78 Chapter 4
mapping between the reduced form and the structural parameters,

the estimation will also provide consistent estimates of the parame-
ters a p , b p , az , and b y as in the method of moment case.
Indirect Inference
For a given value of the parameters we are able to draw supply and
demand shocks from their distribution and to simulate articial data
for prices and demand conditional on observed weather and income.
This is done using expression (4.4). Denote the observed data as
T
fqt ; pt ; yt ; zt gt1 . Denote the simulated data as fqts ; pts gt1...; T; s1;...; S .
Denote the set of parameters of the structural system (4.3) as y
fa p ; az ; bp ; b z g. For simplicity we assume that the parameters sD , sS ,
and rDS are known.
Next, we need an auxiliary model that is simple to estimate.
We could use the system (4.3) as this auxiliary model. For both the
observed and the simulated data, we can regress the quantities on
the prices and the income or the weather. Denote the rst set of
auxiliary estimate c^T and the second one c~Ts , s 1; . . . ; S. These vec-
tors contains an estimate for the effect of prices on quantities and the
effect of weather and income on quantity from both the supply and
the demand equations. These estimates will undoubtedly be biased
given the simultaneous nature of the system. However, we are
interested in these auxiliary parameters only to get to the structural
ones (y). The next step is to nd y that brings the vector c~TS
PS ~ s
1/S s1 cyT as close as possible to c^T . Econometric theory tells
us that this will produce a consistent estimate of the parameters of
interest, a p , az , b q , by . Again, we rely here on the assumption of
ergodicity. As will become apparent in section 4.3.3, the estimator
will be less efcient than maximum likelihood or the method of
moments, unless one relies on a very large number of simulations.
Nonidentication
If the weather has no inuence on supply, meaning az 0, then
the reduced form equations only expresses p and q as a function of
income and shocks only. In this case the system is underidentied.
We can only recover part of the original parameters:

B1 q B1
ap ; sp2 V :
A1 A1 p
Econometrics 79
Further manipulation gives
b y B1 A1 b p : 4:10
There is an innity of pairs fb y ; b p g that satises the equality above.
Hence we cannot recover the true values for these two parameters.
From (4.10) it is easy to see that there is an identication problem.
When the estimation involves moment matching or minimization
of a likelihood function, nonidentication may not be as straight-
forward to spot. Some estimation routines will provide an estimate
for the parameters whether the system is identied or not. There is
no reason to think that these estimates will coincide with the true
values, as many sets of parameter values will satisfy the rst-order
conditions (4.8). If the estimation routine is based on a gradient cal-
culation, nding the minimum of a function requires one to calculate
and to nd the inverse Hessian of the criterion function Ly. If
az 0, the Hessian will not be of full rank, as the cross derivatives of
L with respect to az and the other parameters will be zero. Hence one
should be suspicious about the results when numerical problems
occur such as invertibility problems. As the Hessian matrix enters
the calculation of the standard errors, a common sign is also abnor-
mally imprecise coefcients. If the estimation routine is not based on
gradients (e.g., the simplex algorithm), the problem will be more
difcult to spot, as the estimation routine will come up with an esti-
mate. However, these results will usually look strange, with some
coefcients taking absurdly large values. Moreover the estimation
results will be sensible to the choice of initial values.
exercise 4.3 Build a computer program that creates a data set of
prices and quantities using (4.4) for given values z and y. Use this
program to create a data set of size T, the true data set, and then to
construct a simulated data set of size S. Next construct the objective
function for the indirect inference case as suggested in section 4.3.3.
What happens when you set az to zero?
4.3 Estimation Methods and Asymptotic Properties
In this section we present in more detail the asymptotic properties

of each estimator discussed in the previous section. First, we review
the generalized method of moments, which encompasses most of the
classic estimation methods such as maximum likelihood or nonlinear
80 Chapter 4
least squares. Then, we present methods using simulations. All the

methods are illustrated using simple dynamic programming models
as we did in the cake-eating problem of chapters 2 and 3.
In the following discussion we assume that there is a true
model, xut ; y, parameterized by a vector y of dimension k. ut is a
shock that makes the model probabilistic. For instance, the shock ut
can be a taste shock, a productivity shock, or a measurement error.
We observe a sequence of data generated by this model at the true
value of the parameters, which we denote by y0 , and at the true
T
value of the shocks ut0 . Let fxut0 ; y0 gt1 be the observed data, which
T 5
we also denote as fxt gt1 for simplicity. We are interested in recov-
ering an estimate of y0 from the observed data and making statistical
inferences.
4.3.1 Generalized Method of Moments
In the method of moments presented in section 4.2 we minimized

the distance between an empirical moment and the predicted one.
This way we could exploit the fact that on average, the difference
between the predicted and the observed series (or a function of these
series) should be close to zero at the true value of the parameter y0 .
Let is denote this difference as hy; xt , so that
Ehy0 ; xt 0: 4:11
This identifying equality is called an orthogonality restriction. We

denote the sample average of hy; xt as
1X T
gy hy; xt :
T t1
Then an estimate of y can be found as
y^ arg min Qy arg min gy 0 WT1 gy:

y y
In the expression above WT1 is a weighting matrix, which might

depend on the data, and hence the T subscript. If gy is of size q 1,
then WT1 is of size q q. For instance, if we want to match the rst
two moments of the process fxt g, the function h can be written as
5. Here we view T as the length of the data for time series applications and as the
number of observations in a cross section.
Econometrics 81

xt y xt
hy; xt :
xt y 2 xt2
Averaging this vector over the sample will yield gy

xy x; xy 2 x 2 .
Economic theory often provides more restrictions that can be used
in the estimation method. They often take the form of rst-order
conditions, such as Euler equations, that can be used as an ortho-
gonality restriction as in (4.11). This is the intuition that guided the
Hansen and Singleton (1982) study of consumption, as we will see in
chapter 6 (section 6.3.3). Here we summarize that approach with an
example.
Example 4.5 In a standard intertemporal model of consumption

with stochastic income and no borrowing constraints, the rst-order
condition gives
u 0 ct bREt u 0 ct1 :
One can use this restriction to form hy; ct ; ct1 u 0 ct bRu 0 ct1 ,
where y is parameterizing the utility function. On average,
hy; ct ; ct1 should be close to zero at the true value of the parameter.
The Euler equation above brings actually more information than
we have used so far. Not only should the differences between the
marginal utility in period t and t 1 be close to zero, but it should
also be orthogonal to information dated t. Suppose that zt is a vari-
able that belongs to the information set at date t. Then the rst-
order condition also implies that on average, hy; ct ; zt :u 0 ct
bRu 0 ct1 should be close to zero at the true value of the parameter.
If we have more than one zt variable, then we can exploit as many
orthogonality restrictions.
For further examples, we refer the reader to section 8.4.3.
Asymptotic Distribution
Let y^T be the GMM estimate, that is, the solution to (4.3.1). Under
regularity conditions (see Hansen 1982):
0
y^T is a consistent estimator of the true value y0 .
0
The GMM estimator is asymptotically normal:
p d
T y^T y0 ! N0; S;
82 Chapter 4
1 0 1
where S DWy D and where
( )
0 qgy; YT
D plim :
T qy 0 yy0
The empirical counterpart of D is

^ 0 qgy; YT
D T
qy 0 ^ :
yy T
This means that asymptotically, one can treat the GMM estimate y^T
as a normal variable with mean y0 and variance S^ /T:
!
^
S
y^T @ N y0 ; :
T
Note that the asymptotic properties of the GMM estimator are

independent of the distribution of the error term in the model. In
particular, one does not have to assume normality.
Optimal Weighting Matrix

We have not yet discussed the choice of the weighting matrix WT1 .
The choice of the weighting matrix does not have any bearing on the
convergence of the GMM estimator to the true value. However, a
judiciously chosen weighting matrix can minimize the asymptotic
variance of the estimator. It can be shown that the optimal weighting
matrix WT produces the estimator with the smallest variance. It is
dened as
1X T X y

Wy lim hy0 ; yt hy0 ; ytl 0 :
T!y T t1 ly

Empirically one can replace Wy by a consistent estimator of this
^ :
matrix W
T
q
X
^ G0; T n
W T 1 Gn; T Gn;0 T
n1
q1
with
1 X T
Gn; T hy^; yt hy^; ytn 0 ;
T tn1
Econometrics 83
which is the Newey-West estimator (see Newey and West 1987 for a
more detailed exposition).
Overidentifying Restrictions
If the number of moments q is larger than the number of parameters
to estimate k, then the system is overidentied. One would only
need k restrictions to estimate y. The remaining restrictions can be
used to evaluate the model. Under the null hypothesis that the
model is the true one, these additional moments should be empiri-
cally close to zero at the true value of the parameters. This forms the
basis of a specication test:
L
Tgy^T 0 W
^ 1 gy^T !
T w 2 q k:
In practice, this test is easy to compute, as one has to compare T

times the criterion function evaluated at the estimated parameter
vector to a chi-square critical value.
Link with Other Estimation Methods

The generalized method of moments actually encompasses most
estimation methods such as OLS, nonlinear least squares, instru-
mental variables, and maximum likelihood because it involves
choosing an adequate moment restriction. For instance, the OLS
estimator is dened such that the right-hand side variables are not
correlated with the error term, and this provides a set of orthogonal
restrictions that can be used in a GMM framework. In a linear model
the GMM estimator dened in this way is also an OLS estimator. The
instrumental variable method exploits the fact that an instrument is
orthogonal to the residual.
4.3.2 Maximum Likelihood
In contrast to the GMM approach, the maximum likelihood strategy

requires an assumption on the distribution of the random variables.
Denote by f xt ; y the probability of observing xt given a parameter
y. The estimation method is designed to maximize the likelihood of
observing a sequence of data X fx1 ; . . . ; xT g. Assuming iid shocks,
the likelihood for the entire sample is
Y
T
LX; y f xt ; y:
t1
84 Chapter 4
It is easier to maximize the log of the likelihood:
X
T
lX; y log f xt ; y:
t1
Example 4.6 Consider the cake-eating problem, dened by the Bell-

man equation below, where W is the size of the cake, r is a shrink
factor, and e is an iid shock to preferences:
VW; e maxeuW; EVrW; e 0 :

V: represents the value of having a cake of size W, given the real-
ization of the taste shock e. The equation above states that the indi-
vidual is indifferent between consuming the cake and waiting if the
shock is e W; y EVrW; e 0 /uW, where y is a vector of parame-
ters describing preferences, the distribution of e, and the shrink
factor r. If e > e W; y, then the individual will consume the cake.
e W; y has no analytical expression but can be solved numerically
with the tools developed in chapter 3. The probability of not con-
suming a cake of size W in a given period is then
Pe < e W; y Fe W; y;
where F is the cumulative density of the shock e. The likelihood of

observing an individual i consuming a cake after t periods is then
Y
t1
li y 1 Fe r t W1 ; y Fe r l W1 ; y:
l1
Suppose that we observe the stopping time for N individuals. Then

the likelihood of the sample is
Y
N
Ly li y:
i1
The maximization of the likelihood with respect to y gives the esti-

mate, y^.
For additional examples, we refer the reader to the second part of the
book, and in particular, section 5.5.4.
exercise 4.4 Use the stochastic cake-eating problem to simulate
some data. Construct the likelihood of the sample and plot it against
different possible values for r.
Econometrics 85
Asymptotic Properties
To derive the asymptotic properties of the maximum likelihood esti-
mator, it is convenient to regard the maximum likelihood as a GMM
procedure. The rst-order condition for the maximum of the log-
likelihood function is
XT
q log f xt ; y
0:
t1
qy
This orthogonality condition can be used as a basis for a GMM esti-

mation, where hy; xt q log f xt ; y/qy. The rst derivative of the
log-likelihood function is also called the score function.
From the GMM formula the covariance matrix is D ^ T S^1 D
^ 0 , with
T T

^ 0 qgy 1XT
q 2 log f xt ; y
D 0 I;
T
qy yy^T T t1 qyqy 0
where I is also known as the information matrix, namely minus the

second derivative of the log-likelihood function:
1X T
S^T hxt ; y^T hxt ; y^T 0 I:
T t1
So we get
p L
T y^T y0 ! N0; I 1 :
The maximum likelihood estimator is asymptotically normal, with

mean zero and a variance equal to I 1 /T.
4.3.3 Simulation-Based Methods
We review here estimation methods based on simulation. This eld is

growing, so we will concentrate on only a few methods. For a more
in-depth discussions of these methods, we refer the reader to Gour-
ieroux and Monfort (1996) and Pakes and Pollard (1989), McFadden
(1989), Laroque and Salanie (1989), and McFadden and Ruud (1994)
(see also Lerman and Manski 1981 for an early presentation).
These methods are often used because the calculation of the
moments are too difcult to construct (e.g., multiple integrals in
multinomial probits as in McFadden 1989 or Hajivassiliou and Ruud
86 Chapter 4
1994, or because the model includes a latent (unobserved) variable as

in Laroque and Salanie 1993). Or, it might be that the model My
has no simple analytic representation so that the mapping from the
parameters to moments must be simulated.
Example 4.7 Consider the cake-eating problem studied in section
4.3.2, but where the taste shocks e are serially correlated. The Bell-
man equation is expressed as
VW; e maxeuW; Ee 0 j e VrW; e 0 :
Here the expectations operator indicates that the expectation of next

periods shock depends on the realization of the current shock. We
can still dene the threshold shock e W Ee 0 j e VrW; e 0 /uW, for
which the individual is indifferent between eating and waiting. The
probability of waiting t periods to consume the cake can be written
as
Pt Pe1 < e W1 ; e2 < e rW1 ; . . . ; et > e r t W1 :
In section 4.3.2 the shocks were iid, and this probability could easily
be decomposed into a product of t terms. If e is serially correlated,
then this probability is extremely difcult to write as et is correlated
with all the previous shocks.6 For t periods we have to solve a mul-
tiple integral of order t, which conventional numerical methods of
integration cannot handle. In this section we will show how simu-
lated methods can overcome this problem to provide an estimate of y.
The different simulation methods can be classied into two groups.
The rst group of methods compares a function of the observed data
to a function of the simulated data. Here the average is taken both on
the simulated draws and on all observation in the original data set at
once. This approach is called moment calibration. It includes the
simulated method of moments and indirect inference.
6. For instance, if et ret1 ut with ut @ N0; s 2 , the probability that the cake is
eaten in period 2 is
p2 Pe1 < e W1 ; e2 > e W2
Pe1 < e W1 Pe2 > e W2 j e1 < e W1
! y e
e1 W1 1 1 1
F p p exp 2 u rv 2 du dv:
s/ 1 r 2 2ps e 2 y 2s
If r 0, then the double integral resumes to a simple integral of the normal
distribution.
Econometrics 87
The second set of methods compare the observed data, observa-

tion by observation, to an average of the simulated predicted data,
where the average is taken over the simulated shocks. This is called
path calibration. Simulated nonlinear least squares or maximum
likelihood fall into this category.
The general result is that path calibration methods require the
number of simulations to go to innity to achieve consistency. In
contrast, moment calibration methods are consistent for a xed
number of simulations.
Simulated Method of Moments

denition This method was developed by McFadden (1989), Lee
T
and Ingram (1991), and Dufe and Singleton (1993). Let fxut ; y0 gt1
s
be a sequence of observed data. Let fxut ; yg, t 1; . . . ; T, s 1; . . . ;
S, or xts y for short, be a set of S series of simulated data, each of
length T conditional on a vector of parameters y. The simulations
are done by xing y and by using the TS draws of the shocks uts
(drawn once and for all). Denote by mxt a vector of functions of the
observed data.7 The estimator for the SMM is dened as
" !#0
XT
1X S
^
yS; T W arg min mxt s
mxut ; y
y
t1
S s1
" !#
X
T
1X S
WT1 mxt mxuts ; y :
t1
S s1
This criterion is similar to the one presented for the method of

moments in section 4.2.1. The difference is that we can avoid the
calculation of the theoretical moments mxt y directly. Instead, we
are approximating them numerically with simulations.
Example 4.8 Say the cake example has serially correlated shocks.
Suppose that we have a data set of T cake eaters for which we
observe the duration of their cake Dt , t 1; . . . ; T.
Given a vector of parameter y that describes preferences and the
process of e, we can solve numerically the model and compute
the thresholds e W. Next we can simulate a series of shocks and
7. For instance, mx x; x 2 if one wants to focus on matching the mean and the
variance of the process.
88 Chapter 4
determine the duration for this particular draws of the shock. We

can repeat this step in order to construct S data sets each containing
T simulated durations.
To identify the parameters of the model, we can, for instance, use
the mean duration and the variance of the duration. Both of these
moments would be calculated from the observed data set and the
simulated data. If we want to identify more than two parameters,
we can try to characterize the distribution of the duration better and
include the fraction of cake eaten at the end of the rst, second, and
third periods.
For further examples, we refer the reader to the second part of the
book, and in particular, to sections 6.3.6 and 7.3.3.
exercise 4.5 Construct a computer program to implement the

approach outlined in example 4.8. First, use as moments the mean
and the variance of the duration. Increase then the number of
moments for the fraction of cakes eaten after the rst and second
period. As the model is overidentied, test the overidentication
restrictions.
properties When the number of simulation S is xed and T ! y,
0
y^ST W is consistent.
p
0
T y^ST y0 ! N0; QS W, where

1 qm 0 1 qm 1 qm 0 1 qm
QS W 1 E0 WT E0 W Sy0 WT1 0
S qy qy 0 qy T qy

qm 0 1 qm 1
E0 W :
qy T qy 0
In the expression above Sy0 is the covariance matrix of

p P T
1/ T T1 t1 mxt E0 mxts y .
The optimal SMM is obtained when W ^T S
^ T . In this case

1 qm 0 1 qm 1
QS W 1 E0 W :
S qy qy 0
When S increases to innity, the variance of the SMM estimator is the
same as the variance of the GMM estimator. Note that when S tends
Econometrics 89
to innity, the covariance matrix of the estimator converges to the

covariance matrix of the standard GMM estimator.
In practice, the optimal weighting matrix can be estimated by
" # " #0
1 XT
1 XS
1 XS
W^
T mxt mxts y^ST mxt mxts y^ST
T t1 S s1 S s1
" #
1 1X S X T
s ^ 1X L
l ^
mxt yST mxt yST
S T s1 t1 L l1
" #0
1 XL
mxts y^ST mxtl y^ST ;
L l1
where xts y and xtl y are simulations generated by independent

draws from the density of the underlying shock. W ^ is a consistent
T

estimate of Wy for T ! y and L ! y. Note that the SMM requires
a large number of simulations to compute the standard errors of the
estimator, even if the estimator is consistent for a xed number of
simulation.
Simulated Nonlinear Least Squares
denition We could estimate the parameters y by matching, at

each period, the observation xt with the prediction of the model
xuts ; y, where uts is a particular draw for the shock. There are two
reasons why the predicted data would not match the observed
one. First, we might evaluate the model at an incorrect parameter
point (i.e., y 0 y0 ). Second, the true shock ut0 is unobserved, so
replacing it with a random draw uts would lead to a discrepancy.
In trying to minimize the distance between these two objects, we
would not know whether to change y or uts . To alleviate the problem,
we could use S simulated shocks and compare xt with xtS y
PS
1/S s1 xuts ; y. A natural method of estimation would be to mini-
mize the distance between the observed data and the average pre-
dicted variable:
1X T
min xt xtS y 2 :
T t1
90 Chapter 4
Unfortunately, this criterion does not provide a consistent estima-

tor of y, for a xed number of simulation S, as the sample size T
increases to innity.8
Laffont et al. (1995) proposes to correct the nonlinear least square
objective function by minimizing the following criterion:
" #
1X T
S 2 1 XS
s S 2
min xt xt y xut ; y xt y : 4:12
y T
t1
SS 1 s1
The rst term is the same as the one discussed above, the distance
between the observed variable and the average predicted one. The
second term is a second-order correction term that takes into account
the bias introduced by the simulation for a xed S.
Example 4.9 Consider a continuous cake-eating problem dened as
VW; e max euc bEe 0 j e VW c; e 0 ;

c
where W is the size of the cake, c is the amount consumed, and e is a

taste shock. The optimal policy rule for this program is of the form
c cW; e. Suppose that we can observe over time both an individ-
uals consumption level and the size of the cake, f^ ^ tg
ct ; W t1;...; T . The
8. To see this, dene yy , the solution to the minimization of the criterion above, when
the sample size T goes to innity:
1X T
yy arg min lim xut ; y0 xy 2
y T T t1
arg min Exu; y0 xy 2

y
arg min Exu; y0 2 xy 2 2xu; y0 xy

y
arg min Vxu; y0 Vxy Exu; y0 Exy 2 :

y
This result holds as Exx ExEx, meaning the covariance between ut and uts is zero.
Differentiating the last line with respect to y, we obtain the rst-order conditions sat-
ised by yy :
q q
Vxyy 2 Exyy Exyy Exu; y0 0:
qy qy
If yy y0 , this rst-order condition is only satised if qVxy0 /qy 0, which is not
guaranteed. Hence yy is not necessarily a consistent estimator. This term depends on
the (gradient of) variance of the variable, where the stochastic element is the simulated
shocks. Using simulated paths instead of the true realization of the shock leads to this
inconsistency.
Econometrics 91
taste shock remains unobserved. To estimate the vector of param-

eter y that describes preferences, we can use the simulated nonlinear
least square method. We simulate S paths for the taste shock,
fets gt1;...; T; s1;...; S which are used to construct simulated predictions
for the model fxWt ; ets gt1;...; T; s1;...; S . At each period we construct
the average consumption conditional on the observed size of the
cake, cW ^ t , by averaging out over the S simulated taste shocks. This
average is then compared with that of the observed consumption
level c^t , using formula (4.12).
For further examples on the simulated nonlinear least square
method, we refer the reader to section 7.3.3.
asymptotic properties For any xed number of simulation S,
0
y^ST is consistent.
p d
0
T y^ST y0 ! N0; SS; T .
A consistent estimate of the covariance matrix SS; T can be obtained
by computing
^ S; T A^1 B^ ^1
S S; T S; T AS; T ;
where A^S; T and B ^S; T are dened below. To this end, denote x s
t
s
qxut ; y/qy, the gradient of the variable with respect to the vector of
PS
parameters, and xt S1 s1 xts , its average across all simulations:
" #
^ 1X T
0 1 XS
s s 0
AS; T xt xt xt xt xt xt ;
T t1 SS 1 s1
XT
^S; T 1
B dS; t y dS; t y 0 ;
T t1
with dS; t a k-dimensional vector
1 XS
dS; t y xt xt yxt y xuts ; y xyxts y:
SS 1 s1
Simulated Maximum Likelihood
denition We write the model as xut ; y, where y is a vector

of parameters and ut is an unobserved error. The distribution of ut
92 Chapter 4
implies a distribution for xut ; y; call it fxt ; y. This can be used to

evaluate the likelihood of observing a particular realization xt . In
many cases the exact distribution of xy; ut is not easily determined,
as the model may be nonlinear or might not even have an explicit
analytical form. In this case we can evaluate the likelihood using
simulations.
We use the simulated maximum likelihood (SML) method to
approximate this likelihood. Let f~xt ; u; y be an unbiased simulator
of fxt ; y:
1X S
Eu f~xt ; u; y lim f~xt ; u s ; y fxt ; y:
S S
s1
The SML estimator is dened as

" #
XT
1 XS
y^ST arg max log f~xt ; uts ; y :
y
t1
S s1
asymptotic properties
0
The SML estimator is consistent if Tpand
S tend to innity. When
both T and S go to innity and when T /S ! 0, then
p d
T y^ST y0 ! N0; I 1 y0 :
The matrix Iy0 can be approximated by
PS ~
1X T
q 2 log1/S s1 fxt ; uts ; y
0 :
T t1 qyqy
0
It is inconsistent if S is xed.
The bias is then
1 1
Ey^ST y0 @ I y0 Eaxt ; y;
S
where
Eu qf~/qyVu f~ covu qf~/qy; f~

axt ; y
Eu f~ 3 Eu f~ 2
The bias decreases in the number of simulations and with the preci-
sion of the estimated parameters, as captured by the information
Econometrics 93
matrix. The bias also depends on the choice of the simulator, through
the function a. Gourieroux and Monfort (1996) propose a rst-order
correction for the bias. Fermanian and Salanie (2001) extend these
results and propose a nonparametric estimator of the unknown like-
lihood function based on simulations.
Indirect Inference
When the model is complex, the likelihood is sometimes intractable.
The indirect inference method works around it by using a simpler
auxiliary model, which is estimated instead. This auxiliary model
is estimated both on the observed data and on simulated data. The
indirect inference method tries to nd the vector of structural
parameters that brings the auxiliary parameters from the simulated
data as close as possible to the one obtained on observed data. A
complete description can be found in Gourieroux et al. (1993; see
also Smith 1993).
Consider the likelihood of the auxiliary model f~xt ; b, where b is a
vector of auxiliary parameters. The estimator b^T , computed from the
observed data is dened by
Y
T
b^T arg max f~xt ; b:
b
t1
Under the null the observed data are generated by the model at the
true value of the parameter y0 . There is thus a link between the aux-
iliary parameter b0 (the true value of the auxiliary parameter) and
the structural parameters y. Following Gourieroux et al. (1993), we
denote this relationship by the binding function by. Were this
function known, we could invert it to directly compute y from the
value of the auxiliary parameter. Unfortunately, this function usu-
ally has no known analytical form, so the method relies on simula-
tions to characterize it.
The model is then simulated by taking independent draws for the
shock uts . This gives S articial data sets of length T: fx1s y; . . . ;
xTs yg, s 1; . . . ; S. The auxiliary model is then estimated out of the
simulated data, to get b^sT :
Y
T
b^sT y arg max f~xts y; b:
b
t1
94 Chapter 4
Dene b^ST the average value of the auxiliary parameters, over all
simulations:
1X S
b^ST b^ y:
S s1 sT
The indirect inference estimator y^ST is the solution to

y^ST arg min b^T b^ST y 0 WT b^T b^ST y;
y
where WT is a positive denite weight matrix which converges to a

deterministic positive denite matrix W.
Example 4.10 Consider the cake problem with serially correlated
shocks. The likelihood of the structural model is intractable, but
we can nd an auxiliary model that is easier to estimate. As the data
set consists of durations, a natural auxiliary model is the standard
duration model. Suppose that we chose an exponential model, which
is a simple and standard model of duration characterized by a con-
stant hazard equal to b. The probability of observing a particular
duration is bebDt . The log-likelihood of observing a set of durations
Dt , t 1; . . . ; T is
X
T
ln L ln be bDt :
t1
This likelihood can be maximized with respect to b. Straightfor-

PT
ward maximization gives b^T t1 Dt /T. In this case the auxiliary
parameter is estimated as the average duration in the data set. Given
a value for the structural parameters of our model of interest y, we
can construct by simulation S data sets containing T observations.
For each articial data set s, we can estimate the auxiliary duration
model to obtain b^sT . Using the procedure above, we are then able to
obtain an estimate of y, such that the auxiliary parameters on both
observed and simulated data are as close as possible. Note that with
the simple auxiliary model we use, the indirect inference proce-
dure turns out to be the same as the simulated method of moments
because we are matching the average duration.
We used the exponential duration model for simplicity of exposi-
tion. This model is parameterized by only one parameter, so we can
identify at best only one structural parameter. For more parameters
we would estimate a duration model with a more exible hazard.
Econometrics 95
For other examples on the indirect inference method, we refer the

reader to the second part of the book, in particular, to sections 5.5.3
and 8.6.1.
Gallant and Tauchen (1996) develop an efcient method of
moments based on the use of an auxiliary method. Instead of
matching on a set of auxiliary parameters, they propose to minimize
the score of the auxiliary model, meaning the rst derivative of the
likelihood of the auxiliary model:
1X S
1X T
q
my; bT ln f~xts y; b^T :
S s1 T t1 qb
The structural parameter are obtained from

y arg min my; b^T 0 Wmy; b^T ;
y
where W is a weighting matrix. Gourieroux et al. (1993) show that

the EMM and the indirect inference estimators are asymptotically
equivalent.
properties For a xed number of simulations S as T goes to

innity, the indirect inference estimator is consistent and normally
distributed:
p
T y^ST y0 ! N0; QS W;
where

1 qb 0 y0 qby0 1 qb 0 y0 1 qby0
QS W 1 W 0 WJ0 I0 K0 J01 W
S qy qy qy qy 0
0
qb y0 qby0 1
W :
qy qy 0
PT
Denote cT y; b t1 log f~xts y; b. The matrices I0 ; J0 , and K0
are dened as
q 2 cT y; b
J0 plim ;
T qbqb 0

p qc y; b
I0 lim V T T ;
T qb
96 Chapter 4
" !#
p q XT
K0 lim V E T 0 f~xt ; b ;
T qb t1
qb 0 y0 q 2 cT y0 ; by0
J01 lim :
qy T qbqy 0
The last formula is useful for computing the asymptotic covariance
matrix without calculating directly the binding function. As in the
GMM case there exists an optimal weighting matrix such that the
variance of the estimator is minimized. The optimal choice denoted
W is
W J0 I0 K0 1 J0 :
In this case the variance of the estimator simplies to

1 qb 0 y0 qby0 1
QS W 1 J0 I0 K0 1 J0 ;
S qy qy 0
or equivalently
1
1 q 2 cy y0 ; by0 2
1 q cy y0 ; by0
QS W 1 I0 K0 :
S qyqb 0 qbqy 0
The formula above does not require us to compute explicitly the
binding function. Note that the choice of the auxiliary model matters
for the efciency of the estimator. Clearly, one would want an auxil-
iary model such that qb 0 y/qy is large in absolute values. If not, the
model would poorly identify the structural parameters.
In practice, by0 ) can be approximated by b^ST y^ST . A consistent
estimator of I0 K0 can be obtained by computing
T X
S
I0d
K0 Ws WWs W 0
S s1
with
qcT y^; b^
Ws ;
qb
1X S
W Ws ;
S s1
Econometrics 97
see Gourieroux et al. (1993, app. 2). Now, if the number of parame-
ters to be estimated in the structural model is equal to the number
of parameters in the auxiliary parameters, the weighting matrix W
plays no role, and the variance QS W simplies to

1 qb 0 y0 qby0 1
QS W 1 W :
S qy qy 0
specication tests A global specication test can be carried out
using the minimized
TS
zT min b^T b^ST y 0 WT b^T b^ST y;
1S y
which follows asymptotically a chi-square distribution with q p

degrees of freedom.
4.4 Conclusion
In this chapter we considered methods to use in estimating the

parameters of a model. We reviewed classical methods such as
maximum likelihood and the generalized method of moments as
well as simulation-based methods. In general, in dynamic program-
ming models the likelihood function or the analytical form of the
moments is difcult to write out, so simulated methods are of great
help. However, they come at a cost, for simulated methods are quite
time-consuming. The computation of the value function and the
optimal policy rules often requires the use of numerical techniques.
If, in addition, simulation estimation methods are used, the estima-
tion of a full edged structural model can take hours, and even days.
The choice of method depends on the problem and the data set.
Path calibration methods such as nonlinear least squares and maxi-
mum likelihood use all the information available in the data, as each
observation is needed in the estimation procedure. The drawback is
that one has to specify the entire model up to the distribution of the
unobserved shock. To have tractable likelihood functions, one must
often impose a normal distribution for the shocks, and this might
impose too much structure on the data. On the other hand, moment
calibration methods such as the method of moments use only part
of the information provided by the data. These methods concentrate
on particular functions of the data, as the mean or the variance, for
98 Chapter 4
instance. In contrast to maximum likelihood, the method does not

necessarily requires the specication of the whole model.
Both methods can be justied. The researcher might be interested
in only a subset of the parameters, such as the intertemporal elastic-
ity of consumption. As in example 4.5, the GMM method allows one
to estimate this parameter, without specifying the distribution of the
income shock. However, calibration methods require the choice of
moments that identify the parameters of the model. When the model
is simple, this is not very difcult. When the models are more com-
plex, for instance, when unobserved heterogeneity is present, it is
not that straightforward to nd informative moments. In such cases
the maximum likelihood may be more desirable. Finally, if the data
are subject to measurement errors, taking moments of the data can
reduce the problem. When using simulation methods, calibration
methods also present the advantage of requiring only a xed num-
ber of simulations to get consistent estimates, so the computation
time is shorter.
Overview of Methodology
In the rst three chapters we presented theoretical tools to model,
solve, and estimate economic models. Ideally, to investigate a par-
ticular economic topic, a research agenda would include all three
parts, building on economic theory and confronting it with the data
to assess its validity.
Figure 4.6 summarizes this approach and points to the relevant
chapters. The gure starts with an economic model, described by a
set of parameters and some choice structure. It is important at this
stage to characterize the properties of that model and the rst-order
conditions or to write it as a recursive problem. The model under
consideration might be difcult to solve analytically. It is some-
time necessary to use numerical methods as developed in chapter
3. One can then derive the optimal policy rules, namely the optimal
behavior given a number of predetermined variables.
From the policy rules9 the parameters can be estimated. This is
usually done by comparing some statistics built both from the
observed data and from the model. The estimated parameters are
9. The specication of the model should also be rich enough so that the estimation
makes sense. In particular, the model must contain a stochastic element that explains
why the model is not tting the data exactly. This can be the case if some character-
istics, such as taste shocks, are unobserved.
Econometrics 99
Figure 4.6
Overview of methodology
produced by minimizing the distance between the observed and the

predicted outcome of the model. Once the optimal parameters are
found, the econometric task is not over. One has to evaluate the t
of the model. There are various ways of doing this. First, although
the models are often nonlinear, one can construct a measure such as
the R 2 to evaluate the percentage of the variance explained by the
model. A higher value is seen as a better t. However, the model
may be very good at reproducing some aspects of the data but fail
miserably in other important dimensions. For instance, in the dis-
crete cake-eating problem, the t of the model could be considerably
increased in the rst T periods if one were to construct time depen-
dent utility functions with T dummy variables for each time period.
100 Chapter 4
Such a model would generate a perfect t when it comes to predict

the fraction of cakes eaten in the rst periods. However, the model
could do very poorly for the remaining periods. A second way to
evaluate the estimated model is to use overidentication restrictions
if the model is overidentied. Finally, one can also perform out of
sample forecasts.
Once one is condent that the estimated model is a convincing
representation of reality, the model can be used to evaluate different
scenarios. The next chapters present examples of this strategy using
a number of relevant topics.
II Applications
5 Stochastic Growth
5.1 Overview
We begin our exploration of applications of dynamic programming

problems in macroeconomics with the stochastic growth model.
This is a natural starting point. This framework has been used for
understanding uctuations in the aggregate economy since it was
introduced by Kydland and Prescott (1982). In order to do so, the
researcher must understand the mapping from the parameters of
preferences and technology to observations, perhaps summarized
by pertinent moments of the data. The model also provides an ana-
lytic structure for policy evaluation.1
The stochastic growth model provides our rst opportunity to
review the techniques of dynamic programming, numerical methods
and estimation methodology. We begin with a review of the non-
stochastic model to get some basic concepts straight, and then we
enrich the model to include shocks and other relevant features.
5.2 Nonstochastic Growth Model
Consider the dynamic optimization problem of a very special

household. This household is endowed with one unit of leisure each
period and supplies this inelastically to a production process. The
household consumes an amount ct each period that it evaluates
using a utility function uct . Assume that u is strictly increasing
and strictly concave. The households lifetime utility is given by
1. In the standard real business cycle model, however, there is no rationale for such
intervention.
104 Chapter 5
X
y
b t1 uct : 5:1
1
The household has access to a technology that produces output y

from capital k, given its inelastically supplied labor services. Let
y f k be the production function. Assume that f k is strictly
increasing and strictly concave.
The capital input into the production process is accumulated from
forgone consumption. That is, the household faces a resource con-
straint that decomposes output into consumption and investment it :
yt c t i t :
The capital stock accumulates according to

kt1 kt 1 d it ;
where d A 0; 1 is the rate of physical depreciation.

Essentially the households problem is to determine an optimal
savings plan by splitting output between these two competing uses.
Note that we have assumed that the household maintains a concave
production function rather than simply renting labor and capital in a
market for factors of production. This way the model of the house-
hold is very special, and often it is referred to as a Robinson Crusoe
economy as the household is entirely self-sufcient. Nonetheless, one
could argue (see below) that the model is informative about market
economies as the resulting allocation can be decentralized as a com-
petitive equilibrium. For now, our focus is on solving for this allo-
cation as a dynamic optimization problem.
To do so, we use the dynamic programming approach and con-
sider the following functional equation:
Vk max
0
u f k 1 dk k 0 bVk 0 for all k: 5:2
k
Here the state variable is the stock of capital at the start of the period
and the control variable is the capital stock for the next period.2
With f k strictly concave, there will exist a maximal level of capi-
tal achievable by this economy given by k where
k 1 dk f k:
2. Equivalently we could have specied the problem with k as the state, c as the con-
trol, and then used a transition equation of k 0 f k 1 dk c.
Stochastic Growth 105
This provides a bound on the capital stock for this economy and thus
guarantees that our objective function, uc, is bounded on the set of
feasible consumption levels, 0; f k 1 dk. We assume that both
uc and f k are continuous and real-valued so that there exists a
Vk that solves (5.2).3
The rst-order condition is given by
u 0 c bV 0 k 0 : 5:3
Of course, we dont know Vk directly, so we need to use (5.2) to
determine V 0 k. As (5.2) holds for all k A 0; k, we can take a deriva-
tive and obtain
V 0 k u 0 c f 0 k 1 d:
Updating this one period and inserting this into the rst-order con-
dition yields
u 0 c bu 0 c 0 f 0 k 0 1 d:
This is an Euler condition that is not unlike the one we encountered
in the cake eating problem. Here the left-hand side is the cost of
reducing consumption by e today. The right-hand side is then the
increase in utility in the next period from the extra capital created by
investment of the e. As in the cake-eating example, if the Euler
equation holds, then no single period deviations will increase the
utility of the household. As in that example, this is a necessary but
not a sufcient condition for optimality.4
From the discussion in chapter 2, Vk is strictly concave. Conse-
quently, from (5.3), k 0 must be increasing in k. To see why, suppose
that current capital increases but future capital falls. Then current
consumption will certainly increase so that the left-hand side of (5.3)
decreases. Yet with k 0 falling and Vk strictly concave, the right-
hand side of (5.3) increases. This is a contradiction.
5.2.1 An Example
Suppose that uc lnc, f k k a , and d 1. With this special

structure, we can set up the nonstochastic growth model. As in Sar-
3. This follows from the arguments in chapter 2.

4. As noted in the discussion of the cake-eating problem, this is but one form of a
deviation from a proposed optimal path. Deviations for a nite number of periods also
do not increase utility if (5.2) holds. In addition a transversality condition must be
imposed to rule out deviations over an innite number of period.
106 Chapter 5
gent (1987) we choose a value function (a guess)
Vk A B ln k for all k:
We want to show that this function satises (5.2). If it does, then the
rst-order condition, (5.3), can be written as
1 bB
0:
c k
We use the resource constraint k a c k 0 to obtain
bBk a k 0 k 0
or

0 bB
k k a: 5:4
1 bB
So, if our choice of Vk is correct, then (5.4) is the policy function.

From this policy function we can verify whether or not our choice
of Vk satises the functional equation (5.2). Substitution of (5.4)
into (5.2) yields

1 a bB
A B ln k ln k b A B ln ka for all k:
1 bB 1 bB
5:5
Here we can use c y k 0 so that

1
c k a:
1 bB
Grouping constant terms yields

1 bB
A ln b A B ln :
1 bB 1 bB
Next we group terms that multiply ln k:
B a bBa:
Hence B a=1 ba. Now A can be determined. So we have a
solution to the functional equation. As for the policy functions, using
B, we nd that
k 0 bak a
and
c 1 bak a :
It is important to understand how this type of argument works.
We started by choosing (guessing) the value function. Then we
derived a policy function. Substituting this policy function into the
functional equation gave us an expression (5.5) that depends only on
the current state k. As this expression must hold for all k, we
grouped terms and solved for the unknown coefcients of the pro-
posed value function.
exercise 5.1 To see how this approach to nding a solution to the
nonstochastic growth model could fail, argue that the following
cannot be solutions to the functional equation:
1. Vk A
2. Vk B ln k
3. Vk A Bk a
5.2.2 Numerical Analysis
The nonstochastic growth model is too simple to seriously take to

the data. Still it provides us with a starting point for applying the
contraction mapping theorem and nding a numerical solution to a
dynamic programming problem. These techniques are quite valuable
since the set of economies for which we can obtain an analytic solu-
tion to (5.2) is too small. These techniques will be used again to
study versions of the stochastic growth model that can be taken
to the data.
The Matlab code grow.m solves (5.2) for certain functional forms
by a value function iteration routine.5 The code has four main sec-
tions, which we discuss in turn below.
Functional Forms
There are two primitive functions that must be specied for the
nonstochastic growth model. The rst is the production function and
the second is the utility function of the household. The grow.m code
assumes that the production function is given by
f k k a :
5. That code and explanations for its use are available on the Web site for this book.
108 Chapter 5
Here a is restricted to lie in the interval 0; 1 so that f k is strictly

increasing and strictly concave.
The households utility function is given by
c 1s
uc :
1s
With this utility function the curvature of the utility function,
u 00 cc

u 0 c
is equal to s.6 We assume that s is positive so that uc is strictly

increasing and strictly concave. When s 1, uc is given by lnc.
Parameter Values
The second component of the program species parameter values.
The code is written so that the user can either accept some baseline
parameters (which you can edit) or change these values in the exe-
cution of the program. Let
Y a; b; d; s
denote the vector of parameters that are inputs to the program. In an
estimation exercise, Y would be chosen so that the models quanti-
tative implications match data counterparts. Here we are simply
interested in the anatomy of the program, and thus Y is set at some-
what arbitrary values.
Spaces
As noted earlier, the value function iteration approach does require
an approximation to the state space of the problem. That is, we need
to make the capital state space discrete. Let k represent the capital
state space. We solve the functional equation for all k A k with the
requirement that k 0 lie in k as well. So the code for the nonstochastic
growth model does not interpolate between the points in this grid
but rather solves the problem on the grid.
The choice of k is important. For the nonstochastic growth model
we might be interested in transition dynamics: if the economy is not
at the steady state, how does it return to the steady state? Let k be
6. In the discussion of King et al. (1988), this term is often called the elasticity of the
marginal utility of consumption with respect to consumption.
the steady state value of the capital stock. From (5.2), it solves
1 baka1 1 d:
This is the value of steady state computed in grow.m. The state
space is built in the neighborhood of the steady state through the
denitions of the highest and lowest values of the capital stock,
which are khi and klow in the code.7 A grid is then set up between
these two extreme values. The user species the neness of the grid
with the consideration that a ner grid provides a better approxi-
mation but is expensive in terms of computer time.8
Value Function Iteration

The fourth section of the program solves (5.2) by a value function
iteration routine. We initiate the routine by supplying a guess on the
value function. The program uses the value of the one-period prob-
lem in which the household optimally consumes all output as well as
the undepreciated capital stock (termed ytot in grow.m) as the initial
guess.9
From our initial guess, a loop is set up to perform the value func-
tion iteration, as described in chapter 3. Here the program requires
two additional inputs. The rst is the total number of iterations that
is allowed, termed T. The second is the tolerance to use in determin-
ing whether the value function iteration routine has converged.
This tolerance is called toler, and this scalar gure is compared
against the largest percent difference between the last two calcu-
lations of the value function V and v in the grow.m program.
Evaluating the Results

Once the program has converged, aspects of the policy function can
be explored. The program produces two plots. The rst plot (gure
5.1) is the policy function: k 0 as a function of k. The policy function is
7. One must take care that the state space is not binding. For the growth model we
know that k 0 is increasing in k and that k 0 exceeds (is less than) k when k is less than
(exceeds) k . Thus the state space is not binding.
8. This trade-off can be seen by varying the size of the state space in grow.m. In many
empirical applications, there is a limit to the size of the state space in that a ner grid
doesnt inuence the moments obtained from a given parameter vector.
9. A useful exercise is to alter this initial guess and determine whether the solution of
the problem is independent of it. Making good initial guesses helps solve the func-
tional equation quickly and is often valuable for estimation routines involving many
loops over parameters.
110 Chapter 5
Figure 5.1
Policy function
upward sloping as argued earlier. The second plot (gure 5.2) is the
level of net investment (k 0 k) for each level of k in the state space.
This line crosses zero at the steady state and is downward sloping.
So, for value of k below the steady state, the capital stock is increas-
ing (net investment is positive), while for k above k , net investment
is negative.
The program also allows one to calculate transition dynamics
starting from an (arbitrary) initial capital stock. There are two useful
exercises that help one practice this piece of code.
exercise 5.2
1. Show how other variables (output, consumption, the real interest
rate) behave along the transition path. Explain the patterns of these
variables.
2. Show how variations in the parameters in Y inuence the speed
and other properties of the transitional dynamics.
Figure 5.2
Net investment
5.3 Stochastic Growth Model
We build on the discussion of the nonstochastic growth model

to introduce randomness into the environment. We start from a
specication of the basic economic environment. Our intention is
to make clear the nature of the intertemporal choice problem and
the assumptions underlying the specication of preferences and
technology.
We then turn to the planners optimization problem. We take
the approach of a planner with an objective of maximizing the ex-
pected lifetime utility of a representative agent.10 This way we can
characterize allocations as the results of a single optimization prob-
lem rather than through the solution of a competitive equilibrium.
Since there are no distortions in the economy, it is straightforward
to determine the prices that support the allocation as a competitive
10. Later in this chapter we move away from this framework to discuss economies
with distortions and heterogeneity.
112 Chapter 5
equilibrium. We do this later in a discussion of the recursive equi-

librium concept.
5.3.1 Environment
The stochastic growth model we study here is based on an economy

with innitely lived households. Each household consumes some of
the single good (ct ) and invests the remainder (it ). Investment aug-
ments the capital stock (kt ) with a one-period lag: investment today
creates more capital in the next period. There is an exogenous rate of
capital depreciation denoted by d A 0; 1. For now, we assume there
is a single good which is produced each period from capital and
labor inputs.11 The capital input is predetermined from past invest-
ment decisions and the labor input is determined by the household.
Fluctuations in the economy are created by shocks to the process
of producing goods. Thus good times represent higher produc-
tivity of both labor and capital inputs. The planner will optimally
respond to these variations in productivity by adjusting household
labor supply and savings (capital accumulation) decisions. Of course,
investment is a forward-looking decision, since the new capital is
durable and is not productive until the next period. Further the ex-
tent to which the labor decision responds to the productivity varia-
tion depends, in part, on whether capital and labor are likely to be
more productive in the future. Consequently the serial correlation
properties of the shocks are critical for understanding the responses
of employment and investment.
More formally, the households preferences over consumption (ct )
and leisure (lt ) are given by
X
y
b t uct ; lt ;
t0
where the discount factor b A 0; 1. We will assume that the function

uc; l is continuously differentiable and strictly concave. The house-
holds face a constraint on their time allocation:
1 lt n t ;
where the unit time endowment must be allocated between leisure
and work (nt ).
11. Later in this chapter we discuss extensions that include multiple sectors.
The production side of the economy is represented by a constant

returns to scale production function over the two inputs. Since scale
is not determined, we model the economy as if there is a single
competitive rm that hires the labor services of the households (Nt )
and uses the households capital in the production process. The pro-
duction function is expressed as
Yt At FK t ; Nt ;
where FK; N is increasing in both inputs, exhibits constant returns
to scale, and is strictly concave. Variations in total factor productiv-
ity, At , are the source of uctuations in this economy. Here upper-
case letters refer to economywide aggregates and lowercase letters to
household (per capita) variables.
Finally, there is a resource constraint: the sum of consumption and
investment cannot exceed output in each period. That is,
Yt Ct It :
For characterizing the solution to the planners problem, this is all

the information that is necessary. The planners problem can be
specied from the statement of preferences, the production function,
and the time and resource constraints. A natural approach might be
to allow the planner to choose a sequence of history-dependent
functions that describe the choices of consumption, investment, and
employment for all time periods conditional on the state of the
economy at that point in time. The description of the state would
then include all productivity shocks and the value of the capital
stock.
Instead of solving a planners problem in which the choice is a
sequence of state-contingent functions, the tools of dynamic pro-
gramming can be used. We turn to that approach now.
5.3.2 Bellmans Equation
To begin the analysis, we assume that labor is inelastically supplied

at one unit per household. Thus we consider preferences represented
by uc. This allows us to focus on the dynamics of the problem. Of
course, we will want to include a labor supply decision before con-
fronting the data, else we would be unable to match any moments
with labor variations. Hence we turn to the more general endoge-
nous labor supply formulation later.
114 Chapter 5
In this case we use the constant returns to scale assumption on

FK; N to write per capita output yt as a strictly concave function
of the per capita capital stock kt :

Kt
yt 1 At F ; 1 1 At f kt :
N
As FK; N exhibits constant returns to scale, f k will be strictly

concave. Bellmans equation for the innite horizon stochastic
growth model is specied as
VA; k max
0
uA f k 1 dk k 0 bEA0 j A VA0 ; k 0
k
for all A; k: 5:6
Here the transition equation used to construct (5.6) is k 0 A f k

1 dk c.
An important element of this model is the multiplicative produc-
tivity shock. Through the introduction of this shock, the model is
constructed to capture procyclical uctuations in productivity. An
important question is whether the uctuations in output, employ-
ment, consumption, investment, and so forth, induced by these
shocks match relevant features of the data.
For the quantitative analysis, we assume that A is a bounded, dis-
crete random variable that follows a rst-order Markov process. The
transition matrix is given by P, and this is implicitly used in the
conditional expectation in (5.6).12
As in the general discussion of chapter 2, one important question
is whether there exists a solution to the function equation. A second
is characterizing the optimal policy function.
For the growth model it is important to be sure that the problem is
bounded. For this, let k solve
k A f k 1 dk; 5:7
where A is the largest productivity shock. Since consumption must
be nonnegative, the k that solves this expression from the transition
equation is the largest amount of capital that this economy could
12. Some of these restrictions are stronger than necessary to obtain a solution. As
we are going to literally compute the solution to (5.6), we will eventually have to cre-
ate a discrete representation anyways. So we have imposed some of these features at
the start of the formulation of the problem. The assumptions on the shocks parallel
those made in the presentation of the stochastic dynamic programming problem in
chapter 2.
accumulate. Since f k is strictly concave, there will exist a unique

nite value of k that satises (5.7). This implies that the largest level
of consumption is also k: the largest feasible consumption occurs
when the largest capital stock is consumed in a single period. Thus
we can bound utility by uk.
Since we have bounded the problem, the results from chapter 2
will apply provided that the discount factor is less than one and that
the shocks follow a bounded, rst-order Markow process. Thus we
know that there exists a unique value function VA; k that solves
(5.6). Further we know that there is a policy function given by
k 0 fA; k.
Our goal is to learn more about the properties of this solution.
An important characteristic of the policy function is that it provides
a bridge from the optimization problem to the data. The policy
function itself depends on the underlying structural parameters
and generates relationships between variables, some of which are
observable. So the inference problem is: What can be determined
about the structural parameters from observations on output, capi-
tal, consumption, productivity, and the like?
5.3.3 Solution Methods
Linearization
We could formulate a solution to the stochastic growth model above
through analysis of the resource constraints and the intertemporal
Euler equation. The latter is a necessary condition for optimality, and
it can be obtained directly from the sequence problem representation
of the planners problem. Alternatively, from Bellmans equation, the
rst-order condition for the planner is written as
u 0 A f k 1 dk k 0 bEA0 j A Vk 0 A0 ; k 0 for all A; k: 5:8
Though we do not know VA; k, we can solve for its derivative.

From (5.6), we have
Vk A; k u 0 cA f 0 k 1 d:
Substituting this into (5.8) and evaluating it at A0 ; k 0 yields
u 0 c bEA0 j A u 0 c 0 A0 f 0 k 0 1 d; 5:9
where
c A f k 1 dk k 0 5:10
116 Chapter 5
and c 0 is dened accordingly. These two expressions, along with

the evolution of A (specied below), denes a system of equations.
So one can represent the optimal growth model as a system of rst-
order stochastic difference equations in c; k; A.
An approximate solution is obtained by linearizing this condition
and the resource constraints around the steady state, c ; k .13 In
this approach we x A at the mean value, A. The steady state value
of the capital stock will then satisfy
1 bA f 0 k 1 d: 5:11
Further in steady state k 0 k k , so the steady state level of con-
sumption satises c A f k dk .
Following King et al. (1988), we let c^t , k^t and A^t denote percent
deviations from their steady state values respectively. For example,
x^t 1 xt x =x . Assume that in terms of deviations from mean, the
shocks follow a rst-order autoregressive process, A^t1 rA^t et1
with r A 0; 1.
Then we can rewrite the Euler condition, (5.9), as
xc^t xc^t1 nrA^t nwk^t1 ; 5:12

where x is the elasticity of the marginal utility of consump-
tion, x 1 u 00 c c =u 0 c . The parameter n 1 bA f 0 k , which equals
1 b1 d in the steady state. The parameter r is the serial cor-
relation of the deviation of the shock from steady state, and w 1
f 00 k k = f 0 k is the elasticity of the marginal product of capital
with respect to capital.
The resource condition (5.10) can be approximated by
1 d sc
k^t1 k^t A^t ct :
d^ 5:13
b 1 sc 1 sc
Here sc is consumptions steady state share of output.

If the researcher species a problem such that preferences and the
production function exhibit constant elasticities, then x and w are
xed parameters, and one does not have to ever solve explicitly for
a steady state. For example, if the production function is Cobb-
Douglas where a is capitals share, then w is simply a 1. Likewise
n just depends on the discount factor and the rate of physical capital
13. The problem is similar to that described by King et al. (1988), though here we have
not yet introduced employment.
depreciation. The consumption share sc is just a function of the

parameters of the economy as well.
For example, in the Cobb-Douglas case, (5.11) can be written as

y
1 b a 1 d ;
k
where y is the steady state level of output. Since the steady state
level of investment i dk , this can be rewritten as
1 bad=1 sc 1 d:
Solving this, we obtain
bad
1 sc ;
1 b1 d
so sc can be calculated directly from the underlying parameters.

This approach delivers a log-linearized system whose parameter
are determined by the underlying specication of preferences, tech-
nology, and the driving processes of the economy. This system can
be simplied by solving out for c^t yielding a stochastic system char-
acterizing the evolution of the state variables; put differently, the
system can be written solely in terms of A^t ; k^t . At this point the
response of the system to productivity innovations can be evaluated
using the data.14
Value Function Iteration

Instead of obtaining an approximate solution by log-linearization,
we could pursue the dynamic programming solution more directly.
First we need to more fully characterize the solution. For this we
often resort to specic examples or numerical analysis.
As a lead example, suppose that uc lnc and that the rate of
depreciation of capital is 100 percent. Assume that the process for
the shocks is given by
ln A0 r ln A e;
where r A 1; 1 so that the process is stationary. Now suppose that

the production function has the form f k Ak a .
14. The discussion in the appendix of King et al. (1988) is recommended for those who
want to study this linearization approach in some detail.
118 Chapter 5
With these restrictions the Euler equation (5.9) reduces to

0 0a1
1 A ak
bEA0 j A : 5:14
c c0
Note that here we take the expectation, over A0 given A, of the ratio
since future consumption c 0 will presumably depend on the realized
value of the productivity shock next period.
To solve for the policy function, we make a guess and attempt to
verify it.15 We claim that the policy function k 0 fA; k is given by
fA; k lAk a ;
where l is an unknown constant. That is, our guess is that the future
capital is proportional to output, which is similar to the policy func-
tion we deduced for the example of the nonstochastic growth model.
The resource constraint then is
c 1 lAk a :
To verify this guess and determine l, we substitute this in the pro-
posed policy function (5.14). This yields
0 0a1
1 A ak
bE 0
A jA :
1 lAk a 1 lA0 k 0 a
Solving for the policy function yields
k 0 baAk a : 5:15
Hence our guess is veried and l ab. This implies that consump-
tion is proportional to income:
c 1 baAk a : 5:16
We could also show that the value function that solves (5.6) is
given by
VA; k G B lnk D lnA for all A; k;

where G, B, and D are the unknown constants that we need to solve
for.
Then using (5.15) and (5.16), we would write the functional equa-
tion as
15. Here we formulate the guess of the policy function rather than the value function.
In either case the key is to check that the functional equation is satised.
G B lnk D lnA ln1 baAk a bG B lnbaAk a
DEA0 j A lnA0 for all A; k: 5:17

Note that there is no maximization here as we have substituted the
policy function into the functional equation.16 Since EA0 j A lnA0
r ln A, we make use of the fact that this relationship holds for all
A; k and group terms together as we did in the analysis of the
nonstochastic growth model. So the constants must be the same on
both sides of (5.17):
G ln1 ba bG bB lnba:
Similarly, for the coefcients multiplying lnk, we must have
B a bBa:
Finally, with respect to lnA,
D 1 bB bDr:
As we solve for G, B, and D in this system of equations, we solve
the functional equation. The solution is unique, so we verify our
guess. Although tedious, we could solve this as
ln1 ba ba=1 ba lnba

G ;
1b
a
B ;
1 ba
1
D :
1 br1 ba
Here we see the importance of discounting: if b 1, then G is inn-

ity. However, this is a very special case. We will use discounting
again when we discuss the empirical implications of the stochastic
growth model.
exercise 5.3 Show that if there is less than 100 percent deprecia-
tion, the solution given by fA; k lAk a fails.
Outside of the special examples, we are left with a direct analysis

of (5.6). It is straightforward to apply the analysis of chapter 2 to the
16. Alternatively, one could start from this guess of the value function and then use it
to deduce the policy function.
120 Chapter 5
nonstochastic problem and solve the functional equation.17 We could

even show that the value function is a strictly concave function of k.
Consequently the policy function would be increasing in k. To see
this, we refer to (5.8). An increase in k will increase the left-hand side
of the equation. If k 0 does not rise, then (5.8) will not hold since the
right-hand side, because of the concavity of VA; k, is a decreasing
function of k 0 .
Further details about the policy function require numerical analy-
sis. A stochastic version of the program, termed grow.m, could be
built as discussed above. This is a good exercise for the reader who
knows and how to write a value function iteration program.18 We
return to the value function iteration program in the next section
after we introduce to the model a labor supply decision.
exercise 5.4 Write a value function iteration program that draws
on grow.m to nd the solution to (5.6).
5.3.4 Decentralization
To study the decentralized economy, the households problem must

be supplemented by a budget constraint, and the sources of income
(labor income, capital income, prots) would have to be specied
along with the uses of these funds (consumption, savings). Likewise
the rms demands for labor and capital inputs would have to be
specied as well. We discuss these in turn using the recursive equi-
librium concept.19
The rms problem is static as we assume that the households hold
the capital. Thus the rm rents capital from the household at a price
of r per unit and hires labor at a wage of o per hour. The wage and
rental rates are all in terms of current period output. Taking these
prices as given, the representative rm will maximize prots by
choosing inputs K; N such that
A fN K; N o and A fK K; N 1 d r:
17. Given that uc and f k are both strictly concave, it is straightforward to see that
the value function for the one-period problem is strictly concave in k. As we argued in
chapter 2, this property is preserved by the TV mapping used to construct a solution
to the functional equation.
18. See Tauchen (1990) for a discussion of this economy and a comparison of the value
function iteration solution relative to other solution methods.
19. See also the presentation of various decentralizations in Stokey and Lucas (1989).
Here we stipulate that the capital rental agreement allows the rm to

use the capital and to retain the undepreciated capital, which it then
sells for the same price as output in the one-sector model. Due to
the constant returns to scale assumption, the number and size of
the rms is not determined. We assume, for simplicity, that there is a
single rm (though it acts competitively) that employs all the capital
and labor in the economy, denoted by uppercase letters.
For the households, their problem is
VA; k; K max urKk oK P k 0 bEA0 j A VA0 ; k 0 ; K 0 ;

k0
5:18
where P is the ow of prots from the rms to the households. This

is a different expression than (5.6) as there is an additional state
variable, K. Here k is the households own stock of capital while K is
the per capita capital stock economywide. The household needs to
know the current value of K, since factor prices depend on this
aggregate state variable through the factor demand equations. This
is indicated in (5.18) by the dependence of rK and oK on K.
Let K 0 HA; K represent the evolution of the aggregate capital
stock. As the household is competitive, it takes the evolution of the
aggregate state variable as given. Thus the household takes current
and future factor prices as given.
The rst-order condition for the households capital decision is
u 0 c bEVk A0 ; k 0 ; K 0 : 5:19
Here the household uses the law of motion for K. From (5.18) we
know that Vk rKu 0 c, so the rst-order condition can be written
as
u 0 c bEr 0 u 0 c 0 : 5:20
A recursive equilibrium is comprised of
0
factor price functions: rK and oK
0
individual policy functions: hA; k; K from (5.18)
0
a law for motion for K: HA; K
such that
0
households and rms optimize
0
markets clear
0
HA; k hA; k; k
122 Chapter 5
From these rst-order conditions for factor demand of the operating

rm, it is easy to see that the solution to the planners problem is a
recursive equilibrium.
5.4 A Stochastic Growth Model with Endogenous Labor Supply
We now supplement our version of the stochastic growth model

with an endogenous labor supply decision. For now, we retain the
perspective of the planners problem and discuss decentralization
later in this section.
5.4.1 Planners Dynamic Programming Problem
We can supplement preferences and technology with a labor input

and thus modify the planners problem as
VA; k max
0
uA f k; n 1 dk k 0 ; 1 n bEA0 j A VA0 ; k 0
k ;n
for all A; k: 5:21

Here the variables are measured in per capita terms: k and n are the
capital and labor inputs per capita.
The optimization problem entails dynamic choice between con-
sumption and investment. Recall that this was the key to the sto-
chastic growth model with xed labor input. In addition given k 0 ,
(5.21) has a static choice of n.20 This distinction is important when
we turn to a discussion of programming the solution to this func-
tional equation.
For given A; k; k 0 , dene sA; k; k 0 from
sA; k; k 0 max uA f k; n 1 dk k 0 ; 1 n; 5:22

n
and let n fÂ; k; k 0 denote the solution to the optimization prob-

lem. The rst-order condition for this problem is given by
uc c; 1 nA fn k; n ul c; 1 n: 5:23
This condition equates the marginal gain from increasing employ-
ment and consuming the extra output with the marginal cost in
terms of the reduction in leisure time. This is clearly a necessary
20. Of course, this is static for a given k 0 . The point is that the choice of n does not
inuence the evolution of the state variable.
condition for optimality: in an optimal solution this type of static

variation should not increase welfare.
Thus, for the current productivity shock and current capital stock
and for a level of future capital, fÂ; k; k 0 characterizes the employ-
ment decision. We can think of sA; k; k 0 as a return function given
the current state A; k and control k 0 .
The return function from this choice of the labor input can then be
used to rewrite the functional equation as
VA; k max
0
sA; k; k 0 bEA0 j A VA0 ; k 0 for all A; k: 5:24
k
This has the same structure as the stochastic growth model with
a xed labor supply, though the return function, sA; k; k 0 , is not a
primitive object. Instead, it is derived from a maximization problem
and thus inherits its properties from the more primitive uc; 1 n
and f k; n functions. Referring to the results in chapter 2, we know
there is a solution to this problem and that a stationary policy func-
tion will exist. Denote the policy function by k 0 hA; k.
The rst-order condition for the choice of the future capital stock is
given by
sk 0 A; k; k 0 bEA0 j A Vk 0 A0 ; k 0 0;
where the subscripts denote partial derivatives. Using (5.24), we can

solve for EA0 j A Vk A0 ; k 0 . The result is a Euler equation
sk 0 A; k; k 0 bEA0 j A sk 0 A0 ; k 0 ; k 00 :
Using (5.22), we can rewrite this in more familiar terms as
uc c; 1 n bEA0 j A uc c 0 ; 1 n 0 A0 fk k 0 ; n 0 1 d; 5:25

where c A f k; n 1 dk k 0 and c 0 is dened similarly. This
Euler equation is another necessary condition for an optimum: else a
variation in the level of savings would increase lifetime expected
utility.
The policy functions will exhibit a couple of key properties
revolving around the themes of intertemporal substitution and con-
sumption smoothing. The issue is essentially understanding the
response of consumption and employment to a productivity shock.
By intertemporal substitution, the household will be induced to
work more when productivity is high. But, due to potentially off-
setting income and substitution effects, the response to a productiv-
124 Chapter 5
ity shocks will be lower the more permanent are these shocks.21
By consumption smoothing, a household will optimally adjust con-
sumption in all periods to an increase in productivity. The more
persistent is the shock to productivity, the more responsive will
consumption be to it.22
5.4.2 Numerical Analysis
A discussion along the same lines as that for the stochastic growth
model with xed labor input applies here as well. As in King et al.
(1988) one can attack the set of necessary conditions(5.23), (5.25),
and the resource constraintthrough a log-linearization procedure.
The reader is urged to study that approach from their paper.
Alternatively, one can again simply solve the functional equation
directly. This is just an extension of the programming exercise given
at the end of the previous section on the stochastic growth model
with xed labor supply. The outline of the program will be dis-
cussed here leaving the details as an additional exercise.
The program should be structured to focus on solving (5.24)
through the value function iteration. The problem is that the return
function is derived and thus must be solved for inside of the pro-
gram. The researcher can obtain an approximate solution to the
employment policy function, given previously as fÂ; k; k 0 . This is
achieved by specifying grids for the shocks, the capital state space
and the employment space.23 As noted earlier, this is the point of
approximation in the value function iteration routine: ner grids
yield better approximations but are costly in terms of computer time.
Once fÂ; k; k 0 is obtained, then
sA; k; k 0 uA f k; fÂ; k; k 0 1 dk k 0 ; 1 fÂ; k; k 0

can be calculated and stored. This should all be done prior to start-
21. Preferences are actually often specied so that there is no response in hours worked
to permanent shocks. Another specication of preferences, pursued by Hansen (1985),
arises from the assumption that employment is a discrete variable at the individual
level. Rogerson (1988) provides the basic framework for the indivisible labor model.
22. We will see this in more detail in the following chapter on household savings and
consumption where there is stochastic income.
23. For some specications of the utility function, fÂ; k; k 0 can be solved analytically
and inserted into the program. For example, suppose that uc; 1 n Uc x1 n,
where x is a parameter. Then the rst-order condition is A fn k; n x, which can be
solved to obtain fÂ; k; k 0 given the production function. To verify this, assume that
A f k; n is a Cobb-Douglas function.
ing the value function iteration phase of the program. So, given
sA; k; k 0 , the program would then proceed to solve (5.24) through
the usual value function iteration routine.
The output of the program is then the policy function for capital
accumulation, k 0 hA; k, and a policy function for employment,
n fA; k, where
fA; k fÂ; k; hA; k:

Hence both of these policy functions ultimately depend only on the
state variables, A; k. These policy functions provide a link between
the primitive functions (and their parameters) and observables. We
turn now to a discussion of exploiting that link as the stochastic
growth model confronts the data.
5.5 Confronting the Data
Since its presentation by Kydland and Prescott (1982), macro-

economists have debated the empirical success of the stochastic
growth model. This debate is of interest both because of its impor-
tance for the study of business cycles and for its inuence on empir-
ical methodology. Our focus here is on the latter point as we use
the stochastic growth model as a vehicle for exploring alternative
approaches to the quantitative analysis of dynamic equilibrium
models.
Regardless of the methodology the link between theory and data
is provided by the policy functions. To set notation, let Y denote
a vector of unknown parameters. We will assume that the produc-
tion function is Cobb-Douglas and is constant returns to scale. Let a
denote capitals share. Further assume
uc; 1 n lnc x1 n
as the utility function.24 The parameter vector then is
Y a; d; b; x; r; s;
where a characterizes the technology, d determines the rate of
depreciation of the capital stock, b is the discount factor, and x
24. The interested reader can go beyond this structure, though the arguments put
forth by King et al. (1988) on restrictions necessary for balanced growth should be
kept in mind. Here the function x1 n is left unspecied for the moment, though we
assume it has a constant elasticity given by h.
126 Chapter 5
parameterizes preferences. The technology shock process is para-

meterized by a serial correlation r and a variance s. To make
clear that the properties of this model economy depend on these
parameters, we index the policy functions by Y: k 0 hY A; k and
n fY A; k. At this point we assume that for a given Y these policy
functions have been obtained from a value function iteration pro-
gram. The question is then how to estimate Y.
5.5.1 Moments
One common approach to estimation of Y is based on matching

moments. The researcher species a set of moments from the data
and then nds the value of Y to match (as closely as possible) these
moments. A key element, of course, is determining the set of
moments to match.
Kydland and Prescott (1982) give one version of this approach,
termed calibration. Kydland and Prescott consider a much richer
model than that presented in the previous section as they include
a sophisticated time to build model of capital accumulation, non-
separable preferences, and a signal extraction problem associated
with the technology shock. They pick the parameters for their econ-
omy using moments obtained from applied studies and from low
frequency observations of the U.S. economy. In their words, Our
approach is to focus on certain statistics for which the noise intro-
duced by approximations and measurement errors is likely to be
small relative to the statistic.
Since the model we have studied thus far is much closer to that
analyzed by King, Plosser, and Rebelo (1988), we return to a discus-
sion of that paper for an illustration of this calibration approach.25
King, Plosser, and Rebelo calibrate their parameters from a variety
of sources. As do Kydland and Prescott, the technology parameter
is chosen to match factor shares. The Cobb-Douglas specication
implies that labors share in the National Income and Product
Accounts should equal 1 a. The rate of physical depreciation is set
at 10 percent annually, and the discount rate is chosen to match a
6.5 percent average annual return on capital. The value of x is set so
that, on average, hours worked are 20 percent of total hours corre-
25. King, Plosser, and Rebelo, however, build a deterministic trend into their analysis,
which they remove to render the model stationary. As noted in section 3.2.1 of their
paper, this has implications for selecting a discount factor.
Table 5.1
Observed and predicted moments
KPR calibrated
Moments U.S. data model
Std relative to output
Consumption 0.69 0.64
Investment 1.35 2.31
Hours 0.52 0.48
Wages 1.14 0.69
Cross correlation with output
Consumption 0.85 0.82
Investment 0.60 0.92
Hours 0.07 0.79
Wages 0.76 0.90
sponding to the average hours worked between 1948 and 1986. King,
Plosser, and Rebelo use variations in the parameters of the stochastic
process (principally r) as a tool for understanding the response of
economic behavior as the permanence of shocks is varied. In other
studies, such as Kydland and Prescott (1982), the parameters of the
technology shock process is inferred from the residual of the pro-
duction function.
Note that for these calibration exercises the model does not have
to be solved in order to pick the parameters. That is, the policy
functions are not actually used in the calibration of the parameters.
Instead, the parameters are chosen by looking at evidence that is
outside of business cycle properties, such as time series averages.
Comparing the models predictions against actual business cycle
moments is thus an informal overidentication exercise.
Table 5.1 shows moments from U.S. data as well as the predictions
of these moments from the King, Plosser, and Rebelo model para-
meterized as described above.26 The rst set of moment is the stan-
dard deviation of key macroeconomic variables relative to output.
The second set of moments is the correlation of these variables with
output.
In the literature this is a common set of moments to study. Note
that the stochastic growth model, as parameterized by King, Plosser,
26. Specically the moments from the KPR model are taken from their table 4, using
the panel data labor supply elasticity and r 0:9. The standard deviation of the tech-
nology shock (deviation from steady state) is set at 2.29.
128 Chapter 5
and Rebelo exhibits many important features of the data. In particu-

lar, the model produces consumption smoothing as the standard
deviation of consumption is less than that of output. Further, as in
U.S. data, the variability of investment exceeds that of output. The
cross correlations are all positive in the model as they are in the data.
One apparent puzzle is the low correlation of hours and output in
the data relative to the model.27 Still, based on casual observation,
the model does well. However, these papers do not provide tests
of how close the moments produced by the model actually are to the
data.
Of course, one can go a lot further with this moment-matching
approach. Letting C D be the list of eight moments from U.S. data
shown in table 5.1, one could solve the problem of
minC D C S YWC D C S Y 0 ; 5:26
Y
where C S Y is a vector of simulated moments that depend on the

vector Y which parameterizes the stochastic growth model. As dis-
cussed in chapter 4, W is a weighting matrix. So, for their parame-
terization, the C S Y produced by the King, Plosser, and Rebelo
model is simply the column of moments reported in table 5.1. But,
as noted earlier, the parameter vector was chosen based on other
moments and evidence from other studies.
exercise 5.5 Solve (5.26), using a version of the stochastic growth
model to create the mapping C D .
5.5.2 GMM
Another approach, closer to the use of orthogonality conditions in

the GMM approach, is that of Christiano and Eichenbaum (1992).
Their intent is to enrich the RBC model to encompass the observa-
tions that the correlation between the labor input (hours worked)
and the return to working (the wage and/or the average product
of labor). In doing so, they add shocks to government purchases,
nanced by lump-sum taxes. Thus government shocks inuence the
labor choice of households through income effects. For their exercise,
this is important as this shift in labor supply interacts with variations
in labor demand, thereby reducing the excessively high correlation
27. See King et al. (1988) for a discussion of this.

between hours and the return to work induced by technology shocks

alone.
While the economics here is, of course, of interest, we explore the
estimation methodology employed by Christiano and Eichenbaum.
They estimate eight parameters: the rate of physical depreciation (d),
the labor share of the Cobb-Douglas technology (a), a preference
parameter for households marginal rate of substitution between
consumption and leisure (g), as well as the parameters characteriz-
ing the distributions of the shocks to technology and government
spending.
Their estimation routine has two phases. In the rst, they estimate
the parameters, and in the second they look at additional implica-
tions of the model.
For the rst phase they use unconditional moments to estimate
these parameters. For example, using the capital accumulation equa-
tion, they solve for the rate of depreciation as
kt1 it
d1 :
kt
An estimate of d can be obtained as the time series average of

this expression with data on the capital stock and on investment.28
Note that since there is just a single parameter in this condition, d
is estimated independently of the other parameters of the model.
Building on this estimate, Christiano and Eichenbaum use the
intertemporal optimality conditionon the assumption that uc
lncto determine capitals share in the production function. They
proceed in this fashion of using unconditional movements to identify
each of the structural parameters.
Christiano and Eichenbaum then construct a larger parameter
vector, termed F, which consists of the parameters described above
from their version of the stochastic growth model and a vector of
second moments from the data. They place these moments within
the GMM framework. With this structure, they use GMM to estimate
the parameters and obtain a variance covariance matrix that they
then use to produce standard errors for their parameter estimates.29
28. As the authors indicate, this procedure may only uncover the depreciation rate
used to construct the capital series from observations on investment.
29. In contrast to many studies in the calibration tradition, this is truly an estimation
exercise, complete with standard errors.
130 Chapter 5
The point of their study is to confront observations on the correla-

tion of hours and the average product of labor, corr y=n; n, and the
relative standard deviations of the labor input and the average pro-
ductivity of labor, sn =sy=n . They test whether their model, at the
estimated parameters, is able to match the values of these moments
in the data. Note that this is in the spirit of an overidentication test,
although the model they estimate is just identied. They nd that the
stochastic growth model with the addition of government spending
shocks is unable (with one exception) to match the observations for
these two labor market statistics. The most successful version of the
model is estimated with establishment data. In this specication they
assume that the labor input is indivisible and government spending
is not valued at all by the households.30
5.5.3 Indirect Inference
Smith (1993) illustrates the indirect inference methodology by a ver-

sion of the simple stochastic growth model with xed employment,
as in (5.6). There is one important modication: Smith considers an
accumulation equation of the form
k 0 k1 d Zt it ;
where Zt is a second shock in the model. Greenwood et al. (1988)

interpret this as a shock to next investment goods, and Cooper and
Ejarque (2000) view this as an intermediation shock.
With this additional shock, the dynamic programming problem
for the representative household becomes

1 dk k 0
VA; Z; k max u A f k; n ; 1 n
k 0; n Z
bEA0 j A VA0 ; Z 0 ; k 0 : 5:27
Note the timing here: the realized value of Z is known prior to the
accumulation decision. As with the stochastic growth model, this
dynamic programming problem can be solved by a value function
iteration or by linearization around the steady state.
From the perspective of the econometrics, by introducing this
second source of uncertainty, the model has enough randomness to
30. The model cannot be rejected at a 15 percent level based on the J-statistic com-
puted from the match of these two moments.
avoid zero likelihood observations.31 As with the technology shock,

there is a variance and a serial correlation parameter used to charac-
terize this normally distributed shock. Smith assumes that the inno-
vations to these shocks are uncorrelated.
To take the model to the data, Smith estimates a VAR(2) on log
detrended quarterly U.S. time series for the period 1947:11988:4.
The vector used for the analysis is
xt yt it 0 ;
where yt is the detrended log of output and it is the detrended log of

investment expenditures. With two lags of each variable, two con-
stants and three elements of the variance-covariance matrix, Smith
generates thirteen coefcients.
He estimates nine parameters using the SQML procedure. As out-
lined in his paper and in chapter 3, this procedure nds the struc-
tural parameters that maximize the likelihood of observing the data
when the likelihood function is evaluated at the coefcients pro-
duced by running the VARs on simulated data created from the
model at the estimated structural parameters. Alternatively, one
could directly choose the structural parameters to minimize the dif-
ference between the VAR(2) coefcients on the actual and simulated
data.
5.5.4 Maximum Likelihood Estimation
Last, but certainly not least, is a version of the stochastic growth

model estimated using the maximum likelihood approach. As in the
indirect inference approach, it is necessary to supplement the basic
model with additional sources of randomness to avoid the zero like-
lihood problem. This point is developed in the discussion of maxi-
mum likelihood estimation in Kocherlakota et al. (1994). Their goal
is to evaluate the contribution of technology shocks to aggregate
uctuations.
Kocherlakota et al. (1994) construct a model economy that includes
shocks to the production function and stochastic depreciation. In
particular, the production function is given by Yt At Kta Nt Xt 1a
Qt . Here Xt is exogenous technological progress, Yt is the output of
the single good, K t is the capital stock, and Nt is the labor input. The
31. The reason is that the empirical analysis focuses on output and investment
uctuations.
132 Chapter 5
transition equation for capital accumulation is K t1 1 dt K t It ,

where dt is the rate of depreciation and It is the level of investment.
The authors rst consider a version of the stochastic growth model
without a labor input. They show that the linearized decision rules
imply that consumption and the future capital stock are proportional
to the current stock of capital.32
They then proceed to the estimation of their model economy with
these three sources of uncertainty. They assume that the shocks fol-
low an AR(1) process. Kocherlakota et al. (1994) construct a repre-
sentation of the equilibrium process for consumption, employment,
and output as a function of current and lagged values of the shocks.
This relationship can then be used to construct a likelihood function,
conditional on initial values of the shocks.
Kocherlakota et al. (1994) x a number of the parameters that one
might ultimately be interested in estimating and focus attention on
S, the variance-covariance matrix of the shocks. This is particularly
relevant to their exercise of determining the contribution of technol-
ogy shocks to uctuations in aggregate output. In this regard they
argue that without additional assumptions about the stochastic pro-
cess of the shocks, they are unable to identify the relative variances
of the shocks.
There are a number of other papers that have taken the maximum
likelihood approach.33 Altug (1989) estimates a version of the Kyd-
land and Prescott (1982) model with a single fundamental shock to
technology and measurement error elsewhere. Altug (1989) nds
some difculty matching the joint behavior of labor and other series.
Hall (1996) studies a version of a labor-hoarding model, which is
then compared to the overtime labor model of Hansen and Sargent
(1988). While the Hall (1996) paper is too complex to present here, it
is particularly noteworthy for the comparison of results from esti-
mating parameters using GMM and maximum likelihood.
5.6 Some Extensions
In the nal section of this chapter we consider extensions of the basic

models. These are provided here partly as exercises for readers
32. When employment is variable and wages are observed, then (5.23) has no error
term either. So researchers include taste shocks. This way they nd that current con-
sumption can be written as a function of current output and lagged consumption
without any error term. This prediction is surely inconsistent with the data.
33. See Hansen et al. (1994) for a general formulation of this approach.
interested in going beyond the models presented here.34 One of the

compelling aspects of the stochastic growth model is its exibility in
terms of admitting a multitude of extensions.
5.6.1 Technological Complementarities
As initially formulated in a team production context by Bryant

(1983) and explored subsequently in the stochastic growth model by
Baxter and King (1991), supplementing the individual agents pro-
duction function with a measure of the level of activity by other
agents is a convenient way to introduce interactions across agents.35
The idea is to introduce a complementarity into the production pro-
cess so that high levels of activity in other rms implies that a single
rm is more productive as well.
Let y represent the output at a given rm, Y be aggregate output, k
and n the rms input of capital and labor respectively. Consider a
production function of
y Ak a n f Y g Y1
e
; 5:28
where A is a productivity shock that is common across producers.
Here g parameterizes the contemporaneous interaction between
producers. If g is positive, then there is a complementarity at work:
as other agents produce more, the productivity of the individual
agent increases as well. In addition this specication allows for a
dynamic interaction parameterized by e. As discussed in Cooper and
Johri (1997), this may be interpreted as a dynamic technological
complementarity or even a learning by doing effect. This production
function can be embedded into a stochastic growth model.
Consider the problem of a representative household with access to
a production technology given by (5.28). This is essentially a version
of (5.21) with a different technology.
There are two ways to solve this problem. The rst is to write the
dynamic programming problem, carefully distinguishing between
34. Each of these extensions creates an environment that the interested reader can
use in specifying, solving, and estimating parameters of a dynamic programming
problem.
35. Cooper (1999) explores a wide variety of ways to model complementarities.
Enriching the neoclassical production function is the one closest to existing models.
See the discussion in Benhabib and Farmer (1994) and Farmer and Guo (1994) on the
use of these models to study indeterminacy. Manski (1993) and Cooper (2002) discuss
issues associated with the estimation of models with complementarities and multiple
equilibria.
134 Chapter 5
individual and aggregate variables. As in our discussion of the

recursive equilibrium concept, a law of motion must be specied for
the evolution of the aggregate variables. Given this law of motion,
the individual households problem is solved and the resulting
policy function compared to the one that governs the economywide
variables. If these policy functions match, then there is an equilib-
rium. Else, another law of motion for the aggregate variables is
specied and the search continues.36
Alternatively, one can use the rst-order conditions for the indi-
viduals optimization problem. As all agents are identical and all
shocks are common, the representative household will accumulate
its own capital, supply its own labor, and interact with other agents
only because of the technological complementarity. In a symmetric
equilibrium, yt Yt . As in Baxter and King (1991), this equilibrium
condition is neatly imposed through the rst-order conditions when
the marginal products of labor and capital are calculated. From the
set of rst-order conditions, the symmetric equilibrium can be ana-
lyzed through by approximation around a steady state.
The distinguishing feature of this economy from the tradi-
tional RBC model is the presence of the technological complemen-
tarity parameters, g and e. It is possible to estimate these parameters
directly from the production function or to infer them from the
equilibrium relationships.37
5.6.2 Multiple Sectors
The stochastic growth model explored so far has a single sector of

production. Of course this is just an abstraction that allows the
research to focus on intertemporal allocations without being very
precise about the multitude of activities arising contemporaneously.
As an example, suppose that there are two sectors in the economy.
Sector one produces consumption goods and second two produces
investment goods.38 Let the production function for sector j 1; 2 be
given by
36. In contrast to the contraction mapping theorem, there is no guarantee that this
process will converge. In some cases the households response to an aggregate law of
motion can be used as the next guess on the aggregate law of motion. Iteration of this
may lead to a recursive equilibrium.
37. See Cooper (1999) and the references therein.
38. For now think of these are producer durables, though one could also add con-
sumer durables to this sector or create another sector.
yj A j f k j ; n j :
Here there are sector-specic total factor productivity shocks. An

important issue for this model is the degree of correlation across the
sectors of activity.
Assuming that both capital and labor can be costlessly shifted
across sectors of production, the state vector contains the aggregate
stock of capital rather than its use in the previous period. Further
there is only a single accumulation equation for capital. The dynamic
programming problem for the planner becomes
VA 1 ; A 2 ; k max uc; 1 n bEA 10 ; A 20 j A 1 ; A 2 VA 10 ; A 20 ; k 0 5:29
fk j ; n j g
subject to
c A 1 f k 1 ; n 1 ; 5:30
k 0 k1 d A 2 f k 2 ; n 2 ; 5:31
n n1 n2; 5:32
1 2
k k k : 5:33
This optimization problem can be solved using value function
iteration, and the properties of the simulated economy can, in prin-
ciple, be compared to data. For this economy the policy functions
will specify the state contingent allocation of capital and labor across
sectors.
Economies generally exhibit positive comovement of employment
and output across sectors. This type of correlation may be difcult
for a multisector economy to match unless there is sufcient correla-
tion in the shocks across sectors.39
This problem can be enriched by introducing costs of reallocating
capital and/or labor across the sectors. At the extreme, capital may
be entirely sector specic. In that case the state space for the dynamic
programming problem must include the allocation of capital across
sectors inherited from the past. By adding this friction to the model,
we reduce the ow of factors across the sectors.
exercise 5.6 Extend the code for the one sector stochastic growth
model to solve (5.29). Use the resulting policy functions to simulate
39. Similar problems of matching positive comovements arise in multiple-country real

business cycle models.
136 Chapter 5
the model, and compute moments as a function of key parameters,

such as the correlation of the shocks across the sectors. Relate these
to observed correlations across sectors.
5.6.3 Taste Shocks
Another source of uncertainty that is considered within the stochas-

tic growth model allows for randomness in tastes. This may be a
proxy for variations in the value of leisure brought about by tech-
nology changes in a home production function. Here we specify a
model with shocks to the marginal rate of substitution between con-
sumption and work. Formally, we consider
VA; S; k max
0
uc; 1 n; S bEA0 ; S 0 j A; S VA0 ; S 0 ; k 0 5:34
fk ; ng
subject to the usual production function and capital accumulation

equations. Here S represents the shocks to tastes. This problem may
be interpreted as a two-sector model where the second sector pro-
duces leisure from time and a shock (S). Empirically this type of
specication is useful as there is a shock, internal to the model, that
allows the intratemporal rst-order condition to be violated pro-
vided that S is not observable to the econometrician.
As usual, the policy functions will specify state-contingent em-
ployment and capital accumulation. Again, the model can be solved,
for example, through value function iteration, and then parameters
selected to match moments of the data.
exercise 5.7 Extend the code for the one sector stochastic growth
model to solve (5.34). Use the resulting policy functions to simulate
the model, and compute moments as a function of key parameters,
including the variance/covariance matrix for the shocks. Relate
these to observed correlations from U.S. data. Does the existence of
taste shocks help the model t the data better?
5.6.4 Taxes
One important extension of the stochastic growth model intro-

duces taxes and government spending. These exercises are partly
motivated as attempts to determine the sources of uctuations. Fur-
ther, from a policy perspective, the models are used to evaluate
the impacts of taxes and spending on economic variables. Further,
because the models are based on optimizing households, one can

evaluate the welfare implications of various policies.
McGrattan (1994) and Braun (1994) study these issues. We sum-
marize the results and approach of McGrattan as we elaborate on the
maximum likelihood estimation of these models.
McGrattan species a version of the stochastic growth model
with four sources of uctuations: productivity shocks, government-
spending shocks, capital taxes, and labor taxes. The governments
budget is balanced each period by the use of lump-sum taxes/
transfers to the households. So household preferences are given by
Uc; g; n, where c is private consumption, g is public consumption,
and n is the labor input.40 The budget constraint for the household in
any period t is given by
ct it 1 ttk rt kt 1 ttn wt nt dttk kt Tt ; 5:35
where it is investment by the household and the right side is repre-
sents income from capital rentals, labor supply, depreciation allow-
ances, and a lump-sum transfer. Here ttk and ttn are the period t tax
rates on capital and labor respectively. Because of the presence of
these distortionary taxes, McGrattan cannot use a planners opti-
mization problem to characterize optimal decision rules, and thus
must work directly with a decentralized allocation.
As in the preceding discussion of recursive equilibrium, the idea
is to specify state-contingent transitions for the aggregate variables
and thus, in equilibrium, for relative prices. These prices are, of
course, relevant to the individual through the sequence of budget
constraints, (5.35). Individual households take these aggregate vari-
ables as given rules and optimize. In equilibrium the representative
householdss choices and the evolution of the aggregate variables
coincide.41
McGrattan estimates the model using maximum likelihood tech-
niques. Here the fundamental shocks are supplemented by measure-
ment errors through the specication of a measurement equation.
McGrattan assumes that innovations are normally distributed, and
so writes down a likelihood function for the model economy. The
parameters of the model are estimated using quarterly observations
40. McGrattan (1994) allows for past labor to enter current utility as well.
41. See McGrattan (1994) and the references therein for a discussion of computing
such equilibria.
138 Chapter 5
on output, investment, government purchases, hours, capital, and

the tax rates on capital and labor. Included are parameters that
characterize the utility function, the production function, and the
stochastic processes for the shocks in the system. McGrattan nds a
capital share of 0.397, a discount factor of 0.9927, and a capital de-
preciation rate of about 0.02. Interestingly government purchases do
not appear to enter directly into the households utility function.
Further the log utility specication cannot be rejected.
5.7 Conclusion
The models in this chapter are some of the simpler versions of

the stochastic growth model. This is one of the most useful models
of macroeconomics. There is an enormous literature on it and its
solution techniques. The intention here was more to provide in-
sights into the solutions and estimations of these models using
the dynamic programming approach than to provide a case for or
against the applications of these models in the evaluation of aggre-
gate uctuations.
There are almost endless ways to extend this basic framework. The
approach of this chapter allows the researcher to solve stochastic
growth models numerically and to begin the task of confronting the
models with data.
6 Consumption
6.1 Overview and Motivation
In the next two chapters we study consumption. We devote a

number of chapters to this topic because of its importance in macro-
economics and also because of the common (though unfortunate)
separation of consumption into a study of (1) nondurables and ser-
vices and (2) durables.
From the perspective of business cycle theory, consumption is
the largest component of total expenditures. One of the main aspects
of consumption theory is the theme of consumption smoothing
(dened below). This is evident in the data as the consumption of
nondurables and services is not as volatile as income. In the GDP
accounts durable expenditures is one of the relatively more volatile
elements. Our theories and estimated models must confront these
important facts.
This chapter focuses on the consumption of nondurables and
services. We start with a simple two-period model to build intuition.
We then progress to more complex models of consumption behavior
by going to the innite horizon, adding various forms of uncertainty
and also considering borrowing restrictions. In keeping with the
theme of this book, we pay particular attention to empirical studies
that naturally grow out of consideration of these dynamic optimiza-
tion problems.
6.2 Two-Period Problem
The two-period problem is, as always, a good starting point to build

intuition about the consumption and savings decisions. We start
with a statement of this problem and its solution, and then discuss
some extensions.
140 Chapter 6
6.2.1 Basic Problem
The consumer maximizes the discount present value of consumption

over the two-period horizon. Assuming that preferences are separa-
ble across periods, we represent lifetime utility as:
X
1
b t uct uc0 buc1 ; 6:1
t0
where b A 0; 1 and is called the discount factor. Recall from the

optimal growth model that the taste factor is tied to the marginal
product of capital as part of an equilibrium allocation. It is treated
here as a xed parameter. Period 0 is the initial period, so we make
use of b 0 1.
The consumer is endowed with some initial wealth at the start of
period 0 and earns income yt in period t 0; 1. For now we treat
these income ows as exogenous; we later discuss adding a labor
supply decision to the choice problem. We assume that the agent can
freely borrow and lend at a xed interest rate between each of the
two periods of life. Thus the consumer faces a pair of constraints,
one for each period of life, given by
a1 r 0 a 0 y 0 c0
and
a2 r1 a1 y1 c1 :
Here yt is period t income and at is the agents wealth at the start
of period t. It is important to appreciate the timing and notational
assumptions made in these budget constraints. First, rt represents the
gross return on wealth between period t and period t 1. Second,
the consumer earns this interest on wealth plus income less con-
sumption over the period. It is as if the income and consumption
decisions are made at the start of the period and then interest is
earned over the period. Nothing critical hinges on these timing
decisions, but it is necessary to be consistent about them.
There are some additional constraints to note. First, we restrict
consumption to be nonnegative. Second, the stock of assets remain-
ing at the end of the consumers life a2 must be nonnegative. Else,
the consumer would set a2 y and die (relatively happily) with an
enormous outstanding debt. We leave open the possibility of a2 > 0.
Consumption 141
This formulation of the consumers constraints are similar to the

ones used throughout this book in our statement of dynamic pro-
gramming problems. These constraints are often termed ow con-
straints, since they emphasize the intertemporal evolution of the
stock of assets being inuenced by consumption. As we will see, it is
natural to think of the stock of assets as a state variables and con-
sumption as a control variable.
There is an alternative way to express the consumers constraints
that combines these two ow conditions by substituting the rst into
the second. After some rearranging, it yields
a2 c1 y1
c0 a 0 y 0 : 6:2
r1 r 0 r 0 r0
The left side of this expression represents the expenditures of the

consumer on goods in both periods of life and on the stock of assets
held at the start of period 2. The right side measures the total
amount of resources available to the household for spending over its
lifetime. This is a type of sources as opposed to uses formulation of
the lifetime budget constraint. The numeraire for this expression of
the budget constraint is period 0 consumption goods.
Maximization of (6.1) with respect to c0 ; c1 subject to (6.2) yields
u 0 c0 l br 0 u 0 c1 6:3
as a necessary condition for optimality, where l is the multiplier on
(6.2). This is an intertemporal rst-order condition (often termed the
consumers Euler equation) that relates the marginal utility of con-
sumption across two periods.
It is best to think of this condition as a deviation from a proposed
solution to the consumers optimization problem. So, given a candi-
date solution, suppose that the consumer reduces consumption by a
small amount in period 0 and increases savings by the same amount.
The cost of the deviation is obtained by u 0 c0 in (6.3). The household
will earn r 0 between the two periods and will consume those extra
units of consumption in period 1. This leads to a discounted gain in
utility given by the right side of (6.3). When this condition holds,
lifetime utility cannot be increased through such a perturbation from
the optimal path.
As in our discussion of the cake-eating problem in chapter 2, this
is just a necessary condition, since (6.3) captures a very special type
142 Chapter 6
of deviation from a proposed path: reduce consumption today and

increase it tomorrow. For more general problems (more than two
periods) there will be other deviations to consider. But, even in the
two-period problem, the consumer could have taken the reduced
consumption in period 0 and used it to increase a2 .
Of course, there is another rst-order condition associated with
(6.1): the choice of a2 . The derivative with respect to a2 is given by
l f;
where f is the multiplier on the nonnegativity constraint for a2 .

Clearly, the nonnegativity constraint binds f > 0 if and only if
the marginal utility of consumption is positive l > 0. That is, it is
suboptimal to leave money in the bank when more consumption is
desirable.
This (somewhat obvious but very important) point has two impli-
cations to keep in mind. First, in thinking about perturbations from a
candidate solution, we are right to ignore the possibility of using the
reduction in c0 to increase a2 as this is denitely not desirable. Sec-
ond, and perhaps more important, knowing that a2 0 is also critical
to solving this problem. Looking at the Euler equation (6.3) alone
guarantees that consumption is optimally allocated across periods,
but this condition can hold for any value of a2 . So, applying (6.3) is
but one necessary condition for optimality; a2 0 must be included
as well.
With a2 0, the consumers constraint simplies to
c1 y1
c0 a 0 y 0 1 w 0 ; 6:4
r0 r0
where w 0 is lifetime wealth for the agent in terms of period 0 goods.

The optimal consumption choices depend on the measure of lifetime
wealth w 0 and the intertemporal terms of trade r 0 . In the absence
of any capital market restrictions, the timing of income across the
households lifetime is irrelevant for their consumption decisions.
Instead, variations in the timing of income, given w 0 , are reected in
the level of savings between the two periods.1
1. This has a well-understood implication for the timing of taxes. Essentially a government
with a xed level of spending must decide on the timing of its taxes. If we interpret the in-
come ows in our example as net of taxes, then intertemporal variation in taxes (holding
xed their present value) will only change the timing of household income and not its present
value. Thus tax policy will inuence savings but not consumption decisions.
Consumption 143
As an example, suppose utility is quadratic in consumption:
d
uc a bc c 2 ;
2
where we require that u 0 c b dc > 0. In this case the Euler con-

dition simplies to
b dc0 br 0 b dc1 :
With the further simplication that br 0 1, we have constant con-
sumption: c0 c1 . Note that this prediction is independent of the
timing of income over periods 0 and 1. This is an example of a more
general phenomenon termed consumption smoothing. The smooth-
ing effect will guide our discussion of consumption policy functions.
6.2.2 Stochastic Income
We now add uncertainty to the problem by supposing that income in

period 1 y1 is not known to the consumer in period 0. Further we
use the result of A2 0 and rewrite the optimization problem more
compactly as
max Ey1 j y 0 uc0 buR 0 A 0 y 0 c0 y1 ;

c0
where we have substituted for c1 using the budget constraint. Note

that the expectation is taken here with respect to the only unknown
variable y1 conditional on knowing y 0 , period 0 income. In fact we
assume that
y1 ry 0 e1 ;
where jrj A 0; 1. Here e1 is a shock to income that is not forecastable
using period 0 information. In the optimization problem the con-
sumer is assumed to take the information about future income con-
veyed by observed current income into account.
The Euler equation for this problem is given by
u 0 c0 Ey1 j y 0 bR 0 u 0 R 0 A 0 y 0 c0 y1 :
Note here that the marginal utility of future consumption is stochas-
tic. Thus the trade-off in the Euler equation reects the loss of utility
today from reducing consumption relative to the expected gain,
which depends on the realization of income in period 1.
144 Chapter 6
The special case of quadratic utility and bR 0 1 highlights the

dependence of the consumption decision on the persistence of
income uctuations. For this case the Euler equation simplies to
c0 Ey1 j y 0 c1 R 0 A 0 y 0 c0 Ey1 j y 0 y1 :
Solving for c0 and calculating Ey1 j y 0 y1 yields
R 0 A 0 y 0 ry 0 R0A0 R 0 r
c0 y0 : 6:5
1 R 0 1 R 0 1 R 0 1 R 0
This expression relates period 0 consumption to period 0 income

through two separate channels. First, variations in y 0 directly affect
the resources currently available to the household. Second, varia-
tions in y 0 provide information about future income (unless r 0).
From (6.5) we have
qc0 R 0 r
:
qy 0 1 R 0
In the extreme case of iid income shocks r 0, consumers will

save a fraction of an income increase and consume the remainder.
In the opposite extreme of permanent shocks r 1, current con-
sumption moves one for one with current income. Then savings
does not respond to income at all. The sensitivity of consumption to
income variations depends on the permanence of those shocks.2
Both extremes reect a fundamental property of the optimal
problem of consumption smoothing. By this property, variations
in current income are spread over time periods in order to satisfy
the Euler equation condition that marginal utility today is equal
to the discounted marginal utility of consumption tomorrow, given
the return R 0 . In effect, consumption smoothing is the intertemporal
expression of the normality of goods property found in static
demand theory.
But our example helps highlight an interesting aspect of con-
sumption smoothing: as the persistence of shocks increases, so does
the responsiveness of consumption to income variations. This actu-
ally makes good sense: if income increases today are likely to persist,
there is no need to save any of the current income gain as it will
reappear in the next period. These themes of consumption smooth-
2. If r > 1, then qc 0 =qy 0 will exceed 1.

Consumption 145
ing and the importance of the persistence of shocks will reappear

throughout our discussion of the innite horizon consumer opti-
mization problem.
6.2.3 Portfolio Choice
A second extension of the two-period problem is the addition of

multiple assets. Historically there has been a close link between the
optimization problem of a consumer and asset pricing models. We
will explain these links as we proceed. We begin here with a savings
problem in which there are two assets.
Assume that the household has no initial wealth and can save
current income through these two assets. One is nonstochastic and
has a one period gross return of Rs . The second asset is risky with a
return denoted by R ~ r and a mean return of R r . Let a r and a s denote
the consumers holdings of asset type j r; s. Assets prices are nor-
malized at 1 in period 0.
The consumers choice problem can then be written as
max ~r a r Rs a s y1 :
uy 0 a r a s ER~ r buR
r s
a ;a
Here we make the simplifying assumption that y1 is known with

certainty. The rst-order conditions are
~ r a r R s a s y1
u 0 y 0 a r a s bRs ER~ r u 0 R
and
~ r u 0 R
u 0 y 0 a r a s bER~ r R ~ r a r Rs a s y1 :
Note we have not imposed any conditions regarding the holding of

these assets. In particular, we have allowed the agent to buy or sell
the two assets.
Suppose that uc is strictly concave so that the agent is risk
averse. Further suppose that we search for conditions such that the
household is willing to hold positive amounts of both assets. Then
we would expect that the agent would have to be compensated for
the risk associated with holding the risky asset. This can be seen by
equating these two rst-order conditions (which hold with equality)
and then using the fact that the expectation of the product of two
random variables is the product of the expectations plus the co-
variance. This manipulation yields
146 Chapter 6
covR~ r ; u 0 R
~ r a r Rs a s y1
Rs Rr : 6:6
~ r a r R s a s y1
ER~ r u 0 R
The sign of the numerator of the ratio on the right depends on the
sign of a r .
If the agent holds both the riskless and the risky asset (a r > 0 and
a s > 0), then the strict concavity of uc implies that the covariance
must be negative. In this case, Rr must exceed Rs : the agent must be
compensated for holding the risky asset.
If the average returns are equal, then the agent will not hold the
risky asset ar 0 and (6.6) will hold. Finally, if Rr is less than Rs ,
the agent will sell the risky asset and buy additional units of the
riskless asset.
6.2.4 Borrowing Restrictions
A nal extension of the two-period model is to impose a restriction

on the borrowing of agents. To illustrate, consider a very extreme
constraint where the consumer is able to save but not to borrow:
c0 a y 0 . Thus the optimization problem of the agent is
max uc0 buR 0 A 0 y 0 c0 y1 :

c 0 ay 0
Denote the multiplier on the borrowing constraint by m, the rst-

order condition is given by
u 0 c0 bR 0 u 0 R 0 A 0 y 0 c0 y1 m:
If the constraint does not bind, then the consumer has nonnegative
savings and the familiar Euler equation for the two-period problem
holds. However, if m > 0, then c0 y 0 and
u 0 y 0 > bR 0 u 0 y1 :
The borrowing constraint is less likely to bind if bR 0 is not very large

and if y 0 is large relative to y1 .
An important implication of the model with borrowing con-
straints is that consumption will depend on the timing of income
receipts and not just W 0 . That is, imagine a restructuring of income
that increased y 0 and decreased y1 , leaving W 0 unchanged. In the
absence of a borrowing restriction, consumption patterns would not
change. But, if the borrowing constraint binds, then this restructur-
ing of income will lead to an increase in c0 and a reduction in c1 as
Consumption 147
consumption follows income. To the extent that this change in the

timing of income ows could reect government tax policy ( yt is
then viewed as after-tax income), the presence of borrowing restric-
tions implies that the timing of taxes can matter for consumption
ows and thus for welfare.
The weakness of this and more general models is that the basis for
the borrowing restrictions is not provided. So it is not surprising that
researchers have been interested in understanding the source of
borrowing restrictions. We return to this point in a later section.
6.3 Innite Horizon Formulation: Theory and Empirical

Evidence
We now consider the innite horizon version of the optimal

consumption problem. We are interested in seeing how the basic
intuition of consumption smoothing and other aspects of optimal
consumption allocations carry over to the innite horizon setting. In
addition we introduce empirical evidence into our presentation.
6.3.1 Bellmans Equation for the Innite Horizon Problem
Consider a household with a stock of wealth denoted by A, a current

ow of income y, and a return on its investments over the past
period given by R1 . The state vector of the consumers problem is
A; y; R1 , and the associated Bellman equation is
vA; y; R1 max uc bEy 0 ; R j R1 ; y vA 0 ; y 0 ; R for all A; y; R1 ;
c
where the transition equation for wealth is given by
A 0 RA y c:
We assume that the problem is stationary so that no time subscripts
are necessary.3 This requires, among other things, that income and
returns be stationary random variables and that the joint distribution
of y 0 ; R depend only on y; R1 .
The transition equation has the same timing as we assumed in the
two-period problem: interest is earned on wealth plus income less
3. We assume that there exists a solution to this function equation. This requires, as always,
that the choice be bounded, perhaps by a constraint on the total debt that a household can
accumulate.
148 Chapter 6
consumption over the period. Further the interest rate that applies is
not necessarily known at the time of the consumption decision. Thus
the expectation in Bellmans equation is over the two unknowns
y 0 ; R 0 where the given state variables provide information on fore-
casting these variables.4
6.3.2 Stochastic Income
To analyze this problem, we rst consider the special case where

the return on savings is known and the individual faces uncer-
tainty only with respect to income. We then build on this model by
adding in a portfolio choice, endogenous labor supply, and borrow-
ing restrictions.
Theory
The case we study is
vA; y max uc bEy 0 j y vA 0 ; y 0 ; 6:7
c
where A 0 RA y c for all A; y. The solution to this problem

is a policy function that relates consumption to the state vector:
c fA; y. The rst-order condition is
u 0 c bREy 0 j y vA A 0 ; y 0 ; 6:8
which holds for all A; y, where vA A 0 ; y 0 denotes qvA 0 ; y 0 =qA 0 .
Using (6.7) to solve for Ey 0 j y vA A 0 ; y 0 yields the Euler equation
u 0 c bREy 0 j y u 0 c 0 : 6:9
The interpretation of this equation is that the marginal loss of

reducing consumption is balanced by the discounted expected mar-
ginal utility from consuming the proceeds in the following period.
As usual, this Euler equation implies that a one-period deviation
from a proposed solution that satises this relationship will not
increase utility. The Euler equation, (6.9), holds when consumption
today and tomorrow is evaluated using this policy function. In the
special case of bR 1, the theory predicts that the marginal utility of
consumption follows a random walk.
In general, one cannot generate a closed-form solution of the
policy function from these conditions for optimality. Still, some
4. If there are other variables known to the decision maker that provide information on
y 0 ; R, then these variables would be included in the state vector as well.
Consumption 149
properties of the policy functions can be deduced. Given that uc is

strictly concave, one can show that vA; y is strictly concave in A. As
argued in chapter 2, the value function will inherit some of the cur-
vature properties of the return function. Using this and (6.8), the
policy function, fA; y, must be increasing in A. Else, an increase in
A would reduce consumption and thus increase A 0 . This would con-
tradict (6.8).
As a leading example, consider the specication of utility
c 1g 1
uc ;
1g
where g 1 is the special case of uc lnc. This is called the con-

stant relative risk aversion case (CRRA), since cu 00 c=u 0 c g.
Using this utility function, we rewrite (6.9) as
0 g
c
1 bRE ;
c
where the expectation is taken with respect to future consumption

which, through the policy function, depends on A 0 ; y 0 . As dis-
cussed in some detail below, this equation is then used to estimate
the parameters of the utility function, b; g.
Evidence
Hall (1978) studies the case where uc is quadratic so that the mar-
ginal utility of consumption is linear. In this case consumption itself
is predicted to follow a random walk. Hall uses this restriction to test
the predictions of this model of consumption. In particular, if con-
sumption follows a random walk, then
ct1 ct et1 :
The theory predicts that the growth in consumption et1 should be

orthogonal to any variables known in period t: Et et1 0. Hall uses
aggregate quarterly data for nondurable consumption. He shows
that lagged stock market prices signicantly predict consumption
growth, which violates the permanent income hypothesis.5
Flavin (1981) extends Halls analysis, allowing for a general
ARMA process for the income. Income is commonly found as a pre-
dictor of consumption growth. Flavin points out that this nding is
5. Sargent (1978) also provides a test for the permanent income hypothesis and rejects the
model.
150 Chapter 6
not necessarily in opposition with the prediction of the model. Cur-

rent income might be correlated with consumption growth, not
because of a failure of the permanent income hypothesis but because
current income signals changes in the permanent income. However,
she also rejects the model.
The importance of current income to explain consumption growth
has been seen as evidence of liquidity constraints (see section 6.3.5).
A number of authors have investigated this issue.6 However, most
of the papers use aggregate data to test the model. Blundell et al.
(1994) test the model on micro data and nd that when one controls
for demographics and household characteristics, current income
does not appear to predict consumption growth. Meghir and Weber
(1996) explicitly test for the presence of liquidity constraints using a
U.S. panel data and do not nd any evidence.
6.3.3 Stochastic Returns: Portfolio Choice
We have already considered a simple portfolio choice problem for

the two-period problem, so this discussion will be intentionally brief.
We elucidate the empirical evidence based on this model.
Theory
Assume that there are N assets available. Let R1 denote the N-
vector of gross returns between the current and previous period and
let A be the current stock of wealth. Let si denote the share of asset
i 1; 2; . . . ; N held by the agent. Normalizing the price of each asset
to be unity, the current consumption of the agent is then
X
cA si :
i
Substituting this into the Bellman equation, we have

! !
X X
vA; y; R1 max u A si bER; y 0 j R1 ; y v R i si ; y 0 ; R ;
si
i i
6:10
where Ri is the stochastic return on asset i. Note that R1 is only in

the state vector because of the informational value it provides on the
return over the next period, R.
6. See, for instance, Zeldes (1989b) and Campbell and Mankiw (1989).
Consumption 151
The rst-order condition for the optimization problem holds for

i 1; 2; . . . ; N, and it is
!
X
0 0
u c bER; y 0 j R1 ; y Ri vA Ri si ; y ; R ;
i
where again vA is dened as qv =qA. Using (6.10) to solve for

the derivative of the value function, we obtain
u 0 c bER; y 0 j R1 ; y Ri u 0 c 0 for i 1; 2; . . . ; N;

where, of course, the level of future consumption will depend on the
vector of returns, R, and the realization of future income, y 0 .
This system of Euler equations forms the basis for nancial
models that link asset prices to consumption ows. This system is
also the basis for the argument that conventional models are unable
to explain the observed differential between the return on equity and
relatively safe bonds. Finally, these conditions are also used to esti-
mate the parameters of the utility function, such as the curvature
parameter in the traditional CRRA specication.
This approach is best seen through a review of Hansen and
Singleton (1982). To understand this approach, recall that Hall uses
the orthogonality conditions to test a model of optimal consumption.
Note that Halls exercise does not estimate any parameters as the
utility function is assumed to be quadratic and the real interest rate
is xed. Instead, Hall essentially tests a restriction imposed by his
model at the assumed parameter values.
The logic pursued by Hansen-Singelton goes a step further.
Instead of using the orthogonality constraints to evaluate the pre-
dictions of a parameterized model, they use these conditions to esti-
mate a model. Evidently, if one imposes more conditions than there
are parameters (i.e., if the exercise is overidentied), the researcher
can both estimate the parameters and test the validity of the model.
Empirical Implementation
The starting point for the analysis is the Euler equation for the
households problem with N assets. We rewrite that rst-order con-
dition here using time subscripts to show the timing of decisions and
realizations of random variables:
u 0 ct bEt Rit1 u 0 ct1 for i 1; 2; . . . ; N; 6:11

where Rit1 is dened as the real return on asset i between periods
152 Chapter 6
t and t 1. The expectation here is conditional on all variables

observed in period t. Unknown t 1 variables include the return on
the assets as well as period t 1 income.
The power of the GMM approach derives from this rst-order
condition. What the theory tells us is that while ex post this rst-
order condition need not hold, any deviations from it will be
unpredictable given period t information. That is, the period t 1
realization say, of income, may lead the consumer to increase con-
sumption is period t 1, thus implying ex post that (6.11) does not
hold. This deviation is consistent with the theory as long as it is not
predictable given period t information.
i
Formally, dene et1 y as
i bRit1 u 0 ct1
et1 y 1 1 for i 1; 2; . . . ; N: 6:12
u 0 ct
i
Thus et1 y is a measure of the deviation for an asset i. We have
added y as an argument in this error to highlight its dependence on
the parameters describing the households preferences. Household
optimization implies that
i
Et et1 y 0 for i 1; 2; . . . ; N:
Let zt be a q-vector of variables that are in the period t information
set.7 This restriction on conditional expectations implies that
i
Eet1 y n zt 0 for i 1; 2; . . . ; N; 6:13
where n is the Kronecker product. So the theory implies the Euler

equation errors from any of the N rst-order conditions ought to be
orthogonal to any of the zt variables in the information set. There are
Nq restrictions created.
The idea of GMM estimation is then to nd the vector of structural
parameters y such that (6.13) holds. Of course, applied economists
only have access to a sample, say of length T. Let mT y be an Nq
vector where the component relating asset i to one of the variables
j
in zt , zt , is dened by
1X T
j
e i yzt :
T t1 t1
7. The theory does not imply which of the many possible variables should be used when
employing these restrictions in an estimation exercise. That is, the question of which mo-
ments to match is not answered by the theory.
Consumption 153
The GMM estimator is dened as the value of y that minimizes
JT y mT y 0 WT mT y:
Here WT is an Nq Nq matrix that is used to weight the various
moment restrictions.
Hansen and Singleton (1982) use monthly seasonally adjusted
aggregate data on U.S. nondurable consumption or nondurables and
services between 1959 and 1978. They use as a measure of stock
returns, the equally weighted average return on all stocks listed on
the New York Stock Exchange. They choose a constant relative risk
aversion utility function uc c 1g =1 g. With this specication
there are two parameters to estimate, the curvature of the utility
function g and the discount factor b. Thus y b; g. The authors use
as instruments lagged values of ct ; Rit1 and estimate the model
with 1, 2, 4, or 6 lags. Depending on the number of lags and the
series used, they nd values for g which vary between 0.67 and 0.97
and values for the discount factor between 0.942 and 0.998. As the
model is overidentied, there is scope for an overidentication test.
Depending on the number of lags and the series used, the test gives
mixed results as the restrictions are sometimes satised and some-
times rejected.
Note that the authors do not adjust for possible trends in the esti-
mation. Suppose that log consumption is characterized by a linear
trend:
ct expatc~t ;
where c~t is the detrended consumption. In that case equation (6.12) is
rewritten as
g
i beag Rit1 c~t1
et1 y 1 1 for i 1; 2; . . . ; N:
c~tg
Hence the estimated discount factor is a product between the true

discount factor and a trend effect. Ignoring the trend would result in
a bias for the discount rate.
6.3.4 Endogenous Labor Supply
Of course, it is natural to add a labor supply decision to this model.

We can think of the stochastic income, taken as given above, as
154 Chapter 6
coming from a stochastic wage w and a labor supply decision n.

In this case consider the following functional equation:

A0
vA; w max U A wn ; n bEw 0 j w vA 0 ; w 0 for all A; w:
A 0; n R
Here we have substituted in for current consumption so that the
agent is choosing labor supply and future wealth.
Note that the labor supply choice, given A; A 0 , is purely static.
That is, the level of employment and thus labor earnings has no dy-
namic aspect other than supplementing the resources available to
nance current consumption and future wealth. Correspondingly
the rst-order condition with respect to the level of employment
does not directly involve the value function, and it is given by
wUc c; n Un c; n: 6:14

Using c A wn A 0 =R, this rst-order condition relates n to
A; w; A 0 . Denote this relationship as n jA; w; A 0 . It can then be
substituted back into the dynamic programming problem yielding a
simpler functional equation:
vA; w max
0
ZA; A 0 ; w bEw 0 j w vA 0 ; w 0 ;
A
where
ZA; A 0 ; w 1 UA wjA; w; A 0 A 0 =R; jA; w; A 0 :

This simplied Bellman equation can be analyzed by standard
methods, thus ignoring the static labor supply decision.8 Once a
solution is found, the level of employment can then be determined
from the condition n jA; w; A 0 .
By a similar model MaCurdy (1981) studies the labor supply of
young men using the Panel Study on Income Dynamics (PSID). The
estimation of the model is done in several steps. First, the intraperiod
allocation (6.14) is estimated. The coefcients are then used to get at
the intertemporal part of the model.
To estimate the parameters of the utility function, one has to
observe hours of work and consumption, but in the PSID, total con-
sumption is not reported. To identify the model, MaCurdy uses a
8. This is similar to the trick we used in the stochastic growth model with endogenous
employment.
Consumption 155
utility function that is separable between consumption and labor

supply. The utility function is specied as uct ; nt g1t cto1 g2t nto2 ,
where g1t and g2t are two deterministic functions of observed char-
acteristics that might affect preferences such as age, education, and
number of children.
With this specication the marginal utility of leisure, Un c; n,
is independent of the consumption decision. Using (6.14), hours of
work can be expressed as
ln wt 1
lnnt ln Uc ct ; nt ln g2t ln o2 :
o2 1 o2 1
While the rst term in the right-hand-side is observed, the second

term contains the unobserved marginal utility of consumption.
Uc ct ; nt can be expressed as a function of the Lagrange multiplier
associated with the wealth constraint in period 0:
l0
Uc ct ; nt t :
b 1 r1 . . . 1 rt
The author treats the unobserved multiplier l 0 , as a xed effect
and uses panel data to estimate a subset of the parameters of the
utility function from rst differences. In a next step the xed effect is
backed out. At this point some additional identication assumptions
are needed. A specic functional form is assumed for the Lagrange
multiplier, written as a function of wages over the life cycle and ini-
tial wealth, all of them being unobserved in the data set. The author
uses then xed characteristics such that education or age to proxy
for the Lagrange multiplier. The author nds that a 10 percent in-
crease in the real wage induces a one to 5 percent increase in hours
worked.
Eichenbaum et al. (1988) analyze the time series properties of a
household model with both a savings and a labor supply decision.
They pay particular attention to specications in which preferences
are nonseparable, both across time and between consumption and
leisure contemporaneously. They estimate their model using GMM
on time series evidence on real consumption (excluding durables)
and hours worked. They nd support for nontime separability in
preferences, though in some cases they found little evidence against
the hypothesis that preferences were separable within a period.
156 Chapter 6
6.3.5 Borrowing Constraints
The Model and Policy Function

The extension of the two-period model with borrowing constraints
to the innite horizon case is discussed by Deaton (1991).9 One of the
key additional insights from extending the horizon is to note that
even if the borrowing constraint does not bind in a period, this does
not imply that consumption and savings take the same values as
they would in the problem without borrowing constraints. Simply
put, consumers anticipate that borrowing restrictions may bind in
the future (i.e., in other states), and this inuences their choices in
the current state.
Following Deaton (1991), let x A y represent cash on hand.
Then the transition equation for wealth implies that
A 0 Rx c;
where c is consumption. In the event that income variations are iid,

we can write the Bellman equation for the household as
vx max uc bEy 0 vRx c y 0 6:15

0acax
so that the return R is earned on the available resources less con-

sumption, x c. Note that income is not a state variable here as it is
assumed to be iid. Hence cash on hand completely summarizes the
resources available to the consumer.
The borrowing restriction takes the simple form of c a x so that the
consumer is unable to borrow. Of course, this is extreme and entirely
ad hoc, but it does allow us to explore the consequences of this
restriction. As argued by Deaton, the Euler equation for this problem
must satisfy
u 0 c maxfu 0 x; bREu 0 c 0 g: 6:16

Thus, either the borrowing restriction binds so that c x or it doesnt
so that the more familiar Euler equation holds. Only for low values
of x will u 0 x > bREu 0 c 0 , and only in these states, as argued for
the two-period problem, will the constraint bind. To emphasize an
important point: even if the u 0 x < bREu 0 c 0 so that the standard
condition of
9. See also Wright and Williams (1984) and Miranda and Helmberger (1988) for an early
contribution on this subject, including numerical solutions and simulations of these models.
Consumption 157
u 0 c bREu 0 c 0
holds, the actual state dependent levels of consumption may differ

from those that are optimal for the problem in which c is not
bounded above by x.
Alternatively, one might consider a restriction on wealth of the
form: A b Amin s where s is the state vector describing the house-
hold. In this case the household may borrow, but its assets are
bounded below. In principle, the limit on wealth may depend on the
state variables of the household: all else the same, a household with
a high level of income may be able to borrow more. One can look at
the implications of this type of constraint and, through estimation,
uncover Amin s. (see Adda and Eaton 1997).
To solve the optimal problem, one can use the value function iter-
ation approach, described in chapters 2 and 3, based on the Bellman
equation (6.15). Deaton (1991) uses another approach, working from
the Euler equation (6.16). The method is similar to the projection
methods presented in chapter 3, but the optimal consumption func-
tion is obtained by successive iterations instead of solving a system
of nonlinear equations. Although there is no formal proof that iter-
ations on the Euler equation actually converge to the optimal solu-
tion, the author note that empirically convergence always occur.
Figure 6.1 displays the optimal consumption rule in the case of a
serially correlated income. In this case the problem has two state
variablesthe cash on hand and the current realization of income
that provide information on future income. The policy rule has been
computed using a (coarse) grid with three points for the current
income and with 60 equally spaced points for the cash on hand.
When cash on hand is low, the consumer is constrained and is forced
to consume all his cash on hand. The policy rule is then the 45 degree
line. For higher values of the cash on hand, the consumer saves part
of the cash on hand for future consumption.
Figure 6.2 displays a simulation of consumption and assets
over 200 periods. The income follows an AR(1) process with uncon-
ditional mean of 100, a persistence of 0.5, and the innovations to
income are drawn from N0; 10. The path of income is asymmetric
as good income shocks are smoothed by savings, whereas the
liquidity constraints prevents the smoothing of low income realiza-
tions. Consumption is smoother than income, with a standard devi-
ation of 8.9 instead of 11.5.
158 Chapter 6
Figure 6.1
Consumption and liquidity constraints: Optimal consumption rule
Figure 6.2
Simulations of consumption and assets with serially correlated income
Consumption 159
Table 6.1
GMM estimation based on the Euler equation
Liquidity constrained
g periods g^GMM
0.5 80% 2.54
1 50 3.05
2 27 3.92
3 23 4.61
4 11 5.23
5 9 5.78
6 8 6.25
Note: r 0, s 10, m 100, b 0:9, r 0:05. Estimation done on 3,000 simulated
observations.
An Estimation Exercise
In section 6.3.3 we presented a GMM estimation by Hansen and
Singleton (1982) based on the Euler equation. Hansen and Singleton
(1982) nd a value for g of about 0.8. This is under the null that the
model is correctly specied, and in particular, that the Euler equa-
tion holds in each period. When liquidity constraints are binding, the
standard Euler equation does not hold. An estimation procedure that
does not take into account this fact would produce biased estimates.
Suppose that the real world is characterized by potentially binding
liquidity constraints. If one ignores them and consider a simpler
model without any constraints, how would it affect the estimation of
the parameter g?
To answer this question, we chose different values for g, solved the
model with liquidity constraints, and simulated it. The simulated
consumption series are used to get an estimate g^GMM such that
1X T
cg
t1
g^GMM arg min et g with et g b1 r 1:
g T t1 cg
t
The results are displayed in table 6.1. When g is low, the consumer
is less risk averse, consumes more out of the available cash on hand,
and saves less. The result is that the liquidity constraints are binding
more often. In this case the bias in the GMM estimate is the biggest.
The bias is decreasing in the proportion of liquidity-constrained
periods, as when liquidity constraints are almost absent, the stan-
dard Euler equation holds. From table 6.1 there is no value of g that
160 Chapter 6
would generate a GMM estimate of 0.8 as found by Hansen and

Singelton.
6.3.6 Consumption over the Life Cycle
Gourinchas and Parker (2002) investigate the ability of a model of

intertemporal choice with realistic income uncertainty to match
observed life cycle proles of consumption. (For a related study see
also Attanasio et al. 1999.) They parameterize a model of consump-
tion over the life cycle, which is solved numerically. The parameters
of the model are estimated using a simulated method of moments
method for data on household consumption over the life cycle. We
rst present a simplied version of their model. We then discuss the
numerical computation and the estimation methods.
Following Zeldes (1989a),10 the log income process is modeled as
a random walk with a moving average error. This specication is
similar to the one used in empirical work (see Abowd and Card
1989) and seems to t the data well. Denote Yt as the income of the
individual:
Yt Pt Ut ;
Pt Gt Pt1 Nt :
Income is the product of two components. Ut is a transitory shock

which is independently and identically distributed and takes a value
of 0 with a probability p and a positive value with a probability
1 p. Pt is a permanent component that grows at a rate Gt , which
depends on age. Nt is the innovation to the permanent component.
ln Nt and ln Ut , conditionally on Ut > 0, are normally distributed
with mean 0 and variance sn2 and su2 respectively. The consumer
faces a budget constraint
Wt1 1 rWt Yt C t :
The consumer can borrow and save freely. However, under the
assumption that there is a probability that income will be zero and
that the marginal utility of consumption is innite at zero, the con-
sumer will choose never to borrow against future income. Hence the
outcome of the model is close to the one proposed by Deaton (1991)
10. See also Carroll (1992).

Consumption 161
and presented in section 6.3.5. Note that in the model the agent can
only consume nondurables. The authors ignore the durable decision,
or equivalently assume that this decision is exogenous. This might
be a strong assumption. Fernandez-Villaverde and Krueger (2001)
argue that the joint dynamics of durables and nondurables are
important to understand the savings and consumption decisions
over the life cycle.
Dene the cash on hand as the total of assets and income:
Xt Wt Yt ; Xt1 RXt C t Yt1 :

Dene Vt Xt ; Pt as the value function at age T t. The value func-
tion is indexed by age as it is assumed that the consumer has a nite
life horizon. The value function depends on two state variables: the
cash on hand, which indicates the maximal limit that can be con-
sumed, and the realization of the permanent component, which pro-
vides information on future values of income. The program of the
agent is dened as
Vt Xt ; Pt maxuC t bEt Vt1 Xt1 ; Pt1 :
Ct
The optimal behavior is given by the Euler equation:
u 0 C t bREt u 0 Ct1 :
As income is assumed to be growing over time, cash on hand and
consumption are also nonstationary. This problem can be solved by
normalizing the variables by the permanent component. Denote
xt Xt =Pt and ct C t =Pt . The normalized cash on hand evolves as
R
xt1 xt ct Ut1 :
Gt1 Nt1
Under the assumption that the utility function is uc c1g =1 g,

the Euler equation can be rewritten with only stationary variables:
u 0 ct bREt u 0 ct1 Gt1 Nt1 :

As the horizon of the agent is nite, one has to postulate some ter-
minal condition for the consumption rule. It is taken to be linear in
the normalized cash on hand: cT1 g 0 g1 xT1 .
Gourinchas and Parker (2002) use this Euler equation to compute
numerically the optimal consumption rule. Normalized consumption
is only a function of the normalized cash on hand. By discretizing
162 Chapter 6
Figure 6.3
Optimal consumption rule
the cash on hand over a grid, the problem is solved recursively by

evaluating ct x at each point of the grid using
u 0 ct x

0 R
bR1 p u ct1 x ct U Gt1 N dFN dFU
Gt1 N

R
bRp u 0 ct1 x ct Gt1 N dFN:
Gt1 N
The rst term on the right-hand-side calculates the expected value of

the future marginal utility conditional on a zero income, while the
second term is the expectation conditional on a strictly positive
income. The integrals are solved by a quadrature method (see chap-
ter 3). The optimal consumption rules are obtained by minimizing
the distance between the left-hand side and the right-hand side.
Figure 6.3 displays the consumption rule at different ages.11
11. The gure was computed using the following parameterization: b 0:96, g 0:5,
su2 0:0212, sn2 0:044, p 0:03. g 0 0:0196, and g1 0:0533. We are grateful to Gour-
inchas and Parker for providing us with their codes and data.
Consumption 163
Figure 6.4
Observed and predicted consumption proles
Once the consumption rules are determined, the model can

be simulated to generate average life cycle proles of consumption.
This is done using the approximated consumption rules and by
averaging the simulated behavior of a large number of households.
The simulated proles are then compared to actual proles from U.S.
data. Figure 6.4 displays the predicted consumption prole for two
values of the intertemporal elasticity of substitution as well as the
observed consumption proles constructed from the U.S. Consumer
Expenditure Survey.12
More formally, the estimation method is the simulated method of
moments (see chapter 4). The authors minimize the distance between
observed and predicted consumption at different ages. As neither the
cash on hand nor the permanent component of income are directly
observed, the authors integrate out the state variables to calculate
the unconditional mean of (log) consumption at a given age:

ln C t y ln C t x; P; y dFt x; P; y;
12. See footnote 11 for the parameterization.

164 Chapter 6
where y is the vector of parameters characterizing the model and

where Ft is the density of the state variables for individuals of
age t. Characterizing this density is difcult as it has no closed form
solution. Hence the authors use simulations to approximate ln C t y.
Denote
1X It
1X S
gy ln Cit ln C t Xts ; Pts ; y:
It i1 S s1
The rst part is the average log consumption for households of age
t, and It is the number of observed household in the data set. The
second part is the average predicted consumption over S simulated
paths. y is estimated by minimizing
gy 0 Wgy;
where W is a weighting matrix.

The estimated model is then used to analyze the determinant of
savings. There are two reasons to accumulate savings in this model.
First, it cushions the agent from uninsurable income shocks, to avoid
facing a low marginal utility. Second, savings are used to nance
retirement consumption. Gourinchas and Parker (2002) show that
the precautionary motive dominates at least until age 40, whereas
older agents save mostly for retirement.
6.4 Conclusion
In this chapter we demonstrated how to use the approach of

dynamic programming to characterize the solution of the house-
holds optimal consumption problem and to link it with observations.
The discussion went beyond the savings decision to integrate it with
the labor supply and portfolio decisions.
As in other chapters, we gave numerous extensions that are
open-ended for the researcher to consider. (In the next chapter we
turn to one of these, durable goods.) Further, there are numerous
policy exercises that can be evaluated using an estimated model of
the household consumption choice, included a variety of policies
intended to inuence savings decisions.13
13. Rust and Phelan (1997) provide a good example in exploring the effects of social security
policies on labor supply and retirement decisions in a dynamic programming framework.
7 Durable Consumption
7.1 Motivation
So far the consumption goods we looked at are classied as either

nondurables or services. This should be clear since consumption
expenditures affected utility directly in the period of the purchase
and then disappeared.1 However, durable goods play a prominent
role in business cycles as durable expenditures are quite volatile.2
In this chapter we study two approaches to understanding durable
consumption. The rst is an extension of the models studied in the
previous chapter in which a representative agent accumulates dura-
bles to provide a ow of services. Here we present the results of
Mankiw (1982) which effectively rejects the representative agent
model.3
The second model introduces a nonconvexity into the households
optimization problem. The motivation for doing so is evidence that
households do not continuously adjust their stock of durables. We
explore this optimization problem through the specication and
estimation of a dynamic discrete choice model.
1. In a model of habit formation, past consumption can inuence current utility even
if the consumption is of a nondurable or a service. Then the state vector is supple-
mented to keep track of that experience. For durable goods we will supplement the
state vector to take the stock of durables into account.
2. According to Baxter (1996), the volatility of durable consumption is about ve times
that of nondurable consumption.
3. To be complete, as we explain, there are also maintained assumptions about pref-
erences, shocks, and the lack of adjustment costs.
166 Chapter 7
7.2 Permanent Income Hypothesis Model of Durable

Expenditures
We begin with a model that builds on the permanent income

hypothesis structure that we used in the previous chapter to study
nondurable expenditures. We rst provide the theoretical properties
of the model and then discuss its empirical implementation.
7.2.1 Theory
To model expenditures on both durable and nondurable goods, we

consider household behavior in which the consumer has a stock of
wealth A, a stock of durable goods D, and current income y. The
consumer uses wealth plus current income to nance expenditures
on current nondurable consumption c and to nance the purchase of
durable goods e at a relative price p.
There are two transition equations for this problem. One is the
accumulation equation for wealth given by
A 0 RA y c pe:
The accumulation equation for durables is similar to that used for

capital held by the business sector:
D 0 D1 d e; 7:1
where d A 0; 1 is the depreciation rate for the stock of durables.
Utility depends on the ow of services from the stock of durables
and the purchases of nondurables. In terms of timing, assume that
durables bought in the current period yield services starting in the
next period. So, as with capital, there is a time lag between the order
and the use of the durable good.4
With these details in mind, the Bellman equation for the house-
hold is given by
VA; D; y; p max
0 0
uc; D bEy 0 ; p 0 j y; p VA 0 ; D 0 ; y 0 ; p 0
D ;A
for all A; D; y; p 7:2

with
4. Of course, other possible assumptions on timing are implementable in this frame-

work as we show later in the chapter.
Durable Consumption 167
A0
cAy pD 0 1 dD 7:3
R
and the transition for the stock of durables given by (7.1). The maxi-
mization gives rise to two rst-order conditions:
uc c; D bREy 0 ; p 0 j y; p VA A 0 ; D 0 ; y 0 7:4
and
uc c; Dp bEy 0 ; p 0 j y; p VD A 0 ; D 0 ; y 0 :
In both cases these conditions can be interpreted as equating the

marginal costs of reducing either nondurable or durable consump-
tion in the current period with the marginal benets of increasing the
(respective) state variables in the next period.
Using the functional equation (7.2), we can solve for the deriva-
tives of the value function and then update these two rst-order
conditions. This implies that
uc c; D bREy 0 ; p 0 j y; p uc c 0 ; D 0 7:5
and
puc c; D bEy 0 ; p 0 j y; p uD c 0 ; D 0 p 0 1 duc c 0 ; D 0 : 7:6

The rst condition should be familiar from the optimal consump-
tion problem without durables. The marginal gain of increasing
consumption is offset by the reduction in wealth and thus consump-
tion in the following period. In this specication the marginal utility
of nondurable consumption may depend on the level of durables.
So, to the extent there is an interaction within the utility function
between nondurable and durable goods, empirical work that looks
solely at nondurable consumption may be inappropriate.5
The second rst-order condition compares the benets of buying
durables with the marginal costs. The benets of a durable expendi-
ture comes from two sources. First, increasing the stock of durables
has direct utility benets in the subsequent period. Second, as the
Euler equation characterizes a one-period deviation from a proposed
solution, the undepreciated part of the additional stock is sold and
consumed. This is reected by the second term on the right-hand
5. That is, movement in the marginal utility of consumption of nondurables may be

the consequence of variations in the stock of durables. We return to this point in the
discussion of empirical evidence.
168 Chapter 7
side. The marginal cost of the durable purchase is the reduction in

expenditures on nondurables that the agent must incur.
A slight variation in the problem assumes that durables purchased
in the current period provide services starting in that period. Since
this formulation is also found in the literature, we present it here as
well. In this case the dynamic programming problem is
VA; D; y; p max
0 0
uc; D 0 bEy 0 ; p 0 j y; p VA 0 ; D 0 ; y 0 ; p 0
D ;A
for all A; D; y; p; 7:7

with c dened as in (7.3).
Manipulation of the conditions for optimality implies (7.5) and
puc c; D 0 uD c; D 0 bEy 0 ; p 0 j y; p p 0 1 duc c 0 ; D 00 : 7:8
For constant prices p p 0 , the result is

uD c; D 0 bREy 0 j y uD c 0 ; D 00 :
The optimal condition corresponds to a variation where the stock of

durables is reduced by e in the current period, the resources are
saved and then used to purchase durables in the next period.6 As in
nondurable consumption, in the special case of bR 1, the marginal
utility from durables follows a random walk.
Note that regardless of the timing assumption, there are some
connections between the two Euler equations, particularly if utility
is not separable between durables and nondurables ucD 0 0. Also
shocks to income will inuence both durable and nondurable
expenditures.
7.2.2 Estimation of a Quadratic Utility Specication
Mankiw (1982) studied the pattern of durable expenditures when

uc; D 0 is separable and quadratic. In this case Mankiw nds that
durable expenditures follows an ARMA1; 1 process given by
et1 a0 a1 et et1 1 det ;

where a1 bR. Here the MA piece is parameterized by the rate of
depreciation. In estimating the model using U.S. data, Mankiw nds
6. This condition does not obtain under the previous timing because of the time to
build aspect of durables assumed there.
Table 7.1
ARMA1; 1 estimates on U.S. and French data
No trend Linear trend
Specication a1 d a1 d
U.S. durable expenditures 1.00 (0.03) 1.5 (0.15) 0.76 (0.12) 1.42 (0.17)
U.S. car registration 0.36 (0.29) 1.34 (0.30) 0.33 (0.30) 1.35 (0.31)
France durable expenditures 0.98 (0.04) 1.20 (0.2) 0.56 (0.24) 1.2 (0.36)
France car expenditures 0.97 (0.06) 1.3 (0.2) 0.49 (0.28) 1.20 (0.32)
France car registrations 0.85 (0.13) 1.00 (0.26) 0.41 (0.4) 1.20 (0.41)
Notes: Annual data. For the United States: FRED database, 1959:11997:3. For France:
INSEE, 1970:11997:2. U.S. registration: 19681995.
that empirically, d is quite close to 1. So durables may not be so

durable after all.
Adda and Cooper (2000b) study the robustness of Mankiws
results across different time periods, for different frequencies, and
across countries (United States and France). Their results are sum-
marized in table 7.1.
In the table the rows pertain to both aggregated durable expendi-
tures and estimates based on cars (for France, both total expendi-
tures on cars and new car registrations). The model is estimated with
and without a linear trend. The rate of depreciation is close to 100
percent each year for most of the specications. Mankiws puzzle
turns out to be robust across categories of durables, countries, time
periods, and the method of detrending.
Over the past few years there was considerable effort made in the
literature to understand Mankiws result. One interesting approach
was to embellish the basic representation agent model by the addi-
tion of adjustment costs and shocks other than variations in income.
Another, coming from Bar-Ilan and Blinder (1992) and Bertola and
Caballero (1990), was to recognize that at the household level dura-
ble expenditures are often discrete. We describe these two lines of
research in turn below.
7.2.3 Quadratic Adjustment Costs
Bernanke (1985) goes beyond the quadratic utility formulation by

adding price variations and costs of adjustment. As he observes, it
is worthwhile to look jointly at the behavior of durable and non-
170 Chapter 7
durable expenditures as well.7 Consider the dynamic optimization

problem
VA; D; y; p max
0 0
uc; D; D 0 bEy 0 j y VA 0 ; D 0 ; y 0 ; p 0
D ;A
for all A; D; y; p; 7:9
where the functional equation holds for all values of the state
vector. Bernanke assumes a quadratic utility function with quadratic
adjustment costs of the form
1 a d
uc; D; D 0 c c 2 D D 2 D 0 D 2 ;
2 2 2
where ct is nondurable consumption and Dt is the stock of durables.

The adjustment cost is part of the utility function rather than the
budget constraints, for tractability reasons. Because of the quadratic
structure, the model (7.9) can be solved explicitly as a (nonlinear)
function of the parameters. Current nondurable consumption is a
function of lagged nondurable consumption, of the current and
lagged stock of durables, and of the innovation to the income pro-
cess. Durables can be expressed as a function of the past stock of
durables and of the innovation to income. The two equations with an
equation describing the evolution of income are estimated jointly by
nonlinear three-stage least squares. Lagged measures of prices, non-
durable consumption, durable stocks and disposable income current
income, nondurable consumption and the stock of durables, were
used as instruments to control for simultaneity and measurement
error.
The model is rejected because overall it does not suit the data
when testing the overidentifying restrictions. The estimation of the
adjustment cost gives conicting results (as described in some detail
by Bernanke 1985). The nonlinear function of this parameter implies
a signicant adjustment cost, whereas the parameter itself is not sta-
tistically different from zero.
Bernanke (1984) tests the permanent hypothesis model at the
micro level by looking at car expenditures for a panel of households.
Although Bernanke does not reject the model for this type of data,
the result is at odds with real-life observations (described below) as
7. See also Eichenbaum and Hansen (1990).

it predicts continuous adjustment of the stock whereas car expendi-

tures are typically lumpy at the individual level.
exercise 7.1 Write a program to solve (7.9). Give decision rules
by the household. Use these decision rules to create a panel data
set, allowing households to have different realizations of income.
Consider estimating the Euler equations from the households opti-
mization problem. If there are nonseparabilities present in uc; D; D 0 ,
particularly, ucD 0 0, that were ignored by the researcher, what
incorrect inferences would be reached?
7.3 Nonconvex Adjustment Costs
The model explored in the previous section is intended to capture

the behavior of a representative agent. Despite its theoretical ele-
gance the model does not match two characteristics of the data. First,
as noted above, Mankiws estimate of close to 100 percent depre-
ciation suggests rejection of the model. Second, there is evidence at
the household level that adjustment of the stock of durables is not
Figure 7.1
s; S rule
172 Chapter 7
continuous. Rather, households purchases of some durables, such as

cars as studied by Lam (1991), are relatively infrequent. This may
reect irreversibility due to imperfect information about the quality
of used durable good, the discrete nature of some durable goods, or
the nature of adjustment costs.
Bar-Ilan and Blinder (1992) and Bar-Ilan and Blinder (1988)
present a simple setting in which a xed cost of adjustment implies
inaction from the agent when the stock of durable is not too far from
the optimal one. They argue that the optimal consumption of dura-
bles should follow an S; s policy. When the durable stock depreci-
ates to a lower value s, the agent increases the stock to a target value
S as depicted in gure 7.1.
7.3.1 General Setting
To gain some insight into the importance of irreversibility, consider

the following formalization of a model in which irreversibility is
important. By this we mean that because of friction in the market for
durables, households receive only a fraction of the true value of a
product they wish to sell. This might be thought of as a version of
Akerlofs famous lemons problem.8
In particular, suppose that the price of durables is normalized to 1
when they are purchases e but that the price of durables when they
are sold s is given by ps < 1. The Bellman equation for the house-
holds optimization problem is
VA; D; y maxV b A; D; y; V s A; D; y; V i A; D; y; 7:10
where

b A0
V A; D; y max0 u A y e; D
e; A R
bEy 0 j y VA 0 ; D1 d e; y 0 ; 7:11

A0
V s A; D; y max0 u A y ps s; D
s; A R
bEy 0 j y VA 0 ; D1 d s; y 0 ; 7:12
8. See House and Leahy (2000) for a model of durables with an endogenous lemons
premium.

i A0
V A; D; y max u A y ;D
A0 R
bEy 0 j y VA 0 ; D1 d; y 0 for all A; D; y: 7:13
This is admittedly a complex problem as it includes elements of

a discrete choice (to adjust or not) and also an intensive margin
(given adjustment, the level of durable purchases (sales) must be
determined).
The gap between the buying and selling price of durables will
create inaction. Imagine a household with a substantial stock of
durables that experiences an income loss due to a layoff. In the
absence of irreversibility ps 1, the household may optimally sell
off some durables. If a job is found and the income ow returns, then
the stock of durables can be rebuilt. However, in the presence of
irreversibility, the sale and subsequent purchase of durables is costly
due to the wedge between the buying and selling price of durables.
Thus in response to an income shock the household may be inactive
and not adjust its stock.
The functional equation in (7.10) cannot be solved by linearization
techniques as there is no simple Euler equation that treats the dis-
crete choice nature of the problem. For that reason value function
iteration techniques are needed. As in the dynamic discrete choice
problem specied in chapter 3, we would start with initial guesses of
the values of the three options and then induce VA; D; y through
the max operator. From these initial solutions, the iteration proce-
dure begins. As there is also an intensive margin in this problem
(given adjustment, the stock of durables one can choose is a contin-
uous variable), a state space for durables as well as assets must be
specied. This is a complex setting but one that the value function
iteration approach can handle.
So the policy functions can be created using a vector of parameters
that describes preferences and the stochastic processes. In principle,
in an estimation exercise, these parameters can generate moments
that are matched with observations. This method is described in
some detail, for a different model, in the subsequent subsections.
7.3.2 Irreversibility and Durable Purchases
Grossman and Laroque (1990) develop a model of durable consump-

tion and also consider an optimal portfolio choice. They assume that
174 Chapter 7
the durable good is illiquid as the agent incurs a proportional trans-

action cost when selling the good. The authors show that under the
assumption of a constant relative risk aversion utility function, the
state variable is the ratio of wealth A over the stock of durables D.
The optimal behavior of the agent is to follow an s; S rule, with a
target s A s; S. The agent does not change the stock of durable if the
ratio A=D is within the two bands s and S. If the ratio drifts out of
this interval, the agent adjusts it by buying or selling the good such
that A=D s .
Eberly (1994) empirically investigates the relevance of some
aspects of the Grossman-Laroque model. She uses data from the
Survey of Consumer Finances which reports information on assets,
income and major purchases. She estimates the bands s and S. These
bands can be computed by observing the ratio A=D for individuals
just before an adjustment is made. The target s can be computed as
the average ratio just after adjustment. Eberly (1994) estimates the
band width and investigates its determinants. She nds that the year
to year income variance and the income growth rate are strong pre-
dictors of the width of the band.
Attanasio (2000) develops a more elaborate estimation strategy for
these bands, allowing for unobserved heterogeneity at the individual
level. This heterogeneity is needed as, conditional on household
characteristics and the value of the ratio of wealth to consumption,
some are adjusting their stock and some are not. The estimation is
done by maximum likelihood on data drawn from the Consumer
Expenditure Survey. The width of the bands are functions of house-
hold characteristics such as age and race. The estimated model is
then aggregated to study the aggregate demand for durables.
Caballero (1993) uses the Grossman and Laroque (1990) approach
to investigate the aggregate behavior of durable goods. The individ-
ual agent is assumed to follow an s; S consumption rule because
of transaction costs. In the absence of transaction costs, the agent
would follow a PIH type behavior as described in section 7.2.
Caballero postulates that the optimal behavior of the agent can be
described by the distance between the stock of durables held by the
agent and the target dened as the optimal stock in the PIH
model. The agent adjusts the stock when the gap between the real-
ized and the desired stock is big enough. In this setting the state
variables are the stock of durables and the target. The target stock is
assumed to follow a known stochastic process. Hence in this model
it is assumed that the evolution of the target is a sufcient statistic

to inform of all the relevant economic variables such as prices or
income.
The aggregate demand for durables is the sum of all agents who
decide to adjust their stock in a given period. Hence Caballero
stresses the importance of the cross sectional distribution of the gap
between the target and the realized stock. When there is an aggre-
gate shock on the target, the aggregate response depends not only on
the size of the shock but also on the number of individuals close to
the adjustment line. The aggregate demand for durables can there-
fore display complicated dynamic patterns. The model is estimated
on aggregate U.S. data.
7.3.3 A Dynamic Discrete Choice Model
Suppose that instead of irreversibility, there is a restriction that

households can have either no car or one car.9 Thus, by assumption,
the household solves a dynamic discrete choice problem. We discuss
solutions of that problem, estimation of parameters and aggregate
implications in this section.10
Optimal Behavior
We start with the dynamic programming problem as specied in
Adda and Cooper (2000b). At the start of a period the household
has a car of a particular age, a level of income and a realization of a
taste shock. Formally, the households state is described by the age
of its car, i, a vector Z p; Y; e of aggregate variables, and a vector
z y of idiosyncratic variables. Here p is the relative price of the
(new) durable good. Current income is given by the sum Y y,
where Y represents aggregate income and y represents idiosyncratic
shocks to nondurable consumption that could reect variations in
household income or required expenditures on car maintenance and
other necessities.11 The nal element in the state vector is a taste
shock, e.
9. The assumption that one car is the max is just for convenience. What is important is
that the car choice set is not continuous.
10. This presentation relies heavily on Adda and Cooper (2000b).
11. Adda and Cooper (2000b) explicitly view this as a household specic income
shock, but a broader interpretation is acceptable, particularly in light of the iid as-
sumption associated with this source of variation.
176 Chapter 7
At every point in time the household decides whether to retain a

car of age i, trade it, or scrap it. If the household decides to scrap the
car, then it receives the scrap value of p and has the option to pur-
chase a new car. If the household retains the car, then it receives the
ow of services from that car and cannot, by assumption, purchase
another car. Thus the household is constrained to own at most a
single car.
Formally, let Vi z; Z represent the value of having a car of age i to
a household in state z; Z. Further let Vik z; Z and Vir z; Z represent
the values from keeping and replacing an age i car in state z; Z.
Then
Vi z; Z maxVik z; Z; Vir z; Z;
where
Vik z; Z usi ; y Y; e b1 dEVi1 z 0 ; Z 0
bdfEV1 z 0 ; Z 0 us1 ; y 0 Y 0 ; e 0
us1 ; y 0 Y 0 p 0 p; e 0 g 7:14
and
Vir z; Z us1 ; y Y p p; e b1 dEV2 z 0 ; Z 0

bdfEV1 z 0 ; Z 0 us1 ; y 0 Y 0 ; e 0
us1 ; y 0 Y 0 p 0 p; e 0 g:
In the denition of Vik z; Z, the car is assumed to be destroyed
(from accidents and breakdowns) with probability d leading the
agent to purchase a new car in the next period. The cost of a new car
in numeraire terms is p 0 p, which is stochastic since the price of a
new car in the next period is random. Further, since it is assumed
that there is no borrowing and lending, the utility cost of the new car
is given by us1 ; y 0 Y 0 ; e 0 us1 ; y 0 Y 0 p 0 p; e 0 which exceeds
p 0 p as long as u is strictly concave in nondurable consumption.
It is precisely at this point that the borrowing restriction appears as
an additional transactions cost.
Adding in either borrowing and lending or the purchase and sale
of used cars presents no modeling difculties. But adding in wealth
as well as resale prices as state variables certainly increases the
dimensionality of the problem. This remains as work in progress.
exercise 7.2 Reformulate (7.14) to allow the household to borrow/

lend and also to resell cars in a used car market. What additional
state variables would you have to add when these choices are in-
cluded? What are the new necessary conditions for optimal behavior
of the household?
Further Specication
For the application the utility function is dened to be additively
separable between durables and nondurables:
" #
g ec=l 1x
usi ; c i ;
1x
where c is the consumption of nondurable goods, g is the curvature

for the service ow of car ownership, x the curvature for con-
sumption, and l is a scale factor. In this specication the taste shock
e inuences the contemporaneous marginal rate of substitution
between car services and nondurables.
In order for the agents optimization problem to be solved, a sto-
chastic process for income, prices, and the aggregate taste shocks
must be specied. Aggregate income, prices, and the unobserved
preference shock are assumed to follow a VAR(1) process given
by12
Yt mY rYY Yt1 rYp pt1 uYt ;
pt m p rpY Yt1 rpp pt1 upt ;

et me reY Yt1 rep pt1 uet :
The covariance matrix of the innovations u fuYt ; upt ; uet g is

2 3
oY oYp 0
W 4 opY op 0 5:
0 0 oe
As the aggregate taste shock is unobserved, we impose a block

diagonal structure on the VAR, which enables us to identify all the
parameters involving prices and aggregate income in a simple rst-
step regression. This considerably reduces the number of parameters
12. Here only a single lag is assumed to economize on the state space of the agents
problem.
178 Chapter 7
to be estimated in the structural model. We allow prices and income

to depend on lagged income and lagged prices.13
The aggregate taste shock potentially depends on lagged prices
and income. The coefcients of this process along with oe are esti-
mated within the structural model. By allowing a positive correla-
tion between the aggregate taste shock and lagged prices, given that
prices are serially correlated, we can reconcile the model with the
fact that sales and prices are positively correlated in the data. This
allows us to better capture some additional dynamics of sales and
prices in the structural estimation. An alternative way would be to
model jointly the producer and consumer side of the economy, to get
an upward-slopping supply curve. However, solving for the equi-
librium is computationally very demanding.
Solving the Model

The model is solved by the value function iteration method. Starting
with an initial guess for Vi z; Z, the value function is updated by
backward iterations until convergence.
The policy functions that are generated from this optimization
problem are of an optimal stopping variety. That is, given the state
of the household, the car is scrapped and replaced if and only if
the car is older than a critical age. Letting hk zt ; Zt ; y represent the
probability that a car of age k is scrapped, the policy functions imply
that hk zt ; Zt ; y d if k < Jzt ; Zt ; y and hk zt ; Zt ; y 1 otherwise.
Here Jzt ; Zt ; y is the optimal scrapping age in state zt ; Zt when y is
the vector of parameters describing the economic environment.
In particular, for each value of the idiosyncratic shock z, there is an
optimal scrapping age. Integrating over all possible values of this
idiosyncratic shock produces an aggregate policy function that indi-
cates the fraction of cars of a given vintage being scrapped when the
state of the world is Zt :
13. As in Adda and Cooper (2000b), we assume that the costs of production are inde-
pendent of the level of production. Combined with an assumption of constant mark-
ups, this implies that the product price is independent of the cross-sectional
distribution of car vintages.
This assumption of an exogenous price process greatly simplies the empirical im-
plementation of the model, since we do not have to solve an equilibrium problem. We
have even found that adding information on the moments of the cross-sectional dis-
tribution of car vintages has no explanatory power in forecasting car prices in the
French case. Results are mixed for the U.S. case, as the average age of cars signicantly
predicts future prices.

Hk Zt ; y hk zt ; Zt ; yfzt dzt ;
where f is the density function of zt taken to be the normal distri-

bution. Hk is an increasing function of the vintage and bounded
between d and 1. The aggregated hazard can be used to predict
aggregate sales and the cross-sectional distribution of car vintages
over time. Letting ft k be the period t cross-sectional distribution of
k, aggregate sales are given by
X
St Zt ; y Hk Zt ; y ft k: 7:15
k
From an initial condition it is possible to generate a time series for

the cross-sectional distribution given a particular parameterization
of the hazard function. The evolution of ft k is given by
ft1 k; Zt ; y 1 Hk Zt ; y ft k 1 for k > 1 7:16
and
ft1 1; Zt ; y St Zt ; y:
Thus, for a given y and a given draw of T aggregate shocks, one can
simulate both sales and the cross-sectional distribution. This can be
repeated N times to produce N simulated data sets of length T,
which can be used in the estimation. Dene Stn Zt ; y St pt ; Yt ; ent ; y
as the predicted aggregate sales given prices, aggregate income and
PN
unobserved taste shock ent . Dene St Zt ; y 1=N n1 Snt Zt ; y as
the average aggregate sales conditional on prices, aggregate income
and period t 1 cross-sectional distribution.
Estimation Method and Results

In total there are eight parameters to estimate: y fg; d; l; z; sy ; reY ;
rec ; oe g. The estimation method follows Adda and Cooper (2000b)
and is a mix between simulated non-linear least squares and simu-
lated method of moments. The rst part of the criterion matches
predicted sales of new cars with the observed ones, conditional on
prices and aggregate income. The second part of the criterion
matches the predicted shape of the cross-sectional distribution of
car vintages to the observed one. The objective function to minimize
is written as the sum of the two criteria:
180 Chapter 7
LN y aLN1 y LN2 y;
where N is the number of simulated draws for the unobserved

aggregate taste shock ent . The two criteria are dened by
" #
1 1X T
2 1 XN
2
LN y St St y Stn y St y ;
T t1 NN 1 n1
X
LN2 y a i F i F i y 2 ;
if5; 10; 15; AR; MAg
where St y is the average F i , i 5; 10; 15 is the average fraction of

cars of age i across all periods, and F i , i AR; MA are the autore-
gressive and moving average coefcients from an ARMA1; 1 esti-
mated on aggregate sales.
The estimation uses two criteria for identication reasons. Match-
ing aggregate sales at each period extracts information on the effect
of prices and income on behavior and helps identify the parameter of
the utility function as well as the parameters describing the distri-
bution of the aggregate taste shock. However, the model is able to
match aggregate sales under different values for the agents optimal
stopping time. In other words, there can be different cross-sectional
distributions that produce aggregated sales close to the observed
ones. In particular, the parameter d is poorly identied by using only
the rst criterion. The second criterion pins down the shape of the
cross-sectional distribution of car vintages.
The data come from France and the United States and include
besides the cross-sectional distribution of car vintages over time, the
aggregate sales of new cars, prices, and aggregate income. The esti-
mated aggregate hazard functions Ht Z over the period 1972 to
1995 for France and 1981 to 1995 for the United States are shown in
gures 7.2 and 7.3. Note that the probability of replacement for
young cars, which is equal to the d, is estimated at a low value
between 5 to 10 percent. Hence, in contrast with the estimated PIH
models described in section 7.2, the model is able to produce a sen-
sible estimate of the rate of depreciation. Moreover, in estimating an
ARMA1; 1, as in section 7.2.2, on the predicted aggregate sales, we
nd that the MA coefcient is close to zero as in the observed data.
Hence, from a PIH perspective, the model appears to support a 100
percent depreciation rate at the aggregate level, but not at the micro
level where the depreciation rate is low.
Figure 7.2
Estimated hazard function, France
Figure 7.3
Estimated hazard function, United States
182 Chapter 7
Once the model is estimated, Adda and Cooper (2000b) investigate

the ability of the model to reproduce a number of other features such
as the impulse response of sales to an increase in prices. They also
use the estimated model to decompose the source of variation in
aggregate sales. Within the model there are two main sources, the
endogenous evolution of the cross-sectional distribution and the
effect of aggregate variables such as prices or income. Caballero
(1993) seems to imply that the evolution of the cross-sectional distri-
bution is an important determinant. However, the empirical decom-
position shows that its role is relatively minor, compared with the
effect of income and prices.
The Impact of Scrapping Subsidies

Adda and Cooper (2000a) uses the same framework to analyze the
impact of scrapping subsidies introduced rst in France and later in
a number of European countries such as Spain or Italy.
From February 1994 to June 1995 the French government offered
individuals 5,000 francs (approximately 5 to 10 percent of the value
of a new car) for the scrapping of an old car (ten years or older) and
Figure 7.4
Sales of new cars, in thousands, monthly
the purchase of a new car. Sales of new cars which had been low in
the preceding period (see gure 7.4) increased markedly during the
period the policy was in place. In September 1995 to September 1996,
the government re-introduced the policy, with an age limit of eight
years. After September 1996, the demand for new cars collapsed at a
record low level.
As is evident in gure 7.4, the demand for cars is cyclical and fol-
lows the business cycle. The increased demand for new cars during
the period 1994 to 1996 could be due either to the policy or to the
cyclical nature of demand. If the latter is true, the French govern-
ment was wasting money on car owners who would have replaced
their cars during that period anyway. Even if the increased demand
was entirely fueled by the scrapping subsidies, the government was
paying out money to car owners who would have replaced their car
in the periods ahead anyway. The effect of the policy was then to
anticipate new sales and create future, potentially bigger cycles of
car demand. As a huge number of new cars were sold in that period,
demand for new cars dropped when the policy was ended. How-
ever, a peak in demand is expected to appear in about ten years after
the cars bought in 1995 and 1996 are scrapped.
Adda and Cooper (2000a) estimate the model in section 7.3.3 for
the prepolicy period. The scrapping price p was a constant low value
(around 500 French francs) before 1993. With the policy in place, the
scrapping price increased and was age specic:
pi 500 if i < 10;
pi 5;000 if i b 10:
In the estimated model the effect of the policy can be simulated as

well as the counterfactual without the policy in place. This is done
State tomorrow
1 2 3 4
1 0.01 0.01 0.01 0.97
2 0.01 0.01 0.01 0.97

State
today
3 0.225 0.225 0.1 0.45
4 0.01 0.01 0.01 0.97
Figure 7.5
Transition matrix for p
184 Chapter 7
Figure 7.6
Expected aggregate sales, relative to baseline
Figure 7.7
Expected government revenue, relative to baseline
conditional on the cross-sectional distribution of cars at the begin-

ning of the period and conditional on the realized income and
prices, as the prices of new cars are assumed to be independent of
the policy. (This effect is debatable, however, for empirical evi-
dence suggests that prices remained stable throughout the period
mainly because the government negotiated a stable price with car
producers.)
While the rst scrapping subsidy was largely unexpected by con-
sumers, the second was somewhat anticipated, since after the rst
subsidy, there was discussion on whether to implement another such
subsidy. This is taken into account in the model by adding the
scrapping price pi as a stochastic state variable. More precisely, p is
assumed to follow a rst order Markov process with four states.
These four states are shown in gure 7.5. The rst state models the
1994 reform and the second one the 1995 reform. State 3 is a state
with heightened uncertainty, in which there are no subsidies. State 4
is the baseline state. In state 1, the scrap value is set at 5,500F for cars
older than ten years. This state is not assumed to be permanent:
there is only a 1 percent chance that the subsidy will be in effect in
the next period, conditional on being in force in the current period.
In state 2, the scrap value is also 5,500F but for cars older than eight
years old.
Figures 7.6 and 7.7 show the predicted sales and government rev-
enue relative to the baseline. The model captures the peak in sales
during the two policies, as well as the decline in between due to the
uncertainty. The sales are lower for about ten years, with little evi-
dence of a subsequent peak. This result is in line with that discussed
in section 7.3.3 where it was found that over time the cross-sectional
distribution had little effect on aggregate sales.
Government revenues are lower over the whole period. The gov-
ernment revenue is formed by the value-added taxes from the pur-
chase of new cars minus the paid-out scrapping subsidies. From the
perspective of government revenues, the policy is clearly undesirable.
The subsidies accounted for about 8 to 10 percent of the increased
demand in sales.
8 Investment
This chapter studies capital accumulation. Investment expenditures

are one of the most volatile elements of the aggregate economy.
From the perspective of policy intervention, investment is also a key
issue. The dependence of investment on real interest rates is critical
to many discussions of the impact of monetary policy. Further many
scal policy instruments, such as investment tax credits and accel-
erated depreciation allowances, act directly through their inuence
on capital accumulation.
It should seem then that macroeconomics would have developed
and evaluated numerous models to meet this challenge. Yet, relative
to the enormous work done on consumption, research on investment
lags behind. As noted in Caballero (1999), this has changed dramat-
ically in the last 10 or so years.1 Partly, we now have the ability to
characterize investment behavior in fairly rich settings. Combined
with plant-level data sets, researchers are able to confront a rich set
of observations with these sophisticated models.
Investment, with its emphasis on uncertainty and nonconvexities
is a ripe area for applications of dynamic programming techniques.
In this chapter we rst analyze a general dynamic optimization
problem and then focus on special cases of convex and nonconvex
adjustment costs. This sets the stage for the empirical analyzes that
follow. We also discuss the use of these estimates for the analysis of
policy interventions.
1. There are numerous surveys of investment. See Caballero (1999) and Chirinko
(1993), and the references therein, for further summaries of existing research.
188 Chapter 8
8.2 General Problem
The unit of analysis will be the plant though for some applications
(e.g., consideration of borrowing constraints) focusing on the rm
may be more appropriate. The manager is assumed to maximize
the value of the plant: there are no incentive problems between the
manager and the owners. The problem involves the choice of factors
of production that are rented for the production period, the hiring of
labor and the accumulation of capital. To focus on the investment
decision, we assume that demand for the variable inputs (denoted
by x) is optimally determined given factor prices (represented by the
vector w) and the state variables of the plants optimization problem,
represented by A; K. Here the vector of exible factors of produc-
tion might include labor, materials, and energy inputs into the pro-
duction process.
The result of this optimization leaves a prot function, denoted by
PA; K, that depends solely on the state of the plant, where
PA; K max RA^; K; x wx:

x
Here RA^; K; x denotes revenues given the inputs of capital (K ), the

variable factors (x), and a shock to revenues and/or productivity,
denoted by A^. The reduced form prot function thus depends on
the stochastic variable A that encompasses both A^ and w, and the
stock of physical capital (K). Thus we often refer to A as a prot-
ability shock since it reects variations in technology, demand and
factor prices.
Taking this prot function as given, we consider variations of the
following stationary dynamic programming problem:
VA; K; p max
0
PA; K CK 0 ; A; K pK 0 1 dK
K
bEA 0 ; p 0 j A; p VA 0 ; K 0 ; p 0 for all A; K; p; 8:1
where K 0 K1 d I is the capital accumulation equation and I is

investment. Here unprimed variables are current values and primed
variables refer to future values. In this problem the manager chooses
the level of the future capital stock denoted K 0 . The timing assump-
tion is that new investment becomes productive with a one-period
Investment 189
lag. The rate of depreciation of the capital stock is denoted by

d A 0; 1. The manager discounts the future at a xed rate of b.2
exercise 8.1 Suppose that in contrast to (8.1), investment in period
t is productive in that period. Compare these two formulations of the
investment problem. Assuming that all functions are differentiable,
create Euler equations for each specication. Explain any differences.
exercise 8.2 How would you modify (8.1) to allow the managers
discount factor to be inuenced by variations in the real interest
rate?
There are no borrowing restrictions in this framework. So the
choice of investment and thus future capital is not constrained by
current prots or retained earnings. We return to this issue later in
the chapter when we discuss the implications of capital market
imperfections.
There are two costs of obtaining new capital. The rst is the direct
purchase price, denoted by p. Notice that this price is part of the
state vector as it is a source of variation in this economy.3
Second, there are costs of adjustment given by the function
CK 0 ; A; K. These costs are assumed to be internal to the plant and
might include installation costs, disruption of productive activities in
the plant, the need to retrain workers, the need to recongure other
aspects of the production process, and so on. This function is general
enough to have components of both convex and nonconvex costs of
adjustment as well as a variety of transactions costs.
8.3 No Adjustment Costs
To make clear the contribution of adjustment costs, it is useful to

start with a benchmark case in which these costs are absent:
CK 0 ; A; K 1 0 for all K 0 ; A; K. Note, though, that there is still a time
to build aspect of investment so that capital accumulation remains
forward looking. The rst-order condition for the optimal invest-
ment policy is given by
2. This is corresponds to the outcome of a stochastic growth model if there are risk
neutral consumers. Otherwise, a formulation with variable real interest rates may be
warranted.
3. In many economies it is also inuenced by policy variations in the form of invest-
ment tax credits.
190 Chapter 8
bEA 0 ; p 0 j A; p Vk A 0 ; K 0 ; p 0 p; 8:2
where subscripts on the functions denote partial derivatives. This

condition implies that the optimal capital stock depends on the real-
ized value of protability, A, only through an expectations mecha-
nism: given the time to build, current protability is not relevant
for investment except as a signal of future protability. Further the
optimal capital stock does not depend on the current stock of capital.
Using (8.1) to solve for EA 0 ; p 0 j A; p Vk A 0 ; K 0 ; p 0 yields
bEA 0 ; p 0 j A; p Pk A 0 ; K 0 1 dp 0 p: 8:3
This condition has a natural interpretation. The cost of an additional
unit of capital today p is equated to the marginal return on capital.
This marginal return has two pieces: the marginal prots from the
capital, Pk A 0 ; K 0 , and the resale value of undepreciated capital at
the future price, 1 dp 0 .
Substituting for the future price of capital and iterating forward,
we nd that
X
y
pt b b1 d t EAtt j At PK Ktt1 ; Att1 ;
t0
where pt is the price of capital in period t. So the rms investment

policy equates the purchase price of capital today with the dis-
counted present value of marginal prots in the future. Note that
in stating this condition, we are assuming that the rm will be opti-
mally resetting its capital stock in the future so that (8.3) holds in all
subsequent periods.
While simple, the model without adjustment costs does not t
the data well. Cooper and Haltiwanger (2000) argue that relative
to observations, this model without adjustment costs implies exces-
sive sensitivity of investment to variations in protability. So one of
the empirical motivations for the introduction of adjustment costs
is to temper the otherwise excessively volatile movements in invest-
ment. Further this model is unable to match the observation of inac-
tion in capital adjustment seen (and discussed below) in plant-level
data. For these reasons various models of adjustment costs are
considered.4
4. Moreover the special case of no adjustment costs is generally nested in these other
models.
Investment 191
8.4 Convex Adjustment Costs
In this section we assume that CK 0 ; A; K is a strictly increas-

ing, strictly convex function of future capital, K 0 .5 The rm chooses
tomorrows capital K 0 using its conditional expectations of future
protability, A 0 . Of course, to the extent that A 0 is correlated with A,
current prots will be correlated with future prots.
Assuming that VK; A; p exists, an optimal policy, obtained by
solving the maximization problem in (8.1), must satisfy
CK 0 K 0 ; A; K p bEA 0 ; p 0 j A; p VK 0 A 0 ; K 0 ; p 0 : 8:4
The left side of this condition is a measure of the marginal cost of

capital accumulation and includes the direct cost of new capital as
well as the marginal adjustment cost. The right side of this expres-
sion measures the expected marginal gains of more capital through
the derivative of the value function. This is conventionally termed
marginal Q and denoted by q. Note the timing: the appropriate
measure of marginal Q is the expected discounted value for the fol-
lowing period due to the one-period investment delay.
Using (8.1) to solve for EA 0 ; p 0 j A; p VK 0 A 0 ; K 0 ; p 0 , we can simplify
(8.4) to a Euler equation:
CK 0 K 0 ; A; K p
bEA 0 ; p 0 j A; p fPK K 0 ; A 0 p 0 1 d CK 0 K 00 ; A 0 ; K 0 g: 8:5

To interpret this necessary condition for an optimal solution, con-
sider increasing current investment by a small amount. The cost of
this investment is measured on the left side of this expression: there
is the direct cost of the capital p as well as the marginal adjustment
cost. The gain comes in the following period. The additional capital
increases prots. Further, as the manager returns to the optimal
path following this deviation, the undepreciated capital is valued at
the future market price p 0 and adjustment costs are reduced.
exercise 8.3 Suppose that the problem had been written, more
conventionally, with the choice of investment rather than the future
capital stock. Derive and analyze the resulting Euler equation.
5. In some applications the cost of adjustment function depends on investment and is

written CI; K where I K 0 1 dK.
192 Chapter 8
8.4.1 Q Theory: Models
One of the difcult aspects of investment theory with adjustment

costs is empirical implementation. As the value function and hence
its derivative is not observable, (8.4) cannot be directly estimated.
Thus the theory is tested either by nding a suitable proxy for the
derivative of VA; K; p or by estimating the Euler equation, (8.5). We
focus here on the development of a theory that facilitates estimation
based on using the average value of the rm as a substitute for the
marginal value of an additional unit of capital.
This approach, called Q theory, imposes additional structure
on (8.1). In particular, following Hayashi (1982), we assume that
PK; A is proportional to K and that the cost of adjustment function
is quadratic.6 Further we assume that the price of capital is constant.
Therefore we have

g K 0 1 dK 2
VA; K max AK K
K0 2 K
pK 0 1 dK bEA 0 j A VA 0 ; K 0 : 8:6
As always, Bellmans equation must be true for all A; K. Suppose

that the shock to protability, A, follows an autoregressive process
given by
A 0 rA e 0 ;
where jrj < 1 and e 0 is white noise. The rst-order condition for the
choice of the investment level implies that the investment rate in
i 1 I=K is given by
1
i bEA 0 j A VK A 0 ; K 0 p: 8:7
g
Here EA 0 j A VK A 0 ; K 0 is again the expected value of the derivative

of the value function, a term we called marginal Q. To solve
this dynamic programming problem, we can guess at a solution and
verify that it works. Given the linear-quadratic structure of the
problem, it is natural to guess that
VA; K fAK;
6. Abel and Eberly (1994) contain further discussion of the applicability of Q theory
for more general adjustment cost and prot functions.
Investment 193
where fA is some unknown function. This guess allows us to write

the expected marginal Q as a function of A:
EA 0 j A VK A 0 ; K 0 EA 0 j A fA 0 1 f~A:
Note that in this case the expected values of marginal and average Q
(dened as VA; K=K fA) are the same.7 Using this value func-
tion in the Euler equation, we write
1
i bf~A p 1 zA:
g
This expression implies that the investment rate is independent of

the current level of the capital stock.
To verify our guess, we substitute this investment policy function
into the original functional equation, which implies that
g
fAK AK zA 2 K pzAK bf~AK1 d zA
2
must hold for all A; K. Clearly, the guess that the value function
is proportional to K is correct: the value of K cancels out. So, from
our conjecture that VA; K is proportional to K, we nd an optimal
investment policy that conrms the suggested proportionality.
The remaining part of the unknown value function fA is given
implicity by the expression above.8
The result that the value function is proportional to the stock of
capital is, at this point, a nice property of the linear-quadratic for-
mulation of the capital accumulation problem. In the discussion of
empirical evidence it forms the basis for a wide range of applica-
tions, since it allows the researcher to substitute the average value of
Q (observable from the stock market) for marginal Q (unobservable).
8.4.2 Q Theory: Evidence
Due to its relatively simple structure, the convex adjustment cost

model is one of the leading models of investment. In fact, as dis-
cussed above, the convex model is often simplied further so that
7. Hayashi (1982) was the rst to point out that in this case average and marginal Q
coincide, though his formulation was nonstochastic.
8. Interestingly the natural conjecture that fA A does not satisfy the functional
equation.
194 Chapter 8
adjustment costs are quadratic, as in (8.6). Necessary conditions for

optimality for this model are expressed in two ways.
First, from the rst-order conditions, the investment rate is linearly
related to the difference between the future marginal value of new
capital and the current price of capital, as in (8.7). Using the argu-
ments from above, this marginal value of capital can under some
conditions be replaced by the average value of capital. This sets the
basis for the Q theory empirical approach discussed below.
Second, one can base an empirical analysis on the Euler equation
that emerges from (8.6). It naturally leads to estimation using GMM
as discussed below.
The discussion of estimation based on Q theory draws heavily on
two papers. The rst by Gilchrist and Himmelberg (1995) provides
a clean and clear presentation of the basic approach and evidence
on Q theory based estimation of capital adjustment models. A theme
in this and related papers is that empirically investment depends on
variables other than average Q, particularly measures of cash ow.
The second by Cooper and Ejarque (2001) works from Gilchrist
and Himmelberg (1995) to explore the signicance of imperfect
competition and credit market frictions.9 This paper illustrates the
use of indirect inference.
Tests of Q theory on panel data are frequently conducted using an
empirical specication of
Xit
I=Kit ai0 a1 bEqit1 a2 uit : 8:8
K it
Here the i subscript refers to rm or plant i and the t subscript rep-

resents time. From (8.7), a1 should equal 1=g. This is an interesting
aspect of this specication: under the null hypothesis one can infer
the adjustment cost parameter from this regression. There is a con-
stant term in the regression that is plant specic. This comes from a
modication of the quadratic cost of adjustment to
2
0 g K 0 1 dK
CK ; K ai K
2 K
as in Gilchrist and Himmelberg (1995).10
9. We are grateful to Joao Ejarque for allowing us to use this material.

10. The error term in (8.8) is often ascribed to stochastic elements in the cost of
adjustment function; then ai is modied to become ait ai eit .
Investment 195
Finally, this regression includes a third term, Xit =K it . In fact Q

theory does not suggest the inclusion of other variables in (8.8), since
all relevant information is incorporated in average Q. Rather, these
variables are included as a means of testing the theory, where the
theory predicts that these variables from the information set should
be insignicant. Hence researchers focus on the statistical and eco-
nomic signicance of a2 . In particular, Xit often includes nancial
variables as a way of evaluating an alternative hypothesis in which
the effects of nancial constraints are not included in average Q.
The results obtained using this approach have been mixed. Esti-
mates of large adjustment costs are not uncommon. Hayashi (1982)
estimates a1 0:0423 and thus g of about 25. Gilchrist and Himmel-
berg (1995) estimate a1 at 0.033.
Further many studies estimate a positive value for a2 when Xit is a
measure of prots and/or cash ow.11 This is taken as a rejection of
the Q theory, which of course implies that the inference drawn about
g from the estimate of a1 may not be valid. Moreover the signicance
of the nancial variables has lead researchers to conclude that capital
market imperfections must be present.
Cooper and Ejarque (2001) argue that the apparent failure of
Q theory stems from misspecication of the rms optimization
problem: market power is ignored. As shown by Hayashi (1982), if
rms have market power, then average and marginal Q diverge.
Consequently the substitution of marginal for average Q in the stan-
dard investment regression induces measurement error that may
be positively correlated with prots.12 Cooper and Ejarque (2001)
ask whether one might nd positive and signicant a2 in (8.8) in a
model without any capital market imperfections.
Their methodology follows the indirect inference procedures
described in Gourieroux and Monfort (1996) and Gourieroux et al.
(1993). This approach to estimation was discussed in chapter 4. This
is a minimum distance estimation routine in which the structural
parameters of the optimization problem are chosen to bring the
reduced form coefcients from the regression on the simulated data
close to those from the actual data. The key is that the same reduced
form regression is run on both the actual and simulated data.
11. Hubbard (1994) reviews these ndings.

12. Cooper and Ejarque (2001) do not attempt to characterize this measurement error
analytically but use their simulated environment to understand its implications. See
Erickson and Whited (2000) for a detailed and precise discussion of the signicance of
measurement error in the Q regressions.
196 Chapter 8
Cooper and Ejarque (2001) use the parameter estimates of

Gilchrist and Himmelberg (1995) for (8.8) as representative of the
Q theory based investment literature. Denote these estimates from
their pooled panel sample using the average (Tobins) Q measure by
a1 ; a2 0:03; 0:24.13 Cooper and Ejarque (2001) add three other
moments reported by Gilchrist and Himmelberg (1995): the serial
correlation of investment rates (0.4), the standard deviation of prot
rates (0.3), and the average value of average Q (3). Let C d denote the
vector moments from the data. In the Cooper and Ejarque (2001)
study,
C d 0:03 0:24 0:4 0:3 3:

The estimation focuses on two key parameters: the curvature of
the prot function a and the level of the adjustment costs g. So
other parameters are set at levels found in previous studies: d 0:15
and b 0:95. This leaves a; g and the stochastic process for the
rm-specic shocks to protability as the parameters remaining to
be estimated. Cooper and Ejarque (2001) estimate the serial correla-
tion r and the standard deviation s of the protability shocks
while the aggregate shock process is represented process as a two-
state Markov process with a symmetric transition matrix in which
the probability of remaining in either of the two aggregate states is
0.8.14
As described in chapter 4, the indirect inference procedure pro-
ceeds, in this application, as follows:
0
Given a vector of parameters, Y 1 a; g; r; s, solve the rms
dynamic programming problem of

g K 0 1 dK 2
VA; K max AK a
K pK 0 1 dK
K0 2 K
bEA 0 j A VA 0 ; K 0 for all A; K 8:9
using value function iteration. The method outlined in Tauchen

(1986) is used to create a discrete state space representation of the
13. Cooper and Ejarque (2001) have no unobserved heterogeneity in the model so that
the constant from the regression as well as the xed effects are ignored. The remaining
coefcients are taken to be common across all rms.
14. The estimates are not sensitive to aggregate shocks. The model is essentially esti-
mated from the rich cross-sectional variation as in the panel study of Gilchrist and
Himmelberg (1995).
Investment 197
Table 8.1
Estimated structural parameters
Structural parameters
a g r s y
GH95
CE 0.689 (0.011) 0.149 (0.016) 0.106 (0.008) 0.855 (0.04) 2
Table 8.2
Regression results and moments
Reduced form coefcient estimates/moments
I p
a1 a2 sc std q
K K
GH95 0.03 0.24 0.4 0.25 3
CE 0.041 0.237 0.027 0.251 2.95
shock process given r; s. Use this in the conditional expectation of

the optimization.
0
Given the policy functions obtained by solving the dynamic pro-
gramming problem, create a panel data set by simulation.
0
Estimate the Q theory model, as in (8.8), on the simulated model,
and calculate relevant moments. Let C s Y denote the corresponding
moments from the simulated data.
0
Compute JY dened as
JY C d C s Y 0 WC d C s Y; 8:10
where W is an estimate of the inverse of the variance-covariance

matrix of C d .
0 ^ , that solves
Find the estimator of Y, Y
min JY:
Y
The second row of table 8.1 presents the estimates of struc-

tural parameters and standard errors reported in Cooper and Ejar-
que (2001).15 Table 8.2 reports the resulting regression results and
15. The computation of standard errors follows the description in chapter 4 of Gour-
ieroux and Monfort (1996).
198 Chapter 8
moments. Here the row labeled GH95 represents the regression

results and moments reported by Gilchrist and Himmelberg (1995).
The model, with its four parameters, does a good job of matching
four of the ve estimates/moments but is unable to reproduce the
high level of serial correlation in plant-level investment rates. This
appears to be a consequence of the fairly low level of g which implies
that adjustment costs are not very large. Raising the adjustment costs
will increase the serial correlation of investment.
The estimated curvature of the prot function of 0.689 implies a
markup of about 15 percent.16 This estimate of a, and hence the
markup, is not at variance with results reported in the literature.
The other interesting parameter is the estimate of the level asso-
ciated with the quadratic cost of adjustment, g. Relative to other
studies, this appears quite low.
However, an interesting point from these results is that the esti-
mate of g is not identied from the regression coefcient on average
Q. From table 8.1, the estimated value of g 0:149 is far from the
inverse of the coefcient on average Q (about 4). So clearly the iden-
tication of the quadratic cost of adjustment parameter from a2 is
misleading in the presence of market power.
exercise 8.4 Write a progam to solve

a g K 0 1 dK 2
VA; K max AK K
K0 2 K
pK 0 1 dK bEA 0 j A VA 0 ; K 0 8:11
using a value function iteration routine given a parameterization of

the problem. Use the results to explore the relationship of investment
to average Q. Is there a nonlinearity in this relationship? How is
investment related to protability in your simulated data set?
8.4.3 Euler Equation Estimation
This approach to estimation shares with the consumption applica-

tions presented in chapter 6 a simple but powerful logic. The Euler
16. Cooper and Ejarque (2001) show that if p yh is the demand curve and
y Ak f l1f the production function, maximization of prot over the exible factor l
leads to a reduced form prot function where the exponent on capital is fh 1=
1 f1 h 1. Here f 0:33 and h 0:1315, implying a markup of about 15
percent.
Investment 199
equation given in (8.5) is a necessary condition for optimality. In the

quadratic cost of adjustment model case this simplies to

1 g 2
it b Et pK At1 ; Kt1 pt1 1 d it1 g1 dit1 pt :
g 2
Let et1 be dened from realized values of these variables:

1
et1 it b pK At1 ; Kt1 pt1 1 d
g

g 2
it1 g1 dit1 pt : 8:12
2
Then the restriction imposed by the theory is that Et et1 0. It

is precisely this orthogonality condition that the GMM procedure
exploits in the estimation of underlying structural parameters, y
b; g; d; a.
To illustrate, we have solved and simulated a model with qua-
dratic adjustment costs g 2 with constant investment good prices.
That data set allows us to estimate the parameters of the rms
problem using GMM.
To make this as transparent as possible, assume that the researcher
knows the values of all parameters except for g. Thus we can rely on
a single orthogonality condition to determine g. Suppose that we use
the lagged protability shock as the instrument. Dene
1X
Wg et1 gAt : 8:13
T t
The GMM estimate of g is obtained from the minimization of Wg.

This function is shown in gure 8.1. Clearly, this function is mini-
mized near g 2.17
Whited (1998) contains a thorough review and analysis of existing
evidence on Euler equation estimation of investment models. As
Whited notes, the Euler equation approach certainly has a virtue
over the Q theory based model: there is no need to try to measure
17. The program to estimate this model is very simple. Once Wg is programmed, it is
simply a basic routine to minimize this function. Obtaining Wg is easy too, using the
information on parameters plus observations in the data set on investment rates and
the ratio of output to capital (to determine marginal prot rates). The minimization
may not occur exactly at g 2 because of a sampling error. The interested reader can
extend this analysis to create a distribution of estimates by redrawing shocks, simu-
lating, and then re-estimating g from the GMM procedure.
200 Chapter 8
Figure 8.1
Function Wg
marginal Q. Thus some of the restrictions imposed on the estima-

tion, such as the conditions specied by Hayashi, do not have to be
imposed. Estimation based on an investment Euler equation gener-
ally leads to rejection of the overidentifying restrictions, and as in
the Q theory based empirical work, the inclusion of nancial con-
straints improves the performance of the model.
The point of Whited (1998) is to dig further into these results.
Importantly, her analysis brings xed adjustment costs into the
evaluation of the Euler equation estimation. As noted earlier and
discussed at some length below, investment studies have been
broadened to go beyond convex adjustment costs to match the
observations of nonadjustment in the capital stock. Whited (1998)
takes this into account by dividing her sample into the set of rms
that undertakes positive investment. Estimation of the Euler equa-
tion for this subset is much more successful. Further Whited (1998)
nds that while nancial variables are important overall, they are
also weakly relevant for the rms with ongoing investment.
Investment 201
These results are provocative. They force us to think jointly about

the presence of nonconvex adjustment costs and nancial variables.
We now turn to these important topics.
8.4.4 Borrowing Restrictions
Thus far we have ignored the potential presence of borrowing

restrictions. These have a long history in empirical investment anal-
ysis. As in our discussion of the empirical Q theory literature, nan-
cial frictions are often viewed as the source of the signicance of
prot rates and/or cash ow in investment regressions.
There is nothing particularly difcult about introducing borrowing
restrictions into the capital accumulation problem. Consider

a g K 0 1 dK 2
VA; K max AK K 8:14
K 0 A GA; K 2 K
pK 0 1 dK bEA 0 j A VA 0 ; K 0 for all A; K; 8:15

where GA; K constrains the choice set for the future capital stock.
For example, if capital purchases had to be nanced out of current
prots, then the nancial restriction is
K 0 1 dK a AK a 8:16
so that
GA; K 0; AK a 1 dK: 8:17
The dynamic optimization problem with a restriction of (8.17) can

certainly be evaluated using value function iteration techniques. The
problem of the rm can be broadened to include retained earnings as
a state variable and to include other nancial variables in the state
vector. There are a number of unresolved issues though that have
limited research in this area:
0
What are the GA; K functions suggested by theory?
0
For what GA; K functions is there a wedge between average and
marginal Q?
The rst point is worthy of note: while we have many models of
capital accumulation without borrowing restrictions, the alternative
model of investment with borrowing restrictions is not on the table.
Thus the rejection of the model without constraints in favor of one
with constraints is not as convincing as it could be.
202 Chapter 8
Table 8.3
Descriptive statistics, LRD
Variable LRD
Average investment rate 12.2%

Inaction rate: Investment 8.1
Fraction of observations with negative 10.4
investment
Spike rate: Positive investment 18
Spike rate: Negative investment 1.4
The second point, related to work by Chirinko (1993) and Gomes

(2001), returns to the evidence discussed earlier on Q theory based
empirical models of investment. The value function VA; K that
solves (8.15) contains all the information about the constrained opti-
mization problem. As long as this function is differentiable (which
restricts the GA; K function), marginal Q will still measure the
return to an extra unit of capital. The issue is whether the borrowing
friction introduces a wedge between marginal and average Q.18
Empirically the issue is whether this wedge between marginal and
average Q can create the regression results such as those reported in
Gilchrist and Himmelberg (1995).
8.5 Nonconvex Adjustment: Theory
Empirically one nds that at the plant level there are frequent peri-
ods of investment inactivity and also bursts of investment activity.
Table 8.3, taken from Cooper and Haltiwanger (2000), documents
the nature of capital adjustment in the Longitudinal Research Data-
base (LRD), a plant-level U.S. manufacturing data set.19
Here inaction is dened as a plant-level investment rate less
than 0.01 and a spike is an investment rate in excess of 20 percent.
Clearly, the data exhibit both inaction as well as large bursts of
investment.
As argued by Caballero et al. (1995), Cooper et al. (1999), and
Cooper and Haltiwanger (2000), it is difcult to match this type of
evidence with a quadratic cost of adjustment model. Thus we turn to
alternative models which can produce inaction. In the rst type of
18. If, in the example above, a 1, then the constraint is proportional to K. In this case
it appears that average and marginal Q are equal.
19. Cooper and Haltiwanger provide a full description of the data.
Investment 203
model we relax the convex adjustment cost structure and assume

that the costs of adjustment depend only on whether investment has
been undertaken, and not its magnitude. We then consider a second
type of model in which there is some type of irreversibility. The next
section reports on estimation of these models.
8.5.1 Nonconvex Adjustment Costs
For this formulation of adjustment costs, we follow Cooper and

Haltiwanger (1993) and Cooper et al. (1999) and consider a dynamic
programming problem specied at the plant level as
VA; K; p maxfV i A; K; p; V a A; K; pg for all A; K; p; 8:18
where the superscripts refer to active investment a and inactivity i.

These options, in turn, are dened by
V i A; K; p PA; K bEA 0 ; p 0 j A; p VA 0 ; K1 d; p 0
and
V a A; K; p max
0
PA; Kl FK pK 0 1 dK
K
bEA 0 ; p 0 j A; p VA 0 ; K 0 ; p 0 :
Here there are two costs of adjustment that are independent of the
level of investment activity. The rst is a loss of prot ow equal to
1 l. This is intended to capture an opportunity cost of investment
in which the plant must be shut down during a period of investment
activity. The second nonconvex cost is simply subtracted from the
ow of prots as FK. The inclusion of K here is intended to capture
the idea that these xed costs, while independent of the current level
of investment activity, may have some scale aspects to them.20 In
this formulation the relative price of capital p is allowed to vary as
well.
Before proceeding to a discussion of results, it might be useful to
recall from chapter 3 how one might obtain a solution to a problem
20. See Abel and Eberly (1994) for a model in which xed costs are proportional to K.
If these costs were independent of size, then large plants would face lower adjustment
costs (relative to their capital stock) and thus might adjust more frequently. So, as
in the quadratic specication, the costs are scaled by size. This is nevertheless an
assumption, and the relationship between plant size and investment activity is still
an open issue.
204 Chapter 8
such as (8.18).21 The rst step is to specify a prot function, say

PA; K AK a , for which we set the parameters F; b; l; a; d as well
as the stochastic processes for the random variables A; p. Denote
this parameter vector by Y. The second step is to specify a space for
the state variables A; K; p and thus for control variable K 0 . Once
these steps are complete, the value function iteration logic (sub-
scripts denote iterations of the mapping) takes over:
0
Provide an initial guess for V1 A; K; p, such as the one-period
solution.
0
Using this initial guess, compute the values for the two options,
V1a A; K; p and V1i A; K; p.
0
Using these values, solve for the next guess of the value function:
V2 A; K; p maxfV1a A; K; p; V1i A; K; pg.
0
Continue this process until convergence.
0
After the value function is found, compute the set of state variables
such that the action (inaction) is optimal and that the investment
level in the event adjustment is optimal.
0
From these policy functions, simulate the model and create either a
panel or a time series data set.
The policy function for this problem will have two important
dimensions. First, there is the determination of whether the plant
will adjust its capital stock or not. Second, conditional on adjust-
ment, the plant must determine its level of investment. As usual, the
optimal choice of investment depends on the marginal value of cap-
ital in the next period. However, in contrast to, say, the quadratic
cost of adjustment model, the future value of additional capital
depends on future choice with respect to adjustment. Thus there is
no simple Euler equation linking the marginal cost of additional
capital today with future marginal benet, as in (8.5), since there is
no guarantee that this plant will be adjusting its capital stock in the
next period.
Note that the two types of costs have very different implications
for the cyclical properties of investment. In particular, when adjust-
ment costs interfere with the ow of prots l < 1, then it is more
expensive to invest in periods of high protability. Yet, if the shocks
21. Recall the outline of the basic value function iteration program for the non-
stochastic growth model and the modication of that for nonconvex adjustment costs
in chapter 3.
Investment 205
are sufciently correlated, there is a gain to investing in good times.

In contrast, if costs are largely lump sum, then given the time-to-
build aspect of the accumulation decision, the best time to invest is
when it is protable to do so (A is high) assuming that these shocks
are serially correlated. Thus whether investment is procyclical or
countercyclical depends on both the nature of the adjustment costs
and the persistence of shocks.
We will discuss the policy functions for an estimated version
of this model later. For now we look at a simple example to build
intuition.
Machine Replacement Example

For an example of a simple model of machine replacement, we turn
to a modied version studied by Cooper and Haltiwanger (1993).
Here there is no choice of the size of the investment expenditure.
Investment means the purchase of a new machine at a net price of p.
By assumption, the old machine is scrapped. The size of the new
machine is normalized to 1.22
To further simplify the argument, we assume that new capital
becomes productive immediately. In addition the price of new capi-
tal good is assumed to be constant and can be interpreted as includ-
ing the xed cost of adjusting the capital stock. In this case we can
write the Bellman equation as
VA; K maxfV i A; K; V a A; Kg for all A; K;

where the superscripts refer to active investment a and inactivity i.
These options, in turn, are dened by
V i A; K PA; K bEA 0 j A VA 0 ; K1 d
and
V a A; K PA; 1l p bEA 0 j A VA 0 ; 1 d:
Here action means that a new machine is bought and is immedi-
ately productive. The cost of this is the net price of the new capital
and the disruption caused by the adjustment process. Let DA; K be
the relative gains to action, so
22. As discussed by Cooper and Haltiwanger (1993) and Cooper et al. (1999), this
assumption that a new machine has xed size can be derived from a model with
embodied technological progress that is rendered stationary by dividing through by
the productivity of the new machine. In this case the rate of depreciation measures
both physical deterioration and obsolescence.
206 Chapter 8
DA; K 1 V a A; K V i A; K PA; 1l PA; K p
bEA 0 j A VA 0 ; 1 d EA 0 j A VA 0 ; K1 d:
The problem posed in this fashion is clearly one of the optimal
stopping variety. Given the state of protability A, there is a criti-
cal size of the capital stock K A such that machine replacement
occurs if and only if K < K A. To see why this policy is optimal,
note that by our timing assumption, V a A; K is in fact independent
of K. Clearly, V i A; K is increasing in K. Thus there is a unique
crossing of these two functions at K A. In other words, DA; K is
decreasing in K, given A with DA; K A 0.
Is K between 0 and 1? With PA; 0 sufciently small, V i A; K <
V A; K for K near 0. Hence K > 0. Further, with the costs of
a
acquiring new capital p > 0; l < 1 large enough and the rate of
depreciation low enough, capital will not be replaced each period:
K < 1. Thus there will be a replacement cycle in which there is a
burst of investment activity followed by inactivity until the capital
ages enough to warrant replacement.23
The policy function is then given by zA; K A f0; 1g, where
zA; K 0 means inaction and zA; K 1 means replacement.
From the argument above, for each A there exists K A such that
zA; K 1 if and only if K a K A.
With the assumption that capital becomes productively immedi-
ately, the response of K A to variations in A can be analyzed.24
Suppose, for example, that l 1 and A is iid. In this case the depen-
dence of DA; K on A is solely through current prots. Thus DA; K
is increasing in A as long as the marginal productivity of capital is
increasing in A, PAK A; K > 0. So K A will be increasing in A and
replacement will be more likely in good times.
Alternatively, suppose that l < 1. In this case, during periods of
high productivity, it is desirable to have new capital, but it is also
costly to install it. If A is positively serially correlated, then the effect
of A on DA; K will reect both the direct effect on current prots
and the effects on the future values. If the opportunity cost is large
(a small l) and shocks are not persistent enough, then machine
replacement will be delayed until capital is less productive.
23. Cooper and Haltiwanger (2000) and Cooper et al. (1999) argue that these features
also hold when there is a one-period lag in the installation process.
24. Cooper et al. (1999) analyze the more complicated case of a one-period lag in the
installation of new capital.
Investment 207
Aggregate Implications of Machine Replacement

This model of capital adjustment at the plant level can be used to
generate aggregate implications. Let ft K be the current distribution
of capital across a xed population of plants. Suppose that the shock
in period t, At , has two components, At at et . The rst is aggregate
and the second is plant specic. Following Cooper et al. (1999),
assume that the aggregate shock takes on two values and the plant
specic shock takes on twenty values. Further assume that the idio-
syncratic shocks are iid. With this decomposition, write the policy
function as zat ; et ; Kt , where zat ; et ; Kt 1 signies action and
zat ; et ; Kt 0 indicates inaction. Clearly, the decision on replace-
ment will generally depend differentially on the two types of shocks,
since they may be drawn from different stochastic properties. For
example, if the aggregate shock is more persistent than the plant-
specic one, the response to a variation in at will be larger than the
response to an innovation in et .
Dene

Hat ; K zat ; et ; K dGt e;
e
where Gt e is the period t cumulative distribution function of the

plant-specic shocks. Here Hat ; K is a hazard function representing
the probability of adjustment for all plants with capital K in aggre-
gate state at . To the extent that the researcher may be able to observe
aggregate but not plant-specic shocks, Hat ; K represents a hazard
that averages over the f0; 1g choices of the individual plants so that
Hat ; K A 0; 1.
Using this formulation, let Iat ; ft K be the rate of investment in
state at given the distribution of capital holdings ft K across plants.
Aggregate investment is dened as
X
Iat ; ft K Hat ; K ft K: 8:19
K
Thus total investment reects the interaction between the average

adjustment hazard and the cross-sectional distribution of capital
holdings.
The evolution of the cross-sectional distribution of capital is given
by
gt1 1 dK 1 Hat ; Kgt K: 8:20

208 Chapter 8
Expressions such as these are common in aggregate models of dis-

crete adjustment; see, for example, Rust (1985) and Caballero et al.
(1995). Given an initial cross-sectional distribution and a hazard
function, a sequence of shocks will thus generate a sequence of
aggregate investment levels from (8.19) and a sequence of cross-
sectional distributions from (8.20).
Thus the machine replacement problem can generate a panel data
set and, through aggregation, time series as well. In principle, esti-
mation from aggregate data supplements the perhaps more direct
route of estimating a model such as this from a panel.
exercise 8.5 Use a value function iteration routine to solve the

dynamic optimization problem with a rm when there are non-
convex adjustment costs. Suppose that there is a panel of such rms.
Use the resulting policy functions to simulate the time series of
aggregate investment. Then use a value function iteration routine to
solve the dynamic optimization problem with a rm when there are
quadratic adjustment costs. Create a time series from the simulated
panel. How well can a quadratic adjustment cost model approximate
the aggregate investment time series created by the model with
nonconvex adjustment costs?
8.5.2 Irreversibility
The specications considered thus far do not distinguish between

the buying and selling prices of capital. However, there are good
reasons to think that investment is at least partially irreversible so
that the selling price of a unit of used capital is less than the cost of a
unit of new capital. This reects frictions in the market for used
capital as well as specic aspects of capital equipment that may
make them imperfectly suitable for uses at other production sites. To
allow for this, we alter our optimization problem to distinguish the
buying and selling prices of capital.
The value function for this specication is given by
VA; K maxfV b A; K; V s A; K; V i A; Kg for all A; K;
where the superscripts refer to the act of buying capital b, selling
capital s and inaction i. These options, in turn, are dened by
V b A; K max PA; K I bEA 0 j A VA 0 ; K1 d I;
I
Investment 209
V s A; K max PA; K ps R bEA 0 j A VA 0 ; K1 d R;

R
and
V i A; K PA; K bEA 0 j A VA 0 ; K1 d:
Under the buy option, the plant obtains capital at a cost normal-
ized to one. Under the sell option, the plant retires R units of capital
at a price ps . The third option is inaction, so the capital stock depre-
ciates at a rate of d. Intuitively the gap between the buying and sell-
ing price of capital will produce inaction. Suppose that there is an
adverse shock to the protability of the plant. If this shock was
known to be temporary, then selling capital and repurchasing it
in the near future would not be protable for the plant as long as
ps < 1. Thus inaction may be optimal. Clearly, though, the amount
of inaction that this model can produce will depend on both the size
of ps relative to 1 and the serial correlation of the shocks.25
8.6 Estimation of a Rich Model of Adjustment Costs
Using this dynamic programming structure to understand the opti-

mal capital decision at the plant (rm) level, we confront the data
on investment decisions allowing for a rich structure of adjustment
costs.26 To do so, we follow Cooper and Haltiwanger (2000) and
consider a model with quadratic adjustment costs, nonconvex ad-
justment costs and irreversibility. We describe the optimization
problem and then the estimation results obtained by Cooper and
Haltiwanger.
8.6.1 General Model
The dynamic programming problem for a plant is given by
VA; K maxfV b A; K; V s A; K; V i A; Kg for all A; K; 8:21

where, as above, the superscripts refer to the act of buying capital b,
selling capital s and inaction i. These options, in turn, are dened by
25. An interesting extension of the model would make this gap endogenous.
26. The data set is described by Cooper and Haltiwanger (2000) and is for a balanced
panel of U.S. manufacturing plants. Comparable data sets are available in other coun-
tries. Similar estimation exercises using these data sets would be of considerable
interest.
210 Chapter 8
g
V b A; K max PA; K FK I I=K 2 K
I 2
bEA 0 j A VA 0 ; K1 d I;
g
V s A; K max PA; K ps R FK R=K 2 K
R 2
bEA 0 j A VA 0 ; K1 d R;
and
V i A; K PA; K bEA 0 j A VA 0 ; K1 d:
Cooper and Haltiwanger (2000) estimate three parameters, Y 1

F; g; ps and assume that b 0:95, d 0:069. Further they specify
a prot function of PA; K AK y with y 0:50 estimated from a
panel data set of manufacturing plants.27 Note that the adjustment
costs in (8.21) exclude any disruptions to the production process so
that the PA; K can be estimated and the shock process inferred
independently of the estimation of adjustment costs. If these addi-
tional adjustment costs were added, then the prot function and
the shocks would have to be estimated along with the parameters of
the adjustment cost function.
These parameters are estimated using an indirect inference rou-
tine. The reduced form regression used in the analysis is
iit a i c0 c1 ait c2 ait 2 uit ; 8:22
where iit is the investment rate at plant i in period t, ait is the log of a
protability shock at plant i in period t, and a i is a xed effect.28 This
specication was chosen as it captures in a parsimonious way the
nonlinear relationship between investment rates and fundamentals.
The protability shocks are inferred from the plant-level data using
the estimated prot function.29 Cooper and Haltiwanger document
the extent of the nonlinear response of investment to shocks.
27. See the discussion by Cooper and Haltiwanger (2000) of the estimation of this
prot function.
28. More recent versions of the Cooper-Haltiwanger paper explore adding lagged
investment rates to this reduced form to pick up some of the dynamics of the adjust-
ment process.
29. This is an important step in the analysis. Determining the nature of adjustment
costs will depend on the characterization of the underlying protability shocks. For
example, if a researcher is trying to identify nonconvex adjustment costs from bursts
of investment, then getting the distribution of shocks right is critical.
Investment 211
Table 8.4
Parameter estimates
Structural parameter Parameter estimate
estimates (s.e.) for (8.22)
Speci-
cation g F ps c0 c1 c2
LRD 0.013 0.265 0.20
All 0.043 0.00039 0.967 0.013 0.255 0.171
(0.00224) (0.0000549) (0.00112)
F only 0 0.0333 1 0.02 0.317 0.268
(0.0000155)
g only 0.125 0 1 0.007 0.241 0.103
(0.000105)
ps only 0 0 0.93 0.016 0.266 0.223
(0.000312)
Table 8.4 reports Cooper and Haltiwangers results for four dif-
ferent models along with standard errors. The rst row shows the
estimated parameters for the most general model. The parameter
vector Y 0:043; 0:00039; 0:967 implies the presence of statistically
signicant convex and nonconvex adjustment costs (but nonzero)
and a relatively substantial transaction cost. Restricted versions of
the model are also reported for purposes of comparison. Clearly, the
mixed model does better than any of the restricted models.
Cooper and Haltiwanger argue that these results are reasonable.30
First, as noted above a low level for the convex cost of adjustment
parameter is consistent with the estimates obtained from the Q
theory based models due to the presence of imperfect competition.
Further the estimation implies that the xed cost of adjustment
is about 0.04 percent of average plant-level prots. Cooper and Hal-
tiwanger nd that this cost is signicant relative to the difference
between adjusting and not adjusting the capital stock. So in fact the
estimated xed cost of adjustment, along with the irreversibility,
produces a large amount of inaction. Finally the estimated selling
price of capital is much higher than the estimate report in Ramey
and Shapiro (2001) for some plants in the aerospace industry.
Cooper and Haltiwanger (2000) also explore the aggregate impli-
cations of their model. They contrast the time series behavior of the
30. The results are robust to allowing the discount factor to vary with the aggregate
shock in order to mimic the relationship between real interest rates and consumption
growth from a households Euler equation.
212 Chapter 8
estimated model with both convex and nonconvex adjustment costs

against one in which there are only convex adjustment costs. Even
though the model with only convex adjustment costs does relatively
poorly on the plant-level data, it does reasonably well in terms of
matching time series. In particular, Cooper and Haltiwanger (2000)
nd that over 90 percent of the time series variation in investment
created by a simulation of the estimated model can be accounted for
by a quadratic adjustment model. Of course, this also implies that
the quadratic model misses 10 percent of the variation.
Note too that this framework for aggregation captures the smooth-
ing by aggregating over heterogeneous plants but misses smoothing
created by variations in relative prices. From Thomas (2002) and
Kahn and Thomas (2001) we know that this additional source of
smoothing can be quite powerful as well.
8.6.2 Maximum Likelihood Estimation
A nal approach to estimation follows the approach in Rust (1987).

Consider again, for example, the stochastic machine replacement
problem given by
VA; K; F maxfV i A; K; F; V a A; K; Fg for all A; K; F; 8:23
where
V i A; K; F PA; K bEA 0 j A VA 0 ; K1 d; F 0
and
V a A; K; F max
0
PA; Kl FK pK 0 1 dK
K
bEA 0 j A VA 0 ; K 0 ; F 0 :
Here we have added the xed cost of adjustment into the state
vector as we assume that the adjustment costs are random at the
plant level. Let GF represent the cumulative distribution function
for these adjustment costs. Assume that these are iid shocks. Then,
given a guess for the functions fVA; K; F; V i A; K; F; V a A; K; Fg,
the likelihood of inaction can be computed directly from the cumu-
lative distribution function G. Thus a likelihood function can be
constructed that depends on the parameters of the distribution of
adjustment costs and those underlying the dynamic optimization
Investment 213
problem. From there, a maximum likelihood estimate can be

obtained.31
8.7 Conclusion
The theme of this chapter has been the dynamics of capital accumu-
lation. From the plant-level perspective, the investment process is
quite rich and entails periods of intense activity followed by times
of inaction. This has been documented at the plant level. Using the
techniques of the estimation of dynamic programming models, this
chapter has presented evidence on the nature of adjustment costs.
Many open issues remain. First, the time series implications of
nonconvexities is still not clear. How much does the lumpiness at the
plant-level matter for aggregate behavior? Put differently, how much
smoothing obtains from the aggregate across heterogeneous plants
as well as through variations in relative prices?
Second, there are a host of policy experiments to be considered.
What, for example, are the implications of investment tax credits
given the estimates of adjustment cost parameters?
exercise 8.6 Add in variations in the price of new capital into the
optimization problem given in (8.21). How would you use this to
study the impact of, say, an investment tax credit?
31. The interested reader should read closely the discussion of Rust (1987) and the
papers that followed this line of work. Note that often assumptions are made on G
to ease the computation of the likelihood function.
9 Dynamics of Employment
Adjustment
9.1 Motivation
This chapter studies labor demand. The usual textbook model of

labor demand depicts a rm as choosing the number of workers
and their hours given a wage rate. But the determination of wages,
employment, and hours is much more complex than this. The key is
to recognize that the adjustment of many factors of production,
including labor, is not costless. We study the dynamics of capital
accumulation elsewhere in this book and in this chapter focus atten-
tion on labor demand.
Understanding the nature of adjustment costs and thus the factors
determining labor demand is important for a number of reasons.
First, many competing models of the business cycle depend crucially
on the operation of labor markets. As emphasized in Sargent (1978),
a critical point in distinguishing competing theories of the business
cycle is whether labor market observations could plausibly be the
outcome of a dynamic market-clearing model. Second, attempts to
forecast macroeconomic conditions often resort to consideration of
observed movements in hours and employment to infer the state
of economic activity. Finally, policy interventions in the labor market
are numerous and widespread. These include restrictions on wages,
restrictions on hours, costs of ring workers, and so forth. Policy
evaluate requires a model of labor demand.
We begin the chapter with the simplest models of dynamic labor
demand where adjustment costs are assumed to be convex and con-
tinuously differentiable. These models are analytically tractable as
we can often estimate their parameters directly from rst-order con-
ditions. However, they have implications of constant adjustment
216 Chapter 9
that are not consistent with microeconomic observations. Nickell

(1978) argues:
One point worth noting is that there seems little reason to suppose costs per
worker associated with either hiring or ring increase with the rate at which
employees ow in or out. Indeed, given the large xed costs associated with
personnel and legal departments, it may even be more reasonable to sup-
pose that the average cost of adjusting the workforce diminishes rather than
increases with the speed of adjustment.
This quote is supported by recent evidence in Hamermesh (1989)

and Caballero et al. (1997) that labor adjustment is rather erratic at
the plant level with periods of inactivity punctuated by large adjust-
ments. Thus this chapter goes beyond the convex case and considers
models of adjustment which can mimic these microeconomic facts.
9.2 General Model of Dynamic Labor Demand
In this chapter we consider variants of the following dynamic pro-

gramming problem:
VA; e1 max RA; e; h oe; h; A Ce; e1

h; e
bEA 0 j A VA 0 ; e for all A; e1 : 9:1

Here A represents a shock to the protability of the plant and/or
rm. As in our discussion of the investment problem, this shock
could reect variations in product demands or variations in the pro-
ductivity of inputs. Generally, A will have a component that is com-
mon across plants, denoted a, and one that is plant specic, denoted
e.1 The function RA; e; h represents the revenues that depend on the
hours worked h and the number of workers e as well as the
protability shock. Other factors of production, such as capital, are
assumed to be rented and optimization over these inputs are incor-
porated into RA; e; h.2
The function oe; h; A is the total cost of hiring e workers
when each supplies h units of labor time. This general specication
1. Here we are also assuming that the discount factor is xed. In general, the discount
would depend on a and a 0 .
2. In contrast to the chapter on capital adjustment, here we assume that there are no
costs to adjusting the stock of capital. This is for convenience only, and a complete
model would incorporate both forms of adjustment costs.
Dynamics of Employment Adjustment 217
allows for overtime pay and other provisions. Assume that this
compensation function is increasing in both of its arguments and is
convex with respect to hours. Further we allow this compensation
function to be state dependent. This may reect a covariance with
the idiosyncratic protability shocks (due, perhaps, to prot-sharing
arrangements) or an exogenous stochastic component in aggregate
wages.
The function Ce; e1 is the cost of adjusting the number of
workers. Hamermesh (1993) and Hamermesh and Pfann (1996) pro-
vide a lengthy discussion of various interpretations and motivations
for adjustment costs. This function is meant to cover costs associated
with the following:
0
Search and recruiting
0
Training
0
Explicit ring costs
0
Variations in complementary activities (capital accumulation, reor-
ganization of production activities, etc.)
It is important to note the timing implicit in the statement of the
optimization problem. The state vector includes the stock of workers
in the previous period, e1 . In contrast to the capital accumulation
problem, the number of workers in the current period is not pre-
determined. Instead, workers hired in the current period are imme-
diately utilized in the production process: there is no time to build.
The next section of the chapter is devoted to the study of adjust-
ment cost functions such that the marginal cost of adjustment is
positive and increasing in e given e1 . We then turn to more general
adjustment cost functions that allow for more nonlinear and discon-
tinuous behavior.
9.3 Quadratic Adjustment Costs
Without putting additional structure on the problem, particularly

the nature of adjustment costs, it is difcult to say much about
dynamic labor demand. As a starting point, suppose that the cost of
adjustment is given by
h
Ce; e1 e 1 qe1 2 ; 9:2
2
218 Chapter 9
so Ce; e1 is convex in e and continuously differentiable. Here, q is

an exogenous quit rate.
In this specication of adjustment costs, the plant/rm incurs a
cost of changing the level of employment relative to the stock of
workers 1 qe1 that remain on the job from the previous period.
Of course, this is a modeling choice: one can also consider the case
where the adjustment cost is based on net rather than gross hires.3
The rst-order conditions for (9.1) using (9.2) are
Rh A; e; h oh e; h; A; 9:3
Re A; e; h oe e; h; A he 1 qe1 bEVe A 0 ; e 0: 9:4
Here the choice of hours, given in (9.3), is static: the rm weighs

the gains to the increasing labor input against the marginal cost
(assumed to be increasing in hours) of increasing hours.
In contrast, (9.4) is a dynamic relationship since the number of
employees is a state variable. Assuming that the value function is
differentiable, EVe A 0 ; e 0 can be evaluated using (9.1), leading to
Re A; e; h oe e; h; A he 1 qe1
bEhe 0 1 qe1 q 0: 9:5

The solution to this problem will yield policy functions for hours
and employment given the state vector. Let e fA; e1 denote the
employment policy function and h HA; e1 denote the hours
policy function. These functions jointly satisfy (9.3) and (9.5).
As a benchmark, suppose there were no adjustment costs, h 1 0,
and the compensation function is given by
oe; h; A eo
~ h:
Here compensation per worker depends only on hours worked.

Further, suppose that revenues depend on the product eh so that
only total hours matters for the production process. Specically,
~eh
RA; e; h AR 9:6
with R ~eh strictly increasing and strictly concave.
In this special case, the two rst-order conditions can be manipu-
lated to imply that
3. We can study the implications of that specication by setting q 0 in (9.2) to study

the alternative.
~ 0 h
o
1h :
~ h
o
So, in the absence of adjustment costs and with the functional forms
given above, hours are independent of both e and A. Consequently
all variations in the labor input arise from variations in the number
of workers rather than hours. This is efcient given that the marginal
cost of hours is increasing in the number of hours worked while
there are no adjustment costs associated with varying the number of
workers.
At another extreme, suppose there are adjustment costs h 0 0.
Further, suppose that compensation is simply
oe; h; A eh;
so there are no costs to hours variation. In this case (9.3) implies

AR~ 0 eh 1. Using this, we see that (9.5) is clearly satised at a con-
stant level of e. Hence the variation in the labor input would be
only in terms of hours, and we would never observe employment
variations.
Of course, in the presence of adjustment costs and a strictly convex
(in h) compensation function, the plant/rm will optimally balance
the costs of adjusting hours against those of adjusting the labor force.
This is empirically relevant since in the data both employment and
hours variation are observed. Note, though, that it is only adjust-
ment in the number of workers that contains a dynamic element.
The dynamics in hours is derived from the dynamic adjustment of
employees.4 It is this trade-off between hours and worker adjustment
that lies at the heart of the optimization problem.
Given functional forms, these rst-order conditions can be used in
an estimation routine that exploits the implied orthogonality con-
ditions. Alternatively, a value function iteration routine can be used
to approximate the solution to (9.1) using (9.2). We consider below
some specications.
A Simulated Example
Here we follow Cooper and Willis (2001) and study the policy func-
tions generated by a quadratic adjustment cost model with some
particular functional form assumptions.5 Suppose that output is
4. Add to this the dynamic adjustment of other factors, such as capital.

5. As discussed later in this chapter, this model is used by Cooper and Willis (2001)
for their quantitative analysis of the gap approach.
220 Chapter 9
a Cobb-Douglas function of total labor input eh and capital, and

assume the rm has market power as a seller. In this case consider
RA; e; h Aeh a ; 9:7
where a reects labors share in the production function as well as

the elasticity of the demand curve faced by the rm. Further, impose
a compensation schedule that follows Bils (1987):
oe; h w e w0 h w1 h 40 w2 h 40 2 ; 9:8
where w is the straight-time wage.
Instead of working with (9.5), Cooper and Willis (2001) solve
the dynamic programming problem (9.1) with the functional forms
above, using value function iteration. The functional equation for the
problem is
h e e1 2
VA; e1 max Aeh a oe; h
h; e 2 e1
bEA 0 j A VA 0 ; e for all A; e1 : 9:9
In this analysis, decisions are assumed to be made at the plant

level. Accordingly we assume that the protability shocks have
two components: a piece that is common across plants (an aggregate
shock) and a piece that is plant specic. Both types of shocks
are assumed to follow rst-order Markov processes. These are
embedded in the conditional expectation in (9.9).
In this formulation the adjustment costs are paid on net changes
in employment. Further the adjustment costs depend on the rate of
adjustment rather than the absolute change alone.6
The policy function that solves (9.9) is given by e fA; e1 . This
policy function can be characterized given a parameterization of
(9.9).
Cooper and Willis (2001) assume the following:
0
Labors share is 0.65 and the markup is 25 percent so that a in (9.7)
is 0.72.
0
The compensation function uses the estimates of Bils (1987) and
Shapiro (1986): fw0 ; w1 ; w2 g f1:5; 0:19; 0:03g; the straight-time wage
6. The literature on labor adjustment costs contains both specications. Cooper and
Willis (2001) nd that their results are not sensitive to this part of the specication.
Figure 9.1
Employment policy functions: Quadratic costs
w is normalized to 0.05 for convenience. The elasticity of the wage

with respect to hours is close to 1, on average.
0
The protability shocks are represented by a rst-order Markov
process and are decomposed into aggregate A and idiosyncratic
components e. A A f0:9; 1:1g and e takes on 15 possible values. The
serial correlation for the plant-level shocks is 0.83 and is 0.8 for the
aggregate shocks.7
This specication leaves open the parameterization of h in the cost
of adjustment function. In the literature this is a key parameter to
estimate.
The policy functions computed for two values of A at these
parameter choices are depicted in gure 9.1. Here we have set h 1
which is at the low end of estimates in the literature. These policy
functions have two important characteristics:
7. Alternatively, the parameters of these processes could be part of an estimation

exercise.
222 Chapter 9
0
fA; e1 is increasing in e1 .
0
fA; e1 is increasing in A. As protability increases, so does the
marginal gain to adjustment, and thus e is higher.
The quadratic adjustment cost model can be estimated either from
plant (rm) data or aggregate data. To illustrate this, we next discuss
the approach of Sargent (1978). We then discuss a more general
approach to estimation in a model with a richer specication of
adjustment costs.
exercise 9.1 Write down the necessary conditions for the optimal
choices of hours and employment in (9.9). Provide an interpretation
of these conditions.
Sargent: Linear Quadratic Specication

A leading example of bringing the quadratic adjustment cost model
directly to the data is Sargent (1978). In that application, Sargent
assumes there are two types of labor input: straight-time and over-
time workers. The production function is linear-quadratic in each of
the two inputs, and the costs of adjustment are quadratic and sepa-
rable across the types of labor. As the two types of labor inputs do
not interact in either the production function or the adjustment cost
function, we will focus on the model of straight-time employment in
isolation. Following Sargent, assume that revenue from straight-time
employment is given by
R1 2
RA; e R0 Ae e : 9:10
2
Here A is a productivity shock and follows an AR(1) process. Sar-

gent does not include hours variation in his model except through
the use of overtime labor. Accordingly there is no direct dependence
of the wage bill on hours. Instead, he assumes that the wage rate
follows an exogenous process (with respect to employment) given by
X
in
wt n0 ni wti zt : 9:11
i1
In principle, the innovation to wages can be correlated with the

shocks to revenues.8
8. The factors that help the rm forecast future wages are then included in the state
space of the problem; that is, they are in the aggregate component of A.
With this structure the rms rst-order condition with respect to

employment is given by

R1 1
bEt et1 et 1 b et1 wt R0 At : 9:12
h h
From this Euler equation, current employment will depend on

the lagged level of employment (through the cost of adjustment) and
on (expected) future values of the stochastic variables, productivity,
and wages, as these variables inuence the future level of employ-
ment. As described by Sargent, the solution to this Euler equation
can be obtained so that employment in a given period depends on
lagged employment, current and (conditional expectations of) future
wages, and current and (conditional expectations of) future pro-
ductivity shocks. Given the driving process for wages and produc-
tivity shocks, this conditional expectation can be evaluated so that
employment in period t is solely a function of lagged employment,
current and past wages. The past wages are relevant for predicting
future wages.
Sargent estimates the resulting VAR model of wages employment
using maximum likelihood techniques.9 The parameters he esti-
mated included R1 ; h; r, where r is the serial correlation of the pro-
ductivity shocks. In addition Sargent estimated the parameters of the
wage process.
The model is estimated using quarterly data on total U.S. civilian
employment. Interestingly he also decided to use seasonally unad-
justed data for some of the estimation, arguing in effect that there is
no reason to separate the responses to seasonal and nonseasonal
variations. The data are detrended to correspond to the stationarity
of the model.
He nds evidence of adjustment costs insofar as h is signicantly
different from zero.10 Sargent argues that these results . . . are mod-
erately comforting to the view that the employment-real-wage
observations lie along a demand schedule for employment (p. 1041).
9. Sargent (1978) estimates a model with both regular and overtime employment. For
simplicity we presented the model of regular employment alone.
10. He also discusses at length the issue of identication and nds multiple peaks in
the likelihood function. Informally the issue is distinguishing between the serial cor-
relation in employment induced by lagged employment from that induced by the
serial correlation of the productivity shocks.
224 Chapter 9
exercise 9.2 There are a number of exercises to consider working

from this simple model.
1. Write a program to solve (9.9) for the employment and hours
policy functions using value function iteration. What are the prop-
erties of these policy functions? How do these functions change as
you vary the elasticity of the compensation function and the cost of
adjustment parameter?
2. Solve (9.9) using a log-linearization technique. Compare your
results with those obtained by the value function iteration approach.
3. Consider some moments such as the relative variability of hours
and employment and the serial correlations of these two variables.
Calculate these moments from a simulated panel and also from a
time series constructed from the panel. Look for studies that charac-
terize these moments at the micro and/or aggregate levels. Or, better
yet, calculate them yourself. Construct an estimation exercise using
these moments.
4. Suppose that you wanted to estimate the parameters of (9.9) using
GMM. How would you proceed?
9.4 Richer Models of Adjustment
In part, the popularity of the quadratic adjustment cost struc-

ture reects its tractability. But the implications of this specica-
tion of adjustment costs conict with evidence of inactivity and
bursts at the plant level. Thus researchers have been motivated to
consider a richer set of models. Those are studied here and then are
used for estimation purposes below. For these models of adjustment
we discuss the dynamic optimization problem and present policy
functions.
9.4.1 Piecewise Linear Adjustment Costs
One of the criticisms of the quadratic adjustment cost specication

is the implications of continuous adjustment. At the plant level, as
mentioned earlier, there is evidence that adjustment is much more
erratic than the pattern implied by the quadratic model. Piecewise
linear adjustment costs can produce inaction.
For this case the cost of adjustment function is

g D e if D e > 0;
Ce; e1 9:13
g D e if D e < 0:
The optimal policy rules are then determined by solving (9.1) using
this specication of the adjustment cost function.
The optimal policy rule will look quite different from the one
produced with quadratic adjustment costs. This difference is a con-
sequence of the lack of differentiability in the neighborhood of zero
adjustment. Consequently small adjustments will not occur since
the marginal cost of adjustment does not go to zero as the size of
the adjustment goes to zero. Further this speciciation of adjust-
ment costs implies there is no partial adjustment. Since the mar-
ginal cost of adjustment is constant, there is no basis for smoothing
adjustment.
The optimal policy is characterized by two boundaries: e A and
e A. If e1 A e A; e A; then there is no adjustment. In the event
of adjustment, the optimal adjustment is to e A if e1 < e A and
is to e A if e1 > e A.
Following Cooper and Willis (2001) and using the same basic
parameters as described above, we can study the optimal policy
function for this type of adjustment cost. Assume that g g
0:05, which produces inaction at the plant level in 23 percent of
the observations.11 Then (9.1) along with (9.13) can be solved using
value function iteration and the resulting policy functions evaluated.
These are shown in gure 9.2. Note that there is no adjustment
for values of e1 in an interval: the employment policy function
coincides with the 45-degree line. Outside of that internal there are
two targets: e A and e A. Again, this policy function is indexed
by the values of g and g . So these parameters can be estimated
by matching the implications of the model against observations of
employment adjustment at the plant and/or aggregate levels. We
will return to this point below.
exercise 9.3 Specify the dynamic programming problem for labor
adjustment using a piecewise linear adjustment cost structure. What
determines the region of inaction? Study this model numerically by
solving the dynamic programming problem and obtaining policy
functions.
11. This inaction rate is too high relative to observation: the parameterization is for
illustration only.
226 Chapter 9
Figure 9.2
Employment policy functions: Piecewise linear adjustment costs
9.4.2 Nonconvex Adjustment Costs
The observations of inactivity at the plant level that motivate the

piecewise linear specication are also used to motivate consideration
of xed costs in the adjustment process. As noted by Hamermesh
and Pfann (1996), the annual recruiting activities of economics
departments provide a familiar example of the role of xed costs. In
the United States, hiring requires posting of an advertisement of
vacancies, extensive review of material provided by candidates,
travel of a recruiting team to a convention site, interviews of leading
candidates, university visits, and nally a vote to select among the
candidates. Clearly, there are xed cost components to many of these
activities that comprise the hiring of new employees.12
12. This depiction motivates consideration of a search model as the primitive that
underlies a model of adjustment costs. See the discussion of Yashiv (2000) in chapter
10.
As a formal model of this, consider
VA; e1 maxV a A; e1 ; V n A; e1 for all A; e1 ; 9:14

where V a A; e1 represents the value of adjusting employment and
V n A; e1 represents the value of not adjusting employment. These
are given by
V a A; e1 max RA; e; h oe; h F bEA 0 j A VA 0 ; e; 9:15
h; e
V n A; e1 max RA; e1 ; h oe1 ; h bEA 0 j A VA 0 ; e1 : 9:16

h
In this specication the rm can either adjust the number of

employees or not. These two options are labeled action V a A; e1
and inaction V n A; e1 . In either case, hours are assumed to be
freely adjusted and thus will respond to variations in protability
even if there is no adjustment in the number of workers. Note too
that this specication assumes adjustment costs depend on gross
changes in the number of workers. In this way the model can
potentially match the inaction in employment adjustment at the
plant level dened by zero changes in the number of workers.
The optimal policy has three dimensions. First, there is the choice
of whether to adjust or not. Let zA; e1 A f0; 1g indicate this choice
where zA; e1 1 if and only if there is adjustment. Second, there is
the choice of employment in the event of adjustment. Let fA; e1
denote that choice where fA; e1 e1 if zA; e1 0. Finally, there
is the choice of hours, hA; e1 , which will reect the decision of the
rm whether or not to adjust employment. As these employment
adjustments depend on A; e1 through e fA; e1 , one can always
consider hours to be a function of the state vector alone.
There are some rich trade-offs between hours and employment
variations embedded in this model. Suppose that there is a positive
shock to protability: A rises. If this variation is large and perma-
nent, then the optimal response of the rm will be to adjust employ-
ment. Hours will vary only slightly. If the shock to protability is
not large or permanent enough to trigger adjustment, then by de-
nition, employment will remain xed. In that case the main variation
will be in worker hours.
These variations in hours and employment are shown in gure 9.3.
The policy functions underlying this gure were created using a
228 Chapter 9
Figure 9.3
Employment policy functions: Nonconvex adjustment costs
baseline parameters with xed costs at 0.1 of the steady state

prots.13
exercise 9.4 Specify the dynamic programming problem for labor
adjustment using a nonconvex adjustment cost structure. What
determines the frequency of inaction? What comovement of hours
and employment is predicted by the model? What features of the
policy functions distinguish this model from the one with piecewise
linear adjustment costs? Study this model numerically by solving the
dynamic programming problem and obtaining policy functions.
9.4.3 Asymmetries
As discussed in Hamermesh and Pfann (1996), there is certainly evi-

dence in favor of asymmetries in the adjustment costs. For example,
13. At this level of xed costs, there is about 50 percent employment inaction. Again,
the parameterization is just for illustration.
there may be a cost of advertising and evaluation that is propor-

tional to the number of workers hired but no costs of ring workers.
Alternatively, it may be of interest to evaluate the effects of ring
costs on hiring policies as discussed in the context of some European
economies.
It is relatively straightforward to introduce asymmetries into the
model. Given the approach to obtaining policy functions by solving
(9.1) through a value function iteration routine, asymmetries do not
present any additional difculties. As with the other parameter-
izations of adjustment costs, these model can be estimated using a
variety of techniques. Pfann and Palm (1993) provide a nice example
of this approach. They specify an adjustment cost function of
1
Ce; e1 1 e gD e gD e hD e 2 ; 9:17
2
where D e 1 e e1 . If g 1 0, then this reduces to (9.2) with q 0.

As Pfann and Palm (1993) illustrate, the asymmetry in adjustment
costs is controlled by g. For example, if g < 0, then ring costs exceed
hiring costs.
Using this model of adjustment costs, Pfann and Palm (1993) esti-
mate parameters using a GMM approach on data for manufacturing
in the Netherlands (quarterly, seasonally unadjusted data, 1971.I
1984.IV) and annual data for U.K. manufacturing. They have data on
both production and nonproduction workers, and the employment
choices are interdependent, given the production function.
For both countries they nd evidence of the standard quadratic
adjustment cost model: h is positive and signicantly different from
zero for both types of workers. Moreover there is evidence of
asymmetry. They report that the costs of ring production workers
are lower than the hiring costs. But, the opposite is true for non-
production workers.
9.5 The Gap Approach
The work in Caballero and Engel (1993b) and Caballero et al. (1997)
pursues an alternative approach to studying dynamic labor adjust-
ment. Instead of solving an explicit dynamic optimization problem,
they postulate that labor adjustment will respond to a gap between
the actual and desired employment level at a plant. They then test
for nonlinearities in this relationship.
230 Chapter 9
The theme of creating an employment target to dene an employ-

ment gap as a proxy for the current state is quite intuitive and pow-
erful. As noted in our discussion of nonconvex adjustment costs,
when a rm is hit by a protability shock, a gap naturally emerges
between the current level of employment and the level of employ-
ment the rm would choose if there were no costs of adjustment.
This gap should then be a good proxy for the gains to adjustment.
These gains, of course, are then compared to the costs of adjust-
ment, which depend on the specication of the adjustment cost
function. This section studies some attempts to characterize the
nature of adjustment costs using this approach.14
The power of the gap approach is the simplication of the
dynamic optimization problem as the target level of employment
summarizes the current state. However, as we will see, these gains
may be difcult to realize. The problem arises from the fact that the
target level of employment, and thus the gap, is not observable to
the researcher.
To understand this approach, it is useful to begin with a discus-
sion of the partial adjustment model. We then return to evidence on
adjustment costs from this approach.
9.5.1 Partial Adjustment Model
Researchers often specify a partial adjustment model in which the

rm is assumed to adjust the level of employment to a target.15 The
assumed model of labor adjustment would be
et et1 le et1 : 9:18

So here the change in employment et et1 is proportional to the
difference between the previous level of employment and a target,
e , where l parameterizes how quickly the gap is closed.
Where does this partial adjustment structure come from? What
does the target represent? Cooper and Willis (2001) consider a
dynamic programming problem given by
e e 2 k
Le ; e1 min e e1 2 bEe0 j e Le0 ; e; 9:19
e 2 2
14. This presentation draws heavily on Cooper and Willis (2001). We are grateful to
John Haltiwanger and Jon Willis for helpful discussions on this topic.
15. The structure is used to study adjustment of capital as well.
where the loss depends on the gap between the current stock of
workers e and the target e . The target is taken as an exogenous
process, though in general it reects the underlying shocks to prot-
ability that are explicit in the optimizing model. In particular, sup-
pose that e follows an AR(1) process with serial correlation of r.
Further assume that there are quadratic adjustment costs, para-
meterized by k.
The rst-order condition to the optimization problem is
e e ke e1 bkEe 0 e 0; 9:20

where the last term was obtained from using (9.19) to solve for
qL=qe. Given that the problem is quadratic, it is natural to conjecture
a policy function in which the control variable e is linearly related
to the two elements of the state vector e ; e1 :
e l1 e l2 e1 : 9:21
Using this conjecture in (9.20) and taking expectations of the future
value of e yields
e e ke e1 bkl1 re l2 1e 0: 9:22
This can be used to solve for e as a linear function of e ; e1 with

coefcients given by
1 bkl1 r
l1 9:23
1 k bkl2 1
and
k
l2 : 9:24
1 k bkl2 1
Clearly, if the shocks follow a random walk r 1, then partial

adjustment is optimal l1 l2 1. Otherwise, the optimal policy
created by minimization of the quadratic loss is linear but does not
dictate partial adjustment.
9.5.2 Measuring the Target and the Gap
Taking this type of model directly to the data is problematic as the

target e is not observable. In the literature (e.g., see the discussion in
Caballero and Engel 1993b) the target is intended to represent the
232 Chapter 9
destination of the adjustment process. There are two representations

of the target.
One, termed a static target, treats e as the solution of a static
optimization problem, as if adjustment costs did not exist. Thus e
solves (9.5) with h 1 0 and hours set optimally.
A second approach treats e as the level of employment that the
rm would choose if there were no adjustment costs for a single
period. This is termed the frictionless target. This level of employ-
ment solves e fA; e, where fA; e1 is the policy function for
employment for the quadratic adjustment cost model. Thus the tar-
get is the level of employment where the policy function, contingent
on the protability shock, crosses the 45-degree line as in gure 9.1.
Following Caballero et al. (1997), dene the gap as the difference
between desired ei; t and actual employment levels (in logs):
~zi; t 1 ei; t ei; t1 : 9:25
Here ei; t1 is number of workers inherited from the previous period.
So ~zi; t measures the gap between the desired and actual levels of
employment in period t, prior to any adjustments, but after any rel-
evant period t random variables are realized as these shocks are
embedded in the target and thus the gap.
The policy function for the rm is assumed to be16
D ei; t f~zi; t : 9:26

The key of the empirical work is to estimate the function f.
Unfortunately, estimation of (9.26) is not feasible as the target,
and thus the gap are not observable. So the basic theory must be
augmented with a technique to measure the gap. There are two
approaches in the literature corresponding to the two notions of a
target level of employment, described earlier.
Caballero et al. (1997) pursue the theme of a frictionless target.
To implement this, they postulate a second relationship between
another (closely related) measure of the gap, ~zi;1 t and plant-specic
deviations in hours:
~zi;1 t yhi; t h: 9:27
Here ~zi;1 t is the gap in period t after adjustments in the level of e have
been made: ~zi;1 t ~zi; t D ei; t .
16. Based on discussions above, the policy function of the rm should depend jointly
on A; e1 and not on the gap alone.
The argument in favor of this approach again returns to our dis-

cussion of the choice between employment and hours variation in
the presence of adjustment costs. In that case we saw that the rm
chose between these two forms of increasing output when prot-
ability rose. Thus, if hours are measured to be above average, this
will reect a gap between actual and desired workers. If there was
no cost of adjustment, the rm would choose to hire more workers.
But in the presence of these costs the rm maintains a positive gap,
and hours worked are above average.
The key to (9.27) is y. Since the left-hand side of (9.27) is also not
observable, the analysis is further amended to generate an estimate
of y. Caballero et al. (1997) estimate y from
D ei; t a yD hi; t ei; t ; 9:28

where the error term includes unobserved changes in the target level
of employment D ei; t as well as measurement error. Caballero et al.
(1997) note that the equation may have omitted variable bias as the
change in the target may be correlated with changes in hours. From
the discussion in Cooper and Willis (2001), this omitted variable bias
can be quite important.
Once y is estimated, Caballero et al. (1997) can construct plant
specic gap measures using observed hours variations. In principle,
the model of employment adjustment using these gap measures can
be estimated from plant-level data. Instead, Caballero et al. (1997)
focus on the aggregate time series implications of their model. In
particular, the growth rate of aggregate employment is given by

DEt zFz ft z; 9:29
z
where Fz is the adjustment rate or hazard function characterizing

the fraction of the gap that is closed by employment adjustment.
From aggregate data this expression can be used to estimate Fz.
As discussed in Caballero et al. (1997), if Fz is, say, a quadratic,
then (9.29) can be expanded implying that employment growth will
depend on the rst and third moments of the cross-sectional distri-
bution of gaps.
The ndings of Caballero et al. (1997) can be summarized as
follows:
0
Using (9.28), y is estimated at 1.26.
234 Chapter 9
0
The relationship between the average adjustment rate and the gap
is nonlinear.
0
There is some evidence of inaction in employment adjustment.
0
Aggregate employment growth depends on the second moment of
the distribution of employment gaps.
In contrast, Caballero and Engel (1993b) do not estimate y. Instead,

they calibrate it from a structural model of static optimization by a
rm with market power. In doing so, they are adopting a target that
ignores the dynamics of adjustment. From their perspective, the
gap is dened using (9.25) where ei; t corresponds to the solution of
a static optimization problem over both hours and employment
without any adjustment costs. They argue that this static target will
approximate the frictionless target quite well if shocks are random
walks. As with Caballero et al. (1997), once the target is determined,
a measure of the gap can be created.
This approach to approximating the dynamic optimization prob-
lem is applied extensively because it is so easy to characterize. Fur-
ther it is a natural extension of the partial adjustment model. But, as
argued in Cooper and Willis (2001), the approach may place exces-
sive emphasis on static optimization.17
Caballero and Engel (1993b) estimate their model using aggre-
gate observations on net and gross ows for U.S. manufacturing
employment. They nd that a quadratic hazard specication ts the
aggregate data better than the at hazard specication.
The key point in both of these papers is the rejection of the at
hazard model. Both Caballero et al. (1997) and Caballero and Engel
(1993b) argue that the estimates of the hazard function from aggre-
gate data imply that the cross-sectional distribution matters for
aggregate dynamics. Put differently, both studies reject a at hazard
specication in which a constant fraction of the gap is closed each
period.
Given that this evidence is obtained from time series, this
implies that the nonconvexities at the plant-level have aggregate
implications. This is an important nding in terms of the way mac-
roeconomists build models of labor adjustment.
17. This point was made some years ago. Nickell (1978) says: . . . the majority of
existing models of factor demand simply analyze the optimal adjustment of the rm
towards a static equilibrium and it is very difcult to deduce from this anything
whatever about optimal behavior when there is no equilibrium to aim at.
To the extent that the at hazard model is the outcome of a qua-

dratic adjustment cost model, both papers reject that specication in
favor of a model that generates some nonlinearities in the adjust-
ment process. But, as these papers do not consider explicit models of
adjustment, one cannot infer from these results anything about the
underlying adjustment cost structure.
Further, as argued by Cooper and Willis (2001), the methodology
of these studies may itself induce the nonlinear relationship between
employment adjustment and the gap. Cooper and Willis (2001) con-
struct a model economy with quadratic adjustment costs. They as-
sume that shocks follow a rst-order Markov process, with serial
correlation less than unity.18 They nd that in using either the Cab-
allero et al. (1997) or Caballero and Engel (1993b) measurements of
the gap, the cross-sectional distribution of employment gaps may be
signicant in a time series regression of employment growth.
9.6 Estimation of a Rich Model of Adjustment Costs
Thus far we have discussed some evidence associated with the

quadratic adjustment cost models and provided some insights into
the optimal policy functions from more complex adjustment cost
models. In this section we go a step further and discuss attempts to
evaluate models that may have both convex and nonconvex adjust-
ment costs.
As with other dynamic optimization problems studied in this
book, there is, of course, a direct way to estimate the parameters of
labor adjustment costs. This requires the specication of a model
of adjustment that nests the variety of special cases described above
along with a technique to estimate the parameters. In this subsection
we outline this approach.19
Letting A represent the protability of a production unit (e.g., a
plant), we consider the following dynamic programming problem:
VA; e1 max RA; e; h oe; h; A CA; e1 ; e bEA 0 j A VA 0 ; e:

h; e
9:30
18. The process is taken from the Cooper and Haltiwanger (2000) study of capital
adjustment. As these shocks were measured using static labor rst-order condition,
Cooper and Willis (2001) study the robustness of their results to variations in these
Markov processes.
19. This discussion parallels the approach in Cooper and Haltiwanger (2000).
236 Chapter 9
As above, let
RA; e; h Aeh a ; 9:31

where the parameter a is again determined by the shares of capital
and labor in the production function as well as the elasticity of
demand.
The function oe; h; A represents total compensation to workers
as a function of the number of workers and their average hours. As
before, this compensation function could be taken from other studies
or perhaps a constant elasticity formulation might be adequate:
w w0 w1 h z .
The costs of adjustment function nests quadratic and nonconvex
adjustment costs of changing employment
8
>
> n e e1 2
>F H
> e1 if e > e1 ;
< 2 e1
CA; e1 ; e 2 9:32
>
> n e e
>
> F 1
:F e1 if e < e1 ;
2 e1
where F H and F F represent the respective xed costs of hiring and
ring workers. Note that quadratic adjustment costs are based upon
net not gross hires. In (9.32), n parameterizes the level of the adjust-
ment cost function. This adjustment cost function yields the follow-
ing dynamic optimization problem:
VA; e1 maxfV H A; e1 ; V F A; e1 ; V N A; e1 g for all A; e1 ;
9:33
where N refers to the choice of no adjustment of employment. These
options are dened by
V H A; e1 max RA; e; h oe; h; A F H
h; e

n e e1 2
e1 bEA 0 j A VA 0 ; e if e > e1 ;
2 e1
V F A; e1 max RA; e; h oe; h; A F F

h; e

n e e1 2
e1 bEA 0 j A VA 0 ; e if e < e1 ;
2 e1
V N A; e1 max RA; e1 ; h oe1 ; h; A bEA 0 j A VA 0 ; e1 :

h
Problem (9.33) looks formidable. It contains both an extensive

(adjustment or no adjustment) as well an an intensive (the choice of
e, given adjustment) margin. Further there is no simple Euler equa-
tion to study given the nonconvex adjustment costs.20
But, given the methodology of this book, attacking a problem like
this is feasible. In fact one could build additional features into this
model, such as allowing for a piecewise linear adjustment cost a
structure.21
From our previous discussion, we know that solving a model
with this complexity is relatively straightforward. Let Y represent
the vector of parameters necessary to solve the model.22 Then, for a
given value of this vector, a value function iteration procedure will
generate a solution to (9.30).
Once a solution to the functional equation is obtained, then policy
functions can be easily created. Figure 9.4 produces a policy function
for the case of h 1 and F F F H 0:01.
One can obtain correlations from a simulated panel. For this
parameterization, some moments of interest are: corre; A 0:856;
corrh; A 0:839, and corre; h 0:461. Clearly, employment and
hours adjustments are both positively related to the shock. Further
we nd that the correlation of hours and employment is positive
indicating that the adjustment toward a target, in which the correla-
tion is negative, is offset by the joint response of these variables to a
shock.
Computation of these moments for a given Y opens the door to
estimation. If these moments can be computed for a given Y, then:
0
It is easy to compute other moments (including regression
coefcients).
0
It is easy to nd a value of Y to bring the actual and simulated
moments close together.
The techniques of this book are then easily applied to a study of
labor market dynamics using either panel or time series data.23 Of
course, this exercise may be even more interesting using data from
20. However, see the discussion of Aguirregabiria (1997) for progress in this direction.
21. Of course, it then becomes a question of identication: Can one distinguish
between the nonconvex and piecewise linear models.
22. Note that Y would include the parameters of the stochastic processes.
23. This is the goal of an ongoing project.
238 Chapter 9
Figure 9.4
Employment policy functions: Mixed adjustment costs
countries other than the US who, through institutional constraints,

have richer adjustment costs.
9.7 Conclusion
This point of this chapter has been to explore the dynamics of labor
adjustment. In the presence of adjustment costs, the conventional
model of static labor demand is replaced by a possibly complex
dynamic optimization problem. Solving these problems and esti-
mating parameters using either plant-level or aggregate observations
is certainly feasible using the techniques developed in this book.
In terms of policy implications, governments often impose
restrictions on employment and hours. The dynamic optimization
framework facilitates the analysis of those interventions.24 Further
24. In some cases a more general equilibrium approach is needed to assess the com-
plete implications of the policy.
these policies (such as restrictions on hours and/or the introduction

of ring costs) may provide an opportunity to infer key structural
parameters characterizing labor adjustment costs.25
25. This suggestion is along the lines of the so-called natural experimental approach to
estimation where the researcher searches for exogenous events that may allow for
the identication of key parameters. Evaluating this approach in the context of struc-
tural model is an exercise of some interest.
10 Future Developments
This nal section of this book covers an assortment of additional

topics. These represent active areas of research which utilize the
approach of this book. In some cases the research is not yet that far
along. Examples of this would include ongoing work on the inte-
gration of pricing and inventory problems or the joint evolution of
capital and labor. In a second category are search models of the labor
market that illustrate the usefulness of empirical work on dynamic
programming, though generally these models are not part of stan-
dard courses in applied macroeconomics.
Consequently the presentation is different than other chapters.
Here we focus mainly on the statement of coherent dynamic opti-
mization problems and properties of policy functions. To the extent
that there are empirical studies, we summarize them.
10.2 Price Setting
We begin with a very important problem in macroeconomics, the

determination of prices. For this discussion we do not rely on the
Walrasian auctioneer to miraculously set prices. Instead, we allow
rms to set prices and study this interaction in a monopolistic com-
petition setting.1
The natural specication includes a xed cost of adjusting prices
so that the rm optimally chooses between adjusting or not. Hence
1. Early formulations of the framework we discuss include Benassy (1982), Blanchard

and Kiyotaki (1987), Caballero and Engel (1993a), Caplin and Leahy (1991) and Caplin
and Leahy (1997).
242 Chapter 10
we term this the state-dependent pricing model. These have been

most recently termed menu cost models in the literature to highlight
the fact that a leading parable of the model is one where a seller
nds it costly to literally change the posted price. In fact this termi-
nology is somewhat unfortunate as it tends to trivialize the problem.
Instead, it is best to view these costs as representing a wide range of
sources of frictions in the pricing of goods.
Besides presenting a basic optimization problem, this section sum-
marizes two empirical exercises. The rst reports on an attempt to
use indirect inference to estimate the cost of price adjustment for
magazine prices. The second is a study of the aggregate implications
of state dependent pricing.
10.2.1 Optimization Problem
Consider a dynamic optimization problem at the rm level where,

by assumption, prices are costly to adjust. The rm has some market
power, represented by a downward-sloping demand curve. This
demand curve may shift around so that the price the rm would set
in the absence of adjustment costs is stochastic. The question is:
How, in the presence of adjustment costs, do rms behave?
Suppose, to be concrete, that product demand comes from the CES
(constant elasticity of substitution) specication of utility so that the
demand for product i is given by
g
p D
qid p; D; P : 10:1
P P
Here all variables are nominal. The price of product i is p while the
general price level is P. Finally, nominal spending, taken to be exog-
enous and stochastic, is denoted D.
Given this specication of demand and the realized state, p; D; P,
the rms real prots are
p
p p; D; P qid p; D; P cqid p; D; P; 10:2
P
where c is assumed to be a strictly increasing and strictly convex

function of output.
The dynamic optimization problem of the rm, taking the current
values and evolution of D; P as given, is
Future Developments 243
Vp; D; P; F maxfV a p; D; P; F; V n p; D; P; Fg for all p; D; P; F;

10:3
where
V a p; D; P; F max p p~; D; P F bED 0 ; P 0 ; F 0 j D; P; F V p~; D 0 ; P 0 ; F 0 ;

p~
10:4
V n p; D; P; F p p; D; P bED 0 ; P 0 ;F 0 j D; P; F Vp; D 0 ; P 0 ; F 0 : 10:5
Here the state vector is p; D; P; F. The cost of changing a price is

F. It enters the state vector, since in this specication we allow this
adjustment cost to be stochastic.2
The rm has two options. If the rm does not change its price, it
enjoys a prot ow, avoids adjustment costs, and then, in the next
period, has the same nominal price. Of course, if the aggregate price
level changes P 0 P 0 , then the rms relative price will change over
time. Note that the cost here is associated with adjustment of the
nominal price.
Alternatively, the rm can pay the menu cost F and adjust its price
to p~. This price change becomes effective immediately so that the
prot ow given adjustment is p p~; D; P. This price then becomes
part of the state vector for the next period.
The policy function for this problem will have two components.
First, there is a discrete component indicating whether or not price
adjustment will take place. Second, conditional on adjustment, there
is the policy function characterizing the dependence of p~ on the state
vector D; P; F. Interestingly the choice of p~ is independent of p once
the decision to adjust has been made.
There is a very important difference between this optimization
problem and most of the others studied in this book. From (10.3) we
know that the choice at the individual rm level depends on the
choices of other rms, summarized by P. Thus, given the specica-
tion of demand, the behavior of a single rm must depend on the
behavior of other rms.3 This feature opens up a number of alterna-
tive ways of solving the model.
As a starting point, one might characterize the exogenous evolu-
tion of P, perhaps through a regression model, and impose this in the
2. This is similar to the stochastic adjustment cost structure used by Rust (1987).
3. As discussed, for example, in Blanchard and Kiyotaki (1987), there is a com-
plementarity that naturally arises in the pricing decisions in this environment.
244 Chapter 10
optimization problem of the rm.4 In this case the individual opti-

mizer is simply using an empirical model of the evolution of P.
By this approach, there is no guarantee that the aggregate
evolution of P assumed by the individual agent actually accords
with the aggregated behavior of these agents. This suggests a second
approach in which this consistency between the beliefs of agents
and their aggregate actions is imposed on the model. Essentially this
amounts to the following steps:
1. Solve (10.3) given a transition equation for P.

2. Use the resulting policy functions to solve for the predicted evo-
lution of P.
3. Stop if these functions are essentially the same.
4. Iterate if they are not.
There is a point of caution here. For the dynamic programming
problem we can rely on the contraction mapping property to guar-
antee that the value function iteration process will nd the unique
solution to the functional equation. We have no such theorem to
guide us in the iterative procedure described above. Consequently
nding an equilibrium may be difcult, and further there is no rea-
son to suspect that the equilibrium is unique.5
10.2.2 Evidence on Magazine Prices
Willis (2000a) studies the determination of magazine price adjust-

ment using a data set initially used by Cecchetti (1986). The idea is to
use data on the frequency and magnitude of magazine price adjust-
ment to estimate a dynamic menu cost model.6
Willis postulates a theory model similar to that given above. For
the empirical analysis, he species an auxiliary equation in which
the probability of price adjustment is assumed to depend on the
following:
4. This may entail adding additional elements to the state space; see Adda and Cooper
(2000a) and Willis (2000a) for discussions of this point.
5. Ball and Romer (1990) provide an example. John and Wolman (1999) study these
issues in a dynamic setting of price adjustment.
6. The contribution here is bringing the dynamic menu cost model to the data. Bils
and Klenow (2002) provide further evidence on price-setting behavior based on BLS
price data.
0
Number of years since the last price adjustment.
0
Cumulative ination since the last price adjustment.
0
Cumulative growth in industry demand since the last price
adjustment.
0
Current ination.
0
Current industry demand.
This specication is partly chosen as it mimics some of the key

elements of the specication in Cecchetti (1986). Further the cumu-
lative ination and demand since the last price change are, from the
dynamic programming problem, key elements in the incentive to
adjust prices. Interestingly there seems to be little support for any
time dependence, given the presence of the proxies for the state
variables.
Willis estimates this auxiliary model and then uses it, through an
indirect inference procedure, to estimate the structural parameters of
his model. These parameters include the following:
0
The curvature of the prot function.
0
The curvature of the cost function.
0
The distribution of menu costs.
Willis (2000a) nds that magazine sellers have a signicant
amount of market power but that production is essentially constant
returns to scale. Finally, Willis is able the distinguish the average
adjustment cost in the distribution from the average that is actually
paid. He nds that the former is about 35 percent of revenues, while
the latter is only about 4 percent of revenues.7
10.2.3 Aggregate Implications
A large part of the motivation for studying models with some

form of price rigidity reects the arguments, advanced by macro-
economists, that inexible prices are a source of aggregate inef-
ciency. Further rigidity of prices and/or wages provides a basis for
the nonneutrality of money, thus generating a link between the stock
7. For this specication there is assumed to be no serial correlation in the adjustment

costs. See Willis (2000a) for further discussion of this point and for estimates that relax
this restriction.
246 Chapter 10
of nominal money and real economic activity. But these arguments

rest on the presence of quantitatively relevant rigidities at the level
of individual sellers. Can these costs of adjusting prices explain
observations at both the microeconomic and aggregate levels?
One approach to studying these issues is to model the pricing
behavior of sellers in a particular industry. This estimated model can
then be aggregated to study the effects of, say, money on output. An
alternative, more aggregate approach is to specify and estimate a
macroeconomic model with price rigidities.
At this point, while the estimation of such a model is not complete,
there is some progress. A recent paper by Dotsey et al. (1999) studies
the quantitative implications of state-dependent pricing for aggre-
gate variables. We summarize those results here.
The economy studied by Dotsey et al. (1999) has a number of key
elements:
0
As in Blanchard and Kiyotaki (1987), the model is based on
monopolistic competition between producers of nal goods.
0
Sellers face a (stochastic) iid xed cost of adjusting their price
(expressed in terms of labor time).
0
Sellers meet all demand forthcoming at their current price.
0
There is an exogenously specied demand for money.
At the individual level, rms solve a version of (10.3) where the cost
of adjustment F is assumed to be iid. Further heterogeneity across
rms is restricted to two dimensions, F; p. That is, rms may be in
different states because they began the period with a different price
or because their price adjustment cost for that period is different
from that of other rms. There is a very important consequence of
this restricted form of heterogeneity: if two rms choose to adjust,
they select the same price.
Interestingly Dotsey et al. (1999) solve the dynamic optimization
problem of a rm by using a rst-order condition. This is somewhat
surprising as we have not used rst-order conditions to characterize
the solutions to dynamic discrete choice problems. Consider the
choice of a price by a rm conditional on adjustment, as in (10.4).
The rm optimally sets the price taking into account the effects on
current prots and on the future value.
In the price-setting model the price only effects the future value if
the rm elects not to adjust in the next period. If the rm adjusts its
price in the next period, as in (10.4), then the value of the price at the
start of the period is irrelevant.
So there is a rst-order condition that weighs the effects of
the price on current prots and on future values along the no-
adjustment branch of the value function. As long as the value func-
tion of the rm along this branch is differentiable in p~, there will be
a rst-order condition characterizing this optimal choice given by
qp p~; D; P bGF ED 0 ; P 0 ; F 0 j D; P; F qV n p~; D 0 ; P 0 ; F 0

0; 10:6
qp qp
where GF is the state contingent probability of not adjusting in the

next period.
This is not quite an Euler equation as the derivative of the future
value remains in this expression. Dotsey et al. iterate this condition
forward, and using a restriction that the rm eventually adjusts, they
nd that derivatives of the primitive prot function can substitute
for qV n p~; D 0 ; P 0 ; F 0 =qp.8
The solution of the optimization problem and the equilibrium
analysis relies on a discrete representation of the possible states of
the rms. Given a value of p, there will exist a critical adjustment
cost such that sellers adjust if and only if the realized value of F is
less than this critical level. So, given the state of the system, there
is an endogenously determined probability of adjustment for each
seller. Dotsey et al. (1999) use this discrete representation, these
endogenous probabilities of adjustment, and the (common) price
charged by sellers who adjust to characterize the equilibrium evolu-
tion of their model economy.
Details on computing an equilibrium are provided in Dotsey et al.
(1999). They nd the effects of money on output as follows:
0
If the ination rate is constant at 10 percent, then prices are
adjusted at least once every ve quarters.
0
Comparing different constant ination rate regimes, the higher
the ination rate, the shorter is the average time to adjustment, and
the markup only increases slightly.
8. In principle, one could use this condition for estimation of some parameters of
the model taking orthogonality conditions as moments. See the discussion of this point
by Pakes (1994) and Aguirregabiria (1997), where the latter paper includes a labor
example.
248 Chapter 10
0
An unanticipated, permanent monetary expansion leads to higher
prices and higher output at impact, and there is some persistence in
the output effects.
0
As the money shocks become less persistent, the price response
dampens and consequently the output effect is larger.
This discussion of the aggregate implications of monetary shocks
in an environment with state-dependent prices nicely complements
our earlier discussion of the estimation of a state-dependent pricing
model using micro data. Clearly, there is an open issue here con-
cerning the estimation of a state-dependent pricing model using
aggregate data.9
10.3 Optimal Inventory Policy
The models we have studied thus far miss an important element

of rm behavior, the holding of inventories. This is somewhat ironic
as the optimal inventory problem was one of the earlier dynamic
optimization problems studied in economics.10
We begin with a traditional model of inventories in which a seller
with a convex cost function uses inventories to smooth production
when demand is stochastic. We then turn to models which include
nonconvexities. The section ends with a brief discussion of a model
with dynamic choices over prices and inventories.
10.3.1 Inventories and the Production-Smoothing Model
The basic production smoothing argument for the holding of inven-

tories rests on the assumption that the marginal cost of production is
increasing. In the face of uctuating demand, the rm would then
prot by smoothing production relative to sales. This requires the
rm to build inventories in periods of low demand and to liquidate
them in periods of high demand.
Formally, consider
Vs; I max rs cy bEs 0 j s Vs 0 ; I 0 for all s; I: 10:7

y
9. The ndings of Dotsey et al. (1999) are based on a parameterization of the adjust-
ment cost distribution and the other assumptions noted above. Whether these prop-
erties obtain in an estimated model is an open issue. See Willis (2000b) for progress on
this issue.
10. See the discussion in Arrow et al. (1951) and the references therein.
Here the state vector is the level of sales s and the stock of inven-
tories at the start of the period, I. The level of sales is assumed to be
random and outside of the rms control. From sales, the rm earns
revenues of rs. The rm chooses its level of production y where
cy is a strictly increasing, strictly convex cost function. Inventories
at the start of the next period are given by a transition equation
I 0 RI y s; 10:8
where R is the return on a marginal unit of inventory (which may
be less than unity).11 From this problem, a necessary condition for
optimality is
c 0 y bREs 0 j s c 0 y 0 ; 10:9
where future output is stochastic and will generally depend on the
sales realization in the next period.
To make clear the idea of production smoothing, suppose that
sales follow an iid process: Es 0 j s s is independent of s. In this case the
right-hand side of (10.9) is independent of the current realization of
sales. Hence, since (10.9) must hold for all s, the left-hand side must
be constant too. Since production costs are assumed to be strictly
convex, this implies that y must be independent of s.
exercise 10.1 Solve (10.7) using a value function iteration routine

(or another for comparison purposes). Under what conditions will
the variance of production be less than the variance of sales?
Despite its appeal, the implications of the production smoothing
model contrast sharply with observation. In particular, the models
prediction that production will be smoother than sales, but the data
do not exhibit such production smoothing.12
One response to this difference between the models predictions
and observation is to introduce other shocks into the problem to
increase the variability of production. A natural candidate would
be variations in productivity or the costs of production. Letting A
denote a productivity shock, consider
11. Taken literally R in excess of unity means that inventories accumulate on their
own, which may seem odd. The literature is much more explicit about various mar-
ginal gains to holding inventories. If R is less than unity, than output will be inde-
pendent of the state but will be rising over time. This policy may require negative
inventories, an issue we address below.
12. See Blinder (1986), Blinder and Maccini (1991), and the references therein, for the
extensive literature on these points.
250 Chapter 10
Vs; I; A max rs c y; A bEA 0 ; s 0 j A; s Vs 0 ; I 0 ; A 0 10:10

y
so that the cost of producing y units is stochastic. In this case, (10.9)

becomes
cy y; A bREA 0 ; s 0 j A; s cy y 0 ; A 0 : 10:11
In this case inventories are used so that goods can be produced dur-
ing periods of relatively low cost and, in the absence of demand
variations, sold smoothly over time.13
Kahn (1987) studies a model of stock-out avoidance. Note that in
(10.7), the seller was allowed to hold negative inventories. As dis-
cussed in Kahn (1987), some researchers add a nonnegativity con-
straint to the inventory problem, while others are more explicit
about a cost of being away from a target level of inventories (e.g., a
fraction of sales). Kahn (1987) nds that even without a strictly con-
vex cost function, the nonnegativity constraint alone can increase the
volatility of output relative to sales.
exercise 10.2 Solve (10.10) using a value function iteration routine

(or another for comparison purposes). Under what conditions on the
variance of the two types of shocks and on the cost function will the
variance of production be less than the variance of sales? Supple-
ment the model with a nonnegativity constraint on inventories and/
or an explicit target level of investment. Explore the relationship
between the variance of sales and the variance of output.
Alternatively, researchers have introduced nonconvexities into

this problem. One approach, as in Cooper and Haltiwanger (1992), is
to introduce production bunching due to the xed costs of a pro-
duction run. For that model consider a version of (10.7) where the
cost of production is given by
8
<0 for y 0;
c y K ay for y A 0; Y; 10:12
:
y otherwise.
Here Y represents the total output produced if there is a produc-

tion run. It represents a capacity constraint on the existing capital.
Production is naturally more volatile than sales as the rm has an
13. See, for example, the discussion in Blinder (1986), Eichenbaum (1989), and Chris-
tiano (1988).
incentive to have a large production run and then to sell from

inventories until the next burst of production.14
Further, the original inventory models that gave rise to the devel-
opment of the S; s literature were based on a xed cost of order-
ing.15 One dynamic stochastic formalization of the models discussed
in Arrow et al. (1951) might be
vx; y maxfv o x; y; v n x; yg; 10:13

where x measures the state of demand and y the inventories on hand
at the sales site. The optimizer has two options: to order new goods
for inventory v o or not v n . These options are dened as
v o x; y max rs cq K bEx 0 j x vx 0 ; y s q1 d 10:14

q
and
v n x; y rs bEx 0 j x vx 0 ; y s1 d: 10:15
Here s is a measure of sales and is given as the maximum of x; y:

demand can only be met from inventories on hand. The function rs
is simply the revenues earned from selling s units.
If the rm orders new inventories, it incurs a xed cost of K and
pays cq, an increasing and convex function, to obtain q units. In the
case of ordering new goods, the inventories in the next period reect
the sales and the new orders. The rate of inventory depreciation is
given by d.
If the rm does not order inventories, then its inventories in the
following period are the depreciated level of initial inventories less
sales. This is zero as the rm stocks out.
This problem, which is similar to the stochastic investment prob-
lem with nonconvex adjustment costs, can be easily solved numeri-
cally. It combines a discrete choice along with a continuous decision
given that the rm decides to order new goods.16
14. Hall (2000) studies a model of production scheduling using data on automobile
assembly plants and nds some support for hypothesis that nonconvexities in the
production process lie behind the observations on the relative volatility of production
and sales.
15. See Scarf (1959) for developments of this argument.
16. Hall and Rust (2000) examine a model of optimal inventory behavior in an envi-
ronment where there is a xed ordering cost with a stochastic product price. They
argue that a calibrated version of their model ts important aspects of their data from
a U.S. steel wholesaler.
252 Chapter 10
10.3.2 Prices and Inventory Adjustment
Thus far we have treated pricing problems and inventory problems

separately. So, in the model of costly price adjustment, sellers had
no inventories. In the inventory models, sales are usually taken as
given. Yet there is good reason to think jointly about pricing deci-
sions and inventories.17
First, one of the motivations for the holding of inventories is to
smooth production relative to sales. But there is another mechanism
for smoothing sales: as its demand uctuates, the rm (assuming it
has some market power) could adjust its price. Yet, if prices are
costly to adjust, this may be an expensive mechanism. So the choices
of pricing and inventory policies reect the efcient response of a
prot maximizing rm to variations in demand and/or technology.
At one extreme, suppose that the rm can hold inventories and
faces a cost of changing its price. In this case the functional equation
for the rm is given by
V p; I; S; P maxfV a p; I; S; P; V n p; I; S; Pg; 10:16

where
V a p; I; S; P max p p~; I; S; P F bES 0 ; P 0 j S; P V p~; I 0 ; S 0 ; P 0 ;

p~
10:17
V n p; I; S; P pp; I; S; P bES 0 ; P 0 j S; P Vp; I 0 ; S 0 ; P 0 ; 10:18
where the transition equation for inventories is again I 0

RI y s. In this optimization problem, p is again the price of the
seller and I is the stock of inventories. These are both controlled by
the rm. The other elements in the state vector, S and P, represent a
shock to prots and the general price level respectively. The function
p p; I; S; P represent the ow of prot when the rm charges a price
p, holding inventories I when the demand shock is S and the general
price level is P.
Here, in contrast to the inventory problems described above, sales
are not exogenous. Instead, sales come a stochastic demand function
that depends on the rms price p and the price index P. From
this we see that the rm can inuence sales by its price adjustment.
But, of course, this adjustment is costly so that the rm must balance
17. Kahn (1987) includes a period of price predetermination.

meeting uctuating demand through variations in inventories, vari-

ations in production or through price changes. The optimal pattern
of adjustment will presumably depend on the driving process of the
shocks, the cost of price adjustment and the curvature of the pro-
duction cost function (underlying p p; I; S; P).
exercise 10.3 A recent literature asserts that technology shocks are
negatively correlated with employment in the presence of sticky
prices. Use (10.19) to study this issue by interpreting S as a technol-
ogy shock.
At the other extreme, suppose that new goods are delivered infre-
quently due to the presence of a xed ordering cost. In that case the
rm will seek other ways to meet uctuations in demand, such as
changing its price. Formally, consider the optimization problem of
the seller if there is a xed cost to ordering and, in contrast to (10.13),
prices are endogenous:
Vp; I; S; P maxfV o p; I; S; P; V n p; I; S; Pg; 10:19
where
V o p; I; S; P max p p~; I; S; P K cq bES 0 ; P 0 j S; P V p~; I 0 ; S 0 ; P 0 ;

p~; q
10:20
V n p; I; S; P max p p~; I; S; P bES 0 ; P 0 j S; P V p~; I 0 ; S 0 ; P 0 : 10:21
p~
The transition equation for inventories is again I 0 RI q s.

Aguirregabiria (1999) studies a model with menu costs and lump-
sum costs of adjusting inventories. This research is partly motivated
by the presence of long periods of time in which prices are not
adjusted and by observations of sales promotions.
Interestingly the model has predictions for the joint behavior of
markups and inventories even if the costs of adjustment are inde-
pendent. Aguirregabiria (1999) argues that markups will be high
when inventories are low. This reects the effects of stock-outs on
the elasticity of sales. Specically, Aguirregabiria assumes that
s minDp; q I; 10:22
where, as above, s is sales, q is orders of new goods for inventory,
and I is the stock of inventories. Here D p represents demand that
depends, among other things, on the current price set by the seller.
254 Chapter 10
So, when demand is less than output and the stock of inventories,
then sales equal demand and the price elasticity of sales is equal to
that of demand. But, when demand exceeds q I, then the elasticity
of sales with respect to price is zero: when the stock-out constraint
binds, realized demand is very inelastic. In the model of Aguirre-
gabiria (1999) the rm chooses its price and the level of inventories
prior to the realizations of a demand shock so that stock-outs may
occur.
Aguirregabiria (1999) estimates the model using monthly data on
a supermarket chain. His initial estimation is of a reduced form
model for the choice to adjust prices and/or inventories. In this dis-
crete choice framework he nds an interesting interaction between
the adjustments of inventories and prices. The level of inventories
are signicant in the likelihood of price adjustment: large inventories
increases the probability of price adjustment.
Aguirregabiria (1999) estimates a structural model based upon a
dynamic programming model.18 He nds support for the presence of
both types of lump-sum adjustment costs. Moreover he argues that
the costs of increasing a price appear to exceed the cost of price
reductions.
10.4 Capital and Labor
The grand problem we consider here allows for adjustment costs for
both labor and capital.19 Intuitively many of the stories of adjust-
ment costs for one factor have implications for the adjustment of the
other. For example, if part of the adjustment cost for capital requires
the shutting down of a plant to install new equipment, then this
may also be a good time to train new workers. Moreover we observe
inaction in the adjustment of both labor and capital and bursts as
well. So it seems reasonable to entertain the possibility that both
factors are costly to adjust and that the adjustment processes are
interdependent.
For this more general dynamic factor demand problem, we assume
that the dynamic programming problem for a plant is given by
18. The estimation methodology is complex and the reader is urged to study Aguir-
regabiria (1999).
19. Estimation of this more general structure using plant level data is part of ongoing
research of R. Cooper and J. Haltiwanger. See Sakellaris (2001) for some interesting
facts concerning the interaction of capital and labor adjustment.
VA; K; L max
0 0
PA; K; L 0 ; h oL 0 ; h; K; A CA; K; L; K 0 ; L 0
K ;L ;h
bEA 0 j A VA 0 ; K 0 ; L 0 for all A; K; L: 10:23

Here the ow of prots, PA; K; L 0 ; h, depends on the protability
shock A, the predetermined capital stock K, the number of workers
L 0 , and the hours workers h. The function oL 0 ; h; K; A represents
the total state-dependent compensation paid to workers. Finally, the
general adjustment cost function is given by CA; K; L; K 0 ; L 0 .
To allow the model to capture inaction, the adjustment cost func-
tion in (10.23) contains convex and nonconvex adjustment costs for
both labor and capital. Further one or both of these components
might be interactive. So, for example, there may be a xed cost of
adjusting capital that may cover any adjustments in labor as well.
Or, within the convex piece of the adjustment cost function, there
may be some interaction between the factors.
Writing down and analyzing this dynamic optimization problem
is by itself not difcult. There are some computational challenges
posed by the larger state space. The key is the estimation of the
richer set of parameters.
One approach would be to continue in the indirect inference spirit
and consider a VAR estimated from plant-level data in, say, hours,
employment, and capital. As with the single factor models, we might
also include some nonlinearities in the specication. We could use
the reduced form parameters as the basis for indirect inference of the
structural parameters.
One of the interesting applications of the estimated model will
be policy experiments. In particular, the model with both factors will
be useful in evaluating the implications of policy that directly inu-
ences one factor on the other. So, for example, we can study how
restrictions on worker hours might inuence the demand for equip-
ment. Or, how do investment tax credits impact on labor demand?
10.5 Technological Complementarities: Equilibrium Analysis
Here we continue discussion of a topic broached in chapter 5 where

we studied the stochastic growth model. There we noted that
researchers, starting with Bryant (1983) and Baxter and King (1991),
introduced interactions across agents through the production func-
tion. The model captures, in a tractable way, the theme that high
256 Chapter 10
levels of activity by other agents increases the productivity of each

agent.20
Let y represent the output at a given rm, Y be aggregate output, k
and n the rms input of capital and labor respectively. Consider a
production function of
y Ak a n f Y g Y1
e
; 10:24
where A is a productivity shock that is common across producers.

Here g parameterizes the contemporaneous interaction between pro-
ducers. If g is positive, then there is a complementarity at work: as
other agents produce more, the productivity of the individual agent
increases as well. In addition this specication allows for a dynamic
interaction as well parameterized by e, where Y1 is the lagged level
of aggregate output. As discussed in Cooper and Johri (1997), this
may be interpreted as a dynamic technological complementarity or
even as a learning-by-doing effect. This production function can be
embedded into a stochastic growth model.
Consider the problem of a representative household with access
to a production technology given by (10.24). This is essentially a ver-
sion of the stochastic growth model with labor but with a different
technology.
There are two ways to solve this problem. The rst is to write the
dynamic programming problem, carefully distinguishing between
individual and aggregate variables. As in our discussion of the
recursive equilibrium concept, a law of motion must be specied for
the evolution of the aggregate variables. Given this law of motion,
the individual households problem is solved and the resulting
policy function compared to the one that governs the economywide
variables. If these policy functions match, then there is an equilib-
rium. Else, another law of motion for the aggregate variables is
specied and the search continues. This is similar to the approach
described above for nding the equilibrium in the state-dependent
pricing model.21
20. This is the underlying theme of the macroeconomic complementarities literature,

as in Cooper (1999).
21. In contrast to the contraction mapping theorem, there is no guarantee that this
process will converge. In some cases the households response to an aggregate law of
motion can be used as the next guess on the aggregate law of motion. Iteration of this
may lead to a recursive equilibrium.
Alternatively, one can use the rst-order conditions for the indi-
viduals optimization problem. As all agents are identical and all
shocks are common, the representative household will accumulate
its own capital, supply its own labor, and interact with other agents
only due to the technological complementarity. In a symmetric
equilibrium, yt Yt . As in Baxter and King (1991), this equilibrium
condition is neatly imposed through the rst-order conditions when
the marginal products of labor and capital are calculated. From the
set of rst-order conditions, the symmetric equilibrium can be ana-
lyzed through by approximation around a steady state.
The distinguishing feature of this economy from the traditional
real business cycle model is the presence of the technological com-
plementarity parameters, g and e. It is possible to estimate these
parameters directly from the production function or to infer them
from the equilibrium relationships.22
10.6 Search Models
This is a very large and active area of research in which the struc-
tural approach to individual decision making has found fertile
ground. This partly reects the elegance of the search problem at the
individual level, the important policy question surrounding the pro-
vision of unemployment insurance and the existence of rich data sets
on rms and workers. This subsection will only introduce the prob-
lem and briey touch on empirical methodology and results.
10.6.1 A Simple Labor Search Model
Our starting point is a model in the spirit of McCall (1970).23 A pro-

spective worker has a job offer, denoted by o. If this job is accepted,
then the worker stays in this job for life and receives a return of
uo=1 b. Alternatively, the offer can be rejected. In this case the
worker can receive unemployment benets of b for a period and then
may draw again from the distribution. Assume that the draws from
the wage distribution are iid.24
22. See Cooper (1999) and the references therein.

23. Interestingly McCall mentions that his paper draws on Stanford notes from a class
taught by K. Arrow on the reservation wage property.
24. This model is frequently used for expositional purposes in other presentations of
the search process. It can be enriched in many ways by adding res, quits, and costly
search, for example.
258 Chapter 10
The Bellman equation for a worker with a wage offer of o in hand

is

uo
vo max ; ub bEvo 0 for all o: 10:25
1b
The worker either accepts the job, the rst option, or rejects it in
favor of taking a draw in the next period.
Given the assumption of iid draws, the return to another draw,
Evo 0 , is just a constant, denoted k. It is intuitive to think of this
functional equation from the perspective of value function iteration.
For a given value of k, (10.25) implies a function vo. Use this to
create a new expected value of search and thus a new value for k.
Continue to iterate in this fashion until the process converges.25
Clearly, the gain to accepting the job is increasing in o, while
the return associated with rejecting the job and drawing again is
independent of o. Assuming that the lower (upper) support of the
wage offer distribution is sufciently low (high) relative to b, there
will exist a critical wage, termed the reservation wage, such that the
worker is indifferent between accepting and rejecting the job. The
reservation wage, w is determined from
uw
ub bk; 10:26
1b
where
y
k Evw vw dFw
y
y
uw
Fw ub bk dFw: 10:27
w 1 b
For wages below the reservation wage, the value v is constant

and independent of w as the individual chooses to stay in unem-
ployment. For wages above w , the individual accept the offer and
gets the utility of the wage forever.
exercise 10.4 Write a program to solve (10.25) using the approach
suggested above.
25. Writing a small program to do this would be a useful exercise. Note that this
dynamic programming model is close to the discrete cake-eating problem presented
in chapters 2 to 4.
10.6.2 Estimation of the Labor Search Model
There is now a large literature on the estimation of these models.

Here we focus on estimating the simple model given above and then
discuss other parts of the literature.
The theory implies that there exists a reservation wage that
depends on the underlying parameters of the search problem:
w Y.26 Suppose that the researcher has data on a set of I individu-
als over T periods. In particular, suppose that an observation for
agent i in period t is zit A f0; 1g, where zit 0 means that the agent is
searching and zit 1 means that the agent has a job. For the purpose
of discussion we assume that the model is correct: once an agent has
a job, he keeps it forever.
Consider then the record for agent i who, say, accepted a job in
period k 1. According to the model, the likelihood of this is
Fw k 1 Fw : 10:28
The likelihood function for this problem is equivalent to the coin-
ipping example that we introduced in chapter 4. There we saw that
the likelihood function would provide a way to estimate the proba-
bility of heads but would not allow the researcher to identify the
parameters that jointly determine this probability.
The same point is true for the search problem. Using (10.28) for all
agents in the sample, we can represent the likelihood of observing
the various durations of search. But, in the end, the likelihood will
only depend on the vector Y through w .
Wolpin (1987) estimates a version of this search model with a
nite horizon and costly search. This implies, among other things,
that the reservation wage is not constant as the problem is no longer
stationary. Instead, he argues that the reservation wage falls over
time.27 This time variation in the reservation wage is useful for
identication since it creates time variation in the acceptance proba-
bility for given Y.
Wolpin (1987) also assumes that agents receive an offer each
period with a probability less than one. In order to estimate the
model, he species a function for the likelihood an agent receives an
26. Here Y would include the parameters for the individual agent (e.g., those charac-
terizing uw as well as b) and the parameters of the wage distribution.
27. Sometimes unobserved heterogeneity is added to create the same effect.
260 Chapter 10
offer in a given period. This probability is allowed to depend on the

duration of unemployment.
Wolpin uses data on both duration to employment and accepted
wages. The addition of wage data is interesting for a couple of rea-
sons. First, the lowest accepted wage yields an upper bound on
the reservation wage. Second, the researcher generally observes the
accepted wage but not the offered wage. Thus there is an interesting
problem of deducing the wage distribution from data on accepted
wages.
Wolpin (1987) estimates the model using a panel from the 1979
NLS youth cohort. In doing so, he allows for measurement error in
the wage and also species a distribution for wage offers. Among
other things, he nds that a log-normal distribution of wages ts
better than a normal distribution. Further the estimated hazard
function (giving the likelihood of accepting a job after j periods of
search) mimics the negative slope of that found in the data.
10.6.3 Extensions
Of course, much has been accomplished in the search literature over

the recent years. This includes introducing equilibrium aspects to the
problem so that the wage distribution is not completely exogenous.
Other contributions introduce bargaining and search intensity, such
as in Eckstein and Wolpin (1995). Postel-Vinay and Robin (2002)
develop an equilibrium model where the distribution of wage offers
is endogenous to the model and results from heterogenous workers
and rms and from frictions in the matching process. The model is
then estimated on French data by maximum likelihood techniques.
The simple model of labour search (10.25) can be extended to
include transitions into unemployment, learning by doing and expe-
rience effects, as well as the effect of unobserved heterogeneity.
The model of labor search can also be extended to education choices.
The education choices can be a function of an immediate cost of
education and the future rewards in terms of increased wages. Eck-
stein and Wolpin (1999) develop such a model.
Wages and Experience

The model in (10.25) can also be extended to understand why wages
are increasing in age. An important part of the labor literature has
tried to understand this phenomenon. This increase can come from
two sources, either through an increase in productivity, through

general experience or possibly seniority within the rm, or through
labor market mobility and on-the-job search. Altonji and Shakotko
(1987), Topel (1991), Altonji and Williams (1997), and Dustmann and
Meghir (2001) explore these issues, although in a nonstructural
framework.
Distinguishing the effect of experience from seniority is mainly
done by comparing individuals with similar experience but with
different seniority. However, seniority depends on the job-to-job
mobility which is a choice for the agent, possibly inuenced by het-
erogeneity in the return to experience. Hence seniority (and experi-
ence) has to be considered as an endogenous variable. It is difcult to
nd good instruments that can deal with the endogeneity. Altonji
and Shakotko (1987) instrument the seniority variable with devia-
tions from job means, while Dustmann and Meghir (2001) use
workers who are red when whole plants close down as an exoge-
nous event.
We present a structural model below that can potentially be used
to distinguish between the two sources of wage determinants. The
wage is a function of labor market experience X, of seniority in the
rm S, of an unobserved xed component e, which is possibly indi-
vidual specic, and a stochastic individual component h which is
specic to the match between the agent and the rm and is poten-
tially serially correlated. An employed individual earns a wage
wX; S; e; h. At the end of the period the agent has a probability d of
being red. If not, in the next period the individual receives a job
offer represented by a wage wX 0 ; 0; e; h~0 . This is compared to a wage
within the rm of wX 0 ; S 0 ; e; h 0 . The value of work and of unem-
ployment are dened as28
V W X; S; e; h wX; S; e; h bdV U X 0 ; e
b1 dEh 0 j h; h~0 maxV W X 0 ; S 0 ; e; h 0 ; V W X 0 ; 0; e; h~0 ;

10:29
V U X; e bX bEh 0 maxV U X; e; V W X; 0; e; h 0 :
When employed, the labor market experience evolves as X 0 X 1,
and seniority, S, evolves in a similar way. When unemployed, the
individual earns an unemployment benet bX and receive at the
28. Adda et al. (2002) estimate a related model using panel data of German workers.
262 Chapter 10
end of the period a job offer characterized by a wage wX; 0; e; h 0 .

The individual then chooses whether to accept the job or to remain
for at least an additional period in unemployment.
An important issue is the unobserved heterogeneity in the return
to experience. The model capture this with the term e. Here the
identication of the different sources of wage growth comes from
the structural framework and no instruments are needed. This
model could be solved numerically using a value function iteration
approach and then estimated by maximum likelihood, integrating
out the unobserved heterogeneity. This can be done as in Heckman
and Singer (1984) allowing for mass point heterogeneity (e.g., see
Eckstein and Wolpin 1999 for an implementation in the context of
a structural dynamic programming problem).
Equilibrium Search
Yashiv (2000) species and estimates a model of search and match-
ing. The important feature of this exercise is that it accounts for the
behavior of both rms and workers. In this model unemployed
workers search for jobs and rms with vacancies search for workers.
Firms have stochastic prot functions and face costs of attracting
workers through the posting of vacancies. Workers have an objective
of maximizing the discounted expected earnings. Workers too face a
cost of search and choose their search intensity. These choices yield
Euler equations which are used in the GMM estimation.
The key piece of the model is a matching function that brings the
search of the workers and the vacancies of the rms together. The
matching function has inputs of the vacancies opened by rms
and the search intensity by the unemployed workers. Though all
agents (rms and workers) take the matching probability as given,
this probability is determined by their joint efforts in equilibrium.
Empirically an important component of the analysis is the estima-
tion of the matching function. Yashiv (2000) nds that the matching
function exhibits increasing returns, contrary to the assumption
made in much of the empirical literature on matching.
There is a very interesting link between this research and the dis-
cussion of dynamic labor demand. While researchers have specied
labor adjustment costs, the exact source of these costs is less clear.
The analysis in Yashiv (2000) is a step toward bridging this gap: he
provides an interpretation of labor adjustment costs in the estimated
search model.
10.7 Conclusion
The intention of this book was to describe a research methodology

for bringing dynamic optimization problems to panel and time series
data. In this chapter we have described some ongoing research pro-
grams that utilize this methodology.
Still, there are many avenues for further contributions. In particu-
lar, the applications described here have generally been associated
with the dynamic optimization problem of a single agent. Of course,
this agent may be inuenced by relative prices, but these prices have
been exogenous to the agent.
This does not present a problem as long as we are content to study
individual optimization. But, as noted in the motivation of the book,
one of the potential gains associated with the estimation of structural
parameters is the condence gained in the examination of alternative
policies. In that case we need to include policy-induced variations
in equilibrium variables. That is, we need to go beyond the single-
agent problem to study equilibrium behavior. While some progress
has been made on these issues, estimation of a dynamic equilib-
rium model with heterogeneous agents and allowing for nonconvex
adjustment of factors of production and/or prices still lies ahead. 29
Related to this point, the models we have studied do not allow
any strategic interaction between agents. One might consider the
estimation of a structure in which a small set of agents interact in
a dynamic game. The natural approach is to compute a Markov-
perfect equilibrium and use it as a basis for estimating observed
behavior by the agents. Pakes (2000) provides a thorough review of
these issues in the context of applications in industrial organization.
Again, extensions to applied problems in macroeconomics lie ahead.
29. As noted earlier, Willis (2000b) makes some progress on this in a pricing prob-
lem, and Thomas (2000) studies some of these issues in the context of an investment
problem.
Bibliography
Abel, A., and J. Eberly. 1994. A unied model of investment under uncertainty. Amer-
ican Economic Review 94: 136984.
Abowd, J., and D. Card. 1989. On the covariance structure of earnings and hours
changes. Econometrica 57: 41145.
Adda, J., and R. Cooper. 2000a. Balladurette and Juppette: A discrete analysis of
scrapping subsidies. Journal of Political Economy 108(4): 778806.
Adda, J., and R. Cooper. 2000b. The dynamics of car sales: A discrete choice approach.
NBER Working Paper 7785.
Adda, J., C. Dustmann, C. Meghir, and J.-M. Robin. 2002. Human capital investment
and job transitions. Mimeo. University College London.
Adda, J., and J. Eaton. 1997. Borrowing with unobserved liquidity constraints: Struc-
tural estimation with an application to sovereign debt. Mimeo. Boston University.
Aguirregabiria, V. 1997. Estimation of dynamic programming models with censored
dependent variables. Investigaciones Economicas 21: 167208.
Aguirregabiria, V. 1999. The dynamics of markups and inventories in retailing rms.
Review of Economic Studies 66: 275308.
Altonji, J., and R. Shakotko. 1987. Do wages rise with job seniority? Review of Economic
Studies 54(3): 43759.
Altonji, J., and N. Williams. 1997. Do wages rise with job security? Review of Economic
Studies 54(179): 43760.
Altug, S. 1989. Time to build and aggregate uctuations: Some new evidence. Interna-
tional Economic Review 30: 889920.
Amman, H. M., D. A. Kendrick, and J. Rust. 1996. Handbook of Computational Econom-
ics, vol. 1. Amsterdam: Elsevier Science, North-Holland.
Arrow, K. J., T. Harris, and J. Marschak. 1951. Optimal inventory policy. Econometrica
19(3): 25072.
Attanasio, O. 2000. Consumer durables and inertial behaviour: Estimation and

aggregation of S; s rules for automobile purchases. Review of Economic Studies 67(4):
66796.
266 Bibliography
Attanasio, O., J. Banks, C. Meghir, and G. Weber. 1999. Humps and bumps in lifetime
consumption. Journal of Business and Economic Statistics 17(1): 2235.
Ball, L., and D. Romer. 1990. Real rigidities and the non-neutrality of money. Review
of Economic Studies 57(2): 183204.
Bar-Ilan, A., and A. Blinder. 1988. The life-cycle permanent-income model and con-
sumer durables. Annales dEconomie et de Statistique (9): 7191.
Bar-Ilan, A., and A. S. Blinder. 1992. Consumer durables: Evidence on the optimality
of usually doing nothing. Journal of Money, Credit and Banking 24: 25872.
Baxter, M. 1996. Are consumer durables important for business cycles? Review of Eco-
nomics and Statistics 77: 14755.
Baxter, M., and R. King. 1991. Production externalities and business cycles. Discussion
Paper 53. Federal Reserve Bank of Minneapolis.
Bellman, R. 1957. Dynamic Programming. Princeton: Princeton University Press.
Benassy, J. 1982. The Economics of Market Disequilibrium. New York: Academic Press.
Beneveniste, L., and J. Scheinkman. 1979. On the differentiability of the value function
in dynamic models of economics. Econometrica 47(3): 72732.
Benhabib, J., and R. Farmer. 1994. Indeterminacy and increasing returns. Journal of
Economic Theory 63: 1941.
Bernanke, B. 1984. Permanent income, liquidity and expenditures on automobiles:
Evidence from panel data. Quarterly Journal of Economics 99: 587614.
Bernanke, B. 1985. Adjustment costs, durables and aggregate consumption. Journal of
Monetary Economics 15: 4168.
Bertola, G., and R. J. Caballero. 1990. Kinked adjustment cost and aggregate dynamics.
NBER Macroeconomics Annual 5: 23795.
Bertsekas, D. 1976. Dynamic Programming and Stochastic Control. New York: Academic
Press.
Bils, M. 1987. The cyclical behavior of marginal cost and price. American Economic
Review 77: 83855.
Bils, M., and P. Klenow. 2002. Some evidence on the importance of sticky prices. NBER
Working Paper 9069.
Blackwell, D. 1965. Discounted dynamic programming. Annals of Mathematical Statis-
tics 36: 22635.
Blanchard, O., and N. Kiyotaki. 1987. Monopolistic competition and the effects of
aggregate demand. American Economic Review 77: 64766.
Blinder, A. 1986. Can the production smoothing model of inventory behavior be
saved? Quarterly Journal of Economics 101: 43153.
Blinder, A., and L. Maccini. 1991. Taking stock: A critical assessment of recent research
on inventories. Journal of Economic Perspectives 5(1): 7396.
Blundell, R., M. Browning, and C. Meghir. 1994. Consumer demand and the life-cycle
allocation of household expenditures. Review of Economic Studies 61: 5780.
Bibliography 267
Braun, R. 1994. Tax disturbances and real economic activity in post-war United States.
Journal of Monetary Economics 33(3): 44162.
Bryant, J. 1983. A simple rational expectations Keynes-type model. Quarterly Journal of
Economics 97: 52529.
Caballero, R. 1999. Aggregate investment. In Handbook of Macroeconomics, ed. by J.
Taylor and M. Woodford. Amsterdam: North-Holland.
Caballero, R., and E. Engel. 1993a. Heterogeneity and output uctuation in a dynamic
menu-cost economy. Review of Economic Studies 60: 95119.
Caballero, R., and E. Engel. 1993b. Heterogeneity and output uctuations in a dynamic
menu-cost economy. Review of Economic Studies 60: 95119.
Caballero, R., E. Engel, and J. Haltiwanger. 1995. Plant level adjustment and aggregate
investment dynamics. Brookings Papers on Economic Activity 1995(2): 139.
Caballero, R., E. Engel, and J. Haltiwanger. 1997. Aggregate employment dynamics:
Building from microeconomic evidence. American Economic Review 87: 11537.
Caballero, R. J. 1993. Durable goods: An explanation for their slow adjustment. Journal
of Political Economy 101: 35184.
Campbell, J., and G. Mankiw. 1989. Consumption, income and interest rates: Reinter-
preting the time series evidence. NBER Macroeconomic Annual 4: 150.
Caplin, A., and J. Leahy. 1991. State dependent pricing and the dynamics of money
and output. Quarterly Journal of Economics 106: 683708.
Caplin, A., and J. Leahy. 1997. Durable goods cycles. Mimeo. Boston University.
Carroll, C. D. 1992. The buffer-stock theory of saving: Some macroeconomic evidence.

Brookings Papers on Economic Activity 2: 61156.
Cecchetti, S. 1986. The frequency of price adjustment: A study of newstand prices of
magazines. Journal of Econometrics 31: 25574.
Chirinko, R. 1993. Business xed investment spending. Journal of Economic Literature
31: 18751911.
Christiano, L. 1988. Why does inventory investment uctuate so much? Journal of
Monetary Economics 21(2): 24780.
Christiano, L., and M. Eichenbaum. 1992. Current real-business-cycle theories and
aggregate labor market uctuations. American Economic Review 82: 43050.
Cooper, R. 1999. Coordination Games: Complementarities and Macroeconomics. Cam-
bridge: Cambridge University Press.
Cooper, R. 2002. Estimation and identication of structural parameters in the presence
of multiple equilibria. NBER Working Paper 8941.
Cooper, R., and J. Ejarque. 2000. Financial intermediation and aggregate uctuations:
A quantitative analysis. Macroeconomic Dynamics 4: 42347.
Cooper, R., and J. Ejarque. 2001. Exhuming Q: Market power vs. capital market
imperfections. NBER Working Paper 8182.
268 Bibliography
Cooper, R., and J. Haltiwanger. 1993. On the aggregate implications of machine

replacement: Theory and evidence. American Economic Review 83: 36082.
Cooper, R., and J. Haltiwanger. 2000. On the nature of the capital adjustment process.
Cooper, R., J. Haltiwanger, and L. Power. 1999. Machine replacement and the business
cycle: Lumps and bumps. American Economic Review 89: 92146.
Cooper, R., and A. Johri. 1997. Dynamic complementarities: A quantitative analysis.
Journal of Monetary Economics 40: 97119.
Cooper, R., and J. Willis. 2001. The economics of labor adjustment: Mind the gap.
Cooper, R. W., and J. C. Haltiwanger. 1992. Macroeconomic implications of produc-
tion bunching. Journal of Monetary Economics 30(1): 10727.
De Boor, C. 1978. A Practical Guide to Splines. New York: Springer-Verlag.
Deaton, A. 1991. Savings and liquidity constraints. Econometrica 59: 122148.
Dotsey, M., R. King, and A. Wolman. 1999. State-dependent pricing and the general
equilibrium dynamics of prices and output. Quarterly Journal of Economics 114: 65590.
Dufe, D., and K. Singleton. 1993. Simulated moment estimation of Markov models of
asset prices. Econometrica 61(4): 92952.
Dustmann, C., and C. Meghir. 2001. Wages, experience and seniority. IFS Working
Paper W01/01.
Eberly, J. C. 1994. Adjustment of consumers durables stocks: Evidence from automo-
bile purchases. Journal of Political Economy 102: 40336.
Eckstein, Z., and K. Wolpin. 1989. The specication and estimation of dynamic sto-
chastic discrete choice models. Journal of Human Resources 24: 56298.
Eckstein, Z., and K. Wolpin. 1995. Duration to rst job and the return to schooling:
Estimates from a search-matching model. Review of Economic Studies 62: 26386.
Eckstein, Z., and K. I. Wolpin. 1999. Why youths drop out of high school: The impact
of preferences, opportunities and abilities. Econometrica 67(6): 12951339.
Eichenbaum, M. 1989. Some empirical evidence on the production level and produc-
tion cost smoothing models of inventory investment. American Economic Review 79(4):
85364.
Eichenbaum, M., L. Hansen, and K. Singleton. 1988. A time series analysis of repre-
sentative agent models of consumption and leisure choice under uncertainty. Quarterly
Journal of Economics 103: 5178.
Eichenbaum, M., and L. P. Hansen. 1990. Estimating models with intertemporal sub-
stitution using aggregate time series data. Journal of Business and Economic Statistics 8:
5369.
Erickson, T., and T. Whited. 2000. Measurement error and the relationship between
investment and Q. Journal of Policy Economy 108: 102757.
Farmer, R., and J. T. Guo. 1994. Real business cycles and the animal spirits hypothesis.
Journal of Economic Theory 63: 4272.
Bibliography 269
Fermanian, J.-D., and B. Salanie. 2001. A nonparametric simulated maximum likeli-

hood estimation method. Mimeo. CREST-INSEE.
Fernandez-Villaverde, J., and D. Krueger. 2001. Consumption and saving over the life
cycle: How important are consumer durables? Mimeo. Stanford University.
Flavin, M. 1981. The adjustment of consumption to changing expectations about future
income. Journal of Political Economy 89: 9741009.
Gallant, R. A., and G. Tauchen. 1996. Which moments to match? Econometric Theory
12(4): 65781.
Gilchrist, S., and C. Himmelberg. 1995. Evidence on the role of cash ow for invest-
ment. Journal of Monetary Economics 36: 54172.
Gomes, J. 2001. Financing investment. American Economic Review 91(5): 126385.
Gourieroux, C., and A. Monfort. 1996. Simulation-Based Econometric Methods. Oxford:
Oxford University Press.
Gourieroux, C., A. Monfort, and E. Renault. 1993. Indirect inference. Journal of Applied
Econometrics 8: S85118.
Gourinchas, P.-O., and J. Parker. 2002. Consumption over the life cycle. Econometrica
70(1): 4789.
Greenwood, J., Z. Hercowitz, and G. Huffman. 1988. Investment, capacity utilization
and the real business cycle. American Economic Review 78: 40217.
Grossman, S. J., and G. Laroque. 1990. Asset pricing and optimal portfolio choice in
the presence of illiquid durable consumption goods. Econometrica 58: 2551.
Hajivassiliou, V. A., and P. A. Ruud. 1994. Classical estimation methods for LDV
models using simulation. In Handbook of Econometrics, ed. by D. McFadden and R.
Engle, vol. 4. Amsterdam: North-Holland, pp. 2383441.
Hall, G. 1996. Overtime, effort and the propagation of business cycle shocks. Journal of
Hall, G. 2000. Non-convex costs and capital utilization: A study of production sched-
uling at automobile assembly plants. Journal of Monetary Economics 45: 681716.
Hall, G., and J. Rust. 2000. An empirical model of inventory investment by durable
commodity intermediaries. Carnegie-Rochester Conference Series on Public Policy 52:
171214.
Hall, R. E. 1978. Stochastic implications of the life cyclepermanent income hypothe-

sis: Theory and evidence. Journal of Political Economy 86: 97187.
Hamermesh, D. 1989. Labor demand and the structure of adjustment costs. American
Economic Review 79: 67489.
Hamermesh, D. 1993. Labor Demand. Princeton: Princeton University Press.
Hamermesh, D., and G. Pfann. 1996. Adjustment costs in factor demand. Journal of
Economic Literature 34: 126492.
Hansen, G. 1985. Indivisible labor and the business cycle. Journal of Monetary Economics
16: 30927.
270 Bibliography
Hansen, G., and T. Sargent. 1988. Straight time and overtime in equilibrium. Journal of
Hansen, L. P., E. McGrattan, and T. Sargent. 1994. Mechanics of forming and estimat-
ing dynamic linear economies. Federal Reserve Bank of Minneapolis, Staff Report 182.
Hansen, L. P., and K. J. Singleton. 1982. Generalized instrumental variables estimation
of nonlinear rational expectations models. Econometrica 50: 126986.
Hayashi, F. 1982. Tobins marginal Q and average Q: A neoclassical interpretation.
Econometrica 50: 21524.
Heckman, J., and B. Singer. 1984. A method for minimizing the impact of distri-
butional assumptions in econometric models for duration data. Econometrica 52(2):
271320.
House, C., and J. Leahy. 2000. An s; S model with adverse selection. NBER Working
Paper 8030.
Hubbard, G. 1994. Investment under uncertainty: Keeping ones options open. Journal
of Economic Literature 32(4): 181631.
John, A., and A. Wolman. 1999. Does state-dependent pricing imply coordination
failure? Mimeo. Federal Reserve Bank of Richmond.
Judd, K. 1992. Projection methods for solving aggregate growth models. Journal of
Economic Theory 58: 41052.
Judd, K. 1996. Approximation, perturbation and projection methods in economic
analysis. In Handbook of Computational Economics, ed. by H. M. Amman, D. A. Ken-
drick, and J. Rust. Amsterdam: Elsevier Science, North-Holland.
Judd, K. 1998. Numerical Methods in Economics. Cambridge: MIT Press.
Kahn, A., and J. Thomas. 2001. Nonconvex factor adjustments in equilibrium business
cycle models: Do nonlinearities matter? Mimeo. University of Minnesota.
Kahn, J. 1987. Inventories and the volatility of production. American Economic Review
77(4): 66779.
Keane, M. P., and K. I. Wolpin. 1994. The solution and estimation of discrete choice
dynamic programming models by simulation and interpolation: Monte Carlo evi-
dence. Review of Economics and Statistics 76(4): 64872.
King, R., C. Plosser, and S. Rebelo. 1988. Production, growth and business cycles I. The
basic neoclassical model. Journal of Monetary Economics 21: 195232.
Kocherlakota, N., B. F. Ingram, and N. E. Savin. 1994. Explaining business cycles: A
multiple shock approach. Journal of Monetary Economics 34: 41528.
Kydland, F., and E. Prescott. 1982. Time to build and aggregate uctuations. Econo-
metrica 50: 134570.
Laffont, J.-J., H. Ossard, and Q. Vuong. 1995. Econometrics of rst-price auctions.
Econometrica 63: 95380.
Lam, P. 1991. Permanent income, liquidity and adjustments of automobile stocks:
Evidence form panel data. Quarterly Journal of Economics 106: 20330.
Bibliography 271
Laroque, G., and B. Salanie. 1989. Estimation of multi-market x-price models: An

application of pseudo maximum likelihood methods. Eca 57(4): 83160.
Laroque, G., and B. Salanie. 1993. Simulation based estimation models with lagged
latent variables. Journal of Applied Econometrics 8: S11933.
Lee, B.-S., and B. F. Ingram. 1991. Simulation estimation of time-series models. Journal
of Econometrics 47: 197205.
Lerman, S., and C. Manski. 1981. On the use of simulated frequencies to approximate
choice probabilities. In Structural Analysis of Discrete Data with Econometric Applications,
ed. by C. Manski and D. McFadden. Cambridge: MIT Press, pp. 30519.
Ljungqvist, L., and T. Sargent. 2000. Recursive Macroeconomic Theory. Cambridge: MIT
Press.
MaCurdy, T. E. 1981. An empirical model of labor supply in a life-cycle setting. Journal

of Political Economy 89(6): 105985.
Mankiw, G. N. 1982. Halls consumption hypothesis and durable goods. Journal of
Manski, C. 1993. Identication of endogenous social effects: The reection problem.
Review of Economic Studies 60(3): 53142.
McCall, J. 1970. Economics of information and job search. Quarterly Journal of Econom-
ics 84(1): 11326.
McFadden, D. 1989. A method of simulated moments for estimation of discrete
response models without numerical integration. Econometrica 57: 9951026.
McFadden, D., and P. A. Ruud. 1994. Estimation by simulation. Review of Economics
and Statistics 76(4): 591608.
McGrattan, E. 1994. The macroeconomic effects of distortionary taxes. Journal of Mon-
etary Economics 33: 573601.
McGrattan, E. R. 1996. Solving the stochastic growth model with a nite element
method. Journal of Economic Dynamics and Control 20: 1942.
Meghir, C., and G. Weber. 1996. Intertemporal non-separability or borrowing restric-
tions? A disaggregate analysis using US CEX panel. Econometrica 64(5): 115181.
Miranda, M. J., and P. G. Helmberger. 1988. The effects of commodity price stabiliza-
tion programs. American Economic Review 78(1): 4658.
Newey, W. K., and K. D. West. 1987. A simple, positive, semi-denite, hetero-

skedasticity and autocorrelation consistent covariance matrix. Econometrica 55: 7038.
Nickell, S. 1978. Fixed costs, employment and labour demand over the cycle. Economica
45(180): 32945.
Pakes, A. 1994. Dynamic structural models, problems and prospects: Mixed continu-
ous discrete controls and market interactions. In Advances in Econometrics, Sixth World
Congress, ed. by C. Sims. Cambridge: Cambridge University Press, pp. 171259.
Pakes, A. 2000. A framework for applied dynamic analysis in I.O. NBER Working
Paper 8024.
272 Bibliography
Pakes, A., and D. Pollard. 1989. Simulation and the asymptotics of optimization esti-
mators. Econometrica 57: 102757.
Pfann, G., and F. Palm. 1993. Asymmetric adjustment costs in non-linear labour
models for the Netherlands and U.K. manufacturing sectors. Review of Economic Studies
60: 397412.
Postel-Vinay, F., and J.-M. Robin. 2002. Equilibrium wage dispersion with worker and
employer heterogeneity. Econometrica, forthcoming.
Press, W., B. Flannery, S. Teukolsky, and W. Vetterling. 1986. Numerical Recipes: The
Art of Scientic Computing. Cambridge: Cambridge University Press.
Ramey, V., and M. Shapiro. 2001. Displaced capital. Journal of Political Economy 109:
95892.
Reddy, J. 1993. An Introduction to the Finite Element Method. New York: McGraw-Hill.
Rogerson, R. 1988. Indivisible labor, lotteries and equilibrium. Journal of Monetary
Economics 21: 316.
Rust, J. 1985. Stationary equilibrium in a market for durable assets. Econometrica 53(4):
783805.
Rust, J. 1987. Optimal replacement of GMC bus engines: An empirical model of

Harold Zurcher. Econometrica 55(5): 9991033.
Rust, J., and C. Phelan. 1997. How social security and medicare affect retirement
behavior in a world of incomplete markets. Econometrica 65(4): 781832.
Sakellaris, P. 2001. Patterns of plant adjustment. Working Paper 2001-05. Finance and
Economics Discussion Series, Division of Research and Statistics and Monetary Affairs,
Federal Reserve Board, Washington, DC.
Sargent, T. 1978. Estimation of dynamic labor demand schedules under rational
expectations. Journal of Political Economy 86(6): 100944.
Sargent, T. 1987. Dynamic Macroeconomic Theory. Cambridge: Harvard University
Press.
Scarf, H. 1959. The optimality of S; s policies in the dynamic inventory problem. In
Mathmematical Methods in Social Sciences, ed. by S. K. K. Arrow and P. Suppes. Stan-
ford: Stanford University Press, pp. 196202.
Shapiro, M. 1986. The dynamic demand for labor and capital. Quarterly Journal of Eco-
nomics 101: 51342.
Smith, A. 1993. Estimating nonlinear time-series models using simulated vector
autoregressions. Journal of Applied Econometrics 8: S6384.
Stokey, N., and R. Lucas. 1989. Recursive Methods in Economic Dynamics. Cambridge:
Harvard University Press.
Tauchen, G. 1986. Finite state Markov-chain approximation to univariate and vector
autoregressions. Economics Letters 20: 17781.
Tauchen, G. 1990. Solving the stochastic growth model by using quadrature methods
and value-function iterations. Journal of Business and Economic Statistics 8(1): 4951.
Bibliography 273
Tauchen, G., and R. Hussey. 1991. Quadrature-based methods for obtaining approxi-
mate solutions to nonlinear asset pricing models. Econometrica 59: 37196.
Taylor, J. B., and H. Uhlig. 1990. Solving nonlinear stochastic growth models: A
comparison of alternative solution methods. Journal of Business and Economic Statistics
8: 117.
Thomas, J. 2002. Is lumpy investment relevant for the business cycle? Manuscript.
Carnegie-Mellon University, Journal of Political Economy 110(3): 50834.
Topel, R. 1991. Specic capital, mobility, and wages: Wages rise with job seniority.
Journal of Political Economy 99(1): 14576.
Whited, T. 1998. Why do investment Euler equations fail? Journal of Business and Eco-
nomic Statistics 16(4): 47988.
Willis, J. 2000a. Estimation of adjustment costs in a model of state-dependent pricing.

Working Paper 00-07. Federal Reserve Bank of Kansas City.
Willis, J. 2000b. General equilibrium of monetary model with state dependent pricing.
Mimeo. Boston University.
Wolpin, K. 1987. Estimating a structural search model: The transition from school to
work. Econometrica 55(4): 80118.
Wright, B. D., and J. C. Williams. 1984. The welfare effects of the introduction of stor-
age. Quarterly Journal of Economics 99(1): 16992.
Yashiv, E. 2000. The determinants of equilibrium unemployment. American Economic
Review 90(5): 12971322.
Zeldes, S. 1989a. Optimal consumption with stochastic income: Deviations from cer-
tainty equivalence. Quarterly Journal of Economics 104: 27598.
Zeldes, S. P. 1989b. Consumption and liquidity constraints: An empirical investiga-
tion. Journal of Political Economy 97: 30546.
Index
Adjustment costs. See Costs of Calibration

adjustment stochastic growth model, 126
Aggregate implications Capital
durable purchases, 179 capital accumulation, 187
machine replacement, 207 costs of adjustment, 189
menu costs, 242 convex, 191
Aggregation, 179, 207 nonconvex, 202
Asymptotic properties, 7997 quadratic, 192
GMM, 80 imperfection in capital markets, 195,
indirect inference, 93 201
maximum likelihood, 83 labor adjustment, 25455
simulated maximum likelihood, 91 Car sales, 179
simulated method of moments, 87 Certainty equivalence, 151
Autocorrelation, 57 CES (constant elasticity of substitution),
Autoregressive process, 47, 58 242
Average Q, 192 Chebyshev polynomial, 4345
Coin ipping, 61
Balladurette, 182 Collocation method, 4345
Bellman equation Complementarities
denition, 17, 26 technology, 25557
example, 84, 86, 113, 150, 168, 178, 203, technology and stochastic growth, 133
252 Consumption
numerical solution, 37, 55 borrowing restrictions, 146, 156
Blackwells conditions, 27 durables (see Durable consumption)
Borrowing restrictions endogenous labor supply, 153
consumption, 146, 156 evidence, 149, 151, 159
investment, 201 GMM estimation, 15153
innite horizon, 14760
Cake-eating problem life cycle, 16064
dynamic discrete choice, 22 portfolio choice, 145, 150
example, 14 random walk, 149
nite horizon, 12 smoothing, 143, 144
innite horizon, 16 stochastic income, 143, 148
example, 18 two-period model, 13947
innite horizon with taste shocks, Contraction mapping, 27
20 Control space, 35
overview, 915 Control variable, 16
276 Index
Control vector, 24 piecewise linear adjustment costs, 224

Convergence rate, 37, 41 quadratic adjustment costs, 217
Convex adjustment costs Dynamic programming theory
durables, 169 Blackwells sufcient conditions, 27
investment, 191 cake-eating example, 9, 14
Costs of adjustment control vector, 24
capital, 189 discounting, 25, 28
convex, 191 nite horizon cake-eating problem, 12
nonconvex, 202 general formulation, 24
quadratic, 192 innite horizon cake-eating problem, 16
capital and labor, 25455 monotonicity, 27
durables optimal stopping problem, 22
nonconvex, 171 overview, 6
quadratic, 169 state vector, 24
employment stochastic models, 29
asymmetric adjustment costs, 228 transition equation, 25
convex and nonconvex adjustment value function, 12
costs, 235 value function iteration, 28
nonconvex, 226
piecewise linear, 224 Education choice, 260
quadratic, 217 Efcient method of moments, 95
CRRA (constant relative risk aversion), Elasticity
35, 149 demand curve, 220
intertemporal, 98, 153, 159, 163
Decentralization, 120 labor supply, 153
Demand (and supply), 74 Employment adjustment, 21639
Depreciation rate, 138, 166, 169 asymmetric adjustment costs, 228
Discount factor, 9, 140 convex and nonconvex adjustment
Discounting, 25, 28 costs, 235
Discrete cake-eating problem gap approach, 229
estimation, 84, 87, 88, 90, 94, 99 general functional equation, 216
numerical implementation, 46 nonconvex adjustment costs, 226
overview, 2224 partial adjustment model, 230
Distortionary taxes, 136 piecewise linear adjustment costs,
Durable consumption, 16585 224
dynamic discrete choice, 175 quadratic adjustment costs, 217
estimation quadratic costs of adjustment
dynamic discrete choice, 179 simulated example, 219
with quadratic utility, 168 Sargents linear quadratic model, 222
irreversibility, 173 Endogenous labor supply
non convex costs, 17185 consumption, 153
PIH, 16671 growth, 122
scrapping subsidies, 18285 Equilibrium analysis, 121, 256
Duration Equilibrium search, 262
search, 259 Ergodicity, 76, 78
Duration model, 94 Euler equation
Dynamic discrete choice consumption, 143, 161
durables, 175 consumption and borrowing
Dynamic labor demand, 21639 constraints, 156
estimation, 235 consumption and portfolio choice,
linear quadratic specication, 222 151
nonconvex adjustment costs, 226 durables, 167
partial adjustment, 230 employment adjustment, 223
Index 277
estimation, 81, 198 linear, 53

nite horizon cake-eating problem, 11 splines, 54
investment, 190, 191 Intertemporal elasticity of substitution,
nonstochastic growth model, 105 98, 153, 159, 163
projection methods, 4146 Inventory policy, 24854
stochastic growth model, 115, 123 prices, 252
Experience, return to, 260 production smoothing, 248
Exponential model, 94 Investment
borrowing restrictions, 201
Finite element method, 45 convex adjustment costs, 191
Functional equation, 17 convex and nonconvex adjustment
costs, 209
Galerkin method, 43 costs of adjustment, 189
Gap approach Euler equation with no adjustment
employment adjustment, 229 costs, 190
Gauss-Legendre quadrature, 5556 functional equation, 188
Generalized method of moments general formulation, 188
example GMM estimation of quadratic
capital and quadratic adjustment costs, adjustment cost model, 198
198200 irreversibility, 208
consumption, 81, 151, 155, 159 machine replacement problem, 205
employment adjustment, 229 aggregate implications, 207
stochastic growth model, 128 maximum likelihood estimation,
orthogonality restriction, 80 212
theory, 6466, 8083 no adjustment costs, 189
Government spending, 136 nonconvex adjustment costs, 202
Q theory, 192
Heterogeneity, 246, 262 evidence, 193
Howards improvement algorithm, 40 Irreversibility
durables, 172, 17375
Identication, 70, 7879 investment, 208
Imperfection, in capital market, 146, 156, IV. See Instrumental variable
195, 201
Inaction, 202, 224 Job mobility, 261
Income specication, 154 Job offer, 257
Indirect inference Juppette. See Balladurette
example
cake-eating problem, 94 Labor market
dynamic capital demand, 209 experience, 26062
Q theory of investment, 195 mobility, 26062
stochastic growth model, 130 search, 25763
supply and demand, 78 transitions, 26062
specication test, 97 wage, 26062
theory, 6870, 9397 Labor supply
Indirect utility, 7 endogenous, 122, 15355
Innite horizon Least squares interpolation, 53
consumption model, 14760 Life cycle consumption, 16064
Information matrix, 85 Likelihood, 62, 77, 84
Instrumental variable, 75, 83 Linear interpolation, 53
Integration methods, 5559 Linear quadratic model of labor demand,
quadrature method, 55 222
Interpolation methods, 5255 Linearization, 115
least squares, 53 Logit model, 69
278 Index
Machine replacement Optimal stopping problem, 22, 178, 206

aggregate implications, 207 Optimal weighting matrix, 82, 89
model, 205 Orthogonality restriction, 76, 80
Magazine prices, 244 Overidentication test, 83
Mapping, 28
Marginal q, 192 Partial adjustment model
Market power, 195, 198, 220, 236 employment, 230
Markov chain, 64 Permanent income hypothesis
as approximation, 56 durables, 16671
example, 183, 196, 221 Permanent vs. transitory shocks, 124,
simulation, 59 144, 149, 157, 227
Maximum likelihood, 6264, 7778, 83 PIH. See Permanent income hypothesis
85 Planners problem, 111
asymptotic properties, 85 Policy evaluation, 18285
example Policy function
coin ipping, 62 consumption, 158
discrete cake-eating problem, 84 denition, 18
employment adjustment, 223 Policy function iterations, 40
investment, 212 Policy rule, 42
stochastic growth model, 131 Portfolio choice, 145, 150
supply and demand, 77 and durables, 174
simulated (see Simulated maximum Price setting, 24148
likelihood) Principle of optimality, 13
Menu costs, 24148 Production smoothing, 248
aggregate implications, 245 Projection methods, 41
evidence, 244
model, 242 Q theory
Method of moments, 65, 75, 80 evidence, 193
orthogonality condition, 76 model, 192
Misspecication, 159, 195 Quadratic adjustment costs
Mobility, 261 durables, 169
Moment calibration, 86 employment, 217
Moments Quadrature methods, 55
stochastic growth model, 126
Monotonicity, 27 Random walk in consumption
Multiple sector model, 134 durables, 168
nondurables, 149, 150
Newey-West estimator, 83 Rate of convergence, 37
Nonconvex adjustment costs Recursive equilibrium, 121
durables, 171 Reduced form, 74
employment, 226 Reservation wage, 258
investment, 202 Return to
Nonstochastic growth model, experience, 260
103 tenure, 260
Euler equation, 105
example, 105 Sales of new cars, 182
Matlab code, 107 Score function, 85, 95
preferences, 104 Scrapping subsidies, 182
technology, 104 Search model, 25762
value function, 104 duration, 259
Numerical integration, 5559 Seniority, 261
quadrature methods, 55 Sequence problem, 10
Index 279
nite horizon, 10 multiple sectors, 134

Serial correlation, 56, 86 overview, 103
Simulated maximum likelihood, 66 taste shocks, 136
theory, 66, 9193 technological complementarities, 133
Simulated method of moments technology, 104
asymptotic properties, 8789 value function iteration, 117, 124
efcient method of moments, 95 Stochastic income, 143, 148
example Stochastic returns, 150
cake-eating problem, 87 Supply and demand, 74
consumption, 163
durables, 17981 Taste shock
theory, 67, 8788 aggregate, 176
Simulated nonlinear least squares cake-eating problem, 20, 4647, 86
example durables, 176
cake-eating problem, 90 in estimation, 84
durables, 17981 stochastic growth model, 132, 136
theory, 8991 Tax credits, 187
Simulation methods, 59 Taxes, 136, 142, 185
Solution methods Technological complementarities, 255.
linearization, 115 See also Complementarities
projection methods, 4146 Tenure, return to, 260
value function iteration, 3640, 4750, Transition equation, 16, 25
117 Transition matrix, 21
Specication test Transversality condition, 105
GMM, 83
indirect inference, 97 Uncertainty
Spline interpolation, 54 consumption/saving choice, 14345,
s; S models, 172, 174 14850
State space, 35, 47, 108 Unobserved heterogeneity, 174, 210,
large, 50 26062
State variable, 16 Utility
State vector, 24 quadratic, 143, 168
Stationarity, 17, 25 Utility function
Stochastic cake-eating problem adjustment costs, 170
projection methods approach, 40 CRRA, 35, 149
value function approach, 34, 47 quadratic, 143
Stochastic growth model, 103, 111
calibration, 126 Value function
confronting the data, 12532 implementation, 36, 47
calibration, 126 Value function iteration, 28, 33
GMM, 128 example, 178
indirect inference, 130 nonstochastic growth model, 107
maximum likelihood, 131 stochastic growth model, 117, 124
decentralization, 120 VAR, 177
endogenous labor supply, 122
example, 117 Wage offer, 260
functional equation, 114 Weighting matrix, 82, 89, 95
GMM, 128
indirect inference, 130
intermediation shocks, 130
investment shocks, 130
linearization, 115

Dynamic Economics: Quantitative Methods and Applications

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dynamic Economics: Quantitative Methods and Applications

Uploaded by

Copyright:

Available Formats

This page intentionally left blank

Quantitative Methods and

Jerome Adda and

The MIT Press

Library of Congress Cataloging-in-Publication Data

2 Theory of Dynamic Programming 7

3.4 Extensions and Conclusion 50

5 Stochastic Growth 103

5.6 Some Extensions 132

7 Durable Consumption 165

8.3 No Adjustment Costs 189

9 Dynamics of Employment Adjustment 215

10 Future Developments 241

10.5 Technological Complementarities: Equilibrium

In this book we study a rich set of applied problems in economics

this model are estimated, then various policy experiments can be

application in chapters 5 through 10 contains four elements: pre-

go to Victor Aguirregabiria, Yan Bai, Joyce Cooper, Dean Corbae,

The mathematical theory of dynamic programming as a means of

2.2 Indirect Utility

Intuitively, the approach of dynamic programming can be under-

Consumer choice theory focuses on households that solve

where c is a vector of consumption goods, p is a vector of prices and

where l is the multiplier on the budget constraint and uj c is the

It is in this sense that the indirect utility function summarizes the

Suppose that a rm must choose how many workers to hire at a

A labor demand function results that depends on w; p; k. As with

2.3 Dynamic Optimization: A Cake-Eating Example

Here we will look at a very simple dynamic optimization problem.

where 0 a b a 1 and b is called the discount factor.

for t 1; 2; . . . ; T. How would you nd the optimal path of con-

2.3.1 Direct Attack

One approach is to solve the constrained optimization problem

subject to the transition equation (2.1), which holds for t 1; 2; 3; . . . ;

The nonnegativity constraints are simpler: ct b 0 for t 1; 2; . . . ; T

where f is the multiplier on the nonnegativity constraint on WT1 .

Combining the equations, we obtain an expression that links con-

This is a necessary condition of optimality for any t: if it is violated,

above: l f > 0. This nonnegativity constraint serves two important

2.3.2 Dynamic Programming Approach

Suppose that we change the problem slightly: we add a period 0

Finite Horizon Problem

By adding a period 0 to our original problem, we can take advan-

for t 1; 2; . . . ; T 1. Using these two conditions together yields

for t 0; 1; 2; . . . ; T 1, a familiar necessary condition for an optimal

and the resource constraint is

Working with these two conditions, we have

Using (2.6) evaluated at a cake of size W2 , we can solve for V20 W2

Here c2 the consumption level in the second period of the three-

This plus the resource constraint allows us to construct the solution

2.4 Some Extensions of the Cake-Eating Problem

Here we go beyond the T-period problem to illustrate some ways

2.4.1 Innite Horizon

along with the transition equation of

VW max uc bVW c for all W:

Here uc is again the utility from consuming c units in the current

Alternatively, we can specify the problem so that instead of

Either specication yields the same result. But choosing tomorrows

6. As you may already know, stationarity is vital in econometrics as well. Thus

above, that uc lnc. From the results of the T-period problem,

VW A B lnW for all W:

Using this in (2.8) results in

EnextV=EnextV+nextVPi[i_y] Store the expected future