International Journal of Forecasting 25 (2009) 602–628

www.elsevier.com/locate/ijforecast

Multi-step forecasting in emerging economies: An investigation of

the South African GDP
Guillaume Chevillon ∗
ESSEC Business School, Paris, France
CREST–INSEE, France

Abstract

To forecast at several, say h, periods into the future, a modeller faces a choice between iterating one-step-ahead forecasts
(the IMS technique), or directly modeling the relationship between observations separated by an h-period interval and using
it for forecasting (DMS forecasting). It is known that structural breaks, unit-root non-stationarity and residual autocorrelation
may improve DMS accuracy in finite samples, all of which occur when modelling the South African GDP over the period
1965–2000. This paper analyzes the forecasting properties of 779 multivariate and univariate models that combine different
techniques of robust forecasting. We find strong evidence supporting the use of DMS and intercept correction, and attribute
their superior forecasting performance to their robustness in the presence of breaks.
c 2008 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

Keywords: Multi-step forecasting; Intercept correction; Structural breaks

1. Introduction the context of exponential smoothing. Klein (1971)

When a forecaster uses a model with a given
periodicity, but wishes to forecast at several, say
h > 1, periods into the future, she is faced with
a choice between iterating one-step-ahead forecasts
(iterated multi-step, or IMS, forecasting), or directly
modelling the relationship between the end-of-sample
observation and its hth successor in order to forecast
the latter (direct multi-step, or DMS). This direct
technique was first suggested by Cox (1961) in
∗ Corresponding address: ESSEC Business School, Paris, France.
E-mail address: chevillon@essec.fr.
then applied it to dynamic forecasting, and Johnston
(1974) put a temporary end to the analysis by
concluding that, when a quadratic loss function
is used as the criterion for both estimation and
forecast accuracy, the former constitutes a “reliable
indicator of prediction efficiency”, and hence direct
methods are inefficient for forecasting. However, first
Findley (1983) and then Weiss (1991) re-examined the
previous results in the context of misspecified models,
and found asymptotic relevance in matching criteria
of estimation and forecast efficiency, as with the direct
multi-step method. The renewed interest these authors
brought to the topic spurred new theoretical analyses

0169-2070/$ - see front matter
c 2008 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.
doi:10.1016/j.ijforecast.2008.12.004
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
by, inter alia, Clements and Hendry (1996) and Tiao
and Xu (1993) for ARMA processes; Bhansali
(1996, 1997), Bhansali and Kokoszka (2002), and
Brodsky and Hurvich (1999) for long memory
processes; Haywood and Tunnicliffe-Wilson (1997)
in the frequency domain; Schorfheide (2002) for
asymptotically vanishing misspecification; Findley,
Potscher, and Wei (2004) and Ing (2003, 2004) for
very general settings; and Chevillon and Hendry
(2005), Chevillon (2008), and Proietti (2008) for
misspecified (AR)IMA processes; see Chevillon
(2007) for a survey. Another stream of articles on
the relative merits of DMS versus IMS, proposes
new techniques such as Partial Least-Squares (see Lin
& Tsay, 2005), or design ex ante tests for potential
improvements using DMS (see Clark & McCracken,
2005, Giacomini & White, 2006 and Haywood &
Tunnicliffe-Wilson, 2004).
Despite the number of theoretical results in
favor of DMS, few empirical analyses exhibit
successful uses of direct multi-step methods, except,
notably, Tiao and Tsay (1994) on the US monthly
price index for food, Tsay (1993) on the quarterly
US unemployment rate, and Liu (1996) on the
monthly unemployment rate in Taiwan. The results
from other authors were more mixed, such as Kang
(2003) and Lin and Tsay (1996), who found
evidence both in favour and against; while the
major analysis of 171 US monthly macroeconomic
time series performed by Marcellino, Stock and
Watson (2006) strongly favoured iterative forecasting
techniques. Also working on US series, Proietti (2008)
showed a net improvement from using DMS for
consumer inflation, but not for GDP. Other recent
partially successful uses of DMS techniques include,
inter alia, Eklund and Karlsson (2005), Jorda and
Marcellino (2007) and Schumacher and Breitung
(2008). It should be noted that the authors who
found evidence against direct multi-step methods
mostly used post-war US data, although over extended
periods. The United States has not undergone
massive shifts in this era, and by aggregation over
such a large economy, some stability is expected.
However, several of the theoretical analyses – such
as Peña (1994) for breaks, Bhansali (1997), whose
framework of long memory can be seen as relevant
to regularly occurring breaks in the light of Perron
and Qu (2007), and Chevillon and Hendry (2005)
603
for occasional location shifts that induce negative
serial correlation – point towards deterministic shifts
as a potential source of success of DMS in finite
samples.
The aims of this paper are threefold: first, we run
a forecast competition over 779 different univariate
and multivariate methods, the latter having rarely been
used in conjunction with DMS (some exceptions exist,
such as Haug & Smith, 2007). Second, we specifically
assess the DMS model, which was developed by
Aron and Muellbauer (2002) (AM henceforth) for
forecasting the South African GDP. Unlike Marcellino
et al. (2006), who differenced all of the integrated
series to ensure that they were working with stationary
variables, we focus on accuracy in forecasting the
level of the variable, since differencing corrects for
some location shifts at the cost of higher variance—
in effect transforming step dummies into blips. Thus,
AM established a relationship which allowed them, in
spite of the many breaks experienced by the economy,
to forecast the annual change in the quarterly GDP
series over the last twenty years. Third, we establish
which features improve the forecast accuracy of
the standard IMS and DMS methods (in level,
differences or equilibrium-correction form), with
additional robustifying techniques such as intercept
correction (see Clements & Hendry, 1999), double-
differencing, and the use of various indicator variables
and deterministic or stochastic trends.
We find that DMS and methods that are designed
to be robust to breaks, such as intercept correction,
improve forecast accuracy in multivariate models.
There also appear to be very different patterns for
multivariate and univariate techniques, and the former
prove to be more accurate in this experiment. Also,
our results favour the use of fractional integration in
ARMA models.
The plan of this paper is as follows. First, we review
the South African context and the model developed
by Aron and Muellbauer (2002). In Section 3, we
derive alternative multivariate IMS and DMS models.
We then proceed to a general comparison of the
forecasting techniques, analysing the methods that
perform best, with a special focus on their relationship
to breaks. A conclusion follows. Supplementary tables
to those that appear in this paper are available from the
IJF website. www.forecasters.org/ijf.
604 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
2. Forecasting the South African GDP
2.1. Thirty years of breaks
South Africa has undergone a profound transition
in the last thirty years, and hence, any model
of its economy is subject to frequently occurring
breaks. From 1976, with the government’s policy
of Apartheid, the country began suffering from
increasing international isolation, which culminated,
between late 1985 and the ‘free’ elections of 1994, in
a period of almost no access to international capital.
These factors, when taken in combination with the
high degree of reliance of the economy on mineral
exports, might explain some of the shocks and large
variations observed in the economic variables (see
the articles by Aron and Muellbauer for an extended
analysis of the South African context).
Following Aron and Muellbauer, we can distin-
guish three main monetary regimes since the 1960s.
Until the late 1970s, there existed quantitative con-
trols on interest rates and credit, and the main
criteria used for monetary policy were liquid asset ra-
tios, while the corrective effect of interest rates was
largely neglected by the regulatory authorities. Finan-
cial liberalization and the transition towards a more
flexible, cash-reserves based system took place during
the first half of the 1980s. From 1986 onwards, the
monetary authorities made use of the discount rate to
influence the interbank overnight refinancing market
in order to achieve pre-declared monetary targets. The
credit growth which followed the financial liberaliza-
tion soon lessened the usefulness of monetary targets,
thus leading, from 1998 onwards, to a new regime.
The South African Reserve Bank (SARB) then
offered some amount at the daily tender for repurchase
transactions, thus signalling its preferences on short-
term interest rates via an auction mechanism. In early
2000, inflation targeting was reinstated as part of the
medium-term monetary objectives.
Following the development of monetary policy, the
foreign exchange market experienced various regimes
pointing toward greater flexibility. From US dollar
or pound Sterling pegs, combined with restrictions
on resident and nonresident capital flows, the system
moved, in 1979, to a regime of dual currency. Most
non-resident transactions operated at the floating
‘financial’ exchange rate, while a ‘commercial’ rate
Table 1
Monetary policy and exchange rate policy regimes (Aron &
Muellbauer, 2002).
Period Monetary policy regimes
1960–81 Liquid asset ratio-based system with quantitative
controls on interest rates and credit
1981–85 Mixed system during transition
1986–98 Cost of cash reserves-based system with
pre-announced M3 targets
1998–99 Daily tenders of liquidity through repurchase
transactions (repo system), plus pre-annouced M3
targets and targets for core inflation
2000– Repo system with inflation targeting
Period Exchange rate policy regime
1961(1)–71(2) Pegged to fixed pound Sterling
1971(3)–74(2) Pegged in episodes to floating US dollar/pound
Sterling
1974(3)–75(2) ‘Controlled independent float’: devaluations every
few weeks
1975(3)–79(1) Fixed Regime: pegged to the US dollar
1979(2)–82(4) Dual foreign exchange system: controlled floating
commercial rand and floating financial rand
1983(1)–85(3) Unification to a controlled floating rand
1985(4)–95(1) Return to a dual system
1995(2)– Unification to a controlled floating rand
was instated and announced in line with market forces.
The latter became market determined in 1983 and the
dual rates were soon re-unified. A debt crisis and the
collapse of the Rand provoked a return to the dual
currency system after 1985. In 1995, unification of
the dual currency was initiated under a managed rate,
which became fully floating at the introduction of
inflation targeting.
We reproduce Table 1 of AM (Table 1 here),
where they present the various regimes experienced
by the South African economy. In order to measure
the degree of non-stability in the economy, we also
estimate a series that represents the break intensity. For
this, we focus on five key variables: the logarithm of
real GDP, CPI inflation (measured as log-differences),
the short-term interest rate, and the logarithms of the
real exchange rate (with the US dollar) and of narrow
money. For both the level and the first difference
of each of the variables, we estimate autoregressive
processes of order 1, AR(1), with a constant, over
rolling windows of 30 observations (dated according
to their endpoint). For each of these subsamples,
we use the technique developed by Bai and Perron
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Fig. 1. Break intensity in the South African economy.
(2003)1 to estimate the number of breaks the AR(1)
undergoes, assuming that breaks affect either the mean
or the autoregressive coefficient. We therefore obtain
ten series which, when added up, give the break
intensity of the South African economy. Fig. 1 records
the resulting variable.
In accordance with Table 1, we observe that the
several exchange rate regimes that were experienced in
the 1970s, and in particular the recurring devaluations
toward the middle of the decade, imply a non-
negligible break intensity. The monetary transition of
the early 1980s results in a large break intensity (BI)
that specifically affects the interest rate. The more
autarkic 8–10 year period starting in 1986 was quieter,
and BI is indeed temporarily lower until the beginning
of the political transition, as new economic policies
were installed in the second half of the 1990s. This
measure of break intensity will find its use when
we assess the relationship between the most accurate
forecasting techniques and breaks.
2.2. The AM model
Aron and Muellbauer developed a model for
forecasting the annual change of the quarterly South
African real GDP (Y or y in log) via an equilibrium
1 We also use the Gauss code that they provide on their website.
605
correction equation which depends on a set of
variables {X i }:
k
!
X
∆4 yt = α yt−4 − µt−4 − γi X i,t−4 − β
i=1
k
X
+ δi Z i,t−5 + t−4 , (1)
i=1
where α < 0, t is white noise and µt is a
smooth stochastic trend which aims to capture the
underlying production capacity of the economy. This
trend is defined, as in Harvey (1993) and Harvey
and Jaeger (1993), by a smooth I(2) extracted from
yt using STAMP (see Koopman, Harvey, Doornik &
Shephard, 2000). AM study several sets of variables
for {X i }, and settle on a few. They find that their
equation, estimated using quarterly data over the
period 1963(1)–2000(2), is stable over the various
regimes, using as regressors the four-quarter moving
average (RPRIMA) of the real prime interest rate
(RPRIME), and the fourth lag thereof, a twelve-
quarter moving average of the government surplus to
GDP ratio (RGSURMA12), and the ratio of current
account surplus to current GDP (RCASUR). The non-
integrated {Z i } series comprise the annual change
(∆4 RPRIME) in the real prime interest rate; the
long-term growth in terms of trade (∆12 tot); the first
difference of a financial liberalization indicator (a
spline indicator variable FLIB); a monetary regime
606 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Fig. 2. Variables entering the AM equation; see Table 4 for definitions.
shift dummy (for the 1983(2)–85(4) period, denoted
by N) interacting with both a four-quarter moving
average and an annual difference of RPRIME;
and finally an indicator dummy variable for the
1991(3)–92(2) drought (DUM92). These variables are
presented in Fig. 2. The proximity of the I(2) trend to
yt clearly plays a role in the forecasting accuracy of
Eq. (1): the correlation between these variables is very
large, at 0.994.
In order to assess the forecasting power of this
equation, we need to develop alternative techniques.
This is what we turn to in the next section. However,
it should be noted that several dummy variables
which enter the equation could only be defined
ex-post. Hence, for ex-ante forecast comparisons,
these should be omitted from the equation (this
involves at least DUM92, and potentially N and FLIB,
although the latter two could be kept, since they
were ‘predictable’ from the government’s decisions).
Finally, there remains the issue of the I(2) trend:
in AM, this is computed over the whole sample.
For ex ante forecasting we resort to estimating it
recursively. To stress the importance of this trend, we
also assess the forecasting models with the smooth
trend estimated over the whole sample, with a linear
deterministic trend, and with no trend at all.
Thus, the AM technique and its variants are
not seen as well-specified models of the process
generating the South African GDP, but rather as
misspecified yet parsimonious DMS forecasting tools.
3. Competing forecasts
The forecasting equation developed by Aron and
Muellbauer includes a few exogenous variables which
ought to be modelled for a proper IMS technique to
be used. Unfortunately, doing so might increase the
degree of misspecification, which would in turn be
detrimental to our assessment of forecast accuracy.
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Hence, we resort to three simpler multivariate models
which provide the possibility for both IMS and
DMS forecasting. The first two models define small
monetary systems for South Africa, while the third
uses the main variables of the AM equation.
Another issue arises regarding the variable to be
forecast: the AM technique provides forecasts of
∆4 yT +4 from an end of sample T . This annual
difference does not fit the definition of the IMS
forecast as it is usually referred to. Indeed, if a model
in differences provides some ∆b xT +1 = b B∆xT , with
4 real exchange rate are recorded in Fig. 4. We see that
yt = Pxt , then the IMS forecast is ∆b xT +4 = bB ∆xT ,
and hence ∆4b
P4
xT +4 = i=1 b
i
B ∆xT . The IMS forecast
is also nonlinear in the parameters when, as in AM,
is ∆b
the
 model  xT +1 = AxT , for which ∆4b xT +4 =
b
(b
A + I)4 − I xT . Thus, this choice of target seems
to benefit direct multi-step estimation. In order not to
favour DMS a priori, we assume that the forecasts to
be evaluated concern the levels of the GDP and not
the annual differences. Hence, from the AM equation
∆4b yT +4 = b c1 yT + bCxT , we retrieve b yT +4 = (1 +
c1 )yT + b
b CxT .
We also consider forecasting at various horizons
in order to mitigate the issue of levels versus
differences, and potential differences in seasonal
adjustments (all of the variables are seasonally
adjusted, but the changes in regime and breaks imply
that seasonality may not be taken into account well).
In the comparisons, we will see that whether h =
2, 3, 4, 5 or 8 is important for the relative forecast
accuracy. Hence, we will estimate Eq. (1) with the
left-hand side (LHS) variable defined as ∆h yt for the
different horizons h. We will also produce iterated
multi-step forecasts by cumulating the one-step-ahead
growth forecasts ∆b yt+1 . All of our methods use
quarterly data and are estimated recursively over the
period 1965(2)–2000(2), starting with an initializing
sample of 30 observations.
3.1. A small monetary model
3.1.1. Data description
The variables which we include in our model are
the M1 narrow money aggregate (denoted by M1 ) or
quasi-money (denoted by M∗ ), the consumer price
index (CPI), the 3-month treasury bill interest rate
(per annum, R), and the South African Rand/US
dollar exchange rate, which were obtained from the
607
International Financial Statistics (IFS) database. In
order to maintain a unique series for forecasting
across models, we used the real GDP series from AM
throughout.
Following Jonsson (2001), we conduct a cointegra-
tion analysis of five variables2,3 : the log of real money
m k − cpi, for k ∈ {1, ∗}; the log of real exchange
rate r er ; the log of real GDP y; the nominal trea-
sury interest rate R; and inflation ∆cpi. Fig. 3 presents
graphs of real money, together with the inflation rate
and their differences; interest rates, the GDP and the
annual changes are much more persistent than first dif-
ferences, but that they also exhibit large fluctuation
amplitudes. This may benefit DMS models.
We also notice the large contractions of real narrow
money in the late 1970s and mid 1980s during periods
of higher inflation, and the continuous depreciation of
the Rand with respect to the US dollar. Real M∗ also
contracted severely over the period.
3.1.2. A VAR system
In addition to the five economic variables, where
the analysis below is conducted using M1 (cases where
there are differences in using a model that contains M∗
instead of M1 are reported where they occur), we allow
for a constant and a trend to enter the cointegration
space unrestrictedly. We restrict our attention to a
VAR(2), as tests showed that lags beyond 2 are not
significant. The VAR in levels, estimated over the
period 1965(1)–2000(2), fits the data reasonably well,
as is shown in Fig. 5. Indeed, except for inflation, the
vector of variables is well explained by its past, as can
been seen from the figure.
A corresponding test summary is presented in
Table 2, which records statistical information about
the VAR, namely the equation residual standard
errors (bσ ); single-equation evaluation statistics for
2 The variables modelled by Jonsson (2001) were the short and
long interest rates, real income, the exchange rate, broad money
and prices. Broad money may seem an interesting variable to
use in the presence of a financial liberalization; however, we
were unfortunately unable to work with it, for reasons of vintage
comparatibility. We use quasi-money instead, noting that our aim
is not so much to model the economy as to obtain an operational
model which we can easily modify to see which features matter for
forecast accuracy.
3 Computations and tests were conducted using PcGive and
OxMetrics.
608 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628

Fig. 3. Quarterly South African monetary series and CPI inflation; see Table 4 for definitions.

Table 2
Specification statistics for the VARSAM1 model with a linear trend estimated as a VAR(2).

Statistic R ∆cpi (m − cpi) y r er VAR

σ (%)
b 8.03 0.86 4.84 1.11 6.29
Far (5, 127) 0.517 1.22 2.03* 0.108 1.41 –
Farch (4, 124) 2.39 0.684 0.715 0.740 0.625 –
Fhet (22, 109) 1.80 0.773 0.931 1.25 1.09 –
χnd
2 (2) 40.6** 10.8** 0.805 7.48* 10.1** –
v (125, 511)
Far – – – – – 1.06
v
Fhet (330, 1228) – – – – – 1.06
χnd
2v (10) – – – – – 66.1**
∗ Denotes significance at the 5% level.
∗∗ Denotes significance at the 1% level.

no serial correlation (Far , against 5th-order residual
autoregression); no ARCH (Farch , against fourth-
order); no heteroscedasticity (Fhet , see White, 1980);
and a test for normality (χnd2 , see Doornik & Hansen,
1994). Analogous system (vector) tests are labelled as
‘v ’. Hence, the normality tests fail here for most of the
variables and the system; this reflects what can be seen
from the graphs in the third column of Fig. 5, namely
the presence of large outliers. Given our interest in
breaks, and since there seems to be non-normality
caused by the outliers, we therefore retain our model.
Replacing m 1 with m ∗ yields the same results, with the
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 609

Fig. 4. Quarterly seasonally adjusted series of South African interest rates, real GDP and real exchange rates; see Table 4 for definitions. The
left column records the series in first and fourth differences (the latter are only reported over the sample that is used for estimation).

exception of Far for (m ∗ − cpi), which rejects at the
1% level, and χnd
2 , which is significant for (m − cpi)
∗ shown by Gonzalo (1994) to have good finite sample
but not for y.
Finally, as regards parameter constancy, Fig. 6
presents the equation residuals obtained by recur-
sive estimation and their 0 ± 2b σ boundaries, which
approximately represent their 95% confidence inter-
vals if the VAR is stationary. Fig. 7 records the
1↑ and N↓ (BreakPoint) Chow constancy tests for
the individual equations and the VAR system (see
Chow, 1960), together with the 1% bound. We no-
tice that, according to both figures, we can be rea-
sonably confident of the overall constancy of the
relationships, although they do exhibit occasional
breaks, and we therefore retain this system for further
analysis.
3.1.3. Stationarity analysis
Assuming that the VAR is reasonably specified,
we investigate the cointegration properties of the five
variables, noting that the Johansen procedure has been
properties even in the presence of non-normal errors.4
The cointegration statistics reported in Table 3 –
where a constant and a trend enter the cointegration
space unrestrictedly and restrictedly, respectively –
support the hypothesis that there are two cointegrating
relationships (see Johansen, 1995).
We then follow Jonsson (2001), and identify two
cointegrating vectors:
c1t = Rt − 0.126 r ert − 76.5 ∆cpi t − (m 1t − cpi t )
c2t = r ert − (yt − 0.019t) − 20.1 ∆cpi t
+ 2.11 Rt − −.357(m 1t − cpi t ),
which are presented in Fig. 8. When replacing m 1
with m ∗ , the estimated rank of cointegration is altered:
4 I thank an anonymous referee for pointing this reference out to
me.
610 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628

Fig. 5. Goodness of fit of the VAR(2) model, with the VARSAM1 variables listed in Table 4, with an unrestricted constant and restricted linear
trend. This is estimated over the period 1965(1)–2000(2).

Table 3 cointegrating relationships which experience breaks

Cointegration statistics: eigenvalues (λ), log-likelihood (l), Trace tends to reduce the accuracy of the forecasts.
statistic (Tr) and the corresponding p-values. The model is a VAR(2)
for VARSAM1 with a restricted linear trend. South Africa may provide such an example, as,
when estimating the model over various subperiods,
Rank λ l Tr p-value
the trace statistic provides justification for varying
0 – 1457.141 156.95** [0.000]
numbers of cointegrating vectors. However, in the
1 0.453 1499.968 71.293** [0.009]
2 0.243 1519.728 31.773 [0.407] following we will compare the forecasting properties
3 0.124 1529.099 13.031 [0.737] of the VAR in level (VAR) and the VAR in differences
4 0.058 1533.359 4.5108 [0.671] taking cointegration into account (VEC) or not (DV).
5 0.031 1535.615 – –
The parameters of the cointegrating vectors will be
∗∗ Denotes significance at the 1% level.
recursively estimated under the constraints that yt and
the trend do not belong to c1t . As for the AM equation,
we cannot reject the presence of a unique common
we will compare models with four types of trends; for
trend, i.e., four cointegration relationships. Recursive
consistency, this choice will also be imposed in c2t
estimation of the VAR model with quasi money
appears to be much less stable than with narrow instead of the linear trend.
money; following the evidence of Jonsson (2001), we In addition, we explore two types of DMS
only retain two cointegrating vectors, bearing in mind forecasting in the VAR in differences: one consists of
that we might thus underestimate their number. modelling ∆xt conditional on It−h , while the other
Unfortunately, as has been shown by Clements focuses on ∆h xt |It−h as the AM equation. We refer
and Hendry (1999) and Hendry (2006), the use of to the former as DMS and the latter as DMSh.
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Fig. 6. Residuals from recursive estimation of the VAR(2) model of VARSAM1.
3.2. An IMS version of AM
In order to analyze the properties of the multi-
step AM forecasting procedure, below we develop
a multivariate equivalent. It must be noted that the
parsimonious version of the AM only uses a few
variables: besides the real GDP itself and dummies,
the regressors entering the equation are RPRIME (via
transformations), RCASUR, RGSUR, the log of TOT,
and an I(2) trend (see the definitions in Section 2.2
above). Hence, it seems natural to develop a VAR
model which includes these variables. As regards
the N and FLIB indicators, they could be either
included or not, depending how acute the forecaster’s
perception of the economic environment could be
assumed to have been at the time. We also consider
various models for the trend.
Augmented Dickey–Fuller tests reject the presence
of a unit root in the RCASUR and RGSUR series,
but fail to reject it as regards y, RPRIME and TOT.
Thus, a cointegration analysis should include the latter
three variables, and a trace test for reduced rank
611
claims the existence of one cointegrating relationship
between the three, when a trend is allowed to enter
the cointegration space. By contrast, if we allow N
and FLIB to enter the system unrestrictedly, then
we only marginally reject the hypothesis that the
matrix is of full rank, and hence that the variables
are not cointegrated ( p-value of 4%, given by PcGive
in estimation over the period 1960(3)–2000(2), for
a VAR(2) model, with the lags beyond 2 being
statistically insignificant).5 This small model, also
detailed in Table 4, allows us to generate IMS – and
DMS – forecasts which we can compare to the solved
out AM equation.
3.3. Univariate methods
3.3.1. Robust techniques
We follow the results from the research of
Clements and Hendry, and use alternative forecasting
5 This is why RGSUR and RCASUR appeared among the set of
{X i } variables in Eq. (1).
612 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628

Fig. 7. 1↑ and N↓ (BreakPoint) Chow constancy tests for the individual equations and the system. The model is VARSAM1 and is estimated
as a VAR(2) with a restricted linear trend over the period 1965(1)–2000(2).

Fig. 8. Cointegrating vectors from the VAR(2) including m 1 .

 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 613

Table 4
Description of the data, sources and model definitions.

Data definitions
Source: Aron and Muellbauer (2002), seasonally adjusted
y Log Real GDP
RPRIME Real prime interest rate
RCASUR Ratio of current account surplus to current GDP
RGSUR Government surplus to GDP ratio
tot Log terms of trade
FLIB Spline financial liberalization indicator
N Monetary regime shift dummy for 1983(2)–85(4)
DUM92 Indicator variable for the 1991(3)–92(2) drought
Source: International Financial Statistics, seasonally adjusted
m1 Log M1 monetary aggregate
m∗ Log quasi money
cpi Log Consumer price index
R Short term interest rate
rer Log Real exchange rate (with US)

Models (estimated recursively over 1965(1)–2000(2)) Regressors

AM for ∆h yt ∆4 PRIMEt−h
P3
RPRIMAt−h = 14 i=0 RPRIMEt−h−i and RPRIMAt−h−4
RGSURMA12t−h = 12 1 P11 RGSUR
i=0 t−h−i
∆12 (tot)t−h
∆(FLIB)t−h
AMIMS (yt , RPRIMEt , RCASURt , RGSURt , tott )
VARSAM1 (m 1t , cpit , Rt , rert )
VARSAM∗ (m ∗t , cpit , Rt , rert )

Dummies

1 (DUM92t , N t , FLIBt )
Pt Pt
2 ( i=1 DUM92i , i=1 N i , FLIBt )
3 (N × ∆4 PRIMEt , N ×PRIMAt )
∅ No dummy

Trend

I2 Smooth I(2) trend estimated over the whole sample

recI2 Smooth I(2) trend estimated recursively
L Linear trend
∅ No trend

techniques, focusing on two classes: those of by:

differencing and intercept correcting. Thus, if we wish
to forecast yT +4 from a forecast origin T , it has been (DV) : yT +4 = yT ,
b (2a)
shown that the models: (DDVIMS) : yT +4 = yT + 4(yT − yT −1 ),
b (2b)
∆yt = ζ1t , (DV) (DDVDMS) : yT +4 = yT + (yT − yT −4 ).
b (2c)
∆∆yt = ζ2t , (DDVIMS)
∆4 ∆4 yt = ζ4t , (DDVDMS), By contrast, intercept correcting constitutes an
adjustment to an existing model, such that if b
yT +4 and
where the ζit are assumed to be independent white yT +4 are forecasts from the IMS and DMS models:
e
noise, can exhibit some degrees of robustness to
breaks. They respectively lead to forecasts b
yT +4 given IMS : b b 4 yT ,
yT +4 = Ψ
614 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
DMS : e e4 yT ,
yT +4 = Ψ
then the intercept corrected forecasts are defined as the
original forecasts corrected for the in-sample forecast
errors of the latest available observations:
b 4 yT + (yT − Ψ
yT +4 = Ψ
ICIMS : b b 4 yT −4 ),
e4 yT + (yT − Ψ
e4 yT −4 ).
ICDMS : eyT +4 = Ψ
More generally, we also consider intercept correction
for all forecasting methods.
3.3.2. ARIMA models
In addition to the robust methods presented above,
we also consider ARFIMA( p, d, q) models, where
d is restricted to be 0 or 1, or is estimated as
a fractional parameter, using ARFIMA in Ox, see
Doornik and Ooms (2003). For the order of the
AR and MA polynomials, we allowed for either
fixing them at values between 0 and 2, or choosing
them in-sample by minimizing the Schwarz criterion.
The resulting forecasts were also possibly intercept
corrected, but we did not allow for dummy variables
or trends in these univariate models. We denote as
ARIMA( p, d, q) the models with estimated ( p, q)
and ARIMA(ar, ma) for fixed values (ar, ma) ∈
{0, 1, 2}2 . IMS, DIMS and FIMS then refer to d = 0, 1
or being estimated; the same holds for DMS, DDMS
and FDMS. With or without intercept correction (IC),
univariate ARFIMA models provide 120 different
techniques.
4. Forecast comparison
4.1. Techniques
We proceed to a comparison of ex-ante forecast
accuracy using the various methods delineated above.
There are 779 techniques in total, which are labelled
as follows. The models are AM (univariate Aron &
Muellbauer), AMIMS (multivariate version of AM),
VARSAM1 (the monetary VAR with narrow money),
VARSAM∗ (with quasi money), ARIMA( p, d, q)
(estimated lag orders), and ARIMA(ar, ma) (with
(ar, ma) replaced with their values). For each of
the previous multivariate models, the representations
can be VAR (in levels), DV or VEC (the last for
VARSAM1 and VARSAM∗ only; the cointegrating
vectors are specified using the whole sample but
are estimated recursively). Then, the trend can be
the overall I(2) (labelled as I2), a linear trend (L),
a recursively estimated I(2) trend (recI2), or not
included (∅). For the dummies, we include the three
discussed: DUM92, N and FLIB; the models with
the three dummies are labelled as d1. We also allow
for the cumulated DUM92 and N to be used instead,
so that their first difference provides the version in
AM; together with FLIB this gives the models d2.
We also P4tried, in d3, using N × ∆4 PRIME and
N × 14 i=1 RPRIMEt−i . Finally, ∅ refers to models
without dummies; see Table 4 for a definition of the
models.
The forecasting technique can be any of IMS,
ICIMS (with intercept correction), DMS, DMSh,
ICDMS or ICDMSh. We also used DV, DDVDMS and
DDVIMS. In the ARIMA( p,d,q) and ARIMA(ar,ma)
cases, we denote the three cases with, respectively,
d = 0, 1 and estimated d ∈ (0, 1), by IMS, DIMS and
FIMS. The same holds for DMS, DDMS and FDMS,
where the ARFIMA models are augmented with addi-
tional h lags, and the first h − 1 are restricted to zero.
In total, we obtain 779 models, each of which is es-
timated recursively over the period 1965(1)–2000(2)
and assessed for h ∈ {2, 3, 4, 5, 8} over the period
1974(2)–2000(2); see Table 5 for a complete descrip-
tion of the forecasting techniques. Should we forecast
using expanding or rolling windows? There does not
seem to be a clear cut answer, but the evidence of
Pesaran and Timmermann (2005) points towards the
use of expanding windows, hence our use thereof. We
report below the key features arising from the exer-
cise; additional tables are available from the IJF web-
site (www.forecasters.org/ijf).
4.2. Forecast accuracy
Prior to ranking the individual performances of the
forecasting techniques, we first assess, in Table 6,
how these methods perform out-of-sample compared
to their in-sample fit. The table presents the average
ratios, over the set of forecasts and for all possible
choices of trends and dummy variables, of the root
mean square forecast error (RMSFE) over the in-
sample estimated standard error of the residuals. In
order to compare them at several horizons √ h, we scaled
the ratios forPh IMS and ARIMA by h, since the
variance of i=1 εt+i is hV(εt ) if εt is white noise
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Table 5
Description of the forecasting techniques.
(an assumption which can clearly be violated for the
successive forecast errors over the forecast horizon).
It is clear from the table that for most techniques
the ratios increase with the forecast horizons, which
means that forecast accuracy worsens, and the in-
sample fit provides a less reliable gauge for DMS and
DMSh than for IMS. The univariate methods are the
only ones for which the ratios remain close to unity,
especially when the lag orders are not chosen from
the data. ARIMA models in levels perform particularly
badly when assessing ex-ante vs ex-post fit. This is due
to the exceptional fit of these flexible models in some
subsamples. The ratios, whose average is reported
615
in Table 6, often have almost zero denominators.6
This sits well with the choice made by authors such
as Marcellino et al. (2006) to work with stationary
series only.
It is important to stress that Table 6 reports
averages over the models with the different types
of dummies and trends that we consider in the
exercise. These averages conceal various patterns.
In particular, with respect to the types of trends,
the ratios tend to be smallest when using the
overall I(2); then either a linear or no trend yield
approximately similar reliability of in-sample fit.
However, when a recursively estimated I(2) trend
is used, the RMSFE tends to be much larger than
the in-sample residual standard errors. We will see
below that, in addition, recursive I(2) trends do not
bring much forecast accuracy benefit. By contrast,
the choice of dummy variables seems irrelevant
in the context of predictability of the forecast
error.
4.3. Rankings
Turning to forecast accuracy, we evaluate it via
RMSFE in Tables 7–9. We record the rankings for the
45 most accurate forecasts at horizons h = 2 and 4,
and these ‘league tables’ differ significantly. Tables
for h = 3 and 5 (not given here) show that there
is a continuity which allows one to approximately
interpolate over these values of the horizon. Noticing
that the best forecasts are generally obtained using
models where the smooth trend is estimated over
the whole sample, we also report the same rankings
excluding models with I2. We comment on these
tables below, analysing each of the four points that
we believe need stressing out of this exercise in turn.
Also, and this holds for all of the tables, whether we
consider the Mean Absolute Prediction Error (MAPE)
or RMSFE does not modify the relative rankings
significantly; hence our focus on the RMSFE, with the
unreported MAPE tables being available from the IJF
website.
6 The median IMS ratios of ex-post MSFE over ex-ante residual
standard error – taking values between 417 and 468 when scaled by
h – are also large, but for DMS, the equivalent scaled medians are
between 1.6 and 2.4.
 616

Table 6
Average ratios of the RMSFE over the in-sample residual standard error.
IMS DMS DMSh
Estimation Model h=2 h=3 h=4 h=5 h=8 h=2 h=3 h=4 h=5 h=8 h=2 h=3 h=4 h=5 h=8
VAR AMIMS 1.445 1.638 1.803 1.986 2.32 2.252 2.750 3.223 3.824 5.151
VARSAM1 1.289 1.448 1.612 1.815 2.40 2.204 2.632 3.093 3.814 5.715
VARSAM∗ 1.276 1.424 1.602 1.805 2.34 2.225 2.628 2.979 3.698 5.939
ARIMA( p, d, q) 82 155 80 617 77 570 70 892 59 773 59 177 451 75 38 310 17.4 3.07
ARIMA(ma, ar ) 1.152 1.154 1.143 1.142 1.141 0.943 31.2 0.814 206 1.154

DV AMIMS 1.081 1.189 1.280 1.411 1.575 1.232 1.593 1.914 2.234 3.060 1.387 1.832 1.996 1.819 3.009
VARSAM1 0.990 1.113 1.186 1.298 1.530 1.183 1.519 1.867 2.408 3.118 1.273 1.528 1.602 1.843 3.184
VARSAM∗ 0.986 1.115 1.184 1.293 1.524 1.215 1.530 1.872 2.487 3.029 1.343 1.595 1.766 1.985 3.649
AM 1.298 1.520 1.720 1.884 2.36 1.679 1.947 2.252 2.420 3.824
ARIMA( p, d, q) 1.113 1.150 1.245 1.266 1.295 1.197 1.017 0.836 1.923 1.369
ARIMA(ma, ar ) 0.450 0.506 0.524 0.572 0.652 0.654 0.777 0.591 0.814 1.086

VEC VARSAM1 1.112 1.243 1.329 1.429 1.648 1.332 1.692 2.139 2.714 3.725 1.446 1.782 1.900 2.102 3.783
VARSAM∗ 1.131 1.263 1.373 1.487 1.743 1.341 1.673 2.086 2.692 3.508 1.464 1.861 1.921 2.234 3.992
Fractional ARIMA( p, d, q) 2.72 2.71 2.67 2.57 3.42 2413 640 12.7 4.58 2.49 2.29
ARIMA(ma, ar ) 0.968 1.056 1.130 1.188 1.272 0.853 1.000 0.901 1.985 1.912
G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 617

Table 7
RMSFE Rankings, ranked for h = 2.

Overall Excluding the I2 trend

Model Dum Trend Method h = 2 h = 4 h = 8 Model Dum Trend Method h = 2 h = 4 h = 8
VARSAM1 VEC ∅ I2 DMS 1 116 116 VARSAM1 VEC ∅ L DMS 4 128 99
VARSAM1 VEC 1 I2 DMS 2 136 117 VARSAM1 VEC 1 L DMS 13 156 107
VARSAM∗ VEC ∅ I2 DMS 3 192 81 VARSAM1 DV ∅ L DMS 14 60 92
VARSAM1 VEC ∅ L DMS 4 128 99 VARSAM1 DV ∅ ∅ DMS 14 62 92
VARSAM∗ VAR ∅ I2 IMS 5 4 3 VARSAM∗ VEC ∅ L DMS 16 144 77
VARSAM1 DV ∅ I2 IMS 6 15 7 AMIMS DV ∅ L DMS 18 160 68
VARSAM∗ DV ∅ I2 IMS 7 16 6 AMIMS DV ∅ ∅ DMS 18 162 68
AMIMS DV ∅ I2 IMS 8 14 5 ARIMA ( p, d, q) ∅ ∅ FIMS 21 81 43
AMIMS DV ∅ I2 DMSh 9 2 19 ARIMA (1, 1) ∅ ∅ FIMS 24 109 59
VARSAM1 DV ∅ I2 DMS 10 51 110 VARSAM∗ DV ∅ L DMS 25 75 61
AMIMS VAR ∅ I2 IMS 11 3 1 VARSAM∗ DV ∅ ∅ DMS 25 77 61
VARSAM∗ VEC 1 I2 DMS 12 207 83 VARSAM∗ DV ∅ L ICIMS 27 44 37
VARSAM1 VEC 1 L DMS 13 156 107 VARSAM∗ DV ∅ ∅ ICIMS 27 46 37
VARSAM1 DV ∅ L DMS 14 60 92 VARSAM∗ VEC 1 L DMS 31 169 84
VARSAM1 DV ∅ ∅ DMS 14 62 92 ARIMA (2, 0) ∅ ∅ FIMS 32 95 45
VARSAM∗ VEC ∅ L DMS 16 144 77 VARSAM1 DV 1 L DMS 35 70 94
VARSAM∗ DV ∅ I2 DMS 17 58 66 VARSAM1 DV 1 ∅ DMS 35 72 94
AMIMS DV ∅ L DMS 18 160 68 ARIMA (2, 1) ∅ ∅ FIMS 37 167 115
AMIMS DV ∅ ∅ DMS 18 162 68 VARSAM1 DV ∅ L ICIMS 38 40 34
AMIMS DV ∅ I2 DMS 20 150 86 VARSAM1 DV ∅ ∅ ICIMS 38 42 34
ARIMA (1, 1) ∅ ∅ FIMS 21 81 43 ARIMA (1, 0) ∅ ∅ FDMS 41 138 122
VARSAM1 DV 1 I2 DMS 22 57 112 VARSAM1 DV ∅ L IMS 42 86 50
VARSAM∗ DV ∅ I2 ICIMS 23 19 10 VARSAM1 DV ∅ ∅ IMS 42 88 50
ARIMA (2, 0) ∅ ∅ FIMS 24 109 59 VARSAM∗ VEC ∅ recI2 DMS 44 158 387
VARSAM∗ DV ∅ L DMS 25 75 61 VARSAM∗ DV ∅ L IMS 45 92 53
VARSAM∗ DV ∅ ∅ DMS 25 77 61 VARSAM∗ DV ∅ ∅ IMS 45 94 53
VARSAM∗ DV ∅ L ICIMS 27 44 37 VARSAM1 VEC ∅ recI2 DMS 47 114 147
VARSAM∗ DV ∅ ∅ ICIMS 27 46 37 VARSAM1 DV ∅ recI2 DMS 48 61 133
VARSAM1 DV ∅ I2 DMSh 29 5 22 VARSAM1 VAR ∅ L IMS 52 84 60
VARSAM1 VAR ∅ I2 IMS 30 7 2 AMIMS DV ∅ L IMS 53 78 47
VARSAM∗ VEC 1 L DMS 31 169 84 AMIMS DV ∅ ∅ IMS 53 80 47
ARIMA (2, 1) ∅ ∅ FIMS 32 95 45 VARSAM1 VAR ∅ ∅ IMS 55 143 230
VARSAM∗ DV 1 I2 DMS 33 59 67 AMIMS DV ∅ L ICIMS 56 64 40
VARSAM1 DV ∅ I2 ICIMS 34 20 9 AMIMS DV ∅ ∅ ICIMS 56 66 40
VARSAM1 DV 1 L DMS 35 70 94 VARSAM1 DV ∅ L DMSh 58 37 57

4.3.1. Models
Comparing the three different VAR models that we
considered, none is always preferred. However, when
using the I2 trend, AMIMS tends to come first in the
tables for h ≥ 4, whereas VARSAM1 then VARSAM∗
are more accurate for shorter (h < 4) horizons,7 and
whether M1 or M∗ is used does not matter much. When
excluding I2, AMIMS performs worse than the other
7 When writing h < 4, we also comment on the unreported tables
for h = 3. The same holds for h > 2 and the unreported table with
h = 5.
two VAR models. This seems to indicate that I2 is key
for the economic model that was obtained by Aron
and Muellbauer. Indeed, we see from the left columns
of Table 8 that their univariate AM technique is the
most accurate at horizon h = 4, but that there is
no need here for dummy variables. It also forecasts
well at horizons h > 2, provided that I2 is used.
When excluding this type of trend, its performance
unfortunately drops in the tables at short horizons,
but retains some good medium-term – here over eight
quarter – forecasting power, since the two versions
618 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628

Table 8
RMSFE Rankings, ranked for h = 4.

Overall Excluding the I2 trend

Model Dum Trend Method h = 2 h = 4 h = 8 Model Dum Trend Method h = 2 h = 4 h = 8
AM ∅ I2 DMSh 90 1 4 VARSAM1 VEC ∅ recI2 ICIMS 431 33 561
AMIMS DV ∅ I2 DMSh 9 2 19 VARSAM1 DV ∅ L DMSh 58 37 57
AMIMS VAR ∅ I2 IMS 11 3 1 VARSAM1 DV ∅ recI2 DMSh 436 38 82
VARSAM∗ VAR ∅ I2 IMS 5 4 3 VARSAM1 DV ∅ ∅ DMSh 58 39 57
VARSAM1 DV ∅ I2 DMSh 29 5 22 VARSAM1 DV ∅ L ICIMS 38 40 34
VARSAM∗ DV ∅ I2 DMSh 50 6 23 VARSAM1 DV ∅ recI2 ICIMS 366 41 521
VARSAM1 VAR ∅ I2 IMS 30 7 2 VARSAM1 DV ∅ ∅ ICIMS 38 42 34
VARSAM1 VEC ∅ I2 DMSh 191 8 36 VARSAM∗ DV ∅ L ICIMS 27 44 37
AMIMS DV ∅ I2 ICDMSh 177 9 46 VARSAM∗ DV ∅ recI2 ICIMS 367 45 522
VARSAM∗ VEC ∅ I2 DMSh 179 10 39 VARSAM∗ DV ∅ ∅ ICIMS 27 46 37
VARSAM∗ VAR ∅ I2 DMS 210 11 443 VARSAM∗ VEC ∅ recI2 ICIMS 460 47 554
AMIMS VAR ∅ I2 DMS 207 12 17 VARSAM1 VEC ∅ L ICIMS 68 48 30
VARSAM1 DV ∅ I2 ICDMSh 315 13 91 AMIMS VAR ∅ recI2 ICDMS 827 55 780
AMIMS DV ∅ I2 IMS 8 14 5 AMIMS VAR ∅ ∅ ICDMS 295 56 55
VARSAM1 DV ∅ I2 IMS 6 15 7 VARSAM1 DV ∅ L DMS 14 60 92
VARSAM∗ DV ∅ I2 IMS 7 16 6 VARSAM1 DV ∅ recI2 DMS 48 61 133
VARSAM∗ DV 1 I2 DMSh 390 17 486 VARSAM1 DV ∅ ∅ DMS 14 62 92
VARSAM∗ VAR 1 I2 DMS 318 18 482 AMIMS DV ∅ L ICIMS 56 64 40
VARSAM∗ DV ∅ I2 ICIMS 23 19 10 AMIMS DV ∅ recI2 ICIMS 398 65 524
VARSAM1 DV ∅ I2 ICIMS 34 20 9 AMIMS DV ∅ ∅ ICIMS 56 66 40
VARSAM∗ DV ∅ I2 ICDMSh 321 21 49 VARSAM1 DV 1 L DMS 35 70 94
VARSAM∗ VEC ∅ I2 ICDMSh 362 22 78 VARSAM1 DV 1 recI2 DMS 79 71 140
VARSAM1 VEC ∅ I2 ICDMSh 355 23 111 VARSAM1 DV 1 ∅ DMS 35 72 94
VARSAM1 DV 1 I2 DMSh 383 24 444 VARSAM∗ DV ∅ L DMS 25 75 61
VARSAM∗ VEC ∅ I2 IMS 40 25 8 VARSAM∗ DV ∅ recI2 DMS 62 76 142
AMIMS VAR ∅ I2 ICDMS 120 26 426 VARSAM∗ DV ∅ ∅ DMS 25 77 61
AMIMS DV 1 I2 ICDMSh 523 27 432 AMIMS DV ∅ L IMS 53 78 47
VARSAM∗ VEC 1 I2 DMSh 358 28 489 AMIMS DV ∅ recI2 IMS 446 79 535
VARSAM∗ VAR ∅ I2 ICDMS 93 29 519 AMIMS DV ∅ ∅ IMS 53 80 47
AMIMS DV ∅ I2 ICIMS 51 30 12 ARIMA (1, 1) ∅ ∅ FIMS 21 81 43
VARSAM∗ VEC 2 I2 DMSh 644 31 901 VARSAM1 VAR ∅ L IMS 52 84 60
VARSAM∗ DV 1 I2 ICDMSh 549 32 497 VARSAM1 VEC 1 recI2 ICIMS 521 85 571
VARSAM1 VEC ∅ recI2 ICIMS 431 33 561 VARSAM1 DV ∅ L IMS 42 86 50
VARSAM1 DV 1 I2 ICDMSh 536 34 465 VARSAM1 DV ∅ recI2 IMS 415 87 531
VARSAM∗ VEC ∅ I2 ICIMS 112 35 14 VARSAM1 DV ∅ ∅ IMS 42 88 50

with no trend, or a recursively estimated I2, are the
best in Table 9.
Univariate ARIMA models also feature in the
tables, though more so when I2 is excluded. Also, the
autoregressive lag order is always at least as large as
the moving average order, and only once – at h =
2 when the I2 trend is excluded – is a model with
endogenously chosen lag orders (ARIMA( p, d, q))
present in the rankings. As the literature on ARIMA
forecasting has shown, these models are therefore
good, yet never best here. We will comment on them
further in the next subsection.
4.3.2. Best techniques
First, we notice that VEC models are the best at
short horizons h = 2, with or without I2, but that
their relative performance worsens as h increases. This
corroborates the risks posed by equilibrium-correction
in a non-stationary world, see Hendry (2006). The
fact that VEC models remain in the tables even
for rankings over longer horizons is in line with
the noncausal single or double difference models –
Eqs. (2a)–(2c) – being absent: Hendry (2006) shows
that both types should not be as accurate, and hence
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 619

Table 9
RMSFE Rankings, ranked for h = 8.

Overall Excluding the I2 trend

Model Dum Trend Method h = 2 h = 4 h = 8 Model Dum Trend Method h=2 h=4 h=8
AMIMS VAR ∅ I2 IMS 11 3 1 AM ∅ ∅ DMSh 88 250 25
VARSAM1 VAR ∅ I2 IMS 30 7 2 AM ∅ recI2 DMSh 299 249 28
VARSAM∗ VAR ∅ I2 IMS 5 4 3 VARSAM1 VEC ∅ L ICIMS 68 48 30
AM ∅ I2 DMSh 90 1 4 ARIMA (2, 0) ∅ ∅ ICFIMS 359 273 32
AMIMS DV ∅ I2 IMS 8 14 5 ARIMA (1, 0) ∅ ∅ ICFIMS 372 298 33
VARSAM∗ DV ∅ I2 IMS 7 16 6 VARSAM1 DV ∅ L ICIMS 38 40 34
VARSAM1 DV ∅ I2 IMS 6 15 7 VARSAM1 DV ∅ ∅ ICIMS 38 42 34
VARSAM∗ VEC ∅ I2 IMS 40 25 8 VARSAM∗ DV ∅ L ICIMS 27 44 37
VARSAM1 DV ∅ I2 ICIMS 34 20 9 VARSAM∗ DV ∅ ∅ ICIMS 27 46 37
VARSAM∗ DV ∅ I2 ICIMS 23 19 10 AMIMS DV ∅ L ICIMS 56 64 40
VARSAM1 VEC ∅ I2 IMS 84 49 11 AMIMS DV ∅ ∅ ICIMS 56 66 40
AMIMS DV ∅ I2 ICIMS 51 30 12 VARSAM1 VEC 1 L ICIMS 187 132 42
VARSAM1 VEC ∅ I2 ICIMS 96 36 13 ARIMA (1, 1) ∅ ∅ FIMS 21 81 43
VARSAM∗ VEC ∅ I2 ICIMS 112 35 14 VARSAM1 VEC ∅ L IMS 110 166 44
VARSAM1 VEC 1 I2 IMS 118 50 15 ARIMA (2, 1) ∅ ∅ FIMS 32 95 45
VARSAM1 VEC 1 I2 ICIMS 203 54 16 AMIMS DV ∅ L IMS 53 78 47
AMIMS VAR ∅ I2 DMS 207 12 17 AMIMS DV ∅ ∅ IMS 53 80 47
VARSAM∗ VEC 1 I2 IMS 117 53 18 VARSAM1 DV ∅ L IMS 42 86 50
AMIMS DV ∅ I2 DMSh 9 2 19 VARSAM1 DV ∅ ∅ IMS 42 88 50
VARSAM∗ VEC 1 I2 ICIMS 242 83 20 VARSAM∗ VEC ∅ L ICIMS 197 108 52
VARSAM∗ DV 1 I2 IMS 86 67 21 VARSAM∗ DV ∅ L IMS 45 92 53
VARSAM1 DV ∅ I2 DMSh 29 5 22 VARSAM∗ DV ∅ ∅ IMS 45 94 53
VARSAM∗ DV ∅ I2 DMSh 50 6 23 AMIMS VAR ∅ ∅ ICDMS 295 56 55
VARSAM∗ DV 1 I2 ICIMS 105 74 24 VARSAM1 VEC 1 L IMS 219 231 56
AM ∅ ∅ DMSh 88 250 25 VARSAM1 DV ∅ L DMSh 58 37 57
VARSAM1 DV 1 I2 IMS 89 89 26 VARSAM1 DV ∅ ∅ DMSh 58 39 57
AMIMS DV 1 I2 IMS 67 63 27 ARIMA (2, 0) ∅ ∅ FIMS 24 109 59
AM ∅ recI2 DMSh 299 249 28 VARSAM1 VAR ∅ L IMS 52 84 60
VARSAM1 DV 1 I2 ICIMS 181 110 29 VARSAM∗ DV ∅ L DMS 25 75 61
VARSAM1 VEC ∅ L ICIMS 68 48 30 VARSAM∗ DV ∅ ∅ DMS 25 77 61
AMIMS DV 1 I2 ICIMS 178 137 31 VARSAM∗ DV 1 L DMS 60 96 63
ARIMA (2, 0) ∅ ∅ FIMS 359 273 32 VARSAM∗ DV 1 ∅ DMS 60 98 63
ARIMA (1, 0) ∅ ∅ FIMS 372 298 33 AMIMS DV ∅ L DMS 18 160 68
VARSAM1 DV ∅ L ICIMS 38 40 34 AMIMS DV ∅ ∅ DMS 18 162 68
VARSAM1 DV ∅ ∅ ICIMS 38 42 34 AMIMS DV 1 L DMS 65 189 70

we can be confident that any location shifts that the
South African economy may have experienced over
the sample are not too detrimental to short horizon
forecasting, although they matter for larger h.
In contrast, level forecasting, such as with VAR
techniques, performs best at h = 8 in Table 9, in
the left-hand columns, and, interestingly, we notice
that AMIMS, VARSAM1 and VARSAM∗ are then
the best performing; they are also generally good at
shorter horizons. Models in differences that exclude
cointegrating relationships, DV, also appear in the
tables, never as the most accurate, but always among
the best techniques.
How do these models interact with the choice of
iterated or multi-step forecasting, with or without
intercept correction? When VEC is most accurate, it is
using either DMS (or occasionally DMSh) over short
horizons, or ICIMS for larger h. At short horizons,
ICDMSh also appears later on in the table when the
DMSh version of VEC is best (see Table 8, overall
ranking). Interestingly, when forecasting the level
(VAR) performs well, it is mostly associated with IMS
(although DMS is present in Table 8, overall ranking)
620 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
when the I2 trend is used, but with ICIMS when it is
not. This implies that ICIMS provides a robust way
to account for the evolutions of the trend that occur
immediately following the forecast origin, especially
at longer horizons.
As for DV models, all combinations occur.
However, it emerges that at short horizons, DV IMS
is best if combined with I2, while DV DMS is
best if not; DMSh is better at intermediate horizons
(an annual horizon for quarterly data with residual
seasonal patterns); and for h = 8, again, DV IMS
is best if combined with I2, but if not, DV ICIMS is
preferred.
The above results, which show that methods
deemed robust to breaks, namely DMS and IC,
feature prominently, are all the more interesting when
compared to the univariate models that perform well.
Indeed, it is striking that the ARIMA models that
appear in the tables are without exception fractionally
integrated. Recent work by Perron and Qu (2007) has
shown how occasional breaks can yield a pattern akin
to long memory. R. Bhansali had already shown in
several papers, see e.g. Bhansali and Kokoszka (2002),
that DMS could prove useful when dealing with long
range dependence, such as that arising from fractional
integration; see also Brodsky and Hurvich (1999) and
Bos, Franses, and Ooms (2002). The results of our
experiment are not fully satisfactory in this respect,
since most of the occurrences of ARFIMA models in
the tables involve FIMS, and occasionally ICFIMS,
but only once FDMS. However, the clear link between
breaks and long memory shows promise for further
research.
4.3.3. Horizons
Clearly, horizons matter, and in particular as
regards the forecasting technique developed by AM. It
performs best for its horizon of focus, h = 4, and over
longer horizons, but not for h < 4. It is interesting to
notice that DMSh, which essentially models seasonal
differences, like AM, performs particularly well at
horizons that are multiple of 4, i.e. (pluri-)annual
horizons. This shows that, although the data we
use are seasonally adjusted, there must remain some
seasonal effects. It is obvious that in an economy that
has experienced many breaks, seasonal components
cannot be perfectly estimated.
More generally, it appears that the most accurate
methods vary with h. In particular, very short horizons,
h = 2, do not rank the same forecasting techniques as
those where h > 2, see Table 7: when preferred at
h = 2, the techniques perform increasingly worse as
h grows. In contrast, most models which are ranked at
h = 8 do not fare so badly for h = 2. Hence, good
medium term forecast accuracy is a reasonable gauge
for shorter-term quality, but the converse does not
seem to hold, particularly when short-run techniques
involve cointegrating relationships. The latter should
therefore not be referred to as long-term equilibria,
as implicitly argued by Hendry (2006), since they are
subject to intermittent breaks.
4.3.4. Trends and dummies
The I(2) trend, when estimated over the whole
sample, incorporates bilateral information at each
point in time t, and hence alters the forecasts for the
better. Indeed, I2 features most of the time in the left
columns of Tables 7–9. When this trend is excluded,
the relative performances of the various techniques
are altered; IMS in particular fares less well if not
combined with intercept correction. Surprisingly, no
alternative trend specification dominates. Both the
linear trend (L) and no trend at all (∅) feature
prominently, and each model appears twice, first with
L then without a trend. Hence, linear trends are
preferable to no trend, but this does not change the
rankings of the models. Only at horizon h = 4 does
the recursive I(2) appear consistently, and it then tends
to lie between the other two, and provides little gain,
except for the very best models.
For each model, either three different sets of
dummy variables were used or none. In all of the
tables, the models without dummies perform best.
The same techniques appear again further down in
the tables, but with the dummies DUM92, N and
FLIB included. Hence, there does not seem to be an
argument favouring their use here.
4.4. Forecast accuracy improvements
We now assess the average improvement in forecast
accuracy that is brought by the choice of method.
Tables 10–13 record the average ratios of RMSFEs
for the methods delineated in the first three columns
over that of a reference model, e.g., with respect to
recI2 trend in Table 10. We analyze four elements of
improvement in turn. Extended tables with h = 3, 5
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 621

Table 10
Forecast accuracy comparisons: average RMSFE ratios over the corresponding models with a recursively estimated I(2) trend; the averages are
computed AMIMS and VARSAM models (VEC is only for the latter).

Model I2 trend Linear trend No trend

h=2 h=4 h=8 h=2 h=4 h=8 h=2 h=4 h=8
VAR IMS 0.853 0.82 0.848 0.988 1.106 1.194 1.104 1.408 1.852
DMS 0.671 0.665 0.764 0.779 0.794 0.882 0.844 0.825 0.923
ICIMS 0.946 0.912 1.006 1.050 1.180 1.443 0.998 0.916 1.041
ICDMS 0.591 0.567 0.699 0.695 0.686 0.876 0.783 0.666 0.711
DV IMS 0.805 0.741 0.613 0.816 0.772 0.690 0.816 0.772 0.690
DMS 0.939 0.941 0.937 0.947 0.962 0.929 0.947 0.962 0.929
DMSh 0.739 0.757 0.928 0.715 0.924 0.974 0.715 0.924 0.974
ICIMS 0.852 0.772 0.630 0.829 0.773 0.688 0.829 0.773 0.688
ICDMS 0.967 0.970 0.978 0.969 0.995 0.996 0.969 0.995 0.996
ICDMSh 0.79 0.734 0.931 0.758 0.867 0.977 0.758 0.867 0.977
VEC IMS 0.837 0.744 0.622 0.856 0.757 0.660 0.906 0.916 0.999
DMS 0.895 0.916 0.889 0.935 0.911 0.886 0.703 0.693 0.911
DMSh 0.786 0.854 0.818 0.761 1.091 0.941 0.959 0.921 1.043
ICIMS 0.863 0.752 0.629 0.832 0.726 0.641 0.639 0.655 0.844
ICDMS 0.961 0.934 0.948 0.970 0.957 0.970 0.814 0.764 0.639
ICDMSh 0.804 0.822 0.848 0.771 1.015 0.968 0.936 0.945 0.935

Table 11 IMS in the level of the VAR, where recI2 improves

Forecast accuracy comparisons: average RMSFE ratios over the accuracy over both the linear trend and no trend.
corresponding models without taking cointegration into account.
The main message from Table 10 is that there is
Model Forecast horizon an ordering between the trends: I2 is best, followed
2 4 8 by L, ∅, and finally recI2. Exceptions to this rule,
VARSAM1 VEC IMS 0.942 0.938 0.924 besides that which we reported above, concern (i) IC
DMS 0.885 0.989 0.936 methods in VAR level forecasting, where at longer
DMSh 0.989 1.013 1.009 horizons it is better not to include a trend than to
ICIMS 0.941 0.929 0.918 use L; and (ii) all methods, bar IMS, with the model
ICDMS 0.908 0.945 0.923
ICDMSh 0.962 1.008 0.966
in differences without (DV) or with cointegration
VARSAM∗ VEC IMS 1.064 1.035 1.005
(VEC, except for DMSh) at short horizons: methods
DMS 0.957 1.091 1.056 including an I(2) trend, no matter how it is estimated,
DMSh 0.972 0.941 1.042 are not preferable.8
ICIMS 1.077 1.033 0.999 Three recommendations follow: (i) since the I2
ICDMS 0.972 1.029 1.024 trend cannot be estimated ex ante, a modeller should
ICDMSh 0.972 0.945 1.024 avoid computing it recursively when forecasting with
any method that is meant to improve robustness to
misspecification. In contrast, (ii) recI2 proves useful
that also report the MAPE are available from the IJF when forecasting nonstationary variables in level
website. with IMS, but if intercept correction is used for
the nonstationary variables, then any kind of trend
4.4.1. Trends should be omitted. Finally, (iii) when dealing with
When comparing the various models with respect the differenced series, including the differences in the
to the choice of trend, as in Table 10, it is clear that
the recursively estimated I(2) trend leads to the worst 8 A linear trend and no trend are equivalent for the model
accuracy in most cases. However, its performance estimated in differences without cointegration (DV), since a
improves at longer horizons, in particular when using constant is always included.
622 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628

Table 12
Forecast accuracy comparisons: average RMSFE ratios over the corresponding models estimated in differences rather than in levels.

Model I2 trend recI2 trend Linear or no trend

h=2 h=4 h=8 h=2 h=4 h=8 h=2 h=4 h=8
AMIMS or VARSAM IMS 0.978 0.955 1.162 0.950 0.907 0.910 1.178 1.388 1.839
DMS 1.352 1.315 1.588 1.903 1.823 1.899 1.616 1.544 1.863
ICIMS 0.862 0.819 1.008 0.769 0.649 0.556 0.951 0.899 1.043
ICDMS 1.118 1.278 1.716 1.821 2.147 2.350 1.387 1.475 1.889

Model Overall ar = 0 ar 6= 0
h=2 h=4 h=8 h=2 h=4 h=8 h=2 h=4 h=8

ARIMA( p, d, q) IC/no IC IMS 0.919 0.807 0.538

FIMS 0.717 0.593 0.48
DMS 7.84e07 1.60e07 4.49
FDMS 6.16e06 13.646 1.091
ARIMA(ar, ma) no IC IMS 7.820 6.392 4.633 0.718 0.753 0.77
FIMS 2.066 2.307 2.283 0.666 0.669 0.662
DMS 0.528 0.668 0.901 1.357 1.581 1.598
FDMS 0.521 0.569 0.574 0.979 1.062 1.091

ARIMA(ar, ma) with IC IMS 245 1.016 1.036 2.274 2.283 2.125
FIMS 0.833 0.878 0.92 1.148 1.053 0.955
DMS 1.611 1.081 0.766 0.469 0.362 0.214
FDMS 0.754 0.858 0.794 0.526 0.459 0.193
Note: for the ARIMA models, the comparison is with the model imposing d = 0.

stochastic trend reduces accuracy and does not capture
the potential breaks in mean growth.
4.4.2. Cointegration
We now turn to the usefulness of cointegration
relationships: they were specified using the whole
sample, and, despite reasonable constancy of the
cointegration vectors, will necessarily be misspecified
in subsamples. Table 11 shows three key results.
First, neither the gains nor the losses in forecast
accuracy are as substantial as in other comparisons:
RMSFE ratios vary between 0.88 and 1.077. Second,
if cointegration relationships improve accuracy at all,
they do so mostly at short forecast horizons for
multi-step methods and at longer horizons for iterated
methods. The ratios do indeed decrease with h for
(IC)DMS(h) and increase with h for (IC)IMS. Third,
the differences between VARSAM1 and VARSAM∗
do not permit us to conclude which of VEC or DV
seems better overall. However, as we noted that the
rank of cointegration may be underestimated in the
case of VARSAM∗ , the table suggests that including
relevant equilibrium correction mechanisms benefits
accuracy, and that it is costly to underestimate the
number of cointegrating vectors, or, even worse, omit
them altogether. Care must be taken with these results:
the analysis that we have carried out here is by no
means sufficient to obtain a definitive answer, and
questions remain as to whether or not equilibrium
relationships are useful in forecasting, see Hendry
(2006). All we can say is that the forecast failure that
they potentially induce may not be as prevalent as was
feared: South Africa has experienced numerous breaks
over its recent history, and yet cointegration is useful
in forecasting.
4.4.3. Differences versus levels
Now for a vexed issue: is it preferable to forecast
in differences or in levels? Marcellino et al. (2006)
chose to compare forecasts with differenced variables,
but we also considered levels in our analysis.
Table 12 records the ratios of accuracy statistics
for level forecasts to those for the differences.
The discrepancy between multivariate models and
their univariate counterparts is striking. It appears
that DMS forecasting in levels does not compare
advantageously with targeting the differences: the
latter is more accurate for all models, except in the
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 623

Table 13
Forecast accuracy comparisons: average RMSFE ratios over the corresponding IMS models.

Model Forecast horizon

2 4 8
AMIMS and VARSAM VAR DMS 1.266 0.947 0.965
ICDMS 1.709 1.662 1.899
DV DMS 0.834 0.716 0.751
DMSh 1.069 1.050 1.141
ICDMS 1.034 0.839 0.855
ICDMSh 1.236 1.091 2.462
DDVDMS 0.858 0.762 0.622

VEC DMS 0.740 0.715 0.766

DMSh 1.548 1.0145 3.047
ICDMS 0.938 0.797 0.850
ICDMSh 1.297 0.995 2.814
AM dum 6= 2 DMSh 1.039 0.748 0.779
dum = 2 DMSh 1.129 0.938 1.932
ARIMA( p, d, q) DMS 8.21e07 9.53e06 6.611
DDMS 0.946 0.487 0.783
FDMS 7.85e06 11.31 1.779
ARIMA(ar, ma) ar = 0 DMS 0.094 0.155 0.331
DDMS 2.945 1.583 1.947
FDMS 0.346 0.380 0.473
ARIMA(ar, ma) ar 6= 0 DMS 1.760 1.749 2.049
DDMS 2.810 3.290 5.981
FDMS 2.004 2.213 2.206

cases of (i) IMA models, and (ii) ARIMA with IC
and nonzero autoregressive lag order. In both cases,
the improvements from using levels over differences
are substantial, and are similar for integrated and
fractionally integrated models. The other instances
where levels perform unambiguously better are
(iii) for ICIMS, with stochastic trends (I2 or recI2),
and in a few cases with a linear or no trend,
(iv) IMS with recI2, and (v) ARIMA( p, d, q) models
with IMS or FIMS. It should be noted that when
using a recursive I(2) trend, and this type of trend
only, using levels rather than differences enhances
the accuracy of ICIMS, and in most cases of
IMS, and that this improvement increases with the
horizon.
Two conclusions follow from these results. First,
DMS is to be preferred in levels for univariate models
(but do not let the Schwarz criterion pick the model
in this case!) and in differences for multivariate
models (and maybe with cointegration, as noted in the
previous subsection).
Second, in multivariate models, ICIMS works best
in levels (without a trend, as we saw in Section 4.4.1),
but IMS should be used in differences, unless a
stochastic trend is included in the model, in which case
the levels are preferable. The recommendation is the
opposite in univariate models: it is better to use the
levels when forecasting with IMS in autoregressive
processes, and the differences when using ICIMS,
with the exception that levels are always preferable in
conjunction with (IC)IMS when lag orders are chosen
using the Schwarz criterion.
4.4.4. IMS versus DMS
Finally, we consider the question at the core of
this paper, that of IMS versus DMS, as in Table 13.
We consider the various versions in turn. First, the
standard version of DMS tends to perform better
than IMS, and especially so at intermediate horizons
(2 < h < 5). The notable exception is ARIMA
models with an imposed nonzero autoregressive lag
order. The relative accuracy improvement of DMS
over IMS is stronger for DV and VEC models than for
624 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
VAR. Indeed, as a rule DMS cannot be recommended
for the multivariate models in levels. For univariate
models, the picture is more mixed: gains are not
straightforward for AM, although DMSh marginally
improves upon IMS (cumulated dummies, d2, strongly
modify the results, but the accuracy deteriorates when
they are used). DMS also performs well for IMA
models, but not for the other reported cases. However,
in ARIMA models, DMS performs appallingly when
used for models estimated in differences, unless the
Schwarz criterion is used for specification, in which
case they perform much better than IMS. Note that
fractional DMS is not a good idea: it is better to reserve
fractional models for IMS.
It also appears from the table that DMSh does
improve upon DMS in general. Also, intercept
correction is better used in conjunction with IMS
rather than DMS, although both ICDMS and ICDMSh
exhibit patterns similar to DMS: they are better at
intermediate horizons.
Finally, robust double differencing seems to
perform well in its direct multi-step version. However,
DDVDMS never appears in the models with the best
results in terms of forecast accuracy, and hence it is
not clear that its use should be recommended as a rule.
The conclusions that we draw are twofold:
DMS is better than IMS in multivariate models
in differences or equilibrium-correction forms, and
at longer horizons for models in levels. Additional
corrections to DMS are not necessary. However, these
corrections may benefit IMS and render it more
accurate than DMS (see the previous discussions).
DMS is also useful in IMA models, but not so much in
ARIMA models with fixed lag orders. This confirms
the results of Stoica and Nehorai (1989) and Tiao
and Xu (1993) for IMA models and Ing (2003) for
autoregressive models. As pointed out by a referee, it
may be possible that data mismeasurement plays a role
here. Measurement errors may indeed induce negative
serial correlation, and this could explain why DMS
performs well with IMA models.
4.5. Robustness to breaks
We now address the question which underlies the
choice of South Africa as a country for the analysis: is
robustness to breaks essential to forecast accuracy? To
answer this, we contrast the forecast performance of
the most accurate techniques with the break intensity,
as defined in Section 2.1.
We compute the correlation between the absolute
ex ante forecast errors and break intensity, over the
whole sample and for each technique.9 We then order
the estimated correlations according to the overall
forecast accuracy of each technique (as reported in the
left columns of Tables 7–9). For each of h ∈ {2, 4, 8},
we obtain an ordered sample of 779 correlations.
We then consider recursive samples of the n first
correlation coefficients (i.e., corresponding to the n
most accurate techniques), where n varies from 10
to 500. For each of these recursive samples, Fig. 9
records the quartiles of the distribution of the n
correlations, where n is reported on the horizontal
axis. The purpose of this exercise is to check
whether the accuracy is negatively related to the
break intensity, and whether this holds for the best
performing techniques and only for them. We use
quartiles because some techniques which have low
accuracy may also, by chance or not, exhibit negative
correlation, but we do not expect this to occur too
much as we move to the least accurate methods.
Fig. 9 records these quartiles for rankings at
h = 2 and h = 8, while Fig. 10 does the same
for h = 4, distinguishing two subperiods. For the
choice of these subsamples, we refer to Table 1,
so that the two periods exhibit different features:
the first covers 1973(4)–1986(4), when the South
African economy went through many changes and the
breaks were rather frequent; whereas in period 2, from
1986 to 1995, the country was not very involved in
the international economy, and the system (i.e., the
legal–political environment) did not evolve as fast
(which is not to say that the economy did not suffer,
say, from the 1992 drought). From 1995 to 1998, after
the democratic elections, the economic environment
(banking and financial sectors) was relatively stable,
and deregulation took place again afterward. The
columns on the right of the figures present magnified
versions of the left columns, focusing on the 100 most
accurate techniques.
From these figures, two remarks can be drawn.
First, there is a large contrast between short (h = 2)
9 In computing the correlation, we consider break intensity
measured over the 30 observations prior to the value to be forecast
(not the forecast origin). Hence, we also allow for breaks occurring
over the forecast horizon.
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628 625

Fig. 9. Quartiles of the distribution of coefficients of correlation between the absolute forecast error and the break intensity. The quartiles are
computed for the sample of the n most accurate techniques at that horizon, where n is on the horizontal axis. Dotted lines represent the first and
third quartiles, crosses are used for the median.

and long horizons (h = 8). Indeed, robustness to
breaks does not seem to matter for h = 2, whereas we
observe a clear pattern at h = 8: the 80 most accurate
forecasting techniques do better in the presence of
breaks, but this feature disappears when considering
the remaining methods. Similar patterns arise at h =
4 on either subperiod for the best 40 techniques,
although this is clearer in the second subperiod.
From the figures, we can infer that, at longer
horizons, being accurate in the presence of breaks is
key to forecast accuracy. It does not seem to matter
at short horizons. This in line with the discussions
above, namely that equilibrium correction is useful at
short horizons (Hendry, 2006) and that both DMS and
ICIMS improve upon IMS at long horizons.
5. Conclusions
The purpose of our analysis was to observe,
in an empirical exercise, whether direct multi-step
estimation can improve the accuracy of forecasts, and
if so, when it can. We knew from theoretical results
obtained by, inter alia, Peña (1994) and Schorfheide
(2002), that amongst the conditions beneficial to DMS
are those of model misspecification and non-constancy
of the DGP. Given that Aron and Muellbauer (2002)
have derived an equation for forecasting the South
African GDP which uses direct multi-step estimation,
and that the national economy of this country has
experienced several regime changes and extraneous
shocks, we decided to build our forecast accuracy
comparison drawing on this research. The strategy on
which we settled was to derive 779 models and to
record measures of their corresponding historical ex-
ante forecast accuracy.
The results are, essentially, that the direct
equation derived by Aron and Muellbauer (2002)
has impressive forecasting power at horizons at
least as large as that for which it was designed,
viz. four quarters ahead. It is therefore worth the
626 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628

Fig. 10. Quartiles of the distribution of coefficients of correlation between the absolute forecast error and the break intensity. The quartiles are
computed for the sample of the n most accurate techniques at that horizon, where n is on the horizontal axis. Dotted lines represent the first and
third quartiles, crosses are used for the median. Correlations are computed over two subsamples: pre-1987 (period 1) and post-1986 (period 2).

effort of designing a parsimonious economic equation
aimed at DMS forecasting. However, as these authors
specified their model using the whole sample, it is
only reasonable that it should have good forecasting
properties: it is difficult to judge its performance
further since this model does not forecast so well
at all horizons. At short horizons, we found that
equilibrium-correction models rank among the most
accurate when combined with DMS. However, the
best techniques at h = 2 are no guide for
their performances at longer horizons (though the
converse holds). Indeed, it is multivariate models for
the differences of the variables that perform best,
with DMS at intermediate horizons, and with IMS
combined with intercept correction at large horizons.
It is hence important to study multi-step forecasting in
multivariate models further, as in Jorda and Marcellino
(2007), and more needs to be discovered of the
properties of IC and IMS. Univariate ARFIMA models
also rank high. In accordance with these results, we
showed that robustness to breaks was achieved by
these methods at intermediate and long horizons, and
that this seems to be a significant feature for forecast
accuracy.
Acknowledgements
I am grateful to David Hendry, Janine Aron, John
Muellbauer, Michael Clements and three anonymous
referees for helpful suggestions.
References
Aron, J., & Muellbauer, J. (2002). Interest rate effects on output:
Evidence from a GDP forecasting model for South Africa. IMF
Staff Papers, 49, 185–213.
Bai, J., & Perron, P. (2003). Computation and analysis of multiple
structural change models. Journal of Applied Econometrics, 18,
1–22.
 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Bhansali, R. J. (1996). Asymptotically efficient autoregressive
model selection for multistep prediction. Annals of the Institute
of Statistical Mathematics, 48, 577–602.
Bhansali, R. J. (1997). Direct autoregressive predictors for multistep
prediction: Order selection and performance relative to the plug-
in predictors. Statistica Sinica, 7, 425–449.
Bhansali, R. J., & Kokoszka, P. (2002). Computation of the forecast
coefficients for multistep prediction of long-range dependent
time series. International Journal of Forecasting, 18, 181–206.
Bos, C. S., Franses, P. H., & Ooms, M. (2002). Inflation, forecast
intervals and long memory regression models. International
Journal of Forecasting, 18(2), 243–264.
Brodsky, J., & Hurvich, C. M. (1999). Multi-step forecasting for
long-memory processes. Journal of Forecasting, 18, 59–75.
Chevillon, G. (2007). Direct multi-step estimation and forecasting.
Journal of Economic Surveys, 21(4), 746–785.
Chevillon, G. (2008). Multi-step forecasting in the presence of weak
trends. Mimeo. ESSEC Business School.
Chevillon, G., & Hendry, D. F. (2005). Non-parametric direct
multi-step estimation for forecasting economic processes.
International Journal of Forecasting, 21, 201–218.
Chow, G. C. (1960). Tests of equality between sets of coefficients in
two linear regressions. Econometrica, 28, 591–605.
Clark, T. E., & McCracken, M. W. (2005). Evaluating direct
multistep forecasts. Econometric Reviews, 24(4), 369–404.
Clements, M. P., & Hendry, D. F. (1996). Multi-step estimation
for forecasting. Oxford Bulletin of Economics and Statistics, 58,
657–683.
Clements, M. P., & Hendry, D. F. (1999). Forecasting non-
stationary economic time series. Cambridge, MA: The MIT
Press.
Cox, D. R. (1961). Prediction by exponentially weighted moving
averages and related methods. Journal of the Royal Statistical
Society, B 23, 414–422.
Doornik, J. A., & Hansen, H. (1994). An omnibus test for univariate
and multivariate normality. Discussion paper. Nuffield College.
Doornik, J., & Ooms, M. (2003). Computational aspects of
maximum likelihood estimation of autoregressive fractionally
integrated moving average models. Computational Statistics and
Data Analysis, 42, 333–348.
Eklund, J., & Karlsson, S. (2005). Forecast combination and model
averaging using predictive measures. Discussion paper no.
5268. CEPR.
Findley, D. F. (1983). On the use of multiple models for multi-
period forecasting. Proceedings of Business and Economic
Statistics, American Statistical Association, 528–531.
Findley, D. F., Potscher, B. M., & Wei, C. Z. (2004). Modeling of
time series arrays by multistep prediction or likelihood methods.
Journal of Econometrics, 118, 151–187.
Giacomini, R., & White, H. (2006). Tests of conditional predictive
ability. Econometrica, 74(6), 1545–1578.
Gonzalo, J. (1994). Five alternative methods of estimating long-
run equilibrium relationships. Journal of Econometrics, 60,
203–233.
Harvey, A. C. (1993). Time series models (2nd ed.). Hemel
Hempstead: Harvester Wheatsheaf.
627
Harvey, A. C., & Jaeger, A. (1993). Detrending, stylized facts
and the business cycle. Journal of Applied Econometrics, 8,
231–247.
Haug, A. A., & Smith, C. (2007). Local linear impulse responses
for a small open economy. Discussion paper 2007/09. Reserve
Bank of New Zealand.
Haywood, J., & Tunnicliffe-Wilson, G. (1997). Fitting time series
model by minimizing multistep-ahead errors: A frequency
domain approach. Journal of the Royal Statistical Society, Series
B, 59, 237–254.
Haywood, J., & Tunnicliffe-Wilson, G. (2004). A test for
improved multi-step forecasting. Mimeo. Victoria University of
Wellington.
Hendry, D. F. (2006). Robustifying forecasts from equili-
brium–correction systems. Journal of Econometrics, 135,
399–426.
Ing, C.-K. (2003). Multistep prediction in autoregressive processes.
Econometric Theory, 19, 254–279.
Ing, C.-K. (2004). Selecting optimal multistep predictors for
autoregressive process of unknown order. The Annals of
Statistics, 32, 693–722.
Johansen, S. (1995). Likelihood-based inference in cointegrated
vector auto-regressive models. Oxford: Oxford University Press.
Johnston, H. N. (1974). A note on the estimation and prediction
inefficiency of dynamic estimators. International Economic
Review, 15, 251–255.
Jonsson, G. (2001). Inflation, money demand and purchasing power
parity in South Africa. IMF Staff Papers, 42, 243–265.
Jorda, O., & Marcellino, M. (2007). Path forecast evaluation.
Mimeo. UC Davis.
Kang, I.-B. (2003). Multi-period forecasting using different models
for different horizons: An application to US economic time
series data. International Journal of Forecasting, 19, 387–400.
Klein, L. R. (1971). An essay on the theory of economic prediction.
Chicago, IL: Markham.
Koopman, S. J., Harvey, A. C., Doornik, J. A., & Shephard, N.
(2000). STAMP: Structural time series analyser, modeller and
predictor. London: Timberlake Consultant Press.
Lin, J., & Tsay, R. (2005). Comparisons of forecasting methods with
many predictors. Mimeo. Academia Sinica.
Lin, J. L., & Tsay, R. S. (1996). Co-integration constraint and
forecasting: An empirical examination. Journal of Applied
Econometrics, 11, 519–538.
Liu, S. I. (1996). Model selection for multiperiod forecasts.
Biometrika, 83(4), 861–873.
Marcellino, M., Stock, J. H., & Watson, M. W. (2006). A
comparison of direct and iterated multistep AR methods
for forecasting macroeconomic time series. Journal of
Econometrics, 135, 499–526.
Peña, D. (1994). Discussion: Second-generation time-series model,
A comment. Journal of Forecasting, 13, 133–140.
Perron, P., & Qu, Z. (2007). An analytical evaluation of the log-
periodogram estimate in the presence of level shifts. Mimeo.
Boston University.
Pesaran, M. H., & Timmermann, A. (2005). Small sample properties
of forecasts from autoregressive models under structural breaks.
Journal of Econometrics, 129, 183–217.
628 G. Chevillon / International Journal of Forecasting 25 (2009) 602–628
Proietti, T. (2008). Direct and iterated multistep AR methods for
difference stationary processes. Mimeo. University of Rome,
Tor Vergata.
Schorfheide, F. (2002). VAR forecasting under local misspecifica-
tion: Loss function estimation and model selection. Working pa-
per. Economics department, University of Pennsylvania.
Schumacher, C., & Breitung, J. (2008). Real-time forecasting of
German GDP based on a large factor model with monthly
and quarterly data. International Journal of Forecasting, 24(3),
386–398.
Stoica, P., & Nehorai, A. (1989). On multi-step prediction errors
methods for time series models. Journal of Forecasting, 13,
109–131.
Tiao, G. C., & Tsay, R. S. (1994). Some advances in non-linear
and adaptive modelling in time-series analysis. Journal of
Forecasting, 13, 109–131.
Tiao, G. C., & Xu, D. (1993). Robustness of maximum likelihood
estimates for multi-step predictions: The exponential smoothing
case. Biometrika, 80, 623–641.
Tsay, R. S. (1993). Comment: Adpative forecasting. Journal of
Business and Economic Statistics, 11(2), 140–142.
Weiss, A. A. (1991). Multi-step estimation and forecasting in
dynamic models. Journal of Econometrics, 48, 135–149.
White, H. (1980). A heteroskedasticity-consistent covariance matrix
estimator and a direct test for heteroskedasticity. Econometrica,
48, 817–838.