Professional Documents
Culture Documents
KWAKU ABROKWAH n this article, we use a large sample of volume surprise explains very little of
is an analyst, Equity
Execution Strategies, at
Goldman, Sachs & Co. in
New York, NY.
kwaku.abrokwah@gs.com
I order executions to better understand
“shortfall surprises”: the difference
between actual and expected trading
costs. Our trading cost measure is execution
shortfall, including both liquidity impact and
the shortfall surprise (only 0.3%), indi-
cating that higher-than-expected volume
does not reduce trading costs.
• Volatility surprise. Higher-than-expected
volatility over the execution horizon
GEORGE SOFIANOS the opportunity cost of slow executions.1 should result in higher-than-expected
is a vice president, Equity Exhibit 1 summarizes our main findings. In shortfall. We find that the volatility
Execution Strategies, at
Goldman, Sachs & Co. in
our sample, on average, only 20% of actual surprise explains only 0.1% of the short-
New York, NY. shortfall is predictable pre-trade. Focusing on fall surprise.
george.sofianos@gs.com the non-predictable component, we investi- • Spread surprise. Higher-than-expected
gate the following possible reasons for the quoted spreads over the execution horizon
shortfall surprise: should result in higher-than-expected
shortfall. We find that the spread surprise
• Price surprise. A higher-than-expected explains 1.8% of the shortfall surprise.
price increase over the execution horizon
should result in a higher-than-expected Price surprises, therefore, are by far the
actual shortfall on buy orders and lower- most important factor explaining shortfall
than-expected shortfall on sell orders. surprises; the volume, volatility, and spread
We find that, on average, the price sur- surprises have little explanatory power. These
prise explains 42% of the shortfall sur- findings have important implications for post-
prise. For orders in large-cap stocks that trade analysis, pre-trade tools, the development
take more than an hour to execute, the of algorithms, and the choice of execution
price surprise explains 73% of the short- strategies.
fall surprise. Splitting the price surprise In interpreting post-trade execution
into its alpha and market components, quality, for example, we must somehow con-
the alpha surprise explains 38% of the trol for the underlying execution-horizon price
shortfall surprise, and the market sur- move. For pre-trade analysis, our findings sug-
prise only 4%. gest that the only way to improve the pre-trade
• Volume surprise. Higher-than-expected t-cost estimates is to better predict the alpha-
volume over the execution horizon move over the execution horizon. Better pre-
should result in lower-than-expected trade volatility and volume estimates will not
shortfall. Surprisingly, we find that the help much. The same applies for algorithm
1
Sample period June 1 through December 31, 2006; 241,610 orders.
2
R-square from univariate regression of actual on expected shortfall.
3
Incremental R-square from multivariate regression analysis of shortfall surprise on the five factor surprises: volatility, spread, volume, EH-alpha and EH-market.
development and the choice of execution strategies; only orders, execution shortfall is the strike price minus the exe-
better predictors of execution-horizon alpha will signif- cution price as percent of the strike price. In Exhibit 2, we
icantly reduce execution shortfall. introduce a hypothetical order execution that we will use
In the next section, we develop our framework for throughout this section to illustrate our framework. The
explaining the shortfall surprise, followed by a section on trader receives an order to buy 60,000 shares. The strike
our data sample and the construction of our variables. price at order arrival is $25.00. The trader executes the
The following section presents our empirical findings. In order over time in three executions and the volume-
the section after that, we focus on our puzzling finding weighted execution price is $25.06. The execution short-
that volume surprises do not affect trading costs. The next fall in this example is six cents or 24 basis points (bps).
section elaborates on the distinction between volatility In addition to liquidity impact, execution shortfall
and price surprises. We conclude with a discussion of the includes the opportunity cost of delayed execution. The
implications of our empirical findings, and possible exten- opportunity cost arises because the price may move away
sions of our analysis. from the trader over the execution horizon. This price
move has both a market and a stock-specific (alpha) com-
A FRAMEWORK FOR ANALYZING ponent and we estimate both. Exhibit 3 shows the three
THE SHORTFALL SURPRISE components of execution shortfall: liquidity impact, alpha
loss, and market loss. In our example, the trader is buying
We begin by formally defining execution shortfall, our in a rising market, so the opportunity cost is positive and
t-cost measure. For buy orders, execution shortfall is the increases the shortfall. If the price were falling, the oppor-
execution price minus the prevailing mid-quote at order tunity cost would be negative, reducing the shortfall, and
arrival (strike price) as percent of strike price.2 For sell possibly resulting in negative shortfall. Unlike liquidity
impact, which is never negative, execution shortfall can 1. Stock capitalization: large-cap, mid-cap, and small-
be negative.3 cap stocks.
In this article, we analyze the shortfall surprise (SS) 2. Listing market: NYSE, AMEX, and NASDAQ
that we define as actual shortfall (SA) minus expected stocks.
shortfall (SE): 3. Order size: the actual dollar value of the order
executed.
SS = SA – SE (1) 4. Volatility over the execution horizon.
5. Quoted spreads over the execution horizon.
To generate the shortfall surprise, we must first esti- 6. Trading volume over the execution horizon.
mate expected shortfall. Expected shortfall is the short-
fall a trader can predict with only pre-trade information: The first three factors (stock capitalization, listing
when the trader receives the order, but before execution. market, and order size) we know with certainty pre-trade.
To estimate expected shortfall, we need estimates for both The other factors (volatility, spread, and volume) we do
liquidity impact (LE) and the expected underlying price not know with certainty pre-trade, and we must use their
move over the execution horizon (EH-price or PE): expected values to estimate liquidity impact (LE):
We estimate liquidity impact using the Goldman where Y are the factors known with certainty pre-
Sachs t-cost model.4 The model uses the following six trade, and XE are the factors not known with certainty
factors to predict liquidity impact: pre-trade.
We next discuss our EH-price concept. In Exhibit 4, trade. Our empirical analysis below confirms this is indeed
we assume the market component of EH-price is zero the case.
and show how we measure the stock-specific alpha move In Exhibit 4, we show how we use the closing
over the execution horizon (EH-alpha). Continuing the price to calculate EH-alpha. We first calculate the alpha-
Exhibit 2 example, the trader received the order at 12:00, to-close. For buy orders, alpha-to-close is the closing
executed 20,000 shares immediately, another 20,000 shares price minus the strike price as percent of the strike
at 13:00, and the final 20,000 shares at 14:00. The exe- price. For sell orders, alpha-to-close is the strike price
cution horizon is two hours (order arrival to last execu- minus the closing price as percent of the strike price.
tion), and the volume-weighted execution turnaround The alpha-to-close in our example is 40 bps. We then
time is one hour. The execution turnaround time is the “allocate” the alpha-to-close to the order in propor-
order’s half-life, taking into account that the order execu- tion to the order’s half-life. We derive the allocation
tion is spread over the two-hour horizon.5 We define EH- factor (Φ) by dividing the order’s half-life (one hour)
alpha as the price move over the order’s half-life, aside by the time from arrival to market close (four hours).6
from the liquidity impact of the trade itself. So, in our example, the allocation factor is 1/4 and EH-
In calculating EH-alpha, it is critical to use a post- alpha is 10 bps.
trade price after the liquidity impact of the trade sub- In practice, the underlying price move has both
sides. In our example, liquidity impact subsides at 15:00, market and alpha components:
so any price after that will work. In our empirical analysis,
we use the closing price to measure the actual EH-alpha EH-price = EH-alpha + βEH-market (4)
for each order in our sample. This procedure assumes
that the closing price is not affected by the impact of the where β is the intra-day stock beta.
1
If the time of last execution is after 16:00, it is possible for the ratio to exceed 1. In these cases (129 orders), we set the ratio to 1.
In our empirical analysis, we use S&P 500 ETF We next derive the shortfall surprises factor model
prices to decompose the underlying price move into EH- we use in our analysis. In Equation 5, replacing the
market and EH-alpha. In Exhibit 5, continuing our expected values of the various factors by their post-trade
example, the market-to-close move is 16 bps and allo- actual values, the actual shortfall (SA) is given by:
cating over the execution horizon EH-market is four bps.
Combining Equations (2) to (4) and, for illustration SA = a + bY + cXA + β MA + AA + U (6)
purposes, using a linear version of the liquidity impact
model L (Y, X), expected shortfall is given by: Actual shortfall is also influenced by other factors,
(U), not included in our empirical analysis. Most of these
SE = a + bY + c XE + β ME + AE (5) other factors are random. Some of them, however, may
be systematic but difficult to quantify even post-trade; for
where ME is the expected EH-market and AE is the e.g., trader skill or the presence of natural counterparties.
expected EH-alpha. Subtracting expected shortfall (Equation 5) from
Suppose the liquidity impact is 10 bps. To get the actual shortfall (Equation 6) we get our basic shortfall sur-
total shortfall, we must add the EH-price move. In prises model:
Exhibit 5, the EH-price move is 14 bps so the total
shortfall is 24 bps: 10 bps impact and 14 bps price loss. SS = c (XA – XE) + β (MA – ME) + (AA – AE) + U (7)
Or, writing out the factors in full: in the model is one, and the coefficient of the market sur-
prise is β, the intra-day stock beta.
S s = c1 (VOLA – VOLE ) + c 2 (SPREADA – SPREADE ) In deriving our shortfall surprise model (8), we assume
Volatility surprise Spread surprise that the liquidity impact model we use to generate expected
+ c 3 (VLM – VLM ) + β(EH-market A – EH-market E )
A E
impact is correctly specified. But the model may be mis-
Volume surprise Market surprise specified; we may have omitted systematic variables (U in
A
+ (EH-alpha – EH-alpha ) + U E
(8) our specification), or the model coefficients may be biased.
Alpha surprise
Model misspecification creates another possible source of
shortfall surprises. In the Appendix, we discuss this issue
In our analysis of the shortfall surprises, we focus on more formally and decompose shortfall surprises into a
the five input surprises: 1) the volatility surprise, 2) the misspecification component and the input surprises com-
spread surprise, 3) the volume surprise, 4) the market sur- ponent. In our empirical analysis, we carefully constructed
prise, and 5) the alpha surprise. The expected EH-market our sample to minimize the risk of model misspecification.
and EH-alpha are hard to estimate and, in our empirical
analysis, we assume they are zero. We describe the sur- SAMPLE CONSTRUCTION
prises in greater detail in the next section. Note that in AND DESCRIPTION
the surprises model, the factors known with certainty
pre-trade (Y) drop out. Also, because of the way we con- Our final estimation sample consists of 241,610 client
structed EH-price, the coefficient of the alpha surprises orders executed by Goldman Sachs over seven months,
1
Sample period June 1 through December 31, 2006; (241,610 orders).
2
Simple mean.
3
Value-weighted mean.
to last execution. To generate our impact estimates, we to the order’s execution horizon, by using the same
must also specify pre-trade estimates for volatility, quoted allocation factor Φ that we use in calculating EH-
spreads and trading volume: alpha (Exhibit 4).
• Quoted spread is the time-weighted spread over the
• Volatility is measured as the percent difference execution horizon, taking into account the spread
between the intra-day high and low price, adjusted smile (the U-shaped intraday spread pattern).11
EXHIBIT 8
Closing Prices are Not Affected by Liquidity Impact
Sample period June 1 through December 31, 2006; 241,610 buy and sell orders (Sign of sell orders flipped). Value-weighted averages.
EXHIBIT 9
Scatter plot of Actual Vs Expected Shortfall
EXHIBIT 10
The Five Factors: Actual Values1
1
Sample period June 1 through December 31, 2006; (241,610 orders).
2
Simple mean.
3
Value-weighted mean.
1
Sample period June 1 through December 31, 2006; 241,610 orders.
2
One minus the R-square from univariate regression of actual to expected.
1
Sample period June 1 through December 31, 2006; t-statistics in brackets below the coefficients.
factor surprise by dropping it from the full multivariate In Exhibit 13, rows two to four, we divide our
regression. The overview results in Exhibit 1 are from sample by stock capitalization: orders in large-cap stocks
the incremental approach and are similar to the uni- (>$7.5 billion), mid-cap stocks, and small-cap stocks
variate regression results. (<$1 billion). As we move from large-cap to small-cap
The last three columns in Exhibit 13 show another stocks, the accuracy of the expected shortfall forecasts as
example of the incremental approach. Column 10 shows measured by the root mean-square error (RMSE) falls.14
the full five-factor regression, and the regression in column The expected shortfall RMSE is 13 bps for orders in large-
nine drops the two EH-price surprises (market and alpha). cap stocks, but 49 bps for orders in small-cap stocks, indi-
Dropping the EH-price surprises reduces the explanatory cating that the shortfall surprise is higher in small-cap
power of the regressions from 44% to only 4.5%, con- stocks.
firming that the price surprises account for almost all of Our findings on the five factor surprises are similar
the explanatory power in our shortfall surprise regressions. across stock capitalization buckets. In all three buckets,
EXHIBIT 13
Shortfall Surprise Attribution: An Overview1
1
Sample period June 1 through December 31, 2006.
2
Root mean-square error: the square root of 1/N Σ(SAi – SEi)2, where SAi is the post-trade actual shortfall and SEi is the pre-trade expected shortfall for each order.
3
The average executed order size in this bucket is 775 shares.
1
Sample period June 1 through December 31, 2006; (241,610 orders).
2
Log of ratio of actual to expected volume.
the EH-price surprise explains most of the shortfall sur- the much smaller shortfall surprise. But still, across the
prise. The volatility and spread surprises explain more of five factor surprises, the EH-price surprise is by far the
the shortfall surprise in small-cap stocks than in the other most important factor in explaining the shortfall surprise:
two buckets, but the numbers are still small. In the uni- volatility, spread, and volume surprises explain 2.3%, while
variate regressions, for example, the spread surprise EH-price explains 14.7%.
explains 4.3% of the shortfall surprise in small-cap stocks, In Exhibit 15, we focus on the time dimension of
but only 0.1% of the surprise in large-cap stocks. The order executions. A quick execution exposes the order
volume surprise explains very little of the shortfall surprise to little EH-price movement, so we expect the EH-price
across all stock capitalizations. surprise to explain less of the shortfall surprise. At the
Because we construct EH-price using closing prices, other end, an order that takes several hours to execute is
one concern is that the closing prices are affected by liq- exposed to potentially large price moves, so we expect
uidity impact and therefore are not a pure measure of the EH-price to be much more important in explaining the
underlying price move. In Exhibit 9, we showed that, at shortfall surprise. The results in Exhibit 15 confirm this
least on average, the closing prices in our sample are not hypothesis.
affected by liquidity impact. The liquidity impact of the Exhibit 15 is divided in four panels: Panel A covers
larger trades in our sample is more likely to affect the orders in all stocks, Panel B focuses on orders in large-cap
closing price.15 So, in Exhibit 13, rows five to six, we fur- stocks, Panel C focuses on mid-cap stocks and Panel D
ther test the sensitivity of our results to the quality of the focuses on small-cap stocks. The top row in each panel
closing price by dropping the larger trades in our sample. replicates the information in Exhibit 13. The next two
In row five, we drop all orders greater than 25% of ADV rows in each panel divide the orders into orders with half-
(297 orders) and in row six, we drop all orders greater life more than 30 minutes and orders with half-life less than
than 15% of ADV (1,228 orders). In both cases, dropping 30 minutes. The last two rows in each panel repeat this
the outlier large orders does not affect our results. split, using a 60-minute cut-off.
In the last row in Exhibit 13, we focus on the small The first thing to notice in Exhibit 15 is that the
orders in our sample: 177,191 orders less than 0.1% ADV forecasting accuracy of expected shortfall falls sharply as
(average order size 775 shares). As expected, for these the execution horizon increases (column 4). In Panel A,
small orders, the shortfall forecasting error is low: the for example, the RMSE for orders with half-life more
RMSE is only 7 bps compared to 23 bps overall. For these than 60 minutes is 89 bps compared to only 15 bps for
small orders, the five factor surprises explain only 17% of orders with half-life less than 30 minutes. For orders in
1
Sample period June 1 through December 31, 2006.
small-cap stocks (Panel D), the RMSE for orders with Panel A, for example, EH-price explains 63.5% of the
half-life more than 60 minutes is even higher: 118 bps. shortfall surprise in orders with half-life more than 60
The forecasting accuracy results in Exhibits 13 and 15 minutes, but only 6.3% of the surprise for orders with
strikingly quantify what most traders know from experi- half-life less than 30 minutes. For orders in large-cap stocks
ence: most of the shortfall surprise comes from large orders with half-life more than 60 minutes, EH-price explains
that take time to execute. a remarkable 73% of the shortfall surprise.
In Exhibit 15, column 11 shows the explanatory Looking at the two components of EH-price
power of the full five-factor regression model. Across the (market and alpha), the results in Exhibit 15 across all
various buckets, the R-square of the full model ranges buckets are consistent with the results in Exhibit 13: EH-
from 75.3% (orders >60 minutes, large-cap stocks) to alpha is by far the most important factor in explaining
8.0% (orders <30 minutes, large-cap stocks). We next the shortfall surprise. For orders in large-cap stocks with
examine the contribution of the five-factor surprises in half-life more than 60 minutes, for example, EH-alpha
explaining the shortfall surprise. The last column in explains 65.3% of the shortfall surprise while EH-market
Exhibit 15 shows that, consistent with our hypothesis, explains only 12.5%. For orders with half-life less than
the EH-price surprise is much more important in 30 minutes, EH-alpha explains 6.1% of the shortfall sur-
explaining the shortfall surprise for slow executions. In prise while EH-market explains only 0.8%.
EXHIBIT 17
High Volume Surprises1
1
Sample period June 1 through December 31, 2006.
2
|VLMA – VLME| > one standard deviation.
1
Sample period June 1 through December 31, 2006.
EXHIBIT A1
Further Disaggregating the Shortfall Surprise1
1
Sample period June 1 through December 31, 2006; 241,610 orders.