You are on page 1of 18

A Framework for Measuring the Operational

Risk of Trade Entry Errors


Neil Walter
Quantitative Analyst
ConocoPhillips Commercial Division
R.N.Walter@ConocoPhillips.com
281-293-1565

Slide 1
Neil Walter

What is Operational Risk


Operational risk is defined as the risk of loss resulting from
inadequate or failed internal processes, people and
systems or from external events(1)
Monitoring errors and error rates and measuring their impact
focuses resources on eliminating systematic errors, which
not only helps managers understand operational risk but
also helps them manage the risk. Although not a
replacement for other forms of measuring operational risk,
the ability to systematically eliminate errors stands in stark
contrast to acknowledging a firm is susceptible to
operational risk and therefore simply reporting it.
Slide 2
Neil Walter

Six sections covered

Terminology
Three reasons why this presentation matters
Additional considerations for physical deals
A framework for measurement
Insights
References

Slide 3
Neil Walter

Terminology

Time frame does matter


Risk Management/Traders
Concerned with day to day mark-to-market. Errors persisting
beyond the transaction date may be the most relevant to this
constituency.

Finance
Concerned with monthly accrued changes. Errors crossing
reporting periods (monthly or quarterly) may be the most relevant
to this constituency.

Accounting
Concerned with settlement and reconciliation. Errors persisting to
settlement (physical or financial) may be the most relevant metric
to this constituency.
Slide 4
Neil Walter

Terminology

Late versus error versus rebook


Given a daily mark-to-market timeframe:
Late
Entered into trade capture system after date of execution.

Error/Correction
Change to any portion of the trade after the day it was entered into
the system.

Rebook
Change to the trade so that the risk exposure is altered after the
day it was entered into the system (a subset of error).

Nostro Break
Discrepancy between expected and actual cash settlements.

Slide 5
Neil Walter

Three reasons why this presentation matters

#1 It is not whether or not, it is how many errors


ISDA 2004 Operations Benchmarking Survey says error rates are high (2)
Average front-office error rates
Large Firms
Medium Firms
Small Firms
All Respondents

2003
8
15
6
10%

2004
14
7
12
10%

Trades that need to be rebooked


Large Firms
Medium Firms
Small Firms
All Respondents

2003
11
6
3
7%

2004
14
7
12
8%

Significant Error Rates for


OTC commodity derivatives
"Rebooking is significant from a risk
management point of view because it implies
that the trade data entered into the accounting
and risk management systems are in error and
therefore give an inaccurate picture of risk
exposure.
--ISDA 2004 Operations Benchmarking Survey p 7

Slide 6
Neil Walter

Three reasons why this presentation matters

#2 Errors take time to correct


ISDA 2004 Operations Benchmarking Survey says errors persist for more
than one week on average (2)
Proportion of confirmations sent
100% 100%
89%
83%

93%

3.4
2.5

83%

78%
73%
63%
59%

+
5.9

business days to process incoming confirms


business days to resolve trade discrepancies
and rebook trade
business days to get trade rebooked from
the time confirm is sent out

30%
15%

Trade Date

2003

T+1

T+2

T+3

T+4

T+5

2004

Slide 7
Neil Walter

Three reasons why this presentation matters

#3 Rates and time compound the impact by 7.5


Using the same ISDA data, the expected number of errors at any given
point of time is 75% of the daily deal entry rate!
Days to process confirm
Days to resolve discrepency
Days to correct error after confirm sent
Error Rate
End of business day
% Deals In
Confirms Sent
Errors Entered
Errors Corrected
Error rate for deals entered
on day 0 as of days 0-11

0
89%
15%
8.9%
0

1
100%
59%
1.1%
0

2
100%
78%
0%
0

1.5% = 15% * 10%

3.4
2.5
6
10%
3
100%
89%
0%
0

Errors Corrected=Confirms Sent * Error Rate

5
100%
100%
0%
0

6
100%

7
100%

8
100%

9
100%

10
100%

11
100%

0%
1.5%

0%
4.4%

0%
1.9%

0%
1.1%

0%
0.4%

0%
0.7%

8.9% 10.0% 10.0% 10.0% 10.0% 10.0%

8.5%

4.1%

2.2%

1.1%

0.7%

0.0%

Total errors as a % of the daily deal entry rate 75.5%


Weighted average business days an error persists
7.66
Total errors as a % of the monthly deal entry rate 3.8%
Plus
% of one day's deals not entered at any given time 11.0%

4
100%
93%
0%
0

Significant Error Rates for mark-tomarket calculations


Slide 8
Neil Walter

Additional considerations for physical deals

Generic Financial Deal Flow Diagram


Financial Deal Flow Diagram
Trade is
documented
(phone
recording, other
methods)

Trade entered
into system

Confirmation is
sent out

Expiry

Trade is
executed by
trader

Check out with


counter party &
exchange cash

Late trade MTM P&L exposure exists until corrected or


expiry
Operational risk exists until omission is corrected

Confirmation is primary
control as parties indicate
they agree the terms are
correct.

The remaining operational risk is


a nostro breakthe discrepancy
between expected and actual
cash settlements.

Slide 9
Neil Walter

Additional considerations for physical deals

Generic Physical Deal Flow Diagram (natural gas)


Physical Deal Flow Diagram
Trade is
documente
d

Trade
entered into
system

Confirmation
is sent out

Schedule
gas on EBB

Delivery

Trade is
executed
by trader

Balance
pipeline

Invoice
generated

Pay/receive
cash

Late trade MTMP&L exposure exists until corrected or


nominated
Operational risk exists until omission is corrected

Confirmation remains primary


control as parties indicate
they agree the terms are
correct.

Operational risk remains as


physical gas has yet to flow
as well as the possibility of
nostro breaks

Slide 10
Neil Walter

Additional considerations for physical deals

Material Attributes on Commodity Trades


Common to Financial and Physical

Physical Trades (in addition to those at left)

Commodity type (may be more than


one if spread or basket)
Executed Price
Contract size
Volume (may be more than one if
spread or basket)
Location (more than one if spread or
basket)
Maturity/Expiration
Options: Strike/Volatility/Correlations
Commission
Counter party
Trade Book
Trade date
Trade Type
Settlement Type

Delivery start
Delivery End
Fixed transportation charges
Variable transportation charges
Transportation path
Additional balancing transactions

If any of these parameters are


incorrect at the time of entry, the
P&L may be impacted and
decision making may be altered
based on erroneous information

Slide 11
Neil Walter

A framework for measurement

What is the impact? Rethink variance analysis


We would like to think that
Price

Value1-Value0=mark-to-market
Value1

P1

Mark-to-market is actually:
Q0(P1-P0)

P0

Value0

Quantity

Q0 Q1

+ Qi (P1-Pi)
+ Qi (P1-Pi)
+ Qi-new(P1-Pi-new)
Qi-old(P1-Pi-old)

price on orig. portfolio


P&L since inception (new deals)
P&L since inception (late deals)
P&L on corrected trades

Old P&L of trade now

corrected

Mark-to-market

Slide 12
Neil Walter

A framework for measurement

Why measure? The expected error is zero


Even if your expected error is zero, it is not the expected
value, but rather the variance that is relevant.

As with VaR, presuming the


expected change in
portfolio mark-to-market
is zero does not render
the VaR estimate
useless.
-2.5

-2

-1.5

-1

-0.5

0.5

1.5

2.5

Slide 13
Neil Walter

A framework for measurement

Estimate risk using your own data


Historical Simulation
Calculate the MTM impact of errors for each day over a
significant period of history.
Utilize this distribution to construct a cumulative
distribution function.
For any given day, determine the impact by drawing from
the uniform distribution. Determine means to find the
value that corresponds with the smallest CDF probability
larger than the uniform distribution observation.

Slide 14
Neil Walter

A framework for measurement

Measuring risk: an actuarial approach


In this example:
1. Model the frequency of errors. (Poisson or NB dist)
2. Model the number of days for each error to be corrected. (Poisson or NB
dist)

3.
4.

Model the MTM impact of each error given the number of days it
took to get corrected. (Normal dist)
Infer an aggregate distribution for the errors.

For this example, assumptions are consistent with the ISDA data
provided on OTC commodity derivatives.
For more information on aggregate distributions and the generalized methodology used here

Klugman, Panjer and Willmot. Loss Models: From Data to Decisions. New York: Wiley, 1998. Chapter 4
A sample model similar to the one used for this presentation is available upon request. Please contact Xianqiao
Chen at xianqiao.chen@conocophillips.com
Sahay and Dev A Forward-Looking Adjustment for Operational Risk Quantification Risk, April 2005 Vol. 18, No 4
Samad-Khan, Ali Why COSO is Flawed Operational Risk Magazine, January 2005

Slide 15
Neil Walter

A framework for measurement

Aggregate loss distributions highlight variance

12%

10%

8%

6%

The distribution of daily errors ~N(0,25000) is


significantly smaller than the aggregate distribution--so
much so that the normal distribution is distorted when
plotted using the points of this histogram.
Negative Binomial returns more days
where there are 0 errors corrected.
The 0 corrected trade days result in a
disproportionate number of 0 MTM
observations.

Assumptions
Mean errors (per 100 trades)
Stdev errors
mean # days to correct (per trade)
Stdev # days
mean MTM impact (per trade)
Stdev MTM impact

Sample Mean
p-value (H0: Mean=0)
Sample Stdev
-1.65 * Sample Stdev

Poisson
Negative Binomial
10.0
10.0
3.2
6.0
7.6
7.6
2.8
5.0
25,000
25,000
Poisson
Negative Binomial
1,169
110
67%
60%
189,512
210,903
(312,696)
(347,991)

Note also the fatter tails resulting


from the Negative Binomial

4%

2%

42
,9
00
12
6,
36
3
20
9,
82
5
29
3,
28
8
37
6,
75
0
46
0,
21
3
54
3,
67
6
62
7,
13
8
71
0,
60
1
79
4,
06
4
87
7,
52
6

0%
(8
75
,1
89
)
(7
91
,7
26
)
(7
08
,2
64
)
(6
24
,8
01
)
(5
41
,3
39
)
(4
57
,8
76
)
(3
74
,4
13
)
(2
90
,9
51
)
(2
07
,4
88
)
(1
24
,0
25
)
(4
0,
56
3)

Frequency Proportion (10,000 simulations)

Poisson versus Negative Binomial Aggregate Histograms

Slide 16
Neil Walter

Insights

Error rates are higher for commodities than for other derivatives. Physical
deals increase the propensity for error.
Errors can persist for long periods of time. Monitoring provides a systematic
methodology for reducing errors and improving business processes.
Portfolio variance is more than price risk. Explaining daily or periodic MTM
changes necessitates assigning variance to corrective actions.
Although the MTM impact of an error for any given trade may be smalland
may even be expected to be 0 over timethe variance for any given day as a
result of errors can represent a significant portion of the total variance and
ought to be explained as such.
The modeling approach can be extended to other types of operational risk
such as nostro breaks (also a frequency/severity problem).
Other disciplines such as operations management and actuarial science
contain valuable methodologies for improving estimation of exposure and
reducing risk over time.
Slide 17
Neil Walter

References
1. Basel Committee on Banking Supervision. International Convergence of Capital Measurements and
Capital Standard: A Revised Framework, June 2004, P. 137
2. International Swaps and Derivatives Association. ISDA 2004 Operations Benchmarking Survey, P. 6-7
http://www.isda.org/c_and_a/pdf/ISDA-Operations-Survey-2004.pdf
For more information on aggregate distributions and the generalized methodology used here
Klugman, Panjer and Willmot. Loss Models: From Data to Decisions. New York: Wiley, 1998. Chapter 4
For more information on modeling operational risk
Kindall, Walter, and Chen. The Operational Impact of Belated Trades GARP Risk Review
November/December 04, Issue 21, P.6
A sample model similar to the one used for this presentation is available upon request. Please contact
Xianqiao Chen at xianqiao.chen@conocophillips.com
See the Basel Committee guidelines for operational risk. Basel Committee on Banking Supervision.
International Convergence of Capital Measurements and Capital Standard: A Revised Framework,
June 2004
Sahay and Dev A Forward-Looking Adjustment for Operational Risk Quantification Risk, April 2005
Vol. 18, No 4
Samad-Khan, Ali Why COSO is Flawed Operational Risk Magazine, January 2005
Slide 18
Neil Walter

You might also like