You are on page 1of 7

Chapter 3

Investing in the future: amortized


analysis of operations (on data
stru tures)

3.1 What is amortized analysis?


In previous ourses you have probably had o asion to analyze the running time of a single
exe ution of an algorithm. Typi ally the analysis attempts to answer one, or both, of the
following questions:
 what performan e is guaranteed - how long will the algorithm run on the most diÆ ult
input of size n. This kind of analysis is usually alled worst ase analysis.
 what performan e is expe ted - how long will the algorithm run on the average if all
possible inputs of size n are equally likely to o ur. This kind of analysis is usually
alled average ase analysis.
For example, a straightforward implementation of Qui ksort has a worst ase running time
of O(n2 ) and an average- ase running time of O(n log n).
Impli it in these analyses is the assumption that when a person uses a program to solve a
parti ular problem it is often a one-time a air. And even if she runs the program repeatedly
 the data used by the algorithm in two di erent exe utions bear no relation to ea h
other, and
 the algorithm retains no memory of any previous run.
In ontrast, when an algorithm makes use of an Data Type it is rather to be expe ted
that
 the algorithm will make a sequen e of (di erent) requests on the Data Type;
15
16 CHAPTER 3. AMORTIZED ANALYSIS

 the data ontained in the Data Type will stay more or less the same between operations,
and
 previous operations in uen e the urrent state of the data stru ture used by the Data
Type.
After all, it is the purpose of some operations to hange the ontent, and sometimes even
the internal organization, of the data stru ture. In this ontext, then, it is appropriate to
worry not so mu h about the running time of a single operation as about the running time
of a sequen e of operations, in whi h an o asional lengthy operation may be overshadowed
by long sequen es of short operations. Let's all the running time of an operation its a tual
ost. In a worst ase analysis of an algorithm the a tual ost of an operation is repla ed by
its worst ase ost. In ontrast

amortized analysis aims at providing an amortized ost per operation in su h a way that at
any time the total amortized ost of the sequen e of operations performed so far
is not less than the a tual ost in urred.

In the following se tions we will explore several methods for omputing upper bounds
on the amortized ost of an operation. I emphasized the words \upper bound" be ause I
want you to be aware that that's all we're omputing. So do not be surprised, or worried,
if di erent methods give you di erent bounds. That is the advantage of having di erent
methods at your disposal: you pi k the one that gives you the best result - in this ase the
lowest upper bound. I will illustrate the methods on an example, the pw-sta k, whi h may
seem to be arti ial at rst glan e, but in a tuality appears, in various guises, in a number
of algorithms. Se tion 3.5 presents one su h guise.

3.2 A basi onstru t: the pop-while sta k


The pop-while sta k, for short pw-sta k (read pee-wee-sta k), augments the usual data stru -
ture \sta k" with the operation pop while. The interfa e is therefore:
interfa e PW Sta k f
publi void Push(Obje t x) - pushes obje t x on top of the sta k ;
publi Obje t Top() - returns the value of the top element of S;
publi Obje t Pop() - pops the top element o the sta k, and returns its value;
publi boolean IsEmpty() - returns true if the sta k is empty, false otherwise.
In addition there is the operation
publi Obje t Pop-While(b) - pops obje ts o the sta k, one by one, as long as
boolean variable b remains true.
Returns the value of the last element popped.
Running time proportional to number of elements popped.
Copyright 2001 by Avraham. A. Melkman 17

Whereas the standard sta k operations run in O(1) time, the pop-while operation an
on eivably pop the ontents of the entire sta k and has therefore a running time that is
proportional to the size of the sta k (at that time), in the worst ase.
Suppose now that the algorithm that employs the pw-sta k exe utes a sequen e of n
operations in all, onsisting of some mixture of the permissible operations. What then is the
total running time? A rude analysis goes as follows: if the sequen e ontains O(n) pushes
the sta k may ontain as many as O(n) obje ts; a single pop-while ould therefore take as
long as O(n). If the sequen e ontains k pop-whiles it looks like the running time ould be
as bad as O(kn), whi h for k = O(n) is O(n2 )! In a tuality, however, the running time is
mu h less.

Theorem 3.2.1 Any sequen e of n operations on the pw-sta k runs in time O(n).

The following two se tions give ea h a di erent proof of this theorem. Sin e the running
time is bounded by a linear fun tion of the number of individual pushes and pops that are
performed, we an, and will, simplify the analysis by assigning ea h a ost of one unit (a
pw-unit) and analyze the running time in terms of these abstra t osts.

3.3 The harge-now-pay-later method


The basis of the harge method is to re ord the ost of ea h individual operation, the harge,
and to sum up all re orded harges at the end of the sequen e. There are two ways in whi h
harges an be re orded.

1. The harge-per-operation method, whi h lists the harge for ea h operation separately,
mu h as it is on your redit ard statement. Let's see how this works with the pw-sta k.
Suppose there were k pushes ea h osting one pw-unit, and ` pops and pop-whiles of
whi h the i-th operation popped ni items (ni = 1 for a straight pop), and therefore
P P
osted ni pw-units. Then the total of the re orded harges is k + `i=1 ni . But `i=1 ni
k be ause no more items an be popped o the sta k then were pushed on. Thus the

total number of harges does not ex eed 2k. This proves Theorem 3.2.1 sin e k n. 
2. The distributed- harge method, in whi h the harge for ea h operation is distributed
among all, or some, of the elements that were a e ted by the operation. At the end of
the sequen e all harges atta hed to individual elements are olle ted. In the pw-sta k
example,
Pp
suppose p items are in use, and that item number i is pushed mi times, with
i=1 mi = k . Then the harges olle ted by that item do not ex eed 2mi , be ause the
item annot P be popped more often than it is pushed. Hen e the a umulated harges
are at most pi=1 2mi = 2k, the same bound as obtained by method 1.
18 CHAPTER 3. AMORTIZED ANALYSIS

3.4 The vou her method


In ontrast to the previous method, the vou her method attempts to derive an upper bound
on the ost of a sequen e by prepaying the osts of the operations. The idea is to allot ea h
type of operation a (possibly di erent) standard number of vou hers, its amortized ost.
For some operations this standard number of vou hers is more than what is needed to pay
for their ost; the remainder of the vou hers is then stored in appropriate pla es in the data
stru ture. For other operations the standard number of vou hers is not suÆ ient to pay for
their ost, and they are supposed to pi k up additional vou hers from data stru ture. For
this method to work, therefore, it must be shown that there are always a suÆ ient number
of vou hers to pay for ea h operation, no matter how ostly.
Let me illustrate, again, with the pw-sta k. Suppose ea h push is out tted with two
vou hers; one vou her is used to pay for the ost of the operation itself, the other an be
thought of as a vou her left with the obje t just stored (on top of the plate). The pop and
pop-while operations, on the other hand, are sent on their way with no vou hers at all!
The ost of these operations is payed for with vou hers that were left in the sta k. Namely,
ea h obje t to be popped arries with it the vou her left there by the push operation that
pla ed it on the sta k. This shows that, no matter how many obje ts are popped o by a
pop(-while) operation, there are suÆ ient vou hers to pay for its ost. We have shown,
therefore, that the a tual ost of ea h operation is ompletely overed by a ombination of
the vou hers given to the operation and the ones "stored" in the sta k. Consequently the
sum of the osts of the operations does not ex eed the sum of the vou hers provided to the
sequen e. The latter is simply twi e the number of pushes, 2k, the same bound as derived
in the previous se tion.

3.5 Interlude: an appli ation to nding the onvex hull


Given a set of points P in the plane, imagine that a nail is driven (perpendi ularly) into
ea h point. Now take a rubber band, stret h it so that it surrounds all nails, and let go.
The form that the rubber band takes on is a onvex polygon, C (P ), en losing a shape in the
plane that is alled the onvex hull of P . (Sin e the onvex hull is ompletely determined
by C (P ) the latter itself is often referred to as the onvex hull). Figure 3.5.1 is a bird's eye
view of su h a set of nails and the taut rubber band.
In this se tion we will look at an algorithm for nding C (P ), given P . Spe i ally, we
will nd the sequen e of its verti es as it is traversed in, say, ounter lo kwise order. In
gure 3.5.1 that would be, for example p2 ; p5 ; p1 ; p6 ; p9 . Note that we an start this sequen e
at any of the verti es of the onvex hull. To summarize, we want to solve the following
problem
 Instan e: A set = f
P p1 ; : : : pn g of points in the plane.
 Find: The verti es of ( ) in ounter lo kwise sequen e.
C P
Copyright 2001 by Avraham. A. Melkman 19

p1

p5
p3
p7
p6

p4

p8

p2

p9

Figure 3.5.1: A set of points, and its onvex hull

The algorithm proposed by R. Graham for solving this problem omprises two stages:
I. Renumber the points so that
1. p1 is the point with least x- oordinate (and in ase of ties, with least y - oordinate

among them); this point is guaranteed to be on the hull.


2. p2 ; : : : ; pn are sorted by polar angle with respe t to p1 in ounter lo kwise fashion.
In gure 3.5.2 you an see this renumbering for the points in gure 3.5.1. This stage
is not heap, sin e it involves a sorting of n obje ts, whi h takes O(n log n) ompar-
isons. Moreover, the angle omparison of two points needed in 2 appears to be rather
expensive, sin e (naively) it seems that one has to use the inverse tangent to ompute
these angles.
Let's take a moment to address this last issue, an important one for planar omputational geometry.
Noti e that the a tual values of the angles are irrelevant- we only want to label them a ording to
in reasing angle. Now, the angle that 3 makes with 1 is greater than the angle that 2 makes with
p p p

p1 if and only if 3 is to the left of 1 2 , the dire ted line from 1 to 2 . This an be done as follows:
p p p p p

ompute
1

1 1
x y

 = 1 2 2

x y
1
3 3
x y

Then  = the area of the triangle 1 2 3, whereas () 0 if and only if 3 is to the left of the
j j p p p sign > p

dire ted line 1 2.


p p

Finger exer ise: Prove that  is invariant under translation and rotation. We may therefore assume,
without loss of generality, that 1 = 0 and 2 = ( 2 0); what is the value of () in this ase?
p p x ; sign

Let's skip now to the se ond stage, be ause there appears what I'm really interested
in: a use of the pw-sta k. Observe that if one \ onne ts the dots" after stage I the
20 CHAPTER 3. AMORTIZED ANALYSIS

p9

p8

p1 p7

p6
p5

p3

p4

p2

Figure 3.5.2: Renumbering the points, and onne ting the dots

sequen e < p1 ; : : : ; pn ; p1 > des ribes a simple polygon; this polygon, however, is not
ne essarily onvex, see gure 3.5.2 (it is, however, a star-shaped polygon: one in whi h
one spe ial point, in this ase the pivot, an "see" the whole polygon). Whi h is why
we need the next stage. Observe also that p1 ; p2 , and pn are guaranteed to be verti es
of C (P ).

II. Exe ute the following "Graham S an":


Build the hull, by mar hing around the star-shaped polygon, adding edges when we
make a left turn, and ba k-tra king when we make a right turn.

push(p1 ); push(p2 ); push(p3 );


i=4;
while f
the sta k ontains the verti es of C( f g) in ounter lo kwise
g
p1 ; : : : ; pi 1
order, with pi on top and p1 on bottom

1
i n do
pop-while(pi not left of pt 1 pt );
push(pi );
i++;
endwhile.

Here pt represents the vertex at the top of the sta k, and pt 1 the one just below the
Copyright
2001 by Avraham. A. Melkman 21

top.

I will leave it to you to prove the orre tness of this s an, i.e., to prove the invariant
(guidan e: note that the angle of pi is greater than any that ame before, so that it ne essarily
is outside the onvex hull assembled so far). My point here is that the pop-while onstru t
does appear in a tual algorithms.

3.6 The potential method


3.7 A simple appli ation: dynami ve tors
3.8 Summary
Amortized analysis deals with the running time of sequen es of operations. When we say
that the amortized ost (running time) of an operation on an abstra t data stru ture is
O (log n) we mean that, even though the ost of an individual operation may be mu h larger

than O(log N ), any sequen e of m operations runs in a total time bounded by m log n. Even
if the sequen e was leverly arranged by a endish enemy to be the most diÆ ult to exe ute.
It is therefore a kind of worst- ase analysis - a worst sequen e s enario. In ontrast, when
we say that the worst- ase ost of an operation is O(log N ) we mean that the running time
of ea h and every operation is bounded by a onstant times log n. Unless the algorithm that
employs these operations is real-time, an amortized ost of O(log N ) is just as good as a
worst- ase ost of O(log N ), be ause they guarantee the same overall performan e.

3.9 Key on epts

You might also like