Understanding cover trees∗

Alcides Viamontes Esquivel

June 28, 2009

Imagine a ock of birds in the middle of the air. Without any external guidance, they
are able to self-assemble while they y. Imagine a train. It goes through a xed trail,
having no other destination than the one pre-xed by the trail itself. Think of walking
in a modern city, with well dened blocks, and of walking in ancient ruins spread in a
desert.
A cover tree is most useful if you have to pinpoint ruins in the desert, where you can't
give an address saying St John and 26th Avenue square. All what a cover tree needs is
distance from point to point. Auto-assembling does the rest, like in the ock of birds.
A cover tree is an arrangement of points, or better said, nodes spanned by points.
As the name suggests, it is a tree, meaning that there is a special node which acts as
the ancestor of all the others. The seminal paper [I] introduces them in terms of two
representatios: implicit and explicit. The rst one considers innite trees, which can
hardly be considered trees at all. But that viewpoint contributes to understandability,
which is why I'm going to stick to it at rst and then unfold it in practical implementation
details of the explicit representation.

∗
Online version at http://dignorsign.wordpress.com/2009/06/28/understanding-cover-trees/

Figure 1: Think of walking in a modern city, with well dened blocks, and of walking in ancient ruins
spread in a desert

1
 level 2

v=1.00 lev= 2

level 1

v=4.00 lev= 1 v=1.00 lev= 1

level 0

v=1.00 lev= 0 v=3.00 lev= 0 v=2.00 lev= 0

level -1

v=2.00 lev= -1

level -2

v=2.00 lev= -2

level -3

v=2.00 lev= -3 v=2.20 lev= -3

level -4

v=2.20 lev= -4

level -5

v=2.20 lev= -5 v=2.17 lev= -5

Figure 2: ...in a cover tree, the stored entities are nodes. Each node has a level. Each point in S is
represented in the cover tree by a (virtual) innite set of nodes which spans from a given
maximum level down to minus innite. I will call this innite set of nodes a chain, and the
starting node of it, with the maximum level, its head. Each chain has thus a unique element
of S associated to it. Also, each node has a parent, head nodes of a chain have as parent
nodes in other chains. There is a headless chain, the one of the root node; it spans from plus
innity down to minus innity....Cover trees made of points over the reals (v represents the
value). Dashed lines represent chains of nodes which come from plus innity (only for the
root chain) or which goes to minus innity (all the chains). Thin dotted lines represent links
between chains corresponding to distinct points.

The starting data for a cover tree is, as said before <put link>, a set of objects (which
will be called points from here on) S such that it is possible to assign a distance
1 to

each pair of points in S.

In the implicit representation of a cover tree, the stored entities are nodes. Each node
has a level. Each point in S is represented in the cover tree by a (virtual) innite set of
nodes which spans from a given maximum level down to minus innite. I will call this
innite set of nodes a chain, and the starting node of it, with the maximum level, its
head. Each chain has thus a unique element of S associated to it. Also, each node has a
parent, head nodes of a chain have as parent nodes in other chains. There is a headless
chain, the one of the root node; it spans from plus innity down to minus innity.
Nodes in a cover tree follow certain rules, which are enforced by the point insertion
and the point deletion algorithms. These rules are exploited by the search routines. Here
they are:

1. (nesting) The points on any level are also points of the level below. In other words,

1
This number should be assigned according to a valid metric

2
Figure 3: ...it also implies that the distance from a point p represented by a node np at level i to a
point q represented by a node nq which is descendant of np is lesser than 2i+1 ...

if a point is at level i then it is also at level i − 1.

2. (covering) Each node have a parent (in the self chain or in the parent chain). The
corresponding points are at most at distance 2i one of the other, where i is the
level of the parent node.

3. (separation) Any two nodes at the same level i are separated by a distance 2i or
greater.

Rule 1 makes the chains innite in the downward direction. Rule 2 makes the structure
to looks like a tree; it also implies that the distance from a point p represented by a node
np at level i to a point q represented by a node nq which is descendant of np is lesser
than 2i+1 . See gure 3 to understand this better. The third property, separation, is a
nice side eect which can be used for extracting evenly distributed subsets of S.
Note that the three previous properties does not make mention of anything but the
distance between elements of S. The work of Beygelzimer et al makes this explicit. There
are however certain properties of the set S that, without involving implementation, aect
the asymptotic complexity of construction and query: the intrinsic dimension of the
metric space.
There are multiple denitions for spatial intrinsic dimension . All of them orbit around
the following idea: suppose that there is a way of measuring a set, i.e., counting the
number of elements that it have if nite, or giving either its area or its volume otherwise.
Then, the intrinsic dimension characterizes, via logarithms, how fast the measure of a
sphere grows with its radius. For ordinary 2 and 3d Euclidean spaces, the area of a
4
circumference is πr2 and the volume of a sphere is
3
3 πr , so a certain intrinsic dimension
for these spaces might assign to them the numbers 2 and 3. Note, however, that most
denitions of intrinsic dimension does not assume the existence of a measure but instead
use clever tricks for getting an equivalent number in more general cases. Also, often
the goal is to measure the intrinsic dimension of certain subset, not the entire space.
Once in the explicit representation, the cover tree have a nice property: the ammount
of memory required is proportional to the number of elements it holds. The number of

3
operations (time) required for insertion and deletion of a single point is proportional to
the logarithm of the number of elements already present in the cover tree. The propor-
tionality constant multiplying log n is the combination of an implementaion dependent
parameter α (read: how good your computer and your coding is) and the intrinsic di-
mension c of the set S. More about that in a following post.