Design and Analysis of Algorithms

out: February 6, 2014

Problem Set 3
Lecturer: Daniel A. Spielman

due: 2:35 PM, February 13, 2014

Homework Policy
Collaboration is not allowed on these problems: you are expected to solve them on your
own.
You should cite any sources that you use on the problem sets other than the textbook,
TA and instructor. You must not search the web for solutions to similar problems given
out in other classes. If you think this policy needs any clarification, please let me know.

Turning in the problem set

The problem sets are due at the beginning of class. Please finish them before class
begins. If you bring your problem set to class, please separate it into pieces, one for
each problem. There will be separate boxes for each problem. Make sure that your name
appears on your solution to each problem, and that each problem lands in the correct
box.
Alternatively, you may submit the problem set electronically via the Classes server. If
you do so, please submit as a pdf. And, please put your name on every
problem, and put every problem on a separate page.
Late problem sets will not be accepted as solutions will be handed out at the class during
which they are due.

Do 3 of 4
While you should think about all 4 problems assigned, you should only turn in 3 of them.
Each problem is worth 10 points.

Problem Set 3: February 6, 2014

Corrections
I have made one significant correction to the problem set. In problem 4, I now ask you
to assume that S only contains two elements.

Problem 1: Nearest Neighbors, in General

There are many situation in which computing nearest neighbors becomes complicated.
Sometimes the distance between objects isnt even given by coordinates, but must be
computed! For example, consider the distance given by the number of inversions. In
these cases, we must often compare every object to every other object. If we are going
to do that much work, we should at least try to optimize the cache behavior.
Say that we have n objects, numbered 1 through n, that we need to compare all pairs
of them, and that we have some function, comp, that makes the comparisons. Lets not
worry for now about what we do with the results of these comparisons.
The most natural way to make all of these comparisons is with a loop of the following
form.
For i from 1 to n
For j from i + 1 to n
comp (i,j)
However, this loop is horrible from a cache perspective. For those who dont know much
about caches, the important thing to know is that modern computers have many levels of
memory, with the faster levels holding less. This means that is takes less time to access
a recently accessed memory item. In the loops above, item i will have been recently
accessed every time it is needed, except the first. However, item j will have last been
used long in the past.
In this problem, you will design a loop that has better cache performance. We will use a
crude model of cache behavior: we will assume that the cost of retrieving an item from
memory is the logarithm of how long ago we last used it. We will measure how long
ago by number of comparisons. If an item was used in the previous comparison, then
it is still in a register and the cost of using it is free. This agrees with log(1) = 0. In
general, the cost of retrieving items i and j to perform comp(i, j) will be the logarithm
of the number of comparisons ago that i was used, plus the same term for j. The first
time that an item is used, we assume that the cost of using it is log n. For example, in
the above loop with n = 3, we have comparisons and costs as follows:

Problem Set 3: February 6, 2014

comparison
(1,2)
(1,3)
(2,3)

cost of i cost of j
log 3
log 3
0
log 3
log 2
0

I will define the memory cost of an loop to be the sum of all of these logarithmic terms.
You can see that the memory cost of the naive loop is some constant times n2 log n. You
may assume that n is a power of 2.
a. (5 points) Give a loop that reduces the memory cost to O(n2 ).
b. (5 points) Prove that it does this.
If you cant get all the way down to cost O(n2 ), you will receive half credit if you can
get it down to O(n2 log(log n)) (and if you can explain why).
Hint: You can actually be rather wasteful in this problem, as long as the memory cost
is O(n2 ). For example, you could execute both comp(i, j) and comp(j, i) for all i and j.
You will loose a factor of 2, but can produce cleaner code. In my analysis, I found the
following inequality to be useful. You can use it to
X i
= 2.
i
2
i1

Problem 2: Computing a strange function

The following is a problem that I once had to solve. I wont try to motivate it for you.
The input is a vector of n numbers, x0 , . . . , xn1 , where n is a power of 2. However, we
are going to index them differently. As n is a power of 2, let k = log2 n. So, the indices 0
through n 1 can all be written as base-2 numbers that are k digits long. Alternatively,
we can just record the set of positions that have a 1 in the base-2 representation. For
example, here are some translations with k = 4.
index i base-2 representation
0
0000
1
0001
12
1100
1111
15

set (numbering positions from right to left)

{1}
{3, 4}
{1, 2, 3, 4}

Problem Set 3: February 6, 2014

So, I will think of the elements of the vector as being indexed by the subsets of {1, . . . , k}.
This way, if I write xS , I mean xi where
X
i=
2j1 .
jS

Heres the problem. Compute

X

xS xT .

S{1,...,k} T {1,...,k}:ST =

That is, we compute the product of xS and xT over all pairs of sets S and T that are
disjoint.
Design an algorithm that computes this product in time O(n log2 n). Explain why
your algorithm is correct and why it runs in the stated time.
You should definitely assume that n is a power of 2.
Hint: The most natural way to divide the sets in half is to separately consider those
that do and do not contain element k.

Problem 3: Where paths meet

Trees appear in a huge number of algorithms, and so we have developed many ways of
manipulating them. In this problem, we will consider trees with a root. Given two nodes,
a and b, it is useful to be able to compute their meet: the node that is closest to the root
on the path between them. This can also be defined as the first node where the paths
from a and b to the root meet.

Problem Set 3: February 6, 2014

The naive way of computing the meet of a and b is to just walk from each of a and b to
the root, and see where the paths first meet. This is fast if the tree has low depth. But,
not all trees have low depth (for example, consider a long path).
One way to speed up computations in trees is to create a list of pointers at each node
that point to nodes that are closer to the root. For example, for every node a, we define
parent(a) to be the node adjacent to a that is closer to the root. Imagine that we have
computed the parent of a for every node a. We can then compute the grandparent of a
by the following loop:
For every node a, grandparent(a) = parent(parent(a)).
So that this process is well-behaved, we define the parent of the root to be the root.
Equivalent, we say that the root is its own parent. We can then compute grandparents
of grandparents, and so on. I will call these hyperparents.
Given a tree of depth d (the depth is the longest distance of a node from the root), we
compute all of these, as follows.
1. For every node a, set hyperparent(a, 1) = parent(a).
2. For i from 2 to dlog2 de,
a. For every node a, set
hyperparent(a, i) = hyperparent(hyperparent(a, i 1), i 1).
We can compute all these hyperparents in time O(n log2 d).
Your job is to figure out how to use them. Given access to the tree and all these hyperparents, your job is to design an algorithm that on input nodes a and b will compute the
meet of a and b.
Design an algorithm that computes the meet of a and b in time O(log2 d). Explain
why your algorithm is correct and runs in the stated time.
You will get half credit for an algorithm that runs in time O((log2 d)2 ).
Note: You do not need to look at the whole input. Really, your algorithm should just
query the data structure. So, it should just ask for hyperparents. You may assume that
your algorithm knows d.

Problem Set 3: February 6, 2014

Problem 4: Simplifying Grid Graphs

One of the simplest graphs is the grid graph. The n-by-n grid graph has n2 vertices,
labeled by pairs (i, j) with 1 i n and 1 j n. The edges appear between adjacent
vertices. That is the edges are




(i, j), (i + 1, j) : for 1 i < n and 1 j n , and




(i, j), (i, j + 1) : for 1 i n and 1 j < n .

Grid graphs arise in many (mostly physical) applications. However, they usually have
some extra information that makes them more interesting. In our case, we will assume
that every edge can have a different (positive) length. That is, we are given as input a
grid graph G = (V, E, l), where l assigns a length to every edge.
As in problem 3 from the last problem set, we are only interested in the distances between
pairs of vertices in a set S V . So, we are going to remove all of the vertices that are
not in S. You should assume that S only has two elements. In the last problem
set, we proved that the set of edges in the resulting graph does not depend on the order
in which we remove vertices. However, the time required by the removal process does
depend on this order.
The time required to remove a vertex v is proportional to the square of its degree at the
time it is removed. Note that this degree can change during the course of the algorithm.
As the graph has n2 nodes, no node will ever have degree more than n2 . So, the time
required for all the removals will be at most O(n6 ). I would like you to design a faster
algorithm.
Describe an ordering in which to remove vertices so that the total time to perform the
removals is at most O(n3 ). Prove that your answer is correct. You may assume that
n is a power of 2 if you prefer. In fact, you are free to give an algorithm that makes
any assumption you like about n, as long as it works for an infinite sequence of n. My
solution considers n = 2k 1, for integers k.

Problem Set 3: February 6, 2014

Hint 1: In my solution to this problem, S is essentially irrelevant except that I need to

remember not to remove vertices in S, and that S only has two elements.
Hint 2: You should solve this by a divide-and-conquer strategy.