CSCE 310: Transform and Conquer Algorithms

September 22, 2016

Ryan Patrick

Outline
Transform & Conquer
Instance Simplification
Gaussian Elimination
Balanced Search Trees
Rotating a Binary Search Tree
2-3 Trees

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

Insertion

Rotations
2-3 Trees
Insertion

Heaps & Heapsort

Heaps
Heapsort

Heaps/Heapsort
Heaps
Heapsort

39

Outline
Transform & Conquer
Instance Simplification
Gaussian Elimination
Balanced Search Trees
Rotating a Binary Search Tree
2-3 Trees

Transform/Conquer
Patrick
3

Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

Insertion

Rotations
2-3 Trees
Insertion

Heaps & Heapsort

Heaps
Heapsort

Heaps/Heapsort
Heaps
Heapsort

39

Transform & Conquer

Transform/Conquer
Patrick
4

Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Divide/Decrease and Conquer Kept Structure, but Reduced Size

Balanced Search
Trees

Transform and Conquer Modifies Structure, but Keeps Size

Rotations
2-3 Trees
Insertion

Transform Structure Easier to Solve

Heaps/Heapsort
Heaps
Heapsort

39

Calculating an Average in MATLAB

Transform/Conquer

Arithmetic Mean

Patrick
5

Gaussian
Elimination

i=1

Balanced Search
Trees

Weighted Arithmetic Mean

Rotations
2-3 Trees
Insertion

x =

N
X

Heaps/Heapsort

wi xi

Heaps
Heapsort

i=1

where

Transform/Conquer
Instance
Simplification

N
1 X
xi
x =
N

N
X

wi = 1

i=1
39

Calculating an Average in MATLAB

Example
Transform/Conquer
Patrick
6

Transform/Conquer
Instance
Simplification

300 Students

Gaussian
Elimination

5 Assignments (each worth 10% of final grade)

Balanced Search
Trees

2 Midterm Exams (each worth 10% of final grade)

Rotations
2-3 Trees
Insertion

1 Final Exam (worth 20% of final grade)

Heaps/Heapsort

5 Quizzes (each worth 2% of final grade)

Heaps
Heapsort

Loops are slow in MATLAB

Matrix multiplications are fast in MATLAB

39

Example Solution
Transform/Conquer
Patrick
7

A1,1 A1,2
A2,1 A2,2

.
..
..
.
A300,1 A300,2

. . . A1,5 Q1,,1 Q1,2

. . . A2,5 Q2,,1 Q2,2
.
..
..
..
. ..
.
.
. . . A300,5 Q300,,1 Q300,2

. . . Q1,5 M1,1 M1,2 F1

. . . Q2,5 M2,1 M2,2 F2

..
..
..
..
..
.
.
.
.
.
. . . Q300,5 M300,1 M300,2 F300

w A1
wA
2
.
.
.

w A5

wQ 1

wQ 2

..
.

wQ
5
wM

1
wM
2
wF

Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Balanced Search
Trees

x1
x2

..
.
x300

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Example Solution
Transform/Conquer
Patrick
8

A1,1 A1,2
A2,1 A2,2

.
..
..
.
A300,1 A300,2

. . . A1,5 Q1,,1 Q1,2

. . . A2,5 Q2,,1 Q2,2
.
..
..
..
. ..
.
.
. . . A300,5 Q300,,1 Q300,2

. . . Q1,5 M1,1 M1,2 F1

. . . Q2,5 M2,1 M2,2 F2

..
..
..
..
..
.
.
.
.
.
. . . Q300,5 M300,1 M300,2 F300

0.1
0.1

.
.
.

0.1

0.02

0.02

..
.

0.02

0.1

0.1
0.2

Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Balanced Search
Trees

x1
x2

..
.
x300

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Types of Transformations
Transform/Conquer
Patrick
9

Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Instance Simplification: Simpler (More Convenient) Instance of Same

Problem

Balanced Search
Trees
Rotations
2-3 Trees
Insertion

Representation Change: Different Representation of Same Instance

Problem Reduction: Instance of a Different Problem (with Known
Algorithm)

Heaps/Heapsort
Heaps
Heapsort

39

Instance Simplification


Question: If a problem can be solved as is in O n2 time, but simplified in O (n log n) time and
the simplified version can be solved in O (n) time, is the simplification worth it?
Transform/Conquer

A. Yes
B. No

Patrick
Transform/Conquer
10

Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees
Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Presorting
Transform/Conquer
Patrick
Transform/Conquer
11

Instance
Simplification

Gaussian
Elimination

Basic Instance Simplification

Balanced Search
Trees

Sort in (n log n) Time

Rotations
2-3 Trees
Insertion

Solve in (n log n) Time?

Heaps/Heapsort
Heaps
Heapsort

39

Presorting
Examples
Transform/Conquer
Patrick
Transform/Conquer
12

Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

Duplicate Elements

Rotations
2-3 Trees
Insertion

Mode

Heaps/Heapsort
Heaps
Heapsort

39

Outline
Transform & Conquer
Instance Simplification
Gaussian Elimination
Balanced Search Trees
Rotating a Binary Search Tree
2-3 Trees

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification
13

Gaussian
Elimination
Balanced Search
Trees

Insertion

Rotations
2-3 Trees
Insertion

Heaps & Heapsort

Heaps
Heapsort

Heaps/Heapsort
Heaps
Heapsort

39

Gaussian Elimination
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification
14

Initial Problem: n Equations, n Unknowns

Gaussian
Elimination
Balanced Search
Trees

LU Decomposition

Rotations
2-3 Trees
Insertion

Matrix Inverse

Heaps/Heapsort

Determinant

Heaps
Heapsort

39

Outline
Transform & Conquer
Instance Simplification
Gaussian Elimination
Balanced Search Trees
Rotating a Binary Search Tree
2-3 Trees

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
15

Insertion

Balanced Search
Trees
Rotations
2-3 Trees
Insertion

Heaps & Heapsort

Heaps
Heapsort

Heaps/Heapsort
Heaps
Heapsort

39

Balanced Search Trees

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

AVL Tree: Binary Search Tree (BST) with Subtree Differences 1

16

Balanced Search
Trees
Rotations
2-3 Trees
Insertion

Adelson-Velsky and Landis

Heaps/Heapsort
Heaps
Heapsort

39

Keeping Binary Trees Balanced

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
17

What do we do when an AVL Tree becomes unbalanced?

Balanced Search
Trees
Rotations
2-3 Trees
Insertion

How do we do that?

Heaps/Heapsort
Heaps
Heapsort

39

Example Trees
Inserting the Values 1, 2, and 3
Transform/Conquer
Patrick

Input: 1, 2, 3
1

Transform/Conquer

Input: 2, 1, 3 or 2, 3, 1

Instance
Simplification

Gaussian
Elimination

Balanced Search
Trees

2
18

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

Rotating the BST on the left can result in the BST on the right.

39

Need for Rotations

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Efficiency Requires Balance

Order of Insertions/Deletions Affects Balance

Balanced Search
Trees
19

Rotating a Binary Search Tree Can Restore Balance

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Rotating a BST
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Rotation replaces a node with one of its child nodes . . . and updates the
subtrees accordingly

Balanced Search
Trees
20

The direction that the parent node moves (left/right) describes the type of
rotation performed

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Example: Left Rotation

Transform/Conquer
Patrick
Transform/Conquer
1

Instance
Simplification

Gaussian
Elimination

Balanced Search
Trees
21

1. Current roots right child ( current root) becomes new root

2. Old root (and its left children) become new roots left child ( new root)

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

3. New roots left children become old roots right children ( old root)
Right Rotation is Mirror of Left Rotation

39

Rotating a BST
Double Rotation
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Single Rotations Dont Always Balance Trees

Left-Right/Right-Left Rotations Fix Problem

Balanced Search
Trees
22

Left-Right: Left Rotate Left Subtree, Right Rotate Entire Resulting Tree

Rotations
2-3 Trees
Insertion

Heaps/Heapsort

Right-Left: Right Rotate Right Subtree, Left Rotate Entire Resulting Tree

Heaps
Heapsort

39

Rotating a BST
Double Rotation Example
Transform/Conquer

Insert 38, 14, 32, 19 into AVL+1 Tree

Delete 19

Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees
23

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

2-3 Trees
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

2-nodes and 3-nodes

Balanced Search
Trees

2-nodes are Binary Nodes (Left Children <, Right Children >)
3-nodes have 2 Keys: (Left Children < Key1 , Right Children > Key2 ,
Middle Children > Key1 and < Key2 )

24

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

All Leaves Must be on Same Level

39

Insertion into a 2-3 Tree

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Find Position in Tree

Balanced Search
Trees

If Position is 2-node: Make New Value a Key in 3-node

If Position is 3-node: . . .

25

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Insertion into a 2-3 Tree

Insertion at a 3-node
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

1. Smallest Current/Candidate Key into Left Child

Balanced Search
Trees

2. Largest Current/Candidate Key into Right Child

3. Middle Current/Candidate Key into Parent. . .

26

Rotations
2-3 Trees
Insertion

Heaps/Heapsort

4. If Parent was a 3-node, Repeat Step 1 on Parent

Heaps
Heapsort

39

Example
Figure 6.8, Page 225
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

27

Rotations
2-3 Trees
Insertion

Heaps/Heapsort
Heaps
Heapsort

39

Outline
Transform & Conquer
Instance Simplification
Gaussian Elimination
Balanced Search Trees
Rotating a Binary Search Tree
2-3 Trees

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

Insertion

Heaps & Heapsort

Heaps
Heapsort

Rotations
2-3 Trees
Insertion
28

Heaps/Heapsort
Heaps
Heapsort

39

Need for Heaps

Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Priority Queue

Balanced Search
Trees

Branch & Bound

Rotations
2-3 Trees
Insertion

Job Scheduling
Any Task Where Order is Important

Heaps/Heapsort
29

39

Heaps
Heapsort

Heap
Definition
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Binary Tree

Gaussian
Elimination

One Key per Node

Balanced Search
Trees
Rotations
2-3 Trees
Insertion

Shape Property: Essentially Complete Tree (Some Rightmost Leaves

Allowed at Last Level)
Parental Dominance: KeyParent >KeyChildren

Heaps/Heapsort
30

39

Heaps
Heapsort

Heap
Properties
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

Key Values Ordered Top-Down

Rotations
2-3 Trees
Insertion

No Left-to-Right Order

Heaps/Heapsort
31

39

Heaps
Heapsort

Heap
Construction: Bottom-Up
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Insert Keys Top-Down

Balanced Search
Trees

Enforce Rules Bottom-Up

Rotations
2-3 Trees
Insertion

Efficiency?

Heaps/Heapsort
32

39

Heaps
Heapsort

Heap
Construction: Top-Down
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

Proper Insertion of Every Key into Structured Heap

Balanced Search
Trees

1. Place New Node after Last Leaf

2. Move New Node into Correct Position

Rotations
2-3 Trees
Insertion

Efficiency?

Heaps/Heapsort
33

39

Heaps
Heapsort

Heap
Node Deletion
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination

1. Swap Node with Last Node in Tree

Balanced Search
Trees

2. Remove Node

Rotations
2-3 Trees
Insertion

3. Heapify Tree

Heaps/Heapsort
34

39

Heaps
Heapsort

Heapsort
Algorithm
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

1. Construct Heap from Array

Rotations
2-3 Trees
Insertion

2. Delete Root Node n 1 Times

Heaps/Heapsort
35

39

Heaps
Heapsort

Heapsort
Special Properties
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees

In Place

Rotations
2-3 Trees
Insertion

(n log n) in Average and Worst Case

Heaps/Heapsort
36

39

Heaps
Heapsort

Heapsort
Example: Figure 6.14, Page 232
Transform/Conquer
Patrick

For array elements 1 to n, element i is greater than elements 2i and 2i + 1.

Transform/Conquer
Instance
Simplification

29

Gaussian
Elimination
Balanced Search
Trees

96

Rotations
2-3 Trees
Insertion

78

Heaps/Heapsort
37

62

..
.
9

87

7
39

Heaps
Heapsort

Heapsort
Example: Figure 6.14, Page 232 (Continued)
Transform/Conquer

Delete root node n 1 times.

Patrick

978526

Transform/Conquer
Instance
Simplification

Gaussian
Elimination

9
7
8
5
7
2
6
5
2
9

6
6
6
6
6
6
2
2

8
8
7
7
5
7
5

2
2
2
2
2

5
5
5

622

Balanced Search
Trees

8755

Rotations
2-3 Trees
Insertion

Heaps/Heapsort

38

39

Heaps
Heapsort

Any Questions?
Transform/Conquer
Patrick
Transform/Conquer
Instance
Simplification

Gaussian
Elimination
Balanced Search
Trees
Rotations
2-3 Trees
Insertion

Heaps/Heapsort
39

39

Heaps
Heapsort