Integer

Programming
(Pure & Binary)
Taufik Djatna, Dr.Eng.
 Integer programming ::
Methods
Relaxation
Branch and Bound
Cutting planes
Column generation
Heuristics

Close the
Min Max Theorem gap!
 Integer linear programming
Input:
A linear objective function f: Zn Z
A set of linear constraint functions: gi: Zn Z
A set of constraint values: bi

Goal:
Find x in Zn which:
1. maximizes f(x)
2. satisfies gi(x) <= bi
 LIP Area of Application
Binary Integer Model:: Plant Expansion decision

Z Max(total Net Present Value= 9x1 5 x2 6 x3 4 x4 )

where
1 if decision j is YES
xj j 1, 2,3, 4
0 if decision j is NO
with subject to
6 x1 3 x2 5 x3 2 x4 10
x3 + x 4 1
x1 +x 3 0
xj
 LIP Area of Application
Investment Analysis/Capital Budgeting
Site Selection
Designing a Production and Distribution
Network
Dispatching Shipments
Scheduling Interrelated Activities
Airline applicationFleet Assignment
problem
 Branch-and-
Bound Technique
for Solving
Integer Problems
 Basic Concepts
The basic concept underlying the branch-and-bound technique is to
divide and conquer.
Since the original large problem is hard to solve directly, it is
divided into smaller and smaller sub problems until these sub
problems can be conquered.
The dividing (branching) is done by partitioning
the entire set of feasible solutions into smaller and smaller
subsets.
The conquering (fathoming) is done partially by
(i) giving a bound for the best solution in the subset;
(ii) discarding the subset if the bound indicates that it cant
contain an optimal solution.
These three basic steps branching, bounding, and fathoming are
illustrated on the following example.
x1 Example of Branch-and-Bound
6
x1 + x2 = 6 Max Z = 5x1 + 8x2
5 s.t. x1 + x2 6
5x1 + 9x2 45
4 (2.25, 3.75) x1 , x2 0 integer

Z=41.25
2 5x1 + 9x2 = 45

1
Z=20

0 1 2 3 4 5 6 7 8 9
x2
 Utilizing the information about the
optimal solution of the LP-relaxation
Fact: If LP-relaxation has integral optimal solution x*,
then x* is optimal for IP too.
In our case, (x1, x2) = (2.25, 3.75) is the optimal solution of the LP-
relaxation. But, unfortunately, it is not integral.

The optimal value is 41.25 .

Fact:
OPT(LP-relaxation) OPT(IP)
(for maximization problems)
That is, the optimal value of the LP-relaxation
is an upper bound for the optimal value of the integer
program.In our case, 41.25 is an upper bound for OPT(IP).
 Branching step
In an attempt to find out more about the location of the IPs
optimal solution, partition the feasible region of the LP-
relaxation.
Choose a variable that is fractional in the optimal solution to the
LP-relaxation say, x2 . Observe that every feasible IP point
must have either x2 3 or x2 4 .
With this in mind, branch on the variable x2 to create the
following two subproblems:
Subproblem 1 Subproblem 2
Max Z = 5x1 + 8x2 Max Z = 5x1 + 8x2
s.t. x1 + x2 6 s.t. x1 + x2 6
5x1 + 9x2 45 5x1 + 9x2 45
x2 3 x2 4
x1 , x2 0 x1 , x2 0
Solve both subproblems
(note that the original optimal solution (2.25, 3.75) cant recur)
 Branching step (graphically)
Z=41
Subproblem 1: Opt. solution (3,3) with value 39
5 Subproblem 2: Opt. solution (1.8,4) with value 41
Subproblem 2
4
(1.8, 4)

3 (3, 3)

2
Subproblem 1

1
Z=20 Z=39

0 1 2 3 4 5 6 7 8
 Solution tree
S1: x2 3
For each subproblem, we record
All
(3, 3) int.
the restriction that creates the subproblem Z=39
(2.25, 3.75)
the optimal LP solution
Z=41.25 S2: x2 4
the LP optimum value
(1.8, 4)
Z=41
The optimal solution for Subproblem 1 is integral: (3, 3).
If further branching on a subproblem will yield no useful
information, then we can fathom (dismiss) the subproblem.
In our case, we can fathom Subproblem 1 because its solution is integral.
The best integer solution found so far is stored as incumbent.
The value of the incumbent is denoted by Z*.
In our case, the first incumbent is (3, 3), and Z*=39.
Z* is a lower bound for OPT(IP): OPT(IP) Z* .
In our case, OPT(IP) 39. The upper bound is 41: OPT(IP) 41.
Next branching step (graphically)
- Fathom Subproblem 1.
5 - Branch Subproblem 2 on x1 :
(1, 4.44)
Subpr. 4
Subpr. 3 Subproblem 3: New restriction is x1 1.
4 Opt. solution (1, 4.44) with value 40.55
Subproblem 4: New restriction is x1 2.
The subproblem is infeasible
3
Z=40.55

1
Z=20

0 1 2 3 4 5 6 7 8
 Solution tree (cont.)
S1: x2 3
int.
All (3, 3) S3: x1 1
(2.25, 3.75) Z=39
(1, 4.44)
Z=41.25
S2: x2 4 Z=40.55

(1.8, 4)
Z=41 S4: x1 2
infeasible

If a subproblem is infeasible, then it is fathomed.

In our case, Subproblem 4 is infeasible; fathom it.

The upper bound for OPT(IP) is updated: 39 OPT(IP) 40.55 .

Next branch Subproblem 3 on x2 .

(Note that the branching variable might recur).
Next branching step (graphically)
(0, 5) Branch Subproblem 3 on x2 :
5 Subproblem 5: New restriction is x2 4.
Feasible region:
4 the segment joining (0,4) and (1,4)
(1, 4) Opt. solution (1, 4) with value 37
Subproblem 6: New restriction is x2 5.
3 Feasible region is just one point: (0, 5)
Opt. solution (0, 5) with value 40
2

1
Z=20

0 1 2 3 4 5 6 7 8
 Solution tree (final)
S1: x2 3 S5: x2 4
int.
All (3, 3) S3: x1 1 (1, 4) int.
(2.25, 3.75) Z=39 Z=37
(1, 4.44)
Z=41.25
S2: x2 4 Z=40.55 S6: x2 5
(1.8, 4) (0, 5)
int.
Z=41 S4: x1 2 Z=40
infeasible
If the optimal value of a subproblem is Z*, then it is fathomed.
In our case, Subproblem 5 is fathomed because 37 39 = Z*.
If a subproblem has integral optimal solution x*,
and its value > Z*, then x* replaces the current incumbent.
In our case, Subproblem 5 has integral optimal solution, and its value
40>39=Z*. Thus, (0,5) is the new incumbent, and new Z*=40.
If there are no unfathomed subproblems left, then the current
incumbent is an optimal solution for (IP).
In our case, (0, 5) is an optimal solution with optimal value 40.
 y objective

optimum of
LP relaxation

IP optimum
feasible rounding down optimum
solutions = of LP relaxation

x
Integer Program
Solving MIPs: Branch and Bound
Consider the following integer program:
Maximize x + y + 2z
Subject to 7x + 2y + 3z 36 (IP0)
5x + 4y + 7z 42
2x + 3y + 5z 28
x, y, z 0, integer
 Branch & Bound: Example
3
x 1
11
obj 11
5 IP0
y 0 11
z 5 1
11
x 1 x2
Best IP
value

/ 10/ 11
1
0 obj 11.4 obj 10 2 1
5.2 IP2 IP1 2 0 0
0 Best IP
4 4 5
solution
z 5 z6 Integral
1
1 1
obj 11 IP4 IP3
3 3
5
y0 y 1 Infeasible

1 obj 11 obj 11.2 1

0 IP6 IP5 1
5 4.6

Integral z4 z 5 z 5

1 obj 9 IP8 IP7 obj 11 0

2 1
5
4

Obj. Obj.
incumbent incumbent
Diet Problem

The Pill Salesman

A pill salesman offers Bob energy, protein, and

calcium pills to fulfill his nutritional needs. We
will represent the costs of each of the pills as
follows:
y1 cost (in dollars) of 1 kcal energy pill
y2 cost (in dollars) of 1 g protein pill
y3 cost (in dollars) of 1mg calcium pill
Diet Problem

LP constraints

Nutritional requirements

110x1 + 205x2 + 160x3 + 160x4 + 420x5 + 260x6 2000 y1

4x1 + 32x2 + 13x3 + 8x4 + 4x5 + 14x6 55 y2
2x1 + 12x2 + 54x3 + 285x4 + 22x5 + 80x6 800 y3
Diet Problem

LP constraints
Minimize Cost
0.3x1 + 2.40x2 + 1.30x3 + 0.90x4 + 2.0x5 + 1.9x6

Nutritional requirements

110x1 + 205x2 + 160x3 + 160x4 + 420x5 + 260x6 2000 y1 kcal

4x1 + 32x2 + 13x3 + 8x4 + 4x5 + 14x6 55 y2 protein
2x1 + 12x2 + 54x3 + 285x4 + 22x5 + 80x6 800 y3 calcium

x1 = servings of oatmeal: The cost of the nutrients in one

serving of oatmeal shouldnt exceed the cost of just buying
one serving of oatmeal:
110y1 + 4y2 + 2y3 0.3 (4 y2 = cost of protein in oatmeal)
Diet Problem

The salesmans requirements

The pill salesman wants to make as much money as possible,
given Bobs constraints. He knows Bob wants 2000 kcal, 55g
protein, and 800 mg calcium, so his problem is as follows:

Maximize 2000y1 + 55y2 + 800y3

Subject to 110y1 + 4y2 + 2y3 0.3
205y1 + 32y2 + 12y3 2.4
160y1 + 13y2 + 54y3 1.3
160y1 + 8y2 + 285y3 0.9
420y1 + 4y2 + 22y3 2.0
260y1 + 14y2 + 80y3 1.9
y1 , y2 , y3 0
Diet Problem

Solution

Solving this LP gives us the following pill

prices:

$0.27 for 1 kcal energy pill

$0.00 for 1 g protein pill
$0.16 for 1mg calcium pill

Total cost = 0.27 (2000) + 0.16 (800) = $6.71

THE SAME AS THE LOWEST COST DIET!
Diet Problem

Weak Duality:
Cost of any feas. diet Price of any feas. pills

Strong Duality: (Von Neumann, 1947)

Optimal cost of diet = Optimal cost of pills
Diet Problem

MORAL:
The values of the dual variables, which are
always available as part of LP solution, equal
the marginal value of corresponding
resources.
 Summary
Summary of branch-and-bound for integer programs
Which variable to branch on?
Which open sub problem to solve first?
Updating the bounds (lower and upper) on OPT(IP)
Summary of fathoming criteria
Optimality test
Branch-and-bound applied to
binary integer programs
mixed integer programs
Mixed integer 0-1 programs
 Types of integer programs
All integer programs have linear equalities
and inequalities and some or all of the
variables are required to be integer.
If all variables are required to be integer, then it
is usually called a pure integer program.
If all variables are required to be 0 or 1, it is
called a binary integer program, or a 0-1 integer
program.
if some variables can be fractional and others
are required to be integers, it is called a mixed
linear integer program (MILP) 36
VARIOUS CASES
IN INTEGER
PROGRAMMING
 Example
Stockco is considering 6 investments. The
cash required from each investment as well as
the NPV of the investment is given next. The
cash available for the investments is $14,000.
Stockco wants to maximize its NPV. What is
the optimal strategy?

An investment can be selected or not. One

cannot select a fraction of an investment.
38
 Data for the Stockco Problem
Investment budget = $14,000

Investment 1 2 3 4 5 6

Cash
Required $5 $7 $4 $3 $4 $6
(1000s)
NPV
added $16 $22 $12 $8 $11 $19
(1000s)

39
 Integer Programming
Formulation
What are the decision variables?
1, if we invest in i 1,...,6,
xi
0, else

Objective and Constraints?

Max 16x1+ 22x2+ 12x3+ 8x4+ 11x5+ 19x6

5x1+ 7x2+ 4x3+ 3x4+ 4x5+ 6x6 14

xj e {0,1} for each j = 1 to 6 40
 Possible constraints in integer
programs
The previous constraints represent
economic indivisibilities, either a project
is selected, or it is not. There is no selecting
of a fraction of a project.
Similarly, integer variables can model
logical requirements (e.g., if stock 2 is
selected, then so is stock 1.)

41
 How to model logical constraints

Exactly 3 stocks are selected.

If stock 2 is selected, then so is stock 1.
If stock 1 is selected, then stock 3 is not
selected.
Either stock 4 is selected or stock 5 is selected,
but not both.

42
 Formulating Constraints
Exactly 3 stocks are selected

x1+ x2+ x3+ x4+ x5+ x6=3

43
If stock 2 is selected then so is stock 1
The integer programming
constraint:
A 2-dimensional
representation

x1 x2
Stock 2

Work with your

partner for 5 minutes
Stock 1 trying to model the
other constraints. 44
If stock 1 is selected then stock 3
is not selected
The integer programming
constraint:
A 2-dimensional
representation

x1 + x3 1
Stock 3

Stock 1
45
Either stock 4 is selected or stock
5 is selected, but not both.
The integer programming
A 2-dimensional constraint:
representation

x4 + x5 = 1
stock 5

stock 4
46
 Representing Non-linear functions
Suppose that the cost of computers is as
follows:
$2,000 each if you buy 1 to 10
$1,800 for each computer from 11 to 25 s.t. 0 x1 10
$1,700 for each computer from 26 to 50 0 x2 15
No more than 50 may be purchased.
0 x3 25
Let x = number of computers bought
x1 = number bought at $2,000 If x1 < 10,
x2 = number bought at $1,800 then x2 = 0.
x3 = number bought at $1,700 If x2 < 15,
then x3 = 0.
Minimize 2,000 x1 +1,800 x2 + 1,700 x3
47
Representing Non-linear functions

Minimize 2,000 x1 +1,800 x2 + 1,700 x3

s.t. 0 x1 10
0 x2 15 w1 Let w1 = 1 if x1 = 10
0 x3 25 w2 w1 = 0 if x1 < 10
If x1 < 10, then x2 = 0.
0 w1 x1/10 Let w2 = 1 if x2 = 15
If x2 < 15, then x3 = 0.
0 w2 x2/15 w2 = 0 if x2 < 15

w1, w2 {0, 1}

If x1 < 10, then x2 = 0. If x2 < 15, then x3 = 0.

If w1 = 0, then x2 = 0. If w2 = 0, then x3 = 0. 48
 Is this formulation correct?
It models the xs correctly.
It does not model the ws
perfectly.
Suppose x1 = 10, x2 = x3 = 0;
Then w1 = 0 or 1
Cost = 20,000, regardless.
49
 Fire Station Problem

Locate fire stations

1 2 3 so that each district
has a fire station in
it, or next to it.
5 6
7
4 8 9 Minimize the
number of fire
stations needed.
11 12 13
10
14 15 16
50
 Representation as Set Covering
Problem
Set Covers
1 2 3
1 1, 2, 4, 5

5 6 2 1, 2, 3, 5, 6
7
3 2, 3, 6, 7
4 8 9

11 12 13
10
14 15 16 16 13, 15, 16
51
 Representation as Graph Cover
Problem
Replace each district with
1 2 3 a node.

Two nodes are adjacent if

5 6 their districts are adjacent
7
4 8 9 A node covers itself and its
neighbors. Thus, node 16
covers nodes 13, 15, 16.
11 12 13
10 What is the minimum size
14 15 16
16 of a subset of nodes that
covers all of the nodes?52
Representation as Integer program
xj = 1 if node j is selected
xj = 0 otherwise
11 2 3
Minimize x1 + x2 + + x16
5 6
7 s.t. x1 + x2 + x4 + x5 1
4 8 9 x1 + x2 + x3 + x5 + x6 1

11 12 13
10
14 15 16 x13 + x15 + x16 1

xj {0, 1} for each j. 53

 skip
Warehouse location problem
n warehouses
cost fj of opening warehouse j
m customers
customer i has a demand of di
unit shipping cost cij of serving customer i via
warehouse j.

Variables:
let yj = 1 if warehouse j is opened
Let xij = amount of demand for customer i satisfied
via warehouse j. 54
Suppose you knew which warehouses were
open. S = set of open warehouses
xij = demand satisfied for customer
i at warehouse j minimize Si,j cijxij
yj = 1 for j in S, SjS fj
yj = 0 for j not in S.

subject to:
customers get their demand Sj xij = di
satisfied
no shipments are made from an
empty warehouse
xij di if yj = 1
xij = 0 if yj = 0
and x 0
55
 More on warehouse location
yj = 1 if warehouse i is
opened
yj = 0 otherwise
minimize Si,j cijxij
xij = flow from i to j
Sj fi yi
subject to:
customers get their demand
satisfied
Si xij = dj
each warehouse is either 0 yj 1
opened or it is not (no partial yj integral for all j.
openings)
xij dj yi for all i, j
no shipments are made from
an empty warehouse and x 0
56
Two key aspects of using integrality
in the model
Costs: we include the cost of the
warehouse only if it is opened.
Si fi yi

Constraints: We do not allow shipping

from warehouse j if it is not opened.
xij dj yi for all i, j

57
 More on warehouse location
The above is a core subproblem occurring in
supply chain management, and it can be enriched
more complex distribution system
capacity constraints
non-linear transportation costs
delivery times
multiple products
business rules
and more

58
 Using Excel Solver to Solve
Integer Programs
Add the integrality constraints (or add that a variable
is binary)
Set the Solver Tolerance. (The tolerance is the
percentage deviation from optimality allowed by
solver in solving Integer Programs.)
The default is 5%
The default is way to high
It often finds the optimum for small
problems
59
 Some Comments on IP models
There are often multiple ways of modeling
the same integer program.

Solvers for integer programs are extremely

sensitive to the formulation. (not true for
LPs)

60
 Example
constraint A: 2x1 + 2x2 + + 2x50 51
constraint B: x1 + x2 + + x50 25
assume that x is binary

constraints C: x1 y, x2 y, , x50 y
(where y is binary)
constraint D: x1 + + x50 50 y

B dominates A, C dominates D
It is not obvious why, until you see the algorithms.

61
 Summary on Integer Programming
Dramatically improves the modeling capability
Economic indivisibilities
Logical constraints
Modeling nonlinearities
classical problems in capital budgeting and in
supply chain management
Not as easy to model
Not as easy to solve.

62
Integer Programming

Review
Branch and
Bound 66
 Overview of Techniques for
Solving Integer Programs
Enumeration Techniques
Complete Enumeration
list all solutions and choose the best
Branch and Bound
Implicitly search all solutions, but cleverly eliminate the vast majority
before they are even searched
Implicit Enumeration
Branch and Bound applied to binary variables
Cutting Plane Techniques
Use LP to solve integer programs by adding constraints to eliminate the
fractional solutions.

67
 Overview of this lecture
Explain and illustrate branch and bound.
It is the starting point for all solution techniques for integer
programming
Lots of research has been carried out over the past 40 years to
make it more and more efficient
But, it is an art form to make it efficient. (We shall get a sense
why.)
Integer programming is intrinsically difficult.

68
 Capital Budgeting Example
Investment budget = $14,000

Investment 1 2 3 4 5 6

Cash
Required $5 $7 $4 $3 $4 $6
(1000s)
NPV
added $16 $22 $12 $8 $11 $19
(1000s)

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6

subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14
69
xj binary for j = 1 to 6
 Complete Enumeration
Systematically considers all possible values
of the decision variables.
If there are n binary variables, there are 2n
different ways.
Usual idea: iteratively break the problem in
two. At the first iteration, we consider
separately the case that x1 = 0 and x1 = 1.

70
 An Enumeration Tree
Original
problem
x1 = 0 x1 = 1

x2 = 0 x2 = 1 x2 = 0 x2 = 1

x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

71
 On complete enumeration
Suppose that we could evaluate 1 billion
solutions per second.
Let n = number of binary variables
Solutions times
n = 30, 1 second
n = 40, 17 minutes
n = 50 11.6 days
n = 60 31 years
72
 On complete enumeration
Suppose that we could evaluate 1 trillion
solutions per second, and instantaneously
eliminate 99.9999999% of all solutions as
not worth considering
Let n = number of binary variables
Solutions times
n = 70, 1 second
n = 80, 17 minutes
n = 90 11.6 days

73
n = 100 31 years
 Branch and Bound
The essential idea: search the
enumeration tree, but at each node
1. Solve the linear program at the node
2. Eliminate the subtree (fathom it) if
1. The solution is integer (there is no need to go
further) or
2. The best solution in the subtree cannot be as
good as the best available solution (the
incumbent) or
3. There is no feasible solution
74
 Branch and Bound
Node 1 is the
44 3/7 original LP
1
Relaxation

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6

subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14
0 xj 1 for j = 1 to 6

Solution at node 1:
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7

The IP cannot have value higher than 44 3/7. (Upper bound)

75
 Branch and Bound
Node 2 is the
44 3/7 original LP
1
Relaxation plus
x1 = 0 the constraint
x1 = 0.
44 2

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6

subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14
0 xj 1 for j = 1 to 6, x1 = 0

Solution at node 2:
x1 = 0 x2 = 1 x3 = 1/4 x4 = x5 = 0 x6 = 1 z = 44 76
 Branch and Bound
Node 3 is the
44 3/7 original LP
1
Relaxation plus
x1 = 0 x1 = 1 the constraint
x1 = 1.
44 2 3 44 3/7

The solution at node 1 was

x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
Note: it was the best solution with no constraint on x1.
So, it is also the solution for node 3. (If you add a
constraint, and the old optimal solution is feasible, then
it is still optimal.) 77
 Branch and Bound
Node 4 is the
original LP
44 3/7
1 Relaxation plus
x1 = 0 x1 = 1 the constraints
x1 = 0, x2 = 0.

44 2 3 44 3/7
x2 = 0

42 44

Solution at node 4: 0 0 1 0 1 1 z = 42
Our first incumbent solution! No solution in
No further searching from node 4 because the subtree can
have a value
there cannot be a better integer solution. 78
better than 42.
 Branch and Bound
The incumbent The
is the best incumbent
44 3/7
solution on 1 solution has
hand. value 42
x1 = 0 x1 = 1

44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 44 44 5 44 6 44 1/3 7

We next solved the LPs associated with nodes 5, 6, and 7

No new integer solutions were found.
We would eliminate (fathom) a subtree if we were guaranteed
79
that no solution in the subtree were better than the incumbent.
 Branch and Bound
The
incumbent
44 3/7
1 solution has
x1 = 0 x1 = 1 value 42

44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 44 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43.75 8 43.5 9 43.25 10 43.8 11 44.3 12 - 13

13

We next solved the LPs associated with nodes 8 -13 80

 Summary so far
We have solved 13 different linear
programs so far.
One integer solution found
One subtree fathomed (pruned) because
the solution was integer (node 4)
One subtree fathomed because the
solution was infeasible (node 13)
No subtrees fathomed because of the
bound 81
 Branch and Bound
1 44 3/7 The
incumbent
x1 = 0 x1 = 1 solution has
value 42
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 4 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43.75 8 43.5 9 43.25 10 43.8 11 44.3 12 - 13

43.75 42.66

We next solved the LPs associated with the next nodes.

We can fathom the node with z = 42.66. Why? 82
 Getting a better bound
The bound at each node is obtained by solving an LP.
But all costs are integer, and so the objective value of
each integer solution is integer. So, the best integer
solution has an integer objective value.
If the best integer valued solution for a node is at most
42.66, then we know the best bound is at most 42.
Other bounds can also be rounded down.

83
 Branch and Bound
1 44 3/7 The
incumbent
x1 = 0 x1 = 1 solution has
value 42
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 4 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43.75 8 43.5 9 43.25 10 43.8 11 44.3 12 -. 13

43.75 42.66

84
 Branch and Bound
1 44 The
incumbent
x1 = 0 x1 = 1 solution has
value 42
44 2 3 44
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43 8 43 9 43 10 43 11 44 12 - 13

43 42 43 42 43 43

We found a new incumbent solution!

86
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43
 Branch and Bound
1 44 The new
incumbent
x1 = 0 x1 = 1 solution has
value 43
44 2 3 44
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43 88 43 99 10
43 10 11
43 11 44 12 - 13

43 42 43 42 43 43

We found a new incumbent solution!

87
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43
 Branch and Bound
1 44 The new
incumbent
x1 = 0 x1 = 1 solution has
value 43
44 2 3 44
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43 88 43 99 10
43 10 11
43 11 44 12 - 13

If we had found this incumbent earlier, we could have

saved some searching.
88
 Finishing Up
1 44 The new
incumbent
x1 = 0 x1 = 1 solution has
value 43
44 2 3 44
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43 88 43 99 10
43 10 11
43 11 44 12 - 13

44 14 15 -

44 16 17 -
89
38 18 19 -
 Lessons Learned
Branch and Bound can speed up the search
Only 25 nodes (linear programs) were evaluated
Other nodes were fathomed
Obtaining a good incumbent earlier can be valuable
only 19 nodes would have been evaluated.
Solve linear programs faster, because we start with an
excellent or optimal solution
uses a technique called the dual simplex method
Obtaining better bounds can be valuable.
We sometimes use properties that are obvious to us, such as
the fact that integer solutions have integer solution values
90
 Branch and Bound
Notation:
z* = optimal integer solution value
Subdivision: a node of the B&B Tree
Incumbent: the best solution on hand
zI: value of the incumbent
zLP: value of the LP relaxation of the current node
Children of a node: the two problems created for a node, e.g., by saying
xj = 1 or xj = 0.
LIST: the collection of active (not fathomed) nodes, with no active
children.

NOTE: zI z*
91
 Illustrating the definitions
The incumbent
44 3/7 solution has
1 value 42
x1 = 0 x1 = 1

44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 44 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43.75 8 43.5 9 43.25 10 43.8 11 44.3 12 13

- 13

z* = 43 = optimal integer solution value. (We

found it later in the search)
Incumbent is 0 0 1 0 1 1 zI = 42.
It is the optimal solution for the subdivision 4.
The zLP values for each subdivision are next to the nodes. 92
 Illustrating the definitions
44 3/7
1
The incumbent
x1 = 0 x1 = 1 solution has
value 42

44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1

42 44 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

43.75 8 43.5 9 43.25 10 43.8 11 44.3 12 13

- 13

The children of node (subdivision) 1 are nodes 2 and 3.

The children of node 3 are nodes 6 and 7.
LIST = { 8, 9, 10, 11, 12 } = unfathomed nodes with no
active children
93
 Branch and Bound Algorithm
INITIALIZE LIST = {original problem}
Incumbent: =
zI = -
SELECT:
If LIST = , then the Incumbent is optimal if it exists, and the
problem is infeasible if no incumbent exists;
else, let S be a node (subdivision) from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
e.g., S = {1} 44 3/7 1
e.g., S = {13} - 13

CASE 1. zLP = - (the LP is infeasible)

Remove S from LIST (fathom it)
Return to SELECT 94
 Branch and Bound Algorithm
INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the
problem is infeasible if no incumbent exists;
else, let S be a node from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value

CASE 2. - < zLP zI.

That is, the LP is dominated by the incumbent.

Then remove S from LIST (fathom it)

Return to SELECT

43 88 e.g., the incumbent has value 43, and node

14 is selected. zLP = 43.
43 14 42 95
 Branch and Bound Algorithm
INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the
problem is infeasible if no incumbent exists;
else, let S be a subdivision from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value

CASE 3. zI < zLP and xLP is integral. e.g., node 4 was selected,
That is, the LP solution is integral and and the solution to the LP
dominates the incumbent. was integer-valued.

Then Incumbent := xLP; 42 4

zI := zLP
Remove S from LIST (fathomed by integrality)
Return to SELECT 96
 Branch and Bound Algorithm
INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the
problem is infeasible if no incumbent exists;
else, let S be a subdivision from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value

e.g., select node 3.

CASE 4. zI
< zLP
and is not integral.
xLP
x1 = 1
There is not enough information to fathom S
3 44 3/7
Remove S from LIST x2 = 0 x2 = 1
Add the children of S to LIST
Return to SELECT 6 7

List := List 3 + {6,7} 97

 Different Selection Rules are
Possible
Rule of Thumb 1: Dont let LIST get too big (the solutions
must be stored). So, prefer nodes that are further down in
the tree.

Rule of Thumb 2: Pick a node of LIST that is likely to

lead to an improved incumbent. Sometimes special
heuristics are used to come up with a good incumbent.

98
 Branching
One does not have to have the B&B
tree be symmetric, and one does not
select subtrees by considering variables
in order.

X1 =0 X1 =1 Choosing how to
branch so as to
X2 =0 X2 =1 X8 = 0 X8 = 1 reduce running
time is largely
X =0
art and based
3
3= 0
X
X3 = 0
X3= 1 on experience.
X 3= X 3= 1 X3 =1
X3 = 1 0

. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .

99
 Different Branching Rules are
Possible
Branching: determining children for a node. There are many choices.

Rule of thumb 1: if it appears clear that xj = 1 in an optimal solution, it

is often good to branch on xj = 0 vs xj = 1.
The hope is that a subdivision with xj = 0 can be pruned.

Rule of thumb 2: branching on important variables is worthwhile

100
 Different Bounding Techniques are Possible

We use the bound obtained by dropping the integrality

constraints (LP relaxation). There are other choices.
Key tradeoff for bounds: time to obtain a bound vs quality
of the bound.
If one can obtain a bound much quicker, sometimes we
would be willing to get a bound that is worse
It usually is worthwhile to get a bound that is better, so
long as it doesnt take too long (see next lecture)

101
 What if the variables are general
integer variables?
One can choose children as follows:
child 1: x1 3 (or xj k)
child 2 x1 4 (or xj k+1)

How would one choose the variable j and the value

k
A common choice would be to take a fractional value
from xLP. e.g., if x7 = 5.62, then we may branch on x7
5 and x7 6.
Other choices are also possible. 102
 Summary
Branch and Bound is the standard way of solving
IPs to optimality.
There is art to making it work well in practice.
Much of the art is built into state-of-the-art
solvers such as CPLEX, or others current
available tools such as Lindo, Lingo, LiPs,
Matlab or R, Python even Java

103