You are on page 1of 35

ECEN 687

VLSI Design Automation

Lecture 1 Linear Programming

ECEN 687 Lecture 1 1


Optimization
! What is optimization? Make Merriam-Webster
Definition of OPTIMIZATION
something better or the best? : an act, process, or
! Can be described and solved in methodology of making
something (as a design,
mathematics system, or decision) as fully
! In chip designs, we face many perfect, functional, or
effective as possible;
optimization problems specifically : the mathematical
! Optimization problems are not procedures (as finding the
restricted to, and often far maximum of a function)
involved in this
beyond chip designs

ECEN 687 Lecture 1 2


A Diet Problem
Polly wonders how much money she must spend on food
to get all the energy (2000 kcal) and protein (55g) that
she needs everyday. Of course, she wishes to minimize
the expense.

Serving size Energy (kcal) Protein (g) Price per


serving
Oatmeal 28g 110 4 3
Chicken 100g 205 32 24
Cherry pie 170g 420 22 20
Pork 260g 260 80 19

ECEN 687 Lecture 1 3


Decision Variable
! How many servings of oatmeal? x1
! chicken x2
! cherry pie x3
! pork x4

ECEN 687 Lecture 1 4


Objective Function
Minimize the expense
Serving size Energy (kcal) Protein (g) Price per
serving
Oatmeal 28g 110 4 3
Chicken 100g 205 32 24
Cherry pie 170g 420 22 20
Pork 260g 260 80 19

Minimize 3x1 + 24x2 + 20x3 + 19x4

ECEN 687 Lecture 1 5


Constraints
At least 2000 kcal energy and 55g protein
Serving size Energy (kcal) Protein (g) Price per
serving
Oatmeal 28g 110 4 3
Chicken 100g 205 32 24
Cherry pie 170g 420 22 20
Pork 260g 260 80 19

Subject to: 110x1 + 205x2 + 420x3 + 260x4 2000


4x1 + 32x2 + 22x3 + 80x4 55
x1, x2, x3, x4 0
ECEN 687 Lecture 1 6
Everything Together
Minimize 3x1 + 24x2 + 20x3 + 19x4
Subject to: 110x1 + 205x2 + 420x3 + 260x4 2000
4x1 + 32x2 + 22x3 + 80x4 55
x1, x2, x3, x4 0

A linear programming problem

ECEN 687 Lecture 1 7


Mathematical Programming
Minimize f(x)
Subject to: A(x) 0

ECEN 687 Lecture 1 8


Linear Programming
Minimize f(x)
Subject to: A(x) 0

f(x) is a linear function of x


A(x) is a set of linear functions of x

ECEN 687 Lecture 1 9


Different Forms
! Objective function
! Minimize f(x) " Maximize -f(x)

! Constraint
! A(x) 0 " -A(x) 0

ECEN 687 Lecture 1 10


Inequality vs. Equality Constraint
A(x) = 0 " A(x) 0 and A(x) 0

A(x) 0 " A(x) + s = 0 and s 0


s: slack variable

ECEN 687 Lecture 1 11


LP Standard Form
Minimize c1 x1 + c 2 x 2 + ... + cnxn
Subject to a11 x1 + a12 x 2 + ... + a1nxn b1
a 21 x1 + a 22 x 2 + ... + a 2 nxn b 2
... ...
am1 x1 + am 2 x 2 + ... + amnxn bm
x1, x 2,..., xn 0 Min cT x
s.t. Ax b
x0
ECEN 687 Lecture 1 12
Example
x2
max 3 x1 + 2 x 2
6
s.t. x1 + 2 x 2 8
2 x1 + x 2 6
x1, x 2 0
4
Optimal solution

Feasible
solutions
x1
3 8
ECEN 687 Lecture 1 13
Unbounded
x2

Max x1+x2

Feasible
solutions
x1

ECEN 687 Lecture 1 14


Infeasible
! Constraints
x > 1 and x < -2

ECEN 687 Lecture 1 15


Basic Feasible Solution
x2
max 3 x1 + 2 x 2
s.t. x1 + 2 x 2 8
2 x1 + x 2 6
Basic feasible solution x1, x 2 0

x1

ECEN 687 Lecture 1 16


How to Solve LP?
! Simplex method
! Start from a basic feasible solution
! Move along boundary to another basic feasible solution
! Repeat till reach the optimal
! Interior point method
! Faster

ECEN 687 Lecture 1 17


For EDA R&D
! It is more important to formulate than to solve LP
! There are existing LP solvers
! Sometimes it is not obvious how to formulate an LP
! You may see this in homework and exams

ECEN 687 Lecture 1 18


Why Is LP Important?
! Real world problems are mostly nonlinear
! However, LP theory is the most matured one
! Sometimes NLP can be solved using LP approximately

ECEN 687 Lecture 1 19


Another Example
Max 4 x1 + x 2 + 5 x 3 + 3 x 4 = z
s.t. x1 x 2 x 3 + 3x 4 1 (1)
5 x1 + x 2 + 3 x 3 + 8 x 4 55 (2)
x1 + 2 x 2 + 3 x 3 5 x 4 3 (3)
x1, x 2, x3, x 4 0

The optimal solution is denoted by z*

ECEN 687 Lecture 1 20


Alternative Approach
Max 4 x1 + x 2 + 5 x 3 + 3 x 4 = z
s.t. x1 x 2 x 3 + 3x 4 1 (1)
5 x1 + x 2 + 3 x 3 + 8 x 4 55 (2)
x1 + 2 x 2 + 3 x 3 5 x 4 3 (3) Guess an upper
bound for z* ?
x1, x 2, x3, x 4 0

25 5 40 275
(2)5/3 => x1 + x 2 + 5 x3 + x 4
3 3 3 3
25 5 40
4 x1 + x 2 + 5 x3 + 3x 4 x1 + x 2 + 5 x3 + x 4 z*275/3
3 3 3

ECEN 687 Lecture 1 21


A Tighter Bound
Max 4 x1 + x 2 + 5 x 3 + 3 x 4 = z
s.t. x1 x 2 x 3 + 3x 4 1 (1)
5 x1 + x 2 + 3 x 3 + 8 x 4 55 (2)
x1 + 2 x 2 + 3 x 3 5 x 4 3 (3)
x1, x 2, x3, x 4 0

(2)+(3) => 4 x1 + 3 x 2 + 6 x3 + 3 x 4 58

z*58

ECEN 687 Lecture 1 22


A More Systematic Attempt
! Take a linear combination of (1), (2) and (3)
! y1(1) + y2(2) + y3(3), y1, y2, y30
Max 4 x1 + x 2 + 5 x 3 + 3 x 4 = z
s.t. x1 x 2 x 3 + 3x 4 1 (1)
5 x1 + x 2 + 3 x 3 + 8 x 4 55 (2)
x1 + 2 x 2 + 3 x 3 5 x 4 3 (3)
x1, x 2, x3, x 4 0
( y1 + 5 y 2 y 3) x1 + ( y1 + y 2 + 2 y 3) x 2 +
( y1 + 3 y 2 + 3 y 3) x3 + (3 y1 + 8 y 2 5 y 3) x 4 y1 + 55 y 2 + 3 y 3

ECEN 687 Lecture 1 23


To Obtain Upper Bound
( y1 + 5 y 2 y 3) x1 + ( y1 + y 2 + 2 y 3) x 2 +
( y1 + 3 y 2 + 3 y 3) x3 + (3 y1 + 8 y 2 5 y 3) x 4 y1 + 55 y 2 + 3 y 3

Max 4 x1 + x 2 + 5 x 3 + 3 x 4 = z To form upper bound for z*

s.t. x1 x 2 x 3 + 3x 4 1 (1) y1 + 5 y 2 y 3 4
5 x1 + x 2 + 3 x 3 + 8 x 4 55 (2) y1 + y 2 + 2 y 3 1
x1 + 2 x 2 + 3 x 3 5 x 4 3 (3) y1 + 3 y 2 + 3 y 3 5
x1, x 2, x3, x 4 0 3 y1 + 8 y 2 5 y 3 3

ECEN 687 Lecture 1 24


Minimize the Bound
( y1 + 5 y 2 y 3) x1 + ( y1 + y 2 + 2 y 3) x 2 +
( y1 + 3 y 2 + 3 y 3) x3 + (3 y1 + 8 y 2 5 y 3) x 4 y1 + 55 y 2 + 3 y 3

Max 4 x1 + x 2 + 5 x 3 + 3 x 4 = z y1 + 5 y 2 y 3 4
s.t. x1 x 2 x 3 + 3x 4 1 (1) y1 + y 2 + 2 y 3 1
5 x1 + x 2 + 3 x 3 + 8 x 4 55 (2) y1 + 3 y 2 + 3 y 3 5
x1 + 2 x 2 + 3 x 3 5 x 4 3 (3) 3 y1 + 8 y 2 5 y 3 3
x1, x 2, x3, x 4 0
Min y1 + 55 y 2 + 3 y 3

ECEN 687 Lecture 1 25


The Dual Problem
Primal problem Dual problem
Max 4 x1 + x 2 + 5 x 3 + 3 x 4 Min y1 + 55y2 + 3y3
s.t. x1 x 2 x 3 + 3 x 4 1 s.t. y1 + 5 y 2 y 3 4
5 x1 + x 2 + 3 x 3 + 8 x 4 55 y1 + y 2 + 2 y 3 1
x1 + 2 x 2 + 3 x 3 5 x 4 3 y1 + 3 y 2 + 3 y 3 5
x1, x 2, x 3, x 4 0 3 y1 + 8 y 2 5 y 3 3
y1, y 2, y 3 0

Dual variables

ECEN 687 Lecture 1 26


LP Duality
n
Max c x j j
j =1
n
s.t. a x bij j i (i = 1,2,..., m)
j =1

m
xj 0 ( j = 1,2,..., n)
Min b y i i
i =1
m
s.t. a y c
ij i j ( j = 1,2,..., n)
i =1

yi 0 (i = 1,2,..., m)
ECEN 687 Lecture 1 27
Duality with Equality Constraints
! Figure out by yourself
! Hint: equality constraint " unconstrained dual
variable?

ECEN 687 Lecture 1 28


Shadow Price
! A dual variable is the shadow price: the change in the
objective value of the optimal solution of the primal
problem by relaxing the constraint corresponding to
the dual variable by one unit
! Also known as marginal value or marginal cost

ECEN 687 Lecture 1 29


The Duality Theorem

If the primal problem has an optimal solution ( x1, x 2,..., xn),


then the dual problem has an optimal solution ( y 1, y 2,..., y m)
such that
n m

c x = b y
j =1
j j
i =1
i i

ECEN 687 Lecture 1 30


Relationship between Primal and
Dual
Dual

Optimal Infeasible Unbounded

Primal Optimal Possible Impossible Impossible

Infeasible Impossible Possible Possible

Unbounded Impossible Possible Impossible

ECEN 687 Lecture 1 31


Complementary Slackness
Theorem 1 (it is intuitively related to the concept of shadow price)
Let ( x1, x 2,..., xn) be a feasible solution for the primal problem and
let ( y 1, y 2,..., y m) be a feasible solution for the dual problem.
Necessary and sufficient conditions for simultaneous optimality
of both of the solutions are
m

a y = c
i =1
ij i j or xj = 0 (or both) j = 1,2,..., n

and
n

a x = b
j =1
ij j i or y i = 0 (or both) i = 1,2,..., m

ECEN 687 Lecture 1 32


Complementary Slackness
Theorem 2
A feasible solution ( x1, x 2,..., xn) of the primal problem is optimal
if and only if there are numbers ( y 1, y 2,..., y m) such that
m

a y = c
i =1
ij i j whenever xj > 0
n
y i = 0 whenever a x < b
ij j i
j =1

and such that


m

a y c
i =1
ij i j j = 1,2,..., n

y i 0 i = 1,2,..., m
ECEN 687 Lecture 1 33
Application of Complementary
Slackness
Max 8 x1 9 x 2 + 12 x3 + 4 x 4 + 11x 5
s.t. 2 x1 3x 2 + 4 x3 + x 4 + 3 x5 1
x1 + 7 x 2 + 3x 3 2 x 4 + x 5 1
5 x1 + 4 x 2 6 x 3 + 2 x 4 + 3x 5 22
x1, x 2, x 3, x 4, x 5 0

Is (0, 2, 0, 7, 0) an optimal solution?

ECEN 687 Lecture 1 34


According to Theorem 2
Max 8 x1 9 x 2 + 12 x3 + 4 x 4 + 11x 5
3 y1 + 7 y 2 + 4 y 3 = 9
s.t. 2 x1 3x 2 + 4 x3 + x 4 + 3 x5 1
x1 + 7 x 2 + 3x 3 2 x 4 + x 5 1 y1 2 y 2 + 2 y 3 = 4
5 x1 + 4 x 2 6 x 3 + 2 x 4 + 3x 5 22 y2 = 0
x1, x 2, x 3, x 4, x 5 0
Solve the above equations,
x=(0, 2, 0, 7, 0)T we get y=(3.4, 0, 0.3)T
which is not feasible for the
dual problem
x is not optimal

ECEN 687 Lecture 1 35

You might also like