Julie Eaton

Convex Optimization
Chapter 4, problem 43
Eigenvalue optimization via SDP. Suppose A : Rn Sm is affine, i.e.,
A(x) = A0 + x1 A1 + + xn An where Ai Sm . Let 1 (x) m (x) denote the eigenvalues of A(x).
Show how to pose the following problems as SDPs.

Target form for the SDP

minimize cT x
subject to x1 F1 + . . . xn Fn + G  0
Ax = b
where F1 , . . . , Fn , G Sk and A Rpn .
Generalized Lagrangian Duality

Primal Problem (P) min f0 (x)

x s.t. fi (x)K 0,i=1,...m

Lagrangian L(x, y) = f0 (x) + hy, F (x)i K (y)

Dual objective g(y) = inf L(x, y).
x

Dual Problem (D) sup g(y)

y

(a) Minimize the maximum eigenvalue 1 (x).

Solution: Note that I A(x)  0 if and only if 1 (x) 0. Therefore, 1 (x) is the smallest for which
I A(x)  0. That is, for each x we have

1 (x) = min (1)

A(x)I0

So we may express the problem at hand as follows minxRn 1 (x) = minxRn s.t. A(x)I0 . Let
c = [0, . . . , 0, 1]T Rn+1 , and let x
= [x, ]T . Then we may re-express the minimization problem above as

min cT x
,
A0 +x1 A1 ++xn An +(I)0

which is in the primal form for an SDP.

The Lagrangian is

x, Y ) = cT x
L( + tr ((A0 + x1 A1 + xn An + (I))Y ) Sm
+
(Y )
n
X
= tr(A0 Y ) + xi tr(Ai Y ) + (1 tr(Y )) Sm
+
(Y )
i=1

Therefore,
(
tr(A0 Y ) Sm (Y ) if tr(Ai Y ) = 0 for i = 1, . . . , n and tr(Y ) = 1,
g(Y ) = inf L(
x, Y ) = +
x
otherwise.

So the dual problem is

max tr(A0 Y )
Y Sm
subject to
+
tr(Y ) = 1, tr(Ai Y ) = 0
for i = 1, . . . n

1
(b) Minimize the spread of the eigenvalues, 1 (x) m (x).
Solution: Note that I A(x)  0 if and only if m (x) 0. Therefore,
m (x) = max . (2)
IA(x)0

Negating both sides of Equation 2, and combining the expression for m (x) with that for 1 (x) in Equation
1, we have
minn 1 (x) m (x) = n
min .
xR xR s.t. IA(x)I
     
Ai 0 I 0 0 0
Let Fi = for i = 0, . . . , n, let F1 = , and Fm = .
0 Ai 0 0 0 I
= [x, , ]T and c = [0, . . . , 0, 1, 1]T Rn+2 . Then
Define x
min 1 (x) m (x) = min cT x
,
xRn F0 + m
P
i=1 x1 F1 +F +F 0

which is an SDP. The Lagrangian is

x, Y ) = cT x
L( + tr ((F0 + x1 F1 + + xn Fn + F + F )Y ) S2m
+
)(Y )
n
X
= tr(F0 Y ) + xi tr(Fi Y ) + (1 + tr(F Y )) + (1 + tr(F Y )) S2m
+
(Y )
i=1

Therefore,

tr(F0 Y ) S2m
+
(Y ) if tr(Fi Y ) = 0 for i = 1, . . . , n,
g(Y ) = inf L(
x, Y ) = tr(F Y ) = 1, and tr(F Y ) = 1,
x

otherwise.

The dual problem is

max tr(F0 Y ).
Y S2m
subject to
+
tr(F Y ) = 1, tr(F Y ) = 1, tr(Fi Y ) = 0
for i = 1, . . . n

Y
 
Y1
If we write Y = , where Y1 , Y2 Sm
+ , then we may re-express the the dual problem as
Y T Y2

max tr(A0 Y1 ) tr(A0 Y2 ).

Y1 , Y2 Sm+ subject to
tr(Y1 ) = 1, tr(Y2 ) = 1, tr(Ai Y1 ) tr(Ai Y2 ) = 0
for i = 1, . . . n

(c) Minimize the condition number of A(x), subject to A(x)  0. The condition number is
defined as (A(x)) = 1 (x)/m (x). You may assume that A(x)  0 for at least one x.
Solution: From Equation 1 and by taking the reciprocal of each side of Equation 2, we have
1 (x)
min = min .
{xRn :A(x)0} m (x) 0IA(x)I
Let s = 1/ and t = /. Then we may express the minimization problem as
min t= min t.
s<0,IsA0 + n
P
s<0,IsA(x)tI i=1 xi sAi tI

2
Letting y = sx, we have
min t= min t.
s<0,IsA0 + n Is0,IsA0 + n
P P
i=1 xi sAi tI i=1 yi Ai tI

Consider the two optimization problems:

(P1 ) min t
Is0,IsA0 + n
P
i=1 yi Ai tI

(P2 ) min t.
Is0,IsA0 + n
P
i=1 yi Ai tI

The second problem is a convex optimization problem, as defined in chapter 4. Both objectives are differ-
entiable. The one difficulty (P1 ) is that its domain is not closed. Note that the proof of the optimality
condition (4.21) does not rely on the fact that the domain of the problem is closedthe same proof works for
a problem with a domain that convex but not closed. Also, the discussion in section 4.6 says that condition
(4.21) may be applied to problems with generalized inequality constraints. Therefore, we may apply this
optimality condition to both P1 and P2 . Suppose [ y , s, t]T solves (P1 ). Then
[0, 1, 0]([y y, t t, s s]T ) = t t 0 (3)
T n+2
Pn
for all [y, t, s] R such that s > 0 and IP sA0 + i=1 yi Ai  tI. Then Equation 3 holds for all
n
[y, t, s]T Rn+2 such that s 0 and I  sA0 + i=1 yi Ai  tI, which implies that [ y , t, s]T solves (P2 ). So,
every solution to (P1 ) is a solution to (P2 ). The converse is not true. In order to use the SDP framework, I
will pose (P2 ) as an SDP.

0 0 0 I 0 0 0 0 0 0 0 0
Let G = 0 I 0 , Fs = 0 A0 0 , Fi = 0 Ai 0 , and Ft = 0 0 0 .
0 0 0 0 0 A0 0 0 Ai 0 0 I
T n+2 T n+2
Let x = [y, t, s] R , and c = [0, , 0, 1, 0] R . Then (P2 ) is equivalent to
min cT x
.
sFs + n
P
i=1 yi Fi +tFt +G0

The Lagrangian is
n
X
x, Y ) = cT x
L( + tr((sFs + yi Fi + tFt + G)Y ) S3m
+
(Y )
i=1
X
= str(Fs Y ) + yi tr(Fi Y ) + t(1 + tr(Ft Y )) + tr(GY ) S3m
+
(Y )
i=1n

Therefore,
(
tr(GY ) S3m (Y ) if tr(Fi Y ) = 0 for i = 1, . . . , n and tr(Ft Y ) = 1,
g(Y ) = inf L(
x, Y ) = +
x
otherwise.
So the dual problem is
max tr(GY ).
Y S3m
+

subject to
tr(Fs Y ) = 0, tr(Fi Y ) = 0, tr(Ft Y ) = 1
for i = 1, . . . n

Y1 A B
If we write Y = AT Y2 C , where Y1 , Y2 , Y3 Sm + , then we may re-express the the dual problem as
BT CT Y3
max tr(Y2 ).
Y1 , Y2 , Y3 Sm
+ subject to
tr(Y1 A0 Y2 + A0 Y3 ) = 0, tr(Ai (Y3 Y2 )) = 0, tr(Y3 ) = 1
for i = 1, . . . n

3
(d) Minimize the sum of the absolute values of the eigenvalues, |1 (x)| + + |m (x)|.
Solution: For each x, A(x) is a linear mapping from Rm Rm . Since A(x) is symmetric, it has an orthogonal
basis of eigenvectors (which depend smoothly on x). Let E+ denote the Minkowski sum of the eigenspaces
of A(x) corresponding to the nonnegative eigenvalues of A(x) (let E+ be 0 if there are no nonnegative
eigenvalues). Likewise, let E denote the Minkowski sum of the eigenspaces of A(x) corresponding to
negative eigenvalues of A(x) (and let it be 0 if all the eigenvalues are nonnegative). Let U = A(x)|E and
+

let V = A(x)|E . Then U , V S+ and A(x) = U

m V . Furthermore, the eigenvalues of U
are the nonnegative
eigenvalues
Pm of A(x) and the eigenvalues of V are the absolute values of the negative eigenvalues of A(x). Then
m
i=1 |i (x)| = tr(U + V ). Suppose U, V S+ are such that A(x) = U V . We may write U = U+ + U
and V = V+ + V , where U+ = U |E+ , etc. Note that U+  0, etc. Then A(x) = (U+ V+ ) (V U ),
which implies that U = U+ + V+ and V = V U . So,
m
X
+ V ) = tr(U+ V+ + V U )
|i (x)| = tr(U
i=1
= tr(U+ ) tr(V+ ) + tr(V ) tr(U ) tr(U+ ) + tr(V )
tr(U ) + tr(V ).
Pm
Therefore, the problem minx i=1 |i (x)| is equivalent to

min tr(U + V ).
U,V 0,A0 + n
P
i=1 xi Ai =U V