ENEE631 Digital Image Processing (Spring'04)

Unitary Transforms
Spring 04 Instructor: Min Wu

ECE Department, Univ. of Maryland, College Park
www.ajconline.umd.edu (select ENEE631 S04)
minwu@eng.umd.edu
Based on ENEE631 Spring04
Section 9
ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [3]
Image Transform: A Revisit
With A Coding Perspective
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [4]
Why Do Transforms?
Fast computation
E.g., convolution vs. multiplication for filter with wide support
Conceptual insights for various image processing
E.g., spatial frequency info. (smooth, moderate change, fast change, etc.)
Obtain transformed data as measurement
E.g., blurred images, radiology images (medical and astrophysics)
Often need inverse transform
May need to get assistance from other transforms

For efficient storage and transmission
Pick a few representatives (basis)
Just store/send the contribution from each basis
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [5]
Basic Process of Transform Coding
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)
ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [6]
1-D DFT and Vector Form
{ z(n) } { Z(k) }
n, k = 0, 1, , N-1
W
N
= exp{ - j2t / N }
~ complex conjugate of primitive N
th
root of unity
Vector form and interpretation for inverse transform
z = E
k
Z(k) a
k

a
k
= [ 1 W
N
-k
W
N
-2k
W
N
-(N-1)k
]
T
/ \ N
Basis vectors
a
k
H
= a
k
* T
= [ 1 W
N
k
W
N
2k
W
N
(N-1)k
] / \ N


Use a
k
H
as row vectors to construct a matrix F
Z = F z z = F
*T
Z = F
*
Z

F is symmetric (F
T
=F) and unitary (F
-1
= F
H
where F
H
= F
*T
)

=
=

=
1
0
1
0
) (
1
) (
) (
1
) (
N
k
nk
N
N
n
nk
N
W k Z
N
n z
W n z
N
k Z
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [7]
Basis Vectors and Basis Images
A basis for a vector space ~ a set of vectors
Linearly independent ~ E a
i
v
i
= 0 if and only if all a
i
=0
Uniquely represent every vector in the space by their linear combination
~ E b
i
v
i
( spanning set {v
i
} )
Orthonormal basis
Orthogonality ~ inner product <x, y> = y
*T
x= 0
Normalized length ~ || x ||
2
= <x, x> = x
*T
x= 1

Inner product for 2-D arrays
<F, G> = E
m
E
n
f(m,n) g
*
(m,n) = G
1
*T
F
1
(rewrite matrix into vector)
!! Dont do FG ~ may not even be a valid operation for MxN matrices!
2D Basis Matrices (Basis Images)
Represent any images of the same size as a linear combination of basis
images
A vector space consists of a set of vectors, a
field of scalars, a vector addition operation,
and a scalar multiplication operation.
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [8]
1-D Unitary Transform
Linear invertible transform
1-D sequence { x(0), x(1), , x(N-1) } as a vector
y = A x and A is invertible

Unitary matrix ~ A
-1
= A
*T

Denote A
*T
as A
H
~ Hermitian
x = A
-1
y = A
*T
y = E a
i
*T
y(i)
Hermitian of row vectors of A form a set of orthonormal basis vectors
a
i
*T
= [a
*
(i,0), , a
*
(i,N-1)]
T


Orthogonal matrix ~ A
-1
= A
T

Real-valued unitary matrix is also an orthogonal matrix
Row vectors of real orthogonal matrix A form orthonormal basis vectors
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [9]
Exercise Question for Today:
Which is unitary or orthogonal?






2-D DFT
Write the forward and inverse transform
Express in terms of matrix-vector form
Find basis images
(

u u
u u
cos sin
sin cos
(
(
(
(

2
1
2
1
2
1
2
1
(

2 1
3 2
inv(A1) = [2, 3; -1, 2]
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [10]
Properties of 1-D Unitary Transform y = A x
Energy Conservation
|| y ||
2
= || x ||
2

|| y ||
2
= || Ax ||
2
= (Ax)
*T
(Ax)= x
*T
A
*T
A x = x
*T
x = || x ||
2

Rotation
The angles between vectors are preserved

A unitary transformation is a rotation of a vector in an
N-dimension space, i.e., a rotation of basis coordinates
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [11]
Properties of 1-D Unitary Transform (contd)
Energy Compaction
Many common unitary transforms tend to pack a large fraction of signal
energy into just a few transform coefficients

Decorrelation
Highly correlated input elements quite uncorrelated output coefficients
Covariance matrix E[ ( y E(y) ) ( y E(y) )
*T
]
small correlation implies small off-diagonal terms

Example: recall the effect of DFT

Question: What unitary transform gives the best compaction
and decorrelation?

=> Will revisit this issue in a few lectures
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [12]
Review: 1-D Discrete Cosine Transform (DCT)
Transform matrix C
c(k,n) = o(0) for k=0
c(k,n) = o(k) cos[t(2n+1)/2N] for k>0
C is real and orthogonal
rows of C form orthonormal basis
C is not symmetric!
DCT is not the real part of unitary DFT! See Assignment#3
related to DFT of a symmetrically extended signal
N
k
N
N
k n
k k Z n z
N
k n
k n z k Z
N
k
N
n
2
) ( ,
1
) 0 (
2
) 1 2 (
cos ) ( ) ( ) (
2
) 1 2 (
cos ) ( ) ( ) (
1
0
1
0
= =

+
=
(

+
=

=
o o
t
o
t
o
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [13]
Periodicity Implied by DFT and DCT
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)
ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [14]
Example of 1-D DCT

100
50
0
-50
-100
0 1 2 3 4 5 6 7
n
z(n)
100
50
0
-50
-100
0 1 2 3 4 5 6 7
k
Z(k)
DCT
From Ken Lams DCT talk 2001 (HK Polytech)
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [15]
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

Example of 1-D DCT (contd)
k
Z(k)
Transform coeff.
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
Basis vectors
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100
100
0
-100 u=0
u=0 to 1
u=0 to 4
u=0 to 5
u=0 to 2
u=0 to 3
u=0 to 6
u=0 to 7
Reconstructions
n
z(n)
Original signal
From Ken Lams DCT talk 2001 (HK Polytech)
ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [17]
2-D DCT
Separable orthogonal transform
Apply 1-D DCT to each row, then to each column
Y = C X C
T
X = C
T
Y C = E
m
E
n
y(m,n) B
m,n
DCT basis images:
Equivalent to represent
an NxN image with a set
of orthonormal NxN
basis images
Each DCT coefficient
indicates the contribution
from (or similarity to) the
corresponding basis image

U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [18]
2-D Transform: General Case
A general 2-D linear transform {a
k,l
(m,n)}
y(k,l) is a transform coefficient for Image {x(m,n)}
{y(k,l)} is Transformed Image
Equiv to rewriting all from 2-D to 1-D and applying 1-D transform

Computational complexity
N
2
values to compute
N
2
terms in summation per output coefficient
O(N
4
) for transforming an NxN image!

=
=

=
) , ( ) , ( ) , (
) , ( ) , ( ) , (
1
0
1
0
,
1
0
1
0
,
n m h l k y n m x
n m a n m x l k y
N
k
N
l
l k
N
m
N
n
l k
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [20]
2-D Separable Unitary Transforms
Restrict to separable transform
a
k,l
(m,n) = a
k
(m) b
l
(n) , denote this as a(k,m) b(l,n)

Use 1-D unitary transform as building block
{a
k
(m)}
k
and {b
l
(n)}
l
are 1-D complete orthonormal sets of basis vectors
use as row vectors to obtain unitary matrices A={a(k,m)} & B={b(l,n)}

Apply to columns and rows Y = A X B
T
often choose same unitary matrix as A and B (e.g., 2-D DFT)

For square NxN image A: Y = A X A
T
X = A
H
Y A
*

For rectangular MxN image A: Y = A
M
X A
N

T
X = A
M
H
Y A
N
*


Complexity ~ O(N
3
)
May further reduce complexity if unitary transf. has fast implementation
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [21]
Basis Images
X = A
H
Y A
*
=> x(m,n) = E
k
E
l
a*(k,m)a*(l,n) y(k,l)
Represent X with NxN basis images weighted by coeff. Y
Obtain basis image by setting Y={o(k-k
0
, l-l
0
)} & getting X
{ a*(k
0
,m)a*(l
0
,n) }
m,n
in matrix form A*
k,l
= a*
k
a
l
*T
~ a*
k
is k
th
column vector of A
H

trasnf. coeff. y(k,l) is the inner product of A*
k,l
with the image

Exercise
Unitary transform or not?
Find basis images
Represent an image X with basis images
(Jains e.g.5.1, pp137)
(
(
(
(

=
2
1
2
1
2
1
2
1
A (

=
4 3
2 1
X
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [22]
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

Common Unitary Transforms and Basis Images
DFT DCT Haar transform K-L transform
See also: Jains Fig.5.2 pp136
ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [24]
2-D DFT
2-D DFT is Separable

Y = F X F X = F
*
Y F
*

Basis images B
k,l
= (a
k

)

(a
l

)
T

where a
k
= [ 1 W
N
-k
W
N
-2k
W
N
-(N-1)k
]
T
/ \ N

=
=

=
1
0
1
0
1
0
1
0
) , (
1
) , (
) , (
1
) , (
N
k
N
l
mk
N
nl
N
N
m
N
n
mk
N
nl
N
W W l k Y
N
n m X
W W n m X
N
l k Y
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [25]
Summary and Review (1)
1-D transform of a vector
Represent an N-sample sequence as a vector in N-dimension vector space
Transform
Different representation of this vector in the space via different basis
e.g., 1-D DFT from time domain to frequency domain

Forward transform
In the form of inner product
Project a vector onto a new set of basis to obtain N coefficients

Inverse transform
Use linear combination of basis vectors weighted by transform coeff. to
represent the original signal

2-D transform of a matrix
Rewrite the matrix into a vector and apply 1-D transform
Separable transform allows applying transform to rows then columns
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [26]
Summary and Review (1) contd
Vector/matrix representation of 1-D & 2-D sampled signal
Representing an image as a matrix or sometimes as a long vector

Basis functions/vectors and orthonomal basis
Used for representing the space via their linear combinations
Many possible sets of basis and orthonomal basis

Unitary transform on input x ~ A
-1
= A
*T

y = A x x = A
-1
y = A
*T
y = E a
i
*T
y(i) ~ represented by basis vectors {a
i
*T
}
Rows (and columns) of a unitary matrix form an orthonormal basis

General 2-D transform and separable unitary 2-D transform
2-D transform involves O(N
4
) computation
Separable: Y = A X A
T
= (A X) A
T
~ O(N
3
) computation
Apply 1-D transform to all columns, then apply 1-D transform to rows
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [29]
Optimal Transform
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [30]
Optimal Transform
Recall: Why use transform in coding/compression?
Decorrelate the correlated data
Pack energy into a small number of coefficients

Interested in unitary/orthogonal or approximate orthogonal transforms
Energy preservation s.t. quantization effects can be better understood and
controlled
Unitary transforms weve dealt so far are data independent
Transform basis/filters are not depending on the signals we are processing

What unitary transform gives the best energy compaction
and decorrelation?
Optimal in a statistical sense to allow the codec works well with
many images
Signal statistics would play an important role
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [31]
Review: Correlation After a Linear Transform
Consider an Nx1 zero-mean random vector x
Covariance (autocorrelation) matrix R
x
= E[ x x
H
]
give ideas of correlation between elements
R
x
is a diagonal matrix for if all N r.v.s are uncorrelated

Apply a linear transform to x: y = A x
What is the correlation matrix for y ?
R
y
= E[ y y
H
] = E[ (Ax) (Ax)
H
] = E[ A x x
H
A
H
]
= A E[ x x
H
] A
H
= A R
x
A
H


Decorrelation: try to search for A that can produce a
decorrelated y (equiv. a diagonal correlation matrix R
y
)
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [32]
K-L Transform (Principal Component Analysis)
Eigen decomposition of R
x
: R
x
u
k
=
k
u
k
Recall the properties of R
x

Hermitian (conjugate symmetric R
H
= R);
Nonnegative definite (real non-negative eigen values)

Karhunen-Loeve Transform (KLT)
y = U
H
x x = U y with U = [ u
1
,

u
N
]
KLT is a unitary transform with basis vectors in U being the
orthonormalized eigenvectors of R
x


U
H
R
x
U = diag{
1
,
2
, ,
N
} i.e. KLT performs decorrelation
Often order {u
i
} so that
1
>
2
> >
N


Also known as the Hotelling transform or
the Principle Component Analysis (PCA)
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [33]
Properties of K-L Transform
Decorrelation
E[ y y
H
]= E[ (U
H
x) (U
H
x)
H
]= U
H
E[ x x
H
] U

= diag{
1
,
2
, ,
N
}

Note: Other matrices (unitary or nonunitary) may also decorrelate
the transformed sequence [Jains e.g.5.7 pp166]

Minimizing MSE under basis restriction
If only allow to keep m coefficients for any 1s m sN, whats the
best way to minimize reconstruction error?
Keep the coefficients w.r.t. the eigenvectors of the first m largest
eigenvalues

Theorem 5.1 and Proof in Jains Book (pp166)
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
/
2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [34]
KLT Basis Restriction
Basis restriction
Keep only a subset of m transform coefficients and then perform
inverse transform (1s m s N)
Basis restriction error: MSE between original & new sequences


Goal: to find the forward and backward transform matrices
to minimize the restriction error for each and every m
The minimum is achieved by KLT arranged according to the
decreasing order of the eigenvalues of R

U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [35]
K-L Transform for Images
Work with 2-D autocorrelation function
R(m,n; m,n)= E[ x(m, n) x(m, n) ] for all 0s m, m, n, n s N-1
K-L Basis images is the orthonormalized eigenfunctions of R

Rewrite images into vector form (N
2
x1)
Need solve the eigen problem for N
2
xN
2
matrix! ~ O(N
6
)

Reduced computation for separable R
R(m,n; m,n)= r
1
(m,m) r
2
(n,n)
Only need solve the eigen problem for two NxN matrices ~ O(N
3
)
KLT can now be performed separably on rows and columns
Reducing the transform complexity from O(N
4
) to O(N
3
)
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [36]
Pros and Cons of K-L Transform
Optimality
Decorrelation and MMSE for the same# of partial coeff.
Data dependent
Have to estimate the 2
nd
-order statistics to determine the transform
Can we get data-independent transform with similar performance?
DCT

Applications
(non-universal) compression
pattern recognition: e.g., eigen faces
analyze the principal (dominating) components
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [37]
Energy Compaction of DCT vs. KLT
DCT has excellent energy compaction
for highly correlated data
DCT is a good replacement for K-L
Close to optimal for highly correlated data
Not depend on specific data like K-L does
Fast algorithm available

[ref and statistics: Jains pp153, 168-175]

U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [38]
Energy Compaction of DCT vs. KLT (contd)
Preliminaries
The matrices R, R
-1
, and oR
-1
share the same eigen vectors
DCT basis vectors are eigenvectors of a symmetric tri-diagonal matrix Q
c







Covariance matrix R of 1
st
-order stationary Markov sequence with has an
inverse in the form of symmetric tri-diagonal matrix
DCT is close to KLT on 1
st
-order stationary Markov
For highly correlated sequence, a scaled version of R
-1
approx. Q
c

U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
4
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [41]
Summary and Review on Unitary Transform
Representation with orthonormal basis Unitary transform
Preserve energy

Common unitary transforms
DFT, DCT, Haar, KLT

Which transform to choose?
Depend on need in particular task/application
DFT ~ reflect physical meaning of frequency or spatial frequency
KLT ~ optimal in energy compaction
DCT ~ real-to-real, and close to KLTs energy compaction

=> A comparison table in Jains book Table 5.3
U
M
C
P

E
N
E
E
6
3
1

S
l
i
d
e
s

(
c
r
e
a
t
e
d

b
y

M
.
W
u

2
0
0
1
)

ENEE631 Digital Image Processing (Spring'04) Lec12 Unitary Transform [43]