You are on page 1of 35

Chapter 5 : Image Transforms

(from Anil. K. Jain)

CCU, Taiwan Wen-Nung Lie

2-D orthogonal and unitary transforms

Orthogonal series expansion for an NN image u(m, n)


v(k , l ) =

u(m, n)a
m =0 n =0

N 1

N 1

k ,l

( m, n ) ( m, n )

0 k,l N 1
0 m, n N 1

u ( m, n ) =

v(k , l )a
k =0 l =0

N 1

N 1

* k ,l

v(k, l)s are the transform coefficients, V {v(k , l )} represents the transformed image
{a k ,l ( m, n )} is a set of orthonormal functions,

representing the image transform


CCU, Taiwan Wen-Nung Lie 5-1

Orthonormality and completeness

{a k ,l ( m, n )} must satisfies
orthonormality :

a
m =0 n =0

N 1

N 1

k ,l

( m, n )a k*,l ( m, n ) = ( k k , l l )

completeness :

k =0 l =0

N 1

N 1

a k ,l ( m, n )a k*,l ( m , n ) = ( m m , n n )

CCU, Taiwan Wen-Nung Lie

5-2

Matrix representation of image transform

v(k , l ) =

u(m, n)a
m =0 n =0

N 1

N 1

k ,l

( m, n )

U=

v(k , l )A
k =0 l =0

N 1

N 1

* k ,l

u ( m, n ) =

k =0 l =0

N 1

N 1

v ( k , l )a k*,l ( m, n )
*

v(k , l ) =< U, A*k ,l >

< F, G >=

f (m, n) g (m, n)
m=0 n =0

N 1

N 1

is the matrix inner product

CCU, Taiwan Wen-Nung Lie

Image U can be described as a linear combination * 2 A of N matrix k ,l , k, l = 0,...,N-1 A*k ,l are called the basis images v(k, l) can be considered as the projection of u(m, n) on the (k, l)-th basis image
5-3

Basis images

CCU, Taiwan Wen-Nung Lie

5-4

2-D Separable image transformation

{ak ,l (m, n)} is separable


a k ,l (m, n) = a k (m)bl (n) a ( k , m)b(l , n)
v( k , l) =
m= 0 n = 0 N 1 N 1

a ( k , m)u(m, n)a (l , n)
* * k =0 l =0

N 1 N 1

V = AUAT

*T * U = A VA u(m, n) = a ( k , m)v ( k , l )a (l , n)

A {a(k , m)} B {b(l , n)} are unitary matrices


Consider the above as transforming columns of U by A, then transforming rows of the result by A T
5-5

CCU, Taiwan Wen-Nung Lie

Example of image transform

Given

1 1 1 A= , 1 1 2

1 2 U= 3 4

compute transform Basis images


*

1 1 1 1 2 1 1 5 1 V= = 2 1 1 3 4 1 1 2 0

A 0, 0
*

1 1 1 1 1 (1 1) = = 2 1 1 2 1
1 1 1 A 1, 1 = 2 1 1
*

1 1 1 1 1 *T A 0 ,1 = (1 1) = = A 1, 0 1 1 2 2 1
*T *

CCU, Taiwan Wen-Nung Lie

1 1 1 5 11 1 1 2 A VA = = 2 1 1 2 0 1 1 3 4

5-6

Properties of unitary transform

Given v = Au (A is a unitary transform)

v = u
2

, i.e., energy-conserved

a unitary transformation is a rotation of the basis coordinates , or a rotation of the vector u in NDimensional vector space Unitary transform tends to concentrate most of the signal energy on a small set of transform coefficients

v = A u

R v = AR u A*T
5-7

CCU, Taiwan Wen-Nung Lie

Properties of unitary transform (cont)

The transform coefficients tends to be decorrelated, i.e., the off-diagonal elements of R v are approaching zero The entropy of a random vector is preserved under a unitary transformation (i.e., information preserved)

CCU, Taiwan Wen-Nung Lie

5-8

1-D DFT

1-D DFT pair (unitary transformation)


v(k ) = u ( n) = 1

N
n =0

N 1

u ( n)WNkn v(k )WNkn

k = 0,..., N 1 n = 0,..., N 1

N
k =0

N 1

NN transform matrix F for v = Fu


1 kn F= WN , N 0 k, n N 1
5-9

j 2 WN exp N

CCU, Taiwan Wen-Nung Lie

Properties of DFT

Symmetry F = F T F 1 = F * (unitary : F 1 = F *T ) FFT needs O ( N log 2 N ) operations (DFT needs O ( N 2 ) ) Real DFT is conjugate symmetrical about N/2
v* ( N N k ) = v( + k ) 2 2

x 2 ( n ) is the circular convolution between h ( n ) and x1 ( n )

DFT {x 2 ( n )}N = DFT {h ( n )}N DFT {x1 ( n )}N

Extend the length of h ( n ) ( N ) and x1 ( n ) (N) with zeros to have the same length ( M N + N 1 ), the above equation can be used to compute linear convolution

CCU, Taiwan Wen-Nung Lie

5-10

2-D DFT

2-D DFT is a separable transform


1 v(k , l ) = N
1 u ( m, n ) = N

m =0 n =0
N 1 N 1 k =0 l =0

N 1

N 1

u ( m, n )WNkmWNln
v ( k , l )WNkmWNln

0 k,l N 1
0 m, n N 1

In matrix form
V = FUF T = FUF

CCU, Taiwan Wen-Nung Lie

5-11

Properties of 2-D DFT


Symmetric and unitary F = F and F = F Periodic v (k + N , l + N ) = v (k , l ) V = FUF needs 2N times of 1-D FFT DFT of real images is conjugate symmetrical with respect to ( N , N )
T
1 *

v(k , l ) = v * ( N k , N l )

N 2 basis images
1 {WN( km+ln) ,0 m, n N 1} N 0 k,l N 1
5-12

CCU, Taiwan Wen-Nung Lie

Plots of DFT

CCU, Taiwan Wen-Nung Lie

5-13

Cosine transform

DCT (discrete cosine transform)


1 , k = 0,0 n N 1 N c ( k , n) = 2 cos (2n + 1)k , 1 k N 1,0 n N 1 2N N

88 2-D DCT

CCU, Taiwan Wen-Nung Lie

5-14

Properties of DCT

DCT is real and orthogonal C = C and C = C DCT Real {DFT} DCT can be calculated via FFT
*
1

~( n )} ] v ( k ) = Re[ ( k )W2kN/ 2 DFT {u N


~ ( n ) = u ( 2n ) u N 0 n ( ) 1 ~ 2 u ( N n 1) = u ( 2n + 1)

CCU, Taiwan Wen-Nung Lie

For highly correlated data, DCT has good energy compaction. That is, for first-order stationary Markov sequence with correlation 1, DCT is approximate to KL transform
5-15

Sine transform

DST (discrete sine transform)


(k , n) = ( k + 1)( n + 1) 2 , sin N +1 N +1
0 k, n N 1

Properties

DST is real, symmetric, and orthogonal. = = DST and DST-1 are the same in the form (cf. DCT) DST Imagery {DFT} For first-order stationary Markov sequence with correlation (-0.5, 0.5), the energy compaction of DST is approximate to KL transform
*

= 1

CCU, Taiwan Wen-Nung Lie

5-16

(Walsh-) Hadamard transform

Hadamard matrix
1 1 1 H1 = 1 1 2

1 H n1 H n = H n1 H1 = 2 H n1

Kronecker operator

H n1 H n1

Components of HT vector contain only 1 and -1

The number of transitions from 1 to -1 is called sequency (like in the continuous case)
1 1 1 1 1 H3 = 1 8 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

CCU, Taiwan Wen-Nung Lie

0 7 sequency 3 Not in order 4 1 6 2 5 5-17

(Walsh-) Hadamard transform (cont.)

Hadamard transform pair :


v = Hu u = Hv
b( k , m ) =
n 1 i

v(k ) =

u ( m )( 1) N
m =0

N 1

b ( k ,m )

0 k N 1

u(m) =
i

v ( k )( 1) N
k =0

N 1

b ( k ,m )

0 m N 1

k m
i =0

k i , mi = 0,1

Notice the disordering effect in the sequency in HT spectrum when filtering will be performed

CCU, Taiwan Wen-Nung Lie

5-18

Properties of Hadamard transform


Real, symmetric, and orthogonal H = H = H = H Fast computation (only addition is needed) Can be decomposed into product of sparse matrix
* T

~n H = Hn = H
0 1 . . 0 . . 0 1 . . 0 . . . . . 0 . . . . . . . 1 . . . . . . . . 1

n = log 2 N
. . . 1 . . . 1

1 1 0 0 . . ~ 1 0 0 H= 1 1 2 0 0 . . 0 0

~n ~~ ~ v = Hu = H u = HH ...Hu

1 1 0 . .

For highly correlated images, Hadamard transform also has good energy compaction
5-19

CCU, Taiwan Wen-Nung Lie

Plot of Hadamard transform

CCU, Taiwan Wen-Nung Lie

5-20

KL transform of 1-D vector

For a real N1 random vector u with autocorrelation matrix R, KL transform of u is defined as


v = *T u
u = v

where R k = k k , 0 k N 1 , i.e., ks are orthonormalized eigenvectors of R and k is the k-th column of reduces R to its diagonal form, i.e.,
*T R = = Diag{k }

CCU, Taiwan Wen-Nung Lie

If we replace R with R 0 R T, then eigenmatrix of R 0 is the KL transform of u- KL transform is independent of the image data themselves, but related to its 2nd-order statistics
5-21

KL transform of 2-D image


FFT, DCT, DST all are special cases of KL transform For an image u(m, n) of NN pixels, its autocorrelation function denoted as r ( m, n; m , n ) (or an N N R matrix), the basis images of the KL transform are k ,l ( m, n ) :
2 2

m=0 n=0

N 1

N 1

r ( m, n; m , n ) k ,l ( m , n ) = k ,l k ,l ( m, n )

0 k,l N 1
0 m, n N 1

or

Ri = ii
2

i = 0,..., N 2 1

i is the N 1 vector representation of k ,l ( m, n )

CCU, Taiwan Wen-Nung Lie

5-22

KL transform of 2-D image (cont.)

If R matrix is separable, then = {i } of N N is also separable Kronecker R = R1 R 2 = 1 2 operator


2 2

where

j R j*jT = j ,

j = 1,2
*T * 2 1

*T v = u or V = U KL transform :

Advantages of this separability : reduce the eigenvalue problem of R matrix ( N N ) into two eigenvalue problems of R1, R2 ( N N )
2 2

CCU, Taiwan Wen-Nung Lie

5-23

Properties of the KL transform

Decorrelation
{v ( k ), k = 0,..., N 1} are uncorrelated and have zero mean is not unique, not necessarily be the eigenmatrix of R

Minimum basis restriction error


u

A NN

Im 1mN

B NN

Signal u is transformed via A to be v, clip the first m values by Im to form w, and transformed back to z via BThe basis restriction error is defined as

1 J m E N
CCU, Taiwan Wen-Nung Lie

n =0

N 1

2 1 u (n) z (n) = Tr[ E{(u z )(u z )*T }] N

5-24

Properties of the KL transform (cont.)


Find A KL and B KL (KL transform) such that J m is minimized for each value of m [1, N ] . (KL is effective in compacting the energy) Select A = *T and B = , AB = I , and columns of are sorted according to their corresponding eigenvalues 0 1 2 ..... N 1 In a noisy channel, given a distortion specification,
D= 1 E[(u u. )*T (u u. )] N

Minimum transmission rate at a given distortion

KL transform can, among all unitary transformation A (include A = I ), achieve the minimum transmission rate R (*T ) R ( A )
u
CCU, Taiwan Wen-Nung Lie

A (unitary)

v channel

v. A*T

u.
5-25

An example of KL-transform

1 1 1 = = 1 1 2 2 2 E {[ v ( 0 )] } = = 1 + , E {[ v ( 1 )] } =1 v = u 0 1 1 1+ 1 1 ) + max(0, log )] R() = [max(0, log 2 2 2 comparing R(I ) = 1 [2 log ], 0 < <1 4
*T

A 21 vector u, whose covariance matrix is 1 R= <1 , 1 The KL transform is

CCU, Taiwan Wen-Nung Lie

R () < R (I )

Let be small, say < 1

5-26

Comparison of energy distribution between transforms

For first-order stationary Markov process with large , DCT is comparable to KL With small , DST is comparable to KL For any , we can find a faster sinusoidal transform (DCT or DST) to substitute for the optimal KL transform which needs covariance matrix to compute transform basis vectors

CCU, Taiwan Wen-Nung Lie

5-27

CCU, Taiwan Wen-Nung Lie

5-28

CCU, Taiwan Wen-Nung Lie

5-29

Singular Value Decomposition (SVD)

For an NM matrix U (M N), we can find r orthogonal M1 eigenvector { m } by using U T U , or find r orthogonal N1 eigenvector { m } by using UUT (r is the rank of U) UT U m = m m , m = 1,..., r UUT and UTU have the

Um=km

UU T m = m m ,

m = 1,..., r

same eigenvalues

Matrix U can be described as

U = 1 / 2 T =
1/ 2 =

1
. . 0

. .

. .

CCU, Taiwan Wen-Nung Lie

0 r

r m =1

m m T m

Note. In KL, R is symmetrical, but U is not

Called the spectral representation, outer product expansion, or singular value decomposition (SVD) of U
5-30

Singular Value Decomposition (cont)


Note: U is not symmetrical

SVD of U (diagonalization) : 1/ 2 = T U

U can be described as the composition of r matrix (corresponding to different m , called Us singular values)

If U is an NM image, U can be described as an NM1 vector, also as {1m/ 4 m , 1m/ 4 m ; m = 1,..., r} needing totally (M+N)r data and be more efficient than NM or NM1 if r << M

CCU, Taiwan Wen-Nung Lie

5-31

Properties of SVD

We can compute

U m , m = 1,...r

, when knowing

due to the same eigenvalue Energy conservation E = u 2 ( m, n ) = m


m
k

U = 1 / 2 T

T Denote U k m m m , k r, as Us k partial-sum ( m is m =1 ordered) U k is the optimal rank-k description, with approximation error

k = u ( m, n ) uk ( m, n ) = m ,
M N r 2 m =1 n =1 m = k +1

k = 1,..., r

The energy of each SVD spectral band


CCU, Taiwan Wen-Nung Lie

m m T is m m
5-32

Properties of SVD (cont)

For a given image, SVD can compact the largest energy for a specified number of transform coefficients (optimized independently)

The maximal number of basis images for SVD is N, while for other transforms, it is N2

For KL transform, it optimizes one set of ensemble with the same covariance matrix, resulting in the largest average energy in probabilistic sense

CCU, Taiwan Wen-Nung Lie

5-33

Properties of SVD (cont)

Example :
1 2 U= 2 1 1 3

1 = 18.06 2 = 1.94

4.25 0 SVD : = 0 1 . 39
1 2

0.8649 0.5019 1 = , 2 = 0.5019 0 . 8649

1.120 1.94 0 . 935 1 . 62 U 1 = 11 T = 1.549 2.70


1

2 1 = Using DCT : V = C 3 UCT 3 2 12 1

10 2 2 2 1 2 1 1 = 1 3 0 3 2 1 1 1 12 1 2 1 1 3

2 2 3 5

CCU, Taiwan Wen-Nung Lie

k l

v 2 ( k , l ) = 1 + 2
5-34

You might also like