CV Restoration

Computer Vision -
Restoration
Hanyang University

Jong-Il Park

Department of Computer Science and Engineering, Hanyang University
Restoration vs. Enhancement
Restoration
Objective process

A priori knowledge on
degradation model

Modeling the degradation and
applying the inverse process
to recover the original
To improve an image in some predefined sense
Enhancement
Subjective process

Restoration process

Noise models
Assume noise is independent
of spatial coordinates and it is
uncorrelated w.r.t. the image.
Gaussian: electronic circuit noise,
sensor noise
Rayleigh: range images
Exponential and gamma:
laser images
impulse(salt-and-pepper):
faulty switching

Eg. Sample noisy images

Eg. Sample noisy images(cont.)

Periodic noise
Spatially dependent noise
Periodic noise can be
reduced significantly via
frequency domain filtering

Estimation of noise parameters
PDF from small patches

When the only degradation is noise
Periodic noise subtraction gives a good result
Random noise mean filter, order-statistics filter,
) , ( ) , ( ) , (
and
) , ( ) , ( ) , (
v u N v u F v u G
y x y x f y x g
+ =
+ = q

Mean filters
Arithmetic mean filters
For Gaussian or uniform noise
Geometric mean filters
For Gaussian or uniform noise
Harmonic mean filters
Work well for salt noise but fail for pepper noise
Contraharmonic mean filters
Suited for impulse noise but require identification(salt
or pepper)

Arithmetic & Geometric mean filter

Contraharmonic filters
Q<0 : eliminates salt noise
Q=-1 harmonic mean filter
Q=0 : arithmetic mean filter
Q>0: eliminates pepper noise

e
e
+
=
xy
xy
S t s
Q
S t s
Q
t s g
t s g
y x f
) , (
) , (
1
) , (
) , (
) , (

Eg. Contraharmonic filters

Wrong sign in contraharmonic filters
Disaster!

Order-Statistics filters
Median filter
Max filter
Min filter
Midpoint filter
Alpha-trimmed mean filter

Median filters
3x3
median
3x3
median
3x3
median
blurred

Max and Min filter
Max filter Min filter
Removes pepper noise
Removes dark pixels
Removes salt noise
Removes light pixels
Makes dark objects larger

Eg. Comparison
(a) Additive uniform
noise
(b) (a)+additive S&P
5x5 arithmetic
mean
5x5 geometric mean
5x5 median 5x5 alpha-trimmed
Mean(d=5)

Adaptive filters
Behavior changes locally based on statistical
characteristics of local support

Simple adaptive filter based on mean and variance
1. If global_var is zero, then f(x,y)=g(x,y)
2. If local_var>global_var, then f(x,y)=g(x,y) (high local
var edge should be preserved)
3. If local_var==global_var, then arithmetic mean filtering

Eg. Adaptive filter

Adaptive median filter
Cope with impulse noise with large probability
Preserve detail while smoothing non-impulse noise
Level A: A1=z
med
-z
min
A2=z
med
-z
max
If A1>0 AND A2<0, go to level B
Else increase the window size
If window size<=S
max
repeat level A
Else output z
xy

Level B: B1=z
xy
-z
min
B2=z
xy
-z
max
If B1>0 AND B2<0, output z
xy
Else output z
med
Algorithm

Eg. Adaptive median filter
median adaptive median

Periodic noise reduction
By frequency domain filtering
Band reject filter

Eg. Periodic noise reduction

Noise extraction
By bandpass filter
Help understanding noise pattern

Notch filters

Eg. Notch filtering
Removing sensor scan-
line patterns

Optimum notch filtering
First isolating the principal contributions of the
interference pattern
Then subtracting weighted portion of the pattern from
the corrupted image

Eg. Periodic interference(1/3)
Noisy image

Extraction of noise interference pattern

Restored image by subtracting weighted portion of
periodic interference (Refer to the derivation of
weights in pp.250-252)

Linear, Position-Invariant Degradation
) , ( ) , ( ) , ( ) , (
: Domain Frequency
) , ( ) , ( ) , ( ) , (
: Domain Spatial
v u N v u F v u H v u G
y x y x f y x h y x g
+ =
+ - = q

Degradation knowledge
Degradation knowledge about
1. A priori (known)
2. A posteriori (unknown)

blind restoration
or blind deconvolution
f
g
H
Restoration:
determine the original image ,
given the observed image and
knowledge about the degradation (H).

Fundamental issue
Restoration problem

restoration is to find , such that
but, 1. does not exist: singular
2. may exist, but not be unique: ill-conditioned
3. may exist and unique, but there exists ,
which can be made arbitrarily small, such that
which is not negligible

Image restoration is ill-conditioned at best and
singular at worst

g f T = } {
1
T
f g T =
} {
1
1
T
1
T
1
T c
, , } {
1
c o o c >> + = +
f g T

Estimation of degradation function
Approaches

Observation

Experimentation

Mathematical modeling

Estimation by observation
Looking at a small section of the image containing
simple structures and then obtaining degradation
function

Observed sub-image:
Estimate of original image:

) , ( y x g
s
) , (
y x f
s
) , (
) , (
) , (
v u F
v u G
v u H
s
s
s
=

Estimation by experimentation
Possible only if equipment similar to the equipment
used to acquire the degraded images is available

Eg. Use an impulse

Estimation by modeling
Based on either physical characteristics or basic
principles

Eg.1. Physical characteristics: atmospheric turbulence

Eg.2. Math derivation: motion blur
Starting from

After some manipulation(p.259)

Setting the motion model, we obtain the degradation
func.

6 / 5
) (
2 2
) , (
v u k
e v u H
+
=
dt e v u H
T
t vy t ux j
}
+
=
0
)] ( ) ( [ 2
0 0
) , (
t
}
=
T
dt t y y t x x f y x g
0
0 0
)] ( ), ( [ ) , (

Eg.1. Physical model
Atmospheric turbulence

Eg.2. Math modeling
Motion blur

Restoration methods
Inverse filtering
Wiener filtering
Constrained least square filtering
Geometric mean filtering
Etc..

Inverse filtering

Poor performance!
Very sensitive to noise

) , ( ) , ( ) , ( ) , ( v u N v u F v u H v u G + =
) , (
) , (
) , ( ) , (
v u H
v u N
v u F v u F + =
) , (
) , (
) , (
v u H
v u G
v u F =
Noise amplification
when H(u,v) is small

Eg. Inverse filtering

Minimum mean-square error filter
Necessary to handle noise explicitly
Statistical characteristics of noise should be
incorporated into the restoration process
MMSE filter
To find an estimate of the uncorrupted image
such that the mean square error between them is
minimized:

Assume:
the noise and the image are uncorrelated
The one or the other has zero mean
The gray levels in the estimate are a linear function of
the levels in the degraded image
Derivation: Homework
} )
{(
2 2
f f E e =
f
f

MMSE filter (cont.)
Frequency domain expression:

Approximation of the Wiener filter
) , (
) , ( / ) , ( | ) , ( |
) , ( *

) , (
) , ( | ) , ( | ) , (
) , ( ) , ( *
) , (
2
2
v u G
v u S v u S v u H
v u H
v u G
v u S v u H v u S
v u S v u H
v u F
f
f
f
(
(
+
=
(
(
+
=
q
q
Wiener filter
) , (
| ) , ( |
| ) , ( |
) , (
1
) , (
2
2
v u G
K v u H
v u H
v u H
v u F
(
+
=
PS of noise PS of image f

Eg. Wiener filtering
Using the approximation
K is chosen interactively

Eg. Restoration by Wiener filter
motion blur
Severe noise
Moderate noise
Negligible noise

Constrained Least Square Filtering
Difficulty in Wiener filter
The power spectra of the undegraded image and noise
must be known
Minimization in a statistical sense

The constrained LS filtering
requires knowledge of
Mean of the noise
Variance of the noise
Optimal result for each image

Vector-matrix form of convolution
g: MN-vector (lexicographical order of an image)
f: MN-vector
H: MNxMN matrix
Hf g + =

Formulation: Constrained LS filter

To find the minimum of a criterion function C defined as

subject to the constraint

where is the Euclidean vector norm

| |
=
V =
1
0
1
0
2
2
) , (
M
x
N
y
y x f C
2 2
|| || ||
|| f H g =
w w w
T
2
|| ||

Freq. Domain Sol.
: adjustable parameter
: Fourier transform of the Laplacian operator
) , (
| ) , ( | | ) , ( |
) , ( *
) , (
2 2
v u G
v u P v u H
v u H
v u F
(
+
=
) , ( v u P

Eg. Constrained LS filter
Significant improvement over Wiener filter

Procedure for computing
Define a residual vector

Adjust so that

Calculation
a = =
2 2
|| || || || ) ( r |
f H g r

=
] [ || ||
2 2 2
q q
o m MN + =
In general, automatically determined restoration filter
yields inferior results to manual adjustment of filter
parameters
iteration

CV Restoration

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CV Restoration

Uploaded by

Copyright:

Available Formats

Computer Vision -

You might also like