6.

869 Problem Set 3: Statistical models of images

Nikhil Naik naik@mit.edu

Retinex: improving the input for the simple visual system

1. Use the logarithmic representation of mean of RGB image as input i.e. for an RGB image I , Ilog = log(mean(I)) 2. Calculate the x and y derivatives of Ilog and apply a lower threshold on these images to eliminate the low amplitude values which correspond to the gradient changes in illumination. For instance, the thresholded gradient image in x direction, dIx is given by,
dIx =
Ilog x

The Retinex algorithm is applied as follows-

if

Ilog x

> th

else

(1)

3. The output image is obtained by deconvolving dIx and dIy with the derivative kernels. Following code-snippet implements this algorithm
1 2 3 4 5 6 7 8 9 10 11 12 13 14

img = mean(imread('.\images\simpleworld.jpg'),3); imgLog=img; imgLog(img==0)=1; imgLog=log(imgLog); % Derivation filters clear fn fn(:,:,1) = [0 0 0; -1 1 0; 0 0 0]; fn(:,:,2) = [0 0 0; -1 1 0; 0 0 0]'; %% % Derivate out = convFn(imgLog, fn); %Apply threshold th=0.1; out(abs(out)<th)=0;

Figure 1: Input and output images of Retinex algorithm with thresholded gradients in x and y direction

Figure 2: Two column proles after applying Retinex algorithm. The gradient changes due to illumination disappear as seen from the two plots
15 16 17

% Pseudo-inverse (using the trick from Weiss, ICCV 2001; equations 5-7) im=deconvFn(out,fn);

Figure 1 shows the input and output images along with the thresholded gradient images.
1.1 Removing the gradient due to illumination

According to Retinex algorithm, small gradients are classied as illumination while large gradients are classied as reectance. So by setting small gradients to zero, the eect of illumination is removed. This is shown in Figure 2 by plotting two columns of the image from both input and output images. As it can be seen, the white area which belongs to background, and is only aected by slowly varying illumination, becomes at after applying the retinex algorithm.

Wiener Filtering

120 images were selected by doing a random crawl on google images. They were natural images with dierent categories such as nature, buildings, vehicles and so on. An average amplitude spectrum was calculated as used as a prior for the Wiener ltering process. The following code-snippet performs Wiener ltering
1 2 3 4 5 6 7 8 9 10

function imgFilt=img_wiener(img,ps,sd) % img=Image corrupted with zero mean Gaussian noise % ps= the power spectrum of mean amplitude spectrum... % genereated from the dataset %sd= standard deviation of Gaussian noise. imgFFT=fft2(img); % FFT of noisy image imgWF=1./(1+((numel(img)*(sd^2))./ps)); %define Wiener filter imgFilt=ifft2(imgWF.*imgFFT); % Apply filter and get inverse FFT end

The following code-snipper performs Gaussian ltering

1 2 3 4 5 6 7 8

function imgFilt=img_gaussian(img,sd) %img: input image with added zero mean Gaussian noise %sd: standard deviation for the Gaussian Filter gFilt=fspecial('gaussian',size(img,1),sd); % define filter kernel imgFFT=fftshift(fft2(img)); imgFilt=ifft2(ifftshift(gFilt.*imgFFT)); %perform Gaussian Filtering end

Figure 3 and 4 show two images added with zero mean Gaussian noise and their Wiener ltered and Gaussian ltered results.

Histogram equalization
1 N
I

For an image with histogram h(I),we calculate the CDF given by

c(I) = h(i) = c(I 1) +
i=0

1 h(I) N

(2)

c(I).

The CDF is applied as compensation transfer function(CTF) i.e. f (I) =

The main script

%% Read image and convert to luminance img = imread('.\images\office_6.jpg'); img=im2double(img);

1 2 3 4

Figure 3: Image added with Gaussian noise with n = 10 and ltered with Wiener lter as well as Gaussian lter with s = 100. As it can be seen in the zoomed-in version on the right, the noise removal is similar but the high frequncy content such as hair and eyes is preserved better by the Wiener lter. Please see gures generated from the code for details

Figure 4: Image added with Gaussian noise with n = 20 and ltered with Wiener lter as well as Gaussian lter with s = 100. As it can be seen in the zoomed-in version on the right, the noise removal is similar but the high frequency content is similarly degraded in both cases due to the high n of noise. So Wiener lter does not perform signicantly better as compared to Gaussian lter in this case due to high n of noise content. 5

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

% provide fractional value for alpha for... %better results in case of saturated colors alpha=1; %Calculate Luminance image imgLuma=sqrt(img(:,:,1).^2+img(:,:,2).^2+img(:,:,3).^2); %Get Normalized RGB image rgbNorm=calcNormRGB(img,imgLuma);

%% Calculate histogram, CDF and compensation transfer fn (CTF) intLuma=uint8(255*(imgLuma/max(max(imgLuma)))); imgHeq=calcHisteq(intLuma,alpha); %Equalize luminance histogram eps=10^-3; % Ratio for individual color channel adjustment... % obtained from histogram equalization of luminance imgScale=double(imgHeq)./(double(intLuma)+eps); %Perform RGB adjustment using ratio values rgbHeq=adjustRGB(rgbNorm,imgScale,imgLuma); rgbHeq=uint8(255*rgbHeq);

Following function performs the histogram equalization

function imgHeq=calcHisteq(img,alpha) %% Calculate Histogram,CDF and CTF ch=size(img,3); imgHeq=uint8(zeros(size(img))); tempHeq=uint8(zeros(size(img(:,:,1)))); for i=1:ch data(i).imgHist=imhist(img(:,:,i)); % calculate histogram data(i).imgCDF=zeros(size(data(i).imgHist)); data(i).imgCDF(1)=data(i).imgHist(1); %initialize CDF % Calculate CDF for j=2:length(data(i).imgHist) data(i).imgCDF(j)=data(i).imgCDF(j-1)+data(i).imgHist(j); end; % Calculate CTF idnTF=0:255; %Identity transform idnTF=idnTF(:); %CTF=alpha*CDF+(1-alpha)*Identity data(i).imgCTF=uint8((alpha*255*(data(i).imgCDF/data(i).imgCDF(255)))... +(1-alpha)*idnTF);

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

%% Calculate the histogram-equalized image % Apply the CTF curve on the image for x=0:255 tempHeq(img(:,:,i)==x)=data(i).imgCTF(x+1); end;

Figure 5: Luminance channel, equalized luminance channel and histogramequalized RGB image
32 33 34 35

imgHeq(:,:,i)=tempHeq; clear tempHeq; end; end

Figure 6 shows the result obtained for a washed-out color image The RGB equalization function is as followsfunction adjImg=adjustRGB(img,ratio,amp) %% Calculate adjusted RGB images % amp: Amplitude/Luminance image; % ratio: the scale obtained from histogram equalization of luminance adjImg=zeros(size(img)); adjImg(:,:,1)=ratio.*amp.*img(:,:,1); adjImg(:,:,2)=ratio.*amp.*img(:,:,2); adjImg(:,:,3)=ratio.*amp.*img(:,:,3); end

1 2 3 4 5 6 7 8 9 10 11

Figure 6: The toys in the original image are saturated. In second image, with direct histogram equalization i.e. = 1, they get unnatural colors. In the last image, using = 0.3, we get more natural looking colors, with histogram equalized to some extent. .
3.1 Saturated color values

Color values that are clipped in the original image, i.e., have one or more saturated color channels, may appear unnatural when remapped to a non-clipped value. Extend your algorithm to handle this case in some useful way.

This case can be handled by only partially compensating for the histogram unevenness. This is done by using a mapping function as described by the function f (I) = c(I) + (1 )I (3) where I is an identity transform. The saturated colors look more natural by using a fractional value of as seen in Figure 6