DEPARTMENT OF INFORMATION SCIENCE & ENGINEERING

FACE RECOGNITION
U S I N G S I F T F E AT U R E S
Presented by, Nouman Sadiq(1PI08IS055)

Internal guide, Prof. Shylaja S S 1 H.O.D Department of ISE PESIT, Bangalore

Deep Agarwal (1PI08IS034)

Department of ISE PESIT, Bangalore

AGENDA
1 2
3 4 5 6

INTRODUCTION PROJECT BRIEF

IMPLEMENTATION CONCLUSSION

FURTHER ENHANCEMENT
BIBLIOGRAPHY

2 Aug 2012 Dec 2012

Department of Information Science Eng.

PROJECT OVERVIEW
Goal of this project? To extract Scale Invariant Features from a face that can be further used to perform reliable face recognition.
Scope of this project? Distinctive invariant features from a face is obtained. It is applied to the training set of the faces, thus transforming it. The face in the test set is matched.
Aug 2012 Dec 2012 Department of Information Science Eng.

Face recognition??
Face recognition can be categorized into three steps Face detection Feature extraction Face recognition Face recognition is always prone to problems like Change of posture Illumination changes Change of environment
4 Aug 2012 Dec 2012

Department of Information Science Eng.

Introduction - S.I.F.T
Stands

for Scale Invariant Feature Tranformation G. Lowe Introduced this algorithm.

David

Here

image features having properties which makes them suitable for matching differing images of the same face are extracted.
5

Aug 2012 Dec 2012

Department of Information Science Eng.

SIFT ALGORITHM OVERVIEW

Following are the major stages of computation used to generate the set of image features:
1.

Scale-space extrema detection

2.

Keypoint localization
Orientation Assignment

3.

4.

Keypoint Descriptor construction

6

Aug 2012 Dec 2012

Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION
It involves the following steps 1.1 : The face is expressed as octaves of different size. I(x,y) 1.2 : Each images of an octave is smoothed with different scales of the Gaussian function. L(x,y) 1.3 : Compute difference of Gaussian (doG) images from adjacent scales for entire octave D(x,y). 1.4 :From difference-of-Gaussian local extrema detection we obtain approximate values for keypoints (or interesting points)

The input image

The input image with interesting 7 points

Aug 2012 Dec 2012 Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.1 : The face is expressed as octaves of different size

Four Different Octaves

8 Aug 2012 Dec 2012

Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.2 : All the octave images are filtering with Gaussian function thus obtaining different scales.

The scale space of image L(x, y, ), that is produced from the convolution of a variable-scale Gaussian, G(x, y, )with an input image, I(x, y):

Where

9 Aug 2012 Dec 2012

Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.2 : All the octave images are filtering with Gaussian function thus obtaining different scales.
Five Different scales of Gaussian blur()

The second Octave of L images

Aug 2012 Dec 2012 Department of Information Science Eng.

10

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.3 : The difference of the different scale images is found.

The second Octave of D images

Aug 2012 Dec 2012 Department of Information Science Eng.

11

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.3 : The difference of the different scale images is found.

12 Aug 2012 Dec 2012

Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION

13 Aug 2012 Dec 2012

Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.4 :From difference-of-Gaussian local extrema detection we obtain approximate values for keypoints

Maxima and minima of the difference-of-Gaussian images are detected by comparing a pixel (marked with X) to its 26 neighbours in 3x3 regions at the current and adjacent scales (marked with circles)
14 Aug 2012 Dec 2012

Department of Information Science Eng.

STEP1:SCALE-SPACE
EXTREMA DETECTION
1.4 :From difference-of-Gaussian local extrema detection we obtain approximate values for keypoints

15 Aug 2012 Dec 2012

Department of Information Science Eng.

STEP 2:KEYPOINT
LOCALIZATION
Here the keypoints are selected based on measures of their stability.

It involves the following steps 2.1 : The interesting points of very Low contrast are removed 2.2 : Some more points are removed which threshold on ratio of principal curvatures.
The image with the interesting points

The image with 16 the keypoints

Aug 2012 Dec 2012 Department of Information Science Eng.

STEP 2:KEYPOINT
LOCALIZATION
2.1 : The interesting points of very Low contrast are removed

The image with the interesting points

The image with 17 the keypoints

Aug 2012 Dec 2012 Department of Information Science Eng.

STEP 2:KEYPOINT
LOCALIZATION
2.2 : Some more points are removed which threshold on ratio of principal curvatures.

Difference-of-Gaussian function will be strong along edges Some locations along edges will have a large principal curvature across the edge but a small principal of curvature perpendicular to the edge Therefore we need to compute the principal curvatures at the location and compare the two. Then eliminate some of the candidates below threshold 18
Aug 2012 Dec 2012 Department of Information Science Eng.

STEP 2:KEYPOINT
LOCALIZATION

The image with interesting points

The image with 19 the keypoints

Aug 2012 Dec 2012 Department of Information Science Eng.

STEP 2:KEYPOINT
LOCALIZATION

436 interesting points

The points below contrast threshold is eliminated.410 points eliminated 26 interesting points are left

Further points whose curvature ratio is above the threshold is also removed. 13 more points are removed. The final face with the 13 keypoints.
20

Aug 2012 Dec 2012

Department of Information Science Eng.

STEP 3:ORIENTATION ASSIGNMENT

Orientation assigned to each keypoint and neighboring pixels based on local gradient.

3.1: Points in region around

keypoint are selected and magnitude and orientations of gradient are calculated.

The image with Keypoints

The image with 21 the keypoints

Aug 2012 Dec 2012 Department of Information Science Eng.

STEP 3:ORIENTATION ASSIGNMENT

Points in region around keypoint are selected and magnitude and orientations of gradient are calculated.

For image sample, L(x, y), the gradient magnitude, m(x, y), and orientation, teta(x, y), is computed using pixel differences:
m( x, y) ( L( x 1, y) L( x 1, y))2 ( L( x, y 1) L( x, y 1))2

( x, y ) tan 1 (( L( x, y 1) L( x, y 1)) /( L( x 1, y ) L( x 1, y )))

22 Aug 2012 Dec 2012

Department of Information Science Eng.

TRAINING FACE SET

23 Aug 2012 Dec 2012

Department of Information Science Eng.

TEST FACE SET

24 Aug 2012 Dec 2012

Department of Information Science Eng.

EXTRACTING KEYPOINTS
Video 1

25 Aug 2012 Dec 2012

Department of Information Science Eng.

EXTRACTING KEYPOINTS
Video 2

26 Aug 2012 Dec 2012

Department of Information Science Eng.

MATCHING
Two images of the faces of a same person. 10 keypoints are matched

27 Aug 2012 Dec 2012

Department of Information Science Eng.

MATCHING
Two images of the faces of a different persons. 0 keypoints are matched

28 Aug 2012 Dec 2012

Department of Information Science Eng.

CONCLUSION
In this project Scale Invariant Feature Transform (SIFT) is implemented for feature extraction for the purpose of face recognition.

The Comparisons of this approach among other holistic approaches and feature based approaches are yet to be done .

Aug 2012 Dec 2012

Department of Information Science Eng.

29

Further Enhancement
There are many directions for further research in deriving invariant and distinctive face Features, further distinctiveness could be derived from including illumination-invariant color descriptors . Another direction for future research will be to individually learn features that are suited to recognizing particular categories of facial images.

Aug 2012 Dec 2012

Department of Information Science Eng.

30

REFERENCES
[1] D. Lowe. Distinctive image features from scale-invariant keypoints. Int. Journal of Computer Vision, 60(2):91110, January 5,2004. [2] Gary Bradski and Adrian Kaehler, Learning OpenCV computer vision with the OpenCV library, Printed in the United States of America.Published by OReilly Media, Inc. September 2008: First Edition. [3] David G. Lowe, Object Recognition from Local Scale-Invariant Features. Int. Conf. on Computer Vision, pages 11501157,1999.

Aug 2012 Dec 2012

Department of Information Science Eng.

31

Aug 2012 Dec 2012

Department of Information Science Eng.

32