FACE RECOGNITION USING EIGENFACE APPROACH

VINAY HIREMATH Malardalen University, Vasteras, Sweden. vhh09001@student.mdh.se

ASHWINI MAYAKAR Malardalen University, Vasteras, Sweden. amr09002@student.mdh.se

ABSTRACT:
Face recognition can be applied for a wide variety of problems like image and film processing, human-computer interaction, criminal identification etc. This has motivated researchers to develop computational models to identify the faces, which are relatively simple and easy to implement. The existing system represents some face space with higher dimensionality and it is not effective too. The important fact which is considered is that although these face images have high dimensionality, in reality they span very low dimensional space. So instead of considering whole face space with high dimensionality, it is better to consider only a subspace with lower dimensionality to represent this face space. The goal is to implement the system (model) for a particular face and distinguish it from a large number of stored faces with some real-time variations as well. The Eigenface approach uses Principal Component Analysis (PCA) algorithm for the recognition of the images. It gives us efficient way to find the lower dimensional space. Further this algorithm can be extended to recognize the gender of a person or to interpret the facial expression of a person. Recognition could be carried out under widely varying conditions like frontal view, a 45 view, scaled frontal view, subjects with spectacles etc. are tried, while the training data set covers limited views. The algorithm models the real-time varying lighting conditions as well. But this is out of scope of the current implementation. Key Words: Eigen faces, Eigen Vectors, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA).

1. INTRODUCTION:
This paper is a step towards developing a face recognition system which can recognize static images. It can be modified to work with dynamic images. In that case the dynamic images received from the camera can first be converted in to the static ones and then the same procedure can be applied on them. The scheme is based on an information theory approach that decomposes face images into a small set of characteristic feature images called Eigenfaces, which are actually the principal components of the initial training set of face images. Recognition is performed by projecting a new image into the subspace spanned by the Eigenfaces (face space) and then classifying the face by comparing its position in the face space with the positions of the known individuals. The Eigenface approach gives us efficient way to find this lower dimensional space. Eigenfaces are the Eigenvectors which are representative of each of the dimensions of this face space and they can be considered as various face features. Any face can be expressed as linear combinations of the singular vectors of the set of faces, and these singular vectors are eigenvectors of the covariance matrices. In general, face recognition techniques can be divided into two groups based on the face representation they use: 1. Appearance-based, which uses holistic texture features and is applied to either whole-face or specific regions in a face image. 2. Feature-based, which uses geometric facial features (mouth, eyes, brows, cheeks etc) and geometric relationships between them? Among many approaches to the problem of face recognition, appearance-based subspace analysis is one of the oldest, still gives the most promising results. Subspace analysis is done by projecting an image into a lower dimensional space (subspace) and after that recognition is performed by measuring the distances between known images and the image to be recognized. The most challenging part of such a system is finding an adequate subspace. When using appearance-based methods, we usually represent an image of size n x m pixels by a vector in an n x m dimensional space. These (n x m

dimensional) spaces are too large to allow robust and fast object recognition. A common way to attempt to resolve this problem is to use dimensionality reduction techniques. PCA & LDA are the two most popular techniques usually used for dimensionality reduction. In this paper we make use of PCA dimensionality reduction method. The organization of the paper can be summarized as: The Eigenface approach is briefly explained in Section-2.The distinguishing characteristics of PCA & LDA are discussed in Section-3. The working of Face Recognition system using Eigen values is briefly explained in Section-4. The Significance of Eigen face approach is explained in Section-5. Summary is briefly explained in Section-6.Conclusions & Future work explained in Section-7

Repeated Eigenvalues:

In the case where there are r repeated eigenvalues, then a linearly independent set of n eigenvectors exist, provided the rank of the matrix (A-I) (2.5) is rank n-r . Then, the directions of the r eigenvectors associated with the repeated eigenvalues are not unique.

3. PCA Vs LDA:
3.1 Introduction to PCA approach:
Principal Component Analysis [8] is a widely used technique in the fields of signal processing, communications, control theory, image processing, etc. In the PCA approach the component matching relies on good data to build eigenfaces. In other words, it builds M eigenvectors for an N x M matrix. They are ordered from largest to lowest where the largest eigenvalue if associated with the vector that finds the most variance in the image. An advantage of PCA to other methods is that the 90% of the total variance is contained in 510% of the dimensions. To classify an image we want to find the eigenface with smallest Euclidean distance from the input face. To do so we transform the input image to a lower dimension M by computing Where each v_i = w_i * e_i ^T, v_i is the ith coordinate of the facial image in the new space, which came to be the principal component. The vectors e_i, also called eigenimages, can also be represented as images and look like faces. These vectors represent the classes of faces to which we can classify a new instance of a face. Now that we have our transformed M we can find the vector k to which the image is the closest. Let Omega represent the contribution of each eigenface to the representation of an image in a basis constructed from the eigenvectors. Then find k such that Epsion_k = || Omega - Omega_k|| < Theta Where Omega_k is the vector describing the kth face class. When Epsilon_k is less than some threshold value Theta the new face is classified to belong to class k.

2. Eigenface Approach:
2.1 Eigen Values and Eigen Vectors:
In linear algebra, the eigenvectors of a linear operator are non-zero vectors which, when operated on by the operator, result in a scalar multiple of them. The scalar is then called the eigenvalue () [3] associated with the eigenvector(X). Eigen vector is a vector that is scaled by a linear transformation. It is a property of a matrix. When a matrix acts on it, only the vector magnitude is changed not the direction. AX=X (2.1) Where A is a Vector function.

[v_1 v_2 v_M]^T

2.2 Calculations of Eigen Values and Eigen Vectors:

By using (2.1), we have the equation, (A-I)X=0 (2.2)

Where I is the n x n Identity matrix. This is a homogeneous system of equations, and from fundamental linear algebra, we know that a nontrivial solution exists if and only if det (A-I)=0 (2.3) Where det() denotes determinant. When evaluated, becomes a polynomial of degree n. This is known as the characteristic equation of A, and the corresponding polynomial is the characteristic polynomial. The characteristic polynomial is of degree n. If A is n x n, then there are n solutions or n roots of the characteristic polynomial. Thus there are n eigenvalues of A satisfying the equation, AXi=Xi (2.4) Where i=1, 2, 3.n If the eigenvalues are all distinct, there are n associated linearly independent eigenvectors, whose directions are unique, which span an n dimensional Euclidean space.

3.2 Introduction to LDA approach:

Linear Discriminant Analysis (LDA) is a well-known scheme for feature extraction and dimension reduction. It has been used widely in any applications involving high-dimensional data, such as face recognition and image retrieval. An intrinsic limitation of classical LDA is the so-called Singularity problem, that is, it fails when all scatter matrices are singular. LDA finds the vectors in the underlying space that best discriminate among classes. For all samples of all classes the between-

class scatter matrix SB and the within-class scatter matrix SW are defined by:

intersect the ellipses corresponding to the class distributions, PCA yields superior results.

Where Mi is the number of training samples in class i, c is the number of distinct classes, i is the mean vector of samples belonging to class i and Xi represents the set of samples belonging to class i With xk being the k-th image of that class. SW represents the scatter of features around the mean of each face class and SB represents the scatter of features around the overall mean for all face classes

3.3 Comparison of PCA and LDA:

Distinguishing characteristics techniques are as follows: of these two

LDA and PCA are the two popular independent feature extraction methods. Both of them extract features by projecting the original parameter vectors into a new feature space through a linear transformation matrix. They optimize the transformation matrix with different intentions. PCA optimizes the transformation matrix by finding the largest variations in the original feature space. LDA pursues the largest ratio of between-class variation and within-class variation when projecting the original feature to a subspace. PCA has been used in face recognition, handprint recognition, human-made object recognition, industrial robotics, and mobile robotics. LDA has been used in face recognition and mobile robotics. LDA has also been proposed for generic object recognition. The main differences between these two methods are as follows; LDA is based on a single face image as input [1], [2]. That means LDA can perform face recognition for a single input image; hence it does not consider multiple input images. Where as PCA is based on multiple face images as input. Hence it considers multiple input images. PCA is less sensitive where as LDA is more sensitive. PCA takes very less computational time where as LDA takes more computational time. Fig.1 [10] describes the decision thresholds yielded by the nearest-neighbor approach for the two cases are marked DPCA and DLDA. As can be seen by the manner in which the decision thresholds

Fig 1 Where X- axis : Class A and Y axis : Class B Suppose there are two different classes (Class A and Class B) embedded in two differentGaussian-like distributions. However, only two samples per class are supplied to the learning procedure (PCA or LDA). The classification result of the PCA procedure (using only the first eigenvector) is more desirable than the result of the LDA. DPCA and DLDA represent the decision thresholds obtained by using the nearest-neighbor classification. Principle component analysis aims to catch the total variation in the set of training faces, and to explain the variation by a few variables. In fact, observation described by a few variables is easier to understand than one defined by a huge amount of variables and when many faces have to be recognized the dimensionality reduction is important. The other main advantage of PCA is that once you have found these patterns in the data, and you compress the data, ie. By reducing the number of dimensions, without much loss of information. This technique is also used in image compression.

4. Face Recognition System:

4.1 Face space creation:
The accurate reconstruction of the face is not required, so we can now reduce the dimensionality to M instead of M. This is done by selecting the M Eigenfaces which have the largest associated Eigenvalues. These Eigenfaces now span a M-dimensional which reduces computational time.

Where e1, e2,...en are the Lower diagonal elements and e1,e2,en are the Upper diagonal elements. This algorithm uses HouseHolder Vector form which is n x m matrix (This is a way of Tri-diagonalising the Matrix.). It accepts only symmetric matrix as input.

Working of QL algorithm:
According to the largest value it shifts the elements in a matrix among themselves. (That means Upper and Lower diagonal elements are shifted with respect to main diagonal elements).Hence the number of shifts gives the EigenValues. Higher the Eigen values higher the property of that image and diagonal elements will be in a sorted order. Normalization: It takes max Eigen value of the Eigen Vector and divides each Eigen value by max Eigen value.

Eigen Faces

4.3 Face Recognition system:

Face Space (sum of Eigen Faces) Figure 2 Fig-2 describes about Eigen faces. [9] In order to reconstruct the original image from the eigenfaces, one has to build a kind of weighted sum of all eigenfaces (Face Space). That is, the reconstructed original image is equal to a sum of all eigenfaces, with each eigenface having a certain weight. This weight specifies, to what degree the specific feature (eigenface) is present in the original image. If one uses all the eigenfaces extracted from original images, one can reconstruct the original images from the eigenfaces exactly. But one can also use only a part of the eigenfaces. Then the reconstructed image is an approximation of the original image. However, one can ensure that losses due to omitting some of the eigenfaces can be minimized. This happens by choosing only the most important features (eigenfaces). A new image T is transformed into its eigenface components (projected into face space) by a simple operation[11],

wk = ukT (T - )

4.2 Calculation of Eigen Values:

Two algorithms called TRED2 () & QL algorithm are used for calculating Eigen Values. In TRED2 algorithm, Covariance Matrix is given as a input. Here Covariance Matrix is converted into Tri-diagonalised form except Upper, Lower & main diagonal elements all other elements are made zero. Consider an example:

Here k = 1, 2.M. The weights obtained as above form a vector T = [w1, w2, w3 wM] that describes the contribution of each eigenface in representing the input face image. The vector may then be used in a standard pattern recognition algorithm to find out which of a number of predefined face class, if any, best describes the face. The face class can be calculated by averaging the weight vectors for the images of one individual. The face classes to be made depend on the classification to be made like a face class can be made of all the images where subject has the spectacles. With this face class, classification can be made if the subject has spectacles or not. The Euclidean distance of the weight vector of the new image from the face class weight vector can be calculated as follows,

k = || k||
Where k is a vector describing the kth face class. Euclidean distance formula can be found in [2]. The face is classified as belonging to class k when the distance k is below some threshold value . Otherwise the face is classified as unknown. Also it can be found whether an image is a face image or not by simply finding the squared distance between the mean adjusted input image and its projection onto the face space. Where f is the face space and = Ti is the mean adjusted input. With this we can

2 = || - f ||

classify the image as known face image, unknown face image and not a face image. 4.3.1 Section explains the Figure 3 Reference:[9]

Figure 3

4.3.1 Procedure of Recognition system:

Eigenfaces Initialization Eigenfaces Recognition

1. Acquire an initial set of face images (the training set) 2. Calculate the Eigenfaces from the training set, keeping only the M images that correspond to the highest eigenvalues [4], [5], [6], and [7]. These M images define the face space. As new faces are experienced, the Eigenfaces can be updated or recalculated 3. Calculate the corresponding distribution in Mdimensional weight space for each known individual, by projecting their face images onto the face space.

1. Calculate a set of weights based on the input image and the M Eigenfaces by projecting the input image onto each of the Eigenfaces. 2. Determine if the image is a face at all by checking to see if the image is sufficiently close to face space. 3. (Optional) Update the Eigenfaces and/or weight patterns. 4. If it is a face, classify the weight pattern as either a known person or as unknown

To summarize whole process of Face Recognition using Eigenface approach from the above diagram, the training set of images are given as input to find eigenspace. Using these images, the average face image is computed. The difference of these images is represented by covariance matrix. This is used to calculate Eigenvectors and Eigenvalues. These are the Eigenfaces which represent various face features. Sort the eigenvalues, and consider higher of them since they represent maximum variations. This becomes eigenspace spanned by the eigenfaces, Which has lower dimension than original images? Now given two test images are projected onto this eigenspace to give the weight vector also known as Face key for that image. The Euclidean distance between these two face key vectors is calculated. If this is below some threshold value, then two images are said to be matching that means they belong to same person. Depending on this result, False Acceptation Rate (FAR) and False Rejection Rate (FRR) are found. These are used to change value of Threshold. In this way Face Recognition is carried out using Eigenface Approach.

if one adds up all the eigenfaces (features) in the right proportion. Each eigenface represents only certain features of the face, which may or may not be present in the original image. If the feature is present in the original image to a higher degree, the share of the corresponding eigenface in the sum of the eigenfaces should be greater. If, contrary, the particular feature is not (or almost not) present in the original image, then the corresponding eigenface should contribute a smaller (or not at all) part to the sum of eigenfaces. So, in order to reconstruct the original image from the eigenfaces, one has to build a kind of weighted sum of all eigenfaces. That is, the reconstructed original image is equal to a sum of all eigenfaces, with each eigenface having a certain weight. This weight specifies, to what degree the specific feature (eigenface) is present in the original image.

4.3.2 Overview of the PCA algorithm:

The algorithm for the facial recognition using eigenfaces is basically described in figure 4[7]. First, the original images of the training set are transformed into a set of eigenfaces E. Afterwards; the weights are calculated for each image of the training set and stored in the set W. Upon observing an unknown image X, the weights are calculated for that particular image and stored in the vector WX. Afterwards, WX is compared with the weights of images, of which one knows for certain that they are facing (the weights of the training set W). One way to do it would be to regard each weight vector as a point in space and calculate an average distance D between the weight vectors from WX and the weight vector of the unknown image WX. If this average distance exceeds some threshold value , then the weight vector of the unknown image WX lies too far apart from the weights of the faces. In this case, the unknown X is considered to not a face. Otherwise (if X is actually a face), its weight vector WX is stored for later classification. The optimal threshold value has to be determined empirically. By means of PCA one can transform each original image of the training set into a corresponding eigenface. An important feature of PCA is that one can reconstruct any original image from the training set by combining the eigenfaces. Remember that eigenfaces are nothing less than characteristic features of the faces. Therefore one could say that the original face image can be reconstructed from eigenfaces

Figure 4

5. Significance of Eigenface Approach:

We have seen the standard Eigenface Approach for Face Recognition. Here we find out M Eigenvectors for representing training set images. Now it is important to choose only M' Eigenvectors from these M Eigenvectors, such that M' is less than M, to represent face space spanned by images. This will reduce the face space dimensionality and enhance speed for face recognition. Here we are reducing the dimensionality of face images. We can choose only M' Eigenvectors with highest Eigenvalues. Now as the higher Eigenvalues represent maximum face variation in the corresponding Eigenvector direction, it is important to consider this Eigenvector for face space representation. Since, the lower Eigenvalues does not provide much information about face variations in corresponding Eigenvector direction, such small Eigenvalues can be neglected to further reduce the dimension of face space. This does not affect the success rate much and is acceptable depending on the application of face recognition. Here only M' Eigenvectors with higher Eigenvalues are chosen for defining face space. The results of choosing various values for M' are given in Results section. The most important point here is to choose value of M' (Maximum Eigen values chosen from Co-variance Matrix) so that it does not result in high error rates for Face recognition process. The M' should be chosen such that error rates does not increase much and are acceptable depending on application for face recognition.

identification but just low error rate. So instead of searching large database of faces, it is better to give small set of likely matches. By using Eigenface approach, this small set of likely matches for given images can be easily obtained. For given set of images, due to high dimensionality of images, the space spanned is very large. But in reality, all these images are closely related and actually span a lower dimensional space. By using eigenface approach, we try to reduce this dimensionality. The eigenfaces are the eigenvectors of covariance matrix representing the image space. The lower the dimensionality of this image space, the easier it would be for face recognition. Any new image can be expressed as linear combination of these eigenfaces. This makes it easier to match any two images and thus face recognition. We have also seen that taking eigenvectors with higher M' eigenvalues instead of all M eigenvectors, does not affect performance much. So even taking lower dimensional eigenspace for the images is acceptable as error rates in such cases are very low. The important part is making this choice of M' which will be crucial depending on type of application and error rate acceptable. More research needs to be done on choosing the best value of M'. This value of M' may vary depending on the application of Face Recognition. So various methods for making best choice of M' needs to be studied. Further, this approach can be extended to detect age, facial expressions of a person.

8. References 6. Summary
In this paper, the dimensionality problems are solved for face recognition. The approach using Eigenfaces and PCA is quite robust in the treatment of face images with varied facial expressions as well as the directions. It is also quite efficient and simple in the training and recognition stages, dispensing low level processing to verify the facial geometry or the distances between the facial organs and their dimensions. However, this approach is sensitive to images with uncontrolled illumination conditions. One of the limitations for eigenface approach is in the treatment of face images with varied facial expressions and with glasses. [1]Discriminative learning and Recognition of image set classes using Canonical correlations by Tae-Kyum Kim, Josef Kittler, Roberto Cipolla (IEEE members). [2] Intelligent Face Recognition techniques: A Comparative Study by S.T Gandhe, K T Talele, A G Keskar. [3]Wikipedia http://en.wikipedia.org/wiki/Eigenvalue,_eigenvect or_and_eigenspace [4] Recognition of Non-symmetric Faces Using Principal Component Analysis by N Krishnan, GR Jothi, K Pulloor, GL Aaron [5] M.A.Turk and A.P.Pentland, Face Recognition Using Eigenfaces, Proc. Of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 586-591, June 1991. [6] Matthew Turk and Alex Pentland, Eigenfaces for Recognition, Vision and Modeling Group, The Media Laboratory, Massachusetts Institute of

7. Conclusion and Future works:

The Eigenface approach for Face Recognition process is fast and simple which works well under constrained environment. It is one of the best practical solutions for the problem of face recognition. Many applications which require face recognition do not require perfect

Technology, In J. of cognitive neuroscience,1991, vol.3,no.1, pages 71 to 86. [7] Eigenface-based facial recognition by Dimitri PISSARENKO [8] Enhanced Principal Component Analysis Recognition Performance by Sezin Kaymak [9] Nick Pears Thomas Heseltine and Jim Austin. Evaluation of image preprocessing techniques for eigenface based face recognition. ACA Group, Dept. of Computer Science, University of York, 2002. [10] A.M. Martinez and A.C. Kak, PCA versus LDA, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 228 233, Feb. 2001 [11] Turk and A. Pent land. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1)1991a. http://www.cs.ucsb.edu/ mturk/Paper s/jcn.pdf.