IOSR Journal of Computer Engineering (IOSR-JCE) e-ISSN: 2278-0661, p- ISSN: 2278-8727Volume 11, Issue 1 (May. - Jun.

2013), PP 61-64 www.iosrjournals.org

A Literature review on Facial Expression Recognition Techniques

Ms.Aswathy.R
Post Graduate Scholar, Computer Science Department, Nehru college of Engineering , India

Abstract: Automatic facial expression recognition has become a progressive research area since it plays a major
role in human-computer-interaction. The facial expression recognition finds major application in areas like social interaction and social intelligence. A review of various techniques used in facial expression recognition like principal component analysis (PCA), linear discriminant analysis (LDA) etc is done in this paper. The survey is represented in tabular form for quick reference Index Terms Automatic facial expression recognition, Social Intelligence, Human-Computer interaction.

I.

Introduction

In the recent years, many improvements have been accomplished in the areas like face recognition, face tracking and it anticipated the interest in facial expression recognition field. Facial expressions are produced as a result of distortions of facial features due to the constriction of facial muscles. The facial expressions are examined for identifying the basic human emotions like anger, fear, happiness, sadness, disgust and surprise. These expressions can vary between individuals. The facial expression recognition systems consists of four major steps. Face detection and normalization phase detects the face and lighting effect is reduced. Next step is feature extraction which extracts the features and irrelevant features are eliminated in feature selection process. Final step is classification where the expressions are classified in to six basic emotions. Approaches like Principal Component Analysis [1], Linear Discriminant Analysis [2], Independent Component Analysis [3] constitute a major part in the facial expression recognition techniques. These methods are 1-dimensional in nature. Therefore 2- dimensional Principal Component Analysis [4] is introduced. Since these techniques are applicable only in gray scale images, Global Eigen Approach [5] and Sub pattern Extended 2-dimensional Principal Component Analysis [6](E2DPCA) can be extended by traditional approaches to color space. Multilinear Image Analysis [7] introduced tensor concept which allows more than one factor variation in contradiction to PCA. Color Subspace Linear Discriminant Analysis [8] also uses tensor concept but in color space which improves the accuracy. For achieving greater performance, another technique called Gabor Filer Bank [9] is used which outperforms all the other methods. Local Gabor Binary Pattern [10] has improved recognition rate than the gabor filter bank technique. Many studies have revealed that the overall performance is enhanced by color component embedding. But if RGB color space is used, accuracy depend on the angle and light source which reduces the recognition performance. Therefore RGB color space is not always suitable for color information processing. Perceptually uniform color systems can address this problem. Therefore a novel tensor perceptual framework[11] for facial expression recognition is introduced in this paper for better performance. This is done on perceptual color space and can be examined under slight variations in illuminations.

II.

Techniques Used For Facial Expression Recognition

A. Principal Component Analysis Principal Component Analysis (PCA), also known as the eigen face approach is one of the popular method for facial expression recognition[1]. The major goal of PCA is to reduce the dimensionality for effective face indexing and retrieval. Also, PCA uses linear projection,which maximize the projected sample scattering [2]. In this, the identity of the person is the only varying factor.PCA faces difficuilty if other factors like viewpoint, lighting are varied. B. Fishers Linear Discriminant Under severe variation in facial expression and illumination Fishers Linear Discriminant (FLD) is more suitable. FLD reduces the scattering of projected sample since it is a class specific method[2]. Error rate is reduced when compared to PCA. C. Independent Component Analysis Both PCA and LDA generate spatially global feature vectors. But for effective facial expression recognition spatially localized feature vectors is needed. Therefore Independent Component Analysis (ICA)

www.iosrjournals.org

61 | Page

A Literature review on Facial Expression Recognition Techniques

generates statistically independent basis vector[3].The average recognition rate is improved. But ICA is computationally expensive than PCA. D. 2Dimensional Principal Component Analysis In PCA, feature extraction is done based on 1D vectors. Therefore the image matrix need to be transformed into vector. 2dimensional Principal Component Analysis (2DPCA) uses 2D matrix instead of 1D vector[4]. The recognition rate of 2DPCA is higher than PCA. But the storage requirement for 2DPCA is higher than PCA since 2DPCA needs more coefficients for image representation. E. Global Eigen Approach using Color Images Conventional facial expression recognition techniques like PCA, LDA etc uses only the luminance information in face images. Global Eigen Approach uses the color information in face images[5].RGB color space does not provide any improvement in recogniton rate. In HSV color space, H component is removed since it reduces recognition rate. YUV colorspace provides high recognition rate. F. Sub pattern Extended 2-Dimensional PCA The recognition rate of PCA is low and has small sample size problem. For gray facial expression recognition, 2DPCA is extended to Extended 2DPCA. But E2DPCA is not applicable for color images. Therefore Sub pattern Extended 2-Dimensional PCA (SpE2DPCA) is introduced for color face recognition[6]. The recognition rate is higher than PCA, 2DPCA, E2DPCA and problem of small sample size in PCA is also eliminated. G . Multilinear Image Analysis Facial expression recognition needs different factors like pose, lighting, expressions to be considered. But the conventional PCA addresses only variations in single factor. Multilinear image analysis use multilinear algebra[7]. In this, the concept of Tensor faces is used, which separates different factors underlying the formation of an image. Recognition rate is greater when compared to PCA approach. Color information is not incorporated in multilinear image analysis. H . Color Subspace Linear Discriminant Analysis The 1DLDA AND 2DLDA are extended in color space to improve the face recognition accuracy. A 3D color tensor is used to generate color LDA subspace[8]. Horizontal unfolding increases the recognition rate for 2DLDA while vertical unfolding improves recognition rate for 2DPCA.The performance evaluation of various color spaces is not done. I. 2D Gabor filter bank The Gabor filtering is considered as one of the most important feature extraction technique in facial expression recognition[9]. Gabor filter bank performs better in terms of recognition rate than the other methods like PCA, LDA etc[11]. The major limitation of gabor filter is its bandwidth limitation ie. maximum bandwidth is limited to one octave. Gabor filters loss high and low frequency information since it is band pass in nature J. Local Gabor Binary Pattern Appearence based features are being used for face recognition since it encodes specific details about human faces. In this facial image is divided into sub blocks and similarities between sub blocks is obtained[10]. An important advantage of Local Binary Pattern (LBP) is its illumination tolerance. In Local Gabor Binary Pattern(LGBP) method, LBP is extracted from gabor filters for feature vector generation. LGBP achieves better performance than gabor filter method[11]. All these techniques have improved the performance of the facial expression recognition system considerably by incorporating color component and tensor concepts. But none of the color spaces like RGB provide provision for head pose and lighting variations.Therefore perceptual color space like CIELab and CIELuv is used in this paper to improve the performance in varying conditions. The color images are represented using 3D matrix. But 2D filtering operation is needed to be applied to a 3D array, which is complex in nature. As a solution to this problem, 2D filters are to be applied three times over the three color components in images. Instead of doing this, tensor concept is used. The filtering operation is directly applied to the tensor i.e generated from the color image. This paper introduces a Tensor Perceptual Color Framework (TPCF) [11] where color image components are horizontally unfolded to 2D tensors using multilinear algebra and tensor concepts. Log-gabor filters are used for feature extraction since it overcome the limitations of gabor filter based method. For feature selection mutual information quotient method is utilized. Multiclass linear discriminant analysis classifier is www.iosrjournals.org 62 | Page

A Literature review on Facial Expression Recognition Techniques

used for classifying the selected features. TPCF can effectively recognize the facial expressions under different
NAME Low-Dimensional Procedure for Characterization of Human Faces Eigenfaces vs. Fisherfaces : recognition using class specific linear Projection Recognizing Faces with PCA and ICA METHOD Principal Component Analysis Fishers Linear Discriminant PERFORMANCE Recognition rate is low DISADVANTAGES Only single factor can be varied

Recognition rate higher than PCA

Global feature vectors are generated

Independent Component Analysis

Recognition rate is improved compared to PCA and FLD Recognition rate is higher than PCA

Computationally expensive than PCA

Two-dimensional PCA: A new approach to appearancebased face representation and recognition The importance of the color information in face recognition A novel hybrid approach based on sub-pattern technique and E2DPCA for color face recognition Face Recognition using a Color Subspace LDA approach Multilinear Image Analysis for Facial Recognition Gabor Filter Based Face Recognition Technique

2-Dimensional Principal Component Analysis

Storage requirement is higher than PCA

Global Eigen Approach using Color Images

YUV color space has highest recognition rate

RGB color space does not provide any improvement in recognition rate Variation in lighting, pose are not considered

Subpattern Extended 2Dimensional Principal Component Analysis

Recognition rate higher than PCA,2DPCA

Color Subspace Linear Discriminant Analysis

Recognition rate is higher than 2DPCA and LDA

Variation of performance in color spaces is not evaluated

Multilinear Image Analysis 2-Dimensional Gabor Filter Bank

Recognition rate higher than PCA Higher recognition rate than PCA, LDA,2DPCA,Global Eigen Approach Better recognition rate than Gabor filter bank

Less performance than Color Subspace LDA Low and high frequency component attenuation

Local binary patterns for multi-view facial expression recognition

Local Gabor Binary Pattern

Color information is not included

illumination conditions and thus performance can be improved. Table1: Comparison table on literature survey

III.

Conclusion

This paper presents a literature survey on the various techniques involved in facial expression recognition. There are various techniques that can be used for the purpose. These methods are measured on the basis of recognition rate. Higher the recognition rate, greater the performance. The tensor perceptual color framework has the highest recognition rate and has highest performance.

IV.

Acknowledgment

This work is supported and guided by my research guide. I am very thankful to my research guide Ms. Shinu Acca Mani, Assistant Professor, Computer Science Department, Nehru college of Engineering and Research Centre, India for her guidance and support.

www.iosrjournals.org

63 | Page

A Literature review on Facial Expression Recognition Techniques References

[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] L. Sirovich and M. Kirby, Low Dimensional Procedure for Characterization of Human Faces, J. Optical Soc. Am., vol. 4, pp. 519 524, 1987. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman,Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711 720, Jul. 1997. B.A. Draper, K. Baek, M.S. Bartlett, J.R. Beveridge, Recognizing Faces with PCA and ICA, Computer Vision and Image Understanding: special issue on face recognition, in press. J. Yang, D. Zhang, A. F. Frangi, and J. Y. Yang, Two -dimensional PCA: A new approach to appearance-based face representation and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 131137, 2004. L.Torres, J. Reutter, and L. Lorente, The importance of the color information in face recognition, in Proceedings. 1999 International Conference on Image Processing, 1999. ICIP 99., vol. 3. IEEE, 1999, pp. 627 631. Chen S., Sun Y. and Yin B., A Novel Hybrid Approach Based on Sub-pattern Technique and Extended 2DPCA for Color Face Recognition, 11th IEEE International Symposium on Multimedia, pp. 630 -634, 2009. M. Thomas, C. Kambhamettu and S. Kumar, (2008), Face recognition using a color subspace LDA approach, Proceedings International Conference on Tools with Artificial Intelligence. M. A. O. Vasilescu and D. Terzopoulos, Multilinear image analysis for facial recognition, in Proc. Int. Conf. Pattern Recognit., Quebec City, QC, Canada, Aug. 2002, pp. 511 514 Barbu, T, Gabor filter-based face recognition technique, Proceedings of the Romanian Academy, vol.11, no. 3, pp. 277 283, 2010. S. Moore and R. Bowden, Local binary patterns for multi -view facial expression recognition, Comput. Vis. Image Understand., vol. 115, no. 4, pp. 541 558, 2011. S. M. Lajevardi and H. R. Wu, Facial Expression Recognition in Perceptual Color Space IEEE Transactions on Image Processing , vol. 21, no. 8, pp. 3721-3732, 2012.

www.iosrjournals.org

64 | Page