International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering
MenuISSN ONLINE(2278-8875) PRINT (2320-3765)
ISSN ONLINE(2278-8875) PRINT (2320-3765)
Usha Mary Sharma 1, Kaustubh Bhattacharyya 2, Uzzal Sharma 3
|
| Related article at Pubmed, Scholar Google |
Visit for more related articles at International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering
The primary goal of this paper is to make a comparative study on various approaches used to identify the person’s face and thereafter recognize their emotion. Among the various techniques implemented, Neural Networks, Hidden Markov Model and Dimensionality reduction techniques have received lot of attention. The face and facial emotion recognition system would require care and efforts in data acquisition, pre-processing, feature extraction, classification and performance evaluation. The main aim of this review paper is to study and compare the well-known techniques used at different stages to recognize the face and its emotion.
Keywords |
||||
| Face recognition, Emotion recognition, Principal Component Analysis (PCA), Neural Networks (NN), Hidden Markov Model (HMM), Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM). | ||||
INTRODUCTION |
||||
| The emotion of a person is significantly important for effective communication. There are many factors that contribute to understand the emotions of the individual such as face, speech, body posture, behaviors etc.[1]Paul Ekman has mentioned in his research findings that certain emotions can be universally recognized, even if there are cultural differences through the facial expressions like fear, happiness, anger, disgust, sadness and surprise. | ||||
| Facial expressions provide a vital behavioural approach to study the emotion and social interaction. Identification of the person through his/her face and recognition of their emotions have become a promising research area in recent times. Its application includes Human-Computer Interfaces, Human Emotion Analysis, and Medical Care and Cure. Recent advancement in this area has made the researchers optimistic to widen the application of facial emotion recognition to various areas like chat room avatars, video conferencing avatars. Furthermore, the applications of face and facial emotion recognition can be used in many areas ranging from Sign language through Medical Rehabilitation to Virtual Reality, Education purpose, Clinical and Psychological Departments etc. Lot of emphasis is given in Human Computer Interaction (HCI) so that there could be some possibility for the Human and the Computer to interact in more natural way. For this to happen, it is necessary for the computing device to understand the emotions of the human while interacting. An attempt to identify the facial emotion can contribute to the Human and Computer Interaction in more effective manner. | ||||
| As mentioned earlier, there has been a lot of research work going on to recognize the facial expression to deduce the emotions with higher accuracy rate, still it remains difficult to achieve the greater accuracy due to the intricacy, complexity and variability of facial expressions. Hence, the researchers are limiting their approaches in order to obtain better results. | ||||
THEORETICAL BACKGROUND |
||||
| Over the decades, researchers have made many studies on face and facial emotion recognition as it [2] plays an important role in the field of Intelligent Human Computer Interaction by providing an essential medium of behaviour interpretation and emotion modelling. There are many steps involved to identify a face and implement a facial emotion recognition system. These steps are: Pre-processing, Feature extraction and Classification. | ||||
PREPROCESSING |
||||
| The facial image has to be first captured; this is done mostly by any camera. Further, the color images are converted to grey scale and manually cropped. These cropped images are normalized to certain pixels. The images are normalized to a certain size such as in [3], the images from Olivetti and Oracle Research Laboratory (ORL) database were normalized to 56 x 46 pixels. The Histogram Equalization can be performed to eliminate the light effects. The facial images may contain some noise such as Gaussian Noise, Poisson Noise, Salt and Pepper Noise etc. The various noise removal techniques are applied to ensure that the images are noise free. In addition, the de-blurring techniques must be applied to remove any blurring from the image. Various pre-processing steps done in preprocessing could be Size normalization, unified pose and rotation, contrast optimization and face masking. | ||||
FEATURE EXTRACTION |
||||
| Feature extraction is the most important phase while recognizing any facial image and its emotion. This phase extracts the key features that could help in identifying a particular face. Technically,[4] a feature extraction is used to extract its main components to reduce the vector dimension of input feature of the classifier. In other words, the effort taken to remove the redundant data and extract the relevant data from the image (Feature Vector) is known as Feature Extraction. | ||||
| Over the past ten years, researchers have implemented many feature extraction techniques to extract the key features. Some of the most commonly used methods are PCA (Principal Component Analysis) also known as Eigen-Faces, LDA (Linear Discriminant Analysis), 2DPCA (Two Dimensional Principal Component Analysis). | ||||
CLASSIFICATION |
||||
| In this phase, the system is first trained with a certain number of images and later the system is used to classify any given image at a given time, known as testing phase. There are many classifiers used to recognize the face and its emotion. Some of these are K-Means , Nearest Neighbour Classifier, K-Nearest Neighbour Classifier, SVM ( Support Vector Machine), Decision Trees, ANN ( Artificial Neural Network), HMM (Hidden Markov Model), ADA BOOST HAAR Classifier etc. | ||||
EXPERIMENTAL DETAILS |
||||
| Many experiments have been conducted to identify the face and its emotion. In [5],[6], the experiments were accomplished using (PCA) Principal Component Analysis for Feature extraction. The test in [6] was conducted on Japanese Female Facial Expression (JAFFE) database whereas in [5], the algorithm was tested on Olivetti and Oracle Research Laboratory (ORL) face database with 400 images (40 classes). In addition,[7],[8]some of the wellknown techniques used for the face recognition are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Independent Component Analysis (ICA) and [8]SVMs (Support Vector Machines). The features in [8] was extracted using the Canny edge detection algorithm and further the two techniques i.e., Principal Component Analysis (PCA) and neural networks (NN) were implemented, the results were comparatively higher, proving it to be a better approach. The authors in [9] introduced two other approaches i.e., feature based and template based approach. In the feature based approach, set of geometrical features such as nose width and length, mouth position and chin shape were computed. On contrary, the template based approach was based on almost a grey- level matching. | ||||
| The studies carried in [2],[10],[11],[12],[13],[18] showed that a wide range of experiments had been carried out using neural networks (NN) for face recognition and facial expression classification. The authors have emphasized on different types of Neural networks like [5],[10]Feed Forward Back Propagation Neural Network, [4],[11] RBFNN ( Radial Basis Function Neural Network), [14]Generalized Feed Forward Artificial Neural Network (GFFANN) and [2] Hierarchical Radial Basis Function Network (HRBFN). [7],[12]These experiments were carried out at various time, illumination conditions and facial expressions. Some of the other well – known classifiers mentioned were [15] K-NN classifier and [7] Hidden Markov Model (HMM). HMM seems to with stand the variations on the face such as lighting, orientation, facial expression etc. The experimental work in [13] was carried out by implementing the Back Propagation algorithm for training. The database included a variety of faces with different gender, race and features. These varieties in the database such as glasses, moustache and beard had enabled the system to learn to differentiate various levels of facial expressions successfully. | ||||
RESULTS AND DISCUSSION |
||||
| The experimental results carried out in the papers mentioned above shows that in most of the cases, [5] Principal Component Analysis (PCA) method proves to be best only when the frontal face image is considered with no head orientation. [5] It gives a recognition rate of 97.018% which is better as compared with K-means, Fuzzy Ant with fuzzy C-means. The performance function used here was MSE. | ||||
| [5],[6] PCA is considered as one of the most commonly used Dimensionality reduction method for a feature set. In [6],the experimental result showed an achievement of approximately 85% of recognition rate but[16] it involves unwanted variations due to lighting conditions. In [16], it has been stated that PCA is computationally quite expensive and in [3], it is seen through the experiments that 2DPCA improves the speed of the feature extraction significantly. Hence, 2DPCA is computationally more efficient than PCA. However, 2DPCA has a demerit in terms of the storage as it requires more coefficients to represent images in comparison to PCA. On contrary, the outcome in [15]is very different. | ||||
| In [11], it has also been seen that PCA with FLD (Fisher’s Linear Discriminant) technique helps is acquiring lower – dimensional discriminant patterns that increases the accuracy level. PCA and LDA are commonly used for feature representation. [7]However, HMM (Hidden Markov Model) performs better for images having variations such as Illumination, facial expression, orientation. In addition, in [8], the experimental result also shows that PCA and LDA achieves better when associated with SVM. It concludes that LDA and SVM work much better than PCA and SVM. | ||||
| As a classifier, the Artificial Neural Network (ANN) proves to be efficient as compared to the other classification methods. In [12], it was noticed that PCA with Neural network along with Canny Edge detection algorithm gave a better accuracy. This could be because the preprocessing of the image was not considered; hence the preprocessing time was omitted. [3],[10] Feed Forward Back Propagation Neural Network has shown considerably good results with PCA. However, it was also observed that Radial Basis Function Neural Network (RBFNN) along with[10],[11] PCA or DCT [4] had given a descent accuracy rate of more than 90% especially when there is a small training set consisting of high dimension data set. As mentioned already, the RBFNN consumes lot of time to identify the hidden nodes and the equivalent data center. Hence, when the data set is small in size, RBFNN proves to be one the best classification technique. Since, face is one of the important parts to fetch related information to identify the emotion of a person as well as identify them. [2],[14]Hence, there have been approaches taken that could fetch the local and global features to infer the person’s uniqueness. The proposed system in [14] has achieved an accuracy rate of 97%. | ||||
CONCLUSION |
||||
| This paper has shown some of the research work carried in the field of Computer Vision intensively for face identification and facial expression recognition in order to achieve the robust real time system. It is still hard to stick to a particular method for identifying the facial expressions because of the facial variability and its complexity. | ||||
FUTURE SCOPE |
||||
| As mentioned in the experimental results and discussion, there are various methods that perform relatively well at different situations. As a part of the future scope, I would suggest to give more importance while extracting the features. Enough care and appropriate efforts must be taken to extract the key features efficiently. Hence, Neural Network could be used to extract the features along with the other techniques as it has proved to be one of the robust techniques. | ||||
Tables at a glance |
||||
|
||||
|
|
||||
References |
||||
|
