Article Preview
TopIntroduction
FER system has become a prominent area for many types of research. FER is still a progressive area due to its continuous emerging of new contributions. Emotional changes in a human face are essential in expression recognition. Effective recognition of expressions is possible whenever all the changes in facial emotions can be calculated properly since the intensity of the human feelings could be understood only by observing the emotion changes of a face. While communicating with others, these expressions play a vital role in carrying the original intentions of the speaker. And also the communication becomes effective only with emotions in a face. The concept of FER used in many applications and some trends in recent decades are video indexing, human-computer communication, data-driven animation. Since the emotion analysis deals with the environment around the machine and requires a prompt response from the machine using its acquired intelligence, FER is considered as an ambient computing problem. Most Psychology studies depend on the observation of the facial expressions, and they confer that the facial expressions can convey the human intentions by 55% which is more than verbal communication (Ekman et al., 1978), which is 7% only and paralanguage communication is 38%. These observations ensure that facial expressions are contributing a major part in human-machine interactions. Hence, its significance in nonverbal communication, the development of methods that can recognise and analyse the emotions become very eminent in emotion intelligence systems. Automation of recognition and analysis of these facial features is also required for the machines when performing facial expression recognition, but it is not easy, compared to task by the human.
In the traditional facial expression recognition systems, two types of approaches are followed by the researchers. Features are extracted either from the entire image data or from specific geometric positions. When the entire image data is considered, a huge amount of data to be processed and most of it is unwanted. Machine learning algorithms such as support vector machines would neglect most of the data and consider only vital data is considered called support vectors. And traditional artificial neural networks are very bad at small data such as geometric landmark positions. The ANNs require a huge amount of data and a lot of training time for best accuracy. Whereas SVMs are very good at classifying even with a small amount of data and less training times. It is important to choose a fine blend of an appropriate feature extraction technique and the image data, which is either the entire face image or geometric landmark positions. The novelty of the proposed model is the selection of the feature extraction algorithm, which is applied to parts of the face image such as eyes, eyebrows, mouth, and chin. Each modality expresses certain emotions in the face image. Features from these parts are extracted separately and combined to generate the entire feature set.
Popular feature extraction methods such as local binary pattern, Gabor wavelets, and local directional pattern are used in this paper. Sophisticated machine learning techniques such as Support Vector Machine and Artificial Neural Networks were used to exploit potentials of feature fusion techniques in the classification. Local binary patterns are very popular to handle issues with illumination changes, lightning conditions. Local directionality patterns are very popular in solving the orientation, non-monotonic illumination changes and random noise. Gabor wavelets are best known to deal with the issues of different orientation and scale invariance issues.
The paper is organized into six sections. The first section is the introduction to the basics of emotions, face features, feature extraction techniques, classification techniques, applications of FER, techniques used in the paper and their importance. The second section is background, presents a literature review of the FER in the recent past, highlighting the merits and demerits of the methods. The third section is the proposed model for FER, which is feature-level fusion based FER. This section explains the feature extraction and classification techniques used in the model and the fusion mechanism followed. The fourth section is the experimental setup for validating the proposed model. This section explains the datasets used in validation, software tools used and hardware requirements. The fifth section is the results and discussion. Various analysis is performed, and a detailed discussion is provided in this section. The sixth section is the conclusion, where some important findings are highlighted, merits and drawbacks of the proposed models are discussed.