Leveraging Transfer Learning
for Effective Recognition of Emotions
from Images: A Review
Devangi Purkayastha and D. Malathi
AbstractEmotions constitute an integral part of interpersonal communication and
comprehending human behavior. Reliable analysis and interpretation of facial expres-
sions are essential to gain a deeper insight into human behavior. Even though facial
emotion recognition (FER) is extensively studied to improve human–computer inter-
action, it is yet elusive to human interpretation. Albeit humans have the innate capa-
bility to identify emotions through facial expressions, it is a challenging task to
be accomplished by computer systems due to intra-class variations. While most
of the recent works have performed well on datasets with images captured under
controlled conditions, they fail to perform well on datasets that consist of varia-
tions in image lighting, shadows, facial orientation, noise, and partial faces. For all
the tremendous performances of the existing works, there appears to be significant
room for researchers. This paper emphasizes automatic FER on a single image for
real-time emotion recognition using transfer learning. Since natural images suffer
from problems of resolution, pose, and noise, this study proposes a deep learning
approach based on transfer learning from a pre-trained VGG-16 network to signif-
icantly reduce training time and effort while achieving commendable improvement
over previously proposed techniques and models on the FER-2013 dataset. The main
contribution of this paper is to study and demonstrate the efficacy of multiple state-
of-the-art models using transfer learning to conclude which is better to classify an
input image as having one of the seven basic emotions: happy, sad, surprise, angry,
disgust, fear, and neutral. The analysis shows that the VGG-16 model outperforms
ResNet-50, DenseNet-121, EfficientNet-B2, and others while attaining a training
accuracy of about 85% and validation accuracy as high as 67% in just 15 epochs
with significantly lower training time.
D. Purkayastha (B)·D. Malathi
Department of Computer Science and Engineering, SRM Institute of Science and Technology,
Kattankulathur 603203, India
e-mail:
[email protected]
D. Malathi
e-mail:
[email protected]
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022
D. P. Agrawal et al. (eds.),Cyber Security, Privacy and Networking, Lecture Notes
in Networks and Systems 370,https://doi.org/10.1007/978-981-16-8664-1_2
13