Face Recognition Methods based on Convolutional Neural Networks

Convolutional neural networks (CNN) have improved the state of the art in many applications, especially the face recognition area. In this work, we present a review on latest face verification techniques based on Convolutional Neural Networks. In addition, we give a comparison on these techniques regarding their architecture, depth level, number of parameters in the network, and the obtained accuracy in identification and/or verification. Furthermore, as the availability of large-scale training dataset has significantly affected the performance of CNN based recognition methods, we present a preface to the most common large-scale face datasets, and then we describe some of the successful automatic data collection procedures.

1. Face Recognition Methods
based on
Convolutional Neural Networks
Presenter:
Elaheh Rashedi
Advisor:
Prof. XuewenChen
Winter 2016
1

2. Contents
■Background
–Neural network
–Convolutional neural network
–General CNN-based face recognition schema
■Face recognition models based on CNN
–DeepFacemodel
–Web-scaled DeepFacemodel
–DeepIDmodel series
–FaceNetmodel
–VGG model
–Lightened CNN Model
■CNN training and testing dataset
–CASIA-WebFace, MegaFace, IJB-A, VGG, …
■Summary
■Future Work
2

3. Neural Network
Fig 1. (a) General structure of a Neural Network, (b) an example of a neuron
3
(a) (b)

4. Convolutional Neural Networks
■CNN
–A kind of neural network where the input is image
–Contains less fully connectivity between neurons
■CNN layers
–Input layer
–Convolutional layer
–Pooling layer
–Fully connected layer
–Loss layer
4

5. Convolutional Neural Networks (cont. …)
Fig 2. General CNN structure in face recognition problems
5

6. General CNN-based face recognition schema
■Common steps:
–Face detection
Viola-Jones, Cascade CNN, …
–Pre-processing
Geometric & lighting normalization
–CNN training
Supervised vs. unsupervised
–Face identification
Classification problem
–Metric learning
Joint-Bayesian, Cosine similarity, Triplet Similarity, Energy-based similarity, …
–Face Verification
■There are various ways to perform each step!
6

7. 7
Fig 3. General face recognition pipeline

8. Face recognition models based on CNN
■CNN based models are different
–Architecture of CNN
–Depth of neural network
–Number of parameters
–Scale of training dataset
–Similarity metric
–Alignment vs. non-alignment pre-processing
–…
8

9. DeepFaceModel
■First CNN-based face recognition method (2014)
–By Facebook research group
■Includes 4 main steps
–Detection
–3D Alignment
–Feature representation
–Classification
■Similarity metric learning
–Siamese energy based neural network
9

10. DeepFaceModel (cont. …)
10
Fig 4. DeepFace[1]
Fig 5. Siamese network [2]

11. Web-scaled DeepFaceModel
■Based on DeepFace(2015)
–Higher identification acc.
–Lower verification acc.
–Lower feature vector dimension
■Applies bootstrapping on large training dataset
–Select harder recognition cases
–Ignore easy recognition cases
■Claim
–High dimensional feature vectors do not necessarily result in better accuracy!
11

12. DeepIDmodel series
■Inspired by DeepFace
■Model Series
–DeepID(2014)
–DeepID2 (2014)
–DeepID2+ (2015)
–DeepID3 (2015)
12

13. DeepIDmodel
■CNN structure
–4 convolutional layers
–3 max-pooling layers
–1 fully connected layer
■Alignment
–Center of two eyes, two corners of mouth, nose tip.
■Multiple patches
–Extract different features for different part of face
■Similarity metric learning
–Joint-Bayesian
13

14. DeepID2 Model
14
Fig 6. DeepID2 [3]
Fig 7. DeepID[4]

15. DeepID2+ Model
■Based on DeepID2
–More deep network
–Uses supervisory signals
–Uses fully connected layers
■Fully connected layers
–Early feature extraction
■Claim
–Deep CNN based networks are more
robust to corruption of image!
15Fig 8. DeepID2+ [5]

16. DeepID3 Model
■DeepID3
–More Deep than DeepID2+
–Less deep than FaceNet& VGG
■Includes
–Very deep neural networks
–15 feature extraction layers
–Early fully connected layers
■Similarity metric learning
–Joint-Bayesian
16Fig 9. DeepID3 [6]

17. FaceNetModel
■FaceNetmodel: by Google research group
–Same framework for identification and verification
–Very deep network
–No alignment
–Efficient representation of features
■Similarity metric learning
–Triplet loss
17
Fig 10. Triplet loss learning [7]

18. VGG Model
■VGG model: by Visual Geometry Group
–Inspired by the very deep FaceNetnetwork
–Very deep CNN
–36 level of feature extraction
■Similarity metric
–Triplet loss
■Contributions
–Automatic collection of large face dataset
–Publically available pre-trained CNN model
18

19. Lightened CNN Model
■Shallow network
–4 convolutional layers
–4M parameters
■Less computational intensive
–9 times less than VGG model
19

20. CNN Training and Testing Datasets
20

21. Summary
■The size of training dataset
■Alignment
■Joint-Bayesian similarity
■Multiple patches
■Verification performance on video
■Identification performance on larger number of identities
21

22. Future Work
■Using face verification to outperform face detection
■Designing a similarity metric learning
■Designing a face tracking method based on face verification
22

23. References
1. Y.Taigman,M.Yang,M.Ranzato,andL.Wolf,“Deepface:Closingthegaptohuman-levelperformanceinfaceverification,”inTheIEEE
ConferenceonComputerVisionandPatternRecognition(CVPR),June2014.
2. S.Chopra,R.Hadsell,andY.LeCun,“Learningasimilaritymetricdiscriminatively,withapplicationtofaceverification,”inComputerVision
andPatternRecognition,2005.CVPR2005.IEEEComputerSocietyConferenceon,pp.539–546,2005.
3.Y.Sun,Y.Chen,X.Wang,andX.Tang,“Deeplearningfacerepresentationbyjointidentificationverification,”inAdvancesinNeural
InformationProcessingSystems,pp.1988–1996,2014.
4.Y.Sun,X.Wang,andX.Tang,“Deeplearningfacerepresentationfrompredicting10,000classes,”inProceedingsoftheIEEEConferenceon
ComputerVisionandPatternRecognition,pp.1891–1898,2014.
5.Y.Sun,D.Liang,X.Wang,andX.Tang,“Deepid3:Facerecognitionwithverydeepneuralnetworks,”arXivpreprintarXiv:1502.00873,2015.
6.Y.Sun,X.Wang,andX.Tang,“Deeplylearnedfacerepresentationsaresparse,selective,androbust,”inProceedingsoftheIEEEConference
onComputerVisionandPatternRecognition,pp.2892–2900,2015.
7.F.Schroff,D.Kalenichenko,andJ.Philbin,“Facenet:Aunifiedembeddingforfacerecognitionandclustering,”inTheIEEEConferenceon
ComputerVisionandPatternRecognition(CVPR),June2015.
23

24. Thank You!
24