Deep residual learning for image recognition

고려대 IML 연구실에서 진행한 세미나 자료

1. DEEP RESIDUAL LEARNING FOR IMAGE RECOGNITION
KAIMINGHE, XIANGYUZHANG
SHAOQINGREN, JIANSUN
MICROSOFT RESEARCH
2017-08-17
신윤호

2. 2
KAIMINGHE

3. 3
CONTENTS
Motivation
Related Work
Deep Residual Learning
Experiments

4. 4
MOTIVATION
Network depth is of crucial importance,
but it is difficultto train deep neural networks
•There are vanishing/exploding gradients problems
•When deeper networks are able to start converging, a degradationproblem
has been exposed(not caused by overfitting)

5. 5
MOTIVATION
Proposed network address the degradationproblem by introducing a
‘deep residual learning’ framework
They show
1.A simple and clean framework of training ‘very’ deep nets
2.State-of-the-art performance for Image classification/Object
detection/Semantic segmentation/and more…

6. 6
RELATED WORK

7. 7
DEEP RESIDUAL LEARNING

8. 8
DEEP RESIDUAL LEARNING

9. 9
DEEP RESIDUAL LEARNING

10. 10
DEEP RESIDUAL LEARNINGxWxFy
i
 }){,(
If identity mappings are optimal, the solvers drive the weights of
nonlinear layers toward zero
So if the optimal function is closer to an identity mapping, it
should be easier for the solver to find the perturbations
The shortcut connections introduce neither extra parameter nor
computation complexity

11. 11
DEEP RESIDUAL LEARNING
Network Architectures
•It is worth that this model
has fewer filters and lower
complexitythan VGG nets

12. 12
DEEP BOTTLENECK ARCHITECTURES
1 x 1 layers are responsible for reducing and then increasing
dimensions, leaving the 3 x 3 layer with smaller input/output
dimensions
Using this, the number of layers increases and enjoy significant
accuracy gains from increased depth

13. 13
EXPERIMENTS

14. 14
EXPERIMENT
Deeper layers don’t always work. Due to overfitting, the accuracy
rate at 1202 layers is lower than at 110 layers
ResNetshows good generalization performance on Object
Detection also

15. 15
EXPERIMENTS

16. 16
Q&A