InceptionResNet_Presentation_with_Diagrams.pptx

inception reset

1. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, Alexander A. Alemi (Google Research, 2017)

2. Problem & Motivation Deep CNNs drive advances in image recognition. Inception networks: efficient, strong performance. Residual connections (ResNets): improved optimization, ILSVRC 2015 winner. Gap: Unclear whether Inception can benefit from residuals. Goal: Build hybrid Inception-ResNet, simplify Inception-v4, and compare.

3. Method – Key Contributions Introduced Inception-v4: simplified, uniform architecture. Introduced Inception-ResNet-v1/v2: Inception modules + residual connections. Residual scaling (0.1–0.3) stabilized very wide networks. Used TensorFlow: training without replica partitioning. Training tricks: RMSProp, gradient clipping, exponential LR decay.

4. Method – Architectural Overview Inception-v4: streamlined modules (Inception-A/B/C + Reduction). Inception-ResNet: Inception block + residual shortcut. Projection (1×1 conv) aligns dimensions for residual addition. Residuals accelerate convergence, improve optimization.

5. Results – Training Dynamics Residual connections → faster training than pure Inception. Inception-ResNet slightly outperforms similar-cost Inception. Final accuracy depends mainly on model size, but residuals improve efficiency.

6. Results – Quantitative Findings Single-model (Top-1 / Top-5 errors): • Inception-v3: 21.2% / 5.6% • Inception-ResNet-v1: 21.3% / 5.5% • Inception-v4: 20.0% / 5.0% • Inception-ResNet-v2: 19.9% / 4.9% Ensemble: 3× Inception-ResNet + 1× Inception-v4 → 3.08% Top-5 error.

7. Strengths Introduced strong new architectures (Inception-v4, Inception-ResNet). Residual scaling stabilized wide networks. Residuals improved convergence speed and optimization. Achieved SOTA ImageNet performance (3.08% Top-5 error).

8. Limitations Instability: residual networks with >1000 filters 'died' without scaling. Final accuracy depended more on model size than residuals. Comparisons ad hoc: similar cost models, not systematically optimized. Ensembling gains smaller than expected.

9. Takeaways Problem: Can Inception benefit from residuals? Solution: Yes — hybrids train faster, often more accurate. Results: Inception-v4 and Inception-ResNet-v2 matched/exceeded SOTA. Impact: Residuals confirmed as broadly useful, inspiring later hybrids.