Generative Adversarial Networks : Basic architecture and variants

In this presentation we review the fundamentals behind GANs and look at different variants. We quickly review the theory such as the cost functions, training procedure, challenges and go on to look at variants such as CycleGAN, SAGAN etc.

1. Generative Adversarial
Networks
PalacodeNarayana IyerAnantharaman
29 Oct 2018

2. References
•https://github.com/lukedeo/keras-acgan/blob/master/acgan-analysis.ipynb
•https://github.com/keras-team/keras/blob/master/examples/mnist_acgan.py
•https://skymind.ai/wiki/generative-adversarial-network-gan
•https://junyanz.github.io/CycleGAN/
•Self Attention Generative Adversarial Networks: Zhang et al

3. Why GAN?
•GANs can learn to mimic any distribution and generate data
•The data may be images, speech or music
•The outputs from GANs are found to be quite realistic and impressive
•Thus, GANs have a number of applications: From being a feature in products like
Photoshop to generating synthetic datasets for image augmentation

4. GAN Architecture

5. GAN Architecture

6. GAN Workflow

7. Generator
•Generates synthetic images given the input noise z
•G is differentiable
•Typically a Gaussian distribution

8. Training
•Train on 2 mini batches simultaneously
•Training samples
•Generated samples
•Cost

9. Cost Function

10. Different Variants of GAN
Ref: https://github.com/lukedeo/keras-acgan/blob/master/acgan-analysis.ipynb

11. Cycle GAN (2017)
•Original Paper: “Unpaired Image-to-Image Translation using Cycle-Consistent
Adversarial Networks”, Zhu et al

12. Image to Image Translation
•Image to image translation is aimed at finding a mapping
between an input image (X) and its corresponding output
image (Y), where the pair X, Y are provided in the dataset
•This assumes that we are provided with such a labelled
dataset with pairings
•CycleGANattempts to find a mapping between images from
source and target domains in the absence of paired
examples
Learn G: X → Y such that the distribution of images from G(X) is
indistinguishable from the distribution Y using an adversarial
loss.
Couple this with an inverse mapping F: Y→ X and enforce a
cycle consistency loss to enforce F(G(X)) ≈ X

13. CycleGANApproach

14. Cycle GAN: Objective Function
•Two discriminators: D
xand D
ywhere D
xaims to distinguish between images {x}
and translated images {F(y)}. In the same way D
yaims to discriminate between {y}
and {G(x)}
•The objective function has 2 parts representing the losses:
•adversarial losses for matching the distribution of generated images to the data distribution
in the target domain
•Cycle consistency losses that prevent the learned mappings G and F from contradicting each
other

15. Losses

16. Exercises
•Go through the original paper and answer the following:
•How is the model evaluated? What are the metrics?
•What are the main applications discussed in the paper?
•What are the limitations and future work?

17. SAGAN (2018) Zhang et al Abstract
•GANs often use a CNN as a generator
•CNNs capture short range dependencies very well (local receptive fields) but not
effective to capture long distance correlations
•Self Attention Generative Adversarial Networks (SAGAN) is aimed at generating
images that take in to account both short and long distance dependencies in the
source images

18. SAGAN

19. SAGAN Architecture