Lecture 5.pptx......................................

..........................................................................................................................................................................................................................................................................................................................................................................................................................

1. Activation Functions in Deep Learning

2. Introduction An activation function is a function that is added to an artificial neural network in order to help the network learn complex patterns in the data. When comparing with a neuron-based model that is in our brains, the activation function is at the end deciding what is to be fired to the next neuron.

3. Tanh or Hyperbolic Tangent Activation Function

4. ReLU (Rectified Linear Unit) Activation Function

5. Leaky ReLU Activation Function

6. Leaky ReLU Activation Function

7. ELU (Exponential Linear Units) function

8. PRelu (Parametric ReLU )

9. SoftMax

10. Swish (A Self-Gated) Function

11. Maxout

12. Softplus