Cours setting up your Machine learning 2M1.pdf

Cours setting up your Machine learning une resume pour machine learning

1. Setting up your
ML application
Train/dev/test
sets
deeplearning.ai

2. Andrew Ng
Applied ML is a highly iterative process
Idea
Experiment Code
# layers
# hidden units
learning rates
activation functions
…

3. Andrew Ng
Train/dev/test sets

4. Andrew Ng
Mismatched train/test distribution
Training set:
Cat pictures from
webpages
Dev/test sets:
Cat pictures from
users using your app
Not having a test set might be okay. (Only dev set.)

5. Setting up your
ML application
Bias/Variance
deeplearning.ai

6. Andrew Ng
Bias and Variance
high bias “just right”high variance

7. Andrew Ng
Bias and Variance
Cat classification
Train set error:
Dev set error:

8. Andrew Ng
High bias and high variance
!"
!#

9. Setting up your
ML application
Basic “recipe”
for machine learningdeeplearning.ai

10. Andrew Ng
Basic recipe for machine learning

11. Regularizing your
neural network
Regularization
deeplearning.ai

12. Andrew Ng
Logistic regression
min
$,&
'(),*)

13. Andrew Ng
Neural network

14. Regularizing your
neural network
Why regularization
reduces overfitting
deeplearning.ai

15. Andrew Ng
How does regularization prevent overfitting?
!"
!#
!$
%&
high bias “just right”high variance

16. Andrew Ng
How does regularization prevent overfitting?

17. Regularizing your
neural network
Dropout
regularization
deeplearning.ai

18. Andrew Ng
Dropout regularization
!"
!#
$%
!&
!'
!"
!#
$%
!&
!'

19. Andrew Ng
Implementing dropout (“Inverted dropout”)

20. Andrew Ng
Making predictions at test time

21. Regularizing your
neural network
Understanding
dropout
deeplearning.ai

22. Andrew Ng
Why does drop-out work?
Intuition: Can’t rely on any one feature, so have to
spread out weights.
!"
!# $%
!&

23. Regularizing your
neural network
Other regularization
methods
deeplearning.ai

24. Andrew Ng
Data augmentation
4

25. Andrew Ng
Early stopping
# iterations

26. Setting up your
optimization problem
Normalizing inputs
deeplearning.ai

27. Andrew Ng
Normalizing training sets
!"
!#
5
3
!#
!"
!#
!"

28. Andrew Ng
Why normalize inputs?
!",$=
1
'
(
*
+,-
ℒ (01
+,0
(+))
"
$
"
$
"
$
!
Unnormalized:
"
$
!
Normalized:

29. Vanishing/exploding
gradients
deeplearning.ai
Setting up your
optimization problem

30. Andrew Ng
Vanishing/exploding gradients
!"
!#
=
$%

31. Andrew Ng
Single neuron example
!"
!#
!$
!%
&'
(=*(,)

32. Numerical approximation
of gradients
deeplearning.ai
Setting up your
optimization problem

33. Andrew Ng
Checking your derivative computation
!

34. Andrew Ng
Checking your derivative computation
!

35. Gradient Checking
deeplearning.ai
Setting up your
optimization problem

36. Andrew Ng
Gradient check for a neural network
Take !
",$
["],…,!
(,$
(and reshape into a big vector ).
Take +!
",+$
["],…,+!
(,+$
(and reshape into a big vector d).

37. Andrew Ng
Gradient checking (Grad check)

38. Gradient Checking
implementation notesdeeplearning.ai
Setting up your
optimization problem

39. Andrew Ng
Gradient checking implementation notes
-Don’t use in training –only to debug
-If algorithm fails grad check, look at components to try to identify bug.
-Remember regularization.
-Doesn’t work with dropout.
-Run at random initialization; perhaps again after some training.