Back propagation

Artificial Neural Networks (ANN)

1. Back propagation
N NasurudeenAhamed,
Assistant Professor,
CSE

2. Introduction -Back Propagation
•BackpropagationisasupervisedlearningtechniqueforTraininganeural
networks.
•Calculatethegradientofthecostfunctioninaneuralnetwork.
•Usedbygradientdescentoptimizationalgorithmtoadjustweightofneurons.
•Alsoknownasbackwardpropagationoferrorsastheerroriscalculatedand
distributedbackthroughthenetworkoflayers.
•GoalofBackpropagation:Optimizetheweights.

3. Back Propagation

4. Back Propagation
•ActivationFunction:It’sadecisionmakingfunction.
•ThemainpurposeisconverttheinputsignalofanodeinaANNtoanoutput
signal.

5. Back Propagation
•VariantsofActivationFunction
•LinearFunction
•SigmoidFunction
•TanhFunction(TangentHyperbolicfunction)
•RELUFunction(Rectifiedlinearunit)
•SoftmaxFunction

6. Back Propagation
•Bias:Thebiasnodeaconsidereda“pseudoinput”toeachneuroninthe
hiddenandoutputlayer.
•It’susedtoovercometheproblemsassociatedwithsituationswherethevalues
ofaninputpatternarezero.Ifanyinputpatternhaszerovalues,theneural
networkcouldnotbetrainedwithoutabiasnode.
•Bias(threshold)activationfunctionwasproposedfirst.

7. Back Propagation
•Goal:OptimizeTheWeightssothattheneuralnetworkcanlearnhowto
correctlymaparbitraryinputstooutputs.

8. Back Propagation
•ForwardPass:Input–0.05and0.10

9. Back Propagation
•Howwecalculatetotalnetinput:
•ApplyActivationFunction:

10. Back Propagation
•CalculatingtheTotalError:EachOutputNeuronusingtheSquaredError
FunctionandSumthemtogetthetotalerror.

11. Back Propagation
•BackwardsPass:OurGoalistominimizetheerrorforeachoutputneuronand
thenetworkasawhole.
•HowmuchchangeinW5affectstotalError?
•“GradientWithRespecttoW5”-
•ToDecreasetheerror,thensubtractthevaluefromthecurrentweight.

12. Back Propagation
•Next,Wewillcontinuethebackwardspassbycalculatingthenewvaluesfor
W1,W2,W3,andW4.

13. Back Propagation
•Advantages:
•It is simple, fast and easy to program
•Only numbers of the input are tuned and not any other parameter
•No need to have prior knowledge about the network
•It is flexible
•A standard approach and works efficiently
•It does not require the user to learn special functions

14. Back Propagation
•DisAdvantages:
•Backpropagationpossibly be sensitive to noisy data and irregularity
•The performance of this is highly reliant on the input data
•Needs excessive time for training
•The need for a matrix-based method for backpropagationinstead of mini-batch

15. Back Propagation
•Applications:
•The neural network is trained to enunciate each letter of a word and a sentence
•It is used in the field ofspeech recognition
•It is used in the field of character and face recognition