HOPFIELD NETWORK

1. HOPFIELD NETWORK PRESENTED BY : Ankita Pandey ME ECE - 112604

2. CONTENT 10/3/2012 2 PRESENTATION ON HOPFIELD NETWORK

3. INTRODUCTION 10/3/2012 3 PRESENTATION ON HOPFIELD NETWORK

4. INTRODUCTION 10/3/2012 4 PRESENTATION ON HOPFIELD NETWORK

5. INTRODUCTION 10/3/2012 5 PRESENTATION ON HOPFIELD NETWORK

6. PROPERTIES OF HOPFIELD NETWORK 10/3/2012 6 PRESENTATION ON HOPFIELD NETWORK

7. Consider the noiseless, dynamical model of the neuron shown in fig. 1 The synaptic weights represents conductance’s. The respective inputs represents the potentials, N is number of inputs. These inputs are applied to a current summing junction characterized as follows: Low input resistance. Unity current gain. High output resistance. HOPFIELD NETWORK 10/3/2012 7 PRESENTATION ON HOPFIELD NETWORK

8. Σ ADDITIVE MODEL OF A NEURON NEURAL NETWORK MODEL 10/3/2012 8 PRESENTATION ON HOPFIELD NETWORK

9. HOPFIELD NETWORK 10/3/2012 9 PRESENTATION ON HOPFIELD NETWORK

10. By applying KCL to the input node of the nonlinearity , we get ………..(1) The capacitive term add dynamics to the model of a neuron. Output of the neuron j determined by using the non linear relation The RC model described by the eq. (1) is referred to the additive model HOPFIELD NETWORK 10/3/2012 10 PRESENTATION ON HOPFIELD NETWORK

11. HOPFIELD NETWORK 10/3/2012 11 PRESENTATION ON HOPFIELD NETWORK

12. HOPFIELD NETWORK 10/3/2012 12 PRESENTATION ON HOPFIELD NETWORK

13. HOPFIELD NETWORK Let the sigmoid function be defined by the hyperbolic tangent function Which has slope of . refers as the gain of neuron i . The inverse I/O relation of eq.(3) may be written as ………..(4) 10/3/2012 13 PRESENTATION ON HOPFIELD NETWORK

14. HOPFIELD NETWORK Standard form of the inverse I/O relation for a neuron of unity gain is: We can rewrite the eq. (4) in terms of standard relation as 10/3/2012 14 PRESENTATION ON HOPFIELD NETWORK

15. Plot of (a) Sigmoidal Nonlinearity and (b) its inverse UNKNOWN SYSTEM f (.) Σ Model Output E rror (a) (b) 10/3/2012 15 PRESENTATION ON HOPFIELD NETWORK

16. HOPFIELD NETWORK The energy function of the Hopfield network is defined by: Differentiating E w.r.t . time , we get by putting the value in parentheses from eq.2, we get …………..(5) 10/3/2012 16 PRESENTATION ON HOPFIELD NETWORK

17. HOPFIELD NETWORK The inverse relation that defines in terms of is By using above relation in eq. (5), we have …………..(6) From fig. (b) we see that the inverse I/O relation is monotonically increasing function of the output Therefore, 10/3/2012 17 PRESENTATION ON HOPFIELD NETWORK

18. HOPFIELD NETWORK Also, for all . Hence all the factors that make up the sum on R.H.S. of eq (6) are non-negative. Thus the energy function E defined as 10/3/2012 18 PRESENTATION ON HOPFIELD NETWORK

19. HOPFIELD NETWORK 10/3/2012 19 PRESENTATION ON HOPFIELD NETWORK

20. HOPFIELD NETWORK From eq.(6) the derivative vanishes only if for all j. Thus we can say, expect at fixed point ………(7) The eq.(7) forms the basis for following theorem The energy function E of a Hopfield network is a monotonically decreasing function of time. 10/3/2012 20 PRESENTATION ON HOPFIELD NETWORK

21. HOPFIELD NETWORK EXAMPLE Neuron 1 (N1) Neuron 2 (N2) Neuron 3 (N3) Neuron 4 (N4) Neuron 1 (N1) (N/A) N2->N1 N3->N1 N4->N1 Neuron 2 (N2) N1->N2 (N/A) N3->N2 N4->N2 Neuron 3 (N3) N1->N3 N2->N3 (N/A) N4->N3 Neuron 4 (N4) N1->N4 N2->N4 N3->N4 (N/A) Connection of Hopfield Neural Network A Hopfield Neural network: 10/3/2012 21 PRESENTATION ON HOPFIELD NETWORK

22. The connection weights put into this array, also called a weight matrix, allow the neural network to recall certain patterns when presented. For example, the values shown in Table below show the correct values to use to recall the patterns 0101 . HOPFIELD NETWORK EXAMPLE Neuron 1 (N1) Neuron 2 (N2) Neuron 3 (N3) Neuron 4 (N4) Neuron 1 (N1) -1 1 -1 Neuron 2 (N2) -1 -1 1 Neuron 3 (N3) 1 -1 -1 Neuron 4 (N4) -1 1 -1 Weight Matrix used to recall 0101. 10/3/2012 22 PRESENTATION ON HOPFIELD NETWORK

23. Calculating The Weight Matrix Step 1 : Convert 0101 to bipolar Bipolar is nothing more than a way to represent binary values as –1’s and 1’s rather than zero and 1’s. To convert 0101 to bipolar we convert all of the zeros to –1’s. This results in: 0 = -1 1 = 1 0 = -1 1 = 1 The final result is the array (-1, 1, -1, 1) 10/3/2012 23 PRESENTATION ON HOPFIELD NETWORK

24. Calculating The Weight Matrix Step 2 : Multiply (-1, 1, -1, 1) by its Inverse For this step we will consider -1, 1, -1, 1 to be a matrix. Taking the inverse of this matrix we have. Now, multiply these two matrices -1 X (-1) = 1 1 X (-1) = -1 -1 X (-1) = 1 1 X (-1) = -1 -1 X 1 = -1 1 X 1 = 1 -1 X 1 = -1 1 X 1 = 1 -1 X (-1) = 1 1 X (-1) = -1 -1 X (-1) = 1 1 X (-1) = -1 -1 X 1 = -1 1 X 1 = 1 -1 X 1 = -1 1 X 1 = 1 10/3/2012 24 PRESENTATION ON HOPFIELD NETWORK

25. Calculating The Weight Matrix And the matrix is: Step 3 : Set the Northwest diagonal to zero The reason behind this is, in Hopfield networks do not have their neurons connected to themselves. So positions [1][1], [2][2], [3][3] and [4][4] in our two dimensional array or matrix, get set to zero. This results in the weight matrix for the bit pattern 0101 . 10/3/2012 25 PRESENTATION ON HOPFIELD NETWORK

26. Recalling Pattern To do this we present each input neuron, with the pattern. Each neuron will activate based upon the input pattern. For example, when neuron 1 is presented with 0101 its activation will be the sum of all weights that have a 1 in input pattern. The activation of each neuron is: The final output vector then (-2,1,-2,1) a b c d a+b+c+d N1 -1 -1 -2 N2 1 1 N3 -1 -1 -2 N4 1 1 10/3/2012 26 PRESENTATION ON HOPFIELD NETWORK

27. Recalling Pattern 10/3/2012 27 PRESENTATION ON HOPFIELD NETWORK

28. Recalling Pattern 10/3/2012 28 PRESENTATION ON HOPFIELD NETWORK

29. APPLICATION 10/3/2012 29 PRESENTATION ON HOPFIELD NETWORK

30. References Jacek M. Zurada , Introduction To Artificial Neural Systems (10 th edition) Simon Haykin , Neural Networks (2 nd edition) Satish Kumar, Neural Networks; A Classroom Approach (2 nd Edition) http://www.learnartificialneuralnetworks.com/hopfield.html http://www.heatonresearch.com/articles/2/page6.html http://www.thebigblob.com/hopfield-network/#associative-memory http://www.dsi.unive.it/~pelillo/Didattica/RetiNeurali/Introduction_To_ANN_lesson_6.pdf . http://en.wikipedia.org/wiki/Hopfield_network . 10/3/2012 30 PRESENTATION ON HOPFIELD NETWORK

31. THANK YOU. 10/3/2012 31 PRESENTATION ON HOPFIELD NETWORK