Intelligent systems and neutral networks lec2.pptx

Neural network

1. بسم الله الرحمن الرجيم intelligent system Instructor: dr:Eiman omer [email protected] Lecture (2 )

2. Idealized neurons To model things we have to idealize them . Idealization removes complicated details that are not essential for understanding the main principles Allows us to apply mathematics and to make analogies to other, familiar systems. Once we understand the basic principles, its easy to add complexity to make the model more faithful

3. Linear neurons These are simple but computationally limited If we can make them learn we may get insight into more complicated neurons

4. Binary threshold neurons McCulloch-Pitts (1943): influenced Von Neumann! First compute a weighted sum of the inputs from other neurons Then send out a fixed size spike of activity if the weighted sum exceeds a threshold. Maybe each spike is like the truth value of a proposition and each neuron combines truth values to compute the truth value of another proposition!

5. Linear threshold neurons These have a confusing name . They compute a linear weighted sum of their inputs The output is a non-linear function of the total input

6. Sigmoid neurons These give a real-valued output that is a smooth and bounded function of their total input. Typically they use the logistic function They have nice derivatives which make learning easy . If we treat as a probability of producing a spike, we get stochastic binary neurons .

7. Types of connectivity Feed forward networks These compute a series of transformations Typically, the first layer is the input and the last layer is the output. Recurrent networks These have directed cycles in their connection graph. They can have complicated dynamics. More biologically realistic .

8. Simple Neural Nets for Pattern Classification

9. McCulloch-Pitts networks Threshold Logic and McCulloch-Pitts networks based on Threshold Logic. McCulloch-Pitts networks can be use do build networks that can compute any logical function. Although the idea of thresholds and simple computing units are biologically inspired, the networks we can build are not very relevant from a biological perspective in the sense that they are not able to solve any interesting biological problems .

10. The computing units are too similar to conventional logic gates and the networks must be completely specified before they are used. There are no free parameters that can be adjusted to solve different problems. Learning can be implemented in McCulloch-Pitts networks by changing the connections and thresholds of units. • Changing connections or thresholds is normally more difficult that adjusting a few numerical parameters, corresponding to connection weights.

11. In this lecture, we focus on networks with weights on the connections and we will look at a few simple/early networks types proposed for learning weights. These are single-layer networks and each one uses it own learning rule. The network types we look at are: Hebb networks , Perceptrons and Adaline networks.

12. Learning Process/Algorithm In the context of artificial neural networks, a learning algorithm is an adaptive method where a network of computing units self organizes by changing connections weights to implement a desired behavior . Learning takes place when an initial network is “shown” a set of examples that show the desired input-output mapping or behavior that is to be learned. This is the training set. As each example is shown to the network, a learning algorithm performs a corrective step to change weights so that the network slowly “learns” to produce the desired response

13. A learning algorithm is a loop when examples are presented and corrections to network parameters take place. This process continues till data presentation is finished. In general, training should continue till one is happy with the behavioral performance of the network, based on some metrics.

14. Classes of Learning Algorithms There are two types of learning algorithms: supervised and unsupervised . Supervised Learning: – In supervised learning, a set of “labeled” examples is shown to the learning system/network. Unsupervised Learning: - In unsupervised learning, The exact output that the network should produce is unknown.

15. Classification Patterns or examples to be classified are represented as a vector of features (encoded as integers or real numbers in NN ) Pattern classification: • Classify a pattern to one of the given classes • It is a kind of supervised learning

16. Separability in Classification Separability of data points is a very important concept in the context of classification. • Assume we have data points with two dimensions or "features”. • Data classes are linearly separable if we can draw a straight line or a (hyper-)plane separating the various classes .

17. General Architecture The basic architecture of the simplest neural network to perform pattern classification consists of a single layer of inputs and a single output unit. This is a single layer architecture.

18. Decision region/boundary Consider a single-layer neural network with just two inputs. We want to find a separating line between the values for x1 and x2 for which the net gives a positive response from the values for which the net gives a negative response.

19. Hebb Nets