Sppu engineering artificial intelligence and data science semester 6th Artificial Neural network Unit I.pptx

Sppu University Third year AI&DS Artificial Neural Network unit 1

1. Introduction to ANN Unit I

2. Introduction to ANN The term " Artificial Neural Network " is derived from Biological neural networks that develop the structure of a human brain. Similar to the human brain that has neurons interconnected to one another, artificial neural networks also have neurons that are interconnected to one another in various layers of the networks. These neurons are known as nodes.

3. Biological Neural Network Biological Neural Network (BNN) is a structure that consists of Synapse, dendrites, cell body, and axon. In this neural network, the processing is carried out by neurons. Dendrites receive signals from other neurons, Soma sums all the incoming signals and axon transmits the signals to other cells.

4. Multilayer Feed-Forward Neural Network Multilayer Feed-Forward Neural Network(MFFNN) is an interconnected Artificial Neural Network with multiple layers that has neurons with weights associated with them and they compute the result using activation functions. It is one of the types of Neural Networks in which the flow of the network is from input to output units and it does not have any loops, no feedback, and no signal moves in backward directions that is from output to hidden and input layer. The ANN is a self-learning network that learns from sample data sets and signals, it is based on the function of the biological nervous system. The type of activation function depends on the desired output. It is a part of machine learning and AI, which are the fastest-growing fields, and lots of research is going on to make it more effective.

5. The Architecture of the Multilayer Feed-Forward Neural Network: This Neural Network or Artificial Neural Network has multiple hidden layers that make it a multilayer neural Network and it is feed-forward because it is a network that follows a top-down approach to train the network. In this network there are the following layers: Input Layer:  It is starting layer of the network that has a weight associated with the signals. Hidden Layer:  This layer lies after the input layer and contains multiple neurons that perform all computations and pass the result to the output unit. Output Layer:  It is a layer that contains output units or neurons and receives processed data from the hidden layer, if there are further hidden layers connected to it then it passes the weighted unit to the connected hidden layer for further processing to get the desired result.

6. The input and hidden layers use sigmoid and linear activation functions whereas the output layer uses a Heaviside step activation function at nodes because it is a two-step activation function that helps in predicting results as per requirements. All units also known as neurons have weights and calculation at the hidden layer is the summation of the dot product of all weights and their signals and finally the sigmoid function of the calculated sum. Multiple hidden and output layer increases the accuracy of the output.

7. Introduction to Recurrent Neural Network Recurrent Neural Network(RNN) is a type of  Neural Network  where the output from the previous step is fed as input to the current step. In traditional neural networks, all the inputs and outputs are independent of each other. Still, in cases when it is required to predict the next word of a sentence, the previous words are required and hence there is a need to remember the previous words. Thus RNN came into existence, which solved this issue with the help of a Hidden Layer. The main and most important feature of RNN is its  Hidden state , which remembers some information about a sequence. The state is also referred to as  Memory State  since it remembers the previous input to the network. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output. This reduces the complexity of parameters, unlike other neural networks.

8. How RNN differs from Feedforward Neural Network? Artificial neural networks  that do not have looping nodes are called feed forward neural networks. Because all information is only passed forward, this kind of neural network is also referred to as a  multi-layer neural network . Information moves from the input layer to the output layer – if any hidden layers are present – unidirectionally in a feedforward neural network. These networks are appropriate for image classification tasks, for example, where input and output are independent. Nevertheless, their inability to retain previous inputs automatically renders them less useful for sequential data analysis.

9. Recurrent Neuron and RNN Unfolding The fundamental processing unit in a Recurrent Neural Network (RNN) is a Recurrent Unit, which is not explicitly called a “Recurrent Neuron.” This unit has the unique ability to maintain a hidden state, allowing the network to capture sequential dependencies by remembering previous inputs while processing.  Long Short-Term Memory (LSTM)  and  Gated Recurrent Unit (GRU)  versions improve the RNN’s ability to handle long-term dependencies.

10. Types Of RNN There are four types of RNNs based on the number of inputs and outputs in the network. One to One  One to Many  Many to One  Many to Many 

11. One to One  This type of RNN behaves the same as any simple Neural network it is also known as Vanilla Neural Network. In this Neural network, there is only one input and one output. 

12. One To Many  In this type of RNN, there is one input and many outputs associated with it. One of the most used examples of this network is Image captioning where given an image we predict a sentence having Multiple words. 

13. Many to One  In this type of network, Many inputs are fed to the network at several states of the network generating only one output. This type of network is used in the problems like sentimental analysis. Where we give multiple words as input and predict only the sentiment of the sentence as output. 

14. Many to Many  In this type of neural network, there are multiple inputs and multiple outputs corresponding to a problem. One Example of this Problem will be language translation. In language translation, we provide multiple words from one language as input and predict multiple words from the second language as output.

15. Types Of Activation Function in ANN The biological neural network has been modeled in the form of Artificial Neural Networks   with artificial neurons simulating the function of a biological neuron. The artificial neuron is depicted in the below picture:

16. Types of Activation Function : There are different types of activation functions. The most commonly used activation function are listed below:

17. McCulloch-Pitts Model of Neuron The McCulloch-Pitts neural model, which was the earliest ANN model, has only two types of inputs —  Excitatory and Inhibitory.  The excitatory inputs have weights of positive magnitude and the inhibitory weights have weights of negative magnitude. The inputs of the McCulloch-Pitts neuron could be either 0 or 1. It has a threshold function as an activation function. So, the output signal  y out  is 1 if the input  y sum  is greater than or equal to a given threshold value, else 0. The diagrammatic representation of the model is as follows:

18. Simple McCulloch-Pitts neurons can be used to design logical operations. For that purpose, the connection weights need to be correctly decided along with the threshold function (rather than the threshold value of the activation function). For better understanding purpose, let me consider an example: John carries an umbrella if it is sunny or if it is raining. There are four given situations. I need to decide when John will carry the umbrella. The situations are as follows: First scenario: It is not raining, nor it is sunny Second scenario: It is not raining, but it is sunny Third scenario: It is raining, and it is not sunny Fourth scenario: It is raining as well as it is sunny

19. To analyse the situations using the McCulloch-Pitts neural model, I can consider the  input signals as follows: X 1 : Is it raining? X 2  : Is it sunny? So, the value of both scenarios can be either 0 or 1. We can use the value of both weights X 1  and X 2  as 1 and a threshold function as 1. So, the neural network model will look like:

20. The truth table built with respect to the problem is depicted above. From the truth table, I can conclude that in the situations where the value of  y out  is 1, John needs to carry an umbrella. Hence, he will need to carry an umbrella in scenarios 2, 3 and 4. 

21. The Foundation of Neural Networks: Weights I magine a neural network as a complex web of interconnected nodes, each representing a computational unit known as a neuron. These neurons work together to process information and produce output. However, not all connections between neurons are created equal. This is where weights come into play. Weights are numerical values associated with the connections between neurons. They determine the strength of these connections and, in turn, the influence that one neuron’s output has on another neuron’s input. Think of weights as the coefficients that adjust the impact of incoming data. They can increase or decrease the importance of specific information.During the training phase of a neural network, these weights are adjusted iteratively to minimize the difference between the network’s predictions and the actual outcomes. This process is akin to fine-tuning the network’s ability to make accurate predictions. Let’s consider a practical example to illustrate the role of weights. Suppose you’re building a neural network to recognize handwritten digits. Each pixel in an image of a digit can be considered an input to the network. The weights associated with each pixel determine how much importance the network places on that pixel when making a decision about which digit is represented in the image. As the network learns from a dataset of labeled digits, it adjusts these weights to give more significance to pixels that are highly correlated with the correct digit and less significance to pixels that are less relevant. Over time, the network learns to recognize patterns in the data and make accurate predictions. In essence, weights are the neural network’s way of learning from data. They capture the relationships between input features and the target output, allowing the network to generalize and make predictions on new, unseen data. Basic Learning Laws

22. Biases: Introducing Flexibility and Adaptability While weights determine the strength of connections between neurons , biases provide a critical additional layer of flexibility to neural networks. Biases are essentially constants associated with each neuron. Unlike weights, biases are not connected to specific inputs but are added to the neuron’s output. Biases serve as a form of offset or threshold, allowing neurons to activate even when the weighted sum of their inputs is not sufficient on its own. They introduce a level of adaptability that ensures the network can learn and make predictions effectively. To understand the role of biases, consider a simple example. Imagine a neuron that processes the brightness of an image pixel. Without a bias, this neuron might only activate when the pixel’s brightness is exactly at a certain threshold. However, by introducing a bias, you allow the neuron to activate even when the brightness is slightly below or above the threshold. This flexibility is crucial because real-world data is rarely perfectly aligned with specific thresholds. Biases enable neurons to activate in response to various input conditions, making neural networks more robust and capable of handling complex patterns. During training, biases are also adjusted to optimize the network’s performance. They can be thought of as fine-tuning parameters that help the network fit the data better.