Unit3-RNN (recurrent neural network).pptx

RNN

1. Finding suitable Neural Networks for Sequence data/Time series data RNN (Recurrent Neural Networks)

2. What is SEQUENCE DATA? Have you ever tried typing in the Gmail? Yes You might have observed google predicting next best suitable words, HAVE YOU? This is shown in the image next (Lighter shade text is showing) predicted text. (AUTOCOMPLETION FEATURE of GMAIL ) Gmail has RNN Deployed in the Text editor which predicts the next words.

3. What is SEQUENCE DATA? (AUTOCOMPLETION FEATURE of GMAIL ) INPUT (x) : Text typed in the Mail box editor/writing space “Not interested at” OUTPUT (y) : this time INPUT (x) : “will let you know if it changes” OUTPUT (y) : in the future WHAT IS YOUR OBSERVATIONS?

4. Google Translation of Sentences Have you ever tried Google Translate? YES Explain in steps How does it work?

5. NER: Named entity recognition In this problem: We give input a loverly Sentence Neural Network will give away: Person Company Names Time etc

6. Sentiment analysis of reviews/Tweets/Facebook/Feedback in e-commerce product reviews/Instagram Feeds/whatsApp

7. What is the common in the above problems? Any guesses? * Sequential data: Examples: What is the Difference between “HOW ARE YOU?” and “YOU ARE HOW?” “LIVE TO EAT” AND “EAT TO LIVE”? * Time series data Examples 1: ECG (Electro Cardio Gram) reading taken from Heartbeats for further study?

8. What is the common in the above problems? Any guesses? * Time series data Examples 2: Deep Fake video detection A Video is series of Picture frames

9. What are the limitation of simple neural networks (ANN) or CNN? Any body tell me?

10. Lets solve the language translate using traditional neural networks I build this Neural network and get output as expected in HINDI Language from ENGLISH INPUT. WHAT IF MY SENTENCE SIZE CHANGES?

11. Lets solve the language translate using traditional neural networks IN CNN AND ANN Decide How many Inputs and Outputs needed in the first before feeding and training the data. CORRECT? In language translation one can give any size sentences to translate?

12. Lets solve the language translate using traditional neural networks CNN/ANN don’t take input as string/characgters instead they take numbers. You have to encode them to numbers? Convert the words in to the vector of vocaubulory and give as input to NN.

13. Weight sharing in CNN

14. Lets solve the language translate using traditional neural networks Parameters are not shared between inputs

15. Sequence/Order dependent data This Nerual network trained for Bank Transactions Fraud detection. Input : Transactin amount Out of country purchase SSN Correct? Determine output: Fraud or NOT In this If we change the position of Input neurons Will that change the Expected Output? i.e. Supply SSN Correct – N1 Trnasaction amount – N2 Out of country purchanse – N3 Output is predicted correctly by ANN

16. Problems in using ANN/CNN for sequence/time variant data

17. Recurrent neural networks (intuition) Lets take an Example of Named Entity Recognition problem. Input a Sentence: Output: Identify Entities (Persons) Not Verbs Input: Dhaval Loves Baby Yoda Output: Recognised Persons: Dhaval, baby Yoda Notations used: Entity: Mark as 1 Not Entity as 0 (SEE the Picture)

18. Rnn (Intuition) Input whole sentence: Dhaval Loves Baby Yoda RNN Reads Word-by-Word Dhaval => Word Vector Given as Input to RNN. We use RNN which has Input: Dhaval One Hidden Layer Sigma and Activation Functions One Output Layer Output: y dhaval

19. Rnn (Intuition) Input whole sentence: Dhaval Loves Baby Yoda RNN Reads Word-by-Word Loves => Word Vector Given as Input to RNN. We use RNN which has Input: Loves y dhaval One Hidden Layer Sigma and Activation Functions One Output Layer Output: y loves Language needs a context to understand accurately and completely. Shown in diagram has this kind of Architecture facilitates to memorize the Language context. Feeding output of previous input to current layer

20. Rnn (Intuition) Input whole sentence: Dhaval Loves Baby Yoda RNN Reads Word-by-Word Loves => Word Vector Given as Input to RNN. We use RNN which has Input: baby y loves One Hidden Layer Sigma and Activation Functions One Output Layer Output: y baby Y loves Carries the previous state of Memory when we feed as input. i.e. “Dhaval Loves” Statement.

21. RNN (INTUITION) This Figure is now showing 4 HIDDEN LAYER 4 HIDDEN LAYERS in REAL only a SINGLE LAYER But a SEQUENCE WORDS given input to the same HIDDEN LAYER as shown below. y dhaval (NAME ENTITY RECOGNITION) NER OUTPUT: Dhaval – 1 Loves – 0 Baby – 1 Yoda – 1

22. RNN REPRESENTATION

23. RNN tRAINING

24. RNN tRAINING Take set of statements and train RNN. Statement1: Input: Dhaval loves baby yoda Expected Output: 1 0 1 1 Setup the RNN Hidden layer Initilize Weights and Activation Functions Feed sentence One Word at a Time. Compute y’ (For Each word) Computer the loss (Dhaval): Loss1 = y – y’ Determine Total Loss. Apply Gradient descent Use Backpropation to adjust weights and Reduce Losses

25. RNN tRAINING Take set of statements and train RNN. Statement1: Input: Ironman Punched on Hulk’s face Expected Output: 1 0 0 1 1 Setup the RNN Hidden layer Initilize Weights and Activation Functions Feed sentence One Word at a Time. Compute y’ (For Each word) Computer the loss (Ironman): Loss1 = y – y’ Determine Total Loss. Apply Gradient descent Use Backpropation to adjust weights and Reduce Losses

26. Rnn training I’ve 1000 sentences in the Training set. I feed 1 sentence at a time to determine Loss function. Feed 2 nd sentence and do same Feed all 1000 sentences and determine Loss function This completely called One EPOCH. Do gradient descent and backpropogation to adjust Weights and Functions. Run as many Epochs as possible to reduce the error and stop. Now you can say your RNN is trained for NER (Name Entity Recognition)

27. Language translation Supply all input language words This is called encoder Once complete Output starts Translation to target language This is called Decoder

28. Deep rnn (Multi layer hidden network)

29. Types of rnn Many – Many RNN Many – Fixed/One One – Many RNN

30. Types of rnn (Example: Many to many) Input: Many words Output: Has many 0s/1s

31. Many-to-many (encoder-decoder) Language Translation: Input: all words are feed completely. Output will start translate.

32. Many-to-fixed/ONE sentiment analysis Produce Reviews given output as Stars Input: Paragraph Output: Highlighting number of Starts

33. Many-to-fixed/ONE sentiment analysis Generic representation of RNN

34. Music generation (One – Many) Input: Sample music Note Output: RNN Produce Melody Generic representation of RNN for One to Many