natural language processing --INTRODUCTION-unit-1

nlp unit 1

1. NaturalLanguage Processing UNIT-1 Introduction 1

2. UNIT I SYLLABUS Speech: Phonetics Speech Sounds and Phonetic Transcription. Articulator Phonetics Phonological categories and Pronunciation Variation Acoustic phonetics and signals. Automatic Speech Recognition Architecture. Overview and Language Modeling : OVERVIEW: Origins and challenges of NLP-Language and Grammar- Processing Indian Languages-NLP Applications-Information Retrieval. 2

3. W hat is NLP? 3

4. Phases Of NLP 4

5. Phases Of NLP 1. Lexical Analysis and Morphological This phase scans the source code as a stream of characters and converts it into meaningful lexemes(tokens). It divides the whole text into tokens like Words , sentences ,paragraphs. 2 . Syntactic Analysis (Parsing) Syntactic Analysis is used to check grammar, word arrangements, and shows the relationship among the words. Example :  Agra goes to the Poonam In the real world, Agra goes to the Poonam , does not make any sense, so this sentence is rejected by the Syntactic analyzer . 3 . Semantic Analysis Semantic analysis is concerned with the meaning representation. It mainly focuses on the literal meaning of words, phrases, and sentences . 5

6. Phases Of NLP 3 . Semantic Analysis Semantic analysis is concerned with the meaning representation. It mainly focuses on the literal meaning of words, phrases, and sentences . 4. Discourse Integration Discourse Integration depends upon the sentences that proceeds it and also invokes the meaning of the sentences that follow it . 5. Pragmatic Analysis Pragmatic is the fifth and last phase of NLP. It helps you to discover the intended effect by applying a set of rules that characterize cooperative dialogues. For Example:  "Open the door" is interpreted as a request instead of an order. 6

7. NLP Applications Small Spelling correction Medium Word-sense disambiguation Named entity recognition Information retrieval Large Question answering Conversational agents Machine translation 7

8. NLP Applications 1. Question Answering Question Answering focuses on building systems that automatically answer the questions asked by humans in a natural language . 8

9. NLP Applications 2 . Spam Detection Spam detection is used to detect unwanted e-mails getting to a user's inbox. 9

10. NLP Applications 3 . Sentiment Analysis Sentiment Analysis is also known as  opinion mining . It is used on the web to analyze the attitude, behavior, and emotional state of the sender. This application is implemented through a combination of NLP (Natural Language Processing) and statistics by assigning the values to the text (positive, negative, or natural), identify the mood of the context (happy, sad, angry, etc.) 10

11. NLP Applications 4 . Machine Translation Machine translation is used to translate text or speech from one natural language to another natural language. Example :  Google Translator 11

12. NLP Applications 5 . Spelling correction Microsoft Corporation provides word processor software like MS-word, PowerPoint for the spelling correction. 12

13. NLP Applications 6 . Speech Recognition Speech recognition is used for converting spoken words into text. It is used in applications, such as mobile, home automation, video recovery, dictating to Microsoft Word, voice biometrics, voice user interface, and so on. 7. Chatbot Implementing the Chatbot is one of the important applications of NLP. It is used by many companies to provide the customer's chat services . 13

14. NLP Applications 8. Information extraction Information extraction is one of the most important applications of NLP. It is used for extracting structured information from unstructured or semi-structured machine-readable documents. Natural Language Understanding (NLU) It converts a large set of text into more formal representations such as first-order logic structures that are easier for the computer programs to manipulate notations of the natural language processing. 14

15. Difference between NLU and NLP 15 NLU(Natural Language Undrstanding ) NLG(Natural Language Generation) NLU is the process of reading and interpreting language. NLG is the process of writing or generating language. It produces non-linguistic outputs from natural language inputs. It produces constructing natural language outputs from non-linguistic inputs.

16. Why NLU is Hard Natural language is extremely rich in form and structure, and very ambiguous . How to represent meaning, Which structures map to which meaning structures. Ambiguity: input can mean many different things Lexical (word level) ambiguity -- different meanings of words Syntactic ambiguity -- different ways to parse the sentence Interpreting partial information -- how to interpret pronouns Contextual information -- context of the sentence may affect the meaning of that sentence. Many input can mean the same thing. Interaction among components of the input. Noisy input (e.g. speech) 16

17. Difference between Natural Language and Computer Language 17 Natural Language Computer Language Natural language has a very large vocabulary. Computer language has a very limited vocabulary. Natural language is easily understood by humans. Computer language is easily understood by the machines. Natural language is ambiguous in nature. Computer language is unambiguous.

18. Difference between Natural Language and Computer Language 18 Phonetics and phonology The study of language sounds Ecology The study of language conventions for punctuation, text mark-up and encoding Morphology The study of meaningful components of words Syntax The study of structural relationships among words Lexical semantics The study of word meaning Compositional semantics The study of the meaning of sentences Pragmatics The study of the use of language to accomplish goals Discourse conventions The study of conventions of dialogue