Google Duplex

Google Duplex is the technology that gives Google Assistant the ability to make phones calls and sound 'human'. It allows certain users to make a restaurant reservation by phone, but instead of the user speaking directly to the restaurant employee, Google Duplex, with the help of Google Assistant, speaks for the user with an AI-based, but human sounding, voice. Using Google’s Duplex technology, Assistant will place a phone call to your chosen restaurant, have a voice conversation with the employee at the other end, and send you a confirmation that the reservation was successful and is set.

1. Google Duplex By Deepak Sanaka

2. Contents Introduction Abstract Context about Google Duplex Architecture DNNs and RNNs Closed domains and Vanishing gradient problem Process Flow

3. Introduction A long-standing goal of human-computer interaction has been to enable people to have a natural conversation with computers, as they would with each other. In recent years, we have witnessed a revolution in the ability of computers to understand and to generate natural speech, especially with the application of deep neural networks (e.g., Google voice search , WaveNet ).

4. Abstract Google Duplex, It is a new technology for conducting natural conversations to carry out “real world” tasks over the phone. The technology is directed towards completing specific tasks, such as scheduling certain types of appointments. For such tasks, the system makes the conversational experience as natural as possible, allowing people to speak normally, like they would to another person, without having to adapt to a machine.

5. Defining a natural conversation A natural conversation can be described with the following characteristics: Speaker is exhibiting goal-directed, cooperative, rational behavior. Speaker is using the appropriate tone. Speaker can understand and control the conversational flow and use the right timin g.

6. What is Google Duplex? Google Duplex is an artificial intelligence (AI) chat agent that can carry out specific verbal tasks, such as making a reservation or appointment, over the phone. It works to conduct natural conversations to accomplish certain types of tasks.

7. Closed domain operation Google Duplex is not able to carry out random casual conversation. Rather, it was trained to autonomously handle three specific types of tasks: Scheduling a hair salon appointment, Making a restaurant reservation, and Asking about the business hours of a store.

8. How does Google Duplex model natural conversations? Duplex uses a deep neural network (DNN); in more complex cases, it makes use of a recurrent neural network (RNN) which is more expensive, but better at modeling language. At the core of Duplex is a recurrent neural network (RNN) designed to cope with these challenges, built using TensorFlow Extended (TFX).

9. Architecture Incoming sound is processed through an Automatic Speech Recognition (ASR) system. This produces text that is analyzed with context data and other inputs to produce a response text that is read aloud through the Text-to-Speech (TTS) system.

10. Deep Neural Networks (DNNs) DNNs involve an input layer, a hidden layer (the matrix of weights which is trained against data), and an output layer capable of producing what can be interpreted as a prediction or a classification.

11. Recurrent Neural Networks (RNNs) RNNs not only ingest the current input, they also ingest their past hidden state as well. This allows for them to learn sequential patterns. “Rolled up” RNN “Unrolled” RNN

12. DNNs versus RNNs DNNs are good at one-shot prediction—if a single observation is all it takes to produce suitable output. However, oftentimes, data comes in sequences, esp. for a language it arrives in a specific sequence. It’s for this reason that RNNs are used. Since it is very important to remember the context when conducting a longer human-like conversation, RNNs became one of the obvious, go-to choices to do the job.

13. Why closed domain operation is important? Closed domains are loosely defined as any setting that has a limited number of conceivable interactions. Any closed domain has a sort of closed (and well-worn) number of conversational paths and options. When a domain is closed, conversations are pigeonholed—the same sorts of conversations occur over and over, building up a stronger dataset for harder-to-reach features such as natural timing, knowing industry/trade slang, and so on.

14. Advantages of closed domain operation It has a number of advantages, but a major one is that it helps Duplex avoid the “ vanishing gradient problem ,” which is an issue for many DNNs and RNNs alike. It increases the sample size for particular conversational paths in Duplex’s training data.

15. Vanishing Gradient Problem When many hidden layers are stacked such as in a multi-layer DNN or between time steps in an RNN, the network begins to “forget” the past. As the network goes through multiple layers of words, the original context gets lost, so it fails to capture the relationship between the words that stand far apart in a conversation. This happens due to the underlying mechanics of backpropagation.

16. Illustration of vanishing gradients Given a closed domain, the number of times one has to look into the past is constrained. Vanishing gradients aren’t as much of an issue if you don’t need to remember much.

17. Understanding Nuances When many hidden layers are stacked such as in a multi-layer DNN or between time steps in an RNN, the network begins to “forget” the past. In the above example, we can see how the meaning of “OK for 4” changes in different contexts.

18. Process Flow

19. Conclusion Allowing people to interact with technology as naturally as they interact with each other has been a long standing promise. Google Duplex takes a step in this direction, making interaction with technology via natural conversation a reality in specific scenarios.

20. References https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html https://willowtreeapps.com/ideas/an-introduction-to-google-duplex-and-natural-conversations

21. Thank You