The User Experience of Chatbots - Nielsen Norman Group

Neilsen Norman Group's report on chatbot UX

1. 1
The User Experience of Chatbots
Nielsen Norman Group
/김민준
x 2018 Fall

2. UX LAB | SEOUL NATIONAL UNIVERSITY 2

3. UX LAB | SEOUL NATIONAL UNIVERSITY 3

4. UX LAB | SEOUL NATIONAL UNIVERSITY
6 QUALITIES OF AN INTELLIGENT ASSISTANT
4
•Voice Input
•Natural Language Processing
•Voice Output
•Intelligent Interpretation
•Agency
•Integration of all the above

5. UX LAB | SEOUL NATIONAL UNIVERSITY
6 QUALITIES OF AN INTELLIGENT ASSISTANT
5
•Voice Input
•Natural Language Processing
•Voice Output
•Intelligent Interpretation
•Agency
•Integration of all the above
Chatbots

6. UX LAB | SEOUL NATIONAL UNIVERSITY
CHATBOT GROWTH AND POPULARITY
6
1.5ý ŽÈ8 SÞ È : zE (CS)
2.Conversational ²: (WeChat,    ß, Messenger, a)D k (w œ\D ¿•>(
3.A8 Google Assistant, Siri, AlexaD “Ù”5ý ¤?Þ× Ý ¸\
1.Customer-service Bots (ex. 7ï A, UPS, a)
2.Interaction Bots (ex. ™A)
A>Ñ ,D Chatbot

7. UX LAB | SEOUL NATIONAL UNIVERSITY
TWO DIFFERENT TYPES OF CHATBOTS
7
Customer-service Bot Interaction Bot

8. UX LAB | SEOUL NATIONAL UNIVERSITY
STUDY
8
8 US participants
asked them to perform a set of chat-related tasks on mobile (5) and desktop (3).
Tasks
Some of the tasks involved chatting for customer-service purposes with either humans or bots,
and others targeted Facebook Messenger or SMS-based chatbots.

9. UX LAB | SEOUL NATIONAL UNIVERSITY
ATTITUDES TOWARDS BOTS
9
Customer-service bots were perceived as generally less helpful
Customer Service Bots
ranged from neutral to slightly positive*
* Most participants in our US studies had not interacted with such bots before
* Chinese participants were familiar with them.
Offers some advantages: speed
People felt that access to a human shows that the company cares about its customers
“If you have a human [for customer service], it means that you care enough;
if you have a bot, you should have a really good one.”

10. UX LAB | SEOUL NATIONAL UNIVERSITY
ATTITUDES TOWARDS BOTS
10
Interaction bots were usually easily identifiable as bots,
but customer-service bots were harder to recognize.
Interaction Bots
Some businesses do not always disclose upfront to their customers
that they are interacting with a bot. We believe that this is a mistake.
õ?
because they could calibrate both their expectations and their language to a bot.
when users realized they were talking to a bot, they tended to be more direct,

11. UX LAB | SEOUL NATIONAL UNIVERSITY
UIS FOR INTERACTION CHATBOTS
11

12. UX LAB | SEOUL NATIONAL UNIVERSITY
UIS FOR INTERACTION CHATBOTS
12
Links and buttons worked best when they were clearly signaled.
People expected to be able to click on almost any nontext element that was displayed
Interaction Bots

13. UX LAB | SEOUL NATIONAL UNIVERSITY
LINEAR FLOW
13
Interaction chatbots seemed to best resemble Alexa skills:
they were designed to guide the user through a small number of tasks.
Domino’s Pizza Bot

14. UX LAB | SEOUL NATIONAL UNIVERSITY
LINEAR FLOW
14
•When the user is compliant with the flow and provides ‘legal’ answers … 

the experience feels successful and smooth.
Pros
Cons
•Bots have trouble recovering from a problem or an unexpected input
•forces users to start over at the top of the tree and do more work than necessary 

in order to obtain an answer.
•Some bots had trouble making assumptions and establishing the context of a query.
•Bots were also generally not able to take advantage of previously entered information 

when a new task was started

15. UX LAB | SEOUL NATIONAL UNIVERSITY
LINEAR FLOW
15
More Cons
•With simple linear processes that tackle complex tasks, users fear omissions. 

They doubt that the best answer can be gotten through the bot.
•Indeed, bots only have limited display space available, 

and it is unlikely that they would be able to show users all the matches for a query.

16. UX LAB | SEOUL NATIONAL UNIVERSITY
LANGUAGE
16
Politeness
•participants in this study generally dropped the politeness markers 

(“Please,” “Thank you,” indirect language such as “Would it be possible to…”) 

if they were aware that they were interacting with a bot.
;t WS> î× “direct” à ????I? ??I? x?
Chatbots are less anthropomorphized than Siri, Alexa, Google Assistant

17. UX LAB | SEOUL NATIONAL UNIVERSITY
LANGUAGE
17
More Suggestions
•Owning the failure and offering an escape hatch (phone number or a live agent) 

were generally perceived favorably.
Personas
•Flo from Progressive was playful and light. However, one user was disappointed because 

he felt the bot was not able to sustain that tone consistently throughout the conversation.

18. UX LAB | SEOUL NATIONAL UNIVERSITY
LANGUAGE
18

19. UX LAB | SEOUL NATIONAL UNIVERSITY
WHAT IS THE VALUE PROPOSITION FOR CHATBOTS?
19
Companies are better off investing their money in the existing, well-established channels: 

improving the UX of your website or app will bring you higher return on investment than creating a
chatbot that will get little use.
Do chatbots have any advantages?
In theory, interaction chatbots can be useful for power users who may do 

the same type of tasks repeatedly — provided that they discover this channel.
But then again — a well-designed website will have shortcuts in place to help power users
In their current embodiment, they just have one: less information overload.

20. UX LAB | SEOUL NATIONAL UNIVERSITY
UX GUIDELINES FOR DESIGNING CHATBOTS
20
•Be upfront about using a bot and not a human.
•Clearly tell people what tasks the bot can do. Make sure you don’t create false expectations.
•Don’t be overly ambitious: create bots for simple tasks. 

Complexity is not well handled in the limited bot interface.
•Tolerate typos and ambiguity.
•Allow people to interact with the bot both through free-text input and selection of links.
•Allow sorting and filtering to let people narrow down through results.
•Save information from one task to the next.
•Program some flexibility into the bot: infer context and allow people to jump forward 

and backward in the linear flow.
•Be honest about not understanding. Offer an escape hatch in the form of a real human, 

a phone number, or a link to a different interaction channel.

21. Thank you..!
Q & A
Minjoon Kim
Ph.D. Student | Seoul National University || User eXperience Lab
[email protected]
minjoonkim.github.io