Search @ Spotify

At the BCS Search Solutions 2018, I gave a talk about work on search we are doing at Spotify. The talk described what search means in the context of Spotify, how it differs what we know about search, and the challenges associated with understanding user intents and mindsets in an "entertainment" context. The talk also discussed various efforts at Spotify to understand why users submit search queries, what they expect, how they assess their search experience, and how Spotify responds to these search queries. This is work done with many colleagues at Spotify in Boston, London, New York and Stockholm, and our wonderful summer interns.

1. Search @ Spotify.
Mounia Lalmas
and many others at Spotify Boston,
London, New York & Stockholm



November 27, 2018

2. 1
2
3
4
5
Outline
About Spotify.
Search at Spotify.
Infrastructure for search.
Search user journey.
Satisfaction in search.
6
Search as recommendation.

3. About Spotify.

4. Spotify’s mission is to
unlock the potential of
human creativity — by
giving a million creative
artists the opportunity to
live off their art and
billions of fans the
opportunity to enjoy and
be inspired by it.

5. 87 Million
3 Billion + 78 Markets
40 Million +
191 Million
€10 Billion
Number of playlists Spotify is available in
Revenue paid to rightsholders (as at August 31 2018)

Number of songs
Number of subscribers (as at September 30 2018)


Number of active users (as at September 30 2018)

6. http://everynoise.com/

7. User Engagement Mission:
Match fans and artists in a personal and relevant way.
ARTISTS FANS

8. playlists
songs ...
catalog search
browse
users
What does it mean to match fans and artists in
a personal and relevant way?
Artists Fans

9. Search at
Spotify.

10. Large catalog
40M+ songs, 3B+ playlists
2K+ microgenres

Many languages
78 countries

Different modalities
Typed, voice

Heterogeneous content
Music, podcast

Various granularities
Song, artist, playlist

Various goals
Focus, discover, lean-back, mood
Searching for … music

11. Large catalog
40M+ songs, 3B+ playlists
2K+ microgenres

Many languages
78 countries

Different modalities
Typed, voice

Heterogeneous content
Music, podcast

Various granularities
Song, artist, playlist

Various goals
Focus, discover, lean-back, mood
Searching for … audio

12. Large catalog
40M+ songs, 3B+ playlists
2K+ microgenres

Many languages
78 countries

Different modalities
Typed, voice

Heterogeneous content
Music, podcast

Various granularities
Song, artist, playlist

Various goals
Focus, discover, lean-back, mood
Searching for … moods or activities

13. Search is instantaneous … at each keystroke
m my my_ my_f my_fav

14. s sa satt sat sati statis




Search is instantaneous
… the search logs for “satisfaction”
From prefix to query
→ What is the actual query?
→ What is success vs prefix vs query?


prefix
query

15. Infrastructure for
search.

16. Search infrastructure
{q: ‘drake’, user: ‘user1’}
Client
{q: ‘drake’, user: ‘user1’}
Search Service
Search Results
Re-ranking Service
Candidate List
+
Ranked
Candidate List
Retrieval
Service
Candidate List
{q: ‘drake’}
Ranked Candidate List

17. Search results
re-ranking
A prefix
query
A candidate
to be
scored (c
i
)
Metadata
Feature Builder fi,1fi,2...fi,k Scorer si
Ranking model trained on search interaction logs.
Use search sessions that end in a success action as
positive examples.
user, query and item-based features:
-Item popularity
-whether user has searched for this item before
-similarity of the item to the user taste (vector)
-edit distance between prefix query and the
matched item title ...

18. Search
Research.
We discuss three ongoing projects around
understanding how users search for music
to listen to.


Work in progress.
Search user journey
About intent and mindset
Satisfaction in search
About success and effort
Search as recommendation
About voice
1
2
3

19. Search user
journey.

20. Overview of the user journey in search
TYPE/TALK
User
communicates
with us
20
CONSIDER
User evaluates
what we show
them

DECIDE
User ends the
search session
INTENT
What the user
wants to do
MINDSET
How the user
thinks about
results

21. Intents … what the user wants to do
●Play background music
●Fit an activity
●Listen with others
●Prepare for a concert
●Keep up with current
music here and abroad
●Try recommended
music from friends
●Hear a song stuck in
your head
●Fit a mood
●Keep up with favorite
artists
●Explore a niche genre

LISTEN
Have a listening session
ORGANIZE
Curate for future listening
SHARE
Connect with friends
FACT CHECK
Find specific information
●Make a playlist
●Build library
●Follow artists
●Follow playlists


●Send music to a friend
●Follow a friend


●Check own knowledge
●Gather information
●Learn about concerts



Most common Least Common
based on qualitative research

22. Mindsets … how the user thinks about results
FOCUSED
One specific thing in mind
OPEN
A seed of an idea in mind
EXPLORATORY
A path to explore

●Find it or not
●Quickest/easiest
path to results is
important

●From nothing good
enough, good enough
to better than good
enough
●Willing to try things out
●But still want to fulfil
their intent

●Difficult for users to
assess how it went
●May be able to answer
in relative terms
●Users expect to be
active when in an
exploratory mindset
●Effort is expected
Most common Least Common
based on qualitative research

23. A user can approach any intent with any mindset
FOCUSED
One specific thing in mind
OPEN
A seed of an idea in mind
EXPLORATORY
A path to explore
LISTEN
Have a listening session
ORGANIZE
Curate for future listening
SHARE
Connect with friends
FACT CHECK
Find specific information
EXPLORATORY mindset seems rare and likely better served by other features such as Browse.
LISTEN and ORGANIZE are most prominent intents & associated with lean-back vs lean-in behavior.

24. Focused
mindset.

When users know what they want
to find.

The pull paradigm and how it
translates to the music context.

Findings from large-scale in-app
survey + behavioral analysis.

65% of searches were focused.

When users search with a Focused Mindset

Put MORE effort in search.

Scroll down and click on lower rank results.

Click MORE on album/track/artist and LESS
on playlist.

MORE likely to save/add but LESS likely to
stream directly.

Understanding intents helps us understand
search satisfaction (even within a mindset).

25. Satisfaction in search.

26. What drives
user satisfaction
in search?
Findings from qualitative
research.

Focused mindset.


User satisfaction translates into success
and effort.
Good experience is finding, ideally with
little effort.
Bad experience is not finding, not knowing
how to find, or struggling while searching.
Users prioritize success and given success,
they want to minimize effort.

27. Mapping success and effort metrics with the search
user journey
DECIDE
User ends the
search session.

TYPE
User
communicates
with us.
CONSIDER
User evaluates
search results.

“Success” metrics associate with the
decide phase
“Effort” metrics associate with the type and consider
phases

28. Examples of success and effort metrics
DECIDE

TYPE
number of
deletions, ...
CONSIDER
back button
clicks, first and
last click
position, ...

Time to success

“Success” metrics “Effort” metrics

stream
LISTEN
Have a listening session
add to a playlist, save
into a collection, follow
an artist, follow a
playlist, ...
ORGANIZE
Curate for future listening

29. Satisfaction metrics for search (focus mindset)
DECIDE
User ends the
search session.

TYPE
User
communicates
with us.
CONSIDER
User evaluates
search results.

“Success” metrics associate with the
decide phase
“Effort” metrics associate with the type and consider
phases
≅DECIDE metrics ∆ (TYPE metrics ⨁ CONSIDER metrics)

30. Satisfaction in
search.
Going beyond the focused
mindset.


Success and effort in search
shaped by mindsets.
Focused: one specific thing in mind
Open: a seed of an idea in mind

User can approach any intent with
any mindset.

Automatically identify mindsets.
Automatically identify intents.

Explore satisfaction metrics that
incorporates success and effort
with respect to intent and mindset.

31. Search as
recommendation.

32. Users ask for Spotify to play music, without saying
what they would like to hear (open mindset)


Play
Spotify
Play music
Play music
from Spotify
Play me
some music
Play the
music
Play my
Spotify
Play some
music on
Spotify
Play some
music
Play music
on Spotify
Search by
voice.
A type of push paradigm
and how it translates to the
music context.

Findings from qualitative
research.

33. Why users provide non-specific
queries
Open mindset in voice
Private & Confidential, For Internal Use
Only
Why users do not provide a
non-specific query
They want to effortlessly start a lean
back listening session.
They do not want to make a content
decision.
They want to resume a previous
listening session.
They are curious and want to playfully
engage with Spotify.
They did not know that they could
engage with Spotify this way.
They cannot predict what they will get,
and are not willing to give up control.
They have specific tastes, and do not
trust that something that matches their
listening habits will be returned.

34. Search as
recommendation.
Delivering for the open
mindset.



Non-specific querying is a way for a user
to effortlessly start a listening session
via voice.
Non-specific querying is a way to remove
the burden of choice when a user is open
to lean-back listening.
User education matters as users will not
engage in a use-case they do not know
about.
Trust and control are central to a positive
experience. Users need to trust the system
enough to try it out.

35. Some final
words.

36. Searching
for music.
Qualitative & quantitative research has
helped bring a deeper understanding
into how and why users search for
music and how they assess the quality
of their search experience.

Some of these have been and are being
validated and expanded through more
research.

Input to ranking algorithms and metrics.

Much more to come.
1
Multimodality
pull vs push
Satisfaction
success vs effort
Intents
listen vs organize
Mindsets
focused vs open
2
3
4

37. Join the band!
https://www.spotifyjobs.com/search-jobs/