Beyond the Hype Demystifying AI for Practical Market Research

Let’s understand how LLMs work, so we can decide where and how to use them.

1. Alok Jain / Founder - Reveal (doreveal.com) / Oct 2024
Beyond the Hype
Demystifying AI for Practical
Market Research
Let’s understand how LLMs work, so we
can decide where and how to use them.
Copyright, Alok Jain

2. Alok
8 Years - Application of AI
Including LLMs and other forms of machine learning.
28 Years - Research and Design
Capital One, The World Bank, Carfax, Centene, Cisco,
Microsoft and more.
Founder , CEO - Reveal
(doreveal.com)
AI Assisted synthesis of qualitative research.
Sounds like “oak”

3. Guess the next word.
Let’s play a game

4. I’m
No wrong answers. Just think. No need to share.

5. I’m gonna
Hit ! on zoom if you
guessed the same word.

6. I’m gonna make
Hit ! on zoom if you
guessed the same word.

7. I’m gonna make him
Hit ! on zoom if you
guessed the same word.

8. I’m gonna make him an offer
Hit ! on zoom if you
guessed the same words.

9. I’m gonna make him an offer
he
Hit ! on zoom if you
guessed the same word.

10. I’m gonna make him an offer
he can’t
Hit ! on zoom if you
guessed the same word.

11. I’m gonna make him an offer
he can’t refuse.
Hit ! on zoom if you
guessed the same word.

12. This is how LLMs “think”.
I’m
Alok
young
alive
gonna
…
sing
play
make
…
fun
…
him
…
an
…
offer …
LLM = Large Language Model

13. There is more..
https://www.youtube.com/watch?v=zxQyTK8quyY
Positional encoding
Self-attention
Residual connections
Normalization…

14. LLMs don’t understand the
language or the meaning as a
human does.
LLMs understand the
structure of the language.
It’s a mathematical model.
I’m
Alok
young
alive
gonna
…
sing
play
make
…
fun
…
him
…
an

15. How do words become numbers?
Instead of finding related words,
let’s find similar fruits.

16. How do words become numbers?
Instead of finding related words,
let’s find similar fruits.
Circular
Red
Yellow
Cylindrical

17. How do words become numbers?
Circular
Red
Yellow
Cylindrical

18. How do words become numbers?
Circular
Red
Yellow
Cylindrical
5
2
Apple[5,2]
Orange[4,1]
Banana[1,7]

19. How do words become numbers?
Circular
Red
Yellow
Cylindrical
5
2
Apple[5,2]
Orange[4,1]
Banana[1,7]
Many more dimensions

20. LLM
Large Language Models
•Understand the structure of the
language
•Mathematical representation
•Training data
•Slow to update, singular model
Humans
•Understand the meaning behind the
words
•Natural curiosity
•Self awareness and constantly
learning
•Individual, always evolving model
vs Human Brain

21. Applying AI to Research
•AI Moderated Research
•Synthetic Data
•AI-Assisted Synthesis

22. Example 1: AI Moderated Research
AI-generated
follow-up

23. AI Moderated Research - Evaluative
RESEARCHER ENTERS QUESTIONS.
“How was your experience
using …”
RESPONDENT SAYS
“I was impressed
with the speed and
ease of use”
FOLLOW-UP INSTRUCTIONS
“Ask for more details
related the parts that
respondent likes or dislikes”
“impressed”
“likes”
FOLLOW-UP QUESTION
“What aspects of
Reveal's speed
impressed you the
most?

24. AI Moderated Research - Generative (Strategic)
RESEARCHER ENTERS QUESTIONS.
“What is your current
process to …”
RESPONDENT SAYS
“The process is
managed by .. my
team focusses on..”
FOLLOW-UP INSTRUCTIONS
“Ask why the process is set
up this way.”
FOLLOW-UP QUESTION
?

25. AI Moderated Research
Strategic Evaluative
More Unknowns, Needs more judgment Fewer Unknowns, All pre-defined
More unknown
responses. Requires
judgment and curiosity.
Mostly known spectrum
of responses. AI can
work on follow-up specs
Checkout the experiments by Andrea Knight Dolan:
https://www.aknightux.com/thoughts/embracing-ai-moderated-interviews

26. Example 2: Synthetic Data
LLM Data
You are introducing a
new drink that is rich
on protein, low on
sugar, it tastes okay.
Generate a tabular
comparison of how
various target personas
would feel about such
a drink product.
RESEARCHER ASKS

27. Example 2: Synthetic Data
Low quality data, cannot be enhanced much High quality data, can be enhanced meaningfully
AI Enhanced
AI Enhanced

28. Example 2: Synthetic Data
Data Low
Data Density High
Fewer gaps and more
patterns to inform AI.
More Gaps, Greater
chance of error.

29. Example 3: AI-Assisted Synthesis
Language Relationships
I’m
Alok
young
alive
gonna
…
sing
play
make
…
fun
…
him
Training on Research Synthesis Data

30. Example 3: AI-Assisted Synthesis
Research Question:
Abortion and Reproductive Rights

31. Evaluate Your Research Needs
Data Low Fidelity
Data High Fidelity
Interpret
Meaning
Interpret
Words

32. Thank you.
Alok Jain / Founder - Reveal
doreveal.com