Our presentation at Insights Career Network (ICN) - Aug 1

Presentation about how do the LLMs work & What are they good at? Includes walkthrough of AI concepts in easy to understand format and examples.

1. Alok Jain. Founder, Reveal (doreveal.com)
LLMs in the
Research Domain
How do the LLMs work & What are they good at?
Copyright, Alok Jain

2. Alok
8 Years - Application of AI
Including LLMs and other forms of machine learning.
28 Years - Research and Design
Capital One, The World Bank, Carfax, Centene, Cisco,
Microsoft and more.
Founder , CEO - Reveal
(doreveal.com)
AI Copilot of synthesis of qualitative research.
Rhymes with “oak”

3. AI > GenAI > LLM

4. How do LLMs work?

5. LLMs work
by finding the
next word.

6. Kamala
Find the next word

7. Kamala Harris

8. Kamala Harris is a
Woman
Vice President
Wife
Howard Alum
Democrat
..

9. Kamala Harris is
the 49th
of the United States.

10. Kamala Harris is
the 49th Vice President
of the United States.

11. The Prime
Minister
of India is

12. The Prime
Minister
of India isModi.

13. Finding the next word
Data Context

14. LLMs use math
to understand data
and build context.

15. LLMs use mathematics
Instead of finding related words,
let’s find similar fruits.

16. LLMs use mathematics
Instead of finding related words,
let’s find similar fruits.
Circular
Red
Yellow
Cylindrical

17. LLMs use mathematics
Circular
Red
Yellow
Cylindrical

18. LLMs use mathematics
Circular
Red
Yellow
Cylindrical
5
2
Apple[5,2]
Orange[4,1]
Banana[1,7]

19. LLMs use mathematics
Circular
Red
Yellow
Cylindrical
5
2
Apple[5,2]
Orange[4,1]
Banana[1,7]

20. LLMs use mathematics
Circular
Red
Yellow
Cylindrical
5
2
Apple[5,2]
Orange[4,1]
Banana[1,7]

21. LLMs use mathematics
Circular
Red
Yellow
Cylindrical
5
2
Apple[5,2]
Orange[4,1]
Banana[1,7]
Many more dimensions

22. There is more..
https://www.youtube.com/watch?v=zxQyTK8quyY
Positional encoding
Self-attention
Residual connections
Normalization…

23. How do
products like Reveal use LLMs?

24. Your Data
This process is specific to Reveal (doreveal.com).
Prep for LLMs
Transcribe
Multi-lingual
PHI/PII Redaction
Identify Moderators &
Participants
Extract Conversations
Setup Prompts
LLM
Review and
Process
Handle any errors
Review response
quality
Associate with
appropriate research
elements

25. Your Data
This process is specific to Reveal (doreveal.com).
Prep for LLMs
Transcribe
Multi-lingual
PHI/PII Redaction
Identify Moderators &
Participants
Extract Conversations
Setup Prompts
Eliminate Bias and
Hallucinations
LLM
Review and
Process
Handle any errors
Review response
quality
Associate with
appropriate research
elements

26. Opportunities to be more inclusive
•Diversity of thought, Reduce human bias.
•Speed enables explorations of multiple perspectives.
•Multi-Lingual studies.

27. Data Privacy and Security
Data Storage
Encrypt data at rest and in-transit.
Use best-in-class infrastructure.
Data Access
Only you and your team.
Vendor team with your permission.
Data Usage
Only to provide you the service.
Not to train any models.
Data Retention
Tied to your account, giving you the
Control.
Remember the sub-processors. Ask to sign an MNDA.

28. Opportunities to improve privacy
•PII/PHI Identification and Redaction
•AI-generated audio/video (Planned to be released later in 2024)

29. How will AI change our roles?

30. LLMs do a subset of what humans do.
LLM Brain
•Mathematical representation
•Understand the structure of the
language
•Training data
•Slow to update, singular model
Human Brain
•Social and Human representation
•Understands the meaning
•Natural curiosity
•Self awareness and constantly learning
•Individual, always evolving model

31. Human thought vs
Human language
https://www.youtube.com/
watch?v=PZekqr9WBww

32. Let’s take an example
AI Moderated Research

33. AI Moderated Research - Evaluative
RESEARCHER ENTERS QUESTIONS.
“How was your experience
using …”
RESPONDENT SAYS
“I was impressed
with the speed and
ease of use”
FOLLOW-UP INSTRUCTIONS
“Ask for more details
related the parts that
respondent likes or dislikes”
“impressed”
“likes”
FOLLOW-UP QUESTION
“What aspects of
Reveal's speed
impressed you the
most?

34. AI Moderated Research - Generative (Strategic)
RESEARCHER ENTERS QUESTIONS.
“What is your current
process to …”
RESPONDENT SAYS
“The process is
managed by .. my
team focusses on..”
FOLLOW-UP INSTRUCTIONS
“Ask why the process is set
up this way.”
FOLLOW-UP QUESTION
?

35. AI Moderated Research
Strategic Evaluative
More Unknowns, Needs more judgment Fewer Unknowns, All pre-defined
More unknown
responses. Requires
judgment and curiosity.
Mostly known spectrum
of responses. AI can
work on follow-up specs
Checkout the experiments by Andrea Knight Dolan:
https://www.aknightux.com/thoughts/embracing-ai-moderated-interviews

36. Another example
Synthetic Data
LLM Data
You are introducing a
new drink that is rich
on protein, low on
sugar, it tastes okay.
Generate a tabular
comparison of how
various target personas
would feel about such
a drink product.
RESEARCHER ASKS

37. Synthetic data is about filling small gaps
Low quality data, cannot be enhanced much High quality data, can be enhanced meaningfully

38. Synthesizing
Qualitative
Data

39. Thank you!
Alok Jain
[email protected]