Generative AI for the rest of us

This session was presented at the AWS Community Day in Munich (September 2023). It's for builders that heard the buzz about Generative AI but can’t quite grok it yet. Useful if you are eager to connect the dots on the Generative AI terminology and get a fast start for you to explore further and navigate the space. This session is largely product agnostic and meant to give you the fundamentals to get started.

1. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
Massimo Re Ferrè
Senior Principal Technologist, AWS
Generative AI for the rest of us

2. © 2023, Amazon Web Services, Inc. or its affiliates. 2
Mainframes
Zooming out a bit
Technology wave #1
Data center

3. © 2023, Amazon Web Services, Inc. or its affiliates. 3
Mainframes
Zooming out a bit
Personal Computers
Technology wave #2
Technology wave #1
Data center

4. © 2023, Amazon Web Services, Inc. or its affiliates. 4
Mainframes
Zooming out a bit
Phyisical Servers
Virtual Machines
Personal Computers
Technology wave #2
Technology wave #1
Data center

5. © 2023, Amazon Web Services, Inc. or its affiliates. 5
Mainframes
Zooming out a bit
Phyisical Servers
Virtual Machines
Personal Computers
Technology wave #2
Technology wave #1
Technology delivery model
Data center
Cloud

6. © 2023, Amazon Web Services, Inc. or its affiliates. 6
Mainframes
Zooming out a bit
Phyisical Servers
Virtual Machines
Personal Computers
Containers
Functions
Technology wave #2
Technology wave #1
Technology delivery model
Data center
Cloud

7. © 2023, Amazon Web Services, Inc. or its affiliates. 7
Mainframes
Zooming out a bit
Phyisical Servers
Virtual Machines
Personal Computers
Containers
Functions
Generative AI
Technology wave #3
Technology wave #2
Technology wave #1
Technology delivery model
Data center
Cloud

8. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
What is Generative AI?
8

9. © 2023, Amazon Web Services, Inc. or its affiliates.
What is Generative AI (in simple terms)
9
-Traditional AI/ML: “Is this a picture of Rome or Florence?”
-[ Discriminative ]
-Gen AI: “Compare Rome Vs. Florence for someone interested in history”
-[ Generative ]

10. © 2023, Amazon Web Services, Inc. or its affiliates.
Gen AI “prompt”
10
AT ITS VERY CORE (THE LLM -LARGE LANGUAGE MODEL), GEN AI IS A FAKE. BUT A USEFUL ONE
submit

11. © 2023, Amazon Web Services, Inc. or its affiliates.
This is how I like to think about an LLM
11* or any profession that has nothing to do with a job in IT for that matter
qAn LLM is akin to a … windsurfer professional*
qVery proficient in English
qAnd that had memorized all Wikipedia and all IT forums out there (and a lot more)
qThey know Stack Overflow inside out! But don’t have a window to check the
weather (or a watch to check the time, etc)
qOn their own, they have no relation to reality (beyond what they read)
qBut they are great at generating free form content based on what they know
“have seen"

12. © 2023, Amazon Web Services, Inc. or its affiliates.
Why is Gen AI useful? [ the builder view ]
Source of unstructured knowledge
How can I use this knowledge and reason about it to create a new asset?
An asset being a piece of code, a whole program, a blog, an architecture, a troubleshooting
workflow, a db query and more outside of the IT realm (a poem, a picture, a receipt …)

13. © 2023, Amazon Web Services, Inc. or its affiliates.
Why is Gen AI useful? [ the builder view ]
Read and
memorize it all
(LOL – yeah sure)
(1)
Source of unstructured knowledge
You

14. © 2023, Amazon Web Services, Inc. or its affiliates.
Why is Gen AI useful? [ the builder view ]
You
Read and
memorize it all
(LOL – yeah sure)
Search engines (possibly
not relevant and still hard
- you are the integrator
and generator of a new
asset – text or code)
(1)
(2)
Source of unstructured knowledge
asset

15. © 2023, Amazon Web Services, Inc. or its affiliates.
Why is Gen AI useful? [ the builder view ]
15
You
Read and
memorize it all
(LOL – yeah sure) LLM
Train on it
(doable)
(1)
(2)
(3a)
Natural language
conversation
(3b)
Source of unstructured knowledge
asset
asset
Search engines (possibly
not relevant and still hard
- you are the integrator
and generator of a new
asset – text or code)

16. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
My first Gen AI application
16

17. © 2023, Amazon Web Services, Inc. or its affiliates.
Real life use case
17
BACKGROUND: I HATE WHATSAPP VOCAL MESSAGES
!!!!

18. © 2023, Amazon Web Services, Inc. or its affiliates.
Real life use case –the ClickOps version
18
BACKGROUND: I HATE WHATSAPP VOCAL MESSAGES
Audio
file
Text
fileLLM
Audio to text translationText summarization
Text
file

19. © 2023, Amazon Web Services, Inc. or its affiliates.
Real life use case –the ClickOps version
19
BACKGROUND: I HATE WHATSAPP VOCAL MESSAGES
Prompt
Output
(generated asset)

20. © 2023, Amazon Web Services, Inc. or its affiliates.
Real life use case –the application version
20
MY FIRST (NON TUTORIAL-BASED HELLO-WORLD) GENERATIVE AI APPLICATION

21. © 2023, Amazon Web Services, Inc. or its affiliates. 21
Real life use case –the application version
LAMBDA CALLS AN EXTERNAL LLM SERVICE
https://it20.info/2023/08/building-a-generative-ai-application-using-aws-step-functions/

22. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
Making LLMs useful
22

23. © 2023, Amazon Web Services, Inc. or its affiliates. 23
Why are people talking about things like Agents, Tools, RAG..
qThe LLM is just one (fundamental) component of Generative AI
qThe LLM could hallucinate, don’t have knowledge of recent / private / live
information, can’t do advanced math, may have limited reasoning capabilities, etc.
qYou need something to complement its capabilities and guide/help it
qEspecially for “real” business use cases that go beyond “toying around”

24. © 2023, Amazon Web Services, Inc. or its affiliates. 24
Why are people talking about things like Agents, Tools, RAG..
THERE’RE TWO DIMENSIONS THE LLM OPERATES IN (LEVEL OF ABSTRACTION AND DOMAINS)
Developing code
Debugging code
Living life
Deploying code
Domains
Writing a novel
Organizing travels

25. © 2023, Amazon Web Services, Inc. or its affiliates. 25
Why are people talking about things like Agents, Tools, RAG..
THERE’RE TWO DIMENSIONS THE LLM OPERATES IN (LEVEL OF ABSTRACTION AND DOMAINS)
Developing code
Debugging code
Deploying code
Domains
Autocomplete a
function method
Build a new ERP
from scratch
Resolve an error
message
Rearchitect the
app to avoid this
error at scale
Suggest what I
could do today
Organize my
whole life for the
next 10 years
Level of abstractionSimple task Complex task
Writing a novel
Living life
Organizing travelsTell me how long
it takes driving
from Florence to
Rome
Plan in details all
my 1-year long
sabbatical

26. © 2023, Amazon Web Services, Inc. or its affiliates. 26
Why are people talking about things like Agents, Tools, RAG..
THERE’RE TWO DIMENSIONS THE LLM OPERATES IN (LEVEL OF ABSTRACTION AND DOMAINS)
Domains
Level of abstractionSimple task Complex task
Progressive complexity
CompletionChatReasoningActingààà
coverage
DomainA function of the
corpus data

27. © 2023, Amazon Web Services, Inc. or its affiliates. 27
Why are people talking about things like Agents, Tools, RAG..
THERE’RE TWO DIMENSIONS THE LLM OPERATES IN (LEVEL OF ABSTRACTION AND DOMAINS)
Domains
Level of abstractionSimple task Complex task
Large Language model
Smaller
purpose
built/tuned
model
Models may need to be
helped / guided to achieve
goals where task complexity
is too high or simply for
missing domain knowledge

28. © 2023, Amazon Web Services, Inc. or its affiliates. 28
Why are people talking about things like Agents, Tools, RAG..
AN EXAMPLE OF COT (CHAIN OF THOUGHTS)
https://arxiv.org/abs/2201.11903
But sometimes in-prompt Chain of Thoughts (CoT) isn’t enough for the LLM to reason properly
Welcome to the magic world of
“prompt engineering
”

29. © 2023, Amazon Web Services, Inc. or its affiliates. 29
Why are people talking about things like Agents, Tools, RAG..
AN EXAMPLE OF THE FACT CHECKING WITH PROMPT CHAINING PROCESS
https://it20.info/2023/6/the-dark-zone-between-the-magic-genai-experience-and-the-large-language-model/
Q: What is the biggest clock in the world?

30. © 2023, Amazon Web Services, Inc. or its affiliates. 30
Why are people talking about things like Agents, Tools, RAG..
LLM
AN EXAMPLE OF TOOLS
You
“what’s the weather like
today in Rome?”
Math function
code
Web search
code
“Calculate <very complex
formula>”
(1a)
(2a)
(1b)
(2b)

31. © 2023, Amazon Web Services, Inc. or its affiliates. 31
Why are people talking about things like Agents, Tools, RAG..
AN EXAMPLE OF REACT (REASONING AND ACTING)
https://arxiv.org/abs/2210.03629

32. © 2023, Amazon Web Services, Inc. or its affiliates. 32
Why are people talking about things like Agents, Tools, RAG..
LLM
AN EXAMPLE OF REACT (REASONING AND ACTING)
You
Iterating
reasoning
code
“Write the solution for
<very complex task>” (1)
(2)
https://arxiv.org/abs/2210.03629
Tool

33. © 2023, Amazon Web Services, Inc. or its affiliates. 33
Why are people talking about things like Agents, Tools, RAG..
LLM
Vector
DB
AN EXAMPLE OF RAG (RETRIEVAL-AUGMENTED GENERATION)
You
Private corpus of data
embedding
“Write a draft email on <specific
company secret topic>”
(1)
(2)

34. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
Prompt context Vs. RAG Vs. fine-tuning
34

35. © 2023, Amazon Web Services, Inc. or its affiliates. 35
Prompt context Vs. RAG Vs. fine-tuning: I am lost
qFair. There are three ways to increase an LLM answer precision and correctness
1.Provide context in the prompt
2.Augment the LLM with an external source of vectorized data at inference time (RAG)
3.Fine tune the LLM with additional data
qThere isn’t a global right or wrong approach. As often happens, it depends
qAlso they are not mutually exclusive
qThey could (and often should) be used together to achieve optimal results

36. © 2023, Amazon Web Services, Inc. or its affiliates. 36
qRate of the change of the data source
qLimits, cost, latency, speed of prompt context tokens
qCost of fine tuning
qincluding the work required to “prepare the data”
qCost of creating and maintaining the vector store
Prompt context Vs. RAG Vs. fine-tuning: when to use what?

37. © 2023, Amazon Web Services, Inc. or its affiliates. 37
qComplexity of the architecture
qfine tuning may make the architecture easier (with an upfront fine-tuning investment)
qShape and location of the data source
qPrecision of the outcome
qno absolute rules exist, testing may be required
qPersonal experience of the team building the solution
q“I have always used RAG and that’s what I am comfortable with”
Prompt context Vs. RAG Vs. fine-tuning: when to use what?

38. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
Who’s Gen AI for?
38

39. © 2023, Amazon Web Services, Inc. or its affiliates. 39
Who’s Gen AI for?
qFor the developer that is writing code
qe.g. code assistants e.g. AWS CodeWhisperer
qFor the developer that wants to use English as a programming language
qe.g. the example of the WhatsApp vocal messages
qFor the ops person that does not want to write a SQL query to extract data
qe.g. https://www.honeycomb.io/blog/introducing-query-assistant

40. © 2023, Amazon Web Services, Inc. or its affiliates. 40
Who’s Gen AI for?
qFor the business analyst that wants to create a report off of a spreadsheet
qFor the journalist that wants to draft an article on a specific topic
qEtc. etc.
qCome see me later to chat about the story of my plumber impressed by “chat …
chat …. chat something” (true story)

41. © 2023, Amazon Web Services, Inc. or its affiliates. 41
Conclusions
qGet ready for this new wave. It’s coming and (I think) it’s staying.
qLLMs have moved the needle of the art of possible
qBut LLMs alone are not enough. You need to … make LLMs useful.
qGen AI is for everyone, not just for “builders”. It’s for “consumers” of tech too.
qGo explore! Go build!

42. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates.
Massimo Re Ferrè
Senior Principal Technologist, AWS
Twitter: @mreferre
E-mail: [email protected]
Thanks!