Jeff Maruschek: How does RAG REALLY work?

AWS Community Day Midwest 2024
Jeff Maruschek
How does RAG REALLY work?

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jeff Maruschek
Amazon Web Services
Solutions Architect –DoD –Air Force
Retrieval-Augmented Generation (RAG)
Overview and Interactive Demos

2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2
Introduction

3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3
Introduction

4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 4
djmooseon Mixcloud:

5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is RAG?
What are Embeddings?
Amazon Bedrock ChatBotDemo
Amazon Bedrock Knowledge Bases Demo
Agenda
5

6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 6
What book genre is The Three-Body Problem?

7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 7
What instrument does Joshua Bell play?

8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8
What instrument does the musician Joshua Bell
play?

9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9
Who is Joshua Bell?

10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 10

11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11

12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
•Retrieval
▪Fetches the relevant content from the
external knowledge base or data sources
based on a user query
•Augmentation
▪Adding the retrieved relevant context to
the user prompt, which goes as an input to
the foundation model
•Generation
▪Response from the foundation model
based on the augmented prompt.
12
What is RAG?

13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 13

14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What are embeddings?
EMBEDDING
MODEL
0.027-0.011 -0.023…
0.025-0.009 -0.025…
New York
Paris
Vector EmbeddingsHuman Text
•Numerical representation of
text (vectors) that captures
semantics and relationships
between words.
•Embedding models capture
features and nuances of the
text.
•Rich embeddings can be used
to compare text similarity.
•Multilingual Text Embeddings
can identify meaning in
different languages.
-0.0110.021 0.013…Animal
-0.0090.019 0.015…Horse

15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 16

16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 17

17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 18

18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 19

19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20
Amazon Bedrock ChatBotDemo
https://github.com/aws-samples/bedrock-claude-chat

20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 21
Documents Used

21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 22
Architecture

22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
However, when it comes to implementing
RAG, there are challenges…
Creating vector
embeddings for large
volumes of data
Orchestration
Managing
multiple data
sources
Scaling retrieval
mechanism
Coding effort
Incremental
updates to vector
store

23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24
Amazon Bedrock Knowledge Bases Demo
https://aws.amazon.com/bedrock/knowledge-bases/

24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 25
Documents Used

25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
Jeff Maruschek
Solutions Architect –DoD –Air Force
[email protected]
26