17-October-2024 NYC AI Camp - Step-by-Step RAG 101

17-October-2024 NYC AI Camp - Step-by-Step RAG 101

https://github.com/tspannhw/AIM-BecomingAnAIEngineer

https://github.com/tspannhw/AIM-Ghosts

AIM - Becoming An AI Engineer

Step 1 - Start off local
Download Python (or use your local install)
https://www.python.org/downloads/

python3.11 -m venv yourenv
source yourenv/bin/activate
Create an environment
https://docs.python.org/3/library/venv.html

Use Pip
https://pip.pypa.io/en/stable/installation/

Setup a .env file for environment variables

Download Jupyter Lab
https://jupyter.org/

Run your notebook

jupyter lab --ip="0.0.0.0" --port=8881 --allow-root
Running on a Mac or Linux machine is optimal.

Setup environment variables
source .env
Alternatives
Download Conda
https://docs.conda.io/projects/conda/en/latest/index.html

https://colab.research.google.com/

Other languages: Java, .Net, Go, NodeJS
Other notebooks to try
https://zilliz.com/learn/milvus-notebooks
https://github.com/milvus-io/bootcamp/blob/master/bootcamp/tutorials/quickstart/build_RAG_with_milvus.ipynb
References
Guides
https://zilliz.com/learn

HuggingFace Friend
https://zilliz.com/learn/effortless-ai-workflows-a-beginners-guide-to-hugging-face-and-pymilvus

Milvus
https://zilliz.com/milvus-downloads

https://milvus.io/docs/quickstart.md

LangChain
https://zilliz.com/learn/LangChain

Notebook display
https://ipywidgets.readthedocs.io/en/stable/user_install.html

References
https://medium.com/@zilliz_learn/function-calling-with-ollama-llama-3-2-and-milvus-ac2bc2122538
https://github.com/milvus-io/bootcamp/tree/master/bootcamp/RAG/advanced_rag
https://zilliz.com/learn/Retrieval-Augmented-Generation
https://zilliz.com/blog/scale-search-with-milvus-handle-massive-datasets-with-ease
https://zilliz.com/learn/generative-ai
https://zilliz.com/learn/what-are-binary-vector-embedding
https://zilliz.com/learn/choosing-right-vector-index-for-your-project

1. 1 | © Copyright 2024 Zilliz1
Step-by-Step - RAG 101
Tim Spann @ Zilliz
Principal Developer Advocate

2. 2 | © Copyright 2024 Zilliz2
Slides
XX

3. 3 | © Copyright 2024 Zilliz3 3| © Copyright 10/22/23 Zilliz 3| © Copyright 2024 Zilliz
Tim Spann
Principal Developer
Advocate, Zilliz
[email protected]
https://www.linkedin.com/in/timothyspann/
https://x.com/PaaSDev

4. 4 | © Copyright 2024 Zilliz4
Agenda
About Zilliz, Milvus and Vector Databases01
Level 100  Basic RAG
References03
02

5. 5 | © Copyright 2024 Zilliz5 | © Copyright 9/25/23 Zilliz 5
VECTOR DATABASE

6. 6 | © Copyright 2024 Zilliz6 | © Copyright 9/25/23 Zilliz 6
The Forrester Wave™ Vector
Database Providers, Q3 2024
Zilliz is the right partner for
your Vector Database
needs.

7. 7 | © Copyright 2024 Zilliz7 | © Copyright 9/25/23 Zilliz 7
Milvus is an Open-Source Vector Database to
store, index, manage, and use the massive
number of embedding vectors generated by
deep neural networks and LLMs.
contributors
283
stars
30K
docker pulls
67M
forks
2.8K
+
Milvus: The most widely-adopted vector database

8. 8 | © Copyright Zilliz8
Easy Open RAG Stack Highlighted
Framework
Hardware
Infrastructure
Embedding Models LLMs
Software Infrastructure
Vector Database

9. 9 | © Copyright Zilliz9

10. 10 | © Copyright Zilliz10
Image from Nvidia
Vector Search Overview

11. V
n, 1
…
…
…
1
2
3
4
5
Transform into
Vectors
Unstructured Data
Images
User Generated
Content
Video
Documents
Audio
Vector Embeddings
Perform
Approximate
Nearest Neighbor
Similarity Search
Perform Query
Get Results
Store in Vector Database
How Similarity Search Works

12. 12 | © Copyright 2024 Zilliz12 | © Copyright 9/25/23 Zilliz 12
RAG
Retrieval Augmented Generation)

13. 13 | © Copyright 2024 Zilliz13
Basic Idea
Use RAG to force the LLM to work with your
data by injecting it via a vector database like
Milvus

14. 14 | © Copyright 2024 Zilliz14
Basic RAG Architecture

15. 1
5
Retrieval-Augmented Generation (RAG)
2024
A technique that combines the
strength of retrieval-based and
generative models:

●Improve accuracy and relevance
●Eliminate hallucination
●Provide domain-specific
knowledge

16. 1
6
RAG : an economic perspective
2024
A business model that bridges public
data and private data

●Data sovereignty
●You can't and shouldn't give your
private data to others

17. 17 | © Copyright 2024 Zilliz17 | © Copyright 9/25/23 Zilliz 17
Your Turn
Time To Build a Simple RAG App

18. 18 | © Copyright 2024 Zilliz18
Documents From Here

https://bit.ly/3U7Okgu

19. 19 | © Copyright 2024 Zilliz19
Notebook From Here

https://bit.ly/3BV4IKX

20. 20 | © Copyright Zilliz20

21. 21 | © Copyright Zilliz21

22. 22 | © Copyright Zilliz22
Python SDK Connect…

23. 23 | © Copyright Zilliz23
Choose Your Embedding Function

24. 24 | © Copyright Zilliz24
Vector Embedding

25. 25 | © Copyright Zilliz25

26. 26 | © Copyright Zilliz26

27. 27 | © Copyright Zilliz27

28. 28 | © Copyright Zilliz28

29. 29 | © Copyright Zilliz29

30. 30 | © Copyright Zilliz30

31. 31 | © Copyright Zilliz31

32. 32 | © Copyright Zilliz32

33. 33 | © Copyright 2024 Zilliz33 | © Copyright 9/25/23 Zilliz 33
Advanced RAG

34. 34 | © Copyright Zilliz34
https://bit.ly/4eFdMlK https://bit.ly/3BLeLCx

35. 35 | © Copyright Zilliz35
https://bit.ly/3zXW8dX https://bit.ly/3NuK5ru

36. 36 | © Copyright Zilliz36
https://bit.ly/4gZ4Lpn

Metadata Filtering

Hybrid Search

Agents
https://bit.ly/3UbqUqx
https://bit.ly/3YpKd1K
Smart Chunking
Embedding Model Choice

37. 37 | © Copyright 2024 Zilliz37 | © Copyright 9/25/23 Zilliz 37
Resources

38. 38 | © Copyright Zilliz38
Wednesday, October 23, 2024
5:30 PM to 8:30 PM EDT NYC
https://bit.ly/403JIvy
Thursday, October 24, 2024
6:30 PM to 8:00 PM EDT Princeton
https://bit.ly/3NopXY2
November 5-7, 10-12, 2024: CloudX
https://www.developerweek.com/cloudx/

November 15, 2024: Build Stuff
https://www.developerweek.com/cloudx/speakers/

November 19, 2024: Xtreme Python
https://xtremepython.dev/2024/

November 21, 2024: bigdata Conference

November 21, 2024: Unstructured Data NYC

39. 39 | © Copyright Zilliz39
Vector Database Resources
Give Milvus a Star!




Chat with me on Discord!
https://github.com/milvus-io/milvus

40. 40
Unstructured Data Meetup


https://www.meetup.com/unstructured-data-meetup-new-york/

This meetup is for people working in unstructured data. Speakers will come present about related topics
such as vector databases, LLMs, and managing data at scale. The intended audience of this group
includes roles like machine learning engineers, data scientists, data engineers, software engineers, and
PMs.
This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.

41. https://medium.com/@tspann/shining-some-light-on-the-new-milvus-lite-5a0565eb5dd9

42. https://medium.com/@tspann/unstructured-data-processing-with-a-raspberry-pi-ai-kit-c959dd7fff47
Raspberry Pi AI Kit Hailo
Edge AI

43. 43 | © Copyright 2024 Zilliz43
43
This week in Milvus, Towhee, Attu, GPT
Cache, Gen AI, LLM, Apache NiFi, Apache
Flink, Apache Kafka, ML, AI, Apache Spark,
Apache Iceberg, Python, Java, Vector DB
and Open Source friends.
https://bit.ly/32dAJft
https://github.com/milvus-io/milvus

AIM Weekly by Tim Spann

44. 44 | © Copyright 2024 Zilliz44
milvus.io
github.com/milvus-io/
@milvusio
@paasDev


/in/timothyspann
Connect with me! Thank you!

45. 45 | © Copyright 2024 Zilliz45
Milvus
Open Source Self-Managed

Zilliz Cloud
SaaS Fully-Managed

github.com/milvus-io/milvus
Getting Started with Vector Databases
zilliz.com/cloud
29K - Star us on GitHub!

46. 46 | © Copyright 2024 Zilliz46
Get started for free
zilliz.com/cloud

47. 47 | © Copyright 2024 Zilliz47 | © Copyright 9/25/23 Zilliz 47
Deep-Dive

48. 48 | © Copyright Zilliz48
Vector Similarity Measures: L2 Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)
2
+ (0.9-0.7)
2

= √(0.2)
2
+ (0.2)
2

= √0.04 + 0.04
= √0.08 ≅ 0.28

49. 49 | © Copyright Zilliz49
Vector Similarity Measures: Inner Product IP
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78

50. 50 | © Copyright Zilliz50
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Measures: Cosine
??????
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.3
2
+0.9
2
* √0.5
2
+0.7
2

= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03

51. 51 | © Copyright Zilliz51
Hybrid Search

52. 52 | © Copyright Zilliz52