09-03-2024_UnstructuredDataAndAIDiscussion.pdf

https://www.startupgrind.com/events/details/startup-grind-princeton-presents-building-the-future-ai-amp-startups-in-action/cohost-princeton

Building the Future: AI & Startups in Action
Sep 3, 6:00 – 8:00 PM

Princeton

23 Orchard Rd, 23 Orchard Road, Montgomery, 08558

Step into the future! and see AI & Startups in Action. SeeExplore how AI drives innovation with live demos, no-code tools, and insights from startup founders. Join us to see how AI transforms ideas into impactful, real-world solutions!

RAG

1. 1 | © Copyright 2024 Zilliz1
Unstructured Data and AI Discussion
Tim Spann @ Zilliz

2. Slides
X

3. 3 | © Copyright Zilliz3
01
Introduction

4. 4 | © Copyright 2024 Zilliz4 4| © Copyright 10/22/23 Zilliz 4| © Copyright 2024 Zilliz
Tim Spann
Principal Developer
Advocate, Zilliz
[email protected]
https://www.linkedin.com/in/timothyspann/
https://x.com/PaaSDev

5. 5 | © Copyright Zilliz5
Show Me A Demo

6. Unstructured Data is Everywhere
Unstructured data is any data that does not conform to a predefined
data model.
Currently, 90% of unstructured data is never analyzed.

Images Videos and more!Text

7. 7 | © Copyright Zilliz7
…and cannot process increasingly growing unstructured data
Data Source: The Digitization of the World by IDC
20%
Other
newly generated data in 2025
will be unstructured data80%

8. The challenge of unstructured data
●Problem: Unstructured data comes in lots of forms, no easy
way to interact with it all
●Solution: Vector embeddings
●How: Neural networks e.g. embedding models

Vector
Databases

9. 9 | © Copyright Zilliz9
02
Overview of Vector Databases

10. Why a Vector Database?
•Vector database
•Advanced filtering (filtered vector search, chained
filters)
•Hybrid search (e.g. full text + dense vector)
•Durability (any write in a db is durable, a library
typically only supports snapshotting)
•Replication / High Availability
•Sharding
•Aggregations or faceted search
•Backups
•Lifecycle management (CRUD, Batch delete,
dropping whole indexes, reindexing)
•Multi-tenancy
•Vector search library
•High-performance vector search

•How do I support different applications?
•High query load
•High insertion/deletion
•Full precision/recall
•Accelerator support (GPU, FPGA)
•Billion-scale storage

Purpose-built to store, index and query vector embeddings from unstructured
data.

11. V
n, 1
…
…
…
1
2
3
4
5
Transform into
Vectors
Unstructured Data
Images
User Generated
Content
Video
Documents
Audio
Vector Embeddings
Perform
Approximate
Nearest Neighbor
Similarity Search
Perform Query
Get Results
Store in Vector Database
How Similarity Search Works

12. 1
2
2024
A vector database stores embedding vectors and allows for semantic
retrieval of various types of unstructured data.
Vector Database: Making Sense of Unstructured Data

13. 13 | © Copyright Zilliz13
Do you really need a Vector Database?
•50M100M vectors
•PostgreSQL, ElasticSearch, Big
Query, MongoDB, etc with
ANNS plug-ins
Existing Solutions Vector Databases
•Purpose-built for vectors top
support the requirements and
lifecycle of vectors
•Billion+ scale
•CRUD, real-time search,
top-k/range/hybrid search,
multi-modal, mulit-vector query,
distributed
•Semantic Search is core to your
business
ANN Libraries
•FAISS, ANNOY, HNSW
•Supports 1M vectors
•Good for prototyping
Vector Databases are purpose-built to handle
indexing, storing, and querying vector data.

14. 14 | © Copyright Zilliz14
03
A Quick Introduction to Milvus

15. 15 | © Copyright Zilliz15
Milvus Features
Multi-Tenancy

Hardware-
Accelerated
Compute Support
Python, Java,
Golang, NodeJS

Milvus Lite, K8,
Zilliz Cloud, Docker

Scalable and Elastic
Architecture

Diverse Index
Support

Versatile Search
Capabilities

Tunable
Consistency

16. 16 | © Copyright Zilliz16
Technologies for various types of Use
cases
Compute Types


Designed for various
compute powers, such as
AVX512, Neon for SIMD,
quantization cache-aware
optimization and GPU


Leverage strengths of each
hardware type, ensuring
high-speed processing and
cost-effective scalability for
different application needs


Search Types


Support multiple types such
as top-K ANN, Range ANN,
sparse & dense,
multi-vector, grouping,
and metadata filtering

Enable query flexibility and
accuracy, allowing
developers to tailor their
information retrieval needs
Multi-tenancy


Enable multi-tenancy
through collection and
partition management



Allow for efficient resource
utilization and customizable
data segregation, ensuring
secure and isolated data
handling for each tenant
Index Types


Offer a wide range of 15
indexes support, including
popular ones like
Hierarchical Navigable
Small Worlds HNSW, PQ,
Binary, Sparse, DiskANN
and GPU index

Empower developers with
tailored search
optimizations, catering to
performance, accuracy and
cost needs

17. 17 | © Copyright Zilliz17
What is Milvus/Zilliz ideal for?
•Advanced filtering
•Hybrid search
•Multi-vector Search
•Durability and backups
•Replications/High Availability
•Sharding
•Aggregations
•Lifecycle management
•Multi-tenancy
•High query load
•High insertion/deletion
•Full precision/recall
•Accelerator support GPU,
FPGA
•Billion-scale storage

Purpose-built to store, index and query vector embeddings from unstructured data at scale.

18. 18 | © Copyright Zilliz18
Milvus: From Dev to Prod
AI Powered Search made easy
Milvus is an Open-Source Vector
Database to store, index, manage, and
use the massive number of embedding
vectors generated by deep neural
networks and LLMs.
contributors
285
stars
29K
downloads
50M
forks
2.8K

19. 192024
Higher Scalability

10B vectors
of 1536 dimensions
in a single Milvus/Zilliz Cloud
instance

100B vectors
in one of the largest deployment

20. Milvus: decoupling computation and storage

21. 212024
Indexes
Most of the vector index types supported by Milvus use approximate nearest neighbors search ANNS,
●HNSW: HNSW is a graph-based index and is best suited for scenarios that have a high demand for
search efficiency. There is also a GPU version GPU_CAGRA, thanks to Nvidiaʼs contribution.
●FLAT: FLAT is best suited for scenarios that seek perfectly accurate and exact search results on a small,
million-scale dataset. There is also a GPU version GPU_BRUTE_FORCE .
●IVF_FLAT: IVF_FLAT is a quantization-based index and is best suited for scenarios that seek an ideal
balance between accuracy and query speed. There is also a GPU version GPU_IVF_FLAT.
●IVF_SQ8: IVF_SQ8 is a quantization-based index and is best suited for scenarios that seek a significant
reduction on disk, CPU, and GPU memory consumption as these resources are very limited.
●IVF_PQ: IVF_PQ is a quantization-based index and is best suited for scenarios that seek high query
speed even at the cost of accuracy. There is also a GPU version GPU_IVF_PQ.

22. 222024
Indexes Continued.
●SCANN: SCANN is similar to IVF_PQ in terms of vector clustering and product quantization. What makes
them different lies in the implementation details of product quantization and the use of SIMD
Single-Instruction / Multi-data) for efficient calculation.
●DiskANN: Based on Vamana graphs, DiskANN powers efficient searches within large datasets.

23. 23 | © Copyright Zilliz23
04
Consume and Ingest Data

24. 24 | © Copyright Zilliz24
DATA!!!!

25. 25 | © Copyright Zilliz25
DATA

26. 26 | © Copyright Zilliz26
05
Building a local RAG application

27. 27 | © Copyright Zilliz27
Vector embeddings are something
computers can understand

28. 2
8
Retrieval-Augmented Generation (RAG)
2024
A technique that combines the
strength of retrieval-based and
generative models:
●Improve accuracy and relevance
●Eliminate hallucination
●Provide domain-specific
knowledge

29. 2
9
RAG : an economic perspective
2024
A business model that bridges public
data and private data
●Data sovereignty
●You can't and shouldn't give your
private data to others

30. 30 | © Copyright Zilliz30

31. 31 | © Copyright Zilliz31
Embeddings Models

32. 32 | © Copyright Zilliz32
06
Q & A

34. 34 | © Copyright Zilliz34 | © Copyright Zilliz34
RESOURCES

35. 35 | © Copyright Zilliz35
Vector Database Resources
Give Milvus a Star!




Chat with me on Discord!
https://github.com/milvus-io/milvus

36. 36
Unstructured Data Meetup


https://www.meetup.com/unstructured-data-meetup-new-york/

This meetup is for people working in unstructured data. Speakers will come present about related topics
such as vector databases, LLMs, and managing data at scale. The intended audience of this group
includes roles like machine learning engineers, data scientists, data engineers, software engineers, and
PMs.
This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.

37. 37 | © Copyright Zilliz37
https://zilliz.com/learn/generative-ai

39. https://medium.com/@tspann/unstructured-street-data-in-new-york-8d3cde0a1e5b

40. https://medium.com/@tspann/not-every-field-is-just-text-numbers-or-vectors-976231e90e4d

41. https://medium.com/@tspann/shining-some-light-on-the-new-milvus-lite-5a0565eb5dd9

42. https://medium.com/@tspann/unstructured-data-processing-with-a-raspberry-pi-ai-kit-c959dd7fff47
Raspberry Pi AI Kit Hailo
Edge AI

43. 43 | © Copyright 2024 Zilliz43
43
This week in Milvus, Towhee, Attu, GPT
Cache, Gen AI, LLM, Apache NiFi, Apache
Flink, Apache Kafka, ML, AI, Apache Spark,
Apache Iceberg, Python, Java, Vector DB
and Open Source friends.
https://bit.ly/32dAJft
https://github.com/milvus-io/milvus

AIM Weekly by Tim Spann

44. 44 | © Copyright 2024 Zilliz44
milvus.io
github.com/milvus-io/
@milvusio
@paasDev


/in/timothyspann
Connect with me! Thank you!

45. 45 | © Copyright 2024 Zilliz45

46. 46 | © Copyright 2024 Zilliz46

47. 47 | © Copyright 2024 Zilliz47
Join us at our next meetup!
meetup.com/unstructured-data-meetup-
new-york/

48. 48 | © Copyright Zilliz48
T H A N K Y O U

49. 49 | © Copyright Zilliz49
05
What is Similarity Search?

50. 50 | © Copyright Zilliz50
Image from Nvidia
Vector Search Overview

51. 51 | © Copyright Zilliz51
Vector Similarity Measures: L2 Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)
2
+ (0.9-0.7)
2

= √(0.2)
2
+ (0.2)
2

= √0.04 + 0.04
= √0.08 ≅ 0.28

52. 52 | © Copyright Zilliz52
Vector Similarity Measures: Inner Product IP
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78

53. 53 | © Copyright Zilliz53
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Measures: Cosine
??????
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.3
2
+0.9
2
* √0.5
2
+0.7
2

= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03

54. 54 | © Copyright Zilliz54 osschat.io

55. 55 | © Copyright Zilliz55