NYCMeetup07-25-2024-Unstructured Data Processing From Cloud to Edge

NYCMeetup07-25-2024-Unstructured Data Processing From Cloud to Edge

https://www.meetup.com/unstructured-data-meetup-new-york/

https://www.meetup.com/unstructured-data-meetup-new-york/events/301720478/

Details
This is an in-person event! Registration is required to get in.

Topic: Connecting your unstructured data with Generative LLMs

What we’ll do:
Have some food and refreshments. Hear three exciting talks about unstructured data and generative AI.
5:30 - 6:00 - Welcome/Networking/Registration
6:05 - 6:30 - Tim Spann, Principal DevRel, Zilliz
6:35 - 7:00 - Chris Joynt, Senior PMM, Cloudera
7:05 - 7:30 - Lisa N Cao, Product Manager, Datastrato
7:30 - 8:30 - Networking

Tech talk 1: Unstructured Data Processing From Cloud to Edge
Speaker: Tim Spann, Principal Dev Advocate, Zilliz

In this talk I will do a presentation on why you should add a Cloud Native vector database to your Data and AI platform. He will also cover a quick introduction to Milvus, Vector Databases and unstructured data processing. By adding Milvus to your architecture you can scale out and improve your AI use cases through RAG, Real-Time Search, Multimodal Search, Recommendations Engines, fraud detection and many more emerging use cases.
As I will show, Edge devices even as small and inexpensive as a Raspberry Pi 5 can work in machine learning, deep learning and AI use cases and be enhanced with a vector database.

Tech talk 2: RAG Pipelines with Apache NiFi
Speaker: Chris Joynt, Senior PMM, Cloudera

Executing on RAG Architecture is not a set-it-and-forget-it endeavor. Unstructured or multimodal data must be cleansed, parsed, processed, chunked and vectorized before being loaded into knowledge stores and vector DB's. That needs to happen efficiently to keep our GenAI up to date always with fresh contextual data. But not only that, changes will have to be made on an ongoing basis. For example, new data sources must be added. Experimentation will be necessary to find the ideal chunking strategy. Apache NiFi is the perfect tool to build RAG pipelines to stream proprietary and external data into your RAG architectures. Come learn how to use this scalable and incredible versatile tool to quickly build pipelines to activate your GenAI use case.

Tech Talk 3: Metadata Lakes for Next-Gen AI/ML

Speaker: Lisa N Cao, Datastrato

Abstract: As data catalogs evolve to meet the growing and new demands of high-velocity, unstructured data, we see them taking a new shape as an emergent and flexible way to activate metadata for multiple uses. This talk discusses modern uses of metadata at the infrastructure level for AI-enablement in RAG pipelines in response to the new demands of the ecosystem. We will also discuss Apache (incubating) Gravitino and its open source-first approach to data cataloging across multi-cloud and geo-distributed architectures.

Who Should attend:
Anyone interested in talking and learning about Unstructured Data and Generative AI Apps.

When:
July 25, 2024
5:30PM

1. 1 | © Copyright 10/22/23 Zilliz1 | © Copyright 10/22/23 Zilliz
Presented by:
New York
Unstructured Data Meetup

2. 2 | © Copyright 10/22/23 Zilliz2 | © Copyright 10/22/23 Zilliz 2| © Copyright 10/22/23 Zilliz 2| © Copyright 10/22/23 Zilliz
Tim Spann
Principal Developer
Advocate, Zilliz
[email protected]
https://www.linkedin.com/in/timothyspann/
https://x.com/PaaSDev
Unstructured Data Meetup | Host

3. 3 | © Copyright 10/22/23 Zilliz3 | © Copyright 10/22/23 Zilliz
Code of
Conduct
Be respectful and kind
When communicating with all event participants,
speakers, and hosts. Be considerate

All ideas are welcome
Be present and participate actively in discussions. Ask
questions and reach out for help when needed.

Report inappropriate behavior
Any inappropriate behavior is not tolerated at this event.
Inform a Zilliz team member immediately if you see any
behavior deemed inappropriate

4. 4 | © Copyright 10/22/23 Zilliz4 | © Copyright 10/22/23 Zilliz 4| © Copyright 10/22/23 Zilliz 4| © Copyright 10/22/23 Zilliz
Milvus
Open Source Self-Managed

Zilliz Cloud
SaaS Fully-Managed

github.com/milvus-io/milvus

Getting Started with Vector Databases
zilliz.com/cloud

5. 5 | © Copyright 10/22/23 Zilliz5 | © Copyright 10/22/23 Zilliz
Zilliz is
Hiring!

Join our Team

Zilliz.com/careers
•Developer Advocate
•Senior Software Engineer
•Staff Software Engineer
•Solutions Architect

6. 6 | © Copyright 10/22/23 Zilliz6 | © Copyright 10/22/23 Zilliz
Join the
Milvus
Discord!

7. 7 | © Copyright 10/22/23 Zilliz7 | © Copyright 10/22/23 Zilliz
Become a
Speaker!
Interesting in speaking at and/or
sponsoring a Zilliz Unstructured
Data Meetup? Fill out this form!


??????????????????

8. 8 | © Copyright 10/22/23 Zilliz8 | © Copyright 10/22/23 Zilliz
Have you built
something cool
using Milvus or
Zilliz? We want to
hear all about it.
Share Your Story

9. 9 | © Copyright 10/22/23 Zilliz9 | © Copyright 10/22/23 Zilliz


Community
Day Event

September 13, 2024
Computer History Museum, Mountain View

This event celebrates the diverse
capabilities of AWS, showcasing
cutting-edge technologies and practical
applications across various domains.

10. 10 | © Copyright 10/22/23 Zilliz10 | © Copyright 10/22/23 Zilliz
Star Milvus
for a chance
to win a prize
tonight!

11. 11 | © Copyright 10/22/23 Zilliz11 | © Copyright 10/22/23 Zilliz
Share your
photos!
#ZillizUnstructuredData
@zilliz_universe, @milvusio
@Zilliz, @Milvus

12. 12 | © Copyright 10/22/23 Zilliz12 | © Copyright 10/22/23 Zilliz 12| © Copyright 10/22/23 Zilliz 12| © Copyright 10/22/23 Zilliz
Welcome Speakers
Unstructured Data
Processing From Cloud to
Edge
RAG Pipelines with
Apache NiFi
Metadata Lakes for
Next-Gen AI/ML
TECH TALK 1 TECH TALK 2 TECH TALK 3
Tim Spann
Principal Developer Advocate, Zilliz
Chris Joynt
Senior PMM, Cloudera
Lisa N Cao
Product Manager, Datastrato

13. 13 | © Copyright 10/22/23 Zilliz13 | © Copyright 10/22/23 Zilliz 13| © Copyright 10/22/23 Zilliz 13| © Copyright 10/22/23 Zilliz

14. 14 | © Copyright 10/22/23 Zilliz14 | © Copyright 10/22/23 Zilliz
Join us at our next meetup!
lu.ma/unstructured-data-meetup

15. 15 | © Copyright 2024 Zilliz15
Unstructured Data Processing From
Cloud to Edge
Tim Spann @ Zilliz

16. Slides

17. Agenda
In this talk I will do a presentation on why you should
add a Cloud Native vector database to your Data and
AI platform. He will also cover a quick introduction to
Milvus, Vector Databases and unstructured data
processing. By adding Milvus to your architecture you
can scale out and improve your AI use cases through
RAG, Real-Time Search, Multimodal Search,
Recommendations Engines, fraud detection and
many more emerging use cases.

As I will show, Edge devices even as small and
inexpensive as a Raspberry Pi 5 can work in machine
learning, deep learning and AI use cases and be
enhanced with a vector database.

18. 18 | © Copyright 2024 Zilliz18
01Introduction
CONTENTS
02AI Use Cases
03Edge Devices
Unstructured data, vector databases, traditional databases, similarity search
Why Vector Database, Milvus, Use Cases, Infrastructure. Demos.
How Milvus and AI are at the edge powering the future. Demos.

19. 19 | © Copyright Zilliz19
01
Introduction

20. 20 | © Copyright Zilliz20
-Unstructured Data is 80% of data

-Vector Databases are the only type of database
that can work with unstructured data

- Examples of Unstructured Data include text,
images, videos, audio, etc
Why Vector Databases?

21. 21 | © Copyright Zilliz21
Traditional databases were built on exact
search

22. 22 | © Copyright Zilliz22
…which misses context, semantic
meaning, and user intent





VS.
Apple





VS.
Rising dough





VS.
Change car tire
Rising Dough
Proofing Bread
✔
❌

23. 23 | © Copyright Zilliz23
Vector
Databases
Where do Vectors Come From?

24. 24 | © Copyright Zilliz24
The evolution of AI made the semantic
search of unstructured data possible
Search by Probability
Statistical analyses of common
datasets established the foundation for
processing unstructured data, e.g. NLP,
and image classification
AI Model Breakthrough
The advancements in BERT, ViT, CBT
etc. have revolutionized semantic
analysis across unstructured data
Vectorization
Word2Vec, CNNs, Deep Speech pioneered
unstructured data embeddings, mapping the
words, images, videos into high-dimensional
vectors

25. 25 | © Copyright Zilliz25
This new AI breakthrough requires new
databases to fully unleash its potential
Support multiple
use case types
Accommodate diverse data
requirements, enhancing
flexibility and effectiveness in
varied operational contexts
Scale as needed
Enable robust handling of
expanding data volumes and
search demands
Highly performant
Ensures swift and accurate
query responses, crucial for
optimal user experience

26. 26 | © Copyright Zilliz26
https://milvus.io/milvus-demos/reverse-image-search
Show Me

27. 27 | © Copyright Zilliz27 | © Copyright Zilliz27
Introducing Milvus

28. 28 | © Copyright Zilliz28
Why Even Use a Vector DB?
Beyond High-Performance Search
•CRUD Operations: Just like traditional databases, vector
databases allow you to Create, Read, Update, and Delete data.
•Data Freshness: Vector databases ensure your data remains
up-to-date, reflecting the latest information for accurate searches.
•Persistence: Your data is securely stored and persists even if the
system restarts.
•Availability: Your data is readily accessible for search and retrieval
operations.
•Scalability: Vector databases can handle growing data volumes
efficiently.

29. 29 | © Copyright Zilliz29
Complete Data Management
•Data Management: Vector databases provide tools to manage
your data effectively, including data ingestion, indexing, and
querying.
•Backup and Migration: Create backups of your data for disaster
recovery and easily migrate your data between different systems.
Why Even Use a Vector DB?

30. 30 | © Copyright Zilliz30
Operational ease
•Cloud or On-Premise Deployment: Vector databases can be
deployed easily on various platforms, including cloud and
on-premise environments.
•Observability: Monitor the health and performance of your vector
database to ensure optimal operation.
•Multi-tenancy: Support multiple users or applications accessing
the same database instance securely.
Why Even Use a Vector DB?

31. 31 | © Copyright 8/16/23 Zilliz31 | © Copyright 8/16/23 Zilliz
27K+
GitHub
Stars
25M+
Downloads
250+
Contributors
2,600
+Forks
Milvus is an open-source vector database for GenAI projects. pip install on your
laptop, plug into popular AI dev tools, and push to production with a single line of
code.
Easy Setup

Pip-install to start
coding in a notebook
within seconds.
Reusable Code

Write once, and
deploy with one line
of code into the
production
environment
Integration

Plug into OpenAI,
Langchain,
LlmaIndex, and
many more
Feature-rich

Dense & sparse
embeddings,
filtering, reranking
and beyond

32. 32 | © Copyright Zilliz32
Why Not Use a SQL/NoSQL Database?
•Inefficiency in High-dimensional spaces
•Suboptimal Indexing
•Inadequate query support
•Lack of scalability
•Limited analytics capabilities
•Data conversion issues

TL;DR: Vector operations are too computationally intensive for
traditional database infrastructures

33. 33 | © Copyright Zilliz33
Why Not Use a Vector Search Library?
•Have to manually implement filtering
•Not optimized to take advantage of the latest hardware
•Unable to handle large scale data
•Lack of lifecycle management
•Inefficient indexing capabilities
•No built in safety mechanisms

TL;DR: Vector search libraries lack the infrastructure to help you scale,
deploy, and manage your apps in production.

34. 34 | © Copyright Zilliz34
What is Milvus ideal for?
•Advanced filtering
•Hybrid search
•Durability and backups
•Replications/High Availability
•Sharding
•Aggregations
•Lifecycle management
•Multi-tenancy
•High query load
•High insertion/deletion
•Full precision/recall
•Accelerator support (GPU,
FPGA)
•Billion-scale storage

Purpose-built to store, index and query vector embeddings from unstructured data at scale.

35. 35 | © Copyright Zilliz35
We’ve built technologies for various
types of use cases
Compute Types


Designed for various
compute powers, such as
AVX512, Neon for SIMD,
quantization cache-aware
optimization and GPU


Leverage strengths of each
hardware type, ensuring
high-speed processing and
cost-effective scalability for
different application needs


Search Types


Support multiple types such
as top-K ANN, Range ANN,
sparse & dense,
multi-vector, grouping, and
metadata filtering

Enable query flexibility and
accuracy, allowing
developers to tailor their
information retrieval needs
Multi-tenancy


Enable multi-tenancy
through collection and
partition management



Allow for efficient resource
utilization and customizable
data segregation, ensuring
secure and isolated data
handling for each tenant
Index Types


Offer a wide range of 15
indexes support, including
popular ones like HNSW,
PQ, Binary, Sparse,
DiskANN and GPU index

Empower developers with
tailored search
optimizations, catering to
performance, accuracy and
cost needs

36. 36 | © Copyright Zilliz36
Meta Storage
Root Query Data Index
Coordinator Service
Proxy
Proxy
etcd
Log Broker
SDK
Load Balancer
DDL/DCL
DML
NOTIFICATION
CONTROL SIGNAL
Object Storage
Minio / S3 / AzureBlob
Log Snapshot Delta File Index File
Worker Node
QUERY DATA DATA
Message
Storage
Access Layer
Query Node Data Node Index Node
Milvus’ fully distributed architecture is
designed scalability and performance

37. 37 | © Copyright Zilliz37
Milvus: From Dev to Prod
AI Powered Search made easy
Milvus is an Open-Source Vector
Database to store, index, manage, and
use the massive number of embedding
vectors generated by deep neural
networks and LLMs.
contributors
267+
stars
27K+
downloads
25M+
forks
2K+

38. 38 | © Copyright Zilliz38
Retrieval Augmented
Generation (RAG)
Expand LLMs' knowledge by
incorporating external data sources
into LLMs and your AI applications.
Match user behavior or content
features with other similar ones to
make effective recommendations.
Recommender System
Search for semantically similar
texts across vast amounts of
natural language documents.
Text/ Semantic Search
Image Similarity Search
Identify and search for visually
similar images or objects from a
vast collection of image libraries.
Video Similarity Search
Search for similar videos, scenes,
or objects from extensive
collections of video libraries.
Audio Similarity Search
Find similar audios in large datasets
for tasks like genre classification or
speech recognition
Molecular Similarity Search
Search for similar substructures,
superstructures, and other
structures for a specific molecule.
Anomaly Detection
Detect data points, events, and
observations that deviate
significantly from the usual pattern
Multimodal Similarity Search
Search over multiple types of data
simultaneously, e.g. text and
images
…powers searches across various types
of unstructured data

39. 39 | © Copyright Zilliz39
Up to 100 billion vectors with K8s!

40. 40 | © Copyright Zilliz40

41. 41 | © Copyright Zilliz41
Entity

42. 42 | © Copyright Zilliz42
Vector Space

43. 43 | © Copyright Zilliz43
03
Edge Devices

44. 44 | © Copyright Zilliz44
Milvus Lite
pip install pymilvus

45. 45 | © Copyright Zilliz45
Edge AI + Edge Vector Database
Retrieval Augmented
Generation (RAG)
Run local LLM like OLLAMA
Image Similarity Search
Capture and search images at the
edge for no network, local
robotics, remote and secure.
Video Similarity Search
Search for similar videos, scenes,
or objects from local videos.
Audio Similarity Search
Find similar audios in local audio for
tasks like genre classification or
speech recognition for robotics and
sensing
Anomaly Detection
Detect data points, events, audio,
images and observations that
deviate significantly from the usual
pattern at the edge
Facial Recognition
For security applications
Customization
Robots
Benefits
Lower latency
Offline
Security
Localized storage

46. https://medium.com/@tspann/unstructured-data-processing-with-a-raspberry-pi-ai-kit-c959dd7fff47
Raspberry Pi AI Kit Hailo
Edge AI

47. https://medium.com/@tspann/edgeai-edge-vector-database-6a9b5238bffb
https://github.com/tspannhw/AIM-XavierEdgeAI

48. 48 | © Copyright Zilliz48 | © Copyright Zilliz48
RESOURCES

49. 49 | © Copyright Zilliz49
Vector Database Resources
Give Milvus a Star!




Chat with me on Discord!
https://github.com/milvus-io/milvus

50. 50
Unstructured Data Meetup


https://www.meetup.com/unstructured-data-meetup-new-york/

This meetup is for people working in unstructured data. Speakers will come present about related topics
such as vector databases, LLMs, and managing data at scale. The intended audience of this group
includes roles like machine learning engineers, data scientists, data engineers, software engineers, and
PMs.
This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.

52. https://medium.com/@tspann/unstructured-street-data-in-new-york-8d3cde0a1e5b

53. https://medium.com/@tspann/not-every-field-is-just-text-numbers-or-vectors-976231e90e4d

54. https://medium.com/@tspann/shining-some-light-on-the-new-milvus-lite-5a0565eb5dd9

55. Extracting Value from Unstructured Data
Example
•A company has 100,000s+ pages of
proprietary documentation to enable
their staff to service customers.
Problem
•Searching can be slow, inefficient, or
lack context.
Solution
•Create internal chatbot with ChatGPT
and a vector database enriched with
company documentation to provide
direction and support to employees
and customers.
https://osschat.io/chat

56. We provide deployment flexibility for different operational, security and compliance requirements
BRING YOUR OWN CLOUD
Zilliz BYOC
Enterprise-ready Milvus for
Private VPCs
Deploy in your virtual private cloud
Zilliz Cloud
Milvus Re-engineered for the
Cloud
Available on the leading public
clouds
FULLY MANAGED SERVICE
Coming Soon!Coming Soon!
Milvus
Most widely-adopted open
source vector database
Self hosted on any machine with
community support
SELF MANAGED SOFTWARE
Local Docker K8s

57. 57 | © Copyright Zilliz57
Well-connected in LLM infrastructure to enable RAG
use cases
Framework
Hardware
Infrastructure
Embedding Models LLMs
Software Infrastructure
Vector Database

58. 58 | © Copyright 10/22/23 Zilliz58 | © Copyright 10/22/23 Zilliz
58
This week in Milvus, Towhee, Attu, GPT
Cache, Gen AI, LLM, Apache NiFi, Apache
Flink, Apache Kafka, ML, AI, Apache Spark,
Apache Iceberg, Python, Java, Vector DB
and Open Source friends.
https://bit.ly/32dAJft
https://github.com/milvus-io/milvus

AIM Weekly by Tim Spann

59. 59 | © Copyright 10/22/23 Zilliz59 | © Copyright 10/22/23 Zilliz
59
https://flankworkspace.slack.com/

https://join.slack.com/t/flankworkspac
e/shared_invite/zt-2fycjv241-~NRHZDt
dfwDjlfvXK_Bz0A
Join Our Slack and Interact with LLM

60. 60 | © Copyright 10/22/23 Zilliz60 | © Copyright 10/22/23 Zilliz
milvus.io
github.com/milvus-io/
@milvusio
@paasDev


/in/timothyspann
Connect with me! Thank you!

61. 61 | © Copyright 10/22/23 Zilliz61 | © Copyright 10/22/23 Zilliz
Join us at our next meetup!
meetup.com/unstructured-data-meetup-
new-york/

62. 62 | © Copyright Zilliz62
T H A N K Y O U

63. 63 | © Copyright 10/22/23 Zilliz63 | © Copyright 10/22/23 Zilliz
Vector Database Resources
Give Milvus a Star!




Chat with me on Discord!
https://github.com/milvus-io/milvus

64. 64 | © Copyright 10/22/23 Zilliz64 | © Copyright 10/22/23 Zilliz
Separation of storage and compute Yes
Separation of query, insertion, index creation, and coordination. Yes. At the component level (which provides more fine-grained scalability)

Multi-replication Yes
Dynamic segment placement vs. static data sharding Dynamic segment placement
Cloud-native Yes

Regarding scalability, Milvus uses worker nodes for each type of action
(components to handle connections, data nodes to handle ingestion, index
nodes to index, and query nodes to search). Each node has its own assigned
CPU and memory resources. Milvus can dynamically allocate new nodes to an
action group, speeding up operations or reducing the number of nodes, thus
freeing resources for other actions. Dynamically allocating nodes allows for
easier scaling and resource planning and guarantees latency and throughput.
Billion-scale vector support Yes

65. 65 | © Copyright 10/22/23 Zilliz65 | © Copyright 10/22/23 Zilliz
Roll-based Access Control (RBAC) Yes
Disk Index support Yes
Support for multi-vector/ multimodal Yes
Support Search Types ANN, Range, Grouping
Table-level partitions Yes

Milvus is the fastest regarding search latency and throughput, supporting a
billion scale-dataset. In addition, its exceptional approach to supporting multiple
in-memory indexes and table-level partitions results in the high performance
required for real-time information retrieval systems.
Hybrid Search Yes with Scalar filtering and combined Sparse and Dense Vectors
Index type supported
9 (FLAT, IVS_FLAT, IVF_SQ8, IVF_PQ, HNSW, ANNOY, BIN_FLAT, and
BIN_IVF_FLAT)

Purpose-built for Vectors Yes
Database rollback Yes
Data Consistency settings Yes

66. 66 | © Copyright 10/22/23 Zilliz66 | © Copyright 10/22/23 Zilliz
Support for both stream and batch of vector data Yes
Binary Vector support Yes
Multiple SDKs Python, Java, Go, C++, Node.js
Milvus has an efficent Database rollback mechanism to ensure that...

Milvus is a fully open source and independent project, maintained by a number of
companies and individuals, some of whom also offer commercial services and
support. Graduate of LF AI Data.
License: Apache-2.0 license

67. 67 | © Copyright Zilliz67
Choosing Vector Embedding Types
https://milvus.io/docs/metric.md

68. 68 | © Copyright Zilliz68
Semantic Similarity
Image from Sutor et al
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Woman = [0.3, 0.4]
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Man = [0.5, 0.2]
Queen - Woman + Man = King
Queen = [0.3, 0.9]
- Woman = [0.3, 0.4]
[0.0, 0.5]
+ Man = [0.5, 0.2]
King = [0.5, 0.7]Man = [0.5, 0.2]
Neural Word Embeddings

69. 69 | © Copyright Zilliz69
Vector Similarity Measures: L2 (Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)
2
+ (0.9-0.7)
2

= √(0.2)
2
+ (0.2)
2

= √0.04 + 0.04
= √0.08 ≅ 0.28

70. 70 | © Copyright Zilliz70
Vector Similarity Measures: Inner Product
(IP)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78

71. 71 | © Copyright Zilliz71
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Measures: Cosine
??????
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.3
2
+0.9
2
* √0.5
2
+0.7
2

= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03

72. 72 | © Copyright Zilliz72
Why RAG?
RAG vs. LLM
-Knowledge of LLM is out-of-date
-LLM can not get your private knowledge
-Hallucinations
-Transparency and interpretability

RAG vs. Fine-tune
-Fine-tune is expensive
-Fine-tune spent much time
-RAG is pluggable

73. Retrieval-Augmented Generation

74. …and cannot process increasingly growing
unstructured data
*Data Source: The Digitization of the World by IDC
20%
Other
newly generated data in 2025
will be unstructured data80%

75. 75 | © Copyright Zilliz75
Milvus Dependencies
https://zilliz.com/blog/Milvus-server-docker-installation-and-packaging-dependencies
?????? Main Dependencies:
●FAISS � (vector search)
●etcd ?????? (metadata store)
●Pulsar/Kafka ?????? (messaging)
●Tantivy � (text search)
●RocksDB ?????? (storage)
●Object Storage ?????? (Minio/S3/GCS/Azure Blob Storage)
●Kubernetes ?????? (containerization)
●StorageClass & Persistent Volumes ??????(Storage Management for etcd and Pulsar)
●Prometheus & Grafana ?????? (monitoring)
?????? Docker Image Size: ~500MB
?????? Release Frequency: ~1x per month, with frequent minor releases
?????? SDKs Available: Python ?????? , Node ?????? , Go ?????? , C# ?????? , Java ☕ , Ruby ??????
?????? Python SDK Installation: pip install pymilvus
✅ Version Compatibility: Ensure SDK and Milvus server versions match (major.minor)

76. …different types of data and schemas needs to be thoroughly planned ahead of time

77. 77 | © Copyright Zilliz77
•Search Quality - Hybrid Search? Filtering?
•Scalability - Billions of vectors?
•Multi tenancy - Isolating Multi-Tenant data
•Cost - Memory, disk, S3?
•Security - Data Safety and Privacy

TL;DR: Vector search libraries lack the infrastructure to help you scale,
deploy, and manage your apps in production.
Why Not Vector Search Libraries?

78. 78 | © Copyright Zilliz78
Why Not Use a SQL/NoSQL Database?
•Inefficiency in High-dimensional spaces
•Suboptimal Indexing
•Inadequate query support
•Lack of scalability
•Limited analytics capabilities
•Data conversion issues

TL;DR: Vector operations are too computationally intensive for
traditional database infrastructures

79. 79 | © Copyright Zilliz79
What is Milvus/Zilliz ideal for?
•Advanced filtering
•Hybrid search
•Multi-vector Search
•Durability and backups
•Replications/High Availability
•Sharding
•Aggregations
•Lifecycle management
•Multi-tenancy
•High query load
•High insertion/deletion
•Full precision/recall
•Accelerator support (GPU,
FPGA)
•Billion-scale storage

Purpose-built to store, index and query vector embeddings from unstructured data at scale.

80. 80 | © Copyright Zilliz80
Vector Databases are purpose-built to handle
indexing, storing, and querying vector data.

Milvus & Zilliz are specifically designed for high
performance and billion+ scale use cases.
Takeaway:

81. 81 | © Copyright Zilliz81
Inverted File Index
Source:
https://towardsdatascience.com/similarity-search-with-ivfpq-9c6348fd4db3

82. 82 | © Copyright Zilliz82
HNSW
Source:
https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf

83. 83 | © Copyright Zilliz83
SQ

84. 84 | © Copyright Zilliz84
Open Source
Deploy fully managed or “Bring Your
Own Cloud” (BYOC)
Commercial Offerings

Zilliz Cloud
Optimized Milvus with essential data and
security tools for a high-performing vector
search platform
VECTOR SEARCH
ENGINE
VECTORDB
BENCHMARK TOOL
VECTOR DATABASE
SEMANTIC CACHE
FOR LLM QUERIES
GPT-Cache
Product Portfolio
GUI for Milvus

85. 85 | © Copyright Zilliz85
Couple of Customers

86. 86 | © Copyright Zilliz86
Basic Idea
You want to use your data with a large
language model

87. 87 | © Copyright Zilliz87
Cloud
Service
Provider
Data Platform
GenAI Tooling
Chip
Manufacturer
Partner with Industry Leaders

88. 88 | © Copyright Zilliz88
Composite Identifiers
•Combine multiple fields to create a unique identifier.
Example: Combine user_id and timestamp to create unique IDs for
user-generated content. E.g. {"id": "user123_20240606T123000"}

Hierarchical IDs
•Use hierarchical structures for complex data sets.
Example: For a hierarchical document system, use IDs like {"id":
"projectA_chapter1_section2"}
Identifier Strategies