Build AI-powered Search for Every Data Source with Fivetran and Milvus

chloewilliams62 199 views 26 slides Sep 26, 2024
Slide 1
Slide 1 of 26
Slide 1
1
Slide 2
2
Slide 3
3
Slide 4
4
Slide 5
5
Slide 6
6
Slide 7
7
Slide 8
8
Slide 9
9
Slide 10
10
Slide 11
11
Slide 12
12
Slide 13
13
Slide 14
14
Slide 15
15
Slide 16
16
Slide 17
17
Slide 18
18
Slide 19
19
Slide 20
20
Slide 21
21
Slide 22
22
Slide 23
23
Slide 24
24
Slide 25
25
Slide 26
26

About This Presentation

Fivetran now supports Milvus vector database as a destination, making it easier to onboard every data source for RAG and AI-powered search. Fivetran is a cloud-based data movement platform that makes it easy to extract, load, and transform data between a variety of sources and destinations. Fivetran...


Slide Content

AI Powered
Search


2
www.linkedin.com/in/abhijeeth-padarthi
[email protected]
ABHIJEETH PADATHI
Staff R&D Engineer,
Fivetran

1.What is Fivetran?
Guideline on how to use the template

2.Why is a Good Data Architecture Important?


3.Milvus Vector Database Connector


4.Build Search on Business Data
Logos, icons, illustrations, anything you might need to build your presentation.
Record of the latest changes and updates we made for the deck

3
Table of contents

Fivetran
Automated Data Movement
4

6
2012
2013
2014
2018
June 2021
September 2021
October 2021
November 2022
December 2022
February 2023
April 2023
September 2023
November 2023
Fivetran is founded
Fivetran graduates YC
Fivetran builds first connector Salesforce)
Series A
Fivetran acquires Teleport Data
Series D at $565 million
Fivetran acquires HVR
Fivetran releases first HVA connector
Fivetran awarded partner of the year for
Snowflake, Databricks, Google Cloud
Fivetran releases LiteConnectors
Fivetran supports S3 as a destination
Fivetran releases 1 min syncs
Over 400 connectors available
November 2023 Fivetran releases Partner SDK
June 2024 Fivetran launches Managed Data Lake Service
September 2024 Fivetran supports Hybrid Deployment
September 2024 Milvus uses SDK to build a Fivetran Destination
March 2024 Fivetran launches Connector SDK in PP

?????? Data ⇒ ?????? AI
Good Data Architecture Is Important for AI Apps
7

Upgrade version of XYZ to 5.6+
customer
I am facing issue with XYZ.
How do I resolve it? Error : ….
Agent

AI
APPS
SOURCES
DATA WAREHOUSE
/ DATA LAKE
VECTOR
STORE
SEARCH
CHAT
CLEANING &
DENORMALIZING
( TRANSFORMS )
Extract + Load Vectorize
ANALYTICS
Search
EMBEDDING
MODEL
CHATS
DOCUMENTS

Fivetran and Zilliz Enable Better Data Architectures With the
Milvus
Destination
Connector

TRANSFORMATIONS
DATA REPOSITORY
AI
APPS
SOURCES
VECTOR DB
SEARCH
CHAT
Extract + Load
ANALYTICS
Search
CHATS
DOCUMENTS
EMBEDDING
SERVICE

TRANSFORMATIONS
DATA REPOSITORY
AI
APPS
SOURCES
VECTOR DB
SEARCH
CHAT
Extract + Load
ANALYTICS
Search
CHATS
DOCUMENTS
EMBEDDING
SERVICE

TRANSFORMATIONS
DATA REPOSITORY
AI
APPS
SOURCES
VECTOR DB
SEARCH
CHAT
Extract + Load
ANALYTICS
Search
EMBEDDING
SERVICE
CHATS
DOCUMENTS
Extract +
Vectorize +
Load
Milvus Vector Database Connector
for Fivetran
( Built using Fivetran Partner SDK )

How to
Build Search
on Business Data

E + L
Snowflake
(Customer Managed DW / DL)
T E+V+L
Milvus/Zilliz
VectorDB
OpenAI
Embedding Service

18
Step 1
Create a
Snowflake
Destination

19
Step 2
Create a
Github to Snowflake
Fivetran Connector
E + L

20
Step 3
Setup a DBT Core
Transformation
in Fivetran
https://github.com/fivetran/github_dbt_for_rag

21
Step 4
Create a
Milvus Fivetran
Destination

22
Step 5
Create a
Snowflake to Milvus
Fivetran Connector
E+V+L
22

Letʼs go Live

Try it out yourself! -> https://fivetran-zilliz-rag.streamlit.app/
Demo App – CHAT with your Data

Thank you!

26

Want to learn more about Milvus/Zilliz or Fivetan’s capabilities
with the Milvus Vector Database? Book a demo by scanning
the QR codes below!



What’s Next
https://go.fivetran.com/demo https://zilliz.com/contact-sales
Tags