Build AI-powered Search for Every Data Source with Fivetran and Milvus

Fivetran now supports Milvus vector database as a destination, making it easier to onboard every data source for RAG and AI-powered search. Fivetran is a cloud-based data movement platform that makes it easy to extract, load, and transform data between a variety of sources and destinations. Fivetran connects over 500 sources, such as databases/warehouses, SaaS applications, ERPs, and files. The newly introduced Fivetran's Milvus Destination further extends the data landscape to make every data source semantic-searchable. By ingesting the source data from a diverse set of database/warehouses and business apps to Milvus vector database, this integration makes the development of AI workflows easier.

1. AI Powered
Search

2. “
2
www.linkedin.com/in/abhijeeth-padarthi
[email protected]
ABHIJEETH PADATHI
Staff R&D Engineer,
Fivetran

3. 1.What is Fivetran?
Guideline on how to use the template

2.Why is a Good Data Architecture Important?


3.Milvus Vector Database Connector


4.Build Search on Business Data
Logos, icons, illustrations, anything you might need to build your presentation.
Record of the latest changes and updates we made for the deck

3
Table of contents

4. Fivetran
Automated Data Movement
4

6. 6
2012
2013
2014
2018
June 2021
September 2021
October 2021
November 2022
December 2022
February 2023
April 2023
September 2023
November 2023
Fivetran is founded
Fivetran graduates YC
Fivetran builds first connector Salesforce)
Series A
Fivetran acquires Teleport Data
Series D at $565 million
Fivetran acquires HVR
Fivetran releases first HVA connector
Fivetran awarded partner of the year for
Snowflake, Databricks, Google Cloud
Fivetran releases LiteConnectors
Fivetran supports S3 as a destination
Fivetran releases 1 min syncs
Over 400 connectors available
November 2023 Fivetran releases Partner SDK
June 2024 Fivetran launches Managed Data Lake Service
September 2024 Fivetran supports Hybrid Deployment
September 2024 Milvus uses SDK to build a Fivetran Destination
March 2024 Fivetran launches Connector SDK in PP

7. ?????? Data ⇒ ?????? AI
Good Data Architecture Is Important for AI Apps
7

9. Upgrade version of XYZ to 5.6+
customer
I am facing issue with XYZ.
How do I resolve it? Error : ….
Agent

11. AI
APPS
SOURCES
DATA WAREHOUSE
/ DATA LAKE
VECTOR
STORE
SEARCH
CHAT
CLEANING &
DENORMALIZING
( TRANSFORMS )
Extract + Load Vectorize
ANALYTICS
Search
EMBEDDING
MODEL
CHATS
DOCUMENTS

12. Fivetran and Zilliz Enable Better Data Architectures With the
Milvus
Destination
Connector

13. TRANSFORMATIONS
DATA REPOSITORY
AI
APPS
SOURCES
VECTOR DB
SEARCH
CHAT
Extract + Load
ANALYTICS
Search
CHATS
DOCUMENTS
EMBEDDING
SERVICE

14. TRANSFORMATIONS
DATA REPOSITORY
AI
APPS
SOURCES
VECTOR DB
SEARCH
CHAT
Extract + Load
ANALYTICS
Search
CHATS
DOCUMENTS
EMBEDDING
SERVICE

15. TRANSFORMATIONS
DATA REPOSITORY
AI
APPS
SOURCES
VECTOR DB
SEARCH
CHAT
Extract + Load
ANALYTICS
Search
EMBEDDING
SERVICE
CHATS
DOCUMENTS
Extract +
Vectorize +
Load
Milvus Vector Database Connector
for Fivetran
( Built using Fivetran Partner SDK )

16. How to
Build Search
on Business Data

17. E + L
Snowflake
(Customer Managed DW / DL)
T E+V+L
Milvus/Zilliz
VectorDB
OpenAI
Embedding Service

18. 18
Step 1
Create a
Snowflake
Destination

19. 19
Step 2
Create a
Github to Snowflake
Fivetran Connector
E + L

20. 20
Step 3
Setup a DBT Core
Transformation
in Fivetran
https://github.com/fivetran/github_dbt_for_rag

21. 21
Step 4
Create a
Milvus Fivetran
Destination

22. 22
Step 5
Create a
Snowflake to Milvus
Fivetran Connector
E+V+L
22

23. Letʼs go Live

24. Try it out yourself! -> https://fivetran-zilliz-rag.streamlit.app/
Demo App – CHAT with your Data

25. Thank you!

26. 26

Want to learn more about Milvus/Zilliz or Fivetan’s capabilities
with the Milvus Vector Database? Book a demo by scanning
the QR codes below!



What’s Next
https://go.fivetran.com/demo https://zilliz.com/contact-sales