Multi-agent Systems with Mistral AI, Milvus and Llama-agents

With the recent release of Llama Agents, we can now build agents that are async first and run as their own service. During this webinar, Stephen will show you how to build an Agentic RAG System using Llama Agents and Milvus.

1. 1 | © Copyright 8/16/23 Zilliz1 | © Copyright 8/16/23 Zilliz
Multi-agent Systems with
Mistral AI, Milvus and
Llama-agents
Stephen Batifol | Zilliz

Zilliz Webinar, Aug. 8th

2. 2 | © Copyright 8/16/23 Zilliz2 | © Copyright 8/16/23 Zilliz
Stephen Batifol
Developer Advocate, Zilliz/ Milvus
[email protected]
linkedin.com/in/stephen-batifol/
@stephenbtl
Speaker

3. 3 | © Copyright 8/16/23 Zilliz3 | © Copyright 8/16/23 Zilliz
28K
GitHub
Stars
25M
Downloads
250
Contributors
2,600
+Forks
Milvus is an open-source vector database for GenAI projects. pip install on your
laptop, plug into popular AI dev tools, and push to production with a single line of
code.
Easy Setup

Pip-install to start
coding in a notebook
within seconds.
Reusable Code

Write once, and
deploy with one line
of code into the
production
environment
Integration

Plug into OpenAI,
Langchain,
LlmaIndex, and
many more
Feature-rich

Dense & sparse
embeddings,
filtering, reranking
and beyond

4. 4 | © Copyright 8/16/23 Zilliz4 | © Copyright 8/16/23 Zilliz
Seamless integration with all popular AI toolkits

5. 5 | © Copyright 8/16/23 Zilliz5 | © Copyright 8/16/23 Zilliz | © Copyright 8/16/23 Zilliz 5
RAG
Retrieval Augmented Generation)

6. 6 | © Copyright 8/16/23 Zilliz6 | © Copyright 8/16/23 Zilliz
Basic Idea
Use RAG to force the LLM to work with your data
by injecting it via a vector database like Milvus

7. 7 | © Copyright 8/16/23 Zilliz7 | © Copyright 8/16/23 Zilliz
Basic RAG Architecture

8. 8 | © Copyright 8/16/23 Zilliz8 | © Copyright 8/16/23 Zilliz
5 lines starter

9. 9 | © Copyright 8/16/23 Zilliz9 | © Copyright 8/16/23 Zilliz
Naive RAG is limited

10. 10 | © Copyright 8/16/23 Zilliz10 | © Copyright 8/16/23 Zilliz
Naive RAG failure mode
Summarization

11. 11 | © Copyright 8/16/23 Zilliz11 | © Copyright 8/16/23 Zilliz
Naive RAG failure mode
Implicit data

12. 12 | © Copyright 8/16/23 Zilliz12 | © Copyright 8/16/23 Zilliz
Naive RAG failure mode
Multi-part questions

13. 13 | © Copyright 8/16/23 Zilliz13 | © Copyright 8/16/23 Zilliz 13| © Copyright 8/16/23 Zilliz
RAG is necessary but
not sufficient

14. 14 | © Copyright 8/16/23 Zilliz14 | © Copyright 8/16/23 Zilliz
Good dishes come from good ingredients
•Data collection

•Data cleaning

•Parsing & Chunking

15. 15 | © Copyright 8/16/23 Zilliz15 | © Copyright 8/16/23 Zilliz 15| © Copyright 9/25/23 Zilliz15| © Copyright 9/25/23 Zilliz
Simplify and streamline
the conversion of
unstructured data into
state-of-the-art vector
embeddings, using
intuitive UI and Restful
APIs.
Pipelines
Easy. High-quality. Scalable.







Simplify the workflow
for developers, from
converting
unstructured data into
searchable vectors to
retrieving them from
vector databases
Deliver excellence in
every phase of vector
search pipeline
development and
deployment,
regardless of their
expertise
Ensure scalability for
managing large
datasets and
high-throughput
queries, maintaining
high performance with
min. customization or
infra changes
Zilliz Cloud Pipelines

16. 16 | © Copyright 8/16/23 Zilliz16 | © Copyright 8/16/23 Zilliz
Naive RAG Pipeline
⚠ Single-shot
⚠ No query understanding/planning
⚠ No tool use
⚠ No reflection, error correction
⚠ No memory (stateless)

17. 17 | © Copyright 8/16/23 Zilliz17 | © Copyright 8/16/23 Zilliz
First thing first
Measure it before you attempts to improve it!

18. 18 | © Copyright 8/16/23 Zilliz18 | © Copyright 8/16/23 Zilliz 18| © Copyright 8/16/23 Zilliz18| © Copyright 8/16/23 Zilliz
01 Agentic RAG

19. 19 | © Copyright 8/16/23 Zilliz19 | © Copyright 8/16/23 Zilliz
Agentic RAG
✅ Multi-turn
✅ Query / task planning layer
✅ Tool interface for external environment
✅ Reflection
✅ Memory for personalization

20. 20 | © Copyright 8/16/23 Zilliz20 | © Copyright 8/16/23 Zilliz

21. 21 | © Copyright 8/16/23 Zilliz21 | © Copyright 8/16/23 Zilliz

22. 22 | © Copyright 8/16/23 Zilliz22 | © Copyright 8/16/23 Zilliz

23. 23 | © Copyright 8/16/23 Zilliz23 | © Copyright 8/16/23 Zilliz
Conversation Memory

24. 24 | © Copyright 8/16/23 Zilliz24 | © Copyright 8/16/23 Zilliz
ReAct Reasoning and Action) Prompting
Designed to:
•Integrate the reasoning capabilities of LLMs
•Ability to take actionable steps

Itʼs able to:
•Understand and process information
•Evaluate situations, take appropriate actions
•Communicate responses
•Track ongoing situations

25. 25 | © Copyright 8/16/23 Zilliz25 | © Copyright 8/16/23 Zilliz
ReAct Reasoning and Action) Prompting

26. 26 | © Copyright 8/16/23 Zilliz26 | © Copyright 8/16/23 Zilliz
Tool Use

27. 27 | © Copyright 8/16/23 Zilliz27 | © Copyright 8/16/23 Zilliz 27| © Copyright 8/16/23 Zilliz27| © Copyright 8/16/23 Zilliz
02
RAG in action with Milvus
Lite

28. 28 | © Copyright 8/16/23 Zilliz28 | © Copyright 8/16/23 Zilliz
•Framework for building LLM Applications
•Focus on retrieving data and integrating with
LLMs
•Integrations with most AI popular tools
?????? llama-index

29. 29 | © Copyright 8/16/23 Zilliz29 | © Copyright 8/16/23 Zilliz
?????? llama-agents ?????? by llama-index
•Build Stateful apps with LLMs
and Multi-Agents workflow
•Cycles and Branching
•Human-in-the-Loop
•Persistence

30. 30 | © Copyright 8/16/23 Zilliz30 | © Copyright 8/16/23 Zilliz
?????? llama-agents ?????? - Components

31. 31 | © Copyright 8/16/23 Zilliz31 | © Copyright 8/16/23 Zilliz
Mistral AI
•Mistral Embed
•Embedding Model focused on Retrieval, very useful
for RAG
•English only

•Mistral Nemo
•12B model with 128k context length
•Strong Function Calling and Retrieval for its size
•Run Locally

•Mistral Large 2
•123 Billions parameters with 128K context length
•Very strong Function Calling and Retrieval skills

32. 32 | © Copyright 8/16/23 Zilliz32 | © Copyright 8/16/23 Zilliz
pip install pymilvus
Milvus Lite

33. 33 | © Copyright 8/16/23 Zilliz33 | © Copyright 8/16/23 Zilliz | © Copyright 8/16/23 Zilliz 33
Demo!

34. 34 | © Copyright 8/16/23 Zilliz34 | © Copyright 8/16/23 Zilliz
milvus.io
github.com/milvus-io/
@milvusio
@stephenbtl


/in/stephen-batifol
Thank you

35. 35 | © Copyright 8/16/23 Zilliz35 | © Copyright 8/16/23 Zilliz
Meta Storage
Root Query Data Index
Coordinator Service
Proxy
Proxy
etcd
Log Broker
SDK
Load Balancer
DDL/DCL
DML
NOTIFICATION
CONTROL SIGNAL
Object Storage
Minio / S3 / AzureBlob
Log Snapshot Delta File Index File
Worker Node
QUERY DATA DATA
Message Storage
VECTOR
DATABASE
Access Layer
Query Node Data Node Index Node
Milvus Architecture