Multi-agent Systems with Mistral AI, Milvus and Llama-agents

Agentic systems are on the rise, helping developers create intelligent, autonomous systems. LLMs are becoming more and more capable of following diverse sets of instructions, making them ideal for managing these agents. This advancement opens up numerous possibilities for handling complex tasks with minimal human intervention in so many areas. In this talk, we will see how to build agents using llama-agents. We’ll also explore how combining different LLMs can enable various actions. For simpler tasks, we'll use Mistral Nemo, a smaller and more cost-effective model, and Mistral Large for orchestrating different agents.

1. 1 | © Copyright 8/16/23 Zilliz1 | © Copyright 8/16/23 Zilliz
Stephen Batifol | Zilliz

Unstructured Data Meetup, Sept 5th
Multi-agent Systems with
Mistral AI, Milvus and
Llama-agents

2. 2 | © Copyright 8/16/23 Zilliz2 | © Copyright 8/16/23 Zilliz
Stephen Batifol
Developer Advocate, Zilliz/ Milvus
[email protected]
linkedin.com/in/stephen-batifol/
@stephenbtl
Speaker

3. 3 | © Copyright 8/16/23 Zilliz3 | © Copyright 8/16/23 Zilliz
28K
GitHub
Stars
25M
Downloads
250
Contributors
2,600
+Forks
Milvus is an open-source vector database for GenAI projects. pip install on your
laptop, plug into popular AI dev tools, and push to production with a single line of
code.
Easy Setup

Pip-install to start
coding in a notebook
within seconds.
Reusable Code

Write once, and
deploy with one line
of code into the
production
environment
Integration

Plug into OpenAI,
Langchain,
LlmaIndex, and
many more
Feature-rich

Dense & sparse
embeddings,
filtering, reranking
and beyond

4. 4 | © Copyright 8/16/23 Zilliz4 | © Copyright 8/16/23 Zilliz
Seamless integration with all popular AI toolkits

5. 5 | © Copyright 8/16/23 Zilliz5 | © Copyright 8/16/23 Zilliz 5| © Copyright 8/16/23 Zilliz5| © Copyright 8/16/23 Zilliz
01 Key Components

6. 6 | © Copyright 8/16/23 Zilliz6 | © Copyright 8/16/23 Zilliz
pip install pymilvus
Milvus Lite

7. 7 | © Copyright 8/16/23 Zilliz7 | © Copyright 8/16/23 Zilliz
•Framework for building LLM Applications
•Focus on retrieving data and integrating with
LLMs
•Integrations with most AI popular tools
?????? llama-index

8. 8 | © Copyright 8/16/23 Zilliz8 | © Copyright 8/16/23 Zilliz
?????? llama-agents ?????? by llama-index
•Build Stateful apps with LLMs
and Multi-Agents workflow
•Cycles and Branching
•Human-in-the-Loop
•Persistence

9. 9 | © Copyright 8/16/23 Zilliz9 | © Copyright 8/16/23 Zilliz
Mistral AI
•Mistral Embed
•Embedding Model focused on Retrieval, very useful
for RAG
•English only

•Mistral Nemo
•12B model with 128k context length
•Strong Function Calling and Retrieval for its size
•Run Locally

•Mistral Large 2
•123 Billions parameters with 128K context length
•Very strong Function Calling and Retrieval skills

10. 10 | © Copyright 8/16/23 Zilliz10 | © Copyright 8/16/23 Zilliz
Tavily
●Web search API for up-to-date information
●Tavily Search API is a search engine
optimized for LLMs
●Expands agent's knowledge base

11. 11 | © Copyright 8/16/23 Zilliz11 | © Copyright 8/16/23 Zilliz | © Copyright 8/16/23 Zilliz 11
RAG
Retrieval Augmented Generation)

12. 12 | © Copyright 8/16/23 Zilliz12 | © Copyright 8/16/23 Zilliz
Basic Idea
Use RAG to force the LLM to work with your data
by injecting it via a vector database like Milvus

13. 13 | © Copyright 8/16/23 Zilliz13 | © Copyright 8/16/23 Zilliz
Basic RAG Architecture

14. 14 | © Copyright 8/16/23 Zilliz14 | © Copyright 8/16/23 Zilliz
5 lines starter

15. 15 | © Copyright 8/16/23 Zilliz15 | © Copyright 8/16/23 Zilliz
Naive RAG is limited

16. 16 | © Copyright 8/16/23 Zilliz16 | © Copyright 8/16/23 Zilliz
Naive RAG Pipeline
⚠ Single-shot
⚠ No query understanding/planning
⚠ No tool use
⚠ No reflection, error correction
⚠ No memory (stateless)

17. 17 | © Copyright 8/16/23 Zilliz17 | © Copyright 8/16/23 Zilliz
Naive RAG failure mode
Summarization

18. 18 | © Copyright 8/16/23 Zilliz18 | © Copyright 8/16/23 Zilliz
Naive RAG failure mode
Implicit data

19. 19 | © Copyright 8/16/23 Zilliz19 | © Copyright 8/16/23 Zilliz
Naive RAG failure mode
Multi-part questions

20. 20 | © Copyright 8/16/23 Zilliz20 | © Copyright 8/16/23 Zilliz 20| © Copyright 8/16/23 Zilliz
RAG is necessary but
not sufficient

21. 21 | © Copyright 8/16/23 Zilliz21 | © Copyright 8/16/23 Zilliz 21| © Copyright 8/16/23 Zilliz21| © Copyright 8/16/23 Zilliz
01 Agentic RAG

22. 22 | © Copyright 8/16/23 Zilliz22 | © Copyright 8/16/23 Zilliz
Agentic RAG
✅ Multi-turn
✅ Query / task planning layer
✅ Tool interface for external environment
✅ Reflection
✅ Memory for personalization

23. 23 | © Copyright 8/16/23 Zilliz23 | © Copyright 8/16/23 Zilliz
●Self-Reflection ??????
○Check the internet to verify
●Query Routing
○Whether to check our RAG system
●Query Routing with Subquery
○Whether to check our RAG system for different queries
●Conversation Memory
●Tool Use
Agentic RAG

24. 24 | © Copyright 8/16/23 Zilliz24 | © Copyright 8/16/23 Zilliz | © Copyright 8/16/23 Zilliz 24
Demo!

25. 25 | © Copyright 8/16/23 Zilliz25 | © Copyright 8/16/23 Zilliz
milvus.io
github.com/milvus-io/
@milvusio
@stephenbtl


/in/stephen-batifol
Thank you

26. 26 | © Copyright 8/16/23 Zilliz26 | © Copyright 8/16/23 Zilliz
Meta Storage
Root Query Data Index
Coordinator Service
Proxy
Proxy
etcd
Log Broker
SDK
Load Balancer
DDL/DCL
DML
NOTIFICATION
CONTROL SIGNAL
Object Storage
Minio / S3 / AzureBlob
Log Snapshot Delta File Index File
Worker Node
QUERY DATA DATA
Message Storage
VECTOR
DATABASE
Access Layer
Query Node Data Node Index Node
Milvus Architecture