SAP Building Agentic Systems (Milvus Community Meetup)

This is Nicholas Roze-Freitas's speaker slides from the October Milvus Community Meetup.

1. CONFIDENTIAL
Building Agentic Systems
By Nicholas Roze-Freitas

2. My Background
•10 Months in SAP Academy for Product & Engineering
•Studied Data Science and Math at UCSB
•Data Scientist building agents at SAP Concur

3. Agentic Systems?
•Different categories of Agentic Systems
•independent, fully autonomous
•More prescriptive implementation that
follows a flow
•LangChain’s blog writes often times Agentic
Systems in production are a combination

4. Complex
Reasoning
Dynamic
Decision-making
Unstructured
Data
When are Agents good to use?

5. Main Components Of An Agent
•LLM
•Tools
•Memory
•Short term
•Long term
Langgraph

6. Building Agents
•Tons of frameworks
•Langchain / Langgraph
•CrewAI
•Google ADK
•OpenAI Agents SDK
•… and more!
•Agents themselves are pretty simple
•Most frameworks implement the same loop





Also Langgraph

7. The Loop
•Feed Messages to LLM
•If LLM called tools
•Call tools & add results to messages
•If LLM responds with text
•Get user feedback if applicable
•Or do something custom






Again Langgraph

8. Some Hidden Challenges
•Security
•Malformed & incorrect tool calling
•Message history and context management

Agents
Sometimes

9. Hidden Challenge: Security
•LLMs can generate malicious outputs
•User/company data isolation
•Console commands
•Solution?
•Design interactions with implicit guardrails
•User/company based instances of tools
Agents
Sometimes
(again)

10. Hidden Challenge: Malformed Function Calls
•LLMs can fail to generate function calls
•Malformed function calling can break up
flows
•Solution?
•Define tools with primitives
•Implement custom error handling

11. Hidden Challenge: Context
•Agents can forget things
•Context helps agents understand their task and
get information
•Solution?
•Keep most important info at top of context
•Design RAG systems to pull context
dynamically



Agents Thinking
Really Hard

12. My Library of choice: LangChain & LangGraph

13. Why I like Lang(Chain/Graph)
•Offers high and low level APIs
•Represents agentic systems as a graph
•Low level APIs give the ability to build highly
customizable solutions
•Building blocks
•Mitigations
•Compliance
Some Companies Using
LangGraph

14. Getting Started
•LangGraph/Langchain offers high level APIs that are
easy to start with
•create_agent
•middleware
•Or get into the weeds and just start with LangGraph!

15. What I’m Excited About
•Fine-tuned small language models
•Large model inference is expensive & unnecessary
•Agent’s often handle specialized tasks

16. Thank you + Questions
Connect with me on LinkedIn @ Nicholas
Roze-Freitas