API Days Australia - AI Agent Orchestration using Docker

The slide deck used during API Days Australia. This presentation on AI Agent Orchestration using Docker was part of the Kubernetes + AI track.

https://www.apidays.global/events/australia

The presentation covered key features of Docker Model Runner to enable offline Agentic AI development and testing capabilities

1. Nilesh Gule @nileshgule
AI Agent Orchestration
using
Docker

2. $whoami
{
“name” : “Nilesh Gule”,
“role” : “Senior Cloud Solutions Architect at Avanade”
“website” : “https://www.HandsOnArchitect.com",
“github” : “https://GitHub.com/NileshGule"
“twitter” : “@nileshgule”,
“linkedin” : “https://www.linkedin.com/in/nileshgule”,
“YouTube” : “https://www.YouTube.com/@nilesh-gule”
“likes” : “Technical Evangelism, Cricket”,
}

3. Challenges in building Agentic AI Apps
•Dependency on hosted models
•Quotas based on service tiers
•Unpredictable costs
•Additional setup / infrastructure to build and test Agentic
apps

4. AI Grocery Shopper
Meal Planner
Agent
Inventory
Agent
Budget
Agent
Shopper
Agent
Generate meal
plans and
shopping list
based on
preferences and
dietary
restrictions
Compare
inventory items
and build revise
shopping list
Optimize items
based on budget
Optimize items
based on
categories

5. /demo
Docker Model Runner

6. Docker Model Runner Architecture
Docker Model Runner Design

7. Docker Model Runner Endpoints
•From Containers -http://model-runner.docker.internal/
•From host processes - http://localhost:12434/
•Create model - POST /models/create
•List models - GET /models
•Get a model - GET /models/{namespace}/{name}
•Delete local model - DELETE /models/{namespace}/{name}

8. OpenAIEndpoints
•List Models - GET /engines/llama.cpp/v1/models
•Retrieve models - GET /engines/llama.cpp/v1/models/{namespace}/{name}
•List chat completions - POST /engines/llama.cpp/v1/chat/completions
•Create completions - POST /engines/llama.cpp/v1/completions
•Createembeddings-POST /engines/llama.cpp/v1/embeddings

9. Summary -Docker Model Runner
Docker Model Runner General Availability | LinkedIn

10. Resources
•Docker Docks - Model Runner
•Docker Docs - Use AI models in Compose
•Docker Model Runner Design and What’s Next
•Docker Model Runner General Availability Annoncement

11. Nilesh Gule
ARCHITECT | MICROSOFT MVP
“Code with Passion and
Strive for Excellence”
nileshgule
@nileshgule Nilesh Gule
NileshGule
www.handsonarchitect.com
https://www.youtube.com/@nilesh-gule

12. Source Code & slide deck
AI Grocery Shopper
https://github.com/NileshGule/AI-Grocery-Shopper
https://speakerdeck.com/nileshgule/
https://www.slideshare.net/nileshgule/

13. Q&A