MLOPS By Amazon offered and free download

MLOPS by AMAZON

1. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
© 2021, Amazon Web Services, Inc. or its affiliates.
Introduction to MLOps
Bringing DevOps and Automation to Machine
Learning
HeiChow
Solutions Architect

2. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
© 2022, Amazon Web Services, Inc. or its affiliates. 2
Current state of AI/ML

3. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
State of machine learning
3
By end of 2024
•75% of organizations will shift from piloting to
operationalizing AI
-Gartner
Last decade
•Focusing mostly on building ML models
•Operationalization was an afterthought
•Today
•53% of POCs make it into production
•Average 9 months
-Gartner
https://www.idgconnect.com/article/3583467/gartner-accelerating-ai-deployments-paths-of-least-resistance.html

4. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Main Challenges
4
•Publishing a ML model is not
enough.
•Managing the published ML
models is as important as
developing them.
•“IT leaders responsible for AI are
discovering ‘AI pilot paradox’, where
launching pilots is deceptively easy but
deploying them into production is
notoriously challenging.”
•Chirag Dekate, Vice President
Analyst, Gartner

5. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
© 2021, Amazon Web Services, Inc. or its affiliates. 5
From DevOps to MLOps

6. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
The ML process
6
Business
Problem
ML problem
framing
Data collection
Data integration
Data preparation
and cleaning
Data visualization
and analysis
Feature
engineering
Model training and
parameter tuning
Model evaluation
Monitoring and
debugging
Model deployment
Predictions
YESNO

7. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Phase 1: Research/Experiment
7
Typical scenarios
•Scientific projects
•Proof-of-concepts (PoCs)
Question: “Can we use ML to solve this?”
•“Is it possible to … ?”
•“Can we use this data to solve the following
problem?”
•“Surely we must be able to …”
ML Code
Data
Collection
Data
Verificatio
n
Feature
Extraction
Resource
Manageme
nt
Process
Manageme
nt
Analysis
Tools
Serving
Infrastruct
ure
Monitoring

8. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Phase 2: Operational
8
Typical scenarios
•After PoC, bringing your ML models to
production
•Migration of existing models into ML platform
Question: “How do we implement this method
at scale?”
•How do we pipe the data into the model in a timely
fashion?
•How do we collect, store and transform data so
models can be retrained consistently?
•How do we build an A/B testing environment, in
order to test future model iterations?
ML
Code
Data
Collection
Data
Verificatio
n
Feature
Extraction
Resource
Manageme
nt
Process
Manageme
nt
Analysis
Tools
Serving
Infrastruct
ure
Monitoring

9. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
MLOps –Why?
9
Agility Experiments Scalability Time to MarketBusiness Owners
•Strong collaboration
•Improve iterations
•Reduced time-to-market
•Faster planning and
delivery expectations
•Ease integration of
new ML model
•Standarization of code
•Lower operational
costs
•Faster and Controlled
Experiments
•Faster integration of
successful experiments
to other environments
•Continuous and faster
deliveries
•Faster modifications
•Faster bug-fixing

10. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
MLOps –What?
10
ML+ Dev+ Ops = MLOps
Collaborative and experimental in nature |Automate as much as possible |
Continuous improvement of ML Models |Standardize and Scale
ML Dev Ops
+
Models
Data Monitor
+

11. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
ML Code and Data are Independent
11
Model
Data
Training
Algorithm
Model architecture
Configuration
Data validation
Shuffle and split
Transformation and
feature engineering
Model analysis
Model tuning
Model deployment
Code

12. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
How is MLOpsdifferent from DevOps?
12
https://medium.com/analytics-vidhya/mlops-the-epoch-of-productionizing-ml-models-4eec06d93623
DevOps MLOPS
Code versioning ✓ ✓
Compute environment ✓ ✓
Continuous integration/delivery (CI/CD)✓ ✓
Monitoring in production ✓ ✓
Data provenance ✓
Datasets ✓
Models ✓
Hyperparameters ✓
Metrics ✓
Workflows ✓
MLOPS
End-to-end ML
lifecycle
management

13. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
MLOpspractices
13
Data Engineer
DevOps Engineer
Business
Stakeholder
Project
Identified
Data Scientists/
ML Engineer
Software Engineer
Feedback
Data
Data
Preparation
Model
Build
Model
Training
Model
Artifact
Deploy Integrate Operate
Model
Registry
Deployment pipelineTraining pipeline

14. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
ML lifecycle management
14
Code
Model
Data
Model building
Model
evaluation and
experimentatio
n
Productionize
model
Testing and
quality
Deployment
Monitoring and
Observability
Candidate
models
<>
Train
code
Train
data
Test
data
Metrics
Chosen
model
Productionized
model
Model
Test
data
<>
Test
code
<>
Application
code
Code & model
in production
Production
data

15. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
© 2021, Amazon Web Services, Inc. or its affiliates.
Automating ML Workflows
using SageMaker

16. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Technology components in MLOps
16
MLOPS
Task
orchestratio
n
AWS native
Open
source
ML platform
DIY SageMaker
•ML development,
experimentation,
collaboration
•Compute/training
environment
•Model registry
•Feature store
•Model deployment
•Monitoring in
production
•Hyperparameter
optimization
•Dataset management
Amazon
SageMaker
Amazon EKS
Amazon EC2Amazon ECR
Amazon ECS
AWS Deep
Learning AMIs
AWS Deep
Learning
Containers
•Create and manage
workflows
•Automate ML steps
& pipelines
•Implement CI/CD
•Form a Directed
Acyclic Graph (DAG)
B
A
C
D
E

17. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
17
Integrated Workbench
Capabilities designed specifically for ML, data
preparation, experiment management,
andworkflows
Managed Infrastructure
Designed for ultra low latency and high
throughput, automatic scaling, and
distributedtraining
Managed Tooling
Purpose-built from the ground up to
worktogether including auto ML,
collaboration,debugger, profiler, bias
analyzer,and explainability
Amazon SageMaker
Most complete, end-to-end ML service

18. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerFeatures
18
PREPARE
SageMaker Ground Truth
Label training data for machine
learning
SageMaker Data Wrangler
Aggregate and prepare data for
machine learning
SageMaker Processing
Built-in Python, BYO R/Spark
SageMaker Feature Store
Store, update, retrieve, and share
features
SageMakerClarify
Detect bias and understand
model predictions
BUILD
SageMaker Studio
Notebooks
Jupyter notebooks with elastic
compute and sharing
Built-in and Bring
your-own Algorithms
Dozens of optimized algorithms
or bring your own
Local Mode
Test and prototype on your local
machine
SageMaker Autopilot
Automatically create machine
learning models with full
visibility
SageMakerJumpStart
Pre-built solutions for common
use cases
TRAIN & TUNE
Managed Training
Distributed infrastructure
management
SageMaker Experiments
Capture, organize, and compare
every step
Automatic
Model Tuning
Hyperparameter optimization
Distributed Training
Libraries
Training for large datasets
and models
SageMakerDebugger
Debug and profile training runs
Managed Spot Training
Reduce training cost by 90%
DEPLOY & MANAGE
Managed Deployment
Fully managed, ultra low latency,
high throughput
Kubernetes & Kubeflow
Integration
Simplify Kubernetes-based
machine learning
Multi-Model Endpoints
Reduce cost by hosting multiple
models per instance
SageMaker Model Monitor
Maintain accuracy of deployed
models
SageMaker Edge Manager
Manage and monitor models on
edge devices
SageMakerPipelines
Workflow orchestration and
automation
Amazon SageMaker
SageMaker Studio
Integrated development environment (IDE) for ML

19. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Technology components in MLOps
19
MLOPS
Task
orchestratio
n
AWS native
Open
source
ML platform
DIY SageMaker
•ML development,
experimentation,
collaboration
•Compute/training
environment
•Model registry
•Feature store
•Model deployment
•Monitoring in
production
•Hyperparameter
optimization
•Dataset management
AWS Step
Functions
Pipelines
Amazon
SageMaker
Amazon EKS
Amazon EC2
Kubeflow
Apache Airflow
MLflow
Amazon ECR
Amazon ECS
AWS Deep
Learning AMIs
AWS Deep
Learning
Containers
•Create and manage
workflows
•Automate ML steps
& pipelines
•Implement CI/CD
•Form a Directed
Acyclic Graph (DAG)
B
A
C
D
E

20. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Task orchestration
20
Apache
Airflow
Platform to author,
schedule and
monitor workflows
Kubeflow
ML toolkit
for
Kubernetes
AWS Step
Functions
Serverless
pipeline
orchestration
MLflow
Open source
platform for
the ML
lifecycle
Amazon
SageMaker
Pipelines
Managed ML
pipelines in
SageMaker Studio
Native AWS optionsOpen source 3
rd
party options
Native integration with SageMaker
Kubeflow & Kubernetes
•SageMaker Components for
Kubeflow Pipelines
•SageMaker Operators for
Kubernetes
Apache Airflow
•SageMaker Operators in Apache Airflow
•
(managed Airflow service)
Amazon Managed Workflows
for Apache Airflow

21. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerPipelines
Components
Project
Pipelines
End-to-End Traceability & Integration
Automated
Model Build
Workflows
Model Registry
Central Store to
manage models
Model
Deployment
Pipeline
Source Code
Control
Built-In
Triggers
</>
1
2 3

22. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerPipelines
Components –Pipelines
Project
End-to-End Traceability & Integration
Model Registry
Central Store to
manage models
Model
Deployment
Pipeline
Source Code
Control
Built-In
Triggers
</>
1
3
Pipelines
Automated
Model Build
Workflows
2

23. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerPipelines
Components –Pipelines
Supported Steps:
•Processing
•Training
•Tuning
•Conditional
•Register Model
•Create Model

24. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerPipelines
Components –Model Registry
Project
Pipelines
End-to-End Traceability & Integration
Automated
Model Build
Workflows
Model
Deployment
Pipeline
Source Code
Control
Built-In
Triggers
</>
1
2
Model Registry
Central Store to
manage models
3

25. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerPipelines
Components –Model Registry
•Catalog models for production
•Manage model versions
•Associate metadata with a model
•Manage the approval status of a model
•Deploy models to production (with Projects)
•Track model performance metrics

26. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerProjects
High Level Services View
Build, Train, Deploy Template

27. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerProjects
using third-party source control and Jenkins
https://aws.amazon.com/blogs/machine-learning/create-amazon-sagemaker-projects-using-third-party-source-control-and-jenkins/

28. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
End-to-End Traceability & Integration
Pipelines
Automated
Model Build
Workflows
2
Amazon SageMakerPipelines
Built-In Triggers
Project
Model Registry
Central Store to
manage models
Model
Deployment
Pipeline
Source Code
Control</>
1
3
Built-In
Triggers

29. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Creating retraining strategies
29
”We know we want to
train every week on
Saturday at 23:00”
Amazon EventBridge
Scheduled event
Example:
Trigger
retraining
workflow
2 -Event Driven
Example:
1 -Scheduled
3 -Metric Based
Model Quality /
Data Drift Alert
Example:
Amazon CloudWatch
Event (event-based)
Amazon EventBridge
AWS event
AWS Glue
Job Status
Pipelines
Automated
Model Build
Workflows
Model Registry
Central Store to
manage models
Model
Deployment
Pipeline

30. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Amazon SageMakerModel Monitor
30
Supported Features:
•Automatic data collection
•Continuous monitoring
•Flexible Monitoring Rules
•Visual data analysis
•CloudWatch integration

31. © 2022, Amazon Web Services, Inc. or its affiliates.
Amazon
SageMaker
MLOps
Streamline the ML lifecycle
Automate ML workflows to
scale model development
Build CI/CD pipelines for ML to
accelerate model deployment
Catalog model versions, metadata, metrics,
and approvals for traceability and reusability
Track lineage for troubleshooting
and compliance
Maintain accuracy of predictions
after models are deployed
Enhance governance and security

32. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
© 2022, Amazon Web Services, Inc. or its affiliates.
Getting Started
32

33. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Getting started: Next steps
33
Training and
Certification
Discovery and
Get Hands Dirty
Proof of
Concepts (PoC)
AWS Partner
Network (APN)

34. © 2022, Amazon Web Services, Inc. or its affiliates.
INTRODUCTION TO MLOPS
Thank you! Fill in the event survey
and get USD 25 AWS Credits
© 2022, Amazon Web Services, Inc. or its affiliates. 34