Apidays Singapore 2024 - Designing a Scalable MLOps Pipeline by Victoria Lo, WomenWhoCode

Designing a Scalable MLOps Pipeline: Insights and Best Practices
Victoria Lo, Solutions Engineer - WomenWhoCode

Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024)

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

1. Designing a Scalable
MLOps Pipeline:
Insights & Best Practices

| cpidays
April 17, 2024

2. E 2024 SERIES

© April 17 & 18: Singapore - 1000+ participants

dEverything HE

jedFinance

© Moy 28 8 29: Helsinki & North - 500+ participants

HAPlecosystems #DigitalTransforme

© September 18 & 19: London -1,800+ participants #OpenBanking

© October 16 & 17: Australia - 1,500+ participants
usinessNe

@ December 3, 4 & 5: Paris - 4,000+ participants

@ NEW March 2025 - Germany and Dach region
#DigitalTrar v #Aut

formation #TelCo #Fin

e&Manufacturing

March: GERMANY & DACH - April: SINGAPORE
May: NEW YORK - June: HELSINKI

September: LONDON - October: AUSTRALIA - December: PARIS

3. Introduction

“name”: "Victoria Lo",
“stuff ido” : [
"Solutions Engineer",
"Technical Writer",
"Podcaster eragTechDev",
"WomenWhoCode Leader"
l,

“hobbies”: ["games", "books", "tea"],

4. BEFORE WE BEGIN...

Quick Survey!

5. L models work

eo =

ML Model Code

6. predicts alerts

makes > executes
— >
=> e” A

Customer Payment Fraud Detector Risk Score Detection
Model

7. predicts

Customer Payment Fraud Detector Risk Score Detection
Model

8. An example - F

Detection

What if...?

makes

Customer

model changed

> executes

ro

Payment Fraud Detector
Model

alerts
LD —

yA

Risk Score

Detection

9. An example - Fraud Detection

r
What if...?
code changed
makes > executes
— 5 p> — sr
> rf yA
Customer Payment Fraud Detector Risk Score Detection
Model

10. An example - Fraud Detection

What if...?
* Data changed -> Re-training model
« Model changed -> Code needs update
* Code changed -> New model must be deployed

Challenges
+ Model performance starts to decay with time
* A continuous loop
« Hard to maintain and keep model reliable

11. What is MLOps?

4 N

The practice and processes around designing, building, and
deploying ML models into continuous production using
DevOps principles.

12. eo =

Data ML Model Code

13. Compon

Data ML Model Code
pipelines pipelines pipelines

14. Components of MLOps

r

Feature
Ingestion, Deployment,

bestia engineering, m
validation, 9 9 monitoring,

cleaning trainings logging
evaluation

Data ML Model Code
pipelines pipelines pipelines

15. Implementing MLOps

r

$

$ ZenmL

PyTorch
Lightning

come
come

de

vertex.ai

Amazon SageMaker

mlflow

16. vertex.ai

Covers end-to-end processes in the ML workflow from data ingestion to
model deployment/management

Unifies all GCP products under 1 convenient platform for easy use and
integration

Creates pipelines easily with pre-built components

Serverless and fully managed

Save time and costs on infrastructure, solutions-focused

17. Building a Fraud |

Detection

Pipeline with
vertex.ai

18. Step 1: Import Data

Determine the data you need for training and testing your model
based on the outcome you want to achieve.

Great et € fraud.detection
SOURCE ANALY

” Data setinto

Patane ger ope

‘Summary

19. Step 2: Train/Evaluate Model

Train: Set parameters and build your model
Evaluate: Review model metrics.

Train new model

© Training method

Object
Model details Classsieavon

Training options
Hé Please refer othe pricing guide for mare deta (and avaiable deployment options) or

‘Compute and pricing rad

Auto
CANCEL, © Train h

© Custom training (advances)

CONTINUE

20. Step 3: Define a metrics evaluation custom component

Before deploying our model to production, we need
to:

+ Setup our Data, Model and Code pipelines
+ Define a metrics evaluation custom component

Define a metrics evaluation custom component

+ retrieves the classification model evaluation
generated by the AutoML tabular training
process

+ uses given threshold information

o (ie. thresholds_dict)

+ compares that to the evaluation results to
determine whether the model is sufficiently
accurate to deploy

21. Step 4: Create component pipelines

Define and create the following pipelines using vertex.ai pre-built components
(only if threshold checks from Step 3 passed then the pipeline would run):

Data
pipeline

+ TabularDatasetCreateOp:
retrieves given dataset source
from either in Cloud Storage
or BigQuery

Model
pipeline

+ AutoMLTabularTrainingJobRunOp:

kicks off an AutoML training job for
a tabular dataset

+ EndpointCreateOp: creates an
endpoint in Vertex Al

Code
pipeline

+ ModelDeployOp: deploys a
given model to an endpoint in
Vertex Al

+ VertexNotificationEmailOp:
Send notification email(s) when
an upstream task completes

22. Overview

(Ingestion, \

validation,

cleaning /

/ Feature Y

engineering,

training,

\ evaluation )
com
— |

monitoring,

logging)

23. Thank you! Question:

lo-victoria.com

© /victoria2666

@ragtechdev