tiny Machine Learning and Data Centric Approach

A presentation about tiny Machine Learning and Data Centric Approach

1. ICITDA -2021 Owais A. Malik
TinyMLand Importance of Data-
Centric Approach
Owais Ahmed Malik
UniversitiBrunei Darussalam

2. ICITDA –2021 Owais A. Malik
TinyML
Data-
Centric AI
Applications, Opportunities and Challenges

3. ICITDA –2021 Owais A. Malik
*HarvardX
TinyML

4. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future

5. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future
Local Machines/Servers

6. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future
Cloud Computing
Inference
Learning
Raw Data
Output
Local Machines/Servers

7. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future
Cloud Computing
Inference
Learning
Raw Data
Output
Edge-Computing / On-device Analytics
Inference
Learning
Data
Model
Local Machines/Servers

8. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future
Cloud Computing
Inference
Learning
Raw Data
Output
Edge-Computing / On-device Analytics
Inference
Learning
Data
Model
Local Machines/Servers

9. ICITDA –2021 Owais A. Malik
Knowledge-base
C3L
Optimization
TinyML
AI Computing –Past –Present -Future
InferenceLearning
Information
Future!!!

10. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices

11. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Embedded
System
(ES/MC)
Machine
Learning
TinyML

12. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Embedded
System
(ES/MC)
Machine
Learning
TinyML
Embedded
System
(ES/MC)
Limited
Memory
Limited
Computing
Limited
flexibility/
specialized
Very small in
size
Long-life
battery
Pervasive

13. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
1 MB of flash memory and 256 kB of RAM
Arduino® Nano 33 BLE Sense

14. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices

15. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Collect
Data
Pre-Process
Data
Design
Model
Train
Model

16. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Collect
Data
Pre-Process
Data
Design
Model
Train
Model
Evaluate
Model
Convert
Model
Optimize
Model
Deploy
Model at
Edge
Make
Inference
at Edge

17. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Collect
Data
Pre-Process
Data
Design
Model
Train
Model
Evaluate
Model
Convert
Model
Optimize
Model
Deploy
Model at
Edge
Make
Inference
at Edge
Data Engineering Model Engineering
Model Deployment Product Analytics

18. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices

19. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Tensorflow
TensorflowLite TensorflowLiteMicro

20. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Model
Optimization
In MBs
In ~100s of
KB
In ~10s of
KB

21. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Model
Optimization
Post Training During Training
Pruning
Knowledge Distillation
Quantization

22. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Quantization
Quantization is an optimization that
works by reducing the precision of the
numbers that are used to represent a
model's parameters.

23. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Quantization
Smaller Model
Size
Faster
Computation &
Reduce Latency
Better
Portability
Compromise in accuracy

24. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
Quantization
Post Training
Quantization
(PTQ)
Quantization
Aware Training
(QAT)
Weight
Compression
(size)
Inference
Calculation
(latency)
Inject
Quantization
during Training

25. ICITDA -2021 Owais A. Malik
TinyML
Issues with Resource Constrained Devices
TinyML
Pre-
processing
Always-on
system
Larger NN
On when
required
Cloud
processing
Large
Models
Cascade Architecture –Multistage Model
•MitigateEnergy
Consumption
Problem
•SuitableofReal-
timeSystems

26. ICITDA -2021 Owais A. Malik
TinyML
Example Applications

27. ICITDA -2021 Owais A. Malik
TinyML
Example Applications

28. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future
Knowledge-base
C3L
Optimization
InferenceLearning
Model
Information
Future!!!
Essential
Data

29. ICITDA –2021 Owais A. Malik
TinyML
AI Computing –Past –Present -Future
Knowledge-base
C3L
Optimization
InferenceLearning
Model
Information
Future!!!
Essential
Data
•C3L–continuouslifelonglearningapproach
foredgedevices
•Bioinspiredlearning
•Federatedlearning
•Self-supervisedlearning(SSL)

30. ICITDA –2021 Owais A. Malik
TinyML
Challenges
•Communication –with other devices, with base stations
•Security, protection of data –hacking of devices, false alarm generation,
personal data accessibility
•Quality data collection
•Quality of experience
•Digital footprints –leaving digital debris, affecting Green AI

31. ICITDA -2021 Owais A. Malik
Data-Centric AI
•Paradigm shift
•Model-centric to Data-centric AI
AI is driven by
data –not code
Training data is
the new newoil
Models are
becoming a
commodity
Data Programming
http://ai.stanford.edu/blog/data-centric-ai-retrospective/

32. ICITDA -2021 Owais A. Malik
Data-Centric AI
Data
Model 1
Model 2
Model k
Compare
Performance
Reference
Model-centric vs Data-centric

33. ICITDA -2021 Owais A. Malik
Data-Centric AI

34. ICITDA -2021 Owais A. Malik
Data-Centric AI
Reference
Need for a Systematic Approach
https://www.deeplearning.ai/wp-content/uploads/2021/06/MLOps-From-Model-centric-to-Data-centric-AI.pdf

35. ICITDA -2021 Owais A. Malik
Data-Centric AI
•More data is always equivalent to better data???
Volume
•Sufficient amount of relevant data
•Error analysis of the current model before adding more data
Consistency
•Requirement for consistently labelled dataset
•Careful design of annotation/labeling instructions
Quality
•Cover all variations possibly present in deployment data
Characteristics of Data

36. ICITDA -2021 Owais A. Malik
Data-Centric AI
Modify the
erroneous
ones in the
current
dataset
If not,
collect more
data or
synthetically
create those
data points
Check
training
data has
enough
samples to
learn for
such cases
Check for
corner cases
Pick the
instances
where best
model is
failing to
perform
Pick the
best-
performing
model
Understand
how data
behaves
across
candidate
models
Find a
baseline
model
Practical Steps

37. ICITDA -2021 Owais A. Malik
Data-Centric AI Challenges
•Noestablishedframeworkformakingdatasetsthatareaccessiblefor
machinelearningmodels-SnorkelAIanexample
•Availabilityoflargeamountsofunstructuredheterogenousdata
•Requirementforhighlevelofknowledge,tediousmanualworkand
proficientcodingskillstoturnunstructureddataintoMLcode
friendlydata

38. ICITDA –2021 Owais A. Malik
TinyMLand Data-Centric AI
Problem–Design a tinyMLbased ECG classification that can be deployed as a wearable device.

39. ICITDA –2021 Owais A. Malik
TinyMLMeets Data-Centric AI

40. ICITDA -2021 Owais A. Malik
TinyMLand Data-Centric AI
Frantz Bouchereau-MathWorks

41. ICITDA -2021 Owais A. Malik
TinyMLand Data-Centric AI
•An energy efficient system is required
•Data-centric approach to reduce the transfer of data to the cloud
•Research on C3L for embedded device on-going

42. ICITDA –2021 Owais A. Malik

43. ICITDA –2021 Owais A. Malik

44. ICITDA -2021 Owais A. Malik
References/Acknowledgments
•HarvardX–Applications of TinyMLCourse
•http://ai.stanford.edu/blog/data-centric-ai-retrospective/
•https://www.deeplearning.ai/wp-content/uploads/2021/06/MLOps-From-Model-centric-to-Data-
centric-AI.pdf
•https://towardsdatascience.com/tiny-machine-learning-the-next-ai-revolution-495c26463868
•https://elucidata.io/getting-started-with-data-centric-ai-development-tips-from-andrew-ng/
•https://www.embeddedonlineconference.com/speaker/Frantz_Bouchereau