“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/07/how-to-run-audio-and-vision-ai-algorithms-at-ultra-low-power-a-presentation-from-synaptics/

Deepak Mital, Senior Director of Architectures at Synaptics, presents the “How to Run Audio and Vision AI Algorithms at Ultra-low Power” tutorial at the May 2024 Embedded Vision Summit.

Running AI algorithms on battery-powered, low-cost devices requires a different approach to designing hardware and software. The power requirements are stringent at standby, but the device needs to be able to awaken quickly when an event is detected. The device needs to “pseudo” wake up, determine if the event needs attention, and then either go back to standby or become active to classify the event.

This multistage wake-up process and the associated intelligence requires tight orchestration of hardware and software. Apart from runtime software, the AI models must be highly optimized to fit and run on the constrained device. To show how this can be done, Mital presents a solution that combines hardware, software and AI models to enable running audio and video AI algorithms at ultra-low power.

1. How to RunAudio and
Vision AI Algorithms at
Ultra-Low Power
Presenter:
Deepak Mital
Sr. Director, Architecture
Synaptics Incorporated

2. •Many IoT applications do not require “continuous maximum” compute
•Continuous monitoring results in battery drain
•Examples:
•Security camera: Turn on main processing for actual detection only when confirmed
necessary
•Human presence detection (HPD) and identification to turn device on: Run HPD
detection and identification algorithm only when detected “potential” presence
•Predictive maintenance: Enable advanced detection only when initial metrics are met
•Shoplift prevention: Enable detailed analytics only when “potential” threat detected
Problem statement
2© 2024 SynapticsInc

3. •Multistage hardware: Capable of running
Audio and Video AI algorithms
•Highly efficient AI models with different KPIs
for each stage
•Tight orchestration of software to invoke each
stage
Solution
3© 2024 Synaptics Inc
Always-on domain
High performance
High efficiency
Power management
System memories
Security
USB / serial / MIPI
U55 NPUCortex-M55
μNPUCortex-M4
Vision AI
pipeline
JPEG
Audio
VAD
ISP, encoders
Sensing logic
Deep sleep: GPIO (Wake), internal clock
Reset

4. •Ultra-low power: Microwatts hardware,
always on
•Sound detection
•Image change detection
•Critical model requirements are for very
fewfalse negatives
•False negatives will render device
unresponsive
Solution –Stage 1
4© 2024 SynapticsInc
Always-on domain
High performance
High efficiency
Power management
System memories
Security
USB / serial / MIPI
U55 NPUCortex-M55
μNPUCortex-M4
Vision AI
pipeline
JPEG
Audio
VAD
ISP, encoders
Sensing logic
Deep sleep: GPIO (Wake), internal clock
Reset

5. •Mid-to low power –10s of microwatts
hardware, activated by stage 1 via software
•AI algorithms (example):
•Wake-word detection
•Human presence detection
•Critical model requirements are for very
fewfalse negatives and false positives
•False negatives will render device
unresponsive
•False positives will increase power
consumption
Solution –Stage 2
5© 2024 SynapticsInc
Always-on domain
High performance
High efficiency
Power management
System memories
Security
USB / serial / MIPI
U55 NPUCortex-M55
μNPUCortex-M4
Vision AI
pipeline
JPEG
Audio
VAD
ISP, encoders
Sensing logic
Deep sleep: GPIO (Wake), internal clock
Reset

6. •High performance, activated by Stage 2 via
software
•AI algorithms (example):
•Person identification
•Object detection
•Critical model requirements are forvery high
performance at low power
•Slow run times will increase power
consumption
Solution –Stage 3
6© 2024 SynapticsInc
Always-on domain
High performance
High efficiency
Power management
System memories
Security
USB / serial / MIPI
U55 NPUCortex-M55
μNPUCortex-M4
Vision Ai
pipeline
JPEG
Audio
VAD
ISP, encoders
Sensing logic
Deep sleep: GPIO (Wake), internal clock
Reset

7. •Different requirements for AI models at each stage
•Need AI models optimized for different KPIs:accuracy, performance,and size
•NAS-basedmodel generation architecture where the models are purpose built for the
constrained silicon
•Primary factors affecting inference KPI
•Model architecture design
•Model quantization
•Approach: Jointly optimize model architecture and quantization under memory
constraints
AI models
7© 2024 SynapticsInc

8. •Resolution –[28x28 –32x32]
•Kernel size –[3x3, 5x5, 7x7]
•Depth –[2, 3, 4]
•Width (channel expansion factor) –[2, 3, 4]
•Mixed-precision quantization parameters –
[4 bit, 6 bit, 8 bit]
Multi-precision NAS search range for classification
8© 2024 SynapticsInc

9. CIFAR-10 classification –Mixed vs 8-or 4-bit precision
9© 2024 SynapticsInc

10. CIFAR-10 classification comparison
10© 2024 SynapticsInc

11. •Resolution –[320x240 –
640x480]
•Kernel size –[3x3, 5x5, 7x7]
•Depth –[2, 3, 4]
•Width (channel expansion
factor) –[2, 3, 4]
•Mixed-precisionquantization
parameters –[4 bit, 6 bit, 8bit]
Object detection dataset
11© 2024 SynapticsInc

12. COCO person detection –Mixed vs 8-or 4-bit precision
12© 2024 SynapticsInc

13. COCO person detection comparison
13© 2024 SynapticsInc

14. •Model development stage KPI:
•COCO Instance Mask mAP: 0.636
•Latency:92.19 ms
•Resolution:480x640 (VGA)
•Weights: 1.57 M parameters
•Model run on hardware:
•Inference time: 96 ms
•Total frame time: 120 ms
Segmentation run on Stage 3
14© 2024 SynapticsInc

15. •Building full applications running at ultra-low power requires high levels of integration
ofhardware and software
•Multiple levels of processing is needed to wake up silicon components as needed
•Stage 2 and Stage 3 come out of deep sleep based on results from previous stage
•The low-power orchestration demands tight software integration
•Each stage requires AI models with different KPIs on accuracy, model size, and speed
•Need to have NAS-based model generation/training software to enable the complete
solution
•Solution enables battery-powered devices that are AI capable and can run for many
months/years
Summary
15© 2024 Synaptics Inc

16. Resources
16
Synaptics Astra embedded processors
https://www.synaptics.com/products/embedded-processors
Synaptics Astra evaluation Kit
https://synacsm.atlassian.net/servicedesk/customer/portal/543/grou
p/563/create/6387
Synaptics Astra software
https://github.com/synaptics-astra
© 2024 Synaptics Inc