“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using the Latest FPGAs and DRAMs,” a Presentation from Lattice Semiconductor and Etron Technology America

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/squeezing-the-last-milliwatt-and-cubic-millimeter-from-smart-cameras-using-the-latest-fpgas-and-drams-a-presentation-from-lattice-semiconductor-and-etron-technology-america/

Hussein Osman, Segment Marketing Director at Lattice Semiconductor, and Richard Crisp, Vice President and Chief Scientist at Etron Technology America, co-presents the “Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using the Latest FPGAs and DRAMs” tutorial at the May 2024 Embedded Vision Summit.

Attaining the lowest power, size and cost for a smart camera requires carefully matching the hardware to the actual application requirements. General-purpose media processors may appear attractive and easy to use, but often include unneeded features which increase system size, weight, power and cost. “Right-sizing” the camera design for the application requirements can save significant power, cost, size and weight.

In this talk, Osman and Crisp show how you can leverage an advanced power-optimized FPGA incorporating a soft RISC-V core combined with a video-bandwidth, low-pin-count DRAM to cut power consumption roughly in half for endpoint smart cameras used in automotive, industrial and other applications. They examine techniques for reducing power, cost and size including system architecture, memory architecture, packaging, and signaling and termination schemes. They also explore techniques for enhancing system reliability.

1. Squeezing the Last Milliwatt and
Cubic Millimeter from Smart Cameras
Using the Latest FPGAs and DRAMs
Hussein Osman
Segment Marketing Director
Lattice Semiconductor
Richard Crisp
VP & CTO, EtronTechnology America

2. Why FPGA for Edge AI
Accelerating Innovation in Low Power Applications
SCALABLE PERFORMANCE
Multiple use cases in parallel or serial
SECURE
Secure device configuration
HARDWARE PROGRAMMABLE
Adapts to fast changing machine
learning algorithms
FLEXIBLE COMPUTATION RESOURCES
Pre and post processing
ISP, FFT and filtering
ULTRA-LOW POWER
1 milliwatt –1 watt
© 2024 Lattice Semiconductor & Etron Technology America 2

3. FPGAs Speed / Power Optimized AI Innovation
Accelerating Innovation in Low Power Applications
Hardware
Optimization
Algorithm
Optimization
Continuous
Improvement
New Use Cases
& >
AI models are rapidly evolving
© 2024 Lattice Semiconductor & Etron Technology America 3
Source: BENMEZIANE et al.: A COMPREHENSIVE SURVEY ON HARDWARE-AWARE NEURAL ARCHITECTURE SEARCH

4. Edge AI Camera Architectural Options
Accelerating Innovation in Low Power Applications
© 2024 Lattice Semiconductor & Etron Technology America 4

5. Power Efficient FPGA Inferencing Resources
Accelerating Innovation in Low Power Applications
RPC
DRAM®
DDR
© 2024 Lattice Semiconductor & Etron Technology America 5

6. Scalable Efficient CNN Acceleration Engine
Accelerating Innovation in Low Power Applications
Compact, Optimized
or Extended CNN
AXI4 or FIFO interface
Layer Support
•convolution
•max pooling
•global average
pooling layer
•batch normalization
fully connected
© 2024 Lattice Semiconductor & Etron Technology America 6

7. Scalable CNN Acceleration Engine
Accelerating Innovation in Low Power Applications
Performance
Throughput
milliwatts
10s milliwatts
100s milliwatts
0.5-2 watts
© 2024 Lattice Semiconductor & Etron Technology America 7

8. Object Detection and Counting
© 2024 Lattice Semiconductor & Etron Technology America 8
•Accelerated, low-power human presence detection and counting using neural network model
•VGG, MobileNetv1, MobileNetv2, ResNet, and SSD type structures are supported
•TF Lite based implementation for ease of use
•Reference designs are provided to enable design replication and transfer learning
•Total power consumption of less than 200 mW
•Processing at up to 60 FPS and VGA resolution

9. Avoid Overprovisioning Memory:
Look for Opportunity for Size, Weight, Power & Cost Savings
© 2024 Lattice Semiconductor & Etron Technology America 9
RPC
DRAM
9 x 13 mm
48-52 IOs
2 x 4.7 mm
24 IOs
DDR3 DRAM
96 balls @ 0.8 mm pitch
50 balls @ 0.4 mm pitch
“RPC DRAM: Less than half the I/Os
with < 1/10 the footprint”
Right Sized
Memory/FPGA
In CSPs
Conventional DDR & FPGA
in BGA
(Overkill)
Memory Bandwidth
Application Driven
Minimum
Bandwidth/Capacity
Requirement
Memory Burst Read/Write Current
Extra Bits and Excess Memory Bandwidth
increase cost, power dissipation, and WLCSP
memory PCB footprint
“With same # of FPGA I/Os, RPC DRAM can
provide twice the bandwidth of DDR3 at same
clock frequency or the same bandwidth but at
half the clock frequency”

10. Power Savings from Series Termination For Memory Bus:
Applying Basic E.E. Transmission Line Principles
© 2024 Lattice Semiconductor & Etron Technology America 10
Transmission Line: 50 ohms
50 ohm
resistor Vterm=
0.5* VddQ
Receivers
Parallel Termination
Resistor
Termination
CurrentTermination Current = 0.5* Signal Swing / Rterm= (0.4 to
0.5 V)/50 ohms = 9 -10 mA
For a DDR3 two rank system there are 54 terminated signals
= 486 -540 mA termination current!
Conventional Parallel Termination Scheme
(current always flows, including DC)
Driver (Zout<< 50 ohms)
TL: 50 ohms
(incident wave
switched: flexible
receiver
placement)
~900 mV –1 V signal swing
Transmission Line: 50 ohms
Receivers
Driver (Zout= 50 ohms)
TL: 50 ohms (reflected wave
switched: receivers
should be close
together)
Series Termination Scheme
(Only switching current flows, NO DC)

11. Two Rank Configuration SI Comparison:
@ Lower frequencies (like soft FPGA I/F): series termination works fine & saves significant power
© 2024 Lattice Semiconductor & Etron Technology America 11
400 mV
180 ps
700 mV
600 ps
Termination
Current
DQ/DQS
At CTLR
Far DRAM
Drives Bus
Series terminated: 2 Rank RPC w/Soft FPGA I/F @ DDR1066Parallel terminated: 2 Rank DDR3 w/Hard ASIC I/F @ DDR2133
540 mA
2.7 uA
DQ/DQS
At CTLR
Near DRAM
Drives Bus
350 mV
180 ps
700 mV
600 ps
DQ/DQS
At CTLR
Far DRAM
Drives Bus
DQ/DQS
At CTLR
Near DRAM
Drives Bus
Termination
Current
Series Term @ 533 MHz vs Parallel Termination @ 1066 MHz

12. Component Availability and System Reliability
© 2024 Lattice Semiconductor & Etron Technology America 12
28 nm FD-SOI low power; 100x lower SER
FuSacertified FPGA design tools
RISC-V with Green Hillsµ-velOSity
ECC protected memory
AEC Q100 Level 2 qualified FPGAs and memories
Extensive tier 1 automotive environmental testing
DRAM Component Reliability Reports
RPC DRAM and Lattice CertusPro™-NX FPGA components are available NOW in volume
(including from DigiKey)

13. Key Takeaways
© 2024 Lattice Semiconductor & Etron Technology America 13
For minimum Size, Weight, Power and Cost:
•FPGAs offer parallel processing and adaptability suitable for rapidly evolving AI use cases
•Optimizing and tuning edge AI models and image signal processing to reduce complexity while still
meeting application needs is key for reducing power and cost
•Avoid Overprovisioning Compute Horsepower and Memory: cut it back to minimum requirements with a
small margin: no extra credit for unused excess capability vs application requirements
•Using CHIP SCALE packages can enable significant power savings by using Series Bus Termination due to
faster bus settling times vs BGA.
•Parallel termination –power 3.6 W
•Series termination –power 840 mW-> 2.76 W savings!

14. Resources: Find Out More, Where to Buy
© 2024 Lattice Semiconductor & Etron Technology America 14
Buy RPC DRAM from DIGIKEY
https://www.digikey.com/en/products/detail/etron-
technology-inc/EM6GA16LCAEA-12H/13169828
For EtronRPC Design Info
https://etron.com/innovative-dram-pl/rpc-dram//
CrossLink™-NX CertusPro™-NX
CrossLink™-NX
Embedded
Vision Processing
CertusPro™-NX
Advanced
General Purpose
Processing

15. 15© 2024 Lattice Semiconductor & Etron Technology America
Thank You