"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Presentation from Flex Logix

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/maximize-your-ai-compatibility-with-flexible-pre-and-post-processing-a-presentation-from-flex-logix/

Jayson Bethurem, Vice President of Marketing and Business Development at Flex Logix, presents the “Maximize Your AI Compatibility with Flexible Pre- and Post-processing” tutorial at the May 2024 Embedded Vision Summit.

At a time when IC fabrication costs are skyrocketing and applications have increased in complexity, it is important to minimize design risks and maximize flexibility. In this presentation, you’ll learn how embedding FPGA technology can solve these problems—expanding your market access by enabling more external interfaces, accelerating your compute envelope and increasing data security.

Embedded FPGA IP is highly efficient for pre- and post-processing data and can implement a variety of signal processing tasks such as image signal processing (defective pixel and color correction, for example), packet processing from network interfaces and signal processing from data converters (filtering). Additionally, this IP can manage data movement in and out of your AI engine as well as provide an adaptable protocol layer to connect to a variety of external interfaces, like USB and MIPI cameras. Flex Logix eFPGA IP is easy to integrate, high performing, lightweight and supported across more process nodes than any other supplier’s.

1. Maximize Your AI
Compatibility with Flexible
Pre-and Post-Processing
Jayson Bethurem
VP, Marketing and Business Development
Flex Logix

2. Changing Economics Chips Need to Evolve & Adapt
2© 2024 Flex Logix
$
Decreasing IC Selection
IC tapeout costs increasing significantly
Forcing IC manufacturers to reduce # of
chips/family, especially impacting smaller ICs
All while design complexity increases and
emerging technologies continually evolve
2018
2024
That provide reprogrammable algorithm
acceleration that meets SWaPGoals
Skyrocketing Fab Costs
Increasing Complexity
Adaptability Required

3. eFPGA Complements AI and Signal Processing
3© 2024 Flex Logix
Processor /
Signal
Processing /
AI Engine
ADC
Physical Layer
Image
Signal
Proc
Packet
Processing
Filtering
Transform
Data Throttling /
Memory Management
Data Throttling /
Memory Management
Physical Layer
Data Flow
Management
EFLX® eFPGAHard IP
DAC
For export, data needs to be
packetized and buffered
Data requires management,
formatting, throttling
Data sources vary and require
different preprocessing
eFPGA Enables Greater Market Applicability and Differentiation

4. Dynamic Nature of Data & Algorithms
4© 2024 Flex Logix
Changing
Protocols
Evolving
Algorithms
Emerging
Threats
Interfaces Specific Protocols
Distinct Applications Demands
Unique Regional Requirements
Inferencing Algorithms
Adaptive Filtering / Kalman
Compression Solutions
Evolving Threats / Aging Cryptography
Digital Signing / Authentication
PQC Preparedness
1G 100G+
IPv4 HTTP/2
USA vs EU
YOLO v1 v5
Scalar Vector
LZW Dictionary
AES128 AES CGM
Hash vs. Matrix
NIST Competitions

5. Embedded FPGA Has Many Use Cases
5
© 2024 Flex Logix
AXI Bus
Processor
Arm /
RISC-V
Memory
eFPGA
Accelerator
AI Engine
AXI Bridge
PCIe
Ethernet
Hi Speed
ADC
Packet
Processing
Security
eFPGA
Crypto
Agility
Signal
Processing
Memory Mgmt.
Virtualization
Image
Processing
Image /
Video
Programmable
I/O & Bridging
Asynch /
Precision Control
Fixed I/O
GPIO
Real-time /
Deterministic
Interface Flexibility –
Adaptable to different
markets and applications
Eliminate unused ports –
minimizing surface attack
exposure
eFPGA
Co-µP
Adaptable Image
Signal processing
for a variety of
sensors
Interface
Adapter
Protect against
evolving threats &
obfuscate secret IP
Accelerate complex
algorithms
Expand Physical
and Virtual
functions
Bridge & DMA data
Programmable
Data Plane
Support Industrial
Networking
Filtering / Transforms
Redundancy for
Functional Safety
Flex Logix
eFPGA IP

6. Programmable Data Planes Enable SmartNIC Security
6© 2024 Flex Logix
llll
Servers mitigate many malicious attacks, such as Distributed Denial of Service, which
require immediate detection & response
eFPGA is the perfect solution for SmartNICs and adaptable Packet Processing
In-line
(real-time)
Processing
Continuous
ability to
adapt to
evolving
threats
Many packet processing
solutions available from
IP Partners like Dynanic
Interface
Flexibility
QDR / HBM/ DDR
Ethernet
MAC
Packet
Parser
Packet
Filter 0
Packet
Filter [n]
Edit /
Forward
Ultra Fast
DMA
TCAM
100G
200G
400G
LPM
Match
Tracking
TCAM
LPM
Match
Tracking
Packet FIFO
…

7. Adaptable Image Signal Processing
7© 2024 Flex Logix
Parallel pixel processing as well
as multiple video channels
Video sources & formats
can by vary by sensor
Resolution, frame rate and color
depth dependent on application
Image Signal Processing requirements can be dynamic for each application
EFLX® eFPGAHard IP
MIPI
USB
LVDS
ENet
Data Buffer & Protocol Converter
(Raw Bayer, RGB & YCbCr)
Clipper
Block
Level
Statistics
Defective
Pixel
Correction
Adaptive
Noise
Reduction
Block
Level
Correction
Vignette /
Warping
Correction
White
Balancing
Demosaic
White
Balancing
Feedback
Adaptive
Image
Sharpening
Histogram
Statistics
RGB
Gain
Auto
Exposure
Gamma
Correction
Image Signal Processing Pipeline
Scaling
Many ISP solutions
available from IP Partners

8. Flex Logix IP Availablefor Programmable Applications
8© 2024 Flex Logix
Flex Logix IP available on advanced nodes
TSMC 5nm and INTEL 18A
EFLX® eFPGA InferX™ DSP & AI

9. •Flex Logix DSP Blocks 22x22-bit signed real multiplier with 48-bit accumulator
•Pre-adder & post-adder can perform 11-and 24-bit complex signed add/sub
•Built in sign-detection logic and local carry chains for cascading into larger computation
Signal Processing with Flex Logix EFLX DSP
9© 2024 Flex Logix
Flex Logix EFLX DSP block 10-tap symmetrical FIR filter using only 5 DSPs

10. •Scalable number of tensor processors
•Tensor processors are configured dynamically
•optimizes throughput and utilization
•easy to adapt to new operators + workloads
•INT16 mode for DSP with INT40 accumulation
•INT8 mode for AI (with INT16 and BF16 options)
•High level programming
•Silicon proven
InferX IP World Class DSP+AI Processing at Lowest $/W
10© 2024 Flex Logix
AXI
Hard IP
Soft IP
Memory IP

11. InferX Scalability to Meet a Wide Range of Workloads
11© 2024 Flex Logix
InferX 128 TPU
Tile with 128 TPUs / 128 TOPS
InferX
Tile with 1 TPU
1 TOP
InferX 16 TPU
Tile w/16 TPUs
16 TOPs
Scalable DSP
Performance from
1 16 128
TPUs

12. InferX–High Performance, Multi-Operations, Low Latency
12© 2024 Flex Logix
1GHz
Operation
µInferX 1 InferX8 InferX
Complex INT16
1K/2K/4K FFT
500 MS/s
(Megasamples/sec)
8.5 GS/s
(Gigasamples/sec)
68 GS/s
(Gigasamples/sec)
Real INT16x16
FIR 256 taps
0.25 GS/s 4 GS/s 32 GS/s
Real INT16x16
FIR
4096 taps
16 MS/s 0.25 GS/s 2 GS/s
32x32 Complex
INT16 Matrix
Inversion
10K/sec 0.2M/sec 2.6M/sec
Area
(rough est.)
~0.5 mm
2
~3.6 mm
2
~20 mm
2
DSP
1GHz
Operation
2 µInferX1 InferX8 InferX
YOLOv5s
(640x640)
32 IPS 260 IPS 1400 IPS
YOLOv5L6
(1280x1280)
2 IPS 16 IPS 130 IPS
DETR 2020
Transformer
(1024x1024)
3 IPS 26 IPS 195 IPS
Area
(rough est.)
~0.6 mm
2
~2.5 mm
2
~20 mm
2
LPDDR5 1 1 4
Vision AI
Area and performance benchmarks based on TSMC N5/N4 Nodes

13. •Accelerates crypto algorithms w/ parallel processing in eFPGA
•Multiplex cryptography algorithms –Saving die area
•Enhance digital signatures with additive solutions
•Protect “secret” or critical IP with obfuscation
•Mitigate ITAR concerns by adapting to regional requirements
eFPGA Solutions Hold the Key to Security
13© 2024 Flex Logix
Cryptography solutions must be agile as decisions made today will be challenged
Adaptive
Security
Discrete FPGA Problems
Cloning, Overbuilding,
Side Channel Attacks,
Spoofing, Zero Trust (inc.
supply chain), Bitstream
Interception/Mutable
Minimize Attack Surface
Flexible interface
capability eliminates
unused access ports
Algorithm Acceleration
Many crypto algorithms
can be accelerated in
programmable logic
RNGs, PUF generation,
KDF, ECC
Secure Solutions
Proven IP solutions from
partners
Xiphera, Synopsys, CAST
Flex Logix Solution

14. •IP fits compactly into 2x2 EFLX4K Tile Array
with > 90% utilization
•10K LUTs | 5K Registers | 8 BRAMs
•Can be implemented on any ASIC or SoC on any
technology node
•Fast 225 MHz clock rate @ 64b 1.8 GB/s
•Typical, TSMC 7nm
PQC KYBER Encryption Module Flex Logix EFLX IP
14© 2024 Flex Logix
moduleKEM (input
a,b,outputs,
c);
assigns =a^b;
assignc =a &b;
endmodule
eXpreso
Compiler
Flex Logix EFLX 4K Tiles
ML-KEM Implementation Example

15. Secure Critical IP and Algorithms with Obfuscation
15
© 2024 Flex Logix
Memory Protection
DMA Isolation| Dynamic Virtual Memory
Management |Encryption
Algorithm Confidentiality
Cryptography |Signal Processing IP|
Proxy Re-Encryption|Homomorphic Encryption
Interface Obscurity
Protocol Processing|Encoding & Decoding |
Encryption |Packet Filtering|Authentication
(signing)
Reprogrammable IP in critical interfaces,
algorithms and cryptography becomes the
keystone to device security & operation
Ultra Compact
EFLX 200 TileTile
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙
∙
∙
∙
L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙
∙
∙
∙
L
L
L L MY
L L MY
∙
∙
∙
∙
L
L
L
L
DSP
L MY
L MY
L MY
L MY
L MY
L MY
∙
∙
∙
∙
L
L
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L
L
LL MY
LL MY
L
L
L
L
L MY
L MY
L MY
L MY
L MY
L MY
L
L
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L
L
LL MY
LL MY
L
L
L
L
L MY
L MY
L MY
L MY
L MY
L MY
L
L
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L
L
LL MY
LL MY
L
L
L
L
L MY
L MY
L MY
L MY
L MY
L MY
L
L
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L
L
LL MY
LL MY
L
L
L
L
L MY
L MY
L MY
L MY
L MY
L MY
L
L
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

0
6
30
22
0 7 13 19 25 31
∙

∙

∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
Tile
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙
∙
∙
∙
L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

M MY
∙
∙
∙
∙
L
L
M MY
M
M
L L MY
L L MY
∙
∙
∙
∙
L
L
L
L
M
M
L MY
L MY
L MY
L MY
L MY
L MY
M MY
∙
∙
∙
∙
L
L
M MY
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

M MY
L
L
M MY
M
M
LL MY
LL MY
L
L
L
L
M
M
L MY
L MY
L MY
L MY
L MY
L MY
M MY
L
L
M MY
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

M MY
L
L
M MY
M
M
LL MY
LL MY
L
L
L
L
M
M
L MY
L MY
L MY
L MY
L MY
L MY
M MY
L
L
M MY
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

M MY
L
L
M MY
M
M
LL MY
LL MY
L
L
L
L
M
M
L MY
L MY
L MY
L MY
L MY
L MY
M MY
L
L
M MY
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

L MYL
L MYL
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

M MY
L
L
M MY
M
M
LL MY
LL MY
L
L
L
L
M
M
L MY
L MY
L MY
L MY
L MY
L MY
M MY
L
L
M MY
∙

∙

∙

∙

∙

∙

∙

∙

∙

∙

0
6
30
22
0 7 13 19 25 31
∙

∙

∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
∙
DSP DSP DSP DSP
DSP
∙
∙
∙
∙
∙
∙
∙
∙
DSP DSP DSP DSP
Add & update secrets at your
controlled facility

16. Hybrid Solution = ASIC + FPGA = Best Solution
16© 2024 Flex Logix
Solutions today are implemented in either FPGAs or ASICs/SoCs
FPGA ASIC HYBRID
CONS
PROS
High Cost & Power
Supply Chain
Cloneable/Mutable
Side Channel Attacks
Hard to Design
Flexible, Adaptable
Lacks any HW
adaptability or flexibility
High Design Risk
Only SW upgrades
Highest Performance
Lowest Cost & Power
RTL Design Knowledge
for highest
performance
Adaptable Hardware
High Performance
Low Cost & Power
Lowers Design Risk
, but Hybrid offers Best of Both

17. Improve Your ASIC
17© 2024 Flex Logix
Evolving Protocols
Extend product life by adapting to new
interfaces and protocols and supporting
changing workloads
Periodic Bug Fixes
Enable updates to fix pesky bugs
Enable Differentiation
Flexibility and adaptability to
enable unique features vs competition
Regional Requirements
Meet regional specific protocol
and security requirements
Lasting Security
Adapt to evolving security
algorithms and threats
Lifecycle Test & Debug
Built in Logic Analyzer w/ run-time debug,
bring up analysis , RMA analysis
Algorithm Improvement
Many IP continuously improve such as AI
and data plane processing IP
Save Money
FPGA Integration can reduce mask
spins and save engineering cost by
moving risky IP to programmable logic

18. Summary
18© 2024 Flex Logix
Flex Logix eFPGA is the most proven eFPGA Technology on the market
With over 25 working silicon designs
Best PPA in the industry
Best EDA tool chain with Synplify + eXpreso compiler
Providing the best design and customer experience
Proven high-volume deployment with major IC manufacturers
IP available and proven in more fab nodes than any other supplier
TSMC and Intel IP Alliance Partners
Silicon proven solutions for packet processing, security and SW acceleration

19. •See us at booth #310 to see demos in action
•Visit Flex Logix Website @ https://flex-logix.com
•Contact our Sales team: [email protected]
•Test drive your IP with EFLX Compiler
•Set up a Tools Demo
•A free 45-day eval system is available for your use after the demo session
•Get Performance, Power and Area estimates
Learn More and Next Steps
19© 2024 Flex Logix
See for yourself why Flex Logix is #1 for PPA