“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation from Network Optix

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/

Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.

In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.

van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications. He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.

1. Building and Scaling AI
Applications with the NxAI
Manager
Robin van Emden
Senior Director of Data Science
Network Optix

2. 2© 2024 Network Optix
Network Optix
●Founded in 2010 with the aim of
creating a platform for enterprise
video management.
●Offices in Los Angeles, Portland,
London, Taipei, Adelaide, Belgrade.
●Over 4 million connected video
streaming devices in more than 150
countries
●Recently acquired Scailable AI,
which adds Amsterdam 

3. 3© 2024 Network Optix
Network Optix
●Founded in 2010 with the aim of
creating a platform for enterprise
video management.
●Offices in Los Angeles, Portland,
London, Taipei, Adelaide, Belgrade.
●Over 4 million connected video
streaming devices in more than 150
countries
●Recently acquired Scailable AI,
which adds Amsterdam 

4. 4© 2024 Network Optix
Hard Problem: Video

5. Nx Toolkit Makes Video Solutions at Scale Easy
5© 2024 Network Optix
Network Optix Toolkit is a solution that handles the complexities of
edge video, allowing you to concentrate on building your application.
Device / Stream
Discovery
Storage
Management
Decoding /
Visualization
System
Scalability
Enterprise
Scalability
Cyber
Security
Video /Metadata
Capture
OS /Hardware
Support
Platform
Extensibility
Usability /User
Experience

6. Nx Toolkit Makes Video Solutions at Scale Easy
6© 2024 Network Optix
Network Optix Toolkit offers partially open-source
tools and libraries to build custom video solutions.
It provides server, desktop, mobile applications, and
cloud services.

7. Nx Toolkit Makes Video Solutions at Scale Easy
7© 2024 Network Optix
Network Optix Toolkit offers partially open-source
tools and libraries to build custom video solutions.
It provides server, desktop, mobile applications, and
cloud services.

8. 8© 2024 Network Optix
Hard Problem: AI

9. Creating AIapplications at Scale Is Hard
9© 2024 Network Optix
These days, there are numerous user-
friendly tools available for training AI
models…
Data scientists use their preferred tools to
train AI models on laptops, in data centers,
and in the cloud.
... but building an AI solution
for edge devices still takes a lot of time and
effort.
•Diverse edge AI accelerators: GPU, NPU, DPU.
•Input and output interfacing, edge video.
•Data pre-and post-processing.
•Embedded engineering.

10. Nx AI Manager Makes AI Solutions at Scale Easy
10© 2024 Network Optix
Train AI model
Upload to
Nx Cloud
Deploy AI model to
Device(s) with
Nx AI Manager
Nx Client,
Nx Mobile,
Nx Web
Nx Meta Server
Nx AI manager

11. Nx AI Manager Makes AI Solutions at Scale Easy
11© 2024 Network Optix
Train AI model
Upload to
Nx Cloud
Deploy AI model to
Device(s) with
Nx AI Manager
Nx Client,
Nx Mobile,
Nx Web
Nx Meta Server
Nx AI manager

12. Nx AI Manager Makes AI Solutions at Scale Easy
12© 2024 Network Optix
Train AI model
Upload to
Nx Cloud
Deploy AI model to
Device(s) with
Nx AI Manager
Nx Client,
Nx Mobile,
Nx Web
Nx Meta Server
Nx AI manager

13. Nx AI Manager Makes AI Solutions at Scale Easy
13© 2024 Network Optix
Train AI model
Upload to
Nx Cloud
Deploy AI model to
Device(s) with
Nx AI Manager
Nx Client,
Nx Mobile,
Nx Web
Nx Meta Server
Nx AI manager

14. Nx AI Manager -> Open Ai Accelerator eXchange
[OAAX]
14© 2024 Network Optix
NX Cloud
AI Model
Inference Engine
Using OAAX you can use
any hardware accelerator
effortlessly to deploy your
edge AI models.
Convert models in cloud
Standardized interface
to
native accelerator
library
https://github.com/OAAX-standard/OAAX

15. Nx AI Manager -> Open Ai Accelerator eXchange
[OAAX]
15© 2024 Network Optix
NX Cloud
AI Model
Inference Engine
Using OAAX you can use
any hardware accelerator
effortlessly to deploy your
edge AI models.
Convert models in cloud
Standardized interface
to
native accelerator
library
https://github.com/OAAX-standard/OAAX

16. Nx AI Manager Allows for Custom Pipelines
16© 2024 Network Optix
Preprocessing Inference EnginePostprocessing
https://github.com/scailable/sclbl-integration-sdk
Custom postprocessors Preprocessors Custom inference engine
https://nx.docs.scailable.net/oaax-implementation

17. 17© 2024 Network Optix
Deploying AI Models to Heterogeneous Devices

18. Step 1: Edge Devices, IP Cameras, and a Client Device
18© 2024 Network Optix
CPU AARCH64 X86_64 AARCH64 X86_64
XPU
ACCELERATOR
CPU SIMD CPU SIMD NVIDIA ORIN HAILO-8

19. Step 2: Install an Nx Client and Register in the Nx Cloud
19© 2024 Network Optix
> sudoapt install -y ./metavms-client-6.x.x.x-linux_x.deb
(1) https://meta.nxvms.com/download
(2) https://meta.nxvms.com

20. Step 3: Install Nx Server withNX AI Manager
20© 2024 Network Optix
> sudoapt install -y ./metavms-server-6.x.x.x-linux_x.deb
(1) https://meta.nxvms.com/download
CPU AARCH64 X86_64 AARCH64 X86_64
XPU
ACCELERATOR
CPU SIMD CPU SIMD NVIDIA ORIN HAILO-8
INFERENCE
RUNTIME
LIBRARY
ONNXRuntime
with ACL
ONNXRuntime
with oneDNN,
Intel OpenVINO
NVIDIA CUDA
NVIDIA TensorRT
HAILORT

21. Step 4: Train, Convert, Deploy and Run an AI Model
21© 2024 Network Optix
For instance:
https://nx.docs.scailable.net/for-data-scientists/importing-models/from-teachable-machine

22. Step 4: Train, Convert, Deploy and Run an AI Model
22© 2024 Network Optix
Nx AI Cloud
https://admin.sclbl.nxvms.com/

23. Step 4: Train, Convert, Deployand Run an AI Model
23© 2024 Network Optix
Nx AI Cloud
https://admin.sclbl.nxvms.com/

24. Step 4: Train, Convert, Deploy and Runan AI Model
24© 2024 Network Optix
Nx Client
https://github.com/networkoptix

25. 25© 2024 Network Optix
Let Me Show You How: Live Coding Session

26. Step 4: Live Coding Sessions at Our Booth
26© 2024 Network Optix
Booth #304

27. 27© 2024 Network Optix
Takeaway

28. 28© 2024 Network Optix
Building an AI + Video Application? Build on Nx EVOS
•Simplified AI Deployment and Management: Network Optix streamlines the
implementation, deployment, and management of AI models across edge devices,
allowing developers to focus on creating innovative Vision AI solutions.
•Comprehensive Nx Meta Platform: Nx Meta offers a full suite of tools through the Nx
Toolkit, addressing video development challenges, empowering efficient and effective
distributed video application development.
•Versatile AI Model Integration: The Nx AI Managersupports the entire AI model
lifecycle, from training to real-time inference enabling seamless integration of custom
AI models and real-time visualization within the Nx Client.

29. Thank you!
2024 Embedded Vision Summit
Scaling Vision-Based Edge AI Solutions: From
Prototype to Global Deployment
Maurits Kaptein
General Session —Wed, May 22 at 10:00 am
Nx EVOS: A New Enterprise Operating System for
Video and Visual AI
Nathan Wheeler
Exhibit Hall—ET-2Thur, May 23 at 12:00 pm
Visit us at our booth —Booth #304
29© 2024 Network Optix
Continue exploring Nx Meta and more
Nx AI Plugin Documentation
https://nx.docs.scailable.net/
Nx Meta Developer Portal
https://meta.nxvms.com/
Nx tools:
https://www.networkoptix.com/nx-meta
OAAXgithub
https://github.com/oaax-standard