TDC - The Developers Conference - Creating GenAI Apps in Java with SD4J and the ONNX Runtime

TDC - The Developers Conference - Creating GenAI Apps in Java with SD4J and the ONNX Runtime

1. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Criando Aplicativos GenAI
em Java com SD4J e
ONXX
Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist
12 de Junho de 2024
CentroSul –Florianópolis, SC, Brasil
TDC FLORIPA

2. Juarez Barbosa Junior
Senior Principal Java Developer Evangelist
•Coming from Dublin, Ireland
•Developer since 1995 (Java ☕️1997)
•28 years of experience in SW Engineering & DevRel
•Microsoft, Oracle, IBM, Nokia, Unisys, Accenture, startups
•Microsoft Azure Developer Relations Lead
•IBM Watson Tech Evangelist & Cloud Rockstar
•IBM Mobile Tech Evangelist & Global Thought Leader
•Nokia Developers Global Champion
•Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI,
Blockchain, Rust
•Speaker at conferences
•Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft
TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon,
CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit,
Porto TechHub Conf, Pyjamas, JUGs, meetups, etc.
@juarezjunior
@juarezjunior

3. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Workshop Agenda
•Stable Diffusion -Concepts
•Stable Diffusion for Java (SD4J) by
Oracle Labs
•ONNX and ONNX Runtime
Extensions
•SD4J –Demo + Workshop
•Oracle Database 23ai
•AI / ML with Oracle Cloud
•Technical References
Copyright © 2024, Oracle and/or its affiliates

4. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Stable Diffusion -Concepts
•In a nutshell
•Stable Diffusion is a technique under the umbrella of GenAI used to generate
images primarily from textual descriptions.
•A diffusionmodelpredicts the probability distribution of images using a
diffusion process.
•The diffusionprocessiteratively applies noise to an image, gradually
increasing the noise level and then reversing the process to reconstruct the
original image -denoising with guidance, aligning with the textual description.
•One of the advantages of Stable Diffusion is its ability to control the diversity
and quality of generated images through the noise level parameter.
•The noise intensity allows users to adjust the image quality and diversity
trade-offs.
Copyright © 2024, Oracle and/or its affiliates

5. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Stable Diffusion -Concepts
•In a nutshell
•Stable Diffusion finds applications in various domains, such as art, design, and
entertainment, by providing image synthesis, augmentation, and
manipulation.
•It can run on regular computers with good graphics cards, making it more
accessible.
•Most projects are Open-Source, allowing developers to experiment and fine-
tune the model to meet specific needs.
Copyright © 2024, Oracle and/or its affiliates

6. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Stable Diffusion for Java
•SD4J
•A Java implementation of Stable Diffusion (modified port of
a C# implementation) running on top of ONNX Runtime
•Created by Oracle Labs(R&D) as OSS
•It targets ONNX Runtime 1.14
•It supports using CPUs and CUDA GPUs
•It supports SD v1.5, SD v2 and SDXL style models
•It provides a reusable class (com.oracle.labs.mlrg.sd4j.SD4J)
with a clear contract (interface) to allow the creation of
image generation pipelines
Copyright © 2024, Oracle and/or its affiliates

7. Confidential - Oracle Restricted
Confidential - Oracle Restricted
ONNX -Open Neural Network Exchange
•ONNX Format
•An open-source format designed to represent machine learning models.
•Expressed as a Protocol Buffers (protobuf) file containing a
computational graph representing the model.
•Models trained in one framework can be converted to the ONNX format
and used in another.
•The ONNX ecosystem includes a wide range of tools and libraries for
model conversion, optimization, inference, and visualization.
•While ONNX can represent models for training, its primary focus is on
inference, which is the process of using a trained model to make
predictions on new data.
•Promotes interoperability (frameworks, tools, runtime environments).
•Also promotes standardization, efficiency (HW), and ecosystem
development.
Copyright © 2024, Oracle and/or its affiliates

8. Confidential - Oracle Restricted
Confidential - Oracle Restricted
ONNX -Open Neural Network Exchange
•ONNX Runtime
•The ONNX Runtime is an optimized inference engine,
offering efficient execution of ONNX models with low latency
and high throughput.
•It leverages hardware acceleration capabilities, including GPU
acceleration through CUDA, and optimizations for CPUs.
•It provides tools for quantizing and optimizing ONNX models
to improve performance and reduce memory footprint,
making them more suitable for deployment on resource-
constrained devices.
•It is also cross-platform, as the ONNX Runtime supports
multiple operating systems, including Windows, Linux, and
macOS, making it versatile for deployment across different
target environments.
Copyright © 2024, Oracle and/or its affiliates

9. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Copyright © 2023, Oracle and/or its affiliates. All rights reserved
Experimente o poder da IA
gratuitamente na nuvem
Conheça os recursos de IA e ML da Oracle Cloud
com oModo Gratuito.
São US$ 500em créditos para você usar
durante 30 dias –uma condição EXCLUSIVA
para visitantes do TDC Floripa!

10. Confidential - Oracle Restricted
Confidential - Oracle Restricted
New| Oracle Database 23ai
Next-Generation Converged Database Services
Over300 major new features plusthousands of enhancements
Available on OCI and Azure
Major focus on
•AI for Data features
•Developer/Analyst features
•Mission-Critical features
Addresses data management pain points that have frustrated customers forever
Copyright © 2024, Oracle and/or its affiliates

11. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Oracle Database 23ai -AI for Data
Algorithmic AI AI Vector Search Augmented Generative AI (LLMs)
Distributed AI AI Storage AI Developer Tools
Copyright © 2024, Oracle and/or its affiliates

12. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Oracle Database 23ai -Dev for Data
JSON Relational
Duality Views
Property Graph Views
JavaScript
Stored Procedures
Data Intent
Language
Lock-free Consistent Updates,
Long-running Transactions
True
Cache
Copyright © 2024, Oracle and/or its affiliates

13. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Oracle Database 23ai -Mission-Critical Data
RAFT Replication for
Globally Distributed Database
In-Database
Firewall
Real-Time SQL Plan
Management
RAC, Exadata, Data Guard
Simplicity and Scalability
Analytic SQL
Simplicity and Scalability
Priority
Transactions
Copyright © 2024, Oracle and/or its affiliates

14. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Free| Oracle Database 23ai For Developers
Oracle Always Free ADB
Available on OCI
ADB Free Container Image
Available for download
Oracle Database Free
Available as RPM, Docker
Image, VBox VM
Copyright © 2024, Oracle and/or its affiliates | Confidential -Internal
Very Low Cost Supported DeveloperEdition of ADB, BaseDB, ExaDB-D, and ExaDB-C@C

15. Confidential - Oracle Restricted
Confidential - Oracle Restricted
SD4J –Demo + Workshop
•Creating GenAI Apps in Java with SD4J (Stable
Diffusion for Java) and the ONNX Runtime
•https://juarezjunior.medium.com/b72385c59329
Copyright © 2024, Oracle and/or its affiliates

16. Confidential - Oracle Restricted
Confidential - Oracle Restricted
Copyright © 2024, Oracle and/or its affiliates
Technical References
•SD4J –xxx
•ONNX Format and Runtime -xxx
•Oracle AI Vector Search -https://rb.gy/sp41uk
•Oracle AI Vector Search for Java Developers with the Oracle Database 23ai -
https://rb.gy/7ivwxr
•Java / JDBC –https://www.oracle.com/jdbc
•Introduction to Oracle JDBC Driver Support for Virtual Threads -https://bit.ly/3UlNJWP
•Developing an Oracle JDBC app with GraalVM Native Image -https://rb.gy/iy3sgh
•Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver -https://rb.gy/42dnz5
•Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC
•Introduction to JDBC Reactive Extensions with the Oracle Database —Developer
Release –https://rb.gy/qxlrbx
•Pipelined Database Operations -https://rb.gy/iy3sgh

17. Juarez Junior
@juarezjunior