DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime

DSS_ML_Creating GenAI Apps in Java with SD4J and the ONNX Runtime

1. Creating GenAI Apps in Java
withSD4Jand the ONNX
Runtime
Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist, Oracle

2. Juarez Barbosa Junior
Senior Principal Java Developer Evangelist
•Coming from Dublin, Ireland
•Developer since 1995 (Java ☕️1997)
•28 years of experience in SW Engineering & DevRel
•Microsoft, Oracle, IBM, Nokia, Unisys, Accenture, startups
•Microsoft Azure Developer Relations Lead
•IBM Watson Tech Evangelist & Cloud Rockstar
•IBM Mobile Tech Evangelist & Global Thought Leader
•Nokia Developers Global Champion
•Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI,
Blockchain, Rust
•Speaker at conferences
•Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft
TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon,
CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit,
Porto TechHub Conf, Pyjamas, JUGs, meetups, etc.
@juarezjunior
@juarezjunior

3. Agenda
•Stable Diffusion -Concepts
•Stable Diffusion for Java (SD4J) by
Oracle Labs
•ONNX and ONNX Runtime Extensions
•SD4J -Demo
•Java App Dev with the Oracle Database
•Technical References
•Oracle for Java Developers
Copyright © 2024, Oracle and/or its affiliates

4. Stable Diffusion -Concepts
•In a nutshell
•Stable Diffusion is a technique under the umbrella of GenAI used to generate
images primarily from textual descriptions.
•A diffusionmodelpredicts the probability distribution of images using a
diffusion process.
•The diffusionprocessiteratively applies noise to an image, gradually
increasing the noise level and then reversing the process to reconstruct the
original image -denoising with guidance, aligning with the textual
description.
•One of the advantages of Stable Diffusion is its ability to control the diversity
and quality of generated images through the noise level parameter.
•The noise intensity allows users to adjust the image quality and diversity
trade-offs.
Copyright © 2024, Oracle and/or its affiliates

5. Stable Diffusion -Concepts
•In a nutshell
•Stable Diffusion finds applications in various domains, such as art, design, and
entertainment, by providing image synthesis, augmentation, and
manipulation.
•It can run on regular computers with good graphics cards, making it more
accessible.
•Most projects are Open-Source, allowing developers to experiment and fine-
tune the model to meet specific needs.
Copyright © 2024, Oracle and/or its affiliates

6. Stable Diffusion for Java
•SD4J
•A Java implementation of Stable Diffusion (modified port of
a C# implementation) running on top of ONNX Runtime
•Created by Oracle Labs(R&D) as OSS
•It targets ONNX Runtime 1.14
•It supports using CPUs and CUDA GPUs
•It supports SD v1.5, SD v2 and SDXL style models
•It provides a reusable class (com.oracle.labs.mlrg.sd4j.SD4J)
with a clear contract (interface) to allow the creation of
image generation pipelines
Copyright © 2024, Oracle and/or its affiliates

7. ONNX -Open Neural Network Exchange
•ONNX Format
•An open-source format designed to represent machine learning models.
•Expressed as a Protocol Buffers (protobuf) file containing a
computational graph representing the model.
•Models trained in one framework can be converted to the ONNX format
and used in another.
•The ONNX ecosystem includes a wide range of tools and libraries for
model conversion, optimization, inference, and visualization.
•While ONNX can represent models for training, its primary focus is on
inference, which is the process of using a trained model to make
predictions on new data.
•Promotes interoperability (frameworks, tools, runtime environments).
•Also promotes standardization, efficiency (HW), and ecosystem
development.
Copyright © 2024, Oracle and/or its affiliates

8. ONNX -Open Neural Network Exchange
•ONNX Runtime
•The ONNX Runtime is an optimized inference engine,
offering efficient execution of ONNX models with low latency
and high throughput.
•It leverages hardware acceleration capabilities, including GPU
acceleration through CUDA, and optimizations for CPUs.
•It provides tools for quantizing and optimizing ONNX models
to improve performance and reduce memory footprint,
making them more suitable for deployment on resource-
constrained devices.
•It is also cross-platform, as the ONNX Runtime supports
multiple operating systems, including Windows, Linux, and
macOS, making it versatile for deployment across different
target environments.
Copyright © 2024, Oracle and/or its affiliates

9. SD4J –Demo
•Creating GenAI Apps in Java with SD4J (Stable
Diffusion for Java) and the ONNX Runtime
•https://juarezjunior.medium.com/b72385c59329
Copyright © 2024, Oracle and/or its affiliates

10. Oracle Database 23ai Free available for
download now
•Oracle Database 23ai Free accelerates
Oracle’s mission to make it simple to
develop and run all data-driven apps
•Provides developers easy early access to
23ai AppDev features
•Limited database size, memory, and CPU
threads
http://oracle.com/23aiFree
http://oracle.com/23aiFree
Copyright © 2024, Oracle and/or its affiliates

11. SD4J –Demo with the Oracle Database 23ai
•Code walkthrough, SD4J class, and integration with
the Oracle Database
Copyright © 2024, Oracle and/or its affiliates

12. Java App
Dev with
Oracle
Database
Copyright © 2024, Oracle and/or its affiliates

13. Java Database Connectivity with the Oracle
Database
•Java App Development with the Oracle Database
•Converged Database (Vector, Relational, NoSQL, Graph,
Blockchain, XML, Spatial, JSON) and Java App Dev with
Oracle Database
•Oracle JDBC -Support for the Latest Java Versions
•Overview of DB access with Java -JDBC (with Virtual
Threads), GraalVM, Reactive Streams, R2DBC, Reactive
Streams Ingestion Library (RSI), Oracle UCP, Pipelined
Database Operations
•Synchronous and Asynchronous UCs
Copyright © 2024, Oracle and/or its affiliates

14. Overview of Oracle DB Access with Java
Copyright © 2024, Oracle and/or its affiliates

15. Oracle JDBC -Support for the Latest Java
Versions
•Support to the latest Java versions
(11, 17, 21) and GraalVM
•Project Loom -Virtual Threads
support
•Java Virtual Threads and Pipelined
Database Operations
•JDBC Standards -4.2 and 4.3
•Reactive Streams -Java Flow API
support
•R2DBC, Reactive Streams Ingestion
(RSI)
•Pipelined Database Operations
Data access is critical in mission-
critical applications!
Copyright © 2024, Oracle and/or its affiliates

16. Copyright © 2024, Oracle and/or its affiliates
Technical References
•Stable Diffusion
•A latent text-to-image diffusion model -https://github.com/CompVis/stable-diffusion
•Stable Diffusion for Java (SD4J)
•SD4J by Oracle Labs -https://github.com/oracle/sd4j
•ONNX Runtime
•ONNX Runtime -https://onnxruntime.ai/
•Develop Java applications with Oracle Database
•JDBC –https://www.oracle.com/database/technologies/appdev/jdbc.html
•Introduction to Oracle JDBC Driver Support for Virtual Threads -https://bit.ly/3UlNJWP
•Developing an Oracle JDBC app with GraalVM Native Image -https://rb.gy/iy3sgh
•Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver -https://rb.gy/42dnz5
•Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC
•Introduction to JDBC Reactive Extensions with the Oracle Database 23c Free —Developer
Release –https://rb.gy/qxlrbx
•Pipelined Database Operations -https://rb.gy/iy3sgh

17. Oracle LiveLabs
Showcasing how Oracle’s solutions can
solve yourbusiness problems
500+
free workshops,
available or in
development
3.5 million
people have already visited
LiveLabs
developer.oracle.com/livelabs
learn something new …at your pace!
600+
events run
using LiveLabs
workshops

18. 3 membership tiers
Connect: @oracleace facebook.com/[email protected]
500+ technical experts &
community leaders helping peers globally
The Oracle ACE Program recognizes & rewards individuals for
their technical & community contributions to the Oracle community
Nominate
yourself or a candidate:
ace.oracle.com/nominate
Learn more-ace.oracle.com
blogs.oracle.com/ace

19. Juarez Junior
@juarezjunior