DUBJUG_Blazingly Fast GenAI App Development With Java and Spring AI.pdf

DUBJUG_Blazingly Fast GenAI App Development With Java and Spring AI

1. . ?
ORACLE 23° fl ll
ORACLE Database ol

Blazingly Fast GenAl App Development With
Java and Spring Al

ss)
DUBJUG 2024 DUBJUG
Juarez Barbosa Junior @juarezjunior
Senior Principal Java Developer Evangelist
ORACLE
SS

2. [9] ojuarezjunior Juarez Barbosa Junior
Senior Principal Java Developer Evangelist @ Oracle

+ Coming from Dublin, Ireland

+ 28 years of experience in SW Engineering & DevRel
+ Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups
+ Microsoft Azure Developer Relations Lead
+ IBM Watson Tech Evangelist & Cloud Rockstar
+ IBM Mobile Tech Evangelist & Global Thought Leader
+ Nokia Developers Global Champion
. um Python, Cloud, DevOps, SRE, Cloud-native, loT, Al, Blockchain,
ist
+ Speaker at conferences

+ Oracle CloudWorld, Oracle Code, Microsoft Ignite & TechX, ¡Prime,
JCON, DevConf.cz, GeeCon, DevOpsDays, DeveloperWeek, DevOps
Institute, CloudLand, DWX, The Developer's Conference (TDC),
Sec4Dey, JSNation, NodeConf, Conf42, Shift Conf, Global Azure,
Open-Source Lisbon, CodeFrenzy, Mélée Numérique, React Summit,
Tests Summit, Porto TechHub Conf, Pyjamas, MiTechCon, Data
Science Summit, OpenSourceNorth, WeAreDevelopers, Global
Software Architecture Summit, JavaCro, JUGs, OUGs, GDGs,

meetups, hackathons, and customer engagements.

3. Agenda

+ Aland GenAl - Synthetic Content

+ Artificial Intelligence and GenAl

+ OpenAl

+ Spring Al

+ GenAl with the Oracle Database 23ai

+ Demos

+ Oracle for Java / Database Developers
+ Technical References

+ Audience: Business and Technical
audiences (not necessarily Machine
Learning Engineers)

4. # A E

PEUT CRIE

=. _

= Y ÉS :

Al and GenAl — Dublin in Ireland pictured in a futuristic
Synthetic Content cityscape

5. Artificial Intelligence and GenAl

. Core concepts in a nutshell

+ GenAl - Artificial intelligence algorithms and transformer
models with the capability of generating content.

+ Content - Text, images, videos, code, and others.
+ Generation - a result of an input prompt (prompt engineering).

+ Ethical issues — misuse, deep fakes, bias and discrimination,
automation and job replacements, cybersecurity and
cybercriminals.

6. Artificial Intelligence and GenAl

. Core concepts in a nutshell

+ Apps - Applied currently in the fields of content development,
entertainment, drug discovery, software development,
healthcare, material generation sciences, and others.

« Alis now a commodity!
+ Artificial General Intelligence (AGI) Research - ultimate goal

7. OpenAl Wegen

- Generative Al with OpenAl
+ Natural Language Processing (NLP)

+ Powerful, pre-trained and customizable Machine Learning
Models

+ OpenAl API (REST interface)
+ Scalable infrastructure (GPUs)
« https://openai.com/sora

8. © spring

Spring Al (1.0.0 M1)

- Spring Al (Al Concepts)

+ A Java app framework for building Al-related applications. It provides a robust
abstraction layer that encapsulates many of the complexities required to interact with
various GenAl providers, such as OpenAl, Cohere, Google Vertex Al/Gemini, and
others.

+ API support across Al providers for chat, text-to-image, and embedding models.

Its goal is to apply to the Al domain Spring ecosystem design principles such as
portability and modular design and promote using POJOs as the building blocks of an
application to the Al domain.

+ https://spring.io/projects/spring-ai

9. © spring

- RAG (Retrieval-Augment Generation) with Spring Al

Spring Al

+ LLMs' baseline knowledge might have gaps based on their training data. If you ask
an LLM to write about a recent trend or event, the LLM won't know what you're
talking about, so you may get meaningless responses.

+ There's a way to provide LLMs with additional information (including business-
specific data)- that is what RAG is for!

+ Oracle - What Is Retrieval-Augmented Generation (RAG)?
+ Oracle - What Is Generative Al? How Does It Work?

10. © spring

- RAG (Retrieval-Augment Generation) with Spring Al
+ In a nutshell, RAG is a technique that:
+ Reads unstructured data;
+ Converts the data into tokens;

+ It uses embeddings to transform the tokens into floating-point
arrays or vectors so that LLM can process and interpret them;

Spring Al

+ It stores the vectors in Vector Databases.

11. © spring

- The Spring Al API covers a wide range of functionalities
+ Chat Completion API
« Embeddings API
+ Image Generation API
+ Speech / Transcription API
+ Vector Databases
+ Function Calling API

Spring Al

12. Spring Al

- Spring Al - Blazingly Fast Performance
+ GraalVM

Native Image - compile Java code ahead
of time (AOT) to a binary - a native
executable

Developing an Oracle JDBC app with
GraalVM Native Image

mvn -Pnative native:compile
./target/spring-ai-graalvm-oai.exe

plugin
<groupId»org.graalvm.buildtoo1s</grouptd»
<arti factrdonative-maven-pluginc/artifact1é>
<configuration»
<buildargs>
<buildargo-o ./target/spring-ai-graalvm-oaif//buildarg>
<Jouildangs>
</configuration>
</plugin>

© spring

Copyright © 2024, Oracle and/or its affiliates

13. © spring

Spring Al

- Spring Al - Blazingly Fast Performance
+ Spring with Virtual Threads

- Add the following entry in the application.properties file:
spring.threads.virtual.enabled=true

+ Class VirtualThreadTaskExecutor
+ Embracing Virtual Threads

- All together now: Spring Boot 3.2, GraalVM native images, Java 21, and
virtual threads with Project Loom

Copyright © 2024, Oracle and/or its affiliates

14. Wy
New | Oracle Database 23ai CZ

Next-Generation Converged Database Services

Over 300 major new features plus thousands of enhancements
Available on OCI and Azure
Major focus on

+ Al for Data features

ai
+ Developer/Analyst features Database 23

+ Mission-Critical features

Addresses data management pain points that have frustrated customers forever

15. Oracle Database 23ai - Al for Data

Algorithmic Al

TR

Distributed Al

—

Pad
dz

—+

Al Vector Search Augmented Generative Al (LLMs)

a

Al Storage Al Developer Tools

16. Oracle Database 23ai - Dev for Data

SÉËE
JSON Relational
Duality Views

sed

Data Intent
Language

Lock-free Consistent Updates,
Long-running Transactions

JavaScript
Stored Procedures

17. Oracle Database 23ai - Mission-Critical Data

Hl,
EFT
RAFT Replication for
Globally Distributed Database

Ebo

Priority
Transactions

Cape ©2024 Once a/a

NN
soi © ©

Real-Time SQL Plan
Management

RAC, Exadata, Data Guard
Simplicity and Scalability

So

In-Database
Firewall

saL

Analytic SQL
Simplicity and Scalability

18. Free | Oracle Database 23ai For Developers

Oracle Always Free ADB ADB Free Container Image Oracle Database Free

Available on OCI Available for download Available as RPM, Docker
Image, VBox VM

Very Low Cost Supported Developer Edition of ADB, BaseDB, ExaDB-D, and ExaDB-C@C

Cape © 2024 Orca o/o as Cor ral

19. Confidential - Oracle Restricted

Demos

Confidential Orale Restricted

20. Reference Architecture for GenAl
MIC TO
[search apı | chatapı | Prompt ut |

a oros … Reference
= ER |. ES

oon (en En = Architecture
DM

=
=
Se el oe for GenAl Apps

Doe
=
rn ee

=

Source: Architectural Approach for Building Generative Al Applications

21. Image Generation - Spring Al with Open Al

TITAN

- Demo - Direct LLM calls - Image Generation

22. Chatbot - Spring Al with Open Al

IT |

. Demo - Direct LLM calls - Chatbot

23. RAG - Retrieval Augment Generation
Spring Al with Open Al and Oracle Database 23ai

TITAN

- Demo - RAG
- Spring Al + SimpleVectorStore + Open Al

. Spring Al + OracleVectorStore + OpenAl + Oracle
Database 23ai

24. Multimodality - Spring Al with Open Al

IT |

- Demo - Multimodality - Spring Al with Open Al

25. Function Calling - Spring Al with Open Al

TITAN

- Demo -Function Calling - Spring Al with Open Al

26. Confidential - Oracle Restricted

deyeloper.oracle.com
oracle.com/jdbc

Confidential - Oracle Restricted

27. te ©

Grave oca Koon Java App
paa Dev with
Oracle
Database

— Gpgo- 28eecs

Zo

28. Overview of Oracle DB Access with Java

operators (map, reduce, filters),
concurrency modeling,

monitoring, tracing,
User Java code + JDBC

JavaucFlow

Database Access with

|

ut

Async call with non-blocking backpressut Oracle Database

|" Synchronous blocking cas

Copyright © 2024, Oracle and/or its affiliates

29. Oracle JDBC - Support for the Latest Java
Versions

. Support to the latest Java versions

(11,17,

* Project Loom - Virtual Threads
support

; 2 sl
J ava 2 2 + Java Virtual Threads and Pipelined
Database Operations

Launch Stream

1) and GraalVM

+ JDBC Standards - 4.2 and 4.3
+ Reactive Streams - Java Flow API
support

Re Reactive Streams Ingestion
March 19th 17 + Pipelined Database Operations

30. Oracle LiveLabs

Showcasing how Oracle's solutions can
solve your business problems

500+

free workshops,
available or in
development

people have already visited
LiveLabs

600+

events run
using LiveLabs
workshops

developer.oracle.com/livelabs
learn something new ...at your pace!

31. Qos

Oracle ACE Program

3 membership tiers à Oracle ACE

Oracle ACE à Oracle ACE Oracle ACE ás
Director Pro Associate Nominate

yourself or a candidate:

ace.oracle.com/nominate
Learn more - ace.oracle.com

Connect: EP [email protected] E @oracleace facebook.com/OracleACEs — [8] blogs.oracle.com/ace

32. Create your FREE
Cloud Account

* Goto

https://signup.cloud.oracle.com/

ORACLE
Cloud Infrastructure

de

~—_

Oracle Cloud Free Tier

Get started with.

Terms of Use

33. Confidential - Oracle Restricted

Technical References

Spring Al - https://spring.io/projects/spring-ai

OpenAl - https://openai.com/

Oracle Al Vector Search - https://rb.gv/sp41uk

Oracle Al Vector Search for Java Developers with the Oracle Database 23ai -

https://rb.gv/7ivwxr

E u / JDBC - https://www.oracle.com/jdbc
Introduction to Oracle JDBC Driver Support for Virtual Threads - https://bit.ly/3UINJWP

+ Developing an Oracle JDBC app with GraalVM Native Image - https://rb.sv/iy3sgh

+ Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver - https://rb.gy/42dnz5

* Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC

+ Introduction to JDBC Reactive Extensions with the Oracle Database — Developer
Release - https://rb.; xirbx

+ Pipelined Database Operations - https://rb.ev/iv3sgh

+ What's in Oracle Database 23ai for Java Developers? -
https://www.oracle.com/a/tech/docs/database/whats-in-oracledb23ai-for-java-
developers.pdf

confidential - Oracle

34. Juarez Junior

Y | @juarezjunior

35. ORACLE

36. Confidential - Oracle Restricted

Enabling Generative Al
with Al Vector Search in
Oracle Database 23ai

37. Democratizes

SIMPLE

SUPERFAST

El
Oracle Restricted

38. Finding what you’re looking for can be challenging...

| have a photo of an
artwork, but don’t know
the artist or work’s name

| got this error message
ORA-XXXX, but don't
know how to resolve it

| heard a catchy tune
and can even hum a
few bars, but don't
know the title or artist

39. A technology called Vector Search
enables end-user questions
to be mapped to relevant data
in your database

ne —_

40. Confidential - Oracle Restricted

Transformers and Contextual Embeddings
€.8. BERT, GPT, etc.

late 2010s Relational Database
Integration
eg, Oracle
Efficient Search Algorithms late 2010s - 2020s
€.8.. ANN, HNSW, PQ, Annoy
2010s
Deep Learning Era
a ee ae Dedicated Vector Databases
2010s 2.8. Milvus, Pinecone, Vespa
Word Embeddings late 2010s
eg. LSA and LDA
2000s
Vector Space Model
1960s - 1970s Early Multidimensional Indexing
e.g., KD trees, R-trees, Ball trees
1970s - 1990s

.
En E

41. M

Confidential - Oracle Restricted

content

Confidential - Oracle Restricted

42. An intuitive example
Word relatedness in two dimensions

di

AZ Core © 2024 Once doses
Confidential - Oracle Restricted

43. cle Restricted |

Embeddings for images
Convert an image to one or more vector embeddings and search vector database

Compare these against other
vectors in the database to find
similar content

Confidential - Oracle Restricted lo

44. Confidential - Oracle Restricted

Generating embeddings

Use an embedding model

Confidential - Oracle Restricted

Text documents

EB
EN

EN

+

Natu Bug
Embedding
Mod

y

Image

Text Vector Table

id vector text

1 “It was the best of times, it was the
worst of times, it was.”

2 “itis a truth universally
acknowledged, that a single man.”

3 11.3, 0.1, 0.2, -1.1, ... “It was a bright cold day in April, and
the clocks were striking.”

Image Vector Table

id Ss ES

1 [05,15,26,-14, ] =

2 11.0,09,1.6,-13, -] 07

3 106,11,13,.03,.] <=

45. Confidential - Oracle Restricted

Simple vector search
Pipeline

Input Data

ae Embedding

Model

What were looking for.
Text description, image, etc.

4 comen
Confidential - Oracle Restricted

— -Ep
eee D
Matches

Vector Search Data Object Retrieval Relevant
content

‘compare

46. Ml (IL

Role of vector databases with LLMs

Address the hallucination problem inherent in LLM responses
Augment prompt with enterprise-specific content to produce better responses
Avoid exceeding LLM token limits by using most relevant content

a

Vector Database

Broad range of data

from the internet
Snapshot of data from a point in time

Private enterprise data
Current data - frequently updated

47. Role of vector databases with LLMs

Avoid using sensitive customer data for LLM training and fine-tuning

Cheaper than fine-tuning LLMs, which can be expensive to update

Real-time updated knowledgebase

Cache previous LLM prompts/responses to improve performance and reduce costs

Better business outcomes

amp +

Vector Database

48. Confidential - Oracle Restricted

LLM-based chatbot with “enterprise knowledge”
Extended pipeline using retrieval architecture

Enhanced prompt + policy details +
scenario examples + known concerns +

"What's the policy?” expected policy changes...

Enhanced
Prompt

Augmented
Prompt LLM

Embedding

Prompt
Model

+ "What's my corporate
401k policy?”

Chat History 2
; RESPONSE
J Similarity Search Fetch docs for matching IDs
>
=> Vector
Matches

Vector Database Relational Database

50 Cmeex
confidential Grace Restrctes

49. Confidential - Oracle Restricted

Use cases
What do embeddings and vector databases enable?

Similarity Search Data Analytics

Match candidates with jobs Anomaly detection
Image and video retrieval Pattern recognition
Find patients with similar issues
Legal e-discovery

Computer Vision Biomedical Research Geographic

Information Systems
Face recognition Gene/DNA similarity analysis Spatial analysis
Biometric identification Molecular structure search ere
Object detection

confidential-OracleRestrcted TT lo]

50. Al Vector Search can augment your prompt
to enable accuracy with Generative Al