DevTalks Cluj Romania - A Solid Foundation for GenAI Apps.pdf

DevTalks Cluj - A Solid Foundation for GenAI Apps

1. A Solid Foundation for GenAl Apps -
Exploring Architectural Blueprints for LLM
Solutions with Java

Dev<Talks,

DevTalks Cluj 2024

Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist

ORACLE

2. [9] ojuarezjunior Juarez Barbosa Junior
Senior Principal Java Developer Evangelist @ Oracle

+ Coming from Dublin, Ireland

+ 28 years of experience in SW Engineering & DevRel
+ Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups
+ Microsoft Azure Developer Relations Lead
+ IBM Watson Tech Evangelist & Cloud Rockstar
+ IBM Mobile Tech Evangelist & Global Thought Leader
+ Nokia Developers Global Champion
+ Java, Python, Cloud, DevOps, SRE, Cloud-native, loT, Al, Blockchain,
Rust
+ Speaker at conferences
+ Oracle CloudWorld, Oracle Code, Microsoft Ignite & TechX, jPrime,
JCON, DevConf.cz, DevTalks, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer's Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon, CodeFrenzy,
Mélée Numérique, React Summit, Test js Summit, Porto TechHub
Conf, Pyjamas, MiTechCon, Data Science Summit,
OpenSourceNorth, WeAreDevelopers, Global Software Architecture

Summit, JavaCro, JUGs, OUGs, GDGs, meetups, hackathons, and
customer engagements.

3. D
Agenda \

+ Al and GenAl - Synthetic Content |
+ GenAl Concepts and AGI

* OpenAl, SpringAl, LangChain4J

* Oracle Database 23ai (Vector Store)

+ Reference Architecture for GenAl Apps

+ Direct LLM calls (text generation)

+ LLM + RAG with SpringAl and LangChain4J

+ Multimodality Prompts (various media)

+ Function Calling (execution of client-side
functions)

+ Audience: SW Engineers + Business audiences
(not necessarily ML Engineers)

4. Al and GenAl - Cluj-Napoca in Romania pictured ina
Synthetic Content futuristic cityscape

5. GenAl Concepts and AGI

- Core concepts in a nutshell

+ GenAl - Artificial intelligence agora and transformer models with the
capability of generating content.

+ Content - Text, images, videos, code, and others.
+ Generation - a result of an input prompt (prompt engineering).
+ Training data - dataset

- Apps - Applied currently in the fields of content development, entertainment,
drug discovery, softwaré development, healthcare, material generation
sciences, and others.

+ Ethical issues — misuse, deep fakes, bias and discrimination, automation and
job replacements, cybersecurity and cybercriminals.

- Alis now a commodity -> Artificial General Intelligence (AGI)

6. OpenAl Wegen

- Generative Al with OpenAl
+ Natural Language Processing (NLP)
- Powerful, pre-trained and customizable Machine Learning Models
+ OpenAl API (REST interface)
+ Scalable infrastructure (GPUs)
+ https://openai.com/sora

7. © spring

Spring Al (1.0.0 M2)

- Spring Al (Al Concepts)

+ A Java app framework for building Al-related applications. It provides a robust
abstraction layer that encapsulates many of the complexities required to interact with
various GenAl providers, such as OpenAl, Cohere, Google Vertex Al/Gemini, and
others.

+ API support across Al providers for chat, text-to-image, and embedding models.

Its goal is to apply to the Al domain Spring ecosystem design principles such as
portability and modular design and promote using POJOs as the building blocks of an
application to the Al domain.

+ https://spring.io/projects/spring-ai

8. LangChain4J

. LangChain4J
+ Easy interaction with LLMs and Vector Stores

+ All major commercial and open-source LLMs and Vector Stores are easily
accessible through a unified API

« Easy integration with Java frameworks - Quarkus and Spring Boot
+ Al Services, RAG, Tools, Chains
+ https://docs.langchain4j.dev/

9. Retrieval-Augment $ a

Generation (RAG) LS 18-7:

r

- RAG (Retrieval-Augment Generation) with Oracle Cloud / Oracle Database 23ai

+ LLMs' baseline knowledge might have gaps based on their training data. If you ask
an LLM to write about a recent trend or event, the LLM won't know what you're
talking about, so you may get meaningless responses.

+ There's a way to provide LLMs with additional information (including business-
specific data)- that is what RAG is for!

+ Oracle - What Is Retrieval-Augmented Generation (RAG)?
+ Oracle - What Is Generative Al? How Does It Work?

10. Retrieval-Augment
Generation (RAG) -0-

- RAG (Retrieval-Augment Generation) with Spring Al
+ In a nutshell, RAG is a technique that:
+ Reads unstructured data;
+ Converts the data into tokens;

+ It uses embeddings to transform the tokens into floating-point
arrays or vectors so that LLM can process and interpret them;

+ It stores the vectors in Vector Databases.

11. New | Oracle Database 23ai D UY

Next-Generation Converged Database Services

Over 300 major new features plus thousands of enhancements
Available on OCI and Azure
Major focus on

+ Al for Data features

ai
+ Developer/Analyst features Database 23

+ Mission-Critical features

Addresses data management pain points that have frustrated customers forever

12. Oracle Database 23ai - Al for Data

Algorithmic Al

TR

Distributed Al

—

Pad
dz

—+

Al Vector Search Augmented Generative Al (LLMs)

a

Al Storage Al Developer Tools

13. Free | Oracle Database 23ai For Developers

Oracle Always Free ADB ADB Free Container Image Oracle Database Free

Available on OCI Available for download Available as RPM, Docker
Image, VBox VM

Very Low Cost Supported Developer Edition of ADB, BaseDB, ExaDB-D, and ExaDB-C@C

Cape © 2024 Orca o/o as Cor ral

14. Reference Architecture for GenAl
MIC TO
[search apı | chatapı | Prompt ut |

a oros … Reference
==

|, = = (ae Architecture
== * E E

> ER" En for GenAl Apps

LLM Layer Hallucination

Flow 12: Ingestion

Flow aaiirterence MICE TET open ceras
Purpose fant

Source: Architectural Approach for Building
Generative Al Applications

15. Confidential - Oracle Restricted

Demos

Confidential - Oracle Restricted

16. Direct LLM calls - Spring Al with Open Al

TITAN

- Demo - Direct LLM calls - Spring Al with Open Al

17. RAG - Spring Al with Open Al and Oracle
Database 23ai

TITAN

- Demo - RAG - Spring Al with Open Al and Oracle
Database 23ai

18. Function Calling - Spring Al with Open Al

TITAN

- Demo -Function Calling - Spring Al with Open Al

19. Multimodality - Spring Al with Open Al

IT |

- Demo - Multimodality - Spring Al with Open Al

20. Confidential - Oracle Restricted

Technical References

+ Spring Al - https://spring.io/projects/spring-ai
+ LangChain4J - Retrieval-Augmented Generation (RAG) with LangChain4j and the Oracle
Database 23ai https: //rb. dmwmz
+ OpenAl - https://openai.com/
* Oracle Al Vector Search for Java Developers with the Oracle Database 23ai -
https://rb. ivwxr
. en / JDBC - https://www.oracle.com/jdbc
Introduction to Oracle JDBC Driver Support for Virtual Threads - https: 11 bit.ly/3UINJWP
+ Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gv/iy3sgh
+ Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver - https://rb.gy/42dnz5
+ Getting Started with the Java library for Reactive Streams Ingestion (RSI) -

https://bit.Iv/3rEiRn
+ Introduction to JDBC Reactive Extensions with the Oracle Database — Developer
Release - https://rb. xlrbx

+ Pipelined Database Operations - https: //rb.gv/iv3sgh
* Spring Al - https://github.com/kousen / https: //github.com/ThomasVitale/llm-apps-
java-spring-ai

Copy;
Confidenial- Oracle RESTES

21. Juarez Junior

Y | @juarezjunior

22. ORACLE

23. Confidential Oracle Restricted

NA
A

Java Developers

24. VA
A

¿BO
3 Java App
Dev with
EEE ö Oracle
nn Database

20088299

ORACLE 23"
Database

25. Confidential - Oracle Restricted

VA
Overview of Oracle DB Access with Java”

operators (map, reduce, filters),
concurrency modeling,

monitoring, tracing,
User Java code + JDBC

Javau.c Flow Oracle JDBC Driver

Async call with non-blocking backpressut Oracle Database

|" Synchronous blocking cas
Copyright © 202

Confidential - Oracle Restctes lo]

26. Oracle JDBC - Support for the Latest Java x

Versions

Java 22

Launch Stream

March 19th

(11

su

Suppert to the latest Java versions
17, 21) and GraalVM

coe Loom - Virtual Threads

. pe Virtual Threads and Pipelined
Database Operations

JDBC Standards - 4.2 and 4.3
appa Streams - Java Flow API

waBeC Reactive Streams Ingestion
+ Pipelined Database Operations

27. Oracle LiveLabs

Showcasing how Oracle's solutions can
solve your business problems

500+

free workshops,
available or in
development

people have already visited
LiveLabs

600+

events run
using LiveLabs
workshops

developer.oracle.com/livelabs
learn something new ...at your pace!

28. Qos

Oracle ACE Program

3 membership tiers à Oracle ACE

Oracle ACE à Oracle ACE Oracle ACE ás
Director Pro Associate Nominate

yourself or a candidate:

ace.oracle.com/nominate
Learn more - ace.oracle.com

Connect: EP [email protected] E @oracleace facebook.com/OracleACEs — [8] blogs.oracle.com/ace

29. Create your FREE
Cloud Account

* Goto

https://signup.cloud.oracle.com/

ORACLE
Cloud Infrastructure

de

~—_

Oracle Cloud Free Tier

Get started with.

Terms of Use