I Love Tech Romania - The Trinity in GenAI - Spring AI, LangChain4J and OpenAI

I Love Tech Romania - The Trinity in GenAI - Spring AI, LangChain4J and OpenAI

1. ORACLE

The Trinity in GenAl - Spring Al,
LangChain4J and OpenAl

| Love Tech ROMANIA 2024

Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist

ORACLE

Meet our international

2. [9] ojuarezjunior Juarez Barbosa Junior

Senior Principal Java Developer Evangelist

+ Coming from Dublin, Ireland
. ae years of experience in SW Engineering & DevRel

Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups
Microsoft Azure Developer Relations Lead

IBM Watson Tech Evangelist & Cloud Rockstar

IBM Mobile Tech Evangelist & Global Thought Leader
Nokia Developers Global Champion

Java, Python, Cloud, DevOps, SRE, Cloud-native, loT, Al,
Blockchain, Rust

+ Speaker at conferences

Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft
TechX, Prime, JCON, DevConf.cz, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer's Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon,
CodeFrenzy, Mélée Numérique, React Summit, Test.js Summit,
Porto TechHub Conf, Pyjamas, MiTechCon, JUGs, meetups, etc.

3. Agenda

« Al and GenAl - Synthetic Content

+ Artificial Intelligence and GenAl

+ OpenAl, Cohere

+ Spring Al

+ LangChain4J

+ Enabling GenAl with Al Vector Search in
Oracle Database

+ Java App Dev with the Oracle Database
+ Technical References

+ Audience: Business and Technical
audiences (not Machine Learning
Engineers)

4. Bucharest the Capital of Romania
ictured in a futuristic scenario in one
Al and GenAl — image

Synthetic Content

5. Artificial Intelligence and GenAl

Core concepts in a nutshell

+ GenAl - Artificial intelligence alebnthrns and transformer models with the
capability of generating content.

Content - Text, images, videos, code, and others.
Generation - a result of an input prompt (prompt engineering).
Training data - dataset

- Apps - Applied currently in the fields of content development, entertainment,
drug discovery, software development, healthcare, material generation
sciences, and others.

- Ethical issues — misuse, deep fakes, bias and discrimination, automation and
job replacements, cybersecurity and cybercriminals.

+ Alis now a commodity!

6. OpenAl Wegen

- Generative Al with OpenAl
+ Natural Language Processing (NLP)
- Powerful, pre-trained and customizable Machine Learning Models
+ OpenAl API (REST interface)
+ Scalable infrastructure (powered by Microsoft Azure)
- Artificial General Intelligence (AGI) Research - ultimate goal

+ https://openai.com/sora

7. © spring

Spring Al

+ Spring Al
+ Java app framework for building Al-related applications. It encapsulates many
of the complexities required to interact with a number of different GenAl
providers such as OpenAl by providing a robust abstraction layer.

- API support across Al providers for chat, text-to-image, and embedding
models.

Its goal is to apply to the Al domain Spring ecosystem design principles such
as portability and modular design and promote using POJOs as the building
blocks of an application to the Al domain.

+ https://spring.io/projects/spring-ai

8. LangChain4J

. LangChain4J
+ Easy interaction with LLMs and Vector Stores

+ All major commercial and open-source LLMs and Vector Stores are easily
accessible through a unified API

« Easy integration with Java frameworks - Quarkus and Spring Boot
+ Al Services, RAG, Tools, Chains
+ https://docs.langchain4j.dev/

9. Enabling Generative Al
with Al Vector Search in
Oracle Database

10. The following is intended to outline our general product
direction. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions. The development,
release, timing, and pricing of any features or functionality
described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.

‘The materials in this presentation pertain to Oracle Heath, Oracle, race Come, anderer Enz which ae a wholly owed
“Subsidies of Orce Corporation Natangin ths presentation shuld be taken a nating hat any ecc regurängihe
integration of any EMEA Cerner and/ox Emo erties hove been made were ANNEE ADO Pos nt arenay occurred.

Capri ©2024, rc and/or states

11. SUPERFAST

SIMPLE

Democratizes

Ml IL

12. Finding what you’re looking for can be challenging...

| have a photo of an
artwork, but don’t know
the artist or work’s name

| got this error message
ORA-XXXX, but don't
know how to resolve it

| heard a catchy tune
and can even hum a
few bars, but don't
know the title or artist

13. A technology called Vector Search
enables end-user questions
to be mapped to relevant data
in your database

a e —_

14. Transformers and Contextual Embeddings
eg. BERT, GPT, etc.

late 2010s Relational Database
Integration
2.8. Oracle
Efficient Search Algorithms late 2010s - 2020s
€... ANN, HNSW, PQ, Annoy
2010s
Deep Learning Era
ee ene ÓN Dedicated Vector Databases
2010s e. Mivus, Pinecone, Vespa
Word Embeddings late 20108
eg. LSA and LDA
2000s
Vector Space Model
1960s - 1970s Early Multidimensional Indexing
e.g., KD trees, R-trees, Ball trees
1970s - 1990s

15. M

ontent
E

16. An intuitive example
Word relatedness in two dimensions

di

16 Copie 02024 Orde nara at

pl

17. Embeddings for images
Convert an image to one or more vector embeddings and search vector database

Compare these against other
vectors in the database to find
similar content

18. Generating embeddings

Use an embedding model

Text documents

EB
EN

EN

+

Natu Bug
Embedding
Mod

y

Image

Text Vector Table

id vector text

1 “It was the best of times, it was the
worst of times, it was.”

2 “itis a truth universally
acknowledged, that a single man.”

3 11.3, 0.1, 0.2, -1.1, ... “It was a bright cold day in April, and
the clocks were striking.”

Image Vector Table

id Ss ES

1 [05,15,26,-14, ] =

2 11.0,09,1.6,-13, -] 07

3 106,11,13,.03,.] <=

19. Simple vector search
Pipeline

as ay ID BS- -Ep À
Matches

What we're looking for ED ver search Data Object Retrieval Relevant
Text description, image, etc. Content

‘compare

20. Ml

Role of vector databases with LLMs

Address the hallucination problem inherent in LLM responses
Augment prompt with enterprise-specific content to produce better responses
Avoid exceeding LLM token limits by using most relevant content

a

Vector Database

Broad range of data

from the internet
Snapshot of data from a point in time

Private enterprise data

Current data - frequently updated

21. Role of vector databases with LLMs

Avoid using sensitive customer data for LLM training and fine-tuning

Cheaper than fine-tuning LLMs, which can be expensive to update

Real-time updated knowledgebase

Cache previous LLM prompts/responses to improve performance and reduce costs

Better business outcomes

amp +

Vector Database

22. LLM-based chatbot with “enterprise knowledge”
Extended pipeline using retrieval architecture

Enhanced prompt + policy details +
scenario examples + known concerns +

lic ted | che un
"What's the policy?" expected policy changes.
Prompt i |= Embedding Ay
Model
"What's my corporate
401k policy?”

"y

J Similarity Search Fetch docs for matching IDs
—

=> Vector
Matches

Vector Database Relational Database

23. Use cases
What do embeddings and vector databases enable?

Similarity Search Data Analytics
Match candidates with jobs ‘Anomaly detection
Image and video retrieval Pattern recognition
Find patients with similar issues
Legal e-discovery
Computer Vision Biomedical Research Geographic
o e Information Systems
ens DG Amen Aes Spatial analysis
Molecular structure search ee

Object detection

24. Al Vector Search can augment your prompt
to enable accuracy with Generative Al.

25. Sy ©

VA cata Kram Java App
== Dev with

an Kio y = | | Oracle
ES © +. 2 D. Database

— oge- [TeBBES

26. Oracle JDBC - Support for the Latest Java
Versions

. Support to the latest Java versions

(11,17,

* Project Loom - Virtual Threads
support

; 2 sl
J ava 2 2 + Java Virtual Threads and Pipelined
Database Operations

Launch Stream

1) and GraalVM

+ JDBC Standards - 4.2 and 4.3
+ Reactive Streams - Java Flow API
support

Re Reactive Streams Ingestion
March 19th 17 + Pipelined Database Operations

27. Overview of Oracle DB Access with Java

operators (map, reduce, filters),
concurrency modeling,

monitoring, tracing,
User Java code + JDBC

JavaucFlow

Database Access with

|

ut

Async call with non-blocking backpressut Oracle Database

|" Synchronous blocking cas

Copyright © 2024, Oracle and/or its affiliates

28. Technical References

* Spring Al - https://spring.io/projects/spring-ai
« LangChain4J - https://github.com/langchain4j
* OpenAl - https://openai.com/
+ Oracle Al Vector Search - https://rb. hw4m
+ SD4J by Oracle Labs - https://github.com/oracle/sd4j
+ Java / JDBC - https: //www.oracle.com/database/technologies/appdev/idbc.html
+ Introduction to Oracle JDBC Driver Support for Virtual Threads -
https: //bit.Iy/3UINIWP
+ Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gv/iy3sgh
+ Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver - https://rb.gy/42dnz5
+ Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.lv/3rEiRnC
+ Introduction to JDBC Reactive Extensions with the Oracle Database 23c Free —
Developer Release - https://rb.gv/axlrbx
+ Pipelined Database Operations - https://rb.gv/iv3sgh

29. Juarez Junior

Y | @juarezjunior

30. ORACLE

31. Appendix

32. New | Oracle Database 23ai

Next-Generation Converged Database Services

Over 300 major new features plus thousands of enhancements
Available on OCI and Azure
Major focus on

+ Al for Data features

ai
+ Developer/Analyst features Database 23

+ Mission-Critical features

Addresses data management pain points that have frustrated customers forever

33. Oracle LiveLabs

Showcasing how Oracle's solutions can
solve your business problems

500+

free workshops,
available or in
development

people have already visited
LiveLabs

600+

events run
using LiveLabs
workshops

developer.oracle.com/livelabs
learn something new ...at your pace!

34. Qos

Oracle ACE Program

3 membership tiers à Oracle ACE

Oracle ACE à Oracle ACE Oracle ACE ás
Director Pro Associate Nominate

yourself or a candidate:

ace.oracle.com/nominate
Learn more - ace.oracle.com

Connect: EP [email protected] E @oracleace facebook.com/OracleACEs — [8] blogs.oracle.com/ace

35. Create your FREE
Cloud Account

* Goto

https://signup.cloud.oracle.com/

ORACLE
Cloud Infrastructure

de

~—_

Oracle Cloud Free Tier

Get started with.

Terms of Use

36. SD4)J - Demo

TITAN

- Creating GenAl Apps in Java with SD4J (Stable
Diffusion for Java) and the ONNX Runtime

. https:/ /juarezjunior.medium.com/b72385c59329

37. Spring Al with Open Al

TITAN

- Code walkthrough of a sample Spring Al application
that uses Open Al