OSN-Blazingly Fast GenAI App Development With Java and Spring AI

OSN-Blazingly Fast GenAI App Development With Java and Spring AI

1. Blazingly Fast GenAI App Development With
Java and Spring AI
Open SourceNorth 2024
Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist
ORACLE
Copyright © 2024, Oracle and/or its affiliates

2. Juarez Barbosa Junior
Senior Principal Java Developer Evangelist @ Oracle
•Coming from Dublin, Ireland
•28 years of experience in SW Engineering & DevRel
•Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups
•Microsoft Azure Developer Relations Lead
•IBM Watson Tech Evangelist & Cloud Rockstar
•IBM Mobile Tech Evangelist & Global Thought Leader
•Nokia Developers Global Champion
•Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI,
Blockchain, Rust
•Speaker at conferences
•Oracle CloudWorld, Oracle Code, Microsoft Ignite & TechX, jPrime,
JCON, DevConf.cz, GeeCon, DevOpsDays, DeveloperWeek,
DevOps Institute, CloudLand, DWX, The Developer’s Conference
(TDC), Sec4Dev, JSNation, NodeConf, Conf42, Shift Conf, Global
Azure, Open-Source Lisbon, CodeFrenzy, Mêlée Numérique,
React Summit, Test.js Summit, Porto TechHub Conf, Pyjamas,
MiTechCon, Data Science Summit, OSN, JUGs, meetups,
hackathons.
@juarezjunior
@juarezjunior

3. Agenda
•AI and GenAI –Synthetic Content
•Artificial General Intelligence (AGI)
•OpenAI
•Spring AI
•Reference Architecture for GenAI Apps
•Oracle Database 23ai
•Spring AI –Demos
•Technical References
•Java App Dev with the Oracle Database
•Audience: SW Engineers + Business
audiences (not necessarily ML
Engineers)
Copyright © 2024, Oracle and/or its affiliates

4. AI and GenAI –
Synthetic Content
Minneapolis in Minnesota, USA pictured
in a futuristic cityscape
Copyright © 2024, Oracle and/or its affiliates

5. Artificial Intelligence and GenAI
•Core concepts in a nutshell
•GenAI-Artificial intelligence algorithms and transformer
models with the capability of generating content.
•Content-Text, images, videos, code, and others.
•Generation–a result of an input prompt (prompt engineering).
•Ethical issues –misuse, deep fakes, bias and discrimination,
automation and job replacements, cybersecurity and
cybercriminals.
Copyright © 2024, Oracle and/or its affiliates

6. Artificial Intelligence and GenAI
•Core concepts in a nutshell
•Apps–Applied currently in the fields of content development,
entertainment, drug discovery, software development,
healthcare, material generation sciences, and others.
•AI is now a commodity!
•Artificial General Intelligence (AGI) Research –ultimate goal
Copyright © 2024, Oracle and/or its affiliates

7. OpenAI
•Generative AI with OpenAI
•Natural Language Processing (NLP)
•Powerful, pre-trained and customizable Machine Learning
Models
•OpenAI API (REST interface)
•Scalable infrastructure (GPUs)
•https://openai.com/sora
Copyright © 2024, Oracle and/or its affiliates

8. Spring AI (1.0.0 M1)
•Spring AI (AI Concepts)
•A Java app framework for building AI-related applications. It provides a robust
abstraction layer that encapsulates many of the complexities required to interact with
various GenAI providers, such as OpenAI, Cohere, Google Vertex AI/Gemini, and
others.
•API support across AI providers for chat, text-to-image, and embedding models.
•Its goal is to apply to the AI domain Spring ecosystem design principles such as
portability and modular design and promote using POJOs as the building blocks of an
application to the AI domain.
•https://spring.io/projects/spring-ai
Copyright © 2024, Oracle and/or its affiliates

9. Spring AI
•RAG (Retrieval-Augment Generation) with Spring AI
•LLMs' baseline knowledge might have gaps based on their training data. If you ask
an LLM to write about a recent trend or event, the LLM won't know what you're
talking about, so you may get meaningless responses.
•There's a way to provide LLMs with additional information (including business-
specific data)-that is what RAG is for!
•Oracle -What Is Retrieval-Augmented Generation (RAG)?
•Oracle –What Is Generative AI? How Does It Work?
Copyright © 2024, Oracle and/or its affiliates

10. Spring AI
•RAG (Retrieval-Augment Generation) with Spring AI
•In a nutshell, RAG is a technique that:
•Reads unstructured data;
•Converts the data intotokens;
•It usesembeddingsto transform the tokens into floating-point
arrays orvectorsso that LLM can process and interpret them;
•It stores the vectors inVector Databases.
Copyright © 2024, Oracle and/or its affiliates

11. Spring AI
•The Spring AI API covers a wide range of functionalities
•Chat Completion API
•Embeddings API
•Image Generation API
•Speech / Transcription API
•Vector Databases
•Function Calling API
Copyright © 2024, Oracle and/or its affiliates

12. Spring AI
•Spring AI –Blazingly Fast Performance
•GraalVM
•Native Image -compile Java code ahead
of time (AOT) to a binary –a native
executable
•Developing an Oracle JDBC app with
GraalVM Native Image
•mvn -Pnative native:compile
•./target/spring-ai-graalvm-oai.exe
Copyright © 2024, Oracle and/or its affiliates

13. Spring AI
•Spring AI –Blazingly Fast Performance
•Spring with Virtual Threads
•Add the following entry in the application.properties file:
spring.threads.virtual.enabled=true
•Class VirtualThreadTaskExecutor
•Embracing Virtual Threads
•All together now: Spring Boot 3.2, GraalVM native images, Java 21, and
virtual threads with Project Loom
Copyright © 2024, Oracle and/or its affiliates

14. Reference
Architecture
for GenAI Apps
Copyright © 2024, Oracle and/or its affiliates
Source: Architectural Approach for Building Generative AI Applications

15. New| Oracle Database 23ai
Next-Generation Converged Database Services
Over300 major new features plusthousands of enhancements
Available on OCI and Azure
Major focus on
•AI for Data features
•Developer/Analyst features
•Mission-Critical features
Addresses data management pain points that have frustrated customers forever
Copyright © 2024, Oracle and/or its affiliates

16. Oracle Database 23ai -AI for Data
Algorithmic AI AI Vector Search Augmented Generative AI (LLMs)
Distributed AI AI Storage AI Developer Tools
Copyright © 2024, Oracle and/or its affiliates

17. Oracle Database 23ai -Dev for Data
JSON Relational
Duality Views
Property Graph Views
JavaScript
Stored Procedures
Data Intent
Language
Lock-free Consistent Updates,
Long-running Transactions
True
Cache
Copyright © 2024, Oracle and/or its affiliates

18. Oracle Database 23ai -Mission-Critical Data
RAFT Replication for
Globally Distributed Database
In-Database
Firewall
Real-Time SQL Plan
Management
RAC, Exadata, Data Guard
Simplicity and Scalability
Analytic SQL
Simplicity and Scalability
Priority
Transactions
Copyright © 2024, Oracle and/or its affiliates

19. Free| Oracle Database 23ai For Developers
Oracle Always Free ADB
Available on OCI
ADB Free Container Image
Available for download
Oracle Database Free
Available as RPM, Docker
Image, VBox VM
Copyright © 2024, Oracle and/or its affiliates | Confidential -Internal
Very Low Cost Supported DeveloperEdition of ADB, BaseDB, ExaDB-D, and ExaDB-C@C

20. Demos
20 Copyright © 2024, Oracle and/or its affiliates
Spring AI –several scenarios
Oracle Database 23ai
Oracle AI Vector Search
OpenAI
Ollama

21. Image Generation -Spring AI with Open AI
•Demo -Direct LLM calls –Image Generation
Copyright © 2024, Oracle and/or its affiliates

22. Chatbot -Spring AI with Open AI
•Demo -Direct LLM calls -Chatbot
Copyright © 2024, Oracle and/or its affiliates

23. RAG –Retrieval Augment Generation
Spring AI with Open AI and Oracle Database 23ai
•Demo –RAG
•Spring AI + SimpleVectorStore + Open AI
•Spring AI + OracleVectorStore + OpenAI + Oracle
Database 23ai
Copyright © 2024, Oracle and/or its affiliates

24. Multimodality -Spring AI with Open AI
•Demo -Multimodality -Spring AI with Open AI
Copyright © 2024, Oracle and/or its affiliates

25. Function Calling -Spring AI with Open AI
•Demo -Function Calling -Spring AI with Open AI
Copyright © 2024, Oracle and/or its affiliates

26. Copyright © 2024, Oracle and/or its affiliates
Technical References
•Spring AI -https://spring.io/projects/spring-ai
•OpenAI -https://openai.com/
•Oracle AI Vector Search -https://rb.gy/sp41uk
•Oracle AI Vector Search for Java Developers with the Oracle Database 23ai -
https://rb.gy/7ivwxr
•Java / JDBC –https://www.oracle.com/jdbc
•Introduction to Oracle JDBC Driver Support for Virtual Threads -https://bit.ly/3UlNJWP
•Developing an Oracle JDBC app with GraalVM Native Image -https://rb.gy/iy3sgh
•Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver -https://rb.gy/42dnz5
•Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC
•Introduction to JDBC Reactive Extensions with the Oracle Database —Developer
Release –https://rb.gy/qxlrbx
•Pipelined Database Operations -https://rb.gy/iy3sgh
•Spring AI -https://github.com/kousen/ https://github.com/ThomasVitale/llm-apps-
java-spring-ai

27. Juarez Junior
@juarezjunior

29. Java Developers
29 Copyright © 2024, Oracle and/or its affiliates
Oracle for DB Developers

30. Java App
Dev with
Oracle
Database
Copyright © 2024, Oracle and/or its affiliates

31. Overview of Oracle DB Access with Java
Copyright © 2024, Oracle and/or its affiliates

32. Oracle JDBC -Support for the Latest Java
Versions
•Support to the latest Java versions
(11, 17, 21) and GraalVM
•Project Loom -Virtual Threads
support
•Java Virtual Threads and Pipelined
Database Operations
•JDBC Standards -4.2 and 4.3
•Reactive Streams -Java Flow API
support
•R2DBC, Reactive Streams Ingestion
(RSI)
•Pipelined Database Operations
Data access is critical in
mission-critical applications!
Copyright © 2024, Oracle and/or its affiliates

33. Oracle LiveLabs
Showcasing how Oracle’s solutions can
solve yourbusiness problems
500+
free workshops,
available or in
development
3.5 million
people have already visited
LiveLabs
developer.oracle.com/livelabs
learn something new …at your pace!
600+
events run
using LiveLabs
workshops

34. 3 membership tiers
Connect: @oracleace facebook.com/[email protected]
500+ technical experts &
community leaders helping peers globally
The Oracle ACE Program recognizes & rewards individuals for
their technical & community contributions to the Oracle community
Nominate
yourself or a candidate:
ace.oracle.com/nominate
Learn more-ace.oracle.com
blogs.oracle.com/ace

35. Create your FREE
Cloud Account
•Go to
https://signup.cloud.oracle.com/
Copyright © 2024, Oracle and/or its affiliates