langchain4j-cdi: Infuse your Jakarta and MicroProfile applications with all the AI

Generative AI burst on to the public scene in November 2022, over ten years after Andrew Ng’s Machine Learning premiered on Coursera. Java developers have long felt like second class citizens, compared to the cool Python kids. LangChain4J changes the game. Java is cool with AI now! This lecture from 30-year industry veteran Ed Burns explores why it took so long for Java developers to have access to easy-to-use AI libraries, compared to Python developers. This session introduces the new langchain4j-cdi repository within the LangChain4j GitHub organization. Combine the power of Jakarta EE with the reach of LangChain4j. All the idioms you know and love from Jakarta now work with all the AI exposed by LangChain4j: injection, validation, persistence, REST and more!

1. Session code: 2788 Slide at https://aka.ms/txc-2025 Ed Burns Principal Architect for Java, Microsoft Emily Jiang IBM, STSM, Cloud Native Architect and Advocate langchain4j-cdi: Infuse your Jakarta and MicroProfile applications with all the AI

2. Ed Burns professional biography Client NCSA Mosaic (1994) SGI Cosmo Web Authoring Sun Netscape 6 OJI Server J2EE JSF (2002) Oracle Java EE Servlet, JSF, Bean Validation, etc. Cloud Java Champion Jakarta EE 11 committer and release coordinator Langchain4j-cdi committer Books

3. Emily Jiang Java Champion MicroProfile Technical Lead MicroProfile Spec author Jakarta EE committer Langchain4j-cdi committer Book Author Fellowship BCS TechWomen100

4. Agenda 01 02 03 04 05 Background on Jakarta EE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation E

5. Agenda 01 02 03 04 05 Background on Jakarta EE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation E

6. Java SE Activation Java Products and Specifications Tomcat Jetty Undertow Spring Framework Spring Boot WebSocket Concurrency JSON-B Validation Persistence Messaging Transactions Messaging Servlet Pages Tag Libraries Expression Language Annotations Authorization Faces REST JSON-P Interceptors Dependency Injection CDI Jakarta EE Core Profile Batch Connectors Mail EJB Security EJB Lite Authentication Data Debugging Support WebLogic Server WebSphere Liberty Open Liberty JBoss EAP WildFly WebSphere Traditional Quarkus Micronaut Helidon Jakarta EE Platform and Web Profile (aka Java EE/J2EE JEE) A note on Spring and Jakarta EE “Spring /Tomcat profile” (not an official name) “Spring /Tomcat profile” (not an official name) Config Fault Tolerance Metrics Health JWT RBAC Open API REST Client Telemetry MicroProfile Not Cloud Native Older Cloud Native Newer Cloud Native E

7. New to AI? E

8. Don’t panic! E

9. Ed’s fairly obvious observations Recognize the market forces that have always been in play. Vendors want adoption. They’ll make it easy to adopt. This is also true with AI. E

10. James Gosling: I realized if I wanted bankers to use automatic garbage collection, I needed to give them curly braces rather than lots of irritating silly parentheses (LISP). Packaging a new idea for adoption: deliver it on the back of existing ideas. E

11. Agenda 01 02 03 04 05 Background on Jakarta EE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation J

12. AI-enhanced Java applications Why Java? Leverage skills you and your teams already have Scalability, Security, Performance Deeply integrate AI into the logic and flow of your apps AI-enhanced applications Let users ask questions in natural language about documents, policies, etc. Translate content between languages dynamically Summarize meetings or transcripts Answer user-provided information queries Create specialized interactive research assistants J

13. Create an AI-enhanced Java App Simple LLM use in apps Use an LLM to interact with provided input Swap LLMs by changing configuration and/or dependencies Structured Output Directly integrate LLM responses with code lib Natural Language UI LLM J

14. Create an AI-enhanced Java App Retrieval Augmented Generation (RAG) Use Retrieval Augmented Generation pattern to extend what LLM knows lib Embedding Computation Docs Natural Language UI LLM Splitting Embedding Computation Vector Store J

15. What about AI Agents? E

16. Essential Component of AI Agents LLM Large language model that can comprehend and generate human language Actions and Tools AI Agents may require access to external tools, datasets, APIs, and automation systems to complete tasks. Tool calling enables LLMs to interface with structured tools, such as granting the model access to information beyond the data used in training. Memory Without memory, AI agents cannot remember the previous conversations. Session based context memory to memorize chat conversations. Structured knowledge bases, vector embeddings databases for long-term memory and retrieval of historical data. 16 E

17. Tool Tool Create an AI-enhanced Java App Java Methods as Tools Let LLM access tools lib Tool Natural Language UI LLM E

18. Build AI Agents using Langchain4j J

19. Agenda 01 02 03 05 06 Background on Jakarta EE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation E J

20. Langchain4j Initially, inspired by Langchain A Java library to make interactions with AI models and LLMs Provides a unified API to avoid the need for learning and implements specific APIs for each of them Comprehensive toolbox - tools ranging from low-level prompt templating, chat memory management, and function calling to high-level patterns like Agents and RAG Easy interaction with LLMs and Vector Stores https://github.com/langchain4j/ J

21. Langchain4j Architecture https://docs.langchain4j.dev/intro Low level: access to all the low-level components such as ChatModel , UserMessage , AiMessage , EmbeddingStore , Embedding, etc High level: interact with LLMs using high-level APIs like AI Services , which hides all the complexity and boilerplate from you J

22. Langchain4j AIService https://www.sivalabs.in/langchain4j-ai-services-tutorial/ Create an interface without an implementation Use AIServices to retrieve an object of the interface J

23. Improvement How to easily configure with LLMs? How to make the calling LLMs resilient? How to add telemetry? https://github.com/langchain4j/ J

24. Introducing Jakarta EE with AI via Langchain4j-cdi 24 E

25. Langchain4j-cdi with Jakarta EE Declare the AIService via @ RegisterAIService (CDI stereotype). Inject the AIService in other CDI beans via @Inject. Jakarta EE 10 and higher, JDK 17 support and higher. https://github.com/langchain4j/langchainej-cdi E

26. Langchain4j-cdi with MicroProfile Configuration (MicroProfile Config) • Low-level components are supported: ChatModel , ChatMesage , ChatMemory , EmbeddingStore , Embedding, tools https://github.com/langchain4j/langchainej-cdi E

27. Langchain4j-cdi with MicroProfile Fault Tolerance (MicroProfile Fault Tolerance) • @Timeout, @Fallback, @Retry, @ CircuitBreaker , etc. Observability (MicroProfile Telemetry) • OpenTelemetry AI Spans • OpenTelemetry AI Metrics https://github.com/langchain4j/langchain4j-cdi E

28. Demo https://github.com/langchain4j/langchain4j-cdi/tree/main/examples/liberty-car-booking J

29. Agenda 01 02 03 04 05 Background on Jakarta EE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation E

30. Another approach, with MCP E

31. But what is MCP? E

32. But what is MCP? https://modelcontextprotocol.io/ E

33. But what is MCP? https://github.com/modelcontextprotocol/servers Model Context Protocol (MCP) is an open-source standard for connecting AI applications to external systems. Using MCP, an AI enabled application can access tools and data from an MCP server. Not bound by LLM’s training cutoff. An MCP server can be local, communicating over stdio *, or remote, communicating over Streamable -HTTP. MCP servers are composable. E

34. MCP Architecture MCP Host – an AI app that uses MCP Clients MCP Client – a client to connect to an MCP Server MCP Server – a server that acts as a tool registry and can invoke tools (and other MCP things…) MCP Server LLM AI App / MCP Host JSON-RPC Streaming HTTP MCP Client C1 MCP Client C2 MCP Client C3 MCP Tool T1 MCP Tool T2 Service S1 Service S2 Service S3 REST MCP Tools should be built to return only the data needed (LLM performance is best with smaller responses) MCP Tools should be built to return data in a format that is LLM-friendly (JSON is fine, or yaml , or simple text). Structured output can also be used. MCP Tools should be built to combine common sets of actions into one call (reduces the demand on the LLM to get multiple calls right) J

35. Demo https://github.com/mbroz2/liberty-mcp-server J

36. Useful resources LangChain4j https://github.com/langchain4j/langchain4j https://docs.langchain4j.dev/intro https://docs.langchain4j.dev/category/tutorials LangChain4j-cdi https://github.com/langchain4j/langchain4j-cdi Quarkus / LangChain4j https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html Quarkus AI tutorial https://redhat-developer-demos.github.io/quarkus-tutorial/quarkus-tutorial/17_ai_intro.html MCP for beginners https://github.com/microsoft/mcp-for-beginners/ Full Java GenAI app workshop https://quarkus.io/quarkus-workshop-langchain4j E

37. Agenda 01 02 03 04 05 Background on Jakarta EE and AI Java applications with AI and AI agents Langchain4j and Langchain4j-cdi MCP server Q&A IBM TechXchange | © 2025 IBM Corporation

38. Q&A Ed Burns Principal Architect for Java, Microsoft 38 IBM TechXchange | © 2025 IBM Corporation Emily Jiang IBM, STSM, Cloud Native Architect and Advocate emijiang @uk.ibm.com

39. Thank you Emily Jiang IBM, STSM, Cloud Native Architect and Advocate [email protected] Ed Burns Principal Architect for Java, Microsoft

40. Notices and disclaimers IBM, the IBM logo, and ibm.com are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at: www.ibm.com/legal/copytrade.shtml . Certain comments made in this presentation may be characterized as forward looking under the Private Securities Litigation Reform Act of 1995. Forward-looking statements are based on the company’s current assumptions regarding future business and financial performance. Those statements by their nature address matters that are uncertain to different degrees and involve a number of factors that could cause actual results to differ materially. Additional information concerning these factors is contained in the Company’s filings with the SEC. Copies are available from the SEC, from the IBM website, or from IBM Investor Relations. Any forward-looking statement made during this presentation speaks only as of the date on which it is made. The company assumes no obligation to update or revise any forward-looking statements except as required by law; these charts and the associated remarks and comments are integrally related and are intended to be presented and understood together. © 2025 International Business Machines Corporation. All rights reserved. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. Case studies and client examples are presented as illustrations of how customers or IBM has used IBM products in production or test environments and the results they may have observed. Actual performance, cost, savings or other results in other operating environments may vary.  Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. Not all offerings are available in every country in which IBM operates. Any statements regarding IBM’s future direction, intent or product plans are subject to change or withdrawal without notice. IBM TechXchange | © 2025 IBM Corporation 40