Next Generation Apps: Enhancing User Experience with LLMs.pdf

This talk was delivered at PyCon Portugal 2024 (https://2024.pycon.pt/home/).

Large Language Models(LLMs) are good at reasoning based on their knowledge. This talk explores how you can use the power of LLMs to add intelligence like coding assistants, text-to-sequel, etc to existing applications.

One of the simplest ways to start adding intelligence is by using an LLM with fine-tuned prompts. You can find the answers to questions like: - What are some of the things that you need to consider while prompt engineering? - What are the limits of prompt engineering?

After finding out the limits of prompt engineering, let us understand how to augment the knowledge of the LLM using vector databases. You can learn things like: - Ingesting the data into the vector databases. - Considerations in data ingestion to improve the LLM performance.

We will also cover the concept of AI agents that given a set of capabilities or tools can figure out how to use them where relevant in an intelligent fashion. You can learn - How do agents work? - Where are they useful?

After this talk, you will learn how to add intelligence to existing applications with the help of the ever-popular LLMs using open-source frameworks.

1. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved.
Next Generation Apps:
Enhancing UX with LLMs
Nithish Raghunandanan
Developer Advocate
PyCon Portugal 2024

2. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 2
About Me
●Background in Data/ML Engineering
●Developer Advocate @ Couchbase
●Building Integrations with Developer Ecosystem
●Live in Munich, Germany
●Love Olympic Sports


@nithishr

3. 3
Agend
a
●Motivation
●Guiding LLMs: Prompt
Engineering
●RAG: Grounding LLMs
●Dynamic Agents
●Conclusion

4. 4
Motivation

5. 5
What are Large Language Models?
●Models trained on huge corpus of data
●Extremely good at working with text
●Examples: GPT4, Llama3, Claude, Gemini, etc

6. 6
LLMs are Powerful for certain use cases
●Chatbots
●Summarizing content
●Classifying content
●Language Translation
●Image Generation

7. 7
LLMs are Powerful for certain use cases

8. 8
And they have their Pitfalls

9. 9
Guiding LLMs:
Prompt Engineering

10. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 10
Guiding the LLM to solve tasks

●Specific instructions
●Output structuring
●Zero Shot vs Few Shot Prompting
●Review of Prompting Techniques:
https://arxiv.org/pdf/2406.06608
Prompt Engineering

11. 11
Demo
Prompt Engineering

12. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 12
Limitations
●Knowledge limitations: Hallucinations
●Prone to hijacks
Recommendations
●Provide context
●Concise prompts


Prompt Engineering

13. 13
RAG: Grounding
LLMs with Vector
Search

14. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 14
What is Retrieval-Augmented Generation (RAG)?
Supplying an existing LLM with relevant context
Question
Retriever
(Database)
Knowledge
external to the
LLM
LLM
(GPT, Llama3,
Gemini, etc)
Answer
Retrieval
(your enterprise, data, application)
Generation
(3
rd
party LLM)
Augmentation

15. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 15
RAG Workflow

16. 16
Demo
Basic RAG Example

17. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 17
●Ingestion of Documents
○Chunk Data
○Embedding Data using Embedding Models
●Retrieval of Documents
○Vector Search
○Reranker
●Generation of Answers
○Use the ranked documents as Context
Components of a RAG

18. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 18
●Support Follow up questions
●Get Source documents
●Observability
●Code:
https://github.com/couchbase-examples/
qa-bot-demo
Chat with Docs using RAG

19. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 19
RAG
Limitations
●Passive: Only retrieve data
Recommendations
●Measure the performance of RAGs
○Human evaluations
○LLM based evaluations (Ragas)

20. 20
Dynamic Agents
powered by LLMs

21. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 21
●LLMs can interact with the environment
●LLMs augmented with Tools
●ReAct (Reason + Act) Agents
○Reason about the task
○Act based on Reasoning
○Call tools as needed
Agent

22. 22
Demo
SQL++ Generator

23. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 23
●Generate SQL++ queries for Couchbase from
natural language

Text to SQL++ Generation

24. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 24
Limitations
●Complex workflows might be hard
●Might need human intervention
Recommendations
●Combine Agents with RAGs
●Check combination of Prompts & LLMs


Agents

25. 25
Conclusion

26. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2024. All rights reserved. 26
●LLMs can be used to improve UX for end users.
●Privacy Considerations
●Rapidly advancing field & models.


Key Takeaways

27. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2023. All rights reserved. 27
●Q&A Chatbot
○https://github.com/couchbase-examples/qa-bot-demo
●Chat with PDF
○Python: https://github.com/couchbase-examples/rag-demo
○Node.JS: https://github.com/couchbase-examples/vector-search-nodejs
●SQL++ Agent
○https://github.com/nithishr/sql_plus_plus_agent
Resources
Check out demos

28. 28
Thank you!
[email protected]
https://linkedin.com/in/nithishr
@nithishr
Use
Capella
for Free!

29. 29
Q&A