introduction_to_rag_report_eng-233956390.pdf

Intro to RAG by Uni Amsterdam

1. July 24
Retrieval Augmented
Generation (RAG)
Combining Information Retrieval and
Generative AI foraccurateand
contextuallyrelevant responses.
AI Lab
Urban Innovation and R&D
City of Amsterdam
Juli 2024

2. amsterdamintelligence
Introduction
to RAG
•What is RAG?
•How RAG Works
•RAG differs from LLMs
•Types of Data in RAG
What It Is and How It Works

3. 3
amsterdamintelligence
Language models often generate
incomplete, incorrect or outdated
answers.
What if we could help them out by
providing essential information upfront?

4. Whatis RAG?
Retrieval-Augmented Generation (RAG) is
a system that combinesthe strengths of
Information Retrievaland Generative AI
(e.g., LLMs)to generate accurate and
contextually relevant responses.
By grounding responsesin information
extracted from an external knowledge
base, RAG provides more accurate,
relevant, and customized answers.
This method contrasts with relying solely
on a LLM, leading to improved response
quality.

5. How RAG
Works
2.
3.1. 4.
1.UserAsksa Question:Example-
"WhoistheKing of The Netherlands?”
2.FindRelevantInformation:looksforthemo
strelevantdocumentchunks
thatmatchthequestioninthe
knowledgebase.
3.CombineQuestion and Information:
Thequestions andcollected document
chunksare combined and forwarded to the
Generative AI.
4.Generate anAnswer:Generative
AIprovidesa finalanswer based
oncombinedquestion and documents.

6. RAG Differs From
LLMs
LLM Framework
RAG Framework
RAG is different from LLMs. The key
differences are the addition of:
1.Aretrieverto retrieve relevant
document chunks to the question.
2.A knowledge base
wheredocuments are stored.
3.A framework that combines the
questionand relevantchunksto
obtain an answer.

7. amsterdamintelligence
8
Types of Data in RAG
RAG systems can be designed to manage question distillation, document
retrieval,and answer generationacross various data types(Multi-Modal RAG).
Data types that can be utilized include text(documents, articles, reports),
images(photos, diagrams), audio(recordings, speeches), videos (tutorials,
demonstrations), and others(tables, graphs, charts).
By leveraging this diversity, these systems deliver more preciseand
contextually relevantresults, thereby improving the user experiencein
interactive applications.

8. amsterdamintelligence
Benefits and
Use-Cases of
RAG
•Benefits of RAG
•Applications
•Municipal Applications
Why RAG is Useful and How it Can Be Used

9. 10
amsterdamintelligence
Benefits of
RAG
RAG addresses challenges such as outdated information,
lack of source transparency, and AI hallucinations.
•Provides current information.RAG pulls information
from relevant, reliable and up-to-date sources.
•Increases user trust.Users can access the model's
sources, which lets users verify its accuracy.
•Reduces AI hallucinations.Because LLMs are grounded
to external data, the model has less of a chance to make
up or return incorrect information.
•Synthesizes information.RAG synthesizes data by
combining relevant information from retrieval and
generative models to produce a response.
•Easier to train.Because RAG uses retrieved knowledge
sources, the need to train the LLM on a massive amount
of training data is reduced.
•Reduces computational and financial costs.
Organizations don't have to spend time and resources to
continuously train the model on new data.
•Can be used for multiple tasks.Aside from chatbots, RAG
can be adapted for a variety of specific use cases, such as
text summarization and dialogue systems.

10. 11
amsterdamintelligence
Applications
Conversational search
RAG could improve conversational agents
by retrieving relevant information from
external sources, enabling accurate,
context-aware responses. This can
enhance customer service chatbots or
virtual assistants with up-to-date answers,
expert knowledge, and personalized
interactions.
Insights from documents
Gather insights and recommendations by
using RAG to incorporating relevant
external information such as market
trends, industry benchmarks, or customer
feedback. This can particularly be helpful
in specific topics or changing conditions
where up-to-date information is key.

11. 12
amsterdamintelligence
Applications
Assist based on historical cases
RAG can support users by retrieving and
analyzingrelevant examples from
reference cases. It could provide insights,
feedback, or examples, allowing users to
speed up their workflow.
Content creation
Use RAG during content creation,
leveraging your brand’s specific materials
such as blogs, website copy, or internal
documents. This approach allows for
tailored content that targets specific
groups, aligns with your brand’s voice and
style, and ensures consistency across all
content.
Content recommendation
Based on specific context and user
preference, RAG can retrieve relevant
content and generate personalized
recommendations.

12. 13
amsterdamintelligence
Municipal Applications
Personalizedsupport in naturallanguage
Let citizens ask questions in natural
language about municipality information
and answer questions according to
personalized preferences in multiple
modalities like text (simplification) or
audio.
Permit/Subsidyassistance tool
Assists citizens in the application process
by providing step-by-step help, allowing
multimodal input, offering examples of
relevant approved requests (anonymized),
and giving feedback or rephrasing help to
ensure all proposal conditions are met.
Historicalcase reference tool
Help colleagues write responses by
referencing similar past cases. This could
speed up the workflow in contact centers,
permit handling, or objection processing
(bezwaren).

13. amsterdamintelligence
Risks and
Challenges
•Risks in RAG
•Challenges of RAG
Understanding the Risks and Difficulties

14. 15
amsterdamintelligence
Risks in RAG
The use of RAG presents several risks that can impact its
effectiveness and reliability.
•Limited database coverage.If a RAG database is missing
information, the model may not include essential details,
leading to incomplete or inaccurate responses.
•Issues with retrieving crucial documents.If there are
problems with how data is indexed or retrieved,
important documents may be overlooked, compromising
the quality of responses.
•Inaccuracies and biases in retrieved data.The data RAG
uses can be incorrect or biased, which may result in
misleading or biased responses.
•Risk of generating incorrect or sensitive
information.RAG can generate incorrect information or
inadvertently disclose sensitive details, posing risks of
misinformation and privacy breaches.
•Dependency on external sources.RAG depends on
external databases for information. If these sources are
unavailable, outdated, or compromised, it can affect the
model’s reliability and accuracy.

15. 16
amsterdamintelligence
Challenges in
RAG
RAG faces several challenges that can limit its efficiency and
adaptability.
•Integration of diverse information.Combining different
pieces of information smoothly can be challenging,
especially when trying to maintain coherence and
context in responses generated from disparate data
sources.
•System complexity.Designing and maintaining a RAG
system can be complex due to the need to balance
retrieval effectiveness with generative accuracy, requiring
ongoing technical expertise and resources.
•Prioritization of information.It can be difficult to
determine what information is most relevant or
important, potentially leading to overloaded or
unfocused responses.
•Creativity limitations.Heavy reliance on retrieved
documents can limit the system’s ability to generate
creative or innovative responses, as the model may
default to reproducing existing information rather than
generating new insights.
•Handling diverse data formats.Integrating and managing
information from various data formats and sources poses
significant challenges, as inconsistencies and
compatibility issues can affect the quality and reliability
of the outputs.

16. amsterdamintelligence
Future Prospects
and Ethical
Considerations
•Future Prospects
•Ethical Considerations
Looking Ahead and Adressing Ethical Concerns

17. 18
amsterdamintelligence
Future
Prospects
To further enhance its capabilities, RAG can benefit from
advancements in integration, robustness, and multi-
modality.
•Enhancing robustness and scalability.Making RAG more
robust involves improving its ability to handle large
volumes of queries or context lengths without
performance degradation. Additionally, scaling the
system to accommodate more users or larger datasets is
crucial for widespread adoption.
•Integrating with other techniques.Combining RAG with
other AI techniques, such as machine learning models or
deep learning frameworks, can potentially yield better
results by enhancing the accuracy and relevance of the
responses.
•Further Development of Multi-Modal RAG.Further
development of Multi-Modal RAG, which incorporates
various types of data such as text, images, and audio, can
enhance its utility. This approach allows RAG to provide
richer, more contextually appropriate responses by
processing and integrating multiple forms of information.

18. 19
amsterdamintelligence
Ethical
Considerations
RAG systems raise several ethical concerns that can impact
their fairness, transparency, and societal implications.
•Bias in Data and Unfair Outcomes.Biases in training data
can result in AI making unfair or discriminatory decisions,
perpetuating existing inequalities.
•Risk of Exposing Sensitive Information.handling of large
datasets increases the risk of unintentionally exposing
personal or sensitive information.
•High Energy Usage and Environmental Impact.The
substantial computational power required for AI can lead
to high energy consumption, adversely affecting the
environment.
Opacity of AI Decisions.AI models can be opaque, making it
difficult to understand or explain how decisions are reached,
which challenges accountability.
Exploitation and Ghost Work.AI systems can exploit labor,
with human workers hidden behind automated processes.
Job Threats.Automation threatens jobs, potentially leading
to redundancy, devaluing human work, and reducing service
quality.
Concentration of Power in Big Tech.The development and
deployment of AI are often concentrated within large tech
companies, raising concerns about the concentration of
power and control over data and decision-making.

19. amsterdamintelligence
Summary
•Summary
Summarizing Key Points

20. 21
amsterdamintelligence
Summary
Definition andPurpose.RAG isa system that combines the
strengthsofInformationRetrievalandGenerativeAItogenerateaccu
rateandcontextuallyrelevantresponses.by grounding them in
external knowledge bases for more accurate, relevant, and
customized answers.
How it Works.RAG retrieves relevant document chunks when a user
asks a question, combines them with the question, and generates a
final answer using Generative AI.
Types of Data.It can be designed to manage more data types besides
text, including images and audio, enabling precise and contextually
relevant results for multiple applications.
Benefits.RAG provides current, reliable information, increases user
trust through source transparency, reduces AI hallucinations, and is
easier and cheaper to train than LLMs.
Applications.RAG is versatile for tasks such as gathering insights
from documents, conversational search and content creation.
Risks and Challenges.Risks include limited database coverage,
retrieval issues, data biases, and the generation of incorrect or
sensitive information. Challenges involve integrating diverse
information, maintaining system complexity, prioritizing relevant
information, balancing creativity with retrieved data, and managing
various data formats.
Future Prospects.Enhancing robustness and scalability, integrating
with other AI techniques, and further developing Multi-Modal RAG
for richer, contextually appropriate response are among future
prospects.
Ethical Considerations.Some important ethical considerations are
addressing biases, protecting sensitive information, reducing
environmental impact, improving transparency, mitigating
exploitation and job threats, and avoiding concentration of power in
big tech companies.

21. July 24
Website
www.amsterdamintelligence.com
Email
[email protected]
Juli 2024