Neo4j and Generative AI: New Frontiers in Data Intelligence

Neo4j and Generative AI: New Frontiers in Data Intelligence

1. Neo4j and Generative AI:
New Frontiers in Data
Intelligence
Malaysia GraphTalk - Monday 14th October
Bryan Lee, Professional Services Architect

2. 2Neo4j Inc. All rights reserved 2024
●Located in Singapore

●Neo4j Professional
Services Team

●Helping customers
achieve success
throughout their Neo4j
journey

3. Agenda
●Current State of GenAI
●Why RAG?
●Can Knowledge Graph help?
●Why GraphRAG? Why RAG?
●GenAI Use Cases
●LLM Graph Builder Demo
Neo4j Inc. All rights reserved 20243

4. The State of Generative AI
Neo4j Inc. All rights reserved 20244

5. The Good ??????
The State of Generative AI
Neo4j Inc. All rights reserved 20245

6. 6

AI
Breakthroughs
Personalization at Scale
Enhance Decision-Making
Process Optimization
Boost Creativity
Predictive Analytics
Improve Data Retrieval
and Automation
Generate Content and Code
Improve Customer Experiences
Neo4j Inc. All rights reserved 2024

7. The Good ??????
The State of Generative AI
7Neo4j Inc. All rights reserved 2024

8. The Good ??????
The State of Generative AI
The Bad ??????
8Neo4j Inc. All rights reserved 2024

9. The Good ??????
The State of Generative AI
The Bad ??????
9

10. The Good ?????? The Bad ??????
The State of Generative AI
The Ugly ??????
10Neo4j Inc. All rights reserved 2024

11. The State of Generative AI
The Good ?????? The Bad ?????? The Ugly ??????
GenAI Alone != Right Outcomes ??????
11Neo4j Inc. All rights reserved 2024

12. Challenges with GenAI: Stochastic Parrot?
●Lack of enterprise domain knowledge
●Inability to verify answers
●Hallucination
●Ethical and data bias concerns
●and more
12Neo4j Inc. All rights reserved 2024
GenAIPARROT

13. 13

Managing AI risk
is the biggest
barrier to scaling
AI initiatives
1

Skepticism: Over half of business leaders are
skeptical in adopting GenAI.
2
Neo4j Inc. All rights reserved 2024
Explainability: Over 80% of executives worry
about non-transparent nature of GenAI could
result in poor or unlawful decisions.
2

Reliability: Inaccuracy and hallucination are two
of the most-cited risks of adopting GenAI
technology at all levels of an organisation.
3

1. Deloitte’s State of AI in the Enterprise 2. BCG’s Digital Acceleration Index Study 2023 3. McKinsey: The state of AI in 2023

14. 14Neo4j Inc. All rights reserved 2024
How can organizations use
domain-specific knowledge
to rapidly build accurate,
contextual, and explainable
GenAI applications?
The Big
Question

15. Why RAG?
And what is it anyway…
15Neo4j Inc. All rights reserved 2024

16. Retrieval Augmented Generation:

The ability to dynamically query a large
text corpus to incorporate relevant factual
knowledge into the responses generated
by the underlying language model
16Neo4j Inc. All rights reserved 2024

17. Retrieval-Augmented Generation Is Becoming
an Industry Standard
RAG augments LLMs by retrieving
up-to-date, contextual external data
to inform responses:

●Reduce hallucinations with
verified data
●
Provide domain-specific,
relevant responses
●
Enable traceability back
to sources

Retrieval Augmented Generation
Database of Truth
17Neo4j Inc. All rights reserved 2024

18. Why RAG With Vector Databases Fall Short


1
3
2
4
Similarity is insufficient for rich enterprise reasoning
Only leverage a fraction of
your data: Beyond simple
“metadata”, vector databases
alone fail to capture relationships
from structured data
Miss critical context: Struggle to
capture connections across
nuanced facts, making it
challenging to answer multi-step,
domain-specific, questions
Vector Similarity ≠ Relevance:
Vector search uses an incomplete
measure of similarity. Relying on it
solely can result in irrelevant and
duplicative results
Lack explainability:
The black-box nature of
vectors lacks transparency
and explainability
18Neo4j Inc. All rights reserved 2024

19. Why GraphRAG? Why RAG?
And what is it anyway…
19Neo4j Inc. All rights reserved 2024

20. GraphRAG
Technique for richly
understanding text datasets
by combining text extraction,
network analysis, LLM
prompting and summarization
into a single end-to-end
system
Neo4j Inc. All rights reserved 202420

21. Neo4j Inc. All rights reserved 202421
RAG with Neo4j
Find similar documents
and content
Identify entities associated
to content and patterns
in connected data
Improve GenAI inferences
and insights. Discover new
relationships and entities
Unify vector search, knowledge graph and data science
capabilities to improve RAG quality and effectiveness
Vector Search
Graph Data
Science
Knowledge
Graph

22. Elevate relevance with domain
context and inferences
Accelerate GenAI
application development
Build your next
GenAI breakthrough
with context and
deep explainability
Explain and improve
GenAI applications
Neo4j is the GenAI Enabler
22Neo4j Inc. All rights reserved 2024

23. Elevate Relevance with Domain
Context and Inferences
Improve context with facts from a knowledge graph
Enhance personalization and quality
using graph pattern matching
Expand insights and inferences using
graph data science and machine learning
23Neo4j Inc. All rights reserved 2024

24. 24Neo4j Inc. All rights reserved 2024
Improve Context
with Facts from a
Knowledge Graph
Combine native vector search with
multi-hop graph traversals to add
domain-specific context from a
knowledge graph
Incorporate hybrid retrieval, including
text search and lookups based off date,
numeric, and geopoint indexes

25. 25Neo4j Inc. All rights reserved 2024
LLM
Application
User
Neo4j Database
5. Answer provided
1. Asks question
2. Question is translated to
a Cypher statement
4. Result from database is
converted to natural language
3. Generated
Cypher is used
to query Neo4j
database
Improve Context
with Facts from a
Knowledge Graph
Combine native vector search with
multi-hop graph traversals to add
domain-specific context from a
knowledge graph
Convert user questions to Cypher
queries to perform RAG with explicit
query logic
Incorporate hybrid retrieval, including
text search and lookups based off date,
numeric, and geopoint indexes

26. 26Neo4j Inc. All rights reserved 2024
Improve Context
with Facts from a
Knowledge Graph
Combine native vector search with
multi-hop graph traversals to add
domain-specific context from a
knowledge graph
Convert user questions to Cypher
queries to perform RAG with explicit
query logic
Incorporate hybrid retrieval, including
text search and lookups based off date,
numeric, and geopoint indexes
Unified Copilot Service Layer
Including Text2Cypher
data
importer
Customer
applications

27. 27Neo4j Inc. All rights reserved 2024
Use graph patterns to rank and
score vector search results
based on domain knowledge
Retrieve more refined results
to power better personalized
LLM Responses
from langchain.vectorstores.neo4j_vector import
Neo4jVector

kg_personalized_search =
Neo4jVector.from_existing_index(
embedding=embedding_model,

url=NEO4J_URI,username=NEO4J_USERNAME,password=NEO4J_PA
SSWORD,
index_name='product-text-embeddings' ,
retrieval_query=f"""
WITH node AS product, score AS vectorSearchScore
OPTIONAL MATCH
(product)?[:VARIANT_OF]-(:Article)?[:PURCHASED]-

(:Customer)-[:PURCHASED]?l(a:Article)?[:PURCHASED]-
(:Customer {{customerId: '{CUSTOMER_ID}'}})
WITH count(a) AS purchaseScore,??w
RETURN ??w
""")


Enhance
Personalization
and Quality

28. Expand Insights and
Inferences Using
Data Science and ML
Neo4j Inc. All rights reserved 202428
Quickly incorporate context from
structured data in RAG vector search
using Neo4j graph embeddings with a
native vector index
Deeper insights for AI by enriching
your knowledge graph with link
prediction, community detection, and
classification using graph algorithms
and machine learning
Largest Catalog of Graph
Algorithms
Native Graph
Analytics Workspace
Graph Embeddings for
Enhanced Retrieval

29. 29
Explain and Improve GenAI Applications
Easily trace sources and explain retrieval logic
Understand and explain patterns in AI grounding
Improve GenAI grounding data at scale
Neo4j Inc. All rights reserved 2024

30. Easily Trace
Sources
Neo4j Inc. All rights reserved 202330
Add citation metadata
to nodes and relationships
within the knowledge graph
Enable source citation in
LLM responses by
integrating with
frameworks like LangChain
and LlamaIndex

31. Explain
Retrieval Logic
Neo4j Inc. All rights reserved 202431
Inspect and explain retrieval
logic by converting user
questions to explicit Cypher
queries that are human-
readable and traceable

32. 32Neo4j Inc. All rights reserved 2024
Understand and Explain
Patterns in AI Grounding

Easily understand grounding data
by representing it in a Neo4j knowledge
graph - both a human-friendly and
machine/GenAI-readable format
Introspect and visualize connections
between data using Cypher queries
and low-code/no-code interfaces like
Neo4j Bloom
Analyze patterns in grounding data
using graph algorithms like KNN,
community detection and centrality to
understand semantic similarities and
connections between sources

33. 33Neo4j Inc. All rights reserved 2024
Improve GenAI
Grounding Data at Scale

Improve data quality using Cypher and
graph algorithms for duplicate
identification, outlier detection, and
revealing trends and biases
Assess LLM performance and usage
patterns to identify areas of improvement
– analyze the connections between
prompts, responses, and grounding data
using Cypher and graph algorithms

34. 34
Power your AI applications with robust and dynamic data representation
Rapidly build a knowledge graph for AI use cases
Integrate with the GenAI ecosystem
Accelerate GenAI Application Development
34Neo4j Inc. All rights reserved 2024

35. Empower your AI projects
with Robust and Dynamic
Data Representation
35Neo4j Inc. All rights reserved 2024
Harness all your data in one place with a
property graph data model, including
vector storage and search capabilities
Future-proof with an adaptable,
dev-friendly schema to add new data
types and entities, while maintaining data
quality
Enable real-time data ingestion into Neo4j
for immediate use in GenAI apps, with
streaming support via our Kafka Connector

●
Car

●
Person Person
Name: “Andre”
Born: May 29, 1970
Twitter: “@dan”
Name: “Mica”
Born: Dec 5, 1975
Car
Brand “Volvo”
Model: “V70”
Description: “An executive car manufactured and…”
DescEmbedding: [0.1, -0.3, 0.4, …, -0.7]
DescSource:”https://en.wikipedia.org/wiki/Volvo_V70”
Since:
Jan 10, 2011
LOVES
KNOWS
KNOWS
LIVES WITH
OWNS
DRIVES

36. Rapidly Build a
Knowledge Graph
for AI Use Cases
36Neo4j Inc. All rights reserved 2024
Car
Car
Jumpstart knowledge graph
creation from unstructured
data with named entity
recognition
Rapidly model structured data
as nodes and relationships in a
knowledge graph
Ingest text, audio, and other
content embeddings as node
vector properties and index
them for use in RAG
Embedding (Unstructured)
Structured
Unstructured
Data Sources
Named Entity Recognition
Structured Data
Generative AI Models
Embedding Models
Knowledge Graph

37. Integrate with the
GenAI Ecosystem
37Neo4j Inc. All rights reserved 2024

GenAI Stack
Application

Generative AI & Embedding Models

Orchestration

Grounding Knowledge Graph

Neo4jGraph
Neo4jVector
GraphCypherQAChain
Neo4jGraphStore
Neo4jVectorStore
KnowledgeGraphIndex


Neo4j GenAI Integrations
Text | Chat | Embedding
NL Query | Image Gen
Neo4j Drivers
JavaPython JavaScript
Call LLM APIs natively via
Cypher using our GenAI
procedures or open-source
APOC library
Integrate Neo4j with leading LLM
open-source frameworks such as
LangChain and LlamaIndex
Agnostic LLM orchestration
connecting graphs to OpenAI,
AWS Bedrock, GCP Vertex AI,
Azure, Anthropic, Hugging Face,
and other proprietary and open
source foundation models

38. Neo4j Inc. All rights reserved 202438
Summary
Elevate relevance with domain
context and inferences
Accelerate GenAI application
development
Explain and improve GenAI
applications

39. GenAI - Use Cases

40. Neo4j Inc. All rights reserved 202440
Challenge: Global Retailer wants to retain and
grow talent, but finding the right new opportunity
is in the hands of the associate. This is a time
consuming and often limiting approach because
the current system forces the associate to search
and browse roles without visibility into all the
options and without knowledge about how roles
and skills come together

Solution: Build a career opportunity
recommendation system based on where
associates live, work today, their skills,
and past work experience

Impact: Engage associates at all levels of their
career path and guide them to their best next
position. Proactively recruit candidates for
promotion based on their attributes
Roles
(Email, Custom Cards)
Associate
Global Retail Giant uses LLM with Knowledge Graph
to Recommend the Best Opportunities

41. Neo4j Inc. All rights reserved 202441
Investment Bank Summarizes Quarterly Earnings
Reports Across Companies
Challenge: Investment bankers need a
quick way to understand the detail of
quarterly reporting across all the
companies in their portfolios, but data is
stored in multiple formats and locations

Solution: Use vector embeddings to look
for the most relevant information, while
the Neo4j knowledge graph adds the
exact match for financial statements

Impact: Quickly review quarterly reports
across companies
Investment Banker
Prompt
LLM
Embeddings API
Vector Search in
Vector Index
Embedding
Relevant Results /
Documents
Knowledge graph of
financial statements
LLM
Chat API
Response

42. Neo4j Inc. All rights reserved 202342
Pharmaceutical Company Democratize Access to
Supply Chain Risk Analysis
Challenge: Supply Chain risk analysis
knowledge graph was accessible only by 300
experts familiar with graph analysis. They
wanted to open up these insights to
thousands more employees

Solution: Conversational AI solution to allow
natural language access to supply chain KG.
Any employee can ask a question in plain
english. It gets converted into Cypher query
that can pull relevant risk assessment data
from the graph. The query results get passed
back into natural language generator to
create an an easy to understand response to
the initial question

Impact: Democratize access to supply chain
risk analysis
Supply Chain
Knowledge
Graph
Prompt (Description of
Desired Products)
Cypher Query
Relevant Results
Employee
LLM API
LLM API
Response

43. Neo4j Inc. All rights reserved 202343
Government Procurement Entity Automates
Complex Processes

Challenge: Government entities can overspend
or acquire goods and services redundantly
because the volume of RFPs makes reviewing
each of them resource prohibitive

Solution: Use LLM to read the nature of RFP
and classify it accordingly. Compare the new
RFP against the knowledge graph of active
and historic RFPs and associated spend.
Recommend opportunities for consolidation or
terms negotiation with suppliers

Impact: Save tax dollars for other important
projects
Predefined Prompt with
Results
LLM API
Generates personalized
text
User is Emailed with
Recommendation
Procurement Officer
Supplier
LLM API Categorizes
RFP
Enterprise App
Ingests New RFP
Knowledge
graph of RFPs
and spending
patterns

44. 44Neo4j Inc. All rights reserved 2024
FREE to use and test:
https://llm-graph-builder.neo4jlabs.com/

Github:
https://github.com/neo4j-labs/llm-graph-
builder
LLM Graph
Builder

45. Demo!

46. Thank you