Graph Storage: 10x faster & 100x cheaper

Talk from Jakob Pörschmann, AI/ML Customer Engineer, Google Cloud

1. Jakob Pörschmann - October 2024
Graph Storage:
10x faster, 100x cheaper
Building Beyond the Buzz V
??????????????????

2. Proprietary + ConfidentialProprietary + Confidential
Representing data as graphs uncovers
unprecedented insights.

3. Proprietary + Confidential
Extractive (local) questions require different
retrieval than aggregative (global) questions.
Text Embed. RAG
Doc
Chunk
Chunk
Chunk
Vector DB
[0.2, -0.8, …]
[0.5, -0.2, …]
[-0.7, -0.2, …]

User Query [0.1, -0.7, …]
Find Similar
Given some context:
[0.2, -0.8, …]

What is the answer to:
User Query

Original Doc chunks as final answer context.

Great at extractive answers.
Graph RAG
Doc
Entities
Edges
Graph DB
User Query of
“topic 2”
Given only relevant comm. reports:
Community 2

What is the answer to:
User Query

Community
“Topic 1”
Community
“Topic 2”
Intermediate answer to
user query based on all
community reports
Aggregated topics across docs as final context.

Great at aggregation answers.

4. Proprietary + Confidential
Where am I going to store this graph stuff…?
Graph DB hesitation because:
Learning Query lang. (Cypher/Gremlin)
Expanding tech stack
No decent serverless option

5. Proprietary + Confidential
Graphs are just a
special form of
semi-structured data

6. Proprietary + Confidential
If we only had DBs…

… with flexible schema.

… great at read heavy workloads.

… efficient to filter on attribute level.

7. Proprietary + Confidential

8. Proprietary + Confidential
Graphs can be stored in a regular NoSQL
DB with the right data model.
graph2nosql
on Github

9. Proprietary + Confidential
Wikidata5m

10. Proprietary + Confidential
Pricing: Aura (Neo4j) vs. Firestore (graph2nosql)

11. Proprietary + Confidential
Pricing: Aura (Neo4j) vs. Firestore (graph2nosql)
GCP pricing calculator

12. Proprietary + Confidential
Graph Databases build on index-free adjacency,
firestore on single-field indexes
“The graph structure is stored as fixed-length
records: one store for nodes and a similar one
for relationships.
Multiplying the ID of a record by its size in bytes
gives you its offset in the corresponding store
file. This pattern is known as index-free
adjacency [...]”
If no index exists for a query, most
databases crawl through their contents
item by item, [...]. Cloud Firestore
guarantees high query performance by
using indexes for all queries. As a result,
query performance depends on the size of
the result set and not on the number of
items in the database.

13. Proprietary + Confidential
Single-field indexes & data model allowing for
undirected & directed connections ftw.
Who are Bob’s friends?
vs.
“who is friends with Bob?”

14. Proprietary + Confidential
The Graph Storage landscape is underdeveloped
and has a lot of space to grow.
OSS Focus Firestore is private source, MongoDB graph2nosql interface tbd.
Neo4j UI
Cypher A structured query language is a huge advantage for exploration.
Neo4j UI is great for non-tech stakeholder’s exploration.
Cost + Latency Firestore outperforms in simple cases, let’s discuss.
hello-jp.net

15. Proprietary + Confidential
That’s a wrap!

16. Proprietary + Confidential
Try it out or contribute