Full-RAG: A modern architecture for hyper-personalization

Read more: https://zilliz.com/blog/full-rag-modern-architecture-for-hyperpersonalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.

1. Full-RAG:
A modern architecture for
hyper-personalization
Mike Del Balso
CEO & Co-Founder

2. 2
Goal: highly personalized travel recommendations


We think you’ll love
?

3. How can we get a
good suggestion
from a model?

3

4. 4
Fine tune?
Improving model’s
intrinsic knowledge
How can we get a better recommendation?
Prompt Engineer?
Rewording the question,
giving time to think
RAG?
Improving model’s knowledge
about the current situation

5. 5
Low quality
recommendation
content
VectorDB
Candidates: Cities
Recommendation
LLM
Traditional RAG
(Stone age)
City: Paris
Country: France
City: Tokyo
Country: France
City: Johannesburg
Country: South Africa
??????
“Visit Paris!”
Uncontextualized
Candidates
Query: Where should I travel over summer break?

6. 6
Where should I travel over summer break?
Stranger Travel Agent
Low context, High expertise
Paris.
Your Best Friend Travel Agent
High context, High Expertise
You said you loved that
sailing trip last summer, why
not go check out the Rodos
Cup in Greece? Rhodes has
a super cool old town with
lots of great little cafés.

7. 7
Without context (i.e. in Trad RAG),
we just have uncontextualized candidates
Uncontextualized Candidate
City: Paris
Country: France
City: Tokyo
Country: Japan

8. Context is the relevant information
that AI models use to understand
a situation and make decisions.
8

9. 9
Context enriches the candidate with more
information to make it easier to reason about
Uncontextualized Candidate Contextualized Candidate
City: Paris
Country: France
Weather: 20°C, sunny
Activities: Museums, cafes, river tours
Nature: Fontainebleau, Versailles gardens
Events: Fashion Week, Bastille Day
Cuisine: Croissants, escargot
Language: French
Cost/Day: 200 USD
Safety: High
Visit Time: Apr-Jun, Sep-Nov
Accessibility: High, extensive public transport
Historic Sites: Eiffel Tower, Notre Dame
Accommodation Range: Hostels to luxury hotels
Visa Ease: Schengen Area, visa policies vary
Nightlife: Vibrant, diverse options
Family Friendly: Yes, many activities
Art Scene: Louvre, Montmartre
Shopping: Boutiques, flea markets
Internet Access: High-speed, widely available
City: Paris
Country: France

10. City: Paris
Country: France
City: Paris
Country: France
Weather: 20°C, sunny
Activities: Museums, cafes, river tours
Nature: Fontainebleau, Versailles gardens
Events: Fashion Week, Bastille Day
Cuisine: Croissants, escargot
Language: French
Cost/Day: 200 USD
Safety: High
Visit Time: Apr-Jun, Sep-Nov
Accessibility: High, extensive public transport
Historic Sites: Eiffel Tower, Notre Dame
Accommodation Range: Hostels to luxury hotels
Visa Ease: Schengen Area, visa policies vary
Nightlife: Vibrant, diverse options
Family Friendly: Yes, many activities
Art Scene: Louvre, Montmartre
Shopping: Boutiques, flea markets
Internet Access: High-speed, widely available
10
Personalized Context enriches candidates with
user-level information
Without context With context With Personalized Context
City: Paris
Country: France
Weather: 20°C, sunny
Activities: Museums, cafes, river tours
Nature: Fontainebleau, Versailles gardens
…
Preferred Climate: Mild
Interest in History: High
Dining Preference: Gourmet/Fine dining
Cultural Interest: High in arts and fashion
Budget: Luxury
Accommodation Preference: Boutique hotels
Preferred Language: Prefers English-friendly destinations
Activity Level: Moderate, enjoys leisurely strolls and seated
activities
Travel Experience: Seasoned traveler, prefers depth of experience
Travel Group: Solo traveler
Interest in Shopping: High, prefers unique boutiques
Nightlife Interest: Low, prefers quiet evenings
Interest in Local Cuisine: High, enjoys trying national dishes
Interest in Events: Moderate, selectively attends major events
Transportation Preference: Public transport, occasional taxi

11. 11
Examples of context
Destination Insights
Cultural Significance Local customs, events, and holidays at the destination
Safety and Alerts Current travel advisories and safety warnings
Attractions and Activities Information on points of interest and things to do
Lodging and Transport Availability and options for accommodation and local travel
User-Centric Data
Historical Interactions Including search history and previous bookings
Demographic Information Age, language preferences, and other personal data
Travel Patterns Data on past destinations and types of travel
Preferences and Real-Time Data
Activity Monitoring User's current engagement with the platform
Active Input Immediate queries and filter settings
Preference Settings Explicitly stated travel preferences and interests
Situational Context
Geographic Position The user’s current or selected location
Temporal Context Time of day, date, and season
Economic Context
Financial Indicators User's budget range and previous spending habits
Currency Trends Current exchange rates affecting travel costs
External Influences
Event Schedules Local events that could impact or enhance the travel experience
Weather Patterns Forecasted weather conditions for the destination
...
... ...

12. No
context
High
context
Quality of
response
12
Where should I travel over summer break?

13. 13
VectorDB
Candidates
Recommendation
LLM
Create personalized context by enriching
candidates with relevant user data
City: Paris
Country: France
City: Tokyo
Country: France
City: Johannesburg
Country: South Africa
Feature
Platform



Candidate
Data
User Data
Candidates w/
personalized
context
Best Friend-level
Travel Agent
recommendation
“Kyoto is
perfect for you
because…”
The Feature platform
orchestrates
context assembly

14. Favorite cultural activities
Budget for dining experiences
Past scenic preferences, like mountain or beach destinations
Interest in local festivals and seasonal events
Climate preference for vacation spots
Age range of usual travel group
Preferred types of local cuisine
Frequency of city vs. countryside destinations
Likelihood to engage in water sports
Historical landmark visitation history
Language proficiency for non-English destinations
Desire for luxury vs. budget accommodations
Appreciation for local music and performance arts
Engagement with nature and wildlife conservation areas
Interest in volunteer tourism opportunities
Local public transportation efficiency
Accessibility of medical facilities in the destination
Economic stability of the destination country
Political climate's impact on tourist safety
Visa and entry requirements for the destination
Current exchange rate advantages
Local health advisories or travel restrictions
Event timing, such as major sports or cultural events
Availability of direct vs. connecting flights
Seasonal tourist crowd levels
Local peak dining times and availability
Regional security advisories
Cultural norms and attire expectations
Time zone differences affecting activity planning
Environmental sustainability initiatives at the destination
Local telecommunications infrastructure for connectivity
Historical weather patterns for planned travel dates
Area-specific traveler reviews and ratings
Local emergency services and language support
Average local costs for tourists
Destination-specific travel insurance recommendations
Local customs clearance processes for travelers
Recent developments in local tourism facilities
Availability of multilingual guided tours
Best
High personalization → Better recommendations
VectorDB
Candidates
Recommendation
LLM
City: Paris
Country: France
City: Tokyo
Country: France
City: Johannesburg
Country: South Africa
Destination
Data
Candidates
w/ Context
Feature
Platform



14
User Data

15. We think you’ll love
15
Where to stay
?

16. Tonight: Sushi at Festival in Gion!
16
Last-Minute Opening: A few coveted spots at Chef Takumi
Nishimura's 'Sushi Mastery' workshop have just opened up—right
in the heart of Gion, a few minutes walk from you. Seize this rare
chance to handcraft the praised dragonfly roll, adorned with
top-choice sea urchin, as you've keenly blogged about. The
forecast promises a perfect evening with clear skies to enjoy this
gastronomic affair. The workshop has Dassai Umeshu 23 sake that
you've been eager to try. Act now; these tickets won’t last!

Where to stay
First, we have the Hotel Mume
located at 東山区新門前通梅本
町261. This amazing hotel has
an outstanding average rating
of 5.0 based on 8 reviews.
Book now

Following closely is Shiraume
at 東山区祗園新橋白川筋 . Also
boasting an average rating of
5.0 from 12 reviews, it's highly
recommended and beloved by
previous travelers.
Book now

Lastly, we have the SUIRAN
LUXURY COLLECTION HOTEL
KYOTO located at 右京区嵯峨 天
龍寺芒ノ馬場町 12. This
luxurious hotel in Kyoto also
got an average rating of 5.0
based on 8 reviews.
Book now
Why did we suggest this?
Your profile celebrates the art of Japanese cuisine, and we noticed your fondness for unique,
high-quality ingredients—just like the sea urchin featured in tonight's event. The unexpected ticket
availability and tonight’s stellar weather create the perfect, rare opportunity to indulge your senses
in a way that aligns with your exquisite taste and love for spontaneous adventure.
GET A TICKET HOW TO GET THERE

17. How can we build amazing
personalized contexts?

18. 4 Levels of context personalization
18
LEVEL 1
LEVEL 0
LEVEL 3
LEVEL 2

19. 19
●None
Broad, one-size-fits-all recommendations (TRAD RAG)
CONTEXT
LEVEL 1
LEVEL 0
LEVEL 3
LEVEL 2
LEVEL 0:
No Context

20. Generate a travel
recommendation
LEVEL 0:
No Context
20
Recommendation
LLM
Bad
Recommendation
Uh, Paris?
VectorDB
Candidates
w/ no context

21. No context High context
Quality of response
21
Level 0

22. CONTEXT
LEVEL 1:
Batch Context
22
●Batch
LEVEL 1 CONTEXT
●None
LEVEL 0
Personalized insights drawn from past behavior and profile data
Broad, one-size-fits-all recommendations (The dumbest model)
LEVEL 3
LEVEL 2

23. 23
Recommendation
LLM
Candidates w/
Batch Context
Data Warehouse
●Trips history
●User interests
●Favorite activities
LEVEL 1:
Batch Context
Feature
Platform
Candidates
VectorDB

24. 24
Recommendation
LLM
Candidates w/
Batch Context
Data Warehouse
●Trips history
●User interests
●Favorite activities
LEVEL 1:
Batch Context
Feature
Platform
Candidates
Candidate Source
1.Building pipelines to retrieve, serve, and join data
from warehouses / data lakes
2.Creating historical eval data sets for
benchmarking and development
Problems you will encounter

25. 25
Building batch context simply
“What are the last 5 places this person has visited?”
1) Write simple definition trip_history_features.py
2) Create Eval Data
4) Read in real-time
3) Deploy to production $ tecton apply

26. 26
●trips_history
●user_interests
●favorite_activities
“Visit the ancient city of Kyoto.
Given your interest in history and
your extensive travel to historical
sites, you'll appreciate the city’s
rich heritage and numerous
temples.”
LEVEL 1:
Batch Context
Data Warehouse

27. 27
No context High context
Quality of response
Level 0
Level 1

28. ●Batch
●Streaming
LEVEL 2
CONTEXT
●Batch
LEVEL 1 CONTEXT
LEVEL 2:
Batch + Streaming Context
28
●None
LEVEL 0
Personalized insights drawn from past behavior and profile data
Recommendations adapted to the user's current interests and
interaction behavior
Broad, one-size-fits-all recommendations
CONTEXT

29. Recommendation
LLM
Candidates w/
Batch + Streaming Context
VectorDB
Candidates
29
LEVEL 2:
Batch + Streaming Context
Data Warehouse
Personalized
Recommendation
Purchase data
Purchase data
Session interactions
●Products viewed or
interacted with recently
●Purchase trends
●Pricing changes
Purchase data
Search data
Session interactions
Feature
Platform

30. Recommendation
LLM
Candidates w/
Batch + Streaming Context
Candidate Source
Candidates
30
LEVEL 2:
Batch + Streaming Context
Data Warehouse
Personalized
Recommendation
Purchase data
Purchase data
Session interactions
●Products viewed or
interacted with recently
●Purchase trends
●Pricing changes
Purchase data
Search data
Session interactions
Feature
Platform
1.Building, evaluating, productionizing, and
monitoring streaming data pipelines
2.Cost-efficient inference (not just the model!)
Problems you will encounter
??????

31. Building streaming context can also be simple
“In the past hour, what topics did the user watch a video about?”
2) Create Eval Data


4) Read in real-time
3) Deploy to production $ tecton apply
31
1) Simple definition
media_interaction_features.py

32. 2) Create Eval Data


3) Deploy to production $ tecton apply
media_interaction_features.py
4) Read in real-time
1) Simple definition
32
Building streaming context can also be simple
“In the past hour, what topics did the user watch a video about?”
Same workflow for any context

33. 33
LEVEL 2:
Batch + Streaming Context
●locations_viewed_recently
●recent_activities_viewed
●pricing_changes
“Considering you've recently been
looking at trips to Japan and your
recent interest in fine dining, Kyoto's
Gion district presents a unique dining
adventure with its renowned kaiseki
experience. Seasonal ingredients are
masterfully crafted into exquisite
dishes, offering a feast for the senses.
Don’t miss this chance to indulge in
Japan's artful cuisine during your
stay!"
Streaming

34. No context High context
Quality of response
Level 0
Level 1
34
Level 2

35. ●Batch
●Streaming
●Real-time
LEVEL 3
CONTEXT
●Batch
●Streaming
LEVEL 2
CONTEXT
●Batch
LEVEL 1 CONTEXT
LEVEL 3:
Batch + Streaming + Real-time Context
35
●None
LEVEL 0
Personalized insights drawn from past behavior and profile data
Recommendations adapted to the user's current interests and
interactive behavior
Informed, personalized recommendations using live external events,
the user’s current context, and real-time inputs
Broad, one-size-fits-all recommendations
CONTEXT

36. ●Query
●User location
●Local events

User Application
Data provider
●Local Weather
●Traffic + flight info
●Social media trends
Candidates w/
Batch + Streaming
+ Real-time Context
Data Warehouse
Recommendation
LLM
36
Data Warehouse
LEVEL 3: Full RAG
Batch + Streaming + Real-time Context
VectorDB
Candidates
Purchase data
Purchase data
Session interactions
Feature
Platform
Personalized
Recommendation

37. ●Query
●User location
●Local events

User Application
Data provider
●Local Weather
●Traffic + flight info
●Social media trends
Candidates w/
Batch + Streaming
+ Real-time Context
Data Warehouse
Recommendation
LLM
37
Data Warehouse
LEVEL 3:
Batch + Streaming + Real-time Context
Candidate Source
Candidates
Purchase data
Purchase data
Session interactions
Feature
Platform
Personalized
Recommendation
1.Building, evaluating, productionizing, and
monitoring real-time data pipelines
2.Integrating 3rd party real-time data sources
3.Striking the right balance between speed and cost
Problems you will encounter
??????

38. 38
Building real-time context works the same way
“How far is the user from the destination? Same country?”
1) Write simple
definition
device_destination_distance_features.py
…the other steps are
the same

39. 39
Building real-time context works the same way
“What’s the weather like in that place right now?”
…the other steps are
the same
destination_weather_features.py

1) Write simple
definition

40. Last-Minute Opening: A few coveted spots at Chef
Takumi Nishimura's 'Sushi Mastery' workshop have just
opened up—right in the heart of Gion, a few minutes
walk from you. Seize this rare chance to handcraft the
praised dragonfly roll, adorned with top-choice sea
urchin, as you've recently blogged about. The forecast
promises a perfect evening with clear skies to enjoy
this gastronomic affair. The workshop has Dassai
Umeshu 23 sake that you've been eager to try. Act now;
these tickets won’t last!
Real-time
LEVEL 3:
Batch + Streaming + Real-time Context
40
●query
●user_location
●local_events
●local_weather
●traffic_and_flights
●social_media_trends

41. Real-time personalization means more trusted and
valuable recommendations
Tonight: Sushi at Festival in Gion!
41
Last-Minute Opening: A few coveted spots at Chef Takumi
Nishimura's 'Sushi Mastery' workshop have just opened
up—right in the heart of Gion, a few minutes walk from you.
Seize this rare chance to handcraft the praised dragonfly roll,
adorned with top-choice sea urchin, as you've recently
blogged about. The forecast promises a perfect evening with
clear skies to enjoy this gastronomic affair. The workshop has
Dassai Umeshu 23 sake that you've been eager to try. Act
now; these tickets won’t last!

Where to stay
First, we have the Hotel Mume
located at 東山区新門前通梅本町
261. This amazing hotel has an
outstanding average rating of
5.0 based on 8 reviews.
Book now
Following closely is Shiraume
at 東山区祗園新橋白川筋 . Also
boasting an average rating of
5.0 from 12 reviews, it's highly
recommended and beloved by
previous travelers.
Book now
Lastly, we have the SUIRAN
LUXURY COLLECTION HOTEL
KYOTO located at 右京区嵯峨天
龍寺芒ノ馬場町 12. This luxurious
hotel in Kyoto also got an
average rating of 5.0 based
on 8 reviews.
Book now
Why did we suggest this?
Your profile celebrates the art of Japanese cuisine, and we noticed your fondness for unique,
high-quality ingredients—just like the sea urchin featured in tonight's event. The unexpected ticket
availability and tonight’s stellar weather create the perfect, rare opportunity to indulge your senses in a
way that aligns with your exquisite taste and love for spontaneous adventure.
GET A TICKET HOW TO GET THERE

42. 42
●Batch
●Streaming
●Real-time
LEVEL 3
CONTEXT
●Batch
●Streaming
LEVEL 2
CONTEXT
●BatchLEVEL 1
CONTEXT
BONUS LEVEL 4
Real-time Context w/ feedback
●NoneLEVEL 0
Personalized insights drawn from past behavior and profile data
Recommendations adapted to the user's current interests and
interactive behavior
Informed, personalized recommendations using live external
events, the user’s current context, and real-time inputs
Broad, one-size-fits-all recommendations
CONTEXT
●Batch
●Streaming
●Real-time
with feedback
LEVEL 4 CONTEXT
Informed, personalized recommendations using live external
events, the user’s current context, and real-time inputsIN
CONCEPT

43. OK, what did we learn?

44. E-commerce
Tailored shopping experiences
Communication
Conversational AI that understands you
Content
Recommendations that resonate
Health & Wellness
Customized well-being plans
Financial Services
Personal financial advice


44
Context is King!
E-commerce Tailored shopping experiences
Communication Conversational AI that understands you
Content Recommendations that resonate
Health & WellnessCustomized wellbeing plans
Financial ServicesPersonal financial advice

45. 45
Personalizing context can
unlock amazing AI behaviors
and product experiences.
1
Higher degrees of
personalization are more
valuable but harder to build.
2
Feature Platforms can
configure and assemble
personalized context for
LLMs.
3

46. User Application
Data provider
Context
Data Warehouse
Recommendation
model
46
VectorDB
Purchase data
Purchase data
Session interactions
Feature
Platform

47. User Application
Data provider
Context
Data Warehouse
Recommendation
LLM
47
Candidate Source
Purchase data
Purchase data
Session interactions
Feature
Platform
●Versioning
●Collaboration
●Governance
●Debuggability
●Monitoring and Alerting
Other problems you’ll run into on your journey

48. Build a Full RAG today

48
…and solve all your other AI data problems
Get started at tecton.ai/explore

49. ANNOUNCING
Rift is now in Public Preview
Python is all you need
Python transformations
for batch, streaming, & real-time.
Unmatched performance
Millisecond-fresh aggregations
across millions of events.
Try Rift now: tecton.ai/explore
Lightning-fast iteration
Develop & test locally.
Productionize instantly.
The world’s fastest path to real-time AI.