Caching for Performance Masterclass: Caching Strategies

Exploring the tradeoffs of common caching strategies – and a look at the architectural differences.

- Which strategies exist
- When to apply different strategies
- ScyllaDB cache design

1. Caching Strategies
Felipe Mendes
CACHE FORGE

2. Which Strategies Exist?

3. Top Strategies
Alex Yu @ ByteByteGo – https://blog.bytebytego.com/p/top-caching-strategies

4. Applying These Strategies
Cache Aside

5. Applying These Strategies
Cache Aside
DAX
DAX
External Write/Read
Through
DAX
DAX
DAX

6. Applying These Strategies
Cache Aside
DAX
DAX
DAX
External Write/Read
Through
DAX
DAX
DAX
Write Around /
Write Back

7. Applying These Strategies
Cache Aside
DAX
DAX
DAX
External Write/Read
Through
DAX
DAX
DAX
Write Around /
Write Back
Embedded Read
Through

8. ScyllaDB Cache Design

9. Data Flow – Writes
memtable
Write
RAM
Disk

10. Data Flow – Writes
memtable
Write
RAM
Disk
commitlog

11. Data Flow – Writes
memtable
sstable
memtable
Write
RAM
Disk

12. Data Flow – Writes
sstable
memtable
Write
RAM
Disk

13. Data Flow – Reads
RAM
Disk
sstable
sstable
sstable
Read
memtable

14. Data Flow – Reads
■Read consistency easy
○Pin sstables and memtable
○Thanks to collocation
■… But SLOW

RAM
Disk
sstable
sstable
sstable
Read
memtable

15. Buffer Cache?
RAM
Disk
sstable4K

16. Buffer Cache?
Inefficient use of memory:
■Need to cache whole buffers to cache a single row
■Access locality not likely if data set >> RAM

SSTable page (4K)
Row (300B)

17. Buffer Cache?
Poor negative caching:
■Need to cache whole data buffer to indicate absent data

SSTable page (4K)
?

18. Buffer Cache?
Inefficient use of memory:
■Redundant buffers due to LSM
○Read may touch multiple SSTables
○Memory waste remark pronounced

sstable sstablesstable
Read

19. Buffer Cache?
High CPU overhead for reads:
■Reads need to merge data from multiple sstables

sstable sstablesstable
Read

20. Buffer Cache?
High CPU overhead for reads:
■SSTable format optimized for compact storage, not read speed
■Parsing overhead:
○Need to parse index buffers sequentially
○Need to parse the data file

21. Buffer Cache?
Premature cache eviction due to SSTable compaction:
■SSTable compaction removes old files => buffer invalidation
■Hurts read performance by incurring misses


sstable
sstable
sstable
sstable

22. Naturally, we built our own cache
memtable
RAM
Disk
Read
cache
sstable
sstable
sstable

23. ScyllaDB Cache Structure
■Object cache
○Like memtable
■Optimized for low CPU overhead
○Fast reads
■Row-granularity caching
■Reflects data in all relevant SSTables for a given object (e.g. row)

24. ScyllaDB Memory Management
24
■ScyllaDB reserves and manages most of the memory on a node
○Small reserve for the OS
○No use of Linux page cache (only direct I/O)
■Cache uses all available free memory
○Shrinked on pressure from memtable and other allocations

memtablecache other

25. Shard per core
CPU 0
CPU 1
CPU 2
CPU 3

26. Coherency
memtable
Read
cache
task
task
task
■Complex operations on data without dealing with concurrency
■No locking or complex lock-free algorithms
■Data structures and algorithms simple

memtablecache

27. Challenge: Query & Manipulation (DQL/DML)
SELECT * FROM tbl WHERE pk = 0 AND ck >= 2


DELETE FROM table WHERE pk = 0 AND ck >= 2
What Now?

28. Range Queries
2 5
?
SELECT * FROM tbl WHERE pk = 0 AND ck >= 2

29. Range Queries
2 5
?
SELECT * FROM tbl WHERE pk = 0 AND ck >= 2



range continuity

30. Range Deletes
DELETE FROM table WHERE pk = 0 AND ck >= 2

2
range continuity
+ tombstone

31. BYPASS CACHE
31
■Read-through weakness:
○Cold reads may evict important cache items
○Workloads with infrequent access patterns don't
benefit from any caching
■Frequent heavy evictions are bad
■Cache thrashing

SELECT name, occupation FROM users WHERE userid IN (199, 200, 207) BYPASS CACHE;
SELECT * FROM users WHERE birth_year = 1981 AND country = 'FR' ALLOW FILTERING BYPASS CACHE;

32. ScyllaDB Cache Highlights
■ScyllaDB has a fast cache
■Efficient access & maintenance
○Thanks to collocation with replica and design
■Takes care of consistency guarantees
■Handles complexities of data and query model

33. Keep in touch!
Felipe Cardeneti Mendes
Technical Director
ScyllaDB

[email protected]
@felipemendes.dev