Low-Latency Data Access: The Required Synergy Between Memory & Disk

Analytics has moved from internal dashboards to a dashboard inside the product, providing a personalized experience for each user, be it the LinkedIn profile views or Uber’s online order management and inventory. Given the requirement of sub-millisecond response times on user-facing apps, how does one ensure fast analytics on large volumes of data?

There are two main pieces when it comes to data, be it for analytics or transactions – the memory and the disk. Memory or RAM enables fast access to data during active processing, while disk storage offers a much larger storage capacity compared to memory. Both memory and disk are critical components of data processing, each serving different purposes in managing and manipulating data effectively.

In this talk, I will discuss the synergy required between memory and disk to achieve efficient data processing. I will establish a mental model to reason about data organization in memory and disk, for various data access patterns. Further, I will discuss general techniques that databases use for efficient storage and retrieval of data.

1. Low-Latency Data Access: The Required Synergy Between CPU, Memory & Disk Kriti Kathuria Database Researcher

2. Kriti Kathuria ( she/her) Database Researcher Conceptualizing Eventual Durability SQL-gen for Incremental View Maintenance Data Engineer in a past life Good mentorship is powerful and fundamental At scale, the insignificant become significant! ‹#›

3. Motivation At scale, the insignificant becomes significant! A single IO takes insignificant time But when it is GBs of data and thousands of IO ops, the latencies become significant. Thus, p99, at scale, matters. ‹#›

4. Outline Motivation Existing Techniques Aggregation Processing Vectorization Query Compilation Closing Remarks ‹#›

5. Matrix Multiplication ‹#› Example from MIT 6.172, Fall 2018, Lecture 1

6. Matrix Multiplication ‹#› 3 matrices: A x B = C X = i k j k i j

7. Matrix Multiplication ‹#›

8. Matrix Multiplication ‹#›

9. Processing aggregate queries in a database ‹#›

10. Aggregation during run-generation ‹#›

11. Aggregation during run-generation ‹#› tpch sf = 1, 6M rows filter: 5M rows fetched from disk output: 4 rows Thanh Do, Goetz Graefe, and Jeffrey Naughton. 2023. Efficient Sorting, Duplicate Removal, Grouping, and Aggregation. ACM Trans. Database Syst. 47, 4, Article 16 (December 2022), 35 pages. https://doi.org/10.1145/3568027

12. Aggregation during run-generation ‹#› tpch sf = 1, 6M rows filter: 5M rows fetched from disk output: 4 rows tpch sf = 1000, 6B rows filter: 5B rows fetched from disk output: 4 rows Thanh Do, Goetz Graefe, and Jeffrey Naughton. 2023. Efficient Sorting, Duplicate Removal, Grouping, and Aggregation. ACM Trans. Database Syst. 47, 4, Article 16 (December 2022), 35 pages. https://doi.org/10.1145/3568027

13. Aggregation during run-generation ‹#› Run generation for sorting Sorted runs Reduction of sorted data Thanh Do, Goetz Graefe, and Jeffrey Naughton. 2023. Efficient Sorting, Duplicate Removal, Grouping, and Aggregation. ACM Trans. Database Syst. 47, 4, Article 16 (December 2022), 35 pages. https://doi.org/10.1145/3568027

14. Aggregation during run-generation ‹#› In-memory index Unsorted data on disk Thanh Do, Goetz Graefe, and Jeffrey Naughton. 2023. Efficient Sorting, Duplicate Removal, Grouping, and Aggregation. ACM Trans. Database Syst. 47, 4, Article 16 (December 2022), 35 pages. https://doi.org/10.1145/3568027

15. Vectorized Query Processing ‹#›

16. Vectorized Query Processing ‹#›

17. Vectorized Query Processing ‹#›

18. JIT Query Compilation ‹#›

19. JIT Query Compilation ‹#›

20. JIT Query Compilation ‹#›

21. Kriti Kathuria linkedin.com/in/kriti-kathuria/ twitter.com/kaykathuria Thank you! Let’s connect. ‹#›