Lightweight Transactions in Scylla versus Apache Cassandra

Lightweight transactions (LWT) has been a long anticipated feature for Scylla. Join Scylla VP of Product Tzach Livyatan and Software Team Lead Konstantin Osipov for a webinar introducing the Scylla implementation of LWT, a feature that brings strong consistency to our NoSQL database.

In this webinar we will cover the tradeoffs typically made between database consistency, availability and latency; how to use lightweight transactions in Scylla; the similarities and differences between Scylla’s Paxos implementation and Cassandra’s, and what it all means to users.

From attending this live webinar you’ll learn…

The advantages and disadvantages of various consistency options
Scylla lightweight transactions: syntax and semantics
A design and implementation overview, changes in Paxos
Performance comparisons with Apache Cassandra
Scylla’s future roadmap for LWT beyond Paxos

1. Tzach Livyatan, ScyllaDB VP of Product Konstantin Osipov, Software Team Lead Lightweight Transactions in Scylla vs Apache Cassandra

2. ‹#› The Real-Time Big Data Database Drop-in replacement for Apache Cassandra and Amazon DynamoDB 10X the performance & low tail latency Open S ource , Enterprise and Cloud options Founded by the creators of KVM hypervisor HQs : Palo Alto, CA, USA; Herzelia, Israel; Warsaw, Poland About ScyllaDB

3. Presenters Konstantin Osipov , Software Team Lead Kostja is a well-known expert in the DBMS world, spending most of his career developing open-source DBMS including Tarantool and MySQL. At ScyllaDB his focus is transaction support and synchronous replication. Tzach Livyatan , VP of Product Tzach has a 15 year career in development, system engineering and product management. He has worked in the Telecom domain, focusing on carrier grade systems, signalling, policy and charging applications for Oracle and others. ‹#›

4. Agenda About ScyllaDB Eventual consistency in Scylla LWT at a glance LWT benchmarks Under the hood: Scylla optimizations Only 3 round trips in most cases LWT are always durable with commit log Reduced contention Roadmap QA ‹#›

5. Eventual Consistency in Scylla ‹#›

6. NoSQL - By Availability vs Consistency ‹#› Pick Two Availability Partition Tolerance Consistency

7. Data Replication Replication Factor(RF): Number of nodes where data is replicated Done automatically Per Data Center (DC) Keyspace level setting ‹#›

8. Determines number of replica node responses required for a query to be deemed successful CL of 1: Wait for response from one replica node CL of ALL: Wait for response from all replica nodes CL QUORUM: Wait for floor((#replicas/2)+1) CL LOCAL_QUORUM: Wait for floor((#dc_replicas/2)+1) ‹#› Consistency Level (CL)

9. CL of ONE: W ait for at least one of the replicas Client R1 R2 R3 ‹#› Consistency Level - Write

10. CL of QUORUM: Wait for at least two of the replicas R1 R2 R3 ‹#› Consistency Level - Write Client

11. CL of QUORUM: Wait for at least two of the replicas R1 R2 R3 ‹#› Consistency Level - Read Client

12. Strong Consistency: Read CL + Write CL > RF Examples: Read CL = 2, Write CL = 2, RF =3 ⇒ Strong Consistency Read CL = 1, Write CL = 2, RF =3 ⇒ Eventual Consistency ‹#›

13. ‹#› Eventual Consistency - Anti Entropy Hinted Handoff Read Repair Repair ⇒ Node R evive ⇒ Inconsistent read ⇒ Recurrent operation (weekly) Why Repair? The resurrection problem

14. What about Conditional Updates?

15. LWT at a Glance

16. Try Scylla LWT Now ! Download Scylla 3.2, Launch EC2 AMI, or use Docker https://www.scylladb.com/download/open-source/ https://hub.docker.com/r/scylladb/scylla/ Use --experimental and follow the docs: https://docs.scylladb.com/operating-scylla/scylla-yaml/ CQL Reference https://docs.scylladb.com/getting-started/dml/#if-condition ‹#›

17. CQL Avoids Slow Reads > UPDATE employees SET join_date = '2018-05-19' WHERE firstname = 'John' AND lastname = 'Doe' ; > SELECT * FROM employees ...; firstname | lastname | join_date -----------+----------+------------ John | Doe | 2018-05-19 ‹#›

18. CQL Conditional Statement > UPDATE employees SET join_date = '2018-05-19' WHERE firstname = 'John' AND lastname = 'Doe' IF join_date != null ; [a pplied] ----------- False ‹#›

19. What Statements Can Be Conditional? Any INSERT , UPDATE or DELETE can have an IF clause: > UPDATE employees SET join_date = … IF EXISTS ; > INSERT INTO bookings (id, item, client, quantity) VALUES (…) IF NOT EXISTS ; > UPDATE inventory SET state = 'Used' WHERE itemid = ? IF state = 'Unused' AND check = 'Passed' ; > DELETE FROM tasks WHERE project_id = ? AND task_id = ? IF task[ 'state' ] IN ( 'Complete' , 'Abandoned ' ); ‹#›

20. Conditional Batches > BEGIN BATCH > UPDATE tasks SET n_abandoned = 0 WHERE project_id = 1 > IF n_abandoned > > DELETE FROM tasks WHERE project_id = 1 > AND state = 'Abandoned' > APPLY BATCH ; [applied] | project_id | state | task_id | n_abandoned ----------+------------+-----------+---------+------------- True | 1 | Abandoned | 693 | 2 ‹#›

21. Scylla result metadata vs Cassandra Scylla: > INSERT INTO lwt (k,v,x) VALUES ( 1 , 2 , 3 ) IF NOT EXISTS ; [applied] | k | x | v -----------+------+------+------ True | null | null | null Cassandra: > INSERT INTO lwt (k,v,x) VALUES ( 1 , 2 , 3 ) IF NOT EXISTS ; [ applied ] ----------- True ‹#›

22. Steps of conditional statement execution ‹#› Assume leadership for this key [Repair previous round if necessary] Search the old row and then check IF conditions Build mutation with max known timestamp and send it to peers Commit transaction on peers

23. Graphana M etrics ‹#›

24. Metrics CQL counters: scylla_cql_{inserts|updates|deletes|batches} Label: conditional={yes|no} storage proxy metrics: scylla_storage_proxy_coordinator_{read|write}_ {latency|timeouts|unavailable|contention|unfinished_commit|condition_not_met...} ‹#›

25. Performance

26. Setup: Single Region Amazon EC2, availability zone eu-west-1 3 nodes i 3.8xlarge 32 vcores, 244GB RAM, 4 x 1.9 NVMe SSD Replication strategy: Simple Replication factor: 3 5k row size: blogposts use case cassandra-stress user profile=cqlstress-lwt-example.yaml, client t3.8xlarge 1-182 connections ‹#›

27. Uncontended Write - B andwidth ‹#› 

28. Uncontended Write - Latency ‹#› 

29. Under the Hood

30. Introducing Paxos R 1 Can I propose a value? R 2 R 3 Accept new value Learn decision Decision made ok! ok! ok! ack ok! ‹#›

31. Introducing Paxos R 1 Can I propose a value? Check condition R 2 R 3 Accept new value Learn decision Decision made ok! ok! row checksum ok! ack ok! ‹#›

32. Scylla O ptimizations 3 round trips in most cases LWT are always durable with commit log Reduced contention ‹#›

33. Improved Speeds: Fewer Paxos Rounds ‹#› R 1 R 2 R 3 Decision made O k, checksum O k, row ok! ack ok! Can I propose a value? Check condition Accept new value Learn decision

34. Improved Durability: Always Durable Existing commitlog_sync modes: batch or periodic LWT statements always sync the commit log, in any mode Always durable ‹#›

35. Reduced Contention ‹#› R2 R3 R1 Client

36. Future Work

37. Scylla RAFT N ew replication strategy Tablet partitioning scheme Requested explicitly in CREATE TABLE No client-side timestamps Provides isolation for ALL queries ‹#›

38. Today: Scylla 3.2 - Experimental LWT Q1 2020: S c ylla 4.0 - Production ready LWT Q2 2020: Scylla Enterprise 2020.1 - Production ready LWT Road Map to LWT

39. Try Scylla LWT Now ! Download Scylla 3.2, Launch EC2 AMI, or use Docker https://www.scylladb.com/download/open-source/ https://hub.docker.com/r/scylladb/scylla/ Use --experimental and follow the docs: https://docs.scylladb.com/operating-scylla/scylla-yaml/ CQL Reference https://docs.scylladb.com/getting-started/dml/#if-condition ‹#›

40. Q&A

41. Q&A [email protected] [email protected] @ kostja_osipov @ tzachL Konstantin Osipov Tzach Livyatan Stay in touch

42. Thank you