Extreme Elasticity with Tablets, Raft and Kubernetes by Maciej Zimnoch

Recent ScyllaDB versions have improved elasticity using Tablets and Raft-based Consistent Topology Changes, allowing for fast bootstrapping and parallel scaling. A demo presents doubling cluster size and autoscaling after crossing 90% disk utilization.

1. A ScyllaDB Community
Extreme Elasticity with Tablets,
Raft and Kubernetes
Maciej Zimnoch
Senior Software Engineer

2. Maciej Zimnoch
■Scylla Operator maintainer
■Previously worked on Scylla Manager, Instant
Messaging servers, SDN and LTE networks

3. ■Life before Tablets and Raft
■Elasticity improvements
■Demo - Doubling cluster size
■Demo - Autoscaling after crossing 90% disk utilization
Agenda

4. Life before Tablets and Raft

5. Scaling
Scaling was slow due to:
■Synchronous scaling - missing topology change coordinator
■Data streaming - big datasets

6. Bootstrapping with VNodes
slow slow
Bootstrapping Streaming Ready

7. Synchronous scaling

8. Synchronous scaling

9. Synchronous scaling

10. Synchronous scaling

11. Synchronous scaling

12. Synchronous scaling

13. Elasticity improvements

14. Scaling
Recent ScyllaDB versions include multiple elasticity improvements.
Most notable ones:
■Tablets and Tablets Load Balancer
■Consistent Topology Changes - based on Raft

15. Bootstrapping with Tablets
fast
Bootstrapping Ready

16. Parallel scaling

17. Parallel scaling

18. Doubling cluster size
Demo

19. Demo setup
apiVersion: scylla.scylladb.com /v1alpha1
kind: ScyllaDBDatacenter
metadata:
name: scylla
spec:
rackTemplate:
nodes: 1
racks:
- name: a
- name: b
- name: c
rack-a
rack-b
rack-c

20. Demo setup
apiVersion: scylla.scylladb.com /v1alpha1
kind: ScyllaDBDatacenter
metadata:
name: scylla
spec:
rackTemplate:
nodes: 2
racks:
- name: a
- name: b
- name: c
rack-a
rack-b
rack-c

21. Demo

22. Autoscaling - 90% disk utilization
Demo

23. Demo setup
rack-a
rack-b
rack-c clients
write
HPA
HPA
HPA
*HPA - Horizontal Pod Autoscaler
Target - 90% disk utilization

Each HPA* scales rack
independently based on rack disk
usage.

When target utilization is crossed,
new Scylla Node is added.
…

24. Demo

25. Thanks!
Maciej Zimnoch
scylladb/scylla-operator