2023 Facemoji State of Emoji Report2023 Facemoji State of Emoji Report

2023 Facemoji State of Emoji Report

1. dl @scyiia

ScyllaDB Fast Forward:
True Elastic Scale

Dor Laor, CEO and Co-Founder, ScyllaDB
Felipe Cardeneti Mendes, Solutions Architect++, SoyllaDB

2. Poll

How often do you scale your database?

(2 SCYLLA

3. Presenters

Felipe Cardeneti Mendes, Solutions Architect
+ Puppy Lover
+ Open Source Enthusiast
+ ScyllaDB passionate!

Dor Laor, CEO and Co-Founder
+ Puppy Lover
+ Open Source, snowboard and MTB Enthusiast
+ ScyllaDB passionate!

4. Why ScyllaDB?
The power of Cassandra at the speed of Redis with the usability of DynamoDB
Best Price/Performance in the industry Auto-tune - out of the box performance
Best High Availability in the industry Compatible with Cassandra & DynamoDB

[ Best Disaster Recovery in the industry ] | No Lock-in |
Best scalability in the industry Open Source Software
aws o
LL) kubernetes A Azure

sd D À Gone csshas e,

Escrita

5. +408 Gamechangers Leverage ScyllaDB

@oiscora GSAS @Expedio STRAVA hulu Denk
Eubi @ sharechat COptimizeiy TRACTIAN &Zillow

E RisingWave coMGAST Esiotinema 4 paloalto’

“F Fanaties
AEVERSINGLABS
zn

e Urmeccre Grad SADA @ Homar

PU nauto @ Tripadvisor ticketmaster’ @crypto.com

@scyita

6. Agenda

Wasn't ScyllaDB elastic before?
The value proposition

Demo time

Deeper dive

What's next

+ + + + +

® scriia

7. < 6.@ ScyllaDB Releases Were Good

3x4vepuVMs 3xBvcpuVMs 3X 16-vepuVMs 3x32vepuVMs 3x 64vcpuVMs — 3.x 12B-vepu VMs
1Bkeys 78 keys 4B keys BB keys 168 keys 328 keys

i of
\=

8. disrupt_mgmt_repair_cli

E) SCYLLA

9. ScyllaDB < 6.0 Problems

+ Eventual Consistency of cluster metadata
+ Great architecture for scalable, partition-tolerable, widely distributed DB
+ But
+ Topology changes are allowed one-at-a-time
+ Rely on 30+ second timeouts for poor's man linearizability
+ Node failed/down block scale
+ Streaming time is a function of the schema - parsing partitions
+ Additional complex operations: Cleanup, rebuild, etc

Escrita

10. fam eventual

Escrita

11. ScyllaDB 6.0 Value

Elasticity

+ Faster hootstrap/operations
+ Incremental bootstrap
+ Concurrent node operations

Speed

+ Streaming sstables - 30x faster
+ Load balancing of tablets Consistency
+ Small tables fit in few (single) nodes

@scyita

Simplicity

+ Decouple topology operations
+ Transparent cleanups
+
+

Semi transparent repairs
Parallel maintenance operations

TCO

+ Shrink free space
+ Reduce static, over provisioned deployments

12. Demo Time!

a

@scyiia

13. Theory Behind Tablets

Node 1 Node 2

@scyita

14. Raft- Consistent Metadata

Protocol for state machine replication
Total order broadcast of state change commands

4

Escrita

15. Linearizable Token Metadata

node A

node B

node C
. bootstrap

® scyita

16. Changes in the Data Plane

+

Fencing - each write is signed with topology version
+

If there is a version mismatch, the write doesn't go through
Replica

Coordinator

Topology
coordinator

Escrita

17. Linearizable Schema Version

sx 6.x:
oni 66066668
Hash-based schema version TimeUUID-based Schema version

No re-hash of the entire schema on change
10x less CPU with large schemas.

@ scyita

18. J SCYLLA

Erin ther gros
ee
ee
tout
ae
replica replica ropa
ar — ine
tablet | tablet | tablet
Colossus Colossus Colossus
Om Caner Qu Canter Y Du Carter z

19. Consistent Metadata & Tablet Journey

— ES

A

di
Sale topalegy £
changes

@ scyira

20. Implementation - Metadata

+ Introduce a new layer of indirection - the tablets table
+ Each table has its own token range to nade mapping

+ Mapping can change independently of node addition
and removal

+ Different tables can have different tablet counts
+ Managed by Raft

@scyita

21. Implementation - Data Path

+ Each tablet replica is isolated into its own Ga =
memtable+ SSTables
+ Forms its own little Log-Structured Merge Tree Cut ous
+ With compaction and stuff

+ Can be migrated as a unit
+ Migration: copy the unit N Men
+ Cleanup: delete the unit

+ Split/merge as the table grows/shrinks

it
i

@scyita

22. Standard Tables kr E

Sharding function generates
ai good load distribution between

23. Raft Tables < Y
RAFT q

No. 299

7 tablet i i | \ tablet

replica replica
tablet 7

® scyita

24. The Tablets Table

+ Source of truth for the cluster
+ How many tablets for each table
Token boundaries for each tablets
On which nodes and shards do we have replicas
What kind of transition the tablet is undergoing

Which nodes and shards will host tablet replicas
after the transition

+ Managed using the Raft protocol
+ Replicated on every node

+ + + +

Escrita

CREATE TABLE system. tablets (

table_id uuid,

last_token bigint,

Keyspace_name text static,
replicas listétuplecuuid, int>>,
new_replicas listetuplecuuid, ints>,
session uuid,

stage text,

transition text,

table_name text static,
tablet_count int static,

PRIMARY KEY (tablo_id, last_token)

25. Tablets - Balancing

— 90 9

oo

Depends on fault-tolerant,
reliable, and fast topology
‚changes.

© SCYLLA

26. Implementation - Load Balancer

+ Hosted on one node

+ But can be migrated freely if the node is down
+ Synchronized via Raft
+ Collects statistics on tables and tablets

+ Migrates to balance space

+ Evacuates nodes to decommission

+ Migrates to balance CPU load

+ Rebuilds and repairs

E
LP

Escrita

27. Tablet Scheduler Get!

Scheduler globally controls movement,
maintenance operation on a per tablet basis

schedule schedule
I 1

:
|

1
1
1
1
1
1
1
1
1
1

|
|
|

@ scyita

28. Tablets

Resharding is cheap.
SStables split at tablet boundary.
Reassign tablets to shards (logical operation).

® scruun

29. Tablets

Cleanup after topology change is cheap.
Just delete SSTables. Runs automatically,

under the hood

XA

® scriia

30. Tablets - Streaming (Enterprise)

Send files over RPC. No per
schema, per row processing.
Up to 30x faster, saturate links.

31. Driver Support

+ Drivers are tablet-aware
+ But without reading the tablets table
+ Driver contacts a random node/shard

+ On miss, gets updated routing information for
that tablet

+ Ensures fast start-up even with 100,000 entries
in the tablets table

@scyita

32. Authentication and Service Levels on

ScyllaDB 5.x was Manual

ScyllaDB 6.x:
+ Automatically replicated on every node
+ Linearizable with CREATE/DROP

+ No denial of service if a node is down

@scyita

Raft

33. What's Coming

34. Tablets - Repair - Fire & Forget

+ Continuous, transparent controlled by the load balancer
+ Auto GC grace period

35. E) SCYLLA

36. Consistent
Elasticity =

37. Poll

How long does it take for you to scale
your existing database?

@ scyita

38. ONLINE | OCT 23 + 24, 2024

P99 CONF

aDB \ All Things Performance

engineering

scylladb.com/category/engineering

E) SCYLLA

39. Thank you

for joining us today.

f @scyliadd (E soylladb/

Y ascylladb Ak slack scylladb.com

40. New metrics in keyspace dashboard

@ seylla-monitoring provides an updated dashboard that shows tablets over time per instance

and tablet count per instance at the moment

ScyllaDB ENGINEERING SUMMIT 2024 40 \

41. Write workload

ScyllaDB

42. Growing a cluster - Tablets

—

ur

@scyita

43. Growing a cluster - Tablets

~~

ur

@scyita

44. Growing a cluster - Tablets

@ scyita