What Would You Give for Speed: Trade-offs in Eventually Consistent Systems at Fly.io by Somtochi Onyekwere

Drawing on examples from engineering Corrosion (an source distributed system designed to handle fast eventual consistency across Fly.io's cloud platform), this talk examines the nuanced relationship between speed and consistency in eventually consistent databases, what that means for developers in practice, and how to build resilient, high-speed applications that embrace—or carefully mitigate—the quirks of eventual consistency.

1. A ScyllaDB Community
What Would You Give for Speed:
Trade-offs in Eventually Consistent
Systems at Fly.io
Somtochi Onyekwere
Software Engineer

2. <Somtochi Onyekwere> (she/her)

Software Developer, @ Fly.io
■Working on distributed systems and
networking software
■Recently watched: Last of us and Severance
■Learning espanol

3. The need for speed

4. (Else) (Latency) or (Consistency)

5. CAP -> PACELC
(Else) (Latency) or (Consistency)

6. It takes time to agree

7. -Two phase commits
-Quorum reads/writes
-Whenever nodes have to
reach out to other nodes to
agree

8. Conflict-free Replicated Data Types

9. CRDTs
■Any replica can be modified without coordinating with another replicas;

10. CRDTs
■Any replica can be modified without coordinating with another replicas.
■When any two replicas have received the same set of updates, they reach the
same state.

11. CRDTs
■Any replica can be modified without coordinating with another replicas.
■When any two replicas have received the same set of updates, they reach the
same state.
■The data type (+ an algorithm) takes care of any conflict during merge

12. Corrosion

13. Corrosion
■Global replication system for sqlite data

14. Corrosion
■Global replication system for sqlite data
■CRDTs for conflict resolution

16. Corrosion
■Global replication system for sqlite data
■CRDTs for conflict resolution
■Gossip for data dissemination

17. Gossip-Style
Dissemination

18. Corrosion
■Global replication system for sqlite data
■CRDTs for conflict resolution
■Gossip for data dissemination
■Custom syncing protocol

19. Corrosion
■Global replication system for sqlite data
■CRDTs for conflict resolution
■Gossip for data dissemination
■Custom syncing protocol
■SWIM for membership protocol

20. Measuring latency

21. How quickly are we
processing changes

22. How long does it take for
change to get everywhere

23. Hybrid logical clocks

25. Working with eventually-consistent
systems

26. Reduce conflicts

27. Retries

28. Replays

29. Links

●https://github.com/superfly/corrosion
●https://github.com/superfly/cr-sqlite
●https://crdt.tech

30. Thank you! Let’s connect.
Somtochi Onyekwere
[email protected]
@somtochiama