The Strategy Behind ReversingLabs’ Massive Key-Value Migration

ReversingLabs recently completed the largest migration in their history: migrating more than 300 TB of data, more than 400 services, and data models from their internally-developed key-value database to ScyllaDB seamlessly, and with ZERO downtime. Services using multiple tables — reading, writing, and deleting data, and even using transactions — needed to go through a fast and seamless switch. So how did they pull it off? Martina shares their strategy, including service migration, data modeling changes, the actual data migration, and how they addressed distributed locking.

1. The (Zero Downtime) Database Switch Martina Alilovic Rojnic, Software Architect a t ReversingLabs

2. Martina Alilovic Rojnic Software architect at ReversingLabs Distributed systems enthusiast A mathematician at heart Zagreb, Croatia

3. ReversingLabs beginnings and growth The big database migration Distributed locking Presentation Agenda

4. ReversingLabs by the Numbers ~20B classified samples ~300TB of metadata ~400 microservices

5. A Trip to 2011 Cassandra 0.6 MongoDB 1.6 DynamoDB not released yet Postgres – MVCC overhead

6. ScyllaDB to the Rescue Fast Resilient Scalable

7. “Spectacular achievement is always preceded by unspectacular preparation.”

8. Keep i t Simple Change the models as little as possible Don’t change the code

9. Differences in Architecture

10. Data Migration

11. Data Migration Steps

12. Data Migration Steps 1. Stream the old database output to Kafka

13. Data Migration Steps 2. Dump the old database into a specified location 3. Prepare a ScyllaDB table by configuring its structure and settings

14. Data Migration Steps 4. Prepare and load the dump into the ScyllaDB table

15. Data Migration Steps 5. Continuously stream data to ScyllaDB

16. Services Migration

17. Services Migration Steps

18. Services Migration Steps 1. Migrate the APIs one by one 2. Prepare for the big migration day

19. Services Migration Steps 3. Stop the master services

20. Services Migration Steps 4. Migrate the delivery services

21. Services Migration Steps 5. Migrate and start the master services

22. Resource Locking

23. Differences in Architecture

24. Distributed Locking

25. Stay in Touch Martina Alilovic Rojnic [email protected] https://medium.com/@martina.alilovic https://www.linkedin.com/in/martina-alilovic-rojnic/