From 1M to 1B Features Per Second: Scaling ShareChat's ML Feature Store

ShareChat's Ivan Burmistrov walks through how they built a low latency ML Feature Store based on ScyllaDB which initially failed to meet the scalability requirements and failed on 1 million features per second load, but has been successfully scaled 1000 times to handle 1 billing features per second without scaling the underlying database.

1. From 1M to 1B Features Per Second: Scaling ShareChat’s ML Feature Store Ivan Burmistrov Principal Software Engineer Andrei Manakov Staff Software Engineer ShareChat ShareChat

2. Context

3. Moj, a short video app

4. What are the features anyway?

5. The story

6. The architecture

7. Tiles

8. Tiles

9. High-level architecture

10. Why it failed?

11. ScyllaDb Schema (Bad) CREATE TABLE features ( entity_id string, tile_time timestamp, feature_name string, value blob, PRIMARY KEY ((entity_id), tile_time, feature_name))

12. Tiling Configuration (Bad)

13. Let’s do some math

14. Optimisations

15. ScyllaDb Schema (Good) CREATE TABLE features ( entity_id string, tile_time timestamp, features blob, PRIMARY KEY (entity_id, tile_time))

16. Tiling Configuration (Good)

17. Compaction Strategy

18. 4x Scylla Would Be Enough?

19. Cache locality

20. Consistent hashing

21. Consistent hashing: ingress nginx.ingress.kubernetes.io/upstream-hash-by: "$bucket-value" nginx.ingress.kubernetes.io/upstream-hash-by-subset: "true" nginx.ingress.kubernetes.io/upstream-hash-by-subset-size: "3"

22. Consistent hashing: ingress

23. Consistent hashing: subset

24. Consistent hashing ingress: path rewriting nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/rewrite-target: /$1 nginx.ingress.kubernetes.io/upstream-hash-by: $2 hosts: - host: feature-service.internal paths: - path: /(method)(/hash-by-.*)

25. Consistent hashing: ingress

26. Improve cache locality: 27 deployments

27. Improve cache locality: 27 deployments

28. What’s next?

29. Envoy proxy

30. Feature Service Optimisation

31. Envoy proxy - result

32. Conclusion Robust proven technologies pay off (ScyllaDB, Flink,...) Every next step is harder than previous one The simplest and practical solution does work The most optimized solution isn’t human-friendly Don’t be scared to fork a lib and adjust it for your system

33. Ivan Burmistrov burmistrov.ivan @gmail.com @isburmistrov [email protected] Thank you! Let’s connect. Andrei Manakov andection @gmail.com @AndreyManakov andection@threads