Honest Performance Testing with "NDBench" (Vinay Chella, Netflix) | Cassandra Summit 2016

Apache Cassandra makes it possible to execute millions of operations per second in scalable fashion. Harnessing the power of C* leaves many developers pondering about the following:

- Is my data model appropriate and not going to end up as wide partition(s) causing heap pressure and other issues?
- How do I tune my connection pool configuration? What are the optimal settings for my environment ?
- What is my C* cluster capacity in terms of number of IOPs for a given 95th and 99th latency?
- How do I perf-test my data access layer?

In this talk, Vinay Chella, Cloud Data Architect @ Netflix, will share open source tools, techniques and platform(NDBench) that Netflix uses to perf-test their C* fleet with simulations millions of operations per second.

About the Speaker
Vinay Chella Cloud Data Architect, NETFLIX Inc

About Vinay Chella, Cloud Data Architect at Netflix having deeper understanding of Cassandra and other RDBMS. As an Engineer and Architect, working extensively on data modeling, performance tuning and guiding best practices of various persistence stores. Helping various teams @ Netflix building next generation data access layers.

1. Honest Performance Testing with NDBench Netflix Data Benchmark

2. Vinay Chella Cloud Data Architect Cassandra MVP Cloud Database Engineering @ Netflix

3. CDE, Cloud Database Engineering Providing data stores as a service Cassandra Dynomite Elastic s earch and RDS Who are we?

4. 98% of streaming data is stored in Cassandra Data ranges from customer details to Viewing history / streaming bookmarks t o billing and payment Cassandra @ Netflix

5. Agenda Background Why NDBench? Architecture Usage Achievements Roadmap Take away

6. Perf testing persistence layer?

7. Capacity in my existing fleet?

8. Why? Why not already existing Perf testing tools?

9. What is NDBench? Netflix Data Benchmark (NDBench) is a P luggable cloud-enabled benchmarking tool that can be used across any data store system .

10. Side by Side comparison

11. Different d river/ software versions

12. Different i nstance types

13. Dynamically tunable configurations

14. Varying data models

15. Pluggable Patterns & Loads

16. Different Client APIs

17. Netflix homegrown Well integrated with netflix OSS infrastructure

18. Architecture

19. What is Pluggable? Load Patterns Load tests

20. Load Patterns Random Sliding Window

21. Load Tests Cassandra-JavaDriver Elastic Search Dynomite Cassandra-Astyanax In-Memory Test

22. What can be configured? ndbench.driver.numKeys - 1000000 ndbench.driver.dataSize - 200 bytes ndbench.driver.numWriters - 1 ndbench.driver.numReaders - 1 ndbench.driver.writeRateLimit - 100 ndbench.driver.readRateLimit - 200 ndbench.driver.useVariableDataSize - false Many more….

23. Dynamic Script

24. How to use it REST API UI

25. REST API Initialization Initialize: /pappy/driver/init/{DriverName} Init Script: /pappy/driver/initfromscript Perf API Start writes: /pappy/driver/startWrites Start reads: /pappy/driver/startReads Stop everything: /pappy/driver/stop Sanity check Verify Read: /pappy/driver/readSingle/key Verify Write: /pappy/driver/writeSingle/key Verify Read: /pappy/driver/readSingle/key Backfill Data Backfill: /pappy/driver/startDataFill DataBackfill Async: /pappy/driver/startDataFillAsync Status API /pappy/driver/{getRead/Write}Status /pappy/driver/getserverstatus

26. NDBench Demo...

27. NDBench @ Netflix As a Benchmarking Tool Integration Tests Deployment Validation

28. NDBench’s Achievements @ Netflix

29. N+1

30. C* 1.2 → C* 2.0, C*2.0 → C* 2.1

31. C* 2.0 vs C* 2.1 (Reads - Thrift)

32. C* 2.0 vs C* 2.1 (Writes - Thrift)

33. CentOS ---> Trusty

34. CentOS -> Trusty Migration

35. LCS on CentOS vs Trusty (writes)

36. LCS on CentOS vs Trusty (Reads)

37. Java 7 → Java 8

38. C* on Java 7 vs Java 8 (Writes)

39. C* on Java 7 vs Java 8 (Reads)

40. C* AMI Certification Pipeline

41. Dynomite benchmarking Generating Millions of Ops/ sec

42. Dynomite benchmarking

43. Roadmap Performance profile management Automated metrics analysis Dynamic load generation based on destination schema

44. https://github.com/Netflix/ndbench

45. Take away “Test the honesty of your data models, persistence layers in Cloud ecosystem using NDBench”

46. Q & A