FoundationDB - NoSQL and ACID

In this slidecast, Dave Rosenthal from FoundatoinDB, the first commercially available NoSQL database that supports global ACID transactions (found only in relational systems) and multi-data models.

Database technologies are undergoing rapid evolution, with new approaches being actively explored after decades of relative stability. Dave Rosenthal believes that the next generation of NoSQL will continue to employ shared-nothing, distributed architectures with fault tolerance and scalability. However, they will aggressively explore the strong-consistency permitted by the CAP theorem and will support ACID, allowing for multiple data models.

Rosenthal speaks to the history of NoSQL, problems that have arisen due to a lack of data consistency in NoSQL, his views on the CAP Theorem and where it's all headed.

Learn more: https://foundationdb.com
Watch the presentation video: http://youtu.be/1KsZKRcgmuU

1. NoSQL and ACID [email protected] Twitter: @foundationdb

2. NoSQL’s Motivation Make it easy to build and deploy applications. Ease of scaling and operation Fault tolerance Many data models Good price/performance ACID transactions

3. The case for ACID in NoSQL

4. Bugs don’t appear under concurrency ACID means i solation . Reason locally rather than globally . If every transaction maintains an invariant, then multiple clients running any combination of concurrent transactions also maintain that invariant. The impact of each client is isolated.

5. Isolation means strong abstractions Example interface: storeUser (name, SSN) getName (SSN ) getSSN (name) Invariant: N = = getName ( getSSN (N )) Always works with single client. Without ACID: Fails with concurrent clients . With ACID: Works with concurrent clients.

6. Abstractions Abstractions built on a scalable, fault tolerant, transactional foundation inherit those properties . And are easy to build…

7. Remove/decouple data models A NoSQL database with ACID can provide polyglot data models and APIs. Key-value, graph, column-oriented, document, relational, publish-subscribe, spatial, blobs, ORMs, analytics, etc… Without requiring separate physical databases. This is a huge ops win.

8. So why no ACID NoSQL?

9. Historical Perspective: 2008 In 2008 , NoSQL doesn’t really exist yet. 2008

10. Databases in 2008 NoSQL emerges to replace scalable sharding/caching solutions that had already thrown out consistency. BigTable Dynamo Voldemort Cassandra

11. The CAP 2008 theorem “Pick 2 out of 3 ” - Eric Brewer

12. The CAP 2008 theorem “ Data inconsistency in large-scale reliable distributed systems has to be tolerated … [for performance and to handle faults] ” - Werner Vogles (CTO Amazon.com )

13. The CAP 2008 theorem “The availability property means that the system is ‘ online ’ and the client of the system can expect to receive a response for its request.” - Wrong descriptions all over the web

14. CAP 2008 Conclusions? Scaling requires distributed design D istributed requires high availability Availability requires no C So, if we want scalability we have to give up C, a cornerstone of ACID, right?

15. Thinking about CAP 2008 CAP a vailability != High availability

16. Fast forward to CAP 2013 “Why ’2 out of 3’ is misleading ” “CAP prohibits… perfect availability ” - Eric Brewer

17. Fast forward to CAP 2013 “ Achieving strict consistency can come at a cost in update or read latency , and may result in lower throughput … ” - Werner Vogles (Amazon CTO)

18. Fast forward to CAP 2013 “…it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions.“ - Google (Spanner)

19. ACID + NoSQL!

20. The ACID NoSQL plan Maintain both scalability and fault tolerance Leverage CAP 2013 and d eliver a CP system with true global ACID transactions Enable abstractions and many data models D eliver high per-node performance

21. Decomposition approach Decompose the processing pipeline of a traditional ACID DB into individual stages .

22. Decomposition approach Decompose the processing pipeline of a traditional ACID DB into individual stages . Stages: Accept client transactions Apply concurrency control Write to transaction logs Update persistent data representation Upside : Performance Downside : “Ugly” and complex architecture needs to solve tough problems for each stage

23. The performance surprise ACID: 10% of work NoSQL: 90% of work

24. FoundationDB Database software: Scalable Fault tolerant Transactional Ordered key-value API Layers ACID Key-value API

25. A vision for NoSQL The next generation should maintain Scalability and fault tolerance High performance While adding ACID transactions Data model flexibility

26. Thank you [email protected] Twitter: @foundationdb