You’re Doing It All Wrong by Michael Stonebraker

Historically, business apps use a three-tier architecture. Now, cloud-native architectures and DBMS can be combined, allowing for resilient, cost-effective, and high-performance systems. Learn about DBOS and why "you are doing it all wrong." #TechTalk

1. A ScyllaDB Community
You’re Doing it All Wrong
Michael Stonebraker
CTO & Co-founder of DBOS

2. © 2024 dbos.dev
DBOS
A Database Operating System

Mike Stonebraker

3. © 2024 dbos.dev 3
Timeline
●2020 -2023
○Research project at MIT and Stanford
●2023-now
○VC backed startup
●April 2024
○V1 released

4. © 2024 dbos.dev 4
Three major ideas


●New operating system stack with a DBMS at the bottom
○VLDB paper in 2022

●New programming environment (PE) for DBOS
○Cloud native Typescript (Python coming soon)
○Multi-transactions
●Tight integration between the PE, the OS and the DBMS
○Security
○Transaction guarantees everywhere
○SPEED!!!!!

5. © 2024 dbos.dev 5
The Size of OS State
●In the last 40-ish years, the OS state to be managed
○Processors, memory, storage
○Tasks, messages, files

●Has gotten massively larger (think 10
6
bigger)

●This makes managing OS state a DBMS problem!
○Databricks example

6. © 2024 dbos.dev 6
Time to Rethink OSs from the Bare Metal on Up
●Traditionally DBMS has been an application to the operating system

●We run the operating system on top of the DBMS

●You need to rotate your thinking 180 degrees!!

7. © 2024 dbos.dev 7
Distributed DBMS
Microkernel Services
File
System
Scheduler IPC
Other OS
Services
User Applications

8. © 2024 dbos.dev 8
Features
●OS services the same or faster than what you are currently doing

●Log of all events pushed transactionally to a data warehouse
○Supports time travel for the OS
○Useful for recovering from disasters (e.g. ransomware)

●Much simpler
○Linux/Kubernetes gone!

9. © 2024 dbos.dev 9
Idea Number Two


●Apply DBOS ideas to the programming environment
○Put ALL state in the DBMS
○Move application logic close to the DBMS for performance

●And leverage function as a service (FaaS) cloud model
○Workflow model of stateless functions
○In the spirit of AWS/Lambda and step functions

10. © 2024 dbos.dev 10
Programing environment
●Cloud native architecture
○Workflow of typescript functions (python soon)
○You bring the application; we do the rest (orchestration, load balancing, recovery,
…)
○OOPS code (Error logic) largely automatic
○ MUCH simpler for the developer! (one user reported a factor of 30 productivity
boost)

11. © 2024 dbos.dev 11
Example: Workflow for a “Buy Button”
Reserve Inventory
Check Credit
Process Payment
Fulfill Order
Success
The forward logic seems
simple to implement…

12. © 2024 dbos.dev 12
Challenge: The Error Branches
Reserve Inventory
Check Credit
Process Payment
Fulfill Order
Handle Error
Handle Error
Handle Error
Success Handle Error
The forward logic seems
simple to implement…

…but what happens when
operations fail?

13. © 2024 dbos.dev 13
In our experience…..
●The forward logic is 10%
●Everything else is 90%

14. © 2024 dbos.dev 14
Bigger Challenge: The “Oops” Logic
●On a restart from a crash, then ???
●On a disaster or ransomware attack, then ???
●On an overload, then ???

15. © 2024 dbos.dev 15
DBOS Workflow Guarantees
●Once-and-only-once (OAOO)
○Don’t want to pay twice or ship twice
○One regional bank estimated that 40% of their code base implements OAOO
●“Database sagas” (Multi-transactions)
○If bad address for fulfillment, want to unwind everything
○Physical backout (sometimes); often logical backout
○Compensating transaction for decrementing inventory is incrementing inventory
●Workflow finishes even if disaster strikes
○When there is a crash or a ransomware attack or ….

16. © 2024 dbos.dev 16
With these guarantees…..
●You write the operations, the platform handles everything else

○Orchestration (run operations when inputs are available)
○Careful bookkeeping on errors to support OAOO
○Sagas to support logical backout
○“Time travel” to backup everything on disasters and then replay
○Load balancing and scalability

●Worth an order of magnitude!!!!
○According to early DBOS programmers
○“We got done in 2 days what took 2 months in AWS Lambda”

17. © 2024 dbos.dev 17
Implementation – Careful Integration

●Workflow steps are compiled into Postgres stored procedures
(SPs)
○Co-location makes us an order of magnitude faster/cheaper!!
●And problems with SPs go away
○Use Typescript/Python not a new language (e.g. PL-SQL)
○Fancy time-travel debugger (we can back up individual applications as well
as the whole OS) ---see VLDB ’23
○Version control of SPs provided automatically

18. © 2024 dbos.dev 18
Summary
●PE is DBMS-based
●OS is DBMS-based
○OS/PE/DBMS carefully integrated
○And everything is time-traveled
●Co-location everywhere gives you very high performance
○X15 compared to AWS/Lambda
●Everything in the database gives a terrific security foundation
○Tracking PII data automatic
●Programming environment gives Typescript, Python/Postgres
workflows simplicity, performance and multi-transactions

19. © 2024 dbos.dev 19
Summary (2)
●Linux is elderly and doesn’t handle “big state”
●Programming environments are typically not DBMS-based
●Integration of DBMS/PE/OS gives
○Transactions
○Disaster recovery
○Simplicity
○Performance
● Let the revolution begin!!!!