SREDAY London 2024 | Cloud Native Technologies: The Building Blocks of Modern Database Software

Learn how cloud-native technologies like Kubernetes are revolutionizing database software, enabling scalability, resilience, and agility. Join Divine a Data on Kubernetes Ambassador to explore real-world examples and best practices for building the next generation of database solutions.


In the face of ever-growing data volumes and the increasing complexity of modern applications, cloud native technologies are emerging as the cornerstone of next-generation database systems. Last year, Divine joined Severalnines, a company that builds database automation and management software used by hundreds of organizations. Coming from a background of cloud native technologies like Kubernetes and Cilium, Divine joined Severalnines in their journey building cloud native database software for cloud service providers and enterprise organizations. In this talk, Divine will explore the multifaceted (many sides) role of cloud native technologies in shaping modern database systems. Divine will delve into the specifics of how Kubernetes Operators, Container Storage Interfaces (CSI), APIs, etc., are transforming the way databases are built, deployed, and managed. Towards the end, Divine will discuss the benefits these technologies bring, such as enhanced scalability, resilience, agility, and automation, highlighting case studies of organizations across the Data on Kubernetes (DoK) community. This presentation is designed for database administrators, software engineers, DevOps professionals, and anyone interested in the intersection of cloud-native technologies and database systems.

Divine Odazie is a Technology Evangelist at Severalnines with over 5+ years of experience in Technology and a track record in Backend Engineering, DevOps, Cloud Native and Developer Relations on a global scale. He has given talks/workshops at developer conferences like Open Source Summit Europe, KubeCon North America, Cloud Native Rejekts, and Ansible Contributor Summit.

1. 1
Cloud Native
Technologies: The Building
Blocks of Modern Database
Software
Webinar
Divine Odazie
2024 London
20th September 2024

2. 22
Divine Odazie
●Technology Evangelist
at Severalnines
●Electrical/Electronics Engineer
●Certified Kubernetes Application
Developer (CKAD)
●Data on Kubernetes Ambassador
●Certified AWS Solutions Architect
●I’m excited to speak at SREDAY
London 2024!
Database ops, your way

3. 3
1.A brief history of database
deployments
2.Cloud Native Technologies
3.Introduction to Stateful Workloads on
Kubernetes
4.Building a DBaaS on Kubernetes
5.Building a DBaaS with Kubernetes
6.Conclusion

4. 4
Humble beginnings

5. 55
Early Days
●Data was mainly stored in mainframe
computers.

●At this time, limitations included high
cost and rigid data models.

●Oxford-educated mathematician and
computer scientist, Edgar Codd's 1970
paper introduced the relational
database, revolutionizing the field.
Database ops, your way

6. 66
Pre-Cloud
●Organizations had to purchase and
manage their own database hardware
and software.

●Cloud computing emerged as a
potential solution, promising easier and
more affordable database options.

●These on-prem deployments are still
important today, especially for data
sovereignty, compliance, and
low-latency requirements.


Database ops, your way

7. 77
The Cloud Era
●Cloud-managed databases simplified
provisioning and management of
database infrastructure.

●Hardware tasks were offloaded to the
cloud provider, allowing organizations
to focus on their applications.

●Database-as-a-Service (DBaaS) further
abstracted complexity and offered
pay-as-you-go pricing.

Database ops, your way

8. 88
Container
Revolution
●Container technologies, led by Docker,
have revolutionized application and
database deployment.

●It enabled easier and faster deployment
across various platforms.

●Kubernetes emerged as the defacto
orchestration standard for containerized
applications.
Database ops, your way

9. 9
Road to databases on Kubernetes
Database ops, your way

10. 10
Cloud Native
Technologies


Database ops, your way

11. 11
What is Cloud Native?
Cloud native practices empower
organizations to develop, build,
and deploy workloads in
computing environments (public,
private, hybrid cloud) to meet their
organizational needs at scale in a
programmatic and repeatable
manner.
11
Cloud native technologies
Cloud native technologies and
architectures typically consist of
some combination of containers,
service meshes, multi-tenancy,
microservices, immutable
infrastructure, serverless, and
declarative APIs — this list is
non-exhaustive.
Source: Cloud Native Computing Foundation (CNCF)
Definitions
Database ops, your way

12. 1212
The Cloud
Native
Computing
Foundation
(CNCF)The CNCF seeks to drive adoption
of the Cloud Native paradigm by
fostering and sustaining an
ecosystem of open source,
vendor-neutral projects.
It democratize state-of-the-art
patterns to make these innovations
accessible for everyone.

13. 13
Introduction
to Stateful
Workloads on
Kubernetes


Database ops, your way

14. 14
What is a Stateful workload?
A workload that requires persistent data and must maintain its state across
restarts or failures. Examples include:
●Databases: Store transactional data, user information, etc.
●Game servers: Persist player data, game progress, etc.
●AI/ML applications: Store training data, model checkpoints, etc.

Stateful workloads require meticulous orchestration!








Database ops, your way

15. 15
Challenges Stateful workloads face
●Maintaining a consistent identity
○Often for connection to other services
●High & Consistent Availability
○Upgrades must be handled carefully to avoid disruption
○Stateful workloads often have complex start and shutdown processes

Source: Stateful Workloads in Kubernetes: A Deep Dive

Database ops, your way

16. 16
Kubernetes was originally designed
for stateless workloads.

Database ops, your way

17. 17



How is Kubernetes addressing these
challenges?
Deployment
●StatefulSets
○A StatefulSet ensures each
of its pods retains a sticky,
stable, unique identity.
●Jobs
○Training datasets, models in
ML
○Backups

+Other components
Lifecycle and Day 2
Management
●Custom Resource Definitions
(CRD)
●Operators (How Kubernetes runs
CRDs)
○PostgreSQL,
○Kafka,
○MySQL,
○Redis, etc.

Database ops, your way

18. 18
How does Kubernetes
handle storage?



Database ops, your way

19. 1919
Persisting the
Data
Persistent Volumes (PVs): Kubernetes
objects that represent storage resources in
your cluster.
Persistent Volume Claims (PVCs): a request
for storage by a user. PVCs consume PV
resources.
Container Storage Interface (CSI): is a
standard for exposing arbitrary block and file
storage systems to containerized workloads
Getting features like:
●Dynamic provisioning, resizing
●Snapshots, cloning etc.
Source: DoK Whitepaper
Database ops, your way

20. 2020
Source: DoK Whitepaper
Putting it all
together
●The user submits a configuration…
●The Operator processes the request…
●The StatefulSet creates pods in order…
●The PVCs are bound to Persistent
Volumes…
●The database application within the
pods…
●The Operator continuously monitors the
cluster…
Database ops, your way

21. 21
Building a
DBaaS on
Kubernetes


Database Ops, Your Way

22. 22
What does a DBaaS on Kubernetes need?


●Requires a system to manage users and databases
●Needs a method to apply schemas
●Must generate secrets with authentication data and distribute them to clusters and
applications using those databases
●More importantly the ability to automate day 2 ops and standardize daily lifecycle ops
●Requires a user-friendly interface
Database ops, your way

23. 23
Being Independent
●Involves procuring your own
infrastructure, building custom
software to handle each job
within the DBaaS framework.

●Requires expertise and significant
investment in time and resources.


23
You choose
Being Interdependent
●A middle ground between buying
a solution and building everything
from scratch.

●You would integrate your chosen
infrastructure, code, and tools
with off-the-shelf software to
function as the control plane.


https://youtu.be/G8MnpkbhClc?si=-
he2TdIiJI7dKUv4
Database ops, your way

24. 24
Source: How to Build Modern DBaaS Using CNCF Ecosystem Technologies
Interdependency with Cloud Native Technologies


Database ops, your way

25. 25





Do-It-Yourself DBaaS




Database ops, your way

26. 26
Zalando
●PostgreSQL-as-a-Service on
Kubernetes since as far back as
2018. Learn more in this talk.

●Deploying and managing 500+
PostgreSQL HA clusters.

●Creators of the Postgres operator
and Patroni.


26
Who built a DBaaS on Kubernetes?


Shopify
●KateSQL – Shopify’s custom-built
Database-as-a-Service platform
on GKE.

●Supporting hundreds of
production MySQL instances.

●Kubernetes for efficient resource
utilization and scaling.

https://youtu.be/G8MnpkbhClc?si=-
he2TdIiJI7dKUv4
Database ops, your way

27. 27
Not everyone wants to put their
databases on Kubernetes…
YET!



Database ops, your way

28. 28
Building a
DBaaS with
Kubernetes


Database ops, your way

29. 2929
Introducing CCX
CCX is a drop-in, white-label DBaaS that allows
CSPs to offer managed open-source databases to
their end users, using their own infrastructure.
Details:
●Kubernetes-based
●Built on the industry-leading ClusterControl (CC)
platform
●Prioritizes data sovereignty through a loosely
coupled architecture and regulatory compliance
●Quick to set up and integrate
●Provides 24/7 support and high SLA
●Offers an easy-to-use ops and end-user interface.
Database ops, your way

30. 30
Architecture
& Overview
CCX Cloud Service
Provider Architecture
Database ops, your way

31. 31
Deploy CCX
Dependencies
●CCX core DB operators
●NATS
●ExternalDNS
●Ingress controller

1
CCX Installation in 3 Steps
2
Deploy CCX Core
Services
●CCXUI
●Authentication service, etc.
●DB Orchestration service



3
Deploy CCX
Observability Stack
●Grafana
●Alert manager
●Prometheus node exporter



CCX Documentation | Public On Github | ArtifactHUB
Database ops, your way

32. 3232
Elastx: A Swedish Cloud Provider || Lintasarta Cloudeka:
An Indonesian Cloud Provider

●They implemented a full-fledged, premium service quickly
instead of committing a ton of R&D resources and time to
build one from scratch.

●They are now offering DBaaS to
thousands of end users.

●They fully customized the platform
to meet their specific needs.

Who built a DBaaS with CCX?


Database ops, your way

33. 3333
Wrapping up
The journey of database deployment is far
from over.
Cloud-native technologies, particularly
Kubernetes, offer powerful tools for managing
Stateful workloads like databases.
If you are interested in a drop in DBaaS
solution, scan the QR code.
Database ops, your way

34. 34
Tech Evangelist
Divine Odazie
Check out our podcast:
Sovereign DBaaS Decoded

Sovereign DBaaS Decoded
@Severalnines@_Odazie