2024-08-14 - Cloud Native Istanbul - Cloud Native API Control Planes

The Kubernetes Resource Model (KRM) has proven to be amazing at building consistent, high quality APIs. Its extensibility has driven mass adoption by third party “operators”. The kcp project (accepted into the CNCF Sandbox in 2023) extends Kubernetes API concepts beyond container orchestration. This talk will discuss how kcp supercharges platforms with a global control plane for service APIs. As a central API layer, kcp enables a SaaS-like experience between internal service providers and developers, transforming internal developer platforms into a service marketplace. All via concepts and tools that developers working with Kubernetes already know and love. kcp expands the world of platform engineering beyond the limits of single Kubernetes clusters, and therefore transforms the scale at which platform teams and internal service providers can operate. This talk will explore patterns for both service providers and developers and how kcp makes their lives easier.

1. KCP:
Cloud Native API Control Planes
Marvin Beckers

2. 2
Marvin Beckers
Team Lead @ Kubermatic
& KCP Maintainer
github.com/embik
linkedin.com/in/marvinbeckers
hachyderm.io/@embik

3. Agenda
1.Kubernetes as an API Layer
2.Lightweight Clusters to the Rescue
3.What is KCP?
a.Workspaces
4.The API Marketplace
a.Create APIs with APIExports
b.Enable APIs with APIBindings
5.Demo
6.Wrapping Up

3

4. Kubernetes as an API Layer
4

5. REST API Conventions with Kubernetes
5
/apis/<group>/<version>/[namespaces/<namespace>/] <resourcetype>[/<name>]
APIs in Kubernetes
are grouped.
Each API group is
also versioned.
Resources are optionally
namespaced.
Resources have a specific
resource type that defines
their schema.
Resources are
uniquely named.

6. 6
Some Examples

7. 7
The Kubernetes API is pretty awesome!
(that’s it. That’s the tweet post slide)

8. 8
But …

●CRDs are cluster-scoped, so everyone shares them.
●Kubernetes clusters are local, not meant to scale across regions.
●You don’t need the workload orchestration part for APIs.

9. Lightweight Clusters
9
to the rescue?

10. Hosted Control Planes
10
Infrastructure (Kubernetes)
Datastore
CRDs
RBAC
Control Plane
Datastore
CRDs
RBAC
Control Plane
Datastore
CRDs
RBAC
Control Plane

11. 11
What if …

12. 12
… we “virtualized” API servers?

13. 13
“Logical” Clusters
Infrastructure (Kubernetes)
Datastore
RBAC
Bound APIs
CRDs
Logical Cluster
RBAC
Bound APIs
CRDs
Logical Cluster
RBAC
Bound APIs
CRDs
Logical Cluster

14. What is KCP?
14

15. 15
A horizontally scalable control
plane for Kubernetes-style APIs.

16. 16
Sandbox project
(since end of 2023)

17. Workspace
17
A multi-tenancy unit of isolation in kcp.
Each workspaces has its own available API
resource types.
API objects are not shared across workspaces.
Delegation of administrative permissions to
workspace owners.
Workspaces are cheap.

18. 18
https://kcp:6443/clusters/root
https://kcp:6443/clusters/root:org-a
https://kcp:6443/clusters/ root:org-a:team-a
root
org-a …
… …team-a
Workspaces are organized in a tree (or multiple).

19. 19
$ kubectl ws .
Current workspace is "root".

$ kubectl get ws
NAME TYPE REGION PHASE URL AGE
org-a organization Ready https://… 69d
org-b organization Ready https://… 65d

$ kubectl ws org-a
Current workspace is "root:org-a" (type root:organization).

$ kubectl get ws
NAME TYPE REGION PHASE URL AGE
team-a team Ready https://… 3m23s
team-b team Ready https://… 3m18s


$ kubectl ws team-a
Current workspace is "root:org-a:team-a" (type root:team).
And you can navigate them!

20. 20
The API Marketplace

21. 21
Service teams should provide
services, not fiddle with their own
API server implementation.

22. 22
Create APIs with APIExports

23. APIExport
23
apiVersion: apis.kcp.io/v1alpha1
kind: APIExport
metadata:
name: demo.embik.me
spec:
latestResourceSchemas:
- v1.certificates.demo.embik.me
- v1.pizzas.demo.embik.me
Resource schemas define
resources, just like CRDs.

24. Virtual Workspaces for Controllers
24
Proxy that provides a computed view
Controller
Virtual Workspace KCP
ObjectsTransformed Objects

25. 25
APIExport
Virtual
Workspace
Pizza a Pizza b Certificate a
Pizza a Pizza c
Certificate b Pizza b
root
root:org-a
root:org-b
…/clusters/*/…
Access requires special RBAC!

26. How to Build a KCP-aware Controller
26github.com/kcp-dev/controller-runtime
Use kcp-aware client and cache
Reconcile in Virtual Workspace via Cluster
Reconcile with logical cluster in context
ctx = kontext.WithCluster(ctx, logicalcluster.Name(request.ClusterName))
1
2
3
sigs.k8s.io/controller-runtime/pkg/cluster.Cluster
MapperProvider: kcp.NewClusterAwareMapperProvider,
NewClient: kcp.NewClusterAwareClient,
NewCache: kcp.NewClusterAwareCache,
NewAPIReader: kcp.NewClusterAwareAPIReader,

27. Enable APIs with APIBindings
27

28. 28
$ kubectl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
configmaps cm v1 true ConfigMap
events ev v1 true Event
namespaces ns v1 false Namespace
resourcequotas quota v1 true ResourceQuota
secrets v1 true Secret
serviceaccounts sa v1 true ServiceAccount
[...]
workspaces ws tenancy.kcp.io/v1alpha1 false Workspace
workspacetypes tenancy.kcp.io/v1alpha1 false WorkspaceType
[...]
https://kcp:6443/clusters/ root:org-a/api
Available APIs in a Workspace

29. Powered by APIBindings
29
$ kubectl get apibindings
NAME AGE READY
tenancy.kcp.io-3wb5h 30d True
topology.kcp.io-cua3o 30d True
apiVersion: apis.kcp.io/v1alpha1
kind: APIBinding
metadata:
name: tenancy.kcp.io-3wb5h
spec:
reference:
export:
name: tenancy.kcp.io
path: root
This references an APIExport in a different workspace!

30. APIBindings across Workspaces
30
root
org-a …
team-db team-ateam-os
bind API

31. RBAC Extension for APIBindings
31
Binding to exported APIs
requires RBAC permissions
on the APIExport.
apiVersion:
rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: bind-apiexport
rules:
- apiGroups:
- apis.kcp.io
resources:
- apiexports
verbs:
- use
resourceNames:
- demo.embik.me

32. Demo
32
We are using:

●https://github.com/kcp-dev/kcp
●https://github.com/kcp-dev/kcp/tree/main/cli/cmd/kubectl-workspace
●https://github.com/embik/kubectl-switch-ws
●https://github.com/kcp-dev/controller-runtime/tree/kcp-0.18/examples/kcp
Let’s see KCP in action!

33. Wrapping Up
33

34. 34
Software is APIs are eating the world.

35. 35
Let’s use technology we know …

36. 36
… and create open platforms!

37. Wrapping Up
37
1.kcp is building a global control plane for API-driven platforms.
2.Workspaces allow to mirror organizational hierarchy.
3.Providing Kubernetes-style APIs at scale is incredibly easy.

It’s a community project! We welcome everyone to build the project’s future
together.