stackconf 2024 | Building large-scale Internal Developer Platforms: Core Principles by Arnold Bechtoldt

I was hoping for a slightly more detailed explanation of how cloud computing works than – It’s magic” – Many companies build internal developer platforms in the cloud instead of handing over cloud accounts/credentials directly to their development teams. There may be various reasons for this, such as standardization, governance and cost control. Also the reduction of repetitive and complex tasks due to increasingly complex IT toolchains. The Internal Developer Platform developed at RTL Data is based on these motivators. Today, it is used by over 30 very heterogeneous data product teams with technologies such as GCP/AWS/Azure, GKE/Kubernetes, Prometheus/Grafana/OpsGenie, Argo CD, Gitlab CI, Airflow, Terraform, (HashiCorp) Vault and in-house developments. The consistent decentralization & encapsulation of the infrastructure is quite unique at RTL and has already resulted in over 400 GCP projects and 100 GKE clusters in 2-3 years.

1. Building large-scale Internal Developer Platforms:Core Principles
Arnold Bechtoldt
CP-V1.6
1

2. Arnold
Bechtoldt
•>10 years in IT Engineering
•Systems, Software & Architecture
•Networking, Speaker
•Design, #wasmitholz
•Outdoor sports (MTB/Hiking/Motorcycling/Sailing)
2

3. 3

4. 4

5. 5

6. Mission ofRTL Data
End CustomersDaily Business
ForourForour
6

7. 8
• Cl
ou
d Pl
at
for
m E
ngi
ne
er• (
Se
nio
r)
Dat
a E
ngi
ne
er• (
Se
nio
r)
Dat
a S
cie
nt
ist
• (S
eni
or
) Da
ta
Ana
ly
st• (
Se
nio
r)
Pro
du
ct O
wn
er/
Ma
nag
er
• …
Find us at rtldata.de!
RTLdata.de
We are hiring Cloud Platform
Engineers!

8. 9
1/5
Disclaimer

9. 10
2/5
Intro

10. 11
Deploying an application..
Team A
Team B
Team C
Team D
Team E

11. https://internaldeveloperplatform.org/what-is-an-internal-developer-platform/13
Internal
Developer
Platforms (IDPs)
… areconfigured by Ops teams and used by developers. Ops
teams specify what resources start up with what environment or
at what request. They also set base-line templates for
application configurations and govern permissions. This helps
them to automate recurring tasks such as spinning up
environments and resources and makes their setup easier to
maintain by enforcing standards. Developer teams gain
autonomy by changing configurations, deploying, spinning up
fully provisioned environments, and rollback. IDPs can be built or
bought.

12. 14
3/5
101 Building an Internal
Developer Platform

13. Platform Products &
Services
•Cloud Accounts (GCP/AWS/Azure)
•Git + CI/CD (IaC/Source Code)
•Base Infrastructure (IaaS)
•Identity and Access Management (IAM)
•Application Runtime (K8s/Docker/Config/Secrets)
•Monitoring & Alerting
•Backup & Recovery
1
5

14. Job done.
Party now?
1
6

15. 17
Some time later…
Can I have full IAM admin privileges?
I don’t like tool A. Can I use tool B?
You don’t have any docs for this!
Your docs are outdated!
I don’t want to use terraform here,
can I use the UI?
Can somebody help me setup XYZ?
This is too expensive for our stakeholder!
Re-Org #1!
Re-Org #2!
Are we CO2-neutral?
Can we rebuild the whole platform on
Cloud #2?
Do we really have to use this platform?
Budget cuts!
What are these 100s of Prometheus
alerts for?
Can I use Gitflow?This is super urgent!!11! Can you finish
until tomorrow?
#Security
#Datenschutz
Cloud costs are increasing!
We don’t have time/resources for that!
#Betriebsrat

16. 18
Welcome to
Day 2 Operations.

17. 21
It’s not only about tech
TechnologyProduct
People
IDP

18. 22
Stakeholders
Product Owners
Internal
Developer
Platform
Senior/Tech Leads
Architects
C-Level Mgmt
Mid-Level Mgmt/
Teamleads
Engineers
Analysts, Researcher
& Scientists
LegalSecurity
Controlling

19. 23
Daily Considerations
Stability
Automation
Maintainability
Security/Compliance
Standardization
Cost Efficiency
Innovation
Self Service
Time to Market
Autonomy
Flexibility
Creativity

20. 24
4/5
Core Principles
at RTL Data

21. 25
Platform as
product

22. 26
Autonomous
Product Teams

23. 27
Decentralized
but highly
standardized

24. 30
CMDB as single
source of truth

25. 31
Highly opinionated
& integrated
toolstack
… but options to leave the paved road

26. 32
Listen & Inform

27. 33
1.Developers are customers.
2.Platforms are products.
3.Platforms require product development.
tl;dr

28. 34
5/5
Closing

29. Thank you!
3
5
[email protected]@arnisoph