[CNKCD2024] 선언형으로 만드는 멀티 클러스터 - Cluster API 와 App of Apps 패턴

하나의 쿠버네티스 클러스터에서 모든 환경의 서비스를 운영하는 것은 생각보다 어려운 점이 많습니다. 그렇게 결정된 단일 클러스터에서 멀티 클러스터 환경으로의 전환, 이를 위해서는 더 나은 클러스터 프로비저닝 환경과 더 나은 워크로드 배포 구조가 필요합니다.

이 세션에서는 Cluster API를 사용한 선언적 클러스터 프로비저닝 방법, 그리고 Helm 기반으로 만든 ArgoCD App of Apps 패턴에 대해 소개합니다. 각각의 개괄적 구조와 팁, 그리고 리서치와 도입 과정에서 얻은 인사이트를 공유합니다. 이 세션은 쿠버네티스 멀티 클러스터 도입을 고려중인 분들과 쿠버네티스 환경의 플랫폼 엔지니어링에 관심 있는 분들께 많은 도움이 될 것입니다.
---
행사명: Cloud Native Korea Community Day 2024
영상: https://www.youtube.com/watch?v=dWLEUfXloA8

1. ™Ñ
¶ ±5ý EX× cm ÿÞ
Cluster API ì App of Apps "
1

2. Who am I
??
•3?0 ??? ?
•??x ç´9 À² CI/CD3 ?? 1 E2(?)8 ?? Cloud, DevOps À \?3 ?]
Career
•Smilegate Megaport (2022.01 ~ 2024.07)
??? ??? ?B: Stove ??? ?: D ??X, ???m Ô ? ?(
•Kbank (2024.08 ~ now)
SRE: ??X/???m ??D ( ?: ?(
2

3. Index
•Background
•Cluster API ??D k8s Ðý:wì
•Cluster API ⇢ Concept ?P
•IDC Workload ⇢ Cluster API3 >à * ? ?? ?P
•;
•Helm5? ??? App of Apps "
•App of Apps " & ApplicationSet
•ApplicationSet8 sÑ ‹7 I,
•Helm5? ??? App of Apps " ⇢ Sub Chart ì kustomize
• ?
•Lesson Learned
3

4. Background
4

5. cm ?? ? E\?
>b bw ?? L\
1. ?? ?? E\?
⇢ Cluster API ??D Ðý:wì
2. ? ?? ? ?? ???_ / ?: ??
⇢ Helm5? ??? App of Apps "
5

6. Cluster API ??D k8s Ðý:wì
6

7. Cluster API
???m ÄIÐŽIÿ ?;3 ???m L?
7
ÿÞ 3 v?? Custom Resource?X3 C? uXJ?…

8. ? ?? ?d? KÐÄ ?8 ?
;? ?? ?? Provider3 5‘àî.
8

9. ! IDC ? Workload3 uv…
9
Management ÿÞ À² Workload ÿÞ 3 ??/?;

10. IDC ?D Workload
BYOH (Bring Your Own Host) Provider
IDC ƒ À², VMI ??? Baremetal? ???m 3 ??? ? ã.
OS ?? ???? Baremetal Provider( Šðî5a PoCx ?r ?? ?O? I K=.
OS> ??; Baremetal ??3 ?;?? Provider> S8??
⇢ BYOH (y agent3 ?? ? Daemon '5? ??? ? Custom Resource? ?;)
10

11. byoh-agent
BYOH Provider? y agent3 ??5? ?X ??\8 ?;?t,
agent? h 3PD Éá8 ??? ? S?.
•Registration: agent3 >ç ??3 Management ÿÞ D Custom Resource? a?
•Setup: OCI ?? _;3 >ç ?? ? k8s ???_(kubelet, containerd a) ??/?? ?;
•Bootstrap: kubeadm8 ?? ? ??3 k8s ?X? ?? (kubeadm init, join? M:)
agent> ???? Éá
11
"

12. Agent ?? Host a?/?; flow
12

13. Registration: Custom Resource? a?
13

14. 14
Setup: k8s ???_ ??

15. 15
Bootstrap: ?X HQ8 $à X ¸ / ?TX *ï

16. b|7 0–ÞŠ… ??d H?? _??.
Cluster APID ¯ð |\
???m ?D W8 ???m ? ?;?? ??? ?Qb?I S:
•(Q, Ô, _?6 t — À² ???m ???I " À à Iç> ? (Code èðéÑ L(..)
Cluster API> ?;?? ? ?? ??? ‹7 Ô ÉI B? (Ê ?8 zd ç   ã)
•Cluster API> ?> Ijwj ? ?? j7 ??. ?? k8s/KÐÄ ??? _? r? z03 v? & j ? ?
16

17. ?Qb?8 nv? $ç…
???m ? Docs3 ?W
✔︎ ? ' ???m Slack ;?
•Provider?? Slack ;?I “dã (Github Repo README 5?)
•??, ?? K a ??v5? ?v œ: (vL)
✔︎ Kubernetes Docs
•??? Cluster API Docs? Kubernetes Docs LI ;  F?.
•???m 3 ??/???? KI? ? ???m ? WS3  á ?> F:
•Control Plane ?, kubeadm ?, kubelet ? aI ?? x… ?? r7 ? S:.
•?I?; KW: https://kubernetes.io/docs/reference/command-line-tools-reference/
•???_ ?: https://kubernetes.io/docs/reference/config-api/
•? aa…
17

18. Helm5? ??? App of Apps "
18

19. App of Apps "
19

20. ApplicationSet (=AppSet)
20

21. J: Â ApplicationSet8 s?> e ?v
?? ??E… Ô ? ? \ O? ?? |( S »: #
Ô (?, AppD L ? ?? ? à ?;> ¯ð (SyncPolicy a)
•AppSet ;? > ` App À à ControllerJ? 2X?t, ?W> ??? ?8 &
•ex) ??q App\D auto-sync3  á x? ⇢ UI À² ?? ?2v ?:?, AppSet ;? D .?A8 ?< ? ã
•v2.9 I1 × P¶: »ÑE, L 'D G7 ‡Ò ŇI S:
•?? issue: https://github.com/argoproj/argo-cd/issues/9101
?? Template ?? ? à _( Ð>
•h 7PD ?, 4PD ?? , 4PD Git Repo3 ???? ?d8 template5? EX? rI ( _…
•_? template ??8 v ? ? ?? AppSet H? Application5? ?;??q k? ?8??
21

22. Òà, App of Apps "
D? E\? ??…
$
Application ;? ?
7 I? ??E M?Ų…
fjD yaml\I ?? F:…
Iq H m?L?…
22

23. Helm5? ??? App of Apps "
Chart - ArgoCD Application 1PE ?;
ßÒ ArgoCD Application 1PE ?;?? Chart
•d??8 ?L ? S(? mà }Z ?~8 .?A
•?I? ???, ?? ?? a7 template r ? ?; D
⇢ ??I Ò} X?? x ß/ *?a? ? º|8 C?
• ??; Chart? Harbor a OCI Registry À ?? d
$
%
23

24. Helm5? ??? App of Apps "
Sub Chart '5? Chart d (1)
24

25. Helm5? ??? App of Apps "
Sub Chart '5? Chart d (2)
25

26. Helm5? ??? App of Apps "
Sub Chart '5? Chart d (3)
26

27. Helm5? ??? App of Apps "
Sub Chart '5? Chart d (4)
27

28. Helm5? ??? App of Apps "
?
Application ;? 3 helm? kustomize3 ?? ? ??
•Helm: ApplicationD 3 .?A, v?, ???? $ç ?
•Kustomize: ?F7 Helm Chart\8 ?aD App5? ? ??? $ç ?
E\??? z0× – S?E, D? fjD yaml? ;  a: &
•monorepo ? '5? ?? manifest3 ?;?? ? ? \ O ?:
•??xK v> ?? (` $– À ` ?I?5? values.yaml8 X???, kustomize À v>?v =)
•?? yS Application8 L?? ??? x ,; (repoURL, ??, ?? , syncPolicy a L? ??)
28

29. Lesson Learned
W2 ? ߌèI × J?? KI?E, ?? aI àd ??
•Best Practice ì * Ô š À× @ÓI S? ⇢ I? @Ó À² |8 :?q ?
•d ? ? xäà ??3 :ba, ??3 E\ ? a>? jI ?
&?1 ?? ? S:.
If 7ý E0q ??|8 ??? a7 ??? ?< a>? j ? غÞW %
29

30. End Of Document
30