Olawale Olaleye Cloud Native Dublin Sep 2024.pdf

Olawale Olaleye presentation at CNCF Cloud Native Dublin on Sep 11 202

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimizing Kubernetes Cluster
Costs with Intelligent Shutdown
Strategies
Olawale Olaleye
Sr. Specialist Solutions Architect, Containers @ AWS
CNCF Kubestronaut

2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
•Kubernetes customers’ priorities
•Cost optimization within Amazon EKS
•Shutdown strategy: Cluster Autoscaler
•Shutdown strategy: Karpenter
•Additional resources
Agenda

3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes Customers’ priorities
Customers focused on cost optimization work on a continuous
discipline of cost reduction, while maximizing business value.
Innovate Rapidly Reduce CostResiliency and PerformanceSecurity: Environment
isolation

4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost Optimization within
Amazon EKS

5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost Optimization Strategies
Cost optimization within EKS involves multiple areas of improvement including instance
hardware, savings and purchase options, auto scaling mechanisms and visualization of spend
Purchase OptionsHardware ScalingCost Observability

6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Performance advantage with AWS designed chip Graviton

7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Graviton
AWS Graviton processors are custom built by Amazon Web Services using 64-
bit ARM Neoverse cores
Graviton 2Graviton 3
•40% better price performance over
comparable current generation x86-
based instances
•Supported by popular Linux operating
systems
•Provide up to 25% better compute
performance than Graviton2
•Use up to 60% less energy for the
same performance than comparable
EC2 instances.
High performance advantage of Graviton enables customers to spend less on compute resources

8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Purchase Options
Save more with Spot and Savings Plans

9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spot Instances
Same as
On-Demand and
Reserved Instances
Infrastructure
Spare EC2 Capacity
Pricing is based on
long-term supply and
demand (no bidding!)
Pricing
Up to 90% off compared
to On-Demand
Instances can be
interrupted with a 2
minute warning if
capacity is required
Interruption
only interrupted if OD
needs capacity
Diversification
and flexibility is key
Make use of different
instance types, sizes,
Availability Zones, and
times

10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost Visibility
Cost visualization using Kubecost

11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubecost
Real-time visibility into Kubernetes spend
Granular cost visualization at Namespace and Pod level
View cost by organizational units such as business unit or teams
Export data by API/CLI to Finance management system
Kubecost for Kubernetes provides value across three primary areas
üCost VisibilityüOptimization & SavingsüGovernance
Kubecost add-on features include

12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubecost sample report
12

13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling
Kubernetes Autoscaling: HPA, VPA, Cluster Autoscaler, Karpenter

14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes Autoscaling 101
Amazon EKS
Managed-nodegroup-spot-2vCPUs-C
c5.large, c5d.large, c5a.large, c4.large, c3.large
Managed-nodegroup-spot-2vCPUs-R
Managed-nodegroup-spot-2vCPUs-P
p3.2xlarge
r5.large, r5d.large, r5a.large, r4.large, r3.large
Horizontal Pod Autoscaler (HPA) – Scales Application
Vertical Pod Autoscaler (VPA) – Right size Application
Cluster Autoscaler (CA)/Karpenter – Scales underlying
compute infrastructure

15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EC2 - m5.xlargeEC2 - m5.xlargeEC2 - m5.xlarge
Enable consolidation
EC2 - m5.xlarge
Karpenter cost optimization
apiVersion: karpenter.sh/v1beta1
kind: NodePool
spec:
disruption:
consolidationPolicy: WhenUnderutilized

16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EC2 - m5.xlargeEC2 - m5.xlarge
Better utilization of worker nodes – reduced cost
Karpenter cost optimization

17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EC2 - m5.xlargeEC2 - m5.xlargeEC2 - m5.xlarge
Enable consolidation
Karpenter cost optimization

18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EC2 - m5.xlargeEC2 – m5.large
Better selection of worker nodes – reduced cost
Karpenter cost optimization – Pick cheaper nodes

19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown strategy: Cluster Autoscaler

20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown Strategy for Cluster Autoscaler
•CA uses Amazon EC2 Auto Scaling groups.
•A node group contains >1 nodes deployed in
anAmazonEC2Auto Scaling group.
•You can use scheduled scaling to shutdown workloads
20

21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown Strategy for Cluster Autoscaler
asgs=("eks-d123456543")
forasgin"${asgs[@]}"; do
awsautoscalingput-scheduled-update-group-action\
--auto-scaling-group-name"$asg"\
--scheduled-action-nameCOB\
--recurrence"0 18 * * MON-FRI"\
--time-zoneEurope/Dublin\
--min-size0\
--max-size0\
--desired-capacity0
done

22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown Strategy for Cluster Autoscaler

23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown strategy: Karpenter

24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown Strategy for Karpenter
•Automate tasks with Cronjob
•ACronJobcreatesJobson a repeating schedule
•Scheduled actions that terminates all the pods in your cluster
•Karpenter will gracefully terminate nodes if no pod is running

25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shutdown/Scale-in
Shutdown Deployments:
•Cronjob runs at specific time
•Scale deployments with replicas = 0
•Scale-in Karpenter and CoreDNS
deployments last
•Run Cronjob tasks on Fargate or dedicated
Node with no eviction
DaemonSets?
•Add nodeSelector with any non-existing
label
•-p '{"spec": {"template": {"spec":
{"nodeSelector": {"non-existing": "true"}}}}}'

26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale-Out
Scale-out Deployments:
•Cronjob runs at specific time
•DaemonSets are patched*
•Scale deployments with replicas > 0
DaemonSets?
•Remove non-existing label from nodeSelector
•-p='[{"op": "remove", "path":
"/spec/template/spec/nodeSelecto
r/non-existing"}]'

27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale-Out/Scale-In Cost Comparison

28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scheduled Scaling Solutions
Scale down the replicas to zero based on a cron schedule.
•cluster-turndown
•kube-downscaler
•Deployments
•StatefulSets
•HorizontalPodAutoscalers
•CronJobs
https://github.com/kubecost/cluster-turndown
https://codeberg.org/hjacobs/kube-downscaler

29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost Optimization Recommendations
•Autoscaling
•Right-size your cluster
•Choose the right AWS Instance
•GP3 For EBS
•Cost Tags (Automatic/Manual)
•EKS Fargate
•Container Insights
•Kubecost
•Right Processors (Intel/AMD)
•Spot

30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS Cost OptimizationKarpenter Best PracticesKubernetes Cluster Autoscaler
Additional Resources

31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session
survey in the mobile appThank you!
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session
survey in the mobile app
Olawale Olaleye
Twitter: @waleberry
Thank you!