Multi-Arch Infra From the Ground Up.pptx

Webinar: https://www.brighttalk.com/webcast/17792/588393?utm_source=ArmLtd&utm_medium=brighttalk&utm_campaign=588393

Multi-architecture infrastructures enable workloads to run on the best hardware for the task (Arm or x86) and optimize price/performance ratios while boosting design flexibility. However, migrating from a single- to a multi-arch framework can be tricky.

Join me to learn about:
- What is multi-architecture infrastructure?
- The trend for moving cloud workloads to multi-arch
- How early adopters handled the challenges
- The growing software ecosystem of Arm
- Comparing the price-performance of common workloads on x86 vs Arm
- Resources to get you started

This talk prepares developers for the road ahead with the ability to run workloads on the best hardware without needing to be concerned with the underlying architecture.

1. Cheryl Hung, Sr Director of Ecosystem, Infrastructure 25 July 2023 Multi-Arch Cloud Infrastructure from the Ground Up Why you should consider it and how to get started

2. Agenda This presentation is aimed at architects, engineering managers, DevOps and platform engineers. What is Multi-Arch Cloud Infrastructure? Use cases Web server: NGINX on Alibaba Cloud NoSQL data store: Redis on Alibaba Cloud Machine learning on AWS Graviton3 Case studies FusionAuth Honeycomb How to get started Three steps to migrating to multi-arch Example deployments Wordpress + MySQL on Azure Kubernetes Service (AKS) on Arm Github Actions on Arm-based Burstable VMs on Azure

3. About me [email protected] , Senior Director of Ecosystem, Infrastructure My mission is to get Arm to cross the chasm SW support Open source HW availability Standards bodies Developer outreach Previously engineer and eng manager at Google and Apple, and VP at Cloud Native Computing Foundation

4. What is Multi-Arch Cloud Infrastructure?

5. Cloud spending and energy usage are under pressure EU goal of climate-neutral data centers by 2030 Companies need to maximize price-performance and use resources efficiently

6. What is Multi-Arch Infrastructure? Computer architectures are either RISC (Reduced Instruction Set Computers) or CISC (Complex Instruction Set Computers). Arm’s RISC architecture targets power efficiency and performance, and can be licensed for many use cases Mobile devices Automotive Internet of Things Cloud The goal of Multi-Arch Infrastructure is to run workloads on the best hardware for their price-performance needs , without developers needing to be concerned with the underlying architecture

7. What kind of workloads can benefit? Machine learning Distributed memory cache Web servers From the Arm Community Infrastructure Solutions Blog Improve NGINX performance up to 32% by deploying on Alibaba Cloud Yitian 710 instances Improve Memcached performance up to 41% with Alibaba Cloud Yitian 710 instances AWS Graviton3 featuring Arm Neoverse V1 is up to 1.8x faster than x86 for deep learning inference workloads

8. What kind of workloads can benefit? Broad support for Arm across the top used open source projects Arm-based instances are available on every major public cloud provider AWS, Microsoft Azure, Google Cloud, Oracle, Baidu Cloud, Alibaba Cloud, Tencent Cloud, JD Cloud Lots of work to make it really easy to recompile your software for Arm (with a few tweaks). We’ll look at two real life case studies.

9. Summer 2020 Community member investigates running on Raspberry Pi Load tests on Graviton Official support; rolled out to SaaS in June Summer 2021 March 2022 >70% of SaaS instances run on Arm March 2023 A developer-focused API-first auth provider deployable anywhere

10. Technical timeline Finding the JVM that supported ARM, especially Mac ARM chips (Java 17 was the first one to do so). Added Java 17/ARM support to the code base Dec 2021 - Feb 2022 Updating and testing install scripts to use the correct JVM Updating docker to target the ARM architecture with jlink and multi-arch builds Checking for ARM support in public cloud regions when spinning up SaaS Update the application to expose the underlying architecture Performance testing and results Logins are especially CPU intensive due to password hashing, so the team load tested 50k logins. Arm handled 26-49% more logins per second and cost 8-10% less than Intel on EC2

11. “Because we run on Java, our lift was pretty small . We just had to find a JVM built for ARM, and then work out any remaining kinks” - Daniel DeGroff, FusionAuth CTO “I just switched a FusionAuth instance to arm64 and the transition was so smooth I couldn't even tell whether it's actually running the arm64 version” - Hendy Irawan , FusionAuth user and BandungPermaculture.com CIO

12. Full stack observability enabling engineers to deeply understand and debug production software March 2020 First experiments with Graviton2 Ingest workers in production Virtually all workloads and envs on Arm 92% of vCPUs on Arm May 2021 Nov 2021 Turned off last x86 EC2 instances 99% Arm on Lambda April 2022

13. Technical timeline Chose to migrate ingest workers first as they are stateless, performance critical and scales out horizontally. The ingest workers are written in Golang, so compiling for Arm was easy. They wer deployed in dogfood environment and positive results observed. Initially no Kubernetes or container orchestration, everything in Terraform and Chef so could switch Arm Amazon Machine Images (AMIs) by enumerating all the dependencies to update. Next up were own workloads (easiest to re-compile and highest compute spend) and Kafka. The last were ad-hoc one-off services and those difficult to migrate. After one year of work, the last x86 EC2 instance was turned off, and 99% was on AWS Lambda. Testing and results Saved 40% of EC2 instance bill

14. “ Graviton has enabled Honeycomb to scale up our product without increased operational toil , spend less on compute, and have a smaller environmental footprint. ” - Ian Smith, Engineering Manager, Honeycomb.io “I personally approached it as an idle experiment with a few spare afternoons and was surprised by how compelling the results were. Saving 40% on the EC2 instance bill for this service […] is well worth the investment” Liz Fong Jones, Field CTO, Honeycomb.io @oicheryl

15. “48 of the top 50 Amazon EC2 customers use AWS Graviton processors for their workloads” Danilo Poccia, Chief Evangelist (EMEA) AWS, Aug 2022

16. How to get started

17. Lots to consider in a modern cloud native infrastructure! Infra As Code CI/CD, reproducible builds Packaging, binaries, images, registries Testing, scheduling, rollout, performance testing Infrastructure upgrades

18. @oicheryl Three steps to migrating to multi-arch

19. Three steps to migrating to multi-arch Inform 2. Optimize 3. Operate Inventory your software stack (OS, container images, libraries, frameworks, runtimes, tools used to build, deploy and test, and tools used to monitor and manage) Identify hotspots and check each for Arm support (AArch64 in GCC, arm64 in Linux kernel) Provision your test Arm64 environment Upgrade container images and test Performance testing Build K8s cluster Mixed control plane and worker plane nodes Cluster creation Daemonsets Update CI/CD for reproducible Arm64 builds Canary or blue-green deployment Node affinity, taints and tolerations Different limits and requests per architecture

20. Example deployment 1: Deploy WordPress on K8s Simple deployment on Arm-based instances on Azure Kubernetes Service (AKS) Use terraform to set the cluster to deploy Arm-based VMs: Use the scripts in https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/aks-on-arm-based-ampere to deploy K8s, WordPress and MySQL

21. Example deployment 2: Saving on CI/CD Build apps with Github Actions on Arm-based Burstable VMs on Microsoft Azure

22. Example deployment 2: Saving on CI/CD Check burstable VM is accruing credits Commit to CI/CD pipeline to use the accumulated credits

23. Summary

24. Summary Cloud spending and energy usage are under pressure. Companies need to maximize price-performance and use spend efficiently. The goal of Multi-Arch Infrastructure is to run workloads on the best hardware for their price-performance needs, without developers needing to be concerned with the underlying architecture. Real life case studies show 25-40% savings and 20-60% performance increase compared to x86 Arm-based instances are available on all major public cloud providers and Arm is widely supported across the software ecosystem, both open source and commercially. To get started, use the Inform – Optimize – Operate model to find your hotspots and manage the transition smoothly. Check out the Arm Community Infrastructure Solutions Blog for tips and tricks.

25. Cheryl Hung, Sr Director of Ecosystem, Infrastructure 25 July 2023 Thank you [email protected]