Aws Multi-Account, Self-Healing, Self-Bootstrapping DevOps Pipelines

Describes how we can deploy self-healing, self-bootstrapping Pipelines across multiple AWS accounts

1. Multi-Account, Self-Healing, Self-Bootstrapping DevOps Pipelines Stephen Wilding – Cambridge User Group – February 2019

2. Aims Demonstrate how to manage tooling deployment using pipelines across a multi-account, multi- organisational environment Demonstrate how tooling can be self-healing

3. What Problem Are We Trying To Solve? As an engineer I need to ensure that all AWS are bootstrapped with my tools installed As an engineer I need to ensure that I can deploy updates to my tooling across ALL (i.e. potentially 100’s) of accounts via a single action As an engineer I need to ensure my tooling is self-healing

4. Desired Outcomes Tooling is automatically deployed Tooling can self-heal Tooling can be updated via agility and scalability that the cloud demands

5. Context I manage a single production account SEC

6. Context I need to deploy my tooling to all other accounts!!! SEC client client client client client client client client client

7. Requirements Tooling should be deployed at account bootstrap time No tooling = Account is not ready for use Tooling should be deployed via a pipeline Code repository is Microsoft VSTS (Git) Tooling should be deployed to ALL AWS accounts NB: From multiple AWS organisations Potentially using multiple provisioning mechanisms

8. How Should This Look (Logically)

9. Design Principles Must support CI/CD Must support full automation Should be serverless Should ideally be self-healing Should ideally be native (i.e. AWS or Microsoft) Should support agility

10. OK – How Should This Look (Physically) Ingredients A little bit of Lambda A dash of AWS Config (Rules)……(or Cloudwatch events) A sprinkling of S3 A smattering of Cloudformation A drop of DynamoDb , IAM & SNS And finished off with a sprig of Azure DevOps Pipelines MIX IT TOGETHER AND………

11. DEMO

12. Considerations and Next Steps? PoC end to end including more failure scenarios Need to test for multi-region and at high scale How will the pipeline perform? Config Rules can work out expensive Option to replace with Cloudwatch Events plus Lambda?? How do we handle initial artifact deployment (roles/SNS etc )? Key to the whole bootstrap What about updates? Over-time build in more self-healing granularity

13. Questions Thank You!!