Clean Your Cloud with Cloud Custodian slides

One of the great things about using the cloud is how easy it is to spin up resources and use them. The downside is it is really easy to spin up resources and forget about them. Sprawl becomes a real problem. You waste money if you fail to shut them down when you finish with them or even overnight. And it isn’t just costs. Managing a cloud infrastructure can be a real pain. Who spun this up? What project is it for? Does it need to be shut down? When? Shouldn’t it be encrypted? Cloud Custodian is an easy-to-use framework that allows simple YAML files to specify how to gather, filter, and act on cloud resources across accounts and providers. Generate reports, automatically tag items, shut down non-compliant resources, or even reconfigure them to be compliant. Join Gene as he shows you how to manage your cloud resources.

1. Clean Your Cloud
with Cloud Custodian
Gene Gotimer
Principal DevOps Engineer at Praeses, LLC
@OtherDevOpsGene

2. Why?
•Real-time compliance
•Cost control
•Governance-as-Code
@OtherDevOpsGene @techwell #agiledevopscon 2

3. What?
•Simple domain-specific language (DSL)
•Abstracts the API for each cloud provider
•Modify resources
•Enforce policy
•Control resources
•Generate reports
•Control account
•Across accounts
@OtherDevOpsGene @techwell #agiledevopscon 3

4. Where?
•Amazon Web Services (AWS)
•Microsoft Azure
•Google Cloud Platform (GCP)
•Others in beta
@OtherDevOpsGene @techwell #agiledevopscon 4

5. How?
•YAML files
•Cloud-specific actions, resources, and filters
•Similar syntax, but not cloud agnostic
@OtherDevOpsGene @techwell #agiledevopscon 5

6. Installing Cloud Custodian
$pip install c7n
$pip install c7n-azure c7n-gcp c7n-org
@OtherDevOpsGene @techwell #agiledevopscon 6

7. Modify resources
@OtherDevOpsGene @techwell #agiledevopscon 7

8. Scenario: Add project tags
•We’ve been using our AWS account to stand up instances and
storage for our only project
•Another project is starting up
•We want to add tags so we know which resource belongs to which
project
@OtherDevOpsGene @techwell #agiledevopscon 8

9. Policy to add tags
policies:
-name: add-project-tag-to-ec2
resource: aws.ec2
actions:
-type: tag
key: project
value: cloud-custodian-demo
@OtherDevOpsGene @techwell #agiledevopscon 9

10. Applying a policy
$custodian validate 01-add-tags.yml
$custodian run --dry-run --output-dir=output 01-add-tags.yml
$custodian run --output-dir=output 01-add-tags.yml
@OtherDevOpsGene @techwell #agiledevopscon 10

11. Demo: 01-add-tags.yml
@OtherDevOpsGene @techwell #agiledevopscon 11

12. Getting detailed help
$custodian schema aws
$custodian schema aws.ec2
$custodian schema aws.ec2.actions
$custodian schema aws.ec2.actions.tag
@OtherDevOpsGene @techwell #agiledevopscon 12

13. Scenario: Add more project tags
•We’ve been using our AWS account to stand up instances and
storage for our only project
•Another project is starting up
•We want to add tags so we know which resource belongs to which
project
•We want a contact email and which group to bill for each resource
@OtherDevOpsGene @techwell #agiledevopscon 13

14. Demo: 02-add-tags.yml
@OtherDevOpsGene @techwell #agiledevopscon 14

15. Caching
$custodian run --dry-run --output-dir=output 02-add-tags.yml
... policy:add-required-tags-to-ec2 resource:aws.ec2 region:us-east-1 count:0time:0.00
$custodian run --dry-run --cache-period=0--output-dir=output 02-add-tags.yml
... policy:add-required-tags-to-ec2 resource:aws.ec2 region:us-east-1 count:2time:0.32
@OtherDevOpsGene @techwell #agiledevopscon 15

16. Enforce policy
@OtherDevOpsGene @techwell #agiledevopscon 16

17. Scenario: Required tags
•We’ve decided on three tags all team members should use:
•project
•owner
•bill-to
•We want to stop EC2 instances from running if they are missing
those tags
@OtherDevOpsGene @techwell #agiledevopscon 17

18. Policy to stop if tags are missing
policies:
-name: stop-ec2-with-missing-tags
resource: aws.ec2
comment: |
Stop all EC2 instances missing any of our
required tags.
filters:
-or:
-"tag:project": absent
-"tag:owner": absent
-"tag:bill-to": absent
actions:
-stop
@OtherDevOpsGene @techwell #agiledevopscon 18

19. Demo: 03-enforce-tags.yml
@OtherDevOpsGene @techwell #agiledevopscon 19

20. Scheduling
policies:
-name: stop-ec2-with-missing-tags
resource: aws.ec2
mode:
type: periodic
schedule: "rate(1 day)"
role: arn:aws:iam::{account_id}:role/some-role
comment: |
Stop all EC2 instances missing any of our
required tags.
filters:
…
@OtherDevOpsGene @techwell #agiledevopscon 20

21. Delayed actions
•To take action in the future, use:
•action mark-for-op
•filter marked-for-op
•Apply the condition to a new tag with the op and days in the future
•Every day:
•mark-for-op if the condition isn’t met and it isn’t already marked
•remove-tag if the condition has been remedied
•act if the tag is marked-for-op
@OtherDevOpsGene @techwell #agiledevopscon 21

22. Scenario: Required tags, or else
•We’ve decided on three tags all team members should use:
•project
•owner
•bill-to
•We want to stop EC2 instances from running if they are missing
those tags
•We want to terminate those instances after a week
@OtherDevOpsGene @techwell #agiledevopscon 22

23. Demo: 04-enforce-tags-scheduled.yml
@OtherDevOpsGene @techwell #agiledevopscon 23

24. Demo: 05-enforce-tags-manual.yml
@OtherDevOpsGene @techwell #agiledevopscon 24

25. Generate reports
@OtherDevOpsGene @techwell #agiledevopscon 25

26. Seeing what was applied
$custodian report --output-dir=output 05-enforce-tags-manual.yml # --format csv
$custodian report --output-dir=output 05-enforce-tags-manual.yml --format simple
$custodian report --output-dir=output 05-enforce-tags-manual.yml --format grid
$custodian report --output-dir=output 05-enforce-tags-manual.yml --format json
@OtherDevOpsGene @techwell #agiledevopscon 26

27. Policy without action for reporting
policies:
-name: ec2-instances-by-bill-to
resource: aws.ec2
comment: |
Report on EC2 instance by bill-to.
filters:
-"tag:bill-to": not-null
@OtherDevOpsGene @techwell #agiledevopscon 27

28. Demo: 06-billing-report.yml
@OtherDevOpsGene @techwell #agiledevopscon 28

29. Running a custom report
$custodian run --cache-period=0 --output-dir=output 06-billing-report.yml
…policy:ec2-instances-by-bill-to resource:aws.ec2 region:us-east-1 count:5 time:0.49
$custodian report --output-dir=output 06-billing-report.yml --format simple
$custodian report --output-dir=output 06-billing-report.yml --format simple \
--field Owner=tag:owner--field Project=tag:project
@OtherDevOpsGene @techwell #agiledevopscon 29

30. Control resources
@OtherDevOpsGene @techwell #agiledevopscon 30

31. Scenario: Shut down after hours
•We want to stop unused instances at night and on weekends
•We want them started before business hours on weekdays
•We want this opt-in
•We want the schedules to be customizable per-instance
@OtherDevOpsGene @techwell #agiledevopscon 31

32. Demo: 07-off-hours.yml
@OtherDevOpsGene @techwell #agiledevopscon 32

33. Control account
@OtherDevOpsGene @techwell #agiledevopscon 33

34. Across accounts
@OtherDevOpsGene @techwell #agiledevopscon 34

35. Policies across multiple accounts
•Run with c7n-org
•Need a role to run Custodian
•Same named role in each account
•Create an accounts configuration file
•In policies, {account_id} will be automatically replaced
@OtherDevOpsGene @techwell #agiledevopscon 35

36. Account config file
accounts:
-account_id: "983430165732"
name: parent-account
regions:
-us-east-1
-us-east-2
role: arn:aws:iam::983430165732:role/CloudCustodian
-account_id: "732829343588"
name: classroom-account
regions:
-us-east-1
-us-east-2
role: arn:aws:iam::732829343588:role/CloudCustodian
@OtherDevOpsGene @techwell #agiledevopscon 36

37. Applying cross-account policies
# Was
$custodian run --output-dir=output 07-off-hours.yml
# Cross-account
$c7n-org run --config=accounts.yml--output-dir=output --use=07-off-hours.yml
@OtherDevOpsGene @techwell #agiledevopscon 37

38. Wrap-up
@OtherDevOpsGene @techwell #agiledevopscon 38

39. Key takeaways
•Use Cloud Custodian to manage resources.
•Tag your resources to make filtering easy.
•Use schedules to automate policies and cost controls.
@OtherDevOpsGene @techwell #agiledevopscon 39

40. Resources
•Cloud Custodian: https://cloudcustodian.io/
•Reference documentation
•Example policies
•Demos: https://github.com/OtherDevOpsGene/cloud-custodian
•Infrastructure-as-code for the workspace and demos
•Policies
@OtherDevOpsGene @techwell #agiledevopscon 40

41. Please rate
the sessions
Wednesday
Concurrent sessions
(W2) Gene Gotimer -Clean Your Cloud with Cloud Custodian
@OtherDevOpsGene @techwell #agiledevopscon 41

42. Questions?
Gene Gotimer
Principal DevOps Engineer at Praeses, LLC
@OtherDevOpsGene
@OtherDevOpsGene @techwell #agiledevopscon 42