Getting Started with DevOps on AWS [Mar 2020]

AWS DevOps tools are a set of services designed to facilitate continuous integration and continuous delivery (CI/CD) practices, enabling faster and more reliable software development.

This presentation highlights important tools for orchestrating the release process

2. DevOps Defined
DevOps is a combination of cultural philosophies,
practices and tools that increases an organizations
ability to deliver applications and services at high
velocity.
Amazon Web Services

3. This is a Level 100-200 Session

4. Start Anywhere
But
Start Somewhere

5. How Amazon does DevOps?
Decompose for Agility == Microservices
Automate everything == CI/CD

Standardized Tools
Belts and Suspenders == Templates or Compliances
Infrastructure as a Code

6. Google Search Trends for DevOps

7. Developer and IaaC Tools
CodeCommit CodeBuild CodePipeline CodeDeploy Terraform

8. AWS Core Services
●Availability Zones for resiliency
●IAM
●S3
●Key Management Service (KMS)
●CloudTrail
●CloudWatch Logs
●CloudWatch Events
●CloudFormation
●SNS

9. Demo Application Definition
●A Simple NodeJS / Express App
●ECS Fargate with Load Balancer

10. CodeCommit
●Secure and Scalable Git hosted on AWS
●Pay as you Go
●Integrated with IAM for granular control
●Approval Rules for Pull Requests
●Data is Encrypted in Transit and At Rest via KMS
●Core Services in use
○IAM, S3, KMS

11. Setup SSH Access
$>cat ~/.ssh/config
Host appgambit
HostName git-codecommit.us-east-1.amazonaws.com
User APKAY4AJ6VUH4I22GU5Q
IdentityFile ~/.ssh/id_rsa

$>git clone ssh://appgambit/v1/repos/test-app

If you are not able to connect to the repo, run this command to validate your SSH conntion
$>ssh [email protected]

$>cat ~/.ssh/config
Host git-codecommit.*.amazonaws.com
User APKAY4AJ6VUH4I22GU5Q
IdentityFile ~/.ssh/id_rsa

12. IAM Policy - Allow commit to master branch
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codecommit:GitPush",
"codecommit:Merge*"
],
"Resource": "*",
"Condition": {
"StringEqualsIfExists": {
"codecommit:References": [
"refs/heads/master"
]
}
}
}
]
}

13. IAM Policy - Deny Commit to Master branch
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": [
"codecommit:GitPush",
"codecommit:DeleteBranch",
"codecommit:PutFile",
"codecommit:Merge*"
],
"Resource": "arn:aws:codecommit:us-east-2:111111111111:test-app",
"Condition": {
"StringEqualsIfExists": {
"codecommit:References": [
"refs/heads/master"
]
},
"Null": {
"codecommit:References": false
}
}
}
]
}

14. CodeBuild
●Fully managed build service
●Auto-provision, manage and scale build servers
●Pay as you build, pay only for the number of build minutes
●Fully customize the build process with pre_build, build and post_build
commands
●Intergrate with CodePipeline or Jenkins

15. Sample buildspec.yml file

version: 0.2

phases:
install:
runtime-versions:
docker: 18
commands:

pre_build:
commands:
- echo Logging in to Amazon ECR....
- aws --version
- $(aws ecr get-login --no-include-email --region $REGION)
- COMMIT_HASH=$(echo
$CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-8)
- echo Latest commit hash $COMMIT_HASH


build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t test-app:latest .
- docker tag test-app:latest $REPO_URL:latest
- docker tag $REPO_URL:latest $REPO_URL:$COMMIT_HASH

post_build:
commands:
- echo Build completed on `date`
- echo pushing to repo
- docker push $REPO_URL:latest
- docker push $REPO_URL:$COMMIT_HASH

artifacts:
files:
- taskdef.json
- appspec.yaml

16. CodePipeline
Source Build Deploy
CodePipeline
CodeCommit CodeBuild CodeDeploy
Source code changes are
pushed to the repo
Update running environment

17. CodePipeline
●Automate your release process
●Configurable visual workflow
●Number of built-in stages and actions, like Source, Build, Test, and Deploy
●Easy to customize
●Pay as you go, $1 per active pipeline, this is an integration service
●And most important, Rapid Delivery

18. CodePipeline Stages and Actions
●Source
○S3, ECR, CodeCommit, Github, etc
●Build
○CodeBuild, Jenkins, etc
●Test
○CodeBuild, Device Farms, Jenkins, 3rd party services, etc
●Deploy
○S3, CloudFormation, CodeDeploy, ECS, Alexa, etc
●Approve
○Manual
●Invoke
○AWS Lambda

19. Continuous Integration (CI)
So far...
●We have created and pushed our code in CodeCommit
●We are using the CodeBuild to build the Docker Images
●We are using the CodeBuild to upload the Docker Images to ECR
And
●We are using the CodePipeline to automate the above process
●We can add test scripts and commands to validate the integration

20. Let’s check our app using ECS Fargate
●Create ECS Cluster
●Create a Task Definition
●Create a Service
●Validate the application

21. Everything works!
●So far everything works, but our ECS cluster is not taking the new changes
automatically
●It creates the build and pushes the docker image to the ECR repo only

22. CodeDeploy
●Automated Deployments
●Deploy to ECS, Lambda, EC2, On-Prem Instances
●Rolling and Blue/Green Updates
●Stop or Rollback the Deployment

23. Now we have a fully automated application
pipeline...

WITH ONE BIG PROBLEM

But the Infrastructure to support that is entirely
manual...

24. Infrastructure as a Click
●While our app works with automation
●But the infrastructure for this whole setup is done by “clicks”
●Imagine if we need to replicate this same after a few days or few weeks
●Infrastructure Automation or Infrastructure as a Code is MOST IMPORTANT
THING for a growing application or organization

25. Infrastructure as a Code
●Easy to Replicate
●Easy to Manage
●Supports Changes and Rollbacks
●Versioning

26. Terraform
●Declarative configuration files
●Modules which can be imported
●Supports multiple cloud providers
●Uses Hashicorp Configuration Language

27. Terraform VPC Example
provider "aws" {
region = "${var.aws_region}"
}

resource "aws_vpc" "default" {
cidr_block = "10.0.0.0/16"
}

resource "aws_internet_gateway" "default" {
vpc_id = "${aws_vpc.default.id}"
}

resource "aws_route" "internet_access" {
route_table_id = "${aws_vpc.default.main_route_table_id}"
destination_cidr_block = "0.0.0.0/0"
gateway_id = "${aws_internet_gateway.default.id}"
}



resource "aws_subnet" "default" {
vpc_id = "${aws_vpc.default.id}"
cidr_block = "10.0.1.0/24"
map_public_ip_on_launch = true
}

28. Other tools to checkout
AWS Chatbot
AWS CodeStar
Jenkins
Terraform CI/CD template to run containers with ECS Fargate
https://github.com/AppGambitStudio/aws-terraform-cicd
AWS OpsWorks

29. Where to start
●https://www.docker.com/
●https://aws.amazon.com/devops/what-is-devops/
●https://aws.amazon.com/devops/
●https://www.terraform.io/
●https://aws.amazon.com/serverless/

30. Sometimes it’s best to follow the
dotted line, instead of connecting
the dots.
Founder of CTO.ai @ re:Invent 2019