Automating the development pipeline

DevOps Culture
 Dev Ops

3
4
Dev

Wall of Confusion
Ops
Development + Operations = DevOps

Dev Ops

5
What is DevOps?
• DevOps is a combination of:
• Cultural philosophies

• Practices

• Tools

6
DevOps Culture
• Dev & Ops coming together
• No more “silos”
• Shared responsibility
• Ownership
• Visibility and communication

People over Process over Tools

7
DevOps Practices
• Microservices
• Moving away from “monolithic” application architecture to many individual services

8
DevOps Practices
• Continuous integration
• Continuous delivery & deployment

9
DevOps Practices
• Infrastructure as code
• Model your AWS resources using code

10
DevOps Practices
• Monitoring and logging
• Track and analyze metrics and logs

• Understand real-time performance of infrastructure and application

11
Benefits of DevOps

Improved Collaboration Rapid Delivery Reliability

Speed
Security Scale
12
 Comprehensive set of enterprise-grade tools
Comprehensive set of enterprise-grade tools
CI/CD Tools Infrastructure as Code IDE Monitoring & Tracing Web Apps

AWS Cloud Dev.

Kit (CDK)

IDE and DevOps Toolkits CLI and Scripting Tools Languages Mobile

Visual Studio IntelliJ PyCharm Visual Studio Eclipse VSTS Amazon AWS
Code AWS CLI Tools for Corretto Amplify
PowerShell

SDKs

JavaScript Python PHP .NET Ruby Java Go Node.js C++

13
Cloud9
AWS Cloud9

• Cloud based integrated development environment (IDE)

• Allows write, execute, and debug code in your browser
• Share your environment with your team and collaborate in real
time
• Direct access to the terminal
• Great serverless development experience: Conduct local testing,
included SDK, libraries, plug-ins, etc.

16
Full Editor

17
Wide selection of Run Times

18
Full debug

19
Integrated Tools for Serverless Development

20
AWS CloudFormation
AWS CloudFormation
• Simplified way to create and manage a collection of AWS
resources
• Enables orderly and predictable provisioning and updating of
resources
• Enables version control of your AWS infrastructure
• Deploy and update stacks using the AWS Management Console,
the AWS Command Line Interface (CLI), or the AWS API
• Only pay for the resources you create

22
Overview

A running
environment
Template

AWS Stack
CloudFormation

API calls made on

your behalf

• JSON/YAML format template • FREE – you only pay for resources

• Presents template to AWS CloudFormation • All regions
• AWS CloudFormation translates it to an API • APIs are called in parallel
request • Manages dependencies/relationships
• Forms a stack of resources

23
 Infrastructure
Infrastructure as as code
code
Template
• Single source of truth to deploy the whole stack
• Infrastructure that you can replicate, re-deploy, and re-
purpose
• Control versioning on your infrastructure and your
Stack Stack Stack
application together
• Service rolls back to the last good state on failures
• Build your infrastructure and run it through your CI/CD
pipeline

24
Stack creation
• You use a template to create and manage
a stack

• A stack is a collection of AWS resources

that you can manage as a single unit

• AWS CloudFormation ensures all stack

resources are created or deleted as
appropriate

25
 Create and manage a stack using the AWS CLI

Install the AWS CLI using installation guide

aws cloudformation create-stack

--stack-name ec2InstanceCmdLineDemo
--template-url https://s3.amazonaws.com/cf-templates-deloitte-
workshop/Demo-1.json
--parameters ParameterKey=KeyPair,ParameterValue=KeyName

Returns the details of the created stack, in the output format of your choice
Returns the details of the created stack, in the output format of your choice

arn:aws:cloudformation:us-east-1:496891363831:stack/t1/625f07c0-1fef-
11e8-a501-50d5ca63261e

26
Template anatomy
{
"AWSTemplateFormatVersion" : "version date",
"Description" : ”Just another JSON String”,
"Metadata" : {
JSON objects that provide additional information about the template.
},
"Parameters" : {
Specifies values that you can pass in to your template at runtime
(when you create or update a stack )
},

1. Format version "Mappings" : {

Specifies values that you can pass in to your template at runtime

2. Transform (new) },
(when you create or update a stack)

3. Description "Conditions" : {
Defines conditions that control whether certain resources are created

4. Metadata or whether certain resource properties are assigned a value during stack creation
or update.

5. Parameters },
"Resources" : {

6. Mappings Specifies the stack resources and their properties, such as an

Amazon Elastic Compute Cloud instance or an Amazon Simple Storage Service

7. Conditions },
bucket.

8. Resources* (required) "Outputs" : {

Describes the values that are returned whenever you view your stack's properties.

9. Outputs }
}

Reference: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-ref.html
27
Template anatomy

28
Continuous integration &
Continuous deployment
AWS Code Services
Software Release Steps

Source Build Test Production

30
AWS Code Services
Software Release Steps

Source Build Test Production

AWS CodeCommit

31
AWS Code Services
Software Release Steps

Source Build Test Production

AWS CodeBuild

32
AWS Code Services
Software Release Steps

Source Build Test Production

AWS Third Party

CodeBuild Tooling

33
AWS Code Services
Software Release Steps

Source Build Test Production

AWS CodeDeploy

34
AWS Code Services
Software Release Steps:

Source Build Test Production

AWS CodeCommit AWS Third Party

AWS CodeBuild AWS CodeDeploy
CodeBuild Tooling

AWS CodePipeline

35
AWS CodeCommit

• A secure, managed and scalabe Git repository

• Same standard Git tooling
• Scalability, availability and durability from Amazon S3
• Encryption at rest using customer keys
• Unlimited size repository
• Post-commit triggers can invoke Amazon SNS or Lambda

36
Control source code on the cloud

Safe Managed Highly Store

Available any object

37
 AWS CodeCommit

Git objects on
Amazon S3

Git index on
Amazon DynamoDB
SSH or HTTPS

Git pull/push AWS CodeCommit Cryptography keys

on AWS KMS

38
Same Git Experience
$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli
Cloning into 'aws-cli'...
Receiving objects: 100% (16032/16032), 5.55 MiB | 1.25 MiB/s, done.
Resolving deltas: 100% (9900/9900), done.
Checking connectivity... done.
$ nano README.rst
$ git commit -am 'updated README'
[master 4fa0318] updated README
1 file changed, 1 insertion(+)

39
AWS CodeBuild

• Manage build server that compiles source-code, executes tests

and builds software packages
• Continuously scale and process several builds in parallel
• You can create custom build environments using Docker images
• Pay only for what you have used
• Native integration with AWS CodePipeline and Jenkins

40
How it works?
1. Downloads the source code
2. Runs the defined commands from buildspec file using temporary containers ( new
containers are created for each build)
3. Send build logs to the console and Amazon CloudWatch logs
4. Send the built artifacts to a Amazon S3 bucket

41
 File Example buildspec.yml
version: 0.1

environment_variables:
plaintext:
JAVA_HOME: "/usr/lib/jvm/java-8-openjdk-amd64"

phases:
install:
commands:
- apt-get update -y
- apt-get install -y maven
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn install
post_build:
commands:
- echo Build completed on `date`
artifacts:
type: zip
files:
- target/messageUtil-1.0.jar
discard-paths: yes
43
Building your code
• Typically, the Build process refers to the source code developed
using programming languages that needs to be compiled:
• .NET: C#, F#, VB.net, etc.

• Java e JVM: Java, Scala, JRuby

• Go

• iOS: Swift, Objective-C

• We also used the term Build to refer to Docker image creation

EC2

44
And if you don't need to build…
• Several programming languages doesn’t need to be compiled.
They are considered interpreted languages:
• PHP

• Ruby

• Python

• Node.js
• You just need to deploy your source code
EC2

45
AWS CodeDeploy
• Automates code deployments in any instance

• Handles the complexity of updating applications

• Prevents unavailability during deploy

• Performs automatic rollback in case of failure

• Deploy on Amazon EC2 or on-premises servers, using any

language and any operating system

• Integrates with third-party tools and AWS

46
File Example appspec.yml
version: 0.0
os: linux
files:
Send application files to one
- source: /
directory and configuration files to
destination: /var/www/html
another
permissions:
- object: /var/www/html
pattern: “*.html” Set specific permissions on
owner: root directories and files.
group: root
mode: 755
hooks:
ApplicationStop:
- location: scripts/deregister_from_elb.sh
Remove / add instances to Elastic
BeforeInstall:
Load Balancing
- location: scripts/install_dependencies.sh
Install dependency packages
ApplicationStart:
Start Apache
- location: scripts/start_httpd.sh
Confirm deployment success
ValidateService:
- location: scripts/test_site.sh
- location: scripts/register_with_elb.sh

47
Choose speed and deployment group
Dev deployment group

One at a time
v2 v1 v1 v1 v1 v1 Agent Agent

OR
Half at a time Prod deployment group

v2 v2 v2 v1 v1 v1
Agent Agent Agent

All at once
v2 v2 v2 v2 v2 v2 Agent Agent Agent

48
AWS CodePipeline

• Continuous delivery service for fast and reliable application

updates

• Model and visualize your software release process

• Builds, tests, and deploys the code whenever a change occurs

• Integrates with third-party tools and AWS

49
 AWS CodePipeline
MyApplication
Source

Source
GitHub

Build

AWS CodeBuild Stage Pipeline

AWS CodeBuild Action

Transition
Deploy
JavaApp
AWS Elastic
Beanstalk
 AWS CodePipeline
MyApplication
Source

Source
GitHub

Build

AWS CodeBuild Notify Developers

AWS CodeBuild AWS Lambda

Parallel actions
Deploy

JavaApp
Elastic Beanstalk
 AWS CodePipeline
MyApplication
Source

Source
GitHub

Build

AWS CodeBuild NotifyDevelopers

AWS CodeBuild Lambda
Sequential actions
Test API
Runscope

Deploy

JavaApp
Elastic Beanstalk
 AWS CodePipeline
MyApplication
Build
AWS CodeBuild
AWS CodeBuild

Staging-Deploy

JavaApp
Elastic Beanstalk

QATeamReview
Manual Approval
Manual approvals
Review

Prod-Deploy
JavaApp
Elastic Beanstalk
AWS Elastic Beanstalk
AWS Elastic Beanstalk
The easiest way to get your web applications up and running in minutes

Elastic Beanstalk is a managed service that automatically handles

• Infrastructure provisioning and configuration
• Deployment
• Load balancing
• Auto scaling Provision / Manage /
Deploy
Configure Monitor
• Health monitoring
• Analysis and debugging
• Logging

There is no additional charge for Elastic Beanstalk

55
AWS Elastic Beanstalk Supports

Platforms containing the most popular

runtimes as well as Docker images, allowing
you to bring your own

Elastic Beanstalk also supports custom

platforms and custom images

56
How Elastic Beanstalk works

Your code

HTTP Server
Elastic Beanstalk
Application Server
• Bundled your source code together
with the corresponding platform (OS, Language Interpreter
runtime, application server, and web
server) Operating System
• Safely deployed your web application
Host

57
AWS services are plug and play
You can adopt a systematic approach to CI/CD tools and methodologies while leveraging
existing investments

1 2 3 4

AWS
GitHub Jenkins CodeBuild
repository Pipeline

58
Challenge: Extending to CI/CD
Or you can create robust but simple CI/CD across release stages (dev, staging, production)
with increasingly safer deployment status, deployment failure reporting, and ongoing health
monitoring using AWS services.

1 2 3 4

AWS AWS AWS

CodeCommit CodePipeline CodeBuild

59
Questions?
Lab 2
• Go to https://aws.qwiklabs.com
• Login with your account.
• If you don’t have an account, register and then activate the account using the email that you
will receive.
• Start the Lab 2, and read the instructions.
• If you have doubts about the activity, please ask the trainer to receive help.
• You have 45 minutes to complete this lab.

61