End-to-End CICD at Scale With Infrastructure-As-Code On AWS

D1DEV102
End-to-End CI/CD at scale with

Infrastructure-as-Code on AWS
Bhuvaneswari Subramani (she/hers)

Director, Engineering Operations, Infor
AWS DevTools Hero
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Quick Intro
• Bhuvaneswari Subramani (Bhuvana)

• Director, Engineering Operations, Infor
• AWS Hero since 2019
• Organizer, AWS User Group Bengaluru
• Leading cloud computing, DevOps,
and Performance QA
/bhuvanas
@installjournal
https://bhuvana.pro
Agenda
• Why Infrastructure as Code (IaC) for CI CD ?
• CI CD multi-account Architecture
• Build & Deployment Infrastructure Setup
• Best Practices
Why Infrastructure as Code
for CI CD ?
Key Benefits
Speed Consistency Traceability
CI CD with multi-account
architecture
Continuous Integration
Developer Service Account
AWS Cloud
VPC Gateway
VPC
EndPoint
Private subnet
Lin 1
3
1 - Initiate Build process
2 Worker Nodes
Build & Deploy 2 - Get Source code
Server
CI 3 - Compile & generate artifacts
4 - Publish artifacts
Mirror Git
Repo
5 - Run code validation suite
AWS Cloud
4 Publish build
artifacts
VPC Gateway S3 Buckets

VPC
EndPoint
Private subnet
Lin 1
3
2 Worker Nodes
Server
Mirror Git
Repo
AWS Cloud
4 Publish build
artifacts

VPC
EndPoint
Private subnet
Lin 1
3
2 Worker Nodes
5 Server
Mirror Git
Repo
AWS Cloud
4 Publish build
artifacts

VPC
EndPoint
Private subnet
App artifacts
Lin 1
3
Config artifacts
2 Worker Nodes
Build & Deploy
5 Server
Data system artifacts
Mirror Git
Repo Test artifacts
Continuous Deployment – Nano Env Users
https://<env-id>.nano.infornexus.com
Developer Service Account Nano Environment(s) Account

AWS Cloud AWS Cloud
Availability Zone
S3 Buckets
VPC VPC
Internet gateway
Private subnet Public Subnet

Deploy
Lin
Build & Deploy Windows Linux

Server
Peer Peer
Mirror Git
Repo
Nano to Full Scale Environments
Beta
Continuous Integration – Alpha / Beta / Prod Env
AWS Cloud
VPC
Private subnet
Lin 1
3
2 Worker Nodes
Build Server 2 - Get Source code
Mirror Git
Repo
AWS Cloud
4 Publish build
artifacts

VPC
EndPoint
Private subnet
Lin 1
3
2 Worker Nodes
Mirror Git
Repo
AWS Cloud
4 Publish build
artifacts
4
VPC Gateway VPC Interface S3 Buckets
VPC
EndPoint EndPoint
Private subnet Publish container

images
Lin 1
ECR
3
2 Worker Nodes
Mirror Git
Repo
Continuous Deployment – Alpha / Beta / Prod Env
Developer Service Account Prod Account
AWS Cloud AWS Cloud
4 Publish build
artifacts
4
VPC Gateway VPC Interface S3 Buckets
VPC
EndPoint EndPoint
Private subnet Publish container

images
Lin 1
ECR
3
2 Worker Nodes
5
4 - Publish artifacts Deploy Server

Mirror Git
Repo
Developer Service Account Alpha / Beta / Prod Environment Account
AWS Cloud AWS Cloud
ECR
Publish build
artifacts
A
B Publish
VPC container VPC
images
Private subnet Private subnet

App Container Services
Logging & Monitoring

• Web Tier
• Integration Tier
Supply Chain Intelligence

Lin Intermediate Tier
Deploy Nodes
• Caching
Build Server • Lock co-ordination
• Service Discovery
Data Tier
• Queueing
Mirror Git Deploy Server • AWS Hosted Data Services
Repo • AWS Managed Data Services
AWS Cloud AWS Cloud
ECR
Publish build
artifacts
A
B Publish
VPC container VPC
images


• Web Tier

Lin C Intermediate Tier
Deploy Nodes
• Caching
Data Tier
• Queueing
AWS Cloud AWS Cloud
ECR
Publish build
artifacts
A D
D
B Publish
VPC Gateway VPC Interface
VPC container VPC
EndPoint EndPoint
images


• Web Tier

Lin C Intermediate Tier
Deploy Nodes
• Caching
Data Tier
• Queueing
Build & Deployment
Infrastructure Setup
Build Infrastructure Components
IAM Roles
S3 Bucket Mirror Git Repo VPC EndPoints
Secrets
AWS
CloudFormation
Jenkins Win Jenkins Linux ECR

Agent Agent Jenkins Master
Setup Mirror Git Repo using IaC
Create Git repo tar.gz &
Upload to S3
Create EC2 Instance
AWS Mirror Git Repo

CloudFormation Pull Git repo tar.gz from
S3 and place in EC2
Configure Cron job to

sync Git repo
Setup Jenkins Master using IaC
Pipeline
IaC Developers
Linux Agent
Bootstrap
Jenkins
Gitlab Source - IaC Create Salt Jenkins
EC2 minion Jenkins Job
Server Configuration
Windows Agent
Source - Bootstrap
Dev Environment for Jenkins Bootstrapping
Run Jenkins as Container
Install Docker
Setup local repo
Bind mount the directories into

Docker Containers
Install Jenkins as a Container
Configure Jenkins
Install Docker
Setup local repo

Docker Containers
Configure Jenkins
Install Docker
Setup local repo

Docker Containers
Configure Jenkins
Install Docker
Setup local repo

Docker Containers
Configure Jenkins
Install Docker
Setup local repo

Docker Containers
Configure Jenkins
Install Docker
Setup local repo

Docker Containers
Configure Jenkins
Best Practices
Best Practices
• Operational excellence
• Security
• Reliability
• Performance efficiency
• Cost optimization AWS Well-Architected Framework
• Sustainability
Operational excellence
• Continually improving quality process
• Design for failure
• Opportunities for improvement
• Clearly defined Promotion model for IaC
Operational Excellence – Promotion Model
• Holds IaC for services that are live on an Env
Stable • Protected Branch ; Changes needs approval
• Holds IaC for services ready to go live

Staging • Protected Branch ; Changes needs approval
• To apply changes to production or

Feature alpha branch
{Promote to Stable for
go live}
[STAGING]
[FEATURE-1]
{Create} {rebase} {promote}
[ … ]
[FEATURE-N] {Create}
Conventions
o commit
+ branch create
* rebase
^ promote
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. \/ code flow
[ ] branch
{Promote to Stable for
go live}
[STAGING]
[FEATURE-1]
{Create} {rebase} {promote}
[ … ]
[FEATURE-N] {Create}
[STABLE]
{Code flow to Staging}
[FEATURE-1]{Create} {rebase} {promote}

Conventions
[ … ] o commit
[FEATURE-N] + branch create
{Create} * rebase
^ promote
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. \/ code flow
[ ] branch
Security
• Infrastructure protection
• Centralized identities with SAML 2.0
• End-to-end Traceability
• Data Protection
Reliability
• Monitor workload Resources
• Adapt to changes in Demand
• Implement resiliency for reliable workload
Performance Efficiency
• Capability to deploy the workload in multiple
AWS Regions
• Use serverless architectures
• Monitor your resources
Cost Optimization
• Implement cloud financial management
• Monitor Cost and Usage
• Stop spending money on undifferentiated
heavy lifting
Sustainability
• Remove or refactor workload components with
low or no use
• Increase utilization of build environments
• Optimize areas of code that consume the most
time or resources
Key Benefits
Speed Consistency Traceability
Resources
https://aws.amazon.com/devops/
DevOps and AWS
https://docs.aws.amazon.com/whitepapers/latest/introduction-devops-
aws/infrastructure-as-code.html
AWS Infrastructure as Code
https://aws.amazon.com/architecture/well-architected/
AWS Well Architected Framework
https://aws.amazon.com/developer/community/heroes
AWS Heroes
Learn in-demand AWS Cloud skills
AWS Skill Builder AWS Certifications

Access 500+ free digital courses Earn an industry-recognized
and Learning Plans credential
Explore resources with a variety Receive Foundational,

of skill levels and 16+ languages Associate, Professional,
to meet your learning needs and Specialty certifications
Deepen your skills with digital Join the AWS Certified community
learning on demand and get exclusive benefits
Access new
Train now exam guides
Thank you!
Bhuvaneswari Subramani
/bhuvanas
@installjournal
https://bhuvana.pro
!
Please complete
the session survey

End-to-End CICD at Scale With Infrastructure-As-Code On AWS

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

End-to-End CICD at Scale With Infrastructure-As-Code On AWS

Uploaded by

Copyright:

Available Formats

D1DEV102

End-to-End CI/CD at scale with

Bhuvaneswari Subramani (she/hers)

• Bhuvaneswari Subramani (Bhuvana)

• Why Infrastructure as Code (IaC) for CI CD ?

• Build & Deployment Infrastructure Setup

Speed Consistency Traceability

VPC Gateway S3 Buckets

VPC Gateway S3 Buckets

VPC Gateway S3 Buckets

Developer Service Account Nano Environment(s) Account

Private subnet Public Subnet

Build & Deploy Windows Linux

CI 3 - Compile & generate artifacts

VPC Gateway S3 Buckets

CI 3 - Compile & generate artifacts

Private subnet Publish container

CI 3 - Compile & generate artifacts

Private subnet Publish container

4 - Publish artifacts Deploy Server

Private subnet Private subnet

Logging & Monitoring

Supply Chain Intelligence

Private subnet Private subnet

Logging & Monitoring

Supply Chain Intelligence

Private subnet Private subnet

Logging & Monitoring

Supply Chain Intelligence

Jenkins Win Jenkins Linux ECR

Create EC2 Instance

AWS Mirror Git Repo

Configure Cron job to

Setup local repo

Bind mount the directories into

Install Jenkins as a Container

Setup local repo

Bind mount the directories into

Install Jenkins as a Container

Setup local repo

Bind mount the directories into

Install Jenkins as a Container

Setup local repo

Bind mount the directories into

Install Jenkins as a Container

Setup local repo

Bind mount the directories into

Install Jenkins as a Container

Setup local repo

Bind mount the directories into

Install Jenkins as a Container

• Cost optimization AWS Well-Architected Framework

• Continually improving quality process

• Design for failure

• Opportunities for improvement

• Clearly defined Promotion model for IaC

• Holds IaC for services ready to go live

• To apply changes to production or

[FEATURE-1]{Create} {rebase} {promote}

• Centralized identities with SAML 2.0

• Monitor workload Resources

• Adapt to changes in Demand