Jinesh Varia

Technology Evangelist

jvaria@amazon.com

Architectural
Design Patterns in
Cloud Computing

They sent me here to talk

But I am here to listen

Please Send Feedback

jvaria@amazon.com
Twitter: @jinman

Cloud Best Practices Whitepaper

Prescriptive guidance to Cloud Architects

Just Google for Cloud Best

Practices to find the link

Cloud Computing Attributes

What makes the Cloud so attractive
Abstract
Resources
On-Demand
Provisioning

Focus on your needs, not on hardware specs. As your

needs change, so should your resources.
Ask for what you need, exactly when you need it. Get rid
of it when you dont need

Scalability

Scale out or in depending on usage needs.

No Up-Front
Costs

No contracts or long-term commitments.

Pay only for what you use.

Efficiency of
Experts

Utilize the skills, knowledge and resources of experts.

TheCloud
Cloud
The Living and Evolving
AWS services and features
Most Applications Need:
1. Compute
2. Storage
3. Messaging
4. Payment
5. Distribution
6. Scale
7. Analytics

The Living
and Evolving
At Amazon,
Every
Day is a Cloud
Launch Day
New Features and Services

Reserved Instances in EU Region

Elastic MapReduce
SQS in EU Region

New SimpleDB Features

FPS General Availability

AWS Multi-Factor Authentication

Virtual Private Cloud
Lower Reserved Instance Pricing

AWS Security Center

Amazon EC2 with Windows Server

2008,
Spot Instances,
Boot from Amazon EBS
Amazon CloudFront Streaming
Amazon VPC enters Unlimited Beta
AWS Region in Northern California
International Support for AWS
Import/Export

Amazon RDS
High-Memory Instances
Lower EC2 Pricing

Amazon EC2 with Windows

Amazon CloudFront
Amazon Elastic MapReduce
Amazon EC2 in EU Region
Private Content
in Europe
AWS Toolkit for Eclipse
SAS70 Type II Audit
Amazon EC2 Reserved
AWS SDK for .NET
Instances
Amazon EC2 Reserved Instances
AWS Import/Export
EBS Shared Snapshots
with Windows, Extra Large High
New CloudFront Feature
SimpleDB in EU Region
Lower pricing tiers for
Memory Instances
Monitoring, Auto Scaling &
Monitoring, Auto Scaling &
Amazon CloudFront

Amazon
S3 Versioning Feature
Elastic Load Balancing
Elastic Load Balancing in EU
AWS Management Console
Consolidated Billing for AWS
Lower pricing for Outbound Data
Transfer

Scalability
Build Scalable Architecture on AWS

A scalable architecture is critical to take advantage of a scalable

infrastructure

Characteristics of Truly Scalable Service

Increasing resources results in a proportional increase in
performance
A scalable service is capable of handling heterogeneity
A scalable service is operationally efficient
A scalable service is resilient
A scalable service becomes more cost effective when it
grows

Cloud Architecture Lessons

using Amazon Web Services

1.
2.
3.
4.
5.
6.
7.

Design for failure and nothing fails

Loose coupling sets you free
Implement Elasticity
Build Security in every layer
Don't fear constraints
Think Parallel
Leverage different storage options

1. Design for Failure

and nothing will really fail

"Everything fails, all the time"

Werner Vogels, CTO Amazon.com
Avoid single points of failure
Assume everything fails, and design backwards
Goal: Applications should continue to function even if the

underlying physical hardware fails or is removed or replaced.

Design for Failure with AWS

Tools to make your life easier

Use Elastic IP addresses for consistent and re -mappable routes

Use multiple Amazon EC2 Availability Zones (AZs)
Create multiple database slaves across AZs
Use real-time monitoring (Amazon CloudWatch)
Use Amazon Elastic Block Store (EBS) for persistent file systems

YourWebTwoDotZeroName.com

EC2 Instance A

EC2 Instance B

EC2 Instance A

DATA
Volume

Availability Zone 2

Availability Zone 1

YourWebTwoDotZeroName.com

EC2 Instance B

LOG
Volume

LOG
Volume

Amazon S3

DATA
Volume

2. Build Loosely Coupled Systems

The looser they're coupled, the bigger they scale

Independent components
Design everything as a Black Box
De-coupling for Hybrid models
Load-balance clusters

Use Amazon SQS as Buffers

Tight Coupling

Controller A

Loose Coupling
using Queues

Controller B

Q
Controller A

Controller C

Q
Controller B

Controller C

MyWebSite.com
Exterior Firewall Hardware
or Software Solution to open
standard Ports (80, 443)

Web Load Balancer

LB

Hardware or Software solution

to distribute traffic over web
servers

Web Tier
Fleet of machines handling
HTTP requests.

Web Server

Web Server

Backend Firewall Limits

access to application tier from
web tier

App Load Balancer

LB

Hardware or Software solution to

spread traffic over app servers

App Server Tier

Fleet of machines handling
Application specific workloads
Caching server machines can
be implemented at this layer

App Server

App Server

App server

backups stored on
Tapes usually
managed by 3rd
party at their site

Data Tier
Database Server machines with
master and local running
separately, Network storage for
Static objects

Backups on
Tapes Periodic

MySQL
Master

MySQL
(Slave)

Tapes

MyWebSite.com
DNS
Elastic Load Balancer
ELB to spread traffic to Web
Server Auto-scaling groups

ELB: Web Tier LB

Exterior Firewall no longer

needed because EC2 instances
are controlled with Security
Groups

Auto-scaling Web Tier

Group of EC2 instances
handling HTTP requests.

Auto-scaling group : Web Tier

Web Server

Web Server

Auto-scaling group : Web Tier

Web Server

Web Server

Backend Firewall no longer

needed
SLB

App Server Load Balancer

Edge Caching
High Volume
Static
Content is edge
cached using
CloudFront

SLB

Software LB (e.g. HAProxy) on

EC2 instance to spread traffic
over app server cluster

Auto-scaling App Tier

Group of EC2 instances running
the actual app. Instances
belong to Auto-scaling group.
Caching servers instances can
be implemented at this layer

App Server

App Server
Tomcat

App Server

App Server
Tomcat

Auto-scaling group : App Tier

Auto-scaling group : App Tier

RDS
Master

RDS
Slave

Cloud
Front

DB Tier
MySQL RDS DB Instances
(master, local slave, x-AZ slave
for failover) , Automated
backups to S3 all managed by
AWS

Availability Zone #1

RDS
Slave

Amazon
S3

Availability Zone 2
Availability Zone #n

Backups
Amazon S3 used
for storing Static
Objects and
Backups

3. Implement Elasticity
Elasticity is fundamental property of the Cloud

Dont assume health or fixed location of components

Use designs that are resilient to reboot and re -launch
Bootstrap your instances: Instances on boot will ask a
question Who am I & what is my role?
Enable dynamic configuration

Use Auto-scaling (Free)

Use Elastic Load Balancing on multiple layers
Use configurations in SimpleDB to bootstrap instance

Managed
Development
3. Implement
Elasticity
3 Usecases
Environment

Dev/Test

Apps
Prod

Paid
AMI

SaaS

Web 2.0 Product

Managed
Development
Environment

Automated
Deployment
Environment

Cloud-powered
Software Lifecycle
management

AWS Cloud

AWS Cloud

AWS Cloud

ISV

Startup

Enterprise IT

3. Implement Elasticity

Standardized
StandardizedTechnology
Application Stacks Stacks
Apache
IIS
Web
Server
ASP.NET
Mongrel
Tomcat
App
Server
ASP.NET
Struts
RailsMVC
MVC

Your Code
Log4Net
logger
Log4J
Libraries

Spring.NET
RubyGems
Spring
Packages
memcached
nHibernate
Hibernate
DB
Caching
Ruby
Runtime
.NET
JEE
Framework

Windows
Centos
Linux
OS
Java Stack

.NET Stack

RoR stack

3. Implement Elasticity

3 Approaches
to design
MDE
3 approaches to designing
your AMIs
Easier to Setup

Inventory of fully baked AMIs

(Frozen/Ready made)

Golden AMIs with fetch on boot

(Take N Bake)

AMIs with JeOS and Chef Agent

(Made to Order)
More Control
Easier to maintain

3. Implement Elasticity

3 Approaches
to design
MDE
1. Frozen Pizza
Model
IIS
ASP.NET

IIS
IIS

ASP.NET MVC

Your Code

ASP.NET MVC

Your Code

IIS
IIS

IIS

IIS

ASP.NET MVC

IIS
ASP.NET MVC

IIS
IIS

Log4Net

Log4Net

Your Code

Spring.NET
nHibernate

Spring.NET

Spring.NET

.NET

Your
Code
ASP.NET MVC

Your Code

Log4Net
Your Code
Spring.NET
Log4Net
nHibernate
Spring.NET
.NET
nHibernate

Log4Net
Spring.NET
nHibernate
.NET
Windows

Windows
.NET
Windows

.NET
Windows

Windows

.NET Stack

.NET
Windows

nHibernate

nHibernate

ASP.NET MVC

IIS
Log4Net

.NET AMI

Amazon EC2

3. Implement
Elasticity
Golden
AMIs with fetch on
boot

3 Approaches
toPizza
design
2. Papa Murphy
Model MDE
IIS

Source Control
Your Code

Fetch on boot time

IIS
ASP.NET MVC

Your Code

ASP.NET MVC
Log4Net
nHibernate

Spring.NET
IIS

Amazon S3
Log4Net
Spring.NET

IIS
.NET

IIS
IIS

IIS
IIS
IIS

IIS

Windo .NET
ws
Windo .NET
ws
Windo
ws

IIS
.NET
Windo
ws

IIS

nHibernate
.NET

.NET
Windows

Windows

.NET Stack

.NET AMI

Amazon EC2

3. Implement Elasticity

3 Approaches
toPizza
design
3. Made to Order
Model MDE
Apache

Source Control

Cookbooks
Recipes

Your Code

Mongrel
Rails
Your Code
logger

ASP.NET MVC
Log4Net
.NET
IIS
nHibernate
IIS

Chef Server

Spring.NET

Amazon S3
CHEF
Agent

RubyGems

Windows

memcached
CHEF Agent

Ruby Runtime
Windows

Centos
RoR Stack

AMI (JeOS)

Amazon EC2

3. Implement Elasticity

3 Approaches
to design
MDE
3 approaches to designing
your AMIs
Easier to Setup

Inventory of fully baked AMIs

(Frozen/Ready made)

Golden AMIs with fetch on boot

(Take N Bake)

AMIs with JeOS and Chef Agent

(Made to Order)
More Control
Easier to maintain

4. Build Security in every layer

Design with Security in mind

With cloud, you lose a

little bit of physical
control but not your
ownership
Create distinct Security Groups for each Amazon EC2 cluster
Use group-based rules for controlling access between layers
Restrict external access to specific IP ranges
Encrypt data at-rest in Amazon S3
Encrypt data in-transit (SSL)
Consider encrypted file systems in EC2 for sensitive data
Rotate your AWS Credentials, Pass in as arguments encrypted
Use MultiFactor Authentication

5. Don't fear constraints

Re-think architectural constraints
More RAM? Distribute load across machines
Shared distributed cache

Better IOPS on my database?

Multiple read-only / sharding / DB
clustering

Your hardware failed or messed up config?

simply throw it away and switch to new

hardware with no additional cost

Hardware Config
does not match?
Implement Elasticity

Performance
Caching at different levels (Page, Render, DB)

6. Think Parallel
Serial and Sequential is now history

Experiment different architectures in parallel

Multi-treading and Concurrent requests to cloud services
Run parallel MapReduce Jobs
Decompose a Job into its simplest form

6. Leverage many storage options

One size DOES NOT fit all

Amazon
Amazon
Amazon
Amazon
Amazon
Amazon

S3: large static objects

Cloudfront: content distribution
SimpleDB: simple data indexing/querying
EC2 local disc drive : transient data
EBS: persistent storage for any RDBMS + Snapshots on S3
RDS: RDBMS service - Automated and Managed MySQL

6. Leverage many storage options

Which storage option to use when?
Amazon S3 +
CF

Amazon EC2
Ephemeral
Store

Amazon EBS

Amazon
SimpleDB

Amazon RDS

Storing Large
write-once,
read-many
types of
objects, Static
Content
Distribution
Media files,
audio, video,
images,
Backups,
archives,
versioning

Storing nonpersistent
transient
updates

Off-instance
persistent
storage for any
kind of data,

Querying lightweight attribute

data

Storing and
querying
structured
Relational and
referential
Data

Config Data,
scratch files,
TempDB

Clusters, boot
data, Log or
data of
commercial
RDBMS like
Oracle, DB2

Complex
transactional
systems,
inventory
management
and order
fulfillment
systems

Not
recommended
for

Querying,
Searching

Not
recommended
examples

Database, File
Systems

Storing
Database logs
or backups,
customer data
Sensitive data Content
Distribution

Querying,
Mapping,
tagging, clickstream logs,
metadata,
shared-state
management,
indexing
Relational (joins)
query
OLTP, DW cube
rollups

Simple
lookups

Ideal for

Ideal examples

Cloud Architecture Lessons

Best Practices

1.
2.
3.
4.
5.
6.
7.

Design for failure and nothing fails

Loose coupling sets you free
Design for dynamism
Build Security in every layer
Don't fear constraints
Think Parallel
Leverage many storage options

AWS community and Ecosystem

Find help, guidance, assistance when you need it

AWS Ecosystem
AWS Community

Migrating
a Web Application
to AWS

P h o t o : L a Pe d r e ra - C a s a M i l , B a rc e l o n a - A n t o n i o G a u d i

Migrating your Web Application

Step by Step towards AWS

A typical Web App needs:

Compute Power
Storage capacity
Content Distribution
Database storage
Messaging
Load balancing
Monitoring

Migrating your Web Application - 1/8

Typical Web App Architecture

Database

Application Server /
Business Logic
Web Server /
Presentation Layer
Client Browser

Migrating your Web Application - 2/8

Amazon S3 for Storage

Store persistent files in Amazon S3 for

lower costs, higher reliability
Client Browser

Migrating your Web Application - 3/8

CloudFront CloudFront
for distribution
UseAmazon
Amazon

Amazon CloudFront is a content delivery network

that caches data stored in Amazon S3 across a
network of 14 edge locations around the world
Client Browser

Migrating your Web Application - 4/8

Amazon EC2 for your choice of web servers

Client Browser

Configure Amazon EC2 running your

choice of web server to handle all
incoming web requests.

Migrating your Web Application - 4/8

Scale out App servers on Amazon EC2

Configure multiple Amazon EC2

instances running your choice of
application server to process requests.

Client Browser

Use Availability Zones and Elastic IPs

for greater reliability and resiliency.
Utilize Auto-scaling and Elastic LB
service

Migrating your Web Application - 5/8

for Persistent Storage
S3 forDatabase
Snapshots
UseEBSAmazon
EBSandfor

Client Browser

Configure an Amazon EBS device to host

your existing relational database.
Snapshots can be automatically backed up
to Amazon S3.

Migrating your Web Application - 6/8

Amazon Amazon
SQS for queuing requests
Use
SQS

SQS

Client Browser

Amazon SQS makes it easy to coordinate

between the web server and application
servers.

Migrating your Web Application - 7/8

Amazon
SimpleDB for log
files, metadata
Use
Amazon
SimpleDB

SQS

Client Browser

SimpleDB

Amazon SimpleDB can be used to store

metadata, logfiles, and other information
for your site.

Migrating your Web Application - 8/8

Use Amazon SimpleDB

Monitor your Amazon EC2 instances using CloudWatch

SQS

SimpleDB

Amazon CloudWatch to monitoring your

Amazon EC2 instances
Client Browser

Migrating your Web Application

Step by Step towards AWS

A typical Web App needs:

Compute Power
Storage capacity
Content Distribution
Database storage
Messaging
Load balancing
Monitoring

With AWS:

Amazon EC2
Amazon S3
Amazon CloudFront
Amazon EBS
Amazon SQS
Amazon EC2
Amazon CloudWatch

Conclusions

Most Important Lesson From Our Customers:

Start small with a well-defined proof of concept
Build support and awareness in your organization
Once one application is launched others will follow

Photo: Grand Canyon Hopi Point SunSet

The day is not too far when applications

will cease to be aware of physical
hardware. Much like plugging in a
microwave in order to power it doesnt
require any knowledge of electricity, one
should be able to plug in an application to
the cloud in order to receive the power it
Scalability, Security,
High
availability,
Fault-tolerance,
Testability and
needs
to run,
just like a utility.
As an
architect, you will
manage abstract
Elasticity will be configurable
properties
of the application
storage
and network
architecture and compute,
will be an
automated
andresources
intrinsic part of the
instead of physical servers. Applications
platform on which
are tobuilt.
willthey
continue
function even if the
underlying physical hardware fails or is
removed or replaced. Applications will
adapt themselves to fluctuating demand
patterns by deploying resources
instantaneously and automatically, thereby
achieving highest utilization levels at all
times. Scalability, Security, High availability,
Fault-tolerance, Testability and Elasticity
will be configurable properties of the
application architecture and will be an
automated and intrinsic part of the platform
on which they are built.

The day is not too far.

Thank you!

jvaria@amazon.com Twitter:@jinman
Presentation idea and template from @simon

http://aws.amazon.com