Blending Spot and On-Demand Instances and Optimizing Costs - Cobus Bernard

05
Cost and capacity optimization
Cobus Bernard @cobusbernard

Sr Developer Advocate cobusbernard
cobusbernard
AWS
© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS global platform SLA of
99.99% availability
AWS global infrastructure 216 CloudFront PoPs AWS global network

• 23 Regions with 73 Availability Zones • 205 edge locations • Redundant 100 GbE network
• 4 Regions coming soon: Indonesia, Italy, • 11 Regional edge caches
• 100% encrypted between facilities
Japan and Spain • 245 Countries and territories served
• Private network capacity between
all AWS Regions except China
Amazon EC2 instance characteristics
Instance Additional
CPU generation capabilities
Memory
M5d.xlarge
Storage Instance family Instance size
Network performance
Instance type
Broadest and deepest platform choice
Workloads Capabilities Options
270+
(AWS, Intel, AMD)
(up to 4.0 GHz)
+ (up to 24 TiB) + = instance types
(HDD and NVMe)
(GPUs and FPGA)
(up to 100 Gbps)
(Nano to 32xlarge)
Continued rapid pace of innovation
Instance growth
270+
Customer obsessed
90%
of the roadmap originates with customer requests
and is designed to meet specific needs
Optimizing Amazon
EC2 cost and capacity
Optimizing Amazon EC2 cost and capacity
We continue to innovate for our customers
Pricing Capacity Guidance

Capacity management Cost and capacity

made easy on the recommendations
broadest and deepest enable ease of use
compute platform and save time
Amazon EC2 purchase options
the second a significant discount savings of up to 90%

more flexibility
To optimize Amazon EC2, combine purchase options
Spot for fault-tolerant,

flexible, stateless workloads
On-Demand
RIs or a Savings Plan

Deutsche Börse Group
… covers the entire value chain in securities and derivatives trading.

• Pre-IPO and listing • Settlement • Market data
• Trading • Custody • Indices
• Collateral and liquidity • Technology
• Clearing
management
Other Instance Types can make the difference
1
Instance Price $/h
0.8
eu-central-1
0.6
0.4
0.2
0
r5.2xl r5a.2xl r5d.2xl m5.4xl r4.2xl m4.4xl
spot price on-demand price
T7 and Cloud Use-Cases – SmokeTest termination rate
Number of SmokeTest Spot Instances and Terminations
5000
4532
4000
3323
2919
3000
2459
2000
1521
1000
20 25 128 29 71
0
2019-Jan 2019-Feb 2019-Mar 2019-Apr 2019-May
SmokeTest all SmokeTest terminated
Introducing Savings Plans
Easy to use Significant discounts Flexible

Receive discounted rates Select from two types of Savings Make a single commitment that
automatically in exchange Plans to receive discounts of up applies across multiple AWS
for a monetary commitment to 72% on EC2 Instance Savings Plans compute services, even as
and 66% on Compute Savings Plans your requirements change
Flexible purchase option that offers up to 72% discounts

on Amazon EC2 and AWS Fargate usage
Types of Savings Plans
Compute EC2 Instance
Savings Plans Savings Plans
Offer the greatest flexibility, up to 66% off Provide the lowest prices, up to 72% off (same
as Standard RIs) on the selected instance family
(same prices as Convertible RIs)
(e.g., C5 or M5), in a specific AWS Region
 Instance family: e.g., Move from C5 to M5

 Region: e.g., Change from EU (Ireland) to EU  Size: e.g., Move from m5.xl to m5.4xl
(London)  OS: e.g., Change from m5.xl Windows
Flexible  OS: e.g., Windows to Linux Flexible to m5.xl Linux
across  Tenancy: e.g., Switch Dedicated tenancy to across
 Tenancy: e.g., Modify m5.xl Dedicated
Default tenancy to m5.xl Default tenancy
 Compute options: e.g., Move from EC2 to
Fargate
Comparing RIs and Savings Plans
Savings Plans offer all the benefits of RIs as well as improved flexibility and reduced management
Compute EC2 Instance
Convertible RIs* Standard RIs
Savings Plans Savings Plans
Savings over On-Demand Up to 66% Up to 72% Up to 66% Up to 72%
Low price in exchange for
monetary commitment    
Pricing automatically applies
to any instance families    
to any instance size   ** **
to any tenancy or OS    
Automatically apply to
Fargate usage    
across any AWS Region
   
1- and 3-year term
length options
   
* Convertible RIs can be changed across instance families, sizes, OS, and tenancy – they require customers to manually perform exchanges
** Regional Convertible RIs and Regional Standard RIs provide instance size flexibility
Getting started with Savings Plans
AWS Cost Explorer guides you through the purchasing process
Review your Savings Customize Review hourly Eligible Amazon EC2

Plans recommendations recommendations commitment (e.g., $10/hr) and AWS Fargate
in AWS Cost Explorer based on your needs and add to cart usage is charged at a
(type of Savings Plan, discounted Savings
payment option, Plans rate up to your
term length) commitment level
Just like RIs, you can purchase Savings Plans via the RI operations team
Spot, On-Demand capacity reservations, and
Savings Plan together
Cost-effective,
Savings Plan scalable compute
On-Demand capacity
reservations
Spot Instances
Save up to 90% using EC2 Spot Instances
Instances Pricing Usage Capacity

Same infrastructure as Smooth, infrequent Choose different instance Interruptions only
On-Demand and RIs changes, more predictable types, sizes, and AZs in happen if OD
a single fleet or EC2 Auto needs capacity
Scaling group
Pricing is based on long-term supply and demand trends; no bidding!

Why Spot Instances?
Low, predictable prices
Up to 90% discount over On-Demand prices
Faster results
Increase throughput up to 10x while staying in budget
Easy to use
Launch through AWS services (e.g., Amazon ECS, Amazon EKS,
AWS Batch, Amazon SageMaker, Amazon EMR) or integrated
third parties
Handling Spot interruptions
Less than 5% of Spot Instances were interrupted in the last 3 months
Minimal interruptions
Check for 2-minute interruption notification via instance
metadata or Amazon CloudWatch events, and automate by
 Checkpointing
 Draining from ELB
 Using stop-start and hibernate to restart faster
Interruption handlers for Amazon ECS and Amazon EKS
 Connection between termination requests from AWS infrastructure to nodes

Amazon Elastic Amazon Elastic  Tasks running on Spot Instances will automatically be triggered for shutdown
Kubernetes Service Container Service
(Amazon EKS) (Amazon ECS) before the instance terminates, and replacement tasks will be scheduled
elsewhere on the cluster
Flexibility is key to successful Spot usage
Instance flexible Time flexible Region flexible

Customers across different industries and verticals use Spot
Research Consumer Apps AdTech & MarTech
Sports, Media & Entertainment B2B Enterprise Tech Financial Services

Achieve optimal Cost and capacity

price/performance recommendations
with different enable ease of use
purchase models and save time
Amazon EC2 Cost Optimisation non-prod
% Running Time
100.0
100
80
71.4
Up to 70%
savings for non-
60
40
35.7
production
29.8 workloads
20
0
24 x 7 24 x 5 12 x 5 10 x 5
AWS Instance Scheduler
• AWS-provided solution
• Custom start & stop schedules
• Works with EC2 & RDS instances
• Deploy using CloudFormation
• Selectively tag instances to schedule
• Multiple schedules per instance
• 5-minute granularity
https://aws.amazon.com/answers/infrastructure-
management/instance-scheduler/
Using Amazon EC2 Auto Scaling
Automatically scale instances across instance families
and purchase options in a single ASG to optimize cost
Amazon EC2
Auto Scaling
Capacity-optimized
Prioritize deploying Spot Instances into greater Spot pool capacity
order to lower the chance of interruptions
Lowest cost
Prioritize cost by selecting a mix of On-Demand and Spot Instances
to launch based on the lowest available price
Prioritized list AZ1 and AZ2

Use a prioritized list for On-Demand instance types to scale capacity
during an urgent, unpredictable event to optimize performance
Amazon EC2 Spot Instance pools explained
C4 1a 1b 1c
On
demand Each instance family
8XL $0.50 $0.27 $0.29 $1.76 Each instance size
Each availability zone

4XL $0.21 $0.30 $0.16 $0.88
In every Region
2XL $0.08 $0.07 $0.08 $0.44
Is a separate Spot pool
XL $0.04 $0.05 $0.04 $0.22
R5 I3 C5 R5d
R4 M5
L $0.01 $0.01 $0.04 $0.11
M4 i3en R5a
ASG capacity-optimized allocation strategy
Desired capacity: 12 OnDemandBaseCapacity: 0 OnDemandPercentageAboveCapacity: 0
Overrides: [“r5.large”, “m4.large”, ”m5.large”] SpotAllocationStrategy: capacity-optimized
us-east-1a us-east-1b us-east-1c

$$ r5.large $$$ r5.large $ r5.large
$ m4.large $$ m4.large $$$ m4.large
$$$ m5.large $ m5.large $$ m5.large
R5 R5
R5 R5
ASG lowest-price allocation strategy
Desired capacity: 12 OnDemandBaseCapacity: 0 OnDemandPercentageAboveCapacity: 0
Overrides: [“r5.large”, “m4.large”, ”m5.large”] SpotAllocationStrategy: lowest-price
us-east-1a us-east-1b us-east-1c

$$ r5.large $$$ r5.large $ r5.large
$ m4.large $$ m4.large $$$ m4.large
$$$ m5.large $ m5.large $$ m5.large
R5 R5
R5 R5
Instance type overrides and allocation strategies
ASG adjusts to new configuration Instance type overrides: m4.large, m5.large
as it scales up and down
As ASG scales up m4.large m5.large
Launch capacity according to the new

configuration Instance type overrides: m5.large, c5.large
As ASG scales down

Prioritize terminating instances not m4.large m5.large c5.large
matching the new configuration Instance type overrides: m5.large, c5.large

New termination policy:
AllocationStrategy
m4.large m5.large c5.large

To optimize Amazon EC2, combine purchase options
Spot for fault-tolerant,

flexible, stateless workloads
On-Demand
RIs or a Savings Plan

Before: Multiple ASGs to use Spot, On-Demand, and RIs together
m4.large Spot ASG Min: 1 Max: 10
Before, with
three ASGs
—one for each m5.large Spot ASG Min: 1 Max: 10
instance type/
purchase option
c4.xlarge O-D ASG Min: 1 Max: 10
Availability Availability Availability

Zone 1 Zone 2 Zone 3
Then: Spot, On-Demand, and RIs in a single ASG
m4.large Spot Instances
The new way

combines purchase
options, instance m5.large Spot Instances
types, and AZs in
a single ASG
c4.xlarge On-Demand instances
Availability Availability Availability Single ASG

Now: Spot, On-Demand, and RIs in a single ASG with weights
m4.xlarge Spot
Weight of 1
Different m4.2xlarge Spot

instance types Weight of 2
contribute
differently to
total capacity
m4.4xlarge On-Demand
Weight of 4
Availability Availability Availability

Amazon EC2 Spot Instance pools explained
C4 1a 1b 1c
On
demand
Weight
Weighted
price 1a 1b 1c
8XL $0.50 $0.27 $0.29 $1.76 16 $0.11 $0.032 $0.017 $0.018
4XL $0.21 $0.30 $0.16 $0.88 8 $0.11 $0.032 $0.038 $0.02
2XL $0.08 $0.07 $0.08 $0.44 4 $0.11 $0.021 $0.016 $0.02
XL $0.04 $0.05 $0.04 $0.22 2 $0.11 $0.02 $0.025 $0.04
L $0.01 $0.01 $0.04 $0.11 1 $0.11 $0.01 $0.01 $0.04

API parameters
"MixedInstancesPolicy": {
"LaunchTemplate": {
"LaunchTemplateSpecification": { Max
AZ1 and AZ2
"LaunchTemplateName": "MyLaunchTemplate"
},
"Overrides": [
{
"InstanceType": "m4.xLarge", Desired
"WeightedCapacity": "1"
}, 50% On-Demand
{ 50% Spot
"InstanceType": "m4.2xLarge",
},
{
"WeightedCapacity": "4" On-Demand base
}
] Minimum On-Demand (10)
},
"InstancesDistribution": {
"OnDemandAllocationStrategy": "prioritized", Min
"OnDemandBaseCapacity": 10,
"OnDemandPercentageAboveBaseCapacity": 50,
"SpotAllocationStrategy": “capacity-optimized",
"SpotInstancePools": 2
}
}
API parameters
"LaunchTemplate": {
AZ1 and AZ2
},
"Overrides": [
{
}, 50% On-Demand
{ 50% Spot
},
{
}
},
}
}
API parameters
"LaunchTemplate": {
AZ1 and AZ2
},
"Overrides": [
{
}, 50% On-Demand
{ 50% Spot
},
{
}
},
}
}
API parameters
"LaunchTemplate": {
AZ1 and AZ2
},
"Overrides": [
{
}, 50% On-Demand
{ 50% Spot
},
{
}
},
}
}
API parameters
"LaunchTemplate": {
AZ1 and AZ2
},
"Overrides": [
{
}, 50% On-Demand
{ 50% Spot
},
{
}
},
}
}
API parameters
"LaunchTemplate": {
AZ1 and AZ2
},
"Overrides": [
{
}, 50% On-Demand
{ 50% Spot
},
{
}
},
}
}
AWS and third-party integrations with Spot Instances
and EC2 Auto Scaling
Amazon EC2 Amazon EC2 AWS Thinkbox Amazon EMR AWS AWS
Auto Scaling fleet CloudFormation Batch
Amazon Elastic Amazon Elastic Amazon AWS AWS Elastic

Container Service Kubernetes Service SageMaker Fargate Beanstalk
Achieve optimal Capacity management

price/performance made easy on the
with different broadest and deepest
purchase models compute platform
AWS Compute Optimizer
Recommends optimal instances for Amazon EC2 and Amazon EC2 Auto
Scaling groups from 140+ instances from M, C, R, T, and X families
Lower costs Applies insights Saves time

performance
Simplifying compute optimization
AWS resources
metadata
Amazon
CloudWatch
metrics
EC2 Instance
AWS Compute EC2 Auto

Scaling groups
Optimizer
Identify optimal
AWS compute resources Mettle scans your AWS Identifies workload Matches the resource Helps you visualize
for your workloads infrastructure and uses characteristics and requirements of your what-if scenarios
machine learning to profile based on the workloads to optimal based on the
automatically identify data gathered AWS resources with recommended
optimal AWS resources recommendations resources
for your workloads
Easy to choose with AWS Compute Optimizer
New services that recommend optimal AWS compute resources to reduce costs up to 25%
Recommends optimal EC2 instances

Optimizes performance and reduces costs by
making recommendations to help you
right-size compute to your workloads
Analyzes Amazon CloudWatch metrics and
considers Auto Scaling group configuration for
intuitive and actionable recommendations
Up to three recommendations per workload
Available at no additional charge
Analytics and big data
Databases
DevOps—CI/CD
Workloads
on AWS Enterprise applications
IoT
Machine learning
Storage
Websites and web applications
Machine learning
Workloads
on AWS
DevOps—CI/CD

Machine learning (ML)
Get ML solutions to market faster with access to built-in algorithms,
ML frameworks, and custom models
Save up to 90% in training costs

with managed spot training
Automatically manages Spot

capacity on your behalf
Amazon SageMaker
managed spot training All instance types, training models,
and configurations
Machine learning
Workloads
on AWS
DevOps—CI/CD

DevOps—CI/CD
Or, run Jenkins build jobs inside your Kubernetes

clusters and cost-optimize with Spot node groups
https://ec2spotworkshops.com/amazon-ec2-spot-cicd-workshop.html
CI/CD reference architecture
VPC
Spot fleet
Availability Zone A
Jenkins agent
(Spot)
Jenkins agent
Jenkins master (Spot)
Users Application (OD or RI)
Load Balancer
Jenkins agent
Availability Zone B
(Spot)
Jenkins agent
Jenkins master (Spot)
(OD or RI)
https://github.com/awslabs/ec2-spot-jenkins-plugin/
Machine learning
Workloads
on AWS
DevOps—CI/CD

Amazon Elastic Amazon Elastic AWS

Container Service Kubernetes Service Fargate
Amazon EC2
Auto Scaling
Run web services ranging from ad servers to real-time bidding servers
Deploy web applications or services on containers and scale clusters at a fraction of the cost
Use Auto Scaling with Amazon ECS or Amazon EKS to run any containerized workload,
including a web application
Scale in real time, pay per second, save up to 90%

AWS Fargate with EC2 Spot
Run containers without managing servers or clusters
Up to 70% off over regular

Fargate tasks
AWS Fargate
Only pay for the resources you use
by automatically scaling based on
tasks, vCPUs, and memory
VM-level isolation by design

To tie it all together…
Key takeaways
1 2 3 4
Experiment and test How to automate Optimize your Get technical
at a lower cost to cost and capacity workloads by using guidance in an AWS
innovate faster optimization best practices Immersion Day
Spot Instances Auto Scaling and AWS Compute CI/CD, analytics,

Savings Plans Optimizer big data, machine
learning, and
web services
Schedule an AWS
Immersion Day
AWS experts are here to help,

and it’s free!
Learn compute with AWS Training and Certification
Resources created by the experts at AWS to help you build cloud compute skills
20+ free digital courses cover topics related to cloud compute,

including introduction to the following services:
• Amazon Elastic Compute Cloud • AWS Systems Manager
(Amazon EC2)
• AWS Inferentia and Amazon EC2
• Amazon EC2 Auto Scaling Inf1 instances
Compute is also covered in the classroom offering, Architecting

on AWS, which features AWS expert instructors and hands-on
activities
Visit the learning library at https://aws.training

Thank you!
Cobus Bernard
Sr Developer Advocate
Amazon Web Services
@cobusbernard
cobusbernard
cobusbernard
© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Blending Spot and On-Demand Instances and Optimizing Costs - Cobus Bernard

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Blending Spot and On-Demand Instances and Optimizing Costs - Cobus Bernard

Uploaded by

Copyright:

Available Formats

05

Cost and capacity optimization

Cobus Bernard @cobusbernard

AWS global infrastructure 216 CloudFront PoPs AWS global network

(up to 4.0 GHz)

+ (up to 24 TiB) + = instance types

(HDD and NVMe)

(GPUs and FPGA)

(up to 100 Gbps)

Pricing Capacity Guidance

Pricing Capacity Guidance

Capacity management Cost and capacity

the second a significant discount savings of up to 90%

Spot for fault-tolerant,

RIs or a Savings Plan

… covers the entire value chain in securities and derivatives trading.

Easy to use Significant discounts Flexible

Flexible purchase option that offers up to 72% discounts

 Instance family: e.g., Move from C5 to M5

AWS Cost Explorer guides you through the purchasing process

Review your Savings Customize Review hourly Eligible Amazon EC2

Instances Pricing Usage Capacity

Pricing is based on long-term supply and demand trends; no bidding!

Interruption handlers for Amazon ECS and Amazon EKS

 Connection between termination requests from AWS infrastructure to nodes

Instance flexible Time flexible Region flexible

Sports, Media & Entertainment B2B Enterprise Tech Financial Services

Pricing Capacity Guidance

Achieve optimal Cost and capacity

Prioritized list AZ1 and AZ2

8XL $0.50 $0.27 $0.29 $1.76 Each instance size

Each availability zone

Overrides: [“r5.large”, “m4.large”, ”m5.large”] SpotAllocationStrategy: capacity-optimized

us-east-1a us-east-1b us-east-1c

Overrides: [“r5.large”, “m4.large”, ”m5.large”] SpotAllocationStrategy: lowest-price

us-east-1a us-east-1b us-east-1c

As ASG scales up m4.large m5.large

Launch capacity according to the new

As ASG scales down

matching the new configuration Instance type overrides: m5.large, c5.large

m4.large m5.large c5.large

Spot for fault-tolerant,

RIs or a Savings Plan

m4.large Spot ASG Min: 1 Max: 10

c4.xlarge O-D ASG Min: 1 Max: 10

Availability Availability Availability

m4.large Spot Instances

The new way

c4.xlarge On-Demand instances

Availability Availability Availability Single ASG

Different m4.2xlarge Spot

Availability Availability Availability

8XL $0.50 $0.27 $0.29 $1.76 16 $0.11 $0.032 $0.017 $0.018

4XL $0.21 $0.30 $0.16 $0.88 8 $0.11 $0.032 $0.038 $0.02

2XL $0.08 $0.07 $0.08 $0.44 4 $0.11 $0.021 $0.016 $0.02

XL $0.04 $0.05 $0.04 $0.22 2 $0.11 $0.02 $0.025 $0.04

L $0.01 $0.01 $0.04 $0.11 1 $0.11 $0.01 $0.01 $0.04

Amazon Elastic Amazon Elastic Amazon AWS AWS Elastic

Pricing Capacity Guidance

Achieve optimal Capacity management

Lower costs Applies insights Saves time

AWS Compute EC2 Auto

Recommends optimal EC2 instances