Session-5 & 6

Storage Services on AWS

Elastic Block Store ( EBS )

1
2
Elastic Block Store
Introduction
Amazon Elastic Block Store (EBS) is an easy to use, high-performance, block-storage
service designed for use with Amazon Elastic Compute Cloud (EC2) for both
throughput and transaction intensive workloads at any scale.

A broad range of workloads, such as relational and non-relational databases, enterprise

applications, containerized applications, big data analytics engines, file systems, and
media workflows are widely deployed on Amazon EBS.

3
You can choose from six different volume types to balance optimal price and
performance.

You can achieve single-digit-millisecond latency for high-performance database

workloads such as SAP HANA or gigabyte per second throughput for large, sequential
workloads such as Hadoop.

You can change volume types, tune performance, or increase volume size without
disrupting your critical applications, so you have cost-effective storage when you need
it.

4
Persistent and dynamic disks you can define in Amazon's EC2 compute cloud.
Attach them to EC2 instances.
Create using both AWS console as well as ec2 cli.
Once an EBS volume has been attached to an EC2 instance it functions just like a
local hard disk.
Increase the durability of your EBS volumes by frequently backing up them
(called snapshots).
Pay for what you use.

5
Working of EBS
You create a volume from 1GB to 16TB in size and then you mount it on a device
(such as /dev/sdj) on an instance, format it, and use it.

 Later you can detach it and then reattach it to a different instance.

 One volume can be attached with one instance.

6
Benefits of EBS

Scalable
cost-effective
Redundancy to avoid data loss even when multiple hardware failures occur.
Flexible to detach and attach

7
EBS Volume Types
Amazon EBS provides six volume types:
 SSD (Solid State Drive) Based Volume

 HDD ( Hard Disk Drive) Based Volumes

These volume types differ in performance characteristics and price, allowing you to

tailor your storage performance and cost to the needs of your applications.

8
SSD Based Volumes
1. IO2
io2 is the latest generation of the Provisioned IOPS SSD volumes that is designed to
provide 100X durability of 99.999% as well as a 10X higher IOPS to storage ratio of
500 IOPS for every provisioned GB –at the same price as the previous generation
(io1).
io2 is a high performance EBS storage option designed for business-critical, I/O
intensive database applications, such as SAP HANA, Oracle, Microsoft SQL Server,
and IBM DB2 that have high durability requirements.

9
Io2 is designed to deliver a consistent baseline performance of up to 500 IOPS/GB to a
maximum of 64,000 IOPS. io2 volumes also provide up to 1,000 MB/s of throughput per
volume.

To maximize the benefit of io2, we recommend using EBS-optimized EC2 instances.

10
io2 Block Express
io2 Block Express offers the highest performance block storage in the cloud with 4x
higher throughput, IOPS, and capacity than io2 volumes, along with sub-millisecond
latency.
Block Express is the next generation of Amazon EBS storage server architecture
purpose-built to meet the performance and latency requirements of the most demanding
applications.
io2 Block Express is designed to provide 4,000 MB/s throughput per volume, 256K
IOPS/volume, up to 64 TiB storage capacity, and 1,000 IOPS/GB as well as 99.999%
durability making it ideal for your largest, most I/O intensive, mission-critical
deployments of Oracle, SAP HANA, Microsoft SQL Server, and SAS Analytics.

11
Io 1
io1 is backed by solid-state drives (SSDs) and is a high performance EBS storage option
designed for critical, I/O intensive database and application workloads, as well as
throughput-intensive database and data warehouse workloads, such as HBase, Vertica,
and Cassandra.
These volumes are ideal for both IOPS-intensive and throughput-intensive workloads that
require low latency and have moderate durability requirements or include built-in
application redundancy.
io1 is designed to deliver a consistent baseline performance of up to 50 IOPS/GB to a
maximum of 64,000 IOPS and provide up to 1,000 MB/s of throughput per volume

12
GP3

Amazon gp3 volumes are the latest generation of 

general-purpose SSD-based EBS volumes that enable customers to provision
performance independent of storage capacity, while providing up to 20% lower pricing
per GB than existing gp2 volumes.

Volume Type : EBS General Purpose SSD (gp3)

Use Cases : virtual desktops, medium sized single instance databases such as MSFT
SQL Server and Oracle DB, low-latency interactive apps, dev & test, boot volumes
API Name : gp3
Volume Size : 1 GB – 16 TB
13
Durability : 99.8% - 99.9% durability
Max IOPS/Volume :16,000
Max Throughput*/Volume : 1000 MB/s
Max IOPS/Instance : 260,000
Max Throughput/Instance : 7,500 MB/s

14
GP2
gp2 is the default EBS volume type for Amazon EC2 instances.
These volumes are backed by solid-state drives (SSDs) and are suitable for a broad range of
transactional workloads, including dev/test environments, low-latency interactive applications,
and boot volumes.

Volume Type : EBS General Purpose SSD (gp2)

Short Description : General Purpose SSD volume that balances price performance for a
wide variety of transactional workloads
Use Cases : Boot volumes, low-latency interactive apps, dev & test
API Name : gp2
15
Volume Size : 1 GB – 16 TB
Durability : 99.8% - 99.9% durability
Max IOPS**/Volume : 16,000
Max Throughput***/Volume : 250 MB/s
Max IOPS/Instance : 260,000
Max Throughput/Instance : 7,500 MB/s

16
HDD-based volumes
ST1
Volume Type : Throughput Optimized HDD (st1)
Short Description : Low cost HDD volume designed for frequently accessed,
throughput-intensive workloads
Use Cases : Big data, data warehouses, log processing
API Name : st1
Volume Size : 125 GB – 16 TB
Durability : 99.8% - 99.9% durability
Max IOPS**/Volume : 500

Max Throughput***/Volume: 500 MB/s

17
SC1
sc1 is backed by hard disk drives (HDDs) and provides the lowest cost per GB of all EBS
volume types. It is ideal for less frequently accessed workloads with large, cold datasets.
Volume Type : Cold HDD (sc1)
Short Description : Lowest cost HDD volume designed for less frequently accessed
workloads
Use Cases : Colder data requiring fewer scans per day
API Name : sc1
Volume Size : 125 GB – 16 TB
Durability : 99.8% - 99.9% durability
Max IOPS**/Volume : 250

Max Throughput***/Volume: 250 MB/s

18
Simple Storage Service ( S3 )

19
S3 – Basics

• S3 is a safe place to store your files.

• It is Object based storage.
• The data is spread across multiple devices and facilities
• S3 is Object based i.e. allows you to upload files.
• File size can be from 0 Bytes to 5 TB.
• There is unlimited storage
• Files are stored in Buckets.
• Account→ Bucket -> Object

20
• S3 bucket has a universal namespace, that is, names must be unique globally.
• https://s3-eu-west-1.amazonaws.com/ashishrpandeyfirstbucket.
• When you upload a file to S3 you will receive a HTTP 200 code if the upload was
successful.
• S3 provides developers and IT teams with secure, durable, highly-scalable object storage.
• Amazon S3 is easy to use, with a simple web services interface to store and retrieve any
amount of data from anywhere on the web.

21
S3 – Buckets

• You can then upload any number of objects (pics, docs etc) to the bucket.
• An Amazon S3 bucket name is globally unique and buckets are created in a region you
specify.
• You can create up to 100 buckets in each of your AWS accounts.
• There is no limit to the number of objects that can be stored in a bucket and no
difference in performance whether you use many buckets or just a few.

22
• You cannot create a bucket within another bucket.
• You can delete the bucket when not in use and that bucket name can be reused.
• If you want other users or AWS resources to access your bucket you have to make it
public.
• You can have multiple versions of objects in your bucket by enabling version control.

23
BUCKETS
Containers for objects stored in S3

Serve several purposes:

•Organize the Amazon S3 namespace at the highest level

•Identify the account responsible for charges
•Play a role in access control
•Serve as the unit of aggregation for usage reporting

24
Objects
Fundamental entities stored in Amazon S3

Consist of data & metadata Data portion is opaque to Amazon S3

Metadata is a set of name-value pairs that describe the object

Object is uniquely identified within a bucket by a key (name) and a version ID

25
Object URL

Amazon S3 is storage for the Internet, and every Amazon S3 object can be addressed by a unique URL
formed using the web services endpoint, the bucket name, and the object key.
For example, with the URL:

http://mybucket.s3.amazonaws.com/jack.doc
mybucket is the S3 bucket name, and jack.doc is the key or filename.

Amazon S3 Operations
The Amazon S3 API is intentionally simple, with only a handful of common operations. They
include:
Create/delete a bucket
Write an object
Read an object
Delete an object
List keys in a bucket 26
Key

•Unique identifier for an object within a bucket.

•Every object in a bucket has exactly one key
•Combination of a bucket, key & version ID uniquely identify each object

27
NAMESPACES

28
NAMESPACES Example

29
ACCESS CONTROLS

30
31
32
33
Static Website Hosting
A very common use case for Amazon S3 storage is static website hosting. Many
web sites, particularly micro-sites, don’t need the services of a full web server. A
static website means that all of the pages of the website contain only static content
and do not require server-side processing such as PHP, ASP.NET, or JSP.

Static websites have many advantages: they are very fast, very scalable, and can be
more secure than a typical dynamic website. If you host a static website on Amazon
S3, you can also leverage the security, durability, availability, and scalability of
34
To configure an Amazon S3 bucket for static website hosting:

1. Create a bucket with the same name as the desired website hostname.
2. Upload the static files to the bucket.
3. Make all the files public (world readable).
4. Enable static website hosting for the bucket. This includes specifying an Index
document and an Error document.
5. The website will now be available at the S3 website URL:
<bucket-name>.s3-website-<AWS-region>.amazonaws.com.
6. Create a friendly DNS name for the website using a DNS CNAME.
7. The website will now be available at your website domain name.

35
36
37
38
END

39