The Truth about Cloud Price-Performance:

How Misperceptions about Service Costs

Can Derail Your Cloud Strategy

An Executive Brief Sponsored by IBM

Lynda Stadtmueller
Vice President Cloud Services
August 2015
The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

INTRODUCTION

For IT leaders of the past decade or so, the cloud delivery model has been a blessing. The defining
characteristics of the modelscalability, elasticity, speed of deploymentalign with IT delivery needs in a fast-
moving business world. Yet, as IT leaders increase their reliance on the cloud, many are discovering flaws in their
initial cloud purchase criteria. Some businesses that chose a cloud service provider based on low published rates
are experiencing higher costs than expected. Others are fielding complaints about unacceptable application
performance. Still others are realizing they have no idea whether they are paying too much or too little for what
they are getting.

The issue has become more urgent as enterprise IT leaders transform their organizations into a service broker
model. As service brokers to the business, IT experts assess business needs, and identify the optimal
infrastructure environment (e.g., on-premises or hosted; single-tenant or multi-tenant; physical or virtual) for
each workload. Selecting the optimal environment and service provider requires an understanding of the
factors that influence price-performancefactors that many IT leaders have yet to incorporate into their
decision processes.

Certainly, cloud strategies are and will remain important to an optimized IT environment. However, like any
other business-critical decision, selecting a cloud service requires due-diligence and research beyond published
per-unit rates. It requires an in-depth understanding of workload performance characteristics and needs, as well
as the ability to match those needs to the actual offerings of multiple cloud vendors. The task isnt easy. Cloud
service providers have no shared standard units for cloud capacity or common pricing structures, nor are there
common specifications for the underlying hardware that runs the cloud applications. As a result, an assumption of
total workload cost based on a providers basic per-unit rate can easily be off by orders of magnitude.

In this paper, Stratecast provides a context for understanding the factors that influence cloud price-performance.
We dispel some insidious myths that can prevent IT decision-makers from selecting the optimal cloud service for
their workloads. We step through the multi-dimensional considerations (related to the workload and to your
business) that need to be factored into a cloud service decision. We define price performance and how it
impacts your cloud implementation. To provide a vivid view of the magnitude of the issue, we step through real-
life scenarios provided by IBM, comparing actual costs associated with common cloud workloads on IBM
SoftLayer and competitors cloud services. Finally, we offer tips for enterprises looking to ensure they are
selecting optimal cloud services.

MISPERCEPTIONS THAT HINDER THE UNDERSTANDING OF CLOUD

PRICE-PERFORMANCE
Cloud economics appear to be poorly understood by most IT decision-makers. Spurred by the low barrier to
entry, and lack of commitment associated with public cloud services, many IT, and even line-of-business, leaders
have approached cloud choices with less rigor than other technology decisions. Furthermore, they can be easily
blinded by misperceptions about the services to which they are entrusting their workloadsmisperceptions that,
at best, will escalate the planned budget; and, at worst, can negatively impact business operations. Commonly
accepted, but harmful, myths include the following:

2015 Stratecast. All Rights Reserved. 2

 Stratecast | Frost & Sullivan

Myth #1. If its not virtualized, its not a cloud.

Some industry leaders narrowly define cloud as requiring a virtualized infrastructure.1 Semantics aside, the
mistaken belief that cloud benefits and economies are only possible in a virtualized infrastructure can keep you
from exploring bare metal or physical server cloud options that may enable certain workloads to perform better,
at a lower cost, than virtualized options. Bare metal servers provide the highest levels of raw throughput for
high-performance workloads, as well as flexibility to configure storage and network resources. Note that bare
metal servers do not preclude you from leveraging virtualization as needed; if it makes sense for your workloads,
you can deploy your own hypervisor on the cloud providers physical server, just as you would in your own data
center. As a result, you should be sure to include a bare metal option in the price-performance assessment for
many workloads.

Myth #2. Cloud is always cheap.

For IT organizations long accustomed to laying out escalating amounts of capital to add infrastructure to the
premises data center, the cloudany cloudmay seem like a sure-fire way to reduce costs. And in fact, if you
match your workload to the right cloud service, you will probably enjoy price-performance beyond what you
could achieve on-premises. But the right cloud service isnt always public (multi-tenant) cloud, or even
virtualized cloud; depending on the workload, the optimal solution may involve bare metal or dedicated servers
(as noted above), or even a private or hybrid environment. If you dont take the effort to ensure alignment with
the optimal cloud deployment model and service, you could easily end up paying many times more than you need
to.

Myth #3. The lowest per-unit rate for compute and storage provides the
best overall value.
In the era of cloud price wars, in which providers routinely and publicly lower their per-unit rates, it is easy to
focus on those rates as a proxy for overall costs. But the economics do not work that way. Each cloud service
provider designs and prices its cloud services uniquely, with some infrastructure and service components pre-
packaged into bundles of capacity, and others available la carte. To determine what you will pay to run your
workload, you need to understand how each provider prices each relevant component. Consider storage, for
example. Some cloud service providers include persistent storage with all their virtualized and physical cloud
services; others (e.g., low-end AWS instances) provide only temporary storage as part of the basic cloud
compute unit, with blocks of persistent storage available separately at an additional, non-trivial charge. For
network-dependent workloads, data transfer fees may significantly increase overall costs: some providers charge
a per-GB fee for intra-cloud data transfer, while others include unlimited transfer at no charge. In each of these
cases, the actual charges to run a workload may be many times higher than the basic per-unit rates might indicate
at first glance. We will look at specific use cases later in this paper.

Myth #4. All CPUs are alike.

Cloud service providers enable customers to select the right-sized compute unit, based in part on processor
capacity, generally referred to in cores or virtual CPUs. But there are two important facts to remember in
assessing how a cloud providers services match up to your needs:

A virtual CPU is not the same as a physical CPU. In a virtualized environment, the hypervisor consumes

1
Interestingly, the generally accepted cloud definition issued by the National Institute for Standards in Technology (NIST) does not
include virtualization as a defining characteristic. See http://www.nist.gov/itl/csd/cloud-102511.cfm.

3 2015 Stratecast. All Rights Reserved.

The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

some of the processor capacity, so your workload does not have access to the full core listed in the
description. Furthermore, many cloud service providers use Intels Hyper-Threading Technology to
maximize processor throughput. With this technology, a single physical core can support two
hyperthreads, allowing the provider to sell two vCPUs for each physical core. In comparing physical
(bare metal) cloud servers to virtualized servers of the same processor generation (for example, Intel
Haswell), it can be assumed that two vCPUs are roughly equivalent to one physical core.

All virtual cores are not alike. Within a virtualized cloud environment (just as in your on-premises data
center), workload performance will be impacted by the specific processor hardware on which it is
running. A cloud workload running on the latest generation Intel Haswell processor delivers much
higher performance than the same workload running on older-generation hardware, such as
SandyBridge. Its worth investigating the actual or likely processor as you research your cloud service
options. Most cloud service providers readily communicate the type or range of processors that support
their different-sized cloud units; some even charge higher rates for newer generation hardware.

Myth #5. Cloud is simple and fully self-serve.

While cloud is positioned as easy, in fact the vast majority of businesses have neither the staff expertise nor time
and available resources to successfully deploy and maintain a cloud environment. As evidenced by a recent Frost
& Sullivan survey, a large number of IT decision-makers face unexpected challenges and expenses after beginning
their cloud implementations. By underestimating what it will take to achieve the benefits promised by cloud, IT
decision-makers risk not only their business but their own reputations: 31% said they struggled to attain
additional budget to fully implement their cloud strategies; and 21% said they were challenged to gain senior
leadership support to continue executing on those strategies.

Cloud service providers technical support services may be able to help offset internal gaps in expertise, but not
consistently, and often at a cost. Some providers include access to technical support personnel (e.g., for basic
trouble-ticketing) as part of their standard services; others charge a flat fee for such access; still others charge a
fee that scales along with total spend (the higher the usage, the higher the support fee). If you choose to rely on
the providers expertise (a smart choice for many enterprises), then your overall cost assessment for the project
must include any technical fees or services from the vendor; and perhaps even consider the offset for internal
skills and personnel you can forego.

Myth #6. Performance is fairly similar across cloud providers services, so

price should be the definitive selection criterion.
In commodity markets, where products are assumed to be identical across suppliers, it is appropriate to use
price as the primary purchase criterion. But cloud is not a commodity. In fact, if you deploy the exact same
workload into different providers equivalent cloud services, performance can vary significantlythanks to
differences in the way the providers configure their clouds, the infrastructure components they choose, the
procedures and protocols they follow, and other factors. By considering price alone, rather than price-for-
equivalent-performance, IT decision-makers risk selecting cloud services that are not optimal to run their
workloads. Furthermore, the problem can spiral, since IT leaders tend to deal with lower-than-acceptable
workload performance by spinning up additional cloud serversand thus, unnecessarily driving up costs.

For the remainder of this paper, we will look at price performance, measured as cost per unit of work, as
essential for comparing and selecting among cloud services.

2015 Stratecast. All Rights Reserved. 4

 Stratecast | Frost & Sullivan

WHAT GOES INTO ASSESSING CLOUD PRICE-PERFORMANCE

Calculating the true costs of cloud, and (more importantly) comparing the price-performance of various
providers services can be a valuable and eye-opening effort. In this section, we will look at the considerations
that go into a price-performance assessment for a cloud workload.

To get started, begin by looking not at the cloud service providers, but within your own business, with an
assessment of the workload requirements and your business requirements.

Workload Consumption Requirements Calculate what it will take to run the workload; specifically, how
much of each infrastructure component (CPU, RAM, disk input/output, and network) you will need. Note that
this step is where many cloud buyers start, and end, their cloud workload assessmentand yet, it is only
the beginning.

Workload Operating Characteristics For each cloud-destined workload, consider the following:

Of the various infrastructure components (processor, memory disk input/output, network), which will
the workload emphasize? Does the workload require persistent storage, or is the storage only needed
while the processor is running? Some cloud service providers offer specialized cloud units with pre-
packaged bundles of capacity that emphasize one component or another; others enable you to choose
more precisely what you need.

What type of processing demand does the workload require? Is it steady? Spiky, according to a
predictable pattern, or unpredictable? Steady-state workloads return different economics than volatile
workloads.

For what timeframe do you anticipate running the workload in this cloud? Most cloud services do not
require term commitments; however, some providers do offer discounts for long-term commitments,
paid upfront or monthly.

Can or should the workload be virtualized? Many cloud workloads can be virtualized. However, some
workloads cannot or should not be subjected to a hypervisor; for example, performance-sensitive
workloads such as high-throughput database systems, and those with specific security needs that cannot
be achieved in a virtual machine. For such workloads, consider a provider that offers a bare metal (non-
virtualized) option.

Workload Risk How important is the workload to your business? Does the workload need to perform
consistently? What are the business risks if performance goals are not met (e.g., customer dissatisfaction;
employee productivity decreases; regulatory non-compliance)? Do security or compliance issues make a single-
tenant option (in which the processor is not shared with other companies) preferable to a multi-tenant cloud
service?

Business Considerations As noted, some providers offer discounted per-unit rates for long-term
commitments, and even up-front payments. As you calculate the overall price-performance, consider how the
cloud service aligns with your overall business strategy and goals. For example: if a goal in adopting cloud is to
achieve flexibility, does it make sense to make a long-term commitment to a provider? If you make a
commitment, and your business needs change, will the provider allow you to change services or unit
commitments? As an example, AWS Reserved Instances do not offer such flexibility. Furthermore, does paying
upfront for one to three years make sense from an accounting perspective, as you consider net present value of
money?

5 2015 Stratecast. All Rights Reserved.

The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

ASSESSING PRICE-PERFORMANCE FOR COMMON WORKLOADS

Armed with the workload and business requirements, IT decision-makers can establish parameters for choosing
an appropriate cloud service; and even compare similar service units across providers, to start to derive a best-
price scenario. However, price is only part of the equation. For IT leaders that are driving IT efficiency and
accountability in workload delivery, additional price-performance analysis is well justified.

Price-performance has to do with the workload yieldthat is, what it costs to deliver specific units of work.
The metric will, of course, vary by workload; for example, performance may be measured in transactions or
queries or calculations (depending on what function is being performed). The actual performance of the cloud
workload against the metric relies heavily on the physical capability of the processor in use (as discussed earlier);
as well as to factors such as storage type and proximity, unfettered access to processing capacity and network
bandwidth; even physical distance between cloud components and data centers.

Price-performance assessments enable IT leaders to align their workloads with the optimal cloud service type, as
well as select the vendor with the best return.

PRICE PERFORMANCE USE CASES: IBM SOFTLAYER VERSUS AMAZON

WEB SERVICES (AWS)
IBM recently conducted a comprehensive price-performance comparison of common workloads hosted in the
IBM SoftLayer cloud as well as competitors clouds. Stratecast reviewed and validated the results. In this section,
we summarize IBMs assessments, and provide analysis of their results.2

Note: The scenarios on the following pages reflect specific business workloads which were run on IBM SoftLayer
and AWS Elastic Compute Cloud (EC2). We analyze how the particular workload characteristics, provider
pricing structures, and service delivery influence both the base price and the price-performance. However, the
results achieved in these scenarios can not necessarily be extended to other scenarios. Changes in workload
parameters or provider configurations can yield very different results. Nonetheless, our intention is to
demonstrate just how important a thorough price-performance analysis is in achieving optimal cloud resultsand
to show how counter-intuitive the results may be.

Our final caveat will not come as a surprise to infrastructure or operations engineers. IT configurations are
complex, whether in the cloud or in the enterprise data center, and performance measures are imprecise. Two
identical workloads deployed in the same cloud service, with the same processor type, can yield slightly different
results, thanks to performance variations of individual infrastructure components (processor, memory, storage,
network) over which the user has no control. For this reason, many seasoned cloud users over-sample their
cloud deployments, deploying more servers than they need for their performance-sensitive applications. After a
test period, such users decommission the lower-performing servers.

In the following tests, IBM ran duplicate workloads over a period of time to derive average performance metrics.
In deploying your own workloads, Stratecast recommends not only doing your homework upfront to ensure the
best performance from the factors you can control; but also testing actual performance once the workloads are
up and running.

2
For a detailed review of specifications and assumptions used for each scenario, contact IBM.

2015 Stratecast. All Rights Reserved. 6

 Stratecast | Frost & Sullivan

Use Case #1: Web application (virtualized)

In this scenario, the workload is characterized by large numbers of simultaneous, small online transactions (e.g., logins,
dynamic page construction, database queries). Performance is measured by average requests-per-second (RPS). Price
performance is measured in dollars per RPS ($/RPS).

Workload description: Performance-intensive workload; multi-tier, online transaction processing.

Requirements: AppServer plus Database Server, each running approximately 8 vCPU, 16 GB RAM; 200 GB
Block Storage; RedHat Linux.

Solutions: IBM SoftLayer and AWS offered compute units that matched up well with the requirements and with
each other. For both providers, the VMs ran on hosts based on similar Haswell processors. Single-tenant
virtualized services were selected for both. Pricing was the lowest option available for each provider; for IBM
SoftLayer, pricing was based on a monthly commitment; for AWS, pricing was based on a three-year
commitment, paid in advance.

Calculating Price and Price-Performance: 3

Price(over3years)
Costcomponent IBMSoftLayer AWSEC2
Infrastructure(computeandstorage) $34,746 $89,796

Technicalsupport $0 $16,049
Datatransfer(Internet) $62,366 $74,013
Software $137,760 $137,760
Total(3yearcost) $234,872 $317,618

Priceperformance
Performancecomponent IBMSoftLayer AWSEC2
Maximumrequestspersecond(RPS) 19,883RPS 16,079RPS
Averagerequestspersecond 3,314RPS 2,680RPS
Costperunitofwork(RPS) $71/averageRPS $119/averageRPS

ThisisanIBMinternalstudyofabasicwebapplicationworkload.Resultsobtainedundercontrolledconditionstestingresourcesofapub
licly available cloud infrastructure. Customer applications, differences in the stack deployed, differences in data center locations and op
tions,andothersystemsvariationsortestingconditionsmayproducedifferentresults.PricingbasedonpublishedUSlistpricesavailablefor
IBMandcompetitordatacenterlocationspecifiedasof7/6/2015andcalculatedbyprojectingcostsmeasuredoverafixedtestingperiodto
a3yearTCA,includingcompute,network,storage,software,datatransfer,andsupportcharges.Pricingexcludesclientlaborcosts.Avail
ableoptionsandpricingvarybydatacenterlocation.Optionfornonvirtualizedserversnotavailableincompetitorenvironment.

Analyzing the Results: In this scenario, the topline costs for the IBM SoftLayer solution were 26% lower than
AWS EC2. At the same time, the IBM solution processed, on average, 24% more requests per second, for an
overall higher price-performance. We can interpret the results as follows:

Lower price Storage configuration and data transfer costs are largely responsible for the lower
overall costs for IBM SoftLayer. For persistent storage required by the application, AWS requires a
3
Results for this and other scenarios in this analysis have been condensed and interpreted by Stratecast, based on findings of a study
conducted by IBM, and verified by Stratecast.

7 2015 Stratecast. All Rights Reserved.

The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

separate purchase of block storage. In contrast, block storage is included in the basic price for the
SoftLayer virtualized configuration. Also contributing to the cost disparity are data transfer costs
(outbound from the cloud center to the Internet). These costs are lower with IBM SoftLayer due to
IBMs more generous free tier, which covers 500 GB per month per bare metal server, compared with
AWSs allowance of 1GB per month per account (not per instance). Not specifically included in this
scenario, but also to be considered, are costs associated with supporting geographically dispersed end-
users or data sources. IBM SoftLayers cloud centers are directly connected to one another via the
Softlayer global private network. Via that network, IBM cloud customers are able to transfer unlimited
volumes of data between SoftLayer cloud centers, at no charge. In contrast, AWS meters traffic between
its regions and availability zones, charging a per-GB fee for most types of inbound and outbound data
transfer.

Faster performance Because demand varies, we consider both maximum requests per second and
average requests per second as our performance metrics. The two cloud instantiations use the same
processor generation, so it can be expected that raw processing power should be fairly equal. In this
case, IBM SoftLayers consistent performance edge may be attributed to differences in hypervisor
configuration, which can affect the performance of the virtual processor and virtualized I/O, as well as
network speed for different virtual machines. In a virtualized cloud environment, the customer has no
visibility or control over the hypervisor layer (which is managed by the cloud service provider). This is
one reason why bare metal can be an optimal choice, even for virtualized workloads: the customer can
deploy and manage its own hypervisor on the bare metal server, thus retaining control over the virtual
machines.

Note that the calculations for this and the following scenarios do not factor in the net present value of
moneyeven though many enterprise accounting departments would call for such an analysis in light of the
AWS long-term, paid-in-advance pricing option. Net Present Value is a budgeting practice that discounts the time
-value of investments. If Net Present Value had been taken into account in the use cases in this study, the pricing
and associated price-performance gaps would have been larger, with the IBM SoftLayer solution reflecting even
greater savings over the three-year period.

Use Case #2: Web application (non-virtualized)

In this scenario, we look at the price-performance for the same scenario as Use Case #1but we include IBM SoftLayer
bare metal as a comparative offer. Because AWS does not offer a non-virtualized option, we continue to compare against
a single-tenant virtualized version.

Workload description: Performance-intensive workload; multi-tier, online transaction processing.

Requirements: AppServer plus Database Server, each running approximately 8 vCPU, 16 GB RAM; 200 GB
Block Storage; RedHat Linux.

Solutions: The IBM SoftLayer solution comprises bare metal App Server and Database Server, each 4 core,
Haswell-based and including internal SSD storage. The competitive AWS service is the virtualized offering
described in Use Case #1. Pricing parameters were the same as in Use Case #1.

2015 Stratecast. All Rights Reserved. 8

 Stratecast | Frost & Sullivan

Calculating Price and Price-Performance: 4

Price(over3years)
Costcomponent IBMSoftLayer AWSEC2
Infrastructure(computeandstorage) $51,840 $89,796

Technicalsupport $0 $16,049
Datatransfer(Internet) $68,609 $74,013
Software $48,216 $137,760
Totalcost(3years) $168,665 $317,618

PricePerformance
Performancecomponent IBMSoftLayer AWS
Maximumrequestspersecond(RPS) 21,765RPS 16,079RPS
Averagerequestspersecond 3,628RPS 2,680RPS
Costperunitofwork(RPS) $46/averageRPS $119/averageRPS

ThisisanIBMinternalstudyofabasicwebapplicationworkload.Resultsobtainedundercontrolledconditionstestingresourcesofapublicly
availablecloudinfrastructure.Customerapplications,differencesinthestackdeployed,differencesindatacenterlocationsandoptions,and
othersystemsvariationsortestingconditionsmayproducedifferentresults.PricingbasedonpublishedUSlistpricesavailableforIBMand
competitordatacenterlocationspecifiedasof7/6/2015andcalculatedbyprojectingcostsmeasuredoverafixedtestingperiodtoa3year
TCA,includingcompute,network,storage,software,datatransfer,andsupportcharges.Pricingexcludesclientlaborcosts.Availableoptions
andpricingvarybydatacenterlocation;datacenterlocationschosenbasedonavailabilityofoffering.

Analyzing the Results: In this scenario, the introduction of the IBM SoftLayer bare metal option created
dramatic results. Since AWS does not offer a bare metal option, the nearest equivalent is a single-tenant
virtualized solution. The IBM SoftLayer solution was nearly half the cost of the AWS virtualized solution. At
the same time, the bare metal solution boosted performance over the virtualized offer, processing, on average,
35% more requests per second than the competitive offer. We can interpret the results as follows:

Lower price Primary contributors to the lower price of the IBM bare metal solution are storage and
software costs. As with the virtualized example, AWS requires a separate purchase of blocks of
persistent storagea cost that is included in the base monthly rate of the SoftLayer server.
Furthermore, software license costs are significantly lower with bare metal. As discussed earlier, a
physical CPU can be the equivalent of two vCPUs; so, a bare metal solution can require half as many
software licenses as a virtualized solution with similar processing capacity. In addition, as with Use Case
#1, IBMs data transfer costs are lower because of its more generous free tier.

Faster performance It is no surprise that bare metal returns superior performance over a virtual
machine. In a VM, the hypervisor layer consumes a portion of the available processor capacity, leaving
less maximum capacity for workloads. Conversely, when a workload runs on bare metal, it has full and
continual access to all the processing resources of the CPU. This ensures consistent compute power,
which is why bare metal is often the preferred choice for high-performance compute workloads.

4
Ibid.

9 2015 Stratecast. All Rights Reserved.

The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

Use Case #3: Analytics (storage-intensive)

The characteristics of analytics workloads can strain some cloud units. Multiple users may simultaneously execute quick
queries, while others are performing complex analyses. Storage is an important factor, as databases are continually
accessed and updated. In this scenario, we look at the price-performance for an analytics workload with a mix of queries
executing against a 1 TB data warehouse. Performance is measured in reports per hour.

Workload description: Storage-intensive, deep analytics workload; business intelligence; multiple users; mix of
complex and intermediate reports.

Requirements: 20 cores; 64 GB RAM; 1 TB SSD; 10 Gbps network; RedHat Linux.

Solutions: The IBM SoftLayer bare metal solution includes 20 cores running on Haswell processors; 64 GB
RAM, 1.6 TB SSD. The AWS offer is a virtualized, single-tenant unit comprising 36 vCPU (Haswell processors);
60 GB RAM; 1.6 TB SSD; 40K provisioned IOPS. As with the other use cases, pricing was based on the lowest
available rates: for SoftLayer, a monthly commitment; for AWS, a 3-year commitment, paid upfront; as noted
earlier, net present value of money was not factored into the calculations.

Calculating Price and Price-Performance: 5

Price(over3years)
Costcomponent IBMSoftLayer AWSEC2
Compute $41,796 $74,117
Storage $34,128 $100,800
Technicalsupport $0 $17,025
Totalcost(3years) $75,924 $191,942
Costperhour(over3years) $2.89/hour $7.30/hour
PricePerformance
Performancecomponent IBMSoftLayer AWSEC2
Analyticsperformance(reportsperhour) 13.4RPH 12.3RPH
Costperunitofwork(RPH) $0.22/RPH $0.60/RPH

IBMinternalstudyofananalyticsworkloadconsistingofreportsscanningallorportionsofa1TBdenormalizedstarschemadatabase.Com
petitorCloudconfigurationprovisionedforI/Obandwidthof40,000IOPS.Resultsobtainedundercontrolledconditionstestingresourcesofa
publicly available cloud infrastructure. Customer applications, differences in the stack deployed, differences in data center locations and
options,andothersystemsvariationsortestingconditionsmayproducedifferentresults.PricingbasedonpublishedUSlistpricesavailable
forIBMandcompetitordatacenterlocationspecifiedasof7/6/2015andcalculatedbyprojectingcostsmeasuredoverafixedtestingperiod
toa3yearTCA,includingcompute,network,storage,andsupportcharges.Pricingexcludesclientlaborcosts,DB2softwarelicensecosts,
andcloudproviderdatatransfercharges.Availableoptionsandpricingvarybydatacenterlocation;datacenterlocationschosenbasedon
availabilityofoffering.Optionfornonvirtualizedserversnotavailableincompetitorenvironment.

Analyzing the Results: In this scenario, differences in storage contribute to the performance and cost disparity
between IBM SoftLayer and AWS. The cost for the IBM SoftLayer solution is just 40% of the cost for the
AWS solution. The SoftLayer solution also outperformed its competitor in average reports-per-hour, for a
total price-performance edge of over 60%. We can interpret the results as follows:

Lower price Each component in this scenario shows a significant price advantage for IBM SoftLayer.
5
Ibid.

2015 Stratecast. All Rights Reserved. 10

 Stratecast | Frost & Sullivan

As seen in other scenarios, the bare metal option can introduce significant cost efficiencies for the
compute component. In addition, the internal SSD provides a cost, as well as performance, advantage
over AWSs externally configured storage.

Faster performance In this scenario, the performance difference is based on Input/Output

Operations per Second (IOPS). IBMs performance tests and analyses show that with SoftLayers bare
metal solution, 1.6 TB SSD actually delivers 112,419 IOPScompared with just 29,572 IOPS for similar
SSD storage in the AWS offering. The primary reason for the disparity is that, in the AWS solution,
access to the storage discs is via the hypervisor layer, which introduces a performance overhead
penalty that does not exist in the IBM SoftLayer bare metal solution. Another contributing factor is that
the IBM SoftLayer storage is single-tenant, permanent block storage, which is not available with AWS
(AWSs EBS Provisioned IOPS is only available as a multi-tenant storage solution, even when paired with
a single-tenant compute instance). Multi-tenancy can impact performance consistency because multiple
workloads are competing for the same infrastructure resources. Finally, in this scenario, the SoftLayer
bare metal server utilizes internal storage, which can reduce latency compared with the externally-
configured SSD utilized in the AWS solution.

Use Case #4: Messaging (network-intensive)

A messaging applicationin which a messaging server serves the role of passing messages between message producers
and message consumersdepends heavily on network throughput. In comparing performance of messaging workloads,
the specifications of the cloud compute units are not particularly relevant. Instead, we stress-test the environment by
continuing to add messaging volumes until the maximum server capacity has been reached. In a messaging workload,
performance is measured based on the maximum sustained throughputthat is, the highest possible rate of messages
passing through the server. It is measured in messages per second.

Workload description: High-speed passing of non-persistent messages, from sender applications to receiver
applications, via a messaging server.

Specifications: Messaging servers with a 10 Gbps network; each message is 12 kB.

Solutions: The IBM SoftLayer solution is 12-core bare metal, with 10 Gbps. The AWS solution is virtualized, 32
vCPU, 10 Gbps. Both solutions run on Ivy Bridge processors.

Calculating Price-Performance: 6
PricePerformance
Component IBMSoftLayer AWSEC2
Totalcost $128,112 $225,179
Messagespersecond(MPS) 70,925MPS 51,995MPS
Costperunitofwork(MPS) $1.81/MPS $4.33/MPS

This is an IBM internal study of a message passing workload. Results obtained under controlled conditions testing resources of a publicly
availablecloudinfrastructure.Customerapplications,differencesinthestackdeployed,differencesindatacenterlocationsandoptions,and
othersystemsvariationsortestingconditionsmayproducedifferentresults.PricingbasedonpublishedUSlistpricesavailableforSoftLayer
Washington,DCandAmazonUSEastdatacenterlocationsasof7/6/2015andcalculatedbyprojectingcostsmeasuredoverafixedtesting
periodtoa3yearTCA,includingcompute,network,storage,andsupportcharges.Pricingexcludesclientlaborcostsandcloudproviderdata
transfer charges. Available options and pricing vary by data center location; data center locations chosen based on availability of offering.
Optionfornonvirtualizedserversnotavailableincompetitorenvironment.

6
Ibid.

11 2015 Stratecast. All Rights Reserved.

The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

Analyzing the Results: As noted, this scenario is a little different than the previous use cases, in that we are
focused on the network performance, rather than the compute and storage costs. The IBM SoftLayer solution
delivers 36% more messages per second than the competitors solution. The SoftLayer solution also
outperformed its competitor in cost per message per second, for a total price-performance edge of nearly
60%. We can interpret the results as follows:

Lower price In this network-intensive scenario, the availability of high-bandwidth networking plays an
important role in determining the overall costs. With AWS, 10 Gbps port speeds are available only with
larger EC2 compute instances (those with 32-40 vCPUs). In contrast, IBM SoftLayer enables users to
select 10 Gbps ports with a variety of smaller and larger bare metal server sizes (12-40 cores). This
affects costs in two important ways. First, to attain the needed 10 Gbps network capacity, the AWS test
scenario was forced to provision more vCPUs than necessary to run the workload (in this case, 32
vCPUs on AWS versus 12-core bare metal servers on IBM Softlayer) This increased the compute-
instance costs for AWS. Escalating costs even further is the need to license middleware for each core.
As noted earlier, a physical CPU is roughly equivalent to two vCPUs, so software licensing fees can be
expected to be twice as high in a virtual environment compared with a physical environment. When
vCPUs are overprovisioned, the software fees can add tremendously to the costs.

Faster performance IBM SoftLayer bare metal servers utilize a 10 Gbps physical network interface
card, which directly connects to the physical SoftLayer network, for consistent, low-latency
performance. In contrast, the AWS environment utilizes virtualized network access via the hypervisor
layer. The abstraction provided by virtualization introduces performance penalties and inconsistencies.

WHAT ENTERPRISES NEED TO DO TO ENSURE MAXIMUM PRICE-

PERFORMANCE FOR CLOUD WORKLOADS
Cloud services are inconsistent and complex, with multiple factors that impact overall price-performance. Yet, as
the scenarios indicate, when the differences in price and yield can be so significant, it is worthwhile to devote
resources to consider all options before selecting a cloud service. Here is a summary of recommendations for
enterprises to add to their cloud procurement processes:

Do your homework. Dont assume that the provider thats in the news with price decreases will be the
best-priced provider for your workload.

Understand all workload requirements that will impact cloud workload costs and operations (not just
compute and storage). Consider costs associated with licensing software for each core; data transfer to
the Internet or private network; and persistent storage.

Understand how the provider will support geographically-dispersed workloads. If your application must
move data throughout the globe, ensure that the provider not only has data centers in the regions
where you do business, but also a high-performance, private global network. Also consider whether the
provider charges data transfer fees between and among cloud centersany such fees can considerably
add to costs if your company expands globally.

Consider your business requirements, including the agility tradeoff. Can you afford to lock-in to a
specific provider, unit type, volume, or time frame, even if it means a discount?

2015 Stratecast. All Rights Reserved. 12

 Stratecast | Frost & Sullivan

Consider the net present value of money when you evaluate long-term pricing options. Seek input
from your finance department, especially if you are considering an upfront payment option; this will
ensure your comparisons are valid and adhere to your companys accounting rules for net present value.

Factor in the non-workload-specific costs your business will need to run the workload optimally,
including technical support, engineering, and even professional services.

Allow for changing workload needs. You should be able to move workloads as needed from (for
example) bare metal to virtualized servers without a major effort.

Consider the big picture. Each cloud workload should fit into a holistic cloud strategy, one that will likely
comprise multiple deployment models, geographies, and vendors. Even as you consider price-
performance based on the individual cloud workload, consider the providers ability to support your
broader hybrid IT strategy, via OpenStack-compatible platforms, integrated solutions, and seamless
migration across models.

Finally, dont just look at price for the workload; instead, consider price-performance to be your base
unit of comparison, as you consider not only cloud options but any IT option.

13 2015 Stratecast. All Rights Reserved.

The Truth about Cloud Price-Performance: How Misperceptions about Service Costs can Derail Your Cloud Strategy

THE LAST WORD

Business leaders rely on data to make smart decisions. Unfortunately, in the case of the cloud, too often, IT
decision-makers do not know the cost and price-performance data that can help them assess cloud solutions.
Worse, they often do not know what they do not knowinstead, many continue to purchase cloud services
based on common misperceptions, without going through appropriate due-diligence.

The problem with inadequate due-diligence is two-fold: First, IT leaders are already seeing cloud costs spiral well
beyond expectations, without a clear understanding of what is causing the increase. Second, without visibility into
what it costs to run a cloud workload, IT leaders are not in a position to deliver workloads for optimal price-
performance.

As businesses become increasingly dependent on cloud, IT leaders have the responsibility to

ensure that the business derives the maximum value for each cloud workload. The workload must be
aligned with the appropriate cloud deployment optionsingle-tenant or multi-tenant, virtualized or bare metal,
on-premises, hosted, or hybrid. Furthermore, the workload must deliver appropriate levels of performance,
based on business requirements. Finally, the overall price-performance should be calculated and continually
measured, to ensure continuous optimization.

Choosing a cloud service provider should never be based on cost alone; instead, IT leaders need to assess value
based on factors such as performance, breadth of portfolio, purchase experience, customer service, security,
availability, and ease of migration and integration. Yet, price-performance should remain an important factor for
making business investment decisions, and for continuing to assess value. By building price-performance analysis
into their initial cloud purchase and ongoing cloud management processes, IT leaders can help ensure the success
of their cloud strategy.

Lynda Stadtmueller
Vice President Cloud Services
Stratecast | Frost & Sullivan
lstadtmueller@stratecast.com

2015 Stratecast. All Rights Reserved. 14

Silicon Valley San Antonio London
331 E. Evelyn Ave., Suite 100 7550 West Interstate 10, Suite 400 4, Grosvenor Gardens,
Mountain View, CA 94041 San Antonio, Texas 78229-5616 London SWIW ODH,UK
Tel 650.475.4500 Tel 210.348.1000 Tel 44(0)20 7730 3438
Fax 650.475.1570 Fax 210.348.1003 Fax 44(0)20 7730 3343

877.GoFrost myfrost@frost.com
http://www.frost.com

ABOUT STRATECAST

Stratecast collaborates with our clients to reach smart business decisions in the rapidly evolving and hyper-
competitive Information and Communications Technology markets. Leveraging a mix of action-oriented subscription
research and customized consulting engagements, Stratecast delivers knowledge and perspective that is only
attainable through years of real-world experience in an industry where customers are collaborators; todays
partners are tomorrows competitors; and agility and innovation are essential elements for success. Contact your
Stratecast Account Executive to engage our experience to assist you in attaining your growth objectives.

ABOUT FROST & SULLIVAN

Frost & Sullivan, the Growth Partnership Company, works in collaboration with clients to leverage visionary
innovation that addresses the global challenges and related growth opportunities that will make or break todays
market participants. For more than 50 years, we have been developing growth strategies for the Global 1000,
emerging businesses, the public sector and the investment community. Is your organization prepared for the next
profound wave of industry convergence, disruptive technologies, increasing competitive intensity, Mega Trends,
breakthrough best practices, changing customer dynamics and emerging economies? Contact Us: Start the
Discussion

For information regarding permission, write:

Frost & Sullivan
331 E. Evelyn Ave. Suite 100
Mountain View, CA 94041

Auckland Dubai Moscow Silicon Valley

Bahrain Frankfurt Mumbai Singapore
Bangkok Iskander Malaysia/Johor Bahru Oxford Sophia Antipolis
Beijing Istanbul Paris Sydney
Bengaluru Jakarta Rockville Centre Taipei
Buenos Aires Kolkata San Antonio Tel Aviv
Cape Town Kuala Lumpur So Paulo Tokyo
Chennai London Sarasota Toronto
Colombo Manhattan Seoul Warsaw
Delhi / NCR Miami Shanghai Washington, DC
Detroit Milan Shenzhen