Software defined G.

Kandiraju
H. Franke
infrastructures M. D. Williams
M. Steinder
A fundamental component of any large-scale computer system S. M. Black
is infrastructure. Cloud computing has completely changed the
way infrastructure is viewed, offering more simplicity, flexibility,
and monetary benefits compared to a traditional view of
infrastructure. At the core of this transformation is the notion of
virtualization of infrastructure as a whole, with providers offering
infrastructure-as-a-service (IaaS) to consumers. However, just
offering IaaS alone is insufficient for software defined environments
(SDEs). This paper examines infrastructure in the context of SDE
and discusses what we believe are some of the fundamental
characteristics required of such infrastructureVcalled software
defined infrastructure (SDI)Vand how it fits into the larger
landscape of cloud computing environments and SDEs. Various
components of SDI are discussed, including core intelligence,
monitoring pieces, and management, in addition to a brief
discussion on silos such as compute, network and storage.
Consumer and provider points of view are also presented along
with infrastructure-level service-level agreements (SLAs). Also
presented are the design principles and high-level architectural
design of the infrastructure intelligence controller, which constantly
transforms infrastructure to honor consumer requirements (SLAs)
amidst provider constraints (costs). We believe that the insights
presented in this paper can be used for better design of SDE
architectures and of data-center systems software in general.

Introduction Providers have their own advantages in this model.

Cloud computing has gained significant adoption in recent
years with many consumers choosing to acquire cloud-based
infrastructure, platform services, and software services rather
than having them located in-house [1]. A fundamental
advantage for consumers who choose cloud environments
is the shift from spending on capital expenditure (CAPEX)
to only operational expenditure (OPEX), coupled with almost
zero time to acquire these infrastructure, platform, and
software services (in contrast with weeks and even months).
Additionally, this model also offers other advantages and
flexibility as consumers typically need only pay as they use
the infrastructure services (pay-as-you-go model) with no
contracts, and acquiring services as they need, as opposed
to committing to expenditure, time, and resources that they
would in the traditional model.
Digital Object Identifier: 10.1147/JRD.2014.2298133
By dealing with all the services management challenges
themselves, providers remove that burden from the
consumers, offering them simplified services. At the same
time, knowledge and expertise gained in provisioning for
one consumer can be applied to service many consumers
(as opposed to the traditional model where each of the
consumersVwhether they be individual end users or large
corporationsVdeal with managing services and associated
challenges themselves). Also, having the same services
(infrastructure, platform, or software) available to many
consumers fosters efficient utilization of the services and
the knowledge gained in managing these, thus leading to
a high return on investment for the capital investment that
a provider makes. In addition, having continuous consumer
traffic can create a constant stream of revenue. Also, as a
provider, once the management of these services is mastered,
hosting a new service can help retain existing consumers
and also attract new ones.

Copyright 2014 by International Business Machines Corporation. Copying in printed form for private use is permitted without payment of royalty provided that (1) each reproduction is done without
alteration and (2) the Journal reference and IBM copyright notice are included on the first page. The title and abstract, but no other portions, of this paper may be copied by any means or distributed
royalty free without further permission by computer-based and other information-service systems. Permission to republish any other portion of this paper must be obtained from the Editor.

0018-8646/14 B 2014 IBM

IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014 G. KANDIRAJU ET AL. 2:1

Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
 Therefore, while consumers come to providers to minimize
their costs (CAPEX and OPEX), providers seek consumers
to obtain the maximum from the one-time CAPEX
investment. This paradigm has brought a radical change
in the way computing is being performed today, with
the last few years resulting in intense competition
among many companies (e.g., Rackspace [2], Amazon [3],
IBM [4], VMware [5], etc.) to attract consumers and
retain them.
From a business perspective, a key element of this model
relies on attracting consumers and retaining them. In this
regard, a provider makes every attempt to differentiate
from other providers by offering new features, services,
enhancements and a host of other value-adds. However,
it is extremely important for a provider to do this
while still keeping the costs low (for maximum return
on investment). It is exactly this element that we want
to explore in this paper, particularly in the context of
infrastructure-as-a-service (IaaS).
With high consumer adoption of the cloud, acquiring
physical infrastructure is becoming more and more
uncommon, if not a rare phenomenon. However, from
a provider perspective, simply offering IaaS is not a
business differentiator today. We believe that to offer
value, infrastructure provisioning needs to be constantly
transforming or Bmutating[ amidst continuously changing
consumers (and consumer needs) coupled with constantly
evolving physical infrastructures (upgrades, etc.). This
requires intelligence to continuously monitor, analyze,
and execute to both satisfy consumer requirements and
provider requirements to minimize cost. This is precisely
the focus of this paperVwe describe the infrastructure
that honors these requirements and call it software defined
infrastructures (SDIs). In a larger setting of software
defined environments (SDEs), SDI becomes an integral
part of SDE, handling all the infrastructure-level
goals. We discuss what we think are a few important
characteristics of SDIs, along with some important
components, interactions with SDI, and intelligence
in SDI.
The remainder of the paper is organized as follows. The
first section deals with a traditional view of infrastructure,
consumers, providers and their interaction through SLAs
and infrastructure management. The second section
defines and examines SDIs in detail, discussing various
components, with deeper discussions on some of the
components (particularly on the monitoring and the
SDI controller components). The final section concludes
this paper.
What is infrastructure?
Traditionally, infrastructure refers to the physical and
organizational elements put together to serve the needs
of workloads. This includes all the hardware (servers,
2:2 G. KANDIRAJU ET AL.
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
network connectivity, storage, power sources, racks, cooling
elements, etc.) brought together onto a single physical
platform, whether it is a simple Bmachine room[ or a large
data center. All these disparate elements are configured to
work together in a coordinated manner (by connecting them,
configuring the firmware, hypervisors, operating systems,
and higher-level software) to serve the needs of a workload,
and hence of a business.
In the recent years, virtualization has completely
transformed this view of infrastructure. Evolution of cloud
computing particularly presents us with the notion of
virtual infrastructure where consumers request infrastructure
patterns (virtual compute, virtual network, virtual storage,
etc.) on which they deploy their workloads. Infrastructure
hence has become closely associated with the notion of
consumers (those that consume the infrastructure) and
providers (those that provide the infrastructure to the
consumers) [6].
Virtual infrastructure refers to a combination of the
three aspectsVvirtual compute, virtual network, and
virtual storageVto create a complete end-to-end resource
pattern Bon top of[ which a workload can be manifested.
Infrastructure requirements of a workload are expressed
to create an infrastructure pattern along with SLAs of
all the three components. Creating virtual infrastructure
amidst a constantly changing workload environment and
a heterogeneous physical infrastructure is a non-trivial
problem.
Infrastructure consumers and providers
Cloud computing and virtualization (at all levels) has led
to significant changes in the way infrastructure is viewed by
both consumers and providers. We briefly highlight some
important observations in this context.
Consumers
From a consumer viewpoint, infrastructure today Bis just
a click away,[ a phrase that implies that while acquiring
infrastructure used to be a long process consisting of
choosing, ordering, unpacking, assembling and installing,
today it is as simple as choosing and clicking on a provider’s
website. Also, today’s infrastructure is economical (with
no CAPEX, or capital expenditure, and only OPEX,
or operational expenditure). Thus, consumers no longer
need to worry about actual provisioning of infrastructure
(which can be an arduous task in itself) and the cost
associated with it. They simply can Brent[ virtual
infrastructure (in a pay-as-you-go model) without even
knowing where the physical infrastructure actually resides.
Infrastructure today is also flexibleVi.e., consumers can
Bresize[ or Breturn[ or Bacquire[ virtual infrastructure
at any given time in many ways. It also provides certain
guarantees in settings in which consumers are constantly
looking for a better infrastructure provider, with better being
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
defined as infrastructure that comes with stronger guarantees.
These guarantees may include performance guarantees,
for which a virtual resource is expected to have certain
amount of performance (compute speed, network bandwidth,
storage input/output operations per second, etc.). Also,
availability guarantees may involve a virtual resource
that is expected to be available for certain fraction of time
(e.g., 99.99%) or have certain automated bring-up/recovery
services or be associated with certain actions that are
taken based on predicted failures or some other availability
policy. Security guarantees may involve a virtual resource
that is expected to be trustable and should be quarantined
in case of a security attack. Regulatory guarantees may
involve a virtual resource that is expected to conform
to all the regulatory and compliance rules (such as retention
policies, geographic regulations, etc.) associated with an
organization or a government. Other types of guarantees
may also exist.
Providers
While a consumer considers a virtual infrastructure as a
standalone entity, a provider is typically responsible for
provisioning multiple virtual infrastructures onto a single
physical infrastructure and managing them. From a provider
viewpoint, several challenges may be present. Heterogeneity
in the physical infrastructure is probably the first challenge
to address, as physical components may comprise versatile
hardware combinations such as multiple processor
architectures, different kinds of storage and network
devices (solid-state disks, hard disk-drives, etc.), specialized
functional units (such as graphical processing units,
field-programmable gate arrays, processor accelerators),
appliances with specific functionality (Netezza** [7],
IBM V7000U [8]), etc. While one challenge here is the
management of diverse physical components, Resource
Mapping and placement of virtual infrastructure onto such
physical infrastructure is another complex optimization
problem (due to a large number of variables [9–11]).
Considerations here not only include the usual performance
and availability service-level agreements (SLAs) that
consumers request but also other ones including licensing
of software, energy and power savings (for minimal
cost) [12], geographical placement (regulatory constraints
and policies), etc.
Another important challenge that a provider faces is that of
continuous transformation. This refers to two continuously
evolving aspects: the changing behavior of workloads,
which results in dynamically evolving characteristics of its
virtual resources, and the changes in physical resources
(nodes/links going down, hardware upgrades, hypervisor/
operating system upgrades, etc.). These two aspects result in
continuous remapping and change of the virtual-to-physical
resource mapping, requiring continuous transformation of
virtual infrastructure.
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
Providers also need to be extremely aware about potential
cyber-attacks, malicious software, resource compromises,
etc. With elements of cyber attacks not being uncommon
today, providers need to be able to detect, analyze, and
isolate resources at various levels of the infrastructure,
while simultaneously allowing clients to plug in their own
policies for the same. Thus, security becomes another
challenge for providers. In addition, regulatory and legal
matters may place additional constraints on data storage
and compute deployment.
Finally, while a consumer’s goal is to acquire (virtual)
infrastructure with some guarantees, the provider’s main
goal is to satisfy the consumer’s requests incurring minimal
cost. Providers adapt techniques such as over-commitment
to give the notion to consumers they have all the resources
they need, but actually multiplexing a single physical
resource across potentially multiple consumers to save
cost. Providers also embrace policies for energy saving
and consolidation to save power costs.
Consumer-provider interaction
In this section, we focus on consumer-provider interaction
and how the contract for acquiring the infrastructure
is specified. In our opinion, there are two fundamental
considerations here: infrastructure patterns and SLAs
associated with these patterns. We examine each of these
aspects in some detail below.
Infrastructure patterns
Infrastructure pattern refers to the entire infrastructure
specification (compute, network, storage) along with
associated requirements (SLAs).
Figure 1 shows an example of a three-tier application
with a web server, application server, and a database
(indicated by WS, AS, and DB, respectively). At the highest
level, a consumer might only be interested in what the
deployment as a whole provides in terms of the highest-level
metricVi.e., web transactions-per-second (tnxs/sec) in this
case. However, we believe that guaranteeing such a metric
has implications on how the infrastructure needs to be
instantiatedVi.e., infrastructure itself needs to guarantee
certain infrastructure-level metrics (lower-level metrics). It is
this conversion of metrics that an orchestrator (which sits
above infrastructure) performs, handing down lower-level
metrics to the infrastructure, which then is responsible for
honoring these.
In this example of a three-tier application, the orchestrator
Bdecides[ to provision a load balancer in front of the web
server to handle many requests. For the second tier, the
orchestrator decides to place three instances of application
servers on three different nodes, and for the third tier, the
orchestrator makes a decision to create two instances of the
databases. In addition, the orchestrator also specifies the
desired network bandwidth between each of these nodes and
G. KANDIRAJU ET AL. 2:3
 Figure 1
Example of an infrastructure pattern with infrastructure-level SLAs. Here, C1 , C2 , C3 , and C4 denote different compute requirements in terms of
processing power, memory size, etc. N1 and N2 denote different network bandwidth requirements, while N3 denotes the network latency requirement. S1
denotes local storage capacity requirement, and S2 denotes the shared storage bandwidth requirement. S3 denotes the shared storage capacity requirement.

desired storage bandwidth and capacity for local and shared
storage. Required processing power is also specified. The
orchestrator might have decided these parameters using
historical data collected by repeated experiments of such
applications and building a knowledge-base (models such
as neural networks and machine learning techniques can be
used here).
All of the above requirements constitute performance
requirements from the infrastructure point of view. We
refer to these requirements as the SLAs. In addition to
performance SLAs, other SLAs such as availability SLAs,
security SLAs, etc. can be specified as well.
Infrastructure abstractions
Note that in the above example, while the orchestrator
specifies the number of nodes, in our opinion, it will not
specify which nodes. In other words, the infrastructure
provider has the freedom to choose the nodes (while
provisioning compute) and, similarly, corresponding network
and storage choices as well. Thus, the orchestrator at a
higher-level Bsees[ an abstraction of the infrastructure with
actual infrastructure details (e.g., physical details such as
data-center room, rack, chassis, type of network cards, and
manufacturer specifications) hidden.
We think there are various aspects of this abstraction
that can be provided by the infrastructure provider to the
higher-level consumers. Some of these include abstractions
in the following paragraphs.
Performance abstractionsVWith heterogeneity being an
important aspect of a data center, it becomes necessary
to abstract out the real physical details and provide APIs
for the consumers to acquire resources that have been
calibrated under a common denomination. For example,
a data-center hosting systems of different processor
architectures (x86, POWER*, etc.) could abstract out the
exact architectural details by exposing the processing
power as a calibrated number (e.g., a CoreMark** number
that is derived by executing CoreMark [13] benchmarks),
thus making it easy for consumption. Likewise, storage
performance can be calibrated using the FIO (flexible I/O)
[14] benchmark.
Availability abstractionsVVirtual resource availability has
become an important area of research over the years.
Enterprise applications need to be highly available
by potentially replicating processing, memory, or data
across physical machines, multiple data centers, or
even geographic regions around the world. However,
an infrastructure consumer need not be exposed to every
detail of how availability is implemented. Intelligent
abstractions can be put in place that indicate the Bstrength[
(or degree) of availability, thereby allowing a consumer

2:4 G. KANDIRAJU ET AL. IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014

Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
 to choose what is needed (the higher the strength, the
greater the cost). For example, a provider could offer
five different levels of availability abstractions for a
compute resource (a virtual machine or a container):
(i) no availability, (ii) 3-minute outage maximum, but
re-provisioning similar type of compute, (iii) greater
than 3-minute outage, but re-provisioning more robust
compute, (iv) additional prediction capabilities to
avoid several outage scenarios, but re-provisioning like
(iii) above if outage occurs, and (v) no outage with a
total disruption time of less than 3 seconds.
The above availability abstractions can be implemented
by the provider in the following ways. Abstraction (ii)
may simply involve a local re-start of the virtual compute.
Abstraction (iii) may involve a remote re-start of the virtual
compute, but on a node that has been statistically observed
(over a period of time) to have a fewer number of failures
(virtual compute failures). For (iv), in addition to (iii), a
prediction engine is active (see the section BSDI controller[),
and if a failure is predicted, the compute is live migrated
to another node (outage avoided). For (v), a technology
like micro-checkpointing [15] can be in place, constantly
replicating the virtual compute memory onto another node,
thus having a secondary ready in case of primary failure.
In this case, the Bswitch[ from the primary to the secondary
would be less than 3 seconds.
From a provider point of view, the ways to implement
the availability abstractions can change over time (as
technology evolves); therefore, abstractions play an
important role to insulate the consumer from the raw
implementation details.
Service-level agreements
Providers typically provide a notion of SLAs for the
infrastructure consumers. These SLAs serve as a binding
contract between both these parties. SLAs can range from a
single page describing the guarantees that the infrastructure
will offer, to hundreds of pages describing subtle nuances
(pricing subtleties, outage scenarios, etc.). For a provider,
they not only serve as the primary interface to interact with
a consumer, but also to differentiate from other providers.
Note that consumers of SLAs need not only be real
consumers (such as people, businesses, and organizations)
but also the higher-level softwareVi.e., software that needs
infrastructure. Consumers acquiring software-as-a-service
(SaaS) or platform-as-a-service (PaaS) would request such
services, which in-turn would Bmaterialize[ infrastructure
with SLAs as required.
Since different elements of infrastructure may be
associated with different SLAs, an SLA specification is
closely tied to an infrastructure pattern. In fact, the same
template that describes the pattern may also describe
the SLAs. Such templates are then consumed by the provider
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
(i.e., provider software) to provision the infrastructure.
For example, Heat [16] is evolving as an important
template-provisioner (or orchestrator, as it is sometimes
referred to) in this context.
Although we believe that such SLAs should ideally be
defined in terms of the abstractions (defined in the section
BInfrastructure abstractions[), we leave the possibility
open that not every implementation detail might fit into
the ideal notion of abstractions. This may indeed lead to
implementation descriptions becoming part of SLAs. Given
below are a few examples of infrastructure-level SLA
elements that a template may refer to:
• Performance SLAs
 compute: virtual CPU utilization and abstracted
compute power in CoreMark number
 network: bandwidth and latency
 storage: IOPS (input/output operations per second),
capacity, and abstracted storage power in a FIO number
• Availability SLAs
 compute: availability options (indicated above), local/
remote restart, live migration, and availability zones
 network: multi-pathing and abstracted proximity
allocations
 storage: RAID (redundant array of independent disks),
flash-copy, and abstracted redundancy levels
• Security SLAs: trusted hypervisor, trusted compute, and
encrypted data
• Other SLAs (examples): architectural capabilities (hyper-
threading level, etc.) graphical processing units (GPUs),
and solid-state disks (SSDs).
The above list is only a small sample (in the interest of space)
in the face of multitude of possibilities and definitions for
elements that constitute an SLA. In addition, providers
typically package several of these elements (packaging
criteria possibly derived from their business strategies) into
tiers denoted by bronze, silver, gold, platinum, etc.
In fact, it might indeed only be these packages that a
consumer will ultimately see.
Infrastructure management
Infrastructure management (IM) refers to the piece of the
provider software that accomplishes two important tasks
for the provider: first, it forms the user interface and point
of contact for the consumer (website/user-interface with
SLA options, service listing, etc.) and second, it provisions
the infrastructure that consumer has asked for. A third
important piece of the management (which we refer to
later in the paper in the context of SDIs) is continuously
G. KANDIRAJU ET AL. 2:5
 transforming the provisioned virtual infrastructure. In our
opinion, the IM software that is built for clouds must adhere
to the following characteristics.
The first characteristic is availability. IM forms the
gateway to the customer. Its high-availability is of foremost
importance. Typical solutions here include active-active
or active-passive provisioning of IM services. Examples
include efforts in CloudStack HA (High Availability)
[17], OpenStack** HA [18], Eucalyptus** Cloud
Controller [19], etc.
The second characteristic is scalability. With a number
of consumers moving to the cloud and with increasing
size of data-centers, IM scalability becomes an important
factor. Only a highly scalable IM service provider can
guarantee quick provisioning of virtual infrastructures in
large settings.
A third characteristic is decentralization/decomposability.
We believe that the IM software should not be deployed
as a single monolithic piece of software. It should be built
in a highly-componentized manner to compose IM services
as required.
A fourth characteristic is upgradeability. Software
life-cycle management and DevOps of IM are essential
to capture the continuous evolution of physical infrastructure
and to prevent any downtime for the consumers during
the upgrade.
A final characteristic is intuitiveness. IM needs to be
user-friendly, simple and intuitive for consumers.
Later, in the section BManagement services,[ we describe
the implications of SDIs on management services.
Software defined infrastructure
What is software defined infrastructure?
With virtual infrastructure becoming the primary model
of consumption today, the ability to create infrastructures
on-the-fly is an absolute necessity for providers. In addition,
overlaying of these virtual infrastructures on the same
physical infrastructure to satisfy diverse SLAs requirements
(at same time) is a challenging task. On one hand,
this requires intelligent decisions and placement while
provisioning (i.e., virtual to physical mapping). On the
other hand, continuous transformation of workload and
physical infrastructure (discussed in Section 2) mandates
the infrastructure to be continuously mutating, adapting
constantly to honor the SLA guarantees.
A few aspects become important characteristics of
infrastructure today. Among these, the ability to increase/
decrease capacities of virtual resources on-the-fly is a
key characteristic that differentiates physical and virtual
infrastructures. Also required is the flexibility to expand
infrastructure in various dimensions to incorporate novel
capabilities and monitoring (plug-ins for new software, easy
importing of new hardware, etc.). At the same time, the
2:6 G. KANDIRAJU ET AL.
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
ability to trigger such flexibility of resources is governed
by the current system behavior, which involves another
important characteristicVdeep monitoring of compute,
network, and storage and performing this in the context
of diverse SLA facets such as performance, health, security,
etc. (achieved using monitoring at application, guest,
hypervisor, and hardware levels). Data gathered from
such deep monitoring needs to be consumed and analyzed
by a control intelligence to take appropriate actions to
transform/mutate infrastructure, thus making such control
intelligence as a characteristic of infrastructure. The result
of such intelligenceVcontinuous remapping virtual resources
to physical resourcesVcan also be seen as a characteristic
of infrastructure today. Finally, this process of continuous
transformation results in learning by having ability to grow
knowledge-base to become Bmore intelligent[ (with time)
enabling capabilities such as predicting and avoiding
SLA-under-threat scenarios with constant monitoring,
analysis, planning, and execution.
The key goal of infrastructures today is to use all of the
above characteristics (or work amidst these) to honor the
SLAs promised to the consumers, while at the same time,
performing this to maximize provider benefits (such as
minimizing costs). We think that the above mentioned
characteristics more or less constitute our definition
of an SDI.
Software defined infrastructure refers to infrastructure
that is continuously transforming itself by appropriately
exploiting heterogeneous capabilities, using insights gained
from built-in deep monitoring, to continuously honor
consumer SLAs amidst provider constraints.
Components of SDI
Unlike the view of traditional infrastructure where software
plays some role in infrastructure provisioning (with the
other role played by hardware and humans), in SDI, software
plays a major role. In fact, most of the elements of SDI
are expected to be completely automated. However, it is to be
noted that since SDI is indeed a composition of individual
silos (compute, network, and storage), the automation
within each silo becomes an integral part of the end-to-end
automation.
Figure 2 shows a high-level overview of SDI. Several
important components include software defined compute
(SDC), software defined network (SDN), software
defined storage (SDS), resource abstraction, SDI controller,
management services, and monitoring services. We briefly
discuss each of these, and in the interest of space, provide
more information on three of the components here:
management services, monitoring services, and SDI
controller. Resource abstraction was dealt with in the
section BInfrastructure Abstractions,[ whereas the
remaining components are discussed by other papers
in this journal issue.
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014

Figure 2
Software defined infrastructure: overview of components.
Software defined compute
Virtual compute
Virtual compute is defined as a running instance of
an operating system with a certain number of virtual
processors, certain amount of memory, and potentially
other specialized devices such as graphical processing units
(GPUs), field-programmable gate arrays (FPGAs), or other
accelerators, etc. The capacities and capabilities of these
heterogeneous compute resources are expressed in an
abstract manner by calibrating them using specific
benchmarks. An operating system instance itself may be
instantiated as a bare-metal instance, a virtual machine,
or a container. Depending on the constraints in the SLAs
requested by the consumers, appropriate virtual compute
instances are created for use. Virtual compute can be used
to provision with guarantees on metrics such as processor
performance and utilization, compute availability, isolation
level, security, etc.
Software defined compute
Software defined compute refers to virtual compute that is
transforming as needed. This transformation is accomplished
by intelligence that is continuously consuming monitoring
data and validating if SLAs are being met (we revisit
monitoring in detail in the section BMonitoring services[).
When this intelligence (known as SDI controller, discussed
in the section BSDI controller[) determines that there
is a potential threat to the guaranteed SLA or that SLA
is already being violated, it initiates necessary action to
honor/re-enforce the SLAs.
As an example, let us consider a simple scenario where a
virtual machine has been created on a node with a certain
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
capacity. Suppose that the SLA that was requested (and
given) to the consumer for the virtual machine implies
that the VM needs certain (virtual) compute performance.
Now, if the load on other VMs running on the same node
increases, thereby starving this VM, then migrating this
VM to another node could be one way to continue to honor
the SLA for this VM (and possibly SLAs for the other
VMs on the node too). In this case, migrating compute
instances across physical nodes without affecting any
of the higher level layers (i.e., applications running on
the compute instances that constitute the workload layer)
is an action that is completely contained in the purview
of the SDI (although not necessarily within the
compute domain).
Software defined network
Virtual network
The virtual network connects different virtual compute/
storage instances, providing guarantees such as security
and isolation, performance (network bandwidth), availability
(multi-pathing and routing), etc. Creating the virtual network
typically consists of configuring controllers present in
physical switches and hypervisors to create virtual switches,
links, and end-points. Overlaying virtual networks on
physical networks and continuous remapping of this
constitutes an important problem here. Evolution of
standards such as OpenFlow [20] has played an important
role in proliferation of virtual networks for a data-center
in general.
Software defined network
Similar to SDC, SDN is transforming and evolving.
Monitoring data from the network controllers is constantly
being fed to the control intelligence to determine several
variables (flow and bandwidth parameters, link reliability
parameters, etc.). The intelligence is itself constantly
evaluating and potentially mutating the virtual
network(s) by reprovisioning or readjusting them
using the controller API.
For example, a virtual network might have been
provisioned at start to connect two compute instances.
While there might be two different physical paths
connecting the nodes on which these compute instances
were instantiated, one path might have been used to provision
the initial virtual network. However, as the compute
instances communicate, if the control intelligence determines
that the required network SLAs are not being met, it
might use the other physical link to distribute the traffic,
or switch the virtual network to the other link in entirety
(if SLA can be honored). Not only the applications
running on the compute instances, but compute instances
themselves might be completely Boblivious[ to such network
mutations occurring Bunderneath[ (a hypervisor might
G. KANDIRAJU ET AL. 2:7
 probably be aware of this). While some such actions may
not require involvement of other SDI silos, some may have to
be coordinated with SDC/SDS (we will revisit this in the
section BSDI controller[).
Software defined storage
Virtual storage
Virtual storage refers to provisioning and accessing storage
with diverse capabilities. Important variables here include
access bandwidth (IOPS), capacity, and the nature of fault
tolerance. Underlying capabilities (such as tiering, solid-state
disks [21], RAID, object/block storages, and distributed
file-systems) may be appropriately combined and abstracted
to create virtual storage (virtual volumes, etc.) to guarantee
SLAs.
Software defined storage
Similar to SDC and SDN, SDS refers to utilizing the
storage capabilities and transforming the virtual storage
to continuously honor the SLAs. Monitoring information
(such as number of reads/writes, IOPS, locality, etc.) plays
an important role in instantiating virtual storage with
appropriate capabilities.
As an example, let us consider a compute instance
provisioned on a node with a few RAID-enabled hard-disk
drives. Let us consider that due to the required storage SLA
specification of certain storage bandwidth and tolerating a
single-disk failure, the virtual storage was created in a striped
fashion on multiple disks. However, due to an unexpected
failure in the parity disk, the storage controller module
(SDS controller), possibly in conjunction with the
hypervisor, finally assigns a new disk [only available
free disk, but with a lower revolutions per minute (rpm)] as
the parity disk. At this point, while the storage availability
SLA is still honored (single disk failure can still be tolerated),
the performance SLA is under threat as the new parity
disk is a slower one. The control intelligence (see the
section BSDI controller[) in this case Bdecides[ to migrate
the compute instance to another node that has solid-state
disks for caching in order to meet the performance
criteria. Note that in this example, the SLA-under-threat
situation was first attempted to be handled by the SDS
controller. However, after possibilities at the SDS silo
have been exhausted, the SDI controller (a higher-level
entity that coordinates across silos) was notified, which
took the action to migrate the compute instance itself.
Thus, while some issues might be handled at individual
silo-level, others might have to be passed over to the
higher level controller.
SDI controller
Increasing adoption of the cloud makes data-center
expansion inevitable, resulting in large data-centers with
2:8 G. KANDIRAJU ET AL.
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
thousands of nodes and tens of thousands of paraphernalia
(links, switches, racks, chassis, power cables, etc.).
Consequently, maintaining large physical data centers
is a very difficult task, in addition to the much-needed
management of SLAs promised to the consumers. In other
words, some amount of manual intervention particularly
to manage the physical infrastructure is inevitable, such as
adding or removing physical infrastructure, connecting it
(according to consumer specifications at times), nonstandard
bare-metal installations, repairs, hot swaps, power and
cooling related tasks, etc. With the increase in scale,
it becomes practically impossible to manually manage the
system. Doing so would result in potential bottlenecks
and worse human errors. The programmability of the SDI
lends itself naturally to automating many of the tasks of
data-center management, reducing the requirements impact
of manual intervention.
Honoring consumers’ infrastructure SLAs in an automated
way becomes one of the important goals associated with
SDIs. Intelligence is required in SDI that continuously
keeps track of the current state of the infrastructure and
acts (or transforms) virtual and physical infrastructure
so that SLAs are continuously honored. It is this core
intelligence of SDI that we refer to as the SDI controller.
As such, it becomes a critical entity for the provider. For
example, lower costs might be an important benefit of
such automationVon one hand people-related costs
can be reduced, and, on the other hand, savings may be
gained in infrastructure maintenance by fine-grain real-time
automation (consolidation for power and energy, savings
in cooling, etc.). The blueprint of such a controller
incorporates several components that penetrate deep
into the infrastructure. For example, deep monitoring
(of hardware, hypervisors, network links, switches, disks,
etc.) becomes a critical component of the SDI controller.
With disparate sources publishing a variety of monitoring
information, a standard way of publishing, accumulation,
and subscription is required. This also implies a data-center
pervasive message bus for publishing and consuming
information and events, in a scalable and a fault-tolerant
manner. We refer to this component of deep monitoring
and information sharing mechanism as the M or monitoring
part of the SDI controller.
Next, consider the matter of consuming and analyzing this
monitoring information. This is where different analytics
engines plug-in. For example, a health statistics engine
may only be Binterested in[ and subscribe to health
events such as nodes going up and down, VMs crashing
and rebooting, network link failure counts, switch
replacements, hot disk swaps, etc. It can use this information
to derive data-center level health statistics at physical and
virtual levels. Similarly, an infrastructure policy engine
may subscribe to information and events related to power
consumption, CPU usage events, resource provisioning
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
 Figure 3
SDI controller overview.
times, etc. It may use such information to enforce data-center
level policies for energy consolidation, cost limits, IM
scaling, etc. Similarly, a workload policy engine may
subscribe to events related to VM deployment, specific
performance metrics, VM state, etc. It may use this
information to change the way a workload maps to the
physical infrastructure. Other engines that might plug-in
similarly include a prediction engine, security engine,
customer trend analysis engine, etc. This analysis and
accumulation part of the data-center becomes the second
important component of the SDI controller, and we refer
to this component as A or analysis.
An important result of such analyses could be the need to
transform the infrastructure (i.e., change the virtual resources
in some way). This is where a policy engine might apply
certain criteria or solve certain optimization problems to
derive a forward plan for the (virtual) resources. We refer
to this component as the P (planning) part of the SDI
controller. State maintenance and cross-silo coordination
are important aspects to consider here.
Infrastructure-level SLAs may also be taken into account
during both the A and P phases. For instance, moving a
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
virtual machine to a different host will likely have impact on
the network SLA and thus needs to be taken into account
when entire workloads are deployed.
Finally, the last part of the SDI controller is actually
executing the derived plan. In the execution phase, the
controller invokes APIs (potentially across all the silos)
to instantiate the designed plan. This phase becomes
much more management specific with specific adapter
functions getting invokedVfor example, if OpenStack is the
management system in the data center (or part of the data
center), generic operations that were part of the designed
plan become translated into OpenStack APIs to execute
individual operations. We refer to this component as the
E (execution) part of the SDI controller.
The SDI controller is a generic controller that fits in the
model of M , A, P, and E components (collectively referred
to as MAPE Loop [22], as shown in Figure 3) that can
subsume a variety of policy engines. It is possible for one
of these policy engines to govern a critical function (such
as placement) with it being the sole entity performing that
function data-center wide (possibly for consistency). In this
case, we envision that other policy engines will coordinate
G. KANDIRAJU ET AL. 2:9
 Figure 4
SDI controller architectural vision. (ssh: secure shell; VM: virtual machine.)
with it using an API (or the message bus, exposing
operations as events). Figure 4 shows an architectural
overview of the SDI controller that we envision.
All in all, the SDI controller is responsible for maintaining
the SLA of infrastructure patterns provisioned on physical
infrastructure. Generally, such management needs to consider
a very broad set of concerns related to pattern performance,
security, availability, and cost, as well as concerns
pertaining to the operational policies of the data-center
owner, which include electrical energy cost, software license
entitlements, server maintenance schedules, and many others.
Developing a system capable of addressing such broad
considerations has eluded the technical community so far.
As an example, resource mapping solutions have evolved
around independent technologies narrowly scoped to
networking [9], storage [10, 11, 23], cost efficiency [24],
or power [12, 25]. The SDI controller addresses the challenge
of holistic multiobjective management by introducing
clear abstractions that are used to subdivide the problem
space into smaller more manageable components and by
introducing advanced management in each of the following
three layers. The first layer involves the physical IM
responsible for the discovery and management of individual
physical resources and physical resource topologies,
2 : 10 G. KANDIRAJU ET AL.
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
provisioning of the management stack, and implementation
of policies related to health, security, and availability
of physical resources. The second layer involves virtual
resource management responsible for the life-cycle
management and local optimization of virtual resources
of compute, network, and storage. Typically these functions
would be implemented within the domains of SDC, SDN,
and SDS, respectively. The third layer involves infrastructure
pattern management responsible for the holistic orchestration
and optimization spanning the domains of compute,
network, and storage, leveraging the abstracted capabilities
provided by SDC, SDN, and SDS, respectively as well
as the abstractions provided by physical infrastructure.
An example of a component implementing this function is
described in a companion paper [26] (where infrastructure
pattern is referred to as virtual resource topology).
While each component addresses a broad set of
considerations (e.g., security, availability, etc.) they are
scoped to a narrow and well-defined set of management
entities, actions, and inputs. As an example, physical IM
is concerned with the fine-grained state of physical servers,
including server network reachability, state of individual
DIMMs (dual in-line memory modules), cores, and disks,
hypervisor state, provisioning process, etc. However,
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
these details are abstracted to a single flag Bunavailable/
unavailable[ when Bseen[ by an infrastructure pattern
management component.
Figure 4 shows the high-level design of the SDI controller,
with the MAPE loop and individual silos represented in
logical fashion (i.e., compute nodes or storage disks are
not shown). We believe that the SDI controller (and
the policy engines comprising it) are an important
value-add to a data center. This framework can particularly
be applied to enforce infrastructure-level SLAs promised to
the consumers in an automated manner. Also, the ability
to define a diverse set of policies on data-center-wide
universal functions (and APIs) can be extremely useful to
providers and consumers. In fact, we believe that the API to
define these policies should be exposed to consumers so that
they can build and deploy their own policies.
Management services
In this section, we discuss the management services
component of SDI. In the section BInfrastructure
management,[ we highlighted a few important features that
today’s management services must provide: availability,
scalability, decomposability (de-centralized), upgradeability,
and intuitiveness. We feel that management in an SDI setting
must make use of the above five aspects as primitives but
build on them to provide additional capabilities (to make
the infrastructure truly software defined). These capabilities
include the following.
Elasticity
In SDI, the management services need to be elasticVi.e.,
scale out (up) or in (down), based on the need (current load
on the system). Scaling out (up) may be required when the
load on the management services system is highV either
due to external consumers and due to internal infrastructure
mutation discussed earlier. Without scaling out (up), it
may take long time to provision resources (with limited
management services) and hence become a consumer Bpain
point.[ At the same time, scaling in (down) is an essential
feature to save costs (during low loads)Vthere is no need
to run multiple management instances on nodes that are idle.
Therefore, as a provider, this dynamic scaling (up/down or
out/in) is an essential factor for achieving high consumer
satisfaction with low cost. We believe that this is only
possible when such dynamic scaling mechanisms are
automated, and this is precisely what we mean by elasticity.
This would be an essential part of the SDI intelligenceVin
other words, elastic management services would be yet
another policy engine as part of the SDI controller.
We also believe that in order for management services
to be elastic, the core services need to be built, exposing
APIs for scaling, and the APIs are then invoked using
a higher-level intelligence (see the section BSDI controller[),
which uses the monitoring information.
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
Monitoring
Every element of SDI is monitored. This not only includes
deep monitoring of the hypervisor, compute, network,
storage, etc., but also monitoring of resource provisioning
and management services. This is in fact an absolute
requirement to even know how the infrastructure services
are performing as a whole. Monitoring data here might
refer to metrics such as compute provisioning time (which in
itself might be split as image provisioning time, boot time,
individual services start time, etc.), network and storage
provisioning time, time to link virtual resources (such as
associating a virtual volume to compute), metrics in
management services software (authentication time), etc.
Monitoring data forms the basis for the SDI controller,
not only for scaling management services but also
for event and correlation analysis, diagnostics, prediction
models, etc.
SLAs for management services
Having SLAs for management services can be a differentiator
for a cloud providerVi.e., in addition to having SLAs
associated with infrastructure patterns, a consumer can also
be guaranteed that the pattern will be provisioned in a
specified amount of time.
Examples of management services
A number of options for management services exist today,
with several clouds building their own management stack.
Examples of management software include Eucalyptus
CLC (cloud controller), CloudStack Management Server,
VMware vCloud Suite, OpenStack, etc. OpenStack
is evolving as a de facto open source standard for cloud
management. More than 200 companies are part of the
OpenStack project released under the Apache License.
Monitoring services
Monitoring services of SDI (and a data center in general)
form one of the most important components aiding in making
many decisions. In addition, monitoring itself might be
offered as a service to the consumers. Monitoring here not
only refers to performance aspects of the system but also
its health. It is also essential to enforce security and other
regulatory needs of the consumers. As mentioned in the
discussion on SDI controller, there are two fundamental
aspects of monitoring. First, consider the method of
measuring and gathering data; this varies for all the silos
and individual components and statistics that are being
measured. While a data center’s monitoring system will
make available a plethora of default statistics, consumers
might be interested in specific monitoring information
by providing a means to measure the same. Second, consider
the method of sharing the measured information through
a common message busVa bus that is highly scalable,
asynchronous, fault-tolerant and light-weight (does not
G. KANDIRAJU ET AL. 2 : 11
 interfere with system performance) where information and
events can be published and subscribed to.
For either of these aspects, we think that fundamental
primitives (APIs) need to be exposed to build generic
services including policies in the SDI controller discussed
earlier. Some of these services may be provided by the
data center in a generic way (e.g., CloudWatch), whereas
others can be custom-built for the consumers (or built by
consumers themselves). We believe that there are a few
important characteristics that a monitoring framework
for a next-generation data-center needs to obey.
First, consider the end-to-end characteristic. Monitoring
needs to be end-to-end; this includes monitoring of
lower-level hardware, hypervisor, virtual resources (compute,
network, and storage), applications and monitoring of
infrastructure provisioning itself. APIs at all levels need
to be exposed for consumption of monitoring data.
Second, consider the lightweight characteristic. We believe
that simple, lightweight and non-intrusive monitoring system
is a very essential component of data-centers. Simplicity
is not only in terms of functionality, but also install and
upgrade (DevOps). Heavyweight systems that come with
a host of tools and analysis can quickly become an overhead
and intrude system performance.
Third, consider fault-tolerance. With component failures
in data-centers not far from being norm, monitoring system
needs to be a reliable source for monitored data.
Fourth, consider distributed and loose coupling. We feel
that monolithic monitoring infrastructures do not scale and
also increase the complexity in consumption and analysis.
Lower-level simpler monitoring services should be built to
be able to couple together to form higher-level monitoring
services (such as CloudWatch).
Fifth, consider flexible and event driven characteristics.
Monitoring APIs need to be very generic for consumers
to put in data, retrieve data (say, across any time windows,
any ranges, etc.) or receive events using some generic
criteria about metrics at any level (e.g., notification if CPU
utilization on a node crosses 50% or if a node has seen
increasing trend in memory errors in last week, etc.).
Fifth, consider clean APIs. APIs at various levels of
monitoring should be simple and intuitive. RESTful APIs
have become a common model of consumption in general
and they seem to fit in well here in designing loosely
coupled, distributed large-scale monitoring systems.
Monitoring frameworks such as ganglia [27] have gained a
lot of attention in recent years not only due to their scalability
and light-weightedness, but their simplicity to be used to
compose higher-level services.
Conclusion
Virtualization has transformed the view of infrastructure
and its consumption model. With costs of infrastructure
dropping, price may not be the primary differentiator for
2 : 12 G. KANDIRAJU ET AL.
Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.
infrastructure providers. It is the additional value-adds
of availability, security, monitoring, and sophisticated
management combined with intelligence for automation
that we believe will emerge as differentiators in the market.
In addition, the ability for consumers to create or plug-in
their own policy engines as part of this automation using a
standard API exposed by the infrastructure-level would be an
additional benefit for creating consumer-specific solutions.
We believe that the rapid shift to consuming virtualized
solutions necessitates infrastructures to be software defined
and that the characteristics and components discussed in
this paper will aid the design of data-center systems software
for infrastructures.
*Trademark, service mark, or registered trademark of International
Business Machines Corporation in the United States, other countries,
or both.
**Trademark, service mark, or registered trademark of Netezza, Inc.,
Embedded Microprocessor Benchmark Consortium, OpenStack
Foundation, Eucalyptus Systems, Inc., Linus Torvalds, or The Open
Group in the United States, other countries, or both.
References
1. R. Prodan and S. Ostermann, BA survey and taxonomy of
infrastructure as a service and web hosting cloud providers,[ in
Proc. 10th IEEE/ACM Int. Conf. Grid Comput., 2009, pp. 17–25.
2. RackSpace. [Online]. Available: http://www.rackspace.com/
3. Amazon Web Services. [Online]. Available: http://aws.amazon.
com/
4. IBM Cloud Computing Overview, IBM Corporation, Armonk,
NY, USA. [Online]. Available: http://www.ibm.com/
cloud-computing/us/en/
5. Cloud Computing with VMWare Virtualization and Cloud
Technology. [Online]. Available: http://www.vmware.com/
cloud-computing.html
6. Review and Summary of Cloud Service Level Agreements.
[Online]. Available: http://www.ibm.com/developerworks/cloud/
library/cl-rev2sla.html
7. Netezza: IBM Netezza Data Warehouse Appliances, IBM
Corporation, Armonk, NY, USA. [Online]. Available:
http://www-01.ibm.com/software/data/netezza/
8. IBM Storwize V7000 and Storwize V7000 Unified Disk Systems,
IBM Corporation, Armonk, NY, USA. [Online]. Available:
http://www-03.ibm.com/systems/storage/disk/storwize_v7000/
9. O. Biran, A. Corradi, M. Fanelli, L. Foschini, A. Nus, D. Raz, and
E. Silvera, BA stable network-aware VM placement for cloud
systems,[ in Proc. 12th IEEE/ACM Int. Symp. CCGRID,
Washington, DC, USA, 2012, pp. 498–506.
10. M. Korupolu, A. Singh, and B. Bamba, BCoupled placement in
modern data centers,[ in Proc. IEEE IPDPS, 2009, pp. 1–12.
11. D. Yuan, Y. Yang, X. Liu, and J. Chen, BA data placement
strategy in scientific cloud workflows,[ Future Gen. Comput. Syst.,
vol. 26, no. 8, pp. 1200–1214, Oct. 2010.
12. B. Li, J. Li, J. Huai, T. Wo, Q. Li, and L. Zhong, BEnacloud:
An energy-saving application live placement approach for cloud
computing environments,[ in Proc. IEEE Int. Conf. Cloud
Comput., 2009, pp. 17–24.
13. The Embedded Microprocessor Benchmark Consortium
(EEMBC). [Online]. Available: http://www.eembc.org/coremark/
14. FIO Benchmark. [Online]. Available: http://freecode.com/
projects/fio
15. B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson, and
A. Warfield, BRemus: High availability via asynchronous virtual
machine replication,[ in Proc. 5th USENIX Symp. NSDI, Berkeley,
CA, USA, 2008, pp. 161–174.
IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014
16. Heat. [Online]. Available: https://wiki.openstack.org/wiki/Heat
17. CloudStack High Availability. [Online]. Available: http://
cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.1.0/
html/Installation_Guide/feature-overview.html
18. OpenStack High-Availability. [Online]. Available: http://docs.
openstack.org/trunk/openstack-ha/content/ch-intro.html
19. Eucalyptus. [Online]. Available: http://www.eucalyptus.com/
20. Open Networking Foundation. [Online]. Available: https://www.
opennetworking.org/
21. K. El Maghraoui, G. Kandiraju, J. Jann, and P. Pattnaik,
BModeling and simulating flash based solid-state disks for
operating systems,[ in Proc. 1st Joint Int. Conf. Perform. Eng.
WOSP/SIPEW, 2010, pp. 15–26.
22. P. Lalanda, J. A. McCann, and A. Diaconescu, Autonomic
Computing: Principles, Design and Implementation.
London, U.K.: Springer-Verlag, 2013.
23. A. Singh, M. Korupolu, and D. Mohapatra, BServer-storage
virtualization: Integration and load balancing in data centers,[ in
Proc. ACM/IEEE Conf. SC, Piscataway, NJ, USA, 2008,
pp. 53:1–53:12.
24. R. Zhou, F. Liu, C. Li, and T. Li, BOptimizing virtual machine
live storage migration in heterogeneous storage environment,[ in
Proc. 9th ACM SIGPLAN/SIGOPS Int. Conf. VEE, New York,
NY, USA, 2013, pp. 73–84.
25. E. Feller, L. Rilling, and C. Morin, BEnergy-aware ant colony
based workload placement in clouds,[ in Proc. IEEE/ACM
12th Int. Conf. GRID Comput., Washington, DC, USA, 2011,
pp. 26–33.
26. W. C. Arnold, D. J. Arroyo, W. Segmuller, M. Spreitzer,
M. Steinder, and A. N. Tantawi, BWorkload orchestration
and optimization for software defined environments,[ IBM J.
Res. & Dev., vol. 58, no. 2/3, Paper 11, 2014 (this issue).
27. Ganglia Monitoring System. [Online]. Available: http://ganglia.
sourceforge.net/
Received September 25, 2013; accepted for publication
October 27, 2013
Gokul Kandiraju IBM Research Division, Thomas J. Watson
Research Center, Yorktown Heights, NY 10598 USA (gokul@us.ibm.
com). Dr. Kandiraju is a Research Staff Member and Manager for
Data-center Systems Software at the IBM T. J. Watson Research
Center. He received a bachelor’s of technology degree in computer
science from the Indian Institute of Technology, Madras, India in 1999,
and a Ph.D. degree in computer science from The Pennsylvania State
University in 2004. After joining the IBM T. J. Watson Research
Center in 2004, he worked in areas including operating system virtual
memory management, enterprise storage systems, file-system design,
and cloud computing. He has contributed to IBM products and is
an author of several publications and patents. He is also a member of
ACM and a senior member of IEEE.
Michael D. Williams IBM Systems and Technology Group,
Poughkeepsie, NY 12601 USA (mdw@us.ibm.com). Mr. Williams is
a Distinguished Engineer and member of the IBM Academy of
Technology. He joined IBM in 1989 after graduating magna cum laude
from the State University of New York at Oswego with a bachelor’s
degree in computer science. Throughout his career, Mr. Williams
has had a broad set of client facing and development responsibilities
in the area of systems and software. His current focus is software
architecture and design for IBM’s next-generation systems with an
emphasis on software defined environments, cloud, and virtualization.
Malgorzata (Gosia) Steinder IBM Research Division,
Thomas J. Watson Research Center, Yorktown Heights, NY 10598 USA
(steinder@us.ibm.com). Dr. Steinder is a Research Staff Member and
Manager of the Middleware and Virtualization Management Group
at the IBM T. J. Watson Research Center. She has worked on topics
pertaining to the management of computation and data in large-scale
distributed environments. She led a research team that invented
and developed virtual machine management technology in IBM
WebSphere* Cloudburst Appliance and developed a dynamic
VM placement algorithm for IBM VMControl Enterprise Edition.
Dr. Steinder holds a Ph.D. degree in computer science from the
University of Delaware and an M.Sc. degree in computer science
from AGH University of Science and Technology, Krakow, Poland.
S. Mark Black IBM Platform Computing, IBM Canada,
Markham, ON L3R 9Z7 Canada (mblack1@ca.ibm.com). Mr. Black
is an Architect with the IBM Platform Computing Group. He received
an M.E.Sc. degree in signal processing from University of Western
Ontario in 1995, and a B.Eng. degree in electrical engineering from
Ryerson University in 1992. Prior to joining IBM, Mr. Black’s career
started in systems administration, where he gained an understanding
of how systems should be managed. He then went on to work on
systems management automation tools. He has designed and led the
development of a large-scale UNIX** systems administration tool,
several easy to use high-performance computing management tools,
and an enterprise cloud tool and is currently designing cluster
deployment tools. He has authored and coauthored three technical
papers. Mr. Black is an IEEE (Institute of Electrical and Electronic
Engineers) lifetime member award holder.

Hubertus Franke IBM Research Division, Thomas J. Watson

Research Center, Yorktown Heights, NY 10598 USA (frankeh@us.ibm.
com). Dr. Franke is a Research Staff Member and Senior Manager for
Software Defined Infrastructures at the IBM T. J. Watson Research
Center. He received a Diplom degree in computer science from the
Technical University of Karlsruhe, Germany, in 1987, and M.S. and
Ph.D. degrees in electrical engineering from Vanderbilt University
in 1989 and 1992, respectively. He subsequently joined IBM at
the IBM T. J. Watson Research Center, where he worked on the
IBM SP1/2 MPI (Message Passing Interface) subsystem, scalable
operating systems, Linux** scalability and multi-core architectures,
scalable applications, the PowerEN architecture and application
space and, most recently, cloud platforms. He received several IBM
Outstanding Innovation Awards for his work. He is author or coauthor
of more than 30 patents and over 100 technical papers.

IBM J. RES. & DEV. VOL. 58 NO. 2/3 PAPER 2 MARCH/MAY 2014 G. KANDIRAJU ET AL. 2 : 13

Authorized licensed use limited to: UNIVERSITY OF HERTFORDSHIRE. Downloaded on July 23,2021 at 20:40:05 UTC from IEEE Xplore. Restrictions apply.