Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Feng

Feng

Ratings: (0)|Views: 3|Likes:
Published by Nghĩa Zer

More info:

Published by: Nghĩa Zer on May 29, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

05/29/2011

pdf

text

original

 
Peer-to-Peer Bargainingin Container-Based Datacenters
Yuan Feng, Baochun Li
Department of Electrical and Computer EngineeringUniversity of Toronto
{
 yfeng, bli
}
@eecg.toronto.edu
Bo Li
Department of Computer ScienceHong Kong University of Science and Technology
bli@cs.ust.hk 
 Abstract
—In container-based datacenters, failure-prone com-ponents are sealed in pre-packaged shipping containers, andcomponent failures over time reduce the availability of re-sources. From the perspective of services, application instancescan usually be migrated across the boundary of containers asvirtual machines (VMs). In such an environment, it would besometimes beneficial to migrate application instances to takebetter advantage of available resources. In this paper, we firstpoint out that application placement strategies that are singularlyfocused on the efficiency of resource usage may not be beneficial,especially when resources are over-provisioned in container-baseddata centers. We believe that application instances should beplaced based on the
Buffet principle
, using resources in anaggressive fashion. Once failures occur, application instancescan be migrated across containers, by allowing containers tobargain with each other in a peer-to-peer fashion, treatingapplication instances with different resource requirements as“commodities” in a Nash bargaining game. We show that theratio of utilizing resources can improved by establishing suchlocal Nash bargaining games, where two containers bargain andtrade VMs with each other to increase their own utilities. Withsimulation, we verify the effectiveness of the proposed bargaininggames.
I. I
NTRODUCTION
One of the design objectives of container-based (modular)datacenters is the management of complexity and cost of deployment. Still, a basic configuration of the Sun MD S20modular datacenter, for example, is quoted at
$559
,
000
[1],which represents only a fraction of the total cost of ownership.As a consequence, it is important to achieve a high level of re-source utilization, when it comes to computational, bandwidth,and storage resources. Unfortunately, utilization in operationaldatacenters can be remarkably low, even only
10%
[2].Recent success in virtualization technologies provides pos-sible solutions to improve resource utilization. Such tech-nologies encourage datacenters be transformed from existingrigid IT configurations to a more agile infrastructure [3].Server virtualization techniques, such as VMware and Xen,facilitate
application instances
to be packaged inside
virtualmachines
(VMs). These VMs can then be migrated from oneserver to another without any downtime to the applicationrunning within. In addition to virtualizing computational andbandwidth resources, storage virtualization techniques, such asthe IBM SAN Volume Controller and Sun ZFS, are able tovirtualize storage space into virtual disks (Vdisks), which canbe easily migrated from one storage subsystem to another.As live migration of application and storage instancesbecome feasible by using virtualization, existing work (
e.g.,
Singh
et al.
[4]) has proposed centralized algorithms tomanage a load-balanced cluster of servers by migrating VMsacross server boundaries. By migrating VMs off 
overloaded 
servers or storage nodes, called “hotspots,to under-utilizedservers, performance degradation or premature failures dueto overload can be avoided or mitigated. Such centralizedload management and VM migration is triggered by overloadsituations in hotspots, and migration is performed across theboundary of individual servers.In this paper, we argue that such fine-grained centralizedmicromanagement to alleviate hotspots represents a step to-wards the right direction, but is still too microscopic to berealistic in modern
container-based 
, or
modular 
, datacenters.In container-based datacenters, we believe that computational,bandwidth and storage resources are abundantly available, butcomponent failures over time are the norm, rather than theexception [5]. This is due to the fact that, once such server-packed shipping containers are sealed and operational, it isvery difficult to repair or replace their components individu-ally. Though the reliability and maintainability of resourcesare theoretically described by MTBF or MTTF, failures indifferent resource dimensions in distinct containers may followtheir own degradation distributions.In this paper, we advocate the application of the
Buffet prin-ciple
[6] when it comes to launching application instances toutilize abundant resources in container-based datacenters. The
 Buffet principle
stipulates that, rather than carefully optimizingresource usage for efficiency, one can launch as many applica-tion instances as needed to utilize
all
available resources in anaggressive manner, as long as the marginal benefit outweighsthe marginal costs. While the Buffet principle fits well in theunique characteristics of container-based datacenters, are weable to handle a reduction of resource availability over timedue to component failures, if nearly
all
resources are beingutilized?Live migration of VMs comes to our rescue again, butat a much coarser granularity. We argue that VMs shouldbe migrated across the boundary of 
containers
, rather thanservers, and only when it is necessary to do so due to a lack of resources (caused by failures). In this paper, we propose thatcontainers should be treated as
peers
, and
bargain
with eachother in a
peer-to-peer 
fashion in a local trading market, withVMs traded as “commodities.” Such bargaining games shouldonly be triggered when there exists a resource deficiency, andshould terminate when the utilization ratio of resources is
 
2
relatively balanced. Such a design is simple and realistic tobe implemented, and takes place in an autonomic and self-organized manner, precisely as any other bargaining tradingmarkets.The remainder of this paper is organized as follows. InSec. II, we briefly introduce the Buffet principle and theVM placement strategy. After showing the benefits of redis-tributing VMs across different containers, Sec. III describesthe decentralized VM migration algorithm based on the NashBargaining Solution. In Sec. IV, we show the effectiveness of the proposed VM migration algorithm. We conclude this paperin Sec. V.II. A
PPLYING THE
B
UFFET
P
RINCIPLE
Modern applications hosted by container-based datacentersare highly diverse in their resource requirements. For example,video streaming encoders requires substantial CPU compu-tational resources; while one-click online hosting serviceshave a huge appetite for storage space. Generally, to ensureefficiency, the number of application instances, in virtualmachines (VMs), is determined within the context of theapplication. For example, the default number of replicas inGoogle File System is
3
[7], while alternative email servicesmay require as many as
15
replicas.After the number of application instances is determined,each of these instances is then deployed according to certainload balancing considerations. The design objective of thesestrategies is the
efficiency
of utilizing resources, which attemptto find the “sweet spot” of operation, such that the performanceper unit of resource consumed is maximized.However, a singular focus on efficiency may lead to theunder-utilization of resources. Since application demands varyquickly over time, it is difficult to predict such demandsaccurately. To guarantee service quality during peak hours,container-based datacenters routinely over-provision resourceavailability. If resources provisioned in a datacenter has theability to support
5
instances of 
100
different applications, yetonly
3
instances of these applications are deployed within, thestrategy unnecessarily wastes resources even when they arereadily available.We believe that this is a classic example where the
Buffet  principle
[6] should be applied. The Buffet principle arguesthat resources should be utilized as long as the marginal costis lower than the marginal benefit. In the context of designinga strategy to determine how many application instances shouldbe placed in each container, we may simply let each containeraccommodate as many application instances as it can tosaturate nearly all of its available resources, with respect toeither bandwidth, CPU, or storage space. The Buffet principleis a natural fit in this context for the following reasons:
Low marginal costs.
Once provisioned during design,resources in a container would be wasted if not utilized. Thisimplies that the
marginal cost 
of using such resources isexceedingly low. Though performance gains beyond the sweetspot may not be substantial, the marginal benefits — such asimproved resilience to failures if more replicas are maintainedmay still outweigh the marginal costs. With a containercosting millions to acquire and hundreds of kilowatts of powerto keep up, leaving resources unused may not rational.
Simple design.
Based on the Buffet principle, there is noneed to design elaborate schemes to determine the numberof application instances (as VMs) to be deployed in eachcontainer. The maximum number of VMs is deployed to utilizeall available resources.
Enhanced resilience.
With the service demand fluctuatingquickly and the network changing dynamically, the “sweetspot” is hard to be determined accurately. When experiencinga bursty increase of requests within one application, containerswho hold this application currently may be overloaded, leadingto performance degradation. Instead of setting a stringent limiton resource usage, a greedy consumption of resources mayimprove the resilience of the entire datacenter to unanticipatedcircumstances.III. T
HE
VM M
IGRATION
A
LGORITHM
In this section, we propose a distributed VM migrationalgorithm based on Nash Bargaining Solution to alleviate re-source under-utilization incurred by failures. To minimize thetransmission overhead by migrating, the algorithm responds“lazily” and “locally.”
 A. The Benefits of VM Migration
As component failures start to occur over time, and asavailability of resources in one dimension (
e.g.
, bandwidth)may decrease substantially, it is often the case that suchreduced availability makes it harder to fully utilize resources inother dimensions (
e.g.
, storage and CPU). For example, withBCube [8], the failure of an aggregator in one container willcause a decrease of its upload bandwidth; then, the reducedbandwidth may degrade the access to this container’s storageand CPU computing resources.Taking full advantage of the virtualization technology, wemay
migrate
VMs to an alternative container, to achieve ahigher level of resource utilization in the system. The follow-ing example illustrates potential benefits by such migrating.Consider a datacenter which has two containers and twoapplications. Each of the applications is provided by a cor-responding VM: VM
1
and VM
2
, respectively. The availableresources of containers at the beginning and resources requiredto handle one request in a VM are summarized in Table I.
TABLE IA
VAILABLE
R
ESOURCES FOR
E
ACH
C
ONTAINER AND
R
EQUIRED
R
ESOURCES FOR
E
ACH
VMResources Container
1
 / 
2
VM
1
VM
2
CPU (MIPS)
6 3 1
Storage Space (GB)
6 3 3
Bandwidth (Mbps)
6 1 3
Initially VMs are placed as shown in Fig. 1 (a), according tothe application placement strategy we discussed in Sec. II. Acontainer’s resource utilization ratio is defined as the geometricmean of its utilization ratios in three resource dimensions,
 
3
storage space, bandwidth and CPU. In this datacenter, theaverage resource utilization ratio is about
76%
.Then as time goes by, Container
1
loses some computingresources, which has only
3
MIPS left, and Container
2
losessome bandwidth, with
3
Mbps left. If no VM migration isinvolved, requests for Application
1
and
2
directed to the samecontainer can not be satisfied at the same time, which leadsto the average resource utilization ratio decreasing to around
44%
. Nevertheless, with VM migration, we can shift VM
2
inContainer
1
and VM
1
in Container
2
, as shown in Fig. 1 (b).In such scenario, the VM, which requires more resource in onedimension, is fit into the container that holds more availableresource in that dimension. Then, requests directed to thesame container for both Application
1
and
2
can be satisfiedsimultaneously, so that the average resource utilization ratiocan be increased to
87%
.
Container 1Container 2(b) With VM migration(a) Without VM migrationContainer 1Container 2VM1VM2VM1VM2VM2VM2VM1VM1
Fig. 1. The different application fit with and without VM migration.
From this example, we can see that by allowing VMmigration across different containers, the datacenter is not onlyable to gain higher resource utilization with limited system-wide resources, but also gives a more satisfactory performanceas it can handle more requests at the same time.
 B. The VM Migration Algorithm Based on Nash BargainingSolution1) System Model and Algorithm Trigger:
Before introduc-ing the VM migration algorithm, we first present the systemmodel. Let
denote the set of containers in a datacenter. Forevery container
i
∈ N 
, it is associated with a current availablestorage space of 
i
(
t
)
, in Gigabytes; a bandwidth of 
i
(
t
)
,in Mbps; and a CPU computing capability of 
i
(
t
)
, capturingthe amount of processing power it has, in MIPS. Let the setof application instances provided by the datacenter be denotedby
M
. For any
k
M
, VM
k
requires a certain amount of storage space
s
k
, bandwidth of 
r
k
, and computing resource of 
cl
k
to handle one request.
ki
(
t
)
is a binary variable indicatingif VM
k
is provided by container
i
or not at time
t
. And
D
ki
(
t
)
represents the number of requests for VM
k
directedto container
i
at time
t
.Instead of responding to the failures eagerly or regularly,our VM migration algorithm responses
lazily
,
i.e.
, it is onlyoperated when the imbalance of resource utilization ratios indifferent dimensions alters over a certain threshold. To be pre-cise, at time
t
, the utilization ratios in storage, bandwidth, andcomputing resources of each container
i
can be represented asfollows:
r
si
(
t
) =
k
∈M
ki
(
t
)
s
k
D
ki
(
t
)
i
(
t
)
1
r
bi
(
t
) =
k
∈M
ki
(
t
)
r
k
D
ki
(
t
)
i
(
t
)
1
r
ci
(
t
) =
k
∈M
ki
(
t
)
cl
k
D
ki
(
t
)
i
(
t
)
1
,
where inequalities are the
resource constraints
that resourcesutilized by all application instances stored on one container cannot exceeds its available resources. Let
σ
ri
(
t
)
be the standarddeviation of 
r
si
(
t
)
,
r
bi
(
t
)
and
r
ci
(
t
)
. When a container
i
’s
σ
ri
(
t
)
exceeds the pre-defined threshold
σ
threshold
, this containerwill “trigger” the VM migration algorithm at time
t
.
2) The Relaxed Nash Bargaining Solution:
To be practicaland auto-managed, the VM migration algorithm is done in apeer-to-peer fashion. We envision the existence of a bargainingtrading market. VMs are considered as “commodities” in thismarket. Peers, containers who own the commodities, evaluatethe utility of each commodity in the market, then try to bargainwith each other and exchange their commodities to increasetheir own utilities.Our VM migration algorithm is based on the
Nash Bar-gaining Solution
, which is a Pareto efficient solution to atwo-player bargaining game. In such a game, two individualshave the opportunity to collaborate for mutual benefit inmore than one way. By assuming that I) two individuals arehighly rational, II) each can accurately compare its desire forvarious things, III) they are equal in bargaining skill, and IV)each has full knowledge of the tastes and preferences of theother, Nash proposed a solution which should satisfy certainaxioms [9]. Let
u
be the utility function for Player
1
, and
v
theutility function for Player
2
. Under these conditions, rationalagents will choose what is known as the Nash bargainingsolution. Namely, they will seek to maximize
|
u
(
x
)
u
(
d
)
|
and
|
v
(
y
)
v
(
d
)
|
, where
u
(
d
)
and
v
(
d
)
are the
status quo
utilities (
i.e.
, the utility obtained if one decides not to bargainwith the other player).In the datacenter trading market, there are multiple con-tainers,
i.e.,
possible players. The trigger peer will choosethe corresponding player according to the following
PlayerSelection Principle
:
when a container triggers the VM mi-gration algorithm, it first checks out the dimension in whichits resource utilization ratio is the highest; and then choosesthe container with the lowest resource utilization ratio in thisdimension to bargain with
.The rationale behind is when
σ
ri
(
t
)
> σ
threshold
, dimensionwith the highest
r
i
becomes the “bottleneck” of fully utilizingresources in other dimensions. Application instances requirehigh resource usage in the bottleneck dimension should bemoved out to achieve a more balanced resource utilizationin all dimensions. Reasonably, the ideal destination of theseVMs should be a container with relatively sufficient availableresources in the bottleneck dimension.The Nash Bargaining Solution tries to find the optimalownerships of the two players’ commodities, so that both

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->