17th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications
Self-Assessment and Reconfiguration Methods for
Autonomous Cloud-based Network Systems
Kaliappa Ravindran
Department of Computer Science
The City University of New York
New York, NY 10031, USA.
Email: ravi@cs.ccny.cuny.edu
Abstract—A system that is highly dependable under hostile condi-
tions but whose dependability cannot be easily evaluated prior to the
deployment of applications is less desirable than a system with lower
but predictable dependability. This is because a decision-making
on the deployment of high assurance systems is often based on a
risk analysis of application failures. For systems implemented on a
cloud, the problem of system certification assumes added importance
because of third-party control of cloud resources and the attendant
problems of faults, QoS degradations, and security violations. In this
light, our paper focuses on: i) formulating metrics to quantify the
dependability of cloud-based applications; and ii) identifying tech-
niques to measure these metrics prior to deployment of applications.
The paper treats system dependability as an application-level QoS
for management purposes, and advocates a probabilistic evaluation
of dependability. Our approach is corroborated by measurements on
system-level prototypes and simulation analysis of system models in
the face of hostile environment conditions. A case study of replicated
data service anchored on cloud infrastructures is also described.
I. I NTRODUCTION
We consider an application running on top of the com-
putational and communication services realized over one or
more cloud infrastructures. The system as a whole implements
a core functionality, with augmentations from the service
provider to support a variety of para-functional behaviors.
For instance, data replication and content distribution may
be offered as core services that are often associated with,
say, performance, security, and timeliness attributes. Here,
the problem of system certification (i.e., reasoning about
whether a system behaves in the way it is supposed to) has
become important because of the third-party control of cloud
resources and the attendant issues of fault-handling, security,
maintenance, availability, and the like [1].
The dependability of a cloud-based application system S is
a measure of how good S meets its intended QoS objectives
under uncontrolled external environment conditions incident
on S. Say, for example, S is a content distribution network
(CDN) that advertises content delivery to clients within 5 sec
of a request: say, News download. Suppose S achieves the
best QoS of 5 sec guarantee only with a probability of 0.4,
and achieves a latency distributed between 5 and 15 sec in
other cases. Under simplifying assumptions, the dependability
of S in meeting latency specs is estimated as 0.7 on a
normalized scale [0, 1]. If the content storage/delivery backlog
becomes severe, the 5 sec latency is less sustainable (assuming
that other parameters of storage/delivery mechanism do not
1550-6525/13 $26.00 © 2013 IEEE 87
DOI 10.1109/DS-RT.2013.37
change) — and hence S is now much less than 70%-capable in
meeting the latency specs. The system capability may however
be enhanced by installing additional proxy server nodes along
the content distribution topology. Concomitant with this notion
of system dependability is a safety aspect, namely, S should
not reach unsafe states while meeting its objectives: say, in this
example, the client connectivity to a content getting disrupted.
The goal of our paper is to design the software engineering
methods and tools to quantify dependability, and identify the
system-level techniques therein to assess the dependability
of S. Analyzing the dependability of S involves verifying
that the safety requirements are met as S strives to meet its
QoS objectives under various external environment conditions
incident on the underlying cloud services.
Our notion of dependability of S is at the confluence
of QoS, timeliness, and fault-tolerance attributes of S. It
is divorced from a traditional view where the dependability
of S is rigidly tied to the fault-tolerance of S. The QoS
feature depicts an ability of S to control its performance in
response to an underlying infrastructure resource allocation or
a change in the external environment conditions. The QoS-
to-resource mapping relationship should be established in a
quantitative manner under specific environment conditions, in
order to meet the performance objectives in predictable way.
An example is the determination of content delivery latency
over a distribution network set up on a geographically spread-
out cloud of content storage nodes, in the presence of node
failures. Virtualization, which allows realizing the distribution
network as a core service from the cloud provider, does not
by itself prevent fluctuations in the latency behavior (e.g.,
jitter) induced by node failures and outages. Here, a para-
functional goal is to reduce the latency jitter by resorting to
content caching techniques, thereby assuring a stable behavior
of applications. The mapping between the output of S and
platform resources should be known with reasonable accuracy:
either as a closed-form model of S or through a series of
incremental allocate-and-observe invocations on S [2].
The domain-specific core adaptation function in a cloud-
based application system S is viewed as a control-theoretic
feedback loop acting on a reference input Pref : say, for cloud
resource management (such a view is also advocated in [3]).
Figure 1 concretizes this view . The controller C generates
its actions I based on a computational model of S, denoted
 treats observed system as a composition the dependability analysis by H. Here, the service attribute
S { Ap’ g*(..) CLOUD-BASED APPLICATION
exported by the computational algorithm to its applications
adaptation logic Ap’ rests upon the quality of lower-level service received from
reason about
the underlying infrastructure (e.g., resource allocation, failure

system output
system dependability by

system input
(supplied as
(observed as
External observing system behavior final QoS specs of
detection). These core layers together constitute g ∗ (· · ·), with

Pref)
management

P’)
actually desired
entity H achieved QoS A0p housed in the application layer for behavior monitoring &
mapping between internal-state Service interface control. The latter involves exercising the underlying algorithm
visible
and interface events
state layer (and in turn, the infrastructure layer) via signaling points
defined in the service interfaces. The dependability of S

adaptive application system S

Processes, procedures
and their interactions
internal state s*
may be quantified, with suitable metrics (for certification and
control purposes), by analyzing the external state-machine
level signal flows by H. Basically, H reasons about the
sub-system providing
core services Service-oriented output tracking error |Pref − P 0 | under various environment
algorithms conditions, and maps it onto a measure of the dependability
resource/component
modeled as a external environment virtualization interface of S. Say, H is a cloud management station, employing a
black-box conditions incident on dashboard-based supervisory controller. The paper embarks on
compute servers, . . .)
(VMs, network links,

computational
System components

software/hardware system
function (e.g., errors, threat, outage) a case study of cloud-based content distribution networks, with
data storage,

g*(I,O*,s*,E*)
[parametric representation E*]
I: input
O*: output
s*: internal state
infrastructure
Fig. 1. Structure of adaptation processes in a network application system
as: g(I, O, s, E) — where O is the plant output in response
to the trigger I, s is the plant state prior to the incidence
of I, and E depicts the environment condition. Since the
true plant model g ∗ (I, O∗ , s∗ , E ∗ ) is not completely known,
C refines its input action in a next iteration based on the
deviation in observed output O∗ from the expected output O
when action I occurs. Upon S reaching a steady-state (over
multiple control iterations) with output P 0 , the output tracking
error |Pref − P 0 | is analyzed to reason about the system
dependability. Our approach is guided by the concepts and
taxonomy of dependable computing presented in [4].
We treat the dependability as a management attribute of
cloud-based adaptive systems. An external management entity
H views the system S as made up of adaptation processes A0p
wrapped around a core system g ∗ (· · ·), i.e., S ≡ A0p ⊗ g ∗ (· · ·)
— where ’⊗’ denotes the composition in an object-oriented
software view. A0p is embodied in a distributed agent-based
software module that forms the building-block to structure
S ([5] provides an architecture for distributed realization of
the adaptation logic of A0p ). S interacts with its (hidden)
external environment through the core elements g ∗ (· · ·): e.g.,
responding to client queries on a web server, and delivering
content over a network transport connection. Here, the meta-
level signal flows between A0p and g ∗ (· · ·) are visible to H.
The layered software structure of S intrinsic to cloud-based
systems: viz., the infrastructure, service-oriented algorithms,
and adaptive application, stacked in that hierarchy and sep-
arated across well-defined interfaces1 , lends itself well for
1 The system layers correspond to IaaS, PaaS, and SaaS, in the cloud
computing arena [6].
a focus on system-level dependability.
The paper is organized as follows. Section II treats system
Cloud dependability as compliance to the stated non-functional at-
tributes (e.g., QoS). Section III describes a sample application:
replicated data service, from a standpoint of the QoS compli-
ance issues in cloud-based realizations. Section IV presents our
model-based approach to assess the dependability of cloud-
based systems, with a focus on the adaptative fault-tolerance
of replicated data servers. Section V discusses the relation-
ship to existing system management frameworks. Section VI
concludes the paper.
II. D EPENDABILITY OF CLOUD - BASED SYSTEMS
Suppose G depicts the goal to be met by a cloud-based
network system S. The goal G may include a prescription of
one or more non-functional attributes associated with the QoS
delivered by S to an application: such as resilience to external
disturbances, stability against resource fluctuations, and re-
sponsiveness to user-triggered requirements. The dependability
of S is prescribed in terms of such non-functional attributes
— and is hence distinct from the correctness goal of S which
is a functional attribute (yielding a YES/NO result). This
section provides a non-functional characterization of system
dependability. Here, the dependability of S is a measure of
how well the adaptation functions programmed into S adjust
to the changing external environment conditions in meeting an
application-level goal G specified for S.
A. Dependability as application-level QoS
We treat the dependability of a cloud-based system S as a
meta-attribute of application-level QoS achievable under the
current operating conditions. An example is how stable is the
content delivery latency achieved on a distribution link in the
face of bursty demands and resource costs. Another example
is how good is a web service availability in the presence of
failures of one or more server replicas. Dependability assess-
ment is anchored on three interwoven properties associated
with the behavior of S:

88
 • Measurability, wherein S can map its output/state vari-
ables onto to high-level metrics over a wide range of input
conditions (for analysis and reasoning);
• Predictability, wherein S can compute its expected output
under various input and environment conditions (with
reasonable accuracy);
• Adaptability, wherein S can adjust the utility extracted
from its output as the environment conditions become
harsher, thereby heavily dwindling its resources.
These properties depict that S is programmable, i.e., an
external management entity H can exercise control over the
behavior of S in a concrete manner. For instance, even if S is
100% fault-tolerant, an inability to reason about this property
in various fault scenarios reduces the dependability of S from
an application standpoint. In the earlier example of replicated
web service, a 95%-availability depicts a verifiable assurance,
over a suitable time-scale, that no more than 5% of the client
queries incur a latency higher than a set limit, say, 12 sec, even
in the presence of server crashes. Thus, a characterization of
S in terms of behavioral properties improves the reasoning
about how dependable S is.
The adaptation processes in system S embody two main
functionalities: i) core components and resources set up over
the cloud infrastructure that collectively export a service to
the application, and ii) a controller C that exercises the core
components/resources to meet the application needs Pref . An
application interacts with the system core through a service-
oriented interface (made up of APIs), with a certain trust on
the service delivery vis-a-vis its quality. A model of S captures
the following elements — c.f. Figure 1:
∗ ∗ ∗ ∗
• Plant g (I, O , E , s ) made up of computational algo-
rithms that map the infrastructure resources and compo-
nents onto a unified service delivered to applications;
• Controller C that decides on an input I to g (.): say,
∗ of an external event e ∈ E ∗ and shields the application from
resource allocation and/or component assignment in the
infrastructure, based on the current service state s ≈ s∗
and environment condition e ∈ E ∗ , to2 make the ob-
served output O reach close to Pref in a steady-state;
• Sensor that maps the service state s onto the observed
service output O to report back to C;
• Actuator to realize a resource/component allocation de-
cision I as domain-specific actions on the infrastructure.
The service interface basically wraps around the core system
g ∗ (· · ·). A function g(· · ·) that approximates g ∗ (· · ·) and the
errors in sensing/actuation processes are factored in a system
model programmed into the controller C.
A dependability metric, denoted as DS (e, G), is a measure
of how good the system S meets its goal G under the
environment condition e ∈ E ∗ . Here, G is prescribed as a
behavior of S that moves the physical plant to a desired state.
Note that a certification authority quantifying DS (e, G) may
be external to the cloud provider and the application-level user.
The external events E ∗ incident on S can emanate from the
application, service-layer algorithms, and cloud infrastructure.
2 Observed service interface state is mapped from service-internal state s∗ .
89
E ∗ includes the errors in QoS specification (e.g., incomplete
specs), algorithm formulation (e.g., unspecified events), and
resources/components (e.g., a node crash) respectively.
We cast the notion of system dependability in a broader
context, as advocated in [4], than the currently prevalent
approaches that rigidly tie to the system-level mechanisms for
recovery from component-level faults occurring in a system.
B. System dependability, trust, and fault-tolerance
[4] characterizes dependability as a form of trust bestowed
on a system S by its users across the service interface to S
— where a user may be a human entity or a physical world
process or a computational sub-system that acts as the client
of S. It depicts how trustable is the service exported by S, as
perceived by those who depend on S (i.e., the users of S).
We project the output tracking error ² = |Pref − P 0 | onto
a dependability measure (note: P 0 is the final converged O∗ ).
The following correspondences can be readily established:
• ² > ²m depicts a service failure (i.e., S is of no value);
• δ < ² ≤ ²m depicts a service degradation (but S may
still be usable, al beit, in a degraded form);
∗
• s depicts the service-external state (s is a computational
representation of s∗ in the model programmed into C).
The environment E ∗ covers anything outside the control
regime of S but which impacts the operations of S: e.g., the
abrupt shut-down of a business unit in an enterprise system
causing delays in the servicing of customer demands (or, even
scaling down system operations).
Under the above framework, the conventional system-level
faults (e.g., node crashes) can be subsumed as part of an
uncontrolled external event space E ∗ . QoS degradations can
also be viewed as faults that constitute a part of E ∗ (e.g.,
packet loss along a network path). How S reacts to the hostility
the effects e allows us to delineate the hitherto qualitative
notions of system robustness and resilience.
Robustness depicts the ability of S to present a service
behavior to applications as if there are no failures or hostile
events. Here, the application does not see the impact of hostile
events, as captured by: P 0 ≈ Pref . This is enabled by strong
service-layer guarantees provided by S using, say, resource
reservations and/or redundant component deployments in the
infrastructure, and the underlying algorithms to correctly man-
age these system-level mechanisms. The signaling interaction
between the application and service layers occurs only when
an application starts and when it terminates.
In contrast, resilience depicts the ability of S to reconfigure
its internal functions and/or parameters, in a way to continue
operations in a degraded mode. Here, the application does
see the impact of hostile events, al beit, indirectly, by scaling
down its operations to a reduced level, which is captured as:
0 ¿ (Pref − P 0 ) ≤ ²m . The service-layer is programmable,
with S employing parameterizable system-level mechanisms
that are invoked by the application at various time-scales
during run-time as part of its reconfiguration strategies. Ac-
cordingly, a richer API is required to support the signaling

interaction needed between the application and service layers:
namely, notifying the quality degradation |Pref − P 0 | to the
application and controlling the parametric and/or algorithm
changes needed in the service-layer3 .
We project the management-oriented view of adaptive ap-
plications through a prism of model-based system assessment,
and characterize therein the notion of system dependability.
C. Tracking error based dependability specification
Consider an application system S ≡ A0p ⊗ g ∗ (I, s∗ , E ∗ , O∗ )
— c.f. Figure 1. A controller C embodied in A0p operates
on the physical plant g ∗ (· · ·) to generate to an output close to
Pref . E ∗ depicts the external environment, modeled as a set of
parameters, incident on g ∗ (· · ·) to disturb the output behavior
of S (|E ∗ | À 1). C is programmed with a plant model
g(I, s, E, O) that is an approximate representation of the true
model g ∗ (· · ·), where E ⊂ E ∗ . C uses the model g(· · ·) to
compute the plant input I. If P 0 is the output sustainable
by A0p (i.e., stable converged value of O∗ ) in response to a
trigger input Pref , then ² = |Pref − P 0 | denotes the output
tracking error of S under the current environment conditions
E ∗ incident on the plant g ∗ (· · ·) — e.g., failures and outages4 .
For adaptive QoS management systems, P 0 depicts the
stable QoS sustainable by S when attempting to achieve a
desired QoS Pref . In an example of content delivery over
a geographically spread-out distribution network, S attempts
to keep the content latency L0 less than a prescribed limit
L, as determined by various factors: such as the capacity
of content proxy nodes, the bandwidth available on the path
leading to clients, and algorithm employed to move content
from a node to its downstream clients. The output behavior
of S may be associated with non-functional attributes, such
as boundedness and jitter. The above attributes capture the
ability of S to provide a sustainable output behavior in the
presence of changes in user-requirements and/or fluctuations
in external environment incident on the physical plant — and
even under modifications to the plant itself (e.g., changes in
proxy node placement in a CDN). Figure 2 empirically shows
how the tracking error of S varies relative to the hostility of
an environment condition e ∈ E ∗ . The system S is deemed
as highly risky to be trusted upon (i.e., dependability ≈ 0)
when |Pref − P 0 | > ²m . The convex behavior depicts the
increased additional effort (or, cost) incurred by A0p to counter
the disruption caused by an environment change from e to
e + δe: say, extra resource allocation in the infrastructure, de-
ployment of robust infrastructure components, and/or service-
layer algorithm reconfigurations.
The goal G to be met by S is specified in a declarative form
based on the TE observed across one or more input-output
3 Mapping the domain-specific signaling onto a generic cloud API (such as
OCCI) requires a study of many use-cases and requirements [7].
4 The tracking error ² depicts the ability of system S to generate a desired
output Pref . Whereas, the parameter: γ = |O − O∗ | depicts the modeling
error, i.e., how accurate the computational model g(I, O, s, E), embodied in
the controller C of S, is in predicting the actual behavior generated by the
system processes: g(I, O∗ , s∗ , E ∗ ).
90
op1, op2, op3: Different operating points of the plant
op3 is more hostile than op2; op2 is more hostile than op1
Î op3 sustains a lower e than op2, and op2 sustains a lower e than op1
[e(3m) < e(2m) < e(1m)]
Pref
output tracking error (TE) of A’p
Hm
during control of g*(..)
useful range of system behavior
e
3}-
{op
-e
2}
p
{o
-e
{o
p 1}
0
0 e(3m) e(2m) e(1m)
Hostility of external environment condition e  E*
Ap’: Control-theoretic adaptation process incident on physical system g*(..)
exercised by application agents
g*(..): Target network system (i.e., plant)
(higher e means more hostile condition)
Fig. 2. Output tracking error relative to external environment conditions
variables Pref and P 0 — denoted as G ≡ Φ({|Pref − P 0 |}).
A sample specs of G may take an arbitrary form, such as:
G ≡ {|Pref (1) − P 0 (1)| ≤ ²0 (1) ∧ |Pref (2) − P 0 (2)| ≤ ²0 (2)}
∨ {|Pref (3) − P 0 (3)| ≤ ²0 (3)}
in a case of three outputs generated by S. The specs Φ(· · ·) is
wired into the control logic executed by A0p , with the signal
flows between A0p and g ∗ (· · ·) validated against Φ(· · ·).
When the output of S deviates from compliance to G, a
tolerance to this deviation is based on the perceived usefulness
of S to the application in those cases. With this idea, we derive
a generalized measure of system dependability5 .
III. C ASE S TUDY:
A DAPTIVE FAULT- TOLERANCE OF WEB SERVICE
The web service is replicated, with the servlet replicas ca-
pable of independently processing the queries and computing
the results returned to the client. The replication purports
to improve the web service availability and performance
experienced by clients, in the presence of server failures
and workload fluctuations. The system employs voting among
server replicas that handle a client query, in order to generate
a timely and accurate response [8]. See Figure 3. Our study
focuses on MBE elements in an autonomic management of
the server replication process.
A. System-level parameters for server replication
The parameter fm , which depicts the maximum number of
replicas that can be assumed to fail, impacts the choice of
N : the total number of replicas. Though the system designer
does not have exact knowledge of the actual number of failed
5 Dependability of S depicts a continuous behavioral property, characteriz-
ing various levels of robustness of S against hostile conditions (instead of a
binary thing [4]). Fault-tolerance is subsumed in this notion of robustness.
 Trusted, to produce a correct result in a case when fa ≥ d N2 e. Thus, the
neutral query failure probability Pqf may be estimated in terms of the
certifier H probability that at least fm + 1 of the servlets get attacked and
reason about
CLIENT APPLICATIONS exhibit faulty behavior (Pqf = 0 when fa ≤ fm ). Depicting
Log of meta-data
for QoS analysis

how good the QoS

obligations are met
actual QoS this condition, we have:

CONTROL
QoS specs
server task observed µ K−f −1 ¶

PLANE

PLANE
DATA
requests (response time; (response time desired: T d ' m
fault-tolerance) fault-tolerance desired)
XK fa − fm − 1
s)

Pqf = rfa × µ ¶
cti on

T,', . . [hAT T (fa ) × ]. (1)

on

service interface
funlecti

K
fa =fm +1 f a
s & col

(provider of
Server replication DATA SERVICE value-added
ter ta
me da

fm, . . (management of service)

control algorithm server The web service integrity is then: [1−Pqf ]. Consider the earlier
ra ta-

redundancy,
security, . .)
example of N = 5, K = 8, fm = 2, r = 0.2, and uniform
(pa me

SM
mo(syst I
int nito em server instantiation on VMs
erf rin (# of VMs running server:
ac g
e) N=3)
Vc, . .
VM running server instance
Idle VM node (available cycles: Vc)
difficult-to-measure parameters:
# of VMs suffering failure: fa = 2;
VM failure probability: q X: attack on VM
Fig. 3. Cloud-based replicated data repository service
replicas fa , it is reasonable to assume that a probability
distribution hAT T (fa ) is known. This statistical information
can guide a more accurate choice of fm such that fa ≤ fm
with a high probability – and hence N . The QoS is prescribed
in terms of non-functional attributes of the replicated web
system: availability, integrity, performance, and resilience.
Consider the return of a query result by the voting module.
Here, an integrity violation may possibly occur if more than
fm server modules are attacked, i.e., fa > fm . Because,
there is a chance that the query result is incorrect because
the result proposed by a faulty servlet may have generated
enough consents for the voting module to declare the faulty
proposal as a correct one. In another scenario, it is possible
that none of the first (2fm + 1) iterations produce a result
enjoying fm + 1 consents. Here, no result will be delivered to
the client — which reduces the service availability.
The case of a hostile event e ≡ [fa > fm ] when the voting
algorithm is itself vulnerable to failure is our interest. It may
manifest as, say, a non-completion of client queries and a
return of incorrect results — which lowers the availability and
integrity respectively of the ’data service’ provided by S.
B. Analytical modeling of server replication
The algorithm designer for the voting-based replicated pro-
cessing method may conservatively set fm = d N2 e − 1 for a
given N — even though the probability distribution hAT T (fa )
may instill confidence in the designer to set fm optimistically
to a lower value. Though the probability of a correct decision
is increased in the conservative case (by virtue of the increased
margin of consent voting), the voting algorithm may still fail
K > N t 2fm+1
(assuming distribution of hAT T (fa ) over the interval [1, 8]. The estimated
leasing of VMs no byzantine query failure probability is: Pqf = 223.3×10−5 . By increasing
from cloud
failures)
(K=6) the degree of replication, say as N = 7, Pqf = 36 × 10−5 .
Equation (1) is based on a stochastic analysis of the voting
infrastructure interface
system in terms of its internal state variables. Under certain
CLOUD assumptions about the statistical independence of failures, the
X INFRASTRUCTURE results allow a quick estimate of the extent of faults occurring
X (manages resources:
VM, storage, . .) in the system and the impact on client-perceived QoS: namely,
query failure probability.
(provider of
raw service) IV. E VALUATING ADAPTATION IN CLOUD - BASED SYSTEMS
We employ model-based analysis of the control-theoretic
adaptation loops in a complex system S [9] to reason about
system dependability in a cloud setting. Recall the system
compositional view: S ≡ A0p ⊗ g ∗ (I, O∗ , s∗ , E ∗ ), where A0p
and g ∗ (I, O∗ , s∗ , E ∗ ) depict the adaptation and core compu-
tational processes respectively. The reasoning functionality is
embodied in the management module H — c.f. Figure 1.
A. Model-based reasoning about system behavior
H embodies the functionality to assess the effectiveness
of A0p by monitoring the tracking error ² = |Pref − P 0 |. It
is possible that A0p embodies autonomic elements to morph
the controller algorithms in C and/or the plant parameters, if
needed, to reduce ² (and hence increase dependability). This
involves predicting the future plant conditions and environ-
ment, and determining the anticipated control actions therein:
such as dynamic switching of the push/pull algorithm and/or
its parameters in a CDN to meet the changes in client de-
mands [11]. The control-theorectic elements needed for a self-
managing behavior of S [12], if any, and the computational
intelligence aspects of A0p therein [13], are6 captured in an
d²
assessment of ² vis-a-vis e (say, de ) — c.f. Figure 2.
In the absence of exact knowledge about E ∗ in the controller
C, we capture the variation of tracking error TE with respect
to the unknown inputs of the plant (E ∗ −E) by treating TE as
a random variable with probability distribution that is derived
from the native distributions of (E ∗ − E). This is combined
with an application-specific characterization of how TE = ²
captures the usefulness of S in various operating conditions.
6 We treat failures as causing system-level output errors that are indistin-
guishable from the errors arising due to system modeling inaccuracies. This
unifying view allows S to employ feedback-based control techniques to deal
with the failure-induced issues as well.

91
 We surmount the issue of combinatorial complexity faced in
analyzing a networked system S by resorting to a model-based tracking error
engineering (MBE) approach that combines a tractable per- Model-based H=[J- J’]

modeling error
formance analysis with a control-theoretic feedback process, controller C

\=[J”- J’]
in order to converge to a (sub-)optimal resource allocation actual traffic /*
QoS specs (based on combined
in the underlying infrastructure ([14] describes the role of J application requests and
actual environment changes)
feedback loops in self-adaptive systems). The MBE approach uted network
takes into account the cross-layer dependencies by featuring compallocation system
r c e schedule tasks
the functional elements of control loops residing in various resou based ondel 2 to resources at
m mo system nodes
layers with a holistic system-level model of S [15]. syste/,J,s,E) 1
8 performance J’
G(

T(V’,E’)
5
(subject to

G(V,E)
3 4 7 observation error

topology of compute nodes & links

B. Control-theoretic feedback process
The determination of optimal resource allocation is a step-
by-step process, with each step analyzing a an incremental
allocation in terms of performance indices (such as service
latency and overhead) using a computational model of S, and
using the analysis to refine the resource allocation in a next
step for improved performance. The goal is to finally make the
actual system output γ 0 as close as possible to the QoS-specs
γ. The computational model of target system employed as a
building-block should be tractable (in a mathematical sense):
say, closed-form mapping of the resource allocation parameters
to achievable performance indices γ 00 — the latter is the
model-estimated output of system. A closed-form modeling of
S may be feasible: say, from a consideration of Poisson-based
workload traffic: say, the client requests on web service. With a
closed-form as elementary block, the properties of Markovian
processes and traffic aggregation can be exploited to yield
Queuing Analysis based formulas for performance indices.
The idea is based on treating an arbitrary workload traffic
process incident on S as a piece-wise linear superposition of
Poissonian traffic processes, with model-based corrections ex-
ercised on the resource allocations in various control-theoretic
steps. Here, a model-based correction involves observing the
actual performance accrued vis-a-vis the Poisson-estimated
performance and using this observed error in refining the
resource allocation decision in a subsequent control step. See
Figure 4. The controller C implements a Partially-observable
Markov Decision Process (PO-MDP) [10] to determine a
resource allocation based on the observed modeling error
(ME): ψ = [γ 0 − γ 00 ]. The resource allocation decision is
exercised on the actual system S, with the resulting (steady-
state) system output as γ 0 . The system output tracking error
(TE) is given as: ² = [γ − γ 0 ]. The ME ψ is combined with
the TE ² by the controller C in its decision-making.
The inter-leaving of a closed-form analysis with the piece-
wise linearity of Poissonian-type of traffic and a control-
theoretic error correction with system observation-based feed-
back over multiple steps allows us to solve the otherwise-
intractable problem of system analysis for arbitrary traffic
patterns (i.e., workload) incident on S. The above interleaving
of control and modeling steps aids the traversal towards a final
solution. This approach yields a reasonable convergence to
a (sub-)optimal resource allocation, with the system stability
Script of flow specs in actual networked system:
True model G*(/*,J*,s*,E*)
of workload traffic
/={Oi, Oj, Ok, . . } E: parameters space
hostile
assumed for E*
environment E*
incident on system
model-based
system description
(using declarative scripts)
model Poisson-model of 1
2 model-estimated
8 performance
workload workload traffic 5 J”
topology of
generator network system
assumed in
3 4 7 model G(. . .) system model does not consider
the link between nodes 1 and 4
Fig. 4. Functional elements for model-based system analysis & control
determined by the controller parameters7 .
C. Cost considerations for application services
In a cloud-based setting, a distributed application involves
the use of infrastructure resources, viz., compute nodes and
node interconnects, and the running of a domain-specific
distributed algorithm software on them. This incurs costs that
arise from two distinct aspects:
1) Leasing of K nodes and multiple node-interconnects
from the infrastructure provider, out of which N nodes
and their interconnects provide the service needed by
applications, where 2 ≤ N < K;
2) Software development effort in a service-layer algorithm
running on the N -out-of-K nodes and their intercon-
nects, with the remaining (K − N ) nodes and intercon-
nects functioning as redundant units.
The component redundancy managed in algorithm layer pur-
ports to minimize the occurrence of service degradations and
failures: say, by tolerating up to fm node crashes — where
1 ≤ fm < N . Referring to the example of data server
replication in Figure 3, the internal parameters of replication
algorithm are: K = 6 and N = 3 as depicted in the cloud
infrastructure, with fm = 1 determined therefrom.
The total leasing cost may increase either linearly or
monotonically concave with K — which means a constant
or a decreasing per-node cost with an increase in K, as
negotiated with the infrastructure provider (e.g., pay-by-use
7 The cumulative model-based correction mechanism in resource allocation
decisions is an instance of machine-learning embodied in the controller.

92
pricing) [6]. The service layer cost is tied to the development external environment E* { [fa,r]
and running of distributed algorithm software, and hence is servlet
domain-specific. In a push/pull algorithm for CDN, N proxy B: system failure

Management module to assess

bandwidth
nodes in a distribution tree requires (N −1) content forwarding s1 s2 .. sN

observe application-level
interconnects and N 0 content storages (where 1 ≤ N 0 ≤ N ) —

system dependability
l
tro

control module
con B

QoS parameters
besides the base software itself to infuse the proxy capability

replication
N , Replicated fm: Max. # of servlet failures

respon
in the N nodes. In a replicated web service layer, the needed proto web Service assumed by voting protocol
software diversity among N servlet replicas (to deter failures) para col r: degree of faulty behavior of

ses
me
plug ter a failed servlet (0 < r d 1)

que uest
incurs O(N 2 ) cost — besides the base software itself running

req
-in voting protocol
fa: Actual # of servlet failures

ry
on the (N + 1) nodes and N interconnects. The service layer [fm,'
]
sub-system
ma K: Total servlet pool size(K t N)
cost often exhibits a monotonic convex behavior with respect res jorit
po y
to N , and it dominates over the infrastructure costs because QoS-ada ns client
ptive e
web-base x
of the specialized software development efforts involved. informati d
The combined cost incurred at the infrastructure and service on system
layers gets assigned to the applications in one form or the
Prototype system results
other. The incurrence of per-node costs may compel a service
(on UNIX-based LAN)

time-to-complete query
provider to limit the number of nodes N participating in an delayed query result (TTC > ') is less useful
algorithm execution, thereby lowering the attainable QoS. The Î reduces “service availability”

TTC (in msec)

resulting loss of some service value to the application may
itself manifest as a cost for the service provider that possibly
out-weighs the cost savings arising from a lower N and/or K.
D. Revisit: Autonomic control of web server replication
The formulation (1) is programmed into the controller C
to exercise a model-based control of the web server system.
Since the environment parameter fa is not known, C may
resort to state-based online parameter identification to infer
fa . The service-external state observed is the number of voting
steps, L, needed to deliver a query result to the client. Here, a
voting step involves the proposed delivery of query result by
a servelet, and the collection of votes from other servelets for
this result. If QK is the probability that it takes K steps to
decide on delivering a query result to the client, then:
2f
Xa +1
K= QK × K. (2)
K=1
A higher fa leads to an increase in K, with a worst-case failure
scenario incurring K = (2fa + 1) voting steps when all the fa
faulty servelets behave erroneously. Under certain assumptions
about the failure behavior, the observed K is mapped onto
an inference about fa . Thereupon, Pqf is estimated to enable
online control of the web server system. C then sets the
control parameters N and fm , as exercised on the replica
infrastructure and voting algorithm layers.
We employ PO-MDP to deal with the uncertainties in
a replicated web service system caused by server failures.
Equations (1) and (2) are used as black-box relations in the
PO-MDP tools when considering various failure probability
distributions: hAT T (fa ). The analysis allows changing the
number of servers in the replica pool (i.e., increase/decrease
N ) dynamically. As part of changes in the service infrastruc-
ture, the voting algorithm may also increase/decrease fm , as
determined by the observed fault serverity parameters [fa00 , r00 ].
Both the replication control parameters [N, fm ] and the
infrastructure network bandwidth B strongly impact the query
1250
'=500 msec
750
o x fm=3, B=300 kb
x
ps
250 fm=2, B=300 kbo o x
o o
x
ps o
3 4 5 6 7 8 9 10
# of servlets participating in web replication (N)
Fig. 5. Impact of [N, fm ] on replication QoS: Experimental study
latency performance — and hence the service availability.
Figure 5 shows the results from our experimental study on
a UNIX-based replicated data service implementation. For a
given N and B, the query latency exhibits a convex increase
with respect to fm . On the other hand, an increase in B causes
a less-than-linear decrease in the latency. These results, which
agree with the analytical model (2), corroborate our empirical
observation on how a system output error ² varies with respect
to the hostility of environment condition e — c.f. Figure 2.
Given the partial observability of fault-severity parameters
[fa , r], an optimal choice of [N, fm ] under cost contraints B
involves the use of heuristics to search the solution space:
such as greedy and genetic search methods. For model-based
service assessment by H, a specific choice of greedy versus
genetic algorithms is however less important — though the
performance results shed insight into these choices.
V. R ELATED WORKS ON ADAPTIVE CLOUD SYSTEMS
Today’s distributed systems embody both adaptation behav-
iors and functional behaviors. The former deals with adjusting
the system operations according to the environment conditions
(e.g., increasing the number of server replicas to improve
web service performance). Whereas, the latter deals with
requirements such as network robustness and security. In this
light, we broadly categorize the existing works as dealing with:
• Systems engineering for the control-theoretic aspects of
adaptation (such as stability and convergence) [2], [16];

93
 • Software engineering for the verification of application
requirements (including para-functional ones) [17], [18].
In contrast, our work focuses on model-based assessment of
the dependability of cloud-based systems (our earlier work is
on an assessment of Internet-type network systems [19]).
[20] provides a QoS-aware network service composition and
adaptation to infuse a core self-management intelligence for
autonomic operations across heterogeneous networks. How-
ever, cloud components are under third-party control with less-
than-100% trust — which poses challenges in guaranteeing
system-level reliability and robustness.
Chisel [21] provides an adaptation framework by separating
the core system functionality from those dealing with non-
functional behaviors. This allows a seamless and transparent
infusion of new behaviors at run-time to deal with unantici-
pated external events e ∈ E ∗ , and a removal of those behaviors
when the conditions causing e cease. The dynamic frameworks
discussed therein are applicable to cloud-based systems, given
the uncontrolled nature of third-party administered cloud re-
sources. Our emphasis however is on the decision-theoretic
mechanisms to control the behavior of cloud-based systems.
[22] advocates dependability differentiation to identify dis-
tinct classes of cloud-based applications (with appropriate
scheduling of the underlying cloud resources). Here, system
assessment is feasible with our probabilistic treatment of a
control error vis-a-vis its impact on system dependability [23].
VI. C ONCLUSIONS
We treat system dependability as application-level QoS at a
meta-level (regardless of the system complexity). Dependabil-
ity assessment of a cloud-based system involves three aspects:
measurability, predictability, and adaptability of system behav-
ior — which are enabled by the programmability of cloud-
based infrastructures and services. Guided by the concepts
provided in [4], the paper studied model-based engineering
techniques to quantify system dependability, and certify cloud-
based systems therein. System-level fault-tolerance mecha-
nisms, hitherto treated separately, are subsumed in the broader
notion of system dependability.
Given a system S realized on a cloud-based infrastructure,
the paper employed model-based engineering techniques to
analyze the output behavior of S relative to what is expected
of S under uncontrolled environment conditions incident on S
(such as attacks and failures in the infrastructure). A manage-
ment entity H external to S incorporates the observation logic
to reason about capability of S by quantification of control
errors and probabilistic QoS assessment. We also undertook a
case study of replicated data servers to assess the quality of
fault-tolerance. The focus of study was on the assessment of
how good the adaptation processes are against the imperfect
knowledge about system operating conditions: such as server
failures and available network bandwidth.
An advantage of our management structure for the assess-
ment of cloud-based systems is the reduction in development
cost of distributed control software for system management
via model reuse and service-level programming.
94
R EFERENCES
[1] Panel Discussion (K. R. Joshi, G. Bunker, F. Jahanian, A. V. Moorsel, J.
Weinman) Dependability in the Cloud: Challenges and Opportunities. In
IEEE/IFIP Intl. Conf. on Dependable Systems and Networks, June 2009.
[2] B. Li, K. Nahrstedt. A Control-based Middleware Framework for
Quality of Service Adaptations. In IEEE Journal on Selected Areas in
Communications, vol.17, no.9 Sept.1999.
[3] C. C. Lamb, P.A. Jamkhedkar, G. L. Heileman, and C. T. Abdallah.
Managed Control of Composite Cloud Systems. In proc. IEEE Intl. Symp.
on Service-oriented System Engineering, Irvine (CA), Dec.2011.
[4] A. Avizienis, J. C. Laprie, B. Randell, C. Landwehr. Basic Concepts and
Taxonomy of Dependable and Secure Computing. In IEEE Transactions
on Dependable and Secure Computing, 1(1), pp.11-33, Jan. 2004.
[5] P. G. Bridges, M. Hiltunen, and R. D. Schlichting. Cholla: A Framework
for Composing and Coordinating Adaptations in Networked Systems. In
IEEE Transactions on Computers, 58(11), pp.1456-1469, Nov. 2009.
[6] Sun Microsystems. Introduction to Cloud Computing architecture. In
White Paper, 1st ed., June 2009.
[7] T. Metsch. Open Cloud Computing Interface — Use Cases and Require-
ments for a Cloud API. Open Grid Forum, Sept. 2009.
[8] K. Ravindran, K. A. Kwiat, A. Sabbir, and B. Cao. Replica Voting: a
Distributed Middleware Service for Real-time Dependable Systems. In
proc. IEEE/ACM COMSWARE’06, New Delhi (India), January 2006.
[9] M. Cordier, P. Dague, M. Dumas, F. Levy, A. Montmain, M. Staroswiecki,
and L. Trave-massuyes. A comparative analysis of AI and control theory
approaches to model-based diagnosis. In proc. 14th European Conference
on Artificial Intelligence, pp.136-140, 2000.
[10] F. S. Hillier and G. J. Lieberman. ”Non-linear Programming” and ”Meta-
heuristics”. Chap. 12, 13, Introduction to Operations Research, McGraw-
Hill publ. (8th ed.), pp.547-616, 2005.
[11] K. Ravindran. Dynamic Protocol-level Adaptations for Performance and
Availability of Distributed Network Services. in Modeling Autonomic
Communication Environments, Multicon Lecture Notes, Oct. 2007.
[12] Y. Diao, J. L. Hellerstein, G. Kaiser, S. Parekh, and D. Phung. Self-
Managing Systems: A Control Theory Foundation. In IBM Research
Report, RC23374 (W0410-080), Oct.2004.
[13] R. C. Eberhart and Y. Shi. Computational Intelligence. In chap.
2, Computational Intelligence: Concepts to Implementations, Morgan
Kaufman Publ, 2007.
[14] Y. Brun, G.D.M. Serugendo, C. Gacek, H. Giese, H. Kienle, M. Litoiu,
H. Muller, M. Pezze, and M. Shaw. Engineering Self-Adaptive Systems
Through Feedback Loops. Book Chapter, Self-Adaptive Systems, LNCS
5525, Springer-Verlag, pp.48-70, 2009.
[15] C. G. Cassandras and S. Lafortune. Introduction to Discrete Event
Systems, Springer-Verlag publ., 2007.
[16] C. Lu, Y. Lu, T. F. Abdelzaher, J. A. Stankovic, S. H.. Son. Feedback
Control Architecture and Design Methodology for Service Delay Guaran-
tees in Web Servers. In IEEE Trans. on Parallel and Distributed Systems,
17(7), Sept. 2006.
[17] I. Schaefer and A. P. Heffter. Slicing for Model Reduction in Adaptive
Embedded Systems Development. In Workshop on Software Engineering
for Adaptive and Self-managing Systems (SEAMS 2008), May 2008.
[18] J. Yi, H. Woo, J. C. Browne, A. K. Mok, F. Xie, E. Atkins, and C. G. Lee.
Incorporating Resource Safety Verification to Executable Model-based
Development for Embedded Systems. In IEEE Real-time and Embedded
Technology and Applications Symp., pp.137-146, 2008.
[19] K. Ravindran. Model-based Engineering for Certification of Complex
Adaptive Network Systems. In proc. IEEE Workshop on Cyber-Physical
Networking Systems, ICDCS’12, Macau (China), June 2012.
[20] J. Xiao and R. Boutaba. QoS-Aware Service Composition and Adapta-
tion in Autonomic Communication. In IEEE Journal on Selected Areas
in Communications, 23(12), Dec. 2005.
[21] J. Keeney and V. Cahill. Chisel: A Policy-Driven, Context-Aware,
Dynamic Adaptation Framework. In proc. IEEE Intl. Workshop on
Policies for Distributed Systems and Networks (POLICY’03), June 2003.
[22] Ameen Chilwan. Dependability Differentiation in Cloud Services.
Master’s Thesis, Dept. of Telematics, Norwegian University of Science
and Technology, Advisor: P.E. Heegaard, July 2011.
[23] K. Ravindran. Managing Robustness of Distributed Applications Under
Uncertainties: An Information Assurance Perspective. in proc. Cyber
Security and Information Intelligence Research Workshop (CSIIRW),
ACM-ICPS, Oak Ridge National Laboratory (TN), April 2010.