Telecommunication Systems

https://doi.org/10.1007/s11235-019-00616-1

Toward an integrated dynamic defense system for strategic detecting

attacks in cloud networks using stochastic game
El Mehdi Kandoussi1 · Mohamed Hanini1 · Iman El Mir2 · Abdelkrim Haqiq1

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Abstract
In a complex network as a cloud computing environment, security is becoming increasingly based on deception techniques.
To date, the static nature of cyber networks offers to adversaries good opportunities to systematically study the network
environment, launch a cyber-attack effortlessly and wide-spread and finally defeat the target system. In order to resolve the
limitations of the traditional security measures as intrusion prevention or detection systems, firewall, access list, etc., which
did not change the attack surface and cannot avoid zero-days attacks, technics that provide dynamic defense, such virtual
machine migration and honeypot should be deployed. Despite this, with a virtual machine migration technique, not all virtual
machines’ migration between servers enhances security considerably. In this paper, we propose an integrated defense system
combining virtual machine migration and honeypot. The effectiveness of the proposed system is discussed in terms of security
policies. In addition, our proposed model determines the potential attack paths quantitatively then classifies them into two
sub-sets: attack paths explored only and attack paths explored and exploited based on the black box intrusion steps. Thus, to
model the interaction attacker–defender, the attack graph combined with the stochastic game theory is used. Finally, we carry
out some numerical results to demonstrate the effectiveness of the proposed security game model.

Keywords Honeypot · VM migration · Cloud computing · Stochastic game · Bayesian game · Attack graph

1 Introduction Therefore, the responses to these requests provide an idea

To evaluate network security, penetration testing helps the
defender to detect network nodes’ contributing to a success-
ful intrusion. Thus, the defender will be aware of the existing
openness in order to provide effective security measures. On
the other hand, the attacker sends a series of requests to col-
lect as much as possible of details concerning the network.
B El Mehdi Kandoussi
kandoussi.elmehdi@gmail.com
Mohamed Hanini
haninimohamed@gmail.com
Iman El Mir
iman.08.elmir@gmail.com
Abdelkrim Haqiq
ahaqiq@gmail.com
1 Computer, Networks, Mobility and Modeling Laboratory,
Faculty of Sciences and Technology, Hassan 1st University,
Settat, Morocco
2 Laboratory of Advanced Science and Technologies
Polydisciplinary Faculty, University Abdelmalek Essaadi,
Larache, Morocco
about the path in the network that should be followed and the
nature of exploits that should be customized or developed to
take access. For this reason, the defender deploys static and
dynamic security mechanisms to decrease attack surface and
make it more dynamic. Indeed, the dynamicity of the attack
surface increases the complexity to attacks.
The vulnerabilities present an entry point for the attack-
ers to probe and discover the flaws existing in network or
system configuration so as to launch a successful attack. To
meet the requirements of a system security and overcome
the challenges, it is mandatory to develop security models in
order to analyze the vulnerability of network configuration,
to evaluate the attack and defense mechanisms and to find
the optimal security solutions.
The quantitative measures take into account the three
external factors that impact system security, namely the
vulnerability lifecycle, attacker behavior, and administrator
behavior. Several studies have been proposed to allow a quan-
titative assessment of safety during the operational life of the
system. The formalism of the attack graph described in [30],
is based on similar concepts: each state of the graph repre-

123

sents the privileges possessed by the attacker as well as its
collected information on network flaws and configuration.
Attack graphs and honeypots are among the most used tools
for improving network security performance.
The attack graph provides a compact representation of
different possible strategies that the attacker may follow
to penetrate in a computer network target. By exploiting
the known vulnerabilities databases, the attack graphs are
automatically generated to recognize the minimal subset
of vulnerabilities to be patched or to involve new security
measures for attack prevention and estimate their risk or to
determine the potential attack paths. Among the simulator
tools, MulVAL is widely used for automatically attack graphs
construction from network information collected by network
security scanners like Nessus, Nmap, and OpenVAS. Using
the common vulnerability scoring system (CVSS), we can
analyze the primary features of vulnerability and estimate a
numerical score describing its severity.
The honeypots are a suitable tool to collect data that will be
analyzed to disclose new attack pattern and help the managers
and Information Technology organizations (IT) to improve
their defensive solutions [16].
During last years, theoretical and mathematical mod-
els to evaluate the performance and efficiency of pro-
posed approaches in network security have gained enor-
mous research attention. Recently, game theory approach
as a promising modeling tool has been explored by many
researchers so as to enhance network security. The adver-
saries and attackers do not cease to develop sophisticated and
invasive techniques. The game theoretic approaches provide
a strong tool to model the adversarial interaction between
defender and attacker. It is an appropriate tool to find the
different flaws of both players and determine their best
solutions. Moreover, to prevent attack propagation, Mov-
ing Target Defense (MTD) has been designed to increase
the complexity and the uncertainty for diverse attacks. It’s
implemented as a game changer for attack surface reduction.
Markov chains modeling and complex network analysis
for detecting attack propagation patterns have been used
in [5] by Ariel bar and all. They have considered attack
datasets using honeypots. The proposed models involve dif-
ferent attack profiles and interaction patterns between the
deployed sensors in the honeypot system. From that, the stud-
ied patterns may be useful for attack prevention and honeypot
deployment.
Several were the works trying to optimize the number
of migrations by taking into consideration multiple security
parameters but few of them were based on the attack graph
to predict attackers’ potential path. In addition, they used
sequential game to model the interaction attacker–defender.
Thus, the steps of compromising a node in the network were
also omitted in such model. Indeed, this kind of works did not
illustrate what happened really in an intrusion since the steps
123
E. M. Kandoussi et al.
followed by the attacker to compromise a node in the net-
work and actions which were taken simultaneously in each
step were not mathematically well developed. In this paper,
the potential attack paths were identified based on security
parameters deployed in the network and also by taking into
consideration the process of intrusion. In our case, since we
work in cloud computing the parameters related to virtual
machine migration and honeypot are used. In general, using
only the vulnerabilities’ data in the NVD database helps the
defender having only static ideas about network’s threat. In
our model these data are combined with the security param-
eters.
Considering a Cloud Computing environment, we employ-
ed the attack graph representation to construct an exhaustive
list of all attack paths. Then, we use a stochastic game model
with two stages, namely a recognition phase and an exploita-
tion phase. In the first stage, the potential attack path is
determined quantitatively by using a normal form game.
More precisely, the loss and cost of attack used in this model
could be determined by using the CVSS scoring system. Con-
cerning the security investment cost, it depends on security
measures deployed in the network topology. In the second
stage, we show if the VM is effectively exploited based on
profile played in the first stage (i.e. actions played by the
defender and the attacker) by using a Bayesian game model.
In this work, the attacker–defender interaction has been
developed in order to mitigate the attacks by determining
firstly the potential attack paths. Indeed, their identification
allows the defender to prioritize the nodes along the potential
paths by enhancing their security. In addition, the model helps
the defenders to avoid ineffective migrations which have
a negative impact on security. Thus, our approach predicts
attackers’ action even if an attack path is already explored
and in which case a migration should be triggered.
In the following, we discuss the different possible strate-
gies for both players and decide the best solution by
implementing the VM migration-based honeypot against a
well-skilled attacker. The defender uses the honeypot to
deceive the attacker and triggers the VM migration as a
deception technique.
In this work our main contributions are: (1) stochastic
game theory is used to model attack–defense interaction
based on attack graphs, (2) quantitative evaluations of the
different scenarios to determine the potential attack paths
and the cases where VM migration and honeypot have an
effective contribution to enhance security are performed (3)
proposed stochastic game model provide priority nodes in
the network which must be updated, and finally (4) sensitiv-
ity analysis for the proposed model through some numerical
results is studied.
The mathematical model developed below could also be
extended as a dynamic stochastic game in a way to eliminate
all inefficient migrations and to monitor the defender’s secu-
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…
rity configuration based on the attacker’s interaction history
with the virtual machines.
The rest of this paper is organized as follow: Sect. 2 sur-
veys some research works related to VM migration, honeypot
and theoretical security approaches, with a special focus on
game theory and Markov chain. The security mechanism
and attack model process are described in Sect. 3. Section 4
presents the stochastic game model for the studied attack–
defense scenarios and its analysis. Section 5 presents the
numerical results and discussion. Finally, Sect. 6 is devoted
to the conclusion and future works.
2 Related work
The researchers in the network security field are currently
investigating powerful security architectures and mathemat-
ical modeling tools to evaluate the trade-off between security
improvement and the accumulated costs due to defense solu-
tions. Among of the proposed solutions that address these
challenges, an increasing number of migration approaches
have been proposed in order to enhance the performance of
cloud platforms, to improve the security and reduce threats.
This section is devoted for discussing the previous
research works on moving target defense mechanisms, hon-
eypot deployment in a complex network, and their imple-
mentations using different mathematical modeling tools.
2.1 Moving target defense and migration
In a cloud computing environment, moving target defense
appears as an efficient platform for migration deployment
by moving the virtual machines from one physical server to
another with the aim to deceive the attacker, and increase the
efforts of the attacker to successfully launch attacks [13].
For example, in [35] the authors proposed a periodic
migration strategy using game theory. Their aim is to reduce
the capability of an adversary to locate the target VMs in
terms of survivability metric.
In [4], the authors addressed the problem of co-residency
side-channel attacks on cloud computing. As a solution, they
suggested VM migration to defeat the attackers and pro-
tect their resources. They proposed a container management
framework named MIGRATE which migrates in a real-time
manner the cloud tenants’ applications held in Linux contain-
ers between hosts. By implementing the proposed framework
on VMware V-Sphere Cloud, they demonstrated the effi-
ciency of the live container migration mechanism to mitigate
side-channel attacks with minimal overhead.
Adili and his colleagues [1] introduced a deception-based
moving target defense mechanism that enables frequently
virtual machines migrations so as to enhance the data security
while maintains the migration overhead limited. The authors
modeled the deception technique as a signaling game and
found the Nash equilibrium as well as to determine the best
possible strategies for both attacker and defender.
Debroy et al. [13] have presented a MTD framework for
VM migration in order to mitigate the impact of Denial of
Service attacks. The basic idea of their work is to optimize the
migration operation and minimize the losses. By implement-
ing MTD using SDN controller and OpenFlow switches they
migrated the selected application from one VM to another
VM. In this way, all users connected to this application will
be redirected to this new selected VM. In [39], the authors
have basically discussed the concepts of MTD techniques and
their characteristics. They have addressed fundamental issues
related to MTD implementation which are timing problem,
and the problem of how to find and determine the new optimal
network reconfiguration. Based on moving target defense
entropy hypothesis, they conclude that the effectiveness of
the MTD system depends heavily on the importance of the
entropy of the system’s configuration.
MTD technique demonstrated its effectiveness against dif-
ferent types of attacks. For example, in order to mitigate the
impact of the DDoS attack, Venkatesan and his colleagues
[33] have applied the MTD on proxy-based architectures.
They suggested shifting and changing frequently the prox-
ies and remapping out the clients to proxies after network
reconfiguration in such a way that the attacker cannot define
precisely his target and pass more time in the recognition
phase. By means of some experiments and simulation results,
they have approved the feasibility of the proposed solution in
terms of reduction of the probability of proxies’ detection by
the attacker during a certain period as well as the reduction
of the attack surface.
For Lei et al. [21], the aim is the selection of the optimal
hopping strategy. For that, they have formulated the MTD
model based on Markov game. They have used the dynamic
game theory to characterize the multi-phases of MTD hop-
ping process. They modeled the transition within network
states during the MTD hopping process using Markov deci-
sion process.
Beckery et al. [7] have used a game-theoretic framework
to analyze the attack and defense strategies for virtual coor-
dinate systems. They have considered two topology data sets
which differ in terms of size and features so as to model the
attack defense interaction and select the best strategy for both
players.
IP address performs a pivotal role to ensure Internet com-
munication. It allows for attacker an access point to gain
access and exploit and analyze the system vulnerabilities
as well as to spread intrusions within the network; also the
attacker can use the port number as a potential way for sys-
tem attacking. Indeed, MTD techniques have two important
mechanisms, namely IP address hopping and port hopping
[10]. By the way; Carroll et al. [11] have discussed the
advantages of MTD techniques implementation. They have
123

explained that changing periodically between the network
addresses and devices by combining the IP addresses and
port numbers can bring a lot of benefits for security adminis-
trators and prevent the intruder to launch a successful attack.
2.2 Honeypots
The honeypots have been designed as security resources in
cybersecurity to strengthen defenders and conversely weaken
attackers. They are helpful to analyze the different defensive
deception techniques for computer systems, and networks
[2].
For example, to counter the DDoS attack and ensure the
security in Smart Grid system, Kun et al. [34] proposed to use
the honeypots as decoy system to collect attack information
and detect the malicious activities that may affect the system.
They introduced a Bayesian honeypot game model and stud-
ied their proposed model to find the optimal strategies for
both attacker and defender. Through some implementations
they justified the efficiency of the honeypot deployment for
security assessment.
In [6], the authors proposed a honeypot based approach for
intrusion detection and prevention systems. By implement-
ing the IDS features, a honeypot server application has been
developed to operate strongly and for real time data analy-
sis. The proposed system provides clear information on the
network system and acts as a supervisor in real-time man-
ner of network traffic on servers. They took benefits from
virtualization technology so as to optimize the cost due to
configuration and maintenance.
This combination of IDPS and honeypot performance
remains suitable for zero-day attack detection and false pos-
itive rate reduction. The authors [8] have integrated the
honeypots with intrusion detection and nested virtualization.
The choice of nested virtualization is justified by its capabil-
ity to deploy honeypot infrastructures on IaaS clouds based
intrusion detection.
Dongxia et al. [14] investigated the IDS based on hon-
eypot so as to deploy IP tracing technique. They designed
the intrusion detection system taking into consideration the
limitations of conventional IDSs on honeypot systems.
In other research work such [9], the authors introduced an
intrusion detection system capable to automatically generate
the attack signatures. They constructed an attack scenarios
database using a honeypot system in order to detect malicious
behaviors.
From [12], the deception technique consists of a taken
set of actions to deceive the attackers thus to incite them
to execute certain actions that help and make the defense
solutions for computer security too efficient. The honeypot
technique was used for attack detection and so as to inspect
several spiteful activities.
123
E. M. Kandoussi et al.
Apart from that, the honeypots have been deployed to pre-
vent the Blackhole attack by controlling and monitoring the
user identity and password seize attack. The Blackhole attack
relies on collect the password and keeps the collected infor-
mation into databases. By gathering passwords, the hacker
can detect the user’s pattern and execute the next attack and
then have more chance to successfully steal the account of
another site. To improve the prevention process for this kind
of attacks, Mun et al. [28] consider the weakness of the pass-
word and the different procedures to rob the information of
password pool by means of the phishing site.
2.3 Mathematical modeling tools
Decision-making process consists of defining the main objec-
tives and constraints so as to take the best decision. This
finality can be held by using various mathematical for-
malisms such as game theory and Markov decision processes
[20].
For example in [12], the authors used Stochastic Petri
Nets to model an integrated defense system which com-
bines the intrusion detection and prevention systems. They
have defined many scenarios considering single defense or
partially integrated defense approaches to discuss the perfor-
mance of the proposed solution. Through some numerical
results, they showed how to integrate the defense techniques
to maximize the security and minimize the attack perfor-
mance.
The authors in [37] proposed a Markov Decision Process
modeling-based approach to evaluate the different secu-
rity policies and choose the optimal one for moving target
defense implementation. The proposed model incorporates
the defense cost and the possible security policies required
by the MTD. They defined optimal defense policies based on
Bellman Optimality Equations and studied the impact of the
policy change by the cost of the selected policy.
Research studies have focused on the different types
of game by considering many approaches, namely net-
work computing resource deployment and edge computing
[24,36]. To enhance the smoothness of computation and
storage utilization, edge computing has been introduced to
handle data at local computing servers instead of in the cloud.
The authors in [31] applied the Stackelberg game to define
the optimal payment and computation offloading strategies
and to maximize the payoffs of cloud service operator and
edge server owners.
Similarly, in Vehicular Ad-hoc Networks (VANETs)
domain, the game theory has been applied to model the
attacker–defender interaction and detect the malicious of the
network nodes [27]. The objective is to discuss the prior
strategies of both players thus decide the best strategy for
attacker and defender vehicles.
Moreover, new security models have been developed for
system vulnerability analysis and evaluation. Attack Graph
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

Application condition
and attack Tree are the well-known security models that are

MTD and honeypot

widely used to graphically represent all attack scenarios.
Arghavani and his colleagues [3] proposed a novel graphical

IP hopping
security model using game theory named Attacker-Manager
Game Tree AMGT. The proposed framework enables visu-

MTD
MTD

MTD
MTD
MTD
alizing and analyzing the interactions between the attacker
and the network security manager to find the optimal security
solutions. By considering all possible attack defense scenar-
ios, they defined the minimax rule so as to aid the security

Universality
manager to find the best solution.
Many MTD-based techniques have been proposed to

Good

Good
Good
Good
Poor
Poor
Poor
address a widespread number of real-world security chal-
lenges. In the majority of research works in the literature,
many simulations have been conducted to study the effec-
tiveness of the MTD techniques but the problem consists

Multi-stated multi-phased

Multi-stated multi-phased
Multi-stated multi-phased
Multi-stated multi-phased

Multi-stated multi-phased
Mono-state multi-phased
that it is difficult to apply the same evaluation mechanism

Mono-state mono-phase
for different MTD techniques. In other words, how we can
find the optimal time in which VMs should be migrated with
minimal cost.

Dynamicness
Motivated by the previous related works, there are insuf-
ficient research works that propose an integrated defense
system which combines different defense techniques to study
and analyze the behaviors for both attacker and defender.
Thus, it is mandatory to evaluate the effectiveness of MTD
techniques based on a suitable mathematical model for a bet-
Information completeness
ter analysis of MTD.

Incomplete information

Incomplete information
Incomplete information

Incomplete information
Complete information
Complete information

Complete information
To meet this requirement, and knowing that the virtual-
ization technology offers many benefits in terms of security
and reliability of cloud computing against potential attacks,
we propose an integrated defense system that combines the
migration process with the honeypot system. More specif-
ically, we deploy the MTD technique such as platform
migration for moving a selected virtual machine from one
physical server to another physical server in order to deceive
the attacker and increase his effort to successfully launch
Incomplete information sequential game

Incomplete information Markov game

attacks. The honeypot system is used to reduce the chances

Complete information dynamic game

Complete information Markov game

of a successful attack on the real network. The performances

of the proposed system are studied using a stochastic game
modeling approach.
Table 1 Comparison of existing MTD game models

In Table 1, a comparison between some game modeling

Game type modeling

approaches in the security field is given. Compared with the

Multi-phased game

Stochastic game

studied works, it is clear that our model has the strength

Markov game

to combine two security measures, namely VM migration

(as a MTD technique) and honeypot technique. Moreover,
it is a multi-stated and multi-phased approach, which gives
it the ability to reproduce the reality of the network and its
dynamics. In other point of view, not all attack surfaces are
modeled as a Markov game. Indeed, the decision made by
the attacker’s depends on the whole part of the attack path
Proposed model

already explored and its experience. Therefore, the use of

stochastic games is more adequate to such scenario since the
entire attacker’s history is considered.
Work

[26]

[38]
[25]
[22]
[23]
[1]

123

Fig. 1 Security mechanism description
3 Security mechanism and attack model
description
In this section, we introduce honeypot and VM migration as
two combined dynamic security measures. Then, we present
steps followed by an attacker to compromise a system and
finally we illustrate an attack graph of a simple system archi-
tecture showing the exhaustive list of attack paths used for
predicting the potential ones.
Figure 1 describes a Cloud data center that contains three
physical servers. On each one of them, multiple virtual
machines are running. We define three security policies for
the deployed servers, namely more secure server, current
server and less secure server. The servers’ classification is
based on their security scores. More precisely, a server is
qualified secure than another if its vulnerabilities’ complex-
ity is higher, these vulnerabilities are that of the hosted VMs
in the servers. We assume that the VMs are homogeneous
and have the same level of security according to the physical
server on it they are running. In other terms, the security of
the whole components of the server characterizes its security
rank. More precisely, VMs are grouped according to their
common vulnerabilities score in the same sever. Indeed, the
security of a VM affects the security of the others in the same
server, and if the VMs are hosted randomly in servers of a
datacenter, the attacker uses the less secure VM in this envi-
ronment and its relation to a common platform in order to
attack more secured VMs. This problem is known as the neg-
ative externality and was developed in [17]. Concerning the
hypervisors’ security, it is very crucial, since if it is compro-
mised, the likelihood to control all the VMs will be very high
as mentioned in [32]. For example in Fig. 2, it was supposed
that the servers are ordered increasingly by the complexity
to be compromised from “1” to “n” and the attacker has the
123
E. M. Kandoussi et al.
intention to compromise a secured VM located in the server
“n”. To attain his target, he firstly compromises an unsecured
VM already migrated to server “n”, then attacks the hyper-
visor which was less difficult to gain privileges compared to
his first goal and finally exploits the targeted VM.
Concerning the servers and VMs with grey color in Fig. 1
(honeypot part), they are fictitious assets added in the system
to deceive the attacker.
We deploy the honeypots as a deception technique so as
to deceive the attacker, deflect his attempt for compromising
the VM and maximize his attacking cost. Then, we analyze,
in terms of cost, in which case, the attacker exploited the
virtual machine.
In general, the attacks follow a common strategy that starts
from planning to execution. The structure includes two prin-
cipal phases, namely recognition and exploitation phases as
depicted in Fig. 3.
We adopt the recognition phase as a preliminary step
to gain access to the target system and exploit it. During
the passive recognition phase, the attacker collects massive
information about targeted computers and networks with-
out directly interacting with the system. More precisely, the
attacker minimizes any interaction with the target network
which generates an alert in logs.
The active recognition involves interacting with the tar-
get directly. Indeed, the attacker seeks to win back victim
by implementing different tools such as ping to get informa-
tion of IP address, netcat tool to determine the open ports for
listening and traceroute to discover the packet route’s infor-
mation.
At the exploitation phase, attackers apply all their knowl-
edge and experience to gain access or cause another adverse
impact by using a previously discovered vulnerability.
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…
Fig. 2 Negative externality problem description
Fig. 3 Black box intrusion steps
The post exploitation phase provides the possibility to
gather the information about the attacker, look for interesting
files, attempt to elevate our privileges where it is necessary,
and facilitate the upcoming connections to the compromised
VM. This phase includes the analysis of the interaction
between the attackers and the compromised machines.
The defender handles the dynamic migration strategies to
maximize the rewards and reduce the loss costs. Furthermore,
we use Mulval tool to generate the attack graph which is
illustrated in Fig. 4 and Table 2. This latter generates an
exhaustive list of attack paths that the attacker may follow
Fig. 4 Attack graph generated by Mulval
to compromise a targeted VM and gain a successful network
penetration.
In Fig. 4, an oriented graph composed by three types of
nodes is illustrated. The start nodes {12, 18, 18, 19, 20} rep-
resent the resources provided by the network to clients. From
these entry points, a sequence of events are triggered until the
arrival node (consequence 1 or goal). This latter represents
the execution code by the attacker with root privileges, as
depicted in Table 2. Each event is composed from a require-
ment, a rule and a consequence. Hence, the consequence of
a certain event could be a requirement to apply a certain
rule to obtain another consequence until attaining the goal.
For example, with the code execution in the file server as
a root (requirement 8) and the existence of an account with
principal privileges in the file server (requirement 34), then
by sniffing the password (rule 33) the attacker compromises
the system administrator and increases his privileges (conse-
123
 E. M. Kandoussi et al.

Table 2 Attack graph

Node type Label Semantics
description
Requirement 7 canAccessFile(fileServer,root,write,‘/export’)
12 hacl(webServer,fileserver,rpc,100005)
17 hacl(internet,webServer,tcp,80)
18 attackerLocated(internet)
19 networkServiceInfo(webServer,httpd,tcp,80,apache)
20 vulExists(webServer,‘CVE-2008-0074’,httpd,remoteExploit, privEscalation)
21 networkServiceInfo(fileserver,mountcl,rpc,100005,root)
22 vulExists(fileserver,vulID,mountd,remoteExploit,privEscalation)
24 hacl(webServer,fileServer,nfsProtocol,nfsPort)
25 nfsExportInfo(fileServer,‘/export’,write,webServer)
31 hasAccount(sysAdmin,webServer,root)
34 hasAccount(sysAdmin,fileServer,root)
36 nfsMounted(VirtualMachine,‘/usr/local/share’,fileServer,‘/export’, read)
Rule 2 Trojan horse installation
4 NFS semantics
6 execCode implies file access
9 Remote exploit of a server program
11 Multi-hop access
14 Remote exploit of a server program
16 Direct network access
23 NFS shell
26 NFS shell
28 When a principal is compromised any machine he has an account on will also be
compromised
30 Access a host through executing code on the machine
33 Password sniffing
35 Password sniffing
Consequence 1 execCode(virtualMachine,root)
3 accessFile(virtualMachine,write,‘/usr/local/share’)
5 accessFile(fileServer,write,‘/export’)
8 execCode(fileServer,root)
10 netAccess(fileServer,rpc,100005)
13 execCode(webServer,apache)
15 netAccess(webServer,tcp,80)
27 execCode(webServer,root)
29 canAccessHost(webServer)
32 principalCompromised(sysAdmin)

quence 32). Thus, by following the execution of rules of an
attack path “i”, the attacker eventually gets a total control of
the virtual machine. For detailed specifications about Mul-
VAL the authors in [29] provide a clear explanation to each
Datalog clause in Table 2.
4 Model description
In order to predict attacker’s action, provide cases of an
effective migration and evaluate quantitatively the security
performance related to a complex network deploying VM
migration and honeypot as dynamic security mechanisms,
an approach based on game theory is developed. We opt
for a model of stochastic game where the defender and
the attacker are rational. Indeed, the attack–defense con-
frontation by moving target defense and honeypot requires
a multi-phased and a multi-stated game theoretic model. In
reality, the attacker follows steps to compromise the network
which in turn goes through a certain states. Mathematically,
these concepts are formulated by using a multi-stated model
as Markov chain model and stochastic game. In addition, the

123
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…
Fig. 5 Security stochastic game
model
double character of the honeypot imposes the use of games
with incomplete information. Finally, no prior agreement is
done between the defender and the attacker before or dur-
ing the game. Then, the non-cooperative aspect has to be
present in the model. Therefore, the adequate model includ-
ing multi-phased, multi-stated, incomplete information and
the players’ competitiveness is the Bayesian stochastic game
model.
Since our model is based on black box intrusion steps, the
coalition between attackers and defenders is not considered.
Moreover, the cooperation between several types of attack-
ers is not assumed. These modes of intrusion are developed
precisely in [19]. Practically, the use of proxies as a tool
for bypassing traceability systems makes the assumption of
attackers’ cooperation very hard to proof. In our proposed
approach, to model malicious activities, a malicious, experi-
enced and generic attacker’s character is developed.
On the other hand and conversely to the attackers, it seems
natural to consider the cooperation between several secu-
rity measures. The characteristics of the proposed model
being multi-stated and multi-phased with incomplete infor-
mation can be seen as cooperation modeling between many
defenders using different security techniques. Concerning the
technological aspects discussed in the previous section, they
are taken into consideration.
The stochastic game model proposed in this paper is
composed of two phases. The first one corresponds to the
recognition phase while the second one is the exploitation
phase. In this model, the attacker is supposed to start from
the first phase then the decision made in this step influences
his action in the second step. More precisely, an attacker
who decides to compromise a VM by exploring the attack
path “i”, cannot succeed the VM’s exploitation by chang-
ing the ith attack path to another one. Only explored attack
paths in the first phase are used in the second phase. In the
other side, the defender’s decision in the recognition phase
in term of migrating the VM or not influences also his next
action. The Fig. 5 illustrates which games will be played by
the defender and the attacker. For example, if the attacker
chooses to explore A Pi and A P j with probability pi and p j
such that pi + p j = 1 and the defender migrates the VM
with probability q (In this example, the strategy played by
each player is not supposed to be the best response) then the
sub-games played by these latter are: gm i , g j , g i and g j with
m m̄ m̄
probabilities qpi , qp j , (1 − q) pi and (1 − q) pi .
In the following, the components of each game in the Fig. 5
are detailed.
4.1 Stochastic game formulation
The first stage of the stochastic game model is a normal form
game: G, while the second stage is composed of a set of
the
 i Bayesian games (game  with incomplete information):
i : i ∈ {1, . . . , n} with n the total of potential attack
gm , gm
paths. All games which will be presented in this paper are sup-
posed to be simultaneous non-cooperative game and no prior
agreement is done between the attacker and the defender. The
formal definition of the stochastic game, normal form game
and the Bayesian game are given below:
Definition 1 (Stochastic game)  A stochastic game is a tuple
N , S, (Ai , Ai , u i )i∈N , T where:
• N : a set of players (|N | = n),
• S: a set of states,
• For every player i ∈ N , Ai is a set of ith player’s actions,
and Ai : S ⇒ Ai is a set-valued function that assigns to
player i in each state s ∈ S his available actions: Ai (s),
• For every player i ∈ N , u i : S A → R is the
stage payoff function for player i, where: S A =
(s, a) : s ∈ S, a = (ai )i∈N , ai ∈ Ai (s) ∀i ∈ N ,
123
 E. M. Kandoussi et al.

• T : S A → Δ(S): is a transition function, where Δ(S) is Table 4 System parameters

the set of probability distributions over S. Notations Meanings

Li Total lost associated to the ith attack path

Definition 2 (Normal form game) A normal form game is:
αi The percentage of L i associated to the
recognition phase
• N : a set of players (|N | = n), Ci Total cost associated to the ith attack path
• Ai : a set of actions for each player i ∈ N ,
n βi The percentage of Ci associated to the
• u: an application from A = i=1 Ai to Rn ·u i (a1 , . . . , ai , recognition phase
. . . , an ) is the ith player’s payoff when the profile CM Cost of the migration
(a1 , . . . , ai , . . . , an ) is played. h Cost of the honeypot’s deployment
us Probability to identify the VM after a first
Remark A normal form game is a particular case of the secure migration
stochastic game. us Probability to identify the VM after an
unsecure migration
vs Probability to identify the VM after a second
Definition 3 (Bayesian game) A Bayesian game is a tuple secure migration during the exploitation
N , ( Ai , Θi , pi , u i )i∈N where:
phase
θ Probability to detect the real network
• N : a set of players (|N | = n),
For each player i ∈ N , we have: 
n
• Ai : a set of actions, we note A = i=1 Ai ,  i
n
• Θi : a set of types, we note Θ = i=1 Θi , – ADef (G) = {Ms , M, Ms } and ADef gm = ADef
i = {M, M , M },
• pi : a probability function such as pi : Θi → Δ (Θ−i ), gm s n  i
• u i : a payoff function such as: u i : A × Θ → R. – A At (G)
 i= {A Pi /i ∈ {1, . . . , n}} and A At gm =
A At gm = {A, A}
Before we develop the stochastic game in our security con- • Concerning the utility of each player, it will be repre-
text, the notations in Table 3 are used in the rest of this paper. sented as a matrix in each state since we have only two
The components of our stochastic game are defined as players.
follow:
– The stochastic game starts with the state G which is
a normal form game. In our case, only one matrix is
• The set of players: N =
 {At,i Defi },  used in this stage as in Table 5.
• The set of states: S = G, gm , gm /i ∈ {1,
 . . . , n} , – In the exploitation phase, the defender has two types,
• ADef = {Ms , M, Ms , Mn } and A At = A, A, A Pi /i ∈
either he behaves as a honeypot or the attacker
{1, . . . , n}}
deals with the real network. Therefore, two matri-
ces are used per Bayesian sub-game. In addition, the
Table 3 Notations and their meanings exploitation phase depends on the migration’s occur-
rence in the previous stage. In this case, we have two
Notations Meanings i andg i . The utility
classes of Bayesian sub-game gm m
VM Virtual machine matrices of these games are illustrated in Table 6.
A Pi ith attack path
At Attacker
First the stochastic game’s security parameters and their
Def Defender
meaning are represented in the Table 4.
Ms Secure migration: migration to a secure server
In the rest of this paper we adopt the following hypothesis
M No migration
on the system parameters:
Ms Unsecure migration: migration to a less secure server
Mn Fictitious migration
A Attack • L i , Ci , h > 0 and C M > 0
A No attack • 0 ≤ θ ≤ 1, 0 ≤ u s < u s ≤ 1, 0 ≤ vs ≤ 1, 0 ≤ αi ≤ 1
Hp Honeypot and 0 ≤ βi ≤ 1
Rn Real network • Ci < L i , αi L i < (1 − αi ) L i and L 1 < · · · < L i−1 <
Ml Malicious L i < L i+1 < · · · < L n

123
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

The hypothesis above are explained as follows. A migra- Table 5 Initial game: G (recognition phase)
tion is qualified to be secure when a virtual machine migrates ... A Pi ...
to a server more secure than the current server. In this case,
we note the probability to mitigate the attack by u s . More Ms −u s αi L i − C M u s αi L i − βi Ci
precisely, it is the probability to identify again the virtual M −αi L i αi L i − βi Ci
machine after a secure migration. Indeed, the security of Ms −u s αi L i − CM u s αi L i − βi Ci
a destination server depends on how much the hosted vir-
tual machines are secure. In the virtual complex network
as a cloud computing environment, the virtual machines are
phase is detailed quantitatively. The goal in the illustrated
grouped based on their level of security. For example, an
scenario above is to compromise the VM by following an
unsecure virtual machine could not be hosted with secure
attack path “i”. A loss L i is attributed to this latter. A part of
virtual machines since it will be an entry point to compro-
it is due to the recognition phase αi L i and the other part to the
mise other virtual machines (it maximizes externalities). The
exploitation phase (1 − αi ) L i . Hence, without the deploy-
same explanation goes with an unsecure migration with the
ment of such security measures the VM suffers a loss equal to
notation u s . Then, we have 0 ≤ u s < u s ≤ 1. In addition,
the parameter αi (respectively βi ) is the percentage of lost (αi L i ; (1 − αi ) L i ). During the recognition phase, if the VM
migrates, the loss will be reduced to u s αi L i along the attack
(respectively cost) associated with the recognition phase of
path, with u s the probability to be identified again. Next, the
the ith attack path. Therefore, the quantity (1 − αi ) (respec-
attacker tries to exploit the VM. In this case another migration
tively (1 − βi )) is the percentage of lost (respectively cost)
occurred. In addition, the attacker is not aware of the exis-
associated with the exploitation phase. Thus, in this model,
tence of the honeypot. As a result, the attack path leading to
we suppose that the exploitation phase’s lost is higher than
the VM suffers a loss reduced to θ vs u s (1 − αi ) L i .
the recognition phase’s lost and each attack path has a cost in
In Table 5, the recognition phase is modeled as a normal
order to be exploited. Indeed, during the exploitation phase,
form game represented as a matrix since only two players are
the attacker uses scan results gained in recognition phase in
considered. In this stage, the attacker acts only on the part of
order to compromise the network. During this operation, the
attack path loss related to the recognition phase. For example,
normal network behavior is modified. Therefore, different
if the defender chooses to migrate the VM in a secure server
types of damages related to the network affect its availabil-
and the attacker follows the attack path “i”, then the utility
ity, confidentiality or integrity. Contrary to the exploitation
of the defender is −u s αi L i − C M and that of the attacker is
phase, the recognition phase only acts on confidentiality and
u s αi L i − βi Ci . The defender’s utility interpreted as a loss
integrity of the network and its users. Concerning the last
is composed of two parts. The first part related to the attack
inequality, it helps only for game resolution and did not have
path “i” loss during the recognition phase and the second one
any restriction on the model.
to the cost of the migration. On the other hand, the negative
In Fig. 6, the evolution of attack path loss when deploying
quantity −u s αi L i is a positive gain for the attacker which
honeypot and migration as a deception technique to miti-
is equal to u s αi L i . In addition, the cost of the recognition
gate the attack during the recognition phase and exploitation
phase is also considered and it’s equal to −βi Ci . Therefore,

Fig. 6 Steps to reduce attack

path loss

123
 E. M. Kandoussi et al.

the profile played is (Ms , A Pi ) and in the exploitation phase and the attacker’s utility is u s (1 − αi ) L i − (1 − βi ) Ci .
the players will be in the state gm i . Indeed, the defender suffers a loss equal to u s (1 − αi ) L i
associated to VM’s exploitation phase and added negatively
Remark In Fig. 5, we have only represented sub-games when to the honeypot deployment cost − h. On the other hand,
a secure migration or any migration is done in the recognition the negative loss of the defender is counted as positive gain
phase and no sub-game related to non-secure migration is of the attacker. As a result, this latter has a positive utility
represented. We will prove mathematically in the first part of u s (1 − αi ) L i to which the attack’s cost during this phase is
“Appendix B” that unsecure migration will never be a best subtracted (1 − βi ) Ci . Practically, the attacker repeats many
choice for the defender to mitigate the attack. times the entire process of intrusions due to the dynamicity
of the attack surface.
• The final game is composed by 2 × n Bayesian sub- Table 7 presents notations used to express both defender
games. Before we represent them as a matrix, the set of and attacker utility in recognition and exploitation phase.
player’s types: Θ and the probability function: pi should If the attacker follows each attack path with probability pi
be defined (as in definition 3). and the defender migrates the VM with probability q, then
the defender’s total utility and attacker’s total utility are:
– Players’ types in each sub-game: ΘDef = {H p, Rn}

and Θ At = {Ml} G + q n
UDef = UDef m,i n m,i
i=1 pi UDef + (1 − q) i=1 pi UDef
– Actions associated to each players’ type:
Hp  i  Hp  i 
n m,i n m,i
U At = U At
G +q
i=1 pi U At + (1 − q) i=1 pi U At
• ADef gm = ADef gm = {Mn } and
 i  i
ADef gm = ADef gm = {Ms , M}.
Rn Rn n
 i  i with i=1 pi = 1.
At gm = A At gm = {A, A}.
• A Ml Ml

– Probability distribution over players’ types: For example, if the defender migrates the VM and the attacker
Pr (Def = Rn/At = Ml) = θ ⇒ Pr (Def = explores only the ith attack path, then the utility of each
H p/At = Ml) = 1 − θ and Pr (At = Ml/Def = player in the recognition phase is:
H p) = Pr (At = Ml/Def = Rn) = 1

G = −u α L − CM
UDef
• Since the security game model proposed in this paper is s i i
.
G =u α L −β C
U At
composed from only two stages, the transition function s i i i i
will be defined
 in general as follows:T : G A → Δ(S)
with S = G, gm i , g i /i ∈ {1, . . . , n} and G the initial i . In addition, if the
m Therefore, the sub-game played is: gm
game attacker detects the real network and compromises the VM
In addition, if the attacker invests in each attack path and the defender do not migrate the VM during the exploita-
with probability pi and the defender migrates the tion phase, then the utility of each player during this stage
VM with probability q, then the profile played will is:
be: s = (q × Ms , (1 − q) × M), ( p1 × A P1 , . . . , pi
n m,i
×A Pi , . . . , pn × A Pn )) with i=1 pi = 1. If we note UDef = − u s (1 − αi ) L i − h
.
T
⎧ (Gs) = d, then:  i
m,i
U At = u s (1 − αi ) L i − (1 − βi ) Ci
⎨ ∀i ∈ {1, . . . , n}d gm  = qpi
∀i ∈ {1, . . . , n}d gm i = (1 − q) p (see Fig. 5).
i Thus, players’ total utilities are:
⎩
d(G) = 0
G + U m,i
UDef = UDef Def
G + U m,i
.
In Table 6, two sub-games gm i and g i are represented dur-
m
U At = U At At
i
ing the exploitation phase. gm is the sub-game played if the
profile played in the initial game is (Ms , APi ), while i
 gm is 4.2 Game resolution
the sub-game played if the profile played is M, A Pi . In the
case where the players randomize over their actions in the ini- Resolving a normal form game (respectively Bayesian game)
tial game, each sub-game is played with a certain probability consists of finding a Nash equilibrium (respectively Bayesian
as described just above in the mathematical expression. For equilibrium) (or a set of Nash equilibrium or Bayesian equi-
example, we consider that the profile played in the recog- librium). In the following, these notations will be used:
nition phase is (Ms , A Pi ) and the attacker is aware that a
honeypot is deployed. In addition, the defender chooses to • The notation a = (ai , a−i ) is used instead of a =
not migrate the VM while the attacker chooses to exploit this (a1 , . . . , an ) and denotes the action profile played by the
latter. Therefore, the defender’s utility is −u s (1 − αi ) L i −h n players and a−i is the profile played by the (n − 1)

123
 Table 6 Final game (exploitation phase)
i sub-game
gm i sub-game
gm
Hp A A Hp A A

Mn −h −h Mn −h −h
− (1 − βi ) Ci 0 − (1 − βi ) Ci 0

Rn A A Rn A A

Ms −u s vs (1 − αi ) L i − h − C M u s vs (1 − αi ) L i − (1 − βi ) Ci −h − C M 0 Ms −u s (1 − αi ) L i − h − C M u s (1 − αi ) L i − (1 − βi ) Ci −h − C M 0
M −u s (1 − αi ) L i − hu s (1 − αi ) L i − (1 − βi ) Ci −h0 M − (1 − αi ) L i − h (1 − αi ) L i − (1 − βi ) Ci −h0
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

123
 E. M. Kandoussi et al.

Table 7 Defender and attacker

Intrusion phases States Defender’s utilities Attacker’s utilities
utilities notations
Recogition phase Initial game: G G
UDef G
U At
i m,i m,i
Exploitation phase Final sub-game: gm UDef U At
i m,i m,i
Final sub-game: gm UDef U At
Total player’s utility UDef U At

players except the ith player. The same explanation goes u s < u s ≤ 1, the strategy Ms is strictly dominated by Ms
with the vector θ = (θi , θ−i ). (proof in the first part of “B”). Therefore, the strategy space
• Δ(E) is the set of probability distributions over a set E. of the defender will be reduced to A
Def (G) = {M, Ms }. In
• If γ ∈ Δ(E) he set defined as follows Supp(γ ) = {e ∈ addition, we consider two subsets: E 1 and E 2 of A At (G)
E/γ (e) = 0} will be used. defined as follows:
 
First, we will start by introduction some notions related E 1 = A Pi / ∃ j > i, β j C j − βi Ci ≤ 0 and
to the normal form game resolution than those of Bayesian  
β j C j − βi Ci
game. E 2 = A P j / ∃i < j, ≥1
α j L j − αi L i
Definition 4 (Best response, strictly dominated strategy and
Nash equilibrium) For a strategy A Pi ∈ A At (G) such that A Pi ∈ E 1 ∪ E 2 ,
then: ∃i
= i: A Pi is strictly dominated by A Pi
and the
1. A strategy ai∗ is denoted as a best response of the player strategy space of the attacker will be reduced to A
At (G) =
i if:   A At (G)\ (E 1 ∪ E 2 ).  
∀ai ∈ Ai u i ai∗ , a−i ≥ u i (ai , a−i ) We note m = Car d A
At (G) and we conserve the
2. A strategy ai ∈ Ai of the player i is strictly dominated same order of the attack paths as mentioned on the system
by ai
∈ Ai if:   parameters hypothesis after eliminating all strictly dominated
∀a−i ∈ A−i u i (ai , a−i ) <  u i ai
, a−i strategy.
3. The profile strategy a ∗ = a1∗ , . . . , an∗ is a Nash equi-  
librium if: Theorem 1 If λ ∈ Δ A
At (G) and λ is the probability distri-
   
∀i ∈ I , ai ∈ Ai : u i ai∗ , a−i
∗ ∗
≥ u i ai , a−i bution over attack path strategies in Nash equilibrium then:

   
Definition 5 (Pure strategy, mixed strategy and Bayesian ∃i 0 ∈ {1, . . . , m}Supp(λ) ⊂ A Pi0 , {A Pm } , A Pi0 , A Pm .
equilibrium)
  The proof of this theorem is detailed in “Appendix A”.
1. Given a Bayesian game N , ( Ai , Θi , pi , u i )i∈N a pure
i and g i )
The final games (i.e. the Bayesian sub-games gm
strategy for player i is a function which maps player i’s m
type into its action set: ai : Θi → Ai . are resolved in the second part of “Appendix B”.
2. A mixed strategy for player i is: μi : Θi → Δ (Ai ) :
θi → μi (·|θi ).
3. A Bayesian equilibrium is a mixed strategy profile 5 Numerical results
(μi )i∈N such that for every player i ∈ N and every
type θi ∈ Θi , we have: In numerical results section and in the rest of this paper,
 instead of using L = (L i )i∈{1,...,n} and α = (αi )i∈{1,...,n} to
μi (·|θi ) ∈ argmax pi (θ−i |θi ) illustrate the loss repartition between
γ ∈Δ(Ai ) θ ∈Θ  recognition
 phase and
−i −i exploitation phase, we use L I = L iI i∈{1,...,n} and L F =
⎧ ⎫  F
⎨   ⎬ L i i∈{1,...,n} with α = (1)i∈{1,...,n} . The two representations
μ j a j |θ j γ (ai ) u i (a, θ ) . L li
⎩ ⎭ are equivalents since αi = . The same explanation
a∈A j∈N \{i} L i +L iF
I

goes with the two vectors of cost C = (Ci )i∈{1,...,n} and

4.2.1 Initial and final game resolution β = (βi )i∈(1,...,n) .
To validate our model, we suppose that we have only three
The resolution of the initial game starts by eliminating all attack paths in our attack graph. In reality it is composed
strictly dominated strategies. Based on the assumption 0 ≤ from dozens of attack paths. The characteristics of these three

123
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

Table 8 Numerical values’ parameters
Parameters Numerical values
CM 3 More precisely, it gives the potential AP in a cloud environ-
h 2
LI [2.5, 11, 15]
LF [60, 200, 300]
CI [0.5, 3.5, 6.5]
CF [20, 5, 48]
vs 0.5
attack paths are summarized in the Table 8 and are taken in
the same order as in [15] and [18].
In Fig. 7, probability distribution over attack paths with
respect to the probability of identifying the VM after migra-
tion: u s in case of a Nash equilibrium in the initial game
is illustrated in a, b and c. According to Fig. 7a, c the best
response for the attacker is to randomize over A P1 and A P3 .
Indeed, it’s a mixed Nash equilibrium. Practically the A P1
is the attack path that should be followed since it has more
likelihood based on Fig. 7a, c. In another point of view, for
the value of u s ∈ [0, 0.38], A P1 is the best response since
u s has low values and the attacker cannot invest in AP with
the higher cost even if it has a maximum loss. In addition,
this is intuitive because the VM cannot be exploited only if
it is easily identifiable. For values of u s ∈ [0.38, 0.75], we
have a pure Nash equilibrium and A P2 is the best response to
the attacker. For higher value of u s the attacker uses the AP
with the maximum loss (i.e. maximum information gathered)
since it is easily identifiable after migration. For conclusion,
the parameter u s helps the defender predicts attacker’s action.
ment where migration is deployed as a security measure. The
probability distribution over migrating the VM or not in case
of a Nash equilibrium is illustrated in Fig. 7d, e. For values
of u s ∈ [0, 0.8] the migration must be carried out in general.
For a higher value of u s the migration has any effect since the
migration will not mitigate the attack and in addition, it has
a cost that will decrease the utility of the defender. Indeed,
u s gives an idea about the class of servers of destination that
helps to avoid the attack considerably and reduce the cost of
migration.
In Fig. 8, three sub-games are illustrated in case of a migra-
tion was occurred in the recognition phase. Concerning the
attacker’s probability distributions over attacking or not, they
are represented in the first three plots. As depicted in these
three representations, the probability to attack increases with
the increase of the two parameters u s and θs . Indeed, the
attacker will exploit the VM if this latter is identifiable on
the real network. Conversely, for lower values of u s and θs
the attacker will not compromise the VM since he is uncertain
about dealing with a VM in the real network or a honeypot. In
addition, we see that the surfaces of exploiting APs (with red
color) are different from each other, this is due to the cost and
the loss of the AP. Concerning the last three plots, they illus-
trate the probability distribution of migrating a VM during
the exploitation phase. As depicted in these representations,

Fig. 7 Defender’s and attacker’s (a) Probability to follow AP1 (d) Probability to migrate securely
probability distribution over
1 1
actions in the initial game
Pr(AP )
1

Pr(M)

0.5 0.5

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
u u
s s

(b) Probability to follow AP2 (e) Probability to not migrate

1 1
Pr(AP )

Pr(NM)
2

0.5 0.5

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
us us

(c) Probability to follow AP3

1
Pr(AP3)

0.5

0
0 0.2 0.4 0.6 0.8 1
us

123

Fig. 8 Defender’s and attacker’s probability distribution over actions in sub-games where a migration was occurred
the probability to migrate the VM increases with the increase
of u s and θs . Indeed, for higher values of u s and θs a migration
must be carried out since on the other side the attacker will
compromise the VM. Consequently, the combination of these
two parameters helps to avoid some migrations and gives
an idea about the next attacker’s action in an exploitation
phase.
In Fig. 9, the first three plots represent the utility of the
attacker. As depicted, the attacker’s utility is an increasing
function with u s and θs and it takes different maximum
values. Indeed, this is due to the cost and loss associated
with each AP already explored in the recognition phase. In
the other side, the defender’s utility in each sub-game is a
decreasing function with u s and θs . As shown in Fig. 9, the
migration process and honeypot should take values of u s
and θs on which the defender’s utility corresponds to the red
surface. In general, this helps the defender to enhance the
security by choosing the appropriate honeypot and the class
of destination servers.
Concerning the Figs. 10 and 11, the same explanation used
for Figs. 8 and 9 goes with. But in this case where migra-
tion was not occurred in the recognition phase, the defender’s
utility is below the defender’s utility in case of a migration.
This is because the uncertainty of the attacker that was prop-
agated to the exploitation phase and as a result, we can see
that his utility in case of no migration is above his utility in
case of migration in the recognition phase. Concerning the
probability distribution over migrating or not, the illustra-
tions show that a VM that did not migrate in the initial game
has more probability to migrate in the exploitation phase (this
123
E. M. Kandoussi et al.
is viewed by comparing the red surfaces of the two cases).
From a qualitative point of view, the non-deployment of mov-
ing target defense technique as a dynamic security measure
in the network, leaves the honeypot the only defense making
the attack surface more difficult to be exploited. Indeed, in
this scenario this latter adds uncertainty by creating a fake
assets in the network. Hence, by the identification of the real
attack paths, the entire network will be a potential target to
the attacker. Unlike the combined use of migration and hon-
eypot which acts on the real network by moving effectively
the virtual machines, the deployment of honeypot alone in
the network is based on its ability to not be detected and has
not any real impact on the network. Then, the model without
migration will be a set of independent Bayesian sub-games
g i with “i” is the ith attack path and the defender has only
one action which is “M”. On the other hand the attacker has
the possibility to attack or not the network. Concerning the
attack path loss and attack investment, only one quantity will
be used to represent attack path characteristics.
In Fig. 12a, the defender’s utility in case of the deployment
of a honeypot is compared with the case with no deployment
of a honeypot. As depicted in this illustration, the two func-
tions are decreasing with the increase of u s and θs . In addition
for the value of u s ∈ [0, 0.8], the use of honeypot enhance
considerably the security compared to the case of its absence
but for values of u s ∈ [0.8, 1], the deployment of honey-
pot did not improve the security. As a result, the defender
must make some changes in the migration process to avoid
the VMs to be easily identifiable after migration. In general,
this helps to show in which cases thehoneypot will mitigate
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

Fig. 9 Defender’s and attacker’s utility in sub-games where a migration was occurred

Fig. 10 Defender’s and attacker’s probability distribution over actions in sub-games where any migration was occurred

the attacks. Concerning the attacker’s utility, we see that it
is negatively affected by the deployment of the honeypot.
Moreover, for lower values of u s , its utility is negative, and
since the attacker acts rationally, this latter will have no inten-
tion to compromise the VM. In this case, we could say that
the VM is approximately secure based on the set of attack
paths and the metrics of loss associated with.
6 Conclusion and future work
In this paper, we aimed to enhance security in cloud
computing environment by combining optimal strategy for
VM migration and honeypot deployment. We determined
potential attack paths quantitatively and inefficient virtual
machine’s migration based on stochastic game theory, black
box intrusion’s process and attack graph. The proposed

123
 E. M. Kandoussi et al.

Fig. 11 Defender’s and attacker’s utility in sub-games where any migration was occurred

Fig. 12 a Comparison between

defender’s utilities in case of
presence and absence of
honeypot. b Comparison
between attacker’s utilities in
case of presence and absence of
honeypot

dynamic defense system helps the administrator to know A Appendix

in which location in the network new security measures
should be deployed and avoid negative externality. Finally,
we evaluated security measures and showed in which case
this combination improves the system’s security. As future
work, we seek to use a dynamic stochastic game in order to
decrease the number of inefficient migrations and also adjust
the defender’s security configuration based on the attacker’s
interaction history with the virtual machine.
Proof of Theorem 1 For a fixed value of u s we have:
• If Ms is played by the defender then the best response of
the attacker is A Pi0 with: i 0 = argmax u s αi L i − βi Ci .
i∈{1,...,m}
• If M is played by the defender than then best response of
the defender is A Pm .

123
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

Therefore, the Nash equilibrium are given as follows: – If θ (1 − αi ) L i ≥ (1 − βi ) Ci ≥ θ u s (1 − αi ) L i :

⎧  
⎨μ (M |Rn) = 1 1 − (1−βi )Ci
Def s θ(1−αi )L i
• If i 0 = m: 1−u s
⎩μ At (A|Ml) = CM
CM (1−u s )(1−αi )L i
– If u s < 1 − αm L m : N E pur e = (Ms , A Pm ),
  – If (1 − βi ) Ci ≥ θ (1 − αi ) L i ≥ θ u s (1 − αi ) L i :
CM
– If u s ≥ 1 − αm L m : N E pur e = M, A Pm .
• If i 0 = m: μDef (Ms |Rn) = 0
  μ At (A|Ml) = 0
CM
– If u s ≤ 1 − αi 0 L i 0 : N E pur e = Ms , A Pi 0 , • If u s (1 − αi ) L i + C M > (1 − αi ) L i :
 
– If θ (1 − αi ) L i ≥ θ u s (1 − αi ) L i ≥ (1 − βi ) Ci
M
– If u s ≥ 1 − αCm L m : N E pur e = M, A Pm ,
M M or
– If 1 − αiC L i < u s < 1 − αCm L m : we have a mixed
0 0 θ (1 − αi ) L i ≥ (1 − βi ) Ci ≥ θ u s (1 − αi ) L i :
Nash equilibrium
 μDef (Ms |Rn) = 0
N E mixed = x × A Pi0 + (1 − x) × A Pm , y × Ms
+(1 − y) × M Indeed x and y are computed as fol- μ At (A|Ml) = 1
lows: – If (1 − βi ) Ci ≥ θ (1 − αi ) L i ≥ θ u s (1 − αi ) L i :
EDef (Ms ) = EDef (M) ⇔ x = ααmL L m−α
(1−u s )−C

M

L (1−u s) μDef (Ms |Rn) = 0
  m m i0 i0
E At (A Pm ) = E At A Pi0 ⇔ y = 1−u 1
μ At (A|Ml) = 0
 βm Cm −βi 0 Ci 0
 s

× 1 − αm L m −αi L i . i is resolved below

0 0 Secondly, the Bayesian sub-game gm
by using the same approach presented above. Then, we

 obtain:

⎧
⎪
⎪ μDef (M|H p) = 1
B Appendix ⎪
⎪
⎪
⎪ μ 
Def (.|Rn) ∈ argmaxγ ∈Δ A Rn g i
 ∈
⎪
⎪ ( )
⎪
⎪  Def m

⎪
⎪
1. Proof that the strategy Ms is a strictly dominated by ⎨γ (Ms ) −C M + μ At (A|Ml) u s (1 − vs ) (1 − αi ) L i
Ms : −h − μ At (A|Ml) u s (1 − αi ) L i
⎪
⎪
For i ∈ {1, . . . , n}: u s < u s ⇔ u s αi L i < u s αi L i ⇔ ⎪
⎪
⎪
⎪ μ At (.|Ml) ∈ argmaxγ ∈Δ A Ml (gi ) ∈
−u s αi L i < −u s αi L i . ⎪
⎪   At m 
⎪
⎪ γ (A) θ u (1 − α ) L 1 − (1 − vs ) μDef (M|Rn)
Thereby: −u s αi L i − C M < −u s αi L i − C M ⎪
⎪ s i i
⎩
Then the strategy Ms is strictly dominated by Ms . − (1 − βi ) Ci ]
2. The Bayesian sub-game gm i is resolved by using

the mathematical expression related to compute the

Bayesian Nash equilibrium. Hence, we obtain: Then, the probability distributions μDef (.|Rn) and
μ At (.|Ml) are defined as follows:
⎧ • If u s (1 − αi ) L i ≥ vs u s (1 − αi ) L i + C M :
⎪
⎪μDef (M|H p) = 1
⎪
⎪ – If θ u s (1 − αi ) L i ≥ θ u s vs (1 − αi ) L i ≥
⎪
⎪μDef (.|Rn) ∈ argmax    ∈
⎪
⎪ Rn g i
γ ∈Δ ADef
⎪
⎪
⎪  m  (1 − βi ) Ci :

⎨ γ (M ) −C M + μ (A|Ml) (1 − u ) (1 − α ) L
s At

s i i μDef (Ms |Rn) = 1
⎪ −h − μ (A|Ml) (1 − α ) L
⎪
⎪
⎪ At i i
μ At (A|Ml) = 1
⎪
⎪    ∈
⎪
⎪μ At (.|Ml) ∈ argmax
⎪
⎪
⎪
γ ∈Δ A Ml
At gm
i
– If θ u s (1 − αi ) L i ≥ (1 − βi ) Ci ≥ θ u s vs
⎩    
γ (A) θ (1 − αi ) L i 1 − (1 − μs ) μDef (M|Rn) − (1 − βi ) Ci ⎧ − αi ) L i :
(1  
⎨μ (M |Rn) = 1 1 − (1−βi )Ci
Def s 1−vs θu s (1−αi )L i
Then, the probability distributions μDef (.|Rn) and ⎩μ At (A|Ml) = CM
u s (1−vs )(1−αi )L i
μ At (.|Ml) are defined as follows: – If (1 − βi ) Ci ≥ θ u s (1 − αi ) L i ≥ θ u s vs
• If (1 − αi ) L i ≥ u s (1 − αi ) L i + CM: − αi ) L i :
(1
If θ (1 − αi ) L i ≥ θ u s (1 − αi ) L i ≥ (1 − βi ) Ci :
– μDef (Ms |Rn) = 0
μDef (Ms |Rn) = 1 μ At (A|Ml) = 0
μ At (A|Ml) = 1 • If vs u s (1 − αi ) L i + C M ≥ u s (1 − αi ) L i :

123

– If θ u s (1 − αi ) L i ≥ θ u s vs (1 − αi ) L i ≥
(1 − βi ) Ci or
θ u s (1 − αi ) L i ≥ (1 − βi ) Ci ≥ θ u s vs
− αi ) L i :
(1
μDef (Ms |Rn) = 0
μ At (A|Ml) = 1
– If (1 − βi ) Ci ≥ θ u s (1 − αi ) L i ≥ θ u s vs
− αi ) L i :
(1
μDef (Ms |Rn) = 0
. nication Networks and Information Security, 9(3), 345–357.
μ At (A|Ml) = 0
References
1. Adili, M. T., Mohammadi, A., Manshaei, M. H., & Rahman, M. A.
(2017). A cost-effective security management for clouds: A game-
theoretic deception mechanism. In 2017 IFIP/IEEE symposium on
integrated network and service management (IM) (pp. 98–106).
New York: IEEE.
2. Al-Shaer, E., Wei, J., Hamlen, K. W., & Wang, C (Eds.), (2019).
Honeypot deception tactics. In Autonomous cyber deception (pp.
35–45). Berlin: Springer.
3. Arghavani, A., Arghavani, M., Ahmadi, M., & Crane, P. (2018).
Attacker-manager game tree (AMGT): A new framework for visu-
alizing and analysing the interactions between attacker and network
security manager. Computer Networks, 133, 42–58.
4. Azab, M., & Eltoweissy, M. (2016). Migrate: Towards a lightweight
moving-target defense against cloud side-channels. In 2016 IEEE
security and privacy workshops (SPW) (pp. 96–103). New York:
IEEE.
5. Bar, A., Shapira, B., Rokach, L., & Unger, M. (2016). Identify-
ing attack propagation patterns in honeypots using Markov chains
modeling and complex networks analysis. In 2016 IEEE interna-
tional conference on software science, technology and engineering
(SWSTE) (pp. 28–36). New York: IEEE.
6. Baykara, M., & Das, R. (2018). A novel honeypot based security
approach for real-time intrusion detection and prevention systems.
Journal of Information Security and Applications, 41, 103–116.
7. Beckery, S., Seibert, J., Zage, D., Nita-Rotaru, C., & Statey, R.
(2011). Applying game theory to analyze attacks and defenses in
virtual coordinate systems. In 2011 IEEE/IFIP 41st international
conference on dependable systems and networks (DSN) (pp. 133–
144). New York: IEEE.
8. Beham, M., Vlad, M., & Reiser, H. P. (2013). Intrusion detection
and honeypots in nested virtualization environments. In 2013 43rd
annual IEEE/IFIP international conference on dependable systems
and networks (DSN) (pp. 1–6). New York: IEEE.
9. Boulaiche, A., & Adi, K. (2018). An auto-learning approach for
network intrusion detection. Telecommunication Systems, 68(2),
277–294.
10. Cai, G., Wang, B., Wang, X., Yuan, Y., & Li, S. (2016). An intro-
duction to network address shuffling. In 2016 18th international
conference on advanced communication technology (ICACT) (pp.
185–190). New York: IEEE.
11. Carroll, T. E., Crouse, M., Fulp, E. W., & Berenhaut, K. S. (2014).
Analysis of network address shuffling as a moving target defense.
In 2014 IEEE international conference on communications (ICC)
(pp. 701–706). New York: IEEE.
12. Cho, J. H., & Ben-Asher, N. (2018). Cyber defense in breadth:
Modeling and analysis of integrated defense systems. The Journal
of Defense Modeling and Simulation, 15(2), 147–160.
123
E. M. Kandoussi et al.
13. Debroy, S., Calyam, P., Nguyen, M., Stage, A., & Georgiev, V.
(2016). Frequency-minimal moving target defense using software-
defined networking. In 2016 international conference on comput-
ing, networking and communications (ICNC) (pp. 1–6). EEE.
14. Dongxia, L., & Yongbo, Z. (2012). An intrusion detection system
based on honeypot technology. In 2012 international conference
on computer science and electronics engineering (Vol. 1, pp. 451–
454). New York: IEEE.
15. El Mir, I., Kandoussi, E. M., Hanini, M., Haqiq, A., & Kim, D. S.
(2017). A game theoretic approach based virtual machine migration
for cloud environment security. International Journal of Commu-
16. Kaaniche, M., Deswarte, Y., Alata, E., Dacier, M., & Nicomette,
V. (2007). Empirical analysis and statistical modeling of attack
processes based on honeypots. Preprint arXiv:0704.0861.
17. Kamhoua, C.A., Kwiat, L., Kwiat, K.A., Park, J.S., Zhao, M., &
Rodriguez, M. (2014). Game theoretic modeling of security and
interdependency in a public cloud. In 2014 IEEE 7th international
conference on cloud computing (pp. 514–521). New York: IEEE.
18. Kandoussi, E. M., El Mir, I., Hanini, M., & Haqiq, A. (2017). Mod-
eling an anomaly-based intrusion prevention system using game
theory. In International conference on innovations in bio-inspired
computing and applications (pp. 266–276). Berlin: Springer.
19. Khan, M. E., & Khan, F., et al. (2012). A comparative study of
white box, black box and grey box testing techniques. International
Journal of Advanced Computer Science and Applications, https://
doi.org/10.14569/IJACSA.2012.030603
20. Kiennert, C., Ismail, Z., Debar, H., & Leneutre, J. (2018). A survey
on game-theoretic approaches for intrusion detection and response
optimization. ACM Computing Surveys (CSUR), 51(5), 90.
21. Lei, C., Ma, D. H., & Zhang, H. Q. (2017). Optimal strategy
selection for moving target defense based on Markov game. IEEE
Access, 5, 156–169.
22. Lei, C., Ma, D. H., Zhang, H. Q., & Wang, L. M. (2016). Moving
target network defense effectiveness evaluation based on change-
point detection. Mathematical Problems in Engineering, https://
doi.org/10.1155/2016/6391502
23. Lei, C., Zhang, H. Q., Wan, L. M., Liu, L., & Ma, D. (2018). Incom-
plete information Markov game theoretic approach to strategy
generation for moving target defense. Computer Communications,
116, 184–199.
24. Liu, Y., Xu, C., Zhan, Y., Liu, Z., Guan, J., & Zhang, H. (2017).
Incentive mechanism for computation offloading using edge com-
puting: A stackelberg game approach. Computer Networks, 129,
399–409.
25. Maleki, H., Valizadeh, S., Koch, W., Bestavros, A., & van Dijk,
M. (2016). Markov modeling of moving target defense games. In
Proceedings of the 2016 ACM workshop on moving target defense
(pp. 81–92). New York: ACM.
26. Manadhata, P. K. (2013). Game theoretic approaches to attack sur-
face shifting. In S. Jajodia, A. K. Ghosh, V.S. Subrahmanian, V.
Swarup, C. Wang and X. S. Wang (Eds.), Moving target defense II
(pp. 1–13). Berlin: Springer.
27. Mehdi, M. M., Raza, I., & Hussain, S. A. (2017). A game theory
based trust model for vehicular ad hoc networks (vanets). Computer
Networks, 121, 152–172.
28. Mun, H. J., & Han, K. H. (2016). Blackhole attack: User identity
and password seize attack using honeypot. Journal of Computer
Virology and Hacking Techniques, 12(3), 185–190.
29. Ou, X., Govindavajhala, S., & Appel, A. W. (2005). MulVAL: A
logic-based network security analyzer. In USENIX security sym-
posium (Vol. 8, pp. 113–128). Baltimore, MD.
30. Sheyner, O., & Wing, J. (2003). Tools for generating and analyzing
attack graphs. In International symposium on formal methods for
components and objects (pp. 344–371). Berlin: Springer.
Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks…

31. Speicher, P., Steinmetz, M., Backes, M., Hoffmann, J., & Künne- Mohamed Hanini is currently
mann, R. (2018). Stackelberg planning: Towards effective leader– a professor at the Department of
follower state space search. In 32nd AAAI conference on artificial Mathematics and Computer Sci-
intelligence. ence in the Faculty of Sciences
32. Thongthua, A., & Ngamsuriyaroj, S. (2016). Assessment of hyper- and Techniques, Settat, Morocco.
visor vulnerabilities. In 2016 International conference on cloud He obtained his Ph.D. degree in
computing research and innovations (ICCCRI) (pp. 71–77). New mathematics and computer in
York: IEEE. 2013. He is the author and co-
33. Venkatesan, S., Albanese, M., Amin, K., Jajodia, S., & Wright, author of several papers related
M. (2016). A moving target defense approach to mitigate DDoS to the field of modeling and per-
attacks against proxy-based architectures. In 2016 IEEE conference formance evaluation of commu-
on communications and network security (CNS) (pp. 198–206). nication networks, cloud comput-
New York: IEEE. ing and security. He participated
34. Wang, K., Du, M., Maharjan, S., & Sun, Y. (2017). Strategic hon- as TPC member and organizing
eypot game model for distributed denial of service attacks in the committee member in international
smart grid. IEEE Transactions on Smart Grid, 8(5), 2474–2482. conferences and workshops, and he worked as reviewer for several
35. Yusoh, Z. I. M., & Tang, M. (2010). A penalty-based genetic algo- international journals.
rithm for the composite SaaS placement problem in the cloud. In
IEEE congress on evolutionary computation (pp. 1–8). New York:
IEEE. Iman El Mir is currently a pro-
36. Zheng, J., Cai, Y., Wu, Y., & Shen, X. (2018). Dynamic com- fessor at the Department of Com-
putation offloading for mobile cloud computing: A stochastic puter Science in the Polydisci-
game-theoretic approach. IEEE Transactions on Mobile Comput- plinary Faculty, University Abdel-
ing, 18(4), 771–786. malek Essaadi, Larache, Morocco.
37. Zheng, J., & Siami Namin, A. (2018). A Markov decision process to She received the Ph.D. degree in
determine optimal policies in moving target. In Proceedings of the Computer Science in 2018 from
2018 ACM SIGSAC conference on computer and communications Hassan 1st University, Morocco.
security (pp. 2321–2323). New York: ACM. Here research interests are Secu-
38. Zhu, Q., & Başar, T. (2013). Game-theoretic approach to feedback- rity Modelling and Analysis of
driven multi-stage moving target defense. In International con- computer and networks, Cloud
ference on decision and game theory for security (pp. 246–263). Data Center security and secu-
Berlin: Springer. rity in Software Defined Network-
39. Zhuang, R., DeLoach, S. A., & Ou, X. (2014). Towards a theory of ing. She was a member of the
moving target defense. In Proceedings of the 1st ACM workshop Nato Project SPS-984425 entitled
on moving target defense (pp. 31–40). New York: ACM. Cyber Security Analysis and Assurance using Cloud-Based Security
Measurement system. She participated as TPC member and organizing
committee member in international conferences and workshops, and
she worked as reviewer for several international journals.
Publisher’s Note Springer Nature remains neutral with regard to juris-
dictional claims in published maps and institutional affiliations.
Abdelkrim Haqiq has a High
Study Degree (Diplôme des Etudes
Supérieures de troisième cycle)
El Mehdi Kandoussi is a Ph.D. and a Ph.D. (Doctorat d’Etat),
student in IR2M Laboratory. He both in the field of modeling and
works on game security model in performance evaluation of com-
cloud computing environment. In puter communication networks,
2011, he got his bachelor degree from the University of Mohammed
in mathematical sciences, then he V, Agdal, Faculty of Sciences,
continued his studies in prepara- Rabat, Morocco. Since Septem-
tory classes for high schools ber 1995 he has been working
“Mathematics-Physics” until 2013. as a Professor at the department
In 2016, he had successfully com- of Mathematics and Computer at
pleted all requirements and crite- the Faculty of Sciences and Tech-
ria for “Certified Ethical Hacker” niques, Settat, Morocco. He is the
and he graduated from the national Director of Computer, Networks, Mobility and Modeling laboratory.
high school of computer science He is also the General Secretary of the electronic Next Generation
and system analysis “ENSIAS, Networks (e-NGN) Research Group, Moroccan section. He is an IEEE
Morocco”. In December 2017, he was a member of the organizing senior member and an IEEE Communications Society member. He
committee of the 13th international conference on Information Assur- was a co-director of a NATO Multi-year project entitled “Cyber Secu-
ance and Security. rity Analysis and Assurance using Cloud-Based Security Measure-
ment system”, having the code: SPS-984425. Partners: (1) Duke Uni-
versity, Durham, Karoline du Nord, USA. (2) Arizona State Univer-
sity, USA. (3) Canterbury University, Christchurch, New Zealand.

123