You are on page 1of 62

MODELING SECURITY AND RESOURCE

ALLOCATION FOR MOBILE MULTI-HOP


WIRELESS NETWORKS
USING GAME THEORY

• Candidate: Laurent Lavoisier Yamen Njilla

• Major Professor: Dr. Niki Pissinou

• Committee:

• Dr. Kang Yen Dr. Jean Andrian Dr. Deng Pan Dr. Charles Kamhoua

1
Outline

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

2
INTRODUCTION

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

3
What is the Problem?

• There is no infrastructure for the network


– Ad-hoc network
• All network components are mobile or temporary static
• The network does not have a central manager or authority
– The network is distributed
– No Central authority to administer each component
• Mobile nodes are autonomous
– Free to participate in network activities
– Make their own decision
• Mobile nodes can generate and route packets
– Generate information on their own
– Route packets from neighbor nodes
• Node has limited energy and transmission range
– Needs intermediary node in order to communicate with distant node

4
Is Selfish Behavior a Problem?

• Future applications will require autonomous


devices to be interconnected to form a network

• The network will not have a central authority

• Cooperation is a critical issue in multi-hop


networks due to their lack of infrastructure
• Each node in the network is at the same time a
terminal and a router.

5
Challenges?

• There is a cost in battery power and bandwidth to


forward packets

• cooperation involves forwarding costly packets from


other nodes

• It is not in the best interest of the individual node to


forward packets

• The network will collapse if no node forward packets


from others

• The challenge may find solutions using some game


theory concepts

6
Example of MANET

7
RELATED WORK

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

8
RELATED WORK

Credit or Virtual Currency


 Node pays to send or receive packets
 “Nuglet”: Needs tamper-resistant software/hardware to
store the virtual token [Nuglet]
 Credit Clearance Service: Need a central authority for
payment [SPRITE]
Reputation Based
 Reputation is based on participation (packet forwarding)
 Nodes with Bad reputation are ignored/eliminated
 Reputation information can be manipulated

9
RELATED WORK

Game Theory
 Concepts provide solutions when conflict of interests
 Enforce cooperation by repeated interaction
 Packet forwarding game converge to Nash Equilibrium
while all node cooperate
 Basic assumption: Nodes are rational

Evolutionary Game Theory


 Patient Grim Strategy (PGS)
 PGS does not forgive after n defections

10
Dynamics of Data Delivery in MANETs:
A Bargaining Game Approach

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

11
Dynamics of Data Delivery in MANETs:
A Bargaining Game Approach

Nodes are completely mobile or semi mobile

The network topology constantly changing

The result is a limited amount of bandwidth due to the


mobility of neighboring nodes

Node mobility can break down many solutions proposed to


enforce cooperation.

Reputation may not be well-defined due to mobility

Previous research using game theory fail to characterize


the mobility of individual rational and feasible payoff

12
Dynamics of Data Delivery in MANETs:
A Bargaining Game Approach

Game Formulation

 Source node needs to forward data broadcasts its request to


all node in transmission range.

 Neighboring nodes reply to source request with an answer


including some detail attributes:
 Speed, direction, coordinate, energy level, storage
space available.

 Prerequisites: intermediary node can only volunteer its service


if it has the minimal requirements to carry out the work.

13
Dynamics of Data Delivery in MANETs:
A Bargaining Game Approach

The source can select from all the respondents the most
appropriate based on attributes provided

The bargaining between the source and the intermediate node


can start.

There will be no data transfer unless both parties agree on the


splitting rules before data communication starts.

14
Dynamics of Data Delivery in MANETs:
A Bargaining Game Approach

Player 1 has data to be forwarded to destination or AP

P1 also prefers to obtain as much resource as possible

Player 2 is the intermediate node and he has the resource to


carry out the data packets but prefers to keep as much as
possible in order to fulfill other requests

The bargaining game proceeds between players by alternating


offers: by either “Accept” or “Reject”.

15
Our Approach

K is the total amount of resource available to player 2.


Delta (δ) is the discount factor δ ϵ [0, 1]
P1 proposes splitting rule is (x, K-x) to P2 whom can accept
or reject, if rejected
P2 makes an offer to P1
16
The Nash Equilibrium (NE)

There are many NE in this game

Any strategy profile in which P1 splits its data load is a


NE

Not all profiles are subgame perfect equilibrium (SPE)


 For example, if P2 rejects 1st offer at period p=0 and offers
a share x > δ(K-1) in period p=1, then P1 should accept because
of any share bigger than a previous rejected share based on the
worthiness of the share at the period.

17
The Solutions of the Bargaining
Games
 the unique SPE in the last period before device goes
out of range

p is even: the SPE split offered by player i is:

Also, player i accepts any split equal to or less than:

p is odd: the SPE split demanded by player i is:

Also, player i accepts any split equal to or less than:

18
The Unicity of SPE Solutions

 According to Stahl [10], the safety payoff value of a player in a game


is the guaranteed amount the player can get in the bargaining game.

Let m1 and M1 be P1 lowest and highest payoff values in any SPE


where P1 makes an offer.
Denote n1 and N1 be the lowest and highest payoff values for P1 game
in which P2 makes the offer.
m1 ≥ K- δM2;
 M2 ≥ K- δM1 ; P1 will not reject a split of more than δM1
N1 ≥ δM1 because P2 will never offer a share greater than δM1.

Since P1 can obtain at least m1 in the continuation game by rejecting


P2’s offer, P1 will reject any x such that x < δm1. Thus, n1 ≥ δm1
m = M = K/(1+δ); 19
Simulation Results

Algorithm for Bargaining

The OMNET++ is a discrete-event network simulation framework. The goal of


this framework project is to develop a preferred, and open simulation
environment for networking research.

20
Simulation Results

0
-1 10
10
TOPSIS TOPSIS
BGANS BGANS
The Proposed Scheme
Handoff Request Blocking Ratio

The Proposed Scheme

Handoff Occurrence Ratio


-2 -1
10 10

-3 -2
10 10
0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2
Arrival Request Rate Arrival Request Rate

TOPSIS: Technique for Order Preference by Similarity to Ideal Solution


BGANS: Bargaining Game based Access Network Selection

21
Simulation Results

3
BGANS
TOPSIS
2.5
New Request Blocking Ratio The Proposed Scheme

1.5

0.5

0
0 0.2 0.4 0.6 0.8 1 1.2 1.4
Arrival Request Rate

New Request Blocking Ratio

22
Conclusion

 This research shows that each autonomous mobile node


participation in network lifetime depend on their whilst to
route other nodes packets.

 The game theoretic model used in MANET helps


minimize the waste of resources during data transfer.

 The resources allocation takes into consideration multiple


factors: duration of connection, the mobility pattern,
payload, etc..

 We have introduced a Rubinstein-Stahl bargaining game


approach to analyze the resources allocation in MANET.
23
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

24
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Assumptions
 Each node is autonomous and selfish
 Node acts only for its own self-interest
 Node tries to maximize its expected payoff
 Communication channel is bidirectional
 Nodes have the same communication range
 Node has a short radio range
 Node has a wider transmission range
 Energy consumption is high when node uses its wide radio
transmission

25
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Bargaining Game Model:


 The players are the nodes
 An arbitrator is a node with specific functions
 Only arbitrator makes multiple offers
 Arbitrator acts as a cluster head
 Arbitrator generates offers using Dirichlet distribution
 Arbitrator can estimate dwelling time
 Players can ‘accept’ or ‘reject’ offers
 Arbitrator computes decisions from response based on
simple majority
26
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Dirichlet Distribution:
 Captures a sequence of observations of the n possible
outcomes with n positive real parameters xi i=1,…,n n
1 n ki 1 n
xi , xi  0,  xi  1, ki  1. B(k )  B(k1 ,..., k n ) 
  ( ki )
Dir ( x; k )  
B (k ) i 1 i 1  ( k1
i 1

 ...  k n )
,

 During each cycle, offers represent random variables


distributed according to the Dirichlet law.
 The joint density function for 3 players has the form:
(k1  k 2  k3 ) k1 1 k 2 1 k3 1
f ( x1 , x2 , x3 )  x1 x2 x3
(k1 )(k 2 )(k3 )
Where x1+x2+x3 = 1 and k1=k2=k3 = 1 (same weight)

27
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

3-Player Bargaining Game


 An offer from the arbitrator is accepted if at least 2 out of 3
players consent.
 The horizon of negotiations is finite with K shots before a
player is out of range
 Vector o(xi) defines the probability that player I accepts the
current offer xi.
 The utility (Uk) value of the negotiation game for P1 meets:
1
Uk   2 2U k21  6 3U k31
3
at period k

28
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Equilibrium Analysis
 The strategy profile in period t is {(xti, xt-i), gt-i} is offered by
the arbitrator.
 The majority is ruled, Pi “accept” otherwise players “reject”
 The last period T-1, {(xT-1i, xT-1-i), gT-1-i} is a NE if gT-1-i(|xjT-1|)
=“accept” for j≠i and there is no value |xjT-1|> |zjT-1| such that
gjT-1 (|zjT-1|) =“accept” for j≠i leading to the existence of a value
|xjT-1|< |zjT-1|
 Per NE, there is no incentive for Pi to unilaterally increase its
demand because any increase would cause rejection by
another Pj ϵ P-i and the game will not reach the majority
29
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration
Algorithm for Arbitration
Require: # players, N≥3; Time period before out of range, T; storage,
Q; time transferring a packet, t; Cost factor, δ;
Initialization: ;
1: c ← minvalue of player payload
2: While T > 0 do
3: Arbitrator makes offers to players
4: For each Player receiving offer do
5: If accept (Playeri) then increment count End If
6: If count reach Majority Then Data-Transfer
7: Else No Data-Transfer
8: Q ← Q(1 – δ) ;
9: End If
10: Decrement T;
11: End While
12:End
 
30
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Simulation Parameter Parameter values


Simulation Time 500 sec
 MatLab to simulate our analysis.Protocol AODV

 Simulation parameters: Number of Nodes 20, 30, 40


Arbitrator Nodes 5, 10
  Transmission Range 20m, 35m
Node Initial Position Randomly distributed
Mobility Model Random Waypoint
Simulation Area 100m x 100m
Channel Type Wireless Channel
Node Speed interval 0.2 m/sec – 15m/sec
Traffic Type Constant Bite Rate
Time Step 0.1 sec

31
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Simulation Results

Complete Consent Simple Majority


(a) Complete Consent (b) Simple Majority

32
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Simulation Results
Negotiation Game with Three Players
0.34
Simple Majority
Complete Consent
0.32

0.3

 
Player's Utility

0.28

0.26

0.24

0.22
1 2 3 4 5 6 7 8 9 10
Period T

The figure shows the advantage of a simple majority in terms of player’s payoff compared to
complete consent. The network is also beneficial with the fact there is an agreement for the
arbitrator to carry the task of data transfer over

33
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Simulation Results
0.5
Complete Consent
Simple Majority
Average min player's utility

0.45

0.4

0.35
 
0.3

0.25

0.2

0.15

0.1
2 3 4 5
Number of Nodes per Cluster

The figure shows an estimate of the average minimum player’s utility based on the number of
nodes composing the spontaneous cluster. A cluster of two nodes will have almost the same
minimum payoff for each node.

34
Game Theoretic Analysis For Resource
Allocation in Dynamic Multi-Hops
Networks with Arbitration

Conclusion
We model the dynamic resource allocation in a MANET of N
heterogeneous nodes (including arbitrators) as a perfect information
bargaining game.
The arbitrator’s offers of 1/n to all requests enable the players to
accept based on there being no frustrations with other nodes
receiving preferential treatment.
If arbitrators are rational, they can dynamically adapt their decisions
to achieve the best benefit and optimize the network performance

35
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

36
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace

System and Threat Models


3 major players: Users, Providers and Attackers
 Users are customers of service providers
 Users need to trust providers with his private data

 Providers are business entities offering services to users


 Providers need to secure its infrastructure against attackers

 Attackers are malicious users


 Attacker prefer to launch attack on provider infrastructure
then users'

37
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace

User-Defender-Attacker Game Model


User’s strategies: Trust (T) or Distrust (D)
Provider’s strategies: Invest in Security (IS) or Not Invest in Security (NIS)
Attacker’s strategies: Attack (A) or Not Attack (NA)
User trust (T) User distrust (D)
Attacker Attacker
A NA A NA
{G – λ + αλ; {G; {G – λ + αλ - d; {G - d;

IS R – e – λ (1 – α); R - e; R – e – λ (1 – α); R - e;
Provider

b - pα} 0} b - pα} 0}

{G – λ + αλ; {G; {G – λ + αλ - d; {G - d;

NIS R - λ; R; R - λ; R;

b} 0} b} 0}
(a ) User plays action (T) (b) User plays action (D)

38
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace

Game and Equilibrium Analysis


No player can increase his payoff by a unilateral deviation and NE can help predict
the behavior of the player wanting to max their payoff.

R – e – λ(1 – α) > R - λ → e < λ α  Invest in Sec. is the best option


b – pα > 0  the reward of launching an attack should be worth it.

Theorem: If α < α0 = e/λ, then the game admits a pure strategy NE profile (T, NIS, A).

Case 1: if e > λ α, then Uatt(T, NIS, A)–Uatt(T, NIS, NA) > 0. This is not the best for the
provider’s system by not securing its infrastructure.

Case 2: if e/λ < α < b/p, then Uatt(T, IS, A) - Uatt(T, IS, NA) > 0. the attacker prefers to
launch an attack on the provider’s system than not to attack.
Case 3: if e < λα and α > b/p, no strategy profile for a pure NE. (T, IS, A) is not a NE,
the attacker can increase his payoff from A to NA. (T, NIS, A) is not a NE, the provider
can increase his payoff from NIS to IS.  Mixed strategy

39
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace

Mixed Strategy Equilibrium Analysis


0 ≤ x, y, z ≤ 1 are probability variables for user, provider(fournisseur) and attacker
xT + (1 - x)D be the probability mixed strategy NE of the user
yIS + (1 - y)NIS be the probability mixed strategy NE of the provider
zA + (1-z)NA to be the probability mixed strategy of the attacker
the attacker optimal strategy is to randomly choose y such that the provider is indifferent
between strategies IS and NIS. We must have Uprov(IS) = Uprov(NIS)
Attacker: xz(R-e-λ+λα)+x(1-z)(R–e)+(1-x)z(R-e-λ+λα)+(1-x)(1-z)(R–e)= xz(R-λ)
+x(1-z)R+(1-x)z(R-λ)+(1-x)(1-z)R  z = z0 = e/(λα)
Provider : Uatt(A) = Uatt(NA) xy(b-pα)+x(1-y)b+(1-x)y(b-pα)+(1-x)(1-y)b =0  y = y0=b/p

40
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace

Numerical Results

Variation in Provider's Payoff with Probability alpha Variations in Provider's Payoff with the Loss due to Security Breach
1
0.7

0.9
0.65

0.6 0.8
Provider's Payoff

Provider's Payoff
0.55
0.7

0.5
0.6
0.45

0.5
0.4

0.4
0.35

0.3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Probability alpha Loss Lambda to the Provider

Variations in Provider’s payoff with probably α Variations in Provider’s payoff with loss λ due to the breach

41
Game Theoretic Modeling of Security
and Trust Relationship in Cyberspace
Conclusion The 3-player game provide a quantitative
0.6 approach to perform a cost analysis of
security investment
0.4

0.2
The provider does not have the luxury to
not invest in security
0
Provider'sPayoff

An increase in the frequency of attack


-0.2
and provider able to mitigate the loss
might cause the attacker to be detected.
-0.4

-0.6 The limited benefit generated by an


attack may force the attacker not to attack
-0.8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Expense e on Security Investment Users will be less concerned about data
breaches if they do not first hear them
from Mass media.
Variations in Provider’s payoff
42
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

43
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

Assumptions
Each node is autonomous and selfish
 Node acts only for its own self-interest
 Node tries to maximize its expected payoff
Malicious nodes acts against network objectives
Malicious node deliberately wastes others resource
Malicious nodes can regroup as a coalition to attack
Communication channel is bidirectional
Energy consumption is high when node uses its wide radio
transmission

44
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

Bargaining Game Model:


 The players are the nodes
 Players can ‘accept’ or ‘reject’ offers made by the arbitrator
 An arbitrator is a node with specific functions
 Only arbitrator makes multiple offers
 Arbitrator acts as a cluster head
 Arbitrator generates offers randomly
 Arbitrator can estimate dwelling time
 Arbitrator computes decisions from response based on
simple majority
45
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

3-Player Bargaining Game


 An offer from the arbitrator is accepted if at least 2 out of 3
players consent.
 The utility (Uk) value of the negotiation game for P1 meets:
1
Uk   2 2U k21  6 3U k31
3
at period k and U0 << ⅓ for simple majority.
 For complete consent, the utility (Uk) payoff value of the
negotiation for P1 meets: 1
U k  U k 1  (1  3U k 1 ) 3
3

at period k and U0 << ⅓ .


46
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

Simulation Environment
A predetermined amount of nodes colluding to subvert the
arbitrator.
Colluding nodes are one-hop away from each other and move
within the area to keep their connectivity
Minimum number of colluding nodes is 2

47
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

Simulation Results

1 1
Without Collusion Without Collusion
0.9 0.9
With Collusion With Collusion
0.8 0.8

0.7 0.7

0.6 0.6
Successful votes (%)

successful votes (%)


0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0
0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500
time (s) time (s)

Percentage of successful negotiation sessions, Percentage of successful negotiation sessions,


with minimum payload 1667-3334 bytes with minimum payload 3500-4000 bytes

48
A Game Theoretic Approach on Resource
Allocation with Colluding Nodes
in MANETs

Simulation Results

1 1

0.9 0.9

0.8 0.8

0.7 0.7

0.6 0.6
Successful votes (%)

Successful votes (%)


0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2
Without Collusion Without Collusion
0.1 With Collusion 0.1 With Collusion

0 0
0.01 0.05 0.10 0.15 0.20 0.25 0.30 0.40 0.50 0.01 0.05 0.10 0.15 0.20 0.25 0.30 0.40 0.50
discount factor (1 - ) discount factor (1 - )

Percentage of successful negotiation sessions, Percentage of successful negotiation sessions,


with minimum payloads 1667-3334 bytes with minimum payloads 4000-5000 bytes

49
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

50
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Effective defense-in-depth applies multiple layers of defense


throughout the system
Common layers of defense include:
 Cyber agility: pursues techniques that rendered attacks
ineffective
 Attack avoidance, prevention, detection, survivability
 Cyber recovery: seeks to fight-through successful attacks
Goal of agility is to reduce attacks by making it difficult for a
determined and intelligent attacker to succeed.
Agility employs proactive and adaptive defense technique like:
 Randomization, diversity, obfuscation: to increase levels of
complexity and uncertainty
51
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Developing the most robust and agile systems against cyber attacks does
not guarantee the avoidance of all attacks.

Attempting to predict and protect against every conceivable failure and


attack become cumbersome and costly.

Cyber resilience comprise the ability to withstand, minimize, survive and


recover from negative effect of adversity.

Cyber resilience invests in recovery solutions that increase the probability of


assuring MEFs during and after a successful cyber attack.

 Resource allocation in network system between avoidance and recovery is


most pressing for mission-critical systems and

 The challenge faced is to provide equal and adequate systems security


preparedness for both avoidance and recovery
52
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Model Formulation
A system can be in one of the 2 states

A state transition matrix is defined

Transition Probabilities Two-state Markov chain

We can observe that investment in agility will increase the probability q

An increase investment in recovery will increase the probability p

An attacker objective is to switch the system from state 1 to state 0

Make successful attack difficult to reach its objective (increase q) and the system
to remain in state 1

Provide solution to recover from failure and quickly switch to state 1


53
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Markov chain Analysis


By considering a defender who allocates security resources between agility and
recovery maximizes the long-term fraction of time during which the system is in
state 1.
Using probabilities p and q, the steady-state probabilities indicate the long-term
fraction of time.
Given that the system is in state 1 at time 0, there is need to know the probability
that the system remains UP at the time when n very large

54
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Markov chain Analysis

The chain converges to the stationary distribution regardless of where it begins. The
vector π = [π0 π1] is called the equilibrium distribution of the chain.

Tradeoff : Agility vs. Recovery, assuming P10 > 0 or 0<q<1. The administrator
cannot guarantee at 100% that the system will not change from state 1 to state 0

55
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Markov chain Analysis


Probabilities q and p representation. The parameters  and β represent scaling
factors
Probability q and (1-p) as a function of Investment
1

0.9

0.8

The optimal investment in agility and 0.7

Recovery is governed by the relative 0.6 probability 1-p = exp(-0.75*x)

Probability q and 1-p


probability q = 1-exp(-0.75*x)
Value of q and (1-p) 0.5 probability 1-p = exp(-0.5*x)
probability 1-p = exp(-1.5*x)
0.4

0.3

0.2

0.1

0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Security Investment

56
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Markov Decision Process for Resource Allocation


Resource allocation is one of the most challenges in cyber security, especially when
systems have some Service Level Agreement (SLA).
Resource is usually allocated to agility design and implementation.
Consideration has to be made for recovery, if not mission critical system
lifetime may be impaired.
Markov chain analysis in combination of the actions, states of the systems
and reward describe a Markov decision process (MDP)

57
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Value Iteration Algorithm


 1. V0(s) = 0
2. For each state s
3. For each action a
4. Compute Qk(s,a) = R(s,a) + j  P(s,a)V(s’)
5. Until s, |Vi+1(s) – Vi(s)| < 
6. Compute and store *(s) = arg maxa Qk(s,a)
7. Compute and store Vi(s)= Q(s, *(s))
8. Return < *(s), V(s) >

58
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Experimental Results
Cost R10 When Probability p changes Frontier for Optimal Policy Actions 00 and 01
-1 1

-1.2 0.95
Frontier of Optimal Policy with q=0.70; R01=1; R00=-1; R11=1
Frontier of Optimal Policy with q=0.85; R01=1; R00=-1; R11=1
-1.4
0.9
-1.6
0.85
-1.8

Probability p
ard/CostR10

-2 0.8

-2.2
Rew

0.75

-2.4
0.7
-2.6
0.65
-2.8

-3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.8 1 1.2 1.4 1.6 1.8 2
Probability p Reward R01

Resources variation between p and R10 Frontier of Optimal Actions Between p and R01

59
Cyber Security Resource Allocation:
A Markov Decision Process Approach

Experimental Results
Frontier Between Optimal Policy Actions 01 and 00 at State 0 Frontier Between Optimal Policy Actions When q Changes
-2.7 1

-2.75 0.9

-2.8 0.8
Frontier of Optimal Actions from 00 to 01 with R10=-2
Frontier at p=0.9; q=0.9; R00=-1; R01=+1 Frontier of Optimal Actions from 00 to 01 with R10=-3
ard/Cost R10

Probabilityp
-2.85 0.7
Rew

-2.9 0.6

-2.95 0.5

-3 0.4
1 1.5 2 2.5 3 3.5 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Reward/Cost R11 Probability q

Frontier Actions Between Reward R10 and R11 Behavior of Probabilities p and q

60
Conclusion and Future Work

Introduction
Related Work
Dynamics of Data Delivery in MANETs
Resource Allocation in MANETs with Arbitration
Security and Trust Relationship in Cyberspace
Resource Allocation with Collusion in MANETs
Resource Allocation in Cyber Security
Conclusion and Future Work

61
Conclusion and Future Work

We have explored the problem of incentivizing cooperation in MANETs.

We designed an incentive dynamic data delivery for mobile environment, where
mobile a node can move randomly and still participate in data-packet forwarding by
defining a bargaining model, which took into consideration the mobility factor and
parameters like speed, direction and available resource of a node.

We’ve defined the proper evaluation metrics to evaluate the nodes participating in
the overall performance of the MANET and compared to other methods. The
effectiveness was presented with OMNET++ as our simulation environment, where
node mobility is captured as close as possible to reality.

Limitations in design and implementation are the overhead messages and extra
power consumption for a longer bargaining time. Message exchange during
negotiation and the density of the network. When 2 nodes are in bargaining mode,
their speed should be limited.
62

You might also like