This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
Model-Based Quantitative Network Security
Metrics: A Survey
Alex Ramos, Marcella Lazar, Raimir Holanda Filho, and Joel J. P. C. Rodrigues, Senior Member, IEEE
Abstract—Network security metrics (NSMs) based on models
allow to quantitatively evaluate the overall resilience of net-
worked systems against attacks. For that reason, such metrics
are of great importance to the security-related decision-making
process of organizations. Considering that over the past two
decades several model-based quantitative NSMs have been pro-
posed, this article presents a deep survey of the state-of-the-
art of these proposals. First, to distinguish the security metrics
described in this survey from other types of security metrics, an
overview of security metrics, in general, and their classifications
is presented. Then, a detailed review of the main existing model-
based quantitative NSMs is provided, along with their advantages
and disadvantages. Finally, this survey is concluded with an
in-depth discussion on relevant characteristics of the surveyed
proposals and open research issues of the topic.
Index Terms—Security Metrics, Network Security, Model-
Based Metrics, Quantitative Security, Security Level.
I. I NTRODUCTION
In today’s globalized world, people heavily rely on com-
puter networking technologies, since they are crucial to the
operation of the information systems and critical infrastruc-
tures that underpin our society. However, computer networks
are at constant risk from cyber attacks, which are becoming
increasingly severe and sophisticated. Such attacks may enable
an adversary to gain access to private information, degrade
network performance, and, ultimately, take full control of
the target system. Example of cyber attacks that can take
advantage of network vulnerabilities are denial of service,
brute force, spoofing, and buffer overflow [1], [2]. Many of
those attacks are now being targeted not only to computer
applications and enterprise information systems, but also to in-
dustrial control systems at gas pipelines and power plants [3],
[4]. Therefore, possible consequences of cyber attacks can be
devastating, leading to serious economic and social issues, or
even loss of life.
To prevent, or detect and mitigate cyber attacks, network
administrators can take several actions. For instance, they can
make changes to network configurations (e.g., connectivity)
or deploy security solutions (e.g., intrusion detection systems
(IDSs1 ), firewalls [5], [6]). However, properly securing net-
works requires a way to quantify the amount of security
Manuscript received Month XX, 2016; revised Month XX, 2017.
Alex Ramos, Marcella Lazar, and Raimir Holanda Filho are with Univer-
sity of Fortaleza (UNIFOR), CE, Brazil (e-mail: alex.lacerda@edu.unifor.br;
marcellaplazar@edu.unifor.br; raimir@unifor.br).
Joel J. P. C. Rodrigues is with National Institute of Telecommunications
(Inatel), MG, Brazil; Instituto de Telecomunicações, Portugal; ITMO Uni-
versity, Russia; and University of Fortaleza (UNIFOR), CE, Brazil (e-mail:
joeljr@ieee.org).
1 See Appendix for the list of abbreviations used in this survey.
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1
provided by these actions, since “you cannot improve what
you cannot measure” [7].
Given that computer networks are complex systems, mod-
eling techniques can be used to obtain meaningful security
metrics that quantify overall network security. Such techniques
allow to describe a system in terms of an abstract model that
captures only the attributes of interest, based on assumptions
about attacker and system behavior.
For example, a state-based stochastic model can be used to
represent a system in terms of its possible operational states.
These states can indicate when the system is vulnerable, when
it is under attack, etc. The transitions among these states
should also be included. Then, the resulting model could be
parameterized with input data (e.g., success rate of attacks) and
solved to provide the desired security metrics. One example of
security metric that could be derived from such model is the
mean time to compromise the system [8]. A larger value of
this metric indicates a more secure system. One application
example of this metric is to compute its value before and
after a new security mechanism is deployed. This can help
determine whether the overall system security has improved.
In general, the compromised states of a network correspond
to violations of the underlying security policy. In turn, policy
violations occur when attackers gain unauthorized access to
one or more critical components (e.g., database servers). Since
a policy is defined according to the security goals of each
organization, model-based NSMs are used to quantify security
level in a relative context.
Another model commonly used to quantify network security
is the attack graph (AG). An AG represents a system as a set of
vulnerabilities and their interdependencies. This graph-based
model allows to visualize the ways an attacker can exploit
interdependent vulnerabilities to violate a security policy. In
AGs, a vulnerability exploitation usually results in privilege
escalations on hosts (e.g., gaining root access).
As an example, consider the hypothetical enterprise network
configuration presented in Fig. 1. It consists of two servers
and two firewalls. The database server (D) is located in the
internal network while the IIS web server (W) is located
in the demilitarized zone (DMZ) of the enterprise network.
Firewall 1 controls external access from the Internet to the
DMZ. Firewall 2 mediates the access to the internal network.
According to the firewalls rules, external users are only
allowed to access the web server through HTTP (port 80) and
the database server through SSH (port 22). Only the web server
is allowed to access the database server though protocols FTP
(ports 20 and 21) and RSH (port 514). Suppose also that
the security policy for this network states that external users
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 2

should not be able to gain root privilege on the database server. IIS Web Database
Server (W) Server (D)
In Fig. 2, a possible attack graph2 for the example net- http
ssh
ftp
work (and security policy) is shown. In the graph, plaintexts rsh
refer to access rights on hosts, e.g., user(A) indicates that Internet
the attacker has user privilege on host A. Ovals represent
vulnerability exploits. For instance, sshd bof(A,D) indicates Attacker (A) Firewall 1 Firewall 2

the exploitation of an SSH buffer overflow vulnerability (CVE- External DMZ Internal
Network Network
2002-06403 ) executed from host A to host D. The edges of
the graph show how the exploits can be chained together Fig. 1. Sample Enterprise Network.
in different ways to form attack paths. Notice that attack
paths take the network from an initial state to a compromised
sshd_bof(W,D)
state. There are three attack paths in the example AG. The
longest path starts with the exploitation of an IIS bof (IIS IIS_bof(A,W)

buffer overflow vulnerability, CVE-2002-0364) from host A user(A) ftp_rhosts(W,D) rsh(W,D) local_bof(D,D) root(D)
to host W. This allows the attacker to gain access to host
W and exploit an ftp rhosts vulnerability (CVE-1999-0527).
sshd_bof(A,D)
In turn, this enables the attacker to anonymously upload a
list of trusted hosts from host W to host D. Using this new
trust relationship, the attacker executes a remote shell login Fig. 2. Sample Attack Graph.
(rsh) from host W to host D, without providing a password.
Finally, to gain root access on host D, the attacker exploits
general overviews and definitions of the field [11], [12],
a local bof vulnerability (local buffer overflow, CVE-2001-
[13]. Some other works propose taxonomies [14], [15] and
0886). The other two attack paths of the AG are similar to the
classification schemes [16], [17], [18] to categorize the various
one just described, but shorter.
types of existing security metrics. There are also many other
One attack graph-based security metric that could be used
works that focus on specific types of metrics. In [19], [20],
to express the overall security of the example network is the
[21], [22], the authors specify guidelines for using or creating
Shortest Path (SP) metric [10]. This metric corresponds to the
measurement-based security metrics tailored for a particular
length (number of exploits) of the smallest attack path. SP
organization. The works in [23], [24], [25] focus specifically
assumes that a network is no more secure than its smallest
on software security metrics, while the work in [26] focus on
attack path. Therefore, when comparing two network config-
economic security metrics.
urations, the one with shorter SP value is regarded as less
In contrast to these related works, the present survey focuses
secure. Considering the attack graph in Fig. 2, the shortest path
specifically on model-based quantitative security metrics that
(sshd bof(A,D) → local bof(D,D)) has size two. Therefore,
aim to quantify overall network resilience against attacks. To
to increase the security level of the network, one approach
the best of our knowledge, this is the first comprehensive
would be to disable the SSH service of the database server.
survey specifically addressing this type of security metric. This
As a result, the size of the smallest path would change to four.
more concentrated approach enables a more profound analysis
This would supposedly increase the difficulty of attacks and,
and fair comparison of the advantages and disadvantages of the
consequently, make the network more secure.
reviewed proposals. In addition, this survey reviews security
In general, model-based quantitative NSMs can support the
metrics not covered in the aforementioned papers, including
security-related decision-making process of organizations by
more recent proposals.
answering various questions, including, but not limited to:
The rest of the paper is organized as follows. Section II
• Which network configuration is the most secure? provides an introduction to security metrics in general. Sec-
• Is the network achieving its required security level? tion III describes the main model-based quantitative NSM
• How to change network configuration in order to improve proposals that are based on state-based stochastic models. The
security? proposals based on attack graphs are reviewed in Section IV.
• What is the impact of ongoing attacks? Section V covers proposals that are based on models other
Several quantitative NSMs derived from stochastic models, than attack graphs and stochastic models. In section VI, this
attack graphs, and other models (e.g., Bayesian networks) have paper discusses relevant features of the reviewed proposals
been proposed in the literature. This article presents a deep and presents open research issues in the topic of model-based
survey of the state-of-the-art of these proposals. quantitative NSMs. Related work is reviewed in Section VII.
Many other surveys on security metrics have been pub- Finally, Section VIII concludes this study.
lished. However, because the security metrics field has an
extremely broad scope, most of these works focus on providing II. I NTRODUCTION TO S ECURITY M ETRICS
To clarify how model-based quantitative NSMs relate to
2 Attack graphs can be automatically generated by tools like the Topological other types of security metrics, this section provides an
Vulnerability Analysis (TVA) system [9].
3 CVE – Common Vulnerabilities and Exposures, and details of the vulner- overview of the security metrics field. Specifically, we present
ability identifiers used in this paper can found in http://cve.mitre.org/. the desired properties and classifications of security metrics.

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
Then, the Common Vulnerability Scoring System (CVSS)
[27], a vulnerability impact quantification framework used by
several NSMs surveyed in this article, is described. In addition,
a comparison between the security metrics field and other
related areas is also provided.
In order to understand the meaning of security metrics,
it is important to distinguish metrics from measurements. A
measurement is an observable value (associated with a given
property or attribute) which is collected using any appropriate
method that transforms the observation into data [16], [17].
A metric is comprised of a set of measurements along with
a series of predefined rules that allow the interpretation of
the collected data values [22], [16], [17]. In other words,
measurements are raw data, while metrics are generated from
the analysis of those data. For example, the number of spams
detected is a measurement. However, this measurement by
itself is not very useful. Only when it is compared to a
reference and interpreted to draw conclusions, it becomes a
metric. For instance, the number of spams detected this month,
when compared to the number of spams detected last month,
could indicate whether the effectiveness of content filtering
software is improving [7].
In general, security metrics are used to support decision-
making regarding security-related characteristics of a process,
system, or organization. In particular, security metrics can
be applied to compare different security mechanisms, or to
indicate the degree to which security requirements of an
organization are being met. In addition, they can also be used
to systematically improve the security level of a system, or to
predict this security level in a future point in time [14], [15].
Technical personnel (e.g., security engineers or network
administrators), management members, end users, and external
authorities are the main audience of security metrics [14], [18].
Technical personnel can, for example, use security metrics
to decide which configuration change is the most effective
to increase network resilience. Management members can,
for instance, make decisions regarding financial investment
on security, resource allocation, program planning, and other
strategic actions. End users, in turn, can use security metrics
to decide whether they should trust a specific Web application.
Finally, external authorities can use security metrics as part of
the certification process of organizations [14], [18], [28].
A. Desired Properties of Security Metrics
When designing a security metric, the following properties
should be satisfied [29], [17]:
• Granularity: Metrics should allow to distinguish all sys-
tem states according to the differences among their re-
spective attribute values. This makes metrics more accu-
rate because it avoids that systems with different security
levels are regarded as having the same security level;
• Availability: It should always be possible to calculate the
metric for a particular system, preferably in an easy and
efficient way. If a metric is not available when needed, it
will not be useful for decision makers;
• Cost effectiveness: The computation process of a metric
should be cost effective. This is desirable because the
benefits of applying a metric must justify its cost.
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
3
• Localization: A metric should have a clearly defined
scale. This includes determining the type of scale used,
the range of values of the scale, and their respective
interpretation. This is necessary for the correct application
of the metric.
• Validation: This property states that a metric should
be based on a hypothesis that relate the measured or
estimated value to the actual value of the security attribute
being analyzed. Hence, it is important to validate a
metric in order to check whether its calculated values
accurately represent the associated security attribute of
the target system. This is especially necessary for model-
based security metrics, which are derived from system
abstractions, rather than actual system behavior. If the
validation property is not satisfied, the metric may lead
to incorrect decision-making.
More details about the desired properties of security metrics
can be found in [29], [17], [7].
B. Classification of Security Metrics
As can be seen in Fig. 3, it is possible to classify security
metrics according to the following criteria [14], [16], [17]:
1) Target Type: Security metrics can be categorized ac-
cording to the target they evaluate. The most common targets
assessed (and respective security metrics) are the following:
• Process: Refers to the process undertaken to develop a
given product (e.g., software). Process security metrics
quantify the security level of a product by assessing its
associated development process. The idea behind this
type of metric is that a more secure development process
results in a more secure product. Since process security
metrics measure security in an indirect manner, it is hard
to define their actual impact of a process on the security of
a product. One example of process security metric is the
Software Security Engineering Capability Maturity Model
(SSE-CMM) ISO/IEC 21827 international standard [30].
• Software: Software security metrics evaluate software
security by assessing source code defects, software
(mis)configuration, or other vulnerabilities present in soft-
ware components. More specifically, software security
metrics evaluate the security level of individual software,
instead of whole networks. However, software security
metrics can be used as input to network security metrics,
since network security is affected by software vulnerabil-
ities. For example, the VEA-bility metric [31] measures
network security by combining CVSS scores of software
vulnerabilities present in network hosts. Other examples
of software security metrics include number of defects
per line of code [14] and attack surface [32].
• Network: Unlike software security metrics, which are
concerned with individual software, NSMs assess the se-
curity of entire networks or parts thereof. NSMs take into
account vulnerabilities that emerge from the interaction
among applications running in distinct hosts. Although
this survey focus mostly on NSMs for traditional net-
works, the NSMs field also includes metrics for specific
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 4

Security
Metrics

Construction Automation Measurement Measurement Measurement

Target Type Objective Type
Type Level Consistency Type Moment

Measurement
Process Compliance Manual Subjective Quantitative Static
-based

Software Economic Model-based Automatic Objective Qualitative Dynamic

Semi-
Network Effectiveness
automatic

Organization

Fig. 3. Classification of Security Metrics.

types of networks (e.g., wireless sensor networks [33], Examples of such metrics are the false positive and false
[34], [35] and cloud computing environments [36], [37]). negative rates of an intrusion detection system [39].
• Organization: This category includes metrics that eval-
3) Construction Type: Depending on the way security
uate physical security and personnel security. Physical
metrics can be derived, they can be classified as:
security refers to the protection of organizational assets
(e.g., hardware, software, information) against physical • Measurement-based: These metrics (also referred to as
threats. Personnel security refers to the activities devoted empirical metrics [29]) are generated from observation
to the training and posterior monitoring of organization’s of the security property they evaluate, i.e., observation of
staff. Examples of organization security metrics might be real behavior of the system. These metrics are only useful
the percentage of organizational units (e.g., departments) for quantifying properties that can be directly measured.
with contingency planning [18] and the percentage of Furthermore, unlike model-based metrics, measurement-
personnel with proper security training [14]. based metrics are not useful for prediction. Simple coun-
2) Objective Type: Depending on its objective type, a ters such as the number of attacks detected by the IDS
security metric can be categorized as [17], [38], [12]4 : [7] and the number of packets blocked by a firewall [7]
are classical examples of measurement-based metrics.
• Compliance: Compliance metrics attempt to measure how
• Model-based: These metrics, (also referred to as ana-
well the security requirements of a target (e.g., process,
lytical metrics [29]) are needed when the relation be-
network) are being met, in accordance with security
tween the measurements and the security property being
policies, standards, and best practices. One example is
measured is not trivial. In this case, the target being
the metric amount of users with passwords in accordance
evaluated is represented by a formal mathematical model
with the password management security policy [14].
and the metric values result from complex mathematical
• Economic: Metrics related to the financial aspects of secu-
equations. In other words, this type of metric is generated
rity are classified as economic. Such metrics can evaluate,
from the observation of models of the system.
for example, the expected cost required for performing
As depicted in Fig. 4, once a model of the system
successful attacks or applying security countermeasures.
is created, it takes several input parameters (including
The financial loss that results from security incidents or
measurement-based security metrics) and produces as
the financial benefit from investing resources in security
output the desired model-based security metrics [40].
solutions can also be assessed. Examples of such metrics
Examples of models used to evaluate security metrics are
might be the Return on Security Investment [26] and the
attack graphs, Markov models, attack trees, Bayesian net-
Annual Loss Expectancy [26].
works, etc. The required input parameters vary, depending
• Effectiveness: Effectiveness security metrics measure how
on the specification of the model. Examples of such inputs
well the deployed countermeasures are being able to
include information about attackers’ attributes, network
protect from, detect, or respond to security violations.
vulnerabilities, attack data collected by monitoring mech-
4 The
anisms, network topology, among others.
categories of the objective type criteria are not mutually exclusive.

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
Fig. 4. Evaluation Process of Model-Based Security Metrics.
4) Automation Level: Depending on the level of automation
employed to perform the measurements, a security metric can
be classified as manual, automatic, or semi-automatic:
• Manual: This type of metric requires total human in-
volvement, i.e., the collection of the metric values is
entirely executed in a manual fashion. Manual collection
of security measurements may be very time-consuming,
especially if the target being evaluated is very large.
One example of manual security metric is the number
of employees who have received security training [18].
• Automatic: In automatic security metrics, the measure-
ment process requires (almost) no human intervention,
since it can be thoroughly performed by a computerized
system. For that reason, automatic security metrics are
preferable. One example of automatic metric might be
the number of spams detected in e-mail accounts [7].
• Semi-automatic: In this type of metric, some parts of the
measurement process are executed in a automatic manner
while other parts still require manual intervention. Since it
may be hard to develop automatic security metrics, many
metrics tend to be semi-automatic. The attack surface
metric is one example of semi-automatic metric [32].
5) Measurement Consistency: According to the consistency
of the metric values, security metrics can be classified as:
• Subjective: In subjective metrics, different people ap-
plying the same method to evaluate the same property
may produce different results, i.e., determining the metric
values depends on human judgment. An example of sub-
jective metric is the number of training hours necessary
for an employee to securely operate the system [14].
• Objective: Objective metrics do not depend on human
judgments. In this case, different people performing the
same measurement will get the same results. One example
of objective metrics is the number of hours a system takes
to recover from a security failure [41]. Although objective
security metrics are preferable, various security metrics
tend to be quite subjective since several human behavioral
characteristics are inherent to the security field.
6) Measurement Type: According to the type of measure-
ment, security metrics can be classified as:
• Quantitative: Quantitative security metrics are expressed
as percentages or cardinal numbers (i.e., numbers that
count something, instead of ordinal numbers, which only
denote the position occupied by a given object). For
example, the amount of application security vulnerabil-
ities [7] is expressed as a cardinal number that counts
something. The advantage of quantitative values is that
they can be used in algebraic or arithmetic operations in
order to derive more insightful and robust metrics.
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
5
• Qualitative: Qualitative security metrics are expressed by
labels such as high-medium-low values. Ordinal numbers
can also be regarded as qualitative values. For instance,
the high-medium-low values could be converted into
the numbers 1, 2, and 3, respectively. In this case, as
opposed to cardinal (quantitative) numbers, the ordinal
(qualitative) numbers 1-2-3 do not count anything. Hence,
using qualitative values (i.e., labels or ordinal numbers)
in algebraic or arithmetic operations would be mathemati-
cally incorrect. One example of qualitative security metric
is the Common Criteria ISO/IEC 15408 international
standard [42]. This metric provides seven evaluation
assurance levels that vary from EAL1 to EAL7. Other
example is the password quality metric [43], whose values
can be poor, average, good or excellent.
7) Measurement Moment: Depending on the moment that
security metrics are applied to assess a given target, they can
be classified as either static or dynamic:
• Static: Static, or pre-deployment, security metrics are de-
veloped to be measured before the assessed target enters
operation. Such metrics are usually based on static or
architectural security properties that hardly change over
time. This type of metric enables security practitioners
to perform what-if analysis with the aim of determining
which configuration or version of the target should be
deployed. One example of static security metrics might
be the size of the cryptographic keys of hosts [44].
• Dynamic: Dynamic, or run-time, security metrics are
those developed to be constantly measured, during the
operation of the target being evaluated. This type of met-
ric allows the security of the target to be monitored and
improved in real-time. One example of dynamic security
metric is the amount of reported attack attempts [14].
Since security is an attribute that changes dynamically
according to the behavior of the target, dynamic security
metrics are desirable.
C. Common Vulnerability Scoring System
The Common Vulnerability Scoring System (CVSS) is a
public framework that provides a standardized method for as-
signing quantitative values (scores) to security vulnerabilities
according to their severity [27]. A CVSS score is a decimal
number in the range [0.0, 10.0]. This score is derived from
a set of qualitative values assigned by security experts to
predefined attributes (sub-metrics) of a given vulnerability. The
qualitative values assigned are then converted into quantitative
values and combined using a set of special formulas in order
to provide the final score. Each qualitative value of a given
attribute has a predefined corresponding quantitative value.
The sub-metrics defined by the CVSS are divided into three
groups: base, temporal, and environmental. Those groups and
their respective sub-metrics are briefly described as follows:
• Base group: Represents the fundamental and intrinsic
characteristics of the vulnerability that do not change over
time and do not depend on the system environment. This
group is comprised of six sub-metrics:
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
– Access Vector: Describes the means by which the
vulnerability can be exploited. Accepted values are
Local, Adjacent Network, and Network.
– Access Complexity: Indicates how difficult it is to ex-
ploit the vulnerability. The defined values are High,
Medium, and Low.
– Authentication: Measures the number of times an
attacker needs to authenticate to the system in order
to misuse the vulnerability. Options are Multiple,
Single, and None.
– Impact sub-metrics: Three sub-metrics that measure
the impact on confidentiality, integrity, and avail-
ability (respectively) caused by the exploitation of
the vulnerability. Options are None, Partial, and
Complete.
• Temporal group: Represents the characteristics of the
vulnerability that may evolve over time. This group
contains the following sub-metrics:
– Exploitability: Indicates the availability of exploita-
tion techniques or codes. Score options are Proof-of-
concept, Functional, High, and Not Defined.
– Remediation Level: Specifies the availability of fixes
or patches for the vulnerability. Available options are
Official Fix, Temporary Fix, Workaround, Unavail-
able, and Not Defined.
– Report Confidence: Measures the level of credibility
regarding the existence of the vulnerability and its
technical details. Options are Unconfirmed, Uncor-
roborated, Confirmed, and Not Defined.
• Environmental group: Represents the characteristics of
the vulnerability that are specific for the system envi-
ronment. The five sub-metrics comprising this group are:
– Collateral Damage Potential: Indicates the financial
impact on physical assets caused by the exploitation
of the vulnerability. Options are None, Low, Low-
medium, Medium-high, High, and Not Defined.
– Target Distribution: Measures the fraction of system
components affected by the vulnerability. Options are
None, Low, Medium, High, and Not Defined.
– Security Requirements: Three sub-metrics that assess
the damage that results from the loss of confidential-
ity, integrity, and availability (respectively) caused by
the exploitation of the vulnerability. Possible values
are Low, Medium, High, and Not Defined.
Several vulnerability databases, including the US National
Vulnerability Database (NVD) [45], use CVSS to quantify the
severity of reported vulnerabilities. Currently, CVSS is in its
third version [46], launched in 2015. However, the sub-metrics
described above belong to the second version of CVSS [47],
which is used by most of the existing CVSS-based NSMs.
Despite the use of qualitative values for the sub-metrics,
CVSS is regarded as a quantitative framework. It is argued
that these qualitative values are only used to facilitate the
assignment of the actual quantitative values (probabilities)
related to each sub-metric. Furthermore, given that CVSS
is a well-established and widely adopted framework, it can
provide consistent results when used to compare the security of
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
6
different systems. Readers interested in the quantitative values
associated with the qualitative ratings of sub-metrics, the
aggregation formulas, or other details about CVSS version 2.0
may refer to [47], [48].
D. Security Metrics and Other Related Areas
The terms dependability metric [49], quality of protection
[50], and network security situational awareness [51] are
often used interchangeably with the term security metric.
Although closely related, these terms differ from security
metrics in several ways. Hence, to distinguish these terms, this
section provides their definitions, as well as their different,
overlapping, and complementary characteristics with respect
to security metrics.
Computer system dependability is a well investigated area in
which several model-based approaches have been developed
to quantify the effect of failures in computer systems. Unlike
security metrics, which are designed to measure the impact of
intentional failures caused by attacks, dependability metrics
measure the effect of natural or accidental failures [52]. Since
those two areas share common attributes (e.g., availability,
integrity), some works in the literature have proposed single
frameworks to integrate security and dependability measures,
e.g., [53], [54]. Such approaches allow to measure the trust-
worthiness of a system regardless of how failures occur
(i.e., intentionally or accidentally). The similarities between
those two areas is even more evident with regard to model-
based metrics. This is due to the fact that the techniques for
model-based evaluation of security have been derived from the
techniques applied to the dependability domain. A detailed
discussion on the adaptation of model-based metrics from
dependability to security is provided in [52]. Details about the
differences and similarities between security and dependability
are provided in [55].
The term Quality of protection (QoP) has emerged in the
literature, but has not yet been clearly defined. In general, QoP
is used as a synonym for security level [56]. From this point
of view, it can be said that security metrics are a measure
of the quality of protection of systems. The term QoP has
also been used in the domain of mobile networks, with regard
to frameworks that provide a trade-off between security and
performance, according to different user preferences. In such
frameworks, QoP refers to the security levels provided by
different cryptographic algorithms and key sizes [57], [58],
[59]. A similar approach is used in both the Digest Access
Authentication protocol (RFC 2617 [60]) and the WS-Security
[61]. In particular, these protocols provide a QoP field to
specify authentication and integrity options. Hence, QoP can
be viewed as the level of protection a service provides for
transmitted messages. In this context of web services, QoP
has also been defined as a set of security guarantees offered
by service providers with respect to the services they provide.
This allows clients to discover services with the required level
of security [50], [62]. In general, the notion of quality of
protection in security is comparable to the notion of quality
of services (QoS) in networking [63], [64].
Network security situational awareness (NSSA) is a more
general term related to everything that can provide any kind
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
of perception of the security state of a network. NSSA tries
to provide answers to questions like: What is going on in
the network at this moment? What type of different traffic is
appearing on the network? NSSA approaches cover mecha-
nisms such as network traffic monitoring, intrusion detection,
data fusion, and network security visualization [65], [51],
[66]. NSSA tools deal with a large amount of network audit
trace data collected from different sources. As the amount of
data produced grows exponentially, data fusion and correlation
techniques are employed to summarize the data. Visualization
tools are used to present the summarized data (e.g., network
topology, port activity, routing behavior) in a friendly and
interactive manner. All those techniques combined allow an-
alysts to identify relationships, patterns, and anomalies that
are implicit in the underlying data [65]. Recent researches in
NSSA point to the development of algorithms that prioritize
situations, and to an integration of NSSA and security metrics
[65], [67]. This results in tools that can automatically identify
and assess the impact of critical events on network assets.
III. M ODEL -BASED Q UANTITATIVE N ETWORK S ECURITY
M ETRICS BASED ON S TOCHASTIC M ODELS
In the model-based quantitative NSMs domain, state-based
Stochastic models are used to represent the evolution of
systems and attack processes over time [40]. In such models,
the system is expressed as a finite state machine. Each state
represents a security operational mode of the system, indi-
cating whether it is under attack, whether countermeasures
are being executed, whether components of the system are
compromised, etc. Depending on the level of abstraction, the
system’s states are modeled as one or more good states and
one or more security failure states. Good states are those in
which the system is able to deliver the required services, even
in the presence of attacks. On the other hand, in security failure
states, the system has been compromised by attacks in a way
that its intended services can no longer be delivered.
During its lifetime, a system will continuously alternate
between its several possible states. A state transition is trig-
gered by a given event such as the appearance or remediation
of a vulnerability, the successful execution of an attack step
(elementary attack), the responsive actions performed by se-
curity mechanisms, etc. To model the dynamic behavior of the
system, each transition is labeled with a random variable. Such
variable can indicate, for example, the probability or the time
to complete the event that triggered the associated transition.
The uncertainty in the occurrence of events is represented by
the distribution functions describing the random variables.
Most state-based security metric proposals model the system
as a Continuous-Time Markov Chain (CTMC) with discrete
state space. A CTMC is a special type of state-based stochastic
process that possesses the Markov property. According to this
property, the probability distribution of the next state depends
only on the current state and not on the previous states.
Moreover, the time spent in each state takes a continuous set of
values (i.e., non-negative real values), and follows an exponen-
tial distribution. A CTMC can be viewed as a continuous-time
version of the Discrete-Time Markov Chain (DTMC).
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
7
Modeling a system as a stochastic process allows to estimate
valuable information about its resilience to attacks. The work
presented by Littlewood et al. [68] was one of the first
attempts in the direction of operational security quantification.
In particular, the authors have raised several questions that
should be answered to allow the development of security
metrics based on stochastic models. They have also concluded
that an empirical investigation (e.g., collecting data from real
systems in operation) would be required to answer many of
these open questions.
Following the suggestion made by Littlewood et al. [68],
Jonsson and Olovsson [69] have performed various experi-
ments in a real system to better understand the intrusion pro-
cess in networks. The results collected during the experiments
have indicated that the times between security breaches are
exponentially distributed. This implies that stochastic models
used in reliability could be applied to quantify system security,
as initially suggested in [68].
In the following, the main quantitative NSMs based on
stochastic models are reviewed.
A. Time-Based Security Metrics
Time-based security metrics express the amount of time a
system takes to be compromised by a successful attack. A
larger time value implies more security.
1) Privilege Graph-Based Metrics: In [70], [71], Dacier
et al. propose a state-based Markov model to evaluate system
security based on the Mean Time to Failure (MTTF) metric.
Initially, their model highlights the vulnerabilities of a system
by means of a privilege graph. In this type of graph, nodes
represent access rights (privileges) at hosts, and arcs represent
the execution of actions that enable the transition from one
privilege to another. Each arc is labeled with a value (λ) that
represents the success rate of an elementary attack (privilege
escalation). Target nodes (e.g., super user privileges) are
marked in black. Once the privilege graph is constructed, it
is turned into a CTMC containing all possible attack paths to
target nodes. The Markov model is based on the assumption
that the success probability of an elementary attack before
time t is represented by an exponential distribution given by:
P(t) = 1−e−λt . The mean time of succeeding in an elementary
attack is given by 1/λ. The MTTF of the system is computed
by aggregating all the mean times necessary for succeeding in
elementary attacks that lead to the targets.
In [72], Ortalo et al. extend the model proposed by Dacier
et al. [70], [71] to compute the Mean Effort to Failure (METF)
metric. The notion of effort in METF is analogous to the
notion of time in MTTF. The authors compute METF in
three distinct ways, each one based on a different assumption.
In the first assumption, regarded as total memory (TM), at
each elementary attack, the attacker is able to obtain one out
of all privileges reachable from the already visited nodes.
This is the same case considered by Dacier et al. In the
second assumption, referred to as memoryless (ML), at each
stage of the attack process, the adversary can only obtain
privileges reachable from the current node. Finally, in the third
assumption, named shortest path (SP), the adversary selects
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
the path that requires the lowest cumulative effort to reach the
target. To validate the proposed METF metrics, the authors
conduct experiments on a large real system for almost two
years. Their results show that METFT M is more accurate than
METF M L , which, in turn, is more accurate than METFS P .
However, in some cases, it might be impossible to compute
METFT M , due to the huge number of possible attack paths
it generates from the privilege graph. This is known as the
exponential state explosion problem.
Although MTTF and METF provide useful security level
information, they do not take into account partial or interrupted
attacks (which may also cause damage to the system).
2) Metrics Based on a Manually Defined State Space: In
the model proposed by Almasizadeh and Azgomi [40], [73],
[74], countermeasures performed by security mechanisms are
also considered. Such countermeasures take the system from
partially compromised states back to the secure state. In
this model, network’s resilience to attacks is represented by
the Mean Time to Security Failure (MTSF) metric. MTSF
estimates the amount of time the first attack takes to be
successfully performed. To compute MTSF, the security failure
state is modeled as an absorbing state. In addition, the state
space is derived from manually defined attack types and attack
steps. The model needs to take as input probability distribu-
tions for the attack steps. The time interval over which these
distributions are defined also need to be provided. After the
inputs are provided, the model results in a Semi-Markov Chain
(SMC). Then, this SMC is transformed into its corresponding
DTMC and the metric is computed. With SMCs, any type of
distribution can be used in state transitions, unlike CTMCs,
which require probabilities to be exponentially distributed.
In [53], [54], Sallhammar et al. propose a model to estimate
the Mean Time to First Failure (MTFF) metric. Their model
takes into account both security and dependability on a single
framework, instead of treating these two aspects separately.
This allows to measure the trustworthiness of a system regard-
less of how failures occur (i.e., intentionally or accidentally).
In this proposal, the system is modeled as a CTMC. The state
transition rates can be of three types: accidental failure rates,
repair rates, and intentional failure rates. The first two types
of rate are dependability-related measures and can be directly
gathered from systems in operation and given as input to the
model. In contrast, the intentional failure rates are given by
multiplying the accumulated intensity of a given attack step
by the probability that this attack step is executed. In turn, the
accumulated attack intensities can be directly measured from
multiple sources (e.g., historical data). Finally, to compute the
probabilities of attack step execution, the authors apply game
theory techniques based on reward and cost (negative reward)
values assigned to the actions of attacker and defender.
One advantage of this proposal is its focus on the impact
of intrusions, rather than on specific attacks. This enables
the modeling of zero-day attacks in terms of generic state
transitions, i.e., any actions that take the system from a good
state to a failure state.
3) Metrics Based on the Time to Compromise of System
Components: McQueen et al. [8], [75] propose a model to
estimate the Mean Time to Compromise (MTTC) metric based
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
8
on a compromise graph (CG). A CG can be regarded as an
extended version of a privilege graph. According to the CG,
each component (e.g., hosts) of the modeled system possesses
an associated time to compromise (T). The system MTTC is
given by the sum of the T values of the components in the
shortest attack path of the CG.
In turn, T is modeled as a random process comprised of
the following three subprocesses. Process 1 represents the
situation in which the component has at least one known
vulnerability and the attacker has at least one exploit on hand.
Process 2 occurs when there is at least one known vulnerability
on the component, but the attacker has no exploits available.
Finally, Process 3 corresponds to the discovery phase of zero-
day vulnerabilities and exploits.
Each of these three processes is comprised of two parts.
The first part corresponds to the probability that the attacker
is in the respective process. This probability is computed from
information such as total number of existing vulnerabilities,
number of vulnerabilities present in the system component,
and number of available exploits per attacker skill level
(novice, beginner, intermediate, or expert). The second part
refers to the estimated amount of time expended in the respec-
tive process. The time for Process 1 is defined by the authors
as 8 hours. The time for Process 2 depends on the mean time
to create an exploit (given below) and the mean number of
attempts necessary for successfully completing an attack. This
last value is computed based on attacker skill level, number
of exploitable vulnerabilities, and number of vulnerabilities on
the system component. The time for Process 3 is a function of
the faction of exploitable vulnerabilities for a given attacker
skill level, and two constant values derived from applying
an exponential distribution to historical data. These constant
values are 30.42 (mean time between vulnerabilities) and 5.8
(mean time to create an exploit).
Finally, when the probability values (w1 , w2 , and w3 ) and
the time values (t1 , t2 , and t3 ) respective to the three processes
are obtained, the time-to-compromise a component is given by
T = t1 · w1 + t2 · w2 + t3 · w3 .
An extended version of the model proposed by McQueen
et al. [8] is provided by Leversage et al. [76]. The authors use
a variable length Markov chain (VLMC) to compute MTTC,
instead of using a CG. The VLMC is modeled as an attack
path model — a state-space model containing high level attack
paths. In this model, MTTC is given by the sum of the
mean times spent in each attack path. In addition, the authors
introduce a visibility variable (α) to account for the influence
exerted by firewalls in the value of T. The possible values for
α are defined by security experts according to the frequency
of reviews in firewall rules. The authors also introduce a rating
variable (s) to represent attacker skill level. With this variable,
the authors show how to compute the MTTC for three attacker
profiles, namely, beginner (for s = 0.5), intermediate (for
s = 0.9), and expert (for s = 1.0). As an extension to this
idea, Paulauskas and Garsva [77] propose to use range values
to represent s, instead of using singles values. This results in
higher MTTC values for all skill level profiles. The author in
[78] improve upon this approach, by estimating α based on 15
years of vulnerability data (CVSS exploitability scores) from
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
the National Vulnerability Database (NVD).
Despite the modifications introduced by Leversage
et al. [76] and the authors in [77], [78], the MTTC model is
still based on strong assumptions that have not been validated.
B. Probability-Based Security Metrics
Probability-based security metrics usually express the like-
lihood of an adversary compromising the system or the prob-
ability that the system is secure.
In [40], [73], [74], Almasizadeh and Azgomi propose the
steady-state security metric. This metric is derived from the
same model used to calculate MTSF (see Sec. III-A). The only
difference is that the this metric is represented as a transient
state rather than an absorbing state. This means that the
system can recover from a security failure state. Considering a
sufficiently long execution time, the steady-state security will
express the probability that the system stays in the security
failure state. A larger value for this metric implies less security.
In [79], Jha et al. propose the reliability metric, which
represents the probability of an adversary not succeeding in
an attack. This metric is obtained from a CTMC generated
from assigning transition probabilities to the edges of an
attack graph (for a detailed description of attack graphs, refer
to Sec. IV). Formally, the reliability of the network is the
probability that, in a sufficiently long execution time, the
Markov chain will not be in a security failure state. In case not
all transition probabilities are available, due to, for example,
lack of data about attacks, the authors propose a Decision
Markov Process approach to compute the reliability metric.
A CTMC based on attack graphs is also used by Kanoun
et al. [80] to estimate a metric called Success Likelihood (SLH)
of an attacker. Considering that several security failure states
may exist, the attack graph is first decomposed into several
subgraphs, one for each failure state. Then, each subgraph is
turned into a Markov chain. Considering a failure state as
an absorbing state, for each CTMC, SLH is calculated as
a logarithmic function of the mean time necessary for an
attacker to reach the failure state. The key feature of the
proposed model is that it takes into account both observed
and unobserved states. This means that the Markov chain
is updated whenever an elementary attack is detected by
monitoring mechanisms (e.g., IDSs). Other peculiarity of this
model is that transition probabilities are derived from the
(attacker) skill level required for executing an attack step.
Despite the interesting features of this model, the generation
of subgraphs may result in exponential state explosion.
In [81], Xiaohu et al. use a renewal stochastic process to
estimate the likelihood that an adversary exploits a randomly
selected system vulnerability. This likelihood is given by the
q metric, which is calculated when the system is in its steady
state. A lower q value indicates more security. To compute q,
the network is modeled as a vulnerability graph, in which a
vertex represents a vulnerability, and an edge indicates that the
exploitation of a vulnerability may result in the exploitation of
the other. At any given moment, each node can either be in a
secure state or a compromised state. The behavior of the nodes
is described by a series of random variables that represent
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
9
compromise rates and fix rates. After discussing the practical
applications of q, the authors perform several simulations to
validate the analytical results of their model.
Xu et al. [82] further extend this model by weakening the
strong assumptions made in [81] with respect to the indepen-
dence and exponential distribution of the random variables.
IV. M ODEL -BASED Q UANTITATIVE N ETWORK S ECURITY
M ETRICS BASED ON ATTACK G RAPHS
The concept of attack graph (AG) was originally proposed
by Phillips and Swiler [83]. An attack graph is a model
used to represent all possible ways an attacker may violate
a security policy through the exploitation of interdependent
vulnerabilities. Besides the security policy, the minimum in-
formation necessary to build an attack graph includes hosts
connectivity and vulnerabilities on hosts. There are many
tools to automatically build attack graphs for a given network
(e.g., the Topological Vulnerability Analysis (TVA) tool [9]
and the MulVAL Logic-Based Network Security Analyzer
[84]). These tools may take as input vulnerabilities discovered
by vulnerability scanners (e.g., Nessus [85]) and host connec-
tivity information provided by tools like NetSTAT [86].
In general, AGs are represented by security-related condi-
tions and exploits. An exploit is a misused vulnerability, i.e., it
includes the vulnerability and the hosts involved in the realiza-
tion of the vulnerability. Conditions are the network attributes
necessary for the execution of an exploit (preconditions) or the
network attributes that result from the realization of an exploit
(postconditions). Network attributes include information such
as trust relationship between hosts and access rights (privilege
levels) at hosts. Given the interdependence among vulnerabili-
ties, postconditions that result from the execution of an exploit
may act as preconditions for subsequent exploits. Generally,
multiple preconditions for an exploit have a conjunctive rela-
tion (AND-decomposition), while multiple postconditions for
an exploit present a disjunctive relation (OR-decomposition).
In general, an AG can be viewed as a directed graph
presenting all attack paths that take the network from an
initial (secure) state to one or more goal (compromised) states.
Depending on how nodes and edges are represented, attack
graphs can be classified as condition-oriented, exploit-oriented,
or condition-exploit-oriented [87]. Condition-oriented AGs
represent nodes as conditions, and edges as exploits. An
exploit-oriented AG is the opposite of a condition-oriented
AG, i.e., nodes are represented as exploits, and edges as con-
ditions. This type of graph is sometimes referred to as exploit
dependency graph. Some representations of exploit-oriented
AGs choose to omit the conditions, i.e., these representations
connect nodes (exploits) using unlabeled edges. Finally, the
condition-exploit-oriented AG represents both conditions and
exploits as nodes. In this case, an edge may connect a
condition to an exploit, or an exploit to a condition.
Fig. 5 shows a simplified example of condition-exploit-
oriented AG from [88]. There are two hosts in the network
represented by the graph. The security policy of this network
states that a user on host 0 should not be able to gain root
privileges on host 1, otherwise the network will be in a
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
user(0)
ftp_rhosts(0,1)
trust(0,1)
sshd_bof(0,1)
rsh(0,1)
user(1)
local_bof(1,1)
root(1)
Fig. 5. Attack Graph Illustration.
compromised state. In this figure, there are two kinds of nodes,
namely, ovals and plaintexts. Ovals contain predicates repre-
senting exploits. Plaintexts refer to predicates that represent
conditions. For instance, user(0) indicates that the attacker
has user privilege on host 0 and sshd bo f (0, 1) corresponds
to an ssh buffer overflow exploit executed from host 0 to
host 1. Note that there are two attack paths from the initial
state user(0) to the goal state root(1).
Because AGs enumerate all possible attack paths, they
may suffer from exponential state explosion. Indeed, the first
techniques proposed to construct AGs (e.g., model checking
[89], [90]) produced prohibitively large state spaces, even
for relatively small networks. To avoid this issue, Ammann
et al. [91] proposed a more compact version of attack graphs,
based on the monotonicity assumption. This assumption states
that there is no need for the adversary to backtrack. With the
monotonicity assumption, it is possible to generate manageable
AGs for large network, and compute useful security metrics
in a timely manner. Readers interested in detailed information
about the various types of AG and their applications may
refer to [92], [87], [93]. In addition, a survey of attack graph
generation and analysis techniques is provided in [94].
In the following, the main quantitative NSMs based on
attack graph are described.
A. Path Metrics
Security metrics based on the characteristics of attack paths
are regarded as path metrics [10]. This type of metric is
focused on information such as number or size of attack paths.
Phillips and Swiler [83], besides proposing the attack
graphs, also proposed the first security metric based on them,
i.e., the shortest path metric (SP), also mentioned in [72]. Ac-
cording to this metric, the network is modeled as a condition-
oriented attack graph and its security level will correspond
to the length of the smallest attack path that an adversary
can take to reach a desired goal state (which represents a
critical network asset). The intuition behind this metric is
that the whole network is no more secure then its smallest
attack path. Therefore, the longer the shortest path is, the
more secure the networks is. Formally, this metric is given
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
10
by: SP = min(l(p1 ), l(p2 ), ..., l(pk )), where each pi represents
an attack path and the function l provides the length of an
attack path. The authors defined that l(pi ) is given by the
sum of the weights of the arcs along attack path pi . They
also defined that the weight of an arc of the graph could
correspond to input parameters such as success probability of
an exploit, time to succeed on an exploit, or effort required for
executing an exploit. Given that it may be difficult to obtain
these parameters, Idika and Bhargava [10] later suggested that
the length of an attack path should be given by the number of
exploits found on the attack path. In addition, they have also
proposed to use a condition-exploit-oriented attack graph.
The major advantage of the SP metric is its simplicity,
which allows it to be computed in a fast and fully automatic
manner. On the other hand, as first pointed out by Ortalo
et al. [72] and further detailed by Idika and Bhargava [10],
this metric has several drawbacks. Among them, the most
notorious shortcoming is that this metric does not account for
the number of different shortest paths that a network may have.
As a result, a network with a unique shortest path of size 1 will
be regarded as having the same level of security as another
network with ten different shortest paths of size 1. However,
the second network is clearly less secure than the first since
an adversary will have several distinct ways of compromising
the network with a single step.
In [72], Ortalo et al. also proposed the Number of Paths
metric (NP), which has been adapted to attack graphs by
Idika and Bhargava [10]. This metric represents the number of
distinct ways an adversary can compromise a given network
asset. More specifically, this metric counts the total number of
attack paths that exist in the attack graph of a given network
[10]. A larger count implies less security, i.e., an attacker has
more chances of compromising a network if there is a larger
number of different ways to do this. In contrast to the SP
metric, the NP metric provides a more thorough information
about the overall security of a network. However, it does not
take attack effort into account. This means that two networks
with the same number of attack paths will be regarded as
having the same level of security, even if one network has
attack paths that require considerably more effort than the
attack paths of the other network. This drawback have been
confirmed in the experiments conducted in [72].
In [10], Idika and Bhargava detail the Mean of Path Lengths
metric (MPL), which was first introduced by Li and Vaughn
in [95] as the Average Path Length metric. The MPL metric
corresponds to the arithmetic average of all path lengths that
exist within a given attack graph. This metric gives useful
information about attacks in general, since not all of the
adversaries are able to find the shortest path (or because they
try to avoid it in order to escape detection). However, given
that this metric is based on the arithmetic mean, it may not
capture certain changes in the security level of the network. In
other words, two networks with different security levels may
still be assigned with the same Mean of Path Lengths value.
Idika and Bhargava [10] realize that SP, NP, and MPL
could be combined to overcome their shortcomings. Hence,
the authors describe an algorithm to aggregate these metrics
along with a complementary suite of metrics they propose.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
The first metric proposed by them, the Normalized Mean of
Path Lengths (NMPL), is given by the ratio between MPL
and NP. This solves the issue that MPL has of not taking
into account the number of paths in a network. The authors
also propose the three following metrics, Standard Deviation
of Path Lengths (SDPL), Mode of Path Lengths (MoPL), and
Median of Path Lengths (MePL). They then divide all these
metrics into two groups, namely, decision metrics (SP, NP,
and NMPL) and assistive metrics (MPL, SDPL, MoPL, and
MePL). The decision metrics allow to determine which out of
two networks is the most secure. The assistive metrics, which
should only be used in conjunction with decision metrics, help
determine the most secure network when no conclusion could
be drawn from the application of decision metrics alone. If this
is still not enough to generate a final result, then the decision
should be delegated to the network administrator.
The aggregation algorithm proposed by Idika and Bhar-
gava solve several shortcomings of the individual metrics. In
addition, it eases the analysis of the network administrator
by producing useful results, although the algorithm may not
arrive at a conclusion in a few cases. One shortcoming of this
approach is that it does not include the effort an adversary
takes to perform an attack. In other words. the difficulty level
of exploiting a given vulnerability is not considered.
B. Non-path Metrics
As opposed to path metrics, the measurements produced by
non-path metrics do not depend on attack paths. Instead, non-
path metrics are based on more general information such as
the number of hosts that can be compromised.
In [96], Pamula et al. propose the Weakest Adversary metric,
which is based on a condition-exploit-oriented attack graph.
This metric expresses the security strength of a network by
means of the least amount of effort an attacker needs to
compromise a given network asset. The authors consider the
minimum effort expended by an attacker as the strength of
the set of initial conditions (of an attack graph) that enable
the compromise of a network. Therefore, when comparing
two networks, the less secure network will be the one with
the weaker set of initial conditions. Given two sets of initial
conditions A and B, the set A will be regarded as weaker than
the set B if A is a proper subset of B. However, in the case
that neither of the sets is a proper subset of the other, then an
alternative comparison relation can be defined by the network
administrator in order to decide which set is the weaker. For
example, if the cardinality of the sets is used to compare them,
then the set with smaller size would be regarded as the least
secure. Another option would be the assignment of a weight
to each condition of a set, according to their difficulty level.
An issue with this metric is that it provides little insight
about the actual effort required for compromising a network,
since the internal nodes of the attack graph are disregarded.
Another non-path metric is provided by Lippmann
et al. [97]. This metric, called Network Compromise Percent-
age (NCP), represents the percentage of hosts in which an
attacker is able to achieve any given privilege level. NCP
is calculated by dividing the number of hosts that can be
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
11
compromised (Nc ) by the total number of hosts. Nc can be
obtained from a condition-oriented attack graph. Intuitively, a
lower NCP value indicates a more secure network.
Most AG-based security metrics measure network security
level focusing on specific hosts (goal states). In contrast, NCP
considers that all hosts have the same influence on the network
security level. One drawback of NCP is that it does not
consider the amount of vulnerabilities on hosts. Hence, if new
vulnerabilities are discovered on a host that initially had only
one vulnerability, the NCP value will not change, even though
those new vulnerabilities have made the network less secure.
C. Probabilistic Metrics
Probabilistic AG-based metrics either take probability val-
ues as input, or produce probability values as output, or both.
1) PageRank-Based Metric: In [98], Mehta et al. use a
condition-oriented AG to compute a metric named State Rank.
This metric measures the probability of goal states occurring
(i.e., probability of the network being compromised). Specifi-
cally, the authors use the PageRank algorithm to obtain a rank
for each state of the AG. The rank of a given state corresponds
to the probability, in the long run, of that state occurring.
To compute these ranks, the PageRank algorithm needs to
receive as input from the network administrator a probability
value named damping factor (d). The authors consider that
an adversary executes the next exploit (AG arc) in a random
fashion, with each exploit having the same likelihood of being
chosen by an attacker. Hence, d is equally divided among all
the outgoing arcs of a given state. The authors also consider
that once in a while the attacker may restart the attack process
from the initial state (either because the current attack has been
detected and prevented, or because the adversary has decided
to abort the current attack). This situation is represented by
the probability 1 − d. Once the rank of all states have been
computed, the compromise probability of the network can be
given by the sum of the probabilities of all goal states.
Notice that, in general, attackers are smart and carefully
choose their exploits. Hence, the assumption that attacks are
randomly executed might not hold true.
2) Metrics Based on Independent Attack Paths: Consider-
ing that different exploits may have different chances of being
executed, Noel et al. [99] use an exploit dependency graph
to quantify network security by propagating exploit likelihood
scores from initial exploits to the goal exploit. In this approach,
each exploit has an individual score as well as a cumulative
score. The individual score of each exploit is given as input
to the model, and represent the conditional probability of
the exploit occurring when all its preconditions are already
satisfied. In contrast, a cumulative score is a probability value
computed by combining the scores of immediately preceding
exploits, according to the type of relationship that exists
between them (i.e., conjunctive or disjunctive relationship).
When an exploit B is only preceded by a single exploit A,
then it is said that they have a conjunctive relationship, i.e., B
can only occurs if A also occurs. In that case, the cumulative
score of B is given by the product of the individual scores
of A and B. Then, supposing that B also has a conjunctive
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
relationship with an exploit C (with B preceding C), the
cumulative score of C is given by the product of the cumulative
score of B and the individual score of C.
On the other hand, in the case that an exploit Z can only
be realized if either exploits X or Y occur, then it is said
that Z has a disjunctive relationship with X and Y . In such
case, the cumulative score of Z is given by the product of the
individual probability of Z, and the individual probability of
X or Y occurring, i.e., p(Z) · p(XorY ) = p(Z) · (p(X) + p(Y ) −
p(X)· p(Y )).
Once the cumulative scores of all exploits of the graph have
been computed, the security level of the whole network will
correspond to the cumulative score of the goal exploit of the
attack scenario represented by the attack graph.
One strength of this model is that the cumulative proba-
bilities can be easily calculated, provided that the graph has
been populated with all individual probability scores. The
authors state that individual scores can be obtained from online
vulnerability databases or empirical data from real networks.
One drawback of this model is that the authors do not provide
any further details on how to handle cycles in the AG.
An extended version of the cumulative score metric [99] is
provided by Wang et al. in [88]. It is called attack graph
probabilistic (AGP) metric. Wang et al. use a condition-
exploit-oriented AG to compute AGP. Hence, both conditions
and exploits will have individual and cumulative scores. This
implies that the conjunctive relation will happen between an
exploit and its preconditions. In addition, a disjunctive relation
will take place between a postcondition and the exploits that
lead to it. The authors assume that the individual score of
every condition is 1 and that the individual scores of exploits
could be obtained by converting CVSS scores into probability
values, although they do not specify how to do this. The major
contribution of this work is the procedure the authors propose
to handle cycles in the attack graph.
Both AGP [88] and cumulative score [99] are probabilistic
instances of the general security metrics framework [100],
[101] proposed by the same authors. The framework states
that a security metric should be designed to take into account
that: (1) some exploits are easier to execute than others, (2)
the execution of an exploit may make the execution of another
exploit easier, (3) the existence of more attack paths make the
network less secure, and (4) a network is no stronger than
its weakest part. The framework also states that the exploits
should be assigned with scores that express their resistance
level (i.e., difficulty level) and that these scores should be
combined by means of conjunctive and disjunctive operators in
order to compute the final cumulative resistance score. Then,
based on this framework, the authors show an example of
Attack Resistance metric in which the individual resistance
scores of exploits are represented by real numbers. In this
case, the conjunctive operator will correspond to the sum of the
individual scores, while the disjunctive operator will evaluate
to the reciprocal of the sum of the reciprocal of individual
scores. In a second example, the authors show that their
proposed framework can also be viewed as a generalization
of the weakest adversary metric [96].
The major disadvantage of the three aforementioned proba-
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
12
bilistic metrics (i.e., [99], [88], [100]) is that they assume that
the multiple attack paths to a critical asset are independent
from one another, which is not true in most cases, since they
share common exploits.
3) Metric Based on D-separation: Considering that the
attack paths that lead to the goal state of an AG are indeed
interdependent, Homer et al. [102] propose the enhanced
cumulative score metric. This metric is computed based on
the concept of d-separation, which is employed to take ad-
vantage of conditional independencies between sets of nodes.
To understand this proposal, consider the simplified example
of condition-exploit-oriented attack graph from [102] that is
depicted in Fig. 6. Condition c1 corresponds to the adversary’s
initial state, and condition c3 refers to the goal state. The
execution of exploit e1 takes the adversary from c1 to c2 .
From c2 , the adversary can achieve the goal state c3 by
launching any of the exploits e2, ..., e5 . As an example, suppose
that the success probability of each exploit is 0.5 (i.e., the
conditional probability that the exploit is launched given that
its preconditions have been satisfied). Considering that the
the attack paths share dependencies, the probability that the
adversary successfully reaches c3 can be computed as follows:
P(c3 ) = P(c3 |c2 ) · P(c2 ) = (1 − 0.54 ) · 0.5 = 0.47. On the other
hand, if all the attack paths that lead the attacker from c1 to c3
are regarded as independent, as suggested by the probabilistic
security metrics previously presented, then the following cu-
mulative score should be achieved: P(c3 ) = 1 − 0.754 = 0.68.
This deviation could be even greater in larger attack graphs
with a huge number of attack paths.
e2
0.5
e3
0.5
e1 C2
C1 0.5
C3
e4
0.5
e5
0.5
Fig. 6. Example of attack graph with exploit probabilities.
In addition to accounting for conditional relationships
among attack paths, Homer et al. also address cycles in
AGs. Moreover, they suggest that the success probabilities of
exploits that are context-specific should be provided by the
network administrator, while the others should be computed
from the Access Complexity (AC) scores from CVSS. One is-
sue with this approach is that the AC score alone is insufficient
for obtaining these probabilities, since there are other factors to
be considered. For example, availability of exploitation codes,
number of authentications required, etc.
D. Bayesian Network-Based Metrics
A Bayesian Network (BN), also known as belief network,
can be defined as a directed acyclic graph (DAG) with nodes
representing variables of a system and edges representing
causal relationships among these variables [103]. In a BN,
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
each node is assigned with a Conditional Probability Table
(CPT). Hence, an attack graph annotated with values indicat-
ing the conditional probabilities of each vulnerability being
exploited can be modeled as a BN.
The probabilistic security metrics previously presented can
be regarded as special cases of BNs. However, unlike BN-
based metrics, they do not allow to update nodes’ probabilities
in the face of evidences that other nodes have been compro-
mised (attack detections). Moreover, other more complex re-
lationships that are encoded in the CPTs of BN models cannot
be represented by the aforementioned probabilistic approaches.
Thus, the advantage of BNs is that they provide a formal
technique to make inferences under conditions of uncertainty
and they are suitable for real-time security evaluation.
1) Metric Based on OR-Decomposition: In [104], Liu and
Man generate a BN from a condition-oriented AG. In this
BN, each node is a Bernoulli random variable representing
the state of a condition, i.e., True (T) or False (F). Each
arc (exploit) e is assigned with a probability of success
p(e). These probabilities are used to compute the CPT for
each node (condition), following an OR-decomposition. As
an example, assume that a node C has two parent nodes A
and B. Consider also that node C is connected to A through
exploit e1 and to B through exploit e2 . Tab. I shows the CPT
of node C. To compute the conditional probabilities of the
last line of the table, Liu and Man use the noisy-OR operator,
i.e., p(e1 ∪e2 ) = 1−[(1−p(e1 ))·(1−p(e2 ))]. Note that besides the
probability of exploits, the prior probabilities of all root nodes
(initial states) must also be provided. In the given example, the
prior probabilities should be assigned for nodes A and B.
TABLE I
CPT SPECIFICATION FOR NODE C.
C
A B P(C = T | A, B) P(C = F | A, B)
F F 0 1
T F p(e1 ) 1 − p(e1 )
F T p(e2 ) 1 − p(e2 )
T T p(e1 ∪ e2 ) 1 − p(e1 ∪ e2 )
Liu and Man define the unconditional probability of the
target node (goal state) as the security metric of the network,
(i.e., the probability that the target node gets compromised).
Given the prior probabilities of the root nodes and the CPTs
of all non-root nodes, Bayesian inference techniques can be
used to compute the unconditional probabilities of all non-
root nodes, including the target node. In the example BN
given above, the unconditional probability of target node C,
i.e., P(C = T), would represent the network security level.
In addition, it is also possible to calculate the posterior
probability (which corresponds to the updated unconditional
probability) of node A being compromised given an evidence
that node C has been compromised, i.e., P(A = T |C = T). This
backward belief propagation can be useful, for example, to find
out the most probable attack path followed by the attacker,
given that a monitoring tool has detected a malicious event
in the target node. A forward propagation is also possible,
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
13
i.e., to calculate the posterior probability of the target node
being compromised given that a malicious event has been
detected in one or more non-target nodes. Another interesting
characteristic of this approach is that only the exploit prob-
abilities and prior probabilities need to be given as input to
the model (instead of every individual CPT value). However,
these probabilities have to be defined by security experts.
2) CVSS-Based Metrics: Frigault and Wang [105] also
propose a BN-based security metric. Differently from Liu
and Man [104], Frigault and Wang generate the BN from a
condition-exploit-oriented AG and assign success probabilities
to the nodes rather than to the arcs. Another difference is that
the proposed BN model handles not only OR-decompositions
but also AND-decompositions. The model can also handle
other more complex relationships among nodes such as the
case in which a root node can have its prior probabilities
influenced by other nodes. The authors also propose to derive
the success probabilities of exploits from the CVSS standard.
However, they do not specify how this could be done.
In [106], Frigault et al. extend their previous model by using
a dynamic Bayesian network (DBN) in order to address the
temporal aspects of vulnerabilities, such as the development of
new exploitation codes or the release of patches. The authors
also specify that the success probability of exploits should
be generated by dividing the base and temporal CVSS scores
by 10. Once these probabilities are computed, it is possible
that they change as time goes by. In this case, the use of
DBNs comes in handy. This is because in the DBN-based
model proposed by the authors, the network is represented as
a series of BNs, each representing a distinct instant of time. In
this DBN, if the temporal CVSS scores are modified, then the
success probabilities of exploits can be updated accordingly.
In spite of these improvements, the two models proposed by
Frigault and colleagues have not been designed to update the
CPTs when attacks are detected by monitoring mechanisms.
Another BN-based model is proposed by Poolsappasit
et al. [107]. Their model is similar to that of Liu and Man
[104]. One difference is that, like in [105], the CPTs of
nodes can be computed from OR-decomposition and AND-
decomposition. In addition, the success probabilities of ex-
ploits are derived from three CVSS base scores, namely, access
vector, access complexity, and authentication. Another contri-
bution of this work is that it takes into account two additional
kinds of uncertainty (in the CPTs). The first uncertainty is
related to the existence of an attacker. To express this uncer-
tainty, the authors add (root) nodes that represent the attackers,
and their prior probabilities (which are subjectively provided
by the network administrator) indicate the probability that the
attackers are present in the network. The second uncertainty is
related to the possibility that zero-day vulnerabilities exist in
the network. This uncertainty can be expressed by assigning
non-zero probabilities in the CPT of a node for the case in
which all its parents are false.
The work proposed by Xie et al. [108], which is a BN model
based on a condition-exploit-oriented AG, considers two ad-
ditional types of uncertainty (besides the ones considered by
Poolsappasit et al.). These uncertainties are: the error rates of
the IDS (false positive and false negative rates) and the leaky
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
parameters. One example of leaky parameter is the probability
that the attacker gains network access to a specific host by
circumventing the firewall. A leaky parameter can also be used
to represent the chance of zero-day vulnerabilities occurring,
similar to what is done in [107]. Thus, the authors suggest that
when building the CPTs of nodes, the conditional probabilities
should be set up in order to account for the possible existence
of such leaky parameters. The other uncertainty considered by
the authors, the IDS error rates, is addressed by the inclusion
of two nodes, one representing the existence of an attack and
the other representing an IDS. If the attack existence node is
modeled as parent of the IDS node, then the CPT of the IDS
node will coincide with the error rates of the IDS.
As for the input parameters required by this model, the
probabilities of exploits are derived from the CVSS, while the
other uncertainties (e.g., leaky parameters, attacker existence
probability) have to be provided by the network administrator.
In addition, some exploit probabilities that cannot be obtained
from CVSS scores have to be manually provided by the
network administrator as well. The results of the experiments
conducted by the authors show that their model has high ac-
curacy in estimating the desired security metric (compromise
probabilities), even in the face of the several uncertainties
considered. In addition, the experiments have also shown that
the compromise probabilities provided by the model are not
significantly effected by slight inaccuracies in the CPTs values.
3) Metric Based on Subgraphs: A different BN-based ap-
proach is provided by Dantu et al. [109], [110], [111]. In their
approach, the authors propose to model exploit dependency
graphs as BNs to estimate the compromise probability of
critical assets taking into account different attacker profiles.
The authors believe that attackers with different characteristics
(e.g., skill level, available money) will follow different attack
paths. Hence, each attacker profile is manually associated with
a specific subgraph. The model takes attack evidences, attacker
profiles, and the CPT values of each exploit as input. One
major concern is that all CPTs values have to be manually
specified, based on expert knowledge. Such manual activity
may become infeasible in large networks, making it impossible
to apply the BN-based model proposed by the authors.
E. Other Security Metrics Based on Attack Graph
This section presents other types of attack graph-based
network security metrics.
1) Metric Based on Attack Difficulty Level: In [112], Ghosh
and Bhattacharya use a condition-exploit-oriented AG to mea-
sure network security level in terms of the expected difficulty
that a random adversary may encounter while attempting to
compromise the network. To compute the expected difficulty
metric, the authors assume that the attacker chooses exploits
at random and that every security condition of the graph
is equally likely to be the postcondition of a given exploit.
Based on these assumptions, the authors propose an equation
to calculate the expected number of exploits necessary to reach
goal states. This number of exploits is then multiplied by the
average difficulty level of exploits in order to obtain the desired
metric. The authors state that the difficulty level of an exploit
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
14
should be a real number derived from CVSS scores, rather than
a probability, as commonly proposed by other works. However,
the authors do not specify how to derive these real values.
2) Desirability-Based Metric: In [31], Tupper and Zincir-
Heywood propose the VEA-bility security metric. The VEA-
bility score is expressed by a value in the range [0, 10] that rep-
resents the desirability of network configurations. This means
that a higher VEA-bility score implies more security. The VEA
acronym in the name of this metric stands for Vulnerability,
Exploitability, and Attackability — the three sub-metrics that
compose the overall VEA-bility metric. Each of these three
sub-metrics are first computed for each individual host, then
for the whole network. Finally, their values are combined in
order to obtain the overall VEA-bility score.
The first sub-metric, the vulnerability score (V) of the
network, is a value in the range [0, 10] given by V =
min(10, ln evh ), where vh is the vulnerability score of a
Í
host. In turn, vh corresponds to the exponential average of the
severity scores of the vulnerabilities that exist on that host, or a
maximum of 10. The severity score of a vulnerability is given
by the arithmetic mean of the CVSS impact and temporal
scores of this vulnerability.
The second sub-metric, the exploitability score (E) of the
network, also in the range [0, 10], is given by the sum of
the exploitability scores of the network hosts. In turn, the
exploitability of a host is given by multiplying the ratio of
network services available on that host by the exponential
average of the CVSS exploitability scores of all vulnerabilities
on that host, or a maximum of 10.
The third sub-metric, the attackability score (A) of the
network, is given by the sum of the attackability scores of
the hosts (Ah ). In turn, Ah = 10 · na /n p , where na is the
number of attack paths that lead to this host in the underlying
condition-oriented AG, and n p is the total number of existing
network paths. Finally, the overall score is given as follows:
VEA-bility = 10 − ((V + E + A)/3).
The authors provide the results of their experiments on small
network scenarios, which show that this metric accurately rep-
resents the security level of different network configurations.
Nonetheless, this metric may not be suitable for evaluating
more complex network configurations, since the VEA-bitlity
score does not take into account the interdependencies among
different vulnerabilities when combining the CVSS scores.
3) Metrics for Zero-Day Attacks: In [113], [114], Wang
et al. propose a security metric to quantify the resilience of
networks against zero-day attacks. To do so, the authors extend
a traditional condition-exploit-oriented AG to include zero-
day exploits. The proposed metric, k-zero day safety, counts
the minimum number of distinct unknown vulnerabilities that
are needed for compromising critical network assets. A greater
count indicates more security. One limitation of the k-zero day
safety metric is that it requires that the network administrator
specifies an equivalence relation to compare zero-day exploits.
In [115], [116], Zhang et al. propose two security metrics to
measure the resistance of networks against zero-day attacks,
based on the diversity of network resources (e.g., services,
firewalls). To do so, the authors use a modified version of
a condition-exploit-oriented AG, named resource graph. This
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
graph models zero-day attacks rather than known vulnerabil-
ities [117]. The idea behind the proposed metrics is that a
higher resources diversity level makes a network more secure.
The first proposed metric, d2 -Diversity, measures the min-
imum effort needed by an adversary to compromise critical
network resources. This effort is given by the ratio between
the smallest amount of different resources on a path, and the
smallest amount of steps on a path. Note that the most diverse
minimum path has a ratio equal to 1. To compute this metric, a
similarity function to determine the level of similarity between
resources needs to be provided by the network administrator.
Given that d2 -Diversity is focused on the minimum path, it
provides only a partial picture of network security.
To provide a thorough picture of network security, the other
metric proposed by the authors, d3 -Diversity, measures the
average effort an attacker needs to make for compromising
critical resources. This metric is given by the ratio between
the probability that the target asset is compromised, and this
same probability but with the assumption that attackers can
reuse exploits to compromise similar resources. To compute
this metric, the authors transform the resource graph into a
Bayesian network that contains special nodes that directly
influence the conditional probabilities of resource instances
of the same type. A different special node is included for
each shared resource type of the graph. These special nodes
represent the adversary’s capability of exploiting similar re-
sources of a given type. The input parameters required for
computing this metric include a function for determining the
similarity between resources, prior probabilities of exploiting
each resource type, and conditional probabilities of exploiting
resources instances.
The extensive simulations carried out by the authors have
shown that the accuracy of the proposed metrics is satisfactory
in most cases.
V. OTHER M ODEL -BASED N ETWORK S ECURITY M ETRICS
This section describes security metrics based on models
other than attack graphs and stochastic processes.
A. Biodiversity-Based Metric
In [115], [116], Zhang et al. proposes a metric named
d1 -diversity. This metric is inspired on biodiversity metrics
borrowed from the ecology field. Its objective is to measure the
resilience of networks against zero-day attacks. Specifically,
this metric assumes that a more diverse network is more
resistant to zero-day attacks. It also assumes that all network
resources have the same level of importance, instead of focus-
ing on specific critical assets. Hence, the measure provided by
d1 -Diversity corresponds to the percentage of different existing
network resource instances. The computation of this metric
takes into account the degree of similarity among different
resource instances as well as their irregular distribution among
the various resource types. As with d2 - and d3 -Diversity, this
metric also requires the network administrator to provide a
resources similarity function. Although d1 -Diversity can be
used to quantify overall network security, it does not capture
the relationship among network resources, given that it is not
based on an attack graph.
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
15
B. Bayesian Network-Based Metrics
In [118], Zounouz et al. propose an online security analysis
framework called Seclius. This framework uses IDS alerts
to estimate the probability that critical network assets have
been compromised by the detected intrusions. The higher this
probability is, the more insecure the network is. In contrast
to vulnerability-based metrics, which try to predict security
according to the possibility of future vulnerability exploitation,
Seclius is regarded as a consequence-centric metric. This
means that Seclius analyses security focusing on the damage
already caused by intruders. To capture the dependence be-
tween network assets, Seclius employs a dependency graph,
which is a Bayesian network automatically constructed during
an initial training phase. When an IDS alert is received, Seclius
computes the probability that the security of the network
has been affected by using a belief propagation algorithm to
combine the dependency graph with a consequence tree. A
consequence tree is a fault tree manually built by network
administrators to represent the most critical network assets
in the organization. In summary, Seclius calculates overall
network security using a logical propagation of the individual
probabilities that network assets have been compromised.
Seclius also accounts for IDS error rates.
It is important to highlight that the dependency graph
used by Seclius is different from attack graphs. While AGs
represent the relation among vulnerabilities, the dependency
graph is used to represent the information flow among ob-
jects (e.g., files and application processes). Furthermore, the
dependency graph has an additional feature of being able to
automatically compute the conditional compromise probabili-
ties (CPTs) of its nodes, based on the information flow among
represented objects. In [119], a prototype of the proposed
solution is implemented and validated in a real testbed system.
In [120], Chatzipoulidis et al. propose a high-level system
abstraction model with the aim of measuring the risk of
zero-day vulnerabilities in terms of the damage that can be
caused on system information and resources. The proposed
model is represented as a Bayesian network comprised of
five nodes. These nodes respectively correspond to five risk
elements, namely, vulnerability (V), threat (T), damage on
confidentiality (Dc ), damage on integrity (Di ), and damage
on availability (Da ). Considering that a higher number of
vulnerabilities may increase the effectiveness of threats, the
authors propose a method to compute the prior probability
that a given number of vulnerabilities will exist, i.e., P(V).
This vulnerability prediction is performed by means of a
distribution fitting procedure. CVSS scores and historical data
about the occurrence and impact of known vulnerabilities are
used as input to the prediction model. These data are also used
to compute the CPTs of the other nodes of the BN. Once P(V)
and the CPTs are obtained, the unconditional probabilities of
the damage nodes are computed, i.e., P(Dc ), P(Di ), P(Da ).
With this information, the network administrator can be aware
of the damage risk level of the system. To illustrate the use of
this model, the authors show an example of its implementation
on the e-baking sector.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
C. Tree-Based Metrics
In [121], the mean privacy metric is proposed by Alma-
sizadeh and Azgomi. This metric quantifies security by esti-
mating the amount of private network information that can be
leaked to attackers. Examples of private information of interest
for an attacker are system topology, network vulnerabilities,
open ports, available services, among others. Mean privacy
values are non-negative and are given in units of bits. A
smaller mean privacy value indicates more security. Note that
a value of zero indicates a totally secure network. The mean
privacy metric is based on an information theoretic model and
uses an attack tree labeled with probability mass functions that
represent the evolution of the attack process. Accordingly, this
metric can be described as a weighted average of uncertainty
of distinct attack paths. More specifically, the metric value is
given by the entropy of the attack scenario described by the
attack tree. To compute the mean privacy, initially, the entropy
is computed for the leaf nodes of the tree (initial attacker
actions). Then, a recursive algorithm computes the entropy for
all internal nodes (partial attack steps), until reaching the root,
which represents the security compromise state. The value
computed for the root corresponds to the Mean Privacy. One
disadvantage of the proposed model is that attack scenarios
have to be manually identified to generate the attack tree.
In [122], [123] Sahinoglu propose the Security Meter, a
decision-tree model to measure network security risk based
on risk factors like vulnerabilities, threats, and (lack of)
countermeasures. In this model a probabilistic approach is
used to compute the final metric value as a probability or
percentage indicating how insecure the network is. More
specifically, Security Meter organizes the vulnerabilities of
the network in a decision-tree according to their probability
of existence. Possible threats for each vulnerability are also
included in the tree. Each threat has a probability of exploiting
a specific vulnerability. The third component of the tree refers
to the countermeasures used to defend the network against
attacks. Each countermeasure has a probability of failing
to mitigate each given threat. From the decision tree, it is
possible to compute, for each <vulnerability,threat> pair, the
residual risk that remains after the application of the respective
countermeasure. The residual risk is given by the product of
the three aforementioned probabilities, and the final network
risk is given by multiplying the criticality factor (importance
level) of the network by the sum of all residual risks. The
criticality factor is a subjective value that has to be provided
by the network administrator. In [123], the authors propose a
method to obtain the probabilities of vulnerabilities, threats,
and countermeasures from empirical data. However, given the
difficulty to obtain real attack data, the authors conclude that
technology still needs to advance so that accurate empirical
data can be obtained.
One advantage of Security Meter is its generality, which
enables it to measure network security taking into account not
only remote but also physical threats. One disadvantage of this
model is that the decision-tree has to be manually constructed
by the network administrator, who has to know the specific
vulnerabilities, threats, and countermeasures of the network,
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
16
as well as their respective probabilities.
D. Metrics Based on Vulnerability Severity Scores
In [124], Abedin et al. introduce the Policy Security Score
(PSS) metric. This metric evaluates to a value in the range
[0, 10] that indicates the security of a network in terms of the
(CVSS) severity scores of current and past vulnerabilities. To
compute PSS, the authors first compute the severity score x of
the network as the weighted sum of two vulnerability severity
sub-metrics, namely, existing vulnerability measure (EVM)
and aggregated historical vulnerability measure (AHVM). Ac-
cordingly, PSS will correspond to a monotonically decreasing
function of x. This means that a severity value of x = 0
will correspond to a PSS score of 10. The control parameter
required by the monotonically decreasing function, and the
weight assigned to each sub-metric have to be subjectively
defined by the network administrator.
The EVM sub-metric measures the risk posed by the ex-
isting vulnerabilities in the network. To compute this metric,
it is necessary to separate the existing vulnerabilities into two
groups. The first group, EVU , refers to the vulnerabilities that
have no solutions or patches available at the present moment,
and the second group, EVS , corresponds to vulnerabilities
that already have known solutions available. For each of
those groups (EVU and EVS ), the exponential average of the
severity scores of vulnerabilities is taken. Then, it is possible to
calculate EVM, which is given by the weighted sum of the two
exponential averages. The weight of each average considered
in the computation of EVM has to be subjectively informed
by the network administrator, according to the risk posed by
each vulnerability group (EVU and EVS ).
AHVM, the other sub-metric that composes the PSS metric,
measures how vulnerable the network has been in the past so
as to predict new vulnerabilities. Precisely, a network with a
higher AHVM is more likely to have unknown vulnerabilities
in the present. To compute AHVM, the past vulnerabilities of
the network are split into three sets, according to the level of
risk they pose (high, medium, and low). Each vulnerability is
assigned with a weight that is specific to the set in which the
vulnerability is included. Then, in order to account for the age
of the vulnerabilities, an exponential decay function is applied
to their severity scores, such that the vulnerabilities that have
been discovered more recently have a higher influence in the
value of AHVM. Once the decay function has been applied,
the resulting severity scores are combined in order to obtain
the AHVM score.
In [125], Ahmed et al. propose the probabilistic vulnerabil-
ity measure (PVM). This metric estimates the risk that may
be faced by a network in the time coming. The PVM metric
is given by the exponential average of the expected risk (ER)
of each service (si ) of the network. The expected risk of a
given service (E Rsi ) is given by the product of two values, Psi
and E[X si ]. The first value, Psi , denotes the probability that
a vulnerability on service si will be discovered over a given
period of time T. A probability distribution function needs to
be defined in order to determine Psi . The second value, E[X si ]
corresponds to the expected severity score that a vulnerability
on service si will have.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
In [126], Ahmed et al. use the ER, EVM, and AHVM sub-
metrics to derive the Attack Propagation (AP) metric. The
AP metric is a real number that expresses the effect of the
propagation of an attack through the hosts of the network.
Considering D as the set of all hosts that can be reached from
outside the network, the AP metric is given by the following
equation, AP = d ∈D P(d) · SBEd . The P(d) term represents
Í
the probability that a vulnerability is present at host d, and
can be derived from the ER sub-metric. SBEd refers to the
Service Breach Effect, and denotes the effect of an attacker
at host d reaching every other accessible host. The authors
compute the value of SBEd by using a Service Connectivity
Graph — a directed graph that shows how several services
(edges) connect different hosts (nodes) — to combine values
of several distinct input parameters. Such parameters include
EVM, AHVM, protection level of firewalls, false negative rate
of the IDS, and the effect of compromising a host d.
In [126], the authors implement a tool to compute in real-
time the EVM and AHVM sub-metrics proposed by Abe-
din et al. [124] and the PVM metric proposed by Ahmed
et al. [125]. Despite the availability of an automated tool,
it may still be difficult to apply these metrics in a real
environment. This is especially because the authors do not
give any guidelines on how to define the subjective weights
that are necessary for computing the metrics and the control
parameters that are required by the aggregation functions used.
E. Metric Based on Attack Execution Graph
In [127], LeMay et al. propose the ADversary VIew Security
Evaluation (ADVISE) method to produce security metrics that
quantify network resilience against attacks. ADVISE models
an intrusion process as a series of attack steps organized
into an attack execution graph (AEG). The nodes in an AEG
represent five different attributes: attack steps, system access
domains, relevant system knowledge items, adversary skills
required to attack the system, and attack goals. The arrows in
an AEG represent a cause-consequence relationship between
the linked nodes. As opposed to an attack graph, an AEG has
Q UANTITATIVE NSM S BASED ON S TATE -S PACE S TOCHASTIC M ODELS .
Proposed Approach Metric Metric Output
Dacier et al. [70],[71] MTTF time to compromise
Ortalo et al. [72] METF effort to compromise
Almasizadeh and Azgomi
MTSF time to compromise
[40],[73],[74]
Sallhammar et al. [53], [54] MTFF time to compromise
McQueen et al. [8], [75] MTTC time to compromise
Leversage et al. [76] MTTC time to compromise
Almasizadeh and Azgomi Steady-State
compromise probability SMC, DTMC
[40],[73],[74] Security
Jha et al. [79] Reliability security probability
Success
Kanoun et al. in [80] compromise probability CTMC
Likelihood
Xiaohu et al. [81] q compromise probability Renewal Process vulnerability graph
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Communications Surveys & Tutorials
17
a higher abstraction level and includes additional information
regarding attack steps, such as timing and cost values.
ADVISE enables the specification of different attacker pro-
files (e.g., insider, criminal organization, terrorist). To assess
how a given adversary is likely to attack the system, ADVISE
uses an attack execution algorithm that combines attacker
profiles with an AEG. More precisely, this algorithm evaluates
the attractiveness of each attack step in the AEG to determine
the sequence of attack steps chosen by the adversary. The
attractiveness of each attack step is defined according to the
current model state (i.e., access domains, knowledge items,
and attack goals the adversary has achieved) and the attacker
preferences defined in the adversary’s profile. The attack pref-
erences include monetary value of each attack goal, monetary
cost and payoff of each attack step, attacker skill level, attacker
initial state, and probability that the attacker is detected.
The ADVISE method described above allows the definition
of different types of security metrics. In particular, LeMay
et al. use ADVISE to evaluate system resilience against attacks
based on the average amount of time the system remains in a
secure state. In [128], the authors developed a tool for users
who want to use ADVISE.
VI. D ISCUSSION AND O PEN I SSUES
This section presents a discussion and comparative analysis
of the surveyed model-based quantitative NSMs. Open issues
and future research directions in the field are also provided.
A. Discussion and Comparative Analysis
In the following, each of the three great groups of metrics
presented in the previous sections is individually discussed.
Then, a general discussion on all reviewed metrics is provided.
1) Discussion on the Metrics Based on Stochastic Models:
Tab. II summarizes the security metrics described in Sec. III.
This table includes metric output, analytical model(s) used,
auxiliary state-space model, and input parameters.
While state-based stochastic models have been satisfactorily
used in the dependability domain, the adaptation of these
TABLE II
Analytical
State-Space Model Input Parameters
Models
CTMC privilege graph-based success rate of elementary attacks
CTMC privilege graph-based success rate of elementary attacks
manually defined attack
SMC, DTMC probabilities of attack steps, time for attack steps
types and attack steps
CTMC, Game manually defined attack accidental failure rates, repair rates, accumulated attack steps
Theory types and attack steps intensities, reward and cost values for attackers/defenders
# of existing vulnerabilities, # of vulnerabilities on component,
# of available exploits, attacker skill level, # of exploitable
Random Process compromise graph vulnerabilities, faction of exploitable vulnerabilities, 30.42
(mean time between vulnerabilities), 5.8 (mean time to create
an exploit), 8 hours (time for process 1)
visibility variable, attacker skill rating, and all parameters used
VLMC attack path model
by McQueen et al.
manually defined attack
probabilities of attack steps, time for attack steps
types and attack steps
CTMC attack graph-based transition probabilities
attack graph-based skill level required for executing an attack step, IDS alerts
compromise rates, fix rates
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 18

techniques to the security field still presents some important
challenges. We describe some of these challenges below:
• It may be difficult to correctly model an intrusion process
by means of classical stochastic approaches. This is due to
the intentional nature of attacks, as opposed to the random
nature of accidental failures considered in dependability
models.
• Because attacks are intentional, the assumption that the
times between attacks are exponentially distributed might
not be suitable to describe attacker behavior, as discussed
in [129], [52]. Instead, the empirical analysis presented
in [129] suggests that a mixture distribution obtained
from combining an exponential distribution with a Pareto
distribution seems more promising.
• The abstraction level of network, vulnerabilities, attacks,
and defensive controls is determined by the amount of
states defined by the model. Thus, the number of states
must be carefully chosen. In particular, this number must
be sufficiently large to represent all relevant elementary
attack steps. At the same time, it must be small enough
to avoid exponential growth when applied to very large
networks (e.g., cloud computing environments [130]).
Given that stochastic processes take into account the time
variable, the security metrics presented are very useful for
predicting the evolution of the security level of networks
during their operation.
As a final note, it should be noted that although the reviewed
metrics are focused on traditional networks, there also exist a
few stochastic-based metrics proposed specifically for intru-
sion tolerant systems (e.g., [131], [132]). A system designed
to be intrusion tolerant should be able to automatically detect
and mitigate attacks such that the critical services provided
to users are not interrupted. One example of such system is
SITAR [133], an intrusion tolerant architecture for distributed
services. In [131], [132], Madan et al. propose a model to
estimate the mean time to security failure (MTTSF) metric
for SITAR.
2) Discussion on the Metrics Based on Attack Graphs:
The attack graph-based network security metrics reviewed
in Sec. IV are summarized in Tab. III. This table includes
metric output, additional models and algorithms used, type
of AGs used, and input parameters. Notice that the input
parameters necessary for creating the attack graphs have been
omitted from the table, since they are common to all AG-
based metrics. Examples of such parameters include host
connectivity, network vulnerabilities, firewall rules, available
network services, user privileges, critical assets, among others.
The following conclusions can be drawn with respect to the

TABLE III
Q UANTITATIVE NSM S BASED ON ATTACK G RAPHS .

Proposed Approach
Phillips and Swiler [83], and Idika
and Bhargava [10]
Ortalo et al. [72], and Idika and
Bhargava [10]
Li and Vaughn [95], and Idika and
Bhargava [10]
Idika and Bhargava [10]
Idika and Bhargava [10]
Pamula et al. [96]
Lippmann et al. [97]
Mehta et al. [98]
Noel et al. [99]
Wang et al. [88]
Wang et al. [100]
Homer et al. [102]
Liu and Man [104]
Frigault and Wang. [105]
Frigault et al. [106]
Poolsappasit et al. [107]
Xie et al. [108]
Dantu et al. [109], [110], [111]
Ghosh and Bhattacharya [112]
Tupper and Zincir-Heywood [31] VEA-bility
Wang et al. [113], [114]
Zhang et al. [115], [116]
Zhang et al. [115], [116]
Additional
Metric Metric Output Type of Attack Graph Input Parameters
Models
condition-oriented,
Shortest Path path length – –
condition-exploit-oriented
Number of Paths amount of paths – condition-exploit-oriented –
Mean of Path Lengths path length – condition-exploit-oriented –
Normalized mean of path
path length – condition-exploit-oriented –
lengths
Assistive metrics: SDPL,
path length – condition-exploit-oriented –
MoPL, MePL
strength of initial
Weakest Adversary – condition-exploit-oriented comparison relation of initial sets
conditions set
Network Compromise Percent- compromise
– condition-oriented –
age percentage
State Rank compromise prob. PageRank condition-oriented damping factor
Cumulative Score compromise prob. – exploit dependency graph exploit conditional probabilities
AGP compromise prob. CVSS condition-exploit-oriented exploit conditional probabilities
condition-exploit-oriented,
Attack Resistance resistance level – exploit difficulty levels
exploit dependency graph
Enhanced Cumulative Score compromise prob. CVSS condition-exploit-oriented exploit conditional probabilities
exploit conditional probabilities, root nodes
Liu and Man’s metric compromise prob. BN condition-oriented
prior probabilities., IDS alerts
Frigault and Wang’s metric compromise prob. BN, CVSS condition-exploit-oriented exploit conditional probabilities
Frigault and colleagues’ metric compromise prob. DBN, CVSS condition-exploit-oriented exploit conditional probabilities
Poolsappasit and colleagues’ exploit cond. probs., attacker existence prob.,
compromise prob. BN, CVSS condition-oriented
metric zero-day vulnerabilities prob., IDS alerts
exploit cond. probs., attacker existence prob.,
Xie and colleagues’ metric compromise prob. BN, CVSS condition-exploit-oriented
IDS error rates, IDS alerts, leaky parameters
attacker profiles, CPT values for each exploit,
Dantu and colleagues’ metric compromise prob. BN exploit dependency graph
IDS alerts
Expected difficulty difficulty level CVSS condition-exploit-oriented exploit difficulty levels
desirability score CVSS condition-oriented CVSS scores of vulnerabilities
amount of zero-
k-zero day safety – condition-exploit-oriented zero-day exploits equivalence relation
day vulnerabilities
d2-Diversity (least attacking ef- probability of di-
– condition-exploit-oriented resources similarity function
fort) versity
d3-Diversity (average attacking compromise prob- resources similarity function, prior and condi-
BN, CVSS condition-exploit-oriented
effort) ability tional compromise probabilities of resources

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
existing types of attack graph-based network security metrics:
• On one hand, the simplicity of path metrics make them
practical and cheap to apply, given that they do not
require additional input parameters. On the other hand,
path metrics, when applied alone, provide little insight
about network security level. Interestingly, the suite of
metrics and aggregation algorithm provided by Idika and
Bhargava [10] is a first step toward solving this issue.
• Non-path metrics suffer from the same issues as path
metrics. In particular, the weakest adversary metric [96]
has the additional disadvantage of requiring the definition
of a comparison relation.
• Probabilistic metrics seem to provide a natural way for
capturing the relationship among exploits and measuring
their combined effect on network security. They can
also be efficiently computed, even for large networks.
However, these metrics have not been designed to update
the network security level as attacks are detected (in real-
time) by monitoring mechanisms.
• Bayesian-based security metrics are a powerful extension
of probabilistic security approaches. They are not only
able to handle IDS inputs in order to update network
security level in real-time, but they are also capable of
capturing the effects of several uncertainties regarding
the intrusion process. Examples of uncertainties are: ex-
istence of zero-day vulnerabilities, error rates of the IDS,
etc. The challenge, however, is how to accurately obtain
information regarding those uncertainties. Moreover, the
exponential complexity of BN computations is of great
concern, especially in dynamic and large networks.
• Other attack graph-based approaches such as those fo-
cusing on zero-day attacks and network diversity are very
promising, although they still need further improvements.
3) Discussion on the Metrics Based on Other Models:
Tab. IV summarizes the security metrics presented in Sec. V,
in terms of the metrics outputs, used models and input param-
eters. Instead of using attack graphs or state-based stochastic
models, these metrics rely models such as fault trees, Bayesian
networks, decision trees, etc. Some of these approaches are
able to capture the relationship among the several dynamic
aspects that influence the security of a network, such as
TABLE IV
OTHER M ODEL -BASED Q UANTITATIVE NSM S .
Proposed Approach Metric Metric Output
Zhang et al. [115], [116] d1-Diversity % of distinct hosts
Zounouz et al. [118] Seclius compromise prob.
Chatzipoulidis et al. [120] Damage Risk compromise probability
Almasizadeh and Azgomi [121] Mean Privacy number of bits leaked
Sahinoglu [122], [123] Security Meter compromise probability decision-tree
Policy Security
Abedin et al. [124] security score
Score
Probabilistic Vulner-
Ahmed et al. [125] severity score
ability Measure
attack propagation ef- Service Connectivity Graph,
Ahmed et al. [126] Attack Propagation
fect
LeMay et al. [127] ADVISE time to compromise
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Communications Surveys & Tutorials
19
vulnerabilities, exploits, network topology. This is the case
with Seclius, mean privacy, attack propagation, and ADVISE.
On the other hand, the policy security score metric, the
probabilistic vulnerability measure, the damage risk metric,
and the security meter, although combine information about
multiple vulnerabilities, do not take into account their causal
relationships. As a consequence, those approaches are likely
to estimate security level less accurately.
4) Discussion on All Reviewed Metrics: This section pro-
vides a discussion and comparative analysis regarding addi-
tional aspects of all surveyed security metrics.
Metrics Classification. In Sec. II-B, the surveyed metrics
have been classified into the following categories: model-
based, quantitative, and network. Those categories respectively
refer to the following three criteria: construction type, mea-
surement type, target type. Tab. V presents a classification
of the surveyed metrics with regard to the four remaining
classification criteria (i.e., objective type, automation level,
measurement consistence, and measurement moment). Details
on these criteria are described in the following.
Objective Type Criteria: If a larger value of the metric
indicates more security, then the metric has been classified as
compliance, otherwise it has been regarded as non-compliance
(i.e., a larger metric value indicates less security).
Automation Level Criteria: The metrics classified as semi-
automatic are those that require the network administrator
to manually provide input parameters that cannot be easily
gathered or defined (e.g., comparison relations). In contrast,
metrics classified as automatic are those whose parameters can
all be automatically obtained (e.g., from online databases), or
can be easily provided by administrators (e.g., critical assets).
From Tab. V, it is possible to see that the majority of the an-
alyzed security metrics are semi-automatic. The problem with
this type of metric is that it requires the security administrators
to have an advanced knowledge of the network infrastructure
and the particularities of the metric model. It is important
to emphasize that because various aspects of security metrics
usually depend on personal preferences, manual intervention
is almost always necessary. However, human resources should
be preferably expended on the analysis of the metric values
instead of on their computation procedure.
Models Used Input Parameters
Biodiversity model resources similarity function
fault trees, BN IDS alerts, IDS error rates
BN, CVSS CVSS scores of vulnerabs., vulnerabs. historical data
attack trees, information theory attack probability mass functions, attack types, attack steps
decision-tree components, criticality factor, probabilities of vul-
nerabilities, threats, and countermeasures
CVSS subjective weights, control parameters of functions, CVSS scores
period of time T, expected severity of vulnerabilities, probability
CVSS
distribution function, CVSS scores of vulnerabs.
ER, EVM, and AHVM sub-metrics, protection level of firewalls,
false negative rate of the IDS, effect of node compromise, CVSS
CVSS
scores of vulnerabilities
network data for generating AEG and adversary’s profile
Attack Execution Graph
(e.g., attack step cost and payoff, detection probability)
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 20

Measurement Moment Criteria: The metrics developed with
the primary intention of being used before deployment, to
aid system design, have been classified as static. Conversely,
the metrics created specifically for quantifying security during
network operation have been classified as dynamic. One inter-
esting characteristic of dynamic metrics is that they are able
to capture the security-related events that affect security, and
adjust the computed security level accordingly.
Tab. VI presents a list of security-related events and the
corresponding dynamic security metrics that provide updated
values when these events occur. The events that cause a
modification of the graph model of a network (e.g., appearance
or removal of vulnerabilities) usually require the graph to be
totally regenerated, which may result in serious performance
issues. One interesting approach to handle such modifications
is the type of AG proposed by Lippmann et al. [97], referred
to as predictive AG. In this efficient AG, only the edges and
nodes directly affected by the fired events are removed or
added. This means that the graph is edited instead of being
completely reconstructed. Another interesting characteristic
of some of the dynamic security metrics is their capabil-
ity of handling IDS alerts. This characteristic enables them
to provide a more accurate estimation of network security
level. This is because the metric computation procedure takes
into account attacks that are actually occurring, rather than
attempting to measure security based on the possibility of
future exploitation. In addition, dynamic metrics that handle
IDS alerts are more sensitive, since attacks may occur much
more frequently than modifications to the graphs. However,
metrics that handle IDS alerts but do not update the network
graph when other events occur may also provide an inaccurate
security level. Therefore, metrics capable of handling both IDS
alerts and graph modifications, such as the Success Likelihood
metric proposed in [80], are desirable. A final remark about
dynamic metrics is that they can also be applied to measure
security in a static manner, i.e., they can help find out the most
secure network configuration to be deployed.
Measurement Consistence Criteria: Metrics based solely on

TABLE V
C LASSIFICATION OF THE R EVIEWED M ODEL -BASED Q UANTITATIVE NSM S .

Automation Measurement Measurement

Security Metric Objective Type
Level Moment Consistence
MTTF [70], [71] compliance automatic dynamic objective
METF [72] compliance semi-automatic dynamic subjective
MTSF [40], [73], [74] compliance semi-automatic static subjective
MTFF [53], [54] compliance semi-automatic static subjective
MTTC by McQueen et al. [8], [75] compliance semi-automatic static subjective
MTTC by Leversage et al. [76] compliance semi-automatic static subjective
Steady-State Security [40], [73], [74] non-compliance semi-automatic static subjective
Reliability [79] compliance automatic static objective
Success Likelihood [80] non-compliance semi-automatic dynamic subjective
q [81] non-compliance automatic static objective
Shortest Path [83], [10] compliance automatic dynamic objective
Number of Paths [72], [10] non-compliance automatic dynamic objective
Mean of Path Lengths [95], [10] compliance automatic dynamic objective
Normalized Mean of Path Lengths [10] compliance automatic dynamic objective
Assistive metrics: SDPL, MoPL, MePL [10] compliance automatic dynamic objective
Weakest Adversary [96] compliance semi-automatic static subjective
Network Compromise Percentage [97] non-compliance automatic dynamic objective
State Rank [98] non-compliance semi-automatic static subjective
Cumulative Score [99] non-compliance automatic static objective
AGP [88] non-compliance automatic static subjective
Attack Resistance [100] compliance semi-automatic static subjective
Enhanced Cumulative Score [102] non-compliance semi-automatic static subjective
Liu and Man’s metric [104] non-compliance semi-automatic dynamic subjective
Frigault and Wang’s metric [105] non-compliance automatic static subjective
Frigault and colleagues’ metric [106] non-compliance automatic dynamic subjective
Poolsappasit and colleagues’ metric [107] non-compliance semi-automatic dynamic subjective
Xie and colleagues’ metric [108] non-compliance semi-automatic dynamic subjective
Dantu and colleagues’ metric [109], [110], [111] non-compliance semi-automatic dynamic subjective
Expected Difficulty [112] compliance automatic static subjective
VEA-bility [31] compliance automatic static subjective
k-zero day safety [113], [114] compliance semi-automatic static subjective
d2-Diversity (least attacking effort) [115], [116] compliance semi-automatic static subjective
d3-Diversity (avg. attacking effort) [115], [116] non-compliance semi-automatic static subjective
d1-Diversity (% of distinct resources) [115], [116] compliance semi-automatic static subjective
Seclius [118] non-compliance automatic dynamic objective
Damage risk [120] non-compliance automatic static subjective
Mean Privacy [121] non-compliance semi-automatic static subjective
Security Meter [122], [123] non-compliance semi-automatic static subjective
Policy Security Score [124] compliance semi-automatic dynamic subjective
Probabilistic Vulnerability Measure [125] non-compliance semi-automatic dynamic subjective
Attack Propagation [126] non-compliance semi-automatic dynamic subjective
ADVISE [127] compliance semi-automatic static subjective

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
E VENTS THAT C AUSE THE E STIMATED N ETWORK S ECURITY LEVEL TO B E U PDATED .
Event
Modification of the privilege graph (e.g., new vulnerabil-
ity, new user, new attack success rate, etc.)
Modification of the attack graph (e.g., new vulnerability,
new firewall rule, new compromise probabilities, etc.)
Modification of the number of (compromised) hosts
IDS alerts
CVSS score updates
objective input parameters have been categorized as objective,
otherwise they have been regarded as subjective. Tabs. VII and
VIII present, respectively, the subjective and objective input
parameters of each metric along with the respective source.
The disadvantage of subjective parameters is that it may
be difficult to determine appropriate values for them, since
this requires deep knowledge of the metric model, attacker
behavior, and network infrastructure. The definition of these
subjective values will, consequently, demand a huge amount
of time and effort from the network administrator. Moreover,
the estimated security level will vary a lot depending on
who chooses the subjective input values. One solution to
these problems would be the use of standardized and widely
adopted frameworks such as CVSS. The use of CVSS would
also allow to consistently compare the security of different
systems. Moreover, it is possible to automate the computation
of security metrics that are based on CVSS since several
online databases maintain CVSS severity scores for numerous
existing vulnerabilities. The authors in [134] conduct attack
experiments in a real environment in order to study the cor-
relation between several CVSS-based security metrics and the
mean time necessary for compromising a network. Although
CVSS has some drawbacks, this study shows that security
metrics that use more CVSS data are able to estimate network
security level more accurately. Nevertheless, objective input
parameters are still preferable, although they might sometimes
not be available.
Metric’s Additional Features. Additional features of the
analyzed security metrics are presented in Tab. IX. These
features are described in the following.
Indication of Absolute Security: This features indicates
if the metric scale (alone) allows to determine whether a
network is totally secure. The fact that a metric possesses a
value to indicate absolute security only means that, under the
considered assumptions, there is a (hypothetical) situation in
which systems can be regarded as completely secure. However,
in practice, an absolutely secure system is not attainable.
Examples of this type of metric are those whose values are
probabilities (e.g., Success Likelihood metric). Security metrics
marked as “no” are those that only allow to evaluate system
security from a relative point of view. This evaluation can done
by either comparing different systems or different versions
of the same system. Examples of relative security metrics
are those whose values are given in time units (e.g., MTTC
metric).
Handling of Zero-Day Attacks: This feature indicates if
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Communications Surveys & Tutorials
21
TABLE VI
Security Metric
MTTF [70], [71]; METF [72]
Success Likelihood [80]; Shortest Path [83], [10]; Number of Paths [72], [10]; Mean of Path Lengths [95], [10];
Normalized Mean of Path Lengths [10]; Assistive metrics: SDPL, MoPL, MePL [10]
Network Compromise Percentage [97]
Success Likelihood [80]; Liu and Man’s metric [104]; Poolsappasit and colleagues’ metric [107]; Xie and colleagues’
metric [108]; Dantu and colleagues’ metric [109], [110], [111]; Seclius [118]
Frigault and colleagues’ metric [106]; Policy Security Score [124]; Probabilistic Vulnerability Measure [125]; Attack
Propagation [126]
the security metric takes zero-day attacks into account when
computing network security level. A zero-day attack corre-
sponds to the exploitation of a zero-day vulnerability (i.e., a
vulnerability that has not yet been publicly disclosed) [135].
Quantifying network security based on unknown vulnerabil-
ities is at the same time a challenging and important task.
Challenging because, in general, no information about those
vulnerabilities is available before the discovery of an attack.
Important because the risk imposed by zero-day attacks is
very real and seriously affects network security. For example,
attacks exploiting zero-day vulnerabilities may remain unde-
tected for a long time. Thus, security metrics that disregard
the possible existence of zero-day attacks may not accurately
portray the actual security level of a network. In other words,
a network may be regarded as being secure, while it is in fact
susceptible to zero-day attacks.
Despite that, only a few security metrics consider zero-day
vulnerabilities. Considering the details provided in Secs. III,
IV, and V, the works that handle zero-day attacks can be
organized intro three groups. The first group refers to the
works that use distribution fitting procedures to make pre-
dictions about the existence and impact of unknown vulner-
abilities. In general, the predictions are based on currently
available data about known vulnerabilities. MTTC, Damage
Risk, PSS, PVM, and Attack Propagation are included in this
group. The second group includes k-zero day safety, d1−,
d2−, and d3−diversity. Those metrics assume that zero-day
vulnerabilities are already present in every network service.
This can be modeled, for example, by including hypothetical
zero-day vulnerabilities in attack graphs. Finally, the third
group includes MTFF, q, Poolsappasit and colleagues’ met-
ric, and Xie and colleagues’ metric. Those metrics assume
that probability values describing the existence of zero-day
attacks (e.g., the leaky parameters described in [108]) can be
computed elsewhere and be given as input to the models.
Handling of Vulnerabilities Dependency: This feature indi-
cates if the metric model is capable of taking into account
the interdependencies among vulnerabilities. This capability
is important because attacks are rarely performed by the ex-
ploitation of isolated vulnerabilities, as indicates the empirical
analysis conducted in [134].
Handling of Defensive Actions: This feature indicates if the
security metric model takes into account the defensive actions
performed by intrusion detection and response mechanisms.
Defensive behavior of the system can be either encoded in data
captured off-line, or provided during runtime by the deployed
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 22

TABLE VII
S OURCE OF S UBJECTIVE PARAMETERS OF THE R EVIEWED M ODEL -BASED Q UANTITATIVE NSM S .

Security Metric Subjective Parameters and Respective Source

METF [72] success rates of elementary attacks from security experts
MTSF [40], [73], [74] attack types and attack steps from network administrator
MTFF [53], [54] attack types, attack steps, reward and cost values for attackers/defenders from network administrator
information about vulnerabilities from online databases; information about exploits from metasploit website; attacker skill
MTTC by McQueen et al. [8], [75]
provided by the authors; predefined constant values given by the authors
visibility variable provided by security experts; attacker skill rating provided by the authors; other parameters are the same from
MTTC by Leversage et al. [76]
the model of McQueen et al.
Steady-State Security [40], [73], [74] attack types and attack steps from network administrator
Success Likelihood [80] skill level required for executing an attack step from network administrator
Weakest Adversary [96] comparison relation of initial sets provided by network administrator
State Rank [98] damping factor provided by network administrator
AGP [88] exploit conditional probabilities from CVSS or security experts
Attack Resistance [100] exploit difficulty levels from security experts
Enhanced Cumulative Score [102] exploit conditional probabilities from CVSS and network administrator
Liu and Man’s metric [104] exploit conditional probabilities and prior probabilities from security experts and online databases
Frigault and Wang’s metric [105] exploit conditional probabilities from CVSS
Frigault and colleagues’ metric [106] exploit conditional probabilities from CVSS
Poolsappasit and colleagues’ metric [107] exploit conditional probabilities from CVSS; prob. of attacker and zero-day vulnerability existence from network administrator
Xie and colleagues’ metric [108] exploit conditional probabilities from CVSS and network administrator; other uncertainty values from network administrator
Dantu and colleagues’ metric [109], [110], [111] attacker profiles and CPTs for each exploit from security experts
Expected Difficulty [112] exploit difficulty levels from security experts or CVSS
VEA-bility [31] vulnerability scores from CVSS
k-zero day safety [113], [114] zero-day exploits equivalence relation from network administrator
d2-Diversity (least attacking effort) [115], [116] resources similarity function from network administrator
d3-Diversity (avg. attacking effort) [115], [116] resources similarity function from network administrator
d1-Diversity (% of distinct resources) [115], [116] resources similarity function from network administrator
Damage risk [120] vulnerability scores from CVSS; vulnerabilities historical data from online databases
Mean Privacy [121] attack probability mass functions, attack types and attack steps from network administrator
Security Meter [122], [123] criticality factor and decision-tree components from network administrator
Policy Security Score [124] vulnerability scores from CVSS; weights and control parameters of functions from network administrator
period of time T and probability distribution function from network administrator; expected severity of vulnerabilities and
Probabilistic Vulnerability Measure [125]
vulnerability scores from vulnerability databases and CVSS
Attack Propagation [126] protection level of firewalls and node compromise effect from administrator; vulnerability scores from CVSS
ADVISE [127] network data and adversary’s profile from security experts and network administrator

TABLE VIII
S OURCE OF O BJECTIVE PARAMETERS OF THE R EVIEWED M ODEL -BASED Q UANTITATIVE NSM S .

Security Metric Objective Parameters and Respective Source

MTTF [70], [71] success rate of elementary attacks from statistical attack data provided by IDSs
MTSF [40], [73], [74] probabilities and times of attack steps from honeypots, intrusion experiments, or real attack data
MTFF [53], [54] accumulated attack steps intensities from honeypots/intrusion experiments; other rates from real data
Steady-State Security [40], [73], [74] probabilities of attacks and times for attacks from honeypots, intrusion experiments, or real attack data
Reliability [79] transitions probabilities from empirical data
q [81] fix/compromise rates from historical data
Cumulative Score [99] exploit conditional probabilities from empirical data
Xie and colleagues’ metric [108] IDS error rates from historical data
d3-Diversity (avg. attacking effort) [115], [116] prior and conditional compromise probabilities of resources from historical data
Seclius [118] IDS error rates from historical data
Security Meter [122], [123] vulnerability, threat, and countermeasure probabilities from empirical data
Attack Propagation [126] false negative rate of the IDS from historical data

security mechanisms. defined by the respective authors of each metric).

Focus on Critical Assets: This feature indicates if the metric
model represents the security level of an entire network by
means of the security level of one or more specific target
network resources. Security metrics marked as “no” are those
in which all network resources exert the same influence on the
security level of the network.
Handling Of Insider Attacks: This feature indicates if the
metric model accounts for the threats posed by insider attacks
(i.e., attacks from legitimate users or employees).
Definition Of Attacker Profiles: This feature indicates if the
security metric is based on attacker profiles (which are usually
Metric Validation: This feature indicates if the metric has
been validated by its respective authors, either through sim-
ulation or experiments in real environments. The validation
procedure is important because it analyzes how accurate a
metric value portrays the security status of a network.
As can be seen in Tab. IX, validation is one of the biggest
issues of current security metric proposals. Attack data from
real networks (with real attacks) are usually unavailable (due
to privacy policies of organizations). Therefore, commonly
available validation procedures include network simulation
tools and empirical experiments (performed in real environ-

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 23

TABLE IX
A DDITIONAL F EATURES OF THE R EVIEWED M ODEL -BASED Q UANTITATIVE NSM S .

Indication of Handling of Handling of Handling of Focus on Handling Definition of

Metric
Security Metric Absolute Zero-Day Vulnerabs. Defensive Critical of Insider Attacker
Validation
Security Attacks Dependency Actions Assets Attacks Profiles
MTTF [70], [71] no no yes no yes yes no no
METF [72] no no yes no yes yes no yes
MTSF [40], [73], [74] no no yes yes yes no no no
MTFF [53], [54] no yes yes yes yes yes yes no
MTTC by McQueen et al. [8], [75] no yes no no yes no yes no
MTTC by Leversage et al. [76] no yes no no yes no yes no
Steady-State Security [40], [73], [74] yes no yes yes yes no no no
Reliability [79] yes no yes no yes no no no
Success Likelihood [80] yes no yes yes yes no no no
q [81] yes yes yes yes no no no yes
Shortest Path [83], [10] no no yes no yes no no no
Number of Paths [72], [10] yes no yes no yes no no yes
Mean of Path Lengths [95], [10] no no yes no yes no no no
Normalized Mean of Path Lengths [10] no no yes no yes no no no
Assistive metrics: SDPL, MoPL, MePL [10] no no yes no yes no no no
Weakest Adversary [96] no no yes no yes no no no
Network Compromise Percentage [97] yes no yes no no no no no
State Rank [98] yes no yes yes yes no no no
Cumulative Score [99] yes no yes no yes no no no
AGP [88] yes no yes no yes no no no
Attack Resistance [100] no no yes no yes no no no
Enhanced Cumulative Score [102] yes no yes no yes no no yes
Liu and Man’s metric [104] yes no yes yes yes yes no no
Frigault and Wang’s metric [105] yes no yes no yes no no no
Frigault and colleagues’ metric [106] yes no yes no yes no no no
Poolsappasit and colleagues’ metric [107] yes yes yes yes yes no no no
Xie and colleagues’ metric [108] yes yes yes yes yes no no yes
Dantu and colleagues’ metric [109], [110], [111] yes no yes yes yes yes yes no
Expected Difficulty [112] no no no no yes no no no
VEA-bility [31] yes no yes no no no no yes
k-zero day safety [113], [114] no yes yes no yes yes no no
d2-Diversity (least attacking effort) [115], [116] yes yes yes no yes no no yes
d3-Diversity (avg. attacking effort) [115], [116] yes yes yes no yes no no yes
d1-Diversity (% of distinct resources) [115], [116] yes yes no no no no no yes
Seclius [118] yes no yes yes yes no no yes
Damage risk [120] yes yes no no no no no yes
Mean Privacy [121] yes no yes no yes yes no no
Security Meter [122], [123] yes no no yes no no no no
Policy Security Score [124] yes yes no no no no no no
Probabilistic Vulnerability Measure [125] yes yes no no no no no no
Attack Propagation [126] no yes no yes no no no no
ADVISE [127] no no yes yes yes yes yes no

ments but under controlled conditions). However, most of the
works analyzed in this survey do not perform any validation
procedure to measure the accuracy of their proposed metrics.
Instead, they only provide case studies to show how their
security metrics could be used. Only a few of the reviewed
proposals attempt to validate their metrics through simulations.
However, the attacks and network behavior in a simulation
environment may excessively diverge from real environments.
Another very small subset of the current proposed metrics
are validated through empirical experiments. In these exper-
iments, while teams of hackers try to breach the system,
several important information regarding the attacks is col-
lected so that the metrics being evaluated can be validated.
These experiments are usually very effective in determining
the validity of security metrics. However, they are usually
very expensive activities that take too much effort and time
(sometimes years) to be accomplished. Such difficulties may
hamper the validation processes. In addition, the attacker skills
and strategies employed in these experiments may also differ
from those in real attack scenarios. For that reasons, apart from
the efforts of empirical validation works like [134], [72], most
of the current quantitative model-based NSMs proposed in the
literature have not yet been validated.
Decision-making support. In Sec. I, we have listed four
important questions that can be answered by NSMs. In general,
answers to these questions help network administrators in-
crease network security level. In the following, we discuss how
the surveyed model-based NSMs can answer these questions.
Which network configuration is the most secure? Answering
this question can be regarded as the main objective of all
surveyed metrics. Considering that security metrics assign
numeric values to different network configurations, they allow
a network administrator to objectively compare these config-
urations and determine which is the most secure.
How to change network configuration in order to improve
security? The act of improving network security by making

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
configuration changes is referred to as network hardening.
Network hardening measures include activities such as dis-
abling unnecessary services, blocking unnecessary connec-
tivity, patching known vulnerabilities, deploying new IDSs,
among others. An important task of network administrators
is to decide which hardening measures should be applied
to increase network security level. The model-based NSMs
surveyed in this article are a powerful tool to support this
decision, since they can be used to rank different hardening op-
tions according to their relative effectiveness. This enables ad-
ministrators to discard hardening options that do not increase
network security or to apply only the most effective ones.
Often, administrators have a subset of available hardening
options to select from. If this subset is very large (e.g., in large
networks), the selection of suitable options can be formulated
as an optimization problem. Works in [117], [136] propose
algorithms to generate optimal hardening recommendations
with respect to a given network security metric.
Is the network achieving its required security level? Consid-
ering that it is not possible to provide absolute security for a
network, it may suffice to know that the network is secure for a
given threshold. Based on a security metric, administrators can
use their experience to define a desired network security level.
Once such level is defined, the selected security metric can be
used to check whether the underlying network is achieving the
required threshold or not.
What is the impact of ongoing attacks? Unlike the previous
three questions, this question can only be answered by security
metrics that take IDS alerts as input (see Tab. VI). These
metrics automatically update the estimated values when a new
attack occur. Hence, the impact of ongoing attacks can be
obtained by calculating the difference between the last and
the updated metric values.
Other Relevant Questions: It should be noted that the sur-
veyed metrics attempt to estimate security level by answering a
specific question, according to assumptions about attacker and
system behavior. Hence, depending on the specific questions
an administrator wants answered, he or she might need to
use a different metric. For example, to know how much time
a system takes to be compromised, the administrator should
apply a metric like MTTC. On the other hand, if he or she
wants to know the minimum amount of vulnerabilities an
attacker needs to exploit to compromise a system, then the
shortest path metric should be used.
Guidelines on Which Metrics to Use. In general, the
selection of suitable security metrics will depend on the
requirements of each organization. As previously discussed,
knowing which specific questions need to be answered can
help a network administrator choose among different available
security metrics. Additionally, the several characteristics of
the surveyed security metrics discussed in this section (see
Tabs. V, VI, VII, VIII, IX) should also be considered. In
particular, we highlight the following suggestions:
• Some metrics need input parameters defined (or com-
puted) by security experts or very experienced network
administrators (see Tabs. VII, VIII). Moreover, input pa-
rameters obtained from experiments in real environments
usually take a long time to be computed. Hence, enter-
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
24
prises that do not have security experts available or that
need measuring security level in a timely fashion should
not use these metrics. Instead, Seclius, NCP, CVSS-based
metrics, or path metrics should be used;
• If an enterprise is interested in metrics that enable more
intervention from network administrators, BN-based met-
rics are a suitable option. This is mainly because CPTs
can be used to encode subjective beliefs from adminis-
trators along with objective compromise probabilities;
• Organizations interested in measuring security based only
on critical assets, should use metrics that allow the
specification of such assets (see Tab. IX);
• Enterprises that wish to monitor network security level
during operation should consider dynamic security met-
rics (see Tab. V);
• Organizations with few security employees might prefer
automatic metrics, rather than semi-automatic metrics;
It should be noted that only a few of the works reviewed in
this survey have implemented tools or prototypes to support
the use of their proposed metrics. For example, Seclius [119],
ADVISE [128], Probabilistic Vulnerability Measure [126],
MTTF [72].
B. Open Issues
In the following, open issues regarding model-based quan-
titative network security metrics are provided.
Process for Obtaining Objective Input Parameters. All
objective security metric proposals described in this survey
emphasize the need for input parameters based on attack data
collected from honeypots, intrusion experiments, or historical
data. However, none of these proposals describe a process
for effectively obtaining such input parameters for a specific
network. In fact, the development of such a process is very
much an open issue.
Metrics Aggregation. Security is an abstract concept that
is very hard to define and, consequently, to measure. However,
the several factors that influence the security of a system pro-
vide a tangible source of information that can be used for esti-
mating the amount of security possessed by a system. Hence,
security metrics that handle a larger amount of additional
information (e.g., zero-day attacks, vulnerability dependency,
insider threats, defensive actions) may be able to provide more
accurate results. However, current model-based quantitative
NSM are not able to measure overall network security taking
into account all of these relevant factors at once. Hence, an
interesting future direction would be the development of a
scheme for combining several distinct model-based NSMs.
This scheme should consider the possible dependencies and
conflicts that may arise from the combination these metrics.
The resulting model will most likely be able to provide a more
comprehensive and accurate security level estimation.
Objective Definition of Attacker Profiles. Attacker profiles
are another valuable input information. Indeed, the attack pro-
cess is one of the factors that most impact system security. The
simple existence of a vulnerability does not necessarily imply
that a system is compromised. Security will only be affected
if the attacker is able to exploit existing vulnerabilities. For
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
that reason, network security level directly depends on the
attacker profile, i.e., attacker’s skills, motivation and strategies.
However, a sound procedure for objectively deriving profiles
based on attackers’ characteristics is still an open problem.
Effectiveness of Security Mechanisms as Input. System
vulnerabilities largely affect network security and are a inher-
ent part of all networks, including the most simple ones. For
that reason, vulnerabilities are the characteristics most used by
current model-based quantitative NSMs. However, the effec-
tiveness of security mechanisms, which is an equally important
factor that influences network security, is only addressed by a
few security metric models. Security mechanisms are impor-
tant because the greater their effectiveness is, the more difficult
will be for a threat to exploit a given vulnerability. Although a
few proposals take into account information provided by IDSs,
their effectiveness is usually disregarded, as well as the effec-
tiveness of other mechanisms (e.g., key management schemes).
The development of model-based quantitative NSMs that take
into account such information constitutes an interesting future
line of investigation.
Biodiversity-Inspired Security Metrics. Diversity is a
strong security mechanism used to defend a network against
the actions of adversaries. The work proposed by Zhang
et al. [115] presents promising results that have provided in-
sights on how to correctly apply diversity to increase network
security. Their paper is basically the first work to provide a
formal model based on network diversity to quantify security.
The success of their work is mainly because the security
metrics they propose are inspired by biodiversity metrics from
the ecology field. However, the authors have only used a small
part of the available biodiversity concepts. Hence, the devel-
opment of new diversity-based security metrics that borrow
other concepts from the ecology literature would provide even
more interesting results for the security evaluation field.
Enhanced Stochastic Models. As discussed in Sec. III,
another future research direction is the development of an
NSM based on a stochastic model which do not assume
that the times between attacks are exponentially distributed.
Instead, such model would have to allow the use of a mixture
distribution, obtained from combining an exponential distri-
bution with a Pareto distribution, which seems to be more
suitable for describing attacker behavior, as discussed in [129].
It would also be interesting to use an state space derived from
an attack graph, which can be automatically generated and
seems to be a more natural way for describing the attack steps
performed by adversaries. This model should also possess real-
time features of dynamic security metrics such as handling of
attack graph modifications and handling of IDS alerts. Finally,
this model should provide as output a probability value to
express network security level, instead of a time value. This
would allow to estimate the security of the network in an
absolute manner, not only in a relative fashion.
Enhanced Bayesian Networks-Based Models. Another
promising approach for the development of more accu-
rate model-based quantitative NSMs is the use of Bayesian
network-based models. Such models are able to handle IDS
inputs in order to update network security level during runtime.
Moreover, they are also able to capture the effects of several
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
25
uncertainties regarding the intrusion process. These uncertain-
ties include existence of zero-day vulnerabilities, error rates
of the IDS, difficulty level of exploits, etc. Therefore, an
interesting future work would be to study which additional
uncertainties influence security and how these uncertainties
could be encoded in the CPTs of the BN nodes. Moreover,
the exponential complexity of BN computations is of great
concern, especially in very dynamic and large networking
environments.
Inclusion of Additional Attack Classes. Most of currently
available model-based NSMs measure network resilience tak-
ing into account traditional attack classes (e.g., buffer overflow,
spoofing, denial of service). Therefore, one additional open
issue is the development of a security metric model that takes
into account other attack classes such as social engineering,
physical attacks, exploits of client-side applications, phishing,
and user mistakes.
Trustworthiness of Application Data for End Users.
Although most security metrics are designed to be used by
security administrators, the development of NSMs to inform
users about data trustworthiness is an area of considerable
future research. This is especially true for specific architectures
such as sensor networks [137], whose main goal is to provide
reliable sensor data for end users. In this context, the work
proposed by Ramos and Holanda Filho [33] is one of the
first efforts towards estimating the security level of data from
sensor networks. With this information, end users can take
informed decisions about the use of the received sensor data.
Security metrics of this type would also be valuable for users
of other Internet infrastructures such as cloud computing [130]
and, especially, the Internet of things (IoT) [138], whose
security has received increasing attention [139].
VII. R ELATED W ORK AND S UGGESTED R EADINGS
Additional information regarding the security metrics field
in general or specific types of security metrics can be found
in the papers listed below. In contrast to our survey, these
papers focus mostly on qualitative metrics, measurement-based
metrics, or security metrics for targets other than networks
(i.e., process, software, organization).
• A brief overview of the security metrics field along with
a few examples are provided by Stolfo et al. [12]. Several
examples of security metrics, especially measurement-
based metrics, can also be found in [21], [18], [16].
• In [28], Jansen provides future directions regarding
promising research areas that should be explored in order
to improve the development of security metrics.
• A more detailed overview of the security metrics field is
provided by Bayuk and Mostashari [11]. A classification
and review of several qualitative, measurement-based, and
process security metrics is also provided.
• Pfleeger and Cunningham [140] analyze the security
metrics field and discusses several reasons why measuring
security is hard. The authors also present several strate-
gies to overcome those difficulties in the development of
security metrics.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 26

• Boyer and McQueen [20] propose seven security ideals rity metrics. Jaatun [24] evaluates some approaches for
that should be used to guide the development of new measuring software security and makes recommendations
quantitative security metrics. regarding the use of security maturity models. Ouchani
• A self-assessment framework that helps users determine and Debbabi [25] reviews the state-of-the-art regarding
the measurement-based security metrics that are more ap- the security quantification of software modeled using
propriate for a specific system or organization is provided Unified Modeling Language (UML).
by Heinzle and Furnell [19]. • Nicol at al. [52] survey the main model-based evaluation
• Taxonomies for security metrics are provided by Vaughn techniques used to derive system dependability metrics.
et al. [14] and Savola [15]. Those works focus on process, The authors also discuss how these models and techniques
qualitative, quantitative, and measurement-based metrics. can be adapted in order to measure system security. Since
• In [18], Villarrubia et al. propose a metrics classification this work only includes metrics until 2004, our survey
scheme based on different metrics properties. Then they includes more recent papers.
use this scheme to classify several measurement-based
security metrics that exist in the literature. VIII. C ONCLUSION
• Verendel [16], and Rudolph and Schwarz [17] also pro-
Model-based quantitative NSMs are essential for the de-
pose classification schemes to classify existing security
velopment and maintenance of secure systems, since these
metrics in the literature. While Verendel focused on
metrics are able to quantify the overall resilience of networks
quantitative metrics, Rudolph and Schwarz covered both
against attacks. In this survey, a deep literature review of
qualitative and quantitative metrics. Both studies provide
the state-of-the-art of model-based quantitative NSMs has
a comprehensive discussion on the classified metrics and
been presented. Initially, an overview of the security metrics
conclude that most of the quantitative metrics in the liter-
field has been provided, along with a classification scheme.
ature lacks validation. Although both of these works also
Then, the CVSS framework, that is used as input by several
cover model-based quantitative network security metrics
security metric models, has been described. The differences
in their classification schemes, they do not provide a
between the security metrics field and other closely related
description of the metrics classified. In contrast, our
areas have also been provided. Then, more specifically in the
survey provides an in-depth description of each surveyed
realm of model-based quantitative NSMs, a comprehensive
metric. Moreover, Verendel only covers works until 2008
and detailed review of the main metric proposals has been
and Rudolph and Schwarz only cover works until 2010,
presented. The main pros and cons of each reviewed work
while our survey includes more recent papers.
have also been described. Finally, an in-depth discussion on
• In [141], Pendleton et al. provide a comprehensive sur-
the main properties of the reviewed security metrics has been
vey on quantitative system security metrics. Their work
presented, along with open issues and suggestions of future
basically includes software, network, economic, and ef-
research directions. This was followed by a discussion on past
fectiveness security metrics. Initially, the authors propose
related work.
a hierarchical ontology in order to classify the surveyed
Quantifying the overall security of networked systems is
metrics. Then, each metric is briefly described. It should
a very difficult problem [144], [140]. Consequently, there is
be noted that the authors also cover model-based quantita-
some disbelief on the feasibility of security metrics [145],
tive network security metrics. However, due to the broad
[146]. According to what has been presented in this survey, it
focus of their work, in contrast to our survey, they do not
is possible to conclude that the field of model-based quantita-
provide explanations about the technical details of each
tive NSMs is still in development and much more progress still
individual metric. Moreover, unlike our survey, they do
needs to be done. Although none of the current approaches
not provide a comprehensive discussion and comparative
is totally satisfactory, the value of model-based quantitative
analysis of the reviewed metrics.
NSMs is in supporting decision-making, rather than perfectly
• A survey of security management in industrial control
representing the security level of networks. For that reason,
systems is provided by Knowles et al. [142]. Their
the security metrics that have been proposed over the last few
survey focuses on security standards, guidelines, and best
years are of immense practical value. Moreover, the advances
practices. In contrast to our survey, the security metrics
that have already been achieved so far show that this area
addressed by Knowles et al. are mostly qualitative or
is in constant evolution and that the development of feasible
measurement-based.
security metrics is indeed possible.
• In a more specific context of control systems, Cherdant-
seva et al. [143] present a survey of security metrics
and risk assessment methods for Supervisory Control and ACKNOWLEDGMENTS
Data Acquisition (SCADA) systems. The authors review This work has been supported by Coordination for the
qualitative metrics as well as a few quantitative model- Improvement of Higher Education Personnel (CAPES), by
based metrics. However, due to their specific focus on National Funding from the FCT - Fundação para a Ciência e
metrics for SCADA systems, our survey reviews model- a Tecnologia through the UID/EEA/500008/2013 Project, by
based quantitative security metrics not covered in their Government of Russian Federation, Grant 074-U01, and by
paper, including more recent proposals. Finep, with resources from Funttel, grant no. 01.14.0231.00,
• Mellado et al. [23] present a survey on software secu- under the Radiocommunication Reference Center (Centro de

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 27

Referência em Radiocomunicações CRR) project of the [7] A. Jaquith, Security Metrics: Replacing Fear, Uncertainty, and Doubt.
National Institute of Telecommunications (Instituto Nacional Addison-Wesley Professional, 2007.
[8] M. A. McQueen, W. F. Boyer, M. A. Flynn, and G. A. Beitel, “Time-to-
de TelecomunicaçõesInatel), Brazil. Compromise Model for Cyber Risk Reduction Estimation,” in Quality
of Protection Workshop at ESORICS 2005, the flagship European
Symposium on Research in Computer Security. Boston, MA: Springer
A PPENDIX US, 2005, pp. 49–64.
[9] S. Jajodia, “Topological analysis of network attack vulnerability,” in
Tab. X lists the abbreviations used throughout this survey. Proceedings of the 2Nd ACM Symposium on Information, Computer
and Communications Security, ser. ASIACCS ’07. New York, NY,
TABLE X USA: ACM, 2007, pp. 2–2.
L IST OF A BBREVIATIONS . [10] N. Idika and B. Bhargava, “Extending attack graph-based security
metrics and aggregating their application,” IEEE Transactions on
AC access complexity (CVSS sub-metric) Dependable and Secure Computing, vol. 9, no. 1, pp. 75–85, 2012.
ADVISE ADversary VIew Security Evaluation [11] J. L. Bayuk, “Security as a theoretical attribute construct,” Computers
AEG attack execution graph & Security, vol. 37, pp. 155–175, sep 2013.
AG attack graph [12] S. Stolfo, S. M. Bellovin, and D. Evans, “Measuring security,” IEEE
AGP attack graph probabilistic metric Security Privacy, vol. 9, no. 3, pp. 60–65, May 2011.
AHVM aggregated historical vulnerability measure [13] A. Hecker, “On system security metrics and the definition approaches,”
AP attack propagation in 2008 Second International Conference on Emerging Security Infor-
BN Bayesian network mation, Systems and Technologies, Aug 2008, pp. 412–419.
CG compromise graph [14] R. B. Vaughn, R. Henning, and A. Siraj, “Information assurance
CPT conditional probability table measures and metrics - state of practice and proposed taxonomy,”
CTMC continuous-time Markov chain in System Sciences, 2003. Proceedings of the 36th Annual Hawaii
CVE common vulnerabilities and exposures
International Conference on, Jan 2003, p. 10 pp.
CVSS common vulnerability scoring system [15] R. Savola, “Towards a security metrics taxonomy for the information
DAG directed acyclic graph
and communication technology industry,” in Proceedings of the Inter-
national Conference on Software Engineering Advances, ser. ICSEA
DBN dynamic Bayesian networ
’07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 60–
DMZ demilitarized zone
66.
DTMC discrete-time Markov chain
[16] V. Verendel, “Quantified security is a weak hypothesis: A critical survey
ER expected risk
of results and assumptions,” in Proceedings of the 2009 Workshop on
EVM existing vulnerability measure
New Security Paradigms Workshop, ser. NSPW ’09. New York, NY,
IDS intrusion detection system
USA: ACM, 2009, pp. 37–50.
MePL median of path lengths
[17] M. Rudolph and R. Schwarz, “A critical survey of security indicator
METF mean effort to failure
approaches,” in Availability, Reliability and Security (ARES), 2012
MoPL mode of path lengths Seventh International Conference on, Aug 2012, pp. 291–300.
MPL mean of path lengths [18] C. Villarrubia, E. Fernández-Medina, and P. Mario, “Towards a Clas-
MTFF mean time to first failure sification of Security Metrics,” in Proceedings of the 2nd international
MTSF mean time to security failure workshop on security in information systems (WOSIS 2004). In con-
MTTC mean time to compromise junction with ICEIS, Porto, Portugal, 2004, pp. 342—-350.
MTTF mean time to failure [19] B. Heinzle and S. Furnell, “Assessing the Feasibility of Security
NCP network compromise percentage Metrics,” in Proceedings of the 10th International Conference on Trust,
NMPL normalized mean of path lengths Privacy, and Security in Digital Business (TRUSTBUS’13), 2013.
NP number of paths [20] W. Boyer and M. McQueen, “Ideal Based Cyber Security Technical
NSM network security metric Metrics for Control Systems,” in Proceedings of the 9th Interna-
NSSA network security situational awareness tional Conference on Critical Information Infrastructures Security
PSS policy security score (CRITIS’07), 2007.
PVM probabilistic vulnerability measure [21] E. Chew, M. Swanson, K. Stine, N. Bartol, A. Brown, and W. Robinson,
SDPL standard deviation of path length “Security Metrics Guide for Information Technology Systems, NIST
SLH success likelihood Special Publication 800-55 Revision 1,” National Institute of Standards
SMC semi-Markov chain and Technology (NIST), Tech. Rep., 2008.
SP shortest path [22] S. C. Payne, “A Guide to Security Metrics,” SANS Institute, Tech.
VLMC variable length Markov chain Rep., 2006.
[23] D. Mellado, E. Fernández-Medina, and M. Piattini, “A comparison
of software design security metrics,” in Proceedings of the Fourth
European Conference on Software Architecture: Companion Volume,
R EFERENCES ser. ECSA ’10. New York, NY, USA: ACM, 2010, pp. 236–242.
[24] M. G. Jaatun, “Hunting for Aardvarks: Can Software Security Be
[1] S. Hansman and R. Hunt, “A taxonomy of network and computer Measured?” in IFIP WG 8.4, 8.9/TC 5 International Cross-Domain
attacks,” Computers & Security, vol. 24, no. 1, pp. 31–43, feb 2005. Conference and Workshop on Availability, Reliability, and Security,
[2] U. Lindqvist and E. Jonsson, “How to systematically classify computer CD-ARES 2012, 2012.
security intrusions,” in Security and Privacy, 1997. Proceedings., 1997 [25] S. Ouchani and M. Debbabi, “Specification, verification, and quantifi-
IEEE Symposium on, May 1997, pp. 154–163. cation of security in model-based systems,” Computing, vol. 97, no. 7,
[3] S. Zonouz and P. Haghani, “Cyber-physical security metric inference pp. 691–711, jul 2015.
in smart grid critical infrastructures based on system administrators’ [26] R. Böhme and T. Nowey, “Economic Security Metrics,” in Dependabil-
responsive behavior,” Computers & Security, vol. 39, no. PART B, pp. ity Metrics, I. Eusgeld, F. C. Freiling, and R. Reussner, Eds. Berlin,
190–200, nov 2013. Heidelberg: Springer Berlin Heidelberg, 2008, pp. 176–187.
[4] U. Premaratne, J. Samarabandu, T. Sidhu, R. Beresh, and J. C. [27] P. Mell, K. Scarfone, and S. Romanosky, “Common vulnerability
Tan, “Security analysis and auditing of iec61850-based automated scoring system,” IEEE Security Privacy, vol. 4, no. 6, pp. 85–89, Nov
substations,” IEEE Transactions on Power Delivery, vol. 25, no. 4, 2006.
pp. 2346–2355, Oct 2010. [28] W. Jansen, “Directions in Security Metrics Research, NISTIR 7564,”
[5] M. Bishop, “What is computer security?” IEEE Security & Privacy U.S. National Institute of Standards and Technology, Tech. Rep., 2009.
Magazine, vol. 1, no. 1, pp. 67–69, jan 2003. [29] R. Böhme and F. C. Freiling, “On Metrics and Measurements,” in
[6] T. F. Lunt, “A survey of intrusion detection techniques,” Computers & Dependability Metrics, I. Eusgeld, F. C. Freiling, and R. Reussner,
Security, vol. 12, no. 4, pp. 405–418, jun 1993. Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 7–13.

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
[30] ISO/IEC, “Information technology - systems security engineering -
capability maturity model (SSE-CMM), ISO/IEC 21827,” Geneva,
Switzerland, p. 132, 2002.
[31] M. Tupper and a. N. Zincir-Heywood, “VEA-bility Security Metric: A
Network Security Analysis Tool,” 2008 Third International Conference
on Availability, Reliability and Security, pp. 950–957, mar 2008.
[32] P. K. Manadhata and J. M. Wing, “An Attack Surface Metric,” IEEE
Transactions on Software Engineering, vol. 37, no. 3, pp. 371–386,
may 2011.
[33] A. Ramos and R. Filho, “Sensor Data Security Level Estimation
Scheme for Wireless Sensor Networks,” Sensors, vol. 15, no. 1, pp.
2104–2136, jan 2015.
[34] A. Ramos, M. Lazar, R. Holanda Filho, and J. J. P. C. Rodrigues,
“A Security Metric for the Evaluation of Collaborative Intrusion De-
tection Systems in Wireless Sensor Networks,” in IEEE International
Conference on Communications (Accepted for publication), 2017.
[35] A. Ramos, B. Aquino, R. Holanda Filho, and J. J. P. C. Rodrigues,
“Quantifying Node Security in Wireless Sensor Networks under Worm
Attacks,” in Brazilian Symposium of Computer Networks (Accepted for
publication), 2017.
[36] M. Whaiduzzaman and A. Gani, “Measuring security for cloud service
provider: A third party approach,” in Electrical Information and
Communication Technology (EICT), 2013 International Conference on,
Feb 2014, pp. 1–6.
[37] Q. Duan, Y. Wang, F. Mohsen, and E. Al-Shaer, “Private and anony-
mous data storage and distribution in cloud,” in Services Computing
(SCC), 2013 IEEE International Conference on, June 2013, pp. 264–
271.
[38] D. S. Herrmann, Complete Guide to Security and Privacy Metrics:
Measuring Regulatory Compliance, Operational Resilience, and ROI.
Auerbach Publication, 2007.
[39] G. Gu, P. Fogla, D. Dagon, W. Lee, and B. Skorić, “Measuring
intrusion detection capability: An information-theoretic approach,” in
Proceedings of the 2006 ACM Symposium on Information, Computer
and Communications Security, ser. ASIACCS ’06. New York, NY,
USA: ACM, 2006, pp. 90–101.
[40] J. Almasizadeh and M. A. Azgomi, “A stochastic model of attack
process for the evaluation of security metrics,” Computer Networks,
vol. 57, no. 10, pp. 2159–2180, jul 2013.
[41] A. Xie, W. Wen, L. Zhang, J. Hu, and Z. Chen, “Applying Attack
Graphs to Network Security Metric,” in 2009 International Conference
on Multimedia Information Networking and Security, vol. 1. IEEE,
2009, pp. 427–431.
[42] M. S. Merkow and J. Breithaupt, Computer security assurance, using
the common criteria. Clifton Park, NY: Thomson Delmar Learning,
2005.
[43] K. Solic, H. Ocevcic, and M. Golub, “The information systems’
security level assessment model based on an ontology and evidential
reasoning approach,” Computers & Security, vol. 55, pp. 100–112, nov
2015.
[44] Z. Benenson, U. Kühn, and S. Lucks, “Cryptographic Attack Metrics,”
in Dependability Metrics, I. Eusgeld, F. C. Freiling, and R. Reussner,
Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 133–
156.
[45] NIST (National Institute of Standards and Technology), “National
Vulnerability Database.” [Online]. Available: https://nvd.nist.gov/
[46] FIRST, “Common Vulnerability Scoring System v3.0: Specification
Document,” 2015.
[47] P. Mell, K. Scarfone, and S. Romanosky, “A Complete Guide to the
Common Vulnerability Scoring System Version 2.0,” 2007.
[48] K. Scarfone and P. Mell, “An analysis of cvss version 2 vulnerability
scoring,” in 2009 3rd International Symposium on Empirical Software
Engineering and Measurement, Oct 2009, pp. 516–525.
[49] A. Avizienis, J.-c. Laprie, and B. Randell, “Fundamental concepts of
dependability,” LAAS-CNRS N01145, Tech. Rep., 2001.
[50] Y. Bo, Y. Lin, and M. L. Ru, “Quality of protection in web service:
An overview,” in Instrumentation, Measurement, Computer, Communi-
cation and Control, 2011 First International Conference on, Oct 2011,
pp. 495–498.
[51] L. Jibao, W. Huiqiang, and Z. Liang, “Study of network security situa-
tion awareness model based on simple additive weight and grey theory,”
in 2006 International Conference on Computational Intelligence and
Security, vol. 2, Nov 2006, pp. 1545–1548.
[52] D. Nicol, W. Sanders, and K. Trivedi, “Model-based evaluation: from
dependability to security,” IEEE Transactions on Dependable and
Secure Computing, vol. 1, no. 1, pp. 48–65, jan 2004.
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
28
[53] K. Sallhammar, B. E. Helvik, and S. J. Knapskog, “On Stochastic Mod-
eling for Integrated Security and Dependability Evaluation,” Journal of
Networks, vol. 1, no. 5, oct 2006.
[54] ——, “Towards a stochastic model for integrated security and depend-
ability evaluation,” in First International Conference on Availability,
Reliability and Security (ARES’06), April 2006, p. 8 pp.
[55] M. Al-Kuwaiti, N. Kyriakopoulos, and S. Hussein, “A comparative
analysis of network dependability, fault-tolerance, reliability, security,
and survivability,” IEEE Communications Surveys Tutorials, vol. 11,
no. 2, pp. 106–124, 2009.
[56] S. N. Foley, S. Bistarelli, B. O’Sullivan, J. Herbert, and G. Swart,
“Multilevel security and quality of protection,” in Quality of Protection
Workshop at ESORICS 2005, the flagship European Symposium on
Research in Computer Security. Boston, MA: Springer US, 2005, pp.
93–105.
[57] C. S. Ong, K. Nahrstedt, and W. Yuan, “Quality of protection for mobile
multimedia applications,” in Multimedia and Expo, 2003. ICME ’03.
Proceedings. 2003 International Conference on, vol. 2, July 2003, pp.
II–137–40 vol.2.
[58] B. Ksiezopolski, “QoP-ML: Quality of protection modelling language
for cryptographic protocols,” Computers & Security, vol. 31, no. 4, pp.
569–596, jun 2012.
[59] H. Zhu, K. H. Liu, W. He, and K. Ota, “Quality of experience and
quality of protection provisions in emerging mobile networks [guest
editorial],” IEEE Wireless Communications, vol. 22, no. 4, pp. 8–9,
August 2015.
[60] J. Franks, P. Hallam-Baker, J. Hostetler, S. Lawrence, P. Leach,
A. Luotonen, and L. Stewart., “HTTP Authentication: Basic and Digest
Access Authentication.” Internet RFC 2617, 1999.
[61] T. Moses, “Web Services Security Quality of Protection,” 2002.
[Online]. Available: http://xml.coverpages.org/ni2002-09-21-a.html
[62] G. Karjoth, B. Pfitzmann, M. Schunter, and M. Waidner, “Service-
oriented Assurance Comprehensive Security by Explicit Assurances,”
in Quality of Protection, D. Gollmann, F. Massacci, and A. Yautsiukhin,
Eds. Boston, MA: Springer US, 2006, pp. 13–24.
[63] O. Gerstel and G. H. Sasaki, “Quality of protection (QoP): a quanti-
tative unifying paradigm to protection service grades,” in Proc. SPIE
4599, OptiComm 2001: Optical Networking and Communications, aug
2001, p. 12.
[64] “First Workshop on Quality of Protection,” 2005. [Online]. Available:
http://disi.unitn.it/ qop/QoP2008/index.htm
[65] H. Shiravi, A. Shiravi, and A. A. Ghorbani, “A survey of visualization
systems for network security,” IEEE Transactions on Visualization and
Computer Graphics, vol. 18, no. 8, pp. 1313–1329, Aug 2012.
[66] X. Yin, W. Yurcik, Y. Li, K. Lakkaraju, and C. Abad, “Visflowconnect:
providing security situational awareness by visualizing network traffic
flows,” in Performance, Computing, and Communications, 2004 IEEE
International Conference on, 2004, pp. 601–607.
[67] F. Lan, W. Chunlei, and M. Guoqing, “A framework for network secu-
rity situation awareness based on knowledge discovery,” in 2010 2nd
International Conference on Computer Engineering and Technology,
vol. 1, April 2010, pp. V1–226–V1–231.
[68] O. Littlewood and J. Dobson, “Towards Operational Measures of
Computer Security,” Journal of Computer Security,, vol. 2, no. 3, pp.
211–229, 1993.
[69] E. Jonsson and T. Olovsson, “A quantitative model of the security
intrusion process based on attacker behavior,” IEEE Transactions on
Software Engineering, vol. 23, no. 4, pp. 1–11, 1997.
[70] M. Dacier, Y. Deswarte, and M. Kaâniche, Information Systems Secu-
rity: Facing the information society of the 21st century. Boston, MA:
Springer US, 1996, ch. Models and tools for quantitative assessment
of operational security, pp. 177–186.
[71] M. Dacier, Y. Deswarte, and M. Kaâniche, “Quantitative Assessment
of Operational Security: Models and Tools,” LAAS 96493, Tech. Rep.,
1996.
[72] R. Ortalo, Y. Deswarte, and M. Kaaniche, “Experimenting with Quan-
titative Evaluation Tools for Operational Security,” IEEE Transactions
on Software Engineering, vol. 25, no. 5, pp. 633–650, 1999.
[73] J. Almasizadeh and M. A. Azgomi, “A Method for Estimation of
the Success Probability of an Intrusion Process by Considering the
Temporal Aspects of the Attacker Behavior,” in Transactions on
Computational Science IV, 2009, pp. 200–214.
[74] ——, “Intrusion Process Modeling for Security Quantification,” in
2009 International Conference on Availability, Reliability and Security.
IEEE, 2009, pp. 114–121.
[75] M. A. McQueen, W. F. Boyer, M. A. Flynn, and G. A. Beitel,
“Quantitative cyber risk reduction estimation methodology for a small
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017
scada control system,” in Proceedings of the 39th Annual Hawaii
International Conference on System Sciences - Volume 09, ser. HICSS
’06. Washington, DC, USA: IEEE Computer Society, 2006, pp. 226–.
[76] D. J. Leversage and E. J. Byres, “Estimating a System’s Mean Time-
to-Compromise,” IEEE Security & Privacy Magazine, vol. 6, no. 1, pp.
52–60, 2008.
[77] N. Paulauskas and E. Garsva, “Attacker skill level distribution es-
timation in the system mean time-to-compromise,” in Information
Technology, 2008. IT 2008. 1st International Conference on, May 2008,
pp. 1–4.
[78] S. M. Abraham, “Estimating mean time to compromise using non-
homogenous continuous-time markov models,” in 2016 IEEE 40th
Annual Computer Software and Applications Conference (COMPSAC),
vol. 2, June 2016, pp. 467–472.
[79] S. Jha, O. Sheyner, and J. Wing, “Two formal analys s of attack graphs,”
in Proceedings of the 15th IEEE Workshop on Computer Security
Foundations, ser. CSFW ’02. Washington, DC, USA: IEEE Computer
Society, 2002, pp. 49–.
[80] W. Kanoun, S. Dubus, S. Papillon, N. Cuppens-Boulahia, and F. Cup-
pens, “Towards Dynamic Risk Management: Success Likelihood of
Ongoing Attacks,” Bell Labs Technical Journal, vol. 17, no. 3, pp.
61–78, dec 2012.
[81] X. Li, P. Parker, and S. Xu, “A Stochastic Model for Quantitative
Security Analyses of Networked Systems,” IEEE Transactions on
Dependable and Secure Computing, vol. 8, no. 1, pp. 28–43, jan 2011.
[82] M. Xu and S. Xu, “An Extended Stochastic Model for Quantitative Se-
curity Analysis of Networked Systems,” Internet Mathematics, vol. 8,
no. 3, pp. 288–320, aug 2012.
[83] C. Phillips and L. P. Swiler, “A graph-based system for network-
vulnerability analysis,” in Proceedings of the 1998 Workshop on New
Security Paradigms, ser. NSPW ’98. New York, NY, USA: ACM,
1998, pp. 71–79.
[84] X. Ou, S. Govindavajhala, and A. W. Appel, “Mulval: A logic-based
network security analyzer,” in Proceedings of the 14th Conference on
USENIX Security Symposium - Volume 14, ser. SSYM’05. Berkeley,
CA, USA: USENIX Association, 2005, pp. 8–8.
[85] Nessus, “The Nessus Security Scanner.” [Online]. Available:
http://www.nessus.org
[86] G. Vigna and R. A. Kemmerer, “Netstat: A network-based intrusion
detection system,” J. Comput. Secur., vol. 7, no. 1, pp. 37–71, Jan.
1999.
[87] S. Noel and S. Jajodia, “Managing attack graph complexity through
visual hierarchical aggregation,” in Proceedings of the 2004 ACM
Workshop on Visualization and Data Mining for Computer Security,
ser. VizSEC/DMSEC ’04. New York, NY, USA: ACM, 2004, pp.
109–118.
[88] L. Wang, T. Islam, T. Long, A. Singhal, and S. Jajodia, “An attack
graph-based probabilistic security metric,” in Proceeedings of the 22Nd
Annual IFIP WG 11.3 Working Conference on Data and Applications
Security. Berlin, Heidelberg: Springer-Verlag, 2008, pp. 283–296.
[89] O. Sheyner, J. Haines, S. Jha, R. Lippmann, and J. M. Wing, “Au-
tomated generation and analysis of attack graphs,” in Security and
Privacy, 2002. Proceedings. 2002 IEEE Symposium on, 2002, pp. 273–
284.
[90] R. W. Ritchey and P. Ammann, “Using model checking to analyze
network vulnerabilities,” in Security and Privacy, 2000. S P 2000.
Proceedings. 2000 IEEE Symposium on, 2000, pp. 156–165.
[91] P. Ammann, D. Wijesekera, and S. Kaushik, “Scalable, graph-based
network vulnerability analysis,” in Proceedings of the 9th ACM Confer-
ence on Computer and Communications Security, ser. CCS ’02. New
York, NY, USA: ACM, 2002, pp. 217–224.
[92] R. P. Lippmann and K. W. Ingols, “An annotated review of past
papers on attack graphs, ESC-TR-2005-054.” MIT Lincoln laboratory,
Lexington, MA, Tech. Rep., 2005.
[93] V. Shandilya, C. B. Simmons, and S. Shiva, “Use of Attack Graphs
in Security Systems,” Journal of Computer Networks and Communi-
cations, vol. 2014, pp. 1–13, 2014.
[94] M. Barik, A. Sengupta, and C. Mazumdar, “Attack graph generation
and analysis techniques,” Defence Science Journal, vol. 66, no. 6, pp.
559–567, 2016.
[95] W. Li and R. Vaughn, “Cluster security research involving the mod-
eling of network exploitations using exploitation graphs,” in Cluster
Computing and the Grid, 2006. CCGRID 06. Sixth IEEE International
Symposium on, vol. 2, May 2006, pp. 26–26.
[96] J. Pamula, S. Jajodia, P. Ammann, and V. Swarup, “A weakest-
adversary security metric for network configuration security analysis,”
1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
29
in Proceedings of the 2nd ACM workshop on Quality of protection -
QoP ’06. ACM Press, 2006, p. 31.
[97] R. Lippmann, K. Ingols, C. Scott, K. Piwowarski, K. Kratkiewicz,
M. Artz, and R. Cunningham, “Validating and restoring defense in
depth using attack graphs,” in Military Communications Conference,
2006. MILCOM 2006. IEEE, Oct 2006, pp. 1–10.
[98] V. Mehta, C. Bartzis, H. Zhu, E. Clarke, and J. Wing, “Ranking attack
graphs,” in Proceedings of the 9th International Conference on Recent
Advances in Intrusion Detection, ser. RAID’06. Berlin, Heidelberg:
Springer-Verlag, 2006, pp. 127–144.
[99] S. Noel, S. Jajodia, L. Wang, and A. Singhal, “Measuring Security
Risk of Networks Using Attack Graphs,” International Journal of Next
Generation Computing, vol. 1, no. 1, pp. 135–147, 2010.
[100] L. Wang, A. Singhal, and S. Jajodia, “Measuring the overall security of
network configurations using attack graphs,” in Proceedings of the 21st
annual IFIP WG 11.3 working conference on Data and applications
security. Springer-Verlag, jul 2007, pp. 98–112.
[101] ——, “Toward measuring network security using attack graphs,” in
Proceedings of the 2007 ACM Workshop on Quality of Protection, ser.
QoP ’07. New York, NY, USA: ACM, 2007, pp. 49–54.
[102] J. Homer, S. Zhang, X. Ou, D. Schmidt, Y. Du, S. R. Rajagopalan, and
A. Singhal, “Aggregating vulnerability metrics in enterprise networks
using attack graphs,” Journal of Computer Security, vol. 21, no. 4, pp.
561–597, jul 2013.
[103] V. Mihajlovic and M. Petkovic, “Dynamic bayesian networks: A state
of the art, TR-CTI,” University of Twente, Centre for Telematics and
Information Technology, Tech. Rep., 2001.
[104] Y. Liu and H. Man, “Network vulnerability assessment using bayesian
networks,” in Data Mining, Intrusion Detection, Information Assur-
ance, and Data Networks Security 2005, ser. Proc. SPIE, vol. 5812,
Mar 2005, pp. 61–71.
[105] M. Frigault and L. Wang, “Measuring network security using bayesian
network-based attack graphs,” in Computer Software and Applications,
2008. COMPSAC ’08. 32nd Annual IEEE International, July 2008, pp.
698–703.
[106] M. Frigault, L. Wang, A. Singhal, and S. Jajodia, “Measuring network
security using dynamic bayesian network,” in Proceedings of the 4th
ACM Workshop on Quality of Protection, ser. QoP ’08. New York,
NY, USA: ACM, 2008, pp. 23–30.
[107] N. Poolsappasit, R. Dewri, and I. Ray, “Dynamic Security Risk
Management Using Bayesian Attack Graphs,” IEEE Transactions on
Dependable and Secure Computing, vol. 9, no. 1, pp. 61–74, jan 2012.
[108] P. Xie, J. H. Li, X. Ou, P. Liu, and R. Levy, “Using bayesian networks
for cyber security analysis,” in Dependable Systems and Networks
(DSN), 2010 IEEE/IFIP International Conference on, June 2010, pp.
211–220.
[109] R. Dantu, P. Kolan, and J. Cangussu, “Network risk management
using attacker profiling,” Security and Communication Networks, vol. 2,
no. 1, pp. 83–96, jan 2009.
[110] R. Dantu and P. Kolan, Risk Management Using Behavior Based
Bayesian Networks. Berlin, Heidelberg: Springer Berlin Heidelberg,
2005, pp. 115–126.
[111] R. Dantu, K. Loper, and P. Kolan, “Risk management using behavior
based attack graphs,” in Information Technology: Coding and Com-
puting, 2004. Proceedings. ITCC 2004. International Conference on,
vol. 1, April 2004, pp. 445–449.
[112] S. Ghosh and P. Bhattacharya, “Analytical framework for measuring
network security using exploit dependency graph,” IET Information
Security, vol. 6, no. 4, pp. 264–270, dec 2012.
[113] L. Wang, S. Jajodia, A. Singhal, P. Cheng, and S. Noel, “k-Zero
Day Safety: A Network Security Metric for Measuring the Risk of
Unknown Vulnerabilities,” IEEE Transactions on Dependable and
Secure Computing, vol. 11, no. 1, pp. 30–44, jan 2014.
[114] L. Wang, S. Jajodia, A. Singhal, and S. Noel, “K-zero day safety:
Measuring the security risk of networks against unknown attacks,”
in Proceedings of the 15th European Conference on Research in
Computer Security, ser. ESORICS’10. Berlin, Heidelberg: Springer-
Verlag, 2010, pp. 573–587.
[115] M. Zhang, L. Wang, S. Jajodia, A. Singhal, and A. Massimiliano,
“Network Diversity: A Security Metric for Evaluating the Resilience
of Networks against Zero-Day Attacks,” IEEE Transactions on Infor-
mation Forensics and Security, vol. PP, no. 99, pp. 1–1, 2016.
[116] L. Wang, M. Zhang, S. Jajodia, A. Singhal, and M. Albanese, “Mod-
eling Network Diversity for Evaluating the Robustness of Networks
against Zero-Day Attacks,” in Proceedings of the 19th European
Symposium on Research in Computer Security. Wroclaw, Poland:
Springer International Publishing, 2014, pp. 494–511.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/COMST.2017.2745505, IEEE
Communications Surveys & Tutorials
IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. XX, NO. X, , XXXXX QUARTER 2017 30

[117] D. Borbor, L. Wang, S. Jajodia, and A. Singhal, “Diversifying Net- [137] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A
work Services Under Cost Constraints for Better Resilience Against survey on sensor networks,” IEEE Communications Magazine, vol. 40,
Unknown Attacks,” in 30th Annual IFIP WG 11.3 Conference, DBSec. no. 8, pp. 102–114, Aug 2002.
Trento: Springer International Publishing, 2016, pp. 295–312. [138] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and
[118] S. A. Zonouz, R. Berthier, H. Khurana, W. H. Sanders, and T. Yardley, M. Ayyash, “Internet of things: A survey on enabling technologies,
“Seclius: An Information Flow-Based, Consequence-Centric Security protocols, and applications,” IEEE Communications Surveys Tutorials,
Metric,” IEEE Transactions on Parallel and Distributed Systems, vol. 17, no. 4, pp. 2347–2376, Fourthquarter 2015.
vol. 26, no. 2, pp. 562–573, feb 2015. [139] J. Granjal, E. Monteiro, and J. S. Silva, “Security for the internet
[119] C. Vellaithurai, A. Srivastava, S. Zonouz, and R. Berthier, “Cpin- of things: A survey of existing protocols and open research issues,”
dex: Cyber-physical vulnerability assessment for power-grid infrastruc- IEEE Communications Surveys Tutorials, vol. 17, no. 3, pp. 1294–
tures,” IEEE Transactions on Smart Grid, vol. 6, no. 2, pp. 566–575, 1312, 2015.
March 2015. [140] S. Pfleeger and R. Cunningham, “Why measuring security is hard,”
[120] A. Chatzipoulidis, D. Michalopoulos, and I. Mavridis, “Information IEEE Security Privacy, vol. 8, no. 4, pp. 46–54, July 2010.
infrastructure risk prediction through platform vulnerability analysis,” [141] M. Pendleton, R. Garcia-Lebron, J.-H. Cho, and S. Xu, “A survey
Journal of Systems and Software, vol. 106, no. C, pp. 28–41, aug 2015. on systems security metrics,” ACM Comput. Surv., vol. 49, no. 4, pp.
[121] J. Almasizadeh and M. Abdollahi Azgomi, “Mean privacy: A metric 62:1–62:35, Dec. 2016.
for security of computer systems,” Computer Communications, vol. 52, [142] W. Knowles, D. Prince, D. Hutchison, J. F. P. Disso, and K. Jones, “A
pp. 47–59, oct 2013. survey of cyber security management in industrial control systems,”
[122] M. Sahinoglu, “Security meter: A practical decision-tree model to International Journal of Critical Infrastructure Protection, vol. 9,
quantify risk,” IEEE Security and Privacy, vol. 3, no. 3, pp. 18–24, no. C, pp. 52–80, jun 2015.
2005. [143] Y. Cherdantseva, P. Burnap, A. Blyth, P. Eden, K. Jones, H. Soulsby,
[123] ——, “An input-output measurable design for the security meter model and K. Stoddart, “A review of cyber security risk assessment methods
to quantify and manage software security risk,” IEEE Transactions on for SCADA systems,” Computers & Security, vol. 56, pp. 1–27, 2016.
Instrumentation and Measurement, vol. 57, no. 6, pp. 1251–1260, 2008. [144] Computing Research Association (CRA), “Four Grand Challenges
[124] M. Abedin, S. Nessa, E. Al-Shaer, and L. Khan, “Vulnerability analysis in Trustworthy Computing,” Tech. Rep., 2003. [Online]. Available:
for evaluating quality of protection of security policies,” in Proceedings http://archive.cra.org/reports/trustworthy.computing.pdf
of the 2Nd ACM Workshop on Quality of Protection, ser. QoP ’06. New [145] J. McHugh, “Quality of protection: Measuring the unmeasurable?” in
York, NY, USA: ACM, 2006, pp. 49–52. Proceedings of the 2Nd ACM Workshop on Quality of Protection, ser.
[125] M. S. Ahmed, E. Al-Shaer, and L. Khan, “A novel quantitative QoP ’06. New York, NY, USA: ACM, 2006, pp. 1–2.
approach for measuring network security,” in INFOCOM 2008. The [146] S. M. Bellovin, “On the brittleness of software and the infeasibility of
27th Conference on Computer Communications. IEEE, April 2008, pp. security metrics,” IEEE Security Privacy, vol. 4, no. 4, pp. 96–96, July
1957–1965. 2006.
[126] M. S. Ahmed, E. Al-Shaer, M. Taibah, and L. Khan, “Objective Risk
Evaluation for Automated Security Management,” Journal of Network
and Systems Management, vol. 19, no. 3, pp. 343–366, sep 2011.
[127] E. LeMay, M. D. Ford, K. Keefe, W. H. Sanders, and C. Muehrcke,
“Model-based Security Metrics Using ADversary VIew Security Eval-
uation (ADVISE),” in 2011 Eighth International Conference on Quan-
titative Evaluation of SysTems. IEEE, sep 2011, pp. 191–200.
[128] M. D. Ford, K. Keefe, E. LeMay, W. H. Sanders, and C. Muehrcke,
“Implementing the advise security modeling formalism in mobius,” in
2013 43rd Annual IEEE/IFIP International Conference on Dependable
Systems and Networks (DSN), June 2013, pp. 1–8.
[129] M. Kaaniche, E. Alata, V. Nicomette, Y. Deswarte, and M. Dacier,
“Empirical analysis and statistical modeling of attack processes based
on honeypots,” in IEEE/IFIP International Conference on Dependable
Systems and Networks (DSN-2006), 2006, pp. 119–124.
[130] B. P. Rimal, E. Choi, and I. Lumb, “A taxonomy and survey of cloud
computing systems,” in INC, IMS and IDC, 2009. NCM ’09. Fifth
International Joint Conference on, Aug 2009, pp. 44–51.
[131] B. B. Madan, K. Gogeva-Popstojanova, K. Vaidyanathan, and K. S.
Trivedi, “Modeling and quantification of security attributes of software
systems,” in Dependable Systems and Networks, 2002. DSN 2002.
Proceedings. International Conference on, 2002, pp. 505–514.
[132] B. B. Madan, K. Goševa-Popstojanova, K. Vaidyanathan, and K. S.
Trivedi, “A method for modeling and quantifying the security attributes
of intrusion tolerant systems,” Perform. Eval., vol. 56, no. 1-4, pp. 167–
186, Mar. 2004.
[133] F. Wang, F. Jou, F. Gong, C. Sargor, K. Goseva-Popstojanova, and
K. Trivedi, “Sitar: a scalable intrusion-tolerant architecture for dis-
tributed services,” in Foundations of Intrusion Tolerant Systems, 2003
[Organically Assured and Survivable Information Systems], 2003, pp.
359–367.
[134] H. Holm, M. Ekstedt, and D. Andersson, “Empirical Analysis of
System-Level Vulnerability Metrics through Actual Attacks,” IEEE
Transactions on Dependable and Secure Computing, vol. 9, no. 6, pp.
825–837, nov 2012.
[135] L. Bilge and T. Dumitras, “Before we knew it: An empirical study of
zero-day attacks in the real world,” in Proceedings of the 2012 ACM
Conference on Computer and Communications Security, ser. CCS ’12.
ACM, 2012, pp. 833–844.
[136] N. C. Idika, B. H. Marshall, and B. K. Bhargava, “Maximizing network
security given a limited budget,” in The Fifth Richard Tapia Celebration
of Diversity in Computing Conference: Intellect, Initiatives, Insight, and
Innovations, ser. TAPIA ’09. New York, NY, USA: ACM, 2009, pp.
12–17.

1553-877X (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.