A Modern Approach on Information

Security Measurement
Frederik Humpert-Vrielink . Nina Vrielink

CETUS Consulting GmbH

Vriezenveener Strafie 38, 48465 Schuettorf
{frederik.humpert-vrielink I nina.vrielink}@cetus-consulting.de

Abstract
What can't be measured couldn't be management. The old wisdom of management gain more importance
in infonnation security management. The key problem in most organizations is that no one is able to give a
statement whether the whole security management system works over all management levels top down or
not. Mostly security managers raise infonnation about technical goals and other controls applied in an ISO
27001 ISMS. Less than 10% we think can proof the success down to the last level. The approach mentioned
in this text shown a model enabling all managers to measure the success of security through the whole
organization.

1 Introduction
Over the last years one of the key success factors on the maoagement of information security was
not really discovered. All models related to the measurement of information security success are
mostly driven by finaocial performaoce indicators aod not by psychological or other non-eco-
nomic goals.

1his work forces another approach. The main idea is to raise a measurement system that uses
success indicators with defined goals for the information security maoagement system. To deter-
mine these goals the model uses ao approach that includes all over business factors. The success
maoagement system is founded on a three step approach that works as follows:
1. Development of significant performaoce indicators
2. Matching the indicators with the goals of the implemented maoagement system
3. Aggregation of all indicators in a maoagement dashboard

1ms model is distinct from other measurement models, because we include more dimensions
thao ouly the economic ones in the development process.
The need for measurement aod steering tools is normative told in the ISO 27001:2005 [15005].
Measurement explainations are described in the ISO 27004:2009 [15009].

N. Pohlmann, H. Reimer, W. Schneider (Editors): Securing Electronic Business Processes, SpringerVieweg (2012), 48-53

H. Reimer et al. (eds.), ISSE 2012 Securing Electronic Business Processes,

DOl 10.1007/978-3-658-00333-3_5, () Springer Fachmedien Wiesbaden 2012
A Modern Approach on Information Security Measurement 49

2 First Step - develop significant performance

indicators
This chapter introduces our development approach for performance indicators. After finishing
many projects we learned. that this model could be a solution for all those. who don't know how
to measure. manage and operate their ISMS correctly. The development process is separated in
the steps as follows. In our experience those particular steps lead to optimal performance indi-
cators.

2.1 Determine influencing factors

In many organisations information security is not a discipline with only one dimension. There
are many factors which influence the success of the management system or the security level
finally reached These factors are a key chain for success measurement or financial controlling of
IS-Management.

Selected influencing factors could be

• the inforroation technology strategy.
• the business strategy,
• the architecture of information technology systems and
• the business process architecture.

These factors deliver worthy information for developing key success factors in the information
security. E.g. the information technology strategy includes information. which type of systems
or applications will be enabled for the next years. The business strategy gives information about
the business goals for the next years. This derives worthful information which security level is
iruportant to maintain the business strategy by strategic security management

As a method for documentiog and establishing an accepted number of influencing factors expe-
rience shows, that workshops with the second line management are the best way. If organisations
use this way. the management is actively included in the process.

2.2 Translate into KPls

After the influencing factors have been documented security management or controlling have to
translate the factors into real KPIs. Those key performance indicators are the main steering tool
in measuring information security.

There have been many approaches to define KPIs for information security. In the US the NIST-Pa-
per SP 800 [NIST08] gives a good way how to title KPIs and how to find the right one. But this
paper is mostly related to American legal definitions for companies who have demands on secu-
rity of their accounting systems. Mainly the basis of those publications is located in the financial
sector.
50 A Modern Approach on Information Security Measurement

The function of those KPls is multidimensional as shown in table 1. After understanding of these
functional dimensions the next challenge is to define, which relation the indicators should have.
This is done in the next main step during the matching of the measurements with the goals of the
management system.
Table 1: Functional dimension ofKPIs
Dimension 1: lDformationa1 Di:menaion 2: Doc:umentative
KPIs are used to inform their user about the status of the KPIs are used to document the progress of the ISMS over
ISMS timespreads
Dimension 3: SupportinJ Dimension 4: Normative
KPIs support the security management in the steering of KPIs are normatively told and a key demand in the oper-
the ISMS-Proceas and the operation of the ISMS ation of an ISO 27001 ISMS

After understanding those dimensions security and controlling staff is able to translate the KPls.
In our experience the best KPIs are those, which raise information about all factors, this econom-
ic ones even as the psychological ones.

2.3 Document the KPls

To get the right basis for measurement KPls have to be documented. We suggest the documen-
tation follows the documented procedures used in the management system that should be meas-
ured. This makes sure, that the documentation is audit proof and could be used in the further
process.
Overall the documentation gives a good foundation for the usage in a balanced scorecard based
information system.

3 Second step - matching with the ISMS goals

Most of the classic approaches on measurement of information security dmit regard to the goals
of the real management system. Looking into the normative regards for measurement we will
mostly found indicators, which are technical or financial driven. Applying those to classicallyop-
erated ISMS we surely are able to match the goal for teclmical controls, but beyond the teclmical
dimension we face problems.

3.1 Identify the goals of the ISMS

First important step is to identify the goals of the ISMS. Main problem could be the circumstance,
that most ISMS are mainly operated and steered by realizing superficial controls. We learned in
many projects that ISO 27001 [IS005] related ISMS are operated by paperwork that loses its in-
fluence beyond the third management line.

So the first challenge is to identify the goals. If an organisation ouly has the goal to get certificated
it is better to stop here and use classic measurement approaches.
A Modem Approach on Information Security Measurement 51

1b apply the explained measurement model it is very important to get selected goals and fonnu-
late them on "'Management System Stories", Example goals could be:
• We willlive security during all levels of the organisation,
• We will reduce risks on the information security to realize a business related security level,
• Our main goal is to get residual risks reduced to maintainable dimensions.

Only if thme goa1s are documented in an understandable way it is possible to match the key
indicators on the goals. And remember: Without goals management systems are no more than
paperwork without impact on the organisation.

3.2 Clustering KPls in relating dimensions

To get proven infonnation there is clustering of the KPIs necessary. After identifying the func-
tional dimension of a KPI next step is to identify the relating dimension of the indicator. In our
model we developed a four dimension model for the relationship clusters which is shown in
Figure 1.

t
\
Figure 1: Related Dimension for Security KPIs [VriI2]

These four dimensions are now wed to cluster the developed business indicators depending on
the relation to the management system. Table 2 shows some selected indicators,
52 A Modern Approach on Information Security Measurement

-
Table 2: Selected indicators for measuring information security. [Vri12[
XPI Goo! Dr Da 4HaG
SoaIrity To ensure Managementsys- This indicator is to
Badpt the financial tern relevant measure the ratio of
supply of the the security budget
management according to the
system whole IT or Risk
Management Budget.
_eo... Th llinit Technical m- Measuring the num-
traI access to evant ber of unauthorised
information access attempts to
only for em- gain information
ploy<es with about the real. security
clearance level.
A_ Thmeasun: Risk relevant This indicators gain
md 'IDlIlIIII the per- information whether
centageof an awareness cam-
trained per- paign is successful
sonnel in the or not
organization

The form shown above could be used to document all the identified indicators. This documenta-
tion should be very useful for the next step.

4 Step 3 - Integration in a steering tool like Balanced

Scorecard
Overall KPls are a necessary instrument, but they need to be aggregated and displayed in a steer-
ing tool for the management The best tool to analyze KPls and management the achievement of
goals is a balanced scorecard. There are multiple benefits of displaying all indicators in a balanced
scorecard.
At first there is scalability. The KPls model shown above makes sure, that any organization is able
to manage information security individually for business units. So any organization can develop
its own scorecards for any business unit with different goals for the information security. Any
indicators are functional and depending on their relation to the goals clustered. So unit security
managers are able to pick those goals, which are necessary for the unit. Overall those indicators
can be concentrated to all management levels from bottom up. So management will be enabled to
steer information security all over the organization.

Second is the flexibility. Goals management by scorecards could be easily replaced if the manager
realizes that the goal doesn't support business strategy for example. So even depending on the
clusters new KPIs could be qnickly picked and will be easily analyzed and integrated in the man-
agement dashboard over all management levels.

Figure 2 shows a possible scorecard. It is split up in four dimensions to get all relevant goals done
with reference to their relation model.
A Modem Approach on Information Security Measurement 53

Fipre 2: Example scorecard for ISMS Meuurement [VriI2]

5 Conclusion
Infonnation security has to be measured. But classical approaches collapse on lower manage-
ment levels and in the interpretation if the ISMS is sllCCes.sfui. The model shown above is a new
approach to management of security over allleve1s of an organization. It refers to the business
strategy, the IT strategy and other factors, which influence the steering and management of in-
fOrmation security.

References
[IS009] International Organization for Standardization (ISO). Information technology - Security tech-
niques - Information security management - Measurement, ISO/IEC 27004:2009
[ISOO5] International Organization for Standardization (ISO). Information technology - Security tech-
niques - Information security management - Requirements, ISO/IEC 27001:2005
[NIST08] National Institute of Standards and 'lechnology (NIST). Perfonnance Measurement Guide for
Information Security, NIST SP 800-55, Revision I, Jull 2008, Download viahttp://csrc.nist.govl
publiatiomiPuboSPo.html
[VriI2] Humpert-Vrielink, Frederik: Kennuhlen fun ISMS. In: <Ita> 201211, secumedia2OI2, p. 13-17