Security Awareness / Culture Metrics

Tab

Impact Metrics – Behaviors

Impact Metrics - Culture

Impact Metrics – Strategic

Compliance Metrics

Ambassador Program Metrics

Human Risk Score

NOTE: This matrix is based on content from the two day SANS MGT433 (Security Awareness) and MGT521 (S
https://www.sans.org/ssa-events
 Security Awareness / Culture Metrics

Description

These metrics measure the impact of your security awareness training. Specifically, is the
training changing people's behaviors.

These metrics measure the impact of your security awareness program or other security
initiatives. Specifically, are they changing peoples attitudes, beliefs and norms concerning
security.

These metrics measure how your security awareness program is supporting your organization's
overall security program, and ultimately the mission of your organization. These are the types
of metrics senior leadership are more likely to be interested in.

These metrics measure what your awareness program is doing, specifically who you are
training and how. These metrics are most valuable for compliance and auditing purposes.

These metrics measure the activity and impact of a security ambassador program.

Proof of concept Human Metrics Dashboard that measures your overall human risk based on
index of your top human risks. Designed for senior leadership.

n content from the two day SANS MGT433 (Security Awareness) and MGT521 (Security Culture) courses -
ts
 Security Awareness Impact Metrics
When Is It
Metric Name What Is Measured? How Is It Measured? Who Measures? Details
Measured?

Number of people who fall victim to a phishing These attacks replicate the very same ones cyber attackers are using. The goal
simulation. The definition of falling victim is clicking on is to measure who falls victim to such attacks. This number should decrease over
Phishing Click Rate the link or opening an attachment. Phishing assessment Monthly Security Team time as behaviors change.

Number of people who detect and report a phishing Uses the above methodology, but instead of tracking who falls victim, it tracks
email (regardless of whether it's an assessment or real who identifies the attacks and reports them. This number should increase over
Phishing Reporting attack). Phishing assessment Monthly Security Team time. This is developing the Human Sensor.
These individuals represent a high risk to an organization and must be
Number of workforce that repeatedly fall victim to addressed. This can include an escalation in training and consequences, being
Phishing Repeat phishing simulations. These individuals are not Security Awareness moved to a different job role or department, or being managed in some other
Offenders changing behavior and represent a high risk. Phishing assessment Monthly Team way.

Number of employees who understand, follow, and Test how many employees are For many organizations, physical security is a major control in reducing risk,
Facility Physical enforce your policies for restricted or protected access wearing their badges or Monthly or Information Security or especially when dealing with secured facilities. This metric will test and measure
Security to facilities. stopping those who are not. weekly Physical Security people's understanding and enforcement of this control.
When employees connect to an
internal server or use an
external service such as Security or Technology Measure whether people are keeping their devices updated and current,
Updated Devices Percentage of devices that are updated and current. browsercheck.qualys.com Monthly Team especially when concerning BYOD (Bring Your Own Device).

Number of devices (laptops, smartphones, tablets) that Employees should be trained in maintaining physical security of their devices. In
were lost or stolen. What percentage of those devices Reports to security team or by Security Team or Asset addition, if your organization has policies on the use of encryption for devices,
Lost/Stolen Devices were encrypted? physical asset audits Monthly Management this measures whether employees are following them.

Number of employees who are securing their desk Security team does a walkthrough of organizational facilities, checking each
environment before leaving, as per organizational Monthly or Information Security or desktop or separate work environment, and looking to ensure individuals are
Secure Desktop policy. Nightly walkthrough weekly Physical Security Team following organizational desktop policy.

Monthly or Security gains authorized access to system password database (such as AD or

Passwords Number of employees using strong passwords. Password brute forcing quarterly Security Team Unix server) and attempts to brute force or crack password hashes.

Security team calls random employees, attacking them as real cyber attacker
Number of employees who can identify, stop, and would by attempting to social engineer the victim. An example could be
Social Engineering report a social engineering attack. Phone call assessments Monthly Security Team pretending to be Microsoft support and having victim download infected antivirus.

Number of employees posting sensitive organizational Security Team (or Do extensive searches on sites such as Facebook and LinkedIn to ensure
Sensitive Data information on social networking sites. Online searches for key terms Monthly outsource) employees are not posting sensitive organizational information.
Check digital devices that are
disposed of for proper wiping. Any digital devices that are disposed of (donated, thrown out, resold) may contain
Data Wiping or Number of employees who are properly following data Check dumpsters for sensitive Information Security or sensitive data. Check to ensure proper wiping procedures. Check any rubbish
Destruction destruction processes. documents. Random Physical Security bins or dumpsters for any sensitive documents that were not shredded.
Do a physical walkthrough of the While your organization's parking lot may be a secured environment, this
parking lot and identify any cars measures employee behaviors. If they are leaving unsecured or visible devices in
Device Physical Number of employees who left their devices unsecured that have devices that are Information Security or their car at work, they are most likely doing the same when they are at off-site
Security in their cars in the organization's parking lot. visible on a car seat. Monthly Physical Security facilities.

Learning Management To be able to exhibit a behavior, people need to understand what is expected of
Does workforce know and understand what is expected Knowledge assessments and Annual or after or Security Awareness them. Do they know the indicators of a phishing attack? Do they know your
Knowledge of them? online quizzes training Team policies? Do they know how to identify sensitive data?

NOTE: These metrics measure the impact of your security awareness training. Specifically, is the training changing people's
behaviors, attitudes, or perceptions? Be sure to review the other tabs below for other types of security awareness metrics.
 Security Awareness Impact Metrics
When Is It
Metric Name What Is Measured? How Is It Measured? Who Measures? Details
Measured?

Similar to an Engagement Survey, but measuring attitudes towards cybersecurity.

Your organizations shared attitudes, beliefs and Annually or every Security Team or Good for broad understanding of the organization and can quantify the data
Culture Survey perceptions concerning cybersecurity. Survey other year Human Resources (likert scale) but lackes the depth you may be looking for.

Bring a group of employees together and interact with them to better understand
Your organizations shared attitudes, beliefs and Quarterly or Security Team or their thoughts and concerns towards cybersecurity. This data will be hard to
Focus Groups perceptions concerning cybersecurity Focus Groups Annually Human Resources quantify but give indepth understanding of the target group.

Your organizations shared attitudes, beliefs and Quarterly or Security Team or

Interviews perceptions concerning cybersecurity Individual Interviews Annually Human Resources Similar to focus groups but more of an individual level.
Monitoring communication
channels, such as in Slack,
Your organizations shared attitudes, beliefs and Yammer or other Social Media Sometimes the best way to understand peoples attitudes and beliefs is to monitor
Interactions perceptions concerning cybersecurity feeds Continously Security Team how they interact with each other.

Number of requests the security awareness team gets As your workforce grows in its belief in the importance of security awareness, you
to do security briefings for other business units or Security Awareness should see more and more groups within your organization "pulling" or requesting
Engagement teams Tracking by the security team Monthly Team for more cyber-security information.

Your organizations shared attitudes, beliefs and Booths staffed by the security Monthly or Security Team or Security team staffs a booth where anyone from the workforce can approach the
Security Booth perceptions concerning cybersecurity team in high traffic locations Quarterly Human Resources swecurity team, interact and ask questions.

NOTE: These metrics measure the impact of your security awareness training. Specifically, is the training changing people's
behaviors, attitudes, or perceptions? Be sure to review the other tabs below for other types of security awareness metrics.
 Security Awareness Impact Metrics
21

When Is It
Metric Name What Is Measured? How Is It Measured? Who Measures? Details
Measured?

Incident Response
Time to Detect an Standard incident report tracking Team or Security The time to detect an incident should decrease as the Human Sensior is
Incident What is the average time it takes to detect an incident? processes Monthly Operations Center developed. This is a critical metric as it is key to creating a resilient organization.

Number of times workforce violates organizational Standard violation reporting Human Resources or As the workforce better understands organization policies, or as those policies
Policy Violations security policies. processes Monthly Security Team become easier to follow, workforce is more likely to follow them.

Number of times there is a data loss incident, either Standard incident report tracking Security or Data Loss As your workforce better understands the policies and behaivors they are
Data Loss Incidents accidental or due to a deliberate attack. processes Monthly Prevention Team supposed to follow, the number of data loss incidents should fall.

Most infected computers are a result of human behavior (infected attachments,

Help Desk or centralized AV Help Desk or Security malicious links, etc.). This number should go down over time as employees are
Infected Computers Number of infected computers. management software Monthly Operations Center trained.

Privileged Account Number of privileged users that improperly use or Standard violation reporting As your technical workforce better understands the policies and behaivors they
Abuse abuse their privileged access. processes Monthly Security Team are supposed to follow, the number of privileged access violations should fall.

Incident Response As your technical workforce better understands the policies and behaivors they
Misconfigured Number of incidents of systems or applications Standard violation or incident Team or Security are supposed to follow, the number of incidents caused by misconfigurations
Systems misconfigured. reporting processess Monthly Operations Center should fall.

Compliance or Audit Audit, Compliance, or One of the goals of security awareness is to help meet the requirements of
Violations Number of compliance or audit violations or fines. Audit or compliance reports Annual Governance Teams certain standards or regulations.

NOTE: These metrics measure how your security awareness program is supporting your organization's overall security
program, and ultimately the mission of your organization. These are the types of metrics senior leadership are more likely to be
interested in. Be sure to review the other tabs below for other types of security awareness metrics.
 When Is It
Metric Name What Is Measured? How Is It Measured? Who Measures? Details
Measured?

Reports from LMS or sign- Whoever is

Who has or has not completed annual in sheets for on-site responsible for Primary training is when people are taught all awareness material for the first time or in a
Training Completion security awareness training workshops Annually primary training single sitting, usually through online computer-based training (CBT) or on-site workshops.

For a security awareness program to have an impact, it must be communicated to people

on a regular basis. This metric measures other communication methods and training
modalities that repeat and reinforce key learning objectives from the primary training.
Examples of such metrics can include:

• Monthly hits to internal security blog or website

• Distribution of newsletters, posters, or fact sheets
Track and document when • Tip-of-the-day questions
and how security • Number of attendees for lunch-n-learns
awareness materials are • Number of downloads for podcasts/webcasts
Communication Types of reinforcement training, who is communicated to the Security Awareness • Number of posts on internal social media channels (such as Yammer or Slack)
Methods consuming that training, and how often workforce. Monthly Team

Ensuring employees have completed

training, acknowledge they understand the Supervisor and/or From a compliance perspective, you may be required to document that employees not only
Policy Sign-Off training, and will adhere to the policies Signature or sign-off Part of annual review Human Resources receive training, but also acknowledge they understand and will follow the training.

NOTE: These metrics measure what your awareness program is doing, specifically who you are training and how.
These metrics are most valuable for compliance and auditing purposes. Be sure to review the other tabs below for
other types of security awareness metrics.
 Security Awareness Impact Metrics
When Is It
Metric Name What Is Measured? How Is It Measured? Who Measures? Details
Measured?

Tracking matrix of active One of the most visible metrics you can use is measuring how many active
Number of Number of active Ambassadors promoting the security ambassadors managed by Security Awareness ambassadors you have. However at some point leadership will also want to know
Ambassadors awareness program awareness program Monthly Officer the impact those ambassadors are having.

Number of people Feedback and reports from the

Ambassadors are Combine the total number of your workforce that all of ambassadors to the awareness Security Awareness Ambassadors should report back on a monthly basis the activities they conducted
reaching the Ambassadors are reaching. officer. Monthly Officer and the number of people they reached for that month.

Number of times How often the ambassadors are approached with Feedback and reports from the As the ambassador program grows and becomes more visible and engaged, the
workforce is engaging security questions, requests to present locally, or other ambassadors to the awareness Security Awareness local workforce will be reaching out to them for help. Growing engagement
Ambassadors engagement types. officer. Monthly Officer indicates a growing security culture.

Compare the security awareness impact metrics in Focus on key behaviors, such One way to measure the impact of Ambassadors is to measure and compare the
Effectiveness of departments or offices that do have Ambassadors vs. as phishing simulations, and secure behaviors of a department that does have an active Ambassador program
Ambassadors those that do not. compare between departments Quarterly Security Team vs. one that has no Ambassadors. For example, compare click rates.

Number and type of How often does each ambassador engage or Feedback and reports from the
outreach activities by communicate to their local team, and what is the ambassadors to the awareness Security Awareness Ambassadors should report back on a monthly basis the activities they conducted
Ambassadors communication method? officer. Monthly Officer and the number of people they reached for that month.

Feedback and reports from the Sometimes the best way to demonstrate to leadership the impact of your
Real-world stories on how workforce identified and/or ambassadors to the awareness Security Awareness ambassador program is to share success stories you have collected from your
Success stories stopped a real attack. officer. Monthly Officer ambassadors.

Surveys can provide great information but can be hard to deploy as they are
perceived as boring and time-consuming. May have greater impact by having
Surveys Workforce's attitudes, beliefs, and certain behaviors. Culture survey Quarterly Ambassadors someone local do a survey, perhaps even in person or verbally.

A key metric of a mature security program is the ability to detect and respond to
How many incidents were detected, how fast, and by Incident Response Team and/or Security Awareness incidents quickly. Mature awareness programs contribute to this by creating a
Detection of Incidents whom / how? Security Operations Center. Monthly Officer network of human sensors.

Average time spent to Quite often ambassador programs grow too fast, overwhelming the security team
manage each How much time are you spending managing each Security Awareness and in the end, causing it to fail. Track how much time you spend managing each
Ambassador Ambassador? Management tracking matrix Monthly Officer Ambassador, so you have the metrics to get more support.

NOTE: These metrics measure the activity and impact of a security ambassador program. Be sure to review the other tabs
below for other types of security awareness metrics.
 Security Awareness Impact Metrics
21

When Is It Overall Current

Metric Name What Is Measured? How Is It Measured? Who Measures? How is the Risk Rated
Measured? Score

0-3% click rate - Very Low (1)

3-6% click rate - Low (2)
6-10% click rate - Medium (3)
Time to Detect an 10-20% click rate - High (4)
Incident Phishing Click Rate Phishing simulations Monthly Security Team 20%+ click rate - Very High (5) 3

0-2 times a month - Very Low (1)

3-5 times a month - Low (2)
Accidental data disclosure due to auto-complete in 5-10 times a month - Medium (3)
email. In other words, workforce accidently emailing Data Loss Prevent (DLP) or Security Operations 10-20 times a month - High (4)
Auto-Complete the wrong people sensitive data. some other primeter controls Monthly Center 20+ times a month - Very High (5) 4

0-2 times a month - Very Low (1)

3-5 times a month - Low (2)
5-10 times a month - Medium (3)
10-20 times a month - High (4)
Device Loss Incidents Lost or stolen laptops or mobile devices Physical security Monthly Physical Security 20+ times a month - Very High (5) 2

0-2 times a month - Very Low (1)

3-5 times a month - Low (2)
5-10 times a month - Medium (3)
Number of infected computers every month due to Security Operations 10-20 times a month - High (4)
Infected Computers human action Centralized Anti-Virus Solution Monthly Center 20+ times a month - Very High (5) 4

0-2 sensitive documents - Very Low (1)

3-5 sensitive documents - Low (2)
5-10 sensitive document - Medium (3)
Disposal of Sensitive Measuring people are securely disposing of any Information Security or 10-20 sensitive documents - High (4)
Documents sensitive documents into the shred bin Dumpster diving Monthly Physical Security Team 20+ sensitive documents - Very High (5) 3

0-2 times a month - Very Low (1)

3-5 times a month - Low (2)
Measures individuals who violate policies, such as 5-10 times a month - Medium (3)
surfing to restricted websites, sharing passwords or Reported by supervisors to 10-20 times a month - High (4)
Policy Violations other activities. Security or Human Resources Monthly Human Resources 20+ times a month - Very High (5) 4

Measure the workforces perceptions of security, to

include do the feel responsible for security, do they Random 5% of workforce every
Security Perceptions know they are a target. month. Monthly Security team Measured on a Likert Scale of 1-5. 4

98%+ completed - Very Low (1)

95%+ completed - Low (2)
90%+ completed - Medium (3)
What percentage of the workforce has completed the Security or Training 85% completed - High (4)
Completed Training required awareness training. Learning Management System Monthly Team 80% completed - Very High (5) 2

3.25

The purpose of a human risk score is to provide leadership a summary of the current state of your human risk. Leaders may not want the details of all the different metrics
pertaining to different behaviors, or perhaps even strategic metrics. What they may want is an overall score (very high, very low) and if that score is getting better or worse.
One way to approach that is have a metrics scoreboard that combines all the different behavior metrics you care about (i.e. are useful) into an average score, similar to how
the Dow Jones Industrial Average tracks the average value of 30 large, publicly owned companies. The trick is identifying 3-7 different metrics you care about, score them
(think Likert and come up with an average score. Above is one such example. Quickest path to failure is to add metrics here just for the sake of metrics. You are better off
having too few than too many metrics.

NOTE: In this example, the lower the number, the better your score. The numbers provided in the RISK RATINGS column are purely demonstration purposes,
adjust to your own risk tolerance / requirements and the size of your organization.