Professional Documents
Culture Documents
Omer Tene
Senior Fellow, Future of Privacy Forum
Mary Culnan
Board Vice President & Senior Fellow,
Future of Privacy Forum
ACKNOWLEDGMENTS
Introduction_____________________________________________________ 2
Audiences______________________________________________________ 4
Common Metrics_________________________________________________ 6
Risk Management________________________________________________ 10
Conclusion_____________________________________________________ 12
Glossary_______________________________________________________ 12
T he EU’s General Data Protection Regulation (GDPR), Business leaders have traditionally advocated for man-
California Consumer Privacy Act (CCPA), as well agement by measurement. Edwards Deming wrote,
as the constant drumbeat of major data breaches “What gets measured gets done.” Dr. H. James Har-
have ushered in a new era in privacy and data protection rington once said, “If you can’t measure something, you
accountability and awareness, elevating privacy beyond can’t understand it. If you can’t understand it, you can’t
the office of the privacy leader to the attention of boards control it.” Effective measurement helps managers im-
of directors, audit and risk committees, CEOs, CIOs, prove efficiency, streamline processes, prioritize efforts,
CROs, CISOs and more (internally), and to the view of cus- and manage risk. Indeed, some say that measurement
tomers, legislators, regulators, advocates and the media is management.
(externally). Organizations have expanded their privacy
and data protection programs to cultivate customer trust Privacy leaders collect data and use metrics to measure,
and loyalty; to enable secure cross border data transfers; assess, and improve the performance of their privacy
and to comply with evolving and dynamic privacy laws programs. Beyond demonstrating compliance, privacy
and regulations. Over the past few years, privacy gov- metrics have emerged as key to measure and improve
ernance has emerged as a core business value central privacy program performance and maturity in terms of
to customers’ and employees’ trust, as well as for core customer trust, risk mitigation, and business enablement.
corporate values such as transparency and accountabil- Privacy leaders use metrics to benchmark the maturity of
ity. A wave of media maelstroms, diplomatic rows, and their organization’s privacy program against its strategy
litigation crises — as well as competitive developments and goals and demonstrate how privacy contributes to
in markets, companies, and platforms — have shown that its strategy and bottom line. They use metrics internally
inadequate privacy is bad for business. With a new wave to secure budgets and staffing, to measure performance
of regulation on the horizon, including global privacy and to diagnose program status and needs, as well as
legislation as well as rules for artificial intelligence, digital externally to demonstrate accountability and enhance
services, and platforms and non-personal data, the remit trust. They use metrics to measure, for example, how
of privacy officers is set to grow even more. sound privacy practices reduce sales delays, mitigate
Training 24%
P
rivacy metrics are both quantitative and qual- Framework (Specific, Measurable, Actionable, Rel-
itative assessments that privacy leaders use evant and Time Bound) to create SMART privacy
to measure, manage, track, benchmark, report metrics to assess their capabilities and fulfillment
on, and improve their privacy program and its com- of objectives, demonstrating the relevancy of each
ponents and to establish transparency and trust.1 metric while ensuring the completion of tasks within
Specifically, privacy teams may adopt the SMART specified time frames.
P
rivacy programs use metrics for a variety of purpos- Operational uses include measuring, documenting,
es and goals: compliance, operational, business and improving the performance of the privacy program;
imperatives, and external uses. To be effective, managing enterprise privacy risks; and making progress
metrics should be assessed against clearly defined pri- toward meeting the privacy program’s objectives
vacy program objectives and goals. and goals. In the first instance, this means ensuring
Compliance uses include measuring and tracking key the organization complies with its corporate policies,
metrics that demonstrate compliance with various standards, procedures, and applicable privacy and
laws and regulations. This includes tracking progress data protection laws and regulations. Examples include
to achieve compliance with applicable requirements, metrics for incident reporting and measures related
as well as helping to ensure sustainable compliance to data sharing with business partners and service
over time. providers, including cross border data transfers.
1 Not just privacy leaders but also other staff at all levels of the organization collect, use and present privacy metrics. In this report we use “privacy leaders” as
shorthand to address various stakeholders in charge of privacy metrics.
Audiences
P
rivacy program leaders use metrics to commu- reports and how to best leverage the reports. Different
nicate with various stakeholders. A key task for metrics resonate differently depending on the audience.
privacy leaders involves fashioning metrics into For example, members of the privacy team will view dif-
a narrative, that is, telling a relevant story to different ferent reports than those presented to legal, compliance
stakeholders, ranging from the C-Suite, board audit or officers, or business leaders. Business leaders tend to
risk committee and other business or functional de- live and manage by metrics and could be presented, for
partments to external stakeholders such as consumers,
example, with narrowly tailored or comparative reports.
business customers, regulators, privacy advocates,
and the media. As one member put it, “privacy metrics The following chart specifies various audiences for privacy
are less about numbers, more about stories.” Privacy metrics, including the nature of metrics often presented
leaders consider which audiences receive what metric to each group and the purposes for reporting to it:
W
hile different organizations measure different transfer impact assessments; negotiating and executing
activities and trends, there is a common core of data processing agreements; and so forth. Of course,
privacy metrics that many organizations collect collecting accurate and comprehensive data is a key
and report on. Most organizations measure daily privacy step toward assembling effective metrics. Sound met-
program activities: counting the number of individual rics require good data inputs.
complaints and requests; responding to incidents or
breaches; offering privacy compliance and awareness Organizations vary in the categories and specific metrics
training to team members and employees; conducting they collect and report. The chart below, while by no means
data mapping; undertaking privacy, data protection, and comprehensive, is illustrative of some common metrics.
Privacy Impact Assessments (PIAs) • # of identified high risk data processing activities
and Data Protection Impact requiring a DPIA (as a % of total processing
Assessments (DPIAs) activities recorded/as a % of initial screening/
gating assessments)
• # of PIAs/DPIAs completed
• Time vs SLA
While basic metrics measure activities (e.g., number of The next part presents various strategies privacy lead-
DSRs, ROPAs or DPIAs), more advanced metrics display ers use to weave privacy metrics into a narrative in order
trends (e.g., average time to respond to DSRs over time; to advance program goals, such as increasing resources
supplier assessments performed over time). Ultimate- and awareness, ensuring organizational support, and
ly, metrics are used to drive or show outcomes (e.g., enhancing trust.
mitigation of privacy risks; comprehensiveness of data
mapping; sales won with privacy review).
M
embers of the FPF Privacy Metrics Working • Privacy Customer Experience (CX) metrics. This
Group presented various ways to demonstrate includes direct feedback from customers about
the strategic value of privacy initiatives, shifting their privacy experience. For example, privacy
from activity based key performance or risk indicators leaders could measure consumer satisfaction
(KPIs/KRIs) to metrics advancing business value. While (CSAT) scores in B2B as well as for B2C
compliance and operational metrics are the base, more interactions (e.g., measuring CSAT for user queries
mature privacy programs develop business imperative by using a thumbs up/down button).
metrics, including both customer focused and business • A/B testing on privacy messaging and language.
enablement metrics. Privacy leaders conduct focus groups to assess
Examples of customer focused metrics included: user interface, experience, and satisfaction.
• Increase in personalization (as proxy for trust, • Competitive benchmarking. Privacy leaders
which is difficult to measure). Privacy leaders compare indicators against competitors; including,
demonstrate an increase in consent rates, for example, number of cookies by purpose;
consumer opt-in rates or collection of first party security certifications; last update of privacy policy;
data as a measure of better data utilization. company PR relating to privacy/security; and more.
Risk Management
M
embers of the FPF Privacy Metrics Working • technology risks (e.g., increased use of customer
Group discussed using privacy metrics as part collaboration tools, migration to cloud)
of a risk management strategy. Where privacy
• vulnerabilities and threats (e.g., cyber threats,
metrics are considered Key Risk Indicators (KRI) they are
incident history and trends)
reported in a more formal manner, including possibly to
the board audit or risk committee. Once recognized as a • changes to business environment and processes
source of risk, privacy meets the discipline of risk man- (e.g., expansion to overseas markets, COVID-19
agement, which uses quantitative analysis to measure related digital transformation)
reputational, operational, financial, security, and legal • reputational risks (e.g., negative media coverage,
risks. Members recognized the difficulty of introducing possible impact on share price)
the contextual and nuanced concept of privacy into
quantitative risk management frameworks. By using • industry standards and best practices (e.g., ISO,
metrics, privacy leaders can better assess, manage, NIST, PCI/DSS)
mitigate, and present risk factors such as: One member demonstrated the use of a two-by-two
• legal and regulatory risks (e.g., compliance with graphic for assessing privacy risks in various vectors
new laws and regulations, changes introduced by and tracking their mitigation over time, as a measure of
the Schrems II case) performance of privacy teams:
CONSEQUENCE
FUTURE OF PRIVACY FORUM | PRIVACY METRICS REPORT | SEPTEMBER 2021 10
Benchmarking and Maturity Models
P
rivacy leaders use metrics to compare their benchmarking they use:
privacy programs to benchmarks and maturity • Legal frameworks such as:
models. According to the Cisco Survey, orga-
nizations with more mature privacy practices realize – OECD Privacy Principles
greater business benefits from privacy practices than – APEC Privacy Principles
those less mature. Moreover, mature privacy orga-
– GDPR (and UK DPA)
nizations are better equipped to handle changing
privacy requirements around the globe. Of course, – US federal legislation (HIPAA, GLBA, COPPA,
the maturity of a privacy program also depends on an FCRA, VPPA)
organization's overall maturity, which affects the type – US state legislation (CCPA, CPRA, CDPA, BIPA)
and volume of data processed, size of privacy team,
– LGPD (Brazil), PIPEDA (Canada), Privacy
existence of a CPO, etc.
Act (Australia & New Zealand), PIPA (South
• Early stage programs focus on compliance with Korea), etc.
laws and regulations, real time response to inquiries
• Standards such as:
and incidents, and establishing privacy operations.
– AICPA GAPP
• Mid-level programs seek efficiencies in automating
processes, meeting SLAs, responding to DSRs, – NIST Privacy Framework v2
incidents and requests, supporting additional – ISO (27001, 27018, 27701, 29100, ISO/PC317)
business functions and becoming more proactive,
for example, through privacy by design.
• Industry frameworks such as:
M
embers of the FPF Privacy Metrics Working • Automated Privacy Governance Controls
Group discussed various technologies and tools and Processes
they deploy to collect, assemble, maintain, count, • Privacy Mailbox
and analyze data for privacy metrics. They agreed that
the key to creating a robust privacy metrics system is • Privacy Intranet
a shift from manual processes to ticketing systems and • Privacy module on enterprise software. (e.g., setting
other tools with automated collection and processing of up FAQs on privacy, instances to log privacy tickets,
data for metrics. Tools range from commonly used plat- number of contracts, value of contracts)
forms such as email and intranet to enterprise software, • Privacy GRC Tools
Governance, Risk and Compliance (GRC) solutions such
as ServiceNow or RSA Archer, and dedicated privacy • Privacy software tools (e.g., tools for tracking DSR
tools. Tools for collecting, maintaining, and presenting requests/fulfillment, retention, policy management
privacy metrics include: and violation)
P
rivacy leaders use metrics for purposes ranging holders, ranging from employees to senior manage-
from demonstrating and documenting compliance ment and the board, as well as with external parties,
to asserting the value of their privacy program for including customers, consumers, investors, regulators,
data governance, risk management, business enable- advocates, and the media. This report provides a snap-
ment, and customer trust. Metrics range from activity shot of the privacy metrics programs and strategies
based to outcome focused; distinguish between early deployed by industry leaders at the FPF Privacy Metrics
stage and mature programs; help benchmark against Working Group. Additional work will focus on case stud-
industry standards or competitors’ practices; and facil- ies demonstrating how group members use metrics in
itate risk management and mitigation. Privacy leaders various real-world situations.
leverage metrics to communicate with internal stake-
Glossary