PROCESS

IMPROVEMENT
MEASUREMENT
HANDBOOK
March, 2019
 PROCESS IMPROVEMENT MEASUREMENT

Table of Contents
Training Overview.......................................................................................................................................2
Review.........................................................................................................................................................2
Why Measure..............................................................................................................................................3
When to Measure........................................................................................................................................4
What to Measure, Step 1............................................................................................................................5
What to Measure, Step 2............................................................................................................................6
How to Measure..........................................................................................................................................7
Process Data Sources...................................................................................................................................8
Setting Targets.............................................................................................................................................9
Documenting Your Measures....................................................................................................................10
Displaying & Communicating Data............................................................................................................10
Ethics, Risks and Cautions..........................................................................................................................11
Resources..................................................................................................................................................12
Examples...................................................................................................................................................13

Page 1 of 16
 PROCESS IMPROVEMENT MEASUREMENT

Training Overview
Training Objectives
 Learn why measures are important for improvement
 Understand why, when, what, and how to measure processes
 Generate ideas for measuring your processes
 Gain basic ground rules for using and visualizing measures

Course Agenda
Throughout the course we will practice what we learn

1. Welcome and Introductions

2. Why to measure
3. When to measure
4. What to measure
5. How to measure
6. Basic principles of data collection and reporting

Review
Continuous Improvement (CI) is an ongoing effort to improve products, services, and processes.

A process is a series of steps or tasks to achieve an end or result. In this course, we focus on measuring
processes, not programs or organizations.

Page 2 of 16
 PROCESS IMPROVEMENT MEASUREMENT

Why Measure
We want to understand how well the process is performing so that we can improve!

Measures help us to…

MONITOR and REPORT how well a process is working

Example: We have met our standard of processing 95% of transactions within 30 days for each of the last six
months

UNDERSTAND where there are issues, errors, or delays

Example: 75% of customer wait time occurs during the assessment phase

DIAGNOSE the root causes of problems

Example: 98% of errors can be attributed to manual calculation errors

ASSESS whether performance has improved

Example: The improvements we made reduced the proportion of applications that take more than 30 days to
process from 60% to 20%

ESTABLISH reasonable performance standards and models

Example: One staff member can reasonably process 200 transactions per week
Example: We will respond to our customers within 30 days

Page 3 of 16
 PROCESS IMPROVEMENT MEASUREMENT

When to Measure
Continuous improvement is a cycle. Measurement is a part of every phase of the cycle.

Baseline metrics and targets Measurable evidence and validation of

root causes
Details of process performance (when
do delays occur? where do errors
What goals aren't Why is the
occur?)
we meeting? problem
What is the happening?
problem? What Why aren't we
do we want to meeting our
improve? goals?

What changes
will remove or
reduce the
Take Action! causes of the
problem? How
Are we doing what we said we would? can we fix it?Forecasting and modelling
What is the outcome? What has
changed?

Getting Started
Recall the CI Fundamentals:

1. What is your process?

a. What do statutes/regulations require of the process?
2. Who are your customers?
a. What do they want and need?

Before we can identify the right measures, we first need to identify the process, its customers, and their
needs, as well as any statutory requirements that the process must meet.

We need to

1. Identify the process,

2. how it starts and ends, and
3. who the customers of the process are.

Page 4 of 16
 PROCESS IMPROVEMENT MEASUREMENT

What to Measure, Step 1

What do you want to know, and why?
Measures should help you answer important questions from customers, staff, leadership, and other
stakeholders.

What does How will you

What do they
the process Who is it for? know if they
want?
do? got it?

We should measure aspects of our processes that we want to manage or improve.

Measures help us understand how our processes operate, perform, and how customers experience the
process output.

To answer whether a process is working, we need to first define what it means for it to “work”:

Is the process working?

What does it mean for the process to be working?
Does the process meet customer Is the process conducted
& regulatory requirements? efficiently?
How
How many
much do
What How When staff do Waste Errors
we
we have
process

Attitude/
Quality
treatment

In other words,

1. Does the process produce the product or service the customer requires?
a. in compliance with rules/regulations?
b. without errors or within established thresholds?
c. efficiently, within cost constraints, without waste?
2. What aspects of the process do we need to manage?
a. Staff
b. Materials

Page 5 of 16
 PROCESS IMPROVEMENT MEASUREMENT

c. Budget

Requirements-gathering is a lot of work! What’s the point?

Skipping this step can result in unnecessary and burdensome measurement that can
Alert have negative, unintended consequences! The purpose of measurements must be
clear!

If you don’t have a clear reason why you are measuring something, or haven’t confirmed that a measure
aligns with any requirements, you may

 Waste time and money gathering, reporting, and responding to the measure
 Face unintended consequences of measuring the wrong things (more on page )

What to Measure, Step 2

The requirements you’ve identified, and your key questions about meeting them, will dictate what types
of measures you’ll want to use.

How much does

the process How long does Customer What is causing
produce? the process
Operational Performance Experience the problem?
take? Improvement
(how much) (how well) (customer
Doservice)
customers
How many staff
receive what
do we have?
they require?

Are our
How many
solutions
What does the errors occur?
working?
process cost?

All of these types of measures can be important. Customer experience and performance measures can
be more difficult to develop and implement, but may tell us more about the effectiveness of our
processes than operational measures.

Page 6 of 16
 PROCESS IMPROVEMENT MEASUREMENT

A good measure is…

Important to the Objective and Close to the process,

Easy to understand
customer unbiased not lagging
Keep it simple! If they wouldn't care, Don't use measures Measures that
think twice. that hide or skew the provide quick
facts. feedback are
preferable to those
that show what
happened a long
time ago.

How to Measure
Measure Types
The types of measures you select should directly align with the questions you’re trying to answer, and
the requirements you’ve identified.

Customer
Time Cost Quality
Experience

Root
Volume Complexity
Causes
Remember, you can measure at all stages of the process, input, during the process, and output.

Data

Data Data Data Data

Data

Input Process Output

Page 7 of 16
 PROCESS IMPROVEMENT MEASUREMENT

Process Data Sources

Real talk about process data:

 Collecting data about your process does not have to be complicated.

 Process data always exists or can be collected.
 Process data will rarely be perfect.
 The more important the metric, the more effort you can devote to data collection and
maintenance.

REMEMBER
The effort you put into measurement should serve to help you
manage or improve your process. If no one ever looks at, pays
attention to, or cares about a measure, stop collecting it!

The type of data you’re looking for can help you narrow down where to find, or create it.

Time Cost Quality

Process maps Financial reports Database/system reports
Time studies Audits/quality assurance Data
Database/system reports Observational studies

Customer Volume Complexity

Satisfaction/Experience Database/system reports Process maps
Customer surveys Financial reports Process
Complaint data standards/documentation
Observational studies

Root Causes
Check sheets

Page 8 of 16
 PROCESS IMPROVEMENT MEASUREMENT

Setting Targets
Identifying where you want your process to be can be easy or difficult, depending on the situation. If
there are specific statutory requirements or penalties related to time or quality, those will likely come
first. But if those don’t exist, or if you’re already meeting them, you’ll need to take into account what
your leadership demands and what is reasonable. More advanced efforts may use calculations that can
identify an achievable “magnitude of improvement.”

Leadership
Demands
Statutory
Calculations
Requirements

Customer Target What Seems

Demands
s Reasonable

When engaging in improvement efforts set stretch goals that are potentially achievable, and celebrate
all improvement, even if you do not reach your goal!

Page 9 of 16
 PROCESS IMPROVEMENT MEASUREMENT

Documenting Your Measures

You MUST document important information about your measures, so that

 People you are reporting them to (and others who might see them) understand what they do,
and do not, mean.
 Your team understands who is responsible and accountable for collecting and reporting the
measure data.

Key Information to Document

Measure scope Purpose Target

What is included, where does Why are we measuring this? What is our goal?
it start/stop, etc.

Formula/Calculation Frequency of Collection Source

How is the measure How often is the data Where does the data come
calculated? collected? from?

Responsibility for Data Reported To Responsibility for

Collection Where/how/to whom is the Improvement if Target
Who collects the data? data reported? Not Met
Who is responsible if the goal
is not being met?

Displaying & Communicating Data

There are many resources available to learn more about data visualization. Here are a few general tips:

 Sometimes a pretty chart is NOT the right tool for your message. Consider when a single number
(“X improved by 95%!”) or a detailed table would serve your audience better.
 The easiest and cheapest way to assess your visualization is to test it with a person in the
intended audience. Can they understand it? What are their questions? How could you make it
clearer?
 Label your charts and axes and note the data source.
 Generally, always start numerical axes at zero, this shows the true difference between the data
points, rather than overemphasizing small differences.

Page 10 of 16
 PROCESS IMPROVEMENT MEASUREMENT

 Consider when raw numbers or percentages are more effective for communicating your
message, or whether both are important. (“We served 1% more Minnesotans this year” vs. “We
served 20,000 more Minnesotans this year.”)

Use Your Measures!

Share, Display, Report and USE Your Measures

We measure to improve.
Have No Shame! If we are consistently meeting our goals, we may want to set
new goals.
Have No Fear!
If we are NOT meeting our goals, we (should) want to know!
Honest Data Welcome Here!
Sharing data that shows that a process is not working takes
courage, and is absolutely necessary for improvement.

Improvement starts with acknowledging that we are not

where we want to be. This can be difficult in organizational cultures accustomed to hiding or punishing
anything that may put them in a negative light.

Tips for Changing the Narrative

 Always focus on the team’s goal and desire for improvement for their customers, not “look how
bad things are” but “we want to get to x”
 Track and celebrate even small improvements, put charts on the wall
 Contextualize the data, were there events or are there trends that influence it? Make sure to
note these

Ethics, Risks and Cautions

DO NOT use process measures for substantial rewards or punishments.
Measuring individuals or teams and using threats or enticements to
motivate improvement can result in falsified data. (Examples include VA CAUTION
hospital wait times and teacher bonuses based on student test results.)

There are usually multiple, equally accurate, ways to collect and report data
about a process. Usually they will show slightly different results. We can
ethically use whatever accurate data we choose, but is important to use the same data source when
making comparisons, or to very clearly state when comparing different sources.

Page 11 of 16
 PROCESS IMPROVEMENT MEASUREMENT

Resources
Minnesota Office of Continuous Improvement
http://mn.gov/ci

MDH Public Health and Quality Improvement Toolbox

http://www.health.state.mn.us/divs/opi/qi/toolbox/

ASQ Quality Resources

https://asq.org/quality-resources

Results-Based Accountability (RBA) Guide

http://raguide.org/

Page 12 of 16
Examples

EXAMPLE 1 Page 13 of 16
 PROCESS IMPROVEMENT MEASUREMENT

EXAMPLE 2 Page 14 of 16
 PROCESS IMPROVEMENT MEASUREMENT

EXAMPLE 2 Page 15 of 16