1

 Store    Joseph Brannon 



My Account
Explore
Certifications
Membership
Standards & Publications
Learning & Events
Business Solutions
Store
Other PMI Sites


Keeping score

ARTICLE ǀ Risk Management, Quality Management, Earned Value Management ǀ May 2011

PM Network
Bowles, Michelle
Give Feedback

How to cite this article:

Bowles, M. (2011). Keeping score. PM Network, 25(5), 50–54.

Reprints and Permissions

Give Feedback

BY MICHELLE BOWLES

The same old metrics won't cut it anymore. A new triumvirate of project metrics
has risen to the forefront to take its place beside ROI: customer satisfaction, human
resources (HR) and risk management.
 The down economy has certainly affected how metrics are viewed, says Kjell
Rodenstedt, PMP, senior partner, OPG Management, a Stockholm, Sweden-based
organizational project governance consulting firm. “But metrics are also a way for an
organization to improve its maturity,” he says.

YOU SAY YOU WANT AN EVOLUTION?

Even in an economic climate where every dollar, euro and yuan counts, there's an
inherent problem with narrowly focusing on ROI as a metric, says Bruce Webster,
founder and principal at Webster & Associates LLC, an IT risk management firm in
Englewood, Colorado, USA. It's easy to assume a project is ineffective or failed
when only its direct contribution to the bottom line is considered.

Instead, organizations should take a broader view of financial metrics. “Will the
project reduce costs? Are we doing this project because if we don't, we'll fall behind
the competition and become less effective in the marketplace?” he asks.

Financial success measurements aren't the only ones to track. In fact, customer
satisfaction-driven metrics have become the number-one priority for many
organizations, says Bijan Nikravan, PhD, PMP, director of project management
excellence for IT giant Microsoft in Chicago, Illinois, USA. “These metrics may have
previously not been given as much attention,” he says. Effective customer
satisfaction metrics include measuring return business, the number of customer
objectives met, controlled disruption to normal business while projects are underway
and adhering to quality standards that were originally agreed upon.
Give Feedback

Work health index and organization health indexdriven metrics are also becoming
increasingly important. These KPIs measure an organization's health in terms of
individual and team morale, job satisfaction and the like.

“With the downturn of the economy, organizations went through a period of losing
high-potential project staff,” Dr. Nikravan says. As organizations recover and attempt
to get back to “business as usual,” the need to measure employee retention and
satisfaction increases.
 Take a more proactive approach than simply conducting an employee satisfaction
survey once a year.

“Companies such as Microsoft set employee satisfaction metrics and retention

targets at the beginning of each year, and they make ongoing adjustments based on
business and market conditions,” Dr. Nikravan says. “If you have human resource
issues, you can't wait until the annual survey results come out.”

Equally important today are risk management-related metrics, says Cindy Margules,
PMP, project management office director at Convio, a software as a service
company serving not-for-profit organizations, headquartered in Austin, Texas, USA.

Organizations must look beyond merely the number of risks. “Just because you
have identified 300 risks doesn't mean the project is riskier than a project with only
one risk,” she says.

Instead, organizations should track:

How often a risk assessment is completed

The number of risks that are rated low initially but become issues later
How often risk assessments are reviewed
The percentage of actual risks that were identified in advance

DO YOUR METRICS PASS THE TEST?

The need to put hard numbers to project progress—and ultimately portfolio success
Give Feedback

—is twofold: Without solid metrics that relate to key performance indicators (KPIs),
project managers and team members have a difficult time justifying their work, while
executives remain in the dark.

It's a classic syndrome, Mr. Webster says. A lack of tangible metrics means bad
news doesn't get reported upward. “No one wants to go to the boss and tell him or
her that things are really messed up and the team is going to miss the deadline,” he
says. “At the top rank, everyone thinks everything is fine. But down the hierarchy,
the project team knows that's not the case.”
 Metrics help those above the daily trenches definitively know where a project stands
—good or bad. But not just any measurement will do. Mr. Webster says metrics
should pass a three-part litmus test:

▪ Metrics must be informative and predictive. They need to provide relevant,

useful information about what has been achieved or when a milestone will be met.
That sounds like common sense, but plenty of project professionals don't follow this
rule, Mr. Webster says.

“One of the most popular IT project metrics—and one of the worst—is source lines
of code,” he explains. “You end up with a large, bloated code instead of something
that achieves what you really need it to. Project teams will want to achieve the
desired metrics whether or not the metrics truly measure project success.”

▪ Metrics must be objective. They must be calculated consistently and repeatedly,

independent of who's doing the measuring. “Another popular project metric is the
‘walking around’ metric,” Mr. Webster says. “But if you walk around to team
members and ask them how far along they are, they're just going to pull a number
from their hat—and most likely a number that makes them look good or avoid
blame.”

▪ Metrics must be automated. If a metric requires a lot of manual work to be

measured, the project manager or team member responsible for it will eventually
abandon it. Organizations should consider how much time is spent tracking the
metric—it might not be worth the effort.
Give Feedback

THE TROUBLE WTH METRICS

Project metrics can be as vital as they can be imperfect. To get the most out of
hard numbers, organizations must remember:

1 Metrics are often backward-looking. “If organizational leaders have a micro

focus on KPIs, they could miss opportunities or move the organization into failure
because they aren't looking ahead,” says Cindy Margules, PMP, Convio, Austin,
 Texas, USA. Keep in mind that metrics aren't the big picture—they're just a piece of
it.

2 You can't measure everything. Certain project characteristics are intangible and
therefore can't be associated with a metric. “For example, you can't measure
feelings. You might have team members tell you their feelings, but you can't
measure that as a KPI,” says Imad Alsadeq, PMI-RMP, PMP, Thiqah, Riyadh, Saudi
Arabia.

3 Not all metrics are worth the effort. “You shouldn't spend more than what you
get from the measurement,” Mr. Alsadeq says. If a metric takes a significant amount
of time and effort to track, it's probably not the right metric in the first place.

4 Metrics shouldn't be used to evaluate project managers and team members.

If someone's job depends on achieving a certain cost savings within a given project,
he or she will try to accomplish that goal—even if doing so harms other aspects of
the organization, says Bruce Webster, Webster & Associates LLC, Englewood,
Colorado, USA.

5 Negative behaviors can manifest from certain metrics. Ms. Margules worked
with one project team that was given a KPI of not “overserving” clients, measured by
hours dedicated to each. “But in order to not overserve, people were not submitting
their time spent with clients,” she says. “The organization ended up unable to
accurately measure work.” It realized that while the KPI didn't need to change, the
metric used to measure it did.
Give Feedback
 Don't forget about earned value management (EVM) metrics says Bijan
Nikravan, PhD, PMP, Microsoft, Chicago, Illinois, USA. Measure the planned value
against earned value on a regular and ongoing basis, which in most cases is
weekly.

“It is also important to then communicate these metrics with the project team and
stakeholders.”

KEEPING UP WITH KPIs

Give Feedback

“In a tough economy, knowing if a project is on track can make the difference
between profit and loss,” Mr. Webster says. “Gone are the days of launching
projects within a portfolio with unclear objectives, poor scheduling and cost
estimates, and no way to track progress. For those at the higher level, they can't
afford that anymore.”

An organization must create the right portfolio KPIs, and then determine the most
effective project metrics for measuring those indicators.
 “You can't manage what you can't control, and you can't control what you can't
measure,” says Imad Alsadeq, PMI-RMP, PMP, consultation director at Thiqah, a
management and engineering consultancy in Riyadh, Saudi Arabia.

The secret to choosing the correct KPIs is quite simple: Ensure that they're built on
the organization's strategic objectives. “You have to think of the justification for
having the projects or project portfolio,” Mr. Alsadeq says. “That justification will tell
you what you should be measuring.”

Hospital executives, for example, might establish a KPI to increase the number of
operations. “More operations might bring in lots of revenue, but over time, the
hospital might develop a bad reputation from expanding too quickly,” he says.
“Instead, the hospital needs to revisit its vision or mission: Is it looking for money?
Or is its goal to heal patients and help society?” A better KPI should speak to the
hospital's strategic objective to increase the percentage of successful operations.

Creating the right KPIs could make or break an organization. “The wrong KPIs will
drive the organization to an undesirable goal. Efforts will be wasted, and project
teams will perform inefficiently,” Ms. Margules says.

From the beginning, make sure the right people are involved in the decision-making
process. Get input from those who will be tasked with achieving the benchmarks,
including project managers and team members, Mr. Alsadeq suggests.

Not involving those in the trenches creates unnecessary risk—as Mr. Webster
witnessed on a project to launch a new customer-facing website.
Give Feedback

“The company signed a contract with an outside vendor to provide an inexpensive,

out-of-the-box system without talking to its internal IT department,” he says. In the
end, that poor decision meant the IT department had to invest significant time and
effort to address quality issues with the system.

On the surface, the project met the organizational objective for a low-cost system.
Had the IT department been involved early on, though, a more relevant KPI would
likely have been determined.
 Even the “right” metrics can change over time. Some that were appropriate a year
ago may no longer be today. And who better to recognize when a metric is no longer
working than those responsible for tracking it?

“Organizations should not only collect metrics, but after a few months, they should
also collect feedback on metrics,” Ms. Margules says.

This can be done via an anonymous survey or one-on-one meetings, asking project
managers and team members such questions as, “How well is this metric working
for you?” and “What behaviors are you seeing with the project team as a result of
this metric?”

“It's with this kind of feedback that the right adjustments can be made,” Ms.
Margules says.

Above all, choosing the right KPIs and metrics starts in the executive suite. “If a
company doesn't have an overarching goal that everyone can say, ‘This is what we
are working toward,’ then people won't be moving as one unit,” she adds. “Most
organizations don't realize that if they can be crystal-clear on one goal, they can
drive exceptional performance.” PM

EXPLORE MORE ON METRICS IN A CLOSER LOOK

This material has been reproduced with the permission of the copyright owner.
Unauthorized reproduction of this material is strictly prohibited. For permission to
reproduce this material, please contact PMI.
Give Feedback

PM NETWORK MAY 2011 WWW.PMI.ORG

 Quick Certifications Community Membersh Organization
Links ip
Project Management Latest from the About Us
Webinars Professional (PMP)® Community   Membership
  Overview Our Leadership
Certified Associate in Project Discussions  
PMBOK® Management (CAPM)® Become a Collaboration
Guide Templates   Member
PMI Agile Certified Practitioner Leaders Speak
Online (PMI-ACP)® Blogs   Student 
Stay Connected Stay Connected
Courses Membership
Agile Certifications Volunteering Official PMI Blog
         
Events Local 
See All Certifications Chapters
Store What is Project
Support Report PDUs   Membership Management?
Support
Explore FAQs
Contact Us Contact Us
PMI Who are Project
Managers?
Press and Media Press and Media

Careers
Store Help Store Help

© 2023 Project Management Institute, Inc.

PrivacySitemapTerms of usePurchasing TermsAdvertising & Sponsorship
Give Feedback