Introduction

How do you know the current status of your project? How is the current status of the project
communicated to your organization's management team? How do you ensure your clients know the
project's status? When do you introduce change or corrective action into the project based on the
project's current status? When do you decide to allow the project to continue “as is” with no changes
being introduced? The answers to these question stem from the Monitoring and Controlling Process
Group.

The answers to these questions seem easy on the surface. Yet, in my experience, I find that these are
some of the most difficult questions faced by project managers. I'd like to use a real-life example to
illustrate the complexity of these questions. In one of the most politically important projects of my
career, I reported the status as “green” on the weekly status report prepared immediately before I
departed on vacation. Like a stereotypical project manager, I kept in touch with the project team during
my vacation. No new major issues or concerns arose during that week; however, upon my return, the
PMO director overseeing the project suddenly began reporting my project status as “red” to senior
management. He pressured my team to make changes in the project's execution. This experience started
me on the journey of asking how one really knows the status of a project.

At the same time, I rediscovered the systems thinking work of W. Edwards Deming and Peter Senge. I
wondered how systems thinking concepts impacted project management, especially in the area of
monitoring and controlling.

This paper applies system thinking concepts to the Monitoring and Controlling Process Group. Its goal is
to help each of us develop new tools and techniques while re-emphasizing the basics found in the
PMBOK® Guide. Key concepts include:

Reviewing PMBOK® Guide basics

Selecting key performance indicators (KPIs)

Knowing when to implement change or when to leave a project “as is”

Determining whether or not proposed changes will positively or negatively impact the project

PMBOK® Guide Basics Review

The PMBOK® Guide provides a basic foundation regarding the Monitoring and Controlling Process
Group, which states that monitoring and controlling processes accomplish three things:

Track, review, and regulate the progress and performance of the project
Identify any areas in which changes to the plan are required

Initiate the corresponding changes (PMI, 2008, p 59)

Project managers apply monitoring and controlling processes to eight of the nine Knowledge Areas
including Project Scope Management, Project Time Management, Project Cost Management, Project
Quality Management, Project Communications Management, Project Risk Management, Project
Procurement Management, and Project Integration Management. The Monitoring and Controlling
Process Group excludes only the Project Human Resource Knowledge Area.

On the surface this sounds simple enough, until one stops to think about the depth and breadth of the
monitoring and controlling activities described throughout the PMBOK® Guide, which include:

Comparing planned results with actual results

Reporting performance

Determining if action is needed, and what the right action is

Ensuring deliverables are correct based on the previously approved definitions and/or requirements

Acquiring sign-off on deliverables by authorized stakeholders

Assessing the overall project performance

Managing risks

Managing contracts and vendors

In other words, project managers use the monitoring and controlling processes to translate project
execution data from information into knowledge. This knowledge is then used to make the right
management decisions and to take the right actions at the right time. Generally speaking, project
managers face two choices in most situations:

Recommend the implementation of appropriate changes, which are planned and approved by the
change management process

Allow the project to function “as is”

Key Performance Indicators

Key performance indicators (KPIs) are a tool to help project managers synthesize the information
gathered from monitoring and controlling processes into meaningful knowledge. KPIs help the project
manager turn the scope statement into measurable objectives against which the project's success is
judged. As information is gathered by monitoring and controlling processes, the project manager
organizes it and analyzes it in order to understand how the project's current status measures up
compared to the selected KPIs.
Optimally, three to five KPIs should be selected for each project. Determining the exact number of KPIs
for each project is more art than science. A project may require more or less KPIs based on the project's
scope and complexity. Remember that each KPI must be “SMART:” specific, measurable, attainable,
relevant, and timely.

KPIs cannot be arbitrarily selected by the project manager; rather, the project manager applies tools like
brainstorming, interviewing, and the Delphi technique to determine what the project's stakeholders
want the KPIs to be. The project manager assists stakeholders reach a consensus decision regarding KPIs.
This shared decision-making process helps each stakeholder develop a sense of ownership for each KPI.

Consider the following questions when determining the project's KPIs:

What are the project's overall business goals and objectives?

How will the user use the project's deliverables?

What is the impact of not completing the project?

Part of the KPI selection process also involves identifying what the project may need to be “bad” at in
order to be “good” at the important things. For example, one KPI may be that the project deliverables
must be completed by a contractually obligated or market-driven date. The team may decide it needs to
be “bad” at eliminating overtime expenses in order to be “good” at achieving schedule milestones.

A KPI rule I've learned the hard way to is be sure that everyone is on the same page. If a total stranger
stopped a project stakeholder, sponsor, or team member in the hall, can that person articulate clearly
and succinctly the project's KPIs along with the project's status as compared to the KPIs? Think about
Dorothy's journey down the yellow brick road to the Emerald City. All the project team members—
Dorothy, the Scarecrow, the Tin Man, and the Cowardly Lion—could articulate the KPI of reaching the
Emerald City and knew their status based on that goal (Johnson, 2006, p. 48).

To Change or Not to Change?

Projects by their very nature change over time. Oftentimes this change comes directly from the project
team and project manager. We initiate corrective actions or process improvements in the project. Why
do we continuously “change,” “fix,” or “improve” projects? We implement change because we observe
variation from our expectations or target. Yet, not every variation requires that changes be made to the
project. Project managers need to answer the following questions for each project:
How do we decide whether or not to “change,” “fix,” or “improve” the project?

How do we determine if a change will positively and/or negatively impact the project?

The first step to answering these questions is to understand the type of variation observed. Two types of
variation exist: common cause variation and special cause variation. Common cause variation refers to
events that fall inside the upper and lower control limits on a control chart. In other words, this variation
is expected even though the process is under control. Special cause variations are events that fall outside
the upper and lower control limits on a control chart. These events can be unexpected.

A simple example of this is your commute to and from the office (assuming you actually travel into the
office). Each day you drive the same route; yet, the total commute time can be longer some days and
shorter on other days. This is common cause variation. One day your car has a flat tire, which extends
your daily commute time. This is a special cause variation. Notice you make different decisions based on
each situation. For the common cause variation of daily commute time, you make no adjustments to the
time you depart from the house each morning or the process continues “as is.” For the special cause
variation of the flat tire, you must fix the flat tire, or you take corrective action.

Deming reminds us there are two common mistakes made when working with variation:

Mistake 1: To react to an outcome as if it came from a special cause, when actually it came from
common causes of variation.

Mistake 2: To treat an outcome as if it came from common causes of variation, when actually it came
from a special cause (Deming, 1994, p. 174).

It is impossible to completely eliminate the occurrence of either mistake. A control chart can help project
managers avoid these mistakes. Ideally, the control chart's upper and lower control limits relate back to
the project's KPIs.

So, what does this mean to the project manager? If the project manager determines that an observed
variation is within the control limits, generally speaking, no change is required and the project execution
can continue “as is.” If the observed variation is determined to be a special cause variation, the project
manager must lead the project team through identifying and implementing the correct change.

Take, as an example, a situation faced in a typical software development project. During the testing
stage, the test team executes a varying number of test cases each day. Does the project manager need to
initiate a change in the project as a result of this observed variation? The answer is: it depends. Is the
variation within the expected results defined by the control chart's upper control limit or lower control
limit? If so, this is probably common cause variation and no corrective action is required. Is the variation
outside the control limits on the control chart? If so, this is probably a special cause variation and
corrective action needs to be taken.

The ability to make the correct decision whether or not to implement change is a key skill required by
project managers. Project managers can improve this skill by studying the lessons learned from Deming's
red bead experiment and funnel experiment.

Deming's Red Bead Experiment

Deming's red bead experiment demonstrates some assumptions made based on data gathered from
monitoring and controlling. The demonstration is based on a role play using members of the audience.

He asks six willing workers to remove beads from a bin containing a mixture of red and white beads.
Compare this to assigning project team members to a task. The willing worker inserts a paddle with pre-
drilled holes into the bin to remove beads. The goal is for the willing worker to only remove white beads,
because red beads are considered defects. The willing worker can only use the paddle. No human hands
may touch the beads. Three inspectors count the number of red beads on the paddle. Compare this with
project team members executing tasks following established organizational procedures. The inspectors
represent the project's quality processes. A recorder documents the number of red beads. Compare this
with gathering data for monitoring and controlling purposes.

The bead extraction process is repeated multiple times to represent multiple days. Obviously, it is
impossible to meet the goal of only removing white beads from the container. The willing worker
actually exercises no control over how many red beads are on the paddle. However, this does not hinder
the supervisor from rewarding the person with the lowest red bead count and punishing the person with
the highest amount of red beads during each rotation or work day.

The supervisor or project manager also introduces new management-sponsored programs, such as Zero
Defect Day and bonuses. Management determines what corrective action to implement based on their
interpretation of the results recorded by the recorder from the preceding round of the bead selection
process. Compare this with the changes or corrective actions made to a project as a result of the
monitoring and controlling process.

After Day 4, the management decides to implement a significant change in the project by firing the three
worst performing workers and retaining the three top performers. The remaining three workers execute
the task and the variation continues. The results are disappointing causing the foreman to announce the
firm (or the project) will be shut down.

The lessons learned from this experiment apply to project management. These include:

Ranking people is wrong and demoralizing. What is really being ranked is the effect of the process or
project on people.

Paying for performance is futile as it is merely rewarding and punishing the process or project.

Rigid procedures to accomplish assigned tasks are a display of bad management. Workers should be
allowed to continuously improve their performance and the performance of others.

The root cause of the issue was never tackled by working with the supplier of the beads to eliminate or
reduce the red beads in the container.

No basis existed to assume that the best team member today would be the best in the future (Deming,
1994, pp 154–171).

Deming's Funnel Experiment

Deming uses the funnel experiment to illustrate the results of management tampering or making
inappropriate changes to the execution of a task. This experiment requires a funnel, a marble, and a
target on which to mark the marble's resting place.

The first round of the experiment involves the participant to aim the funnel at the target. Keeping the
funnel at the same height and location, the participant drops 50 marbles through the funnel and marks
where each marble hits the target. The participant observes variation. The marble drops form a circle
around the original target. Again, compare this to executing a project task like executing multiple test
cases.

In round two, the participant attempts to compensate for the variation. The participant moves the
funnel from its last position to compensate for the variation. For example, if the marble comes to rest 6
centimeters southeast of the target, the participant moves it 6 centimeters northwest from its last
position. The participant notices that the variation increases from the first round. The diameter of the
circle containing the marble drops is greater than the first circle.

Still not satisfied, in round three the participant moves the funnel using the target as a reference. The
participant moves the funnel the opposite direction from the target of where the marble hits. The
participant observes a back-and-forth pattern variation from the target.
In the final round, the participant moves the funnel to be over the location of the last marble drop. The
drops move farther and farther away from the target in a pattern Deming calls the “Milky Way.”

After three attempts to make changes to the initial idea of holding the funnel steady over the target, the
participant realizes that the first way was indeed the best way to drop the marbles. The main lesson
learned is that sometimes the project should just be left alone. Deming calls the three attempts at
moving the funnel management tampering.

A project manager engages in tampering when taking action based on the belief that the observed
variation is a common cause variation rather than a special cause variation. This over reaction causes
measurable losses due to increased variation.

By the way, Deming does point out some appropriate changes that can be made in the process to
minimize variation. The first is to move the funnel down closer to the target. The second is to use a terry
cloth to keep the marble from rolling (Deming, 1994, pp 190–206).

How to Implement Change

Based on professional experience, we project managers know we will encounter special cause variation,
which will require that we implement change or corrective action in the project. Determining what
change we recommend for approval requires as much thought as evaluating the observed variation.

System thinking teaches us that in a project, Action A affects Action; Action B affects Action C; then,
Action C affects Action D; and, finally, Action D affects Action A. This is referred to as causality. Exhibit 1
illustrates how one action affects another:

Causality illustration

Exhibit 1 – Causality illustration.

This means we as project managers must consider how changing Action A affects all the other actions in
the system or project. Project managers can use Deming's Plan-Do-Check-Action (PDCA) Cycle as a tool
to help better understand the impact of change on the project (Exhibit 2).

Deming's PDCA Cycle

Exhibit 2 – Deming's PDCA Cycle.

Think of Deming's model in terms of executing a scientific experiment to determine whether or not the
proposed change will benefit or harm the project. Using Deming's model, the first step to implement a
proposed change is to plan. The plan details what change is going to be tested and how to test the
change. If multiple options exist, the project manager will need to select which option to test. Ideally,
this option should be the lowest negative risk and/or highest yield change. Then, the project manager
writes a test plan. A hypothesis or anticipated result is the final planning step.

Step two is to execute the plan to test the proposed change. Ideally, the testing is executed on a small
scale or prototyped. The test's duration should only be as long as necessary to confirm or invalidate the
hypothesis. After testing is complete, the project manager studies the test results, asking questions such
as:

How does the actual result compare with the anticipated result?

What did we learn from implementing the change on a small scale?

What went wrong?

Finally, the project manager decides whether or not to act. The change can be adopted or abandoned.
The proposed change may require revision. If this is the case, then the project manager repeats the
cycle.

Unintended Consequences

Beware of unintended consequences when implementing change. Senge refers to unintended

consequences of “fixes that backfire” (Senge, 1994, p. 125). These unintended consequences negatively
impact the project. Often, the negative impact occurs much later. Initially the change may appear to
mitigate an issue, but that issue returns with a vengeance. The stereotypical example of this quoted by
system thinking authors is increasing profits through layoffs. Profits go up in the short term, while the
company may struggle in the long-term due to the loss of experienced workers.

Senge attributes unintended consequences to a failure to understand the root cause of the issue or
special variation being addressed. Rather, the change is a “fix” to a symptom causing the variation. Use
tools such as a fishbone diagram to help identify the variation's root cause.

Conclusion and Application

We've covered a lot of ground in this paper. We reviewed the basic Monitoring and Controlling Process
Group information documented in the PMBOK® Guide, which led to the assertion that project managers
use the monitoring and controlling processes to translate project execution data from information into
knowledge. This knowledge is framed using three to five KPIs identified by the project's stakeholders.

With this knowledge, project managers must determine if a project issue or observed variation is a
common cause variation or a special cause variation. The project should be left “as is” if the variation is a
common cause, whereas changes are required to address special cause variation.

If changes are required, Deming's PDCA experimentation cycle will assist project managers in
determining if the proposed change will positively or negatively impact the project. The project manager
must also keep in mind Senge's warning to beware of unintended consequences cause by fixing a
problem symptom rather than its root cause.

Now I challenge you to apply this discussion to your current project by asking yourself the following
questions:

What are the three to five KPIs for my project? How are those measured?

What is the most important decision that I face on my project right now because of a project issue or
observed variation?

How will I determine if the observed variation is a common cause variation or a special cause variation?

How will this knowledge impact my decisions for the project?

If I decide to implement change on the project, how can I use the PDCA cycle to execute an experiment
to ensure the proposed change addresses the issue's root cause and has no negative impact on the
project?

How will I communicate this decision-making process to project stakeholders?

The Standish Group founder, Jim Johnson, writes: “Learning the word ‘no’ is the hardest lesson for many
project managers” (Johnson, 2006, p. 92). Truly great project managers understand the difference
between special cause variation and common cause variation. Great project managers are not afraid to
let the project continue “as is” rather than make unnecessary, potentially detrimental changes.