Professional Documents
Culture Documents
Management
Member of the
Change Control
The following table documents the actual development stage of this document.
Every change made to this document requires a new issue.
This case study explains the Six Step To Completion – method for progress tracking in
projects. Progress tracking is a project management task, but it has specific effects in
software projects. In this case it was the first time software people had to get used to
metrics and that they draw some benefit out of it.
The critical success factor is to link software quality assurance to progress reporting. A task
is no longer finished when the planned effort had been spent. We want the deliverables
resulting from that task be reviewed and approved. Based on that data we track actual
duration and effort spent in projects and compare with plans.
The effect is surprising: software projects suddenly behave “normal”. Teams stay focused
on the planned tasks and keep to the deadline. Although bad effort estimation cannot be
completely avoided, it is detected in the very early stages of the project. Delivery rates
become apparent from the beginning.
Index of Figures
Figure 1: The Six Step To Completion Model for Measuring Objective Evidence 7
Figure 2: Sample Project Management Overview 11
Figure 3: Sample Progress Track with Issue History 12
Figure 4: Sample Personalized Ticket 13
Figure 5: Wall Sized Print of the Progress Track as Paper Tool 14
The Problem
The Department for Labour and Economy of the Swiss canton of Zurich decided
to undertake a bold step towards e-Government in 2002 when starting a project
to ease the issue of work permits.
Especially in the ICT industry, but as well for banking and pharmaceuticals, the
need to exchange international experts is apparent. In many other aspects of
economical life, having the ability to transfer experts from one country to another
is decisive for the attractiveness of a site. Thus setting up a web site that allows
applying for a work permit through the Internet and – if the application is all–
inclusive for the needed documents and the applicant fulfils the law’s require-
ments – issuing the permits instantly is a major competitive advantage.
The Project
duration of about seven month, and a To-Do list that an impressing tendency to
grow in size.
Thus the project went the way all such projects must go: It hardly made the first
milestone, already with a long list of To-Do’s, and completely missed the second.
The author remembers his first project steering meeting where the project
manager stated that one of these three-month tasks is “98% finished”. The
author remarked that, when this statement had been true the day before the
meeting, this task must have been finished by this very afternoon. Obviously it
was not. In reality not even the hosting was prepared, and the software was
buggy and very late.
The Dilemma
Obviously this was not state–of–the–art project management. The contract did
stipulate, but not substantiate project management. It did neither mention a
professional (PMI) certificate, nor any specific project management method.
When the author joined the project and proposed to use progress metrics to the
supplier, he did not accept the suggestion because it was not foreseen in the
contract.
For the government department, there was no other way than to insist on the
schedule as stated in the contract but they had to let the supplier decide how to
meet its obligations. All they could do, and what they did, was to spend
additional money for a second project manager who should take over after
completion of the contract, and for independent testing, to avoid having to rely
on the supplier’s testing records. We had to endure another three missed
milestones before the supplier finally admitted that the delivery date had passed
and the contract failed.
The Taskforce
Thus the government department installed a taskforce to take over and lead the
project to its goal. The primary objective was that the supplier finished the
delivery of the web application as soon as possible; thus it was not an option to
stop the project. So how do you deal with people from whom you depend and
who can listen neither to you nor to common lore nor even simply to good
reason?
The answer was a very simple metrics that was so simple those people from the
supplier did not even understand that it is a metric indeed.
We started with a To-Do list because that was a terminology people could follow.
However, each of the To-Do’s was connected to a distinctive deliverable, as
every project manager does with tasks in his project plan. That deliverable
uniquely identifies the To-Do and gives objective evidence for completion of the
task. Professionals with an auditing background understand the term “objective
evidence”. Instead of arguing if the task or To-Do had been completed or not,
the evident status of the deliverable tells us without asking people’s opinion.
Figure 1: The Six Step To Completion Model for Measuring Objective Evidence
Input
Assignment
Assignment Template
Template
Idea
Idea Draft
Draft Review
Review Finalization
Finalization App-
App -
App- Operations
Operations
roval
roval
Output Draft
Draft version
version Review
Review report
report Release
Release Distribution
Distribution
All we need is to provide a project repository where to post draft versions, review
reports, released versions, and evidence for operational use such as distribution
lists for documents and build logs for software.
We use the Six Steps to Completion – model to measure progress based on
objective evidence for completion for each of the six steps. The idea stage
accounts for 10% of the total completion metric; the draft stage for another
30%. Then comes the review stage with 15%, the finalization stage with 20%,
approval gives 15% and when put into operational use the last 10%. The last
stage is equivalent for making the deliverable available for developers who have
to rely on it and does not necessarily refer to the end users.
Originally these percentages were just a guess. However, in the very first project
where such completion metrics had been used (1999) we discovered that these
percentages correspond quite neatly with the total task duration (not the effort!)
such that we started using the completion percentages for Gantt chart tracking.
The percentage values set forth in Figure 1: The Six Step To Completion Model
for Measuring Objective Evidence seem to fit pretty well for creating documents,
preparing workshops, training, or writing software. Thus, completion rates give
an accurate estimate for the actual task duration. For example, if the
completion rate is 40%, it means that we still need 60% of the total duration,
i.e. 1 ½ times the duration that had already passed. If that was not foreseen by
the project plan then you better update the plan right now to make it consistent
with reality and manage expectations.
Only tasks such as meetings – with the minutes as deliverable – and testing may
have somewhat different percentages.
We have since 1999 a record of around forty projects in a dozen different
organizations that followed the same pattern with a variation below ten percent.
The grading of the steps varies in nine degrees between “just begun” and
“completed”. Thus it is possible to acknowledge work in progress within the Six
Stages To Completion. Also it often happens that review or testing activities and
finalization go in parallel, thus the familiar pattern of two green stages followed
by two yellow ones, see the last line in Figure 3: Sample Progress Track with
Issue History. More questionable it is when deliverables are already in use while
still under finalization, but it happens too, e.g. in object-oriented development.
The benefit of completion metrics based on reviews and approvals is that they
reflect the degree of consensus reached for the deliverables. Reviews and
finalization work still going on, or missing approvals clearly indicate missing
consensus about the delivered results. Thus progress metrics are very
important for management. This worked well in the e-Government project.
You never forget when the supplier’s project manager first noted that his To-Do,
which he thought having finished by “85%”, counted only for some 30%. The
taskforce met then weekly in a room with the actual Six Steps To Completion
displayed prominently at a wall. There was not a single date missed. And
interestingly: even the taskforce cost remained within budget.
Obviously this was more than just a psychological trick. The reason is very
simple: The Six Steps To Completion represents nothing else than ordinary
quality planning. Every serious Quality Plan must provide exactly that: a
reviewing method and an approval procedure for every single deliverable in the
project.
Often quality planning is done at the beginning of the project (although not in
this instance) but then not executed because of other priorities shovelling up.
The effect of such missing quality assurance is not felt immediately, in contrary.
It might even felt favourably for the moment because the QA resources are now
free to do even more draft development. This seemingly positive experience will
then be iterated again and again until it is too late and the quality problems start
becoming apparent.
Exactly that happened in the early stages of the e-Government project, and was
the reason why the supplier’s people were not able to listen.
The Six Steps To Completion – metrics makes this impossible. It short-circuits
the quality plan to the project plan, or in this case, to the To-Dos. The team
silently agrees on the necessary quality assurance being done at the root,
causing significantly less bad surprises later on.
The web site www.workpermits.zh.ch went on-line 20 February 2003, three
months late with regard to the original schedule, but three weeks earlier than
expected in the revised schedule after the taskforce took over. All To-Dos
including all the quality issues discovered earlier were resolved, the hosting
strategies established, and the site started to receive very excited feedback.
Since 1998, Euro Project Office AG developed a tool1 based on MS–Project and
Excel to integrate the Six Steps To Completion – method into project planning. It
was developed and refined during the sequence of projects that the Euro Project
Office supported.
The basic philosophy was usability. This included the need to let the project
manager use his tool without any restriction. This led to the solution to use the
database of MS–Project to store all relevant project data, such as deliverable
specifications, quality planning, quality records, and issues, on top of the data
already present in MS–Project such as planned and actual schedule, planned
and actual effort and cost, resource allocation and usage. All the MS–Project
functionality is thus available but with significant enhancements.
As already explained, the completion rates collected by the tracking process
become in turn available again showing the actual completion percentage of the
planned tasks.
The choice of Excel for visualization makes it possible to easily adapt more
functionality and analyze the project for later improvements. Excel as GUI is
possibly not the best choice, but using VBA and the (loop-free) formula
programmability of Excel allows for fast implementation at a high Function
Points2 (FP) delivery rate. We make more than one FP per hour, or about six
times more than Java or C++. Such a delivery rate outperforms some of the
shortcomings of VBA.
1 The tracking software is Open Source and can be found in the /ch/open project methodology
(www.ch-open.ch, or www.swisma.ch). However, since it is embedded in MS Office, it does not
install without proper support, and is not self-explanatory. At least, you need training, and you
need a project office to run the tool.
2 Unadjusted Function Points according IFPUG 4.1
se
rU
n
tio
Fo
al
za
w
y
pro
vie
ali
aft
ad
a
Management Overview
Ide
Fin
Ap
Re
Re
Dr
70%
Planned Dates
09.05.2003
20.03.2003
60%
29.01.2003 50%
10.12.2002 40%
21.10.2002 30%
01.09.2002 20%
13.07.2002 10%
13.07.2002
01.09.2002
21.10.2002
10.12.2002
29.01.2003
20.03.2003
09.05.2003
28.06.2003
17.08.2003
06.10.2003
25.11.2003
0%
on
w
al
aft
ea
e
Us
vie
ov
Id
ti
Dr
za
pr
or
Re
Actual Dates
ali
Ap
yF
Fin
ad
Re
These reports substitute all other project status reports. Because it is based on
object evidence, such reports can be generated by the project office and need
but comments by the project manager. Thus it replaces administrative work by a
service, and saves cost in the project.
se
s)
rU
Ra tion
n
ay
tio
o
l
(D
va
yF
te
le
za
ty
w
pro
mp
lay
Start Due
vie
ali
ori
aft
ad
a
Ide
History Records
Fi n
De
Date Date
Ap
Co
Pri
Re
Re
Dr
1 Proposal Development 93%
1.3 Customer Value Analysis PM; RM; 6-Jan 100% 5 K 13-Jan 4 17.07.2002: Who is responsible for gathering marketing data?
TPL1; TPL2; 11.06.2002: Marketing department task force to provide available data until 20.1.
AM
1.4 Quality Plan QM 13-Jan 100% 2 27-Jan 13
1.5 Project Plan PM & Team 17-Jan 100% 3 26-Feb 43 17.07.2002: Delay because of resource problems.
Search for available skills. Result due by 20.3. Final Project Organziation must be
communicated to customer.
1.6 Statement of Work PM & Team 13-Jan 93% 4 25-Feb 46 17.07.2002: Some minor points need clarification by the subcontractor
Subco will provide statement by 18.3. Consolidate with project plan.
1.7 Project Calculation PM 6-Mrz 70% 3 14-Mrz 58 17.07.2002: Too many open points.
We conduct a 2nd reviewby 18.3. to get agreement on the open points.
1.8 Business Plan AM 13-Jan 62% 3 K 14-Jan -1 17.07.2002: Project Calculation complete. Formal approval asap after project calculation is
final.
Get approval immediately after 2nd reviewon 20.3.
1.9 Contractor Qualification Buyer 25-Feb 100% 5 5-Mrz 44
1.11 Proposed Solution & Commercial PM; QM; 7-Mrz 100% 3 T 17-Mrz 46
Offer Arch; AM
2.1.1 Clickable Prototype Dev2; Dev3; 24-Mrz 76% 3 5-Mai 66 21.07.2002: Missing feature added
Arch 12.07.2002: Prototype lacks the priority setting feature
2.1.3 Use Case Analysis Arch; PM; 5-Mai 52% 3 19-Mai 68 08.04.2003: 2nd reviewplanned for 11.4.2003
Customer
In the past project experience, it was possible to integrate with an ERP tool such
as SAP, or with configuration management such as CVS.
The tool is also available as an option to SCODi4P from Triloga1, or as an add-on
to Bugzilla2. Currently we investigate migration to a DBMS to improve the GUI.
The progress-tracking tool is not only integrated with the project repository, but
also with the reporting process of each team member. Every team member gets
his own personalized ticket, either via the Intranet, the project home page or any
other suitable communication tool (including paper).
The tickets integrate all of the reporting needed in a project, including time
reporting.
17.3. Requirements Engineering Priority 10% 30% 15% 20% 15% 10% 100%
Ra tion
n
or
tio
l
va
yF
te
le
za
w
Co se
pro
mp
lay
vie
Result
ali
Due
aft
ad
U
a
Ide
Fin
De
Ap
Re
Re
Dr
ID Deliverable date
Dev2;
2.1.1 Clickable Prototype 24.3 76% 5.5 179
Dev3;
14-Nov-02 30. Okt 01. Nov 10. No v 15. No v 07. Nov
Quality Measurements
Release Identification Version Under Test
Pilot Functionality Cycle 1
Delivery item identification Actual Vers Tool Improvements Function Points
Prototype Solution V02 ppt 12 45 36 8 11 0 130
Quality Assessment Re-Factoring Runs
1 Reviews 1 Test Case 0 Regression Tests
Average Test Success Average Test Completion
3.0 Grade 0 - 5 100%
Assessment Result Reviewed by Approved by
Usability Test passed. Arch Customer
Found that requirements need to be PM
refined for various user categories. Date of Approval
un
r
Bu W o
rm k
i on
let t
o
e rr
ed
A
d
r
Pe d
e
Co ate T
Pe l Wo
rio
let
Th rme
Co ted
Ov
dg
e
lin
mp
mp
rfo
c
rfo
tua
tim
rk
pe
se
is
Wo
Pe
Ba
Ex
Ac
Es
Specification 72 82 8 7 97 25
Working on this task
Rapid prototype displaying the Arch 2 1 0 3Hours
functionality needed. Dev2 32 5 7 44
Dev3 48 2 0 50 8 h/d
0 5 d/w
0 4 w/m
ers and maximizes their sense of responsiveness and ownership. This effect
could be worth considering as a trade off.
Figure 5: Wall Sized Print of the Progress Track as Paper Tool
Bad effort estimation cannot be completely avoided. Using the Six Steps To
Completion method, errors in estimation are detected in the very early stages of
the project. Delivery rates become apparent from the beginning.
1 For a project repository that integrates well see itp.montana (www.itp-solutions.ch, in German)
When the team starts understanding that the progress metrics are not a threat
but a valuable help for them, their reluctance against using metrics vanishes.
The GMC team in Hradec Králové (Czech Republic) integrated the method in the
open source tool Bugzilla. Thus tracking project tasks and bug fixes becomes
the same process. Developers have only one tool that supplies them with new
tasks, with their specifications, test cases, records test results and review
finding, and tracks effort and progress. It provides progress metrics and bug
statistics at once, with no pain.
Clearly we would like to add a software sizing metric to the tool. However this
yields some specific problems. The work breakdown structure of a software
project does not automatically coincide with the software boundaries. Some-
times it is possible to group development tasks such that they fit with logical
boundaries, but it make not always sense.
Nevertheless, such an approach is very promising and deserves further
investigation.
Conclusion
The Six Steps To Completion – metric is easy to use, simple and straightforward.
In the beginning, it is often not even perceived as a metric, it stirs only little
opposition, and almost no fear against being measured.
On the other hand, the effects are tremendous. If planning and estimation was
correct, the schedule will be respected. The metric establishes a short – circuit
between plan and team. The project manager perceives unrealistic assumptions
and estimations within a single reporting period. We most often use one week,
sometimes two weeks as a reporting period. Thus corrective actions can be
taken immediately when needed.
Metrics provide huge benefits for people and organizations. Progress metrics
are the door opener for other metrics dearly needed for today’s complex tasks.
Moreover, many more sophisticated metrics such as software sizing using
Function Points or bug statistics provide higher value when combined with
progress metrics. This is why we us no metrics without progress metrics in our
software projects.