2/24/2020 Using Assessment to Drive Project Management Capability

Certifications  Membership  Learning & Events  PMBOK® Guide & Standards  Business Solutions  Store  About 
 Register Log In
 

Best foot forward--using assessment to drive

project management capability improvement Search


What are y
CONFERENCE PAPER ǀ Quality Management ǀ 11 February 2009
Crawford, Lynn | Aitken, Alicia | Cooke-Davies, Terry

How to cite this article:

Crawford, L., Aitken, A., & Cooke-Davies, T. (2009). Best foot forward—using assessment to drive project management capability
improvement. Paper presented at PMI® Global Congress 2009—Asia Pacific, Kuala Lumpur, Malaysia. Newtown Square, PA:
Project Management Institute.

Abstract
This paper draws on the extensive research and application of project management capability assessment at the project, program,
and organisational levels. The paper will explore the components of assessment that can be applied to the world of projects
including, what they are for, how they work, and why you'd use them. By the end of the paper you will have learned how to build a
tailored roadmap of project management capability assessments designed to improve your specific organisation's project
management delivery capability. This roadmap will form the basis for understanding where you are today to ensure you know how to
build a better tomorrow.

Introduction
Every organization wants to put its best foot forward and this generally means being seen to have a good strategy and, more
importantly, being able to deliver it. Most organizations have a strategy and many of their strategies look similar. It is the ability to
implement the strategy that makes the difference. Key vehicles for implementation of corporate change and strategy are portfolios of
projects and programs. When an organization recognises that business change and corporate strategy can be delivered through
projects, there is realisation that project management capability is a key to competitive advantage (Aitken & Crawford, 2008a;
KPMG, 1997; Nieto-Rodriguez & Evrard, 2004) and that good governance requires assurance of capability. These are two sound
reasons for investing in development and continuous improvement of project management capability.

To develop and improve project management capability you need to know the status of your current capability, and have a plan for
where you want to be. In fact, just maintaining current capability means that you have to be involved in some form of continuous
improvement or evidence suggests it will go backward (Thomas & Mullaly, 2008). All of this requires an understanding of the
components of organizational project management capability, ways of assessing or measuring those components, and standards
and benchmarks against which to judge how well you are performing as a basis for planning improvement. Assessing your capability,
setting a baseline and measuring your progress toward a desired future state are proven ways of improving personal and corporate
performance.

This paper presents guidelines developing a coherent plan for assessment of organizational project management capability that will
drive improvement. It starts by looking at the components of project management capability and what needs to be assessed with
indications from research of key areas to focus attention for both assessment and improvement to get best return on investment. A
framework for different aspects of organizational project management capability and how to assess them is provided.

What to Assess: The Components of Project Management Capability

If we think about project management capability as encompassing everything within an organization that relates to delivery through
projects we realize that it involves the whole of the organization and not just the project management community and the projects
themselves. It involves the capability of the board of directors and senior management who set the corporate strategy, decide on the
portfolio of projects and programs they will undertake, and how they will resource them. The executive team also provide the vital
link between corporate, program, and project governance, between the permanent and temporary or project-based parts of the
organization, through their role as sponsors (Crawford et al., 2008). The permanent organization or business as usual (BAU) needs
to have the capability to accept the fruits of projects and programs and exploit them to achieve the planned benefits.

Historically, there has been a tendency to think of projects in isolation and focus on the development of methodologies, techniques,
and tools required to manage individual projects, typified by A Guide to the Project Management Body of Knowledge (PMBOK®
Guide). But there is growing awareness of the importance of capability to manage multiple projects, recognized by PMI's standards
for portfolio and program management (Project Management Institute, 2006a, 2006b) and of the need for corporate infrastructure
https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 1/7
2/24/2020 Using Assessment to Drive Project Management Capability
and environment to support projects that inspired the Organizational Project Management Maturity Model (OPM3®) Knowledge
Certifications  Membership  Learning & Events  PMBOK® Guide & Standards  Business Solutions  Store  About 
Foundation.

To really understand our corporate project management capability we can't just look at how each project is managed. We need to
assess at various levels. The executive level needs sufficient agility to articulate strategy, respond to the changing external
environment and make decisions about deployment of resources, and provide the leadership and governance to monitor and control
overall performance. The next level of governance and program capability requires effectiveness. The executive team and other
members of senior management need to be able to effectively provide sponsorship to projects and programs, as the governance
conduit between strategy and its implementation. They must have the leadership ability, credibility, communication, interpersonal,
and self-management skills necessary to govern the project, take accountability for the business case, give direction and make
decisions, critically review progress, manage internal and external interfaces, as well as providing support as required to the project
and program managers and their teams (Crawford et al., 2008). And they usually need to do this for more than one project or
program at a time, on a part-time basis while continuing to fulfil their challenging senior management role. Program managers need
to be effective in orchestrating the resourcing and coordination of multiple projects, mentoring project managers and their teams,
managing stakeholders, and monitoring and ensuring the delivery of benefits. At the project level the aim is to define clear goals and
performance expectations and to emphasise efficiency.

This hierarchy of organizational project management capability can be summarized in the following table (Exhibit 1), which illustrates
the results of analysis of three decades of research into the desired outcomes from project-related endeavours and the capabilities
associated with achieving them (Teague & Cooke-Davies, 2007).

Exhibit 1: Capabilities and Desired Outcomes

Using the framework presented in Exhibit 1, the Human Systems team conducted a survey probing the presence or absence of
specific capability areas at each level, and the degree of success accomplished for each criterion at each level. Results are based
on a total of 339 responses from senior managers, sponsors project managers, and project team members from 44 organizations in
27 countries and 23 industry sectors. Of a total of 43 capabilities identified (10 at the Strategic Capability [Agility] level, 20 at the
Governance and Program Capability [Effectiveness] level, and 13 at the Delivery Capability [Efficiency] level), 13 were shown to
have the significant impact on successful achievement of desired outcomes and therefore may be considered the critical capabilities:

Strategy Capability - Agility

1. Capacity to resource the whole portfolio

Governance and Program Capability—Effectiveness

2. Approval of projects only on the basis of a solid business case

3. Fully resourcing each project

4. Considering all strategic options for the delivery of a project

5. Assuring that all benefits promised by the project have committed “owners”

6. Ensuring that each project is fully aligned with the organizational strategy

Delivery Capability—Efficiency

7. Having a competent project manager

https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 2/7
2/24/2020 Using Assessment to Drive Project Management Capability
8. Having proven methods and systems for planning
Certifications  Membership  Learning & Events  PMBOK® Guide & Standards  Business Solutions  Store  About 
9. Ensuring clarity about the technical performance expected from the product of the project

10. Using accurate information as the basis for decisions

11. Effective teamwork

12. Project team having the necessary authority to deliver the project

13. Effective risk management

Portfolio, program, project, and organizational standards give us a detailed picture of what constitutes project management
capability. Research such as that conducted by Human Systems and previously outlined starts to give us an idea of where to focus
our attention for investment in assessment and development of capability for maximum impact. We can further refine our
understanding of what to assess by looking at the different types of capability that are required. From the aspects of capability
previously indicated we can see that capability encompasses portfolio, project, and program management processes, systems, and
practices that are used, as well as the competencies and skills of the people who use them. There are also differences in the
capabilities required to manage a single project and to manage multiple projects, both at the program and portfolio level.

Having reviewed what constitutes project management capability in organizations, the next step is to think about how we can assess
the different types of capability.

How to Assess Project Management Capability

The categorization of project management capability previously introduced provides a good starting point for determining how to
assess project management capability, and Exhibit 2 provides a summary of the types of assessments that can be used for
processes and practices, competencies and skills, single and multiple projects.

Exhibit 2: Categorization of Assessments

Single Project—Processes and Practices

This form of assessment is primarily concerned with the deployment of practices and processes on a particular project and will often
be conducted at various times throughout the life of the project. The assessment may be required by clients to provide assurance
that the project will be delivered as required in terms of time, cost, quality, scope, and safety. Earned value, stage gate, and peer and
external consultant reviews may all be used for this purpose; however, stage gate reviews tend to be concerned with governance
while earned value reviews focus more on time, cost, and scope performance. Project health checks may provide the opportunity to
benchmark the use of processes and practices on the project with other projects within the organization or industry sector. These
forms of assessment are clearly intended to both monitor and drive performance at the project level. Benchmarked project health
checks are generally designed to encourage improvement in the way projects are managed.

Multiple Projects—Processes and Practices

Processes and practices for management of multiple projects encompass governance processes and structures, methodologies and
tools, IT infrastructure, resource management, knowledge management, communication and reporting, risk management, change
management, stakeholder management, competency development and procurement, and contracting processes. When thinking
about the assessment of an organization's overall project management capability there is a tendency to think in terms of models for
assessment. A number of models are available for this purpose and they tend to be in the form of “excellence” or “maturity” models.

https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 3/7
2/24/2020 Using Assessment to Drive Project Management Capability
General management is familiar with “business excellence” models such as the European Foundation for Quality Management
Certifications  Membership  Learning & Events  PMBOK® Guide & Standards Business Solutions  Store About 
(EFQM) Excellence Model and Baldrige Award, which they use to identify, describe, and thenimprove business processes. Similar 
models, focusing on project and program management, can be used to assess the current status of organizational project
management capability to guide development and for periodic reassessment to determine and demonstrate progress. The IPMA
International Project Excellence Awards and the Human Systems Corporate Practices Assessment (CPA), which has been
continuously developed over the last 15 years by the companies that are members of the Human Systems knowledge networks are
“excellence” models developed on principles similar to those driving the EFQM and Baldrige awards. The focus here is on identifying
and assessing those working practices that lead to improved performance (Cooke-Davies, 2004).

The large number of available “maturity” models have been strongly influenced by the Capability Maturity Model (CMM) developed
by the Software Engineering Institute (SEI) of Carnegie Mellon University for the software development process (Humphrey & Sweet,
1987; Paulk, Weber, Curtis, & Chrissis, 1995). Many of these have combined the concept of maturity as established by the SEI CMM
with the view of project management presented in the Project Management Institute's PMBOK® Guide, which tends to present a
specifically project centric or tactical view. Others such as Gareis (2002) and Kerzner (2001) take different views of organizational
project management maturity.

In 2003 the Project Management Institute released its Organizational Project Management Maturity Model (OPM3®) Knowledge
Foundation, which identifies more than 600 “best practices,” more than 3,000 “capabilities,” and more than 4,000 relationships
between capabilities. They have since developed an OPM3® Product Suite and a process for certification of assessors. The OGC
has also developed a Project Management Maturity Model (PMMM) in recognition that maturity questionnaires provide a simple tool
for identifying areas where an organization's processes may need improvement (Cooke-Davies, 2004).

Different organizations in different industries will place emphasis on different aspects or components of project management
capability. Each organization should therefore choose the model and process best suited to its need as a basis for establishing its
current baseline, formulating and driving an improvement plan, and conducting periodic re-assessment.

Project, program, and portfolio management offices (PMOs) of various types and sizes have become an important aspect of
organizational project management capability. Although research (Hobbs & Aubry, 2007) clearly indicates that there is no simple
categorization or formula for PMOs; but there is a growing interest in assessing their performance. The available research (
Crawford, 2004; Hobbs & Aubry, 2007) and benchmarking initiatives such as that of Human Systems provide a basis for comparison
and baselining and this is often combined with assessment of stakeholder satisfaction through questionnaires and interviews.

Of course, programs comprise multiple projects. Program assessments tend to focus on assessment of component projects and
utilize the assessment methods available for single projects. At the program level, there is an opportunity for comparison and
aggregation of results across the program and assessment of features that are more significant for programs than for projects such
as resource utilization, program risk, change management, stakeholder satisfaction, and benefits realization.

Single Project—Competencies and Skills

The results of research presented earlier in this paper highlight the importance of a competent sponsor, project manager, and project
team to effective and efficient project performance. Assessment and development of competence of the people on projects is
therefore a vital aspect of overall organizational project management capability.

Assessment of competence requires standards or guides against which measurement can be made. Knowledge and demonstrable
performance (use of project management practices) are the two aspects of competence that are most widely addressed in project
management standards or guides produced by project management professional associations or standards setting bodies.
Knowledge is the easiest to assess and least controversial aspect of competence and is generally assessed using multiple-choice
questionnaires such as those used for the Project Management Professional (PMP®) exam.

Demonstrable performance or use of project management practices can be assessed against performance-based competency
standards such as those of the Australian, British, and South African governments and the Global Alliance for Project Performance
Standards (GAPPS). Assessment against performance-based competency standards is generally undertaken by trained workplace
assessors. Candidates are required to gather evidence of the use of practices in accordance with performance criteria specified in
the standards. The workplace assessor works with candidates, advises, and assists them in achieving recognition of competence.
Candidates are assessed either as competent at a particular level, or “not yet competent.” If assessed as competent, a candidate
may be awarded a qualification recognized within a government endorsed framework. The GAPPS (www.globalpmstandards.org )
does not award qualifications. It audits and endorses assessment processes and standards as a basis for mutual recognition and
transferability of project management and related qualifications.

Behaviors are the most difficult, expensive, and controversial aspects of competence to assess, although it is behaviors that
differentiate between threshold and superior performance. A number of organizations have developed corporate competency
models, identifying the behaviors that are considered desirable and associated with superior performance within a specific corporate

https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 4/7
2/24/2020 Using Assessment to Drive Project Management Capability
context. However, these are usually applicable across the organization and not project management specific. Assessment Centers
Certifications  Membership Learning & Events PMBOK® Guide & Standards  Business Solutions  Store  About 
are sometimes used to assessbehavioral competenciesand can be extremely effective but they are also very expensive.

When dealing with individuals, assessment and development of competence can often be addressed through the certification and
qualification processes of professional associations and academic institutions. Professional certification will generally ensure a
consistent level of basic knowledge, shared language, and threshold performance across a project team. Superior performance may
emerge as a result of the reflective processes encouraged in some academic programs.

NASA has found that team assessments provide them with the best returns in terms of performance and capability improvement on
projects (information about this is available on the NASA APPEL website).

Multiple Projects—Competencies and Skills

A sensible approach to assessment of project management competence, from an organizational perspective, is to use knowledge
tests and self assessment (Aitken & Crawford, 2008b) as a first-cut assessment across the entire resource pool; performance-based
assessments for a subset of more senior practitioners; and then behavioral assessments for those people seen to have potential as
superior performers. Results can be aggregated to baseline competency profiles across the resource pool, and provide guidance for
training needs.

A number of organizations encourage development of individual project management capability through internal accreditation
processes and some of the larger organizations have established relationships with academic institutions to provide a range of
development and assessment programs for more senior project personnel. There is growing recognition of the role of the program
manager. The Project Management Institute has produced standards for both program and portfolio management (Project
Management Institute, 2006a, 2006b) and offers a program manager qualification, the Program Management Professional (PgMP®).
It is important to recognise that program management is not just a higher level of project manager. Many program managers come to
the role as part of a general management rather than from a project management career path. The Office of Government Commerce
in the UK was one of the first organisations to recognise the program manager role and produced a guide titled Managing Successful
Programmes (Office of Government Commerce (OGC), 2007).

Using Assessment to Drive Project Management Capability Improvement

As the saying goes: “What gets measured gets done.” To maintain and improve capability, you first need to assess or measure your
current capability. You need to know where you are. Using the guidelines previously provided you can determine which aspects of
capability you will assess and how to carry out that assessment. Once you have a baseline you can develop a plan for improvement,
initiate, and resource an improvement program. Benchmarking is important to ensure that your improvement targets are realistic,
achievable, and relevant. They are also important to ensure that you aren't fooling yourself about your capability. Networking with
other organizations can help you to identify improvements that have worked for others.

Assessing once is not enough. You need to have a regular program of assessment whether this is for single projects or programs, for
the PMO, for individuals and teams or for overall project management capability. These assessments drive project management
capability improvement providing evidence of success and targets for moving forward. As research into the value of project
management has demonstrated, organizations that stop focussing on value, or believe that they are “done” stop demonstrating
value, and the act of not enhancing value appears to destroy value (Thomas & Mullaly, 2008). Assessment demonstrates both value
and improvement. A regular program of assessment keeps everyone on their toes and ensures that you keep your best foot forward.

References
Aitken, A., & Crawford, L. H. (2008a). Delivering on strategy—Benchmarking your way to better performance. In AIPM National
Conference, Canberra, October 2008, Sydney, Australia: Australian Institute of Project Management.

Aitken, A., & Crawford, L. H. (2008b). Senior management perceptions of effective project manager behavior: An exploration of a
core set of behaviors for superior project managers. In Proceedings of PMI Research Conference, Warsaw, Newtown Square, PA:
Project Management Institute.

Cooke-Davies, T. J. (2004). Project management maturity models. In Morris, P. W. G. & Pinto, J. K. (Eds.), The Wiley guide to
managing projects. Hoboken, NJ: John Wiley and Sons.

Crawford, L. H. (2004). Supporting delivery capability: The many faces of project support in organisations. In Proceedings of 17th
IPMA World Congress on Project Management, June 2004, Budapest, Hungary: IPMA.

Crawford, L. H., Cooke-Davies, T. J., Hobbs, J. B., Labuschagne, L., Remington, K., & Chen, P. (2008). Situational sponsorship of
projects and programs: An empirical review. Newtown Square, PA: Project Management Institute.

Gareis, R. (2002). Assessment of competences of project-oriented companies: Application of process based maturity model. In
Proceedings of the 32nd Annual Project Management Institute 2002 Seminars & Symposium, San Antonio, October. Newtown Square,
PA: Project Management Institute.

Hobbs, B., & Aubry, M. (2007). A multi-phase research program investigating Project Management Offices (PMOS): The results of
Phase 1. Project Management Journal, 38 (1), 74-86.

https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 5/7
2/24/2020 Using Assessment to Drive Project Management Capability
Humphrey, W. S., & Sweet, W. L. (1987). A method for assessing the software engineering capability of contractors: CMU/SEI-87-TR-
Certifications  Membership Learning & Events  PMBOK® Guide & Standards  Business Solutions  Store  About 
23, ESD/TR-87- 186, SoftwareEngineering Institute, Carnegie Mellon University.

Kerzner, H. (2001). Strategic planning for project management: Using a project management maturity model. New York: John Wiley &
Sons.

KPMG, Editors. (1997). What went wrong? Unsuccessful information technology projects [Web Page]. Retrieved 19 March 2002 at
http://audit.kpmg.ca/vl/surveys/it-wrong.htm .

Nieto-Rodriguez, A., & Evrard, D. (2004). Boosting business performance through programme and project management: First global
survey on the current state of project management maturity in organisations across the world. PricewaterhouseCoopers International
Limited.

Office of Government Commerce (OGC). (2007). Managing successful programmes, Third ed. London: TSO (The Stationery Office).

Paulk, M. C., Weber, C. V., Curtis, B., & Chrissis, M.- B. (1995). The capability maturity model: Guidelines for improving the software
process. Boston, MA: Addison-Wesley Longman Publishing Co., Inc.

Project Management Institute. (2006a). The standard for portfolio management. Newtown Square, PA: Project Management Institute.

Project Management Institute. (2006b). The standard for program management. Newtown Square, PA: Project Management Institute.

Teague, J. A., & Cooke-Davies, T. J. (2007). Developing organizational capability: Pointers and pitfalls. In Proceedings of PMI Global
Congress 2007 – EMEA. Newtown Square, PA: Project Management Institute.

Thomas, J., & Mullaly, M. E. (2008). Researching the value of project management. Newtown Square, PA: Project Management
Institute.

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly
prohibited. For permission to reproduce this material, please contact PMI or any listed author.
© 2009, Lynn Crawford, Alicia Aitken and Terry Cooke-Davies
Originally published as a part of 2009 PMI Global Congress Proceedings – Kuala Lumpur, Malaysia

Related Content

ARTICLE ǀ Quality Management, Risk Management, Resource Management ǀ 1 December 2019

Project Management Journal

Taking a Holistic Exploration of the Project Life Cycle in Public-Private
Partnerships
By Alexander, James | Ackermann, Fran | Love, Peter E. D. ǀ Operations and projects are typically treated as dichotomous
concepts; projects create assets to be operated. The segregation of operations from projects is evidenced in the generic
project life…

ARTICLE ǀ Quality Management ǀ 1 November 2019

PM Network
Quality Control
PM Network queries the project management community about overcoming problems during the testing phases.

ARTICLE ǀ Quality Management, Scheduling ǀ 1 August 2019

PM Network
Going to Extremes
By Fister Gale, Sarah ǀ Blistering desert heat. Arctic chill. Middle-of-nowhere sites. When projects happen in extreme
environments, teams face a test of professional skill—and personal will. Embracing a climate of…

ARTICLE ǀ Quality Management, Leadership ǀ 1 June 2019

Project Management Journal

Critical Success Factors Affecting Project Performance
By Jitpaiboon, Thawatchai | Smith, Sheila M. | Gu, Qiannong ǀ This study examines success factors for a team project.
The influence of three success factors -- (1) project management tools, (2) project management best practices, and (3)
managerial support on…

ARTICLE ǀ Quality Management, Project Integration Management ǀ 1 April 2019

PM Network
External Impact
By Smith, Ronald B. ǀ Most project managers know that testing new software and/or hardware products before they're
released is an important part of risk management. But determining precisely what the project team should…
https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 6/7
2/24/2020 Using Assessment to Drive Project Management Capability

Certifications  Membership  Learning & Events  PMBOK® Guide & Standards  Business Solutions  Store  About 

© 2020 Project Management Institute, Inc. USA

https://www.pmi.org/learning/library/using-assessment-drive-project-management-capability-6879 7/7