You are on page 1of 13

PAPERS The Nature of Risk in

Complex Projects
Terry Williams, Risk Institute, University of Hull, Hull, United Kingdom

ABSTRACT ■ INTRODUCTION

B
usiness is becoming increasingly projectized or project oriented,
Risk analysis is important for complex proj-
with many billions of dollars being spent annually on projects
ects; however, systemicity makes evaluating
around the world. Building new capital assets, carrying out unique
risk in real projects difficult. Looking at the
large-scale enterprises, or developing new technological products,
causal structure of risks is a start, but causal
all require major projects to be undertaken. The benefits of projectization and
chains need to include management actions,
a good approach to project management can clearly be seen in many ways:
the motivations of project actors, and socio-
these include motivation, satisfaction, and giving meaning to the work of
political project complexities as well as
individuals and teams (Thomas & Mullaly, 2008). However, the reputation of
intra-connectedness and feedback. Com-
project management with most people is that it is generally unsuccessful, with
mon practice based upon decomposition-
projects being late, overspent, and often not technically successful. The media
type methods is often shown to point to
delights in reporting on large public construction projects that have suffered
the wrong risks. A complexity structure is
huge cost or time overruns, such as the United Kingdom’s Scottish Parliament
used to identify systemicity and draws les-
(“10 times over budget and more than three years late,” Tempest, 2004), or a
sons about key risks. We describe how to
healthcare project in the United Kingdom with “much uncertainty about the
analyze the systemic nature of risk and how
costs . . . unlikely to complete . . . anywhere near its original schedule.” (House
the contractor and client can understand the
of Commons, 2007)
ramifications of their actions.
This article concentrates on complex projects, because the effects of risks
within such projects are difficult to understand without analysis. We use
KEYWORDS: complexity; systems
the well-known Simon (1982) description of a complex system being one in
thinking; risk; project risk analysis;
which the behavior of the whole is difficult to deduce from understanding
mapping
the inputs to the system. Thus, in a complex project, understanding what
is likely to impact the project does not lead simply to an understanding of
what that impact might be. We therefore use a structured approach to project
complexity to help identify where the risks are occurring.
Although this article concentrates on complex projects, it does not imply
that the uncertainties facing “complex” projects are any greater than those
impacting any other type of project. Rather, these uncertainties are likely to be
more interconnected, likely to change, there will need to be reactions within
a highly paced project, or there will be socio-political risks whose effects are
uncertain. This mixture will create a project risk that is difficult to compre-
hend in its totality, whose key uncertainties are difficult to detect, and for
which risks are likely to compound and cause an overall greater risk picture.
Study has advanced in understanding complexity in projects (Geraldi,
Maylor, & Williams, 2011) and has enabled a transactional-cost understand-
ing of behavior within a project (Brown, Potoski, & Slyke, 2016). This work,
and the study of the behavior of particular projects, has led to the realization
that the consideration of risk before a project starts, and particularly the com-
mon practices in project risk analysis, are woefully inadequate for complex
projects. Indeed, common practices can sometimes divert attention away
from the key risks to less important ones. Understanding comes from looking
Project Management Journal, Vol. 48, No. 4, 55–66 at the lived experience of projects (Cicmil, Williams, Thomas, & Hodgson,
© 2017 by the Project Management Institute 2006) along the lines of Blomquist, Hällgren, Nilsson, and Söderholm’s (2010)
Published online at www.pmi.org/PMJ project-as-practice research.

August/September 2017 ■ Project Management Journal  55

105546_PMJ_04_055-066_Williams.indd 55 7/11/17 1:13 AM


The Nature of Risk in Complex Projects
PAPERS

Risks set up causal chains, often (1990) recognized the systemic structure by the delays caused by design changes
involving human motivational reac- of risks, which was continued by Williams, and delays in owner approvals and linked
tions to events and decision making by Ackermann, and Eden (1997) and, more to the very tight timescale. The delays
the project parties. The problems are recently, by Kwan and Leung (2011) for led to activities being carried out more
significantly exacerbated when these software risks and Fang, Marle, and Zio in parallel than would be appropriate in
chains lead to positive feedback loops. (2012) for engineering projects. Never- engineering terms, leading both to more
Furthermore, understanding the behav- theless, these are still largely single risks, delays (in a feedback loop) and also to a
ior of such projects therefore becomes recognized by standard risk identification delay in the overall system freeze. These,
difficult, taking away the rational basis techniques, which are then sometimes in turn, mean that more work needs to
for decision making by the project par- taken to be in a systemic relationship. be done without the necessary surround-
ties that Brown et al. (2016) assumes. Cavallo and Ireland (2014) working in ing engineering information completed
The risks discussed in this article are a different field (disaster preparedness) and frozen. Further, while some of this
known within the project and infor- start to consider what they call “unfore- work will remain sound, some will need
mation systems literature, and this lit- seeable risks”; they use Soft Systems to be re-worked, hence exacerbating the
erature already considers both residual Methodology (SSM) to understand the delays and increasing the positive feed-
risks and unintended consequences. underlying system of risks. In this article, back. Lyneis and Ford (2007) provided a
However, this article shows the lack we will consider the “softer” human and survey of the use of System Dynamics in
of consideration to some of the rami- socio-economic causal relationships that modeling projects.
fications of such risks, specifically the link these risks and approach the risks as The idea that issues within projects
risks that enlarge rather than mitigate a systemic whole. arise from systemic causal sets of rea-
through actions. Such issues will often Over the past 30 years, our view of the sons is gaining increasing recognition
overwhelm in size the issues described behavior of complex projects has devel- (some recently described by Lefle and
in project risk management practice oped in many ways, in particular, in the Loch, 2010). Keil and Mähring (2010)
and literature. These issues, therefore, use of systemic modeling. This began identified a key problem of project esca-
need attention if the credibility of proj- with Cooper’s work on the Ingalls Ship- lation as seeing problems as isolated
ect risk management is to be salvaged building case (Cooper, 1980) and contin- incidents, so that a piecemeal approach
and indeed the credibility of our ability ued with others at PA Consulting/MIT. to solving the problems is ineffective
to set up and manage complex projects. A second team at Strathclyde University because this approach does not get to
This article will look at the idea (of which this author was a member) the “underlying root causes of prob-
of systemicity in complex projects to started with the Channel Tunnel “Shuttle lems.” They identified many fine strate-
identify the issue and will review the Wagons” project (Ackermann, Eden, & gies without identifying the underlying
current risk management thinking and Williams, 1997) by using mapping to systemicity and vicious-circle nature of
practice to identify the gap. It will then structure causality and providing an inter- the problems addressed. Merrow, bas-
follow Geraldi et al.’s article to look at face with System Dynamics quantitative ing his comments on the analysis of his
the implications of complexity, and how modeling (Howick, Eden, ­Ackermann, & large database of megaprojects, stated:
unexpected risks can become more sig- Williams, 2008). A review of this body
“In projects, bad things tend to happen in
nificant within a project. of work (­Williams, 2005) and its impli-
groups, not individually . . . . . Events that
cations showed not only that project
affect projects in major ways . . . tend to
Projects and Systemicity behavior could be explained by systemic
go together. Even when one of those things
A conventional view of projects breaks inter-related sets of causal factors rather occurs individually, it tends to trigger a
them down into their constituent parts— than linking effects to single causes but, cascade of problematic effects.” (Merrow,
in scope (work breakdown structure), specifically, behavior (resulting from the 2011, p. 327)
time (critical path networks), cost (bud- dynamics set up) that turned into positive
gets), risk (risk registers), and so on. (In feedback loops, or ‘vicious circles.’ This Similarly, Thamhain stated:
some of the other aspects, there has been positive feedback can cause significant
“Undesirable events (contingencies) are
more progress in looking holistically, but over-spends and “runaway.” The Shuttle
often caused by a multitude of problems . . .
risk work still generally revolves around Wagons project, for example, had a spe-
these problems often cascade, compound,
decomposition into individual parts). cific major change following a fire, and
and become intricately linked . . . . clearly
This, however, is inadequate for complex continuous (multiple, small) approval even small and anticipated contingencies
projects, as defined by Simon earlier. delays, leading to a structure such as that . . . can lead to issues with other groups,
There has been recognition in the lit- shown in Figure 1. In this very simplified confusion, organizational conflict, sink-
erature that risks can be inter-related. Dat- illustrative diagram, we can see some ing team spirit, and fading commitment.”
ing back to 1990, Al-Bahar and Crandall simple positive feedback loops initiated (Thamhain, 2013, p. 29)

56  August/September 2017 ■ Project Management Journal

105546_PMJ_04_055-066_Williams.indd 56 7/11/17 1:13 AM


More work on
unfrozen items

Tight
timescale
More Delayed
rework system freeze

More
parallelism

Design
change

Increased Activities more


delay co-related

Approval
delays

Activities
take longer

Figure 1: Generic delays in the Shuttle project.

Thamhein emphasized the impor- any uncertainty that has an effect on (which is an ANSI standard) in its lat-
tance of dealing with these risks early in a project. We are not trying to look for est version makes brief mentions of the
the project life cycle; however, he also definitional distinctions, but recognize existence of methods to deal with inter-
acknowledged the “enormous difficul- two essential points. First, uncertain- relatedness. In the United Kingdom, the
ties” of actually predicting the risk situa- ties are a collection of both aleatoric Association for Project Management’s
tions and understanding their systemic (i.e., those to which probabilities can Project Risk Analysis and Management
complexity. He also noted that senior be objectively related) and epistemic Guide (2004) has an appendix on the
managers rated the performance impact (i.e., those stemming from a lack of suffi- issue but later publications, including
of risks, on average, to be 30% lower cient knowledge) with many combining their guide on Prioritising Project Risks]
than project managers did—possibly both aspects (Williams & Samset, 2010). (Association for Project Management,
showing the higher expectations of the Second, risks might include “good” 2008), are clearly geared toward under-
project manager to manage perturba- opportunities as well as downside risks. standing and prioritizing individual
tions and perhaps also revealing less Risk management has become a core risks. Leitch (2011) points out that ISO
understanding of the cascading nature part of project initiation and execution 31000 offers no recommendations on
of project risks. since its formal recognition in projects aggregating, splitting, or combining
in the 1980s. However, the methods used risks. Indeed, an influential review risk
Current Risk Management in practice still reflect the early reliance management standard in 2005 made no
Thinking and Practice on lists (or “risk registers”) of individual mention of risk combinations (Raz &
In this environment of complex proj- risk items (Williams, 1994). The Project Hillson, 2005). These standards do not
ects, how are risks identified and man- Management Institute’s A Guide to the actually prohibit more systemic think-
aged in practice? We use the term “risk” Project Management Body of Knowledge ing. However, Hodgson’s (2002) Fou-
here in the typical sense as relating to (PMBOK ® Guide) – Fifth Edition (2013) cauldian analysis shows how, although

August/September 2017 ■ Project Management Journal  57

105546_PMJ_04_055-066_Williams.indd 57 7/11/17 1:13 AM


The Nature of Risk in Complex Projects
PAPERS

those advocating project management and analysis of risk are not typical fea- further by a mind-map-type decom-
toolkits claim them to be “universal and tures of the discussion, although the position (although the first dimension,
politically neutral,” their actual use in issues are recognized. The risk chapter “Risks and opportunities,” shows risk
practice enforced by management leads of the project management “state-of-the- as an input to complexity, rather than
to a specific way of thinking and indeed art” handbook by three leading aca- uncertainty being the input and risk
ontology inculcated into an organiza- demics (Winch & Maytorena, 2011) being the resultant issue). The Trea-
tion, thus actually inhibiting systemic provides limited information on risk sury Board of Canada (2013) provides
thinking. inter-relationships. The literature indeed a similar full scoring model but, again,
There is some recognition in stan- addresses the increasing complexity this model does not capture systemic-
dard methodology that comprehending and intra-connectedness of projects, yet ity; similarly, the spreadsheet used by
overall risk is needed to achieve proper the implications for project risk are not the Commonwealth of Virginia, with
understanding. PMI’s Practice Standard drawn out. For example, the chapter on its 15 risk questions and 16 complexity
for Project Risk Management (Project project risk by Loosemore (2006) in Pryke questions. Perhaps the most research-
Management Institute, 2009) and the and Smyth’s work tellingly entitled “Man- based model is the GAPPS CIFTER
United Kingdom’s Association for Proj- aging Project Risks” (in the plural), has a analysis (discussed by Aitken and
ect Management’s (2012) Body of Knowl- deeper understanding of what an indi- Crawford, 2007). These methods offer
edge have started talking about two-level vidual risk represents but does not con- useful pointers to decision makers, and
definitions of risk, covering both indi- sider risk combinations and the resulting can identify broad areas of likely com-
vidual risks and “overall project risk.” In causal chains involving humans. Cooke- plexity, but they do not help us to iden-
general, however, project risk processes Davies’s (2011) edited book states that tify, understand, or model the systemic
still only address the former level, and “complexity in projects probably has its risks of a project and the causal chains.
there is little advice on how to address greatest impact in the sphere of risk man- Given the lack of information in the
the latter. Hopkinson’s (2011) Project agement,” although it does not provide standard methodologies then, does it
Risk Maturity Model states that “achiev- specific advice for the practical use of mean that the topic of this article is unim-
ing the highest RMM [Risk Management the ideas in risk management. Hass’s portant? There are mixed views on the
Maturity] level (Level 4) does depend (2009) popular book on managing com- extent to which current risk management
upon the use of quantitative techniques plex projects describes some of the prob- has an effect on project success. Some
to understand the implications of over- lems covered in this article, stating that authors (for example, de Bakker, Boon-
all project risk. Using a simple qual- “risk management is one of the most stra, & Wortmann, 2010; Cooke-Davies,
itative approach based on managing neglected aspects of managing complex 2000) show that, not surprisingly, even
risks on a risk-by-risk basis will there- projects” and suggests some useful man- some levels of attention to project risk
fore limit a project to having a level 3 agement ideas for retaining a view of the help projects achieve their objectives. de
RMM capability at best”; this, however, risk systemicity. Remington and Pollack’s Bakker, Boonstra, and Wortmann (2012)
still only provides a passing mention (2008) book seeking tools for complex point to the simple identification of risks
in one paragraph (p. 138) to the way projects also recognizes the issue of risk as having the most effect. Beyond this,
that complexity in projects produces inter-dependence, and indeed includes however, the evidence about the rela-
systemic risks. Indeed, while cross-risk a chapter with a technique called “Risk tionship between risk management and
correlation is discussed, Monte Carlo Interdependencies,” which goes some project success is at best mixed (Zwikael
schedule analysis is used without spe- way in recognizing the issues but does & Globerson, 2006, describe it as a “low
cifically modeling management activity not explore the ramified causal chains impact process”; see also, e.g., Ropponen
or human reactions within the project, or the human elements of those causal & Lyytinen, 1997). This suggests that cur-
which is an important element in pro- chains. Ackermann, Howick, Quigley, rent methods are not convincing users
ducing the systemicity. Williams (2004) Walls, and Houghton (2014) recognize of the successful handling of risk and
shows that using Monte Carlo schedule and expand on some of these issues perhaps not providing understanding of
analysis without modeling how man- further. the overall risk to the project. We will
agement reacts to project events and Where complexity within project now therefore try to understand in a
progress provides misleading results. risk is recognized, some try to cap- more structured way what risk means in
The academic project literature ture the complexity by the use of a complex project.
clearly recognizes complexity within simple spreadsheets and question- Hardy and MacGuire (2016) draw
projects. Nonetheless, with the excep- naires; Maylor, Turner, and Murray- upon the work of Foucault and explore
tion of those authors who were quoted Webster (2013) is one example. the “implications of organizations’ being
in the previous section (Thamhain, 2013; Similarly, Maynard (2013) looks at eight situated in a dominant discourse of risk”
Merrow, 2011), the explicit identification dimensions of project complexity, split and the difficulties of changing those

58  August/September 2017 ■ Project Management Journal

105546_PMJ_04_055-066_Williams.indd 58 7/11/17 1:13 AM


ways, even when it is known that they socio-complexity of the project needs to diminishes creativity (To, Fisher, &
are ineffective. The established norms of be considered—in other words, the fact Ashkanasy, 2015). Interpersonal con-
(separate) risk identification and man- that projects are carried out by teams flict, within the project or with the client,
agement risk-by-risk, undertaken by of humans. There are two key elements damages project performance through
particular actors who have legitimacy that need to be considered. negative emotions (Zhang & Huo, 2015).
to identify, quantify, and manage those The first and perhaps more straight- In the backs of many project workers’
risks, means that risk becomes (quoting, forward element is that human project minds, when there are signs that a proj-
Gephart, Van Maanen, & Oberlechner, managers will react to perturbations in ect is not proceeding well, there is the
2009, p. 143) “identifiable through sci- the project. An example of this is the fear of project failure (Shepherd & Car-
entific measurement and calculation, move to increased parallelism or work- don, 2009). Theories of human behavior,
and [can] be controlled using such ing on unfrozen items as shown in such as that by Bourdieu (1998) (see a
knowledge.” Changing this dominant Figure 1. Indeed, it is the ramifications discussion of how this can be used in
discourse means “challenging the privi- of such actions within systemicity that the understanding of project behavior in
leged position” of risk “experts” and means that sometimes they have appar- O’Leary, 2012) can be useful in under-
drawing on “alternative discourses,” ently counter-intuitive effects (Eden, standing the human aspects of project
particularly in the case of systemic risks Williams, Ackermann, & Howick, 2000). behavior. However, more work is needed
where causality is less clear. This is particularly so in projects subject to quantify the relationships between
to high pace or high time constraints. In drivers, such as changes or conflict and
The Structure of Complexity such projects, attempts to accelerate the outcomes such as productivity or error-
A contingency view of projects recog- project increase the parallelism and make rate, and the role of emotions within
nizes the need to take project complex- the project even more difficult to manage these relationships. Nevertheless, it is
ity into account. Williams (2005) calls and less stable, and so costs spiral out of a key feature of the system modeling of
for different ways of understanding and control, making it difficult to relate proj- projects above, that such human aspects
managing projects in situations of high ect spend to individual parts of the proj- are essential links in the causal chains
structural complexity, high uncertainty, ect (Eden, Ackermann, & Williams, 2005). that explain the behavior of projects.
and high pace. Similarly, Shenhar’s More complex is that the project Thus, the second implication:
“Diamond” model (Shenhar & Dvir, 2007) workforce will react to perturbations
divides projects (and thus what is needed within the project. For example, there (2) The human reactions to events need to
to manage them) by project (structural) can be an increase in nugatory work, be accounted for in analyzing risk.
complexity, pace, novelty, and technol- making this work less meaningful,
ogy (although this does not have the increasing errors and, in Thamhein’s The fifth dimension of socio-political
underlying basis of the systemicity rea- words above, there is “confusion, organi- risk acknowledges the increasingly rec-
soning as described earlier). As our guide zational conflict, sinking team spirit, and ognized political effects within projects.
for this article, we refer to Geraldi et al. fading commitment.” In recent years, This is especially so for “softer” types
(2011), who try to define this overall set there has been increasing emphasis of projects, such as IT-enabled change
of concepts that describe the complexity on the importance of effective experi- (e.g., see the analysis of the UK Govern-
of projects according to five dimensions: ence in motivating work. Seo, Barrett, ment so-called “Phoenix programme”
structural, uncertainty, dynamics, pace, and Bartunek (2004) demonstrate the [O’Leary & Williams, 2013]). For mega-
and socio-political. What then are the implications on goal performance of a projects in particular, the (permanent)
implications for how we regard risk? “pleasant” core effect or, conversely, the political environment within which
Structural complexity implies mul- effect of a negative emotional reaction the (temporary) projects sit becomes
tiple interacting elements. Where there when project events occur. Where proj- increasingly important to give the proj-
are uncertainty and dynamics in the ect workers become disheartened, or ect, and thus the work within the proj-
system, then multiple interacting risks fatigued, or start to make mistakes, then ect, meaning. In general, the actions
will be present. Thus, the first straight- these effects are keys to the progress of of the two parties (client and contrac-
forward implication is: the project. Even if these clearly identi- tor) within that environment can have
fied aspects are avoided, creativity and a significant effect on projects. In the
(1) Technical complexity means a system innovation will decrease. Disruption transaction-cost structure expounded by
view of risk needs to be taken. (under high “pace”) can cause a proj- Brown et al. (2016), a key to success of a
ect team to lose its “conceptual slack” contract is whether each party acts in a
However, when the system is further and its ability to sensemake (Grabher “perfunctory” or “consummate” manner
disrupted by pace so that acceleration & Thiel, 2014), and a negative mood if (the former conforms to the “letter” of
decisions need to be made, the internal combined with a lack of empowerment the contract but has small gains for the

August/September 2017 ■ Project Management Journal  59

105546_PMJ_04_055-066_Williams.indd 59 7/11/17 1:13 AM


The Nature of Risk in Complex Projects
PAPERS

party but greater losses for the other side; therefore facilitates a structured under- involved the manufacture of a ship,
the latter conforms to the spirit of “win- standing of why: of a type which was technologically
win” and has small losses for the party advanced and whose design was far
but greater gains for the other side). • Structural complexity implies multiple from finalized. Thus, the two risks were:
How the parties react to project events interacting elements; where there are the state-of-the-art technology being
and risk outturns has a strong effect on uncertainty and dynamics in the sys- used and the immaturity of the design.
the project out-turn. Under conditions tem, risks will then have causal chains There were two additional sources of
of complexity, this is particularly prob- of ramifications, and risks will interact risk within the project: first, the accep-
lematic, as often the ramifications of in multiple inter-connecting ways. tance criteria were not well-defined; and
those reactions is not clear to the party • When the system is further disrupted second, (a probably unidentified risk)
undertaking them, so the party might by pace, acceleration becomes both the client’s inspection team were (the
not realize the damaging effects that it is necessary and problematic because contractor felt) unreasonably demand-
setting up. Thus, the third implication is: actions will interact with the causal ing. Each of these four risks, except for
chains of these ramifications. possibly the first, would probably not
(3) The contracting parties’ reactions • These interactions include socio- have appeared as major risks on a risk
to events need to be accounted for in complexity in the causal chains of register. The combination of all of them,
analyzing risk. ramifications from human reactions however, meant that the inspection
to events. and acceptance process proved very
However, even these do not fully
• These causal chains can interact with problematic, as demanding inspectors
account for the complexity of risk out-
ramifications from the socio-political could claim that novel items, with some
turns in a project. As described above,
complexity of the project environment, aspects not agreed on pre-contract
where risks set up causal chains—which
in particular reactions to events from due to the immature design, did not
might include human motivational
the contracting parties. meet their stringent interpretation of
reactions to events and/or decision
• All of these ramified causal chains the ill-defined criteria. Furthermore, as
making by the project parties—these
make the understanding of risk diffi- well as the risks themselves interact-
can be significantly exacerbated when
cult and requires analysis; however, an ing, the causal chains emanating from
these chains lead to positive feedback
extra dimension of complexity comes these combinations included design-
loops. Indeed, some systems model-
where these causal chains combine ers disagreeing with inspectors, being
ers will say that the feedback system
into detrimental positive feedback. In over-ruled, having to re-work, becom-
provides a good explanation of how
this way, risks collectively become a ing disheartened and thus lowering pro-
the system will behave. That is, where
serious significant overall risk. ductivity and increasing error rates, and
the project is in homeostasis (Thurston,
so on (see Figure 2). Each risk individu-
2016) the project is under control, but
Key Risks in a Project ally might have been manageable; when
once unwanted positive feedback loops
A main conclusion of this article is that they are combined with the resulting
(or “vicious circles”) are set up, the proj-
the risks identified by current methods human reactions then they are not man-
ect becomes out of control. In this way,
as the most important risks might not ageable. Following the logic discussed
seemingly quite small risks, if they come
actually be the key risks in a project. earlier meant that some of these items
about, can have effects that are exac-
When risks are identified and analyzed looped back to create vicious circles.
erbated with ramifications far greater
in a conventional risk analysis, it is those Sometimes significant risks are sim-
than the original risk. When identified,
risks with the greatest direct impact, ply increases in the degree of risks that
the first goal should be to consider how
when considered on their own, that are are accepted as normal project issues:
to “break” such loops—in other words,
always placed as the most important, The approval delays depicted in Figure 1
management actions that can remove
and which thus gain the most atten- were simply an increase of delays over
one or more of the causal loops that
tion of management. In a complex and above the contracted limit, the
form the loop (such as considering a
project, however, the ramifications of cumulative effect of which would be dif-
system freeze as shown in Figure 1).
risks might be greater than the direct ficult to assess. The extreme example of
Thus, the fourth implication is:
impacts. Examples are reviewed next, in 15,000 design changes is quoted below.
(4) Detrimental positive feedback needs to the order as discussed above.
2. Pace and Management Actions
be identified in analyzing risk.
1. The Combination of Risks and The “Shuttle Wagons” work referred
The implication therefore is that risk Human Reactions to earlier (Ackermann et al., 1997)
needs to be treated differently within A project in which the author was de­scribes the need to make manage-
complex projects. The discussion above involved in was an arbitration that ment deci­sions in a high-pace project

60  August/September 2017 ■ Project Management Journal

105546_PMJ_04_055-066_Williams.indd 60 7/11/17 1:13 AM


1 State-of-the-art 2 Immaturity of the
technology design

3 Acceptance criteria 6 Disagreements


not well-defined

5 Multiple
7 Re-work
inspection failures

4 Client's inspection
team unreasonably 8 Disheartened
9 Lower productivity
demanding engineers

10 More errors

Figure 2: Combinations of risks and the resulting human reactions.

(see Figure 1). In this case, this included the start of the project, let alone its true benchmarking and changing documents
parallel working and pre-emptive impact. (Further, even if a client had late in the process, and so on.
designs, which exacerbated the posi- been aware of the changes, would he or
tive feedback loops of work-arounds, she have been aware of the effects of the 5. Contracts between the
rework, and subsequent disruptions changes on the project?) Project Parties
and delays. In this way, initiating risks, Risks can result from different inter-
4. Inter-Personal Relationships pretations of contracts between the
which are manageable in themselves
between the Project Parties customer and contractor. Eden et al.
(such as the effects of the London fire
referred to earlier), create causal loops Some of the risks themselves can be (2005) present an example of a rolling-
that have effects much greater than interpersonal rather than technical; thus stock project, in which the passenger
expected. the character of the project manager, for doors were not sufficiently watertight
example, can cause problems in agreeing to satisfy the customer, because under
3. Individuals within the Parties on designs, changes, and client accep- extreme test conditions there was a
The construction of the Scottish Parlia- tance. The need for trust between cli- small amount of water ingress. The con-
ment infamously resulted in a cost ten ent and contractor (­Kadefors, 2004) and tractor argued that no train had ever met
times higher than the original (under- within an alliance (Krishnan, Martin, & these criteria, and it was clear that the
estimated) budget. At the public enquiry Noorderhaven, 2006) and the effect of contract was ambiguous on the perfor-
in 2004, the project manager was reported lack of trust on the performance of the mance specification. This led to many
as stating that there had been 15,000 project are well-known. A difficult char- tests, studies by independent experts in
design changes to the building (British acter, or troubled interpersonal relation- the field, and a final solution, but this
Broadcasting Corporation [BBC], 2004). ships, can produce delays and the need caused many designs to be revisited and
Each individual change would have had for additional work, which dishearten changed, which created ripple effects
little effect on the project; on the other the team, disrupt the project, and set and schedule delays. Again, there was a
hand, the effect of a deluge of tempo- up vicious circles of delay and disrup- small initiating risk, but there were also
rally overlapping design changes caused tion. Indeed, some would say that trust major ramifications.
considerable problems. These were both between the contractor and owner is
engineering problems and designers’ one of the most important contribu- 6. Culture within the Project Parties
reactions to multiple changes, with their tors to project success, yet trust rarely In cases in which the industrial sec-
work sometimes becoming futile. It is appears on a risk register. Eden, Acker- tor has moved from the public sector
interesting to reflect on whether “lack mann, and Williams (2005) present an to the private sector, it can be said that
of governance within the client” (which example of a paper mill project, in which contractual relationships have changed
was perhaps what allowed the users to there were “endless” talking and meet- to reflect the new environment, yet cul-
specify changes continuously) would or ings, slowing the rate of production and ture and working practices have not.
should have been a risk identified at a customer who insisted on u ­ nnecessary Where a public-sector client is used to

August/September 2017 ■ Project Management Journal  61

105546_PMJ_04_055-066_Williams.indd 61 7/11/17 1:13 AM


The Nature of Risk in Complex Projects
PAPERS

being part of the same organization exe- by unwanted positive feedback, with an item), a causal map of the possible
cuting a project, the associated behav- management actions exacerbating occurrences within a project can be
ior may not match the demands of a these feedback loops, its effects will be built. The key is to include within the
well-defined contractual relationship. A much greater. Humans have significant map the mitigation actions that would
culture that is used to continuous design difficulty in estimating the probabilities be taken (whether or not by conscious
review or ongoing changes mid-project and impacts of epistemic risks (see, for decision) by the project team and the
is at odds with a fixed-price contract. example the, discussion in Winch & expected behavior of the project partici-
This can lead to redesign and rework as Maytorena, 2011, pp. 350–351). It could pants since, as discussed above, these
a project tries to meet increasingly unre- therefore be argued that the degree of can often be critical elements in the
alistic fixed time- and cost-targets, while positive feedback from a risk is a more causality.
requirements keep changing, leading to reliable indicator of importance to the The process of developing the risk
a cycle of increased work, more delay, project, although the organization must map is as useful as the output and
frustration, loss of control and/or mean- be set up to comprehend the full impli- requires clarity, because thought is given
ing felt by designers and engineers, and cations of a risk rather than a siloed to the chaining, yet it also enhances
increased workarounds. view of direct consequences. creativity as different future pathways
are suggested. Positive feedback loops
7. Changes to the Project Parties One Aid: Mapping Risk can be identified, as well as synergistic
Changes in the client’s strategy can be As an aid to looking at risks with the fea- actions that will ameliorate a set of risks.
a significant risk. In December 1994, tures just described, a natural method Figure 3 illustrates a small example of a
following direction from the U.S. gov- is to structure the risks in a “risk map.” map with three risks (with no border),
ernment, Amtrak was given the goal of This term is used here to denote a causal causing chains of consequences—four
eliminating its need for federal oper- map capturing systemicity (rather than of which (with no border but italic) are
ating subsidies. Working through the simplistic probability-impact grids, for human responses to the risks—resulting
implications of this meant that Amtrak example, Jordan, Jorgenese, and Mit- in two outcomes (in oval borders). (This
had to rethink their strategy; at the same terhofer, 2013). Mapping project effects simple map has multiple loops, although
time, it was engaged in a project to is a recognized part of systemic model- removing the element “Engineers dis-
design and produce the new Northeast ing work, and indeed has recently been heartened” removes one third of them.)
Corridor Acela trains. It has been sug- extended into other uses in understand- “Individuals only ever have a par-
gested, therefore, that it is no coinci- ing projects (Ackermann & Alexander, tial view of risk” (Hardy & McGuire,
dence that there were major changes 2016). Initial mapping is often loose, 2016). The method becomes particu-
to this project and thus rework; thus dealing with rough concepts; to trace larly useful in a group, as it aids com-
causing all the natural human reactions causality, however, this needs to be munication and brings out interactions
to major changes to work already done honed into a map of clearly defined vari- as well as cultural differences between
in a project and subsequent significant ables. This should be at least a “Stage 2 groups, thus providing a richer set of
disruption to the project. Again, the Cause Map” and working toward the knowledge. This requires the group to
client’s change in strategic direction “Stage 3 Influence Diagram” of Howick be suitably heterogeneous, and within
is unlikely to have been on the origi- et al. (2008); being able to put “1” for a the group there needs to be an over-
nal project risk register, but can cause positive influence (or “2” for a negative view of the whole project and an under-
major ­ramifications. influence) is an essential part of iden- standing of the management and team’s
It is likely that none of these risks tifying positive feedback (see Sterman, likely reactions to events. Discussion can
would appear on a risk register, let alone 2000). Some positive feedback may be lead to paradigmatic differences or even
a list of “top ten” risks; however, by set- beneficial (“virtuous circles”), but it is incompatibilities as heterogeneous risks
ting up vicious circles of disruption, they the detrimental positive feedback that (including, for example, both engineer-
can cause much more damage to project needs to be identified. ing and psychological concepts) are
performance than simple one-off risks. Risk mapping carried out by an combined—but this in itself aids intra-
The risks likely to produce vicious cir- individual can start from the risks that group communication and establish-
cles are those likely to produce the most would form a risk register and the proj- ment of the overall risk picture.
risk to the project. An individual risk ect objectives that will be affected by The information on the map is valu-
whose effect can be contained will only “risk.” Then, by considering causal able in various ways (Ackermann et al.,
have that effect on the project. Where chains between items on the map, and 2014), including the following:
a risk sets up causal chains of effects “so what if this happens” and “what
through human reactions to the events, would make this happen” (i.e., the (1) The information contains knowl-
and whose ramifications are multiplied causal chains both up to and following edge of the systemicity.

62  August/September 2017 ■ Project Management Journal

105546_PMJ_04_055-066_Williams.indd 62 7/11/17 1:13 AM


Loose Design changes
specification
Manufacturing before
Design errors approval

Changes caused Design re-work Engineers


to other systems disheartened
Timescale brought
Delay in approvals Manufacturing forward
rework

Delays to other
systems
extra skilled Additional Tight time
engineers manufacturing work constraint
Design delays

Delay in product
Lower productivity extra labor
delivery
(engineers)
Training
requirements Training requirements
(engineers) (manufacturing workers)

Increased cost Lower productivity


(manufacturing)

Figure 3: An example risk map (using Decision Explorer® Banxia Software, UK).

(2) Use of mapping software can cat- structured basis for system dynamics in Figure 4 assessed separately) or the
egorize the risks in various ways analysis (Howick et al., 2008), which probability needs to be identified as at
(e.g., as shown in Figure 3) and show can help us understand how causali- time zero, or some other clear defini-
the “big picture” or detail. ties are likely to interact or predict the tion of the probabilities to be estimated.
(3) The map provides understanding, likely behavior of an upcoming proj- In practice, if a significant detrimen-
traceability of the information, and ect (Rodrigues & Williams, 1998). It tal positive-feedback loop is identified,
the identification of synergistic risk can also provide a useful basis for risk then the first discussion will be about
mitigation actions. assessment, particularly since assess- how to ‘break’ that feedback loop; once
(4) The map can be used to identify ing subjective probabilities is fraught that has been done, the remaining risk
prospective feedback loops and with difficulty if the causes are not split can be quantified.
prompt questions on how to ‘break’ out explicitly or if the item whose prob-
such loops. ability is being assessed is part of a posi- Conclusions
(5) The map can help in scenario analy- tive feedback loop (so its very presence Front-end analysis and preparation are
sis as the likely effects of outcomes makes itself more likely). For exam- becoming increasingly important in
are explored. ple, consider that we need to assess the management of complex projects
(6) The map can underpin project cate- a probability that a client will delay (Williams & Samset, 2012). A vital part
gorization (Ackermann et al., 2007). design approvals too much; in this case, of this is understanding the risks that
however, there is positive feedback (as the project faces. Standard methodolo-
The map can also provide a founda- shown in Figure 4) and the probability gies evaluate these risks individually
tion for quantification. This can be a needs very careful definition to esti- and without considering the human
major task, which has been used more mate. In this type of situation, the input ramifications of each risk. Our under-
in post-hoc arbitration than in a priori causes need to be split out (probabili- standing of complex projects shows us
risk analysis. It can provide a traceable ties associated with the different arrows that risks affect projects in combinations

August/September 2017 ■ Project Management Journal  63

105546_PMJ_04_055-066_Williams.indd 63 7/11/17 1:13 AM


The Nature of Risk in Complex Projects
PAPERS

Association for Project Management


More uncertainty More mistakes (2008). Prioritising project risks. Princess
Risborough, UK: Association for Project
Management.
Association for Project Management
Client delays approvals More diagrams to (2012). Body of knowledge. Princess
beyond expected approve Risborough, UK: Association for Project
Management.
Association of Project Management.
(2004). Project risk analysis and
More diagrams drawn More rework management guide, 2nd edition. High
without approval Wycombe, UK: APM Publishing Ltd.
Blomquist, T., Hällgren M., Nilsson,
Figure 4: Risks within feedback loops. A., & Söderholm, A. (2010). Project-
as-practice: In search of project
management research that matters.
and structures of risks. While this has rather than individual risks to under- Project Management Journal, 41(1), 5–16.
been recognized to some extent in the stand properly the risk to projects. Bourdieu, P. (1998). Practical reason
literature, the key here is that the impor- on the theory of action. Cambridge, UK:
tant causal linkages are the reactions to References Polity Press.
risks; in particular, there are reactions Ackermann, F. R., & Alexander, J. British Broadcasting Corporation (BBC)
by the project manager in the context (2016). Researching complex projects: (2004). BBC newsfeed, Tuesday, 2 March
of needing to make fast decisions in the Using causal mapping to take a systems 2004, 16:48 GMT.
heat of the project, and reactions by the perspective. International Journal of Brown, T. L., Potoski, M., & Slyke, D. V.
project team in terms of motivation and Project Management, 34(6), 891–901. (2016). Managing complex contracts: A
fatigue. Often the risks that cause project Ackermann, F. R., Eden, C., & Williams, theoretical approach. Journal of Public
runaway are not individual, separate T.M. (1997). Modelling for litigation: Administration Research and Theory,
risks, but rather combinations of risks Mixing qualitative and quantitative 26(2), 294–308.
in causal chains that, along with man- approaches. Interfaces, 27(2), 48–65. Cavallo, A., & Ireland, V. (2014).
agement actions and team reactions Preparing for complex interdependent
Ackermann, F. R., Eden, C., Williams, T.,
considered, build up “vicious circles” risks: A system of systems approach to
& Howick, S. (2007). Systemic risk
of disruption. If one looks at a typical building disaster resilience. International
assessment: A case study. Journal of the
risk register in practice, the types of Journal of Disaster Risk Reduction, 9,
Operational Research Society, 58(1), 39–51.
risks discussed in this article often don’t 181–193.
Ackermann, F. R., Howick, S., Quigley,
appear at all, although they might be the
S., Walls, L., & Houghton, T. (2014). Cicmil, S., Williams, T. M., Thomas, J., &
critical risks that will bring a project to,
Systemic risk elicitation: Using causal Hodgson D. (2006). Rethinking project
or near, failure. We have used Geraldi
maps to engage stakeholders and build management: Researching the actuality
et al. (2011) to structure where such risks
a comprehensive view of risks. European of projects. International Journal of
might arise. We provide mechanisms for
Journal of Operational Research, 238(1), Project Management, 24(8), 675–686.
understanding such risks, which could
290–299. Cooke-Davies, T. J. (2000). Towards
inform the client and contractor alike
and who, otherwise would not be able Aitken, A., & Crawford, L. H. (2007). improved project management practice:
to fully understand the ramifications A study of project categorisation based Uncovering the evidence for effective
of their actions, and therefore not have on project management complexity. In: practices through empirical research.
a basis for acting “commensurately” Proceedings of IRNOP VIII Conference, Dissertation.com, an imprint of
(Brown et al., 2016), even if so inclined. University of Sussex, 19–21 September Universal-Publishers, Boca Raton, FL.
The logic in this article highlights the 2007, Brighton, UK: University of Sussex. Cooke-Davies, T. J. (ed.) (2011). Aspects
effort to identifying positive feedback Al-Bahar, J., & Crandall, K. (1990). of complexity. Newtown Square, PA:
loops in risk structures and considering Systematic risk management approach Project Management Institute.
how to “break” these as a significant step for construction projects. Journal Cooper, K. (1980). Naval ship
in risk analysis. And, finally, we have of Construction Engineering and production: A claim settled and a
encouraged a view of risk as a system Management, 116(3), 533–546. framework built. Interfaces, 10(6), 20–36.

64  August/September 2017 ■ Project Management Journal

105546_PMJ_04_055-066_Williams.indd 64 7/11/17 1:13 AM


de Bakker, K., Boonstra, A., & Hopkinson, M. (2011). The project risk Lyneis, J., & Ford, D. (2007).
Wortmann, H. (2010). Does risk maturity model. Farnham, UK: Gower. System dynamics applied to project
management contribute to IT project House of Commons (UK). (2007). management: A survey, assessment, and
success? A meta-analysis of empirical House of Commons Committee of Public directions for future research. System
evidence. International Journal of Project Accounts. Department of Health: The Dynamics Review, 23, (2/3), 157–189.
Management, 28(5), 493–503. National Programme for IT in the NHS. Maylor, H. R., Turner, N. W., & Murray-
de Bakker, K., Boonstra, A., Twentieth Report of Session 2006–07. HC Webster, R. (2013). How hard can it
& Wortmann, H. (2012). Risk 390, 11 April 2007. London, England: The be? Actively managing complexity in
management’s communicative Stationery Office Limited. technology projects. Research-Technology
effects influencing IT project success. Howick, S., Eden, C, Ackermann, F., & Management, 56(4), 45–51.
International Journal of Project Williams, T. (2008). Building confidence Maynard, D. (2013). Complexity
Management, 30(4), 444–457. in models for multiple audiences: The modeling to determine project risk
Eden, C. L., Ackermann, F. R., & modelling cascade. European Journal of (part 1). Webinar, Project Management
Williams, T. M. (2005). The amoebic Operational Research, 186(3), 1068–1083. Institute, Newtown Square, PA, July 2013.
growth of project costs. Project Jordan, S., Jørgense, L., & Mitterhofer, Retrieved from http://risk.vc.pmi.org/
Management Journal, 36(2), 15–27. H. (2013). Performing risk and the Webinars/ViewWebinar.aspx?WebinarAc
Eden, C. L., Williams, T. M., Ackermann, project: Risk maps as mediating tion=ViewandWebinarExternalKey=3bec
F. R., & Howick, S. (2000). On the nature instruments. Management Accounting ccfc-e865-47bd-8180-142f34600eb0
of disruption and delay (D&D) in major Research, 24(2), 156–174. Merrow, E. W. (2011). Industrial
projects. Journal of the Operational Kadefors, A, (2004). Trust in project megaprojects: Concepts, strategies, and
Research Society, 51(3), 291–300. relationships: Inside the black box. practices for success. Hoboken, NJ: John
Fang, C., Marle, F., Zio, E., & Bocquet, International Journal of Project Wiley.
J-C. (2012). Network theory-based Management, 22(3), 175–182. O’Leary, T. (2012). Managing business
analysis of risk interactions in large Keil, M., & Mähring, M. (2010). Is change projects: A social practice
engineering projects. Reliability your project turning into a black hole? perspective. Unpublished PhD thesis,
Engineering & System Safety, 106, 1–10. California Management Review, 53(1), University of Southampton 2010,
Geraldi, J. G., Maylor, H., & Williams, 6–31. Southampton, UK.
T.M. (2011). Now, let’s make it really Krishnan, R., Martin, X., & O’Leary, T., & Williams, T. M. (2013).
complex (complicated): A systematic Noorderhaven, N.G. (2006). When does Managing the social trajectory: A practice
review of the complexities of projects. trust matter to alliance performance? perspective on project management.
International Journal in Operations and Academy of Management Journal, 49(5), IEEE Transactions in Engineering
Production Management, 31(9), 966–990. 894–917. Management, 60(3), 566–580.
Gephart, R. P., Van Maanen, J., & Kwan, T. W., & Leung, H. K. N. (2011). Project Management Institute (2009).
Oberlechner, T. (2009). Organizations Risk management methodology Practice standard for project risk
and risk in late modernity. Organization for project risk dependencies. IEEE management. Newtown Square, PA,
Studies, 30(2–3), 141–155. Transactions on Software Engineering, Author.
Grabher, G., & Thiel, J. (2014). Coping 37(5), 635–648. Project Management Institute (2013).
with a self-induced shock: The heterarchic Lefle, S., & Loch, C. (2010). Lost roots: A guide to the project management body
organization of the London Olympic How project management came to of knowledge (PMBOK ® guide) – Fifth
Games 2012. Social Sciences, 3(3), 527–548. emphasize control over flexibility and edition. Newtown Square, PA: Author.
Hass, K. B. (2009). Managing complex novelty. California Management Review, Raz, T., & Hillson, D. (2005). A
projects: A new model. Vienna, VA: 53(1), 32–55. comparative review of risk management
Management Concepts. Leitch, M. (2010). ISO 31000:2009— standards. Risk Management: An
Hardy, C., & Macguire, S. (2016). The new international standard on International Journal, 7(4), 53–66.
Organizing risk: Discourse, power, and risk management. Risk Analysis, 30(6), Remington, K., & Pollack, J. (2008). Tools
“riskification.” Academy of Management 887–892. for complex projects. Aldershot, UK: Gower.
Review, 41(1), 80–108. Loosemore, M. (2006). Managing Rodrigues, A., & Williams, T. M. (1998).
Hodgson, D. E. (2002). Disciplining project risks. In S. Pryke and H. Smyth, Systems dynamics in project management:
the professional: The case of project The management of complex projects: Assessing the impacts of client behaviour
management. Journal of Management A relationship approach. Oxford, UK: in project performance. Journal of the
Studies, 39(6), 803–821. Blackwell. Operational Research Society, 49(1), 2–15.

August/September 2017 ■ Project Management Journal  65

105546_PMJ_04_055-066_Williams.indd 65 7/11/17 1:13 AM


The Nature of Risk in Complex Projects
PAPERS

Ropponen, J., & Lyytinen, K. (1997). Thurston, M. (2016). Keynote address, Zhang, L., & Huo, X. (2015). The
Can software risk management improve 4th International Mega­Projects Workshop: impact of interpersonal conflict on
system development: An exploratory Theory meets practice, LUISS Business construction project performance:
study. European Journal of Information School, Rome, Italy, May 19–20, 2016. A moderated mediation study from
Systems, 6(1), 41–50. Treasury Board of Canada. (2013). China. International Journal of Conflict
Seo, M-G, Barrett, L. F., & Bartunek, J. M. Project complexity and risk assessment tool. Management, 26(4), 479–498.
(2004). The role of affective experience Retrieved from http://www.tbs-sct.gc.ca/ Zwikael, O., & Globerson, S. (2006).
in work motivation. Academy of pm-gp/doc/pcra-ecrp/pcra-ecrp-eng.asp. From critical success factors to critical
Management Review, 29(3), 324–439. Williams, T. M. (1994). Using the risk success processes. International Journal
Shenhar, A. J., & Dvir, D. (2007). register to integrate risk management in of Production Research, 44(17),
Reinventing project management: The project definition. International Journal 3433–3449.
diamond approach to successful growth of Project Management, 12(1), 17–22.
and innovation. Boston, MA: Harvard Williams, T. M. (2004). Why Monte-Carlo Terry Williams, PhD, PMP, initially worked in
Business Review Press. simulations of project networks can operational research for nine years at Engineering
Shepherd, D. A., & Cardon, M. S. (2009). mislead. Project Management Journal, Consultants YARD, where he developed project risk
Negative emotional reactions to project 35(3), 53–61. management and served as risk manager for major
failure and the self-compassion to Williams, T. M. (2005). Assessing and defense projects. In 1992, he joined Strathclyde
learn from the experience. Journal of moving on from the dominant project University, Glasgow, Scotland and became Professor
Management Studies, 46(6), 923–949. management discourse in the light of of Operational Research and Department Head.
Simon, H. A. (1982). Sciences of the project overruns. IEEE Transactions in There he continued research/consultancy modeling
artificial, 2nd ed. Cambridge, MA: MIT Engineering Management, 52(4), 497–508. the behavior of major projects, both pre- and
Press. Sterman, J.D. (2000). Business post-project, and was with a team supporting over
Williams, T. M., Ackermann, F. R.,
dynamics: Systems thinking and modeling US$1.5 billion post-project delay and disruption
& Eden, C. L. (1997). Project risk:
for a complex world. Boston, MA: Irwin/ claims. Terry joined Southampton University
Systemicity, cause mapping and a
McGraw-Hill. in 2005, becoming Director of the School of
scenario approach. In, K. Kahkonen
Management, then joined Hull University Business
Tempest, M. (2004). Scottish Parliament and K.A. Artto (Eds), Managing risks in
School as Dean in 2011; he recently stepped
opens for business. Retrieved from http:// projects (pp. 343–352). London, England:
down to set up a Risk Institute concentrating on
www.guardian.co.uk/politics/2004/ E&FN Spon.
consulting and research.
sep/07/scotland.devolution Williams, T. M., & Samset, K. (2010). Terry speaks and writes on project modeling;
To, M. L., Fisher, C. D., & Ashkanasy, N. M. Issues in front-end decision-making on he has published 80 journal articles on project
(2015). Unleashing angst: Negative mood, projects. Project Management Journal, management/operational research, and a number of
learning goal orientation, psychological 41(2), 38–49. books on various aspects of projects and opera-
empowerment and creative behaviour. Williams, T. M., & Samset, K. (eds) (2012). tional research. He has degrees in mathematics and
Human Relations, 68(10), 1601–1622. Project governance: Getting investments operational research from Oxford and Birmingham
Thamhain, H. (2013). Managing risks in right. Basingstoke, UK: Palgrave. Universities, is a Project Management Professional
complex projects. Project Management Winch, G. M., & Maytorena, E. (2011). (PMP)® certification holder, has edited a leading
Journal, 44(2), 20–35. Managing risk and uncertainty on academic journal, is on the PMI Academic Members
Thomas, J., & Mullaly, M. (2008). projects: A cognitive approach. In, Morris Advisory Group, served on the United Kingdom’s
Researching the value of project P.W.G., Pinto J.K., & Söderlund, J., The Research Excellence Framework panel, and is a
management. Newtown Square, PA: Oxford handbook of project management. Fellow of various institutes. He can be contacted at
Project Management Institute. Oxford, UK: Oxford University Press. terry.williams@hull.ac.uk

66  August/September 2017 ■ Project Management Journal

105546_PMJ_04_055-066_Williams.indd 66 7/11/17 1:13 AM


Copyright of Project Management Journal is the property of Project Management Institute and
its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.

You might also like