Engineering Management Journal

ISSN: 1042-9247 (Print) 2377-0643 (Online) Journal homepage: https://www.tandfonline.com/loi/uemj20

A Comprehensive Approach to Dynamic Project

Risk Management

Alberto Sols

To cite this article: Alberto Sols (2018) A Comprehensive Approach to Dynamic

Project Risk Management, Engineering Management Journal, 30:2, 128-140, DOI:
10.1080/10429247.2018.1450030

To link to this article: https://doi.org/10.1080/10429247.2018.1450030

Published online: 16 May 2018.

Submit your article to this journal

Article views: 1152

View related articles

View Crossmark data

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=uemj20
 A Comprehensive Approach to Dynamic Project Risk Management
Alberto Sols, Universidad Europea de Madrid
Abstract: A large percentage of projects suffer performance,
cost, and/or schedule problems. There are many reasons why
projects fail, including an inadequate approach to risk manage-
ment. Inadequate approaches result when a comprehensive iden-
tification of risks is not performed at project commencement,
when risk assessment fails to consider important aspects, when no
subsequent follow-up is carried out for mitigation strategies
adopted, when no further identification of risks during project
execution is completed, or a combination of these occurs. A
number of publications have dealt with different facets of risk
management, but not in a comprehensive way. This article com-
piles the approaches applied and observed by the author based on
25 years of industry experience; these observations were synthe-
sized and result in a procedure for dynamic risk management
successfully applied to a number of projects. The use of a broad
array of techniques is suggested to identify risks, which are then
thoroughly analyzed. The assessment was performed using a
comprehensive array of criteria. The complete set of criteria
included essential ones such as aggregability factor, triggering
factor, early warning, immediacy, and recoverability. The thor-
ough assessment was done dynamically. The initial assessment
enables the adoption of the most adequate mitigation strategies,
which later on are dynamically validated. New risks are also
detected and dealt with dynamically throughout project execu-
tion. The main contribution of this work is the synthesis of state-
of-the-art knowledge into a state-of-the-practice procedure,
which can be applied by engineering managers in all sectors and
in all phases of project execution.
Keywords: Project, Dynamic, Risk, Management
EMJ Focus Areas: Program and Project Management; Decision
and Risk Management
T
he problems and challenges faced by society are evolving
fast and are rising to unprecedented levels of complexity.
Technologies are also maturing and changing rapidly,
impacting and reshaping our world. That, in turn, brings new
opportunities and poses new challenges. To cope successfully
with that environment, engineering managers need a solid foun-
dation in risk management. Many projects are troubled, exhibit-
ing problems in performance, costs, and/or schedule. Poor risk
management is one cause of troubled projects. The systems
approach to project management requires that a global view of
the managed project be exercised. Risk management is an essen-
tial part of project management and even in projects in which
risks are systematically addressed, the state of the practice has
significant shortcomings. This article presents a procedure for
dynamic risk management based on the personal experience
gathered by the author over more than 20 years of project
management, mainly in the defense sector. The procedure was
developed using a systems engineering approach. In particular,
this approach has a specific goal, uses a global view and
Refereed Research Manuscript. Accepted by Associate Editor Bastian.
128
feedback, and is dynamic in identifying the goal and the effec-
tiveness of the developed solution (Blanchard & Blyler, 2016;
Blanchard & Fabrycky, 1981; Sage, 1992).
A broad array of complementary techniques for risk identi-
fication were identified. The procedure starts looking first into
the human dimension, with tips for fostering the right atmo-
sphere in a project for real-value-adding risk management, and
then moves on to the technical dimension. Some guidelines are
given for better risk identification. The largest contribution of
this article is the comprehensive approach to the assessment
of identified risks, which facilitates risks filtering and adoption
of mitigation strategies.
Literature Review
There is abundant literature on troubled projects and on risk
management. Nevertheless, the state of the practice is insuffi-
cient, as reflected by the large percentage of projects that exhibit
performance, cost, and/or schedule problems. Those problems
have been documented in previous studies. Troubled projects
usually exhibit early warning signs, not always duly perceived by
project managers (Kappelman, McKeeman, & Zhang, 2007).
There are many reasons why projects may fail to achieve their
desired goals, as a number of studies have surfaced (Kappelman
et al., 2007; Pinto & Mantel, 1990; Zuofa & Ochieng, 2014). The
Standish Group CHAOS yearly report on software projects is
particularly notorious. Of over 50,000 projects analyzed in 2015,
19% were considered as failed, 52% as challenged, and only 29%
as successful. These levels of failed and challenged have remained
reasonably stable over the last 5 years (Standish Group, 2015).
Nevertheless, the reports of the Standish Group have received
some criticism, mainly that their definitions of successful and
challenged projects have four major problems: they are mislead-
ing, one-sided, adulterate the estimation practice, and result in
meaningless figures (Eveleens & Verhoef, 2010). As perceived by
the author of this article throughout more than 25 years of
experience in several industry domains, there is widespread
perception among engineering managers that too many projects,
at least among those of significant technical complexity, fail to
unfold as planned and desired, resulting in unacceptable losses.
The large percentage of troubled projects suggests an insuffi-
ciently adequate approach to project management and to poor
risk management practices.
Risks can be identified through a combination of product-
based techniques, such as Failure Mode, Effects and Criticality
Analysis (FMECA), Fault Tree Analysis (FTA), as well as pro-
cess-based techniques, such as Level of Protection Analysis
(LOPA) or the Swiss Cheese Model (Altabbakh, Murray, Gran-
tham, & Damle, 2013).
Risks should not be addressed as stand-alone events. It is
essential that cross-interactions between perceived risks be
addressed. Moreover, new systems often bring unintended,
negative consequences. From the early stages of a project, the
system engineer must search for unintended consequences of
the system under design. That search will likely increase safety,
Engineering Management Journal Vol. 30 No. 2 2018
reduce risks, and improve customer satisfaction. The so-called
Diogenes process has been developed to help systems engineers
identify unintended consequences of new systems. Diogenes
focuses on broad risk identification, addressing the required
system behavior and including use cases, design diagrams, test
procedures, validation, and verification efforts (Bahill, 2012).
Engineering efforts associated with the design and develop-
ment of complex systems have been termed Complex Product
Systems (CoPS) projects. CoPS are multifunctional and require
the integration of multidisciplinary knowledge inputs. Successful
CoPS projects require a high level of core competencies in
systems engineering and in risk management (Hobday, 1998).
A Risk Management Capability Maturity Model (RM-CMM)
has been proposed for CoPS. The number and complexity of
task and human relations in these projects is a major risk source
that can contribute to project failure. The RM-CMM has five
levels of maturity (ad hoc, initial, defined, managed, and opti-
mizing) and three capability area categories (organization, pro-
cess, and technology). The RM-CMM addresses the risk
planning and control processes, the organizational context, and
the technology content of a CoPS project. Improvement of the
risk management capabilities requires that organizations simul-
taneously build capabilities in internal and external contexts,
risk management process, and technologies (Yeo & Ren, 2009).
Identifying risks is just the first step; next, they have to be
evaluated, in order to choose the appropriate remedial actions.
Traditionally, the likelihood of occurrence and the severity
of damages or consequences have been used to select the most
suitable mitigation strategy for identified risks (Bahill, 2010;
Kaplan, Garrick, & Apostolakis, 1981). Other researchers argue
that likelihood of occurrence and severity of consequences
should not have the same weight, computing risk as the product
of likelihood times the square of the severity of the consequences
(Ben-Asher, 2008). It is possible to make the assessment even
more general, defining relative weights wl and ws for the like-
lihood and the severity, respectively. The weights can be set in
each case, depending on the subjective perception of the people
performing the assessments. The value of the assessed risk is
then the product of the likelihood to the wl power, times the
severity of the consequences to the ws power (Chaves & Bahill,
2014). Sometimes the difficulty of detection of a materialized
risk event is also taken into account, multiplying it by the like-
lihood and the severity of consequences to yield a value of the
assessed risk (Carbone & Tippett, 2004). Even with these
approaches for performing risk assessment with varying relative
importance of the likelihood of occurrence and the severity of
the consequences, other data are required in order to perform a
comprehensive assessment of the identified risks.
There are serious drawbacks to the conventional approach
to risk management in which risks are individually assessed. To
name the main drawbacks, combinatorial interactions among
multiple risks are not accounted for; impact of aggregate risks is
not addressed; uncertainty quantification is lacking; and there is
the over-simplicity of the traditional risk management practices
(Reeves, Eveleigh, Holzer, & Sarkani, 2013). Traditional quanti-
tative risk assessments have proven useful when the observations
of past behavior enable the inference of their probabilities of
occurrence. Nevertheless, in complex and ambiguous risk situa-
tions, the probability distributions provide a limited perspective
of identified risks. In these situations, the qualitative assessment
of identified risks is more appropriate. A model was proposed
that provides a broader uncertainty characterization and in
Engineering Management Journal Vol. 30 No. 2
which scenarios are constructed and validated according to
logical consistency, alignment with stakeholders, and concur-
rence with historical data (Aven & Renn, 2009).
Complex systems are characterized by the often-unpredict-
able interaction of people, organizations, and hardware. Inade-
quate understanding and management of these interactions is
what frequently leads to cost overruns, late deliveries, and/or
underperformance. To reduce, to the extent possible, these
undesirable consequences, a principled application of risk man-
agement is required (Pennock & Haimes, 2002). Risks therefore
need to be addressed at the enterprise level. The purpose of
enterprise risk management is to holistically manage all risks
faced by an enterprise: hazard, financial, operational, and
strategic (Elliott, 2014). No industrial sector or domain is free
from troubled projects. Some studies have addressed this issue,
in particular analyzing infrastructure developments in sectors
such as power, transport, telecommunication, and process
plants. Regardless of the sources of funding, a number of pro-
jects analyzed in those domains faced problematic situations
such as serious delays, cost overruns, and, in some cases, even
project cancellation (Lam, 1999). Research, design, and develop-
ment projects have also received attention in the literature. In
many cases, research, design, and development projects have
failed to achieve cost and schedule budgets, in spite of consider-
able effort over the years toward improving project manage-
ment. These failures are, in part, caused by the use of models
that do not treat projects as complex dynamic systems with risks
(Reichelt & Lyneis, 1999). Even when risks are dealt with, there
frequently is a fragmented approach to risk management. Risks
are assessed as stand-alone events and the probability of occur-
rence and severity of consequences is assumed to remain con-
stant over time. It is absolutely necessary to address risks in
models that include feedback and time (Bharathy & McShane,
2014).
Additionally, some authors have studied the probabilities
and consequences of cascading failures in interdependent sys-
tems. Because of these interdependencies, risks identification
assessment is to take structural complexity into account (Katina,
Ariel Pinto, Bradley, & Hester, 2014). Similarly, there could be
interdependencies between risks identified in a project or endea-
vor. In the literature, a method has been proposed for a dynamic
risk assessment, aimed at using new information when it
becomes available. In particular, this method uses Bayesian
inference to estimate the updated expected frequency of identi-
fied risks, but these assessments are based only on risk likelihood
and the severity of consequences (Paltrinieri, Khan, & Cozzani,
2014). The next section explains the dynamic risk management
procedure developed over years of industry practice.
A Dynamic Risk Management Procedure
Although the etymology of the term risk is a bit uncertain, it
seems that risk comes from the old Italian word riscare, running
into danger. Risk in a project is associated with the possibility of
suffering harm, with the quantifiable likelihood of loss or less-
than-expected returns, with the probability or threat of damage,
injury, or liability; or with any other negative occurrence. Since
every project has goals to be achieved, risks must be considered
because of the potential that they represent for project disrup-
tion. In any project, goals are to be achieved and, consequently,
losses are to be minimized. Following strictly this rationale could
seriously limit the freedom of action of the engineering man-
ager. Therefore, it is essential that certain actions are accepted
2018 129
Exhibit1. Dynamic Risk Management Flowchart
Step1. Foster the right
Project start
risk atmosphere.
Initialriskmanagement
Step6. Group
mitigation strategies.
Step8c. Re-group
mitigation strategies.
Step8b. Revise
mitigation strategy.
N
Step8a. Re-assess
risk.
Step10. Compile,
registerand
disseminaterisk-
relatedlessonslearned
at projectclosure.
Ongoingriskmanagement
(namely, certain risks are undertaken) even if they can cause an
avoidable loss, but only if the possible degree of loss seems to be
acceptable (Luhmann, 1993). Consequently, when there is a
need or convenience for compressing the schedule in a project,
certain risks will have to be undertaken. The pivotal point for
the engineering manager is to have a clear understanding of
what those risks are and how they can be mitigated. To mitigate
is to make less severe or painful. Therefore, to mitigate a risk is
either to reduce (or even eliminate) the probability that the risk
happens and/or to diminish the impact of the risk consequence,
should it materialize.
The developed procedure consists of 10 steps. It is impor-
tant to notice that these steps are to be performed in a way that,
in most cases, implies a significant departure from usual prac-
tices. A flowchart showing the steps to be carried out is depicted
in Exhibit 1. The following sections describe the essence of how
these steps are to be carried out.
Foster the Right Atmosphere
The first thing for an engineering manager to do, to deal with
risks in a way that adds real value and contributes to project
success, is to foster the right culture and atmosphere. That
means that every team member should perceive that he or she
is seen as a true asset, capable of contributing to identifying,
assessing and mitigating risks, in an environment in which
psychological safety reigns. Team members must be comfortable
expressing their thoughts and concerns about the project (Gar-
vin, Edmondson, & Gino, 2008). That demands that a proper
failure culture is instilled, led by example by the project manager
and the top managerial layers in the firm. In every project,
failure is an option except in its very final stage (Slegers et al.,
2012). Every failure offers a unique opportunity for learning and
frequently, without certain failures, there can be no project
success. The important thing is to capitalize on all lessons
learned, including the suffered failures. When the factor of
concern is risk, knowledge of failures and mistakes is an
130
Step3. Complete a
Step2. Identifyinitial
comprehensiverisk
risks.
assessment.
Step5. Develop
Step4. Prioritize
preliminary mitigation
assessed risks.
strategies.
Step7a. Follow-up on
Step9. Identify
risksand onadopted
ongoingrisks.
mitigationstrategies.
Step7b. Risk
unchangedand
strategy
fruitful?
Y
Project
Y N
end?
essential piece. It should come as no surprise that knowledge
of failures and mistakes are among the least likely to be openly
shared by project team members. Penalizing team members for
reporting mistakes and failures will disrupt the risk management
process (Davenport & Prusak, 2010). Nobody enjoys failures.
Yet, in many complex projects and endeavors, success can be
achieved only when mistakes happen and are duly capitalized
upon. That creates a climate where team members feel that it is
safe to fail (Syed, 2014). If this atmosphere is not created, risk
management will not render its full potential, no matter what
techniques or methods are applied. A pure technical approach to
risk management will not be fully successful if the human side
has not been adequately brought into the picture. Creating such
environment and culture is a sine qua non condition for excel-
ling at risk management.
Identify Initial Risks
It is necessary to identify risks that may prevent project goals
from being reached, those that could imply some unacceptable
damage to people, to the firm, or to society at large, despite the
project goals having been met. The objective is to generate an
exhaustive list of the risks with potential for project disruption,
in its broadest sense. Six key actions should be undertaken for a
comprehensive risk identification strategy: (1) consider the nat-
ure of the project, (2) consider all types of risks, (3) be exhaus-
tive in the risk identification effort, (4) look to bottlenecks, (5)
look to external sources of risk, and (6) consider the potential
side effects of mitigation strategies if adopted.
It is advisable to consider the classification of the project
using the Novelty, Technology, Complexity and Pace (NTCP)
framework (Dvir, Raz, & Shenhar, 2003; Shenhar & Dvir, 2007).
The project NTCP classification, also known as project dia-
mond, will help to identify risks that are more likely to happen
due to the nature of the project.
Identifying all types of risks is facilitated by following an
appropriate risk taxonomy. One taxonomy suggested by the
Engineering Management Journal Vol. 30 No. 2 2018
International Council on Systems Engineering (INCOSE) classi-
fies risks as programmatic, technical, financial, and schedule
(INCOSE, 2015). As defined by INCOSE, programmatic risks
are those produced by events that are beyond the control of the
project manager. Programmatic risks can be a source of risk in
any of the other three risk categories.
Whether the risk identification effort has been exhaustive or
thorough is difficult to objectively evaluate. A frequent question is
how many risks are to be identified. There is no magic answer to
knowing if ‘enough’ risks have been identified. There are two
equally undesirable situations to be avoided. First, it is undesirable
to detect an artificially large number of risks. So doing may give the
impression that there are plans for all potential contingencies.
Second, it is equally undesirable to have a factitious, low number
of risks detected. Detecting many risks may convey the idea of little
control of the project or little confidence. Detecting few risks
may convey the idea of little understanding of actual project
complexity. Risk identification must be undertaken without any
prejudice as to the number of risks. Whatever risks are identified,
and however many they are, it will be necessary to evaluate
identified risks and to take suitable actions to mitigate them.
Systemic and comprehensive risk identification rests on two
fundamental pillars, both of which are often neglected: the
involvement of all team members and the use a broad array of
risk identification techniques. Involving all team members facil-
itates the capitalization of the knowledge possessed by the group
that exceeds that of any single team member (Surowiecki, 2004).
This involvement can be done by approaching each team mem-
ber separately, as well as by conducting ad-hoc meetings on risk
identification. Normally, these meetings will be conducted as
brainstorming sessions. The Delphi method is one very appro-
priate approach, because it preserves the anonymity of partici-
pants while fostering group discussion and eventual consensus
or reasonable degree of alignment (Dalkey & Rand, 1969; Jen-
sen, 1996; Linstone & Turoff, 2002).
There is a broad array of risk identification techniques.
There are product-based and process-based risk identification
techniques. Very frequently, only product-based techniques such
as FMECA and FTA are employed, neglecting the fact that many
risks arise due to aspects related to processes, culture, or orga-
nization. Therefore, it is essential to rely also on process-based
techniques, such as the Swiss Cheese Model or the LOPA, which
emphasize the adequacy of multiple layers of protection. By
doing so, LOPA ensures a maximum return on risk reduction
investments (American Institute of Chemical Engineers. Center
for Chemical Process, 2014; Marszal & Scharpf, 2002).
Next, it is necessary to look to bottlenecks, as bottlenecks
can become capacity-related risks to project success. It is fre-
quently forgotten that the critical path in the project will change
if modifications are introduced to tasks, either in sequencing
and/or in duration. But another situation that may arise is a
change in bottleneck. As resources are reassigned, bottlenecks
Exhibit 2. Attributes Required for A Comprehensive Risk Assessment
Risk
Early
Code Description Type Likelihood Severity Detectability warning
ri li si di ei
Engineering Management Journal Vol. 30 No. 2
may shift (Goldratt, 1990). It is therefore crucial to identify
project bottlenecks in new scenarios, to prevent unforeseen
problems, originating as a result of limited capacity.
Then, external (or programmatic) sources of risks must be
considered. Much attention is paid to internal sources, within
the boundaries of the project, while problems frequently come
from the outside world (Reeves et al., 2013). In particular, close
attention is to be paid to external dependencies (Sols, 2015).
These are the tasks that are part of the project Work Breakdown
Structure, but whose work is outsourced. These external depen-
dencies may become even more important if suppliers for out-
sourced tasks are not chosen directly by the project manager but
are selected at a higher level in the organization.
Some risks are not detectable, at least immediately. More-
over, many systems and organizations have some fault-tolerance
capability. The consequence is that some risk event may actually
happen and yet no actions are taken. Later, other risk events
materialize. Perhaps the same situation is repeated again.
Finally, one last undesirable situation happens and then the
combined effect of this risk and the previous one that had
remained unnoticed is what yields devastating consequences.
The last action in comprehensive risk identification is to
consider the potential side effects of adopted mitigation strate-
gies. Consequences that may be derived from actions taken to
mitigate identified risks are frequently overlooked. It is necessary
to ascertain whether or not any risk treatment employed intro-
duces another risk. Once all risks have been identified, the next
step is to assess them.
Complete A Comprehensive Risk Assessment
Once risks have been identified, a comprehensive risk assess-
ment must be performed. Traditional risk evaluation methods
consider the likelihood of occurrence of each risk and severity of
consequences, multiplying both to calculate the so-called Risk
Priority Number 2 (RPN 2). In some cases, risk detectability is
also taken into account, resulting the product of the three
attributes on Risk Priority Number 3 (RPN 3). Nevertheless,
the use of RPNs pose a serious conceptual flaw, due to the
lack of commutative property of the likelihood of occurrence
and the severity of the consequences (Reeves et al., 2013).
Furthermore, little research rigorously validates the performance
of risk matrices in actually improving risk management deci-
sions (Cox, 2008). In addition to likelihood of occurrence and
severity of consequences, other attributes are needed for a
proper assessment of identified risks. For a comprehensive eva-
luation of each identified risk, the experience gathered by this
author in many projects indicates that it is necessary to consider
the nine attributes shown in Exhibit 2.
The first three attributes, likelihood of occurrence, severity
of consequences, and detectability, are well known. They can be
assessed qualitatively, as indicated in Exhibits 3–5, respectively.
Normally, they will be assessed via brainstorming sessions but
Attributes
Time Aggregability Trigger
Recoverability Immediacy elapsed factor factor
rci ii ti afi tfi
2018 131
 Exhibit 3. Evaluation of Likelihood of Occurrence
Likelihood Description
1 Frequent Will happen continuously
2 Probable Will happen frequently
3 Occasional Will happen several times
4 Remote Improbable, but it may be expected to
happen
5 Unlikely Improbable, although possible
Exhibit 4. Evaluation of Severity of Consequences
Likelihood Description
1 Catastrophic Death of people, or complete project failure
2 Critical Severe damages to people, or important loses in the
project
3 Substantial Low damages to people, or significant loses in the
system
4 Minor Low project loses
5 Marginal Extremely low project loses
the assessment can also be undertaken using the Delphi Method
(Dalkey & Rand, 1969; Jensen, 1996; Linstone & Turoff, 2002).
Definitions for the six additional attributes required to
completely assess each identified risk were established as part
of the developed dynamic risk management procedure and are
summarized in Exhibit 6.
Among other things, the probability of a risk materializing
is likely to increase over time, if no effective actions have been
taken to reduce the risk, or to eliminate it. The fact that the
probability of occurrence can increase with time is frequently
forgotten and, once a risk is identified and a preliminary assess-
ment of its likelihood of occurrence is made, such estimate is not
often updated. This can have consequences, if risks that were not
initially considered serious end up materializing and with worse
results than initially anticipated. Furthermore, there is the pos-
sibility of a risk combining with others, to yield devastating
effects (perfect storms). These perfect storms are originated by
cross-interactions between risks.
It is also important to factor in the possibility of a risk,
when materializing, of triggering other risks. One of the main
drawbacks of the traditional approach to risk management is
that it deals with one risk at a time. Very frequently, risks that
materialize increase the likelihood of the occurrence of other
risks, creating a domino effect (Khakzad, Khan, Amyotte, &
Cozzani, 2014). For example, the domino effect can happen in
systems whose components are not statistically independent and
that therefore work under shared loads; the failure of a compo-
nent increases the load or stress on the other team members,
making these other components more likely to fail. Experience
in several projects showed that, in projects in which team
members share responsibilities for specific activities, the pro-
longed unavailability of a team member (due to accident, leave,
or similar reason) increased the pressure on others, making it
more likely that mistakes occurred or that more team members
decided to leave, due to burnout.
Another crucial issue to be addressed is the potential exis-
tence of early warnings. The materialization of some risks is
more easily anticipated than others. When early warning signals
exist, it is easier to adopt appropriate actions on top of already
established mitigation strategies. Regardless of the severity of
consequences, risks that happen without any previous notice
can be more worrisome. It may take more time and/or effort
to react to risks that do not have any early warning. For sake of
simplicity, this attribute is binary (yes/no). Two typical risks
associated with the team members in any project are that a
team member suffers an accident that leaves him or her
grounded for a while and that a team member suffers from
burnout syndrome and develops a depression. The former can
simply not be anticipated, but there are warning signals for the
latter, like the person showing a change in mood and self-
confidence, being stressed, failing to deliver as per his or her
usual standard, and so on. The capability of an early warning
signal that the risk is likely to materialize will help determine
appropriate actions to take. In this case, it would be first to assist
the person as needed, alleviating his or her working conditions
and, second, to get ready for the eventuality of the person failing
to recover and being then required for someone else to take over
his or her duties.
In addition, the consequences of materialized risks are not
always realized immediately. The time elapsed between a cause

Exhibit 5. Evaluation of Detectability

Likelihood Description

1 Non-detectable Even if the risk materializes, it will go unnoticed to the project manager and/or team members for a sufficiently long time
(compared to the project timescales); eventually, everything becomes known
2 Very seldom If the risk materializes, there is unlikely that it will be perceived by the project manager and/or team members in a reasonably
detectable short timescale (compared to the project timescales)
3 Detectable in some If the risk materializes, there is a reasonable probability that it will be perceived by the project manager and/or team members
cases in a reasonably short timescale (compared to the project timescales)
4 Detectable in most If the risk materializes, it is very likely that it will be perceived by the project manager and/or team members in a reasonably
cases short timescale (compared to the project timescales)
5 Always detectable Even if the risk materializes, it will always be perceived by the project manager and/or team members in a reasonably short
timescale (compared to the project timescales)

132 Engineering Management Journal Vol. 30 No. 2 2018

Exhibit 6. Definitions of Six Additional Risk Attributes

Element defined Definition

Time elapsed, ti Time that has spanned from the moment the risk was identified until the present time, if the risk has not been
prevented or transferred to a third-party willing to undertake it
Risk aggregability of two risks, The capability that risks have of yielding consequences, when risks materialize reasonably concurrently in time. The
ri and rk consequences are much more severe than the consequence if any risk were to materialize alone. Risk aggregability is
formulated as follows:


1 if siþk >> si and siþk >> sk
ai;k ¼
0 otherwise

The risk aggregability factor for risk ri, denoted as afi, is the ratio of the number of risks that are aggregated with ri, over
the total number of pair of risks that contain ri. If there are n risks, afi is calculated as follows:
Pn
aik
af i ¼ k¼1
n1

Risk ri capable of triggering A risk ri is said to be capable of triggering risk rk if ri materializes and implies that the likelihood of risk rk taking place
risk rk increases substantially. It is denoted as tik and defined as follows:
8
>
> 1 if ri ðith riskÞ happening implies that lk
<
ðlikelihood of kth riskÞ increases substantially
tik ¼
>
>
:
0 otherwise

Risk chain is a subset of risks (ri, rk, . . ., rm, rn) such that tik = . . . = tmn = 1 and such that sn is unacceptable. That is, a risk
chain is a subset of risks in which each one can trigger the following one, being unacceptable the consequences of the
last risk in the chain. The rational is that, whereas some risks may be acceptable, they could trigger others in a chain
reaction, leading to a risk of unacceptable consequences.
The trigger factor of risk ri, denoted tfi, is the product of the number of risk chains that can be initiated by ri times the
total number of downstream risks in the initiated chains
Early warning Risk ri is said to have an early warning when it is possible to anticipate its materialization based on unfolding events
Immediacy The consequences of risk ri are immediate if they take place concurrently in time with the materialization of the risk
Recoverability Risk ri is said to be recoverable if it is technically and financially possible for the project to regain the state it had prior to
the materialization of the risk

and its effects can impact whether or not additional mitigation
actions are taken. For sake of simplicity, this attribute of imme-
diacy of consequences is also binary (yes/no). Consider for
example the risk of losing a very knowledgeable and valuable
team member. One scenario is that that person, assumed to give
a 3-month notice (common in many countries), announces one
day that he or she is leaving the company, which permits the
taking of actions to transfer knowledge, train a replacement, and
reduce the impact of his or her leaving the project. On the other
hand, if that person suffers an accident, contracts a disease, or
any other equivalent situation that leaves him or her immedi-
ately grounded, the project manager has no time to react. In
both scenarios, the likelihood of that situation happening is low,
but the difference in the immediacy of the consequences makes
the second scenario much more worrisome than the first.
Finally, recovering from risks that have actually happened is
sometimes difficult and often times recovery is not possible. This
attribute, of recoverability from a materialized risk, is binary
(yes/no). Let us consider again the previous case. From a reco-
verability point of view, it is clear that if a very talented and key
team member suffers an unfortunate accident or contracts a
disease, hopefully sooner rather than later that person will be
back, bringing again his or her knowledge, skills, and experience.
On the contrary, there is no way back (exceptional
circumstances aside) when that person leaves the company.
That loss in knowledge, skills, and experience is permanent.
The mitigation strategy to be adopted for an identified risk,
ceteris paribus, should depend on risk recoverability or capabil-
ity for returning back to the initial condition, after the risk has
materialized. Once risks have been assessed, they have to be
prioritized.
Prioritize Assessed Risks
Prioritization of the assessed risks is also a fundamental step and
is to be undertaken at the beginning of a project. A number of
risks will be identified at the beginning of a project. During
project execution, new risks will exist. Not all identified risks
represent the same capacity for project disruption. Therefore, it
is essential that risks be prioritized, based on assessments. Prior-
itizing assessed risks paves the way for selecting appropriate
mitigation strategies. The use of thresholds helps in prioritization.
For each of the nine attributes, the project manager and the
team define a lower and an upper threshold. For each attribute
x, there will be a lower threshold xl and an upper threshold xu.
An ABC classification or sorting of risks is performed, based on
the values of the nine attributes for each identified and assessed
risk. The most important risks, or Class-A risks, are those for
which the severity of the consequences, the aggregability factor,

Engineering Management Journal Vol. 30 No. 2 2018 133

and/or the triggering factor exceed the upper threshold. Class-B
risks are those for which those three attributes are lower than
the upper thresholds, with at least one of them above the lower
threshold. Class-C risks are the remaining risks, those that are
neither Class-A, nor Class B.
Develop Preliminary Mitigation Strategies
Since assessing and monitoring each and every identified risk
may be prohibitively expensive, when not simply impractical,
only those considered to be most critical to the project are to be
managed (Pennock & Haimes, 2002). All Class-A risks are to be
mitigated and, if time and other resources permit, also Class-B
risks and, finally, Class-C risks. In terms of their likelihood of
occurrence and/or the severity of their consequences, risks can
be alleviated through the undertaking of a series of mitigation
measures. Nevertheless, any action will incur costs. The cost
associated with the implementation and follow-up of a mitiga-
tion strategy has to pay off compared to the global costs that the
materialization of the considered risk, or group of risks, could
imply. Some costs may be intangible and difficult to quantify or
estimate, such as the cost of loss of reputation, but it is necessary
to perform a financial analysis to determine the economic fea-
sibility and convenience of adopting a risk mitigation strategy.
The essence of risk assessment and management is to maintain a
dynamic balance between risk level and risk treatment costs, and
the balance of the two is determined by the so-called risk
acceptance criterion (Liu et al., 2016). Two rules can be applied,
in the indicated order, to decide whether the adopted mitigation
strategy is to be prevention, reduction, contingency, or accep-
tance (from most active to most reactive), as described in
Exhibit 7.
The first rule is to look at three most important attributes,
severity of consequences, aggregability factor, and triggering
factor, and decide as shown in Equation (1).
Exhibit 7. Risk Mitigation Strategies
Risk mitigation
strategy Description
Acceptance It is accepted that the risk can occur. No action is
taken to prevent the risk and/or to mitigate its effects
This reactive strategy is reasonable for risks of low
probability of occurrence and, mainly, with
consequences of low severity
Contingency The risk is accepted but an action is preplanned if the
risk occurs
This reactive strategy is semi-proactive, because the
compensating mechanism is only triggered by the
actual materialization of the risk
Reduction Measures are taken to reduce the likelihood of the
risk happening, or to mitigate the eventual
consequences of the risk
This strategy is proactive because actions are taken
immediately after risks are identified and assessed
Prevention Measures are taken to eliminate the likelihood of the
risk materializing
This strategy is the most proactive
134
If ½ðsi  su Þ _ ðafi  af u Þ _ ðtfi  tf u Þ
then Prevention ðif not tech-cost-feasible; ReductionÞ (1)
else Reduction ðif not tech-cost-feasible; ContingencyÞ
Then, rule 2 is applied by looking to exceptional situations
arising from the six other attributes, especially when coupled
with the three most important ones, as shown in Equation (2).
           
If li  lh _ di  d l _ ei  el _ rci  r l _ ii  il _ ti  t l ^
 l     
s  si  su _ af l  afi  af u _ tf l  tfi  tf u
then Reduction ðif not tech-cost-feasible; ContingencyÞ
else Contingency ðif not tech-cost-feasible; AcceptanceÞ
(2)
In Equation (2), each attribute (e.g., si, severity of consequences of
risk ri) has its corresponding lower and upper thresholds (sl and
su, in this case). Once the appropriate mitigation strategy has been
preliminary selected (prevention, reduction, contingency, or
acceptance), the adopted strategy is to be described in sufficient
detail so as to allow its implementation and follow-up.
Group Mitigation Strategies
Normally, risks are treated as stand-alone. This means that for
each identified and assessed risk, a suitable alleviation action is
proposed. It is recommended that the nature of the identified
risks and the initially proposed strategies be considered together.
It is possible to group some risks and mitigation strategies so
that one strategy addresses and mitigates multiple risks. This
approach can result in lower overall efforts and lower costs.
Follow-Up on Risks and on Adopted Mitigation Strategies
There are two powerful reasons for following up on risks and
mitigation strategies. Risks may evolve over time, as well as the
understanding of and the information about the risks that the
project manager has. Therefore, identified risks need to be
periodically reassessed, in order to determine the extent to
which adopted mitigation strategies are still appropriate. Valida-
tion is also essential. Even if the nature of a risk has not
changed, the actual effectiveness of the adopted mitigation strat-
egy must be ascertained to determine if alternative alleviation
measures are needed. If risks have changed and/or their corre-
sponding strategies were not fruitful, then proceed to reassess
risks, revise and regroup mitigation strategies. If risks have not
changed and/or their corresponding strategies were fruitful,
then, if not project end, proceed to ongoing risks identification,
and if it were the end of the project, proceed to compile, register,
and disseminate risk-related lessons learned.
Reassess Risks, Revise, and Regroup Mitigation Strategies
For each risk for which new insight is available, a new evaluation
is needed. The assessment is done as explained before. Following
the new assessment, it is necessary to decide whether to main-
tain the initial alleviation strategy or to choose a different strat-
egy. The experience gathered with the results of the initial
strategy should be taken into account. If any of the initial
mitigation strategies have been changed, it may be necessary to
reconsider their new potential groupings.
Identify Ongoing Risks
Risks can occur at any point in time. Risks that could disrupt the
project may go undetected if there is not a proactive approach to
Engineering Management Journal Vol. 30 No. 2 2018
their identification. Sometimes, those risks will be identified as a
result of preplanned, periodic reflections on potential risks,
whereas, in other cases, risks could be detected in real time
due to specific circumstances.
Compile, Register, and Disseminate Risk-Related Lessons
Learned at Project Closure
As important as preventing a project from finishing late, with
underperformance, with a cost overrun, or combination thereof,
is to capitalize on the lessons learned from the risk management
effort, to help the organization grow. The sharing of lessons learned
will contribute to make the company a learning organization
(Edmondson, 2012; Garvin et al., 2008). The previous subsections
describe the final version of the procedure that was gradually
developed by the author of this article through the application of
risk management practices in several projects. An application
example of the final version of the procedure is described next.
An Application Example
The application of the dynamic risk management procedure is
shown using selected data from a defense project in Spain. In the
past decade, the Spanish firm Electroop obtained a contract to
produce, over a period of 7 years, electronic boards and har-
nesses for the Spike anti-tank missile procured by the Spanish
Ministry of Defense from the Israeli firm Rafael. In fulfillment of
a requirement for local content, the contract was awarded to the
Spanish firm General Dynamics Santa Bárbara Sistemas
(GDSBS), being Rafael GDSBS’ technological partner and licen-
sor. The contract signed between Electroop and GDSBS
included a suspensive condition, which established that, within
the first 6 months, a First Article Inspection (FAI) was to be
conducted by Rafael. If the FAI was not passed, a second
opportunity would be given in no more than 3 months. Failure
to pass the FAI on that second instance would render the con-
tract void. Satisfactory project performance required, among
other requirements, the successful passing of the FAI. Conse-
quently, risk management was a pivotal activity in this project.
Risks were identified, assessed, and prioritized to enable the
adoption of suitable mitigation strategies aimed at minimizing
the risk of not passing the FAI.
Risks with the failure to pass the FAI were identified
through the application of several techniques, including
FTA, Hazard Operations (HazOp), and brainstorming. The
approach to successfully passing the FAI was divided into
four areas. First, it was necessary to demonstrate a compre-
hensive control and configuration management. Full trace-
ability was also required. Traceability from the reception of
bills of materials for boards and harnesses, to the shipping of
produced and tested units and associated documentation,
was required. Second, low rejection and scrap rates were
expected to demonstrate the capability and robustness of
the manufacturing processes and to demonstrate competence
as a financially profitable and technically reliable supplier.
Third, high reliability of the functional testing process was
expected. Fulfillment of this requirement was verified
through the ability to correctly discriminate in a blind test
run by Rafael between functionally good and defective
boards. Furthermore, testing capabilities had to identify
what was wrong with faulty boards or harnesses. Fourth, it
was necessary to prove competence in performing a com-
prehensive visual inspection of functionally good boards and
harnesses, to ensure full compliance with requirements
defined in the IPC A 610 Acceptability of Electronic Assem-
blies standard. The 17 risks Electroop identified across these

Exhibit 8. Short List of Selected Risks

FAI-related block Risk Risk description

1 – Control and r1 Purchase of components from unauthorized suppliers

configuration r2 Inadequate in-coming quality inspection for received components, allowing defective or incorrect components to be
management accepted into the inventory
r3 Lack of availability of components due to market obsolescence

2 – Capability and r4 Wrong version of software downloaded in programmable memory chip

robustness of r5 Suboptimized development of the assembly manual
manufacturing r6 Production manager leaves the company shortly before the FAI
processes r7 Damaging a board when manually soldering the filter
r8 Late reception of long lead time components
r9 Inconsistencies between operators in their application of the assembly manual to manufacture board batches
r10 New recruits fail to pass the IPC A 610 exam

3 – Reliability of r11 Accept as good boards that are actually defective (testing error type II)
functional tests r12 Test manager leaves the company shortly before the FAI
r13 Damaging a board when placing the board in the bed of needles

4 – Reliability of r14 Failure to spot quality-wise unacceptable soldering of components due to lack of appropriate training and education
visual inspections r15 Quality manager leaves the company shortly before scheduled FAI
r16 Lack of analysis of the causes of the detected errors and of implementation of the necessary changes to increase the
robustness of the manufacturing and inspection processes
r17 Inconsistencies between quality inspectors in decisions based on visual examinations of board batches

FAI: First article inspection.

Engineering Management Journal Vol. 30 No. 2 2018 135

136
Exhibit 9. Assessment of Selected Risks: Initial (I) and 2 months into the Project (2M)

Risk Attributes

Early Time Aggregability Trigger

Code Description Assessment Likelihood Severity Detectability warning Recoverability Immediacy elapsed factor factor

r1 Purchase of components from unauthorized suppliers I 4 1 4 N Y N – af1,2 = 1

r2 Inadequate in-coming quality inspection of received components, I 4 2 4 N N Y – af2,1 = 1
allowing defective or wrong components to be accepted into the
inventory
r3 Lack of availability of components due to market obsolescence I 2 3 5 Y Y N –
r4 Wrong version of software downloaded in non-erasable I 4 2 5 N N Y –
programmable memory chip
r5 Suboptimized development of the assembly manual I
r6 Production manager leaves the company shortly before the FAI I 5 2 5 N N N – af6,12,15 = 1
r7 Damaging a board when manually soldering the filter I 4 3 3 N N Y – af7,14 = 1
2M 5 3 3 N N Y – af7,14 = 1
r8 Late reception of long lead time components I 3 3 5 N N Y – 2
r9 Inconsistencies between operators in their application of the I 3 3 4 Y N Y –
assembly manual to manufacture board batches
r10 New recruits fail to pass the IPC A 610 exam I 4 3 5 N Y N – 12
r11 Accept as good boards that are actually defective (testing error type II) I 5 3 3 N N Y –
r12 Test manager leaves the company shortly before the FAI I 5 2 5 N N N – af6,12,15 = 1
2M 5 2 5 N N N – af6,12,15 = 1
r13 Damaging a board when placing the board in the bed of needles I 4 3 4 N N Y –

Engineering Management Journal

r14 Failure to spot quality-wise unacceptable soldering of components I 3 2 3 Y Y Y – af14,7 = 1
due to lack of appropriate training and education 2M 4 2 3 Y Y Y – af7,14 = 1
r15 Quality manager leaves the company shortly before scheduled FAI I 5 2 5 N N N – af6,12,15 = 1
r16 Lack of analysis of the causes of the detected errors and of I 4 2 4 Y Y N –
implementation of the necessary changes to increase the robustness
of the manufacturing and inspection processes

Vol. 30 No. 2
r17 Inconsistencies between quality inspectors in decisions based on I 2 3 3 N Y Y –
visual examinations of board batches 2M 3 3 3 N Y Y –

FAI: First article inspection.

2018
 Exhibit 10. Mitigation Strategies Adopted for Selected Risks

Risk Mitigation strategy (ms)

Code Description Code Initial/Updated Type and description

Engineering Management Journal

r7 Damaging a board when manually soldering the filter r7 ms1 Initial Reduction – Increase the training in manual soldering in order to further develop the skills of the
operators (specially those of the new recruits) and reduce the likelihood of damages to the filter
or to other components in the soldering operations
r7 ms2 Initial Reduction – Improve the quality of the assembly manual so that it gives crystal clear instructions,
tips and warnings to the operators, thus reducing the likelihood of interpretation of the way to
perform the soldering, which could lead to component damage. To improve the quality of the
assembly manual, its writing will be assigned to a senior engineer with a good record for

Vol. 30 No. 2
developing technical manuals
r12 Test manager leaves the company shortly before the FAI r12 ms1 Initial Reduction – Hold a meeting with the test manager to assess his overall degree of satisfaction with
the company, aiming at identifying any potential issues of his concern or dissatisfaction, if so then
take the appropriate and feasible actions
r12 ms2 Initial Reduction – Ask the test manager to identify a qualified senior technician in the test department
and have him follow up closely all the test activities related to the contract

2018
r14 Failure to spot quality-wise unacceptable soldering of
components due to lack of appropriate training and
education
r17 Inconsistencies between quality inspectors in their
decisions in visual examinations of board batches
r12 ms3 Initial Contingency – Should the test manager leave, the identified senior technician would be
immediately appointed to take over the responsibilities as new test manager
r14 ms1 Initial Reduction – Increase the training in visual inspections, in order to further develop the skills of the
inspectors (specially those of the new recruits) and reduce the likelihood of failure to spot quality-
wise unacceptable soldering of components
r13 ms2 Initial Reduction – Implement a 2-month mentor project in which each senior inspector takes
responsibility for mentoring two junior ones, in order to share with them skills and tacit
knowledge, to help them go faster up the learning curve
r17 ms1 Initial Reduction – Conduct special sessions chaired by the quality manager at which boards picked at
random were inspected by several technicians, who would then exchange their perceptions,
seeking to harmonize their assessments and interpretations
Updated, Reduction – Organize a workshop to be taught by a recognized expert in the IPC A 610 standard,
2 months into at which the end customer is invited as observer, to strengthen the knowledge and correct
the project interpretation of the standard, as well as to achieve a higher degree of homogeneity in the
assessments done by the inspectors

FAI: First article inspection.

137
4 areas are summarized in Exhibit 8. The assessment of
these identified risks is summarized in Exhibit 9. Two risk
chains were identified. In the first risk chain, risk r8 (late
reception of long lead time components) could trigger risk r5
(suboptimized development of the assembly manual), which
could then trigger risk r9 (inconsistencies between operators
in the application of the assembly manual). The severity of
the consequences of r9 was far worse than those of r8. Since
one chain could be triggered resulting in two downstream
risks, triggering factor for risk r8 is tf8 = 2. In the second
risk chain, risk r10 (failure of new recruitments passing IPC
A 610 exam) could trigger risk r14 (failure to spot quality-
wise unacceptable soldering of components due to lack of
appropriate training and education), risk r17 (inconsistencies
between quality inspectors in decisions based on visual
examinations of board batches), and risk r11 (accept as
good boards ones that are defective – testing error Type
II). Since three chains could be triggered by risk r10, result-
ing in a total of three downstream risks, the triggering factor
of risk r10 is tf10 = 3.
Furthermore, three blocks of risks that could be aggre-
gated were identified. If risk r1 (purchase of components
from unauthorized suppliers) and risk r2 (inadequate in-
coming quality inspection of received components, allowing
defective or wrong components to be accepted into the
inventory) were to happen together, negative consequences
would occur. If risk r6 (production manager leaves the com-
pany shortly before the FAI), risk r12 (test manager leaves
the company shortly before the FAI), and r15 (quality man-
ager leaves the company shortly before the FAI) were to
happen simultaneously, the negative consequences to the
project would be substantial. Third and final, if risk r7
(damaging a board when manually soldering the filter) and
risk r14 (failure to spot quality-wise unacceptable soldering
of components due to lack of appropriate training and
education) were to happen simultaneously, the consequences
to the project would also be substantial. In these cases of
aggregated risks, the consequences of the simultaneous
materialization were assessed subjectively by the project
team. In all cases, the consequences were considered to be
much more negative than those consequences of any of these
risks that could happen individually.
Next, initial mitigation strategies were identified and
selected. Exhibit 10 shows the strategies chosen for selected
risks. Given that the FAI had to be held within the first
6 months, after 2 months into the project a reassessment was
conducted, to validate the goodness of the adopted mitigation
strategies and to ascertain if there were new risks or changes to
risks previously identified. The assessment concluded that the
strategies had produced the desired results, but the risk of
inconsistencies between quality inspectors in decisions based
on visual examinations of board batches (r17) was now perceived
as more likely to happen, based on gathered data by the quality
inspectors. This is reflected in Exhibit 9, in the rows indicated by
2 months into the project. A new mitigation strategy was
adopted, as shown in Exhibit 10, in the row indicated by
2 months into the project. The risk management approach was
very successful. All adopted mitigation strategies yielded the
desired results and the FAI was successfully passed at the first
attempt. Furthermore, the ongoing risk management
138
contributed substantially to the successful deliveries of boards
and harnesses as established in the contract schedule.
Implications for Engineering Managers
Bringing projects to a successful end is the responsibility of
engineering managers. Achieving project goals requires a
number of skills and abilities, including the ability to lead
team members and to proactively manage identified risks.
Even for managers who are aware of the role and relevance
of risk management, traditional approaches to risk manage-
ment have three limitations. First, risk identification often
relies on product-based techniques, such as FMECA, which
overlook risks that only process-based techniques detect.
Second, the assessment of identified risks is done with a
reduced set of criteria. Third, the assessment is done once,
neglecting changes that occur over time. This article
provides engineering managers with a procedure for mana-
ging risks dynamically. Most engineering managers are
aware of the importance of managing risks, but traditional
approaches have limitations. A deep understanding of how
to perform a comprehensive risk assessment, using a com-
prehensive set of criteria and following the steps described
in this article, enables engineering managers to have a more
complete understanding of risks and of the potential impact
of each risk. The successful application in several projects of
the procedure described in this article ensures that its appli-
cation will enable the adoption of mitigation strategies that
are more likely to be effective and comprehensive. Further-
more, a holistic approach to risk management reinforces the
ability of engineering managers to have a global view of the
project by focusing not only on the tasks to be done but also
on what could make the project derail, the global view that
is also known as the conceptual skill (Katz, 1986). Neglect-
ing to adequately manage risks in a project is, in itself, a
very serious risk with consequences. Successfully coping with
the growing complexity of projects requires a dynamic
approach to risk management. The procedure presented in
this article provides an approach for engineering managers
to dynamically manage identified risks.
Acknowledgments
The author wishes to express his most sincere gratitude to
the reviewers and editors. Thanks to their insightful recom-
mendations, the readability and structure of the paper were
significantly enhanced.
References
Altabbakh, H., Murray, S., Grantham, K., & Damle, S.
(2013). Variations in risk management models: A com-
parative study of the space shuttle challenger disasters.
Engineering Management Journal, 25(2), 13–24.
doi:10.1080/10429247.2013.11431971
American Institute of Chemical Engineers. Center for Chemical
Process, S. (2014). Guidelines for initiating events and inde-
pendent protection layers in layer of protection analysis
Retrieved from Ebook Library http://public.eblib.com/
choice/publicfullrecord.aspx?p=1895791
Aven, T., & Renn, O. (2009). The role of quantitative risk
assessments for characterizing risk and uncertainty and
delineating appropriate risk management options, with
Engineering Management Journal Vol. 30 No. 2 2018
 special emphasis on terrorism risk. Risk Analysis, 29(4),
587–600. doi:10.1111/j.1539-6924.2008.01175.x
Bahill, A. T. (2010). Design and testing of an illuminance man-
agement system. ITEA Journal, 31(1), 63–89.
Bahill, A. T. (2012). Diogenes, a process for identifying unin-
tended consequences. Systems Engineering, 15(3), 287–306.
doi:10.1002/sys.20208
Ben-Asher, J. Z. (2008). Development program risk assessment
based on utility theory. Risk Management, 10(4), 285–299.
doi:10.1057/rm.2008.9
Bharathy, G. K., & McShane, M. K. (2014). Applying a systems
model to enterprise risk management. Engineering Man-
agement Journal, 26(4), 38–46. doi:10.1080/
10429247.2014.11432027
Blanchard, B. S., & Blyler, J. E. (2016). System engineering
management (5th ed.). Hoboken, NJ: Wiley.
Blanchard, B. S., & Fabrycky, W. J. (1981). Systems engineering
and analysis. Englewood Cliffs, NJ: Prentice-Hall.
Carbone, T. A., & Tippett, D. D. (2004). Project risk manage-
ment using the project risk FMEA. Engineering Manage-
ment Journal, 16(4), 28–35. doi:10.1080/10429247.2004.
11415263
Chaves, A., & Bahill, A. T. (2014). Comparison of risk analysis
approaches and a case study of the risk of incorporating
solar photovoltaic systems into a commercial electric
power grid. Systems Engineering, 17(1), 89–111.
doi:10.1002/sys.21254
Cox, L. A. (2008). What’s wrong with risk matrices? Risk Ana-
lysis, 28(2), 497–512. doi:10.1111/risk.2008.28.issue-2
Dalkey, N. C., & Rand, C. (1969). The Delphi method: An
experimental study of group opinion. Santa Monica, CA:
Rand Corp.
Davenport, T. H., & Prusak, L. (2010). Working knowledge: How
organizations manage what they know ([Nachdr.] ed.). Bos-
ton, MA: Harvard Business School Press.
Dvir, D., Raz, T., & Shenhar, A. J. (2003). An empirical analysis
of the relationship between project planning and project
success. International Journal of Project Management, 21
(2), 89–95. doi:10.1016/S0263-7863(02)00012-1
Edmondson, A. C. (2012). Teaming how organizations learn,
innovate, and compete in the knowledge economy. San Fran-
cisco, CA: Jossey-Bass.
Elliott, M. W. (2014). Enterprise risk management (1st, 2nd print
ed.). Malvern, PA: The Institutes.
Eveleens, J., & Verhoef, C. (2010). The rise and fall of the chaos
report figures. IEEE Software, 27(1), 30–36. doi:10.1109/
MS.2009.154
Garvin, D. A., Edmondson, A. C., & Gino, F. (2008). Is yours a
learning organization? Harvard Business Review, 86(3),
109–116.
Goldratt, E. M. (1990). Theory of constraints. Great Barington,
MA: North River.
Hobday, M. (1998). Product complexity, innovation and industrial
organisation. Research Policy, 26(6), 689–710. doi:10.1016/
10.1016/10.1016/S0048-7333(97)00044-9
INCOSE. (2015). Systems engineering handbook (4th ed.). INCOSE-
TP-2003-002-04. San Diego, CA: John Wiley & Sons.
Jensen, C. (1996). Delphi in depth. Berkeley, CA: Osborne
McGraw-Hill.
Kaplan, S., Garrick, B. J., & Apostolakis, G. (1981). Advances in
quantitative risk assessment–the maturing of a discipline.
Engineering Management Journal Vol. 30 No. 2
IEEE Transactions on Nuclear Science, 28(1), 944–946.
doi:10.1109/TNS.1981.4331310
Kappelman, L. A., McKeeman, R., & Zhang, L. (2007). Early
warning signs of it project failure: The dominant Dozen.
Edpacs, 35(1), 1–10. doi:10.1080/07366980701238939
Katina, P. F., Ariel Pinto, C., Bradley, J. M., & Hester, P. T.
(2014). Interdependency-induced risk with applications to
healthcare. International Journal of Critical Infrastructure
Protection, 7(1), 12–26. doi:10.1016/10.1016/10.1016/j.
ijcip.2014.01.005
Katz, R. L. (1986). Skills of an effective administrator. Harvard
Business Review, 64(2), 178.
Khakzad, N., Khan, F., Amyotte, P., & Cozzani, V. (2014).
Risk management of domino effects considering
dynamic consequence analysis. Risk Analysis, 34(6),
1128–1138. doi:10.1111/risa.2014.34.issue-6
Lam, P. T. I. (1999). A sectoral review of risks associated
with major infrastructure projects. International Journal
of Project Management, 17(2), 77–87. doi:10.1016/
S0263-7863(98)00017-9
Linstone, H. A., & Turoff, M. (2002). The Delphi method: Tech-
niques and applications Retrieved from //is.njit.edu/pubs/
delphibook/
Liu, G. F., Feng, X. T., Feng, G. L., Chen, B. R., Duan, S. Q.,
& Chen, D. F. (2016). A method for dynamic risk
assessment and management of rockbursts in drill and
blast tunnels. Rock Mechanics and Rock Engineering, 49
(8), 3257–3279. doi:10.1007/s00603-016-0949-5
Luhmann, N. (1993). Risk: A sociological theory. New York,
NY: A. de Gruyter.
Marszal, E. M., & Scharpf, E. W. (2002). Safety integrity level
selection: Systematic methods including layer of protec-
tion analysis. Research Triangle Park, NC: Instrumenta-
tion, Systems, and Automation Society.
Paltrinieri, N., Khan, F., & Cozzani, V. (2014). Coupling of
advanced techniques for dynamic risk management.
Journal of Risk Research, 18(7), 910–930. doi:10.1080/
13669877.2014.919515
Pennock, M. J., & Haimes, Y. Y. (2002). Principles and
guidelines for project risk management. Systems Engi-
neering, 5(2), 89–108. doi:10.1002/(ISSN)1520-6858
Pinto, J. K., & Mantel, S. J. J. (1990). The causes of project
failure. IEEE Transactions on Engineering Management, 37
(4), 269–276. doi:10.1109/17.62322
Reeves, J. D., Eveleigh, T., Holzer, T. H., & Sarkani, S.
(2013). Risk identification biases and their impact to
space system development project performance. Engi-
neering Management Journal, 25(2), 3–12. doi:10.1080/
10429247.2013.11431970
Reichelt, K., & Lyneis, J. (1999). The dynamics of project
performance: Benchmarking the drivers of cost and
schedule overrun. European Management Journal, 17
(2), 135–150. doi:10.1016/S0263-2373(98)00073-5
Sage, A. P. (1992). Systems engineering. New York, NY: Wiley.
Shenhar, A., & Dvir, D. (2007). Reinventing project manage-
ment: The diamond approach to successful growth and
innovation. Boston, MA: Harvard Business School Press.
Slegers, N. J., Kadish, R. T., Payton, G. E., Thomas, J.,
Griffin, M. D., & Dumbacher, D. (2012). Learning
from failure in systems engineering: A panel discussion.
Systems Engineering, 15(1), 74–82. doi:10.1002/sys.v15.1
2018 139
Sols, A. (2015). Balancing the global view with the relevant few in
project management. Huntsville, AL: International Annual
Conference of the American Society for Engineering Man-
agement (ASEM).
Standish-Group. (2015). The Standish Group Report CHAOS.
Retrieved from https://www.projectsmart.co.uk/white-
papers/chaos-report.pdf, March 15, 2017
Surowiecki, J. (2004). The wisdom of crowds: Why the many are
smarter than the few and how collective wisdom shapes
business, economies, societies, and nations (1st ed.). New
York, NY: Doubleday.
Syed, M. (2014). Black box thinking. [Place of publication not
identified]. New York, NY: John Murrey Publishers.
Yeo, K. T., & Ren, Y. (2009). Risk management capability
maturity model for complex product systems (CoPS) pro-
jects. Systems Engineering, 12(4), 275–294. doi:10.1002/sys.
v12:4
140
Zuofa, T., & Ochieng, E. G. (2014). Project failure: The way forward
and panacea for development. International Journal of Business
and Management, 9(11), 59–71. doi:10.5539/ijbm.v9n11p59
About the Author
Dr. Alberto Sols holds an MSc in Naval Architecture and Marine
Engineering from Universidad Politécnica de Madrid, an MSc in
Systems Engineering from Virginia Tech, and a PhD in Systems
Engineering from Stevens Institute of Technology. He has
35 years of combined industry and academic experience and is
the Dean of the School of Architecture, Engineering and Design
at Universidad Europea de Madrid. He has published over 35
technical papers, written four books, and coauthored another.
Contact: Alberto Sols, School of Architecture, Engineering
and Design, Universidad Europea de Madrid, c/Tajo s/n; 28670
Villaviciosa de Odon, Madrid, Spain; alberto.sols@universida-
deuropea.es
Engineering Management Journal Vol. 30 No. 2 2018