See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220636569

How to use a systems diagram to analyse and structure complex problems for
policy issue papers

Article  in  Journal of the Operational Research Society · July 2011

DOI: 10.1057/jors.2010.28 · Source: DBLP

CITATIONS READS

17 564

4 authors:

T.E. Van der Lei B. Enserink

Delft University of Technology Delft University of Technology
28 PUBLICATIONS   235 CITATIONS    78 PUBLICATIONS   1,306 CITATIONS   

SEE PROFILE SEE PROFILE

Wil Thissen Geertje Bekebrede

Delft University of Technology Delft University of Technology & Tilburg University
103 PUBLICATIONS   1,748 CITATIONS    51 PUBLICATIONS   746 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

HARMONICOP View project

EU FP7 FloodProBE View project

All content following this page was uploaded by Wil Thissen on 03 January 2014.

The user has requested enhancement of the downloaded file.

Journal of the Operational Research Society (2011) 62, 1391 --1402 © 2011 Operational Research Society Ltd. All rights reserved. 0160-5682/11

www.palgrave-journals.com/jors/

How to use a systems diagram to analyse and

structure complex problems for policy issue papers
TE van der Lei, B Enserink∗ , WAH Thissen and G Bekebrede
Delft University of Technology, Delft, the Netherlands
Many policy problems are complex in the sense that natural, technological, social and human elements interact.
Problem exploration and structuring are essential as a basis for deliberate and focused approaches towards
problem resolution. The results of problem exploration efforts can be laid down in the form of a policy issue
paper. We have developed a systemic, stepwise approach, which has been elaborated and taught for over a
decade to hundreds of students. This seven-step approach centers on the construction of a system diagram as
a means to provide structure to the conceptualisation of a complex problem situation. The approach is based on a
conscious combination of existing relatively straightforward analytical methods including objectives hierarchy,
means-ends analysis, causal diagramming, stakeholder analysis, and contextual scenarios. The obtained insights
are then summed up in a policy issue paper, which is the basis for further planning and research.
Journal of the Operational Research Society (2011) 62, 1391 – 1402. doi:10.1057/jors.2010.28
Published online 28 April 2010

Keywords: OR education; problem structuring; policy issue paper; policy analysis; system diagram;
complex problems

Introduction In such situations analysts are advised to seriously invest

Many policy problems are wicked, ill-structured or complex
in the sense that natural, technological, social, and human
elements interact. As a result, a variety of problem perceptions
exists, values and interests may be conflicting, and power and
resources to change things are distributed over multiple actors
(Rittel and Webber, 1973, Dunn, 1994; de Bruijn and Porter,
2004; Koppenjan and Klijn, 2004). Such complexity is the
everyday reality of analysts and problem solvers concerned
with such complex socio-technological systems.
Exploration of the problem situation and identification and
selection of the key issues to be addressed are the first step
that must be taken toward tackling the situation. Dunn (1994,
p 106) explains this in the following way:
Whereas well-structured problems permit analysts to use
conventional methods to resolve clearly formulated or self-
evident problems, ill-structured problems demand that the
analyst first take an active part in defining the nature of the
problem itself.
∗ Correspondence: B Enserink, Delft University of Technology, PO Box
5015, 2600GA Delft, the Netherlands.
E-mail: b.enserink@tudelft.nl
The first three authors work at the Policy Analysis section, the fourth
at the Policy, Organisation, Law & Gaming section of the Faculty of
Technology, Policy and Management of Delft University of Technology,
the Netherlands.
in problem formulation activities before jumping to solutions,
to prevent the risk of problems half-solved, growing worse or
even solving the wrong problem (Quade, 1980). This struc-
turing of an unstructured problem and defining an appropriate
scope for further analysis and action are essential skills that
all policy analysts should master. Therefore, when developing
a novel curriculum in Systems Engineering, Policy Analysis
and Management (see, eg, Thissen, 2000a), we were faced
with the challenge to teach our (undergraduate) students a
way of thinking supported by an analytic approach that would
prepare them for their later work as practicing analysts.
As a starting point, we noted that various authors have
proposed to document the findings of problem exploration
efforts in a so-called ‘policy issue paper’. Preparing it should
force the analyst to address all relevant aspects of a problem
situation in an analytically sound way (Ackoff, 1978; Miser
and Quade, 1988; Quade, 1989). The policy issue paper helps
to reach a joint problem understanding between analyst and
client and it proposes a contribution to solving that problem
by providing focus and proposing specific further steps.
A number of authors have formulated requirements of a
policy issue paper (Quade, 1980; Dunn, 1994; Checkland,
1985). All these authors indicate that a broad approach should
be taken, looking at the situation from a variety of angles.
However, these authors provide little guidance on how exactly
to proceed to structure a problem for a policy issue paper.
Although Dunn (1994), Quade (1980), and Checkland
(1985) provide scarce guidance on how to structure a
1392 Journal of the Operational Research Society Vol. 62, No. 7

problem for an issue paper there is a complementary body of to contribute to transparency, as was urged by Eden and
knowledge, called problem structuring methods (PSMs) that Ackermann (2006) as we want students and colleague practi-
focuses on the art and craft of problem structuring. In a recent tioners to understand what analytical and modelling steps may
special issue of the Journal of the Operational Research be taken to construct a systems diagram; thereby inevitably
Society, Shaw et al (2006) while referring to Rosenhead compromising on the level of detail of describing the
(1989) and Rosenhead and Mingers (2001) define PMSs as a method.
collection of participatory modelling approaches that aim to
support a diverse collection of actors in addressing a prob- The system diagram as a means to represent a structured
lematic situation of shared concern. The methods, generally, problem
are participative in character and help to structure a problem
together with the clients. Therefore adequate facilitation of We use the so-called system diagram (Sage, 1992; Sterman,
the group process is crucial for success. An overview of 2000; Walker, 2000; Sage and Olson, 2001) as a core concept
different types of PSMs is given amongst others by Eden and to represent a structured view of a problem situation. All of the
Ackermann (1998) and Checkland and Poulter (2006). Three analytical tools that will be presented as part of our approach
PSMs are dominant: Soft Systems Methodology, Strategic have a relationship to the system diagram.
Options Development and Analysis, and Strategic Choice The system diagram distinguishes the system itself, the
(Rosenhead and Mingers, 2001). Eden and Ackermann (2006, steering factors, the external factors, and the outcomes of
p 766) argue that the majority of the users of these methods interest or criteria (see Figure 1). As we will illustrate in the
tend to pragmatically combine parts of each of these three next sections, the complete diagram is constructed through
methods, with little regard for their theoretical underpinnings. seven steps and iterations, where each step is a sub-analysis.
In addition, they note that PMSs are not just used to structure Although our steps are presented as a neat sequence, in prac-
problems, but rather to seek to facilitate agreements to act. tice there is no perfect order and a lot of iteration.
As a result of the complexity of the ‘craft skills demanding
many different roles for the ORer’, the approaches have A step-wise approach for constructing a systems diagram
become complex, less transparent than would be desirable,
On the basis of the demands put forward in literature (Quade,
and difficult to transfer.
1980; Dunn, 1994; Checkland, 1985) we outline the ques-
The approach presented in this paper is different from the
tions a policy issue paper should address in Table 1. The
development of PSM’s in two respects. First, a policy issue
first four questions help to structure the problem situation
paper is usually written by an individual analyst (or a small
from the perspective of the problem owner. The first ques-
team) for a client or decision maker and is not typically the
tion, treated in the next section is intended to determine the
outcome of a participatory process. Rather, it is a problem
problem level or scope from the perspective of the problem
exploration that might precede such a participatory process.
owner as suggested by Dunn and Checkland (based on Quade,
The purpose of an issue paper is to explore and define a
1982, pp 71–76), by demanding insight into the source and
problem in sufficient depth so that a client may decide to
background of the problem and reasons for attention (Dunn,
do nothing, commission a study, proceed with a participative
1994, p 363; Checkland, 1985, p 169). The subsequent ques-
process in which the identified stakeholders will be involved,
tions address the problem owner’s objectives, means and the
or combinations of these (Dunn, 1994, p 362; Checkland,
system itself. The goals and objectives and measures of effec-
1985, p 168). Second, our approach is based on using and
tiveness are for example demanded by Checkland (1985).
combining analytical concepts rooted in the ‘hard’ systems
Also Dunn mentions the elements of goals and objectives,
traditions (Sage, 1992; Sterman, 2000; Walker, 2000; Sage
measures of effectiveness and potential solutions as part of an
and Olson, 2001). This systems view focuses on analytical
issue paper. Questions five and six relate directly to the envi-
rigour, consistency and conceptual clarity rather than on the
ronment or context of the problem of the problem owner, and
facilitation of stakeholder interaction.
help to assess the extent to which the problem owner depends
The approach has been developed in a period of more than
on other actors and external factors beyond his control. Dunn
10 years of teaching to over 800 policy analysis students.
The courses focus on teaching students how to structure and
analyse a complex problem situation and how to write a External factors
clear policy issue paper based on this analysis. This is part of
the core undergraduate curriculum in Systems Engineering,
Policy Analysis and Management (see, eg Thissen (2000a) for
Steering factors Criteria
more information on the curriculum). An earlier version of the
approach has been presented at an IEEE conference (Thissen System of interest
et al, 2000b). The steps that outline the approach will be
illustrated by a simplified virtual case. By presenting this
simplified stepwise methodological approach we intend Figure 1 The basic system diagram.

Table 1 Questions that a policy issue paper should address
Question
1. Why should the problem be solved?
2. What determines success for the problem owner?
3. What are the means available to the problem owner?
4. What is the system of interest?
5. Who else is involved?
6. How might the problem change in future?
7. What further steps, including possible research, are
recommended?
adds an element of ‘major stakeholders’ and according to
Checkland ‘beneficiaries and losers’ have to be described.
The question about environmental changes is less central in
the outline of an issue paper as proposed by Checkland and
Dunn. However, the environment of ill-structured problems is
dynamic and therefore we take it into account in the analysis.
Question seven, is about what we learned from the analysis,
assesses the information gathered and helps to draw conclu-
sions and formulate recommendations for further research.
This is comparable with Checkland (1985, p 169) ‘framework
for analysis’ and ‘recommendations that may emerge.’ Both
Dunn and Checkland go a step further in their elements in
the issue paper by giving recommendations for solutions. Our
approach ends with supported knowledge gaps, research ques-
tions, and a proposal for follow-on steps.
These seven questions are the backbone of the stepwise
approach that we have developed and used in our teaching
efforts. For each question different analysis methods are
used. We emphasize iteration, that is, re-assessment and
re-formulation of the problem after each step.
To illustrate the use of the analysis methods that belong to
each question a fictional example is provided for an energy
company called LightOn suffering from capacity shortage. We
assumed LightOn has a need to reduce capacity shortage while
taking costs into account. The example serves to illustrate
the principles only and is not intended as a representation
of a complete detailed structuring of a complex empirical
problem. We used a limited set of mostly internet sources for
constructing the case (Goodell, 2001; van Walle, 2006) and
reports by Harrison (2004) and Smaardijk et al (2005).
Question 1: Why should the problem be solved?
Most policy decisions concern the choice of measures, the
creation or modification of facilities or projects that are driven
by a desire to solve problems, or at the very least to make
these problems controllable. Before making such decisions, it
is important to clearly describe what is considered to be the
problem. As a working definition we use: ‘a policy problem
is the gap between an existing or expected situation and
a desired situation (a principle, or norm)’ (Hoogerwerf, 1988).
In other words, a policy problem is the perceived difference
TE van der Lei et al—Using a systems diagram to analyse and structure complex problems 1393
Main focus
To structure the problem and provide focus from the perspective of a
problem owner
To asses how the environment influences the problem of the problem
owner and the possibilities to resolve it
To recommend future steps for the problem owner
between the present or expected future reality, and the desires
or norms of the problem owner(s).
It is of crucial importance to have a clear understanding of
what really matters to the problem owner. Often, problems
are formulated in terms of quite operational needs (such as
the need to reduce the capacity shortage in our example).
As a result, one may get ‘trapped’ into looking into a very
narrow spectrum of solutions. Therefore, first of all, the more
fundamental interests at stake should be explored. To this end,
we use a ‘problem level determination’ diagram, which is
based on the same logic as Keeney’s means-ends objectives
hierarchy (Keeney, 1992), but concentrates purely on the
‘Why- question’; why would the problem owner want this
problem to be solved? The diagram is constructed by starting
from the initial formulation of needs, by searching for the
underlying reasons for that need. Each time, the question is
asked ‘Why is it important to satisfy this need or to solve the
problem as initially articulated?’ This way, the more funda-
mental interests are identified. On the basis of the insights
thus gained, a decision must be made regarding the level
that is used as a starting point for further analysis and explo-
ration of solution options. Choosing a more fundamental
objective will generally open up a broader spectrum of solu-
tion options. The downside is that broader analyses will be
more encompassing and challenging, or even infeasible if a
very broad and fundamental objective is chosen, such as ‘to
improve everybody’s happiness’. Note that any change in the
choice of fundamental objective will imply a change in the
initial problem formulation. Figure 2 below illustrates how
such ‘problem level determination’ diagram may look like
for our ‘LightOn’ example.
Question 2: What determines success for the problem
owner?
Once a leading objective is chosen, criteria are specified that
can be used to ‘measure’ the extent to which the objective
is attained. To this end, a so-called (fundamental) objectives
hierarchy or objectives tree is used (eg Keeney, 1988, 1992;
Dunn, 1994). The primary objective is split out in intermediate
level objectives, and these are further specified until they
can be expressed in measurable indicators. The latter are
1394 Journal of the Operational Research Society Vol. 62, No. 7
To provide long-
term company
continuity
To increase
net profits
To increase
turnover and
income To reduce costs
To reduce
capacity shortage
Figure 2 ‘Problem level determination’ diagram for LightOn.
Figure 3 The objectives hierarchy and its relationship to the
right side of the system diagram.
called criteria and put at the right side or output side of the
system diagram. Objectives are found by asking the ‘What
is-question’; what attributes define this objective? A visuali-
sation of the relationship of the objectives hierarchy and the
system diagram is provided in Figure 3. It is important that
the criteria do not overlap, that is, each must measure another
aspect of what is desired (Keeney and Gregory, 2005). Objec-
tives in the tree need to have a direction like more profit, less
costs, less CO2 pollution, more recycling etc.
Question 3: What are the means or tactics available to the
problem owner?
How can the problem owner solve the problem; what are the
means he can use to solve it and/or to make it smaller? Ideas
for means or measures can be generated in different ways:
through analysis of the means of the problem owner and other
parties, by asking experts, by creative techniques like brain-
storming, by reasoning backwards from the chosen objectives
or by using checklists. Means or tactics can be of different
nature (see also Walker, 1988): for example communica-
tive (launch an awareness campaign); regulatory (create new
environmental legislation); financial (provide subsidies) or
technical (build a new power plant). We suggest to combine
Continuity of the
company
Boundary
Good image High profit
More
High client
sustainable
satisfaction
image
Figure 4 Example of an objectives hierarchy for LightOn.
the various methods and explore a broad range of possible
means.
We illustrate the use of a means-ends diagram or means-
ends objectives hierarchy (Keeney, 1992) by reasoning back-
wards from the objectives stemming from the previously
developed objectives. The question to be answered is: How
(by using which means) can the objective(s) be achieved?
By systematically reasoning backwards from a chosen objec-
tive a (broad) spectrum of means is mapped out that may
contribute to realisation of the objective (Figure 4).
The means-ends diagram is based on assumed causality.
Moreover means are associated with actions; therefore in the
means-ends diagram we use verbs (see Figure 5). In principle
the means can be interlinked (feed into one another) and feed-
back loops can occur.
Implementation of the lowest level means affects the value
of factors that influence the system and its outcomes. These
factors are represented as steering factors on the left side of
the system diagram. A visual example of the relationship of
the means-ends analysis and the system diagram is provided
in Figure 6. In Figure 6 the means are represented as blocks
and the objectives as black dots.
Question 4: What is the system of interest?
The system of interest is further structured and specified with
the help of a causal diagram or causal model (Simon, 1952;
Axelrod, 1976; Eden, 1988, 2004; Sterman, 2000; Bryson
et al 2004). The causal diagram, as we use it, consists of
factors and the influence relationships between these factors.
The relationships can either be positive, depicted with a ‘+’, or
negative, depicted with a ‘−’. A positive influence implies that
if factor A increases, factor B will also increase (supposing
 TE van der Lei et al—Using a systems diagram to analyse and structure complex problems 1395

A influences B). A negative influence relationship means that
an increase of A will lead to a decrease of B. If the character
of the influence is unknown or cannot be determined this is
indicated by a question mark. The causal diagram portrays the
causal mechanisms linking the tactics with the criteria. These
mechanisms are derived from established theories, definitions,
expert knowledge, and/or beliefs of the author(s) of such a
diagram. The intermediate factors represent key attributes of
the system, and are portrayed inside the system box in the
diagram. Besides factors that can be influenced by the problem
owner, other factors may affect the relevant system outcomes,
for example the weather. These factors may be identified by
asking what other factors may influence the relevant factors in
the system. As a result, we distinguish four classes of factors:
(1) the steering factors related to the tactics at the left side
of the diagram;
(2) the criteria at the right of the diagram;
(3) the intermediary or internal factors inside the box; and
(4) the external factors that do influence the intermediary
factors but cannot be controlled by the problem owner,
which are located at the top-side of the diagram.
Continuity
of the
company
More
sustainable High profit
image
A visual of the relationship of the causal diagram and the
system diagram is provided in Figure 7. An elaboration for
the LightOn example is given in Figure 8.
Question 5: Who else is involved and what are their means
to affect the interests of the problem owner?
The analysis of the perspective and interests of the problem
owner suffices when he himself has sufficient means to solve
his problem adequately. But in modern networked societies
most problem owners can only achieve their objectives in
cooperation with others and by preventing strong opposition.
Therefore the analyst should gain insight into what other
parties will or should be engaged, what their interests are,
what relevant means they possess, how they see the situation,
and what their intentions are.
A stakeholder/actor analysis and basic network analysis
serve to identify relevant social, institutional, and political
attributes of the problem situation. An example of the possible
lay-out of the actor and network analysis schemes is provided
in Tables 2 and 3. The actor network analysis does not so much
focus on the system of interest to the stakeholder, but rather
on the wider policy arena, in which the problem owner has to
solve his problem (see Figure 9 below). The relevant social,
institutional, and political attributes found will however often
lead to the identification of additional tactics, external factors,
or criteria and this may lead to extension or modification of
all other elements in the system diagram.
We suggest the following five steps in the execution of
an actor- and network analysis. Departing from the current

Increase
electricity
generation
capacity

Build Build fossil

renewable fuel
resource- capacity
capacity

Install Install
Install Install coal Install gas
solar mCHP
wind farms fired plant turbines
systems units

Figure 7 The causal diagram and its relationship with the system
Figure 5 The means-ends diagram of LightOn. diagram.

Figure 6 The means-ends diagram and its relationship to the left side of the system diagram.
1396 Journal of the Operational Research Society Vol. 62, No. 7

Total
Fuel electricity
price demand

+
Installed
Electricity + spot
coal-fired
Shortage market +
systems
purchases
- +

Installed + Total Amount of

extra Net
mCHP fossil electricity
income on
systems + capacity + delivered
+ + sold
electricity
+ Total
Installed capacity -
gas fired +
turbines Total extra + +
sustainable
+ capacity
total cost
Installed + (investment
solar + plus
systems operational)
+

Amount of
Installed
wind sustainable
+
turbines capacity

Figure 8 A first system diagram for LightOn.

Table 2 Part of actor analysis table for LightOn

Actors Interests Desired Existing or Causes Possibilities to

situation/objectives expected influence/courses
situation/gap of solution
Ministry of the More sustainable Less emissions Increase in CO2 Too much use Make subsidies
Environment energy industry of CO2 emissions without of fossil fuels available for
adequate policy innovative solutions
Ministry of Low energy prices More capacity Capacity shortage Lack of Subsidized investments
Economic to spur economic causes high prices generation in power generation
Affairs growth capacity
Households Low energy bill Pay less for energy Higher energy bill Increase in oil Install more energy
and high security and possibly help price saving equipment
of supply environment

knowledge about the problem, the actors that might have an
interest in the issue are identified. Relevant actors are those
who have a stake in the solution of the problem, those who
are affected by the solution and those who have a legitimate
interest (Enserink, 2000a; de Bruijn et al, 2002; Bryson,
2004). Interviews or reports are a good starting point for the
list. Another method is the ‘reputation method’, where actors,
starting with the problem owner, are being asked to indicate
who else has the resources that can contribute to or may be
affected by the solution of the problem. Second, an outline
of the formal chart is developed: the formal tasks, authori-
ties, and relations of actors. Studying the formal legislation,
procedures, policy pieces, and so on, provides a complemen-
tary indication of the parties that are possibly—or may need
to be—involved. Third, the interests, objectives, and problem
perceptions of the listed actors are determined. For each
 TE van der Lei et al—Using a systems diagram to analyse and structure complex problems 1397

stakeholder the following elements are mapped: interests,
desired situation/goals, present or expected situation, causes
as seen by the stakeholder, means or possibility to influence
the course of events (eg, ‘blocking power’) (see Table 1).
Fourth, the interdependencies between the listed actors are
explored, based on insight into the resources of the stake-
holders. Resources might for example be money, power to
implement or block certain decisions, expertize, or gateway
to the media. Careful analysis of the causal links between
such resources (or means) and criteria may help in identifying
and specifying the most important dependencies. Actors who
possess resources that are essential for solving the problem
owner’s problem are termed ‘critical’ (see Table 2). The
determination or willingness to use these resources deter-
mines whether such an actor should be considered dedicated
or non-dedicated. Fifth, the consequences of these findings
with regard to the problem formulation are determined. The
lesson is that critical actors are important—their concerns and
issues are relevant to the problem owner as their cooperation
is needed. Therefore, it is relevant to take their objectives
into account in any further analysis, and this may be done
by adding additional criteria that show at the right side of
the system diagram. The analysis might also have lead to
reconnaissance of additional means in possession of
supportive actors that might be willing to invest in the solu-
tion of the problem. Those means might be deployed and
could be placed as inputs at the left side of the diagram.1
Note that the addition of new elements (criteria and steering
factors) will necessitate reconsideration and mostly extension
of the causal diagram, and may also lead to the identification
of new external factors.
Tables 2 and 3 illustrate the possible results of an actor
network analysis for LightOn. For reasons of space we
limit ourselves to three actors only: the ministry of the
environment, economic affairs, and the households. Other
examples of actors are: network companies, environmental
groups, and producers of different types of energy generation
technologies.
We assume that the Kyoto protocol forces the national
government to reduce the overall carbon dioxide (CO2 )
emissions, and the ministry of the environment is the prime
champion of this objective. The ministry could, for example,
choose to subsidize sustainable energy generation to reduce
the nation’s CO2 emissions.
In the assessment of criticality (see Table 2) the house-
holds are considered critical but non-dedicated and the two
ministries as critical and dedicated, but in different positions
when it comes to support for traditional power generation.
The households are critical as they can switch to another
supplier thus affecting the LightOn’s business position. The
ministry of the environment is judged to potentially support
LightOn’s when its investment is in line with the ministry’s
CO2 reduction policy. Consequently a potentially interesting
new means might be government subsidies, represented as a
steering factor on the left side of the system diagram. Also,

1 Clearly what is explained here is a rough sketch only. Such a rough

sketch misses all nuances, and gives only a snapshot of the situation at a
specific time while situations may change rapidly. But the scheme does
Figure 9 Policy arena, system diagram, and its relation to the provide valuable insight in the requirements for and type of process that
stakeholder network. needs to be organized and gives a glance towards the recommendations.

Table 3 Table of dedicated and non-dedicated actors for LightOn

Dedicated actors Non-dedicated actors

Critical actors Non-critical actors Critical actors Non-critical actors
Similar perceptions, Actors that will Actors that will Indispensable potential Actors that do not have
interests, and objectives probably participate and probably participate and allies that are hard to to be involved initially.
are potential allies. are potential allies. activate.

Ministry of Economic Households

Affairs
Different perceptions, Potential blockers of Potential critics of Potential blockers that Actors to whom no atten-
interests, and objectives (certain) changes. (certain) changes. will not immediately tion needs to be given
(biting dogs) (barking dogs) jump to action. initially.
(sleeping dogs)
Ministry of the
Environment
1398 Journal of the Operational Research Society Vol. 62, No. 7
Figure 10 Visualisation of scenario logic for LightOn.
an additional indicator for the environmental friendliness of
the company (CO2 emissions prevented) is added to the right
side of the system diagram (see Figure 11).
Question 6: How might the problem change in future?
Most policy decisions are about affecting the future, and it is
important to consider possible future changes and uncertain-
ties more closely. Will the problem persist, worsen or disap-
pear? Under what conditions? There are two main causes of
uncertainty analysis should focus on: (1) changes from within
the system, and (2) changes in the context or environment
of the system (see Walker et al, 2003), for a more extensive
discussion of uncertainty types and sources).
A range of methods can be of help to explore the impacts
of possible uncertainties on future problem developments.
Expert views, trend extrapolation, model-based sensitivity
analyses, and a variety of scenario methods may be used
(Porter et al, 1991). In our educational setting, we generally
focus on the use of contextual scenarios as we feel exogenous
sources of uncertainty are generally important and influen-
tial but often underestimated or even ignored. Contextual
scenarios try to map the effect of the driving forces that have
an influence on the system (Schwartz, 1991; Reibnitz, 1998)
but largely beyond the span of control of the problem owner.
Consequently these scenarios provide images of possible
future environments of the system that can influence the
results of a policy to a large extent. Context scenarios are
generally used to investigate the robustness of the proposed
policies and are usually qualitative in nature (Rosenhead,
1989). In our systems approach to problem structuring a
small set of alternative possible future contexts for the system
of interest is developed, and it is explored how these different
contexts would affect the problem situation (Thissen, 1998;
Enserink, 2000b, 2004).
As a result, sometimes new exogenous factors need to be
taken into account. Figure 10 presents a (simple) scenario
logic or scenario skeleton for the LightOn example. The
three axes span a scenario space wherein an infinite number
of scenarios can be positioned. For LightOn we developed a
limited set of contextual scenarios that were further detailed,
inspired by Schwartz’ approach to scenario composition
(Schwartz, 1992).
The various steps presented above lead to either specifica-
tion of elements of the problem analysis, or modification, for
example by adding criteria, exogenous factors, or means. In
principle, each of these additions may affect other elements
of the problem analysis. Therefore after each step an itera-
tion is made to check and compensate for inconsistencies. For
example, a newly added tactic may affect an actor who had not
been considered thus far. Adding that actor may lead to recon-
sideration of the set of criteria, and this in turn may require
revision of the causal diagram. Therefore, after performing
the preceding steps, a consistency check is advised to verify
whether all relevant actors and factors have been included.
The result is portrayed as a ‘final’ system diagram, and is
accompanied by a verbal reformulation of the problem in
terms of both needs and choice dilemma’s. Figure 11 illus-
trates the diagram for our simplified LightOn example after
several iterations and consequential revisions and updates.
During the analysis process, possible extension of the set
of problem owners needs to be considered. We started from
the perspective of the client as the single problem owner. As
illustrated in the LightOn case the actor analysis may show
that some of the actors in the policy arena possess means that
are relevant to solving the problem, and that these actors may
be willing to apply those means in accord with the problem
owner. In such cases, such actors may be considered as co-
problem owners. Then, the analysis and the corresponding
diagrams should be adapted in accordance. Consequently the
factors under control of the added problem owners should be
added as steering factors at the left side of the system diagram,
and their objectives should be operationalized into criteria
to be added as necessary. In practical real-world situations,
choices like this should be discussed with and agreed with
the problem owner(s).
Question 7: What further steps, including possible research,
are recommended?
The final step in the policy issue paper is to identify a plan
of action to be recommended to the problem owner. Two
different types of actions may be distinguished: so-called
process actions, for example to start talking to or negotiating
with critical actors, and research actions, targeted at resolving
specific crucial knowledge gaps.
To identify relevant knowledge gaps, a ‘walk through’ of
the system diagram is done in order to assess what the analyst
 TE van der Lei et al—Using a systems diagram to analyse and structure complex problems 1399

Tech-
Fuel Total nological
prices electricity progress
demand renewables

Installed +
coal fired
systems Electricity + Spot
+ +
Shortage market
purchases
Installed
mCHP
systems + Total
Amount of Net
extra
+ + electricity income on
fossil delivered
+ capacity + + sold
Installed
+ electricity
gas fired
turbines -
Total + ?
capacity +

Installed + Total cost

solar (investment
systems + Total extra plus
sustainable operational) CO2
capacity - emissions
?
prevented
Installed +
wind +
turbines -
Amount
+ Amount of
of
sustainable
subsidy + capacity
Subsidy

Figure 11 The final system diagram for LightOn.

knows and does not know about the system. This is done by
following the impact of the policies through the factors in the
system on the criteria. Attention should be drawn to those
relationships between factors that are insufficiently known.
Typically there are unknowns about the causalities within the
system, the impacts of policies on the criteria, the influence
of other stakeholders, and how the problems may change over
time.
The knowledge gaps that have been identified are then
translated into one or more main research questions that can
be split into a number of detailed sub-questions. Preferably
this is done in consultation with the problem owner and the
critical actors. For each sub-question a research proposal is
formulated specifying, among other things, the method(s) to
be used (Van der Lei and Slinger, 2006). A timetable and
a budget estimate can also be incorporated for the potential
commissioner of the studies. Another important element is a
communication plan stating the form and level of cooperation
and involvement of problem owner and stakeholders and the
way reporting will be organised.
The analysis performed for LightOn has shown that invest-
ments in sustainable capacity seem to be important for the
image and long-run success of the company. Until now insight
into the specific costs per capacity type are lacking. As a
research question we pose: What are the investment costs per
type of sustainable capacity for LightOn? As the choice of
capacity is an investment problem a cost-benefit analysis is
recommended. In addition, given the significant uncertainties
regarding technology, fossil fuel prices, etc, we recommend to
perform a more thorough and complete uncertainty analysis.
Discussion
There is no unique way to analyse a complex policy problem.
We have taken system analytic thinking as a starting point,
emphasising the substantive aspects of a problem situation,
and the logic and internal consistency of the conceptual
model(s) developed. Answering the seven questions formu-
lated in Table 1 generates the insights in the problem that
need to be reported in the policy issue paper. Attractive about
this stepwise approach is the coherence of the models used
and produced. By using simple analytical modelling tech-
niques to construct the systems model its internal consistency
is guaranteed. By combining them with actor analysis and
uncertainty analysis an integrated model is build up, wherein
content and process aspects are connected. This iterative
1400 Journal of the Operational Research Society Vol. 62, No. 7
process of problem formulation and reformulation guided by
the questions leads to a balanced problem formulation and
delineation.
Political scientists, however, would generally rather focus
on the broader policy arena first (see Figure 9). Such an anal-
ysis of actors, their perceptions, intentions and behaviour,
may lead to quite different framings of problem situations,
for example, the absence of trust in a policy network may
be seen as a prime impediment of progress, or the problem
is framed as a struggle for power between different parties.
Such different approaches may lead to complementary and
relevant insights, and the nature of further action required to
reach a solution (if possible at all) will generally depend on
the nature of the situation, who the problem owner is, etc. For
further exploration of such different approaches, the reader is
referred to the literature (Bobrow and Dryzek, 1987; Dunn,
1994; Mayer et al, 2004).
Limiting ourselves to the system analytic approach outlined
above, we identify some of the recurring difficulties encoun-
tered in the period of more than 10 years in which we have
been working with and teaching the approach to several gener-
ations of students. Table 4 below lists some of the key issues
and choices that time and again appear to be difficult both for
us teachers (when developing examples) and students alike.
In the table, we also have added a number of tips and sugges-
tions for dealing with these issues. Note that some of the
Table 4 Typical problems and ways to tackle these
Steps or questions Problems and common mistakes
Problem level The problem is framed uniquely as a choice between
alternatives
It is difficult to choose an appropriate scope for the
primary objective(s). Sometimes, there is a tendency
to choose for very abstract high-level objectives like
‘increase of welfare of all inhabitants’ or ‘economic
growth’
Sometimes, too narrow a scope is chosen (jumping to
solutions)
Choice and The elaboration of objectives into an objectives
analysis of hierarchy and operational criteria is mixed up with a
objectives means-ends analysis exploring ways or means to
achieve the objectives
Identifying the It is tempting to extend the explorations of means to
means of the the elaboration of implementation measures
problem owner?
Too many tactics and strategies
issues listed relate to the way results from the analysis are
documented and reported in a policy issue paper.
We have developed this approach when faced with the need
to teach our students a systematic, analytic approach to tack-
ling complex policy problems, and found little ready-to-use
material in existing texts. Inspired by some of our own real-
world experiences, we set out to systematically combine some
of the parts of the puzzle we found in literature, and adapted
them to be applicable at the undergraduate level. Although
no systematic empirical testing in real-world contexts has
been done yet, we believe the approach to be of value to
practicing analysts when faced with complex, unstructured
situations. We see our students often successfully apply the
approach in their MSc thesis work which is mostly done
in a real-world context, and we regularly obtained positive
feedback of alumni about how this part of their education
has benefited them in structuring their work as practicing
analysts.
Concluding remarks
We have outlined how a conscious and step-wise combination
of relative simple and straightforward analytical methods can
be used to frame a complex policy problem in the form of a
systems diagram. Seven questions accompanied by relatively
straightforward analytical methods and models constitute the
Tips and tricks
Emphasise that a problem is a gap between desire or
wish and (expected) reality
Start thinking backwards from the chosen objective to
identify means; if the primary objective is too broad,
the spectrum of possible means and tactics becomes
overwhelming
Ask the ‘next general question’: Why does the problem
owner want to achieve the initial objective? Consider
taking the next higher level objective as starting point
instead as it may allow for a broader set of options
Emphasise the difference between a definition relation
(what) and a causal (or intentional) relation (how).
Use appealing examples. Use nouns for objectives and
verbs for means
Emphasise the need to focus on steering factors first,
that is, the key things or factors a problem owner can
influence to change the situation. Elaboration of
implementation measures can follow later, after
analysis has helped identify the most promising tactics
and strategies (see also Walker, 1988)
Limit attention to strategies that have a direct effect on
factors in the system; focus on the main types of actions
only, elaboration and fine tuning may follow later
 TE van der Lei et al—Using a systems diagram to analyse and structure complex problems 1401

Table 4 Continued

Steps or questions Problems and common mistakes Tips and tricks

Determining Sometimes it is unclear where the factors belong. Are
system boundaries they part of or outside the system?
and elaborating the
system diagram The question what the problem owner can influence
and what is under the influence of other actors is
sometimes hard to answer.
A large number of factors inside the system diagram
with a lot of causal relationships between them
What further Research is suggested on resolving the exogenous
research is uncertainties as these appear to be of crucial importance represent the most influential uncertainties; however,
recommended?
General Different problem definitions ‘float’ through the paper.
The argument ends with the same problem description
as it started, with no clear value added from the analysis
Suddenly new problems and new variables are
introduced that do not logically follow from the
analysis
Emphasise as a basic rule that factors only belong in
the system if they can be affected by means or tactics
under the control of the problem owner(s)
It’s a matter of judgment and choice; best is to make a
choice and be explicit about it. If other actors may be
influenced by the problem owner, an option is to
include them as part of the system and consider the
action to influence them as a tactic
Limit the number of factors to a manageable size
(about 15) to prevent ‘spaghetti diagrams’ and to keep
oversight and to support the self-explanatory character
of the diagram. An alternative is to identify
subsystems rather than factors
This may seem logical as exogenous factors often
often more research will not easily resolve these
uncertainties (eg regarding climate change, energy
prices, etc), so the advice should be to accept these
uncertainties and focus research on identifying those
options that best deal with the uncertain environment
Check for consistency in the storyline; make one sheet
describing the main line of argumentation
Either these issues should be mentioned as insights
from earlier analytical steps or as factors showing up in
previous models and diagrams or they should be left out

backbone of a sound problem analysis. The insights derived
are the input for the proposed policy issue paper we expect our
students to write. The approach focuses on the content dimen-
sion of policy problems, and relates this to the multi-actor
environment within which most policy issues are debated. By
iteration the approach leads to an internally consistent systems
model that represents the problem definition and delineation.
Scrutinising the factors and the character of their relations
leads to the identification of knowledge gaps requiring further
scientific, analytical or political research.
The interrelation between the various analytical steps
and the subsequent iterations force the policy analyst to
continuously re-evaluate the problem definition and problem
delineation. Guarding consistency—checking whether the
criteria on the right side of the systems diagram are indeed
the same as the operationalised lowest level objectives in
the objectives hierarchy, whether the steering factors match
the means, to what extent causal mechanisms relating the
means and the criteria are made explicit, and to what extent
the actors related to the various factors and tactics have
been identified—helps the analysts to construct an internally
consistent and convincing formulation of the most relevant
problem situation elements, and their relations. This then
provides a basis for discussion with the problem owner(s)
and for planning further steps.
The systems perspective illustrated in our integrated
problem structuring approach is expected to lead to clear
and sound insight in the character and complexity of prob-
lems; to better problem descriptions and consequently better
proposals for follow-up.
References
Ackoff RL (1978). The Art of Problem Solving. Wiley: New York.
Axelrod R (1976). Structure of Decision: The Cognitive Maps of
Political Elites. Princeton University Press: Princeton, NJ.
Bobrow DB and Dryzek JS (1987). Policy Analysis by Design.
Pittsburgh University Press: Pittsburgh.
Bryson JM (2004). What to do when stakeholders matter. Stakeholder
identification and analysis techniques. Public Mngt Rev 6(1):
21–53.
Bryson JM, Ackerman F, Eden C and Finn CB (2004). Visible
Thinking. Unlocking Causal Mapping for Practical Business
Results. John Wiley & Sons: Chichester.
Checkland PB (1985). Formulating problems for systems analysis.
In: Miser HJ and Quade ES (eds). Handbook of Systems Analysis,
Overview of Uses, Procedures, Applications, and Practice.
North-Holland: New York, pp 151–170.
Checkland P and Poulter J (2006). Learning for Action: A Short
Definitive Account of Soft Systems Methodology and Its Use, for
Practitioners, Teachers and Students. Wiley: Chichester.
1402 Journal of the Operational Research Society Vol. 62, No. 7
de Bruijn H and Porter AL (2004). The education of a technology
policy analyst to process management. Technol Anal Strateg Mngt
16(2): 261–274.
de Bruijn H, ten Heuvelhof EF and in’t Veld R (2002). Process
Management: Why Project Management Fails in Complex Decision
Making Processes. Kluwer Academic Publishers: Dordrecht,
the Netherlands.
Dunn WN (1994). Public Policy Analysis: An Introduction.
Prentice-Hall: Englewood Cliffs, NJ.
Eden C (1988). Cognitive mapping. Eur J Opl Res 36: 1–13.
Eden C (2004). Analyzing cognitive maps to help structure issues or
problems. Eur J Opl Res 159: 673–686.
Eden C and Ackermann F (1998). Making Strategy: The Journey of
Strategic Management. Sage: London.
Eden C and Ackermann F (2006). Where next for problem structuring
methods. J Opl Res Soc 57: 766–768.
Enserink B (2000a). A quick scan for infrastructure planning:
Screening alternatives through interactive stakeholder analysis.
Impact Asses and Proj Appraisal 18(1): 15–22.
Enserink B (2000b). Building Scenarios for the University. Int T Opl
Res 7(6): 569–584.
Enserink B (2004). Thinking the unthinkable—The end of the Dutch
river dike system? Exploring a new safety concept for the river
management. J Risk Res 7(7–8): 745–758.
Goodell J (2001). How coal got its glow back. New York
Times, http://www.blackmesais.org/How Coal Got.html, 22 July,
accessed July 2007.
Harrison J (2004). Micro combined heat & power (CHP)
for housing. SET 2004—3rd International Conference on
Sustainable Energy Technologies. Nottingham. UK, 28–30 June,
http://www.microchp.dk/Jeremy%20Harrison%20micro%20CHP%
20SET%202004.pdf, accessed July 2007.
Hoogerwerf A (red.) (1988). Het ontwerpen van beleid. Een
handleiding voor de praktijk en resultaten van onderzoek, Samsom
H.D. Tjeenk Willink, Alphen aan den Rijn.
Keeney RL (1988). Structuring objectives for problems of public
interest. Opns Res 36: 396–405.
Keeney R (1992). Value-focused Thinking. A Path to Creative Decision
Making. Harvard University Press: Cambridge, MA.
Keeney RL and Gregory RS (2005). Selecting attributes to measure
the achievement of objectives. Oper Res 53(1): 1–11.
Koppenjan J and Klijn EH (2004). Managing Uncertainties in
Networks. Routledge: London, New York.
Mayer IS, van Daalen C and Bots PWG (2004). Perspectives on
policy analyses: A framework for understanding and design. Int J
Technology, Policy Mngt 4: 169–191.
Miser HJ and Quade ES (1988). Handbook of Systems Analysis, Craft
Issues and Procedural Choices. John Wiley & Sons: Chichester,
New York, Brisbane, Toronto, Singapore.
Porter AL, Roper T, Mason T, Rossini F and Banks J (1991).
Forecasting and Management of Technology. Wiley & Sons:
New York.
Quade ES (1980). Pitfalls in Formulation and Modelling. In:
Majone G and Quade ES (eds). Pitfalls of Analysis. John Wiley &
Sons: Chichester.
Quade ES (1982). Analysis for Public Decisions, 2nd edn. North-
Holland: New York.
Quade ES (1989). Preparing an issue-paper. In: Analysis for Public
Decisions. North Holland: New York, pp 73–78.
Rittel WJ and Webber MM (1973). Dilemmas in general theory of
planning. Policy Sci 4: 155–169.
View publication stats
Rosenhead J (1989). Robustness analysis: Keeping your options open.
In: Rosenhead J (ed). Rational Analysis for a Problematic World.
John Wiley & Sons Ltd: Chichester, West Sussex, England.
Rosenhead J and Mingers J (eds) (2001). Rational Analysis in a
Problematic World Revisited. Wiley: Chichester.
Sage AP (1992). Systems Engineering. IEEE Wiley: New York.
Sage AP and Olson SR (2001). Modeling and simulation in systems
engineering. Simulation 76(2): 90–91.
Schwartz P (1991). The Art of the Long View: Planning for the Future
in an Uncertain World. Doubleday: New York.
Schwartz P (1992). Composing a plot for your scenario. Planning
Review 20(3): 4–9.
Shaw D, Franco A and Westcombe M (2006). Problem structuring
methods: New directions in a problematic world. J Oper Res Soc
57: 757–758.
Simon HA (1952). On the definition of the causal relation. J Philos
49: 517–528.
Smaardijk EJ, Fliert EM, Ledeboer A and Jak MJG (2005). Disperse
energie—Liberale Markt, ALTRAN in opdracht van Provincie
Zeeland en Delta N.V.
Sterman JD (2000). Business Dynamics. Systems Thinking and
Modeling for a Complex World. McGraw-Hill Higher Education:
Boston.
Thissen WAH (1998). A scenario approach for identification of
research topics. In: Weijnen MPC and Heuvelhof EP ten (eds).
The Infrastructure Playing Field in 2030. Proceedings of the First
Annual Symposium Delft Interfaculty Research Center Design and
Management of Infrastructures, November, Delft University Press;
Noordwiik, pp 5–10.
Thissen WAH (2000a). Systems engineering education for public
policy. Int J Technol Mngt 19(3–5): 408–419.
Thissen WAH (2000b). Issue formulation in a multi-actor context:
A five-step approach. In: Kawamura K (ed). Proceedings 2000
IEEE International Conference on Systems, Man, and Cybernetics,
Vol. 1, Nashville, TN, USA, pp 301–306.
Van der Lei T and Slinger JH (2006). The system diagram as a
diagnostic aid for quantitative method choice. In: Antunes CH and
Gomes A (eds). CDRom Proceedings, ORMMES 2006 Operational
Research Models and Methods in the Energy Sector, 6–8 September
2006, Institute of Systems Engineering and Computers of Coimbra
and Association of European Operational Research Societies;
Coimbra, Portugal.
von Reibnitz U (1998). Scenario Techniques. McGraw-Hill: New York.
van Walle E, director-general of the Belgian Nuclear Research
Centre SCK-CEN (2006). Speech on the Strategic Forum on
Energy Supply in the 21st century, http://www.leonardo-energy
.org/drupal/node/1530, Brussels, 18 December 2006, accessed
March 2010.
Walker W (1988). Generating and screening alternatives. In: Miser HJ
and Quade ES (eds). Handbook of Systems Analysis. John Wiley
& Sons: Chichester, New York, pp 217–234.
Walker W (2000). Uncertainty: The Challenge for Policy Analysis
in the 21st Century. TU Delft, Faculteit Techniek, Bestuur en
Management: Delft.
Walker WE, Harremoës P, Rotmans J, van der Sluijs JP, van Asselt
MBA, Janssen P and Krayer von Krauss MP (2003). Defining
uncertainty: a conceptual basis for uncertainty management in
model-based decision support. Integr Assess 4(1): 5–17.
Received November 2008;
accepted January 2010 after two revisions