Building Research & Information

ISSN: 0961-3218 (Print) 1466-4321 (Online) Journal homepage: http://www.tandfonline.com/loi/rbri20

Scientific models versus social reality

Nancy Cartwright

To cite this article: Nancy Cartwright (2016) Scientific models versus social reality, Building
Research & Information, 44:3, 334-337, DOI: 10.1080/09613218.2015.1083811

To link to this article: http://dx.doi.org/10.1080/09613218.2015.1083811

Published online: 12 Sep 2015.

Submit your article to this journal

Article views: 465

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=rbri20

Download by: [185.100.85.138] Date: 31 March 2016, At: 14:21

 B UILDING R ESEARCH & I NFORMATION 2016
Vol. 44, No. 3, 334– 337, http://dx.doi.org/10.1080/09613218.2015.1083811

COMMENTARY

Scienti¢c models versus social reality

Nancy Cartwright1,2

1
Department of Philosophy, University of Durham, 50 Old Elvet, Durham DH13HN, UK.
E-mail: nancy.cartwright@durham.ac.uk

2
Department of Philosophy, University of California ^ San Diego, La Jolla, CA 92093-0119, US.
E-mail: nancy.cartwright@durham.ac.uk
Downloaded by [185.100.85.138] at 14:21 31 March 2016

Policy predictions fail for the very many different kinds of case-by-case local factors described in the Building Research &
Information (2015) special issue (vol. 43/4) entitled ‘Closing the Policy Gaps: From Formulation to Outcomes’. Work in
philosophy of science shows that beyond the case by case, general systematic problems loom that make the gap between
theory and practice hard to close. What is needed in response, it is argued here, are ways to cope with the gap and to build
an expectation about it into planning predictions, into planning decisions, into the methods of implementing and
monitoring, as well as into fallback and failsafe plans. Tracking implementation and outcomes is not only useful for
post hoc evaluation but also a powerful tool for getting the intended outcomes in the first place and making the
necessary adjustments.

Keywords: built environment, evidence-based policy, governance, policy design, policy formulation, policy
implementation, public policy

Much of the rhetoric in favour of evidence-based
policy and practice seems to subscribe to the idea: if it
comes from science, it is going to be right. In the Building
Research & Information special issue ‘Closing the Policy
Gaps: From Formulation to Outcomes’ (Table 1),
repeated failures of this idea can be observed, illustrating
the gap between theory and practice. Since the special
issue concentrates on the specific policy area of the built
environment, this might be thought of as a problem that
besets this area especially violently. But this kind of gap
is the rule, not the exception: it is just what should be
expected for social policy. The papers present a variety
of concrete ways, different for different cases, in which
final outcomes do not fit theoretical expectations. Work
in philosophy of science points up a number of reasons
why this is almost bound to occur. Predictions fail for
the very many different kinds of case-by-case local
factors described in the papers collected here; but
beyond the case by case there are general systematic
problems looming.
It is important to recognize the systematic sources of
trouble, otherwise there is an across-the-board likeli-
hood of overbidding the claims that are made. Scienti-
fic evidence can without doubt improve the reliability
of policy predictions. But too frequently the scientific
base is assumed to offer the promise of far more cer-
tainty than it can deliver. There is danger here. If an
unjustified degree of optimism exists about how effec-
tive policies can be, then there is a likelihood of getting
policy deliberation wrong in the delicate balance
between considerations of effectiveness, on the one
hand, and considerations of cost and legitimacy, on
the other. The policy community often expects results
that cannot be achieved and fails to anticipate side
effects that can be very harmful. Both these lead to
wasted money and effort and to heartbreak and
dashed hopes. The wrong projects get chosen. For the
ones that are chosen, there is failure to establish the
monitoring and feedback systems that provide early
warning that things are going wrong. This would
allow for things to be set right or for the project to
be abandoned before too much money and effort has
been expended. It is unrealistic to expect to close the
gap , but it is feasible to narrow it. What is needed
are ways to cope with a gap, to build an expectation
about it into our predictions, into our decisions, into
our methods of implementing and monitoring and
into our fallback and failsafe plans.
The across-the-board problems arise from the relation
between the demands of exact science and the nature of

# 2015 Taylor & Francis


Table 1 Authors and titles of articles in the special issue ‘Closing the Policy Gaps: From Formulation to Outcomes’, Building
Research & Information (2015), vol. 43(4); guest editors: Simon Foxell and Ian Cooper
S. Foxell and I.Cooper
R. Simmons
L. Schweber,T. Lees and J.Torriti
R.Gupta, M.Gregg, S. Passmore and G. Stevens
A.M. Moncaster and P. Simmons
G. Ive, A. Murray and R. Marsh
E.Warwick
G.R.W. Mills, M. Phiri, J. Erskine and A.D.F. Price
K.B. Janda and M.Toupouzi
R. Cohen and B. Bordass
Downloaded by [185.100.85.138] at 14:21 31 March 2016
the blousy world around us that the science is supposed
to help manage. In general, though scientific principles
can be hugely successful in helping to deal with the
world, they do not describe what actually happens.
They describe what happens in models and very often
the model does not sufficiently resemble the world it
is supposed to model. As Janda and Topouzi remark,
‘Idealizations about people and their energy beha-
viours are familiar, but they are not necessarily accu-
rate or true’ (p. 528). There are several perfectly
general reasons for this that need to be borne in
mind. First, even if the principles applied in the
model are flawless, the outcomes that hold in the
model hold in the world only ceteris paribus, i.e. the
outcomes hold only so long as the model takes into
account all the dominant causes of those outcomes
that are at work in the world and none that are not.
This is a tall order, especially in areas of social policy
where social and political causes interact with techno-
logical ones. Researchers and policy-makers usually
have a very loose grip on what all these social causes
are and how they operate, let alone on how the mix
of causes will interact together, as observed throughout
this special issue.
The striking experimental and technological successes
of some of the best basic science are a witness, rather
than a counterexample, to the ceteris paribus nature
of these principles. Testing a scientific theory depends
upon devising (or finding naturally occurring) settings
in which all the factors relevant to the outcome are
describable by the theory, as well as knowing how to
describe these factors. Otherwise, how would it be
possible to know what the theory predicts should
happen in order to compare that with what actually
does happen? This is important in order to ascertain
if the predictions of the theory are borne out. These
are generally highly artificial settings, or highly
managed ones, so-called ‘closed systems’, that are
‘overbuilt’ or otherwise protected against the intrusion
Scienti¢c models versus social reality
Editorial: Closing the policy gaps
Constraints on evidence-based policy: insights from government practices
Framing evidence: policy design for the zero-carbon home
Intent and outcomes from the Retro¢t for the Future programme: key lessons
Policies and outcomes for UK sustainable schools
Cost and value ratios of operating renewed English secondary schools
Policy to reality: evaluation the evidence trajectory for English eco-towns
Rethinking healthcare building design quality: an evidence-based strategy
Telling tales: using stories to remake energy policy
Mandating transparency about building energy performance in use
of causes that distort the outcomes or make them
unpredictable. Most people have an implicit under-
standing of this. For example, they know that dropping
mobile phones onto a tile floor is detrimental, or not to
use galvanized pipes for plumbing near the sea due to
the potential for corrosion. This lack of control
looms large when it comes to predicting the effects of
planning policy. As Trygve Haavelmo, a Nobel Prize
winner for his work in founding econometrics,
explained to me in conversation about predictability:
Physics has it easy. No-one asks a physicist to
predict the course of an avalanche but we econ-
omists are regularly expected to predict the
course of the economy.
Second, the social and political influences that affect
planning outcomes are often at the tail end of a long
chain of events stretching backwards in time, or the
outcomes are at the end of long chains of complicated
social, political and technological causes stretching
into the future. As Mills, Phiri, Erskine and Price
report about one of the studies they describe:
Perhaps unexpected was the complexity of pro-
blems in organizational and collaborative terms.
(p. 509)
But if John Stuart Mill is to be believed, this complexity
should not be unexpected. It is just the shifting and
unpredictable arrangement of the causes that enter
these chains – the causes of the causes of the causes
– that led Mill to argue that political economy must
be a deductive not an inductive science. Even if we
have observed some relatively stable relations
between one set of factors and another in the past,
those relations cannot be relied on to hold in future
because the background of all the many causes
omitted from consideration is not likely to stay fixed.
As Ive, Murray and Marsh note, following Keynes’
335
 Cartwright
(1936) quote about the yield of an investment: ‘Our
knowledge of the factors which will govern the [ . . .
outcome] some years hence is usually very slight and
often negligible’ (p. 471).
A third reason has to do with the kinds of concepts that
good science demands. These must not be vague or
ambiguous, they must be well defined, precise and
unequivocal. But for policy planning it must also be
possible to use them to make predictions. That means
that they must be linked together by principles that
allow for inferences to be made from one to another.
That too is a tall order. It is usually met by an extended
back and forth process of mutual adjustment. Max
Weber noted this process in describing the differences
between the natural sciences and the study of society.
He too, like Haavelmo, thought that physics has it
easy. It can keep adjusting its concepts until it finds
ones that fit into neat principles. But this approach
Downloaded by [185.100.85.138] at 14:21 31 March 2016
cannot be applied when studying society. When it
comes to society there are fixed topics of enquiry (the
things society and its members want to know about
which nowadays includes poverty, social exclusion,
responses to some nudge policy, etc.) and there is
good evidence that these are not governed by the
kind of tight reliable principles found in science.
In the face of all the difficulties for predicting the out-
comes of policies for the built environment recorded
here, Foxell and Cooper (following McGrayne1) place
some hope in Bayes’ theorem to provide ‘the opportu-
nity to use knowledge of past events to produce a prob-
ability rating for a hypothesis or a policy proposition’
(p. 404). I am far less sanguine about this than they,
for just the same general reasons about science,
models and real life mentioned above. Bayes’ theorem
allows for use of real-life data and can treat real-life
outcomes, but it can only do so via the mediation of
an abstract model, though familiarity and practice
with statistical inference may make the modelling
steps in between almost invisible. The model requires
a well-defined event space satisfying various formal
conditions plus a precise measure over them that
satisfies the probability axioms. To be of any practical
use, the connections used for prediction among the
events in the event space (e.g. which are modelled as
probabilistically dependent? which independent? con-
ditional or unconditional independence? conditional
on what?) must reflect real relations among the happen-
ings in the world that the events in the event space are
meant to represent. So all the same problems arise
about how the concepts that describe the events in a
model map onto things that are actually happening in
the world. This is exacerbated by the fact that the prob-
abilistic relations in the model are even further removed
from the world than ordinary scientific models since the
probabilistic relations generally are warranted by some
theoretical model-dependent assumptions about the
relations (e.g. causal relations) among the abstractly
336
defined events in the event space; then those theoretical
assumptions need to be fitted on to the (at best rough)
regularities found in the real world outside the labora-
tory and other specially controlled environments.
Thus I agree with the doubts that Janda and Topouzi
explicitly expressed about faith that:
building scientists can gather (all) the evidence,
understand it and properly communicate it to
policy-makers, who will then develop effective
policies based on this new understanding.
(p. 516)
There are good reasons coming not only from the
experiences of failure recorded in this special issue
but also from the very nature of the scientific enterprise
itself for their concerns. As they suggest, it is very
doubtful that:
accurate information alone will lead to better
future planning in the complex system of the
built environment.
(p. 516)
What can lead to better future planning for the built
environment? Better tracking is part of the answer.
One of the questions in the original call for papers
for this special issue is ‘How might the implementation
of a policy or initiative be tracked and evaluated
against its intended objectives?’. In their editorial,
Foxell and Cooper remark:
Most [papers] also contain overt (or at least deci-
pherable) signposting to how policies and initiat-
ives could be tracked and evaluated against their
intended objectives and outcomes.
(p. 399)
Tracking is often done primarily for the sake of
evaluation, with which it is linked in this remark:
knowing that the right stages between policy and
outcome occurred provides good evidence that the
policy was indeed responsible for producing that
outcome. But, as Eileen Munro argues in the field
of child protection, tracking is not only useful for
post-hoc evaluation, it is a powerful tool for getting
the intended outcomes in the first place. Tracking
can provide alerts to problems that a policy encoun-
ters as events unfold. It provides the opportunity to
make corrections before it is too late or to abandon
the endeavour early when there seems no way to
set it operating as it should.
Second, better evaluation itself can be of help, especially
when tracking is involved to uncover not only whether
the policy worked but also how it worked. Understand-
ing why is the key to better planning in future, to figur-
ing out what policies will work where and when and

what needs to be in place, or be put in place, to make
their success more likely. Without this understanding,
mere evaluation by itself – learning that a policy
worked in this place or in this handful of places – can
be very misleading. It can lead to ‘induction by simple
enumeration’, which is a very poor form of inference
indeed. For example: ‘swan 1 is white, swan 2 is white
. . . so all swans are white’ or ‘all the coins in my
pocket are copper, so all coins are copper’.
More effective interdisciplinary cooperation in the
planning and implementation of policy is a third impor-
tant tool for getting better outcomes. As already noted,
plans fail for a mix of natural and social causes, some
naturally occurring, some artificially, even sometimes
induced by the process itself in a looping effect. The
hard science matters but equally does the knowledge
of the social processes, like the political agenda of
various players highlighted in Warwick’s and in Mon-
Downloaded by [185.100.85.138] at 14:21 31 March 2016
caster and Simmons’ contributions, the structure of
regulation and in particular its enforcement mechan-
isms, as made clear in Cohen and Bordass’s paper, the
kinds of subjective forces that undermine objectivity
described in Simmons’ and in Schweber, Lees and Tor-
riti’s contributions and the effects of complex inter-
actions among different agencies and the need for
detailed understanding of local situations described in
Mills, Phiri, Erskine and Price’s article. The list con-
tinues. It is long and varied. What will make a differ-
ence will not be the same from one context to
another, which makes it difficult to offer advice in
advance about whose expertise to include, when and
how. The important lesson is to be alert to the need
to include interdisciplinary expertise and local knowl-
edge from the very start of the policy planning
process in almost every case.
Fourth, better outcomes will be achieved overall if
policy-makers, strategists and the public have more
Scienti¢c models versus social reality
realistic expectations about what can be achieved and
what are the chances of success. The discussion
above about the theoretical problems with expecting
a high degree of certainty coupled with the huge
array of concrete problems displayed in the papers in
this special issue argue that, despite good use of good
science in policy construction and deliberation,
smooth sailing will not be the rule. Instead, it should
be expected that problems will arise throughout the
entire process, from the start of implementation to
the final outcomes. Given the widespread predictable
failure of models to be as robust as we would like, it
is essential both to hedge our bets and to allow for,
indeed to plan for, the need to correct, to redesign or
to abort our projects. As several authors emphasize,
it is important to keep firmly in mind that (in the
words of Foxell and Cooper):
policy-making is a live activity, “a continuous
process” [ . . . ] needing constant adjustments
and periodic refocusing to keep it on track.
(p. 401)
Disclosure statement
No potential conflict of interest was reported by the author.
Reference
McGrayne, S. B. (2011). The theory that would not die: How
Bayes’ rule cracked the enigma code, hunted down Russian
submarines & emerged triumphant from two centuries of
controversy. New Haven, CT, and London: Yale University
Press.
Endnote
1
McGrayne (2011) makes a case for Bayes’ theorem, but its use
remains controversial and contested.
337