Evidence-based policy-making: getting more from a shrinking envelope

The phrase "getting more from a shrinking envelope", whilst committing several heinous crimes
against mixed metaphors, does at least give a sense of the need to ensure the efficiency and
efficacy of our public services during a time of spending cuts sorry, 0% increases. The phrase was
quoted recently by a consultant who works closely with a number of government departments;
the importance of basing public spending on a secure evidential base was reiterated yesterday in an
article and editorial in The Times.

Professor Jonathan Shepherd of Cardiff University, called, in this article

published yesterday, for a greater use of scientific evidence in the
development and evaluation of public service activity, to bring services
such as policing and teaching closer to the health service in their
approach. He comments that public services need to develop a
research culture similar to that in medicine, where treatments must be
tested for effectiveness and value before they are adopted, but that such
an approach is lacking elsewhere. For those of us who work to support
the use of evidence in improving policy-making, it is always good to hear
high-profile supporters speaking out about this issue.

However, his article raises a number of issues. As the editorial in The

Times responds, a balance needs to be struck between basing activity on
evidence of its success, and the introduction of new and genuinely
innovative policies. Where a policy breaks new ground, it necessarily is
difficult or impossible to evidence its likely success. It's important to
make sure that innovation is not stifled by the demand for evidence-
based policy-making.

A second issue relates to the evidence itself; as a surgeon and criminologist, The Times interprets
Professor Shepherd's call to be for greater use of scientific evidence. Perhaps use of the adjective
'scientific' is designed to reassure the newspaper's readership of the academic rigour on which
policy should be based; it isn't a term which is used by Professor Shepherd himself in the quotes
given in the article. But the issue of what constitutes evidence, particularly outside the health
sector to which he refers, is an important one. Many of the public policies which benefit from
review in the light of evidence, rely on evidence which is not simply 'scientific', quantitative, or
easily gathered, analysed and interpreted. It is always important to ensure that public policy
decisions aren't based solely on the "we can measure this, so it must be important" scale; the
more complex the problem, the more important evidence of different types becomes. Regional
observatories often provide a useful resource for regional policy-makers in this area, introducing
'softer', qualitative or anecdotal evidence to provide a fuller picture whilst at the same time
bearing in mind the limitations of such evidence.

Professor Shepherd uses the health sector as a comparator for policing and teaching, but another
issue arises in terms of the timescales available against which interventions can be judged. The
ideological and political pressures around policing and teaching are arguably often greater than
those around health (political parties being happy to draft policy about teaching methods, for
example, whilst steering clear of writing policy around specific health interventions, like the
article's example of wisdom tooth removal). Policies around policing and teaching are often
developed more quickly, implemented, then reviewed and changed on a much shorter timescale
than in the health sector. This limits the evidence available, and the time that politicians and civil
servants are happy to let elapse before introducing a new policy. Needless to say, the
introduction or amendment of policy may sometimes be based on political expediency, rather than
solid evidence.

My final issue with the article comes from the suggestion made by Professor Shepherd that more
should be done to build links between universities undertaking research, and practitioners and
policy-makers. Having worked closely in the past with a Research Council, and senior academics,
there is often a mismatch between academia and the public sector: the rapid evidence review
versus the painstakingly researched, peer-reviewed paper; the breadth of view versus the
dedicated specialisation. Both are important, and both have their place, but reconciling those
positions is not without its challenges. That's not to say Professor Shepherd's suggestion isn't
correct; I do agree with him. But it is important to recognise that cultural issues will exist, and it
is not as simple as saying 'this is a good idea'. And I believe it is a bit unfortunate that he was
quoted as saying "It’s time for universities to invade the criminal justice and education systems";
while he qualifies this by saying that it is important to ensure that practitioners get the time, space
and support to research new approaches, it can't help but sounding like "universities know best -
let us show you". In common with at least one of those moved to comment on his article, we
need evolution, not revolution, in public services during a spending squeeze; and of course the
best way to support that is to support researchers, policy-makers and practitioners on the ground
to work together, and with academics, to improve public services, and get more from that
shrinking envelope.

Nicola Underdown is writing in a personal capacity; the views expressed do not necessarily represent those
of the Association of Regional Observatories.