This action might not be possible to undo. Are you sure you want to continue?
http://ppa.sagepub.com/ Evidence Based Policy Making: The View From A Centre
Andrew Wyatt Public Policy and Administration 2002 17: 12 DOI: 10.1177/095207670201700302 The online version of this article can be found at: http://ppa.sagepub.com/content/17/3/12
On behalf of:
Public Administration Committee
Additional services and information for Public Policy and Administration can be found at: Email Alerts: http://ppa.sagepub.com/cgi/alerts Subscriptions: http://ppa.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
Downloaded from ppa.sagepub.com at Oxford University Libraries on June 16, 2011
Introduction A more predictable sub-title for this article might have been 'A View from the Centre'. The subtitle as it stands is meant to signal a much more modest project. That there remains a range of views about what legitimately constitutes evidence in this context need not stand in the way of practical improvements in the ways in which policy advisers approach their task.com at Oxford University Libraries on June 16. information. which had been founded in 1970 and since 1989 12 Public Policy and Administration Volume 17 No. it attempts to establish what the rise to prominence of this terminology has meant in practice for officials. It incorporated the Civil Service College. 3 Autumn 2002 Downloaded from ppa. and to establish how it has come to be used as if it denoted a definable method or body ofprofessional practice. The article discusses both the antecedents of this concern. The aim of this article is to set out some thoughts on recent developments in thinking about what constitutes good policy making. is an important starting point.sagepub. It would. Secondly. and in particular the role of 'evidence' in that activity. The need for better use of evidence. research and analysis of all kinds in the development and implementation ofpolicy is widely understood. CMPS was established within the Cabinet Office in 1999. which stresses the needfor improvement in the processes ofpolicy making and the role that should be played by evidence in them. the Centre for Management and Policy Studies (CMPS). and the ways in which it has been elaborated in a range of subsequent reports. have been presumptuous to attempt to offer even a personal perspective from 'the centre' of UK government. 2011 . and in particular the role of 'evidence' in that activity. Modemising Government.Evidence Based Policy Making: The View From A Centre Andrew Wyatt Centre for Management and Policy Studies Abstract This article discusses recent developments in thinking about what constitutes good policy making. however. as observed from the author's own organisation. it seeks first to trace the sources in both academic and official publications of this term. Recognising the current emphasis on evidence based policy-making.
7). think and connect. 2011 13 .gov. 3 Autumn 2002 Downloaded from ppa. in the following terms: Located at the heart of government. And as the second quotation above indicates. Origins It is. (CMPS. what does it mean .sagepub. where does the notion of evidence based policy (or policy making) come from . It is reasonably certain that any civil servant across the United Kingdom whose work impinges on policy making in any sense will be familiar with the phrases 'evidence based' or possibly 'evidence informed' policy.of the current emphasis on evidence based policy making. whose policies and actions are based on sound evidence. and as like many such source documents in public administration reform it is probably more frequently cited than read it may be instructive to Public Policy and Administration Volume 17 No. The organisation further described its role. 1999a. the Modernising Government White Paper of March 1999 proclaimed that as part of the Government's commitment to improve policy making CMPS would 'identify and spread best practice' (Cabinet Office. from this point of view . This article seeks to explore the implications. coherent thought and a firm customer focus. more than ever. Moreover. the notion of 'evidence' plays a prominent part in its approach to this task. but also took on a significantly broader remit.cmps. two questions need to be addressed.why do we so frequently tell ourselves that this is what we should now be doing? And secondly. A new emphasis on or awareness of issues surrounding the uses of evidence in policy making. It is therefore. p. on its website home page at http:llwww.what is it that we think we should be doing? Let us begin by attempting to trace the term back to its source and examining how it came into common currency. Given this uncertainty about the interpretation and implications of the terminology. in fact. 2002. Our aim is to create a world class Civil Service. What is less clear is what they will take these to mean.com at Oxford University Libraries on June 16. p.uk ('About Us'). we seek to capture and absorb the best research evidence and management practice wherever they may be found. an integral part of the central machinery that is endeavouring to push forward the Prime Minister's agenda for change and reform in the public services. certainly .but to speak of an 'initiative' or a 'movement' would sound a little too formal or structured to describe what has been happening over the last couple of years.from a centre. difficult to know quite what label to attach to this phenomenon. The immediate origin of the phenomenon is often taken to be the Modernising Government White Paper itself.6). CMPS is now part of the Corporate Development Group that was formed in the Cabinet Office as a result of further organisational changes during the summer of 2002. or what changes they will have made in their working practices as a result of this awareness. open to new ideas. if not the centre . First. The mission of CMPS was stated as being to work with other units in the Cabinet Office and Departments to help realise the vision of improved service delivery. We work to realise this vision by seeking to change the ways in people work.had been operating as an executive agency.
inclusive and fair) and integration (policies and programmes will tackle the issues facing society in a joined up way. costs and the cumulative burden on business. * the keystones of the government's modernisation strategy will be inclusiveness (policies will be forward looking. * in turn. * information age government: government will use new technology to meet the needs of citizens and business.10). not the convenience of service providers. managing and communicating risks.17). The chain of argument set out in the White Paper is summarised in Figure 1 below. * avoiding imposing unnecessary burdens . programmes and services that will make them more healthy. * learning from experience . and performs all the other functions of a modern government (p. * improving the way risk is managed .decisions will be based on a careful appraisal of benefits. government should lead a debate on improving them. pilot schemes.6). * valuing public service: government will value public service not denigrate it. the commitment to better policy making will mean developing a new and more creative approach. * making sure policies are inclusive .regarding policy making as a continuous learning process. * modernisation is a long-term programme of improvement (p. * this is centred on five key commitments: * policy making: government will be forward looking in developing policies to deliver results that matter. delivers services to citizens and businesses.a focus on outcomes will encourage departments to work together where necessary.and outward-looking . 2011 . Figure 1: The Modernising Government White Paper * the purpose of government is to improve the quality of people's lives by delivering policies. improving contingency planning. evaluation and feedback (p. more secure and better equipped to tackle the challenges they face (p. integrating the EU and international dimension into policy making. 14 Public Policy and Administration Volume 17 No.com at Oxford University Libraries on June 16.rehearse just what it says on the subject. 3 Autumn 2002 Downloaded from ppa.government needs consistently to follow good practice in assessing. * becoming more forward. * involving others in policy making . * quality public services: government will deliver efficient. not simply reacting to short term pressures.policies will be fair and take full account of the needs of all those likely to be affected. regardless of organisational structures) (p.13). * responsive public services: government will deliver public services to meet the needs of citizens. * to achieve that ambition will require modernisation of the way in which government devises policies and programmes. with improved use of evidence and research.rather than defending policies. and will modernise the civil service (p.9).looking beyond what government is doing now.sagepub. and not trail behind technological developments. based on seven key principles: * designing policy around shared goals and carefully defined results . high quality public services and will not tolerate mediocrity.9). learning lessons from other countries.
in general too little effort has gone into making sure that policies are devised and delivered in a consistent and effective way across institutional boundaries. high quality public services. over the past 20 years. As the White Paper itself acknowledges.Much of the White Paper can be seen as a continuation and development of the programmes of reform and change that had been implemented in the UK through the 1980s and 1990s. mark a real and radical departure from most of what had gone before is in the prominence it gives to the need for improvement in policy making: Policy making is the process by which governments translate their political vision into programmes and actions to deliver 'outcomes' . identifying weaknesses in what had been done previously and establishing its own priorities. demonstrates considerable continuity with what had gone before. p. 1988).. while the focus of attention was on management reforms little attention was paid to the policy process and the way it affects government's ability to meet the needs of the people. implemented a series of reforms in the work of government. p. Government cannot succeed in delivering the outcomes people want if the policies and programmes they are implementing are flawed or inadequate (Cabinet Office. the broad thrust of what is said about the need for responsive..all of which we are determined to build on (Cabinet Office.. Where this paper does. a line of development in thinking about public service reform can be traced in various reports and Command Papers.sagepub. Like some other countries the United Kingdom has. 1982). proposals and statements of intent with actual implementation. The Citizen's Charter (Cabinet Office. 3 Autumn 2002 Downloaded from ppa.. customer-focused.com at Oxford University Libraries on June 16. efficient and effective. and relating the choice between Public Policy and Administration Volume 17 No.desired changes in the real world. As long ago as 1970 a White Paper was discussing the need to improve the quality of policy formulation and decision-taking by presenting Ministers. Continuity and Change (Cabinet Office.. through the inception of the Next Steps programme in Improving Management in Government (Efficiency Unit. 15).. Of course the new government that came to power in 1997 wished to assert its own agenda for change.. 1999a. This new concern with the quality of policy work did of course have some precursors. costed where possible.. ICT-oriented. with local freedom to manage and innovate wherever possible. 1996) to Modernising Government.. 2011 15 . and targets and measures focused on end results rather than inputs. however. 1994) and the Green Paper Government.direct (CITU. Nevertheless. It runs from the announcement of the Financial Management Initiative in Efficiency and Effectiveness in the Civil Service (Prime Minister and Minister for the Civil Service. very many new initiatives are described in the White Paper. However. collectively in Cabinet and individually within their departments. This emphasis on management reforms has brought improved productivity. better value for money and in many cases better quality services . The main focus has been on improving value for money in service delivery. with well-defined options. 1999a. p.15).. (Cabinet Office. 1999a. 15). 1991). Whilst it might be imprudent to confuse analysis..
the set of functions that are generally described by civil servants simply as 'policy work'.. or a radical attempt to improve its management (Wyatt. delegation of management responsibility. 3 Autumn 2002 Downloaded from ppa. The changes . Nevertheless.would be beneficial. The present author was undoubtedly not alone in considering that concern with the reform of policy processes would or should become increasingly prominent in the second half of the decade: There is. by the nature of the processes involved. have tended to be concentrated in those areas of Governmental service delivery which are. 1995. What was less foreseeable at that time was that . p. then.. aimed at finding new ways of working and of working 'smarter. 1994.13) .despite increasing criticism from outside government of the quality of policy work. pp.policy work is carried out.... one important field in which the managerial reforms of the 1980s and 1990s have so far taken much shallower root.. 1970).. (Cabinet Office. 2011 .. It is increasingly being realised that the major task of Civil Service reform in the UK for the remainder of the decade will be to identify the kinds of comparable management change and improvement that will be appropriate in those areas of Civil Service activity that are least like commerce and industry . Much later we find that Continuity and Change alludes to the need to 'improve policy making' as well as 'management and service delivery' (Cabinet Office. a clear focus on outputs and outcomes (Cabinet Office.sagepub. 1994. p.com at Oxford University Libraries on June 16. p. The Emergence of 'Evidence Based Policy' It is. not harder' (Wyatt. most like the operations of business (or at any rate some types of business) in the private sector.it would not be until 1999 that a clear agenda was set for the modernisation of policy making across government. in this context of a new (or at least renewed) interest in reforming the ways in which policy is made in central government.4-6) . all these developments amount to little more than changes in the conditions under which .3) and suggests that the development of a more structured relationship between those at the centre of departments and those in agencies has increasingly focused attention on the nature of headquarters tasks and how they can best be discharged. however. that the references to 'evidence' 16 Public Policy and Administration Volume 17 No. 1994.3). 1995. whether in policy making or the strategic management of agencies.13) This White Paper also proposed that the extension to areas engaged in policy work of some of the principles of the Next Steps reforms maximum clarity about objectives and targets. as an essential precondition for improved service delivery. It would be wrong to suggest that the parts of departments involved in policy work have remained untouched by the various waves of Civil Service management reform introduced by successive Governments since 1979.in other words.options to the contribution they can make to meeting national needs (Cabinet Office. They have not in themselves entailed a comprehensive examination of what that work comprises. p. and a range of initiatives undertaken by individual government departments from 1993 onwards.or the structures within which .
1). p. if appropriate. To meet people's rising expectations. But to aim to make policy that is shaped by the evidence rather than by short-term pressures. let alone put into practice. and suggest ways of bringing about the Modernising Government vision of better policy making (Cabinet Office. Whilst in some ways still a quite daunting proposition. 3 Autumn 2002 Downloaded from ppa. new research.sagepub. 2011 17 . something called 'evidence based policy'.. Of course. evaluation of previous policies. It is noteworthy. 15).17). that are forward-looking and shaped by the evidence rather than a response to short-term pressures. existing domestic and international research. A little further on. including the internet. however.com at Oxford University Libraries on June 16. or secondary sources. that tackle causes not symptoms. as does the need to clarify the relationship between whatever is admitted as evidence and other inputs to the policy process. a report on work carried out by the Cabinet Office in 1999 to examine what 'modernised' policy making should look like. provide a snapshot of current good practice in government departments. To meet these demands. 1999a. 1999a. and to improve the use of evidence in order to understand policy problems better. that are measured by results rather than activity. as part of the expansion of the principle of 'learning from experience'. as we shall see shortly. existing statistics. The raw ingredient of evidence is information. The first in line is Professional Policy Making for the 21st Century. it also says that We will improve our use of evidence and research so that we understand better the problems we are trying to address (Cabinet Office.in Modernising Government must be seen. When we look at the sequence of official documents that followed the White Paper we see this basic underlying concept being developed and elaborated further. derived from a variety of sources expert knowledge. The most striking thing about these two statements is their low-key.. policy making must also be a process of continuous learning and improvement (Cabinet Office. is on the face of it a perfectly comprehensible intention. and goes some way in exploring the practical issues for policy makers. it seems to avoid many of the problems that have surrounded attempts to define and understand. government must be willing constantly to re-evaluate what it is doing so as to produce policies that really deal with problems. In its chapter discussing the use of evidence the report expands considerably on the somewhat laconic references in the White Paper. Good quality policy making depends on high quality information. p. the contentious question of what constitutes 'evidence' remains. that are flexible and innovative rather than closed and bureaucratic. Evidence can also Public Policy and Administration Volume 17 No. What the White Paper does say is that People are becoming more demanding. p. stakeholder consultation. 1999b. practical tone: they represent what is on one level quite a modest aspiration.. and that promote compliance rather than avoidance or fraud. The authors take a wide view of what constitutes the evidence relevant for policy making: The White Paper makes it clear that policy decisions should be based on sound evidence. that the term 'evidence based policy' does not itself appear.
com at Oxford University Libraries on June 16.sagepub.35). the incomplete coverage of research in some areas. * uses the best available evidence from a wide range of sources. there is a tendency to think of evidence as something that is only generated by major pieces of research. Very often they will have a clearer idea than the policy makers about why a situation is as it is and why previous initiatives have failed. taking into account the likely effect and impact of the policy in the future five to ten years and beyond. and the fact that in many cases evidence can either be incomplete.35). The constraints include time pressures. The authors conclude that in order to ensure 'that policy making becomes more soundly based on evidence of what works' it will be necessary to tackle two key issues . 1999b. which includes a list of 'nine features of a policy making process which should produce fully effective policies' (Cabinet Office. p. 1999b. Gathering that evidence through interviews or surveys can provide a very valuable input to the policy making process and can often be done much more quickly than more conventional research (Cabinet Office.the need to improve departments' capacity to make best use of evidence. The report culminates in a description of a model 'intended to describe what an ideal policy making process should look like'. Moreover. costings of policy options and the results of economic or statistical modelling (Cabinet Office. p. 1999b. contradictory or inconclusive. 1999b.include analysis of the outcome of consultation. adding to the difficulty of taking informed decisions rather than reducing it (Cabinet Office. European and international situation. p. * takes a holistic view looking beyond institutional boundaries to the government's strategic objectives.70-71). p. 3 Autumn 2002 Downloaded from ppa. pp. is as follows: A policy-making process which is fully effective: * clearly defines outcomes and takes a long term view. 1999b.31). p. and the need to improve the accessibility of the evidence available to policy makers (Cabinet Office. 1999b. willing to question established ways of dealing with things and encourage new and creative ideas. which of course bears quite a close resemblance to the seven key principles from Modernising Government summarised above. In any policy area there is a great deal of critical evidence held in the minds of both front-line staff in departments. information overload. * constantly reviews existing policy to ensure it is really dealing with problems it was designed to solve without having unintended detrimental effects elsewhere. 18 Public Policy and Administration Volume 17 No. agencies and local authorities and those to whom the policy is directed. 2011 . * takes full account of the national.35). The report also acknowledges the differences in the degree of scientific reliability and validity attainable in the data relevant to different policy areas. This checklist.37). * is flexible and innovative. and the fact that 'policy makers' ability to access evidence-based advice is constrained in a number of ways' (Cabinet Office.
and perhaps under the influence of American usage. p. However. 2011 19 . It is worth noting that the definition of this 'competency' makes an interesting addition to what has so far been said about the use of evidence. Here. This formulation immediately draws attention to the slippery nature of the term 'policy maker'. it has become increasingly common to conflate the roles of deciders and advisers in the general term 'policy maker'. p.74).com at Oxford University Libraries on June 16. perhaps because of the lack of a wieldy term to describe collectively officials engaged in policy work. 1999b. to imply there is an entity . supported by the advice of officials: the Civil Service is responsible for advising Ministers on policy options and ensuring the implementation of Ministers' decisions. However. it proposes as part of its definition: Policy makers' advice/decisions are based upon the best available evidence from a wide range of sources (Cabinet Office. this report reminds us that the better use of evidence has implications for the behaviour of politicians as well as of officials. The next step on this road came with the publication of Better PolicyMaking by CMPS in 2001 (Bullock.7). 1994. * involves all key stakeholders at an early stage and throughout its development. 1999b.or at the very least a discernible body of professional practice labelled 'evidence based policy making' which can be defined and readily distinguished from other kinds of policy making. and taken together with what the report says elsewhere about the sources and limitations of relevant evidence provides a practical reminder to anyone engaged in policy work who is keen to emulate best practice. 3 Autumn 2002 Downloaded from ppa.74). This is perhaps one of the starting points for a rather unhelpful subsequent tendency to set this term in concrete.sagepub. one of which is labelled simply 'Evidence based' (Cabinet Office. This Public Policy and Administration Volume 17 No. p. * learns from experience of what works and what doesn't through systematic evaluation. By drawing attention to the two distinct activities involved. (Cabinet Office. hitherto Professional Policy Making has tended to focus on the behaviours and skills of departmental civil servants preparing policy advice. Modernising Government tends to speak of what the characteristics of policies and programmes should be. pp. Ministers must decide on policy issues . 2001).* is fair to all people directly or indirectly affected by it and takes account of its impact more generally. Not very long ago purists in the Civil Service would have insisted that the only people who actually 'make' policy are Ministers (or other office holders in charge of departments). however.71-2). Mountford and Stanley.and are accountable to Parliament and the public for them (Cabinet Office. The prescription that policy making should use the best available evidence from a wide range of sources is unexceptionable. the report goes on to translate its list of 'features' into what it calls 'core competencies'. 1999b.
Mountford and Stanley. public services may be of poor quality 20 Public Policy and Administration Volume 17 No. it is not entirely clear to what extent this represents the 'better use' advocated by the White Paper. thus ensuring strongly evidence-based policy decisions (Bullock. Government analysts and external experts are working together to ensure that policies are informed by a solid and robust evidence base (Bullock. p. however. although the case studies cited illustrate the use of evidence. and expert knowledge (Bullock. 2001) starts from the same position as Modernising Government in emphasising the connection between good policy making and high quality public services: Departments spend some £350 billion a year on a range of services and activities intended to benefit citizens. p. policy evaluation. p. It is again notable that.or indeed how that could be established.49).com at Oxford University Libraries on June 16.7). 3 Autumn 2002 Downloaded from ppa. The key sources of evidence being drawn upon by departments are domestic and international research and statistics. and what support they required (Bullock.49). conveys a sense of innovation and change. p. for example. Mountford and Stanley. by those formulating policy advice and managing implementation processes). but from the National Audit Office. While some of the case studies either state specifically or imply that the procedures described had not previously been employed in that particular area.report was based on the findings of a comprehensive survey of senior civil servants across government. 2011 . Modern Policy-Making (Comptroller and Auditor General. it is difficult to know how far there has been a general shift in practice across government departments . Another important contribution to the effort to improve policy making comes not from within government itself. Evaluation and Expertise' the report: considers ways in which policy-makers. 2001. the development of policies and their implementation. Quite obviously policy making has never been an entirely evidence-free zone. If policies are not well designed and implemented the consequences can be serious. 2001. the emphasis is on the ways in which evidence is used by policy makers (or more specifically. structured on the nine competencies described in Professional Policy Making. A number of instances are described of research and other evidence being used to inform the understanding of policy contexts.sagepub. what policy makers saw as the main issues involved. as in the previous documents upon which this report is intended to build. economic modelling. The prevailing tone of the report. Mountford and Stanley. and the perceptions of practitioners may (once due allowance has been made for the natural inclination of departments to portray developments in a positive light) be the best available evidence in this regard. The purpose of the survey was to obtain examples of what was being done to modernise policy making in departments. There is little sign of anything called 'evidence based policy' per se: the closest we come to a sighting is an isolated reference in one case study: Policy-makers and government research staff have worked very closely. Mountford and Stanley. It is also of interest that. Under the heading of 'What Works: Evidence.50). 2001. 2001.
37). and also discusses that of statisticians. although it appeared in January 2000. p. p. and thorough analysis as essential to good practice. 2000) is concerned with the capabilities of government departments for analysis and modelling and their access to and use of existing data in key policy areas and how these might be improved.8). 2000. For example: Reliable and comprehensive information is essential in determining the need for a new policy or a change of policy. p. 1).. There is one further document in this sequence which. however. analysis involves the examination and interpretation of data and other information. In turn. 2000. however.or not meet users' expectations and those intended to benefit may not do so or groups in society may be excluded. both qualitative and quantitative. 2001. but no advocacy in general terms of 'evidence based policy' as a distinct approach. 2001. p. implementation. In this context.. 2011 21 . Without sound analysis a solution may be designed based on a misunderstanding of the problem resulting in misdirected effort and poor value for money.sagepub. actuaries. Public Policy and Administration Volume 17 No. having reliable data and staff with the appropriate research and analytical skills (Comptroller and Auditor General. As before. particularly from reviews of research.com at Oxford University Libraries on June 16. 2001. 2001. social and operational researchers and scientists where it touches on the work of economists (Performance and Innovation Unit. to provide insights to improve the formulation of policy and the delivery of services. 3 Autumn 2002 Downloaded from ppa.. p. The Performance and Innovation Unit's report Adding It Up (Performance and Innovation Unit. policy design is broken down into four key elements (Comptroller and Auditor General. p. and relates some of its findings to these.34). has not been mentioned heretofore as it presents something of a contrast to those previously considered. especially in the early phases of policy design.33): * identifying the need for a policy * understanding the nature of the problem * assessing how policies are likely to work in practice * identifying and assessing risks to performance and delivery Throughout the report there is a considerable emphasis on good quantitative and qualitative evidence. we find several good examples of the use of evidence to shape policy. It addresses in particular the work of microeconomists in government. Modelling (a subset of analysis) is about establishing causal and formal relationships between variables (Performance and Innovation Unit.but most of its discussion is structured around what it sees as the principal stages of the policy process: design. and ensuring that such decisions are more evidence-based (Comptroller and Auditor General.9). A prerequisite of sound analysis is. This report again draws on the nine characteristics of modern policy making proposed in Professional Policy Making. High quality public services depend on departments designing and implementing cost effective policies (Comptroller and Auditor General. maintenance and evaluation.
It does. however.constitute a rather slender hook on which to hang anything of substance. p. suggest a stronger assumption than in the other documents we have examined that it makes sense to speak of 'evidence-based policy' as a comprehensible phenomenon . 2000. and one reference to the need for links .sagepub. contrasted by implication with some other putative mode of policy making that does not make use of evidence. which is both rigorously critical of the current state of analytical capability within government. There are at least three similar further uses of the term (p. thereafter rehearse any of the findings of those reports. . It is merely to point out that this appears to be one of the sources for the subsequent adoption of 'evidence based policy' as a description of a distinct and desirable mode of policy making. 2000. This is not to quibble in any sense with the thrust of the report.together with that in Professional Policy Making . p.The report presents itself as 'an integral part of a series of reviews and initiatives to prepare the public service for the 21st century' (Performance and Innovation Unit. however.. It acknowledges at the outset its origin in the policy making commitments of Modernising Government. Nevertheless. (Performance and Innovation Unit.9) This may only be a convenient form of shorthand for the broad shift of emphasis towards better use of evidence of all kinds that we have seen advocated elsewhere. and realistic about the constraints imposed by 'intense political pressures to develop quick solutions to policy problems' and by the complications involved in harnessing 'strong a priori convictions' to 'rigorous and detailed analysis of policy options' (Performance and Innovation Unit. training .5). 2000..8. p. between analytical work in different policy areas to ensure consistency of methods and data and thus a coherent overall evidence base for decision-making.13). these references ..as a definable method or body of professional practice to which policy makers should aspire. but carves out its own line of inquiry and recommendation. it also makes frequent reference to the notion of evidence based policy: Its [the report's] conclusions set out a comprehensive and coherent programme for creating the conditions in which rigorous analysis is routinely demanded and delivered. We need therefore to look outside of the narrow boundaries of the official documentation to locate other factors that have contributed to the emergence of 'evidence based policy' as part of the common currency of debate. p. Significantly for our present purposes. should emphasise the importance of analysis for evidence-based policy (Performance and Innovation Unit.26) in the course of the report. p. 3 Autumn 2002 Downloaded from ppa. 22 Public Policy and Administration Volume 17 No.5).com at Oxford University Libraries on June 16. 2000. It does not. 2011 . Implementation of this programme will be a key step towards the commitment to evidence-based policy made in the Modernising Government White Paper and developed in the Cabinet Office paper on Professional Policy Making (Performance and Innovation Unit.. p.5)... 2000. and mentions Professional Policy Making.
and the other convened by the School of Public Policy at University College London in association with the Cochrane Centre at Oxford.sagepub. searching for and then appraising any evidence relevant to that question. It seeks to help policymakers. .5) In other words. and disseminating Public Policy and Administration Volume 17 No. that evidence-based policy is the exception rather than the rule. 1999. This approach. it is apparent that .. What other kind of policy could there be? Yet I know . 2001..3). maintaining. two important national conferences on evidence-based policy were held: one under the auspices of the Association of Research Centres in the Social Sciences (ARCISS).com at Oxford University Libraries on June 16. although itself subject to some criticism. 3 Autumn 2002 Downloaded from ppa. and contained the observation by the editors that 'there is a growing rhetoric about the use of evidence to determine policy and practice' (Davies. Nutley and Smith. 2011 23 .v) We can also discern in this passage another important factor in the increasing prominence of the 'evidence-based' terminology. in Davies. and the public make well informed decisions about policy interventions by preparing. Nutley and Smith. maintaining and ensuring the accessibility of systematic reviews of the effects of health care interventions' (The Cochrane Collaboration. This international collaboration is in itself one manifestation of a broader 'evidence-based medicine' (EBM) movement which has arisen since 1992. This played a major role in stimulating discussion around issues of evidence in policy making. The Cochrane Collaboration was founded in 1993 'to help people make well informed decisions about health care by preparing. The EBM approach involves defining a specific clinical problem from clinical practice. 2000.The Wider Sources An essential starting point for this inquiry must be the special issue of the journal Public Money and Management (1999) that was devoted to 'The Role of Evidence in Public Sector Policy and Practice'. p. (Amann. has been the explicit inspiration for the Campbell Collaboration. practitioners. which was founded in 1999. This international organisation.. Other evidence supports this conclusion: In early 1999. p. Another contributor remarked that Many researchers have already become so used to the term 'evidence-based policy' that perhaps we now miss its ironic tone. p. It in turn. 1999. (Leicester. p. These key events coincided with the finalisation of the ESRC's plans for a new national Resource Centre for Evidence-Based Policy which will draw together high-quality research evidence to support policy makers and practitioners in a variety of policy domains.before the beginning of 1999 there was already a well-established discourse of 'evidence based policy' in academic circles. p.3). applies a systematic review approach similar to that of the Cochrane Collaboration to a wider field of social policy. founded in 1999. was based on the work of a September 1998 seminar on 'Evidence-based policy and practice'. 1999.13). however. and then applying that evidence in delivering care (Davies and Nutley.despite the rather cautious approach to discussion of better uses of evidence displayed by the key official documents from March 1999 onwards .
2001. albeit obliquely.which is to say. rigorous.3). that this term is deployed and its uncertainties debated may be largely responsible for the extent to which civil servants have tended to feel that it has little to do with them. in the run-up to the General Election in May 1997.294). The new concern with the quality of the evidence supporting policy advice and decisions that is evinced in Modernising Government and subsequent documents seems.xv). 2011 . generalizing.an approach where evidence would take centre stage in the decision-making process (Davies. however. The need for greater rigour and more data . crime and justice. and in particular the emergence of evidence-based medicine. In part it is perhaps simply the next step in a long evolving relationship between policy making and research.sagepub..com at Oxford University Libraries on June 16. therefore.. the origins of which can be traced back to the first part of the 19th century.wherever these do 24 Public Policy and Administration Volume 17 No. 2001. p. and cumulative fashion (Gerring. 1999. It is. This is not to say that very real changes do not seem to be under way in the ways in which people carry out policy tasks. and social welfare (Davies and Boruch. p. 3 Autumn 2002 Downloaded from ppa. evidence-based. which seems to obscure more than it illuminates. the coincidence of these factors with developments in other fields. they intend to study human action in a systematic... What We Mean By Evidence Based Policy The foregoing exploration of the origins of the 'evidence based' terminology has shed. the intention was to signal a new 'postideological' approach to public policy making . rather than in the official documentation. At one remove from the debate on policy making. Nutley and Smith.systematic reviews of the effectiveness of social and behavioural interventions in education. some light on our second question: what does it mean? What is it that civil servants think they are being exhorted to do? The indications. In part it probably results from the convergence of a growing awareness within the Civil Service and amongst its critics of the need for a reform of policy processes to match the managerial reforms that had been accomplished. The fact that it is predominantly in the academic and research literature. aspire to science . with the stance of the incoming Labour government: When Tony Blair announced that 'what counts is what works'. p. research and analysis of all kinds in the development and implementation of policy provide a quite clear direction of march in general terms. we find the 'evidencebased' designation appearing in recent discussions of the methodology of the social sciences at large: . It is simply an observation from first-hand experience that officials appear to remain largely unmoved by the terminology. non-subjective. information. of the need for better use of evidence. in a number of official documents. writers in the social sciences . to arise from the confluence of a number of different streams. that has led to the creation by analogy and wide dissemination of the label 'evidence-based policy'.
2011 25 . such as pressure of time or strong a priori convictions .3-5). On the one hand. 2001. The answers offered by various commentators tend to gravitate towards one of two positions. Another formulation of this position offers the definition that Evidence-based policy helps people make well-informed decisions about policy. 3 Autumn 2002 Downloaded from ppa. or one institution promoting one method. and probably unresolvable. broadly defined. Christopher Pollitt identifies three closely-relatedprincipal questions around which debate revolves (Pollitt. programmes and projects by putting the best available evidence from research at the heart of policy development and implementation (Davies.sagepub.. goes on to suggest that international comparisons can more readily be drawn in terms of 'what other countries have been doing to inject more analysisderived information into the policy process' (Pollitt. 2000. which leads in turn to the conclusion that evidence takes the form of 'research'. and what is to be discarded as not sufficiently reliable? * Who says so . The danger of this research-oriented approach is that it will immediately prompt the resistance of those who perceive the problematic nature.366). p. (Davies. apart from 'evidence' . That is.com at Oxford University Libraries on June 16. there is the argument that all evidence. 'must conform to certain scientific rules of proof' (Davies.could it be that one method. and which can be summarised as follows: * What is and what is not 'evidence' . to warrant the name. p. conceptual and methodological debate. Professor Pollitt. this less assumption-laden formulation takes us back to something much closer to the language used in the official reports surveyed earlier. It is when officials turn to the more specialised literature for more guidance on what might constitute evidence in any given context. p. and how it can be best obtained and most effectively used. pp. and the difficulty of making it comprehensible to interlocutors abroad. 2000. Nutley and Smith. but on which they are unlikely to find much comfort in the literature. Public Policy and Administration Volume 17 No. becomes the only licensed adviser to policy makers? * What else goes into policy making. 2000.. that they are likely to find themselves plunged into a quagmire of continuing definitional. Nutley and Smith.how do we combine evidence and other factors such as norms and values? These unresolved. evidence comprises the results of 'systematic investigation towards increasing the sum of knowledge'.3). of systematic investigation in their particular policy domain. 2001. whether for political or methodological reasons.not clash with other imperatives in the policy environment.is (I have no doubt) well understood. Interestingly. p. It might also be taken to imply an ineluctable. questions do indeed point to just the sort of concerns that are likely to bother practitioners who feel the need to clarify what is meant by 'evidence-based policy'. having established the very UK-specific nature of the 'evidence-based' terminology.2).what is to count.5).
sagepub. On the other hand. convictions. but the drawback is that it begs the awkward question of 'what.. on which there is a substantial literature. both in Britain and internationally' (Williams. and there is a corresponding lack of consensus as to the part that it might play (in relation to other considerations and other inputs) in the processes of policy making. unsatisfactory though this degree of uncertainty might be from an analytical perspective. One version of this approach narrows the field of evidence still further. The sources of the terminology. Concepts of what 'evidence' comprises in this context range widely from the rigorously exclusive to the broadly inclusive. as can the variations in the ways in which it is applied.109). In these circumstances certain and uncontested definitions are likely to continue to prove elusive. Nevertheless.86). can be clearly shown. p. allegedly. in both official and academic publications. we need to ask ourselves how much it really 26 Public Policy and Administration Volume 17 No. norms and values. Conclusion It has not been the purpose of this article to explore comprehensively the debate on the nature of evidence-based policy making..mechanistic linkage between research results and policy decisions. Whose opinion or whose judgement counts? And if nothing is excluded. 2002. This formulation provides a framework which can accommodate not only a hierarchy of scientific evidence (drawn from either the natural or the social sciences. is discarded?'. there is broad consensus as to its contents (if not its interpretation) (Davies. we started? One way of preserving the breadth of possible sources of evidence whilst ensuring some rigour in what is legitimately counted as evidence might lie in the suggestion that evidence (however construed) can be independently observed and verified.com at Oxford University Libraries on June 16. p.2). 3 Autumn 2002 Downloaded from ppa. expert opinion. we find what might be called the 'Not only . 2011 . p. by construing evidence-based policy making as being 'designed to promote the more systematic understanding and use in policy-making of evidence from evaluations of previous policy interventions. The intention has rather been to demonstrate how in the space of a few years the language of 'evidence-basing' has entered in the discourse of British public administration. including individual experience. and become practically a standard element in any discussion of good practice in policy making. 2000. are we not back where. then. 1999. as appropriate) but also a range of other factors. again well expressed in the words of Philip Davies: Evidence-based policy means integrating individual experience and expertise with the best available extemal evidence from systematic research (Davies. consultation.. and . Nutley and Smith.. But also' position. Such a broad approach is likely to be very congenial at first sight to policy officials. This would seem to restrict the field of acceptable evidence so far as to rule out research that leads to improved understanding of the nature or context of policy problems. political judgement.
pp. Cm 4310. R. (eds). Centre for Management and Policy Studies website. 'The Relevance of Systematic Reviews to Educational Policy and Practice'. Davies. S. Oxford Review ofEducation.. H. H. (1999). (London: Cabinet Office). and Smith. that officials should be in little doubt as to the sort of questions that they should be asking themselves about the quality of their policy work. P. Cabinet Office (1991).and also that the approach of UK civil servants to understanding and describing the reforms they have been engaged in tends to be under-conceptualised and relentlessly empirical. Davies. (London: Centre for Management and Policy Studies). (2000). Modernising Government. J. Report and Accounts 2001-02. This mind-set undoubtedly has its weaknesses. Cabinet Office (1999b). (2000). H. Public Money and Management. H. and Stanley. (London: Her Majesty's Stationery Office). pp. 'Foreword'. setting out the kind of change and improvement that is required. P. (2000). Better Policy-Making. 0. Vol 19. P. (1999).3-5. The Citizen's Charter: Raising the Standard. Centre for Management and Policy Studies (2002). Bullock.uk CITU (1996). and Smith. The Reorganisation of Central Government. Nos 3 and 4..cmps. in Davies.108-121.9-16. Vol 47. Davies.gov. (London: Her Majesty's Stationery Office). Davies... but among its strengths is that we are (or should be) able to get on with doing something even if the experts cannot tell us with complete precision what it is we are or should be doing. References Amann. British Journal of Educational Studies.matters to the policy practitioners at whom most of the reports and instructions cited in this article are primarily directed. Nutley. H. (London: The Stationery Office).direct: A Prospectus for the Electronic Delivery of Government Services. No 1. P. T.365-378. Cmd 4506. HC 289 Session 2001-2002 (London: The Stationery Office). 'The rise and rise of evidence in health care'. Cabinet Office (1999a). Professional Policy Making for the Twenty-first Century: A report by the Strategic Policy Making Team. R. S. 3 Autumn 2002 Downloaded from ppa. M. It is sometimes said that the traditions of British public administration are essentially pragmatic .. Comptroller and Auditor General (2001). http://www. There is a sufficient volume of guidance in existence. 2011 27 .com at Oxford University Libraries on June 16. pp. and Nutley. Cabinet Office (1994) The Civil Service: Continuity and Change. Modern Policy-Making: Ensuring Policies Deliver Value for Money. 'What is Evidence-Based Education?'. No 1. Public Money and Management. and Smith. (1999). Cm 2627. 'Editorial: What Works? The Role of Evidence in Public Sector Policy and Practice'. Government. What Works?: Evidence-based policy and practice in public services (Bristol: The Policy Press). Vol 19. Mountford. What Works?: Evidence-based Policy and Practice in Public Services. (Bristol: The Policy Press). Cm 1599. pp. Public Policy and Administration Volume 17 No. Vol 26. No 2. Cm 3438 (London: The Stationery Office).sagepub. S. P. (London: The Stationery Office). (eds) (2000). HC 1091. Davies. Cabinet Office (1970). (2001). Nutley S. Nutley. (London: Her Majesty's Stationery Office).
'The Campbell Collaboration: Does for Public Policy What Cochrane Does for Health'. pp. J. 'Evidence and Policy: Towards a New Politico-Administrative Settlement'. and Boruch. (Cambridge: Cambridge University Press). (2001). Leicester. Efficiency and Effectiveness in the Civil Service: Government Observations on the Third Report from the Treasury and Civil Service Committee. or Evidence Is Not Everything . R. HC 236. 'The Seven Enemies of Evidence-Based Policy'.Davies. Gerring. http://www. Paper given at the 'What Works' in Government . Session 1981-82.sagepub. (1995).org/cochrane/cc-broch. N. A.86-97. 28 Public Policy and Administration Volume 17 No. Pollitt. pp. C.cochrane. British Medical Journal. (2001). 3 Autumn 2002 Downloaded from ppa. No 7308. Vol 19.. (London: The Stationery Office). 'International Developments in Evidence-Based Policy.5-7. Social Science Methodology: A Criterial Framework. Improving Management in Government: The Next Steps (London: Her Majesty's Stationery Office). Efficiency Unit (1988). Williams. Vol 73. Public Money and Management. Paper presented at the Annual Conference of the International Association of Schools and Institutes of Administration. (1999).Evidence Based Policy Conference. G. No 1. P.htm (updated on 1 August 2001). Political Quarterly. No 1. London: March. Vol 323. The Cochrane Collaboration (2001).com at Oxford University Libraries on June 16.294-295.. 'Developing the Concept of Policy Management in the UK Civil Service'. Dubai: July. (2001).'. Wyatt. pp. 2011 . Cmnd 8616 (London: Her Majesty's Stationery Office). Performance and Innovation Unit (2000). Prime Minister and Minister for the Civil Service (1982). (2002). Adding It Up: Improving Analysis and Modelling in Central Government.