A guide to evidence based policymaking

Policymakers often have a hard time knowing in advance what will work and what won’t, which is why
building evidence and evaluation into the policymaking process is crucial for success. If you need to
build your knowledge in this area, read on for our guide to evidence based policymaking and how it
can be carried out effectively.

Franklin Roosevelt, the Great Depression and evidence based

policymaking

Picture this: it’s 1932, the height of the Great Depression in the United States. The unemployment rate is
over 20% and growing. American citizens are looking to the government for answers. Franklin Delano
Roosevelt is running for the US presidency against incumbent President Herbert Hoover.

On the campaign trail, FDR remarks:

“The country needs, and, unless I mistake its temper, the country demands bold
persistent experimentation. It is common sense to take a method and try it: If it
fails, admit it frankly and try another. But above all, try something.”

While many would have expected candidates to campaign on very specific policies, FDR decided
instead to campaign on an alternative approach – experimentation and learning. His message was: we
don’t know what works, so let’s try things.

We can figure out what works and what doesn’t along the way and adjust our policies accordingly. We
can measure both successes and failure, and this gathered evidence can and should inform policy
design.

In your role as a policymaker, you can implement and pilot policies in ways that allow for learning and
adjust policy accordingly. Join us as we look at the intricacies of evidence based policymaking.
 Learn more

Find out more about our short online government and public policy courses and how they could
advance your career.

What is evidence based policymaking?

Evidence based policymaking refers to the method of policy development that consults facts and
credible, relevant evidence to make decisions, over political opinion or theory. For example, a
politician taking this approach may use scientific evidence to aid the development of a new healthcare
policy, instead of ideological beliefs.

Ultimately, the premise of evidence based policy and practice is that policy decisions should be
unbiased and driven by community or country needs, rather than a desire to win the backing of voters.

Policymakers following this method might use existing evidence to support policy development or
conduct new research to bolster their arguments.

Below we explore examples of how experimentation and evaluation is carried out in the education
sector in countries across the globe.

Evidence based policymaking example: improving educational

attainment around the world

The past two decades have seen massive improvements in school enrolment around the world. This is
partly due to the United Nations Millennium Development Goals (MDGs), which mobilised action in the
period 2000-2015 before being superseded by the Sustainable Development Goals (SDGs) in 2015. One
of the MDGs was to achieve universal primary education for all girls and boys. Some of the
achievements under this goal include:

an eight percentage-points increase in the primary school net enrolment between the years 2000 to
2015;
nearly 50% decrease in the number of out-of-school children, globally, from 100 million to around
57 million in the same period;
remarkable improvement in primary education in Sub-Saharan Africa since the establishment of the
MDGs.

But as critics have pointed out since, the MDGs targeted increasing enrolment only, not quality. Hence
these global improvements in school enrolment have not been matched by a similar progress in
learning.

For example, in India, where school enrolment is high, only about 44% of Grade 8 students can do a
division equation. Similarly, only about 75% of Grade 8 students can read a Level-2 book. Level 2 is
defined as ‘reading with help’ where engaging stories, longer sentences and language play are used for
developing readers who still need some help. This has led to calls for improving learning in schools.

We see a similar trend in other countries around the world, but few have picked up the mantle of
experimentation like Kenya.
Trying to identify solutions

In an attempt to improve learning in schools in western Kenya, policymakers in the early 2000s
collaborated with researchers to implement policies that they expected to work, and, when a solution
failed, they tried another.

Solution 1: Flipcharts

Hypothesis: Poor education may be due, in part, to the lack of proper school materials.

Intervention: Provide schools with flip charts covering mathematics, geography, health and two charts
for science (agriculture and general science), as well as a teacher’s guide for science. They could be
used in more than one classroom each day.

Findings: Despite a large sample size and two years of follow-up data, the estimated impact of flipcharts
on student test scores was found to be close to zero.

Solution 2: Textbooks

Hypothesis: Again, poor education may be due, in part, to the lack of proper school materials. This time:
textbooks.

Intervention: Provide schools with additional official government textbooks (English, maths and science).
Sharing textbooks is common in Kenya, and two or three students typically share a workspace. Hence, a
0.6 textbook per pupil ratio was provided in English and science, and a 0.5 ratio was provided in maths,
giving nearly all students shared access to a textbook.

Findings: No evidence that textbook provision increased average test scores, or that it reduced either
grade repetition or dropout rates.

Solution 3: Extra teachers

Hypothesis: Poor education may be due, in part, to large class sizes. When primary education was made
free, it caused an influx of new students with little or no preparation, which affected the overall quality
and achievements of all students as teachers had to spend more time bringing everyone up to the same
level.

Intervention: The ETP programme provided funding to hire a local contract teacher to address classroom
overcrowding. School committees were responsible for hiring the contract teachers and were free to
replace or keep the original contract teacher based on performance. The contract teachers were paid
approximately one-quarter of the salary of regular civil service teachers but had similar educational
qualifications.

Findings: Providing school committees with funds to hire an extra teacher on a short-term contract had a
positive effect on learning. Test scores of students in ETP schools were on average higher than those of
students in comparison schools.

Why do we need evidence based policy?

The three case studies above are not so much about ways to improve education, but rather about a
mindset to use evidence and evaluate policy to understand what works where. Politicians can’t predict
the future and therefore don’t know for sure whether their policies will be successful or not; this is why
 evidence and evaluation should be crucial parts of the policymaking process.

Roosevelt’s policy of experimentation helped the US recover from the worst depression in living memory.
At the University of Oxford’s Blavatnik School of Government, we believe evidence is imperative to
address the complex challenges facing policymakers in today’s world, which is why our online short
course on Evidence in Public Policy equips policymakers with the knowledge and tools to implement this
mindset of experimentation.

This article was originally published on Apolitical.

Find out more

Visit the Evidence in Public Policy course page to learn more about how it could benefit your
career.

Read more blogs

   

Discover more about the Blavatnik School of Government

In partnership with

© Copyright 2023 Blavatnik School of Government Safeguarding policy Privacy policy