Public Policy Analysis

PPMN90007

Dr Kate Williams
kate.williams@unimelb.edu.au
2
success and failure
5
6
Who is doing the analysis?
practitioner policy analysts

- professionals working for an institution

- inside the policy process
- motivated by institutional/pragmatic aims

academic policy analysts

- not working for an institution within the policy process

- outside the policy process
- motivated by academic aims –(studying overall policy dynamics)
- this is your default identity in this module
Who is doing the analysis?
practitioner policy analysts

- professionals working for an institution

- inside the policy process
- motivated by institutional/pragmatic aims

academic policy analysts

- not working for an institution within the policy process

- outside the policy process
- motivated by academic aims –(studying overall policy dynamics)
- this is your default identity in this module
Robert Silvers Joshua D. Freilich
Under Secretary, Office of Strategy, Professor, College of
Policy, and Plans Criminal Justice
Department of Homeland Security City University of New York

“With honor and integrity, we “... conducting innovative

will safeguard the American research at the intersection of
people, our homeland, and our criminal justice and public
values” health”
Robert Silvers Joshua D. Freilich
Under Secretary, Office of Strategy, Professor, College of
Policy, and Plans Criminal Justice
Department of Homeland Security City University of New York

“With honor and integrity, we “... conducting innovative

will safeguard the American research at the intersection of
people, our homeland, and our criminal justice and public
values” health”
 Two ideal-types of analysis

PRACTITIONER

● analysis ‘for’ policy

● inside the process

● institutional aims
 Two ideal-types of analysis

PRACTITIONER ACADEMIC

● analysis ‘for’ policy ● analysis ‘of’ policy

● inside the process ● outside the process

● institutional aims ● academic aims

 Two ideal-types of analysis

PRACTITIONER ACADEMIC

● analysis ‘for’ policy ● analysis ‘of’ policy

Seeks to
understand policy
● inside the process ● outside the process
process and make
improvements
● institutional aims ● academic aims
 Two ideal-types of analysis

PRACTITIONER ACADEMIC

● analysis ‘for’ policy ● analysis ‘of’ policy

Seeks to
understand policy
● inside the process ● outside the process
process and make
improvements
● institutional aims ● academic aims
Policy evaluation
- Why – monitor progress, determine results, understand what works
- When - ex ante or ex post
- What - policy process, policy outcomes
- Which criteria for analysis - effectiveness or accountability
- Who - commercial (PWC); public (NAO); academic (UniMelb); NGO (Oxfam); Lobby (Tobacco)
- What is the overall purpose:
- objective consequences (rationalist)
- understanding (interpretivist/constructivist)

16
Evaluation designs
- Administrative monitoring data (e.g. registrations with dentists)

- Survey research (e.g. satisfaction)

- Before-and-after designs

- Difference-in-difference designs

- Randomised control trial (RCT) (random assignment; double blind)

- Interviews/focus groups
Potential problems
Technical

- Research design problems

- Problems of formulating goals against which performance is measured
- Choosing from multiple goals
- Long or short term goals
- The problem of unintended consequences

Political

- Looks like action

- Supports what you want to do anyway
- Gets you off the hook(?)
Technocratic approach
- Using scientific methods to generate
value neutral objective judgements
- Close relationship to stage model
- Emphasis on quantitative data
- Consequences are politically neutral
Critique
- Critique of oversimplistic rationalism
- Facts and values cannot be separated
- Evaluations are constructed
- Implementation changes policy goals
- Politically charged context
20
Technocratic approach
- Using scientific methods to generate
value neutral objective judgements
- Close relationship to stage model
- Emphasis on quantitative data
- Consequences are politically neutral
Critique
- Critique of oversimplistic rationalism
- Facts and values cannot be separated
- Evaluations are constructed
- Implementation changes policy goals
- Politically charged context
21
The Lambeth Cannabis Policing Study
- July 2001 policy of “warning” and not to arrest for possession of cannabis

- Evaluations:

- PRS consultancy group on policing

- Iposos Mori on public attitudes

- Academic study (2010)

- Impact of evaluation

- Value of evaluation (scope; quality)

Key points
Knowledge on success and failure is good, but:

- Effects of evaluation are cumulative and indirect not linear

- Politics is about values: how evaluations are perceived and used will reflect this
- “Does it work?” is part of the currency of debate for policy making – but evaluations
cannot give you the political answers you need for policy making

Our role as academic policy analysts is to use models to understand the policy process
 Activity
- Find a policy evaluation in an area you’re familiar with
- What type of evaluation is it?
- Potential benefits?
- Potential pitfalls?
afternoon sessions
Afternoon exercises
Three steps:

1. Choose a policy case + report back

2. Choose a theory/framework/literature from the day and apply it to your case + report back
3. Answer the question ‘to what extent is it a good fit’ and discuss as a group
Afternoon exercises

1.30pm - 2.30pm: Choose a policy case (40 mins) + report back (20 mins)

2.30pm - 3.00pm: break

3.00pm - 4.10pm: Choose a theory and apply it to your case (40 mins) + report back (30 mins)

4.10pm - 4.30pm: Answer the question ‘to what extent is it a good fit’ and discuss as a group

4.30pm - 5.00pm: Optional consultations

Step 1: choose a policy case
● The problem
○ Explain your real-world, on-the-ground problem. Tell us the cause of the problem, who it affects, and how.
Give us a sense of scale and urgency.

● The current role of government

○ Tell us how the government in your chosen jurisdiction currently intervenes in or interacts with the problem.

Identify the relevant policies, departments, ministers/officials. How has policy in this area evolved, and how
effective is it at present in dealing with the issue.

● Relevant stakeholders
○ Identify the major groups (beyond gov’t) with a stake in the issue. This might be companies, industry
associations, unions, advocacy groups, specific geographic communities, etc. Tell us who the most
important players are, what their stake in the issue is and how they have interacted with the problem.
Step 2: choose theory/framework/literature + apply it
● How can a particular policy model or framework from today be used to understand the
case?
○ You should show those aspects of the case that are explained or illuminated by your
chosen policy model and address any limitations where the model may fall down.
○ The goal is to ensure your policy model allows you to at least partly analyse your case,
and that in turn, the case allows you to critique the policy model.
Step 3: answer the question ‘to what extent is it a good fit’
● Answer the question: to what extent is it a good fit
● What are the challenges, where did you get stuck?
● What are the limitations of the theories, or difficulties with policy cases
● How did the two elements interact?