Probation Circular

QUALITY ASSURANCE FOR RESEARCH
PURPOSE
To advise probation areas of the creation of RDS NOMS. To advise probation areas of a newly introduced system of quality assurance for research. To audit existing research activity. To audit existing resources available within areas for research, evaluation or performance monitoring. To advise areas of a briefing session on research standards. To advise areas on a future training needs analysis for Research and Information for Staff. REFERENCE NO: 59/2005 ISSUE DATE: 8 August 2005 IMPLEMENTATION DATE: Immediate EXPIRY DATE: August 2010 TO: Chairs of Probation Boards Chief Officers of Probation Secretaries of Probation Boards Regional Managers CC: Heads information and research Units Area leads on information and research AUTHORISED BY: Meg Blumsom ATTACHED: Annex A RDS/NOMS details Annex B PQAB forms and advice Annex C Audit of research proforma Annex D Seminar Attendance proforma

ACTION
Chief Officers should ensure that relevant staff in Information and Research units are made aware of this circular. Following the RDS seminar and workshop session, all staff should complete a proforma for any future piece of research that meets the criteria under the quality assurance. Complete annex C and return to NPD by 16 September 2005. To submit applications for staff to attend the research quality briefing seminar described in this circular. Inform staff in Information and Research units that a training analysis form is to be circulated later in the year to identify training needs.

SUMMARY
This circular describes the new quality assurance procedures for research being introduced in the Home Office and how they will affect probation areas.

RELEVANT PREVIOUS PROBATION CIRCULARS
N/A

CONTACT FOR ENQUIRIES
Chloe.Chitty@homeoffice.gsi.gov.uk (0207 035 3421) RDS/NOMS Quality Assurance of research process Danny.Clark@homeoffice.gsi.gov.uk (0207 207 0675) NPD Research Audit

National Probation Directorate
Horseferry House, Dean Ryle Street, London, SW1P 2AW

Background The creation of NOMS has led to the decision to improve the links between research and policy development by embedding staff from the former RDS within the NOMS directorate. RDS NOMS is the part of the Research Development and Statistics Directorate (RDS) in the Home Office which leads on research and analysis for the National Offender Management Service (NOMS). It consists of two teams dealing with research and evaluation, and statistics and analysis (see Annex A for a breakdown of the individual work of the teams). The team is led by Dr Chloë Chitty who is located within the Directorate of Quality and Standards and reports to Peter Wrench. RDS NOMS goals are to: • • Develop, maintain and report on a high-quality knowledge base to inform policy making and practice, to help the National Offender Management Service and the Home Office in England and Wales to achieve their objectives. To provide the public and Parliament with information necessary for informed debate.

Quality of Research As part of the drive to raise the quality of research throughout the Criminal Justice System, a framework (comprising an approval process and set of standards) for evaluating research has been devised by RDS NOMS (see National Offender Management Service ‘What Works’ Briefing 3/05: Understanding research methods and findings http://www.homeoffice.gov.uk/rds/noms/html). This applies to all delivery units including National Probation Service, HMPS (Prison Service) and the Youth Justice Boards. From September 16th areas are asked to note that any research or evaluation that they wish to embark upon should be considered using this framework, and a proposal put to RDS using the documents attached at Annex B. As specified in the attached paper, ‘Stimulating Good Quality Local Analysis’, all Band 4 projects should be cleared through the new RDS NOMS process. Where a project is Band 3 and does not meet the criteria specified below, it should not be submitted for approval. If a Band 3 project does meet one of the criteria outlined below, it should be submitted for approval. With the new framework areas will be required to have a quality review on any piece of work which is either • over 30 days staff time (1) or • over £10K in external costs or • outcome research (2) or • for publication or submission to the Minister Notes (1) an equivalent measure is 200 hours time; this may be more appropriate to smaller areas (2) Outcome research is defined as work which is designed to establish whether there is an impact on reoffending, reconviction or any related outcome measure (e.g. maintaining employment, drug testing) . It does not include qualitative research e.g. offender feedback, staff review Areas need not advise RDS NOMS of other work but should report to NPD annually as part of the annual report on any studies that they have conducted locally. Where staff wish to publish work which was not formally agreed at the time of commissioning they should either publish in a journal or publication where there is a process of peer review in place, or should seek advice from RDS. Research that is already commissioned is not covered by this requirement, although areas may wish to seek advice from RDS NOMS over any piece of work currently underway in order to be able to obtain the “kite mark” if appropriate. RDS NOMS will provide advice on research design questions from areas. Action: Areas are asked to note the pro-forma (at appendix B) and fill them in for any newly commissioned work which meets the criteria (i.e. is an outcome study, or will take more than 30 days staff time, or will cost more than £10K or will be published or submitted to Ministers).

PC59/2005 – Quality Assurance for Research

2

Audit of Existing Research and Research Capability Under the new guidelines for NOMS, Probation areas are asked to identify what research or evaluation is currently underway (see Annex C). This information will also be helpful in developing the framework for the staff seminar. This annex also asks areas to identify the resource that is available to them for 1. 2. 3. 4. Routine data analysis Performance Monitoring work Small scale local evaluations (e.g. less than 30 days staff time) Other research work

Action: Annex C should be filled in and returned to the Nina Marvan OBP Team, Interventions Unit NPD by 16th September 2005 Briefing Session on Research Quality As part of helping to introduce the new pro-forma and arrangements RDS NOMS will be running a seminar for all staff to attend, where possible. This will take place in the last two weeks of September; full details on the seminar will be circulated in due course. It is expected that the seminar will cover different types of research, research standards and briefing on how to complete the attached PQAB documents, using both presentations and workshops to help familiarise staff with the process. Action: Areas are asked to nominate staff using form in Annex D. Training Analysis RDS organise training for research staff within their team. They are able to offer a limited amount of training to areas to help support the personal development of research and information staff. A training needs analysis will be circulated to areas later in the year, to help identify the sorts of opportunities that would be useful. RDS staff would also hope to learn from areas, e.g. to understand the skills needed for routine data collection and analysis.

PC59/2005 – Quality Assurance for Research

3

Annex A
DIRECTORY OF BUSINESS - RDS NOMS Chloë Chitty Assistant Director Tel: 020 7035 3421

1. NOMS Research and Evaluation General email address: RDSNOMSR&E@homeoffice.gsi.gov.uk Gemma Harper Programme Director Courts and Sentencing Research Functions 1. Courts research 2. Information to sentencers 3. Intermittent custody 4. Criminal law 5. Satellite tracking 6. Electronic monitoring 7. Fines and day fines Robert Street Jessica Harris Mark Abram (1-7) (2-7) (1,5)

Offender Management Research Functions 1. Offender management 2. Regionalisation and contestability 3. Implementation of interventions 4. Enforcement, compliance and completion of interventions 5. Bail and remand 6. Female offenders 7. Ethnic minority offenders Samantha Jones SRO SRO Lan-Ho Man (1-7) (1,3-5) (2,3,7) (1,4)

Public Engagement Research Functions 1. Randomised Control Trial (RCT) programme 2. Restorative justice 3. Reparation by offenders 4. Public understanding, engagement and confidence 5. NOMS contact with victims 6. Diversion from NOMS Robin Elliott-Marshall Catherine Nicol Rosalyn Xavier (1-6) (1, 3-6) (2-6)

Annex A
Community Supervision Research Functions 1. Approved premises 2. Accommodation 3. Education, training and employment 4. Substance misusing offenders in the community 5. High risk of harm offenders in the community 6. Spending Review 2006 (NOMS) Rae Sibbitt SRO Julie Wilkinson (2,4,6) (1-3,5)

Rehabilitative Interventions Research Functions 1. Offending behaviour programmes 2. Juvenile offenders 3. Persistent offenders 4. Polygraph testing Mia Debidin Rachel Walmsley Charlotte Allen RO (1-4) (1,3) (2,4)

Prisons, Drugs and Resettlement Research Functions 1. Resettlement of prisoners 2. Family ties 3. Substance misuse in prisons 4. Sex offenders in prisons 5. Custodial behaviour/decency in prisons 6. Parole/lifers research Malcolm Ramsay Chris May Stephen Niven Duncan Stewart (1-6) (1,3) (1,2) (1,2,5)

2. NOMS Statistics and Analysis General email address: RDSNOMSS&A@homeoffice.gsi.gov.uk Pat Dowdeswell Programme Director Sentencing Statistics and NOMS Projections Functions: 1. Projections of the prison population and probation caseloads 2. Ad-hoc NOMS Analytical Modelling 3. Sentencing Statistics 4. Penalty Notices for Disorder statistics 5. Answering requests for information on sentencing, prison, probation, re-convictions and penalty notices for disorder

Annex A
Dr Nisha de Silva Dr Paul Cowell Dr Terence Yiu Wa Chow Paul Worthington Graham Wilkins EO Karl Chads Hehtal Patel (1-5) (1-4) (1-3) (2-4) (3-4) (3-4) (5) (2,3)

Offender Management Analysis Functions: 1. Prison population statistics 2. Data quality for prison population statistics 3. Probation statistics 4. HDC and Parole statistics 5. Breach and Recall statistics Rachel Councell Veronica Hollis HEO Gary Renshaw Adeel Hassan Julia Reay David Clemas Mark Judd Michael Poole Ms Elizabeth Brocklehurst (1-5) (1,2) (4) (1,3) (4,5) (1,2) (3) (1,3) (1) (1)

Reconviction Analysis Functions: 1. Convictions database development and methodology 2. Reconviction rates: Adult offenders for PSA 5 3. Reconviction rates: Young offenders for PSA 5 4. Reconviction rates: other studies, special projects 5. Offenders Index Analysis 6. Police National Computer Analysis 7. Offender Group Reeconviction scale/predicted reconviction rates Warren Evans SRO Georgina Ford RO (1-7) (1,4,5-6)

Data Management Team Functions 1. Creation, maintenance and data quality of databases concerning statistics on sentencing, prison, probation, Offenders Index and penalty notices for disorder 2. Co-ordination of requests for Offenders Index and PNC data 3. General IT support and advice Jonathan Barbour Farid Guessous EO Tasleem Adedamola Olowe Maya Devi Powar (1-3) (1-3) (1-3) (1-3)

Annex A
3. NOMS OASys Data, Evaluation and Analysis Team (O-DEAT) General email address: O-DEAT@homeoffice.gsi.gov.uk Natasha Garnham Head of O-DEAT Functions: 1. Offending history and offence analysis 2. Accommodation, ETE and financial management 3. Relationships and lifestyle 4. Alcohol and drug misuse 5. Emotional wellbeing, thinking and behaviour, and attitudes 6. Risk of harm analysis 7. Sentence plan and targeting 8. Self assessment 9. PSRs and sentencing 10. Offender management and CJA 2003 11. Mentally disorders offenders 12. Low risk offenders 13. High risk offenders 14. Persistent offenders 15. Young offenders 16. Reconviction analysis and interim change analysis Michelle Burns Philip Howard Robin Moore Emma Peart Nicola Vallis (3, 6, 9-10) (1, 12, 14, 16) (4, 7-8, 15) (5, 13) (2, 11)

Annex B

Project Approval Record for Home Office Research1
File reference Project title Project leader RDS lead Note 2 Note 3 Note 5 Note 6 Note 6 Note 8 Note 9 Note 10 Section Section Programme Group Job title Unit Directorate Note 4 Note 1

Issue date: 20 July 2005

Section Head Programme Director Assistant Director Main contact Head of Unit Director

ERA or RDS-xxx Note 7 Note 8 Note 9

Group lead

Aim of project Notes 11-15 Expected output Notes 16-21 What will be done with the output? Note 22 By whom? Say who will take the action referred to in above. Name Note 23-24 Job title Note 25 When is the output required Notes 26-27 Other key information Note 28 Agreement Group PQAB GEB-designated signatory RDS Assistant Director OR Director of RDS Minister OR RDS Assistant Director (see Introductory Notes)
Use no more than two sides. Print double-sided.

Signature (in the order shown)

Name

Date

1

Please read the guidance notes before completing the form. Make sure you are using the current versions of the form and guidance. If in doubt, check the RDS pages on Horizon or call the PQAB secretariat on 020 7035 3285. Guidance on completing the Project Approval Record (PAR) Page 1 Issue date: 20 7 05

Annex B

Guidance on completing the Project Approval Record (PAR)

Issue date: 20 July 2005

The PAR records the “triple key” agreement to conduct a Home Office social research project. The three keys are held by (a) the Home Office Group Executive Board (GEB) lead for the area in which the project falls, (b) Research, Development and Statistics Directorate (RDS) and (c) the appropriate Minister. PQAB (Project Quality Approval Board) approval is the means to secure RDS agreement to the project design.

Please read this guidance before completing the form. Make sure you are using the current
versions of the forms and guidance. If in doubt, check the RDS pages on Horizon or call the PQAB secretariat on 020 7035 3285.

Introduction Do I need a PAR for my project?
A PAR should be completed for all social research projects (including impact evaluations) that are conducted or funded (wholly or partly) by the Home Office. A project may be exempted by the Director of RDS or by the RDS Assistant Director (AD) who is responsible for the research area, if the work will be carried out entirely by Home Office staff and will require a maximum of 10 staff days to complete. Alternatively, in such small projects, the AD may request that a PAR be produced, to be signed off by him/herself and/or the policy lead (SCS level). Any exemption is not automatic and must be allowed only after considering the project risks: a small project can entail large risks, e.g. it might produce results that are counterintuitive, contentious or have major operational resource implications but might nevertheless be liable to public disclosure. In some cases, RDS staff will be advising another party on a project (e.g. a policy colleague letting a research contract). In such cases, the other party would normally be responsible for producing the PAR, consulting with the relevant RDS Programme Director (PD) and other RDS staff as appropriate. Where responsibility for producing a PAR sits outside of RDS, RDS staff should not decline to provide advice on a project solely because a PAR has not yet been approved, so long as there is a request in writing from the relevant policy/operations lead. The extent of such advice should be determined by local RDS management.

Who completes the form?
The form will generally be drafted by the project leader, liaising with line management and policy/operations lead as appropriate. The correct completion of the PAR is the responsibility of the RDS PD in whose programme the project falls. In submitting the PAR for signature, including Project Quality Approval Board (PQAB) approval, the PD signifies that s/he: • understands the policy/operational requirement for the project; • is content with the proposed design of the project and its links to related past, current or planned work; • has identified any diversity issues in the project design and made the policy/operations lead aware; • can deliver the project to the required timescale and quality, with the available resources. This will normally follow discussion with the staff who are directly involved with the project design, and the policy/operations lead. The PAR would generally need to be accompanied by other documents, at the various steps in the approval process, to inform signatories in more detail about the proposed project. Because of differing requirements and expectations across the Home Office, and different roles and types of research, there will inevitably be variation in the formal and informal elements in the process leading to approval by policy/operations lead and Minister. Nevertheless, the discussion will need to cover at least the feasibility and likely value of the research, a suggested design, how the results are to be used, a cost estimate and an estimated timescale. Whatever the contributing processes, the PAR is used to record approval, thus promoting consistency of approach and avoiding confusion as to what has been agreed.

GEB approval
The first “key” is agreement that the project is required for policy/operational purposes. For this, the PD passes the PAR to the GEB-designated signatory (Grade 3 equivalent or above) for the relevant Group (e.g. NOMS, IND) to agree. There should be one designated signatory for each Group – someone who

Guidance on completing the Project Approval Record (PAR)

Page 2

Issue date: 20 7 05

Annex B
can assign priority to projects for the Group as a whole. In the temporary absence of the designated signatory, a GEB-designated deputy may sign. Signature indicates that: • there is policy/operational need for the project; • diversity issues relating to the project have been appropriately addressed; • the project has the required output, in terms of the knowledge that it proposes to generate and the form in which that knowledge will be made available; • the project takes priority over other projects that might alternatively be funded (i.e. either it is in the business plan and remains a priority, or it has been developed since the agreement of the business plan and is now a priority).

RDS approval
The second “key” is agreement that the project has the required level of quality, i.e. it can be expected to produce the knowledge required by the Group in a sufficiently precise, reliable and useful form. This agreement requires signature by the RDS AD representing PQAB or the Director of RDS. Submission to PQAB will require completion of an additional form to accompany the proposal itself.

Ministerial approval
The third “key” is the Minister’s approval, to confirm that the project is required and that there is an appropriate route for using the outcome from the work. Where (a) the work is part of the RDS business plan, previously agreed by the Minister and (b) the project is not one that the Minister has required to see for individual approval, the RDS Director or AD may sign to indicate Minister’s approval. PDs need to be aware of the requirements for their Minister.

Notes on completing the form General points
Complete the shaded parts of the form. Where there is a note reference on the form, delete it and replace it with your text (10 pt Arial). Some of the information requested will be the same on all the PARs that you complete; save a template with this information (but keep track of any updates that RDS makes in the form). Paragraph spacing is automatic: you do not need to press “Return” twice. Here, and elsewhere in the submission, please (on first use) spell out acronyms and abbreviations and explain any terms or organisations that may not be familiar to readers. Proposers should always remember that material in the PAR is likely to be subject to disclosure under the Freedom of Information Act (FOIA).

Project identifiers
1.

This is the reference number of your file for the project. If a file number has been requested but not yet provided, write “Number requested”. Provide a short descriptive title that clearly identifies the project and distinguished it from other projects. Avoid: • starting with phrases that could apply to most projects, such as “Research to ……” or “A study of …….”; • using a title that implies a wider scope than the project actually has. This is the person who will take responsibility for the execution of the project. The project leader is not necessarily in RDS but there should always be an RDS contact. Within RDS, this would normally mean a team managed at Grade 7 level.
Guidance on completing the PQAB submission 2 Issue date: 20 7 05

2.

Responsible staff
3.

4.

Annex B
5.

If the project leader is not within RDS and the main RDS contact is not a Section Head, give the name of the main RDS contact in brackets, e.g. if Jane Black is the Section Head but John White (one of her staff) is the main contact, write “Jane Black (John White)”. These are RDS titles – staff may be known by different titles within embedded teams (e.g. in IND, G7 managers are called Assistant Director and the RDS Assistant Director is call Director). Insert abbreviation of embedded RDS team at xxx, e.g. NOMS, CRCSG. The main contact is the person who takes the lead in contacts with RDS on this project. The main contact can be any grade but needs to have received authority to make most decisions for the policy/operations lead in relation to the project. Normally there should be a single main contact; if there are different contacts for distinct aspects of the project, record both contacts (and their Head of Unit and Director) and state (in the “Other key information” box) what role each is playing. Where the contact has a job title, use that title, not his/her grade. Normally means the manager at SCS PB1 level (IND uses the titles “Director” and “Directorate” respectively for the Head of Unit and the unit itself) Normally means the manager at SCS PB2 level (generally called “Director” but IND uses the title “Senior Director”).

6.

7. 8.

9.

10.

Aim of project
11. 12.

State what the project is for: who needs it and why; what are its expected benefits? This should be a short clear statement of the project aims, phrased as specific questions, outcomes or descriptions of the intended product – not as research hypotheses or general objectives such as “to inform policy on ……..” or “to enhance knowledge of …..”.. Refer to specific RDS objectives or Home Office Aims if relevant in a specific way to this project. Do not describe the project method - that comes later (in the PQAB submission) or, if requested, in supporting material. The aim will generally relate to informing a specific area of policy or practice but it should be explained if it is also (or instead) aimed at: • developing an area of basic knowledge or theory; • collecting a new body of information or data, for example through a new survey series; • developing research methods or techniques; • providing a product such as a website or new software.

13. 14.

15.

Expected output
16.

This is what the project is intended to produce: the new knowledge and the form in which it is to be presented. Describe the output, not the method. The submission must make it sufficiently clear to all parties what is expected from the research, and what cannot be expected: both the research team and the policy/operations colleagues must be clear about what the project will and will not deliver, what is achievable and what is not. If there is pressure of time or finance on a project, it is generally better to limit the objectives and use the best method to meet those objectives, rather than maintain unrealistic objectives by using a weak method. It is particularly important to note any intention to publish and the expected form of publication (e.g. RDS Online Report, journal paper). If a report will be for internal use only, consider stating here that the report will be produced to a technical and presentational quality that would be acceptable for release under FOIA. Remember that, if the work is carried out under contract, the contractor will normally be allowed to publish the work if the Home Office decides not to do so. As far as possible, indicate what kind of conclusions the project is intended to lead to, based on the research questions (e.g. evidence on which of three possible interventions is most effective).

17.

18.

19.

20.

Guidance on completing the PQAB submission

3

Issue date: 20 7 05

Annex B
21.

Always note any limitations of the work and whether they have been accepted by whoever is commissioning the work.

What will be done with the output?
22.

Be as specific as possible, e.g. not just “To inform policy.” but “Policy lead will advise Minister on whether to extend dissemination of information to potential asylum seekers in their home country”. Remember that the person who receives the output may not be the person who originally commissioned it.

By whom
23.

Normally there should be a single name here. If there are different contacts for distinct aspects of applying the project output, record both contacts and (in the “Other key information” box) state what role each is playing. The person responsible for applying the output will usually not be from RDS, the main exception being if the project is developing methodology, to be applied in other projects. The person who receives the output may not be the one who commissioned it – people move on. Therefore, give as specific a job title as possible so that the right person can be identified if there are changes in personnel between initiation and completion.

24.

25.

When is the output required?
26.

Say when the output, or different parts of the output, will be required, so that the project can then be planned in order to meet the target date. For this purpose, an “output” might be internal communication of key findings, rather than a full report or publication. Also state whether the value of the outcome is date-critical, e.g. if the information will no longer be useful if delivered late.

27.

Any other key information
28.

For example, any particular expectations on the part of Ministers, involvement of OGDs, political sensitivities, external obligations that the Home Office has in relation to the project (e.g. legal or contractual requirements) or the existence of related projects.

Guidance on completing the PQAB submission

4

Issue date: 20 7 05

Annex B

PQAB Submission Template
Please read the guidance notes before completing the form.

Issue date: 20 July 2005

Make sure you are using the current versions of the form and guidance. If in doubt, check the RDS pages on Horizon or call the PQAB secretariat on 020 7035 3285.

1. Project Approval Record (PAR)
The PAR must accompany this submission.

2. Project description
PQAB reference: Project title (from PAR) Approval by RDS Programme Director Signature Name Date Note 1

Will the research be conducted by Home Office staff or externally?

External only Internal and external Internal only

If all or part of the work is to be external, put a cross here if the ITT is included with this submission

ITT included

Background
Notes 2-6

Method
Design type Literature review Evaluation Other quantitative research Qualitative research Other (state below)

Notes 7-15 Burden on data providers – put a cross in one box No significant burden Burden identified (as noted in box above) and accepted by provider Burden identified (as noted in box above) and yet to be negotiated

Dissemination and impact
Notes 16-19

Guidance on completing the PQAB submission

5

Issue date: 20 7 05

Annex B
Ethical considerations
Will an ethics committee will be involved in approving the work (e.g. a contractor’s ethics committee)? Yes No Don’t know yet If yes, who will be responsible for applications to it?

Put one X on each row of the following table to indicate whether each of the potential ethical issues is (a) not an issue for this project, (b) an issue that has been considered and dealt with or (c) not dealt with. If there are additional ethical issues for this project, please add rows to the table to cover them Not an issue Assignment of potentially beneficial treatments is influenced by the research Medical interventions Honesty to researchers and subjects about the purpose, methods and uses of the research Informed consent Participant confidentiality and anonymity Data protection The independence and impartiality of researchers in relation to the subject of research Risks to researchers and subjects (e.g. health and safety) Working with vulnerable groups If the responses on the checklist indicate that any ethical issues are not adequately covered in the proposal, state how this will be resolved. Note 20 Specify which professional code(s) of conduct (e.g. GSR, SRA, BPS) has been or will be referred to in addressing ethical questions during the course of the research. Issue dealt with Issue not dealt with

Diversity issues
Will the research address any of the following diversity issues in the sampling, data collection or analysis? Yes Gender Ethnicity Disability No Religion Sexuality Specific age groups Yes No Other (stated below) Yes No

If the answer is yes to any of these, answer the three following questions. What procedures are in place to ensure that the research is undertaken in a way that is sensitive to the needs of participants from different backgrounds/cultures? Notes 21-23 To what extent is the research aiming to use a sampling procedure that would be expected to produce a sample that is representative in terms of sex, age, ethnicity, religion and sexual orientation?

If not fully representative, why is this? Note 24

Guidance on completing the PQAB submission

6

Issue date: 20 7 05

Annex B
Project risks
Main risks

Notes 25-26

Risk assessment/management Hazard
Note 27

P
Note 28

I
Note 29

Mitigation
Note 30

Monitoring
Note 31

Contingency
Note 32

Project plan
Date (MM/YY) Target Project start PQAB submission Contract start (delete row if no contract) Draft final report from contractor (delete row if no contract) Note 33 Contract completion (delete row if no contract) Publication draft to CDU or other publisher (delete row if no publication) Project completion

Notes 34-36

Resource plan
Home Office project team OGD staff Contractors Others Total3 Staff days Cost2 inc. overheads (£) Staff days Cost inc. overheads (£) Cost (£) Cost (£) Days Cost (£) 2005/06 0 0 0 0 0 0 0 0 2006/07 0 0 0 0 0 0 0 0 2007/08 0 0 0 0 0 0 0 0 Other years 0 0 0 0 0 0 0 0 Total 0 0 0 0 0 0 0 0

Other information Notes 37-41 Project Skills Note 42

After completion up to this point, the RDS Programme Director should email this form to Mark Greenhorn and Mike Taylor in CMU, RDS. If urgent response is needed, explain and give deadline.

2 3

Including overheads and all non-contract costs. Right click on total and update field. Guidance on completing the PQAB submission 7 Issue date: 20 7 05

Annex B 3. Feedback record
Section 1: First decision
PQAB decision (cross as appropriate) 1a 1b 2 3a 3b 4 5 Approved without modification Approved subject to minor changes Modify and submit to board Provisional acceptance, pending sight of ITT Provisional acceptance, pending sight of preferred tender Rejected Decision taken after discussion with third party

Design rating (1-5) PQAB PD (responsible for feedback) PQAB AD

Principal areas of weakness / lack of detail Aim definition Account of background / related work Practicability Design Sampling Power calculation Measurement Availability of data Consultation within RDS Issues with third parties (e.g. courts, IS, police) Analysis Dissemination & impact Project plan Timescale Risk assessment

Section 2: Feedback and response
PQAB comments/feedback4 Compulsory: numbered points that must be addressed in the proposal 1.

The PD and Project Leader do not have to accept PQAB comments without question but would need to argue the case for not taking on board all the comments. Guidance on completing the PQAB submission 8 Issue date: 20 7 05

4

Annex B
Further actions needed by the Project Leader: numbered actions (e.g. to discuss with a third party, provide further documentation or further information about particular aspects of the project) 1. Other points, e.g. stylistic, English, points to take into account as the project progresses5 1. Response from Project Leader6 Compulsory points 1. Further actions 1. Other points (response not compulsory unless specifically requested by PQAB) 1. If there is more than one round of comments and response, duplicate Section 2 as necessary.

Section 3. Final decision
Final PQAB decision (tick as appropriate) 1a 1b 3a 3b 4 5 Approved without modification Approved subject to minor changes Provisional acceptance, pending sight of ITT Provisional acceptance, pending sight of preferred tender Rejected Decision taken after discussion with third party

Design rating (1-5) Copies of the feedback record should be circulated at every stage to:
• • •

The PQAB secretariat the relevant Project Leader the relevant Programme Director.

5 6

If appropriate, comments in this category may be provided as comments/tracked changes added electronically to the proposal document. Also list any additional information or provide this in annexes. Guidance on completing the PQAB submission 9 Issue date: 20 7 05

Annex B

Guidance on completing the PQAB Submission

Issue date: 20 July 2005

Please read this guidance before completing the form. Make sure you are using the current

versions of the forms and guidance. If in doubt, check the RDS pages on Horizon or call the PQAB secretariat on 020 7035 3285. managed by the Research, Development and Introduction Statistics Directorate (RDS) and aims to: Do I need PQAB approval for my project? • ensure that any research undertaken has a PQAB approval is required for all social research good likelihood of achieving its stated aims; projects (including impact evaluations) that are • clarify to policy/operations colleagues what is conducted or funded (wholly or partly) by the and is not achievable by a particular approach Home Office unless one or more of the following to a project; exemptions applies. • thus, ensure that more robust and valid • The project has already been exempted from research is undertaken. requiring a Project Approval Record (PAR). Although set in a context of formal approval, the • The project has been exempted by the Director aim of PQAB is to improve research designs and of RDS or by the RDS Assistant Director (AD) assist the Home Office in developing better who is responsible for the research area, on the research capability. grounds that the work will be carried out entirely by Home Office staff and will require a The relevant RDS Programme Director (PD) has maximum of 20 staff days to complete. the dual role of obtaining signatures on the PAR Alternatively, for such small projects, the AD and approving submissions to PQAB. The PD, in may request that a PQAB submission be both cases, signals a belief that the project is produced, to be signed off by him/herself. Any sufficiently well designed for its purpose, can be such exemption is not automatic and must be expected to produce the required result and all allowed only after considering the project risks: parties should sign up to it. a small project can entail large risks, e.g. it Where a proposal originates outside RDS, the might produce results that are counterintuitive, PD may forward it to PQAB with a note of any contentious or have major operational resource concerns, if these have not been adequately implications but might nevertheless be liable to addressed by the project leader in response to public disclosure. the PD’s comments on drafts. • Where there is any doubt as to whether PQAB Linking the PQAB submission to an approval is needed, the PAR should be signed invitation to tender (ITT) by the policy/operations lead and then either (a) signed off by the relevant AD to confirm that There is a certain amount of duplication between PQAB approval is not needed or (b) sent to the the information required in the PQAB submission PQAB secretary (via the AD) to assess whether and in ITTs. Ideally project leaders should have PQAB approval is needed. to record this information only once. To this end, As a minimum, PQAB approval is needed for all social research and impact evaluation to which one or more of the following applies: • national in scope; • intended to be published; • results to be sent to Ministers; • a study of outcomes of policy or operational changes; • a project of a type specified by Home Office GEB areas (e.g. in the case of NOMS, outcome studies). the following options are available to the project leader. (a) Complete the ITT (or at least the parts that overlap with the PQAB submission and provide the ITT as Attachment(s) A1, A2 etc. To the PQAB submission. In this case, the PQAB submission need provide only a summary of what is in the ITT or only the information that is not included in the ITT (but should refer to the specific parts of the ITT where the required information is to be found). This would include, for example: • aspects of the work that is to be carried out by Home Office staff;

The role of PQAB
PQAB reviews and approves Home Office social research (including evaluation) projects. It is

Guidance on completing the PQAB submission

10

Issue date: 20 7 05

Annex B

a project risk assessment (as distinct from a contract risk assessment, which covers the work to be contracted out); information that is for internal use only.

(b) Complete the ITT and transfer the relevant information to the PQAB submission. It is advisable in this case also to provide the ITT (as Attachment(s) A1, A2 etc) since PQAB may wish to see it. (c) Complete the PQAB submission and then transfer the relevant information to the ITT. Again, it is advisable also to provide the ITT (as Attachment(s) A1, A2 etc) since PQAB may wish to see it. It is not compulsory to provide the ITT with the submission but PQAB may, depending on how much of the design is being left to contractors, request sight of the ITT before approving the project.

Both proposers and PQAB members should always remember that material in PQAB submissions and decisions is likely to be subject to disclosure under the Freedom of Information Act (FOIA).

Project identifiers
1.

This is the reference provided by the PQAB secretariat. Use this reference in any correspondence on the project. If the correspondence is by email, put the reference at the start of the subject line of the email (this assists the secretariat in keeping track of projects). This section should explain the reasons for the research, for the benefit of readers who are not familiar with the area. Include the reason(s) for the Home Office’s interest and a brief summary of what is already known on the subject of the research, leading to a description of the issue to be addressed and an explanation as to why answers are needed to the specific research questions: what benefit will the research bring, for whom or what, and why? Clearly formulate the problem (putting it in a clear and appropriate context of policy, practice, knowledge and theory) and the research questions that follow. A full bibliography is not needed. However, if particular documents are essential to the justification of the project, they should be referenced in a footnote. If it is unlikely that PQAB will have access to them, copies should be made available (labelled Attachment B1, B2, B3 etc. and referenced in the text). Identify other parties (e.g. other parts of RDS, the wider Home Office or OGDs) to which the proposed project is relevant. This section should provide a clear, realistic, convincing and well thought out research design that can be related to the project objectives, scale, timing and resources. It should cover all relevant aspects in sufficient detail for PQAB to make a judgement as to whether the approach taken is the most relevant and appropriate, and adequately worked out. This will include reference, as appropriate to the particular project, to all relevant aspects of method, e.g.:

Background
2.

3.

Related projects
If several similar or related projects are being proposed at the same time, they should ideally be combined or cross-referenced to avoid writing (and PQAB reading) duplicate material.

Notes on completing the form General points
Complete the shaded parts of the form. Where there is a note reference on the form, delete it and replace it with your text (10 pt Arial). Paragraph spacing is automatic: you do not need to press “Return” twice. PQAB submissions must be written in plain English: detail and specification may necessitate the use of discipline-specific or technical terminology, but the content should be clear to a wide audience: PQAB members will not necessarily be specialists in the area covered by the proposal. Spelling, grammar and punctuation affect the clarity, and therefore the quality of the submission (refer to the RDS document Guidance for Authors8). On first use, spell out acronyms and abbreviations and explain any terms or organisations that may not be familiar to readers. Alternatively, if there are many acronyms, abbreviations or unfamiliar terms/organisations, these could be put into a glossary.

4.

5.

6.

Method
7.

8

http://horizon/rds/directorate/publications/guide_authors250804.doc

Guidance on completing the PQAB submission

2

Issue date: 20 7 05

Annex B
• • • • • •

• • • • 8.

the research questions; the use of expert advisers or steering groups; the arrangements for any pilot study; what data, materials or information are to be collected; any existing data sources and how they are to be accessed; sampling protocols (sampling frame, population definition, sample sizes, the sample design etc.); methods of recruiting subjects or survey respondents; data collection instruments and procedures (e.g. questionnaires, interviews); internal and external validity; plans for data analysis and interpretation.

from external advisors or internal experts is already available, and is likely to be helpful to PQAB, provide this as Attachments C1, C2 etc. and reference it in the text.
12.

External advice can, of course, be sought for smaller projects but the cost needs to be kept in reasonable proportion to the cost of the research and the possible consequences of getting the research wrong. The method description should explain the role or relevance of contractors, external stakeholders, other parts of RDS, the wider Home Office, OGDs or any other parties on whom the success of the research depends (including the implementation of the findings). The description should also state what discussion has taken place with these parties and, where required, what agreement (if any) has been reached with these parties or what approach will be taken to reaching agreement. For example, if existing data will be used, is it certain that access will be given and is there written confirmation of this? If documentation is available on agreements with other parties, and if it is important for PQAB’s considerations, provide it as Attachments D1, D2 etc. and reference it in the text. Similarly, any likely impact on stakeholders should be noted where it is relevant to the design of the study. For example, this would apply where certain options for the research design have been rejected because of their demands on front-line staff (e.g. in prisons or at ports). The project leader must consider the likely burden on data providers (within and outside Government), and confirm in the box provided that any issues have been resolved. This also needs to be noted in the risk assessment where appropriate. Where the intention is for contractors to recommend a method as part of a tender, or as work under contract, this should be made clear. The PQAB submission (and the ITT, if separate) should provide information on what has currently been decided about the method, what the methodological issues are, and the criteria by which tenders would be judged in relation to their contribution to the method. As far as possible at this stage, the intended impact of the research should be thought about, particularly in the context of

13.

Home Office researchers should not expect less of themselves than they expect of contractors, and hence the level of detail required in PQAB submissions should be similar. Indicate where alternative approaches, designs or methods have been considered and rejected, to record this for future reference and to avoid PQAB members raising questions that have already been answered. For example, if new data collection is proposed, have existing data sources within Government or in public archives been considered (taking account of data quality, validity, reliability, relevance and any restrictive conditions of using the data)? If the project is a review, say how the review will be carried out. If it will not be a systematic review, explain why not and say whether it will be carried out so as to form a basis for a future systematic review without repetition of work. For larger projects in particular (nominally those with a budget of over £250k including internal costs), serious consideration should be given to seeking external guidance when developing the project method and/or an ITT for example by convening a Research Advisory Group or by working with individual academics or researchers in OGDs. This would be less important in PQAB’s considerations if the project, although large, is simple in concept or substantially a repeat of an earlier project. Where the pool of potential contractors is small, the means of obtaining external advice needs to be selected with regard to the risk of further limiting the pool of contractors or giving unfair advantage to some of them. If material
15.

9.

14.

10.

11.

Dissemination and impact
16.

Guidance on completing the PQAB submission

3

Issue date: 20 7 05

Annex B
dissemination strategy. This can provide insight into the aims and direction of the overall project. It means identifying: • what impact is to be achieved (i.e. what change, where and with whom); • what will be the contribution of the research to that impact; • the target audiences who need to be reached/influenced; • the best ways of achieving the required impact; • assessment criteria: how the success of the project will ultimately be assessed.
17. 22.

Similarly, PQAB is not responsible for assessing the impact of the proposed research on different people and/or organisations. Unless diversity is itself the subject of the research, the assessment should be the responsibility of those who have commissioned the research. Therefore the PAR states that the GEB-nominated signatory indicates, by signing, that diversity issues have been appropriately considered. Similarly, the PD signals that s/he has identified any diversity issues in the project design and made the policy/operations lead aware. If the research is going to be contracted out, then the Project Leader should highlight potential or anticipated diversity issues in the submission. Tenderers should then be asked to address these issues. This question does not presume that such representativeness is always required. In some cases, for example, a representative sample may not be relevant to the research, e.g. in a study examining the views of offenders in only one ethnic group. The risk assessment should show that the Project Leader has understood all the significant project risks and has a plan for managing them. In carrying out a risk assessment, Project Leaders must be open to the conclusion that the project should not go ahead. Therefore, the assessment should be commenced as early as possible in the planning of the project. Record as bullet points any significant external/internal issues that might affect the project or constrain its scope. Confidentiality issues should be included where appropriate. Use this list to compose the list of hazards in the following table. Issues to consider include: • temporary or permanent loss of staff; • problems with access to respondents; • unsatisfactory response rates; • non-availability of records, materials or data; • over-run on time or funds; • poor cooperation from third parties (e.g. contractors, data providers, OGDs, agencies, NGOs); • changes of legislation, processes or administrative systems;
4 Issue date: 20 7 05

The approaches considered need to go beyond publications to other means of dissemination (e.g. seminars, training courses, releasing project data, developing a project website) and working with Home Office colleagues or other stakeholders on impact through policy or operations. Therefore, PQAB submissions should note the expected outcomes, what impact these are likely to have on policy, practice and key interest groups, and how the impact is to be achieved and/or mitigated. The needs of stakeholders should be mentioned, together with any potential for adverse impact on stakeholders. Include any handling issues that can be anticipated at this stage. If the expectations for outputs have changed since the PAR was completed, provide an update here. Where the concerns relate to work that is to be contracted out, the details of how they are to be addressed may be left to the contractor. However, in these circumstances, contractors should be subject to two requirements, i.e. that they: • confirm in their tender or expression of interest that they have an ethics procedure, by which their proposal would be vetted; • provide a copy of the decision arising from that procedure before commencing the part of the work that was subject to concern. The legislation on diversity issues is complex and frequently modified. Hence any potential legal problems should be referred by the RDS PD for legal advice. PQAB may wish to note any concerns it has, but these remain the responsibility of the PD.

23.

24.

Risk assessment
25.

18.

19.

Ethical considerations
20. 26.

Diversity issues
21.

Guidance on completing the PQAB submission

• 27.

media intervention.

Annex B Resource plan
37.

The hazard is what could go wrong and have an adverse effect on the project. Add as many lines to the table as are needed. Repeat the table headings if the table runs over more than one page. Use footnotes if you need to include more text than comfortably fits in the table columns. Assess P - the probability that the hazard will in fact arise in the course of this project: write H for high, M for medium or L for low. Assess I - how serious the impact would be if the hazard did arise: write H for high, M for medium or L for low. State what will be done to reduce the likelihood of the hazard arising and/or the seriousness of its impact. State how the project will be monitored for the hazard arising so that the project leader will know as early as possible if something is going wrong. State the effect on the project if, despite the plans to mitigate the risk, the hazard does arise, and whether there is any contingency, i.e. further action that might then be taken (and by whom) or some residual value in the project. The table is a minimal format with the minimum targets required: add rows as necessary. Every project needs a clear project plan, showing how the various parts of the project relate to each other and the timescale for each part. The format and complexity of the plan will depend on the type and complexity of project but a Gantt chart and/or flowchart is usually necessary and should be added as Attachment E and referenced in the text. All projects should be broken down into stages relating to standard business years. Payments on contracts will be linked to targets such as interim outputs and will normally be made on a quarterly basis: this should be shown in the project plan. Indicate what form of progress reporting will be needed and its frequency. Ensure that the plan gives you adequate opportunity to change the direction of a project, or terminate a contract, with clarity on payments due, at any stage. Arrangements for updating all parties connected with the project should also be covered.

28.

The resource plan must provide, as a minimum, a breakdown, by financial year, of the staff days and cost for each party (the Home office project team, the wider Home Office, OGDs, contractors, external stakeholders, or any other parties on whom the success of the research depends). This is given in the table above. If any additional material is provided, include it as Attachments F1, F2 etc. and reference it in the text. The resource plan should follow directly from the project plan to identify all the tasks that make up the project and assign internal and external resources to those tasks in a reasoned and consistent manner. PDs should ensure that the resource plan has been arrived at by reasoned application of an appropriate approach. ADs should satisfy themselves that an appropriate approach is being used (e.g. a spreadsheet with the correct nominal charging rates for RDS staff, with all parts of the work included, from planning to dissemination). The resource plan must include staff and other resources for project design, letting and managing contracts, and the implementation of the findings including publishing reports and other dissemination/impact activities. The resource plan should state the assumed start date of the project and of any contract(s). The resource plan is not to be disclosed to tenderers Note any specific skills that are required to execute the project. Training and development implications may also be outlined.

29.

38.

30.

31.

32.

39.

Project plan
33. 40.

41.

34.

42.

35.

36.

Guidance on completing the PQAB submission

5

Issue date: 20 7 05

Annex C AUDIT OF EXISTING RESEARCH AND RESEARCH CAPABILITY
1. Area Name 2. Contact for queries 3. Current work 4. Please list any current work under the relevant headings1: Small scale studies (Band 3):
• •

Name Telephone number Email

Less than 30 days internal staff time Less than £10K external budget Not for publication by the Home Office or associated agency or submission to the Minister

Larger scale studies (Band 4):
• • • More than 30 days internal staff time More than £10K external costs Outcome studies (including interim outcomes and reconviction studies) For publication through the Home Office or associated agencies or submission to the Minister2

5. Staff resource available for research or evaluation3 6. Staff time, according to grade, spent on each of the four Bands4 i. Routine data analysis (Band 1)5 ii. Performance Monitoring (Band 2)6 iii. Small scale studies (Band 3) iv. Large scale studies (Band 4):

RO

IO

IT/ Admin

PO

SPO

ACO

This form should be returned to Nina Marvan, Interventions Unit, Horseferry House, Tel 0207 217 0676 (email Nina.marvan@homeoffice.gsi.gov.uk) by 16th of September 2005.

See also attached PC. Studies which may be used to inform policy on a national basis or which reports on outcomes (including interim outcomes) of work with offenders. 3 Identify how many staff at each grade and the time allocated to undertaking research using FTE e.g. 0.2 FTE for each identified staff member. 4 Identify using FTE. If more than one member of staff, please identify for each staff member. 5 Routinely collected data used for operational purposes and for reporting as part of the business plan to the centre. 6 Data collection initiated locally and used to support local performance or to improve implementation. Data used for contract management.
2

1

Annex D

RDS/NOMs Research Seminar September 2005

As part of helping to introduce the new pro-forma and arrangements RDS NOMS will be running a seminar for all staff to attend, where possible. This will take place in the last two weeks of September, in London and will be a day event . Full details on the seminar will be circulated in due course. It is expected that the seminar will cover different types of research, research standards and briefing on how to complete the attached PQAB documents, using both presentations and workshops to help familiarise staff with the process. The seminar is directed at research and information staff or staff who manage or commission research. Please complete the form below indicating which members of staff wish to attend the seminar . This form should be returned to Nina Marvan, Interventions Unit, Horseferry House, Tel 0207 217 0676 (email Nina.marvan@homeoffice.gsi.gov.uk) by 1st of September 2005. Probation Area The following members of staff wish to attend the RDS Research Seminar Name Position Contact details (tel , email ) Dietary or other special requirements