Executive Summary

Interim Report of the Evaluation of the Basic Skills Pathfinder Projects Executive Summary

Caroline Hudson, Gráinne McMahon, Geoff Hayward, Colin Roberts and Rosa Fernández
University of Oxford, Probation Studies Unit and the Centre for Criminological Research in conjunction with University of Oxford, Department of Educational Studies and SKOPE

November, 2001

The full interim report can be requested from Robin Elliott at the Research, Development and Statistics Directorate (robin.elliott@homeoffice.gsi.gov.uk).

Executive Summary

Key Findings

•

Screening 31% of the sample at Pre-Sentence Report (PSR) stage (n=3300) were
likely to have basic skills below level one. 37% of those offenders who received a Community Rehabilitation Order, 23% who received a Community Punishment Order and 35% who received a custodial sentence were likely have basic skills needs below level one (functional literacy level). The data indicate significant associations between basic skills needs, employment status and risk of reconviction.

•

Initial Assessment 66% of those assessed using the Basic Skills Agency’s Initial
Assessment (n=96) were likely to be below level one in literacy. 49% of the same sample are below level one in numeracy. 77% of those assessed using a measure of non-verbal ability (Raven’s) were likely have a non-verbal ability which is below average. It is emphasised that the Initial Assessment sample is not representative of the offending population as a whole.

•

Perspectives on Screening at PSR Stage The evidence has shown that it is
feasible to screen for basic skills at PSR stage. Appropriate training should be provided for those conducting screening.

•

Perspectives on Initial Assessment Basic skills tutors thought that in the future
an initial assessment should consist of a standardised and validated assessment of basic skills, some detailed background information particularly on previous educational experience, and probably an assessment of non-verbal ability.

•

Basic Skills Provision Appropriate use should be made of both in-house and
external provision. Effective links developed by pathfinders with external providers should be extended. Consideration should be given to increasing the amount of inhouse group provision and, where appropriate, providing more accommodation for in-house group teaching. It should be ensured that tutors have access to Information and Communication Technology (ICT) to use in their teaching, and that paper-based and ICT teaching resources are aligned to the Basic Skills Standards. Effective strategies developed by pathfinders to motivate offenders to take up and attend regularly at basic skills sessions, should be extended. Tutors should have access to continuing professional development.

•

Effective Infrastructure The system for referral should be consistent across local
offices and individual probation staff, flexible enough to identify any offenders who may have been missed at an earlier stage, and communicated to relevant staff, to minimise attrition between screening at PSR stage, Initial Assessment at the start of supervision and take up of basic skills provision. Training on basic skills should be provided for probation staff. The evidence suggests that referral should be a designated part of someone’s role, senior management should actively promote basic skills, and basic skills should be incorporated into the case manager’s role. In developing basic skills practice, probation areas need to make effective use of monitoring and evaluation data.

2

Executive Summary

Executive Summary
The Basic Skills Pathfinders
There were seven basic skills pathfinder projects, originally based in nine probation areas (Cornwall; Cumbria; Dorset; Lincolnshire; Nottinghamshire; Oxfordshire and Buckinghamshire, and Berkshire; and East and West Sussex). Following amalgamation in April, 2001, the seven pathfinder projects were based in seven probation areas (as Oxfordshire and Buckinghamshire, and Berkshire are now Thames Valley Probation Area, and East and West Sussex are now Sussex Probation Area). Pathfinders were funded from late 1999. The pathfinders were set in a context of large-scale national developments in adult basic skills. There were three main strands to the basic skills pathfinder evaluation:
Screening for Basic Skills Needs at PSR Stage

•

This provided an indication of likely basic skills needs below level one (functional literacy level). The Basic Skills Agency’s (BSA’s) Fast Track 20 Questions (FT 20 Q) was used. Information on offenders’ demographic, criminogenic and educational characteristics was collected using the Background Information (BI) form. This assessment was completed on those offenders whose screening results indicated likely basic skills needs below level one, and who received a Community Rehabilitation Order (CRO) or a Community Punishment and Rehabilitation Order (CPRO). This assessment consisted of the BSA’s Initial Assessment, for basic skills levels; Raven’s Progressive Matrices, for non-verbal ability; the Initial Assessment Information Form, for detailed background information; CRIME-PICS II, for attitudes to crime and perception of problems in everyday life; and the Social Problem Solving Inventory, Revised (SPSI-R) (shortform), for problem-solving skills. The pathfinders were structured so that offenders whose initial assessment results were below level one were referred to basic skills provision. Whilst there were many similarities between basic skills provision across the pathfinders, there was no uniform model of provision.

Initial Assessment at the Commencement of Probation Supervision

•

Basic Skills Provision

•

Preliminary Findings from Screening Data Collected at PSR Stage
These preliminary findings are from data collected at PSR stage using the self-report BI form and FT 20 Q. The findings are based on a sample of 3300 offenders whose screening data have been entered into the SPSS database. In the final report screening data for over 7500 offenders will be analysed. This means that these preliminary findings should be treated cautiously.
Characteristics of the Current Sample (n=3300)

•

Most offenders (86%) were male and the sample was fairly young with 45% aged 25 years or below. Ninety-four per cent classified themselves as white. 3

Executive Summary • • Thirty-eight percent were in a ‘low’ risk category on Offender Group Reconviction Score (OGRS) 2 scores, 37% were in a ‘medium’ risk category and 22% were in a ‘high’ risk category. Twenty-five percent were convicted of violent offences, 21% were convicted of motoring offences, 20% were convicted of theft, 7% were convicted of drugs offences, 6% were convicted of burglary and 6% were convicted of fraud and forgery. (The remainder was convicted of sex offences, robbery, criminal damage and ‘other’ offences.) Thirty-seven per cent received a CRO or a CPRO, 23% received a Community Punishment Order (CPO) and 27% received a custodial sentence. (The remainder received ‘other’ sentences, such as a fine.) Fifty-seven percent attended school regularly and 44% attended irregularly. It is not unsurprising then that Figure 1 shows that 49% have no qualifications, 24% have level one qualifications (GCSE grade D-G or level 1 NVQ), 20% have level two qualifications (GCSE grade A*-C or level 2 NVQ) and 6% have level three qualifications or above (A Level or above).

• •

Figure 1: Offenders' qualifications as reported at PSR stage (n=3112)
60 50 40 30 20 10 0
no qualifications level one qualifications level two qualifications level three qualifications or above percentage

level of qualifications

• •

Thirty-seven percent were unemployed, while 2% were in training/ education, 37% were employed, 15% were out of work because of incapacity, and the remainder (9%) were in ‘other’ employment, e.g. full time carer. Overall, 31% were likely to have basic skills needs. 37% of those offenders who received a CRO or CPRO, 23% who received a CPO and 35% who received a custodial sentence were assessed as likely to have basic skills needs.

Trends in the Data

• •

FT 20 Q score is associated with the age of the offender, school attendance, highest level of qualification achieved, and employment status. Employment status is also associated with the age of the offender, school attendance, and highest level of qualification achieved.

4

Executive Summary • Risk of reconviction (as measured by OGRS 2) is associated with gender, the age of the offender, highest level of qualification achieved, FT 20 Q score and employment status.

Preliminary Findings from Initial Assessment Stage
These preliminary findings are based on the small sample of the BSA’s Initial Assessment (for basic skills’ levels) and Raven’s Progressive Matrices (for non-verbal ability) entered into the SPSS database to date. Given the size of the sample, the findings below should be treated with caution. Furthermore, it should be emphasised that these findings do not represent the offender population as a whole. There will be a thorough analysis of all completed initial assessment tools returned to the evaluation team in the final report.
The BSA’s Initial Assessment (n=96)

•

• •

Fifty-one percent were below level one in reading, 74% were below level one in spelling and 71% were below level one in punctuation. For research purposes, these scores were combined to give an overall literacy level. 66% were below level one in literacy. Forty-nine percent were below level one in numeracy. Using the categories developed by Raven for assessing non-verbal ability (Raven et al., 2000) for the general population (intellectually superior, definitely above average, average, definitely below average, intellectually impaired), 1% is definitely above average, 23% were average and 77% were definitely below average.

Raven’s Progressive Matrices (n=103)

Perspectives on Screening
Main Findings

•

•

• •

•

From the evidence of the basic skills pathfinder, it seems feasible to screen for basic skills needs at PSR stage. However, evidence has not been collected on whether the screening process would be more effective at PSR stage or at the start of supervision. Although FT 20 Q is not a flawless screening tool, it has proved acceptable to most users within the basic skills pathfinders. New screening tools will be reviewed by the National Probation Directorate (NPD) for strengths and weaknesses and for potential appropriateness for use within probation. Evidence indicates that, despite the demands of heavy workloads, it is feasible for Probation Officers (POs) to screen for basic skills needs. If POs screen for basic skills, then there is a training implication, to ensure that screening is carried out as accurately as possible. Whilst most interviewees thought that the training they had received to screen at PSR stage was good, it was suggested that future training could include a greater focus on strategies to motivate offenders to engage in the screening process. The fact that five probation areas have continued to use FT 20 Q to screen at PSR stage, after the collection of screening data for evaluation purposes finished (September, 2001), suggests that the majority of those involved in the basic skills 5

Executive Summary pathfinder think that the screening process has contributed to improved practice with basic skills.

Perspectives on Initial Assessment
Main Findings

•

• • • •

•

The responses of those involved in initial assessment suggest that, for those offenders whose basic skills are likely to be below level one, in the long term initial assessment should be a briefer process than has been the case during the evaluation. The version of the BSA’s Initial Assessment used in the evaluation is now out of date. The strengths and weaknesses of new tools will be carefully reviewed by NPD to select those most appropriate for use within probation. Tutors have found Raven’s helpful in informing learning plans. The feasibility of assessing speaking and listening should be explored, including potential methods to assess speaking and listening. A shortened version of the Initial Assessment Information Form should be made available for use by probation and basic skills staff. Preferably, this information should be collected by a ‘specialised’ assessment tool additional to the Offender Assessment System (OASys) One or Two. Training should be provided on how assessors can access relevant background information about a learner with sensitivity. Basic skills tutors thought that in the future an initial assessment should consist of a standardised and validated assessment of basic skills, some detailed background information particularly on previous educational experience, and probably an assessment of non-verbal ability.

Basic Skills Provision
Given the lack of previous research on what constitutes effective basic skills teaching, it is difficult to draw firm conclusions in this area. However, the following points have emerged through direct observations of basic skills teaching and interviews with tutors. It should be noted that these conclusions do not necessarily reflect Home Office policy.
Main Findings

• • • • • •

It is clear that the basic skills pathfinder projects have benefited greatly from the expertise and dedication of those responsible for basic skills tuition. There are examples of innovative methods to ensure that those offenders, for whom transport to sessions in probation or in the community is problematic, have the opportunity to improve their basic skills. The direct observations of basic skills teaching and practice revealed many examples of lively, well-structured teaching, with the use of a variety of resources, including Information and Communication Technology. Some project managers and tutors have been impressive in keeping up to date with resources and with developments in delivery methods. Some tutors moving towards more group, rather than purely one to one, provision. Many relationships between tutors and offenders were outstanding.

6

Executive Summary • • • • • • • Some tutors have developed an impressive range of strategies to foster take up of and regular attendance at basic skills. There is increasing focus on building effective links with external providers. Many tutors and project managers are energetic in liaising strategically with relevant probation staff. Tutors and mentors are appropriately qualified and most tutors are keen to enhance their professional development. Most supervision of tutors and mentors, by their line managers, appeared to be of a high standard. There are examples of some excellent initiatives to embed basic skills in probation’s wider work. However, evidence from the pathfinders suggests that not all aspects of basic skills provision are of a consistently high standard across the probation areas involved. The issues below need to be considered. Strategic consideration should be given to clarifying what the central aims of basic skills provision for offenders are. There should be greater clarity about the aims of provision for offenders with basic skills at different levels of the adult Basic Skills Standards. Provision should be planned in order to meet aims. Aims and plans should be part of an action plan and should be reviewed regularly. There should be careful consideration of what constitutes worthwhile but realistic progress for offenders with basic skills below level one. In particular, there should be careful consideration of the extent to which it is appropriate for offenders with particularly low skills to work towards accreditation. Consideration should be given to the feasibility of developing a series of basic skills modules for learners below level one, where the content is relevant to this population group. Strategic consideration should be given to whether attendance at basic skills should be voluntary or enforceable. There should be more strategic planning about the extent to which basic skills provision should or should not be based within probation. Planning should include consideration of when different groups of offenders can cope with provision outside probation. Where tuition is provided by the probation area, accommodation should facilitate teaching and learning and should be large enough to accommodate groups. An increase in group provision should be considered. An increase in the diversity of provision should be considered. For instance, the development of ‘bite-sized’ courses, intensive provision or family literacy programmes should be considered. There should be imaginative approaches for how to deliver basic skills to those offenders for whom provision within probation offices is appropriate, but who live too far away to attend tuition in the probation offices where basic skills is provided, or in the community. 7

Strategic Planning

•

• •

• • •

Accommodation

•

Type of Provision

• • •

Executive Summary

Resources and Teaching

• • • • • • • •

Tutors should have sufficient access to computers and to ICT based learning materials, to use in their teaching. A greater use of ICT in tutoring should be developed. Tutors should ensure that they use a variety of resources which are relevant to learners’ needs, and there should not be an over-reliance on photocopied worksheets. Tutors should consider experimenting with a range of strategies for group teaching, to complement individualized learning in a group context. The incorporation of learning styles into teaching should be considered. Tutors should ensure that they incorporate the Adult Basic Skills Curriculum (BSA, 2001a) into their planning and teaching. A greater focus on speaking and listening within tutoring should be considered. Tutors should use the Adult Basic Skills Curriculum (BSA, 2001a) to help structure their teaching on this. Tutors should ensure that individual teaching sessions are part of a well structured learning programme, where the concept of progression over time is central. The Individual Learning Record should be used to facilitate this, and progress should be reviewed regularly. Use of motivational strategies to promote take-up of and attendance at basic skills should be adopted more widely, as appropriate. In particular, motivational techniques to promote take up of and regular attendance at basic skills should be developed. Where feasible, offenders should be able to attend provision for more than an hour a week. Tutors, whether employed through external partnership or by the probation area, should ensure that they liaise effectively with probation staff. Where possible, tutors should have access to the probation area’s IT system. There should be increased focus on integrating basic skills within the wider work of probation. Probation areas should focus on building effective links with external providers, so that as many offenders as possible can be supported into external provision. Tutors should have access to opportunities for CPD. There should be mechanisms, such as a website, group e-mail and regional meetings, through which tutors can share resources and can discuss issues related to the provision of basic skills within probation. There should be training for tutors about teaching basic skills within the context of probation.

Take-up and Attendance

•

•

Links with External Providers

• • • •

Continuing Professional Development (CPD)

• • •

8

Executive Summary

An Effective Infrastructure
Main Findings

The pathfinder evidence underlines that an effective infrastructure depends upon careful attention to detail in planning and implementing the system for referral, strategic roles, procedures for monitoring and evaluation and a training programme for basic skills.
System for Referral

• • • • • •

It should be ensured that the referral of offenders from screening to initial assessment and provision is systematic. Practice with referral should be consistent across individual offices and officers. At the same time, systems should be flexible enough to identify offenders who have been missed at an earlier stage. Referrals should be made as quickly as possible. E-mail should be used when possible. The system for referral should be communicated to all involved. Senior management’s involvement in basic skills should be considered and, where appropriate, increased. Part of the role of the individual responsible for basic skills (project managers, in the pathfinders) could include the promotion of basic skills, proactive liaison with relevant individuals at a range of different levels over basic skills and monitoring of data on basic skills. The role of the senior probation officer could include promotion of basic skills at team and individual officer level by ensuring team meetings are a vehicle for ongoing training on basic skills, (where appropriate) cascading training on basic skills, and monitoring data completed by team members on basic skills. There is some evidence from the pathfinders to suggest that the person responsible for referral should have a vested interest in ensuring the referral is made. The role of the case manager could be developed. For example, strategies to motivate the offender to take up and attend regularly at basic skills and strategies for proactive liaison with the basic skills tutor could be explored. There should be adequate resourcing of administrative support for basic skills. Administrative staff should be involved as appropriate in training and in meetings related to basic skills. Monitoring and evaluation should include the quantity and quality of screenings completed; the number of referrals to initial assessment, initial assessments completed and referrals to provision; attendance at basic skills sessions; and the inclusion of basic skills in the PSR and supervision plan. To maximize the number of screenings and initial assessments completed, monitoring could take place at individual probation office and officer level. Training should be provided to raise awareness of basic skills, to ensure that all those involved in screening and initial assessment know exactly what their role 9

Strategic Roles

•

• • •

Monitoring and Evaluation

•

•

Training

•

Executive Summary consists of, and to clarify to all those involved the infrastructure for referral from screening to initial assessment and provision. Training on basic skills, appropriately tailored to the audience, should be available for everyone in probation. Training should be accompanied by written guidelines, as appropriate, on the system for referral, how to administer assessment tools and the inclusion of basic skills in the PSR and supervision plan. Training should be ongoing and probation areas should put into place arrangements to train new staff.

• • •

The Final Report
The final report will include an analysis of:

• • •

• •

•

•

All screening data returned to the evaluation team. All data on offenders’ characteristics, collected at PSR stage. All initial assessment tools returned to the evaluation team. This should enable detailed profiles to be developed of all offenders who receive a CRO or CPRO, whose basic skills are likely to be below level one, and who complete initial assessment. Take-up of and attendance at basic skills. Whilst the interim report includes an overview of attendance trends, the final report will include an analysis of attendance statistics collected between August 2001 and March 2002. Progress in basic skills made by individual offenders. The interim report discusses the amount of progress in basic skills it would be reasonable to expect within a community sentence, and provides some case studies of individual offenders which illustrate the extent of their progress in basic skills. However, the final report will include an analysis of progress through offender interviews; analysis of the Individual Learning Record; comparison of the results of initial assessment and assessments conducted at the end of basic skills provision (a parallel version of the BSA’s Initial Assessment, SPSI-R (short-form) and CRIME-PICS II); and an analysis of intermediate outcomes (offenders’ participation in education, training and employment). Comparison group data (i.e. the PSRs, supervision plans and quarterly reviews of 50% of offenders who receive a CRO or a CPRO, whose FT 20 Q results indicate their basic skills were likely to be below level one, but who did not attend basic skills provision). The cost-benefit evaluation.

10