The Howard Journal Vol 43 No 3.

July 2004
ISSN 0265-5527, pp. 267–283

Measuring the Impact of Crime

Reduction Interventions Involving
Sports Activities for Young People

GEOFF NICHOLS and IAIN CROW

Geoff Nichols is Lecturer in Leisure Management and Member of the Centre for
Criminological Research, University of Sheffield; Iain Crow is Reader in
Research Methods and Member of the Centre for Criminological Research,
University of Sheffield

Abstract: This article considers the complexity of measuring the impact on crime
reduction of different types of intervention with young people, which use the medium of
sports activity. It draws on the authors’ experience of sports-related programmes.
Interventions are categorised using a combination of Brantingham and Faust’s (1976)
categorisation of programmes as primary, secondary, or tertiary prevention and the
mechanism by which the intervention is likely to work (Pawson and Tilley 1997). This
shows that the impact of some types of intervention is not only inherently more difficult to
measure, but also demands far more resources to do so. This applies particularly to
interventions such as the Youth Justice Board supported summer Splash programmes,
which involve casual participation, and target geographical areas rather than
individuals. The relationship of the categorisation to the technical and practical
difficulties of measuring impact shows that the ideal of evidence-led policy is not easy to
achieve, however, using this categorisation, the article makes some suggestions for
evaluation methods, based on experience. The article also shows that in some cases the
resources required to produce the evidence exceeds those available to many programmes.
The implications of this are considered.

In recent years there have been various attempts to examine whether

sports and leisure activities for young people have an effect on youthful
offending. The results have been mixed. A study of the West Yorkshire
Sports Counselling Programme (Nichols and Taylor 1996) showed that
such a programme could reduce offending by those taking part, although
this was qualified by drawing attention to the specific circumstances of the
programme, notably the fact that it took place over a longer period of time
than most such programmes. However, this study stands out because there
is a dearth of evidence elsewhere of any similar effect. Utting’s (1996)
review concluded that: ‘the case for investment in criminality reduction in
the UK would be strengthened if future development work in this area
were subject to independent rigorous evaluation’ (p.81). More recently,
Long et al. (2002) reviewed a set of programmes designed to reduce social

267
r Blackwell Publishing Ltd. 2004, 9600 Garsington Road, Oxford OX4 2DQ, UK
and 350 Main Street, Malden, MA 02148, USA
exclusion, and as part of this, examined evidence for crime prevention.
Their general conclusion was that: ‘..there is little effective evaluation
against social inclusion outcomes’ (p.81) and, with specific reference to
crime reduction: ‘despite the pressure to demonstrate success, evidence
from the 14 projects is extremely limited’ (p.45). Typical was this comment:
‘for example, although the Leeds Football community Link project is
conceived in large part as a diversionary project, no information is
gathered on behaviour or offending patterns’.
Is this lack of evidence of crime reduction due to:

(i) the fact that sports and activities programmes don’t ‘work’, or
(ii) although sports and activities programmes have the potential to
reduce offending, they are poorly implemented, or
(iii) research is inadequate at establishing causality between programmes
and crime reduction in this complex research situation?

Our main concern in this article is with the last of these.

There has been much discussion about methods of evaluation. Some
believe that the only valid evaluation is one that conforms to the random
controlled trial (Farrington 1983; Farrington and Jolliffe 2002), often referr-
ed to as the ‘gold standard’ in evaluation. Others argue for a methodology
based on scientific realism rather than positivism (Pawson and Tilley 1997).
Others have suggested a ‘theories of change’ approach to evaluation, which
is based on studying ‘not only what the program is expected to achieve, but
also how it expects to achieve it’ (Weiss 1998, p.55).
Our own view incorporates elements of each of these. If one wants to
know whether something ‘works’, then it is important to have clearly
defined and measurable outcome criteria to show what it has achieved in
relation to its objectives. One wants to be able to attribute any changes that
have occurred to the intervention and not to something else. This is made
more convincing if we are able to compare like with like; in terms of
situations with and without a specific programme; or, through detailed
qualitative methods. We would also agree with Pawson and Tilley (1997,
p.5) that the black box methodology of the traditional OXO model is
inadequate by itself, and that to understand how and why something works
it is necessary to consider both the context of the intervention and the
mechanisms by which it operates. Both of these approaches have to be
based on an understanding of the theory that underpins the intervention
(Crow 2000, 2001), and in the present context this involves a consideration
of the likely relationship between leisure activities and crime reduction.1
We would also suggest that the particular form that an evaluation takes
needs to take account of the nature of the intervention itself. This is not to
suggest a relativist approach to evaluation; only that programmes and
initiatives take many forms, and that evaluation should be sensitive to this
and take account of it. As an example, Hollister and Hill (1995) point to the
problems inherent in evaluating programmes intended to have an impact
on communities rather than individuals. We would take this further and
suggest more careful identification of the type of intervention, the potential

268
r Blackwell Publishing Ltd. 2004
mechanisms, and the nature of the likely outcomes. As a further illustration
we now consider how one might go about adjusting one’s evaluation to give
consideration to the various forms that sports programmes for young
offenders, or those ‘at risk’, take.

Classification by Type of Crime Reduction Programme and

Mechanism of Intervention
Juxtaposing crime reduction programme and mechanism of intervention
gives a 3 by 3 typology in which example programmes will be situated.

(i) A Categorisation of Programmes

Brantingham and Faust (1976) categorise programmes in terms of three
types of crime reduction potential: primary, secondary and tertiary. Primary
reduction is directed at the modification of the criminological conditions that
are likely to give rise to offending. Sports programmes of this type would
operate at the level of attempting to improve the community and reduce
neighbourhood disadvantage as a risk factor. Programmes directed towards
secondary reduction focus on the early identification of, and intervention in,
the lives of those in circumstances likely to lead to crime. Sports programmes
that operate in this way would try to target ‘at risk’ groups of young people.
Tertiary programmes work with those who have already been identified as
offenders, and they seek to prevent recidivism. Therefore sports pro-
grammes that seek to do this are likely to take referrals from a criminal
justice agency such as a Youth Offending Team (YOT). Of course these are
pure types, and some programmes may operate in more than one manner.
As examples will show below, it is not always easy to place a programme in
one of the Brantingham and Faust categories. While they have value they
might more realistically be seen as points on a scale.

(ii) A Categorisations of Mechanisms of Crime Reduction

If sports programmes were to have an impact on crime, then it would be
reasonable to ask what it was about them that causes such change. Several
possibilities have been suggested (Nichols 2001a). These include the
possibility that such programmes quite simply and mechanically occupy
young people at a time when they might otherwise be getting into trouble.
They may also offer young people more positive options; they may bring
them into contact with non-offending peers, or with a sports leader who
acts as a role model; they may increase their self-esteem in such a way that
they no longer need to boost their self image by anti-social behaviour; they
may perhaps even lead to the kind of experiences and qualifications that
will enable them to get a worthwhile job. Broadly speaking these
mechanisms can be looked at as falling into three main types.
(a) Diversion
In such programmes the main intervention mechanism involves diverting
the person from a place or at a time when they might otherwise be involved

269
r Blackwell Publishing Ltd. 2004
in crime. It could also include a diversion from boredom, and thus a
reduction in the likelihood that they would get involved in crime. Thus, in
the summer Splash programmes, supported by Youth Justice (Splash
National Support Team 2002) the place and time the programme was run
would be important, but also the fact that it was run in the school holidays
when young people would otherwise be bored.
(b) Deterrence
The mechanism of deterrence works when the person thinks they are
more likely to be caught if they commit an offence. For example, if you run
a programme on school premises people are less likely to throw a brick
through the window when you are actually there. So this mechanism is
place and time specific.
(c) Pro-social development
This mechanism combines the theory juxtaposing risk and protection
factors (Catalano and Hawkins 1996), which underpins the Communities
that Care pilots in the UK (Communities that Care 1997), with a model of
self-development, facilitated by parallel increases in self-esteem, locus of
control, and cognitive skills, directed by pro-social values. This second
model was in turn developed from a study of sports counselling (Nichols
1999a), interviews with participants on a drug rehabilitation programme
(Nichols 1999b), and a review of theoretical work on the mechanism
through which personal development might arise from participation in
challenging outdoor activity programmes (Nichols 2000). Both models
offer a general explanation of the development of both anti- and pro-social
behaviour. Both models put a strong emphasis on the important influence
of value systems of ‘significant others’; whether peers, programme leaders,
or ‘social units’. Both have an emphasis on understanding the behaviour of
young people, and informing programmes directed towards them. Both
theories propose that a set of success factors, or ‘protection factors’, need to
be in place. Crompton and Witt (2003), in the United States, provide a
similar set of ‘characteristics of environments that promote positive youth
development’ (p.6).
The foregoing enable us to produce a typology of crime reduction
mechanisms, which may be exclusive, although one programme might
have elements of all three in different proportions. This typology can then
be used as a basis for considering the most appropriate way to evaluate
sports and activity programmes in terms of both process and outcome
evaluation. Example programmes have been inserted into the typology
below (Table 1), which will be discussed to illustrate these points.

Example of Programmes
Kirklees Splash (Primary/Diversion) is an open access sports activity
programme, aimed at 8- to 18-year-olds, targeted on the areas of Kirklees
which are most socially and economically disadvantaged. The sites used by
Splash are playing fields, parks, and fields or hard surface areas adjacent to
leisure and community centres. Paid staff lead a range of sports and games.

270
r Blackwell Publishing Ltd. 2004
 TABLE 1
A Typology of Programmes and Mechanisms

PRIMARY SECONDARY TERTIARY

DIVERSION (a) (b) (c)

Kirklees Splash Kirklees Podium
Many local authority
holiday schemes

DETERRENCE (d) (e) (f )

Epsom Parks for
All scheme
Kirklees Splash
on school sites

PRO-SOCIAL (g) (h) (i)

DEVELOPMENT Splash, (long term) Fairbridge Derby Positive
Futures
WYSC
Hafotty Wen
14 peaks

Participation is free. The programme has been run by a partnership of

West Yorkshire Police and Kirklees Leisure Services for over ten years and
is typical of many local authority run programmes (Nichols and Booth
2000), apart from its long-term continuity over the last ten years. In some
areas other activities are offered during the other school holidays and after
school in the summer term. There is the potential for regular participants
to attend a sports-related youth group run by the sports development staff
throughout the year, and also to become involved in voluntary sports
leadership. In 2001 and 2002, the programme was expanded with funding
from the Youth Justice Board. The Youth Justice Board (YJB) insisted that
all the programmes it funded nationally were also called Splash.
Kirklees Splash shows how the main mechanism can be extended.
Situating it where crime would otherwise take place can provide a direct
deterrent. This has occurred on some school sites where vandalism to the
schools has been reduced, at least while Splash is on. It has been suggested
that situating Splash in high crime areas and running it at times when most
youth-related crime occurs might be a way of targeting offenders (Loxley,
Curtin and Brown 2002), although again this shows some overlap of the
Brantingham and Faust primary and secondary categorisations. Splash
also has the potential to contribute to long-term pro-social development,
through participants initially acting as volunteers to help the younger ones,
and this leading to potentially taking sports leadership awards. This is
made possible by its long-term continuity.

271
r Blackwell Publishing Ltd. 2004
 Epsom Parks for All scheme (Primary/Deterrence) was instigated by
Epsom Recreation Department in response to local residents who were
apprehensive about using the parks, especially after the removal of on-site
park attendants. Residents complained about vandalism to the park and
young people drinking there at night. A presence in the park during the
day and evening was provided by letting an unused building in the park at
a pepper-corn rent to a martial arts dojo (centre for the instruction and
practice of martial arts).
Kirklees Podium (Secondary/Diversion) is again run by Kirklees
Recreation Department and is sold to Kirklees Youth Offending Team. It
was developed from Kirklees’s experience running West Yorkshire Sports
Counselling and similarly involves sporting activities on a one-to-one basis,
with a sports leader. The project has been operating since April 1998. Up to
April 2000, clients were referred from the probation service. After this date
clients came just from the Youth Offending Team (YOT) who had
contracted to buy 100 places in 2000/2001, on what was essentially the
same programme. One full-time member of staff is responsible for the
programme and works entirely on it, supported by a small pool of casual
staff. The programme is regarded as secondary intervention because,
although all the participants had been referred by the YOT, they had not
been convicted and the focus was on reducing the potential for this to
happen. In-depth interviews with the sports leader and a set of case study
clients showed that the programme was seen as primarily a diversion from
crime. The YOT officers used it because they felt it had a significant impact
on long-term development.
Fairbridge (Secondary/Pro-Social Development) is a charity which
works with disadvantaged and disaffected young people. All agencies
dealing with young people aged 14 to 25 years can make referrals to the
programme. These include the youth service and the probation service;
although some young people refer themselves. Fairbridge aims to develop
young people’s personal and social skills and build their confidence,
helping them to change their lives. A basic week-long course includes
centre-based preparation and up to three days at a residential location.
From this, participants develop a personal action plan for the next six
months. The activities the young person might choose to take part in after
that include workshops to develop life skills, programmes to develop skills
for the workplace, follow-on outdoor activity courses, courses run by
partner organisations such as The Prince’s Trust, and individual counsel-
ling sessions. Thus Fairbridge aims to work through the mechanism of
long-term pro-social development by intervention in the lives of young
people ‘at risk’, and in some cases, those who have already offended. This
shows that the Brantingham and Faust categories are an ideal type, with
some programmes overlapping categories.
Derby Positive Futures (Tertiary/Pro-Social Development) was one of 24
Positive Futures programmes supported by Sport England (2002a, 2002b).
It was run by Sport 2000, a registered charity with four trustees, instigated
by the Chief Constable of Derbyshire constabulary and a group of local
business men. Sport 2000 participants were referred from the local Youth

272
r Blackwell Publishing Ltd. 2004
Offending Team. The programme offered one-to-one sports counselling
over periods which varied depending on the willingness of participants to
take part. This could last as long as a year or more, since participants could
become involved in leading other groups run by the Sport 2000 project by
taking sports leadership awards. This would initially be on a voluntary
basis, but could lead to paid work. This programme was tertiary as it was
directed towards YOT clients who had a long history of offending.
West Yorkshire Sports Counselling (WYSC; Tertiary/Pro-Social Devel-
opment) (Nichols and Taylor 1996) operated from 1993 to 1996. It was
provided by a voluntary sector charity, West Yorkshire Sports Counselling
Association (WYSCA) and delivered by four sports leaders based in local
authority sports development sections. Probation service clients were
referred to the project by their probation officer although participation
was voluntarily. The basic programme was twelve weeks of sports activities
delivered on a one-to-one basis to each participant, involving one meeting
between the sports leader and participant each week. However, in several
cases sports leaders maintained informal contact with participants and this
allowed them to act as a broker of further sporting opportunities as well
as take a more mentoring role. This was a tertiary programme aimed at
pro-social development.
The Hafotty Wen 14 peaks programme (Tertiary/Pro-Social Develop-
ment) was run by what was, at the time, Merseyside Probation Service’s
residential outdoor centre in North Wales. It involved probationers
climbing all the 14, 3,000 foot plus high peaks in North Wales over 24
hours; a fairly demanding feat. The 14 peaks attempt was preceded by two
or three training events. Interviews conducted with some ex-14 peak
participants showed how the achievement of this expedition could provide
both self-esteem and a metaphor to help overcome difficulties in the future
(Nichols 1999b).

Methods in Relation to the Categorisation

The main purpose of this article is to show that different methods of
evaluation are appropriate and practical, depending on the programme’s
position in the typology above. We will look at just four examples, based on
the cells in Table 1.

Cell (a)
These programmes aim to provide primary crime reduction by a mechanism
of diversion. They are low risk, open access, and targeted by area.
A method, which the Youth Justice Board asked programmes they had
supported in 2000 and 2001 to implement, is to compare police records of
crime and incident data of the categories most associated with youth
offending in an area where a scheme has been provided, with the previous
year when it was not. One difficulty with this method is that the
geographical areas used in the collection of police records, called Police
Basic Command Units (BCU), may not match the areas covered by the
Splash scheme. Secondly, collating the evidence requires considerable

273
r Blackwell Publishing Ltd. 2004
resources beyond the capacity of most schemes, and requires a well-funded
research programme with guaranteed access to relevant data. Thirdly,
small numbers may make any change insignificant. Fourthly, a small
change in any one area could be due to just one or two persistent offenders
moving in or out of the area.
These general difficulties are illustrated by reports on the YJB funded
programmes in 2000 and 2001. The Cap Gemmini Ernst and Young
(2001) report on the 2000 YJB funded schemes reported that of 102
schemes, only 43 had provided crime statistics, although producing such
evidence was a condition of funding. The report on the 2000 schemes says
that: ‘comparing August 2000 with August 1999, figures where we have a
large enough sample to be confident in the results show: a reduction in
domestic burglary of 36%, a reduction in ‘youth crime’ of 18%’ (Cap
Gemmini Ernst and Young 2001, pp.3, 8). However it is not clear how
many of the 43 schemes who were able to produce crime data also showed a
large enough sample to be ‘confident in the results’.
In monitoring the 2001 programme, the 2002 Cap Gemmini Ernst and
Young report (2002) compared changes in reported crime in seven
categories of offence between the increased number of Splash scheme
areas and high crime BCUs. These BCUs were a way of comparing with
control areas where Splash had not been run. Again, most programmes
were unable to provide all the data required, and while the report cites
some positive results, it also concludes that: ‘the sample size of schemes
running in both 2000 and 2001 and variations in data reported by schemes
each year, rendered the results statistically insignificant’ (Cap Gemmini
Ernst and Young 2002, p.21).
These problems were apparent in Kirklees where figures comparing
crime in 2000 and 2001 for the three sites funded by the YJB in 2001 were
too small, even if there had been a positive change, to show anything
significant. Any change could always have been due to one or two regular
offenders moving in or out of the area. This illustrates a third limitation,
that such data do not tell us why any change might have occurred. It is
assumed that it is related to the schemes.
In an attempt to overcome these problems, two alternative methods
were used in the summer of 2002 by one of the authors. A self-completion
questionnaire sent to parents of participants asked them if they thought the
Splash programme reduced crime and if so why. It also asked the major
reasons why parents allowed their children to attend Splash. Secondly,
interviews were conducted with participants. Skill was required in handling
the interviews, especially the direct question about the impact on local
crime; but they did elicit some very frank responses. Thirty of the 63
participants interviewed in 2002 reported that Splash reduced crime either
at exactly the same time as it was on, or generally over the period of its
provision. For them, boredom was a major problem in the school holidays.
Thirteen participants from those interviewed in 2002 made remarks which
implied that they themselves had been prevented from getting in trouble.
For example: ‘we used to go out twocing cars (taking without owner’s
consent) and bricking windows’. Parents also commented that: Splash

274
r Blackwell Publishing Ltd. 2004
reduces crime while it is on, ‘because my son isn’t bored, he has something
to do with people he knows’. Other alternative research methods would be
to undertake a survey of local residents, and to interview local police and
other local agencies to measure changed perceptions of crime.
If the programme was targeted on those most likely to offend, in the way
suggested by Loxley, Curtin and Brown (2002), one might record who
attended the programme and see if they were on the records of the local
YOT. If they were, one would know that the programme had attracted
those who were known to have been involved in crime. An initial difficulty,
illustrated by the Kirklees Splash programme, is that in an open access
programme there are not always definitive records of attendance. In the
Kirklees programme, participants had to register by providing their name
and address if they were going to attend an activity which involved
transport away from the main site, but not otherwise. Thus one only had
definitive records of who had attended these, ‘off site’ trips. There were no
records of who had attended on a daily basis, and these were not possible to
keep, given the open access nature of the sessions. Secondly, if one was able
to establish who had attended, and for how many sessions; and if one had
access to the local YOT database to compile profiles of these regular
participants, one would need a point of reference for the local population
as a whole. For example, for any given age cohort in the programme
catchment area, what percentage had been involved with youth justice?
Having established this, one could then see if YOT clients were over- or
under-represented in Splash attendees of the same age. This would give an
indication of targeting. For there to be a statistically significant difference
between YOT involvement for any given age cohort in the local area and
the involvement of the same age of Splash participants, there would have to
be sufficient numbers of participants on any given Splash site. Observation
at Kirklees Splash sites in the summer of 2002 showed that there were up
to 60 participants on any one site at one time. But once one identified
‘regulars’ and took sub-samples by age cohort, numbers would be small.
Even if one identified Splash attendees as known offenders through Youth
Offending Team records (to which it is difficult for a researcher to gain
access), there is always the possibility that criminal activity will merely be
displaced to other times.
A combination of the methods above provides some indication of
programme effectiveness in crime reduction, but the resources required
are considerable and even then, small sample sizes may make changes
statistically insignificant. The questionnaires and interviews gave an
indication of how and why the programme might reduce crime, but did
not show by how much. They were able to inform development of the
programme, in particular, informing the balance between crime reduction
and sports development objectives (Nichols 2003).

Cell (d)
The example in this cell is the Epsom Parks for All scheme, a primary
diversionary scheme relying on the presence of martial arts dojos in the

275
r Blackwell Publishing Ltd. 2004
parks to deter crime. In this case, one of the authors was able to obtain
police records of reported incidents in each of the three parks involved, for
a year before and a year after the introduction of the dojo. Unfortunately,
for the purposes of the research, the number of incidents was very small.
For example, in one park, numbers of incidents were 8, 17, 17 in 1997,
1998, 1999 respectively. In this park the martial arts centre was opened in
1998. Another source of data was the park rangers’ records on vandalism.
These showed a reduction in reported vandalism in the park where a
martial arts centre had been opened in 1995. Reported incidents were 14,
10, 9, 5 in 1995, 1996, 1997, 1998 respectively. So, again, numbers were
small. This example has the advantage that both police records and ranger
records can be related to the specific geographical area, in contrast to the
BCU areas used for collection of data in relation to Splash, and were
routinely collected.
In this example, another source of data could have been surveys of local
residents’ perceptions of crime. One could argue that perceptions and
reality might differ, but in terms of influencing the policy of the local
authority, and thus the allocation of resources to the scheme, perceptions
are more important.

Cell (g)
Examining the long-term impact of the Kirklees Splash programme, how
can one show its impact on pro-social development? Characteristics of the
programme which make this difficult include open access for participants
(they can drop in or out at any time), the open-ended involvement of
participants (they may be involved for a few days or a few years), the variety
of ‘exit routes’ for participants which can be regarded as successful
outcomes, and the general targeting of young people ‘at risk’, the numbers
of which may be relatively small even when accurate records of participants
are available.
One can not use before and after measures of individual participants,
such as measures of self-esteem, changing relationship with the sports
leaders as mentors, or changed attitudes towards crime, because these
require ‘before’ and ‘after’ measures. In a situation where participation is
open ended, and records of participation are unreliable, it is difficult to
identify the group of participants who one would expect to have benefited,
and one may have only a small proportion of participants who completed
both ‘before’ and ‘after’ measures. Given the large number of participants
in many of these open access programmes, the physical environment in
which the programme operates, and the resources required, it is not
efficient to conduct ‘before’ measures with a large sample, many of whom
will not become regular participants over the two years, or cannot be
identified as such. Much of the data collected at the start of the study would
be wasted.
Even if one was able to do this, it would be impossible to set up
retrospectively a control group; matched by age, gender and history of
offending, to see if those attending the programme regularly had changed

276
r Blackwell Publishing Ltd. 2004
significantly in relation to those who had not. A further difficulty, discussed
in more detail with reference to cell (i), is finding an instrument to measure
such changes, which is suitable for this type of participant and situation.
In a similar way to measuring changes in the personal attributes
discussed above, one could examine recorded offending rates of
participants and non-participants over a two-year period. This would
require access to local YOT records. Programme success would be implied
if regular attendees had lower increases in offending rates than a matched
sample of non-participants. However, the same problems of setting up a
retrospective control group remain. In theory this would be possible if one
had access to all youth offending data in a specific geographical area.
However, the effort involved in such an analysis would be beyond the
resources devoted to research by a typical local authority. If one was able to
conduct such an analysis one would still be reliant on large enough sample
sizes to lead to statistically significant results. This would be difficult as once
one had identified a core of regular participants one would be working
with relatively small numbers. One option might be a meta-analysis of a
number of such programmes although a limitation, as with all records of
offending, is that they only record a small proportion of offences. A further
difficulty is that initial contact with a programme may lead to an ‘exit route’
involving long-term participation in another organisation, which confers
the same positive benefits as the original programme. For example, a
summer sports taster session may lead to a long-term involvement in a local
canoe club, but this will be unrecorded by the original programme unless
they are able to conduct extensive follow-up research into all participants.
The resources to do this are not available, and thus this successful
programme outcome would remain concealed. In such a situation it may
also be asked whether any impact should be attributed to the longer-term
involvement, or to the ‘taster’ session for getting the person involved to
start with, or the combination of the two. These difficulties reflect the
criticisms Pawson and Tilley (1997, p.5) have made of the classical
experimental research design. It is not practical in such situations, and
even if it was, it tells us insufficient about the process.
Evidence of the positive impact of participation on a young person’s
pro-social development can come from individual case studies but the
evidence here is of process, that is, how and why a programme has had an
impact on participants, rather than of outcomes such as a reduction in
crime participation by participants overall. Detailed records of participa-
tion are needed even to show how typical these case study participants
were, and therefore how many similar participants attended regularly to
the extent that one might expect them to benefit in the same ways. One of
the present authors conducted qualitative interviews with some long-term
Splash participants in the summer of 2002, but the research context
reduces the ability of such interviews to explore in depth the causal
relationship between the programme and long-term impacts. The
participants’ attention span and willingness to take part in an interview
for more than 15 minutes was limited, thus restricting the interviewer’s
ability to elicit details about the impact of the programme. For any one

277
r Blackwell Publishing Ltd. 2004
participant it was difficult to triangulate data from other sources, such as
parents or sports leaders. Even where there was sufficient detail, the results
show the relevance of this mechanism of pro-social development in a few
cases, showing that the programme has the potential to achieve this, the
most important programme characteristics leading to this, but not how
many participants have benefited in this way.

Cell (i)
Programmes in this cell tend to be easier to evaluate in terms of outcome
measures. They are working with known offenders, and are seeking to
enhance their pro-social development. They involve high-risk offenders
and intensive work. In such instances the main focus is going to be
individual reconviction, although it is unlikely that a good comparison
group will be easily available. There may also be a question about whether
there is a sufficient number of people involved. Nonetheless it may be
possible to use the Offender Group Reconviction Score (OGRS). Both
studies of West Yorkshire Sports Counselling, and Hafotty Wen used this
method. In WYSC, in spite of a small sample size, statistically significant
results were obtained showing that those participants who stayed with the
programme for eight weeks or more were less likely to be reconvicted than
expected (Nichols and Taylor 1996). However, in the Hafotty Wen case the
apparently positive result had to be qualified by the sample size, which
made results inconclusive (Nichols 2001a). However, if the mechanisms of
pro-social development are to be better understood, reconviction data
need to be supplemented by interviews with individual participants and
individual case studies. This was done in both the examples above. In both
cases it showed the great importance of a mentor relationship with a sports
leader or probation officer. The activities acted as a catalyst for this
developing. The relationship extended to supporting the participant
through critical ‘career transitions’ (Craine and Coles 1995) and a change
in self-concept consistent with a reduction in offending (Nichols 1999a).
This is the same type of change as observed by Maruna (2001), and
Graham and Bowling (1995) in studies of desisters from offending.
In this type of programme, it has still been difficult to apply ‘before and
after’ measures of characteristics of participants, as such measures,
especially if academically validated, have often appeared by programme
staff to be onerous to administer and counter-productive to establishing a
positive relationship with participants. These difficulties led to the failure
of such methods in an extensive study of probation service programmes
(Nichols et al. 2000). This difficulty has recently been overcome in an
evaluation of the programme offered by Fairbridge to disadvantaged and
disaffected young people. This has used an internally designed measure; a
14 statement questionnaire using 5-point likert scales. The measure has
been designed in consultation with Fairbridge staff and has been used by
sufficient participants to enable statistical tests to confirm it is measuring
the same thing at the two points it has been applied. Similarly, Neill, Marsh
and Richards (2003) have developed a ‘Life Effectiveness Questionnaire’

278
r Blackwell Publishing Ltd. 2004
which has been similarly validated, and used in the UK in an evaluation of
an Outward Bound programme offered to school children in Scotland to
improve academic performance. It was easier to administer this instrument
in the school environment, although Tucker (2003) has recently used the
same instrument in the context of a development training course in a
residential setting.

Implications
An initial implication of the categorisation above is that it can inform the
selection of research methods in a deductive approach, starting research
from theory. In any one programme, research starts from a general theory
about the mechanism involved and the outcomes one expects, similar to
Pawson and Tilley (1997) starting research from a particular context/
mechanism/regularity configuration.
One problem is that if you start by just looking for one mechanism, then
there is a danger that that is the only one you will find, and other
mechanisms will be ignored. Starting research from this basis can be self-
fulfilling. It is suggested that preliminary research is conducted prior to
evaluation in order to identify what type of evaluation is necessary. There
also needs to be regular checking of material to ensure that subsidiary
mechanisms are not being missed, and to verify whether the programme’s
mechanisms have altered. As shown by the example of the Kirklees Splash
programme, all three mechanisms might be involved. One of the present
authors started examining this programme to look for evidence of long-
term pro-social development, but found that the strongest evidence was of
diversion.
The ability of programmes to provide the evidence needed to evaluate
them will vary. This determines the evidence one can reasonably expect a
programme to provide itself, and that which can be provided by an
external researcher. Just because it is harder to produce evidence for the
effectiveness of programmes in boxes (a) and (g) in reducing crime, does
this mean there is less justification for funding them? A further implication
of the categorisation is in relation to use of the ‘theory of change’ approach.
The position of a programme in the categorisation will influence policy
makers’ judgments of what is acceptable evidence of effectiveness.
Resources are important. As Long et al. (2002) noted, evidence from
programmes is scant, because they do not have the funds or skills to
conduct their own evaluation, and a higher priority is to assure next year’s
funding to allow them to continue. It is possible for programmes to be
intensively evaluated by experts with greater resources than are available
to the programme management (WYSC is a good example, although in
this case the programme itself funded the evaluation, so it was buying
expertise). But in these cases, the generalisation of results will depend on
how precisely programmes can be replicated. It is possible to say that
certain ways of working are more likely to bring the desired results in
particular circumstances, (as the ‘what works’ doctrine) but the problem is
to define the circumstances. This reflects a general methodological

279
r Blackwell Publishing Ltd. 2004
difficulty of the critical realist approach, which is how to define a context-
mechanism-outcome configuration (Nichols 2001b).
A further implication is that the categorisation above can inform the
design of programmes. For example, the major mechanism of the Kirklees
Splash is diversion, so simply extending it from one week of the summer
holidays to five, as has been possible with Youth Justice funding, will
magnify this impact, as long as young people’s interest can be maintained.
Although it is targeted in areas of social deprivation it could be targeted
even more precisely at sites which suffer extensively from vandalism.
Splash also shows how the main mechanism can be extended. Situating it
where crime would otherwise take place could provide a direct deterrent.
Splash also has the potential to contribute to long-term pro-social
development, through participants initially acting as volunteers to help
the younger ones, and this leading to potentially taking sports leadership
awards.
There is one further consideration, and this is whether sports and
activities programmes are, or should be, developed with evaluation in
mind. Many sports and activities programmes have arisen on an intuitive
basis, according to a perceived need to respond to young people and their
environment. They have developed in an ad hoc manner, based on
subjective experience, rather than according to scientific principles. This
should come as no surprise, since they have usually been initiated by
enthusiasts and people whose areas of expertise are in leisure management
or some other field, rather than in criminology. However, over the last
decade, and especially over the last five years the dominant question
regarding interventions in crime reduction has been ‘what works?’. The
discussion above, while initially focusing on the difficulties of evaluating
programmes, with reference to the typology offered by Table 1, has also
tried to offer potential solutions.
The government’s Crime Reduction Strategy was launched in July 1998
(Home Office 1999). It is an initiative that aims to ensure that all the
programmes in operation are focused on those methods that have been
shown to reduce reoffending, and is supported by d21m, which is being
used to develop programmes for the Prison and Probation Services.
Programmes are evaluated so that successful ones can be formally
accredited, and a Joint Prison and Probation Accreditation Panel has been
set up to adjudicate on which programmes should be accredited. In order
to be recognised and accredited, programmes have to meet eleven
demanding criteria concerning their theoretical basis, selection of
participants, methods of operation, management, staffing and monitoring.
Their application has to be supported by five manuals: the Theory Manual,
the Programme Manual, the Assessment and Evaluation Manual, the
Management Manual, and the Staff Training Manual ( Joint Prison/
Probation Accreditation Panel 2000, 2001). This is required partly to
determine whether programmes are well structured and in accordance
with established crime reduction principles, but also to ensure that they are
capable of being monitored and evaluated. It may of course be argued that
Prisons and Probation Services are statutory agencies, run on public

280
r Blackwell Publishing Ltd. 2004
money, and should therefore meet more rigorous criteria than pro-
grammes that are developed by local non-statutory agencies with limited
and uncertain funding. However, if one is talking about the possibilities of
adequate evaluation, then the question has to be asked whether perhaps
there should be some corresponding accreditation process for community-
based sports and activities programmes. No matter how well intentioned,
should their continuation be based on their ability to offer at least prima
facie evidence that they are doing what they set out to do, or is this the tail
wagging the dog? If we only do what we can measure will we lose the
programmes which the typology in Table 1 has shown to be most difficult to
evaluate?

Note
1 In other contexts it might involve considering the relationship between unemploy-
ment and crime where the intervention takes the form of a training initiative, or the
nature of drug misuse where the programme involves drug treatment.

References
Brantingham, P. and Faust, F. (1976) ‘A conceptual model of crime prevention’, Crime
and Delinquency, 22(3), 284–96.
Catalano, R. and Hawkins, J.D. (1996) ‘The social development model: a theory of
antisocial behaviour’, in: J.D. Hawkins (Ed.), Delinquency and Crime, Cambridge:
Cambridge University Press.
Cap Gemmini Ernst and Young (2001) Summer Splash 2000: Final Report, London: Cap
Gemmini Ernst and Young (available at website: www.youth-justice-board.gov.uk/
policy/summer_splash_2000_ final_report.pdf ).
Cap Gemmini Ernst and Young (2002) Splash 2001: Final Report, London: Cap
Gemmini Ernst and Young (available at website: www.youth-justice-board.gov.uk/
policy/splash_2001.pdf ).
Communities that Care (no author) (1997) Communities that Care, London: Commu-
nities that Care (UK).
Craine, S. and Coles, B. (1995) ‘Alternative careers; youth transitions and young
people’s involvement in crime’, Youth and Policy, 48, 6–26.
Crompton, J. and Witt, P. (2003) ‘Positive youth development practices in recreation
settings in the United States’, World Leisure Journal, 2(45), 4–11.
Crow, I. (2000) ‘Evaluating initiatives in the community’, in: V. Jupp, P. Davies and
P. Francis (Eds.), Doing Criminological Research, London: Sage.
Crow, I. (2001) The Treatment and Rehabilitation of Offenders, London: Sage.
Farrington, D.P. (1983) ‘Randomised experiments on crime and justice’, in: M. Tonry
and N. Morris (Eds.), Crime and Justice, Vol. 4, Chicago, IL.: University of Chicago
Press.
Farrington, D.P. and Jolliffe, D. (2002) A Feasibility Study into Using a Randomised
Controlled Trial to Evaluate Treatment Pilots at HMP Whitemoor (Home Office Online
Report 14/02), London: Home Office.
Graham, J. and Bowling, B. (1995) Young People and Crime (Home Office Research
Study No.145), London: Home Office.
Hollister, R.G. and Hill, J. (1995) ‘Problems in the evaluation of community-wide
initiatives’, in: J.P. Connell, A.C. Kubisch, L.B. Schorr and C.H. Weiss (Eds.), New
Approaches to Evaluating Community Initiatives, New York: The Aspen Institute.

281
r Blackwell Publishing Ltd. 2004
Home Office (1999) What Works: Reducing Re-offending: Evidence Based Practice (Home
Office Communication Directorate), London: Home Office.
Joint Prisons/Probation Accreditation Panel (2000) What Works: First Report from the
Joint Prison/Probation Accreditation Panel, 1999–2000, London: HM Prison Service
(available at website: www.hmprisonservice.gov.uk/library/).
Joint Prisons/Probation Accreditation Panel (2001) What Works: Second Report from the
Joint Prison/Probation Accreditation Panel, Towards Effective Practice, 2000–2001,
London: HM Prison Service (available at website: www.hmprisonservice.gov.uk/
library/).
Long, J., Welch, M., Bramham, P., Butterfield, H. and Lloyd, E. (2002) Count Me In,
Department for Culture Media and Sport (available at website: www.lmu.ac.uk/ces/
lss/research/countmein.pdf).
Loxley, C., Curtin, L. and Brown, R. (2002) Summer Splash Schemes 2000: Findings From
Six Case Studies (Crime Reduction Research Series Paper 12), London: Home
Office.
Maruna, S. (2001) Making Good, Washington: American Psychological Association.
Neill, J.T., Marsh, H.W. and Richards, G.E. (2003) The Life Effectiveness Questionnaire
Development and Psychometrics, Sydney, Australia: University of Western Sydney.
Nichols, G. (1999a) ‘Developing a rationale for sports counselling projects’, Howard
Journal, 38, 198–208.
Nichols, G. (1999b) ‘Is risk a valuable component of outdoor adventure programs for
young offenders undergoing drug rehabilitation?’, Journal of Youth Studies, 2(1),
101–16.
Nichols, G. (2000) ‘Risk and adventure education’, Journal of Risk Research, 3(2),
121–34.
Nichols, G. (2001a) ‘The difficulties of justifying local authority sports and leisure
programmes for young people with reference to an objective of crime reduction’,
Vista, 6(2), 152–63.
Nichols, G. (2001b) ‘A realist approach to evaluating the impact of sports programmes
on crime reduction’, in: G. McPherson and G. Reid (Eds.), Leisure and Social
Inclusion: New Challenges for Policy and Provision (LSA Publication No.73) East-
bourne: LSA.
Nichols, G. (2003) ‘Kirklees Splash: balancing objectives of crime reduction and sports
development’, Vista, 8(1), 26–30.
Nichols, G. and Booth, P. (2000) ‘Crime reduction programmes supported by local
authority leisure departments’, Local Governance, 25(4), 227–36.
Nichols, G. and Taylor, P. (1996) West Yorkshire Sports Counselling: Final Evaluation
Report, Halifax: West Yorkshire Sports Counselling Association.
Nichols, G., Taylor, P., Crow, I. and Irvine, D. (2000) ‘Methodological considerations in
evaluating physical activity programmes for young offenders’, World Leisure and
Recreation, 42(1), 10–17.
Pawson, R. and Tilley, N. (1997) Realistic Evaluation, London: Sage.
Splash National Support Team (2002) Splash 2001 Final Report, London: Cap Gemini
Ernst and Young plc.
Sport England (2002a) Positive Futures: A Review of Impact and Good Practice, Summary
Report, London: Sport England.
Sport England (2002b) Positive Futures: A Review of Impact and Good Practice, Individual
Project Reports, London: Sport England.
Tucker, N. (2003) ‘Participants’ and practitioners’ experience of outdoor experiential
personal and social development’, in: B. Humberstone, H. Brown and K. Richards
(Eds.), Whose Journeys? The Outdoors and Adventure as Social and Cultural Phenomena,
Penrith: Institute for Outdoor Learning.

282
r Blackwell Publishing Ltd. 2004
Utting, D. (1996) Reducing Criminality Among Young People: A Sample of Relevant
Programmes in the United Kingdom (Home Office Research Study No.161), London:
Home Office.
Weiss, C. (1998) Evaluation, New Jersey: Prentice Hall.

Date submitted: October 2003

Date accepted: January 2004

283
r Blackwell Publishing Ltd. 2004