Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword or section
Like this
P. 1
The utilisation of evaluations

The utilisation of evaluations

Ratings: (0)|Views: 9 |Likes:
Published by ALNAPEval

More info:

Categories:Types, Research
Published by: ALNAPEval on May 22, 2012
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





Chapter 3
The utilisation of evaluations
Peta Sandison
3 – ALNAP Review of Humanitarian Action90
Although evaluations are generally successful in identifying lessons and buildinginstitutional memory, only a minority of evaluations are effective at introducingevident changes or improvements in performance. If this continues, there is adanger that continued poor utilisation will undermine the credibility of evaluation asa tool for accountability and learning in the humanitarian sector.We do not know even how many evaluations are conducted, let alone how many areused. The source of concern regarding non-use in the sector is mostly anecdotal – from working observations by evaluation managers and users – and proxy, from theapparent lack of impact of evaluation on the sector’s performance. After all, if evaluations are intended to improve performance, then ‘the recurrence of many of the problems seen in Rwanda and other emergency responses’ observed in thesynthesis report of the Tsunami Evaluation Coalition (Telford and Cosgrave, 2006,p23) implies that evaluations are not doing their job.Evaluations of humanitarian action were rare in the 1980s. Twenty years on, thenumber has soared and evaluation has become one of the most visible features of the learning and accountability agenda. The growth continues, complemented by innovations such as joint and real-time evaluation, participatory evaluation and peerreview. We have become increasingly interested in evaluating our performance, butare we using what we find?The utilisation of research and evaluation has been a topic of lively debate in thedevelopment and public sector since the 1970s. ALNAP’s humanitarian membershiphas long been concerned about utilisation, first commissioning a study on the follow-up to evaluations five years ago (van de Putte, 2001). The concern persists.The results of studies by humanitarian agencies and donors on the use of theirevaluations are mixed. Descriptions of limited or absent use are more easily foundthan examples of good practice. In general the literature describes an inconsistentand, in some cases, a dismal record of evaluation use. A Sida study concludes that‘evaluations are useful to a very limited group of stakeholders. For a majority of stakeholders the evaluation process could just as well have been left undone’(Carlsson et al, 1999, p51).
The utilisation of evaluations – 391
But others are more upbeat. WFP’s study on its follow-up to evaluations carried outover a two-year period concludes that 88 per cent of the recommendations had beenimplemented or were in the process of implementation. Some two thirds of therecommendations had led to improved performance, although the reviewerrecommends caution, noting that ‘management units were almost always assessingperformance intuitively’ (WFP, 2005, p7). A 2002 study on the use of evaluations inthe European Commission found that, while the degree of use varied, ‘in no case were evaluations considered to be not at all useful’ (Williams et al, 2002, p12).The picture of utilisation that emerges from this and other studies is complex andoften unexpected. One of the few certainties is that how and why an evaluation iscarried out significantly affects the likelihood of it being used. The studies thatconstituted the background reading for this chapter provide a significant amount of information about factors that promote the utilisation of evaluations. Much of itresonates with van de Putte’s 2001 study, and key references such as MichaelPatton’s
Utilization-Focused Evaluation
, published in 1997.Information on use-promoting approaches is clearly available, but is thehumanitarian sector using it? And if not, why not? Does the main issue continue tobe the quality of evaluations (content and process), suggesting that the evaluationcommunity itself is not learning? Or are there other issues that undermine eventhe best evaluations? Given the position of evaluation as a primary tool foraccountability and learning, is it the right tool for the job? And what kind of job do we expect it to do?
3.1.1Structure of this chapter
This chapter has five main sections. Following this introduction (Section 3.1),Sections 3.2 and 3.3 draw upon existing studies of utilisation to describe the differenttypes of use made of evaluation, and examine a range of factors found to promoteuse. Section 3.4 examines four case studies and the findings of a series of interviewsand a questionnaire survey. These findings are used to expand on the evidence of the preceding studies, exploring the extent to which factors affecting use identifiedin the literature are borne out in practice. Section 3.5 considers the implications forthe future of learning and accountability mechanisms in the sector.

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->