The utilisation of evaluations – 391
But others are more upbeat. WFP’s study on its follow-up to evaluations carried outover a two-year period concludes that 88 per cent of the recommendations had beenimplemented or were in the process of implementation. Some two thirds of therecommendations had led to improved performance, although the reviewerrecommends caution, noting that ‘management units were almost always assessingperformance intuitively’ (WFP, 2005, p7). A 2002 study on the use of evaluations inthe European Commission found that, while the degree of use varied, ‘in no case were evaluations considered to be not at all useful’ (Williams et al, 2002, p12).The picture of utilisation that emerges from this and other studies is complex andoften unexpected. One of the few certainties is that how and why an evaluation iscarried out significantly affects the likelihood of it being used. The studies thatconstituted the background reading for this chapter provide a significant amount of information about factors that promote the utilisation of evaluations. Much of itresonates with van de Putte’s 2001 study, and key references such as MichaelPatton’s
, published in 1997.Information on use-promoting approaches is clearly available, but is thehumanitarian sector using it? And if not, why not? Does the main issue continue tobe the quality of evaluations (content and process), suggesting that the evaluationcommunity itself is not learning? Or are there other issues that undermine eventhe best evaluations? Given the position of evaluation as a primary tool foraccountability and learning, is it the right tool for the job? And what kind of job do we expect it to do?
3.1.1Structure of this chapter
This chapter has five main sections. Following this introduction (Section 3.1),Sections 3.2 and 3.3 draw upon existing studies of utilisation to describe the differenttypes of use made of evaluation, and examine a range of factors found to promoteuse. Section 3.4 examines four case studies and the findings of a series of interviewsand a questionnaire survey. These findings are used to expand on the evidence of the preceding studies, exploring the extent to which factors affecting use identifiedin the literature are borne out in practice. Section 3.5 considers the implications forthe future of learning and accountability mechanisms in the sector.