The role of event evaluation in the context of hosting international events

The purpose of this work is to show the role of event evaluation in the context of international events. This will be achieved by firstly outlining what evaluation is, particularly event evaluation its dimensions, importance and scope. Relevant examples will be presented to illustrate this. Then reasons for event evaluation implementation will be provided, focusing on the different impacts that events have. Appropriate methods and tools for the measurement of these impacts will be described. Following this, a critical discussion will be undertaken and the advantages and limitations of these methods will be examined. Finally, a conclusion will be drawn and recommendations will be offered. Experience shows that no management can function properly without having undertaken research and evaluation. This applies to event management as well. Information plays a key role, as it enables managers to measure, learn from and improve events in the future (Getz 2007). Event evaluation is a crucial process. It is the final part of the event management cycle (see Figure 1), and it allows event organisers to assess event outcomes promptly. Through evaluation, important statistics and feedback can be given, which will be useful for better and more efficient event planning and implementation (Bowdin et al. 2006). Figure 1: Evaluation and the event management process

(Bowdin et al. 2006)

Evaluation is the only way event managers can establish to what extent the set aims and objectives have successfully been accomplished. It can and should occur pre-, peri- and post-event (Allen et al. 2008); however, most businesses concentrate their efforts only on post-event evaluation (Hi2 Events 2012). In this process, economic impacts usually have priority (with social, cultural and environmental impacts often being ignored), because of the pressure from the media, government and other parties to justify costs, benefits or losses (Carlsen et al. 2000). There are many types of events, but those that are argued to offer an incomparable tourist attraction are the so-called special events (Getz 1989). Hallmark events are another kind in these, the place or time of occurrence are rarely repeated (Graham et al. 2001). Because of their very nature - offering unique experiences and escape from ordinary, daily life and their variance in size and duration - they can be expected to have numerous effects on local governments, economies, cultures and communities where they happen (see Figure 2). This in turn dictates the evaluation that needs to be done. One factor determining such assessment is the economic sector that a certain event falls into. It could be assumed that evaluation of a smaller international event in the private sector would not be as comprehensive and thorough as the evaluation of a mega-event, combining the public and third sectors (Anon. 2012).

Figure 2. Impact model of hallmark events

(Cox 1996) For example, S. Kratchanov and M. Georgieva, who are the founders of SEEME (South East European Music Event) held in Sofia, Bulgaria, share that the evaluation they undertake is only post-event, and is mostly economically oriented, as they have to present reports to their key stakeholders their sponsors. They acknowledge the need for more complex evaluation, including gathering opinions from visitors and participants, in order to be able to improve the event for the future; however, they explain the absence of such actions as being a result of lack of time, resources and money (personal communication, 4

March 29, 2012). Indeed, evaluation can be costly: nevertheless, this cost should be predicted and planned for when managers prepare their event budgets. Event evaluation should gain the same status as other fundamental management tools, because it is the ultimate assistant for event organisers to see where their events have been, make a decision regarding the future direction, and choose the vehicles for transport (Hall 1992). Events requiring continuous evaluation are all mega-events. They are valued for their benefits for host destinations in terms of exposure and image enhancement (Jeong and Faulkner 1996). Unfortunately it cannot always be guaranteed that mega-events will deliver these benefits. One of the best examples of such an event is the Olympics. With the upcoming London 2012 games, all promises made and legacies outlined are constantly being discussed, and what will actually be achieved is questioned. In order for a sport event to be given the go-ahead, it has to be financially feasible (Masterman 2004): therefore, the attention is mainly on economic impact studies. However, such studies can be extremely subjective and are often very misleading and manipulative (Matheson 2002). It is no secret that Tessa Jowell, the Olympics minister in the government of Tony Blair, persuaded Parliament to trust her projections for the 2012 Olympics, estimating a cost for the British taxpayers to pay of approximately 2.375bn, when in reality, the latest estimates pointed towards a figure of 12bn (Slavin 2011). Undoubtedly, hosting sport events has its advantages, although they are frequently exaggerated, while negatives, such as crime and disruption, increased traffic congestion and environmental downfall, are seldom mentioned (Lee 2001). Slavin (2011) also adds that by evaluating past Olympics, obvious reoccurring patterns become apparent, mostly those of empty promises and faulty commitments. Because of the size of the event, nothing can stay hidden for long. If any issues emerge, they come to light during the evaluation process and having negative impacts and damaging the image of the event itself and the host location. Usually the production of hallmark events is associated with showing off the host city, though this very frequently turns into showing up (the negative side of) the city (Cox 1996).

Not all mega events are found to leave negative imprints behind after they have been evaluated, and not all event evaluation is primarily concerned with economic outcomes only. The following two examples support this statement. The first one the European Capital of Culture (ECoC) event - is believed to help towards increasing tourism, enhancing the overall image and aiding urban regeneration (Langen and Garcia 2009). The 1990 City of Culture Glasgow event has been claimed as a true success, a role model for other bidding destinations (Garcia 2005). The author presents statistical data showing 90 per cent favourable relation to the event, highlighting the consequent citys appearance transformation (31 per cent), economic benefits (19 per cent) and increasing the number of tourists (17 per cent). The second event to be discussed is the Sydney World Youth Day in 2008 the Catholic Churchs event devoted to youth. Religious tourism is important, particularly religious pilgrimage, as it brings positive economic outcomes (Rotherham 2007). The results from the implemented post-event evaluation show approval towards the event and the pilgrims by Sydneys residents, and positive social and cultural impacts such as local community pride and self-esteem are achieved (Locke and Zahra, 2011). There are many reasons for event evaluation to be undertaken, some of the most obvious being recognising and acting upon issues in the event planning, and providing recommendations so the latter can be resolved, but also for successful event activities to be preserved and enhanced (Salem et al. 2004). The pre-event evaluation standardises event information and allows objective and complete analysis of important aspects, such as demographics, event reputation and history and sponsorship benefits and costs (Madsen and FlynnWakeling 2006, p.5). Peri-event evaluation is concerned with monitoring and controlling the on-going processes, while post-event evaluation gives important information that can be used as a blueprint to provide a solid foundation on which to build future events (Allen 2009). Looking at event impacts in more detail, others emerge as well as the apparent positive and negative ones. Mason and Beaumont-Kerridge (2004) argue that most event research tends to minimise the significance of non-

economic impacts. Speaking of environmental impacts, they are rarely if ever addressed, although there is a legislated environmental impact assessment (EIA) provided for this purpose (Cox 1996). As events can be a significant source of pollution, it is crucial that all potential negative impacts, such as litter, noise, carbon footprint, etc., are closely considered in advance. Similarly, negative political impacts, such as protests and demonstrations, might occur (Hall 1992), thus affecting the image of the event and of the host destination. Moreover, Haralmbopolous and Pigram (1996) explain how visitors might cause disruptions in the daily lives of local residents, using an event as an excuse for their actions, thus affecting the social order of the community. Social impacts are complex and often directly related to healthcare services, accommodation, transport, sport, cultural activities, art, employment, security, media, ticketing, access to information, visitors and residents attitudes and feelings and so on (Keys Young 1995). They might be difficult to estimate, because many cannot be quantified (Fredline et al. 2003) Nevertheless, they could be evaluated through Social Impact Assessment (SIA). A more comprehensive evaluating tool is the Social Impact Body Of Knowledge (SIBOK) model. It acknowledges the various social benefits and costs [ ] and [ ] If this process of information gathering continues over a longer term then it will facilitate the development of expertise, best practice and industry benchmarks (Wood et al. 2006, p.88). Events might have a tremendous social effect on a host destination. Especially challenging is the situation for residents, as they are the ones who must bear the consequences of an event from its beginning until its end (Boyko 2008), and often long after the event has finished. Social and cultural outcomes are often mentioned in the literature as a single term sociocultural impacts - suggesting that they are interlinked. Mason and Cheyne (2000) remark that these impacts are not easy to evaluate, as they are not as quantifiable as the economic ones. An interesting evaluation tool is the Mental Well-being Impact Assessment (MWIA) toolkit, which was developed in relation to the Liverpool Capital of Culture event, and aimed at discovering the positive and negative impacts on the mental well-being of the participants (West

et al. 2007). The results presented in Figure 3 show a vast spectre of sociocultural effects that an event might cause. Figure 3. Well-being Impact Assessment Results

(West et al. 2007) In some studies, authors mention hard (Wood 2005) and softer (Turok and Bailey 2004) assessments of impacts, suggesting that a hard method of evaluation would be a quantifiable one, as it is likely to carry more weight with policy-makers (p. 82). The softer approach would normally be qualitative, evaluating intangible benefits, outweighing the economic ones and promising long-term sustainability. Examples of such softer evaluation methods are semistructured interviews, focus and discussion groups, surveys, press impact analysis, mapping techniques, workshops etc. Their popularity among researchers (Andrews 2003; Garcia 2005; Hamilton et al. 2007; Mason and Beaumont-Kerridge 2004; Pattison 2006) proves their effectiveness; however,

there is still a lack of clear toolkits to evaluate the less tangible sociocultural impacts. Although intangible event impacts tend to be vague and hard to measure, they could still be evaluated. Dwyer et al. (2000) have come up with a framework that makes their accountability possible and is also a way for the tangible, economic benefits and costs to be identified. It is no secret that it is the economic outcomes studies that prevail in the literature. Most international events nowadays are public funded, and in order for governments to account for the given funds, they want to know what the economic returns would be (Faulkner 1993). There are vast numbers of different models, methods and tools for measuring economic impacts.

References: Andrews, N., 2003. Cultureshock, XVII Commonwealth Games North West Cultural Programme. Evaluation report: impact on organisations, audiences and the city/region. Manchester: Arts Council England. Available from: http://web.me.com/probe8/www.cultureprobe.co.uk/resources_files/culture shock%20report%20NA.pdf [Accessed 31 March 2012]. Allen, J., OToole, W., Harris, R. and McDonnell, I., 2008. Festival and special event management. 4th ed. Australia: Wiley. Allen, J., 2009. Event Planning: The Ultimate Guide To Successful Meetings, Corporate Events, Fundraising Galas, Conferences, Conventions, Incentives & Other Special Events. 2nd ed. Ontario: Wiley. Bowdin, G., Allen, J., OToole, W., Harris, R., and McDonnell, I., 2006. Events management. 2nd ed. Oxford: Butterworth-Heinemann. Carlsen, J., Getz, D. and Soutar, G., 2000. Event evaluation research. Event Management, 6 (4), 247257. Cox, G., 1996. Showing off or showing up the city? The social impacts of major events. Social Impact Assessment Newsletter, 40. Available from: www.gamesmonitor.org.uk/files/Cox%20Showing%20off%20the %20City.pdf [Accessed 30 March 2012]. Dwyer, L., Mellor, R., Mistilis, N. and Mules, T., 2000. A framework for evaluating and forecasting the impacts of special events. In: Allen, J., Harris, R., Jago, L. and Veal, A. Events Beyond 2000: Setting the Agenda, ed. Sydney: University of Technology.

Faulkner, B., 1993. Evaluating the tourism impacts of hallmark events. Occasional Papers, 22. Canberra: Bureau of Tourism Research. Available from: www.ret.gov.au/tourism/Documents/tra/Snapshots%20and %20Factsheets/OP%2016.pdf [Accessed 31 March 2012]. Fredline, L., Jago, L. and Deery, M., 2003. The development of a generic scale to measure the social impacts of events. Event Management. 8, 2337.

10

Garcia, B., 2005. Deconstructing the City of Culture: The long-term cultural legacies of Glasgow 1990. Urban Studies, 42(5), 841-868. Getz, D., 1989. Special events. Defining the product. Tourism Management, 10 (2), 125-137. Getz, D., 2007. Event studies: Theory, research and policy for planned events. Oxford: Elsevier Butterworth-Heinemann. Graham, S., Neirotti, L.D. and Goldblatt, J.J., 2001. The ultimate guide to sports marketing. 2nd ed. New York: McGraw-Hill. Hall, C.M., 1992. Hallmark tourist events impacts, management and planning. London: Belhaven Press. Hamilton, C., Galloway, S., Langen, F., Cran, A., MacPherson, C., Burns, M. and Snedden, E., 2008. Evaluation report: Scotlands Year of Highland Culture 2007. Glasgow: University of Glasgow. Available from: http://christinehamiltonconsulting.com/wpcontent/uploads/2011/10/CCPR_Highland_2007_Evaluation_Report_Oct_ 20082.pdf [Accessed 31 March 2012]. Haralmbopolous, N. and Pigram, A., 1996. Perceived impacts of tourism: The case of Samos. Annals of Tourism Research, 23(3), 503526. Available from: www.sciencedirect.com/science/article/pii/0160738395000755 [Accessed 31 March 2012]. Hi2 Events, 2012. Event Evaluation. Available from: www.hi2events.co.uk/assets/173/event_tip__event_evaluation.pdf [Accessed 30 March 2012]. Jeong, G.H. and Faulkner, B., 1996. Resident perceptions of mega-event impacts: The Taejon International Exposition case. Festival Management & Event Tourism, 4, 3-11. Keys Young, KPMG Management Consulting and King, A., 1995. Preliminary Social Impact Assessment of the Sydney 2000 Olympic and Paralympic Games. Sydney: Office of Olympic Co-ordination. Langen, F. and Garcia, B., 2009. Measuring the impacts of large scale cultural events: A literature review. Available from: www.liv.ac.uk/impacts08/Papers/Impacts0811

FLangen_and_BGarcia_May_2009_Events_Review.pdf [Accessed 30 March 2012]. Lee, S., 2001. A review of economic impact studies on sporting events. The Sport Journal, 4(2). Available from: www.thesportjournal.org/article/review-economic-impact-study-sportevents [Accessed 30 March 2012]. Locke, M. and Zahra, A., 2011. Are media reports representative of host community support for mega-events? The case of Sydney World Youth Day 2008. Event Management, 15, 279-292. Madsen, C. and Flynn-Wakeling, L., 2006. Sponsorships: Finding the Right Fit! Organization on Management Workshop. Available from: www.ibtta.org/files/PDFs/Madsen.pdf [Accessed 31 March 2012]. Mason, P. and Beaumont-Kerridge, J., 2004. Attitudes of visitors and residents to the impacts of the 2001 Sidmouth International Festival. In: Yeoman, I., Robertson, M., Ali-Knight, J., Drummond, S. and McMahonBeattie, U., ed. Festival and events management: An international arts and culture perspective. Oxford: Elsevier Butterworth-Heinemann, 311328.

Mason, P. and Cheyne, J., 2000. Resident attitudes to a tourism development. Annals of Tourism Research, 27(2), 391411. Available from: www.sciencedirect.com/science/article/pii/S0160738399000845 [Accessed 31 March 2012].

Masterman, G., 2004. Strategic sports event management an international approach. Oxford: Elsevier Butterworth-Heinemann.

Matheson, V.A., 2002. Upon further review: An examination of sporting event economic impact studies. The Sport Journal, 5 (1). Available from: www.thesportjournal.org/article/upon-further-review-examinationsporting-event-economic-impact-studies [Accessed 30 March 2012]. Pattison, H., 2006. Urban regeneration through the arts: a case study of the Edinburgh Festivals. In: Fleming, S. and Jordan, F., ed. Events and Festivals: Education, Impacts and Experiences. Eastbourne: LSA, 71-80.

12

Putnam, R. (1993) The prosperous community: social capital and economic growth. The American Prospect, 13: 3542. Available from: www.philia.ca/files/pdf/ProsperousCommunity.pdf [Accessed 31 March 2012].

Rotherham, I., 2007. Sustaining Tourism Infrastructures for Religious Tourists and Pilgrims within the UK. In: Raj, R. and Morpeth, N.D., ed. Religious Tourism and Pilgrimage Management: An International Perspective. Cambridge: CAB International, 64-78. Salem, G., Jones, E. and Morgan, N., 2004. An overview of event management. In: Yeoman, I., Robertson, M., Ali-Knight, J., Drummond, S. and McMahon-Beattie, U., ed. Festival and Events Management: An International Arts and Culture Perspective. Oxford: Elsevier ButterworthHeinemann, 14-32. Scottish Office (1999) Social Inclusion: Opening the Door to a Better Scotland. Edinburgh: Scottish Office. Available from: www.actsparl.org/media/23180/1-2%20-scpo%20briefing-social %20inclusion.pdf [Accessed 31 March 2012].

Slavin, M., 2011. World Class Games in East London. An introduction to the social impacts of the Olympics. Available from: www.gamesmonitor.org.uk/node/333 [Accessed 30 March 2012]. Turok, I. and Bailey, N., 2004. Twin track cities? Linking prosperity and cohesion in Glasgow and Edinburgh. Progress in Planning, 62 (3), 135 204.

West, H., Hanna, J., Scott-Samuel, A. and Cooke, A., 2007. Liverpool 08 European capital of culture: mental well-being impact assessment. executive summary. Available from: www.liv.ac.uk/ihia/IMPACT %20Reports/MWIA_Executive_Summary.pdf [Accessed 31 March 2012].

13

Wood, E.H., 2005. Measuring the economic and social impacts of local authority events. International Journal of Public Sector Management, 18(1), 37-53. Available from: www.ataland.com/Files/Articles/118.pdf [Accessed 31 March 2012].

Wood, E.H., Robinson, L.R. and Thomas, R., 2006. Evaluating the social impacts of community and local government events: A practical overview of research methods and measurement tools. In: Fleming, S. and Jordan, F., ed. Events and Festivals: Education, Impacts and Experiences. Eastbourne: LSA, 81-92.

14