Higher Education Outreach to Widen Participation

Toolkits for practitioners

Evaluation
Presentation to Developing Your Access Agreements Conference February 2014 Tristram Hooley
International Centre for Guidance Studies University of Derby www.derby.ac.uk/icegs

Understanding evaluation
The Office For Fair Access (OFFA) has specified that access agreements should set out how institutions plan to monitor and evaluate WP activity and specify that resources should be set aside for this purpose (OFFA, 2012: 24).
HEFCE has also stressed the importance of evaluating WP activity, arguing that it is essential for the effective development of institutions strategies and practices in this area (HEFCE, 2010, Annex A). Evaluation should therefore be important to all HE providers, clearly acknowledged in WP strategies, and apparent in WP practice.

What is evaluation?
When people seek to evaluate what they are doing as part of an attempt to learn and improve, they are usually undertaking a formative evaluation, so called because it is undertaken to inform what is done while the activity is still in progress. We would like to find out how to do these things better When people evaluate to make a judgement on the value of a particular activity and to draw out what has been learnt, it is usually a summative evaluation; so called because it attempts to create a summary of what has been achieved and what the impacts have been.

We would like to find out how well these things work

Why evaluate?
Evaluation enables us to: examine what we do

think about how we can improve it

decide on whether it was worth doing provide others with a summary to help them to understand what was done.

Evaluation as part of reflective practice

Activity: Understanding evaluation

What is your organisations approach to evaluating WP activity? Is it embedded or occasional? Is it part of an organisational approach to all activities or just specifically related to WP? What are the advantages/disadvantages of this approach?

Levels of impact
Results Behaviour Learning Reaction

Planning an evaluation
R Reasons and purposes What are the Reasons and Purposes for this evaluation? Examples: planning, accountability, developing, to gain resources What will be the Uses of the evaluation? Examples: staff development, learning from good practice, strategic planning, PR, completion of Table 7 in the annual monitoring return to OFFA/HEFCE What will be the Foci for our evaluation? Examples: range of activities, emphasis, aspects of evaluation, identification of priority areas What will be the Data and Evidence? Examples: qualitative (interviews, case studies), quantitative (questionnaire) Who will be the Audience? Examples: HEFCE, senior management, other stakeholders What will be the Timing? Examples: once a year, at year end Who will be the Agency conducting the evaluation? Examples: staff developer, course co-ordinators, external facilitator

Uses

Foci

D A T A

Data and evidence Audience Timing Agency

Activity: Planning an evaluation activity using RUFDATA

Working in pairs using the RUFDATA framework consider one activity which you might want to evaluate. Whilst completing the activity consider how you might adopt this approach to inform the evaluation activities in your organisation.

Types of data for evaluation

Activity: Sources of data

In small groups think about the data which is currently gathered within your organisation and how this could be utilised to support your evaluation activity.

What new data might need to be gathered to support your evaluation in terms of input, participation and output data

Publish or perish!
Too little evaluation and impact work on widening participation is published.

Writing up your evaluation for broader circulation is an important way to support the development of the sector. Self publication Journal publication Partnership with academics Using external consultants

Activity: Taking evaluation work forward

In your groups or individually, prioritise three areas, activities or actions to take forward with regards to evaluating your widening partnership activity.

Bibliography
Dent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013). Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition. Bristol: HEFCE. Hooley, T., Marriott, J. and Wellens, J. (2012). What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury Academic. Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU. Moore, N., Christopoulos, M., Dimitriou, K. and Hutchinson, J. (2011). Researching the Aimhigher Greater Manchester Graduate Officer Programme. Derby: International Centre for Guidance Studies.

Moore, N. and Hooley, T. (2011). Building a Progression Culture: Exploring Learning Organisations' Use of The Progression Matrix. Derby: International Centre for Guidance Studies.

Contact details
Dr Tristram Hooley Reader in Career Development International Centre for Guidance Studies University of Derby

Tel: 01332 591268 Email: t.hooley@derby.ac.uk Twitter: @pigironjoe Blog: http://adventuresincareerdevelopment.wordpress.com/