You are on page 1of 26

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 03.10.

11

JISC Project Plan


Project Information Project Identifier Project Title Project Hashtag Start Date Lead Institution Project Director Project Manager Contact email Partner Institutions Project Webpage URL Programme Name Programme Manager Sept. 2011 Institute of Education Dr Mary Stiasny Dr Gwyneth Hughes g.hughes@ioe.ac.uk N/A http://www.ioe.ac.uk/study/departments/lce/59520.html Assessment and Feedback Programme Strand A Lisa Gray End Date Aug. 2014 To be completed by JISC Assessment Careers: enhancing learning pathways through assessment

Document Information Author(s) Project Role(s) Date URL Access Gwyneth Hughes and Martin Oliver Project Manager and Deputy Manager 12.11.11 Filename Projectplanv7 http://assessmentcareers.jiscinvolve.org/wp/ This report is for general dissemination

Document History Version Draft 4 Draft 7 Date 3.10.11 12.11.11 Comments

Page 1 of 26
Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11

Table of Contents
NB : This table of contents auto-populates - to update the table of contents place cursor in the table of contents, right-click your mouse, click update field, select appropriate option

1.

Project Overview .............................................................................................................................. 3 1.1 Project Summary ..................................................................................................................... 3 1.2 Objectives ................................................................................................................................ 3 1.3 Anticipated Outputs and Outcomes ........................................................................................ 4 1.4 Overall Approach .................................................................................................................... 5 1.5 Anticipated Impact ................................................................................................................... 5 1.6 Stakeholder Analysis ............................................................................................................... 7 1.7 Related Projects ...................................................................................................................... 7 1.8 Constraints .............................................................................................................................. 8 1.9 Assumptions ............................................................................................................................ 8 1.10 Risk Analysis ........................................................................................................................... 8 1.11 Technical Development ........................................................................................................... 9 1.12 Standards ................................................................................................................................ 9 1.13 Intellectual Property Rights ..................................................................................................... 9 2 Project Resources .......................................................................................................................... 10 2.1 Project Partners..................................................................................................................... 10 2.2 Project Management ............................................................................................................. 10 2.3 Project Roles ......................................................................................................................... 10 2.4 Programme Support .............................................................................................................. 11 3 Detailed Project Planning .............................................................................................................. 11 3.1 Evaluation Plan ..................................................................................................................... 11 3.2 Quality Assurance ................................................................................................................. 13 3.3 Dissemination Plan ............................................................................................................... 14 3.4 Exit and Embedding Plans .................................................................................................... 15 3.5 Sustainability Plans ............................................................................................................... 16 Appendices ........................................................................................................................................... 17 Appendix A. Project Budget............................................................................................................... 17 Appendix B. Workpackages .............................................................................................................. 26

Page 2 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11

1. Project Overview
1.1 Project Summary
Assessment is often viewed as a series of one-off events. This means that learners do not benefit from feedback, they lack a sense of progress and self-reliance is not encouraged. This project will reconceptualise assessment from the perspective of an assessment career and use this to transform an institutions assessment processes. An assessment career perspective involves taking a longitudinal view of assessment by building on a learners past experiences of assessment as well as current trajectories. Like a working career, an assessment career develops through a series of related events that join up to give a coherent and progressive pathway that is self-directed. Previous initiatives suggest that linking up assessments by giving learners feedback on their progress (ipsative feedback) motivates learners and helps them focus on learning rather than grades. We will develop an Assessment Career framework to include ipsative assessment as well as multi-stage assessment and assessment career audits to help learners overcome negative past experiences and move forward. The VLE will support the framework through assessment templates and improving records of feedback over time. The project will involve three phases. The first will create a baseline using existing institutional practice and use external research and development to refine the Assessment Career framework. In the second phase, five case studies will explore application of the framework in an efficient and sustainable way. In the third phase, the project will scale up innovation and share staff expertise by linking modular assessments, extending the new Assessment Career framework across programme clusters and embedding the framework into institutional structures and procedures. Details of the pilots and their evaluations will be made available to the HE sector, along with a review of how technology was used to support and scale up the Assessment Career framework across the institution.

1.2

Objectives
The project objectives are to:

1.

Document the range of assessment practice across the IOE and, in particular, to identify those assessment practices which students value most with regard to supporting their learning over time. Build on the work of REAP, other projects such as MAC and institutional assessment needs, to develop a conceptual and practical model of the assessment career that can be used at an institutional level to benchmark existing assessment practice and plan development. Apply the Assessment Career model to identify a set of best assessment practices that can be used more widely across IOE Masters and PGCE programmes.

2.

3.

Page 3 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 4. Pilot the new assessment practices in five contrasting Masters modules (such as

those focusing on research, health and clinical education, PGCE and Educational Psychology).
5.

Provide those piloting new assessment practices with systems and technologies for supporting that practice, e.g. assessment templates, multi-stage assessments, ipsative feedback, multiple mixed methods of assessment, peer and self assessment, multi-source feedback and assessment career audits. Evaluate the new pilot assessment practices to identify (a) student perceptions of whether, or in what ways, their learning has been supported as well as (b) teaching and support staff responses to the innovations. Design systems and processes to scale up and embed the changes at an institutional level. Achieve this change in professional assessment and feedback practice though redirecting workloads rather than workload intensification.

6.

7.

8.

All the above objectives will be addressed in the project deliverables and all will be completed within the three years of the project timescale.

1.3

Anticipated Outputs and Outcomes


Brief Description

Output / Outcome Type (e.g. report, publication, software, knowledge built) D1 Baseline report including summary of previous projects and initiatives.

D2 Case Studies and Evaluation reports for each pilot.

D3 Assets from the case studies

D4 Guidance and Support materials for developing an Assessment Career approach for other institutions D5 Institutional Evaluation of scaling up and embedding learning pathway enhancement through assessment.
Page 4 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

This will include: summaries of HEA and CPE funded projects on assessment and a JISC project on e-assessment; collection of data on assessment from the previous two i-graduate surveys; a summary of External Examiner comments on written feedback, information on assessment appeals and plagiarism cases in 2010/11; interviews with programme leaders for examples of innovation in assessment practice. Each innovation will also be subject to an efficiency review. Each report will provide details of how the intervention promotes the assessment career approach, details of students and staff involved in the pilot, evaluation methods, results of evaluation of the pilot including an efficiency review and 4 videos of students. To include videos of students accounts of their assessments career development at the IOE, templates for assessment design for enhancing assessment careers including multi-stage assessment, audit for assessment career on entry to a programme of study, online ipsative feedback scheme(s) and pro-formas. This will include a summary of the Assessment Career framework, evidence for the benefits of its use and a mapping document of the framework onto institutional change processes and costs of implementation. These will be informed by awareness that other contexts are different from the IOE. This will include a comparison of current data with the baseline data collected, evidence from committee reports on the embedding of the new framework, records of staff development, results of assessment career audits, summaries of validations

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 and programme reviews that included assessment reviews, further piloting of the framework across programme clusters and feedback from Faculty Directors of Learning and Teaching and from the Pro-Director (Learning and Teaching and International). While difficult to evidence and measure directly, it is expected that D2 and D3 and D5 will support this claim.

Increased awareness of and buy in to good practice in assessment and feedback across the institution and improved IOE reputation for excellence. Project Blog

Interim reports

D6 Assessment templates

D7 Final Report

A project blog will keep records of meetings, documents, outputs e.g. evaluations and reports and provide space for the team to comment on progress. Interim reports will be prepared every 6 months and sent to the Steering Group, and be presented at Teaching Committee as well as JISC. Templates will be developed that codify examples of practice and make these available for re-use. These may include descriptions of activities or structures that could be incorporated in a course area of a VLE. Final project evaluation report and completion survey

1.4

Overall Approach

The project will consist of 3 distinct phases: Phase 1 October 2011- May 2012: Review of current practice, consolidation of assessment career approach This will begin with an institute-wide review of current practice in assessment and feedback practice to identify how assessment pathways for students are currently established and supported. It will cover innovation in assessment (particularly using technologies), feedback practices, multi-stage assessment (draft assignments commonly receive formative feedback), timing of assessments and receipt of feedback/grades and quality assurance. Each innovation will also be subject to an efficiency review to ensure that staff time is redirected rather than increased. This will be undertaken using an online teaching financial and pedagogic modelling tool, developed through the work of the TLRP-TEL 1 funded Learning Design Support Environment (LDSE) project , and implemented within IOE systems as part of the Open Mode strategy. A sample of programme leaders will be interviewed about new ideas for practice and to develop the research already undertaken on assessment. Data on assessment appeals, external examiner reports and i-graduate survey results, as well as results from research undertaken at the IOE funded by HEA, JISC and CPE, will be collated. Literature on assessment and feedback, experiences of other institutions and other projects such as REAP and MAC will be used to benchmark institutional assessment practice and its shortcomings (Objectives 1 & 2). In parallel with this work the assessment career framework will be developed and adapted in response to the benchmarking data and literature reveiw. In particular, the assessment career model will apply the following principles of good assessment design: principle 4 in encouraging learners to act on feedback in the longer term; 6 in encouraging dialo gue; 7 through developing learners selfassessment of their career and 11 through motivating learners as their progress is made more visible. It is expected that innovation in assessment design will also address other principles, but we

will focus on these as our priorities (Objective 3).

Laurillard, D. (2008) The teacher as action researcher: Using technology to capture pedagogic form, Studies in Higher Education, 33 (2), 139-154.
Page 5 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Phase 2 April 2012-July 2013: Piloting innovations and evaluation Team members will each have responsibility for one of the five pilots. The pilots will begin with a discussion and briefing meeting for all programme teaching staff (approx. 5), administrator and students (20-40 per pilot) involved to ensure that the work addresses stakeholders needs. Each pilot will consist of a review of existing module(s), auditing existing practice, identifying the innovations (documented in phase one) that best address stakeholder needs and existing shortcomings, implementing the innovation and evaluating its impact. Institutional support processes for implementing existing and procuring new technologies will be used. The interventions will be evaluated through comparing changes in practice with previous delivery of the module, using the LDSE-developed resource costing tool to benchmark practice. In this way, the team will ensure that interventions are efficient in that they require re-adjustment of staff workloads. Additionally, students and staff will be invited to complete an online evaluation questionnaire in order to gather evidence of benefits (Objectives 4,5 & 8). These pilots will allow the Assessment Career concept to be mapped against institutional change processes. This mapping will be documented, providing a framework for institutional change that can be shared with other institutions. In Phase 2, this Assessment Career institutional change process will be refined on the basis of the pilots and their evaluation, and the documented change process will be presented to institutional committees, e.g. Teaching Committee and Senate, which have student representation, for consultation and agreement to roll out across the institution for phase three. It is expected that this work will be closely aligned with the IOEs Assessment Review which will be due to report during phase 2. Pilot leaders will present their initiatives to other programmes in their cluster to identify at least 5 further programmes to test the Assessment Career frame work in phase 3 (Objectives 5 & 6). Phase 3 April 2013-July 2014: Supporting and developing staff, scaling up and sustaining the Assessment Career approach This will begin during the piloting phase and will continue into a third unfunded year of the project. A series of staff development workshops will run from the end of phase 2 and into phase 3. These will introduce the Assessment Career framework and will be targeted at teaching teams in particular programmes identified in the clusters associated with the pilots. In phase 3 the staff development workshops will focus on using technology to enhance assessment careers as part of the Learning Technology Units training programme. In order to ensure the relevance of these workshops to stakeholder needs, they will all involve discussion and exchange of views as well as the introduction of documented approaches, templates, technologies, etc. The Institute has, for example, already established a series of Lunchtime Conversations in Learning and Teaching in which topical issues are debated by teaching and support staff. During Phase 3, this series of activities will be devoted primarily to the assessment career theme (Objective 7). The Institute Teaching Committee, chaired by the Director of this project, will have responsibility for reconfiguring assessment processes and documentation and Faculty Quality committees will have the role of monitoring/supporting change process for all programmes to embed the assessment career model (Objectives 7 & 8). The benefits to students of taking an assessment career approach to learning will be evaluated at an institutional level (Objective 6). This will include student evaluations from the 5 pilots from phase 2 and from the 5 or more programmes applying the Assessment Career framework in phase 3 (approx. 400 students in total) and comparisons with baseline data from students (approx. 90). Wider uptake will be enabled by providing templates for activities and/or course structures derived from this work. These will be made available in formats that are as widely accessible as possible, to ensure reusability and avoid problems of interoperability. Such formats may include descriptions, archived course structures or formats consistent with work undertaken through the Learning Design Support Environment project.
Page 6 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Areas not covered are quality assurance of marking and standards, double marking issues, the current grading system and institutional M level marking criteria. Critical Success Factors include: Clarity of the Assessment Career framework Effectiveness of pilots in providing evidence for the benefits of this approach Range of stakeholders supporting the Assessment Career framework Scalability and sustainability of the Assessment Career approach Across the IOE

Anticipated Impact: Impact Area


Maintain teaching & learning excellence and develop excellence in assessment. Be more effective/save money

Be ready for technology needs in the future

Anticipated Impact Description The impact of a longitudinal approach to assessment will enhance learning and teaching by linking assessments, making progress visible and encouraging learner self-reliance and motivation. Increased awareness of the cost-benefit of teaching and assessment activities will ensure that programmes and their assessments are designed to maximise learning for a given resource. Use of technology to enhance and deliver new approaches to assessment will prepare teaching, administrative and support staff for future technological innovation.

1.5

Stakeholder Analysis
Stakeholder Interest / stake Quality assurance of learning and teaching Learning and teaching excellence, student experience, workload Student experience, workload Supporting pedagogic innovation using technologies Learning and teaching excellence, student experience, quality assurance of learning and teaching Use of the Assessment Career framework to improve assessment Impact of technologies on assessment process both pedagogic and workload L H Importance (H/M/L)

Administrators Teaching staff

Students Learning Technologists Pro-Director (Learning and Teaching and International)

H M H

Other institutions including our Moodle partners and other related projects. JISC

M/H

1.6

Related Projects
A longitudinal study of taught postgraduates transitional experiences of assessment and feedback conducted at the IOE from 2009-2011 funded by the Higher Education Academy A scoping study on formative e-assessment (FEASST), funded by JISC A CDE funded study of ipsative assessment for distance learners.

Page 7 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 A project on the development of an institutional infrastructure for digital literacies, funded by JISC. This proposal will extend that work by integrating the new approaches to supporting the development of digital literacy with institution-wide approaches to assessment.

1.7

Constraints

The main constraint will be time for the Project Manager, Deputy Project Manager and the 5 Pilot Leaders. Participants will be encouraged to keep a record of days spent on the project and compare this with the allocated allowance and alert the project leader if they are spending too much/too little time on the project. Institutional quality assurance and registry core staff who have been built into the project will have many other issues to address as well as those emanating from this project and priorities will be agreed with the Pro-Director (Learning and Teaching and International) to ensure that actions are taken within the timescale of the project. It is unlikely that institutional priorities will change over the lifetime of the project, but if so there could be constraints on resourcing and scope.

1.8

Assumptions

Key assumptions are that the pilot leaders will not change during the project and that their programme teams and students will be willing to participate. It is also assumed that the reference to assessment in the IOEs draft Learning and Teaching Strategy will be agreed and that a review of assessment will be a priority for 2011/12.

1.9

Risk Analysis
Probability (P) 15 (1 = low 5 = high) 1 Severity (S) 15 (1 = low 5 = high) 4 Risk Score (PxS) Detail of action to be taken (mitigation / reduction / transfer / acceptance)

Risk Description

One or more pilots do not support the assessment career model Resistance from key section of institution, e.g. QA or Teaching Committee Cost effectiveness not achieved in one or more pilots Key team members leave the project/institution VLE and associated templates unable to support the project

15

1 1

5 3

5 3

Readjust Assessment Career model to allow for variation of practice Ensure buy-in across senior management from an early stage. Consider adjusting Assessment Career model to accommodate opposition Reconsider balance between assessment and teaching and other activities in workload management models A deputy project leader can take over if PL leaves VLE implementation is an institutional priority reporting to the Corporate Systems Board with a risk register compiled. Risks associated with failures of the VLE to support practice and/or activity templates will be managed by (a) using open

Page 8 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 standards wherever appropriate and (b) switching to other systems if required. All pilot modules to be given priority in moving to the new VLE before OCT. 2012. Redistribution of some project co-ordination to project deputy manager e.g. dividing responsibility for pilots.

Insufficient time for project management

12

1.10 Technical Development


No technical development is being undertaken within the project. The project will rely on availability of a VLE; provision of a VLE is an institutional priority, and is currently being reviewed through a PRINCEII-managed project, due to complete in October 2012. Risks of failure within that project are being managed by a board chaired by the Pro-Director (Learning and Teaching and International). As part of this process, a risk register has been compiled. The risks here include issues of integrating the system with existing IOE services (e.g. registry databases), concerns around the migration of existing content from Blackboard to Moodle, additional workload for academic staff, and so on. The approach that has been adopted ensures that mitigating measures are put in place for all major risks. For example, the risk of additional work for academics and subsequent disengagement has been mitigated in two ways: supported migration (automated where possible, and using contracted staff to recreate materials that cannot be recreated automatically) and pedagogic training. The re-created courses will make use of structural templates, allowing us to introduce standardise elements that are recognised as good practice. In the worst case, this will leave the institution with business as usual, wherein existing practices are simply moved across to the new platform; in the best case, this provides an opportunity to intervene in practices to raise the standard of learning and teaching through the promotion of successful practices. Following the initial year, the emphasis will shift, increasingly, towards this best case position: once business as usual has been established, we will concentrate on improving practice through training and the use of templates that represent and share successful practice. This phase coincides with the innovative work that will be undertaken in this project; we will use the projects work to inform and extend our work in developing staff use of Moodle.

1.11 Standards
Name of standard or specification As the project is not undertaking technical development, the only standards that are relevant are HTML Version 4 Notes Some templates may be created using Moodle (for course structures), but in these cases a parallel flat-text description will be provided for those who cannot access this.

1.12 Intellectual Property Rights


Page 9 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 All materials produced by the project (including reports, templates, etc) will be made freely available to the sector, under a creative commons attribution, non-commercial, share-alike licence. All existing copyrights will remain with the copyright holders. However, the data produced during the project will not be released for ethical and data protection reasons, unless project participants choose to waive their rights to privacy.

2 Project Resources
2.1 Project Partners
The Association for Learning Technology will undertake consultation and dissemination activities on behalf of the project. This will be undertaken on a sub-contract basis. The main contact is Seb Schmoller (s.schmoller@alt.ac.uk), the chief executive. A consortium agreement will be provided within three months of the project start date.

2.2

Project Management

The Project leader will manage the project with support from the deputy project leader who will in particular oversee the technical and staff development aspects. Both will report to the Project Director who is on the Senior Management Team of the IOE and who will also chair a steering group. The project leader will: Prepare the project plan with the deputy leader Coordinate and manage project work with the deputy leader Monitor project progress and performance with the deputy leader Ensure that project outputs are delivered on time Identify risks, problems, and issues, and escalate them as appropriate with the deputy leader Manage communication within the project and with the programme manager Prepare progress, final, and other reports with the deputy leader Arrange meetings (e.g. management committee) and write the minutes with the administrator Manage project resources, including the budget with the institutional research administrator Maintain the project web site with the deputy leader and learning technologist Maintain project documentation Ensure that the project abides by the letter of grant, the JISC Terms and Conditions, and the JISC Project Management guidelines.

2.3

Project Roles
Role Project Leader Deputy Leader Pilot Leader Pilot Leader Pilot Leader (provisional) Pilot Leader Pilot Leader (provisional) Contact Details g.hughes@ioe.ac.uk m.oliver@ioe.ac.uk m.newman@ioe.ac.uk i.warwick@ioe.ac.uk a.robertson@ioe.ac.uk y.reynolds@ioe.ac.uk w.gibson@ioe.ac.uk Days per week to be spent on the project 1.5 0.75 0.5 0.5 0.5 0.5 0.5

Team Member Name Dr Gwyneth Hughes Dr Martin Oliver Dr Mark Newman Dr Ian Warwick Anne Robertson Yvonne Reynolds Dr Will Gibson

Page 10 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Dr Mary Stiasny Project Director Dr Kaori Kitagawa Researcher Tim Neumann Learning Technologist Beatrice Hyams Academic support and Quality Officer Donnalyn Morris Project Administrator

m.stiansy@ioe.ac.uk k.kitagawa@ioe.ac.uk t.neumann@ioe.ac.uk b.hyams@ioe.ac.uk d.morris@ioe.ac.uk

8 days total 1 1 5 days total 0.5

The project team will meet 5 times during phase 1 and pilot leaders and project leaders will meet 4 times in phase 2. There will be 2 meetings early in phase 3 to include QA staff and the project Director. In addition the Project Leader will have short updating and briefing meetings with the Director at 6 week intervals throughout the project. The Steering group will meet 3 times during phases 1 and 2. All project team will need training on Moodle access and its assessment related features. This will be provided by the Learning Technology Unit (LTU). The project Steering group consists of: Dr Mary Stiasny (chair), Prof. Diana Laurillard d.laurillard@ioe.ac.uk, Tina Isaacs t.isaacs@ioe.ac.uk, Mandy Bentham (external) [mb68@soas.ac.uk], Sharon Huttley (external) Sharon.Huttly@lshtm.ac.uk, Samuel Mejias (Student Union Rep) s.mejias@ioe.ac.uk. The steering group will steer and advise on the project, discuss risks, agree major changes and represent the interests of the project.

2.4

Programme Support

Support with designing and monitoring pilots and with scaling up and embedding a new approach to assessment is likely to be requested.

3 Detailed Project Planning


3.1 Evaluation Plan
Timing April 2012 Factor to Evaluate Phase 1: The institutions baseline practice and the Assessment Career framework. Questions to Address Have milestones been met on schedule? If not, what should we do to correct this? Is project management effective? Are stakeholders on board? Do they agree with interim findings and clarity of the Assessment Career framework? What lessons have we learned?
Page 11 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Method(s) Presentations to Steering Group and IOE Teaching Committee for peer review. Self evaluation by project leaders.

Measure of Success Consultation with stakeholders will ensure that their needs have been identified, providing an evidence base for subsequent actions and choices by the project team.

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Do we need to change the plan? Do plans build on baseline report and Assessment Career framework? Are they realistic? Will students and staff be involved? How far were the results from each pilot useful for scaling up the framework? Is the approach for each pilot efficient? What changes are needed for scaling up?

June 2012

Pilot plans

Project leaders

See below.

June 2013

Phase 2 The success of the pilots in embedding an assessment career framework

June 2014

Phase 3: The summative project evaluation report will be written by members of the project team.

Have objectives been met? Have outcomes been achieved? What are the key findings? What impact did the project have? What benefits are there for stakeholders? Was our approach effective? What lessons have we learned?

Pilot leaders for peer review Teaching committee and the Steering Group review. The LDSE costing tool will applied to each programme. The project leader and deputy will review the project deliverables and key findings and impact against project outcomes and programme outcomes. The project leader and deputy will review the impact on institutional practice by comparing with the Baseline report

Success will be judged by the effectiveness and efficiency of apply the framework developed in phase 1 and by transferability of interventions to other programmes.

Success will be judged by comparisons made with the baseline position of the institution identified in phase 1 and external feedback from dissemination events.

June 2014

Impact of the Assessment Career (AC) framework

The impact on stakeholders of introducing assessment careers will be reviewed: students and teaching staff (engagement and action related to assessment and feedback), Senior managers and learning technology staff, administrators (engagement and support). Moodle partners and JISC (awareness).

Success will be judged by comparisons made with the baseline position of the institution identified in phase 1

Page 12 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11

Evaluation questions:
1. Is the AC framework widely understood? (Discussion at committee level, pilot evaluations, attendance at staff development events, changes to QA processes). Stakeholders Senior managers, QA staff, teaching staff and students. 2. How far has the AC framework been applied across the institution? (Evidence from formal assessment guidelines and QA documentation, discussion at committee level, use of assessment templates). Stakeholders Senior managers, QA staff, teaching staff and students. 3. What impact has the AC framework had on assessment and feedback? (Evidence from pilot evaluations, external examiner feedback and igraduate or equivalent data) Stakeholders teaching staff and students including students with disabilities, international and widening participation students. 4. What is the impact of applying the AC framework on workload for teaching and administrative staff? (Evidence from pilot evaluations using modelling tool, responses from Programme leaders and programme administrators to new assessment documentation) Stakeholders teaching staff and programme administrators.

3.2

Quality Assurance
D1 Baseline report including summary of previous projects and initiatives. Who will carry out the QA What QA methods / measures will be used? work? Project Leader and deputy Fitness for purpose i.e. as a baseline for future comparisons

Output / Outcome Name When will QA be carried out? April 2012

Output / Outcome Name When will QA be carried out? June 2012

D2 Evaluation reports for each pilot. Who will carry out the QA work? Project Leader and deputy in consultation with Steering group and IOE Teaching Committee and LTU. What QA methods / measures will be used? Use of best practice methods and techniques for development. Adherence to relevant standards and specifications (e.g. to ensure interoperability and accessibility)

Output / Outcome Name When will QA be carried out? July 2012

D3 Assets from the case studies Who will carry out the QA work? Teaching Committee and LTU. What QA methods / measures will be used? Use of best practice methods and techniques for development. Adherence to relevant standards and specifications (e.g. to ensure interoperability and accessibility)

Page 13 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11

Output / Outcome Name When will QA be carried out? July 2013

D4 Guidance and Support materials for developing an Assessment Career approach for other institutions Who will carry out the QA What QA methods / measures will be used? work? Project Leader and deputy Use of best practice methods and techniques for in consultation with development. Steering group and IOE Adherence to relevant standards and Teaching Committee and specifications (e.g. to ensure interoperability and JISC. accessibility)

Output / Outcome Name When will QA be carried out? July 2014

D5 Institutional Evaluation of scaling up and embedding learning pathway enhancement through assessment. Who will carry out the QA What QA methods / measures will be used? work? Project Leader and deputy Fitness for purpose for future comparisons and in consultation with for use by other institutions Steering group and IOE Teaching Committee and JISC.

Output / Outcome Name When will QA be carried out? Jan. 2014

Assessment Templates Who will carry out the QA work? Project leader and deputy in consultation with Steering group and IOE Teaching Committee, Student Union and JISC What QA methods / measures will be used? Review in terms of LDSE project specifications, trials with users (within the IOEs training programme) and compliance with accessibility guidelines.

Output / Outcome Name When will QA be carried out? April 2014

Increased awareness of good practice in assessment and feedback across the institution and improved IOE reputation for excellence. Who will carry out the QA What QA methods / measures will be used? work? Pro- Director (Learning and Evidence for good practice in assessment at the Teaching and International) IOE and an institutional claim for excellence in assessment.

3.3

Dissemination Plan
Timing Dissemination Activity Webinar on the assessment career framework Audience Educational Developers and researchers in HE Purpose Explore the assessment career framework and its potential to enhance Key Message Assessment needs a longitudinal approach

May 2012

Page 14 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 assessment and get feedback Discuss extending assessment career framework Present learner assessment trajectories and get feedback Present assessment career framework and its potential to enhance assessment and get feedback Present learner assessment trajectories to share practice Information on AC approach

May 2013

Programme cluster meetings (working with Assessment Review group) ELISIG Webinar

Programme leaders

June 2013

Educational Developers and practitioners HE Researchers, developers and practitioners HE

Dec. 2012/Sept 2013

Conference presentations e.g. SRHE, ALT-C. Scottish e-assessment event, CAA conference, JISC online conference.

Assessment can be improved without increase in time and resources Assessment careers need to be supported and developed Assessment needs a longitudinal approach

November 2013

ALT Webinar

Teaching and support staff

September 2013

Student handbooks and induction materials working with Assessment Review group.

Students

Ongoing

Project website and blog

Ongoing

IOE Teaching and Learning Newsletter

Educational Developers, researchers, managers and practitioners HE. IOE staff and students. All IOE staff and students

Information on progress and deliverables of the project

Assessment can be oriented to learners rather than to courses Taking a longitudinal approach to assessment will enable learners to get the most from feedback and to become more selfreliant. N/A

Ongoing

Design Studio

Other projects in the programme

Information on progress and key messages of the project Information on progress, sharing problems and ideas

N/A

N/A

3.4

Exit and Embedding Plans


Action for Take-up & Embedding On project website and presented to Action for Exit Project website maintained at

Project Outputs/Outcomes D1 Baseline report

Page 15 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 including summary of IOE Teaching Committee previous projects and initiatives. D2 Evaluation reports for On project website and Design each pilot. Studio D3 Assets from the case On project website and Design studies Studio D4 Guidance and Support On project website and Design materials for developing an Studio Assessment Career approach for other institutions D5 Institutional Evaluation On project website and presented to of scaling up and IOE Teaching Committee and embedding learning Student Union pathway enhancement through assessment. Increased awareness of good practice in assessment and feedback across the institution and improved IOE reputation for excellence. D6 Assessment Templates Embedded in IOE Assessment guidelines and in assessment regulations

IOE until 2017, then archived. IOE archive of committee records. Project website maintained at IOE until 2017, then archived. See above See above

IOE archive of committee records

Guidelines circulated at IOE and stored on IOE website

On project website, LTU website and available through Design Studio

Template availability at IOE managed by LTU and as above

3.5 Sustainability Plans


Project Outputs D2 Evaluation reports for each pilot. Why Sustainable These can continued to be used by other HEIs for research and development work in assessment These can be used to inform students about the assessment career approach and for staff development materials These can continued to be used by other HEIs for institutewide development work in assessment These will continue to be useful in supporting curriculum development. Scenarios for Taking Forward Design Studio and IOE website. Journal article publication e.g. in ALT-C to publicise further or use in book on Assessment Careers. Design Studio and IOE website. Use in Certificate in Learning and Teaching programmes for lectures and other staff development events. Design Studio and IOE website. Journal article publication e.g. in ALT-C to publicise further. Issues to Address Small scale of each pilot.

D3 Assets from the case studies

Videos may become dated

D4 Guidance and Support materials for developing an Assessment Career approach for other institutions D6 Assessment templates

Materials may become dated

Design Studio and IOE website. Use in Certificate in Learning and Teaching programmes for lectures and other staff development events.

Ensure interoperability across commonly used VLEs.

Page 16 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11

Appendix A. Project Budget


To discuss please contact g.hughes@ioe.ac.uk

Page 17 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 03.10.11

Appendix B. Workpackages

JISC WORK PACKAGE


Before completing this template please note: The Project Management Guidelines have detailed instructions for preparing project plans and work packages.. Please expand tables as appropriate. Fill in the information in the document header project name, document version, and date. Text in italics is explanatory and should be deleted in completed documents.

WORKPACKAGES 1: Literature/external project review 2:IOE baseline review 3: Assessment career framework 4: 5 Pilot Plans 5: Baseline Report D1 6: Pilot evaluations D 2 7: Assessment career framework refined 8. Workshops 9. Assets from case studies D3

Month

4 x

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

x x x x x x x x x

Page 18 of 26
Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 WORKPACKAGES Month 8. (cont.) Workshops 10: Guidance and support materials D4 11. Institutional Scaling up report D5 12. Assessment templates D6 13. Final Report D7

25 x x

26

27

28

29

30

31 x

32

33

34

x x x

M=Milestone D=Deliverable

Project start date: Sept.2011 Project completion date: July 2014 Duration: 34 months

Page 19 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11

YEAR 1
Oct 2011 WORKPACKAGE 1: Literature/external project review Objective: To identify good practice to underpin Assessment career framework 1. Summarise previous assessment projects e.g REAP and key relevant literature 2. List good practice themes that can be used in development of Assessment career framework 3. WORKPACKAGE 2: IOE baseline review Objective: To identify IOE baseline position on assessment and feedback 4. Summarise findings of previous IOE research on assessment and feedback 5. Summarise comments on assessment from External Examiner reports over previous 2 years 6. Summarise findings from previous 2 igraduate surveys 7. Summarise assessment practice at IOE from regulations and guidance documents Dec 2011 Nov 2011 Feb 2011 Feb 2011 Dec 2011 Review document to be combined with workpackage 2 M1 Project researcher

Nov 2011

March 2012

Review document and baseline statement

M2

Project researcher

Nov 2011 Nov 2011 Nov 2011 Nov. 2011

Jan 2012 Jan 2012 Jan 2012 Jan 2012

Dec 2011 WORKPACKAGE 3: Assessment career framework Objective: To develop and summarise the
Page 20 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Feb. 2012

Document summarising Assessment career framework. Webpage with visual representation.

M3

Project Leader and deputy in consultation with team

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Assessment career framework 8. Review findings of project review and IOE baseline position 9. Amend and expand assessment career framework Dec 2011 Dec 2011 Jan. 2012 March 2012 5 pilot plan documents on website WORKPACKAGE 4: 5 Pilot Plans Objective: To plan 5 pilots to test use of technology to support aspects of assessment career framework 10. Write draft plans and exchange with others and discuss and amend 11. Finalise plans and present to Teaching Committee 12 Undertake pilots WORKPACKAGE 5: Baseline Report Objective: To report on IOE baseline position 12. Develop a baseline statement that benchmarks the IOE position in relation to sector best practice using REAP principles 13. Synthesise findings from WP 1 and 2 YEAR 2 July 2012 WORKPACKAGE 6: Pilot evaluations Objective: Evaluate 5 pilots against assessment career and success criteria
Page 21 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

M4

Pilot Leaders

March 2012

June 2012

March 2012 May 2012 Sept. 2012

May 2012 June 2012 April 2013 May 2012 Document: Institutional baseline report and literature review D1 Project researcher in consultation with project leader(s)

March 2012 April 2012

May 2012 April 2012

Mapping document using REAP principles

April 2013

M5

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 14. Develop evaluation methodology and criteria including use of costing tool 15. Write up case studies and evaluate against criteria 16. Internal reports of pilots to Teaching Committee and dissemination to programme clusters for further testing with 5 more programmes July 2012 Jan 2013 May 2013 Sept 2012 April 2013 June 2013 Evaluation methodology document Case studies of 5 pilots Internal reports D2 Project leader plus team Pilot leaders Pilot leaders and Faculty Directors of Learning and Teaching (FDTLs) Project leader plus team

WORKPACKAGE 7: Assessment career framework refined Objective: To adjust Assessment career framework in the light of pilot findings 17. Adjust Assessment career framework 18. Consultation on embedding framework in institutional assessment regulations and guidance WORKPACKAGE 8: Workshops Objective: 19. Webinar offered in conjunction with ALT and advertised through SEDA report on assessment career framework, illustrated in relation to baseline data 20. Webinar offered in conjunction with ELESIG, and advertised through ALT & SEDA on learner trajectories and pilot work 21. Workshop/presentation at one or more
Page 22 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

May 2013 May 2013

May2013 Sept. 2013

Amended Assessment career framework document Reports to Teaching Committee

M6

Project leader Team and FDTLs Deputy project leader

Webinar May 2012 April 2013 December September Webinar Presentation summary/Powerpoint

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 conferences (ALT-C, SRHE, etc) 22. Webinar offered in conjunction with ALT and advertised through SEDA report on institutional embedding and scaling up

2012 March 2014

2013 Webinar

WORKPACKAGE 9: Assets from case studies D3 Objective: 23. Capture of data to produce anonymised case studies of learner trajectories; activity templates; and guidance material 24. Release of finalised case studies, templates and guidance via Design Studio and IOE website, and institutional promotion via IOE Certificate in Learning and Teaching programmes for lectures and other staff development events WORKPACKAGE 10: Guidance and support materials Objective: To provide resources for other HEIs to make use of the framework and scaling up processes 25. Support and guidance for using Assessment career framework including benefits and challenges 26. Review scaling up processes and their effectiveness 27. Develop suggestions for use of assets May 2012 August 2012 August 2012

Video, templates, documents with examples of assessments and guidance for use

D3

Learning Technologist plus Deputy project leader

Video Templates, assessment materials examples and guidance documents

July 2013

Oct. 2013

Guidance and Support materials

D4

Project and deputy leaders in consultation with pilot leaders

July 2013 July 2013

Aug. 2013 Oct. 2013

Support document on use of Assessment career framework Guidance document on scaling up Assessment careers Document -guidance on use of assets

Page 23 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 April 2013 WORKPACKAGE 11: Final Report Objective: Prepare final project report for JISC August 2013 Document Final Project Report D7 Project leaders with consultation with Project Director

28. Agree evaluation criteria with JISC YEAR 3 WORKPACKAGE 12: Institutional Scaling up report Objective: 29. Review use of Assessment career approach(es) on further programmes in programme clusters 30. Review changes in assessment practice on other programmes 31. Map the framework onto institutional change processes and include costs of implementation 32. I-graduate survey results 33. External Examiner comments 34. Impact on IOE assessment documentation Nov. 2013 Nov. 2013 Nov. 2013 March 2014 March 2014 March 2014 March 2014 March 2014 March 2014 April 2014 April 2014 April 2014 Summary reports Survey of Programme leaders Mapping diagram Sept. 2013 April 2014 Document Institutional scaling up report M7 D5 Project Leader and Deputy Project Director and QA Officer Pilot leaders Project Director and QA Officer Project Leaders QA officer QA officer Project Director

Comparison with baseline report

Page 24 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Digtial assessment templates WORKPACKAGE13: Assessment templates Objective: To refine assessment templates and make available sector wide Evaluation and revision of IOE use of assessment templates and guidance materials, including review of accessibility, interoperability, etc. 35. Release of revised assessment templates to the sector, with promotion through ALT, SEDA, Design Studio etc. WORKPACKAGE 14: Update to Final Report, completion survey and final budget Objective: Update final project report for JISC September 2013 February 2014 March 2014 April 2014 July 2014 Evaluation report of templates Digtal assessment templates Documents: Final Project Report with updates, survey completed. Final budget. D7 Project leaders with consultation with Project Director D6

36. Finalise project report for JISC 37. Survey 38. Prepare final budget

May 2014

July 2014 July 2014 July 2014

Draft report for feedback Survey Budget report

Research admin team

Members of Project Team: Name Dr Gwyneth Hughes Dr Martin Oliver Dr Mark Newman Dr Ian Warwick Anne Robertson Yvonne Reynolds Dr Will Gibson Dr Mary Stiasny Dr Kaori Kitagawa
Page 25 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0

Role Project Leader Deputy Leader Pilot Leader Pilot Leader Pilot Leader Pilot Leader Pilot Leader (provisional) Project Director Researcher

Assessment careers Version: 7 Contact: g.hughes@ioe.ac.uk Date: 12.11.11 Tim Neumann Beatrice Hyams Donnalyn Morris

Learning Technologist Academic support and Quality (QA) Officer Project Administrator

Page 26 of 26 Document title: JISC Project Plan Template Last updated: Feb 2011 v2.0