How best to evaluate

the effectiveness of a
VLE
Chris Rowell
Technology Enhanced Learning Advisor – Regent’s
College, London.
Introduction
Session outcomes:
Identify ways of evaluating technology
enhanced learning
Understand how to evaluate a VLE using
qualitative data
Identify ways in which effective academic
practice in VLE’s can be supported and
developed
The context
Regent’s College:
Central London
7 Schools
Fees: £13, 000 to 21,000
4000 students
350 academics

My role:
learning technologist
Bb VLE
How do you evaluate your VLE?

Spend 10 mins discussing the following:

Does your institution evaluate its VLE?

How is your VLE evaluated?
Who does it?
Why do they do it?
Methodology
Baume (2010) paraphrasing Kirkpatrick (1992):

 Did people like it?

 What have they learnt from it?
 Have they applied what they learnt to their
practice?
 Have the results improved?
My research project at Regent’s
College:
Quantitative data:

My research:
1. Online questionnaire
2. Interviews
3. Analysis/report
Questionnaire
Interview questions
Your task:
Arrange the interview question in order of
importance.
Interviews
Interview questions for lecturers

Starting Bb
1.a) How easy was it for you to start adding content to your module area?
b) What were the main difficulties you faced when creating a new module area in Bb (if any)
2. Have you experienced any technical difficulties using Bb?
Staff/Student interaction
3. Do you think the students have an increased expectation of your availability since Bb was
introduced?
4. Do you believe that Bb can replace a part of your face-to-face contact with your students?
5. Do you think that Bb affects the attendance in your lessons?
6. In your opinion, has Bb changed the quality of interaction with your students?
7. Has the introduction of Bb affected your relationship with students?
Teaching
8. Do you think that Bb enables you to become more or less student-centred in your teaching?
9. Has Bb affected (or could affect) the way you teach in the classroom?
10. How effective is Bb at supporting collaborative activities on your course?
Assessment
11. How has Bb affected the way you assess students work?
12. Have you changed the amount of feedback to students since the introduction of Bb?
13. Do you use Bb to evaluate how much work the students are doing on your module?
14. Do you use Bb to identify any students who might be “at risk” on your module?
Workload
15. Has the introduction of Bb changed your overall workload?
Future needs
16. Would you like more training or on-to-one support to develop your Bb module? If so, what areas of
Bb would you like to develop?
17. Would you like to make any other comments about Bb?
Report
Revisiting our evaluation criteria
 Did people like it?
 What have they learnt from it?
 Have they applied what they learnt to their practice?
 Have the results improved?
Who gets the report?
Why bother?
Conclusion
What I have got out of it:
 Justification of what I am already doing
 Influenced future direction of the VLE
 Ticks SM’s boxes
 Unintended consequences
Any questions?
Contact me:

rowellc@regents.ac.uk