Srudm

m Educarronal

Evaluation.

Pergamon

Pruned

Vol.

23, No. 2. pp. 141-157.

1997
0 1997 Elsewer
Science Ltd
in Great Britain.
All rights reserved
0191-491x/97
$17.00
+ 0.00

SO191-491X(97)00009-6

EVALUATING
THE EFFECTIVENESS
OF EDUCATIONAL
INNOVATIONS:
USING THE STUDY PROCESS
QUESTIONNAIRE
TO SHOW THAT
MEANINGFUL
LEARNING
OCCURS
David
Hong

Kember,
Margaret
Jan McKay,
Kong

Polytechnic

Charlesworth,
and Vanessa

University,

Hung

Educational

Horn,

Howard
Stott
Kowloon,

Hong

Davies,
Kong

Innovations

This article advocates the use of repeated applications of inventories of approaches

to learning as an appropriate technique for the evaluation of educational innovations. The
method is particularly appropriate for innovations aiming to introduce more meaningful
forms of learning as this is a dimension measured by these instruments. The article presents
case studies showing how the measure can be used for innovations introduced in
naturalistic settings.
There is a voluminous literature on educational innovations. Different teaching
methods, models of teaching, the latest educational media, new curricula, alternative
assessment methods and many other forms of innovation have been proposed as leading
to better teaching and improvements in student learning outcomes. For such claims to be
believed they need some form of evaluation. The traditional approach to seeking evidence
is to use the science-based experiment and control method. This approach, though, has
been subject to criticism and suffers from limitations where the aim is to introduce an
innovation into a genuine educational setting, rather than to conduct a laboratory-type
trial. The first difficulty lies in ensuring that the experiment and control are genuinely
comparable. Educational media comparison studies, for example, have been criticized on
this basis by a number of writers (e.g., Clark, 1983, 1985; Levie & Dickie, 1973; Schramm,
1977). Clark (1983) suggests that unintentional content differences between treatments
often confound results. Compelling evidence for this assertion comes from the observation

141