You are on page 1of 6


Donald H. Naftulin, M.D., o!n E. "a#$, #., and F#an% A. Donn$ll&
ou#nal of M$di'al Edu'ation, (ol. )*, ul& +,-., /. 0.120.3
This article is based on a paper presented at the 11th Annual Conference on Research in
Medical Education at the 83rd Annual Meeting of the AAMC, Miami Beach, Florida,
o!ember ", 1#$%&
'r& aftulin is associate professor and director of the 'i!ision of Continuing Education
in (s)chiatr), *ni!ersit) of +outhern California +chool of Medicine, Mr& -are is
assistant professor of medical education and health care planning and director of
research and e!aluation, +outhern .llinois *ni!ersit) +chool of Medicine, Mr& 'onnell)
is an instructor in ps)chiatr) /ps)cholog)0 in the *+C 'i!ision of Continuing
Education in (s)chiatr)&
Abstract 1 2n the basis of publications supporting the h)pothesis that student ratings of
educators depend largel) on personalit) !ariables and not educational content, the
authors programmed an actor to teach charismaticall) and non substanti!el) on a topic
about 3hich he 4ne3 nothing& The authors h)pothesi5ed that gi!en a sufficientl)
impressi!e lecture paradigm, e!en e6perienced educators participating in a ne3 learning
e6perience can be seduced into feeling satisfied that the) ha!e learned despite irrele!ant,
conflicting, and meaningless content con!e)ed b) the lecturer& The h)pothesis 3as
supported 3hen 77 sub8ects responded fa!orabl) at the significant le!el to an eight1item
9uestionnaire concerning their attitudes to3ard the lecture& The stud) ser!es as an
e6ample to educators that their effecti!eness must be e!aluated be)ond the satisfaction
3ith 3hich students !ie3 them and raises the possibilit) of training actors to gi!e
:legitimate: lectures as an inno!ati!e approach to3ard effecti!e education& The authors
conclude b) emphasi5ing that student satisfaction 3ith learning ma) represent little more
than the illusion of ha!ing learned&
Teaching effecti!eness is difficult to stud) since so man) !ariables must be considered
in its e!aluation& Among the ob!ious are the education, social bac4ground, 4no3ledge
of sub8ect matter, e6perience, and personalit) of the educator& .t 3ould seem that an
educator 3ith the proper combination of these and other !ariables 3ould be effecti!e&
;o3e!er, such a combination ma) result in little more than the educator<s abilit) to
satisf) students, but not necessaril) educate them&
=et5els and >ac4son /10 ha!e stated that the personalit) of the teacher might be the most
significant !ariable in the e!aluation of teaching effecti!eness& -allen and Tra!ers /10
also supported this concept in stating that :3e ha!e tried to demonstrate that patterns of
teacher beha!ior and the teaching methods the) represent are mainl) the products of
forces 3hich ha!e little to do 3ith scientific 4no3ledge of learning&:
+imilarl), =offman /%0 !ie3ed audience recepti!it) to a lecturer as highl) influenced b)
the person introducing him as 3ell as b) the 9ualit) of the introduction& .n addition,
=offman described an audience as influenced b) the spea4er<s :in!oluntar) e6pressi!e
beha!ior: as much as b) the e6pressed information he 3ished to con!e)& This is
especiall) so if the audience has had little time to e!aluate the information&
Conse9uentl), the learner<s impression of the information con!e)er becomes a decisi!e
factor in ho3 he responds to the information con!e)ed&
Rogers /30 stressed the importance of humani5ing our educational institutions b)
bringing :together the cogniti!e and the affecti!e1e6periential: aspects of learning& ;e
also discussed the significance of the educator<s genuineness& ;e feels that the educator
3ho does not present a facade is more li4el) to be effecti!e& The educator, states
Rogers, must ha!e a :direct personal encounter 3ith the learner&:
.n one stud) /?0 in 3hich student perceptions of educators in 1,?%$ se!enth through
1%th grade classes 3ere factor anal)5ed, it 3as reported that the students regarded
:teacher charisma or popularit): as the most important characteristic 3hen rating
teachers& The article further states that :students do not respond directl) to specific
9uestions regarding teacher effecti!eness& Rather a 4ind of halo effect on teacher
charisma or popularit) determines to a large e6tent ho3 students react to 9uestions
about their teacher&:
.f charisma or popularit) ha!e such an effect on the rating of teachers b) 8unior high and
high school students, the authors 3ondered 3hether the ratings of a highl) trained group
of professional educators in a learning situation might be similarl) influenced& .f that
3ere the case, a demonstration of the personalit) factor in percei!ed learning might
ser!e to arouse the group members< concern about the proper combination of st)le and
substance in their o3n teaching&
The h)pothesis for this stud) 3as as follo3s& =i!en a sufficientl) impressi!e lecture
paradigm, an e6perienced group of educators participating in a ne3 learning situation
can feel satisfied that the) ha!e learned despite irrele!ant, conflicting, and meaningless
content con!e)ed b) the lecturer&
To test the h)pothesis, the authors selected a professional actor 3ho loo4ed
distinguished and sounded authoritati!e, pro!ided him 3ith a sufficientl) ambiguous
title, 'r& M)ron @& Fo6, an authorit) on the application of mathematics to human
beha!ior, dressed him up 3ith a fictitious but impressi!e curriculum !itae, and
presented him to a group of highl) trained educators&
The lecture method 3as the teaching format selected since it is one used e6tensi!el) in
the professional educational setting& .t has been described as the one teaching method
during 3hich most of the time the instructor tal4s to the students /10& .ts acceptance as
an effecti!e teaching tool is attributable mainl) to its time1testedness&
'r& Fo6<s topic 3as to be :Mathematical =ame Theor) as Applied to (h)sician
Education&: ;is source material 3as deri!ed from a comple6 but sufficientl)
understandable scientific article geared to la) readers /70& 2ne of the authors, on t3o
separate occasions, coached the lecturer to present his topic and conduct his 9uestion
and ans3er period 3ith an e6cessi!e use of double tal4, neologisms, non se9uiturs, and
contradictor) statements& All this 3as to be interspersed 3ith parenthetical humor and
meaningless references to unrelated topics&
=R2*( .
Ele!en ps)chiatrists, ps)chologists, and social13or4er educators 3ho 3ere gathered for
a teacher training conference in continuing education comprised the learner group& The
purpose of the conference 3as to help this group be more effecti!e educators of other
health professionals b) pro!iding them !arious instructional goals, media, and
e6periences& 'r& Fo6 3as introduced as :the real McCo): to this unsuspecting group,
and he presented his one1hour lecture in the manner described, follo3ed b) a half hour
discussion period 3hich 3as hardl) more substanti!e&
At the end of his performance an authentic loo4ing satisfaction 9uestionnaire 3as
distributed to 3hich all 11 mental health educators 3ere as4ed to respond anon)mousl)
/Table 10& The introduction of the lecturer as 3ell as his lecture and discussion 3ere
!ideotaped for use 3ith other groups&
+ignificantl), more fa!orable than unfa!orable responses to the 9uestionnaire 3ere
obtained /chi1s9uare A 37&#", p B &CC10& The one item 3ith most fa!orable responses
3as the first, :'id he d3ell upon the ob!iousD: .t 3as the feeling of half the group that
he did& The remaining items recei!ed a ma8orit) of fa!orable responses& o respondent
reported ha!ing read 'r& Fo6<s publications& +ub8ecti!e responses included the
E6cellent presentation, en8o)ed listening& ;as 3arm manner& =ood flo3, seems
enthusiastic& -hat about the t3o t)pes of games, 5ero1sum and non15ero sumD Too
intellectual a presentation& M) orientation is more pragmatic&
Because the first group 3as fe3 in number and 9uite select, the authors sought other
sub8ects 3ith similar e6perience and professional identit) 3ho might pro!ide further
data to test the h)pothesis&
=R2*( ..
The second group consisted of 11 sub8ects 3ho 3ere ps)chiatrists, ps)chologists, and
ps)chiatric social 3or4ers, all identified as mental health educators& A !ideotape of the
pre!iousl) described lecture and discussion period as 3ell as the preparator)
introduction 3as sho3n to the group& After the presentation group members responded
to it using the same 9uestionnaire as did the first group /Table 10& Fa!orable responses
far out3eighed unfa!orable responses, and the difference bet3een the t3o 3as /chi1
s9uare A "?&73, p B &CC10& All responded fa!orabl) to the first item, 3hich means that
the) felt he did not :d3ell upon the ob!ious&: There 3ere also significantl) more
fa!orable than unfa!orable responses to the other items and one respondent reported
ha!ing read the lecturer<s publications& +ome sub8ecti!e statements 3ereE
'id not carr) it far enough& @ac4 of !isual materials to relate it to ps)chiatr)& Too much
gesturing& @eft out rele!ant e6amples& ;e misses the last fe3 phrases 3hich . belie!e
3ould ha!e tied together his ideas for me&
+till more sub8ects 3ere sought to further test the h)pothesis&
=R2*( ...
The third group 3as different in that it consisted of 33 educators and administrators
enrolled in a graduate le!el uni!ersit) educational philosoph) course& 2f the 33 sub8ects
in this group, %1 held master<s degrees, eight had bachelor<s degrees, and four had other
degrees 3hich 3ere not specified& Most of these educators 3ere not specificall) mental
health professionals but had been identified as ha!ing counseling e6perience in their
respecti!e schools& The !ideotape of the lecture 3as again presented to this group, after
3hich the educators responded to it b) using the same 9uestionnaire as the first t3o
groups /Table .0&
Again the number of fa!orable responses 3as significantl) greater than the number of
unfa!orable responses /chis9uare A 1C%&83, p B &CC10& The ma8orit) of respondents from
=roup ... also did not feel the lecturer d3elt upon the ob!ious, and the) also responded
fa!orabl) for the most part to the other items& +ub8ecti!e responses, 3hen gi!en, 3ere
again interesting& +ome 3ereE
@i!el) e6amples& ;is rela6ed manner of presentation 3as a large factor in holding m)
interest& E6tremel) articulate& .nteresting, 3ish he d3elled more on bac4ground& =ood
anal)sis of sub8ect that has been personall) studied before& Fer) dramatic presentation&
;e 3as certainl) capti!ating& +ome3hat disorgani5ed& Frustratingl) boring&
*norgani5ed and ineffecti!e& Articulate& Gno3ledgeable&
=i!en the responses of these three groups of educators to the lecture paradigm, the
authors belie!e that the stud) h)pothesis has been supported&
EHAM(@E+ 2F I*E+T.2+ *+E' A' (ERCETA=E 2F RE+(2+E+J F2R
T;REE =R2*(+
Iuestions =roup . =roup .. =roup ...
Kes o Kes o Kes o
'id he d3ell upon the ob!iousD 7C 7C C 1CC %8 $%
'id he seem interested in his sub8ectD 1CC C #1 # #$ 3
'id he use enough e6amples to clarif) his materialD #C 1C "? 3" #1 #
'id he present his material in a 3ell organi5ed formD #C 1C 8% 18 $C 3C
'id he stimulate )our thin4ingD 1CC C #1 # 8$ 13
'id he put his material across in an interesting 3a)D #C 1C 8% 18 81 1#
;a!e )ou read an) of this spea4er<s publicationsD C 1CC # #1 C 1CC
+pecif) an) other important characteristics of his

J :Kes: responses to all but item one are considered fa!orable&
The notion that students, e!en if the) are professional educators, can be effecti!el)
:seduced: into an illusion of ha!ing learned if the lecturer simulates a st)le of authorit)
and 3it is certainl) not ne3& .n a terse but appropriate statement on educators, (ostman
and -eingartner /"0 emphasi5ed that :it is the sign of a competent crap detector that he
is not completel) capti!ated b) the arbitrar) abstractions of the communit) in 3hich he
happened to gro3 up&: The three groups of learners in this stud), all of 3hom had
gro3n up in the academic communit) and 3ere e6perienced educators, ob!iousl) failed
as :competent crap detectors: and 3ere seduced b) the st)le of 'r& Fo6<s presentation&
Considering the educational sophistication of the sub8ects, it is stri4ing that none of
them detected the lecture for 3hat it 3as&
.n addition to testing the h)pothesis, the paradigm 3as to pro!ide these professional
educators 3ith an e6ample of being educationall) seduced and to demonstrate that there
is much more to teaching than ma4ing students happ), A balanced combination of
4no3ledge and personalit) are needed for effecti!e teaching e!en if the student does not
re9uire the former to sustain the illusion that he has learned& .t is hoped that this
e6perience has helped respondents from these three groups to 9uestion their educational
effecti!eness more meaningfull)&
To the authors< 4no3ledge a simulated teaching paradigm such as this 3ith student
responses to subse9uentl) percei!ed learning has not been reported& 'espite the usual
reser!ations about generali5ing data from onl) 77 sub8ects, the results of the stud) raise
some interesting 9uestions& The first in!ol!es the content of the lecture& 'oes a topic
seemingl) short on content and long on ambiguit) or abstraction lend itself more readil)
to such a lecture paradigm than a content1based factual presentation from a more
concrete topic areaD The ans3er is an e9ui!ocal :)es,: as a sub8ect in =roup . noted
after being told of the stud)<s design& ;e said he felt that the lecturer might ha!e had a
tougher time tal4ing nonsense about a more concrete topic but e!en under those
circumstances a fa4e lecture could be :pulled off: 3ith an unsuspecting group& This
raises the ne6t 9uestion&
.f the group 3ere more sophisticated about a more concrete aspect of the lecturer<s
sub8ect matter, in this case mathematics, 3ould he ha!e been as successful in seducing
the respondents into an illusion of ha!ing learnedD (robabl) not& 2r at least the lecturer
3ould ha!e to be e6tremel) s4illful to be successful& The stud) also raises the larger
issue of 3hat mi6 of st)le and substance in the lecture method is optimal for not 8ust
integrating information in a meaningful 3a) but for pro!iding learning moti!ation as
3ell& Although the stud) 3as not specificall) addressed to this 9uestion, the fact that no
respondents sa3 through the hoa6 of the lecture, that all respondents had significantl)
more fa!orable than unfa!orable responses, and that one e!en belie!ed he read 'r& Fo6<s
publications suggests that for these learners :st)le: 3as more influential than :content:
in pro!iding learner satisfaction&
A more ideal assessment of the relati!e !alue of content and st)le in determining
learner1reporter satisfaction might consist of programming the same :lecturer: to
s)stematicall) alter the content of his presentation before three e9ui!alent groups of
learners& +imultaneousl), his :in!oluntar) e6pressi!e beha!ior: 3ould remain constant
for each of the three groups, for e6ample, =roup A 3ould recei!e sufficient content
con!e)ed 3ith sufficient :in!oluntar) e6pressi!e beha!ior,: =roup B moderatel)
insufficient content accompanied b) the same :in!oluntar) e6pressi!e beha!ior: as 3as
displa)ed 3ith =roup A, and =roup C totall) inade9uate content deli!ered in the same
manner as to the first t3o groups, the three groups of learners could then be more
s)stematicall) compared as to learner percei!ed satisfaction&
After the respondents Jin the actual stud) 3ere informed of its purpose, numerous
sub8ects from each group re9uested the article from 3hich the lecturer 3as
programmed& Reported intent of these re9uests ranged from curiosit) to disbelief, but
the authors 3ere told b) some respondents that 'r& Fo6 did stimulate interest in the
sub8ect area e!en after the respondents 3ere told of the stud)<s purpose& 'espite ha!ing
been misinformed, the moti!ation of some respondents to learn more about the sub8ect
matter persisted& Conse9uentl) it is the authors< impression that the :arbitrar)
abstractions: suggested b) (ostman and -eingartner ha!e some initial pump1priming
effect on educational moti!ation&
The relationship of the illusion of ha!ing learned to moti!ation for learning more has
not been full) addressed here, but should a positi!e relationship e6ist, this stud)
supports the possibilit) of training actors to gi!e legitimate lecture as an inno!ati!e
educational approach to3ard student1percei!ed satisfaction 3it the learning process&
The corollar) 3ould be to pro!ide the scholar1educator 3ith more dramatic stage
presence to enhance student satisfaction 3ith the learning process& Either e6treme has a
soap1selling 9ualit) not li4el) to lather the enthusiasm of the pure scholar& ;o3e!er, this
paper is not addressed to him but rather to student1percei!ed satisfaction 3ith ho3 3ell
he has shared his information& More important, as has been noted, it suggests to the
educator that the e6tent to 3hich his students are satisfied 3ith his teaching, and e!en
the degree to 3hich the) feel the) ha!e learned, reflects little more than their illusions
of ha!ing learned&
1& =age, & @& /Ed&0& Handbook of Research on Teaching& e3 Kor4E Rand Mcall),
1#"3, pp& 7C", ?"?, and ?81&
%& =offman, E& The Presentation of Self in Everyday Life& e3 Kor4E 'oubleda), 1#7#&
3& Rogers, C& R& Bringing Together .dea and Feelings in @earning& Learning Today,
5E3%1?3, +pring, 1#$%&
?& Coats, -& '&, and +3ierenga, @& +tudent (erceptions of Teachers& A Factor Anal)tic
+tud)& J. Educ. Res., 5E37$13", April, 1#$%&
7& Rapoport, A& The *se and Misuse of =ame Theor)& Scientific !"erican, #$%E 1C81
11?, 'ecember, 1#"%&
"& (ostman, &, and -eingartner, C Teaching as a Subversive !ctivity& e3 Kor4E
'elacorte (ress, 1#"#, pp& 1117&