Tutoring in a Problem-Based Curriculum:

Expert Versus Nonexpert

Daniela Bochner, D.M.D.; Rachel L. Badovinac, D.M.D.;
Thomas H. Howell, D.D.S., M.M.Sc.; Nadeem Y. Karimbux, D.M.D., M.M.Sc.
Abstract: Many studies have examined whether being an “expert” influences the success of a tutor in a problem-based learning
curriculum. There are, however, no established standards by which to determine expertise. The purpose of this study was to
examine whether students evaluate expert and nonexpert tutors comparably and to determine whether setting different standards
to determine expertise influences the outcome of the above findings. Tutor evaluations, consisting of eight Likert-type questions
completed by first-, second-, and third-year dental students, were analyzed. Tutors were ranked by the authors within three
different categories of expertise based on the highest educational degree they had attained, familiarity with the specific subject
matter, and previous problem-based learning (PBL) experience. Linear regression analyses were then performed between each
category and student evaluation results. A statistically significant difference was found in the way students evaluated experts, but
only when expertise was defined by the tutor’s previous tutorial experience. The findings of this study underscore the importance
of the retention of dental faculty with PBL experience in a PBL-based curriculum.
Dr. Bochner is AEGD Resident, Office of Dental Education; Dr. Badovinac is Research Fellow, Department of Oral Health Policy
and Epidemiology; Dr. Howell is Dean of Dental Education, Office of Dental Education; and Dr. Karimbux is Director of
Predoctoral Curriculum, all at the Harvard School of Dental Medicine. Direct correspondence and requests for reprints to Dr.
Nadeem Karimbux, Office of Dental Education, Harvard School of Dental Medicine, 188 Longwood Avenue, Boston, MA
02115; 617-432-4247 phone; 617-432-3881 fax; nadeem_karimbux@hms.harvard.
Key words: dental education, medical evaluation, evaluation studies, problem-based learning, evaluation, tutor training
Submitted for publication 4/25/02; accepted 9/6/02

P
roblem-based learning (PBL) was introduced
in courses at the Harvard School of Dental
Medicine (HSDM) in 1994. The motivation
behind the adaptation of this educational method was
to encourage enthusiasm and ability to engage in self-
directed learning and to impart an excellent under-
standing of basic science and aptitude in incorporat-
ing psychological and social considerations into the
care of patients.1 PBL was instituted at HSDM via
the New Pathway program, which coordinates lec-
tures, labs, and clinical sessions in an effort to syn-
ergistically increase student understanding, skill, and
motivation.
Integral to the success of this and other PBL
programs is the role of the tutor. In one model of
how PBL functions, the tutor’s performance is con-
sidered (along with students’ prior knowledge and
the quality of the cases discussed) to be a prime de-
terminant of how the tutorial group functions.2
Due to the inherent importance of the tutor to
PBL, there is great interest in identifying character-
istics that contribute to tutor success, which ulti-
mately leads to student success. One specific char-
acteristic that has stimulated major debate is the
concept of the content expert. The basis of the de-
bate is the central importance of the development of
students’ skills in active, self-directed learning in
PBL. According to Barrows, facilitation is key to the
role of the PBL tutor and is critical for student learn-
ing.3 There is concern, however, that content experts
may not be satisfied merely to facilitate, but may also
tend to communicate subject information, thereby hin-
dering group collaboration and self-directed learning.
Schmidt et al. found that content-expert tutors used
their expertise more while nonexperts used their pro-
cess facilitation skills more.4
In a study of tutors from Harvard Medical
School, Silver and Wilkerson found that tutors with
expertise tended to take a more directive role in tu-
torials: they spoke more often and extensively. In
addition, tutor-student interactions predominated
along with fewer student-student interactions.5 An-
other study of the New Pathway at Harvard Medical
School by Calvin and Wetzel found that tutors do
not have to be experts to be successful.6
On the other hand, there are some studies that
find positive benefits in having expert tutors. Davis
et al. found that student test performance was in-
creased when tutors were active in research or had
advanced disciplinary training in the tutorial subject
matter.7 Schmidt et al. reported that students guided
by experts spent more time on self-directed study

1246 Journal of Dental Education ■ Volume 66, No. 11

and had better achievement scores.4 Similarly, Eagle
et al. found that students tutored by experts produced
more than double the issues for self-directed learn-
ing and spent almost double the amount of time en-
gaged in self-study.8 However, Swanson et al. found
that tutor experts had no measurable effects on stu-
dent performance.9
One factor that may contribute to the inconsis-
tent findings is that there are no established standards
by which to determine expertise. Due to the amorphous
nature of the term “expert,” every study established its
own standards by which to determine expertise.
For example, in a study by Schmidt and Moust,
expertise was measured by students’ responses to a
questionnaire that included statements such as “The
tutor used his subject-matter knowledge to guide the
group.”10 In another study by Schmidt, “nonexperts
were tutors whose previous training was unrelated
to the problem at hand” and “tutors who had general
background knowledge regarding the unit but not
specific expertise. Experts were tutors with fairly
specific background knowledge relevant to the
unit.”11 In the study by Davis et al., experts were iden-
tified as faculty who had an active research interest
in the topic at hand.7 In a study by Kaufman and
Holmes, the tutors themselves determined their sta-
tus as experts.12 Both Regehr et al. and Eagle et al.
used the tutor’s experience with a particular case,
whether through case development or clinical prac-
tice, to determine expertise.8,13
Our study had three purposes: first, to investi-
gate whether students evaluated expert and nonex-
pert tutors comparably; second, to determine whether
setting different standards to determine expertise in-
fluences the outcome of the above findings; and third,
to investigate whether the effect of expertise on evalu-
ation is modified by student year (i.e., first-third).
Methods
Data Collection
Between 1996 and 2001, all first-, second-, and
third-year students at HSDM were randomly assigned
to tutorial groups for each of their blocks. First-year
and third-year students participated in five blocks,
and second-year students participated in three blocks.
At the end of each block, each student was requested,
but not required, to complete an anonymous evalua-
tion of his or her block tutor. The tutor evaluation
November 2002 ■ Journal of Dental Education
consisted of a series of eight statements (Q1-8) de-
scribing various aspects of tutor performance in the
tutorials. Students assessed the statements on a five-
point Likert scale, with one defined as excellent and
five defined as poor. Over the period of evaluation
for this study, a total of sixty-five course evaluations
should have been available for analysis. Forty-seven
course evaluations were actually collected and avail-
able for analysis. A total of ninety-seven different
tutors were available for analysis; three of them could
not be classified according to the classification sys-
tem used, so ninety-four tutors were looked at, in a
total of 186 tutorial sessions.
Out of a potential total of 1,504 evaluations (47
available courses x 4 tutorials x 8 students on aver-
age per tutorial), 1,099 were analyzed for this study—
73 percent of the total available. The 27 percent not
analyzed included responses that were not returned
or filled out by students (students were not required
to fill out evaluations during the years of this study)
and evaluations that were incomplete.
Definitions of “Expert”
For the purposes of this study, the term expert
was defined as “having, involving or displaying spe-
cial skill or knowledge derived from training or ex-
perience” (www.merriamwebster.com). The authors
developed three classifications by which to measure
expertise and classified each tutor (for each block
and year) accordingly (Table 1). The standards of
expertise are based both on classifications developed
by previous investigators and on the authors’ estima-
tion of the classifications of expertise that are most
applicable to the dental tutorial. The three different
standards are based upon the highest level of educa-
tion attained by the tutor (classification A), tutor fa-
miliarity with a particular subject area (classifica-
tion B), and previous tutoring experience
(classification C). (See Table 1.)
Within classification A, subclass 1 tutors have
either a D.M.D., D.D.S., or Ph.D. degree only; sub-
class 2 tutors have either a D.M.D. or D.D.S. and are
currently engaged in specialty training or have a Ph.D.
and are engaged in postdoctorate work; and subclass
3 tutors have either a D.M.D. or D.D.S. and have com-
pleted their specialty training or have a Ph.D. and
have completed their postdoctorate work.
Within classification B, subclass 1 tutors have
no previous exposure to the specific subject area dis-
cussed in the tutorial; subclass 2 tutors have mini-
mal exposure to the subject area, with no continuing
1247
 Table 1. Classifications of expertise
Expert Classification A B
Highest Educational
Basis of Classification Degree Attained
Score 1 D.M.D./D.D.S. or
Ph.D. degree only
2 D.M.D./D.D.S. engaged
in specialty training,
Ph.D. engaged in postdoc
3 D.M.D./D.D.S. or Ph.D.
with specialty training or
postdoc complete
education or practice in the area; and category 3 tu-
tors have exposure to the subject area, with continu-
ing education and practice of area. The exposure that
classification B is based upon could be through clini-
cal or research activities or through case develop-
ment. Given the subjective nature of this classifica-
tion system, one of the authors who is familiar with
the educational, research, and clinical history of each
tutor personally and from written records made the
categorization of each tutor within classification B.
A second investigator also independently verified this
classification. Any mismatches or tutors that could
not be classified were not included in the study (a
total of three tutors).
Within classification C, subclass 1 tutors have no
previous tutoring or PBL experience, subclass 2 tutors
have either tutored once before or were PBL students
in the past, and subclass 3 tutors have tutored more than
once before. It should be noted that lower number of
subclasses indicates lower levels of expertise.
The potential for a tutor’s personal engagement
in clinical or scientific pursuits and the interdiscipli-
nary nature of dentistry underlie the importance of
distinguishing between categories A and B. For ex-
ample, consider an introductory block intended to
introduce students to patient diagnosis through his-
tory taking, physical exam, and radiographic analy-
ses. There is little reason to think that a prosthodon-
tist (by degree) has more expertise in this area than a
general dentist, but the prosthodontist would be de-
termined by our ranking system to have greater ex-
pertise within category A.
Statistical Analysis
The statistical analysis assumes there were no
differences between the tutor evaluations that were
and were not submitted by students. Univariate lin-
ear regression analyses were performed between each
1248
C
Familiarity with Previous Tutorial
Specific Subject Experience
No exposure to specific No tutorial experience
subject area
Minimal exposure to Tutored once before or was
specific subject area, with PBL student in past
no continuing education and
practice in subject area
Maximum exposure to subject Tutored more
area with continuing education than once before
and practice of subject area
independent variable, classification A, B, and C, and
academic year, and the outcome variable, Qmean
(={Q1+Q2+Q3+Q4+Q5+Q6+Q7+Q8}/8 for all of
the evaluations). For the univariate analyses, classi-
fications A, B, and C were treated as ordinal cat-
egorical variables. Academic year was treated as a
nominal categorical variable, using two dummy vari-
ables simultaneously entered into the model. For each
outcome, multivariate models were built using aca-
demic year and the expertise variable, classification
A, B, or C, closest to statistical significance. In the
case of the multivariate models, the expertise vari-
ables were treated as nominal categorical variables.
For each model, a product term between ordinal ex-
pertise and ordinal academic year was then added to
assess for multiplicative interaction. In every case,
alpha was set at 0.05. All statistical analyses were
conducted using the STATA statistical package.
Results
In this study, 39.78 percent of the tutorial evalu-
ations were completed by first-year students, 23.12
percent were completed by second-year students, and
37.10 percent were completed by third-year students.
Table 2 summarizes the statistics of the 186 tutorials
that were analyzed. With respect to classification A,
nine tutorials (4.84 percent) had tutors that were in
subclass 1, eighty-seven (46.77 percent) tutorials had
tutors that were in subclass 2, and ninety tutorials
(48.39 percent) had tutors that were in subclass 3.
With respect to classification B, one tutorial (0.54 per-
cent) had a tutor that was in subclass 1, 104 tutorials
(55.91 percent) had tutors that were in subclass 2, and
eighty-one tutorials (43.55 percent) had tutors that were
in subclass 3. With respect to classification C, sev-
enty-four tutorials (39.78 percent) had tutors that were
Journal of Dental Education ■ Volume 66, No. 11
Table 2. Summary statistics of discrete variables
Discrete Variable Frequency Percent Cumulative Percent
Expert Class A 1 9 4.84 4.84
2 87 46.77 51.61
3 90 48.39 100.00
Expert Class B 1 1 0.54 0.54
2 104 55.91 56.45
3 81 43.55 100.00
Expert Class C 1 74 39.78 39.78
2 49 26.34 66.13
3 63 33.87 100.00

Table 3. Summary statistics of continuous variables

Continuous Variable Observations Mean SD
Qmean* 184 1.76 0.64
Q1 186 1.66 0.64
Q2 186 1.61 0.65
Q3 186 1.73 0.70
Q4 185 1.66 0.67
Q5 185 1.80 0.79
Q6 185 2.18 0.85
Q7 184 1.73 0.69
Q8 185 1.70 0.69
* Qmean=Q1+Q2+Q3+Q4+Q5+Q6+Q7+Q8/8

in subclass 1, forty-nine tutorials (26.34 percent) had
tutors that were in subclass 2, and sixty-three tutorials
(33.87 percent) had tutors that were in subclass 3.
Table 3 summarizes the students’ ratings. Out
of the 186 tutorials assessed, 184 tutorials had evalu-
ators that responded to all of the statements. The mean
grade for Qmean was 1.76 with a SD of .64. The means
of the individual statements ranged from 1.61 to 2.18.
Table 4 summarizes the results of the univariate
linear regression analysis of Qmean and classification
A, classification B, classification C, and academic
year. There is no apparent linear relationship between
Qmean and either classification A, classification B, or
academic year when considering the responses of
first-year students compared to those of third-year
students. However, there is a statistically significant
linear relationship between Qmean and both classifi-
cation C (coefficient -.111, SE .054, p-value .043,
95 percent CI {-.218, -.003}) and the responses of
first-year students compared to those of second-year
students (coefficient -.374, SE .120, p-value .002,
95 percent CI {-.670, -.138}).
Table 4 also summarizes the results of the mul-
tivariate linear regression analysis of Qmean with first-
time tutors compared to tutors with one previous tu-
torial experience (classification C1-2), first-time
tutors compared to tutors with more than one previ-
ous tutorial experience (classification C1-3), first-
year students compared to second-year students
(Acad. Yr. 1-2), and first-year students compared to
third-year students (Acad. Yr. 1-3). A statistically sig-
nificant linear relationship was found between Qmean
and both classification C1-3 (coefficient -.211, SE
.107, p-value .050, 95 percent CI {-.421, .000}) and
Acad. Yr. 1-2 (coefficient -.364, SE .119, p-value
.002, 95 percent CI {-.599, -.130}).
Table 5 summarizes the results of linear regres-
sion analysis between Qmean and classification C tu-
tors as they progress from subclass 1 to subclass 2
and from subclass 1 to subclass 3 for each academic
year. A statistically significant relationship exists
between Qmean and the advancement of tutors from
subclass 1 to subclass 3 in the third academic year
(coefficient -.425, p-value .026).
Discussion
In this study, a statistically significant differ-
ence was found in the way students evaluated ex-
perts, but only when expertise was defined by the
tutor’s previous tutorial experience (classification C).
There is also a linear relationship between student
evaluations and academic year in that as students
progress from year one to year two, Qmean decreases,
indicating a more favorable evaluation of tutors. Fi-

November 2002 ■ Journal of Dental Education 1249

 Table 4. Linear regression analysis of Qmean
Qmean
Univariate Coefficient SE p-value 95 percent CI
Class A .067 .064 .295 -.059 .194
Class B .057 .093 .542 -.126 .239
Class C -.111 .054 .043 -.218 -.003
Acad. Yr. 1-2* -.374 .120 .002 -.670 -.138
Acad. Yr. 1-3 -.148 .105 .161 -.355 .059
Multivariate Coefficient SE p-value 95 percent CI
Class C, 1-2** .001 .114 .993 -.224 .226
Class C, 1-3 -.211 .107 .050 -.421 .000
Acad. Yr. 1-2 -.364 .119 .002 -.599 -.130
Acad. Yr. 1-3 -.149 .104 .154 -.354 .057
*Acad. Yr. 1-2 signifies the change in the dependent variable as students progress from academic year one to academic year two.
** In this analysis, Class C, 1-2 signifies the change in the dependent variable as tutors progress from classification C1 to C2.

Table 5. Linear regression analysis of Qmean for each academic year

Class C, 1-2 Class C, 1-3
Academic Year Coefficient p-value Coefficient p-value
1 -.104 .60 -.177 .343
2 -.044 .76 .057 .676
3 -.425 .44 -.425 .026

nally, a significant linear relationship was found be-
tween Qmean and tutor progression from classification
C1 to C3 when third-year students evaluated tutors.
It is interesting to note that as students progress
from year one to two, they grade their tutors more
favorably, whereas a statistically significant relation-
ship could not be established between Qmean and stu-
dent progression from academic year one to year
three. These findings indicate that between year two
and three Qmean increases, indicating that tutors are
evaluated less favorably. It would be interesting to
determine whether using different classifications of
expertise alters the effect of academic year on Qmean.
One possible explanation for this is that as students
gain PBL experience, they become less dependent
on their tutors to facilitate group function and more
interested in obtaining directed, hard information.
As mentioned in the above findings, Qmean de-
creased significantly as tutors progressed from year
one to three, but there was no significant increase
between years one and two. This may indicate that it
takes at least two prior tutoring experiences until
experience contributes to improved tutoring. On the
other hand, it is also possible that poor tutors are
somehow “weeded out” so that the proportion of in-
ferior tutors decreases as classification C progresses
from C1 to C3.
When we assessed whether the effect of expe-
rience on Qmean varies by academic year (Table 5),
we found that the only statistically significant rela-
tionship occurred when there was progression from
classification C1 to C2 during the third year. This
may indicate that experience only affects student
grading in the third year. However, in limiting the
analyses to specific academic years, sample size and
power are decreased, and the statistical insignificance
of p-values may be misleading. This could be rem-
edied by repeating the analysis in the future when
more tutor evaluations are available.
Another area of concern is what seems to be
generous grading of tutors by students. As indicated
in Table 3, the mean for Qmean was 1.76, which seems
to be skewed considering that the grading scale was
from 1 (excellent) to 5 (poor). While we assumed
that the evaluations that were completed were done
at random, a greater spread of scores may aid in iden-
tifying statistically significant relationships.
As seen in Table 2, only one tutor was classi-
fied as B1, and only nine tutors were classified as
A1. These low numbers, especially the B1 category,
are probably related to the interdisciplinary nature
of dentistry. Regardless of their specialties, most
dentists have some familiarity with many aspects of
dentistry. Also, with regard to the low numbers in

1250 Journal of Dental Education ■ Volume 66, No. 11

the A1 category, the academic environment may at-
tract more specialists than general dentists. Increas-
ing the number of tutors in these categories may re-
veal additional statistically significant relationships.
While it is difficult to ascertain the ultimate
causal factor for some of our findings, it is clear that
the retention of tutors—so that they can gain experi-
ence from tutorial to tutorial—is a vital strategy in
the training of future dentists in a PBL environment.
REFERENCES
1. Moore GT, Block SD, Briggs-Style C, Mitchell R. The
influence of the New Pathway curriculum on Harvard
medical students. Acad Med 1994;69:983-9.
2. Gijselaers WH, Schmidt HG. Development and evalua-
tion of a causal model of problem-based learning. In:
Nooman ZM, Schmidt HG, Ezzat ES, eds. Innovation in
medical education: an evaluation of its present status. New
York: Springer Publishing, 1990:95-113.
3. Barrows HS. How to design a problem-based curriculum for
the pre-clinical years. New York: Springer Publishing, 1985.
4. Schmidt HG, Van Der Arend A, Moust JHC, Kokx I, Boon
L. Influence of tutors’ subject matter expertise on student
effort and achievement in problem-based learning. Acad
Med 1993;68:784-9.
5. Silver M, Wilkerson L. Effects of tutors with subject ex-
pertise on the problem-based tutorial process. Acad Med
1991;66:298-300.
November 2002 ■ Journal of Dental Education
6. Calvin RB, Wetzel MS. Pathology in the new pathway of
medical education at Harvard Medical School. Am J Clin
Pathol 1989;92 Suppl:S23-S30.
7. Davis WK, Nairn R, Paine ME, Anderson RM, Oh MS.
Effects of expert and non-expert facilitators on the small-
group process and on student performance. Acad Med
1992;67:470-4.
8. Eagle CJ, Harasym PH, Mandin H. Effects of tutors with
case expertise on problem-based learning issues. Acad
Med 1992;67:465-9.
9. Swanson DB, Stalenhoef-Halling BF, van der Vleuten
CPM. Effect of tutor characteristics on test performance
of students in problem-based curriculum. In: Bender W,
Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, eds. Teach-
ing and assessing clinical competence. Groningen, The
Netherlands: Boek Werk Publications, 1990:129-34.
10. Schmidt HG, Moust JHC. What makes a tutor effective?
a structural-equations modeling approach to learning in a
problem-based curricula. Acad Med 1995;70:708-14.
11. Schmidt HG. Resolving inconsistencies in tutor exper-
tise research: does lack of structure cause students to seek
tutor guidance? Acad Med 1994;69:656-62.
12. Kaufman DM, Holmes DB. The relationship of tutors’
content expertise to interventions and perceptions in a PBL
medical curriculum. Med Educ 1998;32:25-261.
13. Regehr G, Martin J, Hutchison C, Murnaghan J, Cusimano
M, Reznick R. The effect of tutors’ content expertise on
student learning, group process and participant satisfac-
tion in a problem-based learning curriculum. Teach Learn
Med 1995;7:225-32.
1251