@ MEDICAL SCHOOL FACULTY Moderator: James O. Woollscroft, MD Relationships of How Well Attending Physicians Teach to Their Students’ Performances and Residency Choices CHARLES H. GRIFFITH Ill, JOHN F WILSON, STEVEN A. HAIST, and MARY RAMSBOTTOM-LUCIER Like teachers inal diciptine, medical edvemtors hope that their teaching tet in learning. We would like to think if we teach vel, sent Teaming and performances are enhanced. However, the effec of "unity tescing” on siden performances remains empirically unproven Coleg students have scored higher on ex- Sminatons in clases in which they gave istractrs higher rt ings.'- However, these correlations have been criticized because they involved elective clases. Thus motivated student interested in Bria leratare may do better ina British Iteatue class, and the beter performance may be from inherent interest or enths- fm rather than being a rfection ofthe quality of the teacher ikewse the higher ratings given insrictors may feet enths- tum forthe subject and not tir ably os teachers. Several sds fave tented behavior of medical instore amocinted wth beter student ratings such bchaviors inline spending time with student, che ability to capture the lamers attention and involve alllesrmers, understanding the learners needs and knowledge bse, ‘and using case-based teaching scripts.~* However, whether the students of these highly skilled instructors do better on objective treasures of performance ls unknown. Similarly, exporire to outstanding ole models or oustanding teachers is thought to Inence medical siden? carer choices ‘When graduating medical talents are qucied about earerchoie, clerkship role models have cceasinaly been cited rerospectey a, inlvencal faccors=* However, i ificule to know i this i ‘merely fond remembrance or acwal nfluenee. Further, we Jo not lino whether oer sucensexpored othe sme role models were sore likly t0 choose the ame typeof residences or career In dion, several retrospective states have reported that neprive leeahip experiences with role models were infgntal in exter decisions." Two prospective suds fund no influence on caret choice of precincl exposure to family practice instrctor!* Exe posure toa general incr atending physician during the thid- year clerkship has been associated with post-clerkship career choice Of general intemal medicine." owever, «quantitative prospective analysis of the influence of outstanding (or poor) clerkship role todel or eachers on residency choice hs not been reported. ‘The purpose of our project was to quantify the effects of relative teaching quality during athitdyer internal medicine clership on students’ performances and residency choices. We hypothesized that third-year students exposed to one of the “best” teaching at- tending phyilans during the inpatient iteral medicine leship would (1) perform better on clerkship examinations and (2) be tore likely to choose residences n internal medicine ra priay Core dsipline. Conversely, we hypothesed that third-year se dns exposed io one ofthe “wont” teaching attending physicians drng the inpatient intemal medicine cerahip woud (1) peform worse on clerkship examinations and (2) be les Uikly co choose residences in internl medicine ora primary eae diipine. Method Subjects were all 169 students who rotated on the third-year inter. nnal_medicine cleskship during academic years 1993-94 and 1994-95. The intemal medicine clerkship consists of two four week rotations on the general medicine inpatient service, one at sus Acapsmic Mepicine, Vol. 72, the university hospital, andthe other atthe afliated Veteran's Af fairs hospital. (Ambulatory medicine is part ofa separate primary care clerkship, and was not included in this study.) During the medicine clerkship, students were randomly assigned to one of nine ‘general medicine teams. A team consists ofan attending physician, 8 supervising resident, evo interns, and two students. Intensive care and coronary care patients are on separate services without stu- dents. Sixty-two faculty attended on the general medicine inpa- tient service during this period. Specialties of the attending physi- cians were: 21 general internist, ten oncologists, six nepheologist, six endocrinologist, five gastroenterologiss, five infectious disease specialists, chree sheumatologiss, and six medicine chief residents ‘Atending physicians lead daly teaching rounds with the ream, dis: ‘casing management isues regarding the patients on service, either at the bedside or in conference. In addition, attendings are ex- pected to conduct separate teaching rounds with the team eheee times a week, 1.5 hours per session, where focus is on individual pa tients or copies. Attendings are also encouraged to meet separately with students for studentlevel discussions, although this is not mandatory. Thus, students have 15-20 hours of contact with tendings each week, generally in the context of patient care, cluding 2 minimum of fou ro five hours of formal instruction “Teaching quality was measured using a 16-tem, departmentally ‘generated evaluation form completed by students at the end of tach four-week rotation, Students were required to complete the form at the end ofeach month’ rotation, independent of and gen- erally before they had seen their evaluations by faculty. Students, evaluated the attending physician on teaching skills and ability, rapport with students, residents, end patients, and gave an overall rating 8 a teacher and role model. Bach item was rated on a five- point Likert-type scale, with 1 = strongly disagree, 2 = disagree, 3 = neutral, 4= agree, and 5 = strongly agree. The coefficient ‘pha forthe evaluation form was 0.96, indicating that the items of ‘our form have a high degree of internal consistency for rating fc- uly. However, there is little discrimination forthe individual items ‘on the form, with inter-item correlations ranging from 0.71 100.95. ‘Therefore, because ofthis lack of tem discrimination, fr this seudy wwe chose to use the mean rating across the 16 items ofthe form as fone overall rating of teaching quality. The overall faculty score was the mean of the ratings from all the thid-year students the faculty precepted during the two-year study period. ‘We did not expect a perfectly linear relationship beeween teach- ling qualicy and studene performance or residency choice. We be- lieved much of student performance and residency choice would be from innate student qualities (drive, intelligence, interest in pri- mary cate), but that ata certain threshold of teaching ability (both ‘good and bad), the quality ofthe attending physician could be in- fluendal, Therefore, we constructed categories of teaching quality. ‘A priori we defined “best” teaching attendings as those receiving the top 20% of overall student evaluations in the two-year period, and “worst” teaching attendings as those receiving the lowest 20% ‘of overall student evaluations. We chose 20% as the upper and lower cutoff because we believe chat most faculty (> 50%) are nei ther the best nor the worst teachers ‘Que measures of student performance included (1) che student's score on the National Board of Medical Examiners (NBME) Sub- No.10/OctoveR SureLewent 1 1997ject Examination in Internal Medicine, taken at the end of the clerkship, and (2) the student score on a linieal performance ex- amination (CPE) caken at the end ofthe clerkship. The CPE is an 8-station (15 minutes per station) examination, with six to eight stations involving standardized patients, and the other stations testing various clinical skils (eg, x-ray and ECG interpretation). Reliability of the CPE is generally 60-70. In addition, an NBME subject examination in medicine is administered the fist day of the clerkship. Our third measure of student performance was che dif ence in score on the pre-clerkship and postclerkship NBME sub- ject examinations. Residency choice was defined by match-day re- sults. For this study, primary eare was defined as internal medicine, pediatrics, medicine—pediatrcs, and family practice ‘Analysis of student performance was with focused contrast analy- sis of covariance, comparing students exposed to "best” (or “worst”) teachers with other students, controling for the pre-cleskship [NBME subject examination score. Career choice was assessed with Lat chi-square. Results ‘One hundred sixty-nine students completed 291 evaluations (86% response rate) regarding 62 faculty. The mean number of evalua tlons per faculty member was 4.7. Thirty-one of the 62 attending physicians (50%) had four or fewer evaluations, 22 (34%) had five to eight evaluations, and nine (15%) had more than eight evalua- tions. The point precision of our estimate of faulty teaching ability approached a confidence interval of #10 SEM with seven to eight evaluations pe faculey: However, ii important that the number of| evaluations for a faculey member was not significantly correlated with the mean evaluation score (t= «17, p> 20), and therefore all faculy were included in the analysis regardless of the numbers of students they precepted. Twelve ofthe 62 faculty were rated "best” teachers (top 20% of evaluations). These 12 faculty precepted 62 students in the two years, receiving a mean rating of 4.80 (SD 31), with 4 = agree, 5= strongly agree. The 12 faculty rated as “worst” teachers precepted 56 students, with a mean rating of 3.62 (SD “11, with 3 = neutral, 4= age. ‘Table 1 presents measures of cudent performance and residency choice depending on exposure to "best," “worst,” or neither "best” nor “worst” attending physicians ("other"). Eleven students were ‘exposed to both a “best” and "worst teacher, and are included in both categories. As presented, students exposed to “best” teachers hhad significantly greater increases in theit pre- to post-lerkship [NBME subject examination scores. Conversely, students exposed to Tate 1. Students’ Performances and Residency Chloes asa Function of Teac Univesity of Kentucky Col “worst” teachers had lower CPE scores and were less Ukely to choose an internal medicine or a primary care residency. Suba ss showed that 80% of the negative impact on residency choice ‘occurred from exposure to a “wor” attending physiian on the ini- tial inpatient roation. Although several of the "best ceachers were ‘general internists, exposure toa general internist did not influence students’ performances or residency choices. The speci specialty of the attending physician was likewise not influential Discus (Our study suggests that teaching quality indeed has an impact on both seudents’ performances and theie residency choices. Interest- ingly, exposure to “best” teachers and “worst” teachers influenced liffeene domains of studenc performance. Students exposed co "best” teachers performed better on a measure of knowledge at tained during the clerkship, as shown by the increase in pre- 0 post-clerkship scores on the NBME subject examination. Our mea- sure of teaching "quality" encompassed other factors besides each- ing (rapport with residents, students, and patients} and we may be ‘measuring overall attending quality, noe just their ability to teach. However, better teachers may be more adept at establishing a posi tive learning climate, which may include respect for learners and patients. In this better learning climate, students may be more simulated to learn and become more enthusiastic about learning, demonstrated by greater improvement in knowledge. And indeed, although rapport with leaners and patients isnot “teaching” in the sense of transmitting information, role modeling in interactions with patients and leaeners is something from which we hope most ‘of our students and residents lean. Conversely, expose €0 "worst teachers resulted in worse performances on 2 measure of clinical skill, the CPE. Many of the stations on our CPE involve inter viewing skis, and perhaps students exposed to “worst” faculty had less fective role modeling in doctor~patient interactions and communication, resulting in worse performances on the CPE. Likewise, exposure to "best” and “worst” teachers had diferent influences on residency choice. We found no influence of *best” teachers on residency choice, while exposure 0 "worst" teacher was asoclated with being less likely to choose a residency in pri- ‘mary care or intemal medicine. Our findings prospectively confirm the work of Karz, MeMurray, and others who found that role models were important at a negative factor in carer decisions. We found that career decisions were influenced by exposure to these poorly rated faculty: students who had an exposure to a “worst” teacher were much les likely to enter an internal medicine resi- 9 Quallty, 169 Students, lege of Medicine, 1963-1995" Best Teather the Teacher Worst Teacher (a= 12Facaly) (n= 38 acl) (a= 12 Facuy) Exposure xgasure Exposure (n= 62Studons) (a= 164 Studens)__(n-= 56 Students) 2 Perormance NBME subject exam score 455, 449 454 039 Ditferencen pre-toposclership NBME subject exam score att 4 120 0.05 Score on cliaal performance eam. 855 5.1 a36t 008 Fesidency choice % Choosing internal meine a 254 ort oot % Choosing primary eae 500 508 54h 9.03 “Tota ata show ht nhs sty techn uly voy inuece tte perfomance and esi ccs Sete at or epanaion "best each” ae. {Meas tals sian ere ‘arty aces, eral mene, etic, made -pedtes. Acavemic Mepieie, Vot. 72, No.10/Ocrowen Surrtenenr 1 1997 susdency, In our study, most of the negative effet on residency choice ‘occurred during the first ofthe two month-long rotations, suggest- ing that early clerkship exposures have the most impact. ‘One might argue that students who were not inclined to choose primary-care residencies would rate faculty lower, resulting in lower ‘overall faculty ratings for those faculty, and a spurious association of ‘poor ratings with carer choice. Indeed, we have no data on su- dents’ pre-lerkship interests in medicine or primary care, Al- though’ few individual students may have rated attendings based primarily on their own relative interests in medicine, we believe this did not signfieantly influence our results for three reasons: (1) ‘Students had ewo different atzending physicians, and only six (45%) ‘of the students were exposed ro ewo poorly raced attendings. One ‘would suspect an anti—primary-care student would rate both of thei faculty preceptors low, and the other 50 students exposed on. ‘one rotation fo a “worst” tescher did not have exposure toa second poorly rated teacher. (2) *Worst” teacher exposure affected not ‘only career choice, but also measures of performance. Students ex- posed to “worst” attendings had as good or better pre-clekship [NBME scores as the other students, suggesting that students who prior to the clerkship performed as well or beter worked with the worse” attendings. Nevertheless, they performed worse on our CPE. (3) Most students in our study did not choose internal medi cine or primary cate residencies, even those exposed to “best” teachers. Therefore, ro get high ratings, faculty had to impress stu- ddents who did not chose primary care residencies. Conversely, 35% of the students exposed to “worst” attendings chose primary care residencies, and for faulty ro receive such poor ratings, they had to bye rated lower by most ofthe students, even the primary care sti dents Several limitations to our study must be considered when inter- preting the results. Fitst, cis study is fom one instittion and one discipline, and may reflect a particular characteristic of our instiu- ton’ faculty or students. Future studies should evaluate effects of teaching quality in varicy of disciplines and institutions, and in- clude the influence of residents’ of interns teaching quality. See: ‘ond, our study did noe account forthe possible influence of ambula- tory teaching, Our ambulatory experience for students in internal medicine involves an individual student working with a variety of attendings in a variety of clinics and in a separate clerkship, thus making attribution of influence dificult. Institutions where the ‘outpatient experience i limited to one or afew faculty per student could consider a study like ours on the influence of ambulatory teaching quality. Third, our measure of attending teaching “q ity” is based solely on students’ evaluations. A mote comprehens ‘measure of faculty teaching ability may include input from other sources, such as peers and residents. However, in our institution faculty rarely observe each others teaching sills, and most ofa fac- ulty members teaching reputation is based on the reputation gener- aed! by his or her learners: the students and residents. And given the substantial intermingling of students and residents, we would expect residents and students to be similar in thei judgments of “pes” and “worst” faculty. Nevertheless, our study only encom- passes student ratings. Fourth, the difference inthe CPE scores for students exposed 0 a “worst” teaching attending was 15-17 points. While this is modest, it has a significant influence on thei arade ({e, B ro B minus in our grading scale), and for many stu- dents this one or two points is perceived as highly significan. Fifth, residency choice can be considered an intermediary outcome, as it is not necessarily the same as ultimate career choice. However, if students do not choose a primary eare residency, then a career in si20 Acapemic MeoiciNe, Vot primary care will not fllow, Postgraduate experiences may be of ual or greater importance for ultimate carer choice. "Despite thee limitations, we conclude that relative teaching wali can have a messrable impact on stulents performances hd redeney choles. Inthe current climate of incensed fecal ac ‘ounmbiliy, academic medical enters wll have to jl co chem: Selves and managed eare companies thatthe resources allored to their educational mision are newded. Our study provides such an tempi justification. Further, our esearch can provide the bats to \denifycharcteriics and teaching behavior of aay who pose tively tile student’ performance and career choices. Faculty ‘who are notated highly in these areas ean be offered faculty devel {pment in teaching sil. Previous work with faculty development in teaching skills hasbeen limited to slftepor of parcpants Ieremaine tobe seen whether faculty development wll objectively lead to improved teaching. Therefore, fr wae application of fc tity development, our study can provide a basis to measure more prevnly the effec of fecaty development on teaching ability, Tending validity to efforts a instruction in teaching, and tying some ofthe cot deffo of faculty development. Carciponene: Cares Heney Gi I, MD. S13 Kemcky Cline, Lexington, 40596 emai Refs 1. Gahan PA Sean tins of set an en seem a tana tof titan valli tales Rev Bloc Ret 198151261300. 2. Feld KA. Grd lle sale craton of te cone ad teach, esther Ele 1976460-111 5, Cathin WE. IDEA Teche Report No 32: Slee Rags of Teaclng the ewe Revel Manhattan, KS: Kaa State Univer, Coe ely elton and Development, 1995 4 iby DMO Whar clea! sccher: tn mle mead bom Acad Me 194403342 5. Ramey, ila OM, ly DM. Erato link eching inthe mlicne lasipsreaoshpo isto expsene ond ering eng ings teckel.) Go tern Ma 1988933515. 6 Camps Onc D, St, Watkins A), Basti 8. Theale of mada shool comclum fly le cde and boreal reach spo cote Stale pyean caters 3 review an quay aumento the ler: Acad Mek 199570611-9. 1. Wills CC, Wiener NW, Matas KM, Reve JM, Del EL. Mae eden? ‘neiton real meine) Oc ter Mel 199493275 8, Moni, Vela J, Brncy 8, Xo G Fels SK. Masao ad sent orci that infenecsing a enact JAMA. 1D46I 8 9, Roger LQ, Fincher ME, Lewis LA, Ney LD. A sur tdterin et i Ahonelng mail salons cee cle, Aca Med. 2.447 10, Kou LA, Sitch’ RE, Schuur F The ale of magne facts io changes in carer dice by medal sans} Med Blue 198459285-90. 11, Meslay J, Sra: MD, Genero NF, Lise M. The ttstvene of el tains quali ana ofthe experince of onl ra ale ed ‘les Alten Med, 993119812-8 12a SS, Sharman MB, Bland Cj, Fl JA. Ee of ely egos 0 Gly ‘eine an eater oval the cia) Med Ee 1987:2911~7. 1D Besley JW. Des teaching by fly phys In she cand year of eel schol incase tet section of aly racic eens! Fm Mas 1995; sie-3 14, Hendenon MC, Hane DK, Wins JW. Gener neni iene tle shore pir ct eases the poms of role modeling, Am) Me. 19960: ie-53 15, She KM, Sirtos GA, Barman J, Baer MR. Ingoving cline teaching vali of mona diserinson propa Arch Intm Med 1992382 1156-6 22, No,lo/Octoser SurruemenT 1 1997