Original Article Shifa Tames Mini-CEX (Clinical Evaluation Exercise) as an assessment tool at Shifa College of Medicine, Islamabad, Pakistan Nadia Saeed, Nabia Tariq, Tara Jaffery Departments of Medicine, Obstetrics and Gynecology, Shifa College of Medicine, Millat University, Islamabad, Pakistan Objectives: To evaluate role of Mini-CEX as a learning and assessment tool for clinical competencies and to assess assessor's and the student's satisfaction ratings with Mini-CEX at Shifa College of Medicine, Islamabad. Methodology: This observational and psychometric analysis Mini-CEX was carried out at Shifa College of Medicine, Islamabad, Pakistan during the academic year 2011. Ninely six students of class of 2012 going through the junior clerkship were included in the study. Clinical competencies, Internal consistency and Reliability Mini-CEX were assessed. Mean values were calculated for satisfaction scores and time for observation and feed back. Results: There was generally a progressive improvement of clinical competencies in successive rotations as shown by significant paired sample t test between first and last rotation and significant mean differences of all clinical competencies and over all rating across rotations as analyzed by Kruskal-Wallis Rank Sum Test. Comparison between fourth and fifth rotation, however showed some insignificant drop in mean scores of competencies and global ratin Students’ and assessors’ satisfaction with Mini- CEX, on the scale of 9 were 8,0341.44 and 7.8341.22 respectively. Internal consistency reliability through Cronbach's alpha was 0.9 Conclusion: Mini-CEX was found as a useful leaming and assessment tool for demonstrating students’ clinical competencies progression and students and faculty were satisfied with this tool. (Rawal Med J 201;40: 220-224) Key Words: Clinical competence, Mini-CEX, Assessment. INTRODUCTION Mini-CEX (Clinical Evaluation Exercise) is an important tool for performance based formative assessment.!? Mini-CEX has been widely used to assess clinical competence of medical trainees and practicing physicians and has good overall utility for assessing aspects of the clinical encounter in an undergraduate setting. Many studies have proven its validity and reliability." In a typical Mini-CEX the supervising physician or faculty member evaluates a trainee during a patient encounter. This interaction is relatively short in which trainee conducts a focused history, examination and counseling. The evaluator asks about diagnosis and treatment plans and rates the trainee on six competencies and an overall rating. Focus of Mini-CEX may be confined to history and examination or counseling etc depending upon the nature of patients disease and evaluator preferences. At the end, evaluator gives feed back on trainees performance and they mutually discuss future plans for further improvement and it is documented in a validated Mini-CEX form. 220 This interaction occurs as a natural part of the training environment, and allows each resident to be evaluated on several occasions by different faculty members."*In a study students and preceptors agreed that feedback provided by using the Mini-CEX was more valuable than the type of feedback they had been receiving and giving previously." This effective feedback is the sine qua non for the improvement in student performance that we are claiming. We at Shifa College of Medicine are moving towards competency based education, No matter what the learning strategy is, students are assessed throughout the curriculum on competencies required to fulfill their roles as physicians. Considering the importance of Mini-CEX, it was introduced in clinical clerkship program of fourth and final year in 2009. The clinical faculty was trained by workshops for successful implementation of Mini-CEX. This study is about our one year experience of Mini-CE at Shifa College of Medicine to evaluate role of Mini-CEX as a learning and assessment tool for clinical skills and competencies. Rawal Medical Journal: Vol. 40. No. 2, AprilJune 2015,(Mini-CEX (Clinical Evaluation Exercise) as an assessment tool METHODOLOGY This study was conducted on 96 students of class of 2012 (Fourth year) at Shifa College of Medicine, Islamabad, Pakistan during the academic year 2011. For junior clerkship, students were divided into five groups (A to E). Each group was rotated through five Clinical rotations of two months each. These rotations included Medicine, Surgery and combined clerkship for Obstetric/Gynecology and Pediatrics, Ophthalmology and ENT and Family Medicine and Community Medicine. Bach student gave two Mini- CEX per clinical rotation; making a total of 10 Mi CEX encounters in a year. A validated Mini-CEX form comprised of student's demographic data, patient's diagnosis, case setting and complexity, focus of Mini-CEX and ‘or's position. Six competencies including medical interviewing skills, physical examination, humanistic qualities and professionalism, clinical judgment, counseling and communication skills” and organization’ efficiency were assessed and each was scored on a nine-point rating scale. Depending on the satisfaction levels of assessors, this nine point rating scale was further grouped into three categories i: e, unsatisfactory (scores 1 to 3), satisfactory (scores 4 to 6) and superior (scores 7 to 9). Overall rating of clinical assessment on global scale was also documented. Performance of the student was discussed at the end of encounter and a written feedback was given to the student. A suggested action plan was documented by the student, Time taken for observation and feedback was also documented and student and assessor's satisfaction with Mini-CEX was noted on ascale of 10. After the completion of academic year a total of 938 Mini-CEX forms representing data from 96 students of class 2012 were taken from the College record department. All variables in Mini-CEX form were included and SPSS 20 was used for data analysis. Mean differences between two rotations were calculated with paired sample t test. Kruskal-Wallis Rank Sum Test was used to calculate mean values differences across rotations. Intemal consistency Reliability was assessed by Cronbach's alpha. Mean values were calculated for satisfaction scores and time for observation and feed back RESULTS Out of 938 Mini-CEX forms, 700 (74.6%) were taken in ambulatory care while 157 (16.7%) were taken in In patients Department. Eight hundreds and twelve (86.6%) forms showed emphasis on history taking and physical examination. Maximum Mini- CEX were taken by Senior Registrars 422 (45%), while Assistant Professors, Associate Professors and Consultants took 54(5.8%), 30(3.2%) and 58 (6.2%), respectively (Table 1). Mean scores of clinical competencies and over all rating across five rotations are shown in Table 2. ‘Table 1. Clinical Clerkship Program of Class of 2012 (Fourth Year) (n=938). o Clinical Rotations and Number of Mini-CEX encounters (in brackets) 3 T T i Vv Vv. Toa & [Ga 2010 | Dec20I0w | Feb 2011 ‘Ape 2011 te Tun. | Tun. 2011 © Dec. 2010 Feb, 2011 ‘Ape. 2011 2011 Aug. 2011 Medicine Surgery | PacdsiGynae EyeENT G6)_[ Com/Fam Med A G5) G9) fey G6) 180 Surgery | Paeds/Gynae | ByelENTG4) | 2 [ Com/FamMed | Medicine B 2) G5) g 4) G2) 161 Paeds/Gynaet_ | Eye/ENT (29) | Com/Fam. 3 | Medicine 38) ‘Surgery c (68) Med (57) _| > GD 29 ByeENT | Com/Fan | Medicine Surgery Preds/Gynae D 36) Med. (38) G9) 38) G4) 18s ComiFam Medi | Medicine (39) | Surgery PacdsiGynae | EyeENT [177 E 40) doy as) 40) Total 21 180 208 164 179 338 OF Pealatrcs and Obstetrc/Gynecology T Community and Family Medicine Total Mini-CEX encounters in each rotation *Total Mini-CEX ei 221 encounters by each group Rawal Medical Journal: Vol. 40. No. 2, April-June 2015(Mini-CEX (Clinical Evaluation Exercise) as an assessment tool There was generally a progressive improvement in successive rotations as shown by significant paired sample t test between fifth and first rotation (Table 2). Kruskal-Wallis Rank Sum Test across rotations also showed significant mean differences in all clinical competencies and over all rating (Table 3). Comparison between fourth and fifth rotation showed drop in mean scores of all competencies and global rating except Counseling and communication skills. However this drop was insignificant as shown by paired sample t test (Table 2. Table 2. Progression of mean scores with standard deviations over Rotations (N=938). Seventy five percent (703) feed back and suggested action plan was related to improve relevant history taking and examination skills. Students! and assessors’ satisfaction with Mini- CEX, on the scale of 9 were 8.034144 and 7.83+1.22, respectively. Time spent in observation 222 ‘Paised mean scores Paired mean scores Rotations differences between fifth | differences between fifth and and frst rotation fourth rotation Competencies g el 3 8 rf2}3]e¢]fs5] 2] orf 2) &] r da} 2 a &| 4 2 637 | 685 | 697 | 739 | 719 , Medical interviewing kill | 67 | 685 | 697 | 73) 718 | o.826 | 5:72 | 166] 000] -02 | 075 | 154 | o4s 613 | 658 | 670 | 7:5 | 686 , Physical examination kil | &"9 | 638 | 670 | 71S) 686 | o.730 | 476 | 158 | 000 | -029 | -120 | 149 | 023 Humanistic 676 | 7.03 | 732 | 7.68 | 742 aqualitiesiprofessionatism | 21.2 | 21.3 | 21.3 | 21 | 21 | 9662 | 415 | 129 | 0.00] -026 | 0.96 | 120 | 0.35 651 | 694] 710] 758 | 734 5 > Clinical judgment o5) | 698 | THO | 758 | 7 | 0.829 | 435 | 110] 000] -024 | 121 | 106 | 022 Counseling and 635 | 732 | 732 | 747 | 750 communication skills | 22.3 | 21.2 | 213 | 212] 224 | 959 | 469 | 99 [0.00] 003 | 022 | 103 | 0.82 612 | 704 [7.44 | 7.60 | 731 x00 | as Organizationfetticiency | &77 | 704) 744 | 76) 751 | 701 | 3.99 | a5 | 000] -002 | -os2 | 2 | oar 678 | 711729 | 779 1749 | a 7 (Over all rating 678 | TAL | 728 | 778 | 749 | 709 | 497 | 178] 000] -03 | -178 | 162 | 007 "Significance @-ailed) ‘Table 3. Kruskal-Wallis Rank Sum Test across rotation and feed back was 16.425.8 minutes and 9.845.8 (0-938). minutes, respectively. Variables ChiSquare [Df | Pvalues] Internal consistency reliability through Cronbach's Tnierviewing skills 4310 | 4 [000000] alphawas0.9. Physical Examination | 63.230 | 4 | 0.00000 acon [en Tt Pama] piscussion Clinical Tudgment {58.290 [4 F 00000 1 Considering the internationally acceptable ‘Organization 31283} a] 0.000007) Significance of Mini-CEX, Shifa College of ‘Overall Rain 31250[-4 [0.00000] Medicine adapted this tool in senior and junior clerkship as formative assessment tool. Students were assessed on same clinical competencies through out the year with some differences depending on the rotation they were having Students were given frequent and structured feed back on their performances. We can infer that over all clinical competencies grew over year as was Rawal Medical Journal: Vol. 40. No. 2, AprilJune 2015(Mini-CEX (Clinical Evaluation Exercise) as an assessment tool shown by significant mean scores improvement across rotations. We found that paired mean scores differences between fifth and first rotations were higher among medical interviewing skill, clinical judgment and counseling and communication skill. Mean score analysis for individual rotation showed improvement in students’ clinical competencies till the second last or fourth rotation. However in the last rotation there was insignificant drop in mean scores of competencies which needs further evaluation. We assume that it might be due to higher expectations of assessors in last clinical rotation. In addition, at the end of fourth academic year students are assessed for only three disciplines i: e, Pathology, Ophthalmology and ENT. So another assumption may be that since they were preparing for their professional examination, they did not perform well in last clinical rotations. Focus of Mini-CEX was mainly on history taking and examination skills. In sections of "Suggestions for development" and "Agreed action plan", emphases were mostly on improving focused history taking and examination. It is possible that since students were accustomed to a detailed patient encounter in junior clerkships, they found it difficult to obtain relevant medical interview and perform relevant physical examination in a short encounter. However, we did not find such comments from students in any Mini-CEX form. Only 0.2% Mini CEX were carried out in Emergency settings which indicates that efforts should be made to distribute these encounters equally in out door, in patient and emergency departments. Satisfaction levels of our faculty and students with Mini-CEX correlate with satisfaction levels in various international studies revealing its acceptability in our set up.***’ Mini-CEX implementation was an added responsibility for our faculty but their satisfaction levels showed their confidence on this tool. In.a typical Mini-CEX time span can vary from 15 to 20 minutes. We found that the time spent in observation and feed back was 16.4+5.8 minutes and 9.8+5.8 minutes respectively. Our one year Mini-CEX scores data revealed high internal consistency reliability through Cronbach's 223 alpha coefficient. However in order to estimate inter rater reliability, a complex performance assessment tool like Generalisability Theory (GT) analysis should be used." The strength of our study is that we hada large sample size and it was followed through out the year. Our internal consistency was reliable. However our study lacks Generalisability Theory (GT) analysis of scores. Further analysis can also be done to compare our results with other assessment tools in order to determine its validity in our set up. CONCLUSION We found Mini-CEX as a useful learning and assessment too] for demonstrating students’ clinical competencies progression. We also found that students and faculty were satisfied with this tool. [Author Contributions [Conception and design: Nadia Saeed, Nabia Tara, Tara Jatery |Collection and assembly of data: Nadia Saved, Nabia Tariq |Analyss and intorpretalon of data: Nacla Saved, Tara Jafery lDrating of article: Nacka Saeed, Tara Jaffery \Crticalrevsion farce for important intellectual content: All Statistical expertise: Nadia Saced [Final approval and guarantor ofthe article: Al Icorresponaing author email: Saced.nadia@hotmall.com [Contict of interest: None declares Rec. Dare: Jan 18, 2015 Accopt Dale: Feb 18, 2015 REFERENCES 1. Noreini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No, 31. Med Teacher 2007;29:855-71 2. Bruce D. Workplace-based assessment as an educational, tool: guide supplement 31.4 Review. Med Teacher 2010;32:524-5. 3. Hill F, Kendall K, Galbraith K, Crossley J. Implementing the undergraduate mini-CEX: a tailored approach at Southhampton University. Med Educ 2009343:326-34, 4, Kogan JR. Bellini LM. Shea JA. Feasibility, reliability and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 2003;78(10 Suppl):$33-5, 5. Norcini JJ, Blank LL, Amold GK, Kimball HR. The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation. Ann Intern Med 1995;123:795-9. 6. Noteini JJ, Blank LL, Duffy FD, Fortna GS. The mini- CEX: a method for assessing clinical skills. Ann Intern Med 2003;138:476-81 7. Kogan JR, Bellini LM, Shea JA. Implementation of the Rawal Medical Journal: Vol. 40. No. 2, April-June 2015(Mini-CEX (Clinical Evaluation Exercise) as an assessment tool mini-CEX to evaluate medical students’ clinical skills. 9. De Lima AA, Barrero C, Baratta $, Costa YC, Bortman ‘Acad Med 2002377:1156-7. G,Carabajates J, etal. Validity, reliability, feasibility and 8. Nair BR, Alexander HG, MeGrath BP, Parvathy MS, satisfaction ofthe Mini-Clinical Evaluation Exercise for Kilsby EC, Wenzel J, etal. The mini clinical evaluation cardiology residency training. Med Teacher ‘exercise (mini-CEX) for assessing clinical performance 2007;29:785-90. ‘of international medical graduates. Med J Aust 10. Downing SM. Reliability: on the reproducibility of 2008;189:159-61, assessment data, Med Edue 2004;38: 006-12. 224 Rawal Medical Jounal: Vol. 40. No. 2, Apritune 2015,