You are on page 1of 5

SPMC

JHCS

Relationships of training examination results and patient


load to certifying examination results in a Surgery
residency training
Mauro Bravo III,1,2,3 Bernardo Ang1

1DavaoRegional Hospital, Apokon,


Tagum City, Philippines Background. In the Philippines, Surgery residents­in­training are assessed annually for proficiency through
2Democrito O Plaza Memorial
residency in­service training examinations (RITE). Specialty certification, however, is contingent upon the
Hospital, Prosperidad, Agusan del results of the Surgery Certifying Examinations (SCE).
Sur Objectives. To determine the relationships of Surgery residents’ annual RITE performance and patient load
with the results of their written and oral SCE.
3San Francisco Doctors Hospital,
Design. Retrospective cohort study.
Brgy Hubang, National Highway,
San Francisco, Agusan del Sur
Setting. Davao Regional Hospital in Tagum City and Southern Philippines Medical Center in Davao City,
Philippines.
Participants. Thirty­eight residents who graduated from the Mindanao Integrated Surgical Residency Training
Mauro Bravo III Program (MISRTP) and took the SCE in 2000­2012.
maui_brux00@yahoo.com Main outcome measures. Mean percentage of RITE passed, mean annual patient load per resident, SCE
performance, and odds ratios (95% CI) of passing the SCE.
17 April 2015 Main results. Resident physicians passed 76.32 ± 18.48% of the RITE that they took, and performed a mean
of 441 ± 52 surgical operations per resident per training year. Passing rates on first attempt were 25/38
(65.79%) for written SCE and 13/24 (54.17%) for oral SCE. Passing ≥75% of the total number of RITE
24 July 2015 significantly increased the odds ratio of passing the written SCE (OR=24.44, 95% CI 4.18 to 142.95,
p=0.0004), but not the odds ratio of passing the oral SCE. Having a higher than average patient load did not
Bravo M III, Ang B. Relationships significantly change the odds ratio of passing any SCE.
of training examination results and Conclusion. Good performance in the RITE increased the odds ratio of passing the written, but not the oral,
patient load to certifying examination SCE. Patient load did not significantly affect the odds ratios of passing the written or oral SCE.
results in a Surgery residency
training program. SPMC J Health
Care Serv. 2015;1(1):9­13.
Keywords. specialty certification, proficiency assessment, written examination, oral examination

© 2015 M Bravo III, et al. well in the specialty certifying examinations.


In 1991, the Philippine College of Surgeons In the Philippines, residents in Surgery
(PCS) formulated a standardized surgical take the annual written residency in-service
curriculum that was subsequently imple- training examinations (RITE) as part of their
mented among surgery residency training proficiency assessment. Residents who grad-
programs throughout the country.1 The uate from Surgery programs usually appraise
Philippine Society of General Surgeons their preparedness and ability to pass the
(PSGS) assumed the responsibility of over- surgery certifying examinations (SCE) from
seeing the implementation of the PCS their past performances in the RITE. Both the
curriculum and accrediting the training RITE and the SCE are given by the Philippine
programs from PCS in 2002.1
The Mindanao Integrated Surgical
Residency Training Program (MISRTP), a
consortium of the originally independent
surgical residency programs of Southern
Philippines Medical Center (SPMC) and
Davao Regional Hospital (DRH), has been
an accredited training program since 2000. It
is the largest training program in General
Surgery in Mindanao. MISRTP has produced
competent surgeons in General Surgery, as
well as in surgical subspecialties within the
field. However, like other training programs
all over the country, not all its graduates fare

Bravo M III, Ang B. SPMC J Health Care Serv. 2015;1(1):9­13. 9


SPMC JHCS

Board of Surgery (PBS), a body created by the training board, composed of consultant spe-
PCS to formulate and organize examinations cialists from both hospitals, is responsible for
for residents-in-training and for graduates planning and implementing activities that
who are eligible to take the SCE. promote academic and skills learning among
The SCE is a two-part examination. The residents, and for regularly evaluating the
first part is a written examination, which is residents’ performance within the MISRTP.
similar to the RITE. In the second part,
which is an oral examination, examinees an- Participants
swer questions based on theoretical surgical For this study, we included all residents who
cases. The examinees’ performance in the graduated from the MISRTP from 2000 to
oral examination is assessed by a panel of 2012 and took the SCE given by the PBS. A
examiners. An examinee must pass both the total of 40 residents graduated from the
written and oral examinations by the PBS in training program, but two of them did not
order to become a board-certified surgeon. take the written SCE. Of the 38 who took
In-training examinations have been the written SCE, only 24 took the oral SCE.
investigated for their ability to predict per-
formance in post-residency certifying exam- Data collection
inations, with varying results.2-4 Work con- The primary outcome measures in this study
ditions have also been known to influence were mean percentage of RITE passed,
the performance and proficiency of sur- mean annual patient load per resident, SCE
geons.5 6 Surgery residency training in SPMC performance on first take, and odds ratios
and DRH, as in many service-oriented gov- (95% CI) of passing the SCE on first take.
ernment training hospitals, is characterized We obtained data on the annual RITE results
by high patient load because these hospitals from the records of the individual Surgery
are the end-referral centers in Southeastern departments, and the results of the oral and
Philippines. written SCE from the records of the PCS.
We were interested to know whether the Data on annual patient load were taken from
training conditions in MISRTP are adequate- the annual census of surgical procedures
ly preparing the residents for the national done by MISRTP residents, as annually re-
SCE. Hence, we did this study to determine ported by the program to the PSGS.
the relationships of Surgery residents’ annual We took the annual RITE ratings of each
RITE performance and annual patient load resident and compared each rating to the
with the results of their written and oral minimum passing level set by the PBS during
SCE. the year that the RITE was taken, in order to
determine whether the resident passed the
RITE during a particular year. We then de-
Study design and setting termined the individual resident’s percentage
We did a retrospective cohort study by of RITE passed by dividing the number of
reviewing available records on the annual RITE passed by the total number of RITE
RITE performance and workload of resi- taken and multiplying the quotient by 100.
dents in Surgery, and the subsequent results To determine patient load per resident
of their SCE. The study was done within the per year, we took the total number of
MISRTP. In the program, Surgery residents surgical procedures done by the MISRTP
typically rotate in two hospitals in South- residents each year and allocated the sum
eastern Philippines. The first hospital, SPMC, equally to all training residents for that year.
is the only tertiary government hospital in We took the average annual patient load for
Davao City with a 1,200-bed capacity, shared each resident by taking into account all their
by 13 medical specialty departments, in- annual loads during residency training.
cluding Surgery. The second hospital, DRH, For the SCE performance, we reflected
is located in Tagum City and is 60 kilometers the passing or failure status of each resident
away from SPMC. DRH has a 400-bed during the first attempt to take the written or
capacity, with five other medical specialty the oral SCE. Since the numbers of written
departments, apart from Surgery. Surgical SCE takers and oral SCE takers were dif-
residents in the program provide tertiary level ferent, we expressed the outcomes of the
care and perform a total of 16,000 emer- written SCE and the oral SCE separately in
gency and elective operations annually. A the results.

10 Bravo M III, Ang B. SPMC J Health Care Serv. 2015;1(1):9­13.


SPMC JHCS

Table 1 Characteristics of Surgery residents sequently took the oral SCE, 13 (54.17%)
c
passed the examination on first attempt.
Values
Characteristics n=38* Table 2 shows the odds ratios (95% CI)
of passing the certifying examination on first
Sex distribution, frequency (%) take. Having a ‘good performance in the
Female 5 (13.16) RITE’ significantly increased the odds ratio
Male 33 (86.84) of passing the written SCE on first take
Mean annual patient load per resident ± SD, patients 441 ± 52 (OR=24.44, 95% CI 4.18 to 142.95,
Mean percentage of RITE passed ± SD, % 76.32 ± 18.48 p=0.0004), but not the odds ratio of passing
Passed written SCE on first take, frequency (%) 25 (65.79) the oral SCE on first take (OR=1.22, 95%
Passed oral SCE on first take, frequency (%) (n=24) 13 (54.17) CI 0.14 to 10.48, p=0.8548). Having a ‘high
patient load during residency’ did not
*Unless specified otherwise. significantly change the odds ratio of passing
either the written SCE (OR=2.59, 95% CI
0.46 to 14.51, p=0.2796) or the oral SCE
Statistical analysis (OR=0.10, 95% CI 0.01 to 1.06, p=0.0558).
We summarized continuous variables using
means and standard deviations, and categor-
ical variables using frequencies and percent- Key results
ages. We predetermined that a ‘good perfor- We found out that ‘good performance in the
mance in RITE’ should be any percentage of RITE’ increased the odds ratio of passing the
RITE passed equal to or higher than 75%. written SCE, but did not affect the odds ratio
We also classified residents with higher than of passing the oral SCE. Further, ‘high
average annual patient load as having ‘high patient load during residency’ did not
patient load during residency.’ We used significantly change the odds ratio of passing
logistic regression to determine the odds any of the SCE.
ratios (95% CI) of passing the written SCE
and the oral SCE on first take for ‘good Strengths and limitations
performance in RITE’ and ‘high patient load This study provides evidence regarding the
during residency.’ A two-sided p-value of relationships of academic exercises in the
<0.05 was considered statistically significant. form of RITE and patient load during
residency, with outcomes of certifying
examinations that residents take after train-
We included 38 residents who graduated ing. We believe that this is the first systematic
from MISRTP and who eventually took the inquiry on the factors that affect SCE results
written and/or the oral SCE in this analysis. among residents who graduated from the
The characteristics of the residents are MISRTP.
shown in Table 1. Most of the residents In measuring proficiency during
(33/38, 86.84%) were males. The residents residency, we only considered the perfor-
passed a mean of 76.32 ± 18.48% of the mance of residents in the RITE. Several
RITE that they took during residency, and other quantified measures of proficiency,
performed a mean of 441 ± 52 surgical such as quarterly evaluations, or annual
procedures per resident per year. Of the 38 marks considered in the decision to promote
residents who took the written SCE, 25 residents to the next level of training were,
(65.79%) passed the examination on first in fact, obtainable but were not accounted
attempt. Of the 24 residents who sub- for in this study. Our study was also limited

Odds ratios (95% CI) of passing the written and oral SCE on first take

Passing written SCE on first take Passing oral SCE on first take
Variables OR (95% CI) p­value OR (95% CI) p­value
Good performance in the RITE† 24.44 (4.18 to 142.95) 0.0004* 1.22 (0.14 to 10.48) 0.8548
High patient load during residency‡ 2.59 (0.46 to 14.51) 0.2796 0.10 (0.01 to 1.06) 0.0558
*Statistically significant; †resident passed ≥75% of total number of RITE taken; ‡mean annual surgical operations >441 patients (average). RITE – residency in­service training
examinations; SCE – surgery certifying examinations.

Bravo M III, Ang B. SPMC J Health Care Serv. 2015;1(1):9­13. 11


SPMC JHCS

in the sense that, in quantifying patient load sidents’ skills in verbally articulating their
during residency, we only took into academic proficiency in Surgery. Modes of
consideration the number of surgical instruction that can be considered for this
procedures done during training. Performing purpose include grand rounds and mock oral
non-surgical procedures like admitting examinations. The aim should be to provide
patients, doing rounds, doing pre-operative during training a learning activity that is
and post-operative care, etc., may also take comparable to the setup of the oral portion
up a considerable amount of the residents’ of the certifying examinations.
time and effort, which, in turn, may
positively or negatively affect their overall
proficiency in Surgery. Performing well in the RITE during training
significantly increased the odds ratio of
Interpretation passing the written SCE. However, perfor-
Associations between in-training exami- mance in the RITE was not significantly
nations and certifying examinations in Sur- associated with oral SCE results. Moreover,
gery residency have been have been explored patient load during residency was not
in the past.2-4 7 However, these studies re- significantly associated with written or oral
ported different conclusions, perhaps be- SCE results.
cause of the diverse practices of conducting
and evaluating residency training programs. Acknowledgments
Our underlying goal for doing this study We thank the members, the training staff, and consultants of the
was to build evidence for or against the way Mindanao Integrated Surgical Residency Training Program, and Dr
that Surgery residents are being prepared Alvin S Concha for their inputs and guidance in this research. We
during residency to tackle the SCE, which is also thank the Philippine College of Surgeons for granting us access
to their data on certifying examination results of the MISRTP graduates.
an important milestone in their practice. The
fact that this study demonstrates that good Ethics approval
performance in the RITE is associated with a This study was reviewed and approved by the Department of Health
positive outcome in SCE may mean that the XI Cluster Ethics Review Committee (DOHXI CERC reference
two sets of examinations given by the PBS P13112701).
are similar. In a way, the RITE do prepare Reporting guideline used
the residents for the written SCE. The whole STROBE Checklist (http://www.strobe­
system works because would-be takers of the statement.org/fileadmin/Strobe/uploads/checklists/STROBE_checklist_v4
written SCE can simulate taking the cer- _combined.pdf)
tifying examinations by going through the
Article source
RITE during training. Submitted
It is a common notion among residents
that high work load during training gives Peer review
them less time to study and usually translates External
into poor performance in examinations.
Competing interests
However, the results of this study, as well as None declared
those of previous studies, showed otherwise
– that the volume of surgical procedures Access and license
during training does not affect the This is an Open Access article licensed under the Creative Commons
performance of residency graduates during Attribution­NonCommercial 4.0 International License, which allows
others to share and adapt the work, provided that derivative works
their certifying examinations.8 9 bear appropriate citation to this original work and are not used for
commercial purposes. To view a copy of this license, visit
Generalizability http://creativecommons.org/licenses/by­nc/4.0/
While on training, residents need to prepare
for the certifying examinations, which will
eventually assess their competence in be- 1. Philippine Society of General Surgeons. Standardized
coming surgical specialists. The RITE ade- Curriculum in General Surgery 4th edition. 2012.
quately prepares residents in Surgery for the 2. De Virgilio C, Yaghoubian A, Kaji A, Collins JC, Deveney K,
written certifying examination only. Since the Dolich M, et al. Predicting performance on the American Board of
oral portion of the SCE is as important as Surgery qualifying and certifying examinations: a multi­institutional
study. Arch Surg. 2010;145(9):852­6.
the written examination, training programs 3. Corneille MG, Willis R, Stewart RM, Dent DL. Performance on
should consider introducing learning activi- brief practice examination identifies residents at risk for poor
ties in residency that would enhance re- ABSITE and ABS qualifying examination performance. J Surg

12 Bravo M III, Ang B. SPMC J Health Care Serv. 2015;1(1):9­13.


SPMC JHCS

Educ. 2011;68(3):246­9. 7. Grewal SG, Yeung LS, Brandes SB. Predictors of success in a
4. Woloschuk W, Mclaughlin K, Wright B. Predicting performance urology residency program. J Surg Educ. 2013;70(1):138­43.
on the Medical Council of Canada Qualifying Exam Part II. Teach 8. Sturm L, Dawson D, Vaughan R, Hewett P, Hill AG, Graham JC,
Learn Med. 2013;25(3):237­41. et al. Effects of fatigue on surgeon performance and surgical
5. Brandenberger J, Kahol K, Feinstein AJ, Ashby A, Smith M, outcomes: a systematic review. ANZ J Surg. 2011;81(7­8):502­9.
Ferrara JJ. Effects of duty hours and time of day on surgery 9. Lehmann KS, Martus P, Little­elk S, Maass H, Holmer C,
resident proficiency. Am J Surg. 2010;200(6):814­8. Zurbuchen U, et al. Impact of sleep deprivation on medium­term
6. Friedlaender GE. The 80­hour duty week: rationale, early psychomotor and cognitive performance of surgeons: prospective
attitudes, and future questions. Clin Orthop Relat Res. cross­over study with a virtual surgery simulator and psychometric
2006;449:138­42. tests. Surgery. 2010;147(2):246­54.

Bravo M III, Ang B. SPMC J Health Care Serv. 2015;1(1):9­13. 13