Research Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022

Archives of Surgical Research | Systematic Literature Review

Utility of Workplace-Based Assessment Among Surgical Residents: A

Systematic Literature Review
Nabila Talat, Muhammad Usama Aziz

IMPORTANCE This review article aims to assess the usefulness of Workplace-Based Systematic Literature Review
Assessment (WBA) among surgical residents. Multiple studies have been done in the
past to determine the utility of WBAs across different fields of medicine. But it hasn’t
been done among surgical specialties on large scale, and the present paper fills in this
research gap.
METHODS Articles were searched using PubMed and Google Scholar databases up to
30th June 2022. Following headings were used for the literature search i.e work-based
assessment or workplace-based assessment or WBAs or direct observation of
procedural skills or DOPS or procedure-based assessments or PBA or Case-based
discussion or CBD or clinical evaluation exercise or CEX or Mini CEX, and surgical
training. The utility formula of Vleuten was used to assess the usefulness of WBA. It is
a product of validity, reliability, educational impact, acceptability, cost-effectiveness,
and feasibility.
RESULTS For reviewing purposes, 29 studies were selected. There were 8 studies on
PBA, 2 on Mini-CEX, 2 on CBD, 6 on DOPS, and 11 studies on multiple methods of
WBA. PBA had positive satisfaction levels and was valid, reliable, feasible, and
acceptable. CBD. Mini-CEX and CBD had very few studies in which they were proved
to be reliable and valid but had mixed responses in terms of satisfaction levels. DOPS
had a level 1 educational impact and also proved to be valid. When all components of Author Affiliations: Author
WBAs were used together, a negative level 1 educational impact was observed. The affiliations are listed at the end
concerns that were identified include lack of time, training and evidence of validity, of this article.
design of the tool, and perception of WBA as a summative tool.
Corresponding Author:
DISCUSSION Work-based Assessments should be implemented and used properly
Dr. Nabila Talat, MBBS, FCPS,
after the training of faculty. The focus should be on the quality of the assessments, FACS, Professor & Head
not on the quantity. Large-scale studies should be done to assess level 3 and 4 of Department of Pediatric
educational impact from WBAs. Surgery Unit II The Children’s
KEYWORDS Work-based assessment, Surgical resident, Utility, Procedure based assessment, Hospital and the University of
Case-based discussion, Clinical Evaluation Exercise, Direct Observation of Procedural Skills Child Health Sciences, Lahore
nabila.talat@chich.edu.pk
HOW TO CITE Talat N, Aziz MU. Utility of Workplace-Based Assessment Among Surgical
+92-336-6441369
Residents: A Systematic Literature Review. Archives of Surgical Research. 2022, 3 (2):11-18.
https://doi.org/10.48111/2022.02.03
https://doi.org/10.48111/2022.02.03

T
he 21st century is bringing new challenges. There are
expected rapid changes in surgical knowledge and
understanding of diseases, new and better
technologies, and surgical procedures. To keep pace with
the modern world of science, training institutions need
more active scrutiny and error judgment in their training
programs1. In the modern world, the current module of
surgical training has been shaped by different
modifications in medical education and training like
modernizing medical careers and postgraduate education2.
In the past, trainee doctors were mostly assessed for their
knowledge using theory examinations and stimulated
Archives of Surgical Research www.archivessr.com
clinical assessments at the end of their training period.
More emphasis was given on what a person knows rather
than what he or she does in practice. However, doctors
have multiple attributes besides specialty knowledge and
skills, but also attitudes and generic skills like integrity,
honesty, communication, and teamwork. Actual
performance is the only reflection of the competence, i.e.,
day-to-day behavior and performance of a person in
practice3. Workplace-based assessments (WBAs) are the
skills that need to be assessed in clinical settings. Strong
emphasis has been given to the assessment of trainees at
the workplace4.
11
Research
WBAs deal with all forms of work a resident usually does at
his workplace5. The reason behind implementing the WBAs
is to make sure that a trainee learns and develops through
evidence-based progression to attain clinical competency6.
Assessment of clinical skills in WBA correlates with the
“Does” level in Miller’s Pyramid (Fig 1)7.
Fig 1. “Miller’s Pyramid”7
There is always an emphasis on formative assessment of
trainees and WBAs are a source of it. They provide
feedback to improve trainees’ development and learning8.
Consultants or trainee’s supervisors should carry out WBAs
who assess skills, knowledge, and behavior during multiple
daily tasks in a hospital9. (Fig 2)
Fig 2. “Conceptual Framework of Learning Benefits Of Workplace-
Based Surgical Assessment”17
The working environment for Surgical Training is unlike
other training programs. The surgical resident performs
multiple tasks such as treating patients in wards as
inpatients as well as patients in outpatient clinics, doing
surgical procedures, and seeing patients pre and
postoperatively. They require special surgical skills to
complete their training which is not seen in other
specialties. Some of the tasks can be performed
independently by the surgical trainee at workplace in which
he is competent enough, but direct observation of a
Archives of Surgical Research www.archivessr.com
Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022
consultant is required for most of the surgical procedures.
All types of work that a trainee do in his day-to-day tasks,
should be included in WBAs10.
The WBAs include procedure-based assessments (PBA),
case-based discussion (CBD), clinical evaluation exercise
(CEX), and direct observation of procedural skills (DOPS).
(Table 1)5,11–13.
Procedure- (Orthopedic Proficiency Assessment Project (OCAP)
Based initially produced PBA which was then introduced in
Assessment other fields of surgery. PBA form includes feedback
(PBA) from instructor and trainee, several competencies, and
a worldwide assessment. In case a trainee can perform
the procedure with no guidance and deal with it on his
own, then it means that this individual has attained the
highest competence degree.5
Clinical American Board introduced CEX in medical specialties.
Evaluation It was then modified to mini CEX to suit the surgical
Exercise (CEX) clinical activities. The trainee is evaluated while
wearing interactions with patients such as taking
history, medical examination, communication skills,
and professionalism.11
Case-Based CBD is an in-depth discussion with the trainee about
Discussion the management of a patient. The trainee is assessed
(CBD) in terms of clinical knowledge, reasoning, and
judgment.12
Direct Initially, the Royal Colleges of Physicians developed a
Observation of tool to assess operative, technical and professional
Procedural Skills skills in diagnostic and interventional procedures.
(DOPS) Several modifications have been done to include
different surgical procedures in it.13
Table 1: Types of Workplace-Based Assessment
For more than 10 years, WBAs have been in use in general
surgery training. Since its implementation, multiple studies
have been published in the literature on the utility of WBAs
in training. However, in surgery, there are not a lot of
studies, especially on DOPs, CBD, and Mini CEX10. Due to
the heterogenicity of the methods and outcomes of the
previous studies, it is expected that there are very meta-
studies available in the literature. Saedon et al. stated in
their systematic review that WBA provides valuable
feedback which has a constructive effect on clinical
practice. But it involved studies from all medical and
surgical specialties8. Similarly systematic reviews of Barette
et al. and Miler et al. also included articles from all medical
specialties14,15. Aryal et al. published 2 systematic reviews in
2019 and 2021 that focused on the educational impact and
utility of WBAs in higher surgical training residents10,16. But
medical knowledge is everchanging so we needed an
updated review about WBA and its components. This
systematic review was done to assess the insight of the
utility of WBA’s among consultants and their trainees. This
study reviews published articles on the utility of WBAs
among residents of general surgical and surgical specialty
training.
12
Research
METHODS
This study has been reported following the guidelines of
PRISMA (Preferred Reporting Items for Systematic Reviews
and Meta-Analyses)18. Literature was searched on the
databases Pubmed and Google Scholar up to 30th June
2022. The literature search was done under the headings of
work-based assessment or workplace-based assessment or
WBAs or direct observation of procedural skills or DOPS or
procedure-based assessments or PBA or Case-based
discussion or CBD or clinical evaluation exercise or CEX or
Mini CEX, and surgical training.
Inclusion criteria included quantitative or qualitative studies
published in the English language, involving post-graduate
surgical specialties trainees or trainers. Following surgical
specialties were included: General Surgery, Orthopedic
Surgery, Urology, Plastic surgery, Neurosurgery,
Cardiothoracic Surgery, Vascular Surgery, Pediatric Surgery,
Oral and Maxillofacial Surgery, Otolaryngology, and
Ophthalmology. Studies involving, gynecology, pediatrics,
anesthesia, endoscopy, dental, histopathology, medical
students, nursing, and other specialties were excluded.
According to the inclusion and exclusion criteria, both
authors did the initial scrutiny of the articles by titles and
Figure 3. PRISMA 2020 flow diagram
Archives of Surgical Research www.archivessr.com
Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022
abstracts. Then full text of the selected articles was
reviewed. Only those articles were selected which described
the utility or usefulness of WBAs in surgical training. The
final selection of the studies was also done by both authors.
The usefulness of WBA is assessed by the utility formula of
Vleuten. Utility= Validity x Reliability x Educational Impact x
Cost effectiveness x Feasibility x Acceptability19.
“Extent to which a test measures what is intended to be
measured and nothing else” is known as validity20.
Reliability is “a measure of the consistency and precision
with which a test measures what it is supposed to assess”21.
Educational impact may have 4 levels; Satisfaction,
Learning, Behavior, and Results with better patient
outcomes22. Cost-effectiveness of assessment means
evaluating the benefits of a particular assessment against
its cost7. Acceptability means the assessment tool is
acceptable to the participants. Availability of sources, time,
and personals will determine the Feasibility23.
RESULTS
Initially, 257 studies were identified through a database
search. Titles and abstracts of these articles were screened
and 208 studies were excluded. Full-text review of 49
articles was done and out of which 29 were included in this
study. (Fig 3)
13
Research Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022

Procedure-Based Assessment (PBA): We found 8 Roushdi et al. found a high level of confusion among
publications in surgical specialties describing PBAs (Table trainees and trainers regarding the use of PBA27. Hunter et
2). Medeiros et al. showed high validity, reliability, and al. showed that PBA is a useful tool for feedback and
feasibility of PBA among urology residents24. learning but the focus should be on quality, not quantity28.
Rhamhamdany described a modified version of PBA for PBA was proved to be valid, reliable, acceptable, and
orthopedic trainees which had a better formative feasible with a good satisfaction level in other studies29–31.
assessment of surgical skills. At the same time, it was not Overall, PBA showed excellent utility and positive
much acceptable due to the lengthy tick box activity25. satisfaction among surgical residents.
Shalhoub et al. stated that PBA has a high educational
value but it should be perceived as formative tool26.
Author Study Design Values Assessed No of Subject Area Results
Participants
Medeiros et al., Prospective, action- Validity, Reliability, 6 trainees Urology training PBA has high validity, reliability, and feasibility
202124 research study Feasibility 10 trainers
Ramhamadany, Quantitative survey Validity and 12 trainees Orthopedic A modified version of PBA is a better
202025 acceptability surgical formative assessment of surgical skill.
residents Lengthy tool and prolonged tick box exercise
questions its acceptability
Shalhoub et al., Qualitative semi- Validity, 10 trainees Different “PBA has significant educational value when
201726 structured Educational impact Surgical used as a formative tool”
interviews Specialities
Roushdi et al. 201527 Quantitative survey Validity, 39 trainees Orthopedic High level of confusion among trainers and
Educational impact 21 trainers Surgical trainees about both the purpose of PBA and
Training the correct method of using them
Hunter et al., 201528 Quantitative Survey Educational Impact 616 trainees Orthopedic Most residents found PBAs useful as a
Surgical learning tool for the delivery of feedback.
Training Quality should be improved rather than
quantity
Marriot et al., 201129 Quantitative, Validity, reliability, 81 trainees 6 surgical Overall high validity and acceptability, and
Observational acceptability specialties reliability
Beard et al., 201130 Quantitative, Acceptability, 85 trainees 6 surgical High acceptability, validity, reliability, and user
Observational Reliability, Validity specialties satisfaction.
James et al., 200931 Quantitative, Educational 3 trainers Higher surgical Feasible
Observational impact, feasibility 2 trainees training “Focused feedback and targeted training”
Table 2: PBA
Clinical Evaluation Exercise (mini-CEX): proved to be a valid, reliable, acceptable, and feasible tool
We only found 2 studies evaluating mini CEX among and showed positive satisfaction among trainees and
postgraduate trainees in general surgery (Table 3). It trainers32,33.

Author Study Design Values Assessed No of Participants Subject Area Results

Deshpande et Quantitative Educational 18 trainees Orthopedic Mini-CEX improves technical and non-technical
al. 202132 Observational impact, Validity, 10 trainers surgical skills
Reliability, training It is a valid, reliable, acceptable, and feasible tool
Acceptability,
Feasibility
Joshi et al., Quantitative survey Acceptability, 16 trainees Higher surgical Mini-CEX is feasible and well accepted by
201733 feasibility 9 trainers training trainees and trainers

Table 3: Mini-CEX
Case-Based Discussion (CBD) These studies showed that CBD is reliable and valid with
The utility of CBD has been discussed in 2 studies for good educational impact but promotes tick box
surgical specialties which include only trainees (Table 4). mentality34,35.

Author Study Design Values Assessed No of Subject Area Results

Participants
Philips et al., Quantitative survey Utility, Educational 42 trainees Surgical training Provides feedback., Improves learning
201634 impact Promotes tick-box mentality
Awad et al., Retrospective Reliability, Validity 46 trainees Otolaryngology Case-based discussion is a reliable and valid tool
201535 database analysis
Table 4: CBD

Archives of Surgical Research www.archivessr.com 14

 Research Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022

Direct Observation of Procedural Skills (DOPS)
Among DOPS, 6 studies are identified (Table 5). Inamdar et
al. showed that DOPS is a feasible, acceptable, and effective
tool36. A study in Ophthalmology showed positive
satisfaction among trainees37. DOPS have helped improve
surgical skills in multiple studies38,39. Mathew et al. modified
the DOPS tool into Surgical DOPS (S-DOPS) which showed
good uptake and construct validity as compared to the
original version of the tool13. Awad et al. had a view that
DOPS can differentiate between senior and junior
residents40. Overall DOPS proved to be valid, acceptable,
and reliable with a good educational impact in surgical
training.

Author Study Design Values Assessed No of Participants Subject Area Results

Inamdar et al., Quantitative Educational 30 trainees Surgical training DOPS is a feasible, acceptable, and effective
202136 Impact, Feasibility, tool
Effectiveness,
Acceptability
Sethi et al., Quantitative Educational 115 trainees Ophthalmology Improvement in clinical skills with repeated
201937 impact DOPS
Hussain Waqar Sequential Educational 48 trainees Surgical training Improved surgical skills.
et al., 201638 mixed-method impact, validity, Shortage of time and lack of training are the
study feasibility main concerns
Ali et al., Comparative Educational 20 trainees Urology training DOPS is an effective tool for assessing and
201939 study impact, 4 trainers improving the surgical skills
Effectiveness
Matthew et al., Quantitative Construct validity, Phase 1, 1370 trainees Surgical training Good uptake and construct validity of the new
201413 DOPS uptake Phase 2, 16 assessors S-DOPS.
and 15 trainees
Phase 3, 3525 trainees
Awad et al., Quantitative Validity, Reliability 767 DOPS Otolaryngology Reliable and internally consistent.
201440 Trainees not Differentiates between junior and senior
mentioned trainees
Table 5: DOPS
Multiple WBA Methods identified as the most useful tool of assessment among all
When we searched for WBAs having all or maximum of its WBAs and CBD was at the 2nd spot. There were concerns
components within the same study, a total of 11 studies about lack of time, training and evidence of validity, design
were found that highlighted the use of all or most of the of the tool, and perception of WBA as a summative tool.
components of WBAs within the same settings(Table 6). All Multiple WBAs when used together can lead to confusion
of these studies highlighted several problems including and negative satisfaction levels among trainers and
negative satisfaction levels on educational impact. PBA was trainees17,41–50.

Author Study design Type of WBA Value assessed No. of Subject Area Results
Participants
Aryal et al., Mixed method CEX, CBD, Educational Phase 1 - 27 General surgical PBA was the most useful WBA.
202041 study PBA, DOPS impact, Validity trainers, 38 training Useful when practiced with face-to-
trainees Phase 2 face validation, and trainer trainee
- 5 trainers and 5 engagement.
trainees Require time and training
Nathoo et al., Qualitative WBA Educational 4 trainees Ophthalmology WBA is an excellent feedback tool
202042 impact 9 trainers but needs to be designed well
Gaunt et al., Qualitative PBA, DOPS, Educational 42 trainees Surgical specialties Negative feedback by trainees.
201743 interviews CBD, mini-CEX impact
Gaunt et al., Quantitative PBA, DOPS, Perception of 178 trainees 147 General Surgical Both trainers and trainees perceive
201644 survey CBD, mini-CEX WBA trainers training WBA as a summative assessment
tool
Pentlow Quantitative PBA, DOPS, Educational 61 trainers Orthopedics Only PBA was perceived to be
201545 survey CBD, CEX impact 46 trainees Training educationally valuable
Philips et al., Quantitative PBA, DOPS, Utility 64 trainers General Surgical PBA and CBD were the most useful
201546 survey CBD, mini-CEX training tools
Eardley et al., Qualitative PBA, DOPS, Validity, No. of trainees Surgical specialties PBA is the best tool for assessment.
201347 CBD, mini-CEX Reliability, not mentioned Concerns include proper training,
Educational time management, and lack of
impact reliability and validity
Ali et al., ISCP portfolio DOPS. PBA Educational 170 trainees Surgical specialties Lack of feedback is recorded and
201248 impact when recorded, it is not of good
quality
Pereira et al., ISCP portfolio PBA, DOPS, Educational 359 trainees General surgical Negative satisfaction.
Archives of Surgical Research www.archivessr.com 15
Research Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022

201349 Quantitative CBD, mini-CEX impact training Inadequate evidence of validity

Survey
Stepathy et al., Qualitative PBA, DOPS Educational 4 trainees General surgical Concerns about validity and
201117 interviews impact 14 trainers training feasibility of assessment tools
Pereira et al., ISCP portfolio PBA, DOPS, Educational 539 trainees General surgical Poor validity.
200950 Quantitative CBD, mini-CEX impact training 41% reported a negative impact
Survey overall upon their training
Table 6: WBA
DOPS is a feasible, acceptable, and effective tool. Major
concerns were the shortage of time and lack of proper
training13,36–40.
DISCUSSION
There is a vital role of assessment in trainees' progression.
MINI-CEX tool has been suggested for clinical assessment
To improve their clinical skills, several tools have been
developed. Their cognitive knowledge has been assessed as well as teaching because it allows the observation of a
trainee by different observers at different stages of their
by Written and oral exams, while their clinical skills are
training and it also provides effective feedback. As a
assessed by work-based assessment (WBA) methods. These
methods either use real-life patients or simulated patients formative assessment tool, MINI-CEX is well accepted by
the trainers and trainees. Mini-CEX improves technical and
or also provide constructive feedback51.
non-technical skills. It is a valid, reliable, acceptable, and
Out of these 29 studies, only 10 studies discussed purely feasible tool32,33.
general surgery. PBA has been discussed in most of them.
In most of the centers, 40 WBAs per year are being used.
Among these WBAs, PBA appears to be most useful and
then CBD46. PBA is designed to assess the surgical One study suggested that 18 WBAs were used in their
training program every year28. Their quantity should be
competencies of trainees by direct observation and to
adequate to maintain reliability as well as their quality.
provide feedback. PBA is a reliable, valid, acceptable, and
feasible tool to assess trainees learning. PBA has significant Some centers use 80 numbers but they may decrease their
quality10. These studies showed that WBAs should be used
educational value when used as a formative tool. Trainees
as a formative tool to improve their value.
perceive it as a means of targeted feedback and training24–
31.
Miler and Archer stated in their review that WBA leads to
But still, a high level of confusion among trainers and
improved performance of residents but several factors like
trainees exists about both the purpose of PBA and the
correct method of using them. Many of them perceive PBA the presence of facilitation and the design of the feedback
tool can have effects on the response. When multiple tools
as a lengthy tool and prolonged tick box exercise which
(PBA, CBD, DOPS, Mini-CEX) were used together, it did not
questions its acceptability. Emphasis should be on quality
show any improvement in the performance although it had
rather than quantity27,28.
a positive educational impact15. Saedon et al. had a view
that if feedback from workplace-based assessments is well
CBD is a performance-based assessment tool that is used
implemented, it can lead to a positive effect on practice8.
to evaluate clinical assessment and reasoning,
management, communication, and professionalism. CBD is
regarded as a tool for learning because it allows discussion The concept of WBAs among trainees and trainers is
gaining popularity day by day. As most of their users are
of complicated cases, and promotes higher-order thinking.
certain that they help to develop and improve their clinical
CBD is regarded as a reliable and valid tool. It promotes
feedback and evidence-based learning34,35. skills if used correctly. WBAs, when used alone or as a
component, they had an excellent validity, utility, reliability,
acceptability, and high educational impact. But it is not the
DOPS is used to test the "Does" level of the Miller's
Pyramid. In DOPS, a trainee is observed for consent taking, same scenario when these tools are used together. The
possible reasons for this are that WBAs are not used
the indication of the procedure, clinical awareness,
properly, there is always a shortage of time, avoidance of
handling complications, and communication skills, and then
immediate constructive feedback is provided by the face-to-face feedback, lack of faculty training, and use of
WBA as a summative feedback tool17,41–50.
assessor. According to the Royal College of Physicians,
DOPS is a valid and reliable tool in contrast to the old
methods of assessments such as logbooks. DOPS is an The medical field is an everchanging world, which always
demands you to improve yourself and one of the best ways
effective tool for assessing and improving surgical skills.
Archives of Surgical Research www.archivessr.com 16
 Research Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022

to improve is by feedback and assessment. There will
always be a need for a professional assessment of the
trainee and WBAs should be used for this assessment so
that there will be continuous development of clinical skills.
But the point is that they should be properly used so that
most of the benefits can be achieved. We need studies on
larger scales to assess these tools and to define their role in
the training of general surgery residents.
CONCLUSION
Work-based Assessments should be implemented and used
properly after the training of faculty. All components or
methods of WBAs should not be incorporated or used
simultaneously. The focus should be on the quality of the
assessments, not on the quantity. Large-scale studies
should be done to assess level 3 and 4 of educational
impact from WBAs.
Recommendation: We recommend that WBA should be
implemented in all public and private hospitals. The model
of WBA in the UK can be used as a template for this
purpose. Specialty-based relevant methods and tools
should be opted and utilized rather than implementing all
tools together. We also suggest that undergraduate
students should also be included in this setup.

ARTICLE INFORMATION Accepted Medicine. 2009;70(5):290-293. 12. Norcini J, Burch V. Workplace-based

for Publication: June 26, 2022,
Published Online: Jun 30, 2022.
https://doi.org/10.48111/2022.02.03
Open Access: This is an open-access
article distributed under the terms of
the CC-BY License. © 2022 Talat et al
ASR.
Author Affiliations. Department of
Pediatric Surgery Unit II The Children’s
Hospital and the University of Child
Health Sciences, Lahore
Acknowledgements: We acknowledge
the support of pediatric surgery unit 2 of
Children’s Hospital and the University of
Child Health Sciences, Lahore.
Financial Support and Sponsorship: Nil.
Conflicts of Interest: There are no
conflicts of interest
REFERENCES
1. Ahmed I. Post Graduate Surgical Training:
Past, Present and Future in Pakistan.
www.jpmsonline.com
2. childhood MBA of disease in, 2006 undefined.
What’s happening in postgraduate medical
education? adc.bmj.com. Accessed July 5, 2022.
https://adc.bmj.com/content/91/1/68?max-
results=200
3. Swanwick T, Practice NCBJ of G, 2005
undefined. Workplace assessment for licensing
in general practice. bjgp.org. Accessed July 5,
2022.
https://bjgp.org/content/55/515/461.short
4. Swanwick T, Chana N. Workplace-based
assessment. British Journal of Hospital
Archives of Surgical Research
doi:10.12968/HMED.2009.70.5.42235
5. ISCP. Intercollegiate surgical curriculum
overview, The Intercollegiate Curriculum,
Educating the Surgeons of the Future,
Intercollegiate Surgical Curriculum Project,
2017. Accessed in may 2021 from,
https://www.iscp.ac.uk/curriculum/surgica
l/assessment_wbas.aspx. Published 2021.
Accessed July 5, 2022.
https://www.iscp.ac.uk/curriculum/surgica
l/assessment_wbas.aspx.
6. Rowley D, Wass V, Myerson Keith.
Supplementary Guidance Workplace Based
Assessment: A Guide for Implementation.; 2010.
7. Miller GE. The assessment of clinical
skills/competence/performance. Acad Med.
1990;65(9 Suppl):S63-7. doi:10.1097/00001888-
199009000-00045
8. Saedon H, Salleh S, Balakrishnan A, Imray CH,
Saedon M. The role of feedback in improving
the effectiveness of workplace based
assessments: A systematic review. BMC Medical
Education. 2012;12(1). doi:10.1186/1472-6920-
12-25
9. Satpathy A, Kneebone R. Workplace-Based
Assessment Insurgical Training: A Qualitative
Study. The Bulletin of the Royal College of
Surgeons of England. 2011;93(10):1-5.
doi:10.1308/147363511x595082
10. Raj Aryal K. The Utility of Work-Based
assessments in higher general surgical training-
a systematic review. Published online 2019.
doi:10.21203/rs.2.16654/v1
11. Norcini JJ. The Mini-CEX (Clinical Evaluation
Exercise): A Preliminary Investigation. Annals of
Internal Medicine. 1995;123(10):795.
doi:10.7326/0003-4819-123-10-199511150-
00008
www.archivessr.com
assessment as an educational tool: AMEE Guide
No. 31. Medical Teacher. 2007;29(9-10):855-
871. doi:10.1080/01421590701775453
13. Mathew A, Beard J, Bussey M. An evaluation
of the use of direct observation of procedural
skills in the intercollegiate surgical curriculum
programme. Bulletin of The Royal College of
Surgeons of England. 2014;96(9):10-13.
doi:10.1308/147363514x14042954769519
14. Barrett A, Galvin R, Steinert Y, et al. A BEME
(Best Evidence in Medical Education) review of
the use of workplace-based assessment in
identifying and remediating underperformance
among postgraduate medical trainees: BEME
Guide No. 43. Medical Teacher.
2016;38(12):1188-1198.
doi:10.1080/0142159X.2016.1215413
15. Miller A, Archer J. Impact of workplace based
assessment on doctors’ education and
performance: a systematic review. BMJ.
2010;341(sep24 1):c5064-c5064.
doi:10.1136/bmj.c5064
16. Aryal K, Hamed M, Currow C. The usefulness
of work-based assessments in higher surgical
training: A systematic review. International
Journal of Surgery. 2021;94.
doi:10.1016/j.ijsu.2021.106127
17. Satpathy A, Kneebone R. Workplace-Based
Assessment Insurgical Training: A Qualitative
Study. The Bulletin of the Royal College of
Surgeons of England. 2011;93(10):1-5.
doi:10.1308/147363511X595082
18. Page MJ, McKenzie JE, Bossuyt PM, et al. The
PRISMA 2020 statement: an updated guideline
for reporting systematic reviews. Systematic
Reviews. 2021;10(1):89. doi:10.1186/s13643-
021-01626-4
19. van der Vleuten CPM. The assessment of
professional competence: Developments,
17
 Research
research and practical implications. Advances in
Health Sciences Education. 1996;1(1):41-67.
doi:10.1007/BF00596229
20. Fowell SL, Southgate LJ, Bligh JG. Evaluating
assessment: the missing link? Med Educ.
1999;33(4):276-281. doi:10.1046/j.1365-
2923.1999.00405.x
21. Everitt S, Howell C. Encyclopedia of Statistics
in Behavioral Science. 2nd Edition.; 2005.
22. Kirkpatrick. Evaluation of training. In: R.L.
Craig, L.R. Bittel, eds. Training and Development
Handbook. ; 1967:87-112.
23. Crossley J, Humphris G, Jolly B. Assessing
health professionals. Medical Education.
2002;36(9):800-804. doi:10.1046/j.1365-
2923.2002.01294.x
24. Medeiros DCA de, Medeiros PJ de, Diniz RVZ.
Procedure-based assessment (PBA) in urology
residency: initial experience. Revista Brasileira
de Educação Médica. 2021;45(4).
doi:10.1590/1981-5271v45.4-20210346.ing
25. Ramhamadany E. Orthopaedic Trainee
Opinion on the Current Procedure-based
Assessment Compared to its Predecessor: A UK
Trauma Centre Perspective. Cureus. Published
online April 23, 2020. doi:10.7759/cureus.7799
26. Shalhoub J, Marshall DC, Ippolito K.
Perspectives on procedure-based assessments:
a thematic analysis of semistructured interviews
with 10 UK surgical trainees. BMJ Open.
2017;7:13417. doi:10.1136/bmjopen-2016
27. Roushdi I, Tennent D. Current usage patterns
of procedure-based assessments in the
orthopaedic community. Bulletin of The Royal
College of Surgeons of England. 2015;97(2):1-3.
doi:10.1308/147363515x14134529300148
28. Hunter AR, Baird EJ, Reed MR. Procedure-
based assessments in trauma and orthopaedic
training-The trainees’ perspective. Medical
Teacher. 2015;37(5):444-449.
doi:10.3109/0142159X.2014.956055
29. Marriott J, Purdie H, Crossley J, Beard JD.
Evaluation of procedure-based assessment for
assessing trainees skills in the operating
theatre. British Journal of Surgery.
2011;98(3):450-457. doi:10.1002/bjs.7342
30. Beard J, Marriott J, Purdie H, Crossley J.
Assessing the surgical skills of trainees in the
operating theatre: a prospective observational
study of the methodology. Health Technology
Assessment. 2011;15(01). doi:10.3310/hta15010
31. James K, Cross K, Lucarotti ME, Fowler AL,
Cook TA. Undertaking procedure-based
assessment is feasible in clinical practice. Ann R
Archives of Surgical Research
Workplace-Based Assessment Utility Among Surgical Residents: Talat et al, 2022
Coll Surg Engl. 2009;91(2):110-112.
doi:10.1308/003588409X359286
32. Deshpande S, Gudhe M. Utility of mini-CEX in
assessing postgraduates competency in
orthopedics. Journal of Datta Meghe Institute
of Medical Sciences University. 2021;16(1):52-
56. doi:10.4103/jdmimsu.jdmimsu_240_20
33. Joshi M, Singh T, Badyal D. Acceptability and
feasibility of mini-clinical evaluation exercise as
a formative assessment tool for workplace-
based assessment for surgical postgraduate
students. Journal of Postgraduate Medicine.
2017;63(2):100. doi:10.4103/0022-3859.201411
34. Phillips A, Lim J, Madhavan A, Macafee D.
Case-based discussions: UK surgical trainee
perceptions. The Clinical Teacher.
2016;13(3):207-212. doi:10.1111/tct.12411
35. Awad Z, Hayden L, Muthuswamy K, Ziprin P,
Darzi A, Tolley NS. Utilisation and outcomes of
Case-based discussion in otolaryngology
training. Clinical Otolaryngology. 2015;40(2):86-
92. doi:10.1111/coa.12321
36. Inamdar P, Hota PK, Undi M. Feasibility and
Effectiveness of Direct Observation of
Procedure Skills (DOPS) in General Surgery
discipline: a Pilot Study. Indian Journal of
Surgery 2021 84:1. 2021;84(1):109-114.
doi:10.1007/S12262-021-02953-5
37. Sethi S, Badyal DK. Clinical procedural skills
assessment during internship in
ophthalmology. J Adv Med Educ Prof.
2019;7(2):56-61.
38. Hussain Waqar S, Ali SZ. Direct Observation
of Procedural Skills As Assessment Tool Pak
Armed Forces Med. Vol 66.; 2016.
39. Ali L, Ali S, Orakzai N, Ali N. Effectiveness of
Direct Observation of Procedural Skills (DOPS)
in Postgraduate Training in Urology at Institute
of Kidney Diseases, Peshawar. Journal of the
College of Physicians and Surgeons Pakistan.
2019;29(6):516-519.
doi:10.29271/jcpsp.2019.06.516
40. Awad Z, Hayden L, Muthuswamy K, Ziprin P,
Darzi A, Tolley NS. Does direct observation of
procedural skills reflect trainee’s progress in
otolaryngology? Clinical Otolaryngology.
2014;39(3):169-173. doi:10.1111/coa.12251
41. Aryal KR, Currow C, Downey S, Praseedom R,
Seager A. Work-Based Assessments in Higher
General Surgical Training Program: A Mixed
Methods Study Exploring Trainers’ and
Trainees’ Views and Experiences. The Surgery
Journal. 2020;06(01):e49-e61. doi:10.1055/s-
0040-1708062
www.archivessr.com
42. Nathoo NA, Sidhu R, Gingerich A. Educational
Impact Drives Feasibility of Implementing Daily
Assessment in the Workplace. Teaching and
Learning in Medicine. 2020;32(4):389-398.
doi:10.1080/10401334.2020.1729162
43. Gaunt A, Patel A, Rusius V, Royle TJ, Markham
DH, Pawlikowska T. ‘Playing the game’: How do
surgical trainees seek feedback using
workplace-based assessment? Medical
Education. 2017;51(9):953-962.
doi:10.1111/medu.13380
44. Gaunt A, Patel A, Royle J, et al. What do
surgeons and trainees think of WBAs and how
do they use them? The Bulletin of the Royal
College of Surgeons of England.
2016;98(9):408-414.
doi:10.1308/rcsbull.2016.408
45. Pentlow A, Field J. The educational value of
work-based assessments: a survey of
orthopaedic trainees and their consultant
trainers. Journal of Surgical Simulation. 2015;2.
doi:10.1102/2051-7726.2015.0013
46. Phillips A, Jones A. The validity and reliability
of workplacebased assessments in surgical
training. The Bulletin of the Royal College of
Surgeons of England. 2015;97(3):e19-e23.
doi:10.1308/147363515X14134529301147
47. Eardley I, Bussey M, Woodthorpe A, Munsch
C, Beard J. Workplace-based assessment in
surgical training: experiences from the
Intercollegiate Surgical Curriculum Programme.
ANZ Journal of Surgery. 2013;83(6):448-453.
doi:10.1111/ans.12187
48. Ali AS, Bussey M, O’Flynn KJ, Eardley I. Quality
of Feedback Using Workplace Based
Assessments in Urological Training. British
Journal of Medical and Surgical Urology.
2012;5(1):39-43.
doi:10.1016/j.bjmsu.2011.10.001
49. Pereira EAC, Dean BJF. British Surgeons’
Experiences of a Mandatory Online Workplace
Based Assessment Portfolio Resurveyed Three
Years On. Journal of Surgical Education.
2013;70(1):59-67.
doi:10.1016/j.jsurg.2012.06.019
50. Pereira EA, Dean BJ. British surgeons’
experiences of mandatory online workplace-
based assessment. J R Soc Med.
2009;102(7):287-293.
doi:10.1258/jrsm.2009.080398
51. Sultan AS, Ali R, Shakil S, Khan RN. Workplace
Based Assessment: Tools to assess
competencies in a clinical setting. J Pak Med
Assoc. 2021;71(Suppl 1)(1):S89-S93.
18