OSCE design for 3 rd Year Medical student

Dr Nishat Ahmed (2021)

Part 1: OSCE design of a history taking station of 3rd Year Medical

Students

Part 2 : Reflection and critique of the station

Word count: Approximately 2023 ( Excluding the references /

Bibliography and the Table )
OSCE DESIGN:

HISTORY TAKING STATION FOR THIRD YEAR MEDICAL STUDENTS

After studying how OSCEs are designed and used across the world and
applying what I learned in this module, I designed this OSCE station for a
common presentation in Acute Medicine

Assessed learning objectives (Mapped using the third year workbook) and
GMC outcomes for graduates.
The Medicine block workbook for Leicester Medical School outlines the
aims and learning outcomes. Some of these are listed below:
“Students should continue to improve their skills in history taking and
clinical examination, case presentation, communication and diagnostic
reasoning. Students should be able to recognise common medical
conditions, and be able to describe their investigation, treatment and
prevention.”
“Symptom based approach to patients with chest pain, palpitations, limb
pain and swelling and syncope / presyncope.
Recognise, investigate and manage arrhythmias (including peri-arrest),
angina, myocardial infarction / acute coronary syndromes, valvular heart
disease, heart failure, cardiac tamponade, hypertension (including
retinopathy), hyperlipidaemia and peripheral vascular disease” (Workbook
for 3rd year medical students).
The blueprint for aspects of professionalism, communication,
interpersonal skills, and patient safety that are being assessed are taken
from the GMC guide for tomorrow's doctors.

Station History taking

Assessment Breathlessness
Candidate level 3rd Year Medical student
Total time 11 minutes
Aim of the station To assess the history taking skills
of a third year medical student.
The ability to formulate a diagnosis
and differentials based on the cues
given in the history. To assess
communication skills and
professionalism, in the manner
 expected of a 3rd year medical
student who has spent time on the
wards and experienced a few acute
presentations.

Type of patient:
Simulated patient who is able to give clear answers when asked.

Equipment required
Two laminated A4 papers detailing the scenario, one to be kept outside
and one inside the room.
Another A4 paper with patient’s observations, available near the patient’s
bed.
Hand gel on the table.
Two chairs one for the student and one for the examiner. One trolley for
the patient.
Patient is sat on the trolley.
Hospital gown for the patient.

Time structure
This station is for 11 minutes
0 minute- candidate arrives outside room to read instructions.
1 minute- first bell, candidate enters the room.
8 minutes- bell rings to notify the candidate to stop history taking and
summarise the findings to the examiner and answer the rest of the two
questions of the examiner.
11 minutes - final bell, candidate leaves and moves on to next station.

Instructions to Candidates:
This is Mr Charlie Smith, a 55 years old gentleman who has been
admitted to the Medical Admissions Unit. He complains of worsening
shortness of breath over the last three weeks. Please take a full history.
There is no need to examine the patient.
An observation chart will be present next to the patient’s trolley.
Observations:
Temp-36.5, BP- 142/84, HR- 104, Oxygen Saturations are 94 % on RA.

You have 8 minutes to take history and further 2 minutes to summarise

your findings and answer the questions of the examiner.

Instructions to the simulated patient :

You are Mr Charlie Smith 55 years of age.
You will be asked a series of questions to try and determine what is the
cause of your symptoms.
The symptoms that you have noticed are:
You have been suffering from shortness of breath for the last 3 weeks.
This difficulty in breathing is on exertion, and settles with rest. However
in the last couple of days you have started to feel short of breath at rest
as well. You are not able to lie flat and are sleeping in a recliner chair due
to shortness of breath. You occasionally wake up in the middle of the
night gasping for breath.
If asked anything that makes your breathing worse, you can mention that
going upstairs is particularly difficult and also noticed that your breathing
is worse when you lie flat.
Your mobility is limited as a result of the shortness of breath, previously
you were independent and were able to work as a healthcare assistant
attending to the needs of the patients. This is worrying you as you feel
you may be dependent on others which you do not wish to be.
Important negatives:
You do not have chest pain
You do not have Asthma and you are not a smoker.
There is no cough or fever.
There is no weight loss. No change in your bowel or bladder.
No fever. In particular no cough.
You have not been exposed to asbestos and you do not have any pets at
home.
You have not had any extra beats in your heart.
If asked if you travelled or had recent surgery you did not.
Health conditions that you require treatment for:
High blood pressure and you take medication for this. You are pre-
diabetic on diet-control and do not take any medications for this.
You have never experienced anything like this in the past.
Social history:
You drink alcohol socially
You smoked only one or two cigarettes as a teenager until your early 20’s
and since then you have not smoked at all.
You work as a healthcare assistant.
No one in your family that you are aware of had any heart problems.

Your usual health : You would describe yourself as otherwise healthy. You
take your blood pressure medications regularly.
Your medications :
You take Amlodipine 10mg everyday. You do not have any allergies that
you know of. You do not use any inhalers.
Your lifestyle :
Your work involves lifting and turning patients and you have been able to
do work with no problems until the last 4 weeks.
Your mental health : You are worried about your symptoms and that you
may be dependent on others. You expect the candidate to demonstrate
empathy. You are currently off work and find that your work is supportive
to your illness as you are not known to take sick leaves.

Instruction for Assessors :

This station will assess a candidates data gathering and assimilation of
information to form a provisional diagnosis and differential diagnoses.
Based on the cues taken from the history, the candidate is expected to
come up with a list of investigations that will help in diagnosing the
underlying cause and be guided on the investigations for further
treatment.

If candidates ask for any details about the case.

Please ask the candidate to refer to candidate instructions.
If candidate asks for observations, refer to the chart in the vicinity of the
patient.

You will be asking two questions:

1. Summarise your findings and what is your diagnosis or differential
diagnoses.
The candidates should be able to identify the salient features in the
history that point to the diagnosis of heart failure. For example,
SOB on exertion and orthopnoea., along with ankle oedema and
PND. They should list important negatives such as no chest pain,
cough and no weight loss. They should be able to say that the
symptoms point towards a diagnosis of heart failure. Other
diagnosis they would consider is underlying chest infection or lung
diseases such as fibrosis or asthma.
2. What will be your next steps after taking history and what
investigations would you like to order?

Marking Scheme:

The marking in this OSCE station is by a single examiner.

Domains:
• Information gathering
• Clinical reasoning
 • Investigations and next steps to confirm a diagnosis
• Communication
• Professionalism

0 1 2 3 4
Information
Gathering
Clinical
Reasoning
Management
Communication
Professionalism

Mark scheme 0-4

0. None of the descriptors in the domain are met.

1. Few descriptors are met, but does not fulfil the majority of criteria.
2. Reasonable number of descriptors are met but lacks detail.
3. Most of the descriptors are met, with only minor omissions. The
standard is in line with that expected of a Year 4 doctor.
4. All the descriptors for this domain are met.

History taking skills

Anchor Descriptors
Information Gathering
• History of presenting illness:
Starts by asking open questions – what are the main symptoms
and their duration
Explores key symptoms in depth. Focus history to elicit
precipitating factors such as exertional and orthopnoea and
elicits history of paroxysmal nocturnal dyspnoea.
Asks about important negatives such as chest pain.
• Systemic history:
Enquires about past medical history. Explores risk factors for
cardiovascular disease.
Focussed system enquiry. Looks for other causes of
breathlessness
The manner is focused to the presenting illness and relevant to
this case.
• Past medical history : Other risk factors for cardiovascular
diseases such as hypertension and diabetes

Clinical reasoning:
Focuses questions to formulate the likely diagnosis and rule out other
differentials that can cause the symptoms.
Picks up on cues and focusses as appropriate.
Ability to interpret and analyse the given history to formulate a
differential diagnosis of heart failure, lung conditions such as lung
COPD , Asthma or fibrosis, chest infection. Other heart conditions
such as pulmonary hypertension which could be primary or
secondary. Points out the features in history suggestive of heart
failure, for example shortness of breath on exertion, ankle oedema
and orthopnoea. They should list important negatives such as no
chest pain, cough and no weight loss.

Summary and next steps :

Able to summarise the relevant points from the history in a logical
way and able to formulate a list of differential diagnosis. Able to
mention heart failure as a cause of breathlessness in this case.
Next steps: Would mention that they would like to examine for ankle
oedema, JVP and auscultate chest for crackles.
Next appropriate investigations to find out the underlying diagnosis:
Candidate mentions CXR, ECG, Urea and electrolytes specialised tests
such as echocardiogram.

Communication:
Is clear and uses appropriate tone.
Is considerate to the patient’s feelings and addresses concerns
around his symptoms.
Shows interest in what the patient is saying.
Avoids jargon.

Professionalism:
Introduces self to the patient and seeks permission to take a detailed
history of their illness.
Is patient centred.
Demonstrates empathy and compassion
Acknowledges concerns and addresses the concerns.

Global Rating Score

Fail Borderline Pass Good Excellent

Global rating score assesses the total competence or performance

for that station.
This is used to rate the overall competence of the candidate for the
station.

Examiner feedback to the candidates:

Please provide feedback to the candidates of any suggested areas of

improvement and/or what was done well.

Significant Reservations:
Please indicate any concerns about the candidate which you feel
should be reviewed by the circuit lead. You are asked to use your
professional judgement on the seriousness of the concern and
provide details below.
Examples of behaviour that raise the concerns about the candidate
are:
• Any behaviour or action that jeopardises patient safety.
• Inappropriate or offensive language is used towards the
patient or other staff
• Serious professional concerns such as being dishonest in their
dealing with the patient or during answering the examiner.
• If the candidate is being discriminatory or racist or ageist or
sexist in their dealings with the patient or other members of
staff.
Part 2

Student ID : 20416708

Critique of the station

After reading around how OSCEs are used as a method of assessment and
designed and reflecting and also my understanding of what was taught in this
module, I have designed this OSCE station in Acute Medicine which is my
area of expertise.

The critique and reflections of the design based on the available literature
and my own learning during this process are described in this essay

What is an OSCE

Objective Structured Clinical Evaluation (OSCE) is a method of assessment

that is widely used in medical education, since its introduction by Harden et,
al in 1975.

OSCE is a method of examination that comprises of a series of stations that

run as a circuit. The circuit then repeats for the next batch of candidates
(Harden et al., 1975).
OSCEs are now a popular way of assessing candidates not only for licencing
exams but also in undergraduate medicine (Patricio et al., 2013). OSCEs are
a useful tool for ensuring that students are evaluated in relation to their
engagement with patients, which includes communication, empathy, and
other skills essential to progress and complete training to become a doctor.
The GMC uses OSCE for it’s licencing exams.

This has replaced the traditional methods of testing which were long case
scenarios. The main drawback of the long case was that the test was based
on only one case and decision about competence of a candidate was based
on this. The long case was thus low in reliability and validity.

George E. Miller (1990) identified hierarchical tiers of competence that

can be used as a framework for assessment. Miller's model has multiple
layers that represent distinct stages of growth.
Miller visualised competence in the form of a pyramid. The pyramid's base
is made out of factual knowledge. Miller defined the ability to apply
information in a specific environment as "knows how" at a higher level.
This resembles clinical reasoning and problem solving in certain ways. At
a deeper level, "showing how" describes hands-on behaviour in a
simulated or practise environment and displays the person's capacity to
respond effectively in a practical circumstance. OSCEs aim to test the
“show how” tier of the pyramid.
The way OSCEs are designed and learning takes place, it follows that this
type of assessments encourages students to practice more in the clinical
environment (Stephen Mulller 2019). From my experience this
encourages students to engage more with clinical activities and practise
their examination and communication skills with peers and seniors around
them.

Choice of the station and blueprinting:

This will be a history taking station of a case of breathlessness, this a

common presentation in acute medicine. I examined the workbook of third-
year medical students of Leicester medical school, and this book contains
information about the basic learning that is expected of the students during
their clinical placements. The workbook emphasises that students are
required to take histories and also encourages the students to acquaint
themselves with the common presentations in acute admissions. Heart failure
is one among them.

I also considered the GMC outcomes for graduates and GMC outcomes for
tomorrow’s doctors.
Mapping Outcomes for graduates to Tomorrow’s doctors (GMC Website)

Outcomes 2009 Outcomes 2018

The doctor as a practitioner Professional Skills

15. Communicate effectively
with patients and colleagues in
a medical context.
15a. Communicate clearly,
sensitively and effectively with
patients, their relatives or other
carers, and colleagues from the
medical and other professions,
by listening, sharing and
responding.
15c. […] The graduate should
appreciate the significance of
non-verbal communication in
the medical consultation.
We expect doctors to demonstrate appropriate skills in clinical
practice.
Communication and interpersonal skills
10. Newly qualified doctors must be able to communicate
effectively, openly and honestly with patients, their relatives,
carers or other advocates, and with colleagues, applying patient
confidentiality appropriately.
They must be able to:
10a. communicate clearly, sensitively and effectively with
patients, their relatives, carers or other advocates, and
colleagues from medical and other professions, by:
• listening, sharing and responding
• demonstrating empathy and compassion
• demonstrating effective verbal and non-verbal interpersonal
skills
• making adjustments to their communication approach if
needed, for example for people who communicate differently
due to a disability or who speak a different first language
• seeking support from colleagues for assistance with
communication if neede

Initial thoughts when designing the OSCE is that what is it testing? Blueprint
is important to test the OSCE station. What am I testing? The learning
outcomes and course objectives are used to create test blueprints
(Raymond and Grande 2019). Hence, I decided to use the workbook and
GMC outcomes to use as blueprint for this station.
Blueprinting can be done in-house as mentioned in AMEE guide no. 81 the
topics decided by the local assessment team or in high stakes exams a
Delphi or other types of survey are used to come to a consensus of the
selected topics (Khan et al., 2013).
Year 3 workbook explains that students are expected to gain experience
in medical admissions and clerk as many cases as possible and present
prepared cases. Breathlessness is a common presentation in acute
admissions.

Assessment is vital in medical school because it is used to monitor

students' progress toward the expected learning outcomes. In addition
there is evidence in the literature that assessments drive learning and
serves as a key for students to learn (Wormald et al., 2009), this links to
the purpose of blueprinting.

Other considerations in station design:

The timings of the station were kept as 10 minutes for history taking. My initial
thoughts were to assess a focused examination in addition to history taking.
This, however, may have resulted in the station not having enough time for one
activity, which is history taking and could also effect test reliability (the degree
to which the test consistently measures what it is supposed to measure) if there
is too little testing time (Smee 2003; Trejo-Mejia 2016). Validity of a test is the
extent of what it is supposed to measure. There is evidence to suggest the time
and the number of stations are directly related to the validity and reliabilty of
the test (Gromley 2011; Downing 2004). Hence, I decided to keep the 10-
minute time, which is standard practise in most OSCEs that allows sufficient
stations in a realistic time frame to be conducted on the day. A review of the
literature reveals that the number of stations and duration of the examination
have an impact on the test's validity and reliability (Newble 2004, Robert et
al.2006). This is reflected in the GMC guide for assessments in undergraduate
medicine: “Key to increasing the reliability or generalisability of OSCEs is
providing a sufficient number of separate cases or stations”.

This station depicts acute admissions. I decided to create an environment similar

to a ward. With hospital bed and gown for the patient. Patient’s chart is kept at
the bedside.

The scenario is written in clear and big font so that it is easy to read and also a
second one is available in the room, in case the student would like to refer to it
again. One minute time appears to be sufficient to allow the student to read the
scenario.

This station is designed with a view of using a simulated patient to portray the
patient. As long as SP performance is credible and medically realistic, the use of
SPs in healthcare education is commonly recognised (Bowman et al. 1992).
Simulated patients are either lay persons or actors. A study by Sullivan et al.
demonstrates that SPs are motivated to contribute to the important field of
medical education. The simulated patient is briefed on the case. Training of the
simulated patient reduces variability and keeps the test consistent throughout
the OSCE circuit (Perera et al.,2009). Depending on the level of difficulty SPs
can be trained to be vague and requiring the students to ask direct questions to
elicit the answer from the SP (Cleland et al.,2009). Since the student is a novice
learner, I decided that the history taking will be straight forward, and the
simulated patient will give answers to open ended questions such as ‘what
makes your breathlessness worse’. However, it is important that the SP is
consistent in giving the answers, for the station to be marked fairly. That is the
reason, GMC guidance for assessments suggests that examiners and SP’s should
be briefed and monitored for consistency.
Marking scheme:

The marking of this station is by a single assessor. To avoid subjectivity error,

Tavakol and Pinner (2018) have suggested that 2 independent assessors in each
station would be better, however they also recognise that this may not be
feasible and it is better to have more stations with single assessor rather than
less stations with multiple assessors.

There are two ways in which the OSCE’s are marked: checklist or domains in
conjunction with a global rating scale. Initially, checklists were proposed as they
were assumed to be meeting the criteria of the reliability and objectivity of the
marking scheme (Harden 1979). Nevertheless, the idea of checklist ‘ tick boxes’
can lead to students memorising the criteria to tick boxes instead of portraying
of how they will conduct themselves in a real clinical encounter. Sadler (2009)
highlighted some of the disadvantages of using checklists for marking 1.
Moreover, if there is a discrepancy between the checklist scores and global
ratings, means that there is a problem with the station (Pell et al. 2015) In the
domain-type of marking scheme, examiner will give an overview of how well the
candidate performed in each domain. The scale used is of 0-4. There is work to
suggest that global ratings are better in reliability than checklists although the
difference was only small (Cohen et al. 1996). More research is needed to
compare the two and determine whether it is best to use checklist for example in
OSCE’s for procedures such as cannulation and use domains for stations where a
judgement is made on skills such as communication and history taking.

Standardisation is important in an OSCE station to distinguish the pass from the

fail.

The mark sheet scores candidates on 5 domians :

• Information gathering
• Clinical reasoning
• Investigations and next steps to confirm a diagnosis
• Communication
• Professionalism

These domains have descriptors and based on the judgement of the

assessor and whether the criteria in the descriptors are met by the
candidate they are scored on a scale of 0-4. The descriptors of
professionalism and communication are designed to align with the GMC
outcomes of tomorrow’s doctors and outcomes for graduates. The
domains of information gathering and clinical reasoning are set in such
a way that gives an understanding of how well the candidate is able to
corelate the history of orthopnea and exertional dyspnoea with heart
failure. Working in acute medicine, I come across this scenario often
and sometimes referred as pulmonary embolism whereas on carefully
eliciting the history it is suggestive of heart failure. Peterson et al.

1
Indeterminacy in the use of preset criteria for assessment and grading
D. Royce Sadler
 (1992) makes an observation that history will lead to a diagnosis in
76% of cases.

The setting of domains and criteria for clear pass, borderline and fail
can be discussed among peers and the examiners briefed about the
station.

Alongside examiner understanding of the station that is being

assessed, examiner training is vital to improve and standardize the
OSCE ( Reid et al. 2016, Boursicot and Roberts 2005).

Global rating score is used to judge the overall performance of the candidate for
that station. It helps identify errors in measurement and helps identify problems
with the station. If the objective assessment of checklists was good but poor on
the global rating score, it helps understanding the errors in assessment (Pell et
al. 2015) and thus check the quality of the station.

Standard setting:

GMC emphasises that for assessments, medical schools should adopt a robust,
transparent, and consistent approach to standard setting that meets the
objectives of Tomorrow's Doctors (2009).

The AMEE Guide 85 describes a number of methods for carrying out standards
setting on an OSCE. For the sake of simplicity, only three methods are described
here that are commonly used 1) Borderline regression 2) Borderline group and
2) Angoff

Borderline group method uses the scores of borderline candidates to determine

the pass score. The examiner gives an overall global rating of the candidates'
performance in the categories of fail/ borderline/pass. The station's pass mark is
the mean of all borderline scores, and the OSCE's pass mark is the sum of all
pass marks of the individual station. (Newble, 2004). Borderline method is
simpler to use and can be used reliably to define standards ( Kaufmann et al.,
2000). The main drawback is if there are only a few number of borderline
candidates. In such cases borderline regression method is used to set pass
marks. (Chapter 24, Understanding Medical Education)

To summarise, a high fidelity OSCE requires careful designing, wherein the

learning outcomes to be achieved, the marking and scoring of domains and
setting standards to the level expected of the regulatory bodies and general
public to whom the healthcare professionals are accountable.
References/ Bibliography :
1. Outcomes for graduates – GMC 2018

2. Tomorrow’s doctors- GMC 2009

3. Harden RM, Gleeson FA. Assessment of clinical competence using an

objective structured clinical examination (OSCE). Med Educ. 1979
Jan;13(1):41-54. PMID: 763183.

4. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81.
Part II: Organisation & Administration

5. Madalena Folque Patrício, Miguel Julião, Filipa Fareleira & António Vaz
Carneiro (2013) Is the OSCE a feasible tool to assess competencies in
undergraduate medical education?, Medical Teacher, 35:6, 503-
514, DOI: 10.3109/0142159X.2013.774330

6. The long case and its modifications: a literature review Gominda G

Ponnamperuma ,Sean McAleer,Margery H Davis
https://doi.org/10.1111/j.1365-2923.2009.03448.x

7. How the introduction of OSCEs has affected the time students spend studying:
results of a nationwide study
Stefan Müller
8. The Objective Structured Clinical Examination(OSCE): AMEE Guide No. 81.
Part II:Organisation & Administration KAMRAN Z. KHAN

9. A practical guide to test blueprinting Mark R. Raymond & Joseph P. Grande

10.AMEE Guide No. 25: The assessment of learning outcomes for the competent
and reflective physician

11.Assessment Drives Learning: An Unavoidable Truth?

October 2009Anatomical Sciences Education 2(5):199-204
DOI:10.1002/ase.102

12.ABC of learning and teaching in medicine

Skill based assessment
BMJ 2003; 326 doi: https://doi.org/10.1136/bmj.326.7391.703 (Published
29 March 2003)

13.Reliability analysis of the objective structured clinical examination using

generalizability theory
Juan Andrés Trejo-Mejía

14.Summative OSCEs in undergraduate medical education

Gerry Gormley 2011

15.Downing S. Reliability: on the reproducibility of assessment data. Med Educ.

2004

16.Their OSCE, not your Oscar: Simulated Patients’ Perspectives

Clare Sullivan,Claire M. Condron,Laura O'Connor,Teresa Pawlikowska,James
M. Murray. First published: 27 July 2021 https://doi.org/10.1111/tct.13400

17.Training simulated patients: evaluation of a training approach using self-

assessment and peer/tutor feedback to improve performance
Jennifer Perera
18.The use of simulated patients in medical education: AMEE Guide No 42
Jennifer A Cleland
19.https://www.gmc-uk.org/-
/media/documents/Assessment_in_undergraduate_medical_education___gui
dance_0815.pdf_56439668.pdf
20.Enhancing Objective Structured Clinical Examinations through visualisation
of checklist scores and global rating scale
Mohsen Tavakol corresponding author 1 and Gill Pinner 1

21.Indeterminacy in the use of preset criteria for assessment and grading. D.

Royce Sadler

22. Disparity in global and checklist OSCE scores. Godfrey Pell, Matt Homer,
Richard Fuller
23. Contributions of the history, physical examination, and laboratory
investigation in making medical diagnoses. M. C. Peterson, et al.

24. Taking OSCE examiner training on the road: reaching the masses
Katharine Reid

25. How to set up an OSCE

Katharine Boursicot, et al.

26. Pell, G, Homer, MS and Fuller, R (2015) Investigating disparity between

global grades and checklist scores in OSCEs. Medical Teacher. ISSN 0142-159X
https://doi.org/10.3109/0142159X.2015.1009425

27. A Comparison of Standard-setting Procedures for an OSCE in Undergraduate

Medical Education
Kaufman, David M. EdD; Mann, Karen V.

28. Understanding medical education – Chapters 20 and 24.