British Journal of Anaesthesia, 120 (2): 264e273 (2018)

doi: 10.1016/j.bja.2017.09.007
Advance Access Publication Date: 24 November 2017
Review Article

Competency-based assessment tools for regional

anaesthesia: a narrative review
A. Chuan1,2,*, A. S. Wan2, C. F. Royse3,4 and K. Forrest5
1
South Western Sydney Clinical School, University of New South Wales, Sydney, Australia, 2Department of
Anaesthesia, Liverpool Hospital, Sydney, Australia, 3Department of Surgery, University of Melbourne,
Melbourne, Australia, 4Department of Anaesthesia, Royal Melbourne Hospital, Melbourne, Australia and
5
Faculty of Health Sciences and Medicine, Bond University, Queensland, Australia

*Corresponding author. E-mail: dr.chuan@iinet.net.au

Abstract
Competency-based assessment tools are used in regional anaesthesia to measure the performance of study participants,
trainees, and consultants. This narrative review was performed to appraise currently published assessment tools for
regional anaesthesia. A literature search found 397 citations of which 28 peer-reviewed studies met the inclusion criteria
of primary psychometric evaluation of assessment tools for regional anaesthesia. The included studies were diverse in
the type of assessment and the skill set being assessed. The types of assessments included multiple-choice questions,
hand-motion analysis, cumulative sum, visuospatial and psychomotor screening, checklists, and global rating scales.
The skill sets that were assessed included holistic regional anaesthesia technical and non-technical performance
observed at the bedside, to isolated part-tasks, such as needle tip visualisation under ultrasound. To evaluate validity and
reliability, we compared the studies against published medical education consensus statements on ideal assessment
tools. We discuss the relative merits of different tools when used to assess regional anaesthesia, the importance of
psychometrically robust assessment tools in competency-based anaesthesia education, and directions for future edu-
cation research in regional anaesthesia.

Keywords: anaesthesia; conduction; credentialing; psychometrics; reproducibility of results

The assessment of complex anaesthesia procedures, such as
regional anaesthesia, requires evaluation of disparate knowl-
edge, skill sets, and professional attributes. Practically, this is
achieved by the use of assessment tools.1 With the interna-
tional shift towards competency-based postgraduate medical
education, these tools assume a central role in current
anaesthesia curricula by assessing the trainees’ performances
against predefined learning objectives, standards of perfor-
mance, and behaviour.1e6 The performances can range from
the entire nerve block procedure on patients in a clinical
setting, to part-tasks relevant to regional anaesthesia, such as
anatomy knowledge and ultrasound-guided needle skills in
simulation settings.
The design, structure, and quantitative characteristics of
an assessment tool are defined as its psychometric properties.
Evaluating a tool’s psychometrics (‘assessing the assessment
tool’) is necessary for acceptance by all stakeholders. These
properties include validity, reliability, feasibility, educational
stimulus, and acceptability. For clinicians, validity and reli-
ability are required to demonstrate robustness and confidence

Editorial decision: October 26, 2017; Accepted: December 1, 2017

© 2017 British Journal of Anaesthesia. Published by Elsevier Ltd. All rights reserved.
For Permissions, please email: permissions@elsevier.com

264
 Assessment tools for regional anaesthesia - 265

in assessment results. In research studies, validated and reli-

Table 1 Psychometric properties of an ideal competency-
able tools are needed to measure performance changes after
based regional anaesthesia assessment tool. Adapted from
training interventions and for serial measurement of perfor- the 2010 Ottawa Conference for Competence in Medicine and
mance over time.7 Healthcare ‘Criteria for good assessment: consensus state-
Until relatively recently, the principles of assessment and ment and recommendations from the Ottawa 2010 Confer-
the methods of psychometric evaluation were not well ence’ by Norcini and colleagues.8 Further definitions from
disseminated beyond the medical education literature. To Bould and colleagues,1 Ahmed and colleagues,4 Van der
Vleuten,10 and Gallagher and colleagues11
improve the consistency and robustness of studies that sought
to design and evaluate assessment tools, the 2010 Ottawa
Property Description and comments
Conference for Competence in Medicine and Healthcare pub-
lished a consensus statement on ideal assessment tool prop- Validity The tool measures what it purports to
erties,8 describing a framework of reporting standards and measure.
statistical techniques used in psychometric evaluation.9 We A body of evidence, built up over
adopted this framework to appraise the published regional multiple studies, supports validity.
Must also demonstrate reliability
anaesthesia assessment tools. This narrative review dis-
cusses the type and purpose of current tools, and categorizes Face validity Suitability of the tool to the real-life
their psychometric properties, implications for clinical adop- regional anaesthesia clinical reality
tion, and priorities for research. Content Criterion-based regional anaesthesia
validity knowledge, skill set, or behaviour
being tested
Methods Construct Test scores differentiate inexperienced
validity and experienced proceduralists.
Criteria for ideal assessment tools
Discriminate Test scores differentiate novice vs expert
A utility index of psychometric properties of the ideal validity proceduralists.
competency-based assessment tool was first proposed by Van
Concurrent Test scores are consistent with an
der Vleuten10 in 1996. This was officially adopted as a validity external and objective outcome.
consensus statement at the 2010 Ottawa Conference as the Examples have included measures of
document ‘Criteria for good assessment: consensus statement block performance, such as success,
and recommendations from the Ottawa 2010 Conference’.8 time to onset, number of needle
passes, and complication rates.
The properties that defined a good assessment tool included
validity, reliability, feasibility, educational stimulus, and Predictive Test scores are predictive of clinical
acceptability of a tool by stakeholders. Table 1 summarizes validity practice.
these properties with a specific focus on regional anaesthesia. An example is in vitro simulation scores
predict in vivo block performance.
In practice, every assessment tool is a compromise be-
tween competing properties. This interdependent balance is Reliability Reproducibility of test scores if repeated
under similar circumstances
most pertinent for validity and reliability. At a minimum, this
External reliability is more important
requires evidence that the important skill sets of that task are
than internal reliability. In practice,
examined (face and content validity), whether the tool differ- trainees are scored by different
entiate trainees of different abilities (construct validity) and assessors and over multiple occasions
reliability (reproducibility of results).12,13 Of the different types throughout training; thus,
of reliability, external reliability is more important in a clinical demonstration of external reliability is
essential for validity and acceptability.
context, as it concerns consistency of scoring among different
faculty members who assess trainees. Internal Different trainees scoring consistently in
Content validity is typically assured by anchoring with reliability the same section of the tool
expert consensus guidelines. Examples include the regional External Different assessors scoring consistently
anaesthesia learning objectives in the 2015 Accreditation reliability for the same trainee performance
Council for Graduate Medical Education Milestone project.14 Also called between-assessor or inter-
rater agreement
For ultrasound-guided techniques, the joint committees of
the American Society of Regional Anesthesia and Pain Medi- Testeretest Consistency of scores over different times
cine and the European Society of Regional Anaesthesia and reliability for the same performance
Pain Therapy describe minimum competencies.15 Feasibility Minimal time, logistics, financial costs,
and assessor training required
Educational Allows structured feedback, deliberate
Framework for psychometric evaluation catalyst practice, and reflection (formative
Psychometric evaluation is an examination of these assess- assessment)
ment tool properties against how well it conforms to purpose. Acceptability Acceptability dependent on the
These often use quantitative statistical analyses. In research stakeholder and purpose of tool
studies designing new assessment tools, a primary aim is to For example, medical registration boards
evaluate reliability. However, previous reviews have noted accept tools if there is evidence of high
reliability and validity. Feasibility and
limitations in the statistical analyses of reliability used by
educational effect are not as important.
published studies.4,16 These limitations include heterogeneity Clinical tutors will accept a tool for its
and incorrect use of tests, precluding any meta-analyses of educational benefit and high feasibility.
tools. In an effort to improve the quality of medical education Other properties are not as critical.
research, an additional 2010 Ottawa Conference statement on
 266 - Chuan et al.
reliability methods was published in the document ‘Research
in Assessment’.9 Tests of reliability were classified as de-
rivatives of the classical test theory, generalisability theory,
and item-response theory.
The majority of tests familiar to anaesthetists are based on
the classical test theory. It is assumed that the observed score
X is the sum of the true score T and any errors in measurement
E, described by the formula X ¼ T þ E. Reliability coefficients
estimate this amount of error; higher coefficients suggest low
errors and greater reproducibility in observed test scores.13
Examples of classical tests include split-half methodology,
Cronbach’s a, Cohen’s k, and the intra-class correlation coef-
ficient (ICC).
Split-half methodology is based on the correlation between
scores from one-half of questions in a test against scores from
the other half. This is most appropriately used in large banks
of multiple-choice questions (MCQs). Split-half methods can
identify questions with low correlations, prompting a rewrite
or removal.
Cronbach a is a parallel test design applicable for contin-
uous data, and is, in turn, the general form of the
KudereRichardson 20 test used for dichotomous data.17
Despite being a common test, Cronbach a has been incor-
rectly used and its limitations underappreciated. Firstly,
Cronbach a is artificially increased when used for criteria-based
assessments and by having more items in the test.9 Also,
Cronbach a is sensitive to individual inter-item correlations; a
subset of items may weigh disproportionally on the overall
average Cronbach a value for the entire tool. Equally, the test is
susceptible to missing data with resultant low Cronbach a
values. There are recommendations to calculate standard er-
rors of mean with 95% confidence intervals to provide infor-
mation on the absolute agreement of scores.9,11
The analysis of external reliability can take several forms.
In the simplest, the testeretest method measures scoring
consistency at initial testing vs retesting at a later time by the
same assessor. As a measure of score stability, there is a bal-
ance between retested too early (recall and learning bias) and
too late (assessors may have gained experience in scoring). For
reliability between different assessors, Cohen’s k, ICC, and
Pearson correlation tests have been used.
Cohen’s k is typically used when two assessors are
involved. It is a more robust test than simple percentage
comparisons, as Cohen’s k takes into account the possibility of
chance agreement. For three or more assessors, the ICC sta-
tistic can be used. As ICC is a form of analysis of variance
(ANOVA), the test can correct for chance variance, and is able to
measure consistency and absolute agreement between
scores.18 This is particularly important when compared with
Pearson correlation, which can only measure consistency. In
practical terms, Pearson correlation sets the upper limit of ICC
values in ideal situations, but the calculated ICC is much
lower. Studies that only use Pearson correlation, therefore,
risk overstating the reliability of the assessment tool.19
ICC itself has six statistical forms, and the choice of model
will markedly change the result.20,21 Shrout and Fleiss22
demonstrated that the six ICC models produce results
ranging from 0.17 to 0.91 using the same raw data. As an
example of using ICC to analyse a regional anaesthesia
assessment tool, the following factors are considered: is the
assessed task a random selection of all block procedures, and
are the assessors randomly chosen from all possible assessors
(one vs two way, fixed vs random effects); is the test designed
to be scored by a single assessor, or two or more (single vs
average measures); and does the numerical value of scores
carry discrete information (absolute agreement)?
The most stringent model is ICC (2A, 1), used when a single
randomly chosen assessor scores a randomly chosen trainee
performing a randomly selected regional anaesthesia task,
with absolute agreement in scores. Cronbach’s a is simply the
consistency model ICC (C, k).21 As a rule of thumb, ICC should
be 0.80 or higher for high-stakes exams, and only marginally
less for formative assessments.10,13,19
The generalisability theory is the second major statistical
technique to evaluate reliability. This theory is also a form of
ANOVA and calculates the ratio of true variance (true score) to
total variance (true score and errors). Analogous to ICC
analysis, the assumptions used when performing general-
isability modelling have significant effects on the resultant
coefficient.9,23 Generalisability is uncommon in the anaes-
thesia literature. It has been used previously for paediatric
anaesthesia skills24 and validation of the mini-clinical eval-
uation examination tool in anaesthesia practice.25 One
advantage over classical tests includes identifying the source
of scoring inconsistency from differences in patient difficulty
(case specificity), or from differences in assessors (inter-rater
error variance).13,26 For example, generalisability can provide
indication if an otherwise satisfactory trainee received a low
score because of a technically challenging patient, or if
scoring was biased by a ‘hawkish’ assessor. Another advan-
tage is reliability modelling, akin to a sample-size power
calculation. Based on the raw data set, a decision study es-
timates the effect on reliability when changing the number of
assessments performed vs the number of assessors
required.9
The third option to create assessment tools with inherent
reliability is to use item-response theory. This method has
been used recently to create a 47-item multiple-choice
assessment tool for sono-anatomy relevant for ultrasound-
guided regional anaesthesia (UGRA).27 Unlike the previous
statistical methods, item response is based on the analysis of
individual items in the tool rather than scores by assessors.
Each item is assessed to its probability of a correct answer vs
trainee ability in an iterative process of item calibration and
proficiency estimation. The discriminatory power of each item
is determined by the location index, which is a graphical
representation of the ‘item characteristic curve’. The addition
of Rasch one-parameter mathematical modelling allows the
identification of poorly fitting items, which may then be
examined for causes of poor discrimination, and amended
appropriately or deleted. More complex two- and three-
parameter logistic models allow better discrimination be-
tween trainees of variable ability, but require very large sam-
pling sizes.9
Literature search
We undertook an electronic literature search for articles on
regional anaesthesia assessment tools in PubMed, Embase, and
Medline. Articles were deemed eligible for inclusion if the pri-
mary aim was psychometric evaluation of competency-based
assessment tools for regional anaesthesia knowledge, skills,
or behaviours. The following search terms and medical subject
headings were used: regional anaesthesia, conduction anaes-
thesia, plexus block, brachial plexus block, femoral nerve block,
sciatic-nerve block, neuraxial block, spinal anaesthesia,
epidural anaesthesia, paravertebral block, human, clinical
competence, and not limited for year. The search methodology
 Assessment tools for regional anaesthesia - 267

was registered with PROSPERO (CRD42017068029) and per-
formed in May 2017.
All identified citations were independently reviewed by
authors A.C. and A.S.W. Potentially relevant records, based on
title and abstract, were selected. Earmarked citations, chosen
by either of the blinded authors, were combined into a provi-
sional list. We also manually searched references within ar-
ticles for any relevant citations not already included. Only full-
text articles in the English language and published as peer-
reviewed journal articles were accepted. These authors then
jointly scrutinized the methodology of each full-text article
Idenficaon
against the inclusion criteria, to reach a final list. Each article
in this final list was then compared against the criteria for
ideal assessment tools. The framework for psychometric
evaluation was then applied to each article’s statistical
methods and results.
Results
There were a total of 397 records identified, comprising 391
citations from the literature search and six citations from bib-
liographies, as presented in the PRISMA flow diagram (Fig 1).

Records idenfied through Addional records idenfied

database searching through other sources
(n = 391) (n = 6)

Records aer duplicates removed

(n = 350)
Screening

Records screened by tle Records excluded

and abstract (n = 265)
(n = 350)

Full-text arcles excluded:

(n = 50)
- Did not perform psychometric
Eligibility

analysis of assessment tool

Full-text arcles assessed - Lumbar puncture only
for eligibility - Referred to earlier arcle where
(n = 85) primary validaon was performed

Studies of interest:
(n = 7)
- psychometric analysis of
assessment tool was secondary aim
Included

Studies included in
qualitave synthesis
(n = 28)

Fig 1. PRISMA flow diagram of the study selection process. Six additional records identified through other sources were Smith and col-
leagues (2012),28 Duce and colleagues (2016),29 de Oliveira Filho and colleagues (2008),30 de Oliveira Filho (2002),31 Schuepfer and colleagues
(2000),32 and Friedman and colleagues (2008).33
 268 - Chuan et al.
Twenty-eight articles met the inclusion criteria. Multiple types
of assessment tools were used, including MCQs, hand-motion
analysis, cumulative sum (CUSUM), visuospatial and psycho-
motor screening, checklists, and global rating scales (GRSs).
To allow for an easier comparison of the quality of tools
within each type of regional anaesthesia procedure, the arti-
cles were categorized by the purpose of assessment: use in
part-trainer tasks for regional anaesthesia (Supplementary
Table 1); neuraxial-only regional anaesthesia clinical proced-
ures (Supplementary Table 2); and peripheral regional
anaesthesia clinical procedures, including one tool assessing
peripheral and neuraxial blocks (Supplementary Table 3). An
in-depth explanation of the advantages and disadvantages of
specific types of assessment is provided next.
In seven other articles, an assessment tool was purpose-
fully designed for that study, but psychometric evaluation was
not the primary aim. These seven articles are in
Supplementary Table 4 as studies of interest, as their meth-
odologies did include some information regarding the tool’s
psychometric properties.
Multiple-choice questions
MCQ tests have the advantages of testing multiple knowledge
areas and efficiently testing large groups of trainees. The in-
clusion of clinical information into MCQs makes them context
rich and forces trainees to use decision-making skills to
consider clinical issues alongside theoretical considerations.2
Uniquely in regional anaesthesia, Woodworth and col-
leagues27 reduced an initial 81-item MCQ to 50 items through
expert consultation. This was validated by the item-response
theory and Rasch modelling to a robust final 47-item MCQ.
This final form has evidence for face, content, and construct
validity and reliability in testing sono-anatomy knowledge for
femoral, sciatic, and above-clavicle brachial plexus blocks.
Hand-motion analysis
Motion analysis uses the Imperial College Surgical Assessment
Device (Polhemus, Vermont, USA, and Imperial College London,
UK),57 a radiofrequency tracking system to monitor dexterity, via
sensor-embedded gloves worn by the proceduralist. Three end
points are measured: time taken to perform a task, number of
movements, and path length, as a surrogate of economy of
motion. The attraction of hand motion lies with objective end
points, measured by a computerized tracking system. Two
studies in regional anaesthesia demonstrated construct validity
between novices and experts performing ultrasound-guided
supraclavicular brachial plexus blocks (Chin and colleagues49)
and in labour epidurals (Hayter and colleagues43). In both
studies, a secondary aim was concurrent validation with previ-
ously used combined checklist and GRSs.42,48 However, these
were assessed using Pearson or Spearman correlations.
Visuospatial and psychomotor ability screening
The CattelleHorneCarroll theory is a taxonomic system
dividing human intelligence into cognitive strata that include
visuospatial and psychomotor ability.58,59 Visuospatial ability
itself is defined as the cognitive capacity to generate, retain,
retrieve, manipulate, and process visual information.58 Psy-
chomotor ability influences the capacity to precisely control
objects via bimanual dexterity, and incorporates handeeye
coordination, reaction speed, and accuracy.60 In medicine,
individuals with higher levels of visuospatial and psychomo-
tor ability demonstrate greater proficiency in laparoscopic
surgery, general surgery, interventional radiology, anatomy,
endoscopy, and fibre-optic intubation tasks.61e69 Potentially,
screening for these abilities may identify trainees of lower
ability and help target educational interventions to assist
them attain competency.
Four studies have been performed in regional anaesthesia.
The microcomputerised personnel aptitude tester was used to
screen trainees for psychomotor ability and performance of
labour epidurals (Dashfield and colleagues40). Proficiency was
only weakly associated using Pearson correlation, and results
may also be influenced by not using a formally recognized
neurocognitive test.
The other three studies used tasks related to UGRA. Smith
and colleagues28 recruited trainees and screened visuospatial
and psychomotor abilities using a battery of neurocognitive
tests. All trainees performed a needle visualisation task on a
bench-top model, using time and image-quality end points.
Only the block design test (a subset of the Wechsler Adult In-
telligence Scale) was associated with performance, whilst
psychomotor tests were not significant.
Visuospatial ability was again significant in a needle visu-
alisation task (Shafqat and colleagues36) and brachial plexus
sonography (Duce and colleagues29) studies, both finding that
the Mental Rotations Test-A was significant in discriminating
performance. In the former study, the reliability of the com-
posite error score and GRS was also evaluated with ICC and
Cronbach a, with all results 0.90.
Checklists
Checklists are designed to assess the component skills of a
procedure. Checklists can be specific for individual nerve
blocks, or be intended for a range of blocks. Identification of
the core skills can be determined informally through
consensus opinions or formally through the Delphi process.
During pilot and validation of new checklists, a psychometric
analysis of individual items allows the identification of items
with poor reliability.
Checklists provide a rich source for formative feedback, as
weaker component skills within a whole performance can be
identified. Contrariwise, checklists reward thoroughness in
trainees, but not necessarily discriminate between experts, as
the latter group tends to use shortcuts in technique.1
One study in regional anaesthesia used a 61-item lumbar
epidural checklist using consensus opinions from experts
(Sivarajan and colleagues38). They defined eight ‘critical
items’, such as accidental dural puncture that was deemed an
automatic unsatisfactory performance for the entire checklist.
For initial piloting, assessors rated eight videotaped lumbar
epidural insertions. Moderate to high reliability was confirmed
with Cohen’s k, but neither construct validity nor feasibility
was evaluated.
GRSs and derivatives
GRSs are differentiated from checklists by the use of Likert
scales for scoring and inclusion of items assessing non-
technical aspects, such as communication, teamwork, and
efficiency of motion. The theoretical disadvantages of GRSs
include the halo effect (the general impression of the trainee
influencing scoring of individual items) and scale limitation
(assessors marking in a small range of Likert scores instead of

using the entire scale).1 This has led to a belief that dichoto-
mous checklists are objective, whilst the subjectivity inherent
in GRSs is seen as a shortcoming. However, in multiple clinical
studies, and repeated in a recent meta-analysis, both GRSs and
checklists appear to be at least equally reliable.70e73
The original GRS was a seven-item tool scored on a 5-point
Likert scale, introduced in 1997 for objective structured
assessment of technical skills in surgery trainees.37 The direct
observation of procedural skills (DOPS) tool was later intro-
duced in 2002, also based on a Likert scoring scale.6 A type of
DOPS used in the Australian and New Zealand College of
Anaesthetists for regional anaesthesia workplace-based
assessment was evaluated for its psychometric properties for
a mixture of ultrasound-guided peripheral plexus blocks
(Watson and colleagues52). Whilst exhibiting construct validity
and feasibility, the external reliability of the DOPS using ICC
(2A, 1) models was 0.32. The critical impact of an appropriate
ICC model choice was highlighted; if two assessors are nor-
mally used to score trainees, then ICC (2A, k) modelling will
instead provide a markedly improved reliability coefficient of
0.88 (Chuan and colleagues56).
Combined checklists and GRSs
Combining a checklist with a GRS has been suggested as the
most appropriate tool to assess medical procedures.1 A com-
bined tool can assess clinical technical skills and professional
attributes. Chronologically, the first combined tools were
developed for single regional anaesthesia procedures:
landmark-guided obstetric epidural analgesia (Friedman and
colleagues42) and spinal anaesthesia (Breen and colleagues45),
landmark-based neurostimulation-guided interscalene
brachial plexus blocks (Naik and colleagues48), and ultrasound-
guided axillary brachial plexus blocks (Sultan and colleagues50).
The last study is notable in the use of cognitive task analysis, a
format of semi-structured interviews and cross referencing
against observations by experts, to break down complex pro-
cedures into smaller-component tasks.74 This helps avoid un-
intentional omission of critical tasks, which can be as high as
70%, when using informal expert panels.
More recent studies have developed tools to encompass a
range of regional anaesthesia procedures. Cheung and col-
leagues51 used a Delphi technique to construct a combined 22-
item checklist and a nine-item GRS suitable for peripheral
UGRA blocks, but did not undertake any psychometric anal-
ysis. This analysis was subsequently performed in 2014 in two
separate studies with different results for reliability. Whilst
both studies found evidence for construct validity, Wong and
colleagues53 found that the inter-rater reliability was overall
poor (ICC 0.44, single-measure ICC model 2A, 1) despite
piloting, modifications to the descriptors, and assessor
training. In contrast, Burckett-St. Laurent and colleagues54
found moderate reliability for both checklist and GRS in the
clinical setting, but less in the simulation environment (ICC
0.6e0.8, average-measure ICC model not specified). The dif-
ference in reliabilities reported by these two studies is large
and likely because of the use of different ICC models.
Finally, our group designed a tool for use in any regional
anaesthesia procedure (Chuan and colleagues55). Validation
and reliability testing were conducted using videos of neu-
raxial and peripheral nerve blocks, performed by landmark-,
neurostimulation-, and ultrasound-guided needle guidance.
The external reliability was determined by two independent
groups of experts in a testeretest pilot phase (Cohen’s k 0.70)
Assessment tools for regional anaesthesia - 269
and in a clinical validation phase using ICC (2A, k) with an
overall reliability of 0.80.
Cumulative sum analysis
CUSUM analysis is a type of sequential analysis first used to
measure quality assurance in industrial manufacturing. Used
in medicine, a trainee’s performance can be plotted over
successive attempts as trending towards achieving, or moving
away from, an agreed competency criterion.75 CUSUM is
different from the psychometric framework and is not
amenable to the statistical analyses described in this review.
Instead, we summarized the major features of CUSUM and the
insights this tool brings to the assessment of regional
anaesthesia performance.
In the medical literature, the sequential probability-ratio
CUSUM is the most common form. This requires an a priori
decision of the acceptable a error (risk of mislabelling a trainee
as incompetent) and b error (risk of mislabelling an incompe-
tent trainee as competent), and acceptable (p0) and unaccept-
able (p1) failure rates. This generates statistical thresholds for
either attaining or not attaining competency over time,
respectively, denoted as the h0 lower and h1 upper decision
thresholds. The determination of the success or failure for
each attempt is a binary outcome and should use an objective
end point. Performance that is trending towards competency
is graphically represented as a decreasing CUSUM plot
crossing the h0 lower threshold, whilst worsening perfor-
mance causes the plot to increase towards the h1 upper
threshold.
Disadvantages for using CUSUM in medical education have
been raised, including logistical complexity, as multiple at-
tempts per trainee are required to construct the CUSUM plot;
differences in technical difficulty cannot be modelled; and
previous performance is disproportionally weighed, which
may not be reflective of current levels of competency.75 More
recently, Sivaprakasam and Purva44 demonstrated that
acceptable and unacceptable labour epidural failure rates are
sensitive to small changes in a priori variables.
CUSUM articles have described the learning curve for a
range of regional anaesthesia tasks. These have included part-
tasks (ultrasound-guided needle-tip visualisation30,35 and
neuraxial sono-anatomy34) and clinical procedures (lumbar
neuraxial,31,39,41,44,46 caudal blocks,32 and penile blocks47). The
studies show a wide inter-personal variability in learning adult
blocks, but less for paediatric blocks.
Learning curves generated from CUSUM studies have had
practical implications. Novices benefit greatly from deliberate
practice intervention35 vs a discovery-learning protocol30
when learning a needle-guidance task with ultrasound. Mar-
garido and colleagues34 used CUSUM to trial the learning curve
of a deliberate practice intervention for novices to learn neu-
raxial sono-anatomy, resulting in changes to their curriculum.
Drake and colleagues46 used CUSUM to analyse retrospectively
their labour epidural complications, which led them to modify
their informed-consent material.
Discussion
The aims and nature of the assessment in anaesthesia have
fundamentally changed. Internationally, anaesthesia cre-
dentialing bodies have adopted competency-based curricula
based on the principles of the adult lifelong learner.76 In this
context, assessments have assumed importance as both
 270 - Chuan et al.
summative (assessment of learning and high-stakes tests) and
formative (assessment for learning, feedback, and reflection)
tools, anchored on predefined criteria and purposefully
designed.8,77,78 The 2010 Ottawa documents provide a frame-
work to perform psychometric evaluation of these assessment
tools.
The significance of assessing the assessment tools should
not be underestimated. For summative assessments, evidence
of fairness and test score reproducibility is a minimum
requirement for stakeholders. More recent tools have re-used
or modified older checklists and GRSs, and thus, the original
tools should be critically evaluated for validity and reliability.
Lastly, research in regional anaesthesia education would be
greatly advanced if we used accepted, common, and reliable
tools. This would allow pooling of results in meta-analyses,
comparisons between institutions, comparisons over time,
and comparing the relative effectiveness of different educa-
tional interventions.
The literature search found that several types of assess-
ment tools have been used to evaluate regional anaesthesia
performance for different purposes, from part-tasks to
workplace-based clinical assessment. There was recognition
in all articles to conform to the criteria of ideal tools, namely,
validity (in particular face, content, and construct validity) and
reliability. A smaller number of articles evaluated feasibility,
but no article attempted to evaluate educational benefit or
stakeholder acceptability.
The included studies were heterogeneous in methodology,
especially in statistical analysis, which makes interpretation
difficult. A recurring problem identified in this review con-
cerned reliability testing. In our experience, the modelling
assumptions that greatly inflate the ICC coefficient are average
measures and consistency agreement. Unless the assessment
tool was purposefully designed to be scored by multiple as-
sessors simultaneously, single-measure models should be
used. With Likert scales, absolute-agreement models should
be used. Researchers should state the ICC model used in their
methodology, as incorrect modelling will provide a false
assurance of reliability. Similarly, the limitations of Pearson
correlation and Cronbach’s a must be recognized as only
describing reliability in ideal situations, and the use of an
appropriate ICC model is instead encouraged.
Learning a complex procedure, such as UGRA, by decon-
struction into component part-tasks comes from the surgical
literature.79 This systematic approach of pre-training im-
proves learning by concentrating on known quality compro-
mising behaviours seen in UGRA novices.80,81 Following this,
current part-task assessment tools have focused on, first,
competency in theoretical sono-anatomy; second, defining
learning in ultrasound image acquisition; and, third, learning
of real-time needle guidance in low- and high-fidelity phan-
toms. Further MCQs, constructed using the item-response
theory as exemplified by Woodworth and colleagues,27
should be created to address important gaps, such as neu-
raxial sono-anatomy. A relative deficiency is lack of a vali-
dated checklist or GRS tool to assess competency in practical
UGRA skills using part-trainer models.
In contrast to part-tasks, workplace-based assessment
tools aim to evaluate the trainees over their entire regional
anaesthesia clinical performance. Nearly all types of tools
have been used in this context. Motion-analysis tools have
demonstrated the ability to discriminate between novices and
experts in both neuraxial and peripheral regional anaes-
thesia.43,49 However, practical educational interventions
remain unknown, as motion analysis describes what experts
already can do, but not how to progress novices to competent
performance. CUSUM analysis has shown high variability in
learning regional anaesthesia. More recent studies have
explored how to integrate CUSUM techniques into prospective
quality-assurance projects for both novices and experienced
proceduralists,44 and have a potential to provide formative
feedback.
Combined checklists and GRSs have been used for neu-
raxial and peripheral nerve block assessment. An attractive
aspect of the combined tools is the concurrent assessment of
technical and non-technical skills and professional attributes.
Regional anaesthesia experts are characterized not just by
higher technical proficiency, but also more developed cogni-
tive (decision making, situational awareness, resource man-
agement, and anticipation of problems) and interpersonal
skills (communication, teamwork, leadership, and self-
awareness of limitations).82,83
The GRSs used in these combined tools are similar in
design. Encouragingly, across different block types and study
environments, validity and reliability appear to be consistent.
In contrast, appraisal of the checklist component shows that
further research is required. The most recent checklists were
designed to be applicable across multiple block types (Cheung
and colleagues,51 for ultrasound-guided peripheral blocks;
Chuan and colleagues,55 for all peripheral and neuraxial
blocks). In the former tool, the two validation studies showed
marked difference in reliability, whilst in the latter tool a
confirmatory validation study is yet to be published. The
attractiveness of either tool is broader applicability than tools
designed for a particular block type. If consistent validity and
reliability can be demonstrated, these tools could offer a
standardized single-assessment format across multiple
institutions.
There are several avenues for future research using
assessment tools. Individuals learn differently, which con-
tributes to the variability of the UGRA learning curve. Assess-
ments could be repurposed as primary screening tools to
identify novices who are predicted to struggle with UGRA
skills. As an example, our current studies suggest that visuo-
spatial, rather than psychomotor, ability has a better predic-
tive value for future UGRA task performance.28,29,36,40
Educational interventions may then be more productively
directed to these trainees, which is essential in our resource-
and time-poor teaching environment.
Finally, non-technical attributes in regional anaesthesia lag
behind that of our understanding of motor skills develop-
ment.84 Technical performance of nerve blocks is but one
aspect of delivery of anaesthesia care to a patient. As suc-
cinctly described by Neal,85 ‘focusing on technical success says
nothing regarding the resident’s overall competence as a
physician’. Further simulation and best methods to train these
important professional attributes are required.
The limitations of this review include potential bias from
the process of article selection and from the quality of the
included articles. The initial screening of citations was per-
formed independently by blinded authors and the selected
articles were then systematically evaluated. Whilst the pro-
cess was methodical, relevant articles may have been inad-
vertently omitted. More seriously, bias in medical education
articles is not as readily estimated as in clinical trials, because
of quite different study designs. Subsequently, a risk of bias
and quality of evidence evaluation was not performed; we
instead chose to evaluate the statistical rigour and

quantitative results of the included studies against the psy-
chometric framework.
In conclusion, competency-based assessment tools are
used in all aspects of assessing regional anaesthesia perfor-
mance, and provide information on the competency, safety,
and educational progress of practitioners. The psychometric
evaluation of these tools provides credibility of results ob-
tained by the tool. Following published frameworks on
methodology and statistical analysis will improve the report-
ing and interpretation of studies of regional anaesthesia
assessment tools.
Authors’ contributions
Study design derived from a section of a doctoral thesis: A.C.
Supervision of doctoral thesis: C.F.R., K.F.
Electronic search, study selection, and analysis: A.C., A.S.W.
Critical review of manuscript drafts and approval of the final
version: all authors.
Declaration of interest
None declared.
Funding
Australian National Health and Medical Research Council
Postgraduate Fellowship (APP1056280) to A.C.; Australian
Society of Anaesthetists PhD Support Grant (2013) to A.C.
Appendix A. Supplementary data
Supplementary data related to this article can be found at
https://doi.org/10.1016/j.bja.2017.09.007.
References
1. Bould M, Crabtree N, Naik V. Assessment of procedural
skills in anaesthesia. Br J Anaesth 2009; 103: 472e83
2. Epstein R. Assessment in medical education. N Engl J Med
2007; 356: 387e96
3. Glavin R, Maran N. Editorial I: development and use of
scoring systems for assessment of clinical competence. Br
J Anaesth 2002; 88: 329e30
4. Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB.
Observational tools for assessment of procedural skills: a
systematic review. Am J Surg 2011; 202: 469e80. e6
5. Kathirgamanathan A, Woods L. Educational tools in the
assessment of trainees in anaesthesia. Contin Educ Anaesth
Crit Care Pain 2011; 11: 138e42
6. Wragg A, Wade W, Fuller G, Cowan G, Mills P. Assessing
the performance of specialist registrars. Clin Med 2003; 3:
131e4
7. Allen B, Sandberg W. Assessment tools: searching for
purpose. Reg Anesth Pain Med 2015; 40: 299e300
8. Norcini J, Anderson B, Bollela V, et al. Criteria for good
assessment: consensus statement and recommendations
from the Ottawa 2010 conference. Med Teach 2011; 33:
206e14
9. Schuwirth L, Colliver J, Gruppen L, et al. Research in assess-
ment: consensus statement and recommendations from the
Ottawa 2010 conference. Med Teach 2011; 33: 224e33
Assessment tools for regional anaesthesia - 271
10. Van der Vleuten C. The assessment of professional
competence: developments, research and practical im-
plications. Adv Health Sci Educ Theory Pract 1996; 1: 41e67
11. Gallagher AG, Ritter EM, Satava RM. Fundamental princi-
ples of validation, and reliability: rigorous science for the
assessment of surgical education and training. Surg Endosc
2003; 17: 1525e9
12. Downing S. Validity: on meaningful interpretation of
assessment data. Med Educ 2003; 37: 830e7
13. Downing S. Reliability: on the reproducibility of assess-
ment data. Med Educ 2004; 38: 1006e12
14. Accreditation Council for Graduate Medical Education and
the American Board of Anesthesiology. The anesthesiology
milestone project. 2015. https://www.acgme.org/Portals/0/
PDFs/Milestones/AnesthesiologyMilestones.pdf.
[Accessed 6 May 2016]
15. Sites B, Chan V, Neal J, et al. The American Society of
Regional Anesthesia and Pain Medicine and the European
Society of Regional Anaesthesia and Pain Therapy Joint
Committee recommendations for education and training
in ultrasound-guided regional anesthesia. Reg Anesth Pain
Med 2010; 35: S74e80
16. Kogan J, Holmboe E, Hauer K. Tools for direct observation
and assessment of clinical skills of medical trainees: a
systematic review. JAMA 2009; 302: 1316e26
17. Bravo G, Potvin L. Estimating the reliability of continuous
measures with Cronbach’s alpha or the intraclass corre-
lation coefficient: toward the integration of two traditions.
J Clin Epidemiol 1991; 44: 381e90
18. Chuan A, Graham P, Forrest K, et al. Regional anaesthesia
assessment toolsda reply. Anaesthesia 2016; 71: 473e4
19. Cicchetti D. Guidelines, criteria, and rules of thumb for
evaluating normed and standardized assessment in-
struments in psychology. Psychol Assess 1994; 6: 284e90
20. Bartko J. On various intraclass correlation reliability co-
efficients. Psychol Bull 1976; 83: 762e5
21. McGraw K, Wong S. Forming inferences about some
intraclass correlation coefficients. Psychol Methods 1996; 1:
30e46
22. Shrout PE, Fleiss JL. Intraclass correlations: uses in
assessing rater reliability. Psychol Bull 1979; 86: 420e8
23. Briesch A, Swaminathan H, Welsh M, Chafouleas S. Gener-
alizability theory: a practical guide to study design, imple-
mentation, and interpretation. J Sch Psychol 2014; 52: 13e35
24. Fehr J, Boulet J, Waldrop W, Snider R, Brockel M, Murray D.
Simulation-based assessment of pediatric anesthesia
skills. Anesthesiology 2011; 115: 1308e15
25. Weller J, Jolly B, Misur M, et al. Mini-clinical evaluation
exercise in anaesthesia training. Br J Anaesth 2009; 102:
633e41
26. Crossley J, Davies H, Humphris G, Jolly B. Generalisability:
a key to unlock professional assessment. Med Educ 2002;
36: 972e8. erratum, Med Educ 2003; 37: 574
27. Woodworth G, Carney P, Cohen J, et al. Development and
validation of an assessment of regional anesthesia ultra-
sound interpretation skills. Reg Anesth Pain Med 2015; 40:
306e14
28. Smith H, Kopp S, Johnson R, Long T, Cerhan J, Hebl J.
Looking into learning: visuospatial and psychomotor
predictors of ultrasound-guided procedural performance.
Reg Anesth Pain Med 2012; 37: 441e7
29. Duce N, Gillett L, Descallar J, Tran M, Siu S, Chuan A. Vi-
suospatial ability and novice brachial plexus sonography
performance. Acta Anaesthesiol Scand 2016; 60: 1161e9
 272 - Chuan et al.
30. de Oliveira Filho G, Helayel P, da Conceicao D, Garzel I,
Pavei P, Ceccon M. Learning curves and mathematical
models for interventional ultrasound basic skills. Anesth
Analg 2008; 106: 568e73
31. de Oliveira Filho G. The construction of learning curves
for basic skills in anesthetic procedures: an application
for the cumulative sum method. Anesth Analg 2002; 95:
411e6
32. Schuepfer G, Konrad C, Schmeck J, Poortmans G,
Staffelbach B, Johr M. Generating a learning curve for
pediatric caudal epidural blocks: an empirical evaluation
of technical skills in novice and experienced anesthetists.
Reg Anesth Pain Med 2000; 25: 385e8
33. Friedman Z, Siddiqui N, Katznelson R, Devito I, Davies S.
Experience is not enough: repeated breaches in epidural
anesthesia aseptic technique by novice operators despite
improved skill. Anesthesiology 2008; 108: 914e20
34. Margarido CB, Arzola C, Balki M, Carvalho JC. Anesthesi-
ologists’ learning curves for ultrasound assessment of the
lumbar spine. Can J Anaesth 2010; 57: 120e6
35. Barrington M, Wong D, Slater B, Ivanusic J, Ovens M. Ul-
trasound-guided regional anesthesia: how much practice
do novices require before achieving competency in ul-
trasound needle visualization using a cadaver model. Reg
Anesth Pain Med 2012; 37: 334e9
36. Shafqat A, Ferguson E, Thanawala V, Bedforth N,
Hardman J, McCahon R. Visuospatial ability as a predictor
of novice performance in ultrasound-guided regional
anesthesia. Anesthesiology 2015; 123: 1188e97
37. Martin J, Regehr G, Reznick R, et al. Objective structured
assessment of technical skill (OSATS) for surgical resi-
dents. Br J Surg 1997; 84: 273e8
38. Sivarajan M, Lane P, Miller E, et al. Performance evalua-
tion: continuous lumbar epidural anesthesia skill test.
Anesth Analg 1981; 60: 543e7
39. Kestin I. A statistical approach to measuring the compe-
tence of anaesthetic trainees at practical procedures. Br J
Anaesth 1995; 75: 805e9
40. Dashfield A, Coghill J, Langton J. Correlating obstetric
epidural anaesthesia performance and psychomotor
aptitude. Anaesthesia 2000; 55: 744e9
41. Naik V, Devito I, Halpern S. Cusum analysis is a useful tool
to assess resident proficiency at insertion of labour epi-
durals. Can J Anaesth 2003; 50: 694e8
42. Friedman Z, Katznelson R, Devito I, Siddiqui M, Chan V.
Objective assessment of manual skills and proficiency in
performing epidural anesthesiadvideo-assisted valida-
tion. Reg Anesth Pain Med 2006; 31: 304e10
43. Hayter M, Friedman Z, Bould M, et al. Validation of the
imperial college surgical assessment device (ICSAD) for
labour epidural placement. Can J Anaesth 2009; 56: 419e26
44. Sivaprakasam J, Purva M. CUSUM analysis to assess
competence: what failure rate is acceptable? Clin Teach
2010; 7: 257e61
45. Breen D, Bogar L, Heigl P, Rittberger J, Shorten G. Valida-
tion of a clinical assessment tool for spinal anaesthesia.
Acta Anaesthesiol Scand 2011; 55: 653e7
46. Drake E, Coghill J, Sneyd J. Defining competence in ob-
stetric epidural anaesthesia for inexperienced trainees. Br
J Anaesth 2015; 114: 951e7
47. Schuepfer G, Jo € hr M. Generating a learning curve for
penile block in neonates, infants and children: an empir-
ical evaluation of technical skills in novice and experi-
enced anaesthetists. Paediatr Anesth 2004; 14: 574e8
48. Naik V, Perlas A, Chandra D, Chung D, Chan V. An
assessment tool for brachial plexus regional anesthesia
performance: establishing construct validity and reli-
ability. Reg Anesth Pain Med 2007; 32: 41e5
49. Chin K, Tse C, Chan V, Tan J, Lupu C, Hayter M. Hand-
motion analysis using the imperial college surgical
assessment device: validation of a novel and objective
performance measure in ultrasound-guided peripheral
nerve blockade. Reg Anesth Pain Med 2011; 36: 213e9
50. Sultan S, Iohom G, Saunders J, Shorten G. A clinical
assessment tool for ultrasound-guided axillary brachial
plexus block. Acta Anaesthesiol Scand 2012; 56: 616e23
51. Cheung J, Chen E, Darani R, McCartney C, Dubrowski A,
Awad I. The creation of an objective assessment tool for
ultrasound-guided regional anesthesia using the Delphi
method. Reg Anesth Pain Med 2012; 37: 329e33
52. Watson M, Wong D, Kluger R, et al. Psychometric evalu-
ation of a direct observation of procedural skills assess-
ment tool for ultrasound-guided regional anaesthesia.
Anaesthesia 2014; 69: 604e12
53. Wong D, Watson M, Kluger R, et al. Evaluation of a task-
specific checklist and global rating scale for ultrasound-
guided regional anesthesia. Reg Anesth Pain Med 2014; 39:
399e408
54. Burckett-St Laurent D, Niazi A, Cunningham M, et al.
A valid and reliable assessment tool for remote
simulation-based ultrasound-guided regional anesthesia.
Reg Anesth Pain Med 2014; 39: 496e501
55. Chuan A, Graham P, Wong D, et al. Design and validation
of the regional anaesthesia procedural skills assessment
tool. Anaesthesia 2015; 70: 1401e11
56. Chuan A, Thillainathan S, Graham P, et al. Reliability of
the direct observation of procedural skills assessment tool
for ultrasound-guided regional anaesthesia. Anaesth
Intensive Care 2016; 44: 201e8
57. van Hove P, Tuijthof G, Verdaasdonk E, Stassen L,
Dankelman J. Objective assessment of technical surgical
skills. Br J Surg 2010; 97: 972e87
58. McGrew K. CHC theory and the human cognitive abilities
project: standing on the shoulders of the giants of psy-
chometric intelligence research. Intelligence 2009; 37: 1e10
59. Schneider W, McGrew K. The CattelleHorneCarroll model
of intelligence. In: Flanagan P, Harrison P, editors.
Contemporary intellectual assessment: theories, tests, and is-
sues. New York: Guildford Press; 2012. p. 99e144
60. Ackerman P. Determinants of individual differences dur-
ing skill acquisition: cognitive abilities and information
processing. J Exp Psychol Gen 1988; 117: 288e318
61. Enochsson L, Westman B, Ritter E, et al. Objective
assessment of visuospatial and psychomotor ability and
flow of residents and senior endoscopists in simulated
gastroscopy. Surg Endosc 2006; 20: 895e9
62. Luursema J-M, Buzink S, Verwey W, Jakimowicz JJ. Visuo-
spatial ability in colonoscopy simulator training. Adv
Health Sci Educ Theory Pract 2010; 15: 685e94
63. Brandt M, Davies E. Visualespatial ability, learning mo-
dality and surgical knot tying. Can J Surg 2006; 49: 412e6
64. Gallagher A, Cowie R, Crothers I, Jordan-Black J, Satava R.
PicSOr: an objective test of perceptual skill that predicts
laparoscopic technical skill in three initial studies of
laparoscopic performance. Surg Endosc 2003; 17: 1468e71
65. Hedman L, Strom P, Andersson P, Kjellin A, Wredmark T,
Fellander-Tsai L. High-level visualespatial ability for
novices correlates with performance in a visualespatial

complex surgical simulator task. Surg Endosc 2006; 20:
1275e80
66. Van Herzeele I, O’Donoghue K, Aggarwal R, Vermassen F,
Darzi A, Cheshire N. Visuospatial and psychomotor apti-
tude predicts endovascular performance of inexperienced
individuals on a virtual reality simulator. J Vasc Surg 2010;
51: 1035e42
67. Sweeney K, Hayes J, Chiavaroli N. Does spatial ability help
the learning of anatomy in a biomedical science course?
Anat Sci Educ 2014; 7: 289e94
68. Gettman M, Kondraske G, Traxer O, et al. Assessment of
basic human performance resources predicts operative
performance of laparoscopic surgery. J Am Coll Surg 2003;
197: 489e96
69. Dashfield A, Smith J. Correlating fibreoptic nasotracheal
endoscopy performance and psychomotor aptitude. Br J
Anaesth 1998; 81: 687e91
70. Morgan P, Cleave-Hogg D, Guest C. A comparison of global
ratings and checklist scores from an undergraduate
assessment using an anesthesia simulator. Acad Med 2001;
76: 1053e5
71. Cunnington J, Neville A, Norman G. The risks of thor-
oughness: reliability and validity of global ratings and
checklists in an OSCE. Adv Health Sci Educ Theory Pract
1996; 1: 227e33
72. Regehr G, MacRae H, Reznick R, Szalay D. Comparing the
psychometric properties of checklists and global rating
scales for assessing performance on an OSCE-format ex-
amination. Acad Med 1998; 73: 993e7
73. Ilgen J, Ma I, Hatala R, Cook D. A systematic review of val-
idity evidence for checklists versus global rating scales in
simulation-based assessment. Med Educ 2015; 49: 161e73
74. Clark R, Pugh C, Yates K, Inaba K, Green D, Sullivan M. The
use of cognitive task analysis to improve instructional
descriptions of procedures. J Surg Res 2012; 173: e37e42
Assessment tools for regional anaesthesia - 273
75. Norris A, McCahon R. Cumulative sum (CUSUM) assess-
ment and medical education: a square peg in a round
hole. Anaesthesia 2011; 66: 250e4
76. Ebert T, Fox C. Competency-based education in anesthe-
siology: history and challenges. Anesthesiology 2014; 120:
24e31
77. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB.
Systematic review of the literature on assessment, feed-
back and physicians’ clinical performance: BEME guide
no. 7. Med Teach 2006; 28: 117e28
78. Wood T. Assessment not only drives learning, it may also
help learning. Med Educ 2009; 43: 5e6
79. Aggarwal R, Grantcharov T, Darzi A. Framework for sys-
tematic training and assessment of technical skills. J Am
Coll Surg 2007; 204: 697e705
80. Slater R, Castanelli D, Barrington M. Learning and teach-
ing motor skills in regional anesthesia: a different
perspective. Reg Anesth Pain Med 2014; 39: 230e9
81. Gallagher A, Ritter E, Champion H, et al. Virtual reality
simulation for the operating room: proficiency-based
training as a paradigm shift in surgical skills training.
Ann Surg 2005; 241: 364e72
82. Fletcher G, McGeorge P, Flin R, Glavin R, Maran N. The role
of non-technical skills in anaesthesia: a review of current
literature. Br J Anaesth 2002; 88: 418e29
83. Smith A, Pope C, Goodwin D, Mort M. What defines
expertise in regional anaesthesia? An observational
analysis of practice. Br J Anaesth 2006; 97: 401e7
84. Nix C, Margarido C, Awad I, et al. A scoping review of the
evidence for teaching ultrasound-guided regional anes-
thesia. Reg Anesth Pain Med 2013; 38: 471e80
85. Neal J. Education in regional anesthesia: caseloads,
simulation, journals, and politics: 2011 Carl Koller lecture.
Reg Anesth Pain Med 2012; 37: 647e51
Handling editor: J.G. Hardman