You are on page 1of 7

Learning Endotracheal Intubation in a Clinical Skills

Learning Center: A Quantitative Study
John L. Plummer, PhD, AStat, and Harry Owen, MD, FRCA, FANZCA
Department of Anaesthesia, Flinders University of South Australia and Flinders Medical Centre, Bedford Park, Australia
This study aimed to develop statistical models describ-
ing the learning of endotracheal intubation (ETI). We
collected data from 100 subjects undergoing ETI train-
ingwithintubatable medical models andmanikins (air-
way trainers). Trainees initially viewed a video about
ETI andaninstructor demonstratedthe technique. Sub-
jects thenmade upto17 supervisedtrials. Eachtrial was
scoredas a success or failure; this score was the primary
outcome used in analyses. Random effects and
population-averaged logit models, and a learning
model intendedtoquantifythe relative contributions of
failedandsuccessful trials to the learning process, were
fittedtothe data. The logit models providedevidence of
differences in difficulty between different airway train-
ers and differences in success rate related to previous
ETI experience. Trainees became familiar with an air-
way trainer after multiple trials, as demonstrated by a
50% decrease in the odds of successful ETI when start-
ing on a newtrainer. The learning model indicated that
a trainee learns about as much from 1 successful ETI as
from12 (95%confidence interval, 2–23) failedtrials. The
results demonstrate the feasibility of statistical model-
ing of the learning of ETI and provide insight into the
learning process.
(Anesth Analg 2001;93:656–62)
I
n our institution, endotracheal intubation (ETI) is
taught to a range of health professionals, including
medical students and trainees in anesthesia. Initial
training occurs in a Clinical Skills Learning Center
equipped with a variety of manikins and medical
models (subsequently referred to as airway trainers).
Feedback from trainees has been very positive, but
teaching methods should be continually evaluated
and refined. To do this effectively, we must be able to
relate components of the teaching process to the out-
come, at least qualitatively and preferably quantita-
tively. This study was based on the hypothesis that the
process of learning to intubate airway trainers may be
subjected to quantitative analysis, and that such anal-
ysis would provide insight into how instruction could
be further improved.
Methods
This project was approved by the Flinders Clinical
Research Ethics Committee. All subjects provided
written, informed consent. Before beginning the
project, the three instructors involved discussed meth-
ods and criteria for assessment and jointly assessed a
number of intubation attempts to achieve uniform
scoring. Six airway trainers were used (Table 1). Air-
way trainer D was used both with and without a
cervical collar; these configurations were considered
to be different trainers, making a total of seven. Train-
ees attended the training laboratory in groups of two
to four. Each trainee provided demographic details,
history of previous experience in ETI, details of expe-
rience in 14 clinical procedures ranging from head tilt,
oral airway, and peripheral venous cannulation to
cardiac defibrillation, and level of confidence in their
ability to perform these procedures. For each proce-
dure, a score of 1–4 was assigned, according to
whether the trainee had performed the procedure on
patients 0, 1–5, 6–20, or more than 20 times, respec-
tively. The scores were summed to provide a global
clinical experience score. The level of confidence in
ability to perform these procedures was rated on a
scale from 1 (not confident at all) to 5 (extremely
confident). These scores were summed to give a clin-
ical skills confidence score.
The training sessions lasted 1.5 to 2 h. Initially the
aims of the teaching were discussed, and trainees
viewed a brief video presentation about ETI. The tech-
nique was then demonstrated by an instructor, after
which each trainee made up to 17 trials at ETI with
Supported, in part, by grants from the Laerdal Foundation for
Acute Medicine and the Australian and New Zealand College of
Anaesthetists.
Accepted for publication May 1, 2001.
Address correspondence and reprint requests to Dr. J. L. Plum-
mer, Department of Anaesthesia, Flinders Medical Centre, Bedford
Park, SA 5042, Australia. Address e-mail to john.plummer
@flinders.edu.au.
©2001 by the International Anesthesia Research Society
656 Anesth Analg 2001;93:656–62 0003-2999/01
three to six different airway trainers. Instructors 1–3
supervised 89, 7, and 4 trainees, respectively. There
was no fixed sequence of airway trainer use, but gen-
erally trainees commenced with trainers perceived to
be relatively easy (Trainers A and B) and moved to-
ward more difficult ones (e.g., Trainer DC). The in-
structor rated each trial as a “success” or “failure.” To
be scored as a success, the trial had to be considered to
be a satisfactory intubation, defined as an intubation
which, if it had been made on a patient, would have
achieved ETI without significant risk of adverse
events. Factors considered in scoring included han-
dling of the laryngoscope and endotracheal tube, care
taken to avoid trauma to oral structures (force used,
leverage on upper incisors, etc), and the time taken to
complete the ETI.
To identify factors predictive of successful intuba-
tion, three types of statistical models appropriate for a
repeated-measures design with a binary outcome
(here, success versus failure of intubation attempt)
were fitted to the data. These models were population-
averaged logistic regression models, random effects
logistic regression models, and a learning model (1,2)
intended to quantitate the relative contributions of
successful versus failed attempts to the learning pro-
cess. Parameter estimates from the logistic regression
models are presented as odds ratios and 95% confi-
dence intervals (95% CI) in relation to an arbitrarily
chosen reference category. Goodness of fit of different
random effects and learning models was compared by
Akaike’s Information Criterion (AIC) (3). A smaller
AIC corresponds to improved model fit. Further de-
tails of the statistical models are given in Appendix 1.
Initially models were fitted that included variables
believed a priori to affect success of ETI (occupational
group, previous ETI experience, type of airway trainer
attempted, and instructor). Other potential predictors
were then examined by adding them to the model.
Results
One hundred subjects participated in the training ses-
sions. Demographic data are given in Table 2. The
median global clinical experience score was 16 (range,
14–45), and the median clinical skills confidence score
was 24 (range, 14–55).
The rate of successful ETI increased from 6% on the
first trial to approximately 80% after 15 trials. To de-
termine an appropriate approximation to the shape of
the learning curve, population-averaged logistic mod-
els were fitted that included trial number on a linear,
quadratic, or logarithmic scale and covariates adjust-
ing for airway trainer type, previous ETI experience,
occupational category, and instructor. The predicted
success rates from the linear and logarithmic models
are shown together with the observed success rates in
Figure 1. The success rates predicted by the logarith-
mic model were substantially closer to the observed
success rate than were the predictions of the linear
model. The success rates predicted from the quadratic
model were close to observed success rates but still
inferior to those predicted from the logarithmic
model. Although the logarithmic model did not pre-
dict the success rate on the 16th or 17th trial well, only
five subjects and one subject contributed to these
points, respectively. When corresponding random ef-
fects models were fitted, AICs for models with trial
number on a linear, quadratic, or logarithmic scale
were 1171, 1144, and 1140, respectively. Consequently,
logarithm of trial number was used as a covariate in
the logit models.
The basic population-averaged logistic regression
model, which included logarithm of trial number, air-
way trainer, previous experience in ETI, occupation,
and instructor as covariates, gave a reasonably good
fit to the data. There was evidence of a significant
association between each category of predictor and
successful intubation (all P Ͻ 0.05). The only addi-
tional variable identified as an important predictor
was when a subject changed to a new airway trainer.
Results for the model including this variable, Model 1,
are given in Table 3.
Model 1 provided only weak evidence of heteroge-
neity in difficulty level among airway trainers. The
most important component of previous ETI experi-
ence was for trainees who had prior experience intu-
bating patients. There was a significant association
between occupational group and success rate, but no
strong evidence of heterogeneity among medical stu-
dents in Years 1–4 (P ϭ 0.07). This probably reflects
the small numbers of students from Year 1, 3, and 4 in
Table 1. Airway Trainers Used in Training Sessions
Airway
trainer Name/manufacturer
A Intubation Head
Medical Plastics Laboratory, Inc, South
Gatesville, TX
B Laerdal Airway Management Trainer, 25 00 00
Laerdal Medical Corporation, Wappingers Falls,
NY
C Laerdal Adult Intubation Model
Laerdal Medical Corporation
D Laerdal ALS Skillmaster Interactive Manikin
Laerdal Medical Corporation
DC Same as trainer D but with Stiffneck௡ Adjustable
Collar
Laerdal Medical Corporation
E Trauma Intubation Head
Medical Plastics Laboratory, Inc
F Airway Larry airway trainer, LF03699U
Lifeform products, NASCO, Fort Atkinson, WI
ANESTH ANALG TECHNOLOGY, COMPUTING, AND SIMULATION PLUMMER AND OWEN 657
2001;93:656–62 LEARNING ENDOTRACHEAL INTUBATION
the sample. Despite attempts to standardize the as-
sessment procedure, the results indicated that there
was heterogeneity in scoring among different instruc-
tors. When a trainee changed to a new airway trainer,
the odds of successful ETI were halved, suggesting
that trainees adapt their intubation technique to suit
the particular airway trainer being used.
The global clinical experience score and clinical
skills confidence score were not important predictors
when added to Model 1 (odds ratios 1.04 and 1.01 per
unit, P ϭ0.2 and 0.6, respectively). Because the level of
clinical experience is associated with occupational
group, the global clinical experience score was
included in Model 1, but variables denoting occupa-
tional group were omitted. In this context, the global
clinical experience score was an important predictor of
successful intubation (odds ratio 1.07 per unit, 95% CI
1.03–1.11). However, this model did not fit the data as
well as Model 1, as judged by the Hosmer-Lemeshow
test.
A random effects model, Model 2, was fitted by
using the same predictor variables as in Model 1.
Estimates from Model 2 are given in Table 4. Qualita-
tively the parameter estimates were similar to those of
Model 1. There was significant evidence of heteroge-
neity in difficulty among airway trainers, with trainer
D the most difficult.
The learning model described by Bush and Mos-
teller (1) is of the form
P
failure
ϭ ␤
1
s
ϫ ␤
2
f
, (1)
where P
failure
is the predicted probability of failure on the
ith trial for the jth subject, s is the number of successes
and f the number of failures before the ith trial by the jth
subject (subscripts i and j have been omitted from equa-
tion for clarity), and ␤
1
and ␤
2
are variables to be esti-
mated. For this study, this model was modified to allow
the inclusion of airway trainer, occupational group, and
so forth as additional predictors (Appendix 1). Because
of the difficulty in fitting multivariable learning models,
the following variables each were collapsed into two
categories: previous ETI experience (experience on pa-
tients versus no experience on patients), occupational
category (medical student or nurse versus other), and
instructor (Instructor 1 versus Instructor 2 or 3). The
model containing all these predictors, as well as type of
airway trainer, had an AIC of 1167. The best-fitting
model was obtained by removing the variables previous
ETI experience and instructor and adding a variable
denoting whether the trainee had previously attempted
to intubate the same trainer. This final model had an AIC
of 1149. Parameter estimates for this model, Model 3, are
given in Table 5. The predicted probabilities of successful
ETI for Model 3 range from 0 to 0.97, with a predicted
success rate on the first trial of 5% (observed ϭ 6%).
The ratio log(␤
1
)/log(␤
2
) gives the relative contribu-
tion of a successful trial versus a failed trial to the
learning process. Here, this ratio is approximately 12
(95% CI, 2–23), suggesting that a trainee learns about
as much from one successful trial as from 12 failed
trials.
Discussion
The learning of clinical skills has been studied in a
variety of ways. Descriptive studies (4,5) may be ap-
propriate for providing feedback to trainees about
their progress or for summarizing large amounts of
Figure 1. Observed and predicted success rates for endotracheal
intubation. F Observed success rate. Open symbols indicate success
rate predicted by models adjusting for airway trainer type, previous
intubation experience, occupational group, instructor, and trial
number on a linear () or logarithmic (⅙) scale. For Trial 1, the data
point for the logarithmic model overlaps that for the observed
success rate.
Table 2. Demographic Data
Variable
No.
Subjects
Occupation
Medical student (Year 1) 3
Medical student (Year 2) 63
Medical student (Year 3) 15
Medical student (Year 4) 4
Ambulance paramedic trainee 13
Nurse (Royal Flying Doctor Service) 1
Specialist (nonanesthesia) 1
Level of prior ETI experience
None 13
Theory only 24
Practice on airway trainers only 50
Experience intubating patients 13
Primary reason for training
Compulsory 37
To refresh previously acquired skills 1
Will probably be called upon to use ETI 33
Small chance of being called upon to use
ETI
1
General interest 28
ETI ϭ endotracheal intubation.
658 TECHNOLOGY, COMPUTING, AND SIMULATION PLUMMER AND OWEN ANESTH ANALG
LEARNING ENDOTRACHEAL INTUBATION 2001;93:656–62
data into a more comprehensible form. Comparative
studies have demonstrated the efficacy of teaching
methods, including the use of medical models or sim-
ulators (6,7). This study demonstrates that the process
of learning a complex clinical skill, ETI, can be mod-
eled statistically, enabling the identification of factors
affecting the learning process. This statistical model-
ing provides a tool that will allow anesthesiologists to
study the effect of new educational methods or
technologies.
In this study, the subject group was heterogeneous,
three different instructors were involved, and differ-
ent subjects made differing numbers of trials on dif-
fering sets of airway trainers, so the data were
“messy.” This was because the purpose of the training
sessions was educational, and their design and con-
duct was aimed to optimize learning, not to provide a
standardized setting for data collection. Results ob-
tained from this real-world situation are probably
more widely applicable than those obtained in highly
controlled experimental settings. However, interpre-
tation of the results should take the data collection
method into account. For example, the relative diffi-
culty of the different airway trainers may depend on
the sequence in which they are presented. In this
study, the sequence was not balanced or randomized,
but generally progressed from the easier to the more
difficult trainers.
Results obtained from population-averaged and
random effects models were qualitatively similar. The
interpretation of the estimated parameters, however,
differs between these types of models. Population-
averaged models are more appropriate for comparing
groups that receive different teaching methods,
whereas random effects models are more appropriate
for assessing an intervention made at an individual
level. In practice, teaching includes elements at both a
group and an individual level, and so both classes of
models might be considered.
Although the primary goal of this preliminary study
was to establish the feasibility and utility of modeling
the learning process, the results do provide insight
into the learning process. If the odds of successful
intubation are taken as a measure of performance,
then these results conform to the commonly observed
pattern of a linear relationship between logarithm of
performance score and logarithm of the number of
trials made (8). Although this type of learning curve
leads to rapid early learning, the rate of learning rap-
idly declines, essentially reaching a plateau. At this
Table 3. Results for Population-Averaged Model, Model 1 Dependent Variable: Successful Intubation
Predictor
a
Odds ratio 95% CI
Log(trial number) (P Ͻ 0.001) 6.8 4.3–11
Airway trainers (P ϭ 0.09)
Trainer A (reference category) 1
Trainer B 1.3 0.9–2.1
Trainer C 1.1 0.6–2.0
Trainer D 0.1 0.01–1.6
Trainer DC 0.6 0.2–1.6
Trainer E 1.9 0.8–4.1
Trainer F 0.9 0.5–1.6
Previous experience (P ϭ 0.005)
None (reference category) 1
Theory only 0.6 0.3–1.4
Practice on airway trainers only 1.7 0.8–4.0
Experience intubating patients 4.2 1.0–16
Occupation (P Ͻ 0.001)
Medical student Year 1 (reference category) 1
Medical student Year 2 1.4 0.5–4.2
Medical student Year 3 0.7 0.2–2.3
Medical student Year 4 0.3 0.03–1.9
Ambulance paramedic trainee 7.8 2.1–29
Nurse (RFDS) 0.5 0.1–2.3
Specialist 6.1 1.7–21
Instructor (P Ͻ 0.001)
Instructor 1 (reference category) 1
Instructor 2 0.1 0.02–0.3
Instructor 3 0.1 0.01–0.7
Changing to a new airway trainer (P Ͻ 0.001)
No change of trainer (reference category) 1
First trial after change to new trainer 0.5 0.4–0.7
CI ϭ confidence interval; RFDS ϭ Royal Flying Doctor Service.
a
P values given after each category of predictor are fromWald tests of the null hypothesis that none of the predictors in that category is associated with success
of intubation.
ANESTH ANALG TECHNOLOGY, COMPUTING, AND SIMULATION PLUMMER AND OWEN 659
2001;93:656–62 LEARNING ENDOTRACHEAL INTUBATION
Table 5. Results for Learning Model, Model 3 Dependent Variable: Failed Intubation
Predictor
a
Parameter
estimate
b
95% CI
Number of previous successes (P Ͻ 0.001) 0.72 0.68–0.76
Number of previous failures (P Ͻ 0.001) 0.97 0.95–1.00
Airway trainers (P ϭ 0.017)
Trainer A (reference category) 1
Trainer B 0.96 0.88–1.04
Trainer C 0.91 0.75–1.11
Trainer D 2.42 1.48–3.96
Trainer DC 1.75 1.10–2.80
Trainer E 0.77 0.44–1.35
Trainer F 0.92 0.72–1.17
Occupation (P Ͻ 0.001)
Medical student/nurse (reference category) 1
Ambulance paramedic trainee/specialist 0.74 0.62–0.88
New airway trainer (P Ͻ 0.001)
First trial on a new trainer (reference category) 1
Second or subsequent trial on a trainer 0.83 0.76–0.90
CI ϭ confidence interval.
a
P values given after each category of predictor are from likelihood ratio tests of the null hypothesis that none of the predictors in that category is associated
with success of intubation.
b
Because the modeled outcome is probability of a failed trial, the parameter estimates may be interpreted as the factor by which the probability of failure is
reduced. For example, other factors being equal, a trainee who is an ambulance paramedic trainee or specialist has only 74% as much chance of failing an
intubation trial as a medical student or nurse.
Table 4. Results for Random Effects Model, Model 2 Dependent Variable: Successful Intubation
Predictor
a
Odds ratio 95% CI
Log(trial number) (P Ͻ 0.001) 11.4 7.2–18.1
Airway trainers (P ϭ 0.002)
Trainer A (reference category) 1
Trainer B 1.3 0.8–2.0
Trainer C 1.0 0.6–1.9
Trainer D 0.1 0.01–0.4
Trainer DC 0.4 0.2–1.0
Trainer E 1.8 0.7–4.6
Trainer F 0.8 0.4–1.6
Previous experience (P ϭ 0.013)
None (reference category) 1
Theory only 0.7 0.2–1.9
Practice on airway trainers only 1.8 0.7–4.7
Experience intubating patients 5.1 1.2–21
Occupation (P ϭ 0.009)
Medical student Year 1 (reference category) 1
Medical student Year 2 1.5 0.3–8.6
Medical student Year 3 0.7 0.1–4.7
Medical student Year 4 0.2 0.02–2.7
Ambulance paramedic trainee 8.1 1.0–64
Nurse (RFDS) 0.5 0.01–23
Specialist 10.2 0.3–342
Instructor (P Ͻ 0.001)
Instructor 1 (reference category) 1
Instructor 2 0.1 0.02–0.5
Instructor 3 0.1 0.01–0.2
Changing to a new airway trainer (P Ͻ 0.001)
No change of trainer (reference category) 1
First trial after change to new trainer 0.4 0.3–0.6
CI ϭ confidence interval; RFDS ϭ Royal Flying Doctor Service.
a
P values given after each category of predictor are from likelihood ratio tests of the null hypothesis that none of the predictors in that category is associated
with success of intubation.
660 TECHNOLOGY, COMPUTING, AND SIMULATION PLUMMER AND OWEN ANESTH ANALG
LEARNING ENDOTRACHEAL INTUBATION 2001;93:656–62
stage, simply doing more of the same has little edu-
cational value. It may be that an appropriate interven-
tion, such as instruction, reflection, or a break from
training after approximately 12 trials, can be devel-
oped that would initiate a further acceleration in
learning.
Each time a trainee changed to a new airway trainer,
the odds of a successful intubation decreased by half,
suggesting that trainees became familiar with a trainer
on repeated use and tailored their technique to suit it.
Learning studies have shown that variation in the
training task slows acquisition but enhances both re-
tention and transfer to new settings (9,10). This leads
to the hypothesis that if trainees trained on only a
single airway trainer, they would become skilled at
intubating only that airway trainer but would have
more difficulty intubating a patient, and would lose
the skill more easily than a trainee who had trained on
a variety of trainers.
Another interesting result arose from the learning
model. It seems that trainees learn far more from
successful, as opposed to failed, trials at intubation.
This is consistent with the hypothesis that ETI is a
motor skill learned by the practice of the correct tech-
nique, such as with riding a bike. Initially it is difficult,
but by repetition of the correct motor skills it becomes
automatic. Repetition or performance of incorrect
skills is of little value. This supports the intuitive view
that training should commence with the easier airway
trainers and progress to more difficult ones only when
the trainee is able to approach them with a good
prospect of success.
This study focused on the process of learning to
intubate airway trainers. It has identified means by
which such learning might be improved, for exam-
ple, by using a variety of trainers and structuring
the session to maximize the proportion of successful
trials. It remains to be determined whether optimiz-
ing training with airway trainers improves a train-
ee’s ability to intubate a patient or whether a sub-
ject’s performance during training on airway
trainers may be used to predict ability to intubate a
patient. This has major implications for clinical sim-
ulation. Data are currently being collected to exam-
ine these questions.
The authors are grateful to Ms. Val Follows, CN, and Dr. Madhavi
Singh, who supervised a number of training sessions.
Appendix 1
An important feature of data consisting of multiple
measurements on each of a number of individuals is
the lack of independence of observations within sub-
jects. The three types of models examined in this study
represent three different approaches to dealing with
this dependence. Models were fitted by using the xt-
logit or glm (generalized linear models) procedures of
the software package Stata version 7 (Stata Corpora-
tion, College Station, TX).
Population-averaged, or marginal, models model
the mean response of groups of subjects with the same
covariate patterns. These models were fitted by using
the generalized estimating equations method. Param-
eters are estimated assuming that the observations are
independent. The correlations within subjects are es-
timated and used to obtain correct ses and CIs. A
specific structure must be assumed for the correlations
between responses (in this case, an exchangeable
structure was assumed), but by basing ses on the
information sandwich, or robust, estimator, CIs for
parameters will be correct even if the correlation struc-
ture was misspecified (11,12). An odds ratio from a
population-averaged model compares the odds of dif-
ferent groups of subjects having different values for
the covariate of interest. Population-averaged models
are therefore most appropriate for comparison of
groups in which treatments are fairly uniform within
a group but differ between groups. Adequacy of fit of
these models was assessed by comparison of pre-
dicted and observed success rates and by the Hosmer-
Lemeshow test (12). Population-averaged models
have been criticized on a number of grounds, in par-
ticular because the predicted response profiles may
not apply to any possible individual subject (13). For
further information about population averaged mod-
els, see Diggle et al. (14).
The random effects model may be viewed as a
model in which some variables influencing the out-
come have been omitted. The values taken by these
omitted variables must be constant within each subject
but may vary among subjects. For example, in this
study, motor coordination might be an important pre-
dictive variable, but it was not measured and was not
included in the models. The consequence of omitting
these variables is that some subjects perform better
than others for reasons not explicitly explained by the
model. The random effects model absorbs this heter-
ogeneity into a random subject factor; that is, the
unexplained variation among subjects is considered to
follow some probability distribution. In the models
fitted here, this distribution was assumed to be a
normal distribution. An odds ratio from a random
effects logit model applies to subjects having the same
value for the random effect, for example, the odds of a
specific subject observed under different levels of a
covariate (12). Random effects models are most appro-
priate when it is desired to assess the effect of an
intervention made at an individual level. Adequacy of
fit of these models was assessed by comparison of
predicted and observed success rates and by the
Hosmer-Lemeshow test (12). For further information
ANESTH ANALG TECHNOLOGY, COMPUTING, AND SIMULATION PLUMMER AND OWEN 661
2001;93:656–62 LEARNING ENDOTRACHEAL INTUBATION
about random effects models, see Brown and Prescott
(15).
The learning models used here were of the form
P
failure
ϭ ␤
1
s
ϫ ␤
2
f
ϫ ␤
3
x1
ϫ ␤
4
x2
..., (1)
where P
failure
is the predicted probability that the ith
attempt of the jth subject will be a failure, s is the
number of successful and f the number of failed at-
tempts by the jth subject before the ith attempt, and x1,
x2, and so on are predictors denoting airway trainer,
occupational group, and so forth. Within-subject cor-
relation arises from the dependence of an observation
on past values for the same subject. Goodness of fit of
learning models was assessed by comparison of pre-
dicted and observed success rates and by examination
of Pearson and deviance residuals.
References
1. Bush RR, Mosteller F. Stochastic models for learning. New York:
John Wiley & Sons, 1955.
2. Lindsey JK. Models for repeated measurements. 2nd ed. Oxford:
Oxford University Press, 1999.
3. Lindsey JK, Jones B. Choosing among generalized linear models
applied to medical data. Stat Med 1998;17:59–68.
4. Kestin IG. A statistical approach to measuring the competence
of anaesthetic trainees at practical procedures. Br J Anaesth
1996;75:805–9.
5. Konrad C, Schupfer G, Wietlisbach M, Gerber H. Learning
manual skills in anesthesiology: is there a recommended num-
ber of cases for anesthetic procedures? Anesth Analg 1998;86:
635–9.
6. Abrahamson S, Denson JS, Wolf RM. Effectiveness of a simula-
tor in training anesthesiology residents. J Med Educ 1969;44:
515–9.
7. Sajid A, Magero J, Feinzimer M. Learning effectiveness of the
heart sound simulator. Med Educ 1977;11:25–7.
8. Annett J. Skill acquisition. In: Morrison JE, ed. Training for
performance: principles of applied human learning. Chichester:
John Wiley & Sons, 1991:13–51.
9. Catalano J, Kleiner B. Distant transfer in coincident timing as a
function of variability of practice. Percept Mot Skills 1984;58:
851–6.
10. Schendel JD, Hagman JD. Long-term retention of motor skills.
In: Morrison JE, ed. Training for performance: principles of
applied human learning. Chichester: John Wiley & Sons, 1991:
53–92.
11. Huber PJ. The behavior of maximum likelihood estimates under
nonstandard conditions. In: le Cam LM, Neyman J, eds. Pro-
ceedings of the 5th Berkeley Symposium on Mathematical Sta-
tistics and Probability. Berkeley: University of California Press,
1967:221–33.
12. Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed.
New York: John Wiley & Sons, 2000.
13. Lindsey JK, Lambert P. On the appropriateness of marginal
models for repeated measurements in clinical trials. Stat Med
1998;17:447–69.
14. Diggle PJ, Liang K-Y, Zeger SL. Analysis of longitudinal data.
Oxford: Oxford University Press, 1994.
15. Brown H, Prescott R. Applied mixed models in medicine.
Chichester: John Wiley & Sons, 1999.
662 TECHNOLOGY, COMPUTING, AND SIMULATION PLUMMER AND OWEN ANESTH ANALG
LEARNING ENDOTRACHEAL INTUBATION 2001;93:656–62