REVIEW

CURRENT
OPINION Large databases in anaesthesiology
Richard P. Dutton

Purpose of review
The purpose of this study is to review the current state of large database research in anaesthesiology and
to describe the evolution of the National Anesthesia Clinical Outcomes Registry (NACOR) in the USA.
Recent findings
The Anesthesia Quality Institute of the American Society of Anesthesiologists was created to develop a
national anaesthesia registry for the USA. NACOR and the companion Anesthesia Incident Reporting
System are now 5 years old and in daily use by hundreds of US practices. The 30 million cases in
NACOR are an emerging source for ‘big data’ research in anaesthesiology.
Summary
The Information Age is bringing new capabilities for large database research to the specialty of
anaesthesiology, driven by the formation of registries capable of capturing a large fraction of all cases
performed.
Keywords
anaesthesiology, database research, National Anesthesia Clinical Outcomes Registry, quality
improvement, registry

INTRODUCTION potential for learning from this information is vast,

The Information Age has brought ‘big data’ to and the science of doing so is advancing every day
&

healthcare, loosely defined as datasets so large that
they cannot be analysed with traditional statistical
methods. Compared with a prospective randomized
trial of dozens to thousands of patients, big data
research is based on analysing millions of bits of
information. Data can become ‘big’ in several ways.
One is by capturing millions of pieces of infor-
mation on small numbers of patients, as in genetic
databanks that focus on specific diseases or patient
populations. Another approach is to amass giant
datasets from imaging studies or diagnostic
monitors applied to a single patient, for example
microsecond-level capture of the multiple physio-
logic monitors used for intensive or intraoperative
care. But the most common definition of big data
research in healthcare applies to analysis of rela-
tively simple information gathered from very large
numbers of patients, for example in the National
Inpatient Sample (NIS) maintained by the US Agen-
cy for Healthcare Research and Quality, which is a
roll-up of hospital-based patient and procedure
reporting from all 50 states. Large datasets such as
this one are being increasingly mined to create
observations about medical care beyond the con-
trolled study of carefully recruited populations. The
[1 ]. Comparative effectiveness research of different
therapies in similar populations will be enabled, as
will better understanding of the long-term con-
sequences of medical care.
BIG DATA IN ANAESTHESIOLOGY
For anaesthesiology, the Information Age acceler-
ated two already dominant characteristics: evi-
dence-based decision-making and a strong culture
of introspection and perpetual self-improvement.
Detailed recording of the patient’s response to
anaesthesia goes back at least as far as 1895, when
Harvey Cushing and E.A. Codman competed to see
who could maintain the more stable vital signs
during anaesthesia [2]. Anaesthesiologists were early
American Society of Anesthesiologists, University of Chicago, Schaum-
burg, Illinois, USA
Correspondence to Richard P. Dutton, MD, MBA, Clinical Associate,
American Society of Anesthesiologists, University of Chicago, 1061
American Lane, Schaumburg, IL 60068, USA.
E-mail: r.dutton@asahq.org
Curr Opin Anesthesiol 2015, 28:697–702
DOI:10.1097/ACO.0000000000000243

0952-7907 Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved. www.co-anesthesiology.com

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.

Technology, education and safety
KEY POINTS
 Advancing healthcare information technology is
creating new opportunities for knowledge development
in anaesthesiology.
 The Anesthesia Quality Institute was created by the
American Society of Anesthesiologists to collect clinical
information about the practice of anaesthesiology.
 The US National Anesthesia Clinical Outcomes Registry
(NACOR) is based on the routine digital harvest of
administrative data, quality outcomes and granular
clinical information from anaesthesia cases.
 NACOR today includes more than 30 million cases,
and is capturing approximately 33% of all US
anaesthesia practice.
 NACOR data are available for researchers and have
already contributed to articles describing the
demographics and outcomes of modern anaesthesia
practice.
adapters of computer-facilitated documentation in
the late 1980s and early 1990s, leading to substantial
single-institution archives of electronic records at
facilities such as the Cleveland Clinic and Vander-
bilt University. As the rest of healthcare has caught
up, the Anaesthesia Information Management
System (AIMS) is increasingly a component of an
enterprise-wide electronic medical record (EMR)
that reaches from outpatient clinics and primary
care offices all the way to the ICU and operating
rooms [3]. In the USA, incentives offered to physi-
cians and healthcare facilities for ‘meaningful use’ of
electronic records have led to steadily accelerating
uptake of EMRs, to the point at which they are now
ubiquitous in university and large-community hos-
pitals and rapidly gaining in all other venues. Light-
weight tablet computers and cloud architecture
have made it possible to extend the EMR to almost
any environment in which care – including anaes-
thesia – is provided. Further, billing and payment
systems for healthcare have been fully digital for at
least a decade, making it nearly certain that at least
some electronic information is available for every
anaesthetic.
The American Society of Anaesthesiologists
(ASA) took the opportunity in 2008 to invest in a
new infrastructure for understanding and improv-
ing patient care, by creating the Anesthesia Quality
Institute (AQI) as a nonprofit related organization.
The AQI began operations in 2009 with a simple
mission: to use the tools of the Information Age to
capture anaesthesia case data for the purposes of
local quality improvement, inter-practice bench-
marking and scientific research [4].
698 www.co-anesthesiology.com
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
ANESTHESIA QUALITY INSTITUTE
INFRASTRUCTURE
The first product developed by the AQI was the
National Anesthesia Clinical Outcomes Registry
(NACOR), built to capture available electronic data
from every case, every day, in participating anaes-
thesia practices. Participation is open to every anaes-
thesiologist in the USA and is free to members of the
ASA. Data capture in NACOR is through oppor-
tunistic linkage to existing software, in three ‘tiers’
of effort [4]. Every participating group begins by
contributing a minimum dataset of demographic
information from their billing software, which
includes for every case the facility wherein the case
occurred; the patient ZIP code, age, sex and ASA
physical status; a description of the procedure [using
the Current Procedural Terminology (CPT) codes
established by the American Medical Association];
a description of the anaesthetic (also using CPT
codes); the date, time and duration of the procedure;
and coded identifiers for the anaesthesia personnel
involved [5]. These data by themselves provide a
powerful and representative picture of anaesthesia
practice, while also providing denominator infor-
mation useful for benchmarking.
The second tier of NACOR data collection are
the immediate clinical outcomes of anaesthesia,
gathered from the point of care in the operating
room (OR) or postanaesthesia care unit (PACU).
These include case-by-case reporting of the occur-
rence, or not, of haemodynamic instability, dys-
rhythmia, cardiac arrest or mortality; airway and
respiratory complications; neurologic injury or
altered mental status; postoperative nausea and
vomiting; inadequate pain management; and rare
adverse effects ranging from corneal abrasion to
anaphylaxis. Between one-third and half of practices
participating in NACOR gather these data for every
case, through a variety of stand-alone software sys-
tems or – increasingly – as a part of the AIMS.
The highest tier of NACOR participation is trans-
mission of full reports from the AIMS. This includes
intraoperative vital signs, medication doses, pro-
cedural details and specific event timing for each
case. These data are currently captured from about
10% of participating groups, but this proportion is
rising steadily with increased penetration of AIMS
and propagation of report templates through the
vendors. Broad collection of these data will enable
low-cost, high-volume comparative effectiveness
research that links specific anaesthetic interventions
(e.g. choice of medication, haemodynamic manage-
ment) with clinical outcomes such as delerium,
nausea, pain and adverse cardiac events [6]. Cases
have accumulated in NACOR from 1 January 2010.
More than 500 practices are now contributing cases
Volume 28  Number 6  December 2015

from more than 3700 hospitals, outpatient centres
and other anaesthesia venues.
With the basic infrastructure for routine case
surveillance established and scaling up, the AQI
turned its attention to another need: the capture of
narratives from individual exceptional cases. The
Anesthesia Incident Reporting System (AIRS) was
launched in 2011 as a universally accessible web-
based tool for reporting unusual cases, serious adverse
events and near misses. Any anaesthesia practitioner,
anywhere, can enter a case at www.aqiairs.org
by filling in the simple survey tool provided. In the
years since its launch, AIRS has been ‘built-out’ with
the inclusion of subspecialty modules, has been
adapted by several departments as a routine tool
for internal incident capture and has been supple-
mented with a smartphone application usable when
the Internet is not available. Data and narratives
entered into AIRS are protected from legal discovery
as patient-safety work product through the AQI’s
status as a federally designated Patient Safety Organ-
ization [6]. Interesting cases reported to AIRS are
reviewed by a panel of experts, fictionalized to pre-
serve confidentiality, and then presented as quality
improvement learning exercises in the ASA News-
letter. A complete archive of the more than 50 cases
published to date is available at https://www.aqihq.
org/casereportsandcommittee.aspx.
USE OF NACOR DATA FOR RESEARCH
Of the 22 peer-reviewed publications to date that
concern the AQI, the last 10 can truly be said to be
‘big data’ research. With more than 30 million
reported cases to draw from over a 6-year span,
coming from a sample approaching one-third of
all US anaesthetics, NACOR provides unprece-
dented power to understand and improve patient
care. Unlike many registries, NACOR is not biased
towards university-based practice but includes
representative proportions of academic and
private practice and small vs. large facilities.
This helps to avoid the identified bias in big data
&
towards sicker patients [7 ]. The supplementation
of administrative data with clinical data in a
subset of cases can help to identify gaps in effective
reporting, as illustrated by the work of Katznelson
et al. [8].
De-identified data from NACOR are available
upon request to any academic in a participating
&&
practice [9 ]. Figure 1 is a cartoon showing the type
and quantity of data in NACOR as of 1 July 2015.
Figure 2 shows the growth of NACOR over the first
5 years of its existence. In 2015, the registry will
capture approximately 16 000 000 cases, represent-
ing one-third of all US anaesthetics.
0952-7907 Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Large databases in anaesthesiology Dutton
Facility and provider
information from > 500 practices
Case data from
> 30M anesthetics
Clinical outcomes
from > 5M cases
AIMS data from
> 1M cases
FIGURE 1. Data available in the National Anesthesia
Clinical Outcomes Registry. Participating practices report
information on their facilities and providers, billing data on
individual cases, clinical outcomes for each case and
granular information from the Anesthesia Information
Management System (AIMS).
The most obvious use of these data is to establish
a picture of the national demographics of anaesthe-
sia cases, including trends over time. Data from
NACOR are used on a regular basis by ASA and
AQI leaders to determine the size and personnel
in anaesthesia practices (physicians vs. mid-level
providers), the facility types wherein cases are done,
the age, sex and physical status of anaesthetized
patients, the number and duration of procedures
performed and the type of anaesthesia provided to
&&
each [9 ]. Figure 3 shows one emerging trend in the
USA, a steady and significant increase in the per-
centage of patients greater than 64 years of age
presenting for anaesthetics. Figure 4 shows a differ-
ent abstraction from the case data in NACOR: the
mean and interquartile duration for a single com-
mon surgical procedure (total knee replacement)
from more than 100 different facilities where it is
performed. The substantial variation seen – more
than 350% – represents a significant target for
improvement. The local version of this figure is
available in the routine reports accessible by every
participating practice and is of enormous value for
local quality improvement and practice manage-
ment.
NACOR participation fuels improved patient
care by enabling the practice to track key outcome
measures over time, with benchmarking to other
groups and facilities. In 2015, NACOR is expanding
www.co-anesthesiology.com 699
Technology, education and safety

NACOR Cases Facilities

2500
25 000 000
2000
20 000 000
1500
15 000 000
NACOR Cases 1000 Facilities
10 000 000
5 000 000 500
0 0
2010 2011 2012 2013 2014 2010 2011 2012 2013 2014

Providers Practices
25 000
250
20 000
200
15 000
150
10 000 Providers
Practices
100

50
0
2010 2011 2012 2013 2014 0
2010 2011 2012 2013 2014

FIGURE 2. Growth in cases and in participating practices, facilities and providers in the National Anesthesia Clinical
Outcomes Registry, 2010–2014.

its reporting infrastructure to include risk-stratified patients undergoing office-based surgery, while
benchmarks for outcome measures based on facility Fleischut et al. [12] examined the choice of anaes-
type, private vs. academic practice, patient age and thesia type for knee replacement on the basis of the
geographic region. Board-certification status of the attending anaes-
&
Scientific publications based on NACOR data thesiologist. Nunnally et al. [13 ] reported on the
have focused on various elements and subsets. Deiner occurrence of cardiac arrest in the OR and PACU,
&
et al. [10 ] described the demographics of older including patient risk factors, the most commonly
patients in NACOR, including the most common associated anaesthetics and the associated short-
adverse events occurring during anaesthesia. Shapiro term mortality. Howard-Quijano et al. [14] illustrated
&
et al. [11 ] reported demographics and outcomes of a similar approach to studying outcomes from

Patients over 64 by year

35.00%

30.00%
Percentage of cases in NACOR

25.00%

20.00% % over 64
% over 80

15.00%

10.00%

5.00%

0.00%
2010 2011 2012 2013 2014

FIGURE 3. Change in the percentage of patients aged >64 and >80 years in the National Anesthesia Clinical Outcomes
Registry. Anaesthesia practices in the USA are seeing a significant increase in the number of older patients.

700 www.co-anesthesiology.com Volume 28  Number 6  December 2015

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.


Mean case time by practice – Total knee replacement
330
264
Duration (minutes) 198
132
66
0
0 50
IQR
FIGURE 4. The mean and interquartile range of duration in minutes of anaesthesia time for uncomplicated total knee
replacement in facilities reporting this case to the National Anesthesia Clinical Outcomes Registry. Facilities in light grey are
university hospitals.
paediatric heart transplantation from a national
organ-transplant database, one of many smaller regis-
tries that contain anaesthesia data.
NACOR data have been used to describe the
economics of anaesthesia practice, as in the article
by Dexter et al. [15] showing the distribution of case
starts and duration over the course of the workday.
&&
Dutton et al. [16 ] used NACOR to add perspective
to a case series drawn from the ASA Closed Claims
Project registry of anaesthesia malpractice cases. In
this report on events related to massive haemor-
rhage, the frequencies of case types in the Closed
Claims Registry were related to national frequencies
of those case types in NACOR, demonstrating that
obstetric cases and minimally invasive spine
surgeries were associated with a higher risk for mal-
practice litigation due to haemorrhage.
THE FUTURE OF LARGE DATABASES IN
ANAESTHESIA
NACOR and other large repositories of anaesthesia
case information will continue to grow in both
participation and depth of data captured. The shift
to value-based payment in the USA is accelerating
this trend, as every anaesthesia practice will soon be
required to collect performance data and contribute
it to external benchmarking systems in order to be
paid for their services. At the same time, the steady
advance of digital hardware and software, coupled
with painful but progressing standardization and
interoperability of EMRs, will make data collection
faster, less expensive and more comprehensive.
NACOR will progress from gathering short-term
0952-7907 Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Large databases in anaesthesiology Dutton
100 150
Practice ID
University based Average
outcomes into more relevant data such as hospital
length of stay, postoperative functional status and
long-term mortality. Many of these outcomes will
progress from being reported through provider
attestation to automatic capture from the EMR,
for example calculation of ‘haemodynamic instabil-
ity’ from the vital signs rather than from a provider
checkbox. This will produce higher quality data
with less burden on the clinician. Data from the
EMR will be additionally supplemented with
patient-reported outcomes, including overall satis-
faction, that will provide a new – and perhaps more
relevant – perspective on anaesthesia practice.
Large anaesthesia datasets, connected in real time
to ongoing cases, will create a mechanism for power-
ing very large prospective trials at a fraction of the
usual cost. As reported by Panjasawatwong et al. [17],
the AIMS can be used both to provide clinical
decision support based on big-data analysis and to
randomize patients for the necessary validation
studies. The future will also see the rapid advance
of machine learning based on large datasets [18,19].
One of the key characteristics of big data is the
potential for automating the learning process, allow-
ing the computer to identify associations between
disparate data elements while simultaneously con-
trolling for thousands of variables. This approach
will become even more powerful as interoperability
between registries increases, based on standard data
definitions and formatting. Wnent et al. [20] illus-
trate this principle in their discussion of a unifying
European resuscitation ‘meta-registry’. This will
allow researchers to connect anaesthesia process
decisions in real-world populations to surgical and
www.co-anesthesiology.com 701
Technology, education and safety

4. Dutton R, DuKatz A. Quality improvement using automated data sources: the

medical outcomes occurring weeks or months in the
future. Problems that will be studied through this
mechanism include the role that anaesthesia medi-
cations play in cancer metastasis, the risk of infection
and long-term cognitive decline.
CONCLUSION
National-level registries are an emerging research
vehicle that will become increasingly important
for linking actions and outcomes in anaesthesiology
across large populations of ‘real-world’ patients and
cases.
Acknowledgements
The author would like to acknowledge the many indi-
viduals who have contributed to the rapid growth of
NACOR into an engine for modern anaesthesia research.
This is an original work that has not been previously
published in whole or in part.
The Figures have not been previously published.
Financial support and sponsorship
This work was supported by the AQI.
Conflicts of interest
There are no conflicts of interest.
REFERENCES AND RECOMMENDED
READING
Papers of particular interest, published within the annual period of review, have
been highlighted as:
& of special interest
&& of outstanding interest
1. Simpao AF, Ahumada LM, Rehman MA. Big data and visual analytics in
& anaesthesia and health care. Br J Anaesth 2015 [Epub ahead of print]. Article
25627395.
An excellent high-level article describing the potential of big data research for the
future of anaesthesiology.
2. Wright AJ. Early use of the Cushing-Codman anesthesia record. Anesthesiol-
ogy 1987; 66:92.
3. Kadry B, Feaster WW, Macario A, Ehrenfeld JM. Anesthesia information
management systems: past, present, and future of anesthesia records. Mt
Sinai J Med 2012; 79:154–165.
Anesthesia Quality Institute. Anesthesiol Clin 2011; 29:439–454.
5. Grissom TE, DuKatz A, Kordylewski H, Dutton RP. Bring out your data: the
evolution of the National Anesthesia Clinical Outcomes Registry. Int J Comput
Models Algorith Med 2011; 2:51–69.
6. Dutton RP. Quality management and registries. Anesthesiol Clin 2014;
32:577–586.
7. Weiskopf NG, Rusanov A, Weng C. Sick patients have more data: the
& nonrandom completeness of electronic health records. AMIA Annu Symp
Proc 2013; 2013:1472–1477.
A recent publication demonstrating bias in published ’big data’ studies towards
sicker patients and university over community hospitals.
8. Katznelson R, Djaiani G, Tait G, et al. Hospital administrative database
underestimates delirium rate after cardiac surgery. Can J Anaesth 2010;
57:898–902.
9. Liau A, Havidich JE, Onega T, Dutton RP. The National Anesthesia Clinical
&& Outcomes Registry (NACOR). Anesth Analg 2015. (in press).
This is a technical communication describing in detail the current workings of
NACOR, including the recruitment of participant practices, the acquisition of case-
level data, validation and aggregation of results.
10. Deiner S, Westlake B, Dutton RP. Patterns of surgical care and complications
& in elderly adults. J Am Geriatr Soc 2014; 62:829–835.
An early illustration of the power of NACOR to describe the demographics of
anaesthesia practice in the United States.
11. Shapiro FE, Jani SR, Liu X, et al. Initial results from the National Anesthesia
& Clinical Outcomes Registry and overview of office-based anesthesia.
Anesthesiol Clin 2014; 32:431–444.
Another demographics article from NACOR, illustrating the rapid growth of the
most dynamic sector of U.S. anaesthesia practice.
12. Fleischut PM, Eskreis-Winkler JM, Gaber-Baylis LK, et al. Variability in
anesthetic care for total knee arthroplasty: an analysis from the Anesthesia
Quality Institute. Am J Med Qual 2015; 30:172–179.
13. Nunnally ME, O’Connor MF, Kordylewski H, et al. The incidence and risk
& factors for perioperative cardiac arrest observed in the National Anesthesia
Clinical Outcomes Registry. Anesth Analg 2015; 120:364–370.
This study is the first of what will likely be many articles describing the ‘real-world’
complications of anaesthesia. It is also noteworthy in identifying a much higher risk
of intraoperative cardiac arrest in male vs. female patients.
14. Howard-Quijano K, Schwarzenberger JC, Scovotti JC, et al. Increased red
blood cell transfusions are associated with worsening outcomes in pediatric
heart transplant patients. Anesth Analg 2013; 116:1295–1308.
15. Dexter F, Dutton RP, Kordylewski H, Epstein RH. Anesthesia workload
nationally during regular workdays and weekends. Anesth Analg 2015 [Epub
ahead of print]. Article 25923436.
16. Dutton R, Lee L, Stephens L, et al. Massive hemorrhage: a report from the
&& Anesthesia Closed Claims Project. Anesthesiology 2014; 121:450–458.
This study is the first to link incident cases from the Closed Claims Project to
national demographics from NACOR; this likely represents the first example of a
common future approach to understanding the risk for very rare adverse outcomes.
17. Panjasawatwong K, Sessler DI, Stapelfeldt WH, et al. A randomized trial of a
supplemental alarm for critically low systolic blood pressure. Anesth Analg
2015. (in press).
18. Crown WH. Potential application of machine learning in health outcomes
research and some statistical cautions. Value Health 2015; 18:137–140.
19. Tighe PJ, Harle CA, Hurley RW, et al. Teaching a machine to feel post-
operative pain: combining high-dimensional clinical data with machine learn-
ing algorithms to forecast acute postoperative pain. Pain Med 2015;
16:1386–1401.
20. Wnent J, Masterson S, Gräsner JT, et al. EuReCa ONE – 27 nations, ONE
Europe ONE Registry: a prospective observational analysis over one month in
27 resuscitation registries in Europe – the EuReCa ONE study protocol.
Scand J Trauma Resusc Emerg Med 2015; 23:7.

702 www.co-anesthesiology.com Volume 28  Number 6  December 2015

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.