REVIEW

Artificial Intelligence in Musculoskeletal Imaging:

A Paradigm Shift
Joseph E Burns,1 Jianhua Yao,2 and Ronald M Summers2
1
Department of Radiological Sciences, University of California-Irvine School of Medicine, Orange, CA, USA
2
Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences Department, Clinical Center, National
Institutes of Health, Bethesda, MD, USA

ABSTRACT
Artificial intelligence is upending many of our assumptions about the ability of computers to detect and diagnose diseases on
medical images. Deep learning, a recent innovation in artificial intelligence, has shown the ability to interpret medical images with
sensitivities and specificities at or near that of skilled clinicians for some applications. In this review, we summarize the history of arti-
ficial intelligence, present some recent research advances, and speculate about the potential revolutionary clinical impact of the
latest computer techniques for bone and muscle imaging. © 2019 American Society for Bone and Mineral Research. Published
2019. This article is a U.S. Government work and is in the public domain in the USA.

KEY WORDS: RADIOLOGY; SKELETAL MUSCLE; CANCER; ANALYSIS/QUANTITATION OF BONE

Introduction For this review, we will focus on musculoskeletal radiology

A rtificial intelligence (AI), a subfield of computer science, can
be broadly defined as the use of a computer to replicate
human intelligence in performing tasks. Nested within this
umbrella term of “artificial intelligence” is the field of machine
learning. Important goals of machine learning are to impart to
computers the capacity to learn without explicit programming
and to make more accurate predictions from data. Although
machine learning algorithms in general do progressively
improve with experience at performing tasks, they still may need
some human input or guidance to improve performance.
Computer analysis of images has a long history in musculo-
skeletal radiology. The last 5 years have seen a dramatic
improvement in such analyses. In particular, recent develop-
ments in artificial intelligence have caused a paradigm shift in
the ability of computers to detect, characterize, and measure
complex pathologies in the musculoskeletal system.
These advances have arisen in large measure because of
improvements in the field of computer vision, a branch of com-
puter science. The computer scientists in this field have devel-
oped a suite of technologies loosely called deep learning that
are able to recognize objects in natural world images like the
photographs we take while on vacation.(1) It turns out that these
same technologies are highly effective at detecting abnormali-
ties on medical images, including radiology, pathology, and oph-
thalmologic images.(2–5)
images. We performed a search of PubMed for papers with the
following key word set: (machine learning OR deep learning
OR convolutional OR neural network) AND (musculoskeletal OR
skeletal OR bone OR skeleton OR muscle) AND (X-ray OR xray
OR radiography OR computed tomography OR CT OR magnetic
resonance OR MRI). This search yielded more than 800 results
as of this writing, the majority (71%, 581/821) from 2013 and
later. Clearly this field is growing rapidly. In this review, we focus
on the authors’ experience with the application of artificial
intelligence in musculoskeletal imaging and how we believe this
represents a paradigm shift in this field.
Background
Although deep learning has only been applied to radiology over
the past few years, radiology researchers have been developing
computer tools for automated detection and characterization of
musculoskeletal disease on radiologic images for decades. Three
generations of development are loosely defined centered about
the parallel evolution of medical imaging technology and com-
puter hardware and software, and availability of very large
labeled data sets.
First-generation development was inhibited by primary image
acquisition on analog photographic film, with ad hoc digital con-
version, and the markedly slower speeds of available computers.
Some early examples in musculoskeletal radiology include

Received in original form April 11, 2019; revised form July 23, 2019; accepted August 5, 2019. Accepted manuscript online September 12, 2019.
Address Correspondence to: Ronald M Summers, MD, PhD, Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences,
National Institutes of Health Clinical Center, Building 10, Room 1C224D MSC 1182, Bethesda, MD 20892-1182, USA. E-mail: rms@nih.gov
This work was supported in part by the Intramural Research Program of the National Institutes of Health, Clinical Center. The opinions expressed herein are
those of the authors and do not necessarily represent those of the DHHS, NIH, or UCI.
Journal of Bone and Mineral Research, Vol. 35, No. 1, January 2020, pp 28–35.
DOI: 10.1002/jbmr.3849
Published 2019. This article is a U.S. Government work and is in the public domain in the USA

n 28 BURNS ET AL. Journal of Bone and Mineral Research

classifying bone tumors on radiographs (Lodwick and col-
leagues(6)), segmenting the femur and tibia on AP knee radiographs
(Ausherman and colleagues(7)), and assessing bone mineral density
on hand and wrist radiographs (Geraets and colleagues(8)).
Second-generation development is characterized by increas-
ingly prevalent primary digital image acquisition, more capable
computers, and new algorithms. This generation continued until
the early to mid-2010s. Some examples include texture-based
morphometry of osteoporosis on CT (Mundinger and col-
leagues(9)), estimation of skeletal age on radiographs (Pietka
and colleagues(10); Frisch and colleagues(11)), assessment of bone
trabeculae on MRI and CT (Link and colleagues(12)), measurement
of bone mineral density on CT (Lang and colleagues(13); Summers
and colleagues(14)) (Fig. 1), and detection of lytic and sclerotic
metastases on CT (O’Connor and colleagues(15); Burns and
Fig. 2. Lytic spine metastasis detection using second-generation AI soft-
colleagues(16)) (Fig. 2 and Fig. 3). More examples can be found in
ware. There are two lytic lesions, 3.4 cm (5.1 cm2) and 1.2 cm (1.1 cm2),
a recent review.(17)
presumably metastatic, in L1 vertebral body of a 47-year-old man with mel-
In these first two generations, researchers attempted to repli-
anoma. (A) Original axial bone window image showing lesions (arrows).
cate human perception and cognition in the interpretation of
(B) Color-coded image showing lesion detection and classification.
medical images. They did so by handcrafting computer models
(B): Green = computer-aided detection of lytic lesions; red = spine detec-
to mimic human perception of the features of the anatomy or
tion; blue = spinal canal detection; brown = Lytic lesion candidate rejected
pathology of interest. While neural networks were available at
by filter or classifier. Reproduced from O’Connor and colleagues.(15)
this time, use was inhibited because of CPU (rather than graphics
processing unit or GPU) based computation and inability to train
deep networks. Consequently, other analytical techniques such
as support vector machines (SVMs) rose in prominence. Unfortu-
nately, those other techniques were insufficient to achieve medical images without being specifically designed to assess
clinician-level performance. these features (ie, handcrafted features are no longer necessary).
Recent advances have led to the third generation of develop- The third-generation techniques are more accurate and rapidly
ment. These advances include faster and cheaper GPUs, the abil- developed (from years to months) than prior generations.
ity to train deep (as opposed to shallow) neural networks with
many layers, and the increasing availability of very large labeled
data sets. These deep learning systems can learn features from
Imaging Modality Examples

Artificial intelligence techniques have been applied to the vari-

ous modalities of musculoskeletal imaging, including DXA, radi-
ography, CT, and MRI. In this section, we will review a number
of such applications.

DXA
On DXA, applications include detection of osteoporotic verte-
bral compression fractures and femoral segmentation for
assessment of bone density.(18,19) Using semi-automated
vertebral segmentation and appearance models, Roberts
et al. attained 88% sensitivity for compression fracture detec-
tion at a 5% false-positive rate.

Fig. 1. Fully automated phantomless bone mineral densitometry for
osteoporosis assessment on contrast-enhanced CT of a 65-year-old
woman using second-generation AI software. The green oval indicates
the region of interest (ROI) in the L1 vertebral body automatically located
and measured by the software. The ROI is placed in the anterior vertebral
body so as to avoid the basivertebral vein. The green and red boxes indi-
cate the automatically located vertebral body/spinous process and entire
vertebra, respectively.
Radiography
Kim and MacKinnon(20) and Lindsey and colleagues(21) used
deep convolutional neural networks to assess wrist radiographs
for fractures. Kim and MacKinnon attained a sensitivity of 0.9
and specificity of 0.88 in wrist fracture detection with transfer
learning on a deep CNN model pretrained on non-medical
images. Lindsey and colleagues showed that with deep learning,
wrist fractures were correctly detected by emergency medicine
clinicians with a relative reduction in misinterpretation rate
of 47.0%.
Bone age analysis from pediatric hand radiographs has been
performed recently using deep learning.(22,23)

Journal of Bone and Mineral Research ARTIFICIAL INTELLIGENCE IN MUSCULOSKELETAL IMAGING 29 n

Fig. 3. Sclerotic spine metastasis detection using second generation AI software. (A) Sclerotic vertebral body tumors (focal regions of high bone density)
denoted by blue arrows. (B) Watershed method for segmentation of regions of similar bone and soft tissue density. (C) CAD system detected high-density
lesions (green). (D) Reference standard segmentation (blue). (E) CAD system 3D detections (green) with ground truth lesions superimposed.
Computed tomography
Our group and others have focused to a great extent on muscu-
loskeletal pathology of the central body or axial skeleton on
CT. Reasons for this focus include the widespread use of body
CT and the relatively infrequent use of extremity CT. For AI
research, CT also has some benefits over MRI, including fewer
artefacts and the direct physical meaning of the pixel intensities.
For example, CT Hounsfield units (HU) are calibrated daily to a
standard, but MR signal intensity is determined by a host of
machine-specific parameters and characteristics.
Examples of this work include automated detection of traumatic
and compression fractures, degenerative changes, epidural masses,
bone metastases, and bone mineral density of the spine on CT
utilizing both second- and third-generation techniques.(15,16,24–38)
The typical strategy for these applications involves vertebral level
labeling and segmentation, followed by feature- or learning-based
automated measurement or detection.(39–42)
Magnetic resonance imaging
Because MRI is widely used for assessing spinal degenerative
changes in the setting of neck and back pain, significant efforts
have been made toward automated delineation of spinal anat-
omy on MRI images.(43–48) Forsberg and colleagues studied
detection and labeling of cervical and lumbar vertebrae using
third- and second-generation techniques, respectively, with sen-
sitivities of 99.1% to 99.8% for detection and accuracies of 96.0%
n 30 BURNS ET AL.
to 97.0% for labeling.(44) Lu and colleagues segmented the lum-
bar vertebrae and assessed for spinal stenosis using a U-Net
deep learning architecture.(47) For vertebral detection, the Dice
Similarity Coefficient, a measure of segmentation accuracy, was
0.93, and for spinal canal and foraminal stenosis grading, the
accuracies were 80.4% and 78.1%, respectively.
Neoplasia detection in bone on MRI has been performed by
Jerebko and Wang.(49,50) Automated detection of the fascia lata
in the thigh was used to assess for fatty replacement of muscle
in patients with muscular dystrophy.(51) Knee cartilage defects
and meniscal tears have been automatically assessed on MRI.(52)
In the next sections, we discuss specific applications in more
detail.
Fracture detection
There has been considerable interest in developing automated
tools to detect a variety of bone fractures. One such system auto-
matically detected acute traumatic fractures of thoracic and lum-
bar vertebral bodies on CT(24,53,54) (Fig. 4). The sensitivity for
detection of fractures within each vertebra was 81%, with a
false-positive rate of 2.7 per patient in the test-set patients. Acute
pelvic fractures can also be automatically detected.(55,56)
Radiologists may overlook spine compression fractures if they do
not routinely review sagittal midline images on body CT.(57) In
response, a system was designed for the automated detection
and localization of thoracic and lumbar vertebral body compression
Journal of Bone and Mineral Research
Fig. 4. Spine acute traumatic fracture detection using second generation AI software. (A) Spine segmentation. There is a fracture at T12 (green arrow). (B)
2D and (C) 3D cortical shell segmentation maps. Cyan = periosteal surface; red: endosteal surface. (D) Unwrapped cortical shell. (E) Detections on
unwrapped map. Fracture detections 1 and 2 are focal curvilinear regions of low bone density. (F) Detections on axial slice. (G) 3D display of detections.
Adapted from Burns and colleagues.(24)

fractures on CT. In addition to detection, the system determined that scanner. This calibration curve may then be used to convert
Genant classification of the fractures(26,58) (Fig. 5). In a set of CT scans the mean trabecular bone CT attenuation on this now-calibrated
with 210 fractured vertebrae, the sensitivity for detection of verte- scanner to give a BMD estimation without the presence of the
brae with compression fractures was 95.7%, with a false-positive phantom. DXA and automated quantitative CT of the lumbar
rate of 0.29 per patient.(26) spine have been compared for assessment of bone mineral den-
sity on CT with an area under the ROC curve of 0.888.(62)
Bone oncology Osteoporosis and fragility fracture risk have also been
assessed on dental panorex radiographs, hip radiographs, and
Automated detection of lytic, sclerotic, and mixed density meta-
on MRI.(63–65) For example, MRI in combination with FRAX score,
static bone lesions of the spine and sclerotic lesions of the ribs
BMD, and patient physical characteristics has been used to pre-
has been developed (Figs. 2 and 3).(15,16,59–61) In one such
dict osteoporotic bone fractures.(65)
system, the sensitivity (and false-positive rate per patient) was
81% (2.1), 81% (1.3), and 76% (2.1) for sclerotic, lytic, and mixed
lesions of the spine, respectively, using SVM classifiers.(27) This Opportunistic screening
system is a first step toward the quantitative analysis of meta-
Opportunistic screening means the detection of abnormalities
static spine disease for determination of tumor burden, assess-
unrelated to the primary indication for the scan. Examples of oppor-
ment of lesion change over time, and inclusion of bone lesions
tunistic screening include bone mineral densitometry, visceral fat
into treatment response criteria such as RECIST. In other work,
analysis, and sarcopenia assessment on CT scans obtained for colo-
185 sclerotic lesions were identified in patient ribs, and a system
rectal cancer screening or other indications(14,62,66–71) (Fig. 6). Such
with 75.4% sensitivity at an average of 5.6 false-positives per case
opportunistic screening does not require additional radiation expo-
was designed and tested using an SVM classifier.(59) Performance
sure and provides additional information from images that
improvements will be required for such systems to be used
already exist. It is envisioned that such opportunistic screening
clinically.
may lead to early detection, risk assessment, and favorable
treatment outcomes in such conditions as osteoporosis, meta-
Osteoporosis and assessment of bone mineral density
bolic syndrome, and sarcopenia.
Automated bone mineral density determination on CT may be
performed with a densitometry phantom in the field-of-view or
Other applications
with a calibrated scanner(14) (Fig. 1). Calibration curves may be
constructed from the phantoms in dedicated QCT scans, map- Other AI applications include bone strength determination and
ping CT attenuation in HU to bone mineral density in mg/cc on osteoarthritis evaluation.(72)

Journal of Bone and Mineral Research ARTIFICIAL INTELLIGENCE IN MUSCULOSKELETAL IMAGING 31 n

Fig. 5. Vertebral body compression fracture detection and characterization in an osteopenic patient using second-generation AI software. Eighty-six-
year-old female, compression fractures at T3 and T7. (A, B) the program detects the vertebral endplates, creates a coordinate axis, and establishes distances
between endplates along the z axis. (C) Sagittal CT section shows automated spine segmentation and vertebral partitioning. (D, E) Cross section of stacked
(D) sagittal and (E) axial vertebral height compasses. Each compass is an axial sector map of the vertebral height loss. White = no height loss. Gray to
black = increasing height loss. (E) Height compasses of a grade 2 concave fracture at T3 (top), a grade 3 wedge fracture at T7 (middle), and a normal ver-
tebral body at L2 (bottom). A = anterior; P = posterior; L = left; R = right. Adapted from Burns and colleagues.(26)

Purpose-Driven Development: Putting Medicine
on a Scientific Quantitative Basis
Although in many cases, the automated detection of a specific
pathology is the initial goal, the ultimate goal of much of
this research is quantitative characterization of the disease of
interest. In this way assessment of disease can be taken from
the qualitative or semiquantitative historical basis of 19th- and
20th-century medicine into the realm of other scientific disci-
plines using the full power of modern computing. Potential
advantages of doing so include quantitative analysis of static
and dynamic characteristics, reduced diagnostic variability, inte-
gration of imaging, pathology, genomic and laboratory data, and
discovery of hidden correlations heretofore unknown and uninves-
tigated, akin to the advances made in data mining genomic data.
Challenges in Development
As alluded to previously, one of the enabling (as well as limiting)
factors in AI development for musculoskeletal imaging is the
public availability of well-labeled image data sets. Examples of
available data sets include those for bone age assessment, knee

Fig. 6. Muscle volumetry for sarcopenia assessment using third-generation AI software. (A) Original axial contrast-enhanced abdominal CT image of a 65-year-old
woman at the L3 level. (B) Muscle group segmentation automatically segmented and measured by the software (the skeletal muscle index group) (red).

n 32 BURNS ET AL. Journal of Bone and Mineral Research

MRI, bone radiographs, and spine imaging.(41,73–75) Data sets
such as these are often used for competitive challenges typically
in association with an image processing conference. Examples
include SpineWeb and intervertebral disc segmentation
(https://ivdm3seg.weebly.com/) at the MICCAI meeting and the
Pediatric Bone Age Machine Learning Challenge at the 2017
meeting of the RSNA.(41,75)
There are also many possible clinical tasks to be automated,
many of which might be used only infrequently.(76) This leads
to development challenges such as a lack of relevant data sets
and difficulties in attaining high performance. Difficulties in
labeling the reference standard, such as uncertainties in lesion
boundaries or diagnosis, and variabilities in human labeling
accuracies, can also lead to reduced performance.
Challenges in Implementation
There have also been challenges in transitioning computer tech-
nologies from the lab bench (ideation, design, and testing) to the
patient’s bedside (direct clinical application). Computer-aided
diagnosis has had a mixed track record in patient care, dampen-
ing enthusiasm for these technologies in clinical settings. Early
CAD systems, which were FDA-approved and integrated into
direct clinical patient care, predominantly for mammography,
often had high sensitivities but also high false-positive rates,
leading to inefficiencies and unnecessary testing in clinical prac-
tice.(77) The new deep learning-based techniques may reduce
the false-positive rates and provide more accurate diagnoses.
Another challenge to implementing these technologies in clini-
cal settings has been practice workflow and economic factors.
For example, it is not clear how these automated tools would
be paid for because there are few mechanisms for reimburse-
ment for such practice enhancements.
Opportunities
Automated image analysis tools can have other applications
that are underrecognized. For example, population analyses
and epidemiologic assessments can be performed more readily
when images can be converted to measurements. Automated
analyses can enable triage, for example, by quickly identifying
critical exam findings and alerting physicians to their presence.
They can permit better integration with other quantitative clini-
cal measures such as lab values and genomic data. They can also
improve measurement reproducibility, for example, in the setting
of RECIST measurements of solid tumors, which are known to
demonstrate large observer variability.(78)
The Future
The power of the latest machine learning algorithms represents
a paradigm shift over the failed promises of earlier generations
of CAD. There is huge excitement about the application of
machine learning to images. This has led to burgeoning invest-
ment in startup companies in this space. Consequently, we can
expect to see many commercial products and much physician
interest in these technologies. Potential benefits to the patient
include more prompt and accurate diagnoses, more coordina-
tion with other clinical data, and more accurate disease progno-
ses. Better understanding of the natural progression of disease,
an enlarged spectrum of epidemiologic studies, and diffusion
of diagnostic technologies to underserved communities for
Journal of Bone and Mineral Research
global health improvements are other potential benefits. Whether
these benefits will come to pass depends on many factors in addi-
tion to basic research progress, including clinical acceptance and
means of reimbursement.
Disclosures
RMS has pending and/or awarded patents for automated image
analyses, and receives royalty income from iCAD, Ping An,
ScanMed, and Koninklijke Philips. His lab received research sup-
port from Ping An Technology Company Ltd. and NVIDIA. JY has
pending and/or awarded patents for automated image analyses,
and receives royalty income from iCAD and Ping An. He is pres-
ently an employee at Tencent Holdings.
Acknowledgments
We thank Drs Nathan Lay and Daniel Elton for software engineer-
ing. Tatiana Wiese is thanked for Fig. 3.
References
1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):
436–44.
2. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in med-
ical image analysis. Med Image Anal. 2017;42:60–88.
3. Sahiner B, Pezeshk A, Hadjiiski LM, et al. Deep learning in medical
imaging and radiation therapy. Med Phys. 2019;46:e1–36.
4. Madabhushi A, Lee G. Image analysis and machine learning in digital
pathology: challenges and opportunities. Med Image Anal. 2016;33:
170–5.
5. Gulshan V, Peng L, Coram M, et al. Development and validation of a
deep learning algorithm for detection of diabetic retinopathy in ret-
inal fundus photographs. JAMA. 2016;316(22):2402–10.
6. Lodwick GS, Haun CL, Smith WE, Keller RF, Robertson ED. Computer
diagnosis of primary bone Tumors. Radiology. 1963;80(2):273–5.
7. Ausherman DA, Dwyer SJ, Lodwick GS. Extraction of connected
edges from knee radiographs. IEEE Trans Comput. 1972;C-21(7):
753–8.
8. Geraets WGM, van der Stelt PF, Netelenbos CJ, Elders PJM. A new
method for automatic recognition of the radiographic trabecular
pattern. J Bone Miner Res. 1990;5(3):227–33.
9. Mundinger A, Wiesmeier B, Dinkel E, Helwig A, Beck A, Moenting JS.
Quantitative image analysis of vertebral body architecture—Improved
diagnosis in osteoporosis based on high-resolution computed tomog-
raphy. Br J Radiol. 1993;66(783):209–13.
10. Pietka E. Computer-assisted bone age assessment based on features
automatically extracted from a hand radiograph. Comput Med Imag-
ing Graph. 1995;19(3):251–9.
11. Frisch H, Riedl S, Waldhor T. Computer-aided estimation of skeletal
age and comparison with bone age evaluations by the method of
Greulich-Pyle and Tanner-Whitehouse. Pediatr Radiol. 1996;26(3):
226–31.
12. Link TM, Majumdar S, Lin JC, et al. A comparative study of trabecular
bone properties in the spine and femur using high resolution MRI
and CT. J Bone Miner Res. 1998;13(1):122–32.
13. Lang TF, Guglielmi G, van Kuijk C, De Serio A, Cammisa M, Genant HK.
Measurement of bone mineral density at the spine and proximal
femur by volumetric quantitative computed tomography and dual-
energy X-ray absorptiometry in elderly women with and without ver-
tebral fractures. Bone. 2002;30(1):247–50.
14. Summers RM, Baecher N, Yao J, et al. Feasibility of simultaneous com-
puted tomographic colonography and fully automated bone mineral
densitometry in a single examination. J Comput Assist Tomogr. 2011;
35(2):212–6.
ARTIFICIAL INTELLIGENCE IN MUSCULOSKELETAL IMAGING 33 n
15. O’Connor SD, Yao J, Summers RM. Lytic metastases in thoracolumbar
spine: computer-aided detection at CT—preliminary study. Radiol-
ogy. 2007;242(3):811–6.
16. Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, Summers RM. Auto-
mated detection of sclerotic metastases in the thoracolumbar spine
at CT. Radiology. 2013;268(1):69–78.
17. Gundry M, Knapp K, Meertens R, Meakin JR. Computer-aided detec-
tion in musculoskeletal projection radiography: a systematic review.
Radiography. 2018;24(2):165–74.
18. Roberts MG, Pacheco EMB, Mohankumar R, Cootes TF, Adams JE.
Detection of vertebral fractures in DXA VFA images using statistical
models of appearance and a semi-automatic segmentation. Osteo-
poros Int. 2010;21(12):2037–46.
19. Hussain D, Al-antari MA, Al-masni MA, Han S-M, Kim T-S. Femur seg-
mentation in DXA imaging using a machine learning decision tree.
J Xray Sci Technol. 2018;26(5):727–46.
20. Kim DH, MacKinnon T. Artificial intelligence in fracture detection:
transfer learning from deep convolutional neural networks. Clin
Radiol. 2018;73(5):439–45.
21. Lindsey R, Daluiski A, Chopra S, et al. Deep neural network improves
fracture detection by clinicians. Proc Natl Acad Sci U S A. 2018;
115(45):11591–6.
22. Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP.
Performance of a deep-learning neural network model in assessing
skeletal maturity on pediatric hand radiographs. Radiology. 2018;
287(1):313–22.
23. Summers RM. Deep learning lends a hand to pediatric radiology.
Radiology. 2018;287(1):323–5.
24. Burns JE, Yao J, Munoz H, Summers RM. Automated detection, local-
ization, and classification of traumatic vertebral body fractures in the
thoracic and lumbar spine at CT. Radiology. 2016;278(1):64–73.
25. Roth HR, Wang Y, Yao J, Lu L, Burns JE, Summers RM. Deep convolutional
networks for automated detection of posterior-element fractures on
spine CT. Medical Imaging. 2016 Computer-Aided Diagnosis 2016;
9785:97850P-1–97850P-7.
26. Burns JE, Yao J, Summers RM. Vertebral body compression fractures
and bone density: automated detection and classification on CT
images. Radiology. 2017;284(3):788–97.
27. Yao J, Burns JE, Sanoria V, Summers RM. Mixed spine metastasis detec-
tion through positron emission tomography/computed tomography
synthesis and multiclassifier. J Med Imaging. 2017;4(2):024504.
28. Yao J, Muno H, Burns JE, Lu L, Summers RM. Computer aided detec-
tion of spinal degenerative osteophytes on sodium fluoride PET/CT.
Lecture Notes in Computational Vision and Biomechanics. Springer
Nature; 2014 pp 51–60.
29. Roth HR, Yao J, Lu L, Stieger J, Burns JE, Summers RM. Detection of
sclerotic spine metastases via random aggregation of deep convolu-
tional neural network classifications. Recent Advances in Computa-
tional Methods and Clinical Applications for Spine Imaging. Lecture
Notes in Computational Vision and Biomechanics. Springer Nature;
2015 pp 3–12.
30. Roth HR, Lu L, Liu J, et al. Improving computer-aided detection using
convolutional neural networks and random view aggregation. IEEE
TMI. 2016;35(5):1170–81.
31. Liu J, Pattanaik S, Yao J, et al. Computer aided detection of epidural
masses on computed tomography scans. Comput Med Imaging
Graph. 2014;38(7):606–12.
32. Pattanaik S, Liu J, Yao J, et al. Epidural masses detection on computed
tomography using spatially-constrained Gaussian mixture models.
Computational Methods and Clinical Applications for Spine Imaging.
Lecture Notes in Computational Vision and Biomechanics. Springer
Nature; 2014 pp 99–108.
33. Munoz HE, Yao J, Burns JE, Pham Y, Stieger J, Summers RM. Vertebral
degenerative disc disease severity evaluation using random forest
classification. Medical Imaging 2014: Computer-Aided Diagnosis.
2014;90353A. https://doi.org/10.1117/12.2042793.
34. Muñoz HE, Yao J, Burns JE, Summers RM. Detection of vertebral
degenerative disc disease based on cortical shell unwrapping. Med-
ical Imaging 2013: Computer-Aided Diagnosis. 2013;86700C. https://
doi.org/10.1117/12.2008063.
n 34 BURNS ET AL.
35. Stieger J, Burns JE, Yao J, Summers RM. Dynamic cortex stripping for
vertebra evaluation. Medical Imaging 2015: Computer-Aided Diag-
nosis. 2015;94143D. https://doi.org/10.1117/12.2082434.
36. Wang Y, Yao J, Burns JE, Liu J, Summers RM. Detection of degenerative
osteophytes of the spine on PET/CT using region-based convolutional
neural networks. Computational Methods and Clinical Applications for
Spine Imaging. Lecture Notes in Computer Science. Springer Nature;
2016 pp 116–24.
37. Wang Y, Yao J, Burns JE, Summers R. Osteoporotic and neoplastic
compression fracture classification on longitudinal CT. IEEE 13th
International Symposium on Biomedical Imaging (ISBI). 2016.; 2016
pp 1181–4.
38. Wang Y, Yao J, Roth HR, Burns JE, Summers RM. Multi-atlas segmenta-
tion with joint label fusion of osteoporotic vertebral compression frac-
tures on CT. Computational Methods and Clinical Applications for
Spine Imaging. Lecture Notes in Computer Science. Springer Nature;
2016 pp 74–84.
39. Yao J, O’Connor SD, Summers RM. Automated spinal column extrac-
tion and partitioning. 3rd IEEE International Symposium on Biomedical
Imaging: From Nano to Macro – Proceedings. 2006; 2006 pp 390–3.
40. Wang Y, Yao J, Roth HR, Burns JE, Summers RM. Improving vertebra
segmentation through joint vertebra-rib atlases. Medical Imaging
2016: Computer-Aided Diagnosis. 2016;9785:97853O–1-97853O-6.
41. Yao J, Burns JE, Forsberg D, et al. A multi-center milestone study of
clinical vertebral CT segmentation. Comput Med Imaging Graph.
2016;49:16–28.
42. Yao J, Burns JE, Getty S, Stieger J, Summers RM. Automated extrac-
tion of anatomic landmarks on vertebrae based on anatomic knowl-
edge and geometrical constraints. IEEE 11th International Symposium
on Biomedical Imaging (ISBI). 2014; 2014 pp 397–400.
43. Lootus M, Kadir T, Zisserman A. Vertebrae detection and labelling in
lumbar MR images. Computational Methods and Clinical Applica-
tions for Spine Imaging. Lecture Notes in Computational Vision and
Biomechanics. Springer Nature; 2014 pp 219–30.
44. Forsberg D, Sjöblom E, Sunshine JL. Detection and labeling of verte-
brae in MR images using deep learning with clinical annotations as
training data. J Digit Imaging. 2017;30(4):406–12.
45. Szu-Hao H, Yi-Hong C, Shang-Hong L, Novak CL. Learning-based ver-
tebra detection and iterative normalized-cut segmentation for spinal
MRI. IEEE Trans Med Imaging. 2009;28(10):1595–605.
46. Korez R, Likar B, Pernuš F, Vrtovec T. Model-based segmentation of
vertebral bodies from MR images with 3D CNNs. Medical Image Com-
puting and Computer-Assisted Intervention – MICCAI 2016. Lecture
Notes in Computer Science. Springer Nature; 2016 pp 433–41.
47. Lu J-T, Pedemonte S, Bizzo B, et al. Deep spine: automated lumbar
vertebral segmentation, disc-level designation, and spinal stenosis
grading using deep learning. In Finale D-V, Jim F, Ken J, et al., eds.
Proceedings of the 3rd Machine Learning for Healthcare Conference.
Proceedings of Machine Learning Research: PMLR.; 2018 pp 403–19.
48. Zhou Y, Liu Y, Chen Q, Gu G, Sui X. Automatic lumbar MRI detection
and identification based on deep learning. J Digit Imaging. 2018;
2019(32):513. https://doi.org/10.1007/s10278-018-0130-7.
49. Jerebko AK, Schmidt GP, Zhou X, et al. Robust parametric modeling
approach based on domain knowledge for computer aided detec-
tion of vertebrae column metastases in MRI. Information Processing
in Medical Imaging. Lecture Notes in Computer Science. Springer
Nature; 2007 pp 713–24.
50. Wang J, Fang Z, Lang N, Yuan H, Su MY, Baldi P. A multi-resolution
approach for spinal metastasis detection using deep Siamese neural
networks. Comput Biol Med. 2017;84:137–46.
51. Kovacs W, Liu CY, Summers RM, Yao JH. Identification of muscle and
subcutaneous and intermuscular adipose tissue on thigh MRI of mus-
cular dystrophy. IEEE 13th International Symposium on Biomedical
Imaging (Isbi)., vol. 2016; 2016 pp 176–9.
52. Bien N, Rajpurkar P, Ball RL, et al. Deep-learning-assisted diagnosis for
knee magnetic resonance imaging: development and retrospective
validation of MRNet. PLoS Med. 2018;15(11):e1002699.
53. Yao J, Burns JE, Munoz H, Summers RM. Detection of vertebral body
fractures based on cortical shell unwrapping. Med Image Comput
Comput Assist Interv. 2012;15(Pt 3:509–16.
Journal of Bone and Mineral Research
54. Yao J, Burns JE, Muñoz H, Summers RM. Cortical shell unwrapping for
vertebral body abnormality detection on computed tomography.
Comput Med Imaging Graph. 2014;38(7):628–38.
55. Chowdhury AS, Burns JE, Mukherjee A, Sen B, Yao J, Summers RM.
Automated detection of pelvic fractures from volumetric CT images.
9th IEEE International Symposium on Biomedical Imaging (ISBI).,
vol. 2012; 2012 pp 1687–90.
56. Wu J, Davuluri P, Ward KR, Cockrell C, Hobson R, Najarian K. Fracture
detection in traumatic pelvic CT images. Int J Biomed Imaging. 2012;
2012:1–10.
57. Carberry GA, Pooler BD, Binkley N, Lauder TB, Bruce RJ, Pickhardt PJ.
Unreported vertebral body compression fractures at abdominal mul-
tidetector CT. Radiology. 2013;268(1):120–6.
58. Yao J, Burns JE, Wiese T, Summers RM. Quantitative vertebral com-
pression fracture evaluation using a height compass. Medical Imag-
ing 2012: Computer-Aided Diagnosis. 2012;83151X. https://doi.org/
10.1117/12.911703.
59. Yao J, Burns JE, Summers R. Sclerotic rib metastases detection on rou-
tine CT images. 9th IEEE International Symposium on Biomedical
Imaging (ISBI); 2–5 May 2012.; 2012 pp 1767–70.
60. Wiese T, Burns J, Yao J, Summers RM. Computer-aided detection of
sclerotic bone metastases in the spine using watershed algorithm
and support vector machines. IEEE International Symposium on Bio-
medical Imaging: From Nano to Macro. 2011; 2011 pp 152–5.
61. Wiese T, Yao J, Burns JE, Summers RM. Detection of sclerotic bone
metastases in the spine using watershed algorithm and graph cut.
Medical Imaging 2012: Computer-Aided Diagnosis. 2012;831512.
https://doi.org/10.1117/12.911700.
62. Pickhardt PJ, Lee LJ, Muñoz del Rio A, et al. Simultaneous screening
for osteoporosis at CT colonography: bone mineral density assess-
ment using MDCT attenuation techniques compared with the DXA
reference standard. J Bone Miner Res. 2011;26(9):2194–203.
63. Kavitha MS, Asano A, Taguchi A, Kurita T, Sanada M. Diagnosis of
osteoporosis from dental panoramic radiographs using the support
vector machine method in a computer-aided system. BMC Med
Imaging. 2012;12(1).
64. Sapthagirivasan V, Anburajan M. Diagnosis of osteoporosis by extrac-
tion of trabecular features from hip radiographs using support vector
machine: an investigation panorama with DXA. Comput Biol Med.
2013;43(11):1910–9.
65. Ferizi U, Besser H, Hysi P, et al. Artificial intelligence applied to osteo-
porosis: a performance comparison of machine learning algorithms
in predicting fragility fractures from MRI data. J Magn Reson Imaging.
2019;49(4):1029–38.
Journal of Bone and Mineral Research
66. Liu J, Pattanaik S, Yao J, et al. Associations among pericolonic fat,
visceral fat, and colorectal polyps on CT colonography. Obesity.
2015;23(2):408–14.
67. Lee SJ, Liu J, Yao J, Kanarek A, Summers RM, Pickhardt PJ. Fully auto-
mated segmentation and quantification of visceral and subcutane-
ous fat at abdominal CT: application to a longitudinal adult
screening cohort. Br J Radiol. 2018;91:1089.
68. Burns JE, Yao J, Chalhoub D, Chen JJ, Summers RM. A machine learn-
ing algorithm to estimate sarcopenia on abdominal CT. Acad Radiol.
2019 Epub 2019 May 21. https://doi.org/10.1016/j.acra.2019.03.011.
69. Weston AD, Korfiatis P, Kline TL, et al. Automated abdominal seg-
mentation of CT scans for body composition analysis using deep
learning. Radiology. 2019;290(3):669–79.
70. Lee H, Troschel FM, Tajmir S, et al. Pixel-level deep segmentation: arti-
ficial intelligence quantifies muscle on computed tomography for
body morphometric analysis. J Digital Imaging. 2017;30(4):487–98.
71. Pickhardt PJ, Lee SJ, Liu J, et al. Population-based opportunistic oste-
oporosis screening: validation of a fully automated CT tool for asses-
sing longitudinal BMD changes. Br J Radiol. 2019;92(1094):20180726.
72. Gyftopoulos S, Lin D, Knoll F, Doshi AM, Rodrigues TC, Recht MP. Arti-
ficial intelligence in musculoskeletal imaging: current status and
future directions. AJR Am J Roentgenol. 2019;213(3):506–13.
73. Rajpurkar P, Irvin J, Bagul A, et al. MURA: large dataset for abnormality
detection in musculoskeletal radiographs. arXiv preprint arXiv:
171206957. 2017.
74. Zbontar J, Knoll F, Sriram A, et al. FASTMRI: an open dataset and
benchmarks for accelerated MRI. arXiv preprint arXiv:181108839.
2018.
75. Halabi SS, Prevedello LM, Kalpathy-Cramer J, et al. The RSNA pediatric
bone age machine learning challenge. Radiology. 2019;290(2):
498–503.
76. Summers RM. Road maps for advancement of radiologic computer-
aided detection in the 21st century. Radiology. 2003;229(1):11–3.
77. Gao Y, Geras KJ, Lewin AA, Moy L. New frontiers: an update on
computer-aided diagnosis for breast imaging in the age of artificial
intelligence. Am J Roentgenol. 2019;212(2):300–7.
78. Tang Y, Harrison AP, Bagheri M, Xiao J, Summers RM. Semi-automatic
RECIST labeling on CT scans with cascaded convolutional neural net-
works. Medical Image Computing and Computer Assisted Interven-
tion — MICCAI 2018. MICCAI 2018. Lecture Notes in Computer
Science, vol 11073. Cham: Springer International Publishing; 2018
pp 405–13.
ARTIFICIAL INTELLIGENCE IN MUSCULOSKELETAL IMAGING 35 n