ORIGINAL ARTICLE

Characterizing the role of

dermatologists in developing artificial
intelligence for assessment of skin
cancer: A systematic review
George A. Zakhem, MD, MBA,a Joseph W. Fakhoury, BS,b Catherine C. Motosko, MD,a and
Roger S. Ho, MD, MS, MPHa
New York, New York and Detroit, Michigan

Background: The use of artificial intelligence (AI) for skin cancer assessment has been an emerging topic
in dermatology. Leadership of dermatologists is necessary in defining how these technologies fit into
clinical practice.

Objective: To characterize the evolution of AI in skin cancer assessment and characterize the involvement
of dermatologists in developing these technologies.

Methods: An electronic literature search was performed using PubMed by searching machine learning or
artificial intelligence combined with skin cancer or melanoma. Articles were included if they used AI for
screening and diagnosis of skin cancer using data sets consisting of dermoscopic images or photographs of
gross lesions.

Results: Fifty-one articles were included, and 41% of these had dermatologists included as authors.
Articles that included dermatologists described algorithms built with more images versus articles that did
not include dermatologists (mean, 12,111 vs 660 images, respectively). In terms of underlying technology,
AI used for skin cancer assessment has followed trends in the field of image recognition.

Limitations: This review focused on models described in the medical literature and did not account for
those described elsewhere.

Conclusions: Greater involvement of dermatologists is needed in thinking through issues in data

collection, data set biases, and applications of technology. Dermatologists can provide access to
large, diverse data sets that are increasingly important for building these models. ( J Am Acad Dermatol
https://doi.org/10.1016/j.jaad.2020.01.028.)

Key words: artificial intelligence; computer vision; machine learning; melanoma; nevi; pigmented lesions;
skin cancer screening.

O ver the past decade, we have seen a sharp
increase in the use of artificial intelligence
(AI). This rise has been bolstered by
an explosion in the generation of digital data,
substantial advances in computing hardware, and
innovation in algorithm design.1 AI now underlies a
plethora of tasks we perform on a daily basis and is
becoming an increasingly crucial part of the
conveniences of modern life. These technologies
have also begun shaping the global economy and
industries around the world; AI has become a part of
crucial functions in the fields of finance, engineering,

From the Ronald O. Perelman Department of Dermatology, New
York University School of Medicinea; and Wayne State Univer-
sity School of Medicine, Detroit.b
Funding sources: None.
Conflicts of interest: None disclosed.
Accepted for publication January 11, 2020.
Correspondence to: Roger S. Ho, MD, MS, MPH, The Ronald O.
Perelman Department of Dermatology, New York University
School of Medicine, 240 E 38th St, Floor 12, New York, NY
10016. E-mail: Roger.Ho@nyumc.org.
Published online February 26, 2020.
0190-9622/$36.00
Ó 2020 by the American Academy of Dermatology, Inc.
https://doi.org/10.1016/j.jaad.2020.01.028

1
2 Zakhem et al J AM ACAD DERMATOL
n 2020

marketing, and entertainment and has now begun to the full texts of 83 articles were assessed, of
find its way into medicine. which 51 were included in the final analysis (Fig 1
Computer visionda subset of AI that involves and Table I3-52). Overall, 41% of the included articles
computer recognition of imagesdin dermatology had dermatologists included as an author. Articles
has garnered substantial attention over the last that included dermatologists as authors described
2 years, as deep learning has allowed AI to approach algorithms built with a larger average image data set
dermatologist-level classification of skin cancer.2 versus articles that did not include dermatologists
However, the use of these (mean, 12,112 vs 660 images;
models in dermatology has standard deviation, 823 vs
been explored for decades. CAPSULE SUMMARY
28,187; median, 426 vs 304
In this review, we aim to images; interquartile range,
characterize the evolution of d
Artificial intelligence for evaluation of 199 to 866 vs 172.5 to 801,
the use of AI in skin cancer skin cancer has been an emerging topic respectively) (Fig 2). The
screening and assessment in dermatology; however, dermatologist number of publications,
and to characterize the involvement in designing and most common algorithms,
involvement of dermatolo- implementing these technologies has and relative distribution of
gists in developing these not been studied. articles including and not
models. d Dermatologists are underrepresented in including dermatologists
publications describing these among the authors were re-
METHODS technologies. Increased involvement and corded for each year (Fig 3).
An electronic literature leadership of dermatologists are Of the 12 convolutional neu-
search was performed on paramount to designing clinically ral networks identified in the
February 19, 2019, with the relevant and efficacious models. literature, 8 used transfer
PubMed database. Articles learning; the most common
published from inception pretrained models used were
through February 19, 2019, were included in the versions of Google’s Inception architecture (Google,
search. Search terms included machine learning or Mountain View, CA) (n = 3) and Microsoft’s ResNet
artificial intelligence combined with skin cancer or (n = 3). Forty five percent of the articles identified
melanoma. were published over the last 3 years.
Study selection was performed based on inclusion Of the publications identified, 70% did not list a
criteria consisting of (1) full-length trials, (2) use of dermatologist as an author. Of the 45 articles that
machine learning for screening and diagnosis of skin described the disease composition of the image set
cancer, and (3) use of data sets consisting of used for algorithm training, 70% reported a base rate
dermatoscopic images or photographs of gross of disease images of at least 30%, and 26% reported a
lesions. Studies were excluded on the following base rate of at least 50%. Of the 51 articles included,
criteria: (1) duplicate publications, (2) reviews, (3) 38 used data sets of dermoscopic images to train the
abstracts or presentations, (4) no use of artificial described algorithms, 12 used data sets of gross
intelligence, (5) use of imaging modalities other than images, and 1 used a combination of both. The
dermoscopy and gross photography, (6) use of country of origin for each author of the included
histopathologic images, (7) no diagnosis of lesions, articles was also recorded (Fig 4).
(8) not published in English, and (9) use of only
animal models. DISCUSSION
Titles and abstracts were assessed on these criteria Our findings suggest that machine learning
by 2 independent blinded reviewers (GAZ and JWF). algorithms for skin cancer assessment have followed
If the title or abstract did not include enough general trends in the field of image recognition. The
information to apply exclusion criteria, the full text trends we observed reflected technology advances
was assessed. The reviewers compared results, and in the field, as well as growing interest in and
any discrepancies were resolved by arbitration by adoption of these algorithms.
the senior author (RSH). The full text of the In terms of the technologies used to build the
remaining articles was reviewed in detail to algorithms, there appeared to be 3 eras of
determine eligibility based on the inclusion criteria. development in the literature. From 1994 to 2011,
algorithms were built primarily by using classical
RESULTS machine learning, such as logistic regressions,
During the initial search, 463 articles were Bayesian classifiers, and simple feedforward neural
reviewed. After the inclusion criteria were applied, networks. These algorithms were typically built with
J AM ACAD DERMATOL
VOLUME jj, NUMBER j
Abbreviation used:
AI: artificial intelligence
data sets of fewer than 1000 images, required human
feature engineering based most commonly on the
ABCDEs of melanoma, and involved painstaking
image preprocessing before model training. From
2011 to 2016, algorithms were built most commonly
by using support vector machines, which are more
sophisticated, nonlinear classifiers. These systems
typically had automated yet human-engineered
feature extraction and were trained with more
images, yet data sets still seldom consisted of more
than 1000 photos.
Over the last 2 years, AI in skin cancer assessment
has been radically affected by the revolution in deep
learning. Since 2017, there has been a sharp rise in
the number of publications describing new models
in the literature (Fig 3), and the majority have been
built using convolutional neural networksdmodels
consisting of several layers of simple algorithms that
extract and assess features automatically to classify
Fig 1. Search methodology and results of the literature search on artificial intelligence in skin
cancer screening.
Zakhem et al 3
input images.53 However, this increase in publica-
tions has been driven largely by teams of
nondermatologists.
This finding is of particular concern because,
based on the typical composition of the data sets
used, the models described are designed for use by
dermatologists. The majority of data sets used to train
and test these models have a disease base rate
between 30% and 50%, which would be appropriate
for a dermatologist using the model to assess
higher-risk lesions. However, this same base rate, if
used in the primary care setting or a patient-facing
app, could result in high rates of false positive
screening results. Moreover, the majority of models
are built with dermoscopic images; dermoscopes are
much less likely to be available to a primary care
practitioner or patient than to a dermatologist.
Although decision support for dermatology is a
viable use case for these technologies, it should be
dermatologists, rather than data scientists, who
define how this technology is used in the clinical
setting. A robust understanding of the clinical
context is absolutely essential for effective
implementation of these technologies.
Table I. Publications identified through systematic review of the literature

4 Zakhem et al
Transfer
Dermatologists learning
included as Base Image type used for
authors? Images in rate of used to convolutional
Country of origin (number of data set, disease, train the neural network
Authors Year of authors dermatologists) n % algorithm Classification task Algorithm architecture Reference
3
Ercal et al 1994 United States No 326 56.13 Gross Classifies images as Feedforward artificial N/A
melanoma vs benign neural network
melanocytic lesions
4
Rubegni et al 2002 Italy Yes (4) 147 61.22 Dermoscopic Classifies images as Simple single-layer N/A
melanoma vs atypical perceptron artificial
nevus neural network
5
Stanley et al 2003 United States Yes (1) 258 50 Gross Classifies images as Fuzzy-logic color N/A
melanoma vs benign histogram analysis
lesion technique
6
Maglogiannis 2004 Greece No 17 41.18 Gross Classifies images as Support vector machine N/A
et al melanoma vs atypical
nevus
7
Maglogiannis 2005 Greece No 34 41.18 Gross Classifies images as Feedforward neural N/A
et al melanoma vs atypical network
nevus
8
Stoecker et al 2005 United States Yes (3) 512 32.23 Dermoscopic Classifies images as Neural network N/A
melanoma vs benign
melanocytic lesions
9
Stanley et al 2007 United States Yes (1) 226 50 Dermoscopic Classifies images as Relative color histogram N/A
melanoma vs atypical
nevus
10
Surowka et al 2007 Poland No 39 48.72 Dermoscopic Classifies images as Neural network N/A
melanoma vs atypical (multilayer
nevus perceptron-based
binary classifier),
support vector
machine, attributional
calculus
11

J AM ACAD DERMATOL
Iyatomi et al 2008 Japan, United States Yes (4) 199 15.08 Dermoscopic Classifies images as Linear classifier N/A
melanoma vs benign
melanocytic lesions
12
Ruiz et al 2008 Spain No 110 d Dermoscopic Classifies images as Bayesian classifier and N/A
melanoma vs benign multilayer perceptron
melanocytic lesions artificial neural

n 2020
network
 VOLUME jj, NUMBER j
J AM ACAD DERMATOL
13
Khan et al 2009 Australia, Italy, Yes (5) 424 31.6 Dermoscopic Classifies images as Feedforward artificial N/A
United States melanoma vs benign neural network
melanocytic lesions
14
Gilmore et al 2010 Austria, Australia Yes (3) 199 50.75 Dermoscopic Classifies images as Support vector machine N/A
melanoma vs atypical
nevus
15
Surowka et al 2010 Poland No 158 49.37 Dermoscopic Classifies images as Attributional calculus N/A
melanoma vs atypical
nevus
16
Tenenhaus 2010 France No 180 d Dermoscopic Classifies images as Kernel logisticdpartial N/A
et al melanoma vs benign least-squares
melanocytic lesions multinomial
regression
17
Cheng et al 2012 United States Yes (2) 489 53.78 Dermoscopic Classifies images as BCC 3-layer multilayer N/A
vs benign lesion perceptron neural
network, direct neural
dynamic
programming
18
Garnavi et al 2012 Australia No 289 39.44 Dermoscopic Classifies images as Support vector machine, N/A
melanoma vs benign random forest, logistic
melanocytic lesions model tree, hidden
naive Bayes
19
Liu et al 2012 United Kingdom No 351 25.07 Dermoscopic Classifies images as Support vector machine N/A
melanoma vs benign
melanocytic lesions
20
Chang et al 2013 Taiwan Yes (3) 769 22.62 Gross Classifies images as 1 of Support vector machines N/A
19 distinct diagnoses
(14 benign, 5
malignant)
21
Cheng et al 2013 United States Yes (4) 114 30.7 Dermoscopic Classifies images as BCC Backpropagation neural N/A
vs benign lesion network
22
Liu et al 2013 United Kingdom No 307 73.62 Dermoscopic Classifies images as Support vector machine N/A
melanoma vs atypical
nevus
23
Lingala et al 2014 United States Yes (6) 866 19.98 Dermoscopic Classifies images as Support vector machines N/A
melanoma vs benign

Zakhem et al 5
melanocytic lesions
24
Riaz et al 2014 Portugal, Pakistan No 200 20 Dermoscopic Classifies images as Support vector machine N/A
melanoma vs benign
melanocytic lesions
Continued
Table I. Cont’d

6 Zakhem et al
Transfer
Dermatologists learning
included as Base Image type used for
authors? Images in rate of used to convolutional
Country of origin (number of data set, disease, train the neural network
Authors Year of authors dermatologists) n % algorithm Classification task Algorithm architecture Reference
25
Sabouri et al 2014 New Zealand, No 412 42.48 Gross Classifies images as Cascade classifier of 2 N/A
Sweden melanoma vs benign support vector
melanocytic lesions machines
26
Souza and 2014 Brazil No 762 74.67 Gross Classifies images as Paraconsistent artificial N/A
Abe melanoma vs benign neural network
melanocytic lesions
27
Ferris et al 2015 United States Yes (3) 426 50 Dermoscopic Classifies images as Decision forest (1000 N/A
malignant (including decision trees)
melanoma and NMSC)
vs benign lesions
28
Kaur et al 2015 United States Yes (1) 192 59.38 Dermoscopic Classifies images as Logistic regression N/A
melanoma vs benign
melanocytic lesions
29
Rastgoo et al 2015 Spain, France No 180 50 Dermoscopic Classifies images as Support vector machine, N/A
melanoma vs atypical gradient boosting,
nevus random forest
30
Shimizu et al 2015 Japan, United Yes (1) 964 28.22 Dermoscopic Classifies images as Layered linear classifiers N/A
States melanoma, benign
nevus, BCC, or SK
31
Jafari et al 2016 Iran, United States No 170 41.18 Gross Classifies images as Support vector machines N/A
melanoma vs benign
melanocytic lesions
32
Jaworek- 2016 Poland No 300 20 Dermoscopic Classifies images as Support vector machines N/A
Korjakowska melanoma or nevi,
et al recognizing 3
histotypes of benign
nevi
33
Jaworek- 2016 Poland No 200 35 Dermoscopic Classifies images as Support vector machines N/A

J AM ACAD DERMATOL
Korjakowska micromelanoma vs
benign nevus
34
Kefel et al 2016 Algeria, United Yes (5) 1378 29.32 Dermoscopic Classifies images as BCC Logistic regression N/A
States vs benign lesion
35
Nasr-Esfahani 2016 Iran, United States No 170 41.12 Gross Classifies images as Convolutional neural No transfer
et al melanoma vs benign network learning

n 2020
melanocytic lesions
 VOLUME jj, NUMBER j
J AM ACAD DERMATOL
36
Premaladha 2016 India No 992 d Dermoscopic Classifies images as Deep learningebased No transfer
et al melanoma vs benign neural network, learning
lesion support vector
machine, hybrid
Adaboost
37
Sabbaghi 2016 Australia No 814 21.38 Dermoscopic Classifies images as Stacked sparse encoder No transfer
et al melanoma vs atypical of bag of features learning
nevus
2
Esteva et al 2017 United States Yes (3) 129,405 d Gross and Classifies images as 1 of Convolutional neural Inception
dermoscopic 757 distinct disease network version 3
classes composed of architecture
2032 diagnoses
38
Gareau et al 2017 United States Yes (5) 120 50 Dermoscopic Classifies images as Q-learning, 13 machine N/A
melanoma vs atypical learning algorithms
nevus
39
Munia et al 2017 United States No 170 1.12 Gross Classifies images as Support vector machine, N/A
melanoma vs benign k-nearest neighbor,
melanocytic lesions decision tree, random
forest
40
Xie et al 2017 China, United No 600 33.33 Dermoscopic Classifies images as Neural network N/A
States melanoma vs benign ensemble of fuzzy
lesion neural networks
41
Yu et al 2017 China No 1279 29.63 Dermoscopic Classifies images as Fully convolutional Unnamed
melanoma vs benign residual network architecture
lesion (segmentation)/very trained on
deep residual network ImageNet
(classification)
42
Gautam et al 2018 India No 294 46.94 Gross Classifies images as Random forest (best of 4 N/A
melanoma vs benign different types)
melanocytic lesions
43
Gilmore et al 2018 Australia Yes (1) 250 66 Dermoscopic Classifies images as Support vector machine, N/A
melanoma vs atypical random forest, linear
nevus regression
Haenssle et al 2018 France, Germany, Yes (9) [100,000 20 Dermoscopic Classifies images as Convolutional neural Inception 44

United States melanoma or nevus, network version 4

recognizing 8 architecture

Zakhem et al 7
histotypes of
melanoma and 12
histotypes of nevus
Continued
Table I. Cont’d

8 Zakhem et al
Transfer
Dermatologists learning
included as Base Image type used for
authors? Images in rate of used to convolutional
Country of origin (number of data set, disease, train the neural network
Authors Year of authors dermatologists) n % algorithm Classification task Algorithm architecture Reference
45
Han et al 2018 Korea Yes (4) 16,684 d Gross Classifies images as 1 of Convolutional neural Microsoft
12 distinct diagnoses network ResNet-152
(8 benign and 4
malignant)
46
Li et al 2018 China No 2600 18.88 Dermoscopic Classifies images as Convolutional neural No transfer
melanoma vs benign network learning
melanocytic lesions
47
Mahbod et al 2018 Austria, United No 2787 20.02 Dermoscopic Classifies images as Convolutional neural AlexNet,
Kingdom, Sweden melanoma, benign network VGGNet,
nevus, or SK ResNet
48
Rebouças Filho 2018 Brazil, India No 3100 d Dermoscopic Classifies images as Multilayer perceptron N/A
et al melanoma vs benign
lesion
49
Wahba et al 2018 Egypt, United States No 1200 50 Dermoscopic Classifies images as Support vector machine, N/A
melanoma, BCC, multilayer perceptron
nevus, or benign
keratocytic lesion
50
Walker et al 2018 Israel, United States No 482 50 Dermoscopic Classifies images as Convolutional neural Inception
malignant (including network version 2
melanoma and NMSC) architecture
vs benign lesions
51
Yu et al 2018 Korea, United States Yes (4) 724 48.34 Dermoscopic Classifies images as Convolutional neural Unnamed
melanoma vs benign network architecture
melanocytic lesions trained on
ImageNet
52
Yu et al 2018 China, United States No 1279 19.39 Dermoscopic Classifies images as Convolutional neural ResNet
melanoma vs benign network
lesion

J AM ACAD DERMATOL
BCC, Basal cell carcinoma; N/A, not applicable; NMSC, nonmelanoma skin cancer; SK, seborrheic keratosis.

n 2020
J AM ACAD DERMATOL
VOLUME jj, NUMBER j
Fig 2. Comparison of image data set size in publications with and without dermatologist
involvement.
Dermatologist input is also paramount to improve
model efficacy for other reasons, notably the
procurement and validation of larger and more
diverse data sets of images with which to train
and validate models. In our review, we found
that the models described in publications with
dermatologist authorship were built with a higher
average number of images. Moreover, the 3
publications in the literature describing models built
with tens of thousands of images involved extensive
collaboration with dermatology departments.
The number of images used in model training has
become one of the most important factors in
developing the latest generation of deep learning
models. Whereas the performance of classic
machine learning models plateaus after a threshold
number of images, deep learning performance scales
efficiently with the amount of input data (Fig 5).54
Currently, the largest open-source data set contains
Zakhem et al 9
only 10,015 standardized clinical images of skin
lesions,55 which is likely too small to build a reliable
decision-support system. Moreover, this data set
appears to contain exclusively images taken of
light-skinned patients, such that a model trained on
this data set would likely be less effective for patients
of color. If models trained with biased data sets are
eventually used in clinical practice, existing outcome
disparities56,57 could be exacerbated58; furthermore,
it could be difficult for dermatologists to recognize
the cause of this phenomenon without active
involvement in procuring training data sets and
developing these models.
In the field at large, advances in computing
hardware, increased generation of digital data, and
algorithm improvements have been the biggest
drivers of the deep learning revolution1; however,
in dermatology, the increased availability and
improved usability of tools to build deep learning
10 Zakhem et al J AM ACAD DERMATOL
n 2020

Fig 3. Number of computer vision publications in dermatology and with dermatologist

involvement by year. CML, Classical machine learning; CNN, convolutional neural network;
SVM, support vector machine.

Fig 4. Normalized global distribution of dermatology computer vision authorship. This figure
shows the number of authors per country normalized to country population.64 In terms of
absolute numbers, the most authors were affiliated with institutions in the United States
(117 authors), followed by China and South Korea (13 authors each), the United Kingdom
(12 authors), and Australia (10 authors).

models may have been the more relevant catalyst for
this new era in computer vision. For example, open
source tools, such as Google’s Tensorflow and
Inception architecture,59,60 have greatly simplified
the coding necessary to build these models. In
fact, performing machine learning will become
increasingly routine as the next generation of
machine learning products are used to develop
models without any coding from the user.61-63
Instead, the skill sets necessary to build practical
applications of this technology are shifting; critically
thinking through issues in data collection, potential
J AM ACAD DERMATOL
VOLUME jj, NUMBER j
Fig 5. Comparison of the performance and size of data sets
among deep learning and classical machine learning
models. With classical machine learning algorithms, such
as logistic regressions, decisions trees, and simple
feedforward artificial neural networks, model performance
eventually plateaus as the amount of data used in training
increases. With deep learning, performance scales efficiently
with more data. For this reason, access to larger data sets of
imagesdand therefore involvement of dermatologists and
dermatology departmentsdhas become increasingly impor-
tant to train artificial intelligence for skin cancer screening.
data set biases, model assessment, and applications
of the technology are ever more relevant as building
the back end becomes increasingly routine. Under
the leadership of dermatologists, these tools can be
designed to improve patient outcomes, root out
health care disparities, bridge the gap between a
shortage of dermatologists and an abundance of
patients, assist in the allocation of other limited
resources, and allow increased patient traffic in our
practices. Now, more than ever, our guidance is
necessary in the development of these technologies
and in defining how they should fit into the practice
ecosystem.
Limitations
This review focused only on models described in
the medical literature and did not account for
those described in other sources, such as GitHub
or Kaggle. This may have resulted in an
underestimation of the number of models developed
by nondermatologists; however, we believed that
the models most likely to reach the clinical setting
would be more likely to be described in the medical
literature. Furthermore, we could not account for
models that were built without publication of the
results, such as those developed for hackathons or
by private companies. Again, we believed that those
models built by private companies for clinical use
Zakhem et al 11
would likely be described in the literature when fully
developed as part of a potential US Food and Drug
Administration approval process or to raise clinician
awareness. In addition, coauthorship is not a
completely reliable surrogate in determining the
extent of dermatologist involvement. Meaningful
participation in these projectsdincluding extensive
involvement in designing studies, curating and
validating data, and determining the optimal clinical
setting for usedis difficult to assess, and a reliable
surrogate for this has not been described. However,
we believe that having no dermatologist among the
authors is an extremely reliable surrogate for a lack
of meaningful involvement by dermatologists.
We were unable to compare and assess model
efficacy in this review, largely because of a lack of
homogeneity in reporting metrics and target
variables of the models. We observed a lack of
standardization in the reporting of accuracy metrics,
and we believed we were unable to assess potential
distortion of model efficacy caused by overfitting or
data leakage. We were unable to compare clinical
outcomes, because nearly all models were tested
exclusively with retrospective clinical images. We
encourage the development of guidelines on metric
reporting to facilitate the assessment of future
publications in this area.
REFERENCES
1. Brynjolfsson EM, Andrew. What’s driving the machine learning
explosion? Harvard Business Rev. 2017;18:3-11.
2. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level
classification of skin cancer with deep neural networks. Nature.
2017;542(7639):115-118.
3. Ercal F, Chawla A, Stoecker WV, Lee HC, Moss RH. Neural
network diagnosis of malignant melanoma from color images.
IEEE Trans Biomed Eng. 1994;41(9):837-845.
4. Rubegni P, Burroni M, Cevenini G, et al. Digital dermoscopy
analysis and artificial neural network for the differentiation of
clinically atypical pigmented skin lesions: a retrospective
study. J Invest Dermatol. 2002;119(2):471-474.
5. Stanley RJ, Moss RH, Van SW, Aggarwal C. A fuzzy-based
histogram analysis technique for skin lesion discrimination in
dermatology clinical images. Comput Med Imaging Graph.
2003;27(5):387-396.
6. Maglogiannis IG, Zafiropoulos EP. Characterization of digital
medical images utilizing support vector machines. BMC Med
Inform Decis Mak. 2004;4:4.
7. Maglogiannis I, Pavlopoulos S, Koutsouris D. An integrated
computer supported acquisition, handling, and characteriza-
tion system for pigmented skin lesions in dermatological
images. IEEE Trans Inf Technol Biomed. 2005;9(1):86-98.
8. Stoecker WV, Gupta K, Stanley RJ, Moss RH, Shrestha B.
Detection of asymmetric blotches (asymmetric structureless
areas) in dermoscopy images of malignant melanoma using
relative color. Skin Res Technol. 2005;11(3):179-184.
9. Stanley RJ, Stoecker WV, Moss RH. A relative color
approach to color discrimination for malignant melanoma
detection in dermoscopy images. Skin Res Technol. 2007;
13(1):62-72.
12 Zakhem et al
10. Surowka G, Grzesiak-Kopec K. Different learning paradigms for
the classification of melanoid skin lesions using wavelets. In:
Proceedings of the 29th Annual IEEE Engineering in Medicine
and Biology Society Conference. Piscataway, NJ: IEEE; 2007:
3136-3139.
11. Iyatomi H, Celebi M, Oka H, Tanaka M. An Internet-based
melanoma screening system with acral volar lesion support.
In: Proceedings of the Annual International Conference of
the IEEE Engineering in Medicine and Biology Society. Piscat-
away, NJ: IEEE; 2008:5156-5159.
12. Ruiz D, Berenguer VJ, Soriano A, Martin J. A cooperative
approach for the diagnosis of the melanoma. In: Proceedings
of the Annual International Conference of the IEEE Engineering in
Medicine and Biology Society. Piscataway, NJ: IEEE; 2008. 2008:
5144-5147.
13. Khan A, Gupta K, Stanley RJ, et al. Fuzzy logic techniques for
blotch feature evaluation in dermoscopy images. Comput Med
Imaging Graph. 2009;33(1):50-57.
14. Gilmore S, Hofmann-Wellenhof R, Soyer HP. A support vector
machine for decision support in melanoma recognition. Exp
Dermatol. 2010;19(9):830-835.
15. Surowka G. Symbolic learning supporting early diagnosis of
melanoma. In: Proceedings of the Annual International
Conference of the IEEE Engineering in Medicine and Biology
Society. Piscataway, NJ: IEEE; 2010. 2010:4104-4107.
16. Tenenhaus A, Nkengne A, Horn JF, Serruys C, Giron A, Fertil B.
Detection of melanoma from dermoscopic images of naevi
acquired under uncontrolled conditions. Skin Res Technol.
2010;16(1):85-97.
17. Cheng B, Stanley RJ, Stoecker WV, Hinton K. Automatic
telangiectasia analysis in dermoscopy images using adaptive
critic design. Skin Res Technol. 2012;18(4):389-396.
18. Garnavi R, Aldeen M, Bailey J. Computer-aided diagnosis of
melanoma using border and wavelet-based texture analysis.
IEEE Trans Inf Technol Biomed. 2012;16(6):1239-1252.
19. Liu Z, Sun J, Smith L, Smith M, Warr R. Distribution
quantification on dermoscopy images for computer-assisted
diagnosis of cutaneous melanomas. Med Biol Eng Comput.
2012;50(5):503-513.
20. Chang WY, Huang A, Yang CY, et al. Computer-aided diagnosis
of skin lesions using conventional digital photography: a
reliability and feasibility study. PLoS One. 2013;8(11):e76212.
21. Cheng B, Joe SR, Stoecker WV, et al. Automatic dirt trail
analysis in dermoscopy images. Skin Res Technol. 2013;19(1):
e20-e26.
22. Liu Z, Sun J, Smith M, Smith L, Warr R. Incorporating clinical
metadata with digital image features for automated
identification of cutaneous melanoma. Br J Dermatol. 2013;
169(5):1034-1040.
23. Lingala M, Stanley RJ, Rader RK, et al. Fuzzy logic color
detection: blue areas in melanoma dermoscopy images.
Comput Med Imaging Graph. 2014;38(5):403-410.
24. Riaz F, Hassan A, Javed MY, Tavares Coimbra M. Detecting
melanoma in dermoscopy images using scale adaptive local
binary patterns. In: Proceedings of the Annual International
Conference of the IEEE Engineering in Medicine and Biology
Society. Piscataway, NJ: IEEE; 2014. 2014:6758-6761.
25. Sabouri P, GholamHosseini H, Larsson T, Collins J. A cascade
classifier for diagnosis of melanoma in clinical images. In:
Proceedings of the Annual International Conference of the
IEEE Engineering in Medicine and Biology Society. Piscataway,
NJ: IEEE; 2014. 2014:6748-6751.
26. Souza S, Abe JM. Nevus and melanoma paraconsistent
classification. Stud Health Technol Inform. 2014;207:244-250.
J AM ACAD DERMATOL
n 2020
27. Ferris LK, Harkes JA, Gilbert B, et al. Computer-aided
classification of melanocytic lesions using dermoscopic
images. J Am Acad Dermatol. 2015;73(5):769-776.
28. Kaur R, Albano PP, Cole JG, et al. Real-time supervised
detection of pink areas in dermoscopic images of melanoma:
importance of color shades, texture and location. Skin Res
Technol. 2015;21(4):466-473.
29. Rastgoo M, Garcia R, Morel O, Marzani F. Automatic
differentiation of melanoma from dysplastic nevi. Comput
Med Imaging Graph. 2015;43:44-52.
30. Shimizu K, Iyatomi H, Celebi ME, Norton KA, Tanaka M.
Four-class classification of skin lesions with task
decomposition strategy. IEEE Trans Biomed Eng. 2015;62(1):
274-283.
31. Jafari MH, Samavi S, Karimi N, Soroushmehr SM, Ward K,
Najarian K. Automatic detection of melanoma using broad
extraction of features from digital images. In: Proceedings of
the Annual International Conference of the IEEE Engineering in
Medicine and Biology Society. Piscataway, NJ: IEEE; 2016. 2016:
1357-1360.
32. Jaworek-Korjakowska J. Computer-aided diagnosis of
micro-malignant melanoma lesions applying support vector
machines. Biomed Res Int. 2016;2016:4381972.
33. Jaworek-Korjakowska J, Kleczek P. Automatic classification of
specific melanocytic lesions using artificial intelligence.
Biomed Res Int. 2016;2016:8934242.
34. Kefel S, Pelin KS, LeAnder RW, et al. Adaptable texture-based
segmentation by variance and intensity for automatic
detection of semitranslucent and pink blush areas in basal
cell carcinoma. Skin Res Technol. 2016;22(4):412-422.
35. Nasr-Esfahani E, Samavi S, Karimi N, et al. Melanoma detection
by analysis of clinical images using convolutional neural
network. In: Proceedings of the Annual International Conference
of the IEEE Engineering in Medicine and Biology Society. Piscat-
away, NJ: IEEE; 2016. 2016:1373-1376.
36. Premaladha J, Ravichandran KS. Novel approaches for
diagnosing melanoma skin lesions through supervised and
deep learning algorithms. J Med Syst. 2016;40(4):96.
37. Sabbaghi S, Aldeen M, Garnavi R. A deep bag-of-features
model for the classification of melanomas in dermoscopy
images. In: Proceedings of the Annual International Conference
of the IEEE Engineering in Medicine and Biology Society. Piscat-
away, NJ: IEEE; 2016. 2016:1369-1372.
38. Gareau DS, Correa dRJ, Yagerman S, et al. Digital imaging
biomarkers feed machine learning for melanoma screening.
Exp Dermatol. 2017;26(7):615-618.
39. Munia TTK, Alam MN, Neubert J, Fazel-Rezai R. Automatic
diagnosis of melanoma using linear and nonlinear features
from digital image. In: Proceedings of the Annual International
Conference of the IEEE Engineering in Medicine and Biology
Society. Piscataway, NJ: IEEE; 2017. 2017:4281-4284.
40. Xie F, Fan H, Li Y, Jiang Z, Meng R, Bovik A. Melanoma
classification on dermoscopy images using a neural network
ensemble model. IEEE Trans Med Imaging. 2017;36(3):849-858.
41. Yu L, Chen H, Dou Q, Qin J, Heng PA. Automated melanoma
recognition in dermoscopy images via very deep residual
networks. IEEE Trans Med Imaging. 2017;36(4):994-1004.
42. Gautam D, Ahmed M, Meena YK, Ul HA. Machine
learning-based diagnosis of melanoma using macro images.
Int J Numer Method Biomed Eng. 2018;34(5):e2953.
43. Gilmore SJ. Automated decision support in melanocytic lesion
management. PLoS One. 2018;13(9):e0203459.
44. Haenssle HA, Fink C, Schneiderbauer R, et al. Man against
machine: diagnostic performance of a deep learning
J AM ACAD DERMATOL
VOLUME jj, NUMBER j
convolutional neural network for dermoscopic melanoma
recognition in comparison to 58 dermatologists. Ann Oncol.
2018;29(8):1836-1842.
45. Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE.
Classification of the clinical images for benign and malignant
cutaneous tumors using a deep learning algorithm. J Invest
Dermatol. 2018;138(7):1529-1538.
46. Li Y, Shen L. Skin lesion analysis towards melanoma
detection using deep learning network. Sensors (Basel). 2018;
18(2):E556.
47. Mahbod A, Schaefer G, Ellinger I, Ecker R, Pitiot A, Wang C.
Fusing fine-tuned deep features for skin lesion classification.
Comput Med Imaging Graph. 2019;71:19-29.
48. Rebouças Filho PP, Peixoto SA, Medeiros da Nobrega RV, et al.
Automatic histologically-closer classification of skin lesions.
Comput Med Imaging Graph. 2018;68:40-54.
49. Wahba MA, Ashour AS, Guo Y, Napoleon SA, Elnaby MMA. A
novel cumulative level difference mean based GLDM and
modified ABCD features ranked using eigenvector centrality
approach for four skin lesion types classification. Comput
Methods Programs Biomed. 2018;165:163-174.
50. Walker BN, Rehg JM, Kalra A, et al. Dermoscopy
diagnosis of cancerous lesions utilizing dual deep
learning algorithms via visual and audio (sonification) outputs:
laboratory and prospective observational studies.
EBioMedicine. 2019;40:176-183.
51. Yu C, Yang S, Kim W, et al. Acral melanoma detection using a
convolutional neural network for dermoscopy images. PLoS
One. 2018;13(3):e0193321.
52. Yu Z, Jiang X, Zhou F, et al. Melanoma recognition in
dermoscopy images via aggregated deep convolutional
features. IEEE Trans Biomed Eng. 2019;66(4):1006-1016.
53. Zakhem GA, Motosko CC, Ho RS. How should artificial
intelligence screen for skin cancer and deliver
diagnostic predictions to patients? JAMA Dermatol. 2018;
154(12):1383-1384.
Zakhem et al 13
54. Ng A. What data scientists should know about deep learning.
YouTube. Available at: https://www.youtube.com/watch?v=
NKpuX_yzdYs; 2015. Accessed April 25, 2019.
55. Tschandl P, Rosendahl C, Kittler H. The HAM10000 dataset, a
large collection of multi-source dermatoscopic images of
common pigmented skin lesions. Sci Data. 2018;5:180161.
56. Cormier JN, Xing Y, Ding M, et al. Ethnic differences among
patients with cutaneous melanoma. Arch Intern Med. 2006;
166(17):1907-1914.
57. Ward-Peterson M, Acuna JM, Alkhalifah MK, et al. Association
between race/ethnicity and survival of melanoma patients in
the united states over 3 decades: a secondary analysis of SEER
data. Medicine. 2016;95(17):e3315.
58. Adamson AS, Smith A. Machine learning and health care disparities
in dermatology. JAMA Dermatol. 2018;154(11):1247-1248.
59. Abadi M, Barham P, Chen J, et al. TensorFlow: a system for
large-scale machine learning. In: Google Brain, ed. Proceedings
of the 12th USENIX conference on Operating Systems Design and
Implementation. Savannah, GA. November 2-4, 2016.
60. Szegedy C, Sergey I, Vanhoucke V. Inception-v4, Inception-
ResNet and the impact of residual connections on learning.
Paper presented at: The Thirteenth Association for the Advance-
ment of Artificial Intelligence (AAAI) Conference. Phoenix, AZ.
February 12-17, 2016.
61. Boyd E. Making AI real for every developer and every orga-
nization. Available at: https://azure.microsoft.com/en-us/blog/
making-ai-real-for-every-developer-and-every-organization.
Accessed September 16, 2019.
62. DataRobot. Enabling the AI-driven enterprise with automated
machine learning; 2019. Available at: https://www.datarobot.
com. Accessed February 7, 2020.
63. Google. Smart business analytics and AI. https://cloud.google.
com/solutions/#smart-business-analytics-ai; 2019. Accessed
September 16, 2019.
64. The CIA World Factbook. Washington, DC: Central Intelligence
Agency; 2019.