JID: PATREC

ARTICLE IN PRESS [m5G;November 29, 2017;20:57]

Pattern Recognition Letters 0 0 0 (2017) 1–9

Contents lists available at ScienceDirect

Pattern Recognition Letters

journal homepage: www.elsevier.com/locate/patrec

Experiments using deep learning for dermoscopy image analysis

Cristina Nader Vasconcelos a,∗, Bárbara Nader Vasconcelos b
a
Departamento de Ciência da Computação, Instituto de Computação, Universidade Federal Fluminense, Niterói, 24210-346, Brazil
b
Serviço de Dermatologia, Hospital Universitário Pedro Ernesto (Hupe), Universidade Estadual do Rio de Janeiro, Rio de Janeiro 20, 551-030, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: Skin cancer is a major public health problem, as is the most common type of cancer and represents more
Available online xxx than half of cancer diagnosed worldwide. Early detection influences the outcome of the disease and mo-
tivates the research presented in this paper. Recent results show that deep learning based approaches
MSC:
41A05 learn from data, and can outperform human specialists in a set of tasks when large databases are avail-
41A10 able for training. This research investigates the scenario where the amount of data available for training is
65D05 small. It obtains relevant results for the ISBI 2016 melanoma classification challenge (named Skin Lesion
65D17 Analysis for Melanoma Detection) facing the peculiarities of dealing with such a small and unbalanced
biological database. To do this, it explores committees of Deep Convolutional Neural Networks (DCNN),
Keywords:
the augmentation of the training data set by image processing classical transforms and by deformations
Skin lesion classification
guided by expert knowledge about the lesion axis, and it introduces a third class aiming to improve the
Dermoscopy image analysis
Lesion dermoscopic feature extraction and classifiers’ distinction of the region of interest of the lesion. The experiments show that the proposed ap-
classification proach improves the final classifier invariance for common melanoma variations, common skin patterns
Convolutional neural networks and markers, and dermatoscope capturing conditions.
Data augmentation
© 2017 Elsevier B.V. All rights reserved.
Deep learning

1. Introduction The access to digital images of skin lesions annotated with

Skin cancer is the most common type of cancer and accounts
for more than half of all cancer diagnoses. There are two basic
types of skin cancer, non-melanoma and melanoma, much rarer,
however, a much more serious disease. Melanoma is a malignancy
with predominance in white adults in the trunk of men or in the
women lower limbs - but they also appear in other body parts.
Although more common in fair-skinned people, blacks and their
descendants are not free of the disease. Lens and Dawes [14] ob-
serve that the incidence and overall mortality rates have increased
in the last decades, therefore, it represents a substantial public
health problem. Up to one-fifth of patients develop metastatic dis-
ease, which may even lead to death.
Fortunately, the prognosis of patients can be considered good
when melanoma is detected in the early stages. Early detection and
proper excision lead to a cure rate of more than 90% in patients
with low-risk melanoma. Nestle FO [21] pointed out that innova-
tive early detection programs in combination with improved diag-
nostic tools and new immunologic and molecular target treatments
for advanced stages of the disease may influence the outcome of
the disease in the future.
∗
Corresponding author.
E-mail address: crisnv@ic.uff.br (C.N. Vasconcelos).
metadata can be used to educate professionals in melanoma recog-
nition, as well as to directly aid the diagnosis of melanoma through
teledermatology, clinical decision assistance, and support the de-
velopment of automated (or semi-automated) diagnosis tools. In
this direction, dermatoscopy proved to be valuable in visualizing
the morphological structures in pigmented lesions for the diag-
nosis of melanoma. It inspires the development of various Com-
puter Vision techniques for the diagnosis of skin cancer such as
the works of Scharcanski and Celebi [27] and also Masood and Al-
Jumaily [17] to name a few.
More recently, a great effort is being made as part of the “Inter-
national Skin Imaging Collaboration: Melanoma Project” (ISIC [23])
to establish an archive annotated by experts to serve as a public
resource of images for teaching and as a standard for the develop-
ment of automated diagnostic systems. Currently, the ISIC Archive
contains around 13,0 0 0 images associated with clinical metadata
and expert annotations.
In 2016, in order push the development of dermoscopy image
analysis tools for automated diagnosis of melanoma evaluated over
a common standard, the ISIC project organized a challenge dur-
ing the IEEE International Symposium on Biomedical Imaging (ISBI
2016) named Skin Lesion Analysis Towards Melanoma Detection
[8]. The challenge provides 900 annotated images as training data
for participants to engage in three components of lesion image
analysis: lesion segmentation; detection and localization of fea-

https://doi.org/10.1016/j.patrec.2017.11.005
0167-8655/© 2017 Elsevier B.V. All rights reserved.

Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
JID: PATREC
ARTICLE IN PRESS
2 C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
tures; and disease classification (which is the focus of the present
work). It also provides a separate test dataset containing 379 im-
ages so that the results of different methods can be compared.
This work investigates the use of Deep Learning techniques,
more specifically, Deep Convolutional Neural Networks (DCNNs)
trained on the ISBI challenge data set, for the problem of
melanoma classification from dermoscopic images of skin lesions.
CNNs are able to learn hierarchies of invariant features and clas-
sifiers from annotated datasets. They currently represent the state
of the art solution for classification problems over natural images.
That is the case of the Imagenet Large Scale Visual Recognition an-
nual Challenge (ILSVRC), organized by Russakovsky et al. [26], for
object localization/detection and image/scene classification from
images and videos at large scale since the 2012 edition.
The development of biomedical images classifiers has its own
research peculiarities. One of the most critical difficulties for the
direct application of the Deep Learning techniques to biomedical
problems is the fact that public databases with annotations are
considerably smaller than those of natural images. It is well known
that CNNs usually require a large amount of training data in or-
der to avoid overfitting. While the ISBI challenge database used for
training the proposed model contains 900 images, the ILSVRC 2012
participants received 10 million annotated images. Such a huge dif-
ference in the amount of publicly available annotated data drives
the experimental hypotheses raised by the present work that are
listed below.
1.1. Hypotheses
This work investigates how far a well known CNN topology
(GoogLeNet [30]) can be trained on the ISBI 2016 melanoma clas-
sification challenge data set since it is expected that such complex
models suffer from severe overfitting when the number of train-
ing samples available is too small. The present work adopted the
restriction of not using external sources of images. Although the
challenge rules do not impose such a restriction, it allows the ob-
servation of the proposed methods effects in the training of deep
neural networks using small data sets. It investigates the following
hypotheses:
(H1) The training of a deep classifier on biomedical images,
more specifically skin lesions, can be benefited by the use of pre-
trained models trained in a huge set of natural images;
(H2) CNN performance is affected by unbalanced training data
sets, but it is possible to reduce the side effect of the existing nat-
ural imbalance on biomedical datasets by artificially balancing the
number of benign and malignant samples;
(H3) It is possible to improve the classifier performance by in-
troducing a new and independent class containing visual patterns
obtained from the original training samples, but not directly re-
lated to the malignant or benign prognosis of the lesion;
(H4) It is possible to increase the invariance of the CNN classi-
fier to image capturing conditions by augmenting the training data
set with classical image processing operations exploring geometric
and color transforms;
(H5) It is possible to increase the invariance of the CNN classi-
fier to biological patterns by augmenting the training data set with
warped images preserving the characteristics of the classes accord-
ing to specialized knowledge (dermatologist);
(H6) It is possible to improve the overall classification perfor-
mance by creating committees composed of a set of CNNs trained
with different variations of the database;
1.2. Contributions
The presented methodology is based on a known CNN topology
and investigates how a small and unbalanced biomedical dataset
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
can be used to train such a model. It explores: (I) data augmen-
tation using both image processing classical transforms and based
on specialist knowledge; (II) transfer learning, so that CNN weights
are initialized with those obtained from a huge dataset of natu-
ral images; (III) the introduction of an additional class represent-
ing the visual patterns of regions outside the lesion to reduce their
influence on the classification decision; (IV) the analysis of the im-
pact of the data set unbalance and a method to reduce its effect
on the classifier decision; (V) diversity in the composition of com-
mittees;
The results show that it is possible to surpass the average pre-
cision obtained by the participants of the ISBI 2016 melanoma
classification challenge by exploring the hypotheses presented in
Section 1.1 with the presented methodology.
1.3. Organization
This paper is organized as follows. Section 2 presents related
work on skin lesions analysis and the use of deep learning for
biomedical image analysis. Section 3 presents the neural networks
investigated, while Section 4 details the methodology proposed to
train them using a small and unbalanced data set for skin lesion
classification. Section 5 presents the experiments and the results.
Finally, Section 6 presents the conclusions.
2. Related work
There are several methods to the analysis of biomedical im-
ages that explicitly model features that describe visual patterns,
guided by expert knowledge. They are usually based on Computer
Vision pipelines that contain three main steps: an image prepro-
cessing module responsible for image enhancements, such as noise
removal and illumination calibration; a module responsible for the
extraction, selection and description of features that is guided by
specialized knowledge and responsible for encoding the natural in-
variance of the problem; and, a final module, during which Ma-
chine Learning techniques are applied to train a classifier on the
adopted descriptors, where Support Vector Machines (SVM) is one
of the most popular choices.
The analysis of digital images of skin lesions has been investi-
gated for a long time in the literature. Moss et al. [19] proposed
an expert system based on the analysis of texture frequencies and
features obtained by the Fourier transform. In the work of Chang
et al. [3], a typical pipeline is presented where initially the im-
age is cropped and preprocessed; next, 91 image features are ex-
tracted (covering lesion shape, texture, and color); and finally, fea-
tures obtained in a labeled dataset are used to train an SVM. Many
other studies investigate the skin lesions classification under dif-
ferent Computer Vision concepts for feature engineering. Two de-
tailed reviews of such approaches can be consulted in the works
of Scharcanski and Celebi [27] and also of Masood and Al-Jumaily
[17].
The development of a classifier based on Computer Vision
pipelines can be very complex and time-consuming. The feature
engineering requires domain-specific knowledge that may not be
known in advance. It also requires the development of models that
are robust to existing diversity, caused for instance by capturing
conditions or natural intra-class variations.
Recent enthusiasm for deep learning is driven by its character-
istic of providing meta-solutions in which domain-specific knowl-
edge is learned from data. In the case of DCNNs, they are able to
learn from labeled images how to adapt to the application domain
in an integrated manner what formerly represented the image pro-
cessing, feature extraction, and classifier modules. The good perfor-
mance of DCNNs, however, is usually associated with the availabil-
ity of a large training database. DCNNs are very suitable to overfit-
JID: PATREC
ARTICLE IN PRESS
C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
ting when trained from scratch over a small dataset, such as the
ISBI melanoma classification challenge investigated in this work.
The following paragraphs describe the methods found in the lit-
erature to overcome such difficulties that inspire some of the hy-
potheses raised in this study.
Transfer Learning is a set of strategies commonly used to pre-
vent overfitting that exchange knowledge between a source and a
target domain in order to overcome the deficit of training sam-
ples. Different experiments indicate that the visual features pro-
duced by CNNs are very powerful and generic so that they can be
transferred from one context (where more training data is avail-
able) to another. Raina et al. [24] pointed out that pre-training a
CNN in a different dataset can be adopted in supervised and un-
supervised training (ie, on both labeled and unlabeled data sets).
In Razavian et al. [25] work, the authors performed a series of
experiments on the features obtained by a CNN trained for ob-
ject classification on the ILSVRC data set. Extracted features were
used as a generic image representation for the object image classi-
fication, scene recognition, finely grained recognition, attribute de-
tection, and image retrieval tasks applied to a diverse set of data
sets. Their results strongly suggest that the CNN features should be
the primary candidate in most visual recognition tasks. Following
their findings, the present work investigates if such hypothesis is
valid for the skin lesion analysis problem. Additional discussions
on Transfer Learning techniques and performance can be found in
the work of Pan and Yang [22] and Yosinski et al. [31].
Another commonly adopted approach in Machine Learning to
overcome training sample deficits and to improve the classifier
generalization ability is to produce new samples from the original
training set with a data augmentation scheme. In biomedical prob-
lems, increasing training data is crucial to deal with problems aris-
ing from the existence of rare events and with unbalanced or small
data sets. That is the case of the work of Ciresan et al. [5] that took
the first place in the 2012 Mitosis Detection in Breast Cancer His-
tological Images Challenge with a 13-layer CNN architecture. They
created additional training samples performing arbitrary rotations
and mirroring. As detailed in the following sections, this work in-
vestigates the use of even more ostensible data augmentation, en-
compassing geometric and color transformations, as well as image
warps, guided by the skin lesion geometry.
In a different line of investigation, which they call convergent
learning, the work of Li et al. [15] investigates whether DNNs
trained separately learn features that converge to span similar
spaces. One of their findings is of particularly interesting for the
methodology proposed in this work. They note that some features
are learned across multiple networks, but other features are not
consistently learned. Moreover, combining results from multiple
networks in a committee is expected to increase classifier perfor-
mance by smoothing out the effects of non-optimal solutions. In
2010, Ciresan et al. [6] work reduced the error in the MNIST hand-
writing recognition from 0.40% to 0.27% by combining seven CNNs.
Proposals that led the ILSVRC championship in its most recent edi-
tions [10,13,30] also used committees.
The present work investigates Li et al. [15] findings by cloning
each DCNNs training five times. The present methodology also
raises the hypothesis that a better classifier can be obtained by
combining different classifiers, and extends the common use of
the committees. It investigates the combination of CNNs trained
in different partitions of the dataset and also using different data
augmentation schemes suggesting that these compositions benefit
from unique features and greater diversity.
Another topic in the focus of this work is the fact that training
with unbalanced data sets may severely hinder the performance of
a number of classifiers such as Decision Trees, SVMs, Multi-Layer
Perceptrons (MLPs) and Boosting classifiers [9,11]. They all suffer
from poor generalization, especially in the classification of minor-
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
3
ity classes. Regarding expected behavior for CNN-based classifiers
(which are a special case of MLP), this is not a topic explored in
most large-scale image classification challenges, such as CIFAR-10
and CIFAR-100 [12] and ILSVC [26]. These classification challenges
deal with cases where the number of samples available for training
is balanced (or near equilibrium) among existing classes. Despite
this, there are many situations where the number of training sam-
ples per class is imbalanced. This work investigates how the un-
balanced distribution influences a classifier based on a CNN, more
specifically the distribution of the skin lesion benign and malig-
nant labels.
In order to develop the proposed methodology based on ma-
chine learning, it was necessary to choose a skin lesion database
for training it. There are several skin lesion databases, but most of
the publicly available have less than a thousand images, making it
difficult to train a CNN from scratch using them. They can be dis-
tinguished according to the type of image adopted. The two most
commonly used types are clinical images (obtained by conven-
tional photographs) and dermatoscopic images (obtained through
the use of a dermatoscope).
The use of clinical images recently received a lot of attention
due to the opportunity of developing low-cost diagnostic applica-
tions for massive screening using personal cameras, such as smart-
phones. Its classification is extensively explored by classical meth-
ods, such as Chang et al. [3] work which argues that conventional
digital photographs are feasible to provide useful information for
CADx applications. In parallel to the present research, Esteva et al.
[7] obtained a dermatologist-level performance on the classifica-
tion of skin lesions using a single DCNN, trained using an anno-
tated dataset of 129,450 clinical images. The database used is not
publicly available, but their results attest the potential of using a
CNN based approach capable of classifying skin cancer with a level
of competence comparable to dermatologists.
The use of a dermatoscope facilitates the melanoma diagnosis.
It allows a clear inspection of lesions, as it cancels the skin sur-
face reflections. In addition, since they have a magnifier (typically
x10), allow the visualization of clinical dermoscopic features (such
as Milia-like Cysts, streaks and pigment networks), which are rec-
ognized by dermatologists and inform their decision to biopsy sus-
picious skin lesions.
The ISBI Challenge for melanoma classification uses dermato-
scopic images. They are publicly available and provide a common
criterion for comparing different methodologies for lesion classi-
fication. Gutman et al. [8] published the numerical results of the
ISBI challenge, but did not include the description of the methods
that produced the presented statistics. As far as the authors of this
article are aware, those methods with public descriptions to date
are detailed below.
Majtner et al. [16] propose an SVM ensemble over the combina-
tion of hand-crafted features (RSurf and Local Binary Patterns) with
deep learning features obtained from training an AlexnNet [12] on
the ISBI dataset.
Menegola et al. [18] trained a VGGnet [29] using transfer learn-
ing (the training was initialized with weights from the ImageNet
dataset) and data augmentation using classical geometric transfor-
mations (scaling, rotations, and flipping). An extra source of im-
ages was used by Menegola et al. [18] to train and test the tar-
get models, containing both clinical and dermatoscopic images for
more than a thousand cases.
The challenge team called CUMED [32] focused on training a
deeper network. They obtained the highest accuracy on the chal-
lenge submission date using the CNN architecture considered cur-
rent state of the art for image classification: a Residual Network
[10]. They propose a framework where the lesion is initially pre-
segmented, so that a cropped version of the image is used for clas-
sification.
JID: PATREC
ARTICLE IN PRESS
4 C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
Although the cited works are also based on DCNNs, the use
of transfer learning and data augmentation by geometric trans-
forms, the present work investigates additional strategies to im-
prove the classifier performance. Compared to the present work,
none of them directly addressed the database imbalance nor did
them use specialized knowledge and color transforms in data aug-
mentation strategies nor did them investigate the influence of vi-
sual patterns from outside of the skin lesion nor did then address
the diversification of classifiers composing committees. The pro-
posed methodology is presented next.
3. Convolutional neural networks (CNNs)
CNNs are biologically inspired variants of MultiLayer Percep-
trons (MLP) specially designed for learning visual features. Unlike
Computer Vision pipelines, they learn domain-specific features au-
tomatically from training image data sets, without the need for
feature engineering. The CNN architecture is characterized by alter-
nating convolutional with pooling layers and by the organization of
their neurons in a grid.
A convolutional layer implements a set of linear filters and
learns its weights from training data. Each of its neurons observes
a small region at the corresponding input, simulating a local recep-
tive field. A neuron shares its weights with the neighbors of the
same depth of the grid so that together they form a certain convo-
lution filter. In the vicinity along the depth dimension, it observes
the same receptive field as its neighbors, but they encode different
filters.
Pooling layers are responsible for downsampling along the spa-
tial dimensions of the feature maps grids, adding local transla-
tion invariance to the classifier. Together with nonlinear activation
functions, they add nonlinearity to the overall network decision
function. They usually encode a fixed transformation that does not
depend on training (ie. do not increase the number of learning pa-
rameters).
In CNN original architectures, fully-connected layers are added
to learn a classifier over the features hierarchy. As the name im-
plies, in these layers each neuron is connected to all the neurons
in the previous layer, just as in classic MLPs.
The development of a new CNN architecture is not the objective
of this work, but rather to evaluate the extent to which CNN based
solutions can be applied to the melanoma classification problem,
given the ISBI data set peculiarities. A well-known topology, named
GoogLeNet [30], was chosen to implement the proposed methodol-
ogy in order to clarify the contributions in a replicable framework.
It is described next.
3.1. GoogLeNet
GoogLeNet is the ILSVRC 2014 image classification winner de-
veloped by Szegedy et al. [30]. Their main contribution is the In-
ception Module that drastically reduced the number of parameters,
creating a deeper and wider topology. The input signal entering an
inception module is processed by filters of different sizes (includ-
ing 1 × 1 filters which are responsible for aggregating correlated
features), whereas in previous networks the convolution kernels
were typically uniform within a layer.
GoogLeNet is formed with Inception modules stacked on the
top of each other, leading to a 22-layer deep model when only
taking into account layers with learning parameters. Along with
the convolution layers, it also has nonlinear transformations such
as ReLU and pooling layers. Differing from the previous CNNs, the
GoogLeNet architecture replaces the fully connected layers with
sparse ones and spread the classifier decision across the network
(even within the convolutions).
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
In order to keep the network deep and combat the vanish-
ing gradient problem, they include during the training two auxil-
iary output branches connected to the intermediate layers of the
network. The loss is added to the total net loss weighted by a
smaller weight. The auxiliary layers are discarded during the in-
ference time.
4. Methodology
This section describes the foundations behind the experimental
hypotheses raised by this work.
4.1. Transfer learning
A well-known technique to prevent overfitting in CNN classi-
fiers is to initially train them with a large auxiliary dataset and
then transfer the obtained knowledge to the target domain. More
specifically, the weights learned from this initial training are used
as the seed for the parameters in the second round of training.
This time the training is performed in the target database, aiming
to fine-tune the solution to its real purpose.
The transfer learning success depends primarily on the size
and variety of the auxiliary dataset and its similarity to the tar-
get dataset. The experiments evaluate fine-tuning starting from the
weights obtained by training the GoogLeNet in the ILSVC data set
for natural objects classification. This choice is motivated by the
size and extraordinary diversity of images in this database.
The weights from the ILSVC are expected to be very generic in
the early layers of the CNN as they define basic visual features such
as oriented edges and color contrast filters. The open question is
whether the CNN-based skin lesion classifier benefits from them
since a biological data set may or may not be similar enough to
natural images in order to share features (or to ease their learning),
especially in the middle and upper layers.
4.2. Imbalanced dataset
Studies on machine learning from unbalanced data sets state
that their side effect to the classifier should be considered in re-
lation to the complexity and nature of the specific task. [9,11,20].
Even a large imbalance in the number of samples available per
class may not cause problems in cases of well-separated classes
or when there is a large set of training data.
The present case study does not meet these conditions. The
training set is small. It contains 900 images distributed in 727 of
benign lesions and 173 of malignant lesions (approximately 4:1).
The present research assumes that it is not a problem of easily sep-
arated classes based on the methodology adopted by specialized
analysis (dermatologists). They presume the existence of cases that
can not be diagnosed solely by the observation of images of the
lesion (neither classical nor dermoscopy). The diagnosis of these
cases occurs after the biopsy of the lesions. For these reasons, the
proposed method investigates the impact of the imbalanced train-
ing set on the CNN classifier and also how to minimize this effect.
Existing methods to reduce the effect of an imbalanced data set
operate at both the algorithm level and the training data set [4,20].
In the first case, the learning algorithms include a misclassification
weight associated with each class, adjust the decision threshold,
or even modify the existing algorithm to be more sensitive to rare
classes.
This proposal follows the second group of methodologies, based
on the balancing of the training set by some sampling strategy. Bal-
ance through a strategy based on majority-class sub-sampling (i.e.
discarding majority-class samples) is not appropriate for our case
study since the total number of samples available is already small
for training a deep CNN. Hence, the proposed method investigates
JID: PATREC
ARTICLE IN PRESS
C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
Fig. 1. Cropping maintaining lesion aspect ratio and border integrity.
two strategies of oversampling of the minority class: based on the
addition of replicas of their samples, and based on the artificial
synthesis of new samples.
4.3. Data augmentation by image processing techniques
This section and the next (4.4) describe how new training sam-
ples were synthesized. They were included in the training data set
for multiple purpose: to deal with the data set imbalance (as justi-
fied above); to increase the classifier invariance to common visual
patterns observed in dermatoscopy; to avoid overfitting in training
a DCNN model; and to increase the diversity among trained classi-
fiers.
4.3.1. Geometric transformations
The classifier should not alter its results according to the ori-
entation, scale and position of the lesion within the image. Thus,
copies of each training image were rotated (by multiples of 90°),
flipped and cropped to create new examples associated with the
same label as the original sample.
Random cropping is very common in CNN training, as it is ex-
pected to increase the invariance to image translations and scal-
ing. Unlike the free cut used over natural images, the proposed
method enforces the rule that the entire lesion must be pre-
served in the new image produced as lesion boundaries are an-
alyzed by the decision criteria of dermatologists. Therefore, the
random crop is applied in a controlled manner, respecting a re-
stricted area. The lesion bounding box is obtained by processing its
segmentation mask (available per training image in the ISBI data
set). A fixed margin is added around the bounding box to define
the protected area (Fig. 1). Finally, the lesion original aspect ratio
is enforced, as its proportion is also used by specialists in their
melanoma/nonmelanoma analysis.
4.3.2. Color transformations
Augmenting the color variety of an image data set can be
achieved by different techniques. However, in biological data sets,
it is critical that the synthesized colors correspond to plausible in-
stances of the subject portrayed.
Dermoscopic images present a bias in their color distribution,
covering a very specific subset of the color spectrum. It corre-
sponds to the biological variations of skin and lesions observed
in relation to the color distribution of the light source. The ISBI
database images in particular contains two lighting patterns corre-
sponding to whitish and bluish dermatoscope lights.
A color synthesis based on the main axes of the database color
distribution was applied to follow its bias and create plausible im-
ages. They were obtained by Principal Component Analysis (PCA).
The new images were produced by adding vectors to the origi-
nal image colors. The vectors were obtained as the principal com-
ponents weighted by their corresponding eigenvalues and multi-
plied by a random factor. The random factors were sampled from
a Gaussian distribution with mean μ = 0.0 and standard deviation
σ = 0.2. This σ was found by validating the appearance of the syn-
thesized images with specialists, who described images obtained
with higher values of σ as unnatural. Samples created are illus-
trated in Fig. 2.
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
5
Fig. 2. PCA-based color processing and validated by specialists.
Fig. 3. Distortion of the original image keeping the main axes of the lesion.
4.4. Data augmentation based on specialists knowledge
The analysis of skin lesions by specialists is done by observing
the symmetries and patterns around their main orthogonal axes.
There are no requirements for domain specific prior knowledge
nor hand-crafted features in order to produce CNN-based classi-
fiers, as they learn directly from training data. However, specialized
knowledge can be used to augment the available training sam-
ples while preserving the annotation. Consequently, the proposed
method preserves the symmetries of lesions (or anti-symmetries)
in the artificial images produced, since they imply in the final clas-
sification of the lesion.
An additional data augmentation scheme has been included to
synthesize variations of lesions usually found in different individu-
als, or with the development of the lesion. It is based on the dis-
tortion of the lesion main axis sizes, preserving its symmetries. To
obtain the lesion axes the segmentation mask is processed so that
the lesion region is approximated with an ellipse (Fig. 3 (b)). The
main axes of the ellipse are used to control the image distortion
applied by increasing or reducing their sizes by a random factor
while their directions are kept.
A warped image is created by employing a thin plate spline
mapping [28] between nine points of the original image in posi-
tions in the new image. The four corners of the image are held
fixed and included as anchor points. The center of the ellipse ap-
proximates the lesion center and is translated at random. The four
end points of the ellipse axes are shifted following their original
axes by random factors, altering the axes sizes (positively or nega-
tively) but maintaining their original directions (Fig. 3).
4.5. Visual patterns from outside the lesion
There are visual patterns in skin images that are not directly re-
lated to the classification of the lesion but represent natural vari-
ations in the area surrounding the lesion. Some of these occur as
rare cases or even as a single sample in the training data set, but
are recurrent in dermatoscopic images. Fig. 4 illustrates some of
these patterns found in the ISBI dataset presenting variations of
the image and dermatoscope borders, colored stickers, hair, pen
markers and rulers. Neither of which is directly associated with
the malignant or benign prognosis, therefore, should not influence
the classifier’s decision.
The flexibility of deep CNNs allows them to learn observed pat-
terns as inherent characteristics of classes. Patterns that are not
directly related to skin lesion, but which do not occur in a dis-
tributed manner among different classes, may introduce an unde-
sired bias in the learned classifier.
JID: PATREC
ARTICLE IN PRESS
6 C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
Fig. 4. Visual patterns that should not interfere in theprognostic.
Fig. 5. Extra class representing out-of-lesion patterns.
One approach to overcome this effect following the CV pipelines
would be to clean the image in a preprocessing step during both
training and testing. It deals with each pattern modeling it indi-
vidually (for instance, the remotion of visible hair). Another ap-
proach would be to synthesize new images of both labels contain-
ing the out-of-lesion most common patterns. An example would be
the use of hair simulation to amplify the originally existing varia-
tions in the base and produce new samples associated with both
types of labels (malignant and benign). In practice, however, the
syntheses of those patterns are not trivial, considering the need for
a high degree of realism. Neither approaches attended to the pro-
posed methodology requirements of simplicity and easy extension
for new cases. Therefore, another approach is proposed.
An extra class was included and associated with a new, inde-
pendent label. The new class was populated with random crops
from regions outside the lesions captured from both benign and
malignant samples (Fig. 5). For this, the crops were obtained
within the area which is complementary to the lesion segmenta-
tion mask, maintaining the aspect ratio of the original image. The
goal is to force a distinction between patterns that should be as-
sociated with the melanoma prognosis from unrelated ones by re-
designing the problem as a ternary classifier. Thus, it contains im-
ages of healthy skin, rulers, hair, among other patterns found in
the training base, which are similar to the originals, except for the
lack of the lesion itself.
4.6. Committees
Training a DCNN using a small dataset is especially affected by
a dilemma called bias - variance tradeoff [2], that prevents super-
vised learning algorithms from generalizing beyond their training
set. It can be seen as the choice between very flexible models with
low bias and high variance (overfitting), and relatively rigid models
with high bias and low variance (underfitting). Ideally, the tradeoff
is solved by increasing the number of training samples to infinity.
As this is not possible in practice, the methodology proposed in-
creases the number and diversity of training samples using data
augmentation (as presented in previous sections), but also deals
with the bias-variance dilemma by the formation of committees.
In Machine Learning, a committee is responsible for combin-
ing the responses of multiple models (in this case, multiple CNNs)
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
trained for a given problem into a single response. A better re-
sponse is expected from a committee than from its constituent
experts assuming that the model’s individual errors are average
out. The use of committees can be seen as a resolution of the
bias-variance dilemma when applied to networks that individually
present low bias and high variance, but combined produce a low
bias and low variance model.
The proposed committees were built in three steps. First, N
models were designed, including the CNN topology definition, ini-
tialization choices and training data set. Then each model was
trained separately corresponding to a single response. Finally, the
responses of the models were combined into a single one. Differ-
ent strategies have been investigated for each step as detailed in
Section 5.6.
5. Experiments and results
All the experiments performed adopted the same CNN topol-
ogy (from GoogLeNet [30]) and fixed hyper-parameters, in order to
focus exclusively on the improvements obtained by the proposed
method. They were trained for a maximum number of epochs
ranging from 30 to 50 according to the training database size and
using a learning rate fixed at 10−4 .
The experiments presented in this section followed a 5-fold
cross validation. They started with a random partition of the ISBI
training set into five folders of 180 images each, maintaining the
melanoma/non-melanoma ratio of ≈ 0.24. Each model was trained
with images of four of these folders. The fifth is used for valida-
tion, so that its statistics were used to choose the models weights.
The five combinations are identified by the corresponding valida-
tion folder number (fi with i = 0, . . . , 4).
Although accuracy is the measure most commonly adopted in
the CNN literature, it is highly affected by the unbalance of the
data set. It can hide what is known as the accuracy paradox, where
a high accuracy rate only reflects the distribution of the underlying
class. This means that it is very likely to predict the most popular
class, regardless of the data it is asked to classify. For this reason,
each trained CNN was used to produce two models by choosing
the set of weights corresponding respectively to the epoch with
the minimum loss and with the maximum accuracy in the valida-
tion set. The box charts included in this section present a compar-
ison between the two.
The analysis of a single experiment covering each hypothesis
investigated is unreliable as CNNs performance is affected by ran-
dom initialization of its weights and other factors (such as the or-
der in which samples are presented during training). Besides, the
final solution can be benefited by diversity as Li et al. [15] found
that some features are learned across multiple networks, but other
features are not. With these assumptions, five replicas of each
model were trained using the same hyperparameters and input
data. Thus, a total of 50 models were evaluated for each hypoth-
esis investigated to portray a more stable analysis of the investi-
gated events. The performance fluctuation between the 25 differ-
ent replicas of a model is represented in the columns of the box
charts presented below.
5.1. Performance measurements
The performance measurements were taken from 379 unseen
images (75 melanomas and 304 non-melanoma) from the ISBI test
set. They were not used during training nor for validation (i.e. for
any parameter choice or model adjustment) for a realistic estimate
of the generalization error of the proposed method.
The box charts present average precision measurements since
the ISBI challenge participants were ranked based on it alone. Even
JID: PATREC
ARTICLE IN PRESS
C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
Fig. 6. First convolutional layer learned filters.
Fig. 7. Average Precision of the classifiers trained over imbalanced × balanced
datasets.
though not used as criteria of the challenge, in practice, the sensi-
tivity and specificity are of special interest for the specialized com-
munity as represent respectively an estimate of the probability of
a positive test (melanoma) given that the patient has the disease,
and the probability of a negative test (non-melanoma) given that
the patient is well. Thus, results are compared with related works
using average precision, accuracy, sensitivity and specificity mea-
surements.
5.2. Experiments of the transfer learning hypothesis (H1)
The first experimental hypothesis investigates the adoption of
a fine tuning procedure. The best model obtained when training a
CNN from scratch got an accuracy of 83%, but this is a fallacious
number. The observation of its accuracy by class shows a strong
tendency for the benign class (98% × 24%). Some trained models
have reached an accuracy of 100% for the benign class against only
4% to malignant.
The numbers described hide a number of questions raised by
this work, but to clarify the advantage of using a pre-trained set
of weights, Fig. 6 presents a comparison of their first convolu-
tional layer filters. As shown, features learned from scratch do
not converge to semantic patterns using such a small data set,
but are visualized as random noise as opposed to those obtained
by fine-tuning. Consequently, subsequent experiments started with
GoogLeNet’s public weights pre-calculated over the ILSVC 2014
data set. This fine-tuning reduced the overfitting and training time
observed in all realized experiments.
5.3. Experiments of the data set balancing hypothesis (H2)
Fig. 7 presents the average precision of CNNs trained with the
original training data set against an oversampling strategy based
on the inclusion of identical copies of the malignant class sam-
ples. All models have improved their sensitivity and most of them
have improved their average precision with such a simple strategy.
These results encouraged the adoption of a balanced training data
set in the subsequent experiments.
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
7
Fig. 8. Average Precision distributions of the binary × the ternary classifier.
Fig. 9. Average Precision distributions of the proposed data augmentations.
5.4. Experiments of the independent class hypothesis (H3)
Fig. 8 presents the average precision of CNNs trained on the
original classes against CNNs designed with a third class represent-
ing the surroundings of the lesions. The number of test samples
mistakenly associated with the third class introduced was negli-
gible, which indicated that the classifier could identify such an
artificial class extremely well. Thus, a binary output is obtained
from the ternary classifier by normalizing the benign and malig-
nant scores by their sum. Improvements results were observed in
almost all models and partitions evaluated.
5.5. Experiments of the data augmentation hypotheses (H4,H5)
The box charts of Fig. 9 present the average precision of ternary
classifiers, in which the malignant class had their samples over-
sampled by the data augmentation schemes proposed. They corre-
spond to data augmentation experiments using the image geome-
try, color and lesion axis distortion and a fourth experiment with
a random mix of the three. In all the experiments, the weights ob-
tained as the training epoch with minimum loss in the validation
set induced a better generalization.
The augmentation by lesion axis distortion induced the highest
observed performance gains followed by the one based on color.
This should not be interpreted simply as if one of the proposed
augmentation methods is significantly better than the others. By
analyzing the classification of the validation images individually, it
is possible to observe that the networks established different clas-
sification partitions as if they had actually observed different fea-
tures and decision criteria. This result is in line with the proposal
initial design that seeks for diversity.
5.6. Experiments of the committee formation hypothesis (H6)
The analysis of the hypotheses H1 to H5 was used to pre-select
the set of models to be combined in different committees in the
search for the final classifier. That is, the evaluated committees
were composed only of those models obtained by fine-tuning, with
minimum loss in the corresponding validation set, trained in bal-
anced sets, and modeled as ternary classifiers. Fig. 10 shows the
JID: PATREC
ARTICLE IN PRESS [m5G;November 29, 2017;20:57]

8 C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9

Fig. 10. Block diagram of the proposed approach for training the CNNs.

Table 1
Results of the proposed committees versus the first (CUMED) and second
(GTDL) best results of the ISBI challenge and others [8]).

Aver. Prec. Accur. Spec. Sens.

SDAS(geometric) 0.648 0.802 0.819 0.733

SDAS(color) 0.642 0.825 0.858 0.693
SDAS(lesionaxis) 0.629 0.807 0.835 0.693
SDAS (random) 0.628 0.810 0.835 0.706
SDASBM (geometric) 0.661 0.807 0.819 0.760
SDASBM (color) 0.659 0.836 0.865 0.720
SDASBM (lesionaxis) 0.660 0.807 0.842 0.666
SDASBM (random) 0.634 0.791 0.819 0.680
ADAS 0.645 0.817 0.845 0.706
ADAS(withoutRandom) 0.645 0.812 0.838 0.706
ADASBM 0.669 0.825 0.845 0.746
ADASBM (withoutRandom) 0.669 0.810 0.842 0.680
CUMED [32] 0.637 0.855 0.941 0.507
GTDL (unpublished method) 0.619 0.813 0.872 0.573
[16] 0.473 0.826 0.898 0.533
[18] 0.549 0.792 0.881 0.476 Fig. 11. Limitations of the training using only dermatoscopic images: it does not
generalize to the illumination conditions of clinical images, presence of specular
brightness and unconstrained differences in resolution, scale and acquisition equip-
ment.
workflow proposed to train the CNNs that compose the final clas-
sifier. Diversity was introduced in the committees composition by
three training strategies: (A) using the same data set, but varying 5.7. Proposal limitations: experiments with clinical images
CNN initialization and training samples order; (B) using different
partitions of the same dataset; and (C) using different augmen- The ADASBM (with the random augmentations) committee
tation schemes. The following committee compositions were in- trained in the ISBI data set was applied to a set of images ob-
vestigated: (a) gathering all 25 models trained from a single aug- tained from the public DermIS and DermQuest databases by Ame-
mentation scheme (named SDAS); (b) gathering the 5 best models, lard et al. [1] in order to evaluate its performance in clinical image
one from each validation folder partition of a single augmentation analysis. The tests were done by applying the images directly to
scheme (named SDASBM ); (c) gathering all models of all augmenta- CNN classifiers without additional preprocessing steps to be con-
tion schemes (named ADAS), with and without the random combi- sistent with the proposed methodology.
nations database; (d) gathering the best models from each valida- The images in Fig. 11 show some samples in which the com-
tion folder of all augmentation schemes (named ADASBM ), with and mittee trained in the ISBI data set does not generalize well to the
without the random combinations database; The composition rule visual patterns presented in clinical images. These results could be
adopted follows Ciresan et al. [6] approach, where the committee predicted for a number of reasons: by the illumination condition
output is calculated as the average of the individual outputs. variations of the clinical images (both in the direction and in the
Table 1 presents the results of the 12 different committees pro- color spectrum) as opposed to the uniform illumination condition
duced by the proposed methodology, and related works statistics. of the dermatoscopes; due to the fact that in most clinical im-
It shows that the two ADASBM committees proposed obtained the ages, lesions and skin present some level of specular glow occlu-
highest average precision of all of them and that all twelve have a sion, which is usually eliminated by the use of dermoscopy; and
better balance between specificity and sensitivity than the classi- by the unconstrained differences in resolution, scale, and acquisi-
fiers of the ISBI challenge participants. They outperform a ResNet- tion equipment, whose diversity is not covered by the ISBI data set.
based approach [32], recognized for producing better results than Besides, the fact that lesions in the ISBI data set occupy most of
the GoogLenet in a direct comparison between the two, as well as the image area and are of sufficient resolution to observe dermato-
an approach using considerably more training data [18]. scopic features (such as Milia-like Cysts, streaks, and pigment net-

Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
JID: PATREC
ARTICLE IN PRESS
C.N. Vasconcelos, B.N. Vasconcelos / Pattern Recognition Letters 000 (2017) 1–9
works), may have induced the trained networks to look for those
patterns that are not visualized in some of the clinical images eval-
uated.
The potential of a deep learning based approach in the context
of clinical imaging is clear from the results obtained by the work
of Esteva et al. [7]. Thus, the presented limitations do not invali-
date the proposed methodology since it can be replicated without
modifications to train a classifier of clinical images using a suitable
data set, containing its natural variations.
6. Conclusions
Melanoma early detection programs, in combination with im-
proved diagnostic tools, are crucial for the outcome of the disease.
In parallel, Deep Learning approaches are succeeding in several
tasks. The presented methodology investigates in several aspects
how a DCNN based classifier can be employed to deal with a small
and unbalanced biomedical data set.
The contributions presented include data augmentation
schemes, the inclusion of a third class representing out-of-lesion
patterns, the exploration of diversity in committee formation, that
together with the balance of the data set led to a classifier with
the best average precision published so far for the ISBI challenge
in dermatoscopic images classification.
The comparison with the related works strongly validates the
contributions presented since it surpassed those using extra source
of data or a DCNN known for its capacity in achieving better re-
sults (ResNet) than the one adopted by this work (GoogLeNet-v1).
In addition, the proposed methodology led to classifiers with the
best balance between specificity and sensitivity among related ap-
proaches, as it increases the maximum sensitivity published so far
in this dataset from the 57%–76%.
The performance gain obtained is expected to increase further
by replicating the proposal using a ResNet, but it was left as future
work, as well as its extension to clinical images.
Acknowledgments
The authors would like to thank NVidia for the donation of the
Titan X GPU used in this research.
References
[1] R. Amelard, J. Glaister, A. Wong, D. A. Clausi, Melanoma decision support using
lighting-corrected intuitive feature models, pp. 193–219.
[2] C.M. Bishop, Pattern Recognition and Machine Learning (Information Science
and Statistics), Springer-Verlag New York, Inc., 2006.
[3] W.-Y. Chang, A. Huang, C.-Y. Yang, C.-H. Lee, Y.-C. Chen, T.-Y. Wu, G.-S. Chen,
Computer-aided diagnosis of skin lesions using conventional digital photogra-
phy: a reliability and feasibility study, PLOS ONE 8 (11) (2013) 1–9.
[4] N.V. Chawla, Data mining for imbalanced datasets: an overview, in: The Data
Mining and Knowledge Discovery Handbook., 2005, pp. 853–867.
[5] D.C. Ciresan, A. Giusti, L.M. Gambardella, J. Schmidhuber, Mitosis detection in
breast cancer histology images with deep neural networks, in: Medical Image
Computing and Computer-Assisted Intervention - MICCAI 2013 - 16th Interna-
tional Conference, Proceedings, Part II, 2013, pp. 411–418.
[6] D.C. Ciresan, U. Meier, L.M. Gambardella, J. Schmidhuber, Convolutional neural
network committees for handwritten character classification., in: ICDAR, IEEE
Computer Society, 2011, pp. 1135–1139.
Please cite this article as: C.N. Vasconcelos, B.N. Vasconcelos, Experiments using deep learning for dermoscopy image analysis, Pattern
Recognition Letters (2017), https://doi.org/10.1016/j.patrec.2017.11.005
[m5G;November 29, 2017;20:57]
9
[7] A. Esteva, B. Kuprel, R.A. Novoa, J. Ko, S.M. Swetter, H.M. Blau, S. Thrun, Derma-
tologist-level classification of skin cancer with deep neural networks, Nature
542 (7639) (2017) 115–118.
[8] D. Gutman, N.C.F. Codella, E. Celebi, B. Helba, M. Marchetti, N. Mishra,
A. Halpern, Skin lesion analysis toward melanoma detection: a challenge at
the international symposium on biomedical imaging (ISBI) 2016, hosted by the
international skin imaging collaboration (ISIC), CoRR (2016). abs/1605.01397.
[9] H. He, Y. Ma, Imbalanced Learning: Foundations, Algorithms, and Applications,
1st, Wiley-IEEE Press, 2013.
[10] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition,
in: arXiv:1506.01497arXiv prepring arXiv, 2015.
[11] N. Japkowicz, S. Stephen, The class imbalance problem: a systematic study, In-
tell. Data Anal. 6 (5) (2002) 429–449.
[12] A. Krizhevsky, Learning multiple layers of features from tiny images, 2009.
[13] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep con-
volutional neural networks, in: Advances in Neural Information Processing Sys-
tems 25, 2012, pp. 1106–1114.
[14] M. Lens, M. Dawes, Global perspectives of contemporary epidemiological
trends of cutaneous malignant melanoma, Brit. J. Dermatol. 150 (2) (2004)
179–185.
[15] Y. Li, J. Yosinski, J. Clune, H. Lipson, J.E. Hopcroft, Convergent learning:
do different neural networks learn the same representations? CoRR (2015).
abs/1511.07543.
[16] T. Majtner, S.Y. Yayilgan, J.Y. Hardeberg, Combining deep learning and
hand-crafted features for skin lesion classification, in: Sixth International Con-
ference on Image Processing Theory, Tools and Applications, IPTA 2016, 2016,
pp. 1–6.
[17] A. Masood, A.A. Al-Jumaily, Computer aided diagnostic support system for skin
cancer: a review of techniques and algorithms, Int. J. Biomed. Imaging 2013
(2013) 1–22.
[18] A. Menegola, M. Fornaciali, R. Pires, F.V. Bittencourt, S.A.F. de Avila, E. Valle,
Knowledge transfer for melanoma screening with deep learning, in: 14th IEEE
International Symposium on Biomedical Imaging, ISBI 2017, 2017, pp. 297–300.
[19] R. Moss, W.V. Stoecker, S.-J. Lin, S. Muruganandhan, K.-F. Chu, K.M. Poneleit,
C.D. Mitchell, Skin cancer recognition by computer vision, Comput. Med. Imag-
ing Graph. 13 (1989) 31–36.
[20] Y.L. Murphey, H. Guo, L.A. Feldkamp, Neural learning from unbalanced data,
Appl. Intell. 21 (2) (2004) 117–128.
[21] H.A. Nestle FO, Melanoma, in: Dermatology, 2, second edition, 2008,
pp. 1745–1771. Eds. Bolognia JL, Jorizzo JL, Rapini RP.
[22] S.J. Pan, Q. Yang, A survey on transfer learning, IEEE Trans. Knowl. Data Eng.
22 (10) (2010) 1345–1359.
[23] I. project, Public skin lesion image archive, (http://isdis.net/isic-project/
public- image- archive- of- skin- lesions/). Accessed: 2016-09-01.
[24] R. Raina, A. Battle, H. Lee, B. Packer, A.Y. Ng, Self-taught learning: transfer
learning from unlabeled data, in: Proceedings of the 24th International Con-
ference on Machine Learning, ICML’ 07, ACM, 2007, pp. 759–766.
[25] A.S. Razavian, H. Azizpour, J. Sullivan, S. Carlsson, Cnn features off-the-shelf:
an astounding baseline for recognition, in: Proceedings of the 2014 IEEE Con-
ference on Computer Vision and Pattern Recognition Workshops, CVPRW’14,
2014, pp. 512–519.
[26] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpa-
thy, A. Khosla, M. Bernstein, A.C. Berg, L. Fei-Fei, Imagenet large scale visual
recognition challenge, Int. J. Comput. Vision (IJCV) 115 (3) (2015) 211–252.
[27] J. Scharcanski, M.E. Celebi, Computer Vision Techniques for the Diagnosis of
Skin Cancer, Springer Publishing Company, Incorporated, 2013.
[28] R. Sibson, G. Stone, Computation of thin-plate splines, SIAM J. Sci. Stat. Com-
put. 12 (6) (1991) 1304–1313.
[29] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale
image recognition, CoRR (2014). abs/1409.1556.
[30] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S.E. Reed, D. Anguelov, D. Erhan,
V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, CoRR (2014).
abs/1409.4842.
[31] J. Yosinski, J. Clune, Y. Bengio, H. Lipson, How transferable are features in deep
neural networks? in: Advances in Neural Information Processing Systems 27,
2014, pp. 3320–3328.
[32] L. Yu, H. Chen, Q. Dou, J. Qin, P. Heng, Automated melanoma recognition in
dermoscopy images via very deep residual networks, IEEE Trans. Med. Imaging
36 (4) (2017) 994–1004.