Professional Documents
Culture Documents
Thissen 2017
Thissen 2017
Monique Thissen, MD, PhD,1–4,* Andreea Udrea, Eng PhD,5,* Results: On the 108 cases used for evaluation the algorithm
Michelle Hacking, MD,1 Tanja von Braunmuehl, MD, PhD,6 scored 80% sensitivity and 78% specificity in detecting
and Thomas Ruzicka, MD, PhD6 (pre)malignant conditions.
1
Discussion: Although less accurate than the dermatologist’s
Department of Dermatology, Catharina Hospital, Eindhoven, clinical eye, the app may offer support to other professionals
The Netherlands.
2 who are less familiar with differentiating between benign and
Department of Dermatology, Maastricht University Medical
malignant lesions.
Centre+, Maastricht University, Maastricht, The Netherlands.
3
GROW Oncology, Maastricht University, Maastricht, Conclusion: An mHealth application for the risk assessment
The Netherlands. of skin lesions was evaluated. It adds value to diagnosis tools
4
School for Oncology and Developmental Biology, Maastricht of its type by taking into consideration pigmented and non-
University, Maastricht, The Netherlands. pigmented lesions all together and detecting signs of malig-
5
Department of Automatic Control and Systems Engineering, nancy with high sensitivity.
University Politehnica of Bucharest, Bucharest, Romania.
6
Department of Dermatology and Allergology, Ludwig-Maximilian Keywords: automatic skin lesion risk assessment, mHealth
University, Munich, Germany. app, skin cancer, fractal analysis
*These authors are the main contributors to this work.
Introduction
W
ith the advances of mobile technology, an in-
Abstract creasing number of mHealth applications re-
Background: With the advent of smartphone devices, an in- lated to dermatology emerged. A particular
creasing number of mHealth applications that target mela- segment aims for melanoma detection, but
noma identification have been developed, but none addresses there are few clinical studies discussing their accuracy and
the general context of melanoma and nonmelanoma skin this leads to criticism.1,2–5
cancer identification. In this context, this article presents a risk assessment
Introduction: In this study a smartphone application using algorithm for (pre)malignant lesion detection which is inte-
fractal and classical image analysis for the risk assessment grated in SkinVision mHealth application (developed by
of skin lesions is systematically evaluated to determine its Skin Vision B.V., The Netherlands). The app primary targets
sensitivity and specificity in the diagnosis of melanoma and laypersons, but can be also used by nondermatologists.
nonmelanoma skin cancer along with actinic keratosis and The app should be certified by appropriated regulatory
Bowen’s disease. bodies and has been already certified in Europe, New Zeal-
Materials and Methods: In the Department of Dermatology, and, and Australia.
Catharina Hospital Eindhoven, The Netherlands, 341 mela- The algorithm was initially dedicated for the analysis of
nocytic and nonmelanocytic lesions were imaged using melanocytic lesions and melanoma detection. It underwent a
SkinVision app; 239 underwent histopathological examina- clinical study and scored a sensitivity of 73% in detecting
tion, while the rest of 102 lesions were clinically diagnosed as melanoma, while the specificity was 83%.6
clearly benign and not removed. The algorithm has been ca- Now, the algorithm has been adapted and recalibrated for a
librated using the images of the first 233 lesions. The cali- much broader segment of (pre)malignant skin lesion diagno-
brated version of the algorithm was used in a subset of 108 sis. With this study, we assess the sensitivity and specificity
lesions, and the obtained results were compared with the of the recalibrated algorithm in the diagnosis of melanoma
medical findings. and nonmelanoma skin cancer along with in situ melanoma
Table 1. The Pigmented and Nonpigmented Melanocytic and Nonmelanocytic Lesions Collected and Used During the Study
CASES USED FOR THE CASES USED FOR ALGORITHM
CALIBRATION EVALUATION IN TERMS
OF THE ALGORITHM OF SPECIFICITY AND SENSITIVITY
NO. OF NO. OF LESIONS NO. OF NO. OF LESIONS
LESIONS THAT UNDERWENT LESIONS THAT UNDERWENT
NO. LESION TYPE PER TYPE BIOPSY PER TYPE PER TYPE BIOPSY PER TYPE
1 Basal cell carcinoma 94 94 16 16
2 Actinic keratosis 10 10 8 3
4 Bowens disease 9 9 5 5
5 Melanoma 4 4 2 2
6 Melanoma in situ 2 2 1 1
7 Psoriasis 10 5 10 0
8 Eczema 3 1 0 0
10 Histiocytoma 7 6 8 0
11 Folliculitis 10 3 8 0
12 Sebaceous hyperplasia 1 0 3 0
13 Angioma senilis 10 3 6 1
15 Scar 7 1 4 0
16 Verruca vulgaris 1 1 1 1
17 Verruca seborrhoica 12 8 9 1
20 Dysplastic nevus 1 1 1 1
21 Lentigo solaris/senilis 0 0 7 0
and actinic keratosis and Bowen’s disease, as premalignant written informed consent. The study had been approved by the
skin lesions, compared to clinical diagnosis and histopatho- local ethics committee (No. 2014-41).
logical results. Consecutive patients were seen by one dermatologist and
one resident in dermatology. The lesions were selected by the
Materials and Methods dermatologist in case of clinical clear benign lesions or
MATERIALS AND DATA ACQUISITION during total body skin examination in patients with multi-
We included, in total, 341 lesions in 256 consecutive pa- ple skin malignancies in the past and, also, in patients re-
tients seen routinely for different skin problems, including ferred by general practitioner for skin malignancies. The
skin cancer follow-up at the Department of Dermatology, lesions underwent visual and dermatoscopic diagnosis and
Catharina Hospital Eindhoven, The Netherlands in the period in 239 cases either incisional or excisional biopsies were
December 2014–April 2016, after obtaining the patients’ performed, followed by histopathological examination of
Table 3. Results Obtained by The Algorithm Incorporating the Patients’ Answers to the Questionnaire Regarding
the Lesion’s Characteristics Versus Clinical Diagnosis—Eindhoven Database-Test Data Set
MALIGNANT
ALGORITHM OR PREMALIGNANT BENIGN SUM
High risk 28 16 44 Sensitivity (95% CI) 0.8 (0.62–0.90) Positive predictive value (95% CI) 0.63 (0.47–0.77)
Low/medium risk 7 57 64 Specificity (95% CI) 0.78 (0.66–0.86) Negative predictive value (95% CI) 0.89 (0.78–0.95)
Sum 35 73 108
Table 4. Absolute Numbers and Percentage of the Different Subgroups with False Positive and False Negative
Ratings—Eindhoven Database-Test Data Set
ALGORITHM’S RATINGS ALGORITHM’S RATINGS WITHOUT
CONSIDERING THE ANSWERS CONSIDERING THE ANSWERS ALGORITHM’S RATINGS
TO THE QUESTIONNAIRE TO THE QUESTIONNAIRE BEFORE RECALIBRATION
NO. HIGH, MEDIUM, LOW, HIGH, MEDIUM, LOW, HIGH, MEDIUM, LOW,
DIAGNOSES INCLUDED N (%) N (%) N (%) N (%) N (%) N (%) N (%) N (%) N(%)
Basal cell carcinoma 16 11 (69) 4 (25) 1 (6) 9 (56) 6 (38) 1 (6) 3 (19) 11 (69) 2 (12)
Squamous cell carcinoma 3 2 (66) 1 (33) 0 (0) 1 (33) 2 (66) 0 (0) 1 (33) 2 (66) 0 (0)
Actinic keratosis 8 7 (88) 1 (12) 0 (0) 7 (88) 1 (12) 0 (0) 3 (43) 3 (43) 1 (14)
Bowens’ disease 5 5 (100) 0 (0) 0 (0) 5 (100) 0 (0) 0 (0) 1 (20) 3 (60) 1 (20)
Melanoma 2 2 (100) 0 (0) 0 (0) 2 (100) 0 (0) 0 (0) 1 (50) 1 (50) 0 (0)
Melanoma in situ 1 1 (100) 0 (0) 0 (0) 1 (100) 0 (0) 0 (0 0 (0) 0 (0) 1 (100)
Psoriasis 10 0 (0) 0 (0) 10 (100 7 (70) 2 (20) 1 (10) 5 (50) 4 (40) 1 (10)
Eczema 0 — — — — — — — — —
Lichen planus-like keratosis 3 2 (66) 0 (0) 1 (33) 2 (66) 0 (0) 1 (33) 1 (33) 1 (33) 1 (33)
Histiocytoma 8 2 (25) 1 (13) 5 (63) 3 (38) 2 (25) 3 (37) 2 (25) 3 (38) 3 (37)
Folliculitis 8 0 (0) 0 (0) 8 (100) 5 (62) 1 (13) 2 (25) 2 (25) 5 (62) 1 (13)
Sebaceous hyperplasia 3 2 (66) 1 (33) 0 (0) 2 (66) 1 (33) 0 (0) 2 (66) 1 (33) 0 (0)
a
Nevus naevo-cellularis 13 2 (15) 2 (15) 9 (70) 3 (23) 2 (15) 8 (62) 2 (15) 5 (38) 6 (47)
Verruca vulgaris 1 1 (100) 0 (0) 0 (0) 1 (100) 0 (0) 0 (0) 1 (100) 0 (0) 0 (0)
Verruca seborrhoica 9 2 (23) 3 (33) 4 (44) 2 (23) 3 (33) 4 (44) 2 (23) 4 (44) 3 (33)
Senile angioma 6 0 (0) 2 (33) 4 (66) 0 (0) 1 (17) 5 (83) 0 (0) 1 (17) 5 (83)
Clear cell acanthoma 1 0 (0) 0 (0) 1 (100) 0 (0) 0 (0) 1 (100) 0 (0) 0 (0) 1 (100)
Scar 4 0 (0) 0 (0) 4 (100) 4 (100) 0 (0) 0 (0) 1 (25) 3 (75) 0 (0)
Lentigo solaris/senilis 7 3 (42 2 (29) 2 (29) 3 (42) 3 (43) 1 (15) 0 (0) 6 (85) 1 (15)
Total 108
Values shown in bold represent false positive and negative results.
a
All nevi (junctional, dermal, and dysplastic) have been merged in one subgroup.
Fig. 1. (a) BCC rated low risk by the algorithm (b) associated fractal map. Color images Discussion
available online at www.liebertpub.com/tmj This prospective study targeted
melanocytic and nonmelanocytic
about (pre)malignancy or the lesion was excised because of skin lesions to test the ability of an mHealth app in detecting
complaints. Out of 73 benign lesions, however, 14 were rated as melanoma and nonmelanoma skin cancers, specifically BCC
high risk by the application and 11 as medium risk (Table 4). and squamous cell carcinoma (SCC), and their precursors. We
Four of these lesions were examined histopathologically (re- have considered that actinic keratosis and Bowen’s disease
sults: one lichen planus-like keratosis, one dysplastic nevus, should be included in the high risk class, too, to urge the users
one verruca vulgaris, and one verruca seborrhoica). to visit the doctor because these lesions may progress to in-
The algorithm has also been evaluated on the same test set vasive skin cancer in the future.
without patient’s answers to the questionnaire (details per Although no images were excluded from the study due
class in Table 4); in this setup, the sensitivity was 71%, and the to low quality, the image acquisition step revealed to
specificity was 56% (Table 5). be problematic for those skin lesions with ulcerations or
The original algorithm, the one dedicated to melanocytic bleeding on top, surrounded by mottled or extremely tan-
lesion analysis and to melanoma detection, obtained, on the ned skin, localized in skin folds and hairy areas and finally
same set (details per class in Table 4), a sensitivity of 25% and those with other skin lesions in close proximity. Focusing
a specificity of 75% (Table 6). The poor results in this case were on and capturing them proved difficult and the image
inherent due to the fact that the algorithm contained no quality was quite low. This was also the case in four out of
specifications on how to treat nonpigmented lesions. six malignant skin tumors (three BCC and one SCC) re-
To identify the recalibrated algorithm’s performance on ceiving the medium or low risk ratings. The above listed
melanocytic lesions, for melanoma identification only, we rerun situations are defined to be exclusion criteria for partici-
the tests on the database of images with histopathology that was pation in future studies. In addition, in the app itself, these
used during the algorithm’s first investigation performed at criteria are clearly mentioned as factors that might alter the
Table 5. Results Obtained by the Algorithm Without Incorporating the Patients’ Answers to the Questionnaire
Regarding the Lesion’s Characteristics Versus Clinical Diagnosis—Eindhoven Database-Test Data Set
MALIGNANT
ALGORITHM OR PREMALIGNANT BENIGN SUM
High risk 25 32 57 Sensitivity (95% CI) 0.71 (0.53–0.84) Positive predictive value (95% CI) 0.43 (0.30–0.57)
Low/medium risk 10 41 51 Specificity (95% CI) 0.56 (0.44–0.67) Negative predictive value (95% CI) 0.80 (0.66–0.89)
Sum 35 73 108
CI, confidence interval.
Table 6. Results Obtained by the Algorithm for Melanoma Detection and Melanocytic Lesions (the Algorithm
Before Recalibration) Versus Clinical Diagnosis—Eindhoven Database-Test Data Set
MALIGNANT
ALGORITHM OR PREMALIGNANT BENIGN SUM
High risk 9 18 27 Sensitivity (95% CI) 0.25 (0.13–0.43) Positive predictive value (95% CI) 0.33 (0.17–0.53)
Low/medium risk 26 55 81 Specificity (95% CI) 0.75 (0.63–0.84) Negative predictive value (95% CI) 0.67 (0.56–0.77)
Sum 35 73 108
CI, confidence interval.
Table 7. Results Obtained by the Recalibrated Algorithm Versus Histopathological Results on the Ludwig Maximilian
University Database
ALGORITHM MELANOMA BENIGN LESIONS SUM
High risk 23 25 48 Sensitivity (95% CI) 0.884 (0.68–0.96) Positive predictive value (95% CI) 0.47 (0.33–0.62)
Low/medium risk 3 93 96 Specificity (95% CI) 0.788 (0.70–0.85) Negative predictive value (95% CI) 0.96 (0.90–0.99)
assessment quality in the Contraindication section (Terms application might be useful for early detection of the most
and Conditions). common types of skin cancer and their precursors (Fig. 2a, b).
Most of the verruca seborrhoica and lentigo solaris, two The recalibrated algorithm (with or without the patients
harmless but frequently appearing skin lesions, were not answers to the questions regarding the lesion) has a higher
classified as low risk (55% and 71%, respectively). In practice sensitivity than the original algorithm both on the Eindhoven
this means that patients could panic unnecessarily. Because in and LMU data sets. The specificity is low for the Eindhoven
many patients these lesions are multiple, in future studies data set when disregarding the patients input on the lesion.
adjustment of the questionnaire might be helpful for better (The low specificity is due to the fact that in the test set there
differentiation by the application. have been included mainly skin lesions belonging to non-
A limitation of the study is the low number of SCCs. Only melanoma skin cancer differential diagnosis.) Considering the
three tumors were included, one
of them being localized in an area
with extreme skin folds which
may be the reason for receiving a
medium rating. In three sub-
groups of benign lesions the
number of included lesions was
also limited to three for each di-
agnosis (lichen planus-like kera-
tosis, sebaceous hyperplasia, and
verruca vulgaris), which may in-
terfere with the results of the
analysis, too.
According to the answers given
in the questionnaire, 25 out of 35
(pre)malignant lesions appeared
during the last 1 month to 1 year. Fig. 2. (a) BCC rated high risk by the algorithm (appeared in the last 1 to 3 months, no changes, no other
In this context, it is clear that the symptoms) (b) associated fractal map. Color images available online at www.liebertpub.com/tmj
which may result in overtreatment (unnecessary excisions in 3. Wolf JA, Moreau JF, Akilov O, Patton T, English JC, III, Ho J, Ferris LK. Diagnostic
inaccuracy of smartphone applications for melanoma detection. JAMA
case of benign lesions) or incorrect referral to secondary care. Dermatol 2013;149:422–426.
In case they are willing to improve their diagnostic skills,11 the 4. Robson Y, Blackford S, Roberts D. Caution in melanoma risk analysis with
app might support them in accurately recognizing benign smartphone application technology. Br J Dermatol 2012;167;703–704.
lesions and differentiating them from skin malignancies or 5. Hamilton AD, Brady RR. Medical professional involvement in smartphone
‘‘apps’’ in dermatology. Br J Dermatol 2012;167:220–221.
might offer them the opportunity to communicate with der-
6. Maier T, Kulichova D, Schotten K, Astrid R, Ruzicka T, Berking C, Udrea A.
matologists in case of doubt; however, this should be further Accuracy of a smartphone application using fractal image analysis of
investigated in the future. pigmented moles compared to clinical diagnosis and histological result. J Eur
Acad Dermatol Venereol 2015;29:663–667.