Research

JAMA Dermatology | Original Investigation

Evaluation of Point-of-Care Decision Support for Adult Acne Treatment

by Primary Care Clinicians
David G. Li, MD, MBA; Ashley B. Pournamdari, BS; Kristina J. Liu, MD, MHS; Karl Laskowski, MD, MBA;
Cara Joyce, PhD; Arash Mostaghimi, MD, MPH, MPA

Supplemental content
IMPORTANCE Acne is a common reason for referral to dermatologists from primary care
clinicians. We previously modeled the impact of algorithm-based acne care in reducing
dermatology referrals, missed appointments, and treatment delays.
OBJECTIVE To prospectively evaluate the downstream outcomes following a real-time,
algorithm-based electronic decision-support tool on the treatment of patients referred
for acne.

DESIGN, SETTING, AND PARTICIPANTS This prospective cohort study included 260
treatment-naive patients referred to a dermatologist for the chief concern of acne, as well as
the referring primary care clinicians, at 33 primary care sites affiliated with Brigham and
Women’s Hospital from March 2017 to March 2018.

INTERVENTIONS We developed and implemented a decision-support tool into the electronic

medical record system at an academic medical center. The algorithm identified patients
referred to a dermatologist who had not previously been treated for acne and offered Author Affiliations: Department of
guideline-based recommendations for treatment via a real-time notification. Dermatology, Brigham & Women's
Hospital, Harvard Medical School,
MAIN OUTCOMES AND MEASURES Treatment modification by referring clinicians. Boston, Massachusetts (Li,
Pournamdari, Liu, Mostaghimi);
RESULTS Of 260 patients referred for acne, 209 (80.4%) were women, 146 (56.1%) were University of California, San Francisco
non-Hispanic white, and 236 (90.8%) listed English as the preferred language. Patients had a School of Medicine, San Francisco
median (quartile 1-quartile 3) age of 28.8 years (24.4-35.1 years) and 185 of 260 had private (Pournamdari); Department of
Medicine, Brigham & Women’s
insurance (71.1%). In total, the algorithm was associated with cancellation of the initial referral
Hospital, Harvard Medical School,
in 35 of 260 (13.5%) instances and treatment initiation by the referring clinician in 51 of 260 Boston, Massachusetts (Laskowski);
(19.6%) instances. Brigham and Women’s Physicians
Organization, Boston, Massachusetts
CONCLUSIONS AND RELEVANCE This decision-support algorithm was associated with a modest (Laskowski); Loyola University,
reduction in rates of acne-related referrals to dermatologists, and an increased likelihood of Chicago, Illinois (Joyce).
treatment initiation by the referring clinician. Corresponding Author: Arash
Mostaghimi, MD, MPA, MPH,
Department of Dermatology,
JAMA Dermatol. doi:10.1001/jamadermatol.2020.0135 Brigham and Women’s Hospital,
Published online March 4, 2020. 221 Longwood Ave, Boston, MA 02115
(amostaghimi@bwh.harvard.edu).

A
cne is a chronic inflammatory skin disease that affects
more than 85% of adolescents and frequently contin-
ues to adulthood.1-4 Although there is a high diagnos-
tic concordance for acne between dermatologists and pri-
mary care clinicians, acne continues to be a common indication
for referral to a dermatologist.5,6 Although referrals to derma-
tologists for acne are not discouraged, there are conse-
quences of immediate referral by primary care clinicians who
do not provide treatment, including delayed treatment and loss
to follow-up.7
Previous studies have demonstrated the potential of edu-
cation and treatment algorithms to reduce unnecessary refer-
rals to specialty care in nondermatological disease areas.8-12
Importantly, these studies support the use of real-time deci-
sion-support tools to modify physician behavior and reduce
unnecessary referrals, specifically for patients with pediatric
scoliosis or constipation being referred to specialists by pri-
mary care clinicians.9,10 Meanwhile, decision-support tools
have also been used to improve rates of health screening ques-
tions for preventative health and laboratory test orders for
monitoring of adverse drug events.11,12
In dermatology, stepwise treatment algorithms for acne
have been proposed, although implementation has been a
challenge.13-15 In a previous study,7 we modeled the effect of
algorithm-based acne treatment by primary care clinicians, and
demonstrated shared acne care had the potential to decrease
the rates of unnecessary appointments, wait time for treat-
ment, no-show rates, and downstream costs associated with
acne. Despite these predictions, prospective evaluation of al-
gorithm-based care for acne is lacking. From a societal stand-
point, shared care may reduce unnecessary referrals for acne
while reallocating resources to improve access to dermato-

jamadermatology.com (Reprinted) JAMA Dermatology Published online March 4, 2020 E1

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020

 Research Original Investigation
logic care, which has an average wait time of 32.3 days until
the first appointment.16-18
In this study, we prospectively evaluate the association of
implementation of an algorithm-based, decision-support tool
with changes in referral rates and treatment of adult acne
among primary care clinicians in a single health care system
across 33 clinical sites.
Methods
Patient Population and Development of Algorithm
We developed a point-of-care electronic health record–based
algorithm designed to alert clinicians referring treatment-
naive patients with acne to a dermatologist with a pop-up no-
tification (Figure 1). To appropriately identify treatment-
naive patients, the algorithm excluded patients who had a
previously documented prescription acne treatment in their
electronic medical record and patients who had a docu-
mented dermatology visit for acne within 1 year of the refer-
ral date.
If a patient meeting the above criteria was referred to a der-
matologist, the algorithm generated a notification alerting the
clinician that the patient was being referred to a dermatology
department without having trialed prescription treatments for
acne. The notification provided representative photographs
of acne severity (mild, moderate, severe)19 with correspond-
ing recommendations for treatment, based on guidelines of
care for the management of acne vulgaris at the time of study
design in 2015.
The clinician was then given the option of (1) keeping the
referral order without initiating treatment, (2) keeping the re-
ferral order while initiating treatment (using premade order sets
tailored to acne severity), (3) removing the referral order with-
out initiating treatment, or (4) removing the referral order while
initiating treatment (using the premade order set).
Premade order sets consisted of (1) full topical regimen for
patients with mild acne (tretinoin, 0.05% cream nightly and
clindamycin, 1% external solution twice daily), (2) full topical
plus oral antibiotic (tretinoin, 0.05% cream nightly and clinda-
mycin, 1% external solution twice daily, and doxycycline hy-
clate, 100 mg twice daily) for patients with moderate acne, and
(3) an accelerated 2-week referral to a dermatologist for pa-
tients with severe acne (eTable 1 in the Supplement).
Partners Healthcare institutional review board approval
was obtained for this study and written informed consent was
waived because data regarding clinician behavior were col-
lected within the electronic medical record system (EPIC, ver-
sion 2015), and the remainder of clinician and patient data were
examined via medical chart review as part of standard care.
Data were analyzed between October and December, 2018.
Implementation of Algorithm
The point-of-care algorithm was implemented in EPIC for pri-
mary care sites affiliated with Brigham and Women’s Hospi-
tal (Boston, Massachusetts) and identified patient encoun-
ters associated with an ambulatory referral order to the
dermatology department for acne from March 2017 through
E2 JAMA Dermatology Published online March 4, 2020 (Reprinted)
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020
Point-of-Care Decision Support for Adult Acne Treatment by Primary Care Clinicians
Key Points
Question What are the downstream outcomes following
implementation of a real-time, electronic decision-support tool for
the treatment of patients referred for acne?
Findings This prospective cohort study included 260 patients
referred by a primary care clinician for acne. Overall, the algorithm
was associated with cancellation of the initial referral in 35 of 260
(13.5%) instances and treatment initiation by the referring clinician
in 51 of 260 (19.6%) instances.
Meaning This decision-support algorithm was associated with
modestly reduced rates of acne-related referrals to dermatologists
and increased likelihood of treatment initiation by the referring
clinician.
March 2018. Among patients whose clinicians received a
pop-up notification, the algorithm identified and recorded the
patient’s medical record number, the referring clinician’s name,
and the time/date of the pop-up notification.
Data Collection: Initial Presentation for Acne
The medical records of all patients for whom the referring cli-
nician received a real-time notification for acne referral were
reviewed, and patient and clinician demographic informa-
tion was retrieved. Each clinician’s engagement with the real-
time notification (eg, ignored notification, kept referral, re-
moved referral) was documented, and the patient’s medication
list was reviewed to assess treatment rendered by the refer-
ring physician.
Data Collection: Referral to a Dermatologist
Among patients whose clinicians proceeded with the referral
to a dermatologist for acne, patient records were reviewed to
determine whether a dermatologic appointment was made
after and the status of the appointment (seen, not yet seen,
no-show/canceled appointment). For patients seen by a der-
matologist, we documented the diagnosis made by the
dermatologist and any acne medications that were used.
Medical records were reviewed at 60 days after the initial
referral date.
Statistical Analysis
Data were recorded using Research Electronic Data Capture
(version 6, REDCap). All data were analyzed and reported de-
scriptively with median (quartile 1, quartile 3) for continuous
measures and counts (percentage) for categorical variables.
Topical, oral, combination acne treatments, and isotretinoin
as initiated by dermatologists and referring clinicians were
compared using a Fisher exact test.
Results
Patient and Clinician Demographics
We identified 260 unique referrals to the dermatology depart-
ment for acne from across 33 different primary care sites in Mas-
sachusetts. Of these, 188 (72.3%) patients were referred for an
jamadermatology.com
 Point-of-Care Decision Support for Adult Acne Treatment by Primary Care Clinicians
Figure 1. Notification to Referring Clinicians
isolated problem of acne and 72 (27.7%) were referred for acne
and a concurrent dermatologic issue. Among the 260 patients,
209 (80.4%) were women, 146 (56.1%) were non-Hispanic white,
and 236 (90.8%) listed English as the preferred language (Table 1).
Patients had a median (Q1, Q3) age of 28.8 (24.4, 35.1) years and
185 (71.1%) had private insurance.
At the patient encounter level, 177 (68.1%) and 34 (13.1%)
patients were seen by 103 attending physicians and 30 resi-
dent physicians, respectively. The remaining 49 (18.8%) pa-
tients were seen by 28 midlevel clinicians (eg, nurse, physi-
cian assistant) and a midwife (Table 1). One hundred ninety
patients (73.1%) were seen by a female clinician. These demo-
graphic findings were similar between patients referred for acne
and those referred for acne plus a concurrent skin concern.
Decision-Support Notification
and Clinician Treatment of Acne
Overall, the decision-support notification coincided with treat-
ment initiation in 51 of 260 (19.6%) of cases. Of these, treat-
ment was initiated using the notification’s premade order set
in 16 of 51 (31.4%) instances (Figure 2).
Among 188 of 260 patients referred for only acne, the real-
time notification was associated with clinician cancellation of
the initial referral to a dermatologist for acne in 34 of 188 (18.1%)
jamadermatology.com
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020
Original Investigation Research
The pop-up notification occurred in
real time after a clinician placed a
referral order for acne in patients
meeting the prespecified criteria
described in the methods.
cases. Of these 34 cases, the referring clinician then initiated
acne treatment in 26 (76.5%) instances. Eleven of these 34
(42.3%) cases were initiated using the premade order set. In
the other 154 patients whose referrals were continued de-
spite the notification, the referring clinician then initiated acne
treatment in 17 (11.0%) cases. Only 3 of these 17 (17.6%) cases
were initiated via the premade order set.
For the 72 of 260 patients referred for acne plus another
skin problem, the alert was associated with clinician cancel-
lation of the initial referral to a dermatologist for acne in 1 (1.4%)
case, and treatment was provided via an order set. In the other
71 patients whose referrals were continued despite the alert,
the referring clinician then initiated acne treatment in 7 (9.9%)
cases, 1 (14.3%) of which was initiated on an order set.
Health Care Utilization Among Patients
With Referrals for Acne
Of 154 patients referred for an isolated concern of acne whose
referrals were continued, 121 (78.6%) had made an appoint-
ment at 60 days after the date of referral (Table 2). Of these,
20 (16.5%) were no-shows and 5 (4.1%) had not been seen at
60 days postreferral. The remaining 96 (79.3%) patients were
seen by a dermatologist, of which 90 of 96 (93.5%) were di-
agnosed with acne (eTable 2 in the Supplement). Overall,
(Reprinted) JAMA Dermatology Published online March 4, 2020 E3
 Research Original Investigation Point-of-Care Decision Support for Adult Acne Treatment by Primary Care Clinicians

Table 1. Characteristics of Patients and Clinicians Associated With Referral to a Dermatologist for Acne

No. (%)
Characteristic Acne Only Acne Plus Other Total
Encounter-level summary statistics for patients
Age, median (Q1-Q3), y 28.3 (24.3-34.2) 30.3 (24.7-43.8) 28.8 (24.4-35.1)
Female 153 (81.4) 56 (77.8) 209 (80.4)
Race
Non-Hispanic white 100 (53.2) 46 (63.9) 146 (56.1)
Black 40 (21.3) 8 (11.1) 48 (18.5)
Hispanic 23 (12.2) 8 (11.1) 31 (11.9)
Other 25 (13.3) 10 (13.9) 35 (13.5)
Health insurance
Private 138 (73.4) 47 (65.3) 185 (71.1)
Public 46 (24.5) 24 (33.3) 70 (26.9)
Uninsured 2 (1.1) 1 (1.4) 3 (1.2)
Unspecified 2 (1.1) 0 (0) 2 (0.8)
English as preferred language 168 (89.4) 68 (94.4) 236 (90.8)
Encounter-level summary statistics for referring cliniciansa
Years of practice (if attending physician)
<10 28 (21.9) 12 (24.5) 40 (22.6)
Between 10 and 20 56 (43.8) 18 (36.7) 74 (41.8)
>20 44 (34.4) 19 (38.8) 63 (35.6)
Female 132 (70.2) 58 (80.6) 190 (73.1)
Clinician title
Physician (attending) 128 (68.1) 49 (68.1) 177 (68.1)
Resident 29 (15.4) 5 (6.9) 34 (13.1) Abbreviation: Q, quartile.
a
Midlevel 31 (16.5) 17 (23.6) 48 (18.5) Encompasses 33 unique primary
care sites around Boston,
Other 0 (0) 1 (1.4) 1 (0.4)
Massachusetts.

Figure 2. Flowchart of Clinician Behavior Modification After Notification

260 Acne-related encounters

188 Acne-only encounters 72 Acne plus other encounters

34 Canceled referrals 154 Continued referrals 1 Canceled referrals 71 Continued referrals

8 Did not prescribe 137 Did not prescribe 0 Did not prescribe 64 Did not prescribe
treatment treatment treatment treatment

26 Prescribed treatment 17 Prescribed treatment 1 Prescribed treatment 7 Prescribed treatment

11 Treated via BPA order 3 Treated via BPA order 1 Treated via BPA order 1 Treated via BPA order
15 Treated via custom order 14 Treated via custom order 0 Treated via custom order 6 Treated via custom order

BPA indicates best practice alert. The flowchart illustrates the primary outcome having their referral to a dermatologist canceled after the notification,
of behavior modification (eg, placement of premade order set or custom order 2 patients were recommended to contact their local clinician for acne
set) among referring clinicians after being alerted with the notification. Of the 8 treatment, 1 eventually saw a dermatologist despite canceled referral,
patients who did not receive treatment from the referring clinician despite and the remaining 5 were lost to follow-up.

96 of 154 (62.3%) patients referred to a dermatologist for acne similar among patients who were referred to a dermatologist
were seen at 60 days after the referral date. These findings were for acne in addition to another skin problem.

E4 JAMA Dermatology Published online March 4, 2020 (Reprinted) jamadermatology.com

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020

 Point-of-Care Decision Support for Adult Acne Treatment by Primary Care Clinicians Original Investigation Research

Table 2. Health Care Utilization Associated With Continuing Referral to a Dermatologist for Acne Only Despite
Decision Support

Continued Referrals, No. (%)

Patient Statusa Acne Only (n = 154) Acne Plus Other (n = 71) Total (n = 225)
Made appointment 121 (78.6) 54 (76.1) 175 (77.8)
No show 20/121 (16.5) 8/54 (14.8) 28/175 (16.0)
Not yet seen 5/121 (4.1) 8/54 (14.8) 13/175 (7.4) a
Analysis performed at 60 days after
Seen by dermatologist 96/121 (79.3) 38/54 (70.4) 134/175 (76.6) date of referral to dermatology
department.
Acne confirmed by dermatologistb 90/96 (93.5) 27/38 (71.1) 117/134 (87.3)
b
See eTable 2 in the Supplement for a
Did not make appointment 33 (21.4) 17 (23.9) 50 (22.2)
list of misdiagnoses.

ther escalation of care (eg, additional referral to a dermatolo-

Table 3. Comparison of Acne Treatments Initiated by Dermatologists and
Referring Clinicians gist). Of these 4 patients, 3 (75.0%) were prescribed an acne
treatment on a trial basis for fewer than 2 weeks before receiv-
No. (%)
ing a referral to a dermatologist.
Treatment by Treatment via Referring
Dermatologist Clinician after Decision Of 51 patients who were prescribed treatment via the re-
Treatment (n = 117) Support (n = 51) ferring clinician, 15 (29.4%) were eventually evaluated by a der-
Topical medications onlya 62 (53.0) 33 (64.7) matologist at 60 days. Thirteen of 15 patients (86.7%) had their
Oral medications onlyb 6 (5.1) 4 (7.8) regimen changed by the dermatologist. The most frequent
Topical and oral medications 35 (30.0) 10 (19.6) change was the addition of 1 or more topical medications such
Isotretinoin 8 (6.8) 0 as clindamycin (6 of 13), benzoyl peroxide (4 of 13), tretinoin
Otherc,d 6 (5.1) 4 (7.8) (4 of 13), or adapalene (2 of 13) onto the existing regimen. Medi-
Received subsequent referral NA 4 (7.8) cations started by the referring clinician were discontinued and
to a dermatologiste
replaced with alternatives in 4 of 15 patients (26.7%). In 2 of
Abbreviation: NA, not applicable. 15 cases (13.3%), the dermatologist agreed with the initial treat-
a
Includes topical antibiotics, retinoids, benzoyl peroxide, acne moisturizers, ment regimen by the referring clinician and did not alter the
and salicylic acid. Of patients who were prescribed a topical treatment by the
regimen.
referring clinician, 2 were concurrently provided with an accelerated referral to
a dermatologist.
b
Includes oral antibiotics, spironolactone, and oral contraceptives.
c
Four patients who received proposed acne treatment via a dermatologist
declined treatment; 2 patients were recommended by a dermatologist to
Discussion
continue treatment started by the referring clinician.
d
In this prospective study examining the implementation of a
Four patients received an accelerated referral to a dermatologist via a
premade order set. point-of-care decision support tool for adult acne treatment,
e
Of the 4 patients who received subsequent dermatology referral at 60 days we found that real-time notification may be modestly effec-
after referral, 3 of the patients trialed the referring clinician's treatment for tive in modifying clinician referral and treatment behavior. Af-
fewer than 2 weeks before requesting another referral to a dermatologist. ter being presented with the notification containing acne treat-
ment guidelines, referring clinicians often chose an appropriate
Comparison of Treatments by Dermatologists regimen that approximately matched treatments prescribed
and Referring Clinicians by dermatologists. Although these findings have demon-
Of the 117 patients who were seen by a dermatologist and con- strated reductions in immediate referrals to dermatologists for
firmed to have acne, 62 (53.0%) were prescribed at least 1 topi- acne, the maximal impact of the advisory tool was not ob-
cal medication for acne (Table 3). Oral medications including tained, as evident from high rates of referral continuation
oral antibiotics, spironolactone, and/or oral contraceptives were (>80%).
initiated in 6 (5.1%) patients, whereas combined topical and The reason for this limited success is likely multifacto-
oral medications were prescribed in 35 (30.0%) patients. rial. Referring clinicians may have disregarded the real-time
Isotretinoin was started in 8 (6.8%) patients. In the remain- notification because of alert fatigue, which has been shown to
ing 6 patients, 4 (3.4%) declined proposed treatment by the der- explain high physician override rates of decision support
matologist and 2 (1.7%) were recommended to continue the alerts.20-22 Furthermore, because the notification occurred af-
treatment initiated by the referring clinician. ter a decision was already made to refer the patient, cognitive
Among 51 patients who initiated treatment recom- barriers (eg, anchoring bias) may have prevented physician up-
mended by the referring clinician, 33 (64.7%), 4 (7.8%), and take of the notification. Likewise, some referring physicians
10 (19.6%) were prescribed topical, oral, and combined topi- may have placed the referral (and received a notification) af-
cal and oral medications. When classified by these broad cat- ter the patient visit, making them less likely to deviate from
egories, it appeared that treatment selections by referring cli- the original plan as communicated with the patient. It is also
nicians did not differ significantly from those chosen by plausible that clinicians may have declined to change their ex-
dermatologists. At 60 days after the date of referral, only 4 isting decision-making given the inertia of conventional prac-
(7.8%) patients treated by a referring clinician received fur- tice patterns, which can contribute toward nonadherence to

jamadermatology.com (Reprinted) JAMA Dermatology Published online March 4, 2020 E5

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020

 Research Original Investigation
clinical guidelines.23-26 Finally, it is possible that some pa-
tients may have declined treatment recommended by the no-
tification and referring clinician, and that this interaction was
simply not documented.
Despite these challenges, these findings support and ex-
pand on existing data regarding the role of primary care clini-
cians in shared care for evaluation and treatment of acne.6,7
Importantly, we demonstrated the decision support notifica-
tion to correlate with the selection of guideline-based acne
treatments by referring clinicians, which were not signifi-
cantly different from treatments by dermatologists (Table 3).
Although 4 (7.8%) patients treated by referring clinicians re-
ceived subsequent referral to a dermatologist, most of these
patients were prescribed acne medications on a trial basis for
fewer than 2 weeks prior to dermatology assessment.
Early treatment initiation by referring clinicians may also
address the high rates of loss to follow-up. Among patients
whose referring clinicians continued with the dermatologist
referral for an isolated problem of acne, only 96 of 154 (62.3%)
were seen by a dermatologist at 60 days after the initial refer-
ral date. Of the 5 patients with appointments who were not seen
at the time of analysis, 2 were later evaluated by a dermatolo-
gist and prescribed an oral acne treatment (doxycycline and
spironolactone); the remaining 3 patients were lost to follow-
up, further underscoring the potential consequences associ-
ated with immediate referral to a dermatologist for acne.7
Although this point-of-care notification reduced rates of
immediate referral to a dermatologist and improved the like-
lihood of treatment initiation, a comprehensive approach em-
bedded into the training and practice of primary care clini-
cians may be needed to improve the effic acy of this
intervention. For instance, educational initiatives for pri-
mary care clinicians to highlight the opportunities and chal-
lenges associated with shared care for relevant disease mod-
els (eg, acne) may partially overcome the cognitive overload
and alert desensitization associated with electronic health rec-
ord systems, while priming physicians to be more receptive to
best practice notifications.27-31 Initiatives including team-
based didactics and cross-training rotations may also im-
prove clinician preparedness for initiating treatment for acne.
These approaches have the added benefit of helping to main-
tain antibiotic stewardship among physicians who may have
become reliant on the use of antibiotics to treat acne. Finally,
repositioning of future decision-support notifications to ap-
pear earlier in the clinician workflow may improve patient-
physician communication regarding alternatives to immedi-
ate dermatologist referral.
Beyond this point-of-care decision support tool, other op-
portunities to promote shared care for the treatment of acne
should be explored. Whereas our intervention focused on the
use of algorithm-based notifications in the clinical workflow
of primary care clinicians, another potentially viable option
to optimize acne referrals and reduce time to treatment would
be to use live interactive or store-and-forward teledermatol-
ogy during the patient’s initial visit at the primary care office.
Shared patient care among primary care clinicians and live
video teledermatologists have been studied in the literature,
with 1 study of 1500 live interactive teledermatology consul-
E6 JAMA Dermatology Published online March 4, 2020 (Reprinted)
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020
Point-of-Care Decision Support for Adult Acne Treatment by Primary Care Clinicians
tations for unspecified skin diseases resulting in changes to di-
agnosis and treatment in 69.9% and 97.7% of the time,
respectively.32,33 To further reduce the rate of unnecessary re-
ferrals for acne among treatment-naive patients, it may also
be worthwhile to explore the use of live interactive teleder-
matology to facilitate shared care for acne.
Limitations
These findings must be interpreted in the context of study de-
sign. Although this was a multicenter prospective study, all
practice locations were in Massachusetts and thus our results
may not be generalizable given regional and geographical varia-
tions in practice. Specifically, all practice locations included
in the study were affiliated with a large academic medical cen-
ter and results may not be generalizable to different practice
settings. Similarly, generalizability may be limited largely to
adult women because sex distribution among patients in our
study (80.2%) is greater than the reported sex distribution
among patients with acne vulgaris in the United States
(65.2%).34 In addition, the median age of patients in our sample
was greater than the national average for acne primarily be-
cause our outpatient dermatology clinic does not see pa-
tients younger than 16 years. Furthermore, although we could
verify an association between the notification and referral can-
cellation, we could only establish an association between the
notification and treatment initiation by referring clinicians be-
cause it is unclear whether physicians who initiated treat-
ment would not have done so in the absence of the decision-
support tool. Finally, our data do not include referrals for acne
that were coded under different names (eg, facial rash).
Regarding treatment selection, we had included topical and
oral antibiotics in our premade order sets based on regional pre-
scription preferences in efforts to have primary care clini-
cians adopt our algorithm; however, there exist treatments that
would be more optimal and in alignment with current acne
treatment guidelines.35,36 Future studies should consider a
broader selection of commonly prescribed and effective treat-
ments for acne, including benzoyl peroxide, oral contracep-
tives, and spironolactone while also accounting for separate
pathways for male and female patients. Although these op-
tions were not included in the present study to simplify the
premade order sets, they may reduce the risk of inappropri-
ate prescribing of oral antibiotics.
Additional efforts aimed at gathering a larger sample size
would enable granular treatment comparisons between der-
matologists and primary care clinicians. Increasing fol-
low-up time to allow for certain nonhormonal treatments to
take full effect before analyzing the data and implementing a
patient/clinician survey to gain additional insight into the mod-
est success of this intervention would also be of benefit.
Conclusions
These data suggest that this decision-support tool may be mod-
estly effective at reducing the rates of acne-related referrals
to dermatologists among treatment-naive patients, while in-
creasing the likelihood of treatment initiation by the refer-
jamadermatology.com
 Point-of-Care Decision Support for Adult Acne Treatment by Primary Care Clinicians
ring clinician. Not only were treatments by referring primary
care clinicians similar to those selected by dermatologists, the
rates of subsequent referral among these patients were low.
ARTICLE INFORMATION
Accepted for Publication: January 16, 2020.
Published Online: March 4, 2020.
doi:10.1001/jamadermatol.2020.0135
Author Contributions: Drs Li and Mostaghimi had
full access to all the data in the study and take
responsibility for the integrity of the data and the
accuracy of the data analysis.
Study concept and design: Li, Liu, Laskowski.
Acquisition, analysis, or interpretation of data: Li,
Pournamdari, Laskowski, Joyce.
Drafting of the manuscript: Li, Pournamdari.
Critical revision of the manuscript for important
intellectual content: Li, Liu, Laskowski, Joyce.
Statistical analysis: Li, Joyce.
Obtained funding: Li, Laskowski.
Administrative,technical,ormaterialsupport:Laskowski.
Conflicts of Interest: Dr Mostaghimi is on the
editorial board of JAMA Dermatology but played no
role in the acceptance or publication of the article.
No other conflicts are reported.
Funding/Support: This project was supported by
the National Center for Advancing Translational
Sciences, National Institutes of Health, Award
Number TL1TR001062 and by the Brigham and
Women’s Physician Organization Brigham Care
Redesign Incubator and Startup Program (BCRISP).
This project was supported by the National Center
for Advancing Translational Sciences, National
Institutes of Health, Award Number TL1TR001062
(Dr Li) and by the Brigham and Women’s Physician
Organization Brigham Care Redesign Incubator and
Startup Program (BCRISP) (Dr Mostaghimi).
Role of the Funder/Sponsor: The funding agencies
had no role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
approval of the manuscript; and decision to submit
the manuscript for publication.
REFERENCES
1. Moradi Tuchayi S, Makrantonaki E, Ganceviciene
R, Dessinioti C, Feldman SR, Zouboulis CC. Acne
vulgaris. Nat Rev Dis Primers. 2015;1:15029. doi:10.
1038/nrdp.2015.29
2. Bhate K, Williams HC. Epidemiology of acne
vulgaris. Br J Dermatol. 2013;168(3):474-485.
doi:10.1111/bjd.12149
3. Tan JKL, Bhate K. A global perspective on the
epidemiology of acne. Br J Dermatol. 2015;172(suppl
1):3-12. doi:10.1111/bjd.13462
4. Lynn DD, Umari T, Dunnick CA, Dellavalle RP. The
epidemiology of acne vulgaris in late adolescence.
Adolesc Health Med Ther. 2016;7:13-25. doi:10.2147/
AHMT.S55832
5. Tran H, Chen K, Lim AC, Jabbour J, Shumack S.
Assessing diagnostic skill in dermatology:
a comparison between general practitioners and
dermatologists. Australas J Dermatol. 2005;46(4):
230-234. doi:10.1111/j.1440-0960.2005.00189.x
6. Moreno G, Tran H, Chia AL, Lim A, Shumack S.
Prospective study to assess general practitioners’
dermatological diagnostic skills in a referral setting.
Australas J Dermatol. 2007;48(2):77-82. doi:10.1111/
j.1440-0960.2007.00340.x
jamadermatology.com
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Utrecht University Library User on 03/06/2020
Moving forward, further exploration of educational decision
support in combination with more comprehensive initiatives
may optimize shared care for acne.
7. Liu KJ, Hartman RI, Joyce C, Mostaghimi A.
Modeling the effect of shared care to optimize acne
referrals from primary care clinicians to
dermatologists. JAMA Dermatol. 2016;152(6):655-
660. doi:10.1001/jamadermatol.2016.0183
8. Faulkner A, Mills N, Bainton D, et al. A systematic
review of the effect of primary care-based service
innovations on quality and patterns of referral to
specialist secondary care. Br J Gen Pract. 2003;53
(496):878-884. https://www.ncbi.nlm.nih.gov/
pubmed/14702909. Accessed June 3, 2018.
9. Mallon D, Vernacchio L, Trudell E, et al. Shared
care: a quality improvement initiative to optimize
primary care management of constipation. Pediatrics.
2015;135(5):e1300-e1307. doi:10.1542/peds.2014-
1962
10. Vernacchio L, Trudell EK, Hresko MT, Karlin LI,
Risko W. A quality improvement program to reduce
unnecessary referrals for adolescent scoliosis.
Pediatrics. 2013;131(3):e912-e920. doi:10.1542/peds.
2012-2359
11. Burdick TE, Kessler RS. Development and use of
a clinical decision support tool for behavioral health
screening in primary care clinics. Appl Clin Inform.
2017;8(2):412-429. doi:10.4338/ACI-2016-04-RA-
0068
12. Steele AW, Eisert S, Witter J, et al. The effect of
automated alerts on provider ordering behavior in
an outpatient setting. PLoS Med. 2005;2(9):e255.
doi:10.1371/journal.pmed.0020255
13. Practitioners TRAC of G. RACGP - Acne – best
practice management. https://www.racgp.org.au/
afp/201009/39116. Accessed June 3, 2018.
14. Thiboutot D, Gollnick H, Bettoli V, et al; Global
Alliance to Improve Outcomes in Acne. New
insights into the management of acne: an update
from the Global Alliance to Improve Outcomes in
Acne group. J Am Acad Dermatol. 2009;60(5)
(suppl):S1-S50. doi:10.1016/j.jaad.2009.01.019
15. Liao DC. Management of acne. J Fam Pract. 2003;
52(1):43-51. https://www.ncbi.nlm.nih.gov/pubmed/
12540312. Accessed June 3, 2018.
16. Kimball AB, Resneck JS Jr. The US dermatology
workforce: a specialty remains in shortage. J Am
Acad Dermatol. 2008;59(5):741-745. doi:10.1016/j.
jaad.2008.06.037
17. Resneck J Jr, Kimball AB. The dermatology
workforce shortage. J Am Acad Dermatol. 2004;50
(1):50-54. doi:10.1016/j.jaad.2003.07.001
18. 2017 Survey of Patient Appointment Wait
Times. https://www.merritthawkins.com/news-
and-insights/thought-leadership/survey/survey-of-
physician-appointment-wait-times/. Accessed
February 1, 2020.
19. Acne vulgaris | DermNet NZ. https://www.
dermnetnz.org/topics/acne-vulgaris. Accessed
January 12, 2020.
20. van der Sijs H, Aarts J, Vulto A, Berg M.
Overriding of drug safety alerts in computerized
physician order entry. J Am Med Inform Assoc.
2006;13(2):138-147. doi:10.1197/jamia.M1809
21. Ash JS, Berg M, Coiera E. Some unintended
consequences of information technology in health
care: the nature of patient care information
system-related errors. J Am Med Inform Assoc.
2004;11(2):104-112. doi:10.1197/jamia.M1471
(Reprinted) JAMA Dermatology Published online March 4, 2020
Original Investigation Research
22. Ancker JS, Edwards A, Nosal S, Hauser D,
Mauer E, Kaushal R; with the HITEC Investigators.
Effects of workload, work complexity, and repeated
alerts on alert fatigue in a clinical decision support
system. BMC Med Inform Decis Mak. 2017;17(1):36.
doi:10.1186/s12911-017-0430-8
23. Cabana MD, Rand CS, Powe NR, et al. Why
don’t physicians follow clinical practice guidelines?
A framework for improvement. JAMA. 1999;282
(15):1458-1465. doi:10.1001/jama.282.15.1458
24. Main DS, Cohen SJ, DiClemente CC. Measuring
physician readiness to change cancer screening:
preliminary results. Am J Prev Med. 1995;11(1):54-58.
doi:10.1016/S0749-3797(18)30501-4
25. Grol R. National standard setting for quality of care
ingeneralpractice:attitudesofgeneralpractitionersand
response to a set of standards. Br J Gen Pract. 1990;40
(338):361-364. https://www.ncbi.nlm.nih.gov/pubmed/
2265001. Accessed June 3, 2018.
26. The Awareness-to-Adherence Model of the
Steps to Clinical Gu. Medical Care.
https://journals.lww.com/lww-medicalcare/
Abstract/1996/09000/The_Awareness_to_
Adherence_Model_of_the_Steps_to.2.aspx.
Accessed June 3, 2018.
27. Pena S, Hill D, Feldman SR. Use of topical
retinoids by dermatologists and
non-dermatologists in the management of acne
vulgaris. J Am Acad Dermatol. 2016;74(6):1252-1254.
doi:10.1016/j.jaad.2016.01.011
28. Mate KS, Mehta NP. A two-way street: what the
United States can learn from resource-limited
countries to improve health care delivery and
reduce costs. Jt Comm J Qual Patient Saf. 2015;41
(5):236-239. doi:10.1016/S1553-7250(15)41032-3
29. Rush JL, Ibrahim J, Saul K, Brodell RT.
Improving Patient Safety by Combating Alert
Fatigue. J Grad Med Educ. 2016;8(4):620-621.
doi:10.4300/JGME-D-16-00186.1
30. Reducing the Safety Hazards of Monitor Alert
and Alarm Fatigue | AHRQ Patient Safety Network.
https://psnet.ahrq.gov/perspective/reducing-
safety-hazards-monitor-alert-and-alarm-fatigue.
Accessed February 1, 2020.
31. Alarm Fatigue: A Concept Analysis | HIMSS.
https://www.himss.org/alarm-fatigue-concept-
analysis. Accessed February 1, 2020.
32. Lamel S, Chambers CJ, Ratnarathorn M,
Armstrong AW. Impact of live interactive
teledermatology on diagnosis, disease management,
and clinical outcomes. Arch Dermatol. 2012;148(1):61-
65. doi:10.1001/archdermatol.2011.1157
33. Lee JJ, English JC III. Teledermatology: a review
and update. Am J Clin Dermatol. 2018;19(2):253-260.
doi:10.1007/s40257-017-0317-6
34. Yentzer BA, Hick J, Reese EL, Uhas A, Feldman
SR, Balkrishnan R. Acne vulgaris in the United States:
a descriptive epidemiology. Cutis. 2010;86(2):94-99.
35. Zaenglein AL, Pathy AL, Schlosser BJ, et al.
Guidelines of care for the management of acne
vulgaris. J Am Acad Dermatol. 2016;74(5):945-73.e33.
doi:10.1016/j.jaad.2015.12.037
36. Nast A, Dréno B, Bettoli V, et al. European
evidence-based (S3) guideline for the treatment of
acne - update 2016 - short version. J Eur Acad
Dermatol Venereol. 2016;30(8):1261-1268. doi:10.
1111/jdv.13776
E7