Original Investigation | Psychiatry

Machine Learning–Based Prediction of Attention-Deficit/Hyperactivity Disorder

and Sleep Problems With Wearable Data in Children
Won-Pyo Kim, ME; Hyun-Jin Kim, MD; Seung Pil Pack, PhD; Jae-Hyun Lim, MD, PhD; Chul-Hyun Cho, MD, PhD; Heon-Jeong Lee, MD, PhD

Abstract Key Points

Question Can data obtained from
IMPORTANCE Early detection of attention-deficit/hyperactivity disorder (ADHD) and sleep
personal digital devices (ie, wearable
problems is paramount for children’s mental health. Interview-based diagnostic approaches have
devices) in children collected from the
drawbacks, necessitating the development of an evaluation method that uses digital phenotypes in
Adolescent Brain Cognitive
daily life.
Development study be used for
predicting attention-deficit/
OBJECTIVE To evaluate the predictive performance of machine learning (ML) models by setting the
hyperactivity disorder (ADHD) and sleep
data obtained from personal digital devices comprising training features (ie, wearable data) and
problems?
diagnostic results of ADHD and sleep problems by the Kiddie Schedule for Affective Disorders and
Schizophrenia Present and Lifetime Version for Diagnostic and Statistical Manual of Mental Disorders, Findings In this diagnostic study of 79
5th edition (K-SADS) as a prediction class from the Adolescent Brain Cognitive Development (ABCD) children with ADHD and 68 children
study. with sleep problems, circadian rhythm–
based wearable data were useful for
DESIGN, SETTING, AND PARTICIPANTS In this diagnostic study, wearable data and K-SADS data developing a suitable machine learning
were collected at 21 sites in the US in the ABCD study (release 3.0, November 2, 2020, analyzed model. The model showed reasonable
October 11, 2021). Screening data from 6571 patients and 21 days of wearable data from 5725 patients predictive performance.
collected at the 2-year follow-up were used, and circadian rhythm–based features were generated
Meaning The findings of this diagnostic
for each participant. A total of 12 348 wearable data for ADHD and 39 160 for sleep problems were
study suggest that wearable data have
merged for developing ML models.
the potential to detect ADHD and sleep
problems; this approach can facilitate
MAIN OUTCOMES AND MEASURES The average performance of the ML models was measured
the application of digital phenotypes in
using an area under the receiver operating characteristics curve (AUC), sensitivity, specificity, positive
daily life for mental problems in children.
predictive value (PPV), and negative predictive value (NPV). In addition, the Shapley Additive
Explanations value was used to calculate the importance of features.
+ Supplemental content
RESULTS The final population consisted of 79 children with ADHD problems (mean [SD] age, 144.5 Author affiliations and article information are
[8.1] months; 55 [69.6%] males) vs 1011 controls and 68 with sleep problems (mean [SD] age, 143.5 listed at the end of this article.

[7.5] months; 38 [55.9%] males) vs 3346 controls. The ML models showed reasonable predictive
performance for ADHD (AUC, 0.798; sensitivity, 0.756; specificity, 0.716; PPV, 0.159; and NPV,
0.976) and sleep problems (AUC, 0.737; sensitivity, 0.743; specificity, 0.632; PPV, 0.036; and
NPV, 0.992).

CONCLUSIONS AND RELEVANCE In this diagnostic study, an ML method for early detection or
screening using digital phenotypes in children’s daily lives was developed. The results support
facilitating early detection in children; however, additional follow-up studies can improve its
performance.

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502

Open Access. This is an open access article distributed under the terms of the CC-BY License.

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 1/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

Introduction
Mental health problems are prevalent in childhood and are major health and social burdens. The
prevalence of mental problems in children and adolescents worldwide is approximately 10% to 20%,
with most symptoms appearing before age 14 years.1,2 Psychiatric disorders diagnosed in childhood
have a high correlation with lifelong mental problems, making early detection and therapeutic
intervention vital.2-4
Attention-deficit/hyperactivity disorder (ADHD) is prevalent in approximately 5% of children,
and sleep problems are prevalent in approximately 10% of children.3,5,6 Both ADHD and sleep
problems have multifactorial etiologic factors that can cause neurocognitive and functional
impairment and can have negative effects in several areas, such as educational achievements,
socioeconomic outcomes, and parent-child relationships.7,8 Currently, the diagnosis of ADHD and
sleep problems is not based on measurable methods, such as biomarkers, but is diagnosed through
clinician evaluation by confirming the severity, frequency, and duration of symptoms.2 Structured
interview tools and semistructured interview tools are recommended for assessing psychiatric
disorders in children.9,10 These evaluation tools, used by trained professionals, have limitations
during face-to-face evaluations due to time and spatial constraints, as well as differences in results
among the interviewees.11
The development of personal digital devices has enabled moment-by-moment collection of
individual-level behavioral and physiologic data and their integration and analysis for health care as
digital phenotypes.12 Considering the advantage of enabling digital phenotypes in diagnoses and
treatments for personalized health services, the digital health care market is expected to grow
exponentially.13 In particular, digital phenotypes from personal digital devices can provide insights
into an individual’s real-time mental state.14,15 Although recent studies have reported their
advantages, few studies have used digital phenotypes for psychiatric diagnosis.16 Conventional
assessment methods of mental states through questionnaires or interviews vary regarding data
quality and have limitations such as recall bias. Therefore, there is a demand for a new diagnostic
method based on quantitative data by using real-time digital phenotypes gathered from people’s
daily lives.17
The Adolescent Brain Cognitive Development (ABCD) study is a longitudinal observational
study of more than 10 000 adolescents recruited from 21 locations across the US.18 It aims to trace
the trajectory of brain development in children and adolescents and investigate the factors
influencing the onset, course, and severity of various psychiatric disorders.19-21 Several studies have
used data from the ABCD study to investigate mental disorders in children and adolescents.20-22
Particularly, the ABCD study provides wearable data (Fitbit Wearable Wrist Tracker, Google LLC) on
children’s physical activity, sleep, and heart rates. Accordingly, the present study aimed to develop a
machine learning (ML) method to predict ADHD and sleep problems in children using wearable data,
anthropometrics, and the Kiddie Schedule for Affective Disorders and Schizophrenia Present and
Lifetime Version for Diagnostic and Statistical Manual of Mental Disorders, 5th edition (K-SADS) data
from the ABCD study.

Methods
Study Design and Setting
The ABCD study is a large-scale, longitudinal study involving children aged 9 to 11 years from 21 data
collection sites,23 covering 17 states in the US. It was designed considering the demographic
composition of the US and using a school-based recruitment strategy based on epidemiologic
information.18 Starting with the initial 11 878 patients, data for 11 235 patients have been released at
1-year and for 6571 patients at 2-year follow-ups. The ABCD study data are available at the National
Institute of Mental Health data repository. This study used screening data from 6571 patients and 21
days of wearable data (ABCD mobile tech) from 5725 patients released at the 2-year follow-up. The

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 2/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

study was reviewed and approved, with a waiver of informed consent, by the institutional review
board of Chungnam National University Sejong Hospital and followed the Transparent Reporting of a
Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline.

Participants
The study participants were selected using a diagnosis label on the K-SADS from parents and children
without comorbidities. The exclusion criteria were reported comorbidities and missing or abnormal
values in wearable data. Detailed inclusion and exclusion criteria are described in eMethods 1 in
Supplement 1. Following the K-SADS parent data, the ADHD diagnoses showed various symptoms,
such as explosive irritability, avoiding tasks requiring attention, difficulty sustaining attention since
elementary school, easily distracted, and interference with social, academic, or occupational
functioning. Following the K-SADS child data, the sleep problems diagnosis only showed insomnia
symptoms.

Procedures
Data Collection and Feature Extraction
This study used data regarding participants’ sex, wearable device, anthropometrics of youth, and
K-SADS data from the ABCD study (release 3.0, November 2, 2020, analyzed October 11, 2021)
provided by the National Institute of Mental Health database.10 The data were expanded to reflect
participants’ daily circadian rhythms through cosinor analysis24 using physical biomarkers (age,
height, and weight in anthropometrics) and wearable data including 9 variables (heart rate, stage of
sleep for both the 30- and 60-second records, calories, intensity, and metabolic equivalents in the
ABCD mobile tech). The list of generated features is given in Table 1. The features were generated
from circadian rhythm–based physical biomarkers, including arithmetic variables, calorie
consumption, a difference of basal metabolic rates, cosinor analysis-related variables, the least active
5-hour periods, and the most active 10-hour periods.25,26 To generate valid features, the wearable
data recorded for less than 30 minutes per hour and less than 5 hours per day were regarded as
invalid data in this study. In addition, a total of 64 circadian features were created, including bedtime
and daytime heart rates and step counts, duration of naps, and duration of sleep. eMethods 2 and
equations 1 to 6 in the eAppendix in Supplement 1 present the detailed process and formula of
generating the features.

Training Data Set

The wearable data (mobile tech in the ABCD study) comprised 5275 individuals. Data on the children
selected for analysis were included. These individuals were used for linking 21 days of the wearable
data using a unique identifier (subjectkey in the ABCD study) to generate the training data set.
Figure 1 shows the study population and the amount of the training data for each group, and
eMethods 3 in Supplement 1 describes the detailed procedure for making the training data sets.

Statistical Analysis
Python software, version 3.7.10 (Python Software Foundation), was used for all analyses. Categorical
variables are presented as count (percentage); continuous variables are presented as mean (SD).
After conducting the Kolmogorov-Smirnov test to determine the distribution of the data set, the
independent sample test for gaussian or the Mann-Whitney test for nongaussian distribution was
used. Statistical significance was indicated by a 2-sided P value <.05 in this study.
The training data set was split into 70% for the training set, 10% for the validation set, and 20%
for the hold-out test set. Considering that the proportion of the prediction class was highly
imbalanced at 6.62% for ADHD and 1.84% for sleep problems in the training data set, the training set
comprised a strategy of the hyperensemble Smote Undersampled Random Forest (hyperSMURF)
algorithm.27 The hyperSMURF generates the training set by dividing the imbalanced data into N
partitions with a maintained ratio (eg, a stratified k-fold) and balancing the ratio using the synthetic

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 3/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

Table 1. Characteristics of the Training Data Set for Attention-Deficit/Hyperactivity Disorder and Sleep Problems

Mean (SD)
Data set for ADHD Data set for sleep problems
Variables Control (n = 1011) ADHD (n = 79) P value Control (n = 3346) Sleep problems (n = 68) P Value
Demographic characteristic
Sex, No. (%)
Male 513 (50.7) 55 (69.6) 1725 (51.6) 38 (55.9)
<.001 .007
Female 498 (49.3) 24 (30.4) 1621 (48.4) 30 (44.1)
Daily summarized features based on the wearable data
30-s–related features, min
Duration
Of being asleep 217.1 (36.3) 213.3 (37.2) .003 217.9 (36.9) 217.0 (36.0) .60
Of in-bed sleep 245.7 (41.3) 241.8 (42.2) .007 246.4 (41.7) 247.4 (40.7) .39
Of deep sleep 41.7 (17.3) 40.7 (17.7) .14 41.6 (17.7) 41.7 (16.6) .77
Of light sleep 127.1 (27.0) 123.8 (26.2) .001 127.6 (26.9) 128.9 (27.2) .20
Of REM sleep 48.3 (15.2) 48.8 (16.0) .58 48.7 (15.3) 46.5 (15.4) <.001
Of waking for short periods 28.6 (9.8) 28.5 (9.9) .80 28.5 (9.7) 30.5 (9.5) <.001
Of being asleep during a nap 37.5 (37.2) 39.0 (38.2) .25 37.1 (36.5) 47.0 (40.7) <.001
Of being in bed during a nap 44.9 (42.6) 46.4 (44.0) .42 44.7 (41.9) 55.9 (46.5) <.001
Rate, %
Of deep sleep 19.0 (7.0) 18.8 (7.2) .50 18.9 (7.1) 19.1 (6.9) .48
Of light sleep 58.6 (8.4) 58.2 (8.5) .19 58.7 (8.5) 59.4 (8.1) .01
Of REM sleep 22.3 (6.5) 22.9 (6.7) .02 22.4 (6.4) 21.5 (6.6) <.001
Of wake 13.3 (4.3) 13.4 (4.4) .28 13.2 (4.2) 14.1 (4.2) <.001
Duration
Of deep sleep stage 83.4 (34.7) 81.5 (35.5) .14 83.3 (35.4) 83.3 (33.2) .77
Of light sleep stage 254.2 (54.1) 247.5 (52.5) .001 255.2 (53.9) 257.7 (54.3) .20
Of REM sleep stage 57.3 (19.7) 57.0 (19.7) .80 97.3 (30.6) 92.9 (30.9) <.001
Of being wake stage 96.6 (30.5) 97.6 (32.0) .58 57.0 (19.3) 60.9 (18.9) <.001
Quality of sleep, % 86.7 (4.3) 86.6 (4.4) .04 86.8 (4.2) 85.9 (4.2) <.001
60-s–related features, min
Duration
Of being asleep 473.9 (76.4) 465.4 (79.8) .001 475.6 (77.0) 477.1 (73.4) .47
Of restlessness 28.4 (15.1) 28.8 (15.0) .36 27.7 (15.0) 29.8 (14.8) <.001
Of being wake 2.6 (4.6) 2.8 (3.8) .06 2.6 (4.9) 2.6 (3.8) .30
Quality of sleep, % 93.9 (3.1) 93.7 (3.1) .04 94.0 (3.1) 93.6 (3.0) <.001
Calorie-related features, kcal
Difference
Between Harris-Benedict and BMRa −699.4 (411.6) −769.2 (450.6) <.001 −680.2 (506.5) −661.7 (379.5) .18
Between Katch-McArdle and BMRa −842.0 (416.4) −902.1 (457.6) <.001 −826.8 (446.0) −804.6 (381.3) .13
Between Mifflin-St Jeor and BMRa −352.5 (409.0) −404.3 (449.6) <.001 −337.4 (449.3) −314.7 (373.8) .09
Sum of calories 2050.9 (489.3) 2120.3 (517.4) <.001 2023.9 (487.4) 2026.2 (437.5) .35
Heart rate–related features, bpm
Acrophaseb 11.1 (16.4) 11.5 (15.4) .26 11.2 (16.6) 11.4 (12.8) .80
Amplitudeb 12.0 (4.2) 12.4 (4.3) .83 12.1 (4.2) 12.4 (4.3) .17
MESORb 82.7 (7.5) 84.4 (8.0) <.001 83.3 (7.5) 83.7 (8.2) .20
Goodness of fit, %b 45.7 (19.0) 46.9 (19.1) .04 46.3 (19.1) 48.3 (19.0) .002
Bedtime
Maximum 110.1 (15.0) 111.3 (15.7) .07 110.4 (15.0) 112.6 (14.7) <.001
Minimum 58.3 (6.8) 58.9 (7.0) .23 58.7 (6.8) 58.9 (6.9) .36
Mean 72.8 (8.5) 74.4 (9.0) <.001 73.2 (8.5) 74.1 (9.1) .04
Daytime
Maximum 137.5 (16.3) 139.2 (16.6) .001 137.4 (15.9) 137.6 (15.5) .51
Minimum 59.8 (6.3) 59.8 (6.4) .12 60.0 (6.3) 59.8 (6.5) .30
Mean 88.4 (8.7) 89.9 (9.4) <.001 89.0 (8.7) 89.0 (9.3) .94

(continued)

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 4/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

Table 1. Characteristics of the Training Data Set for Attention-Deficit/Hyperactivity Disorder and Sleep Problems (continued)

Mean (SD)
Data set for ADHD Data set for sleep problems
Variables Control (n = 1011) ADHD (n = 79) P value Control (n = 3346) Sleep problems (n = 68) P Value
Maximum 101.8 (7.6) 103.5 (8.5) <.001 102.2 (7.6) 102.9 (8.0) .11
Minimum 69.1 (7.3) 70.2 (7.7) .003 69.6 (7.4) 70.0 (7.9) .18
Mean 82.4 (8.0) 84.0 (8.5) <.001 83.0 (8.0) 83.4 (8.4) .27
Variance 240.7 (105.7) 247.1 (111.0) .10 241.3 (103.5) 242.2 (99.3) .70
Standard deviation 15.2 (3.3) 15.3 (3.5) .10 15.2 (3.2) 15.3 (3.1) .70
Intensity-related features
Maximum 2.3 (0.9) 2.3 (0.9) .05 2.2 (0.9) 2.1 (0.9) .01
Minimumc 0.0 (0.0) 0.0 (0.0) >.99 0.0 (0.0) 0.0 (0.0) >.99
Mean 0.3 (0.1) 0.3 (0.1) .03 0.3 (0.1) 0.2 (0.1) .007
MET-related features, min
Maximum 7.1 (1.6) 7.0 (1.4) .28 7.1 (1.6) 7.1 (1.5) .43
Minimum 1.0 (0.0) 1.0 (0.0) .42 1.0 (0.0) 1.0 (0.0) .31
Average 1.6 (0.3) 1.6 (0.3) <.001 1.6 (0.3) 1.6 (0.3) .03
Duration
Of light activity 105.6 (39.1) 99.1 (38.7) <.001 106.4 (39.9) 108.7 (38.7) .20
Of moderate activity 196.6 (92.0) 211.7 (104.4) <.001 198.3 (91.3) 194.2 (87.3) .23
Of sedentary activity 385.7 (108.8) 393.8 (119.4) .17 384.4 (111.2) 405.9 (103.3) <.001
Of vigorous activity 15.0 (23.0) 17.8 (25.1) .007 14.6 (22.7) 11.5 (20.1) <.001
Steps-related features, steps
Acrophaseb 112.7 (27.3) 114.5 (27.1) .009 113.0 (27.4) 113.8 (33.7) .40
Amplitudeb 444.7 (202.8) 475.4 (228.9) .53 442.7 (197.3) 408.2 (206.2) .10
MESORb 397.4 (166.0) 412.2 (185.3) .08 395.1 (160.3) 364.1 (162.9) <.001
Goodness of fit, %b 29.2 (10.9) 30.0 (10.9) .03 29.6 (10.9) 30.3 (10.4) .04
Sum of bedtime 543.0 (737.7) 541.2 (749.1) .26 545.8 (752.5) 564.5 (612.1) <.001
Sum of daytime 9111.4 (4952.7) 9585.7 (5620.1) .09 9039.5 (4824.0) 8221.0 (4770.3) <.001
Sum of stepsd 9654.3 (5041.8) 10127.0 (5750.4) .11 9585.3 (4918.1) 8785.5 (4811.5) <.001
IS, 0-2 0.2 (0.2) 0.2 (0.3) <.001 0.2 (0.2) 0.2 (0.2) .03
IV, 0-1 1.0 (0.4) 1.0 (0.4) .003 1.0 (0.4) 1.0 (0.4) .92
L5, h 1.7 (3.9) 1.7 (3.5) .18 1.7 (4.1) 2.1 (4.0) <.001
M10 846.8 (412.5) 911.9 (463.0) <.001 840.4 (402.5) 770.6 (409.6) <.001
RA, 0-1 1.0 (0.0) 1.0 (0.0) .33 1.0 (0.0) 1.0 (0.0) <.001
b
Abbreviations: ADHD, attention-deficit/hyperactivity disorder; BMR, basal metabolic These are the estimated values of the cosinor analysis to reflect the circadian rhythm.
rate; IS, interdaily stability; IV, intradaily variability; L5, least-active 5-hour period; M10, Goodness of fit represents how much it fits the cosine graph of the 24-hour period
most active 10-hour period; MESOR, midline estimating statistics of rhythm; MET, in %.
metabolic equivalent; RA, relative activity; REM, rapid eye movement. c
The intensity daily minimum was recorded as 0.0 during sleep, making it impossible to
a
Detailed equations are listed in eEquations 1 to 3 in the eAppendix in Supplement 1. calculate the P value.
Negative values for these features indicate that the daily calorie consumption is higher d
The equation for each value is shown in the eAppendix in Supplement 1.
than the estimated BMR.

minority oversampling technique for the minority class and undersampling for the majority class. For
predicting each diagnosis, the ML models consisted of random forest, extreme gradient boosting,
and light gradient-boosting machine (LightGBM) that were generated 50 times to estimate an
average performance. The model evaluation metrics included area under the receiver operating
characteristic curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative
predictive value (NPV). The best model was estimated using the validation set and measured PPV.
The Shapley Additive Explanations values were calculated to determine the importance of
features.28-30

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 5/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 Figure 1. Study Population and Formation of Training Dataset

A Attention-deficit/hyperactivity disorder B Sleep problems

6571 ABCD study release 3.0 6571 ABCD study release 3.0
from November 2, 2020 from November 2, 2020
4085 Control 5858 Control
461 ADHD 293 Sleep problems

5481 Excluded 3157 Excluded

2603 Control has symptoms 1062 Control has symptoms
2025 Other diagnoses 1482 Other diagnoses
355 ADHD with comorbidities 140 Sleep problems with comorbidities
498 Wearable data missing 473 Wearable data missing
JAMA Network Open | Psychiatry

1090 Final study inclusion 3414 Final study inclusion

1011 Control 3346 Control
79 ADHD 68 Sleep problems

Downloaded From: https://jamanetwork.com/ on 11/14/2023

Merge with 21 d of wearable data Merge with 21 d of wearable data

12 348 Final training data inclusion 39 160 Final training data inclusion
11 530 Control 38 438 Control
818 Sleep problems 722 Sleep problems

8890 Train set 988 Validation set 2470 Test set 28 195 Train set 3123 Validation set 7832 Test set
8301 Control 923 Control 2306 Control 27 675 Control 3075 Control 7688 Control
589 ADHD 65 ADHD 164 ADHD 520 Sleep problems 58 Sleep problems 144 Sleep problems

Attention-deficit/hyperactivity disorder (A) and sleep problems (B) populations. The final training data inclusion represents the generated wearable data linked (merged) with each corresponding diagnosis in individuals following a
unique identifier. ABCD indicates Adolescent Brain Cognitive Development; ADHD, attention-deficit/hyperactivity disorder.

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted)

March 17, 2023
Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

6/14
 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

Results
Participants
Of the 6571 children with screening data, a total of 1090 individuals (16.6%) were included for the
ADHD analysis, 79 (1.2%) as diagnoses (mean [SD] age, 144.5 [8.1] months; 55 [69.6%] males) and
1011 (15.4%) as controls (mean [SD] age, 144.3 [7.7] months; 513 [50.7%] males), using the parent
interviews from the K-SADS. A total of 3414 individuals (52.0%) were included for the sleep problems
analysis, 68 (1.0%) as diagnoses (mean [SD] age, 143.5 [7.5] months; 38 [55.9%] males) and 3346
(50.9%) as controls (mean [SD] age, 143.5 [7.7] months; 1725 [51.6%] males), using the child
interviews from the K-SADS.

Baseline Characteristics of the Training Data Set

The training data set consisted of 12 348 entries for ADHD analysis (818 [6.62%] with ADHD and
11 530 [93.38%] controls) and 39 160 for sleep problems analysis (722 [1.84%] with sleep problems
and 38 438 [98.16%] controls). The training data set consisted of 79 individuals with ADHD, of whom
55 (69.6%) were male and 24 (30.4%) were female, and 68 individuals with sleep problems, of
whom 38 (56.0%) were male and 30 (44.0%) were female (Table 1; eTable 1 in Supplement 1). When
comparing the ADHD diagnoses with the controls, reported as mean (SD), the 30-second sleep
records showed that, in the ADHD cohort, the duration of in-bed sleep was 3.9 (0.9) minutes shorter
(P = .007) and the duration of light sleep was 3.3 (0.8) minutes shorter (P = .001). In addition, the
duration of being asleep was 8.5 (3.4) minutes shorter (P = .001) in the 60-second sleep records; the
heart rate records showed that the bedtime heart rate was 1.6 (0.5) bpm higher (P < .001) and
daytime heart rate was 1.5 (0.7) bpm higher (P < .001). Moreover, the maximum heart rate was 1.7
(0.9) bpm higher (P < .001), mean heart rate was 1.6 (0.5) bpm higher (P < .001), and midline
estimating statistics of rhythm heart rate was 1.7 (0.5) bpm higher (P < .001). In addition, the sum of
calories in the ADHD cohort was 69.4 (28.1) kcal higher (P < .001) in the calorie records. The steps
records showed that the duration of light activity was 6.5 (0.4) minutes shorter (P < .001) in the
ADHD cohort and the duration of moderate activity was 15.1 (12.4) minutes longer (P < .001).
When comparing the participants with sleep problems diagnoses with the controls, the
30-second sleep records showed that the duration of rapid eye movement sleep was 2.2 (0.1)
minutes shorter (P < .001) and the waking for short periods was 2.0 (0.2) minutes longer (P < .001).
Furthermore, the rate of rapid eye movement sleep was 0.9% (0.2) shorter (P < .001) and the rate
of wake sleep was 0.9% (0.0%) higher (P < .001). In addition, the duration of restlessness was 2.1
(0.2) minutes longer (P < .001) and the quality of sleep was 0.4% (0.1%) lower (P < .001) in
60-second sleep records. Regarding the heart rate records, the bedtime maximum in children with
sleep disorders was 2.2 (0.3) bpm higher (P < .001) than in the controls. The metabolic equivalents–
related records showed that the duration of sedentary activity was 21.5 (7.9) minutes longer
(P < .001) and the duration of vigorous activity was 3.1 (2.6) minutes shorter (P < .001). Regarding
the steps records, the sum of bedtime was 18.7 (140.4) steps higher (P < .001), the sum of daytime
was 818.5 (53.7) steps lower (P < .001), and the sum of steps was 799.8 (106.6) steps lower
(P < .001).

Model Performance
The performance of the ML models for predicting the ADHD and sleep problems was measured from
the mean and highest PPV. In the ADHD diagnosis group, the LightGBM model showed the highest
average performance: AUC was 0.791 (95% CI, 0.790-0.792), sensitivity was 0.718 (95% CI,
0.715-0.722), specificity was 0.716 (95% CI, 0.715-0.717), PPV was 0.152 (95% CI, 0.152-0.153), and
NPV was 0.973 (95% CI, 0.972-0.973). In the sleep problems group, the LightGBM model showed
the highest average performance: AUC was 0.735 (95% CI, 0.735-0.736), sensitivity was 0.718 (95%
CI, 0.715-0.720), specificity was 0.628 (95% CI, 0.628-0.629), PPV was 0.035 (95% CI,

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 7/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

0.035-0.036), and NPV was 0.992 (95% CI, 0.992-0.993). The detailed average performance of the
ML models is given in Table 2.
Figure 2 shows the average predictive performance of the 3 ML models and the highest
predictive performance of the ML model. The highest performance was measured from the best
LightGBM model for both the ADHD and sleep problems diagnoses. For the ADHD group, the highest
performance AUC was 0.798, sensitivity was 0.756, specificity was 0.716, PPV was 0.159, and NPV
was 0.976. For the sleep problems group, AUC was 0.737, sensitivity was 0.743, specificity was 0.632,
PPV was 0.036, and NPV was 0.992.

Feature Interpretation
The feature importance was measured by the Shapley Additive Explanations values. Figure 3 shows
the top 20 important features by the best predictive model for each diagnosis. For the ADHD group,
the predictive model included sex (55 [69.6%] males; P < .001), midline estimating statistics of
rhythm heart rate (mean [SD], 84.4 [8.0] bpm; P < .001), maximum heart rate (mean [SD], 103.5
[8.5] bpm; P < .001), daytime minimum heart rate (mean [SD], 59.8 [6.4] bpm; P = .12), and duration
of being asleep 60 seconds (mean [SD], 465.4 [79.8] minutes; P = .001). For the sleep problems
group, the predictive model included duration of being asleep during a nap, 30 seconds (mean [SD],

Table 2. Average Performance Evaluations of Machine Learning Modelsa

Average performance, mean (SD) [95% CI]

Diagnosis Model AUC Sensitivity Specificity PPV NPV
ADHD RF 0.739 (0.011) 0.730 (0.023) 0.634 (0.012) 0.124 (0.004) 0.971 (0.002)
[0.736-0.742] [0.724-0.736] [0.630-0.637] [0.123-0.125] [0.970-0.971]
XGB 0.789 (0.004) 0.719 (0.014) 0.716 (0.005) 0.153 (0.003) 0.973 (0.001)
[0.788-0.790] [0.715-0.723] [0.714-0.717] [0.152-0.153] [0.972-0.973]
LGB 0.791 (0.003) 0.718 (0.014) 0.716 (0.004) 0.152 (0.003) 0.973 (0.001)
[0.790-0.792] [0.715-0.722] [0.715-0.717] [0.152-0.153] [0.972-0.973]
Sleep problems RF 0.641 (0.007) 0.813 (0.018) 0.373 (0.006) 0.024 (0.001) 0.991 (0.001)
[0.639-0.643] [0.808-0.818] [0.372-0.375] [0.024-0.025] [0.990-0.991]
XGB 0.724 (0.003) 0.712 (0.008) 0.614 (0.002) 0.033 (0.000) 0.991 (0.000)
[0.724-0.725] [0.710-0.714] [0.614-0.615] [0.033-0.034] [0.991-0.992]
LGB 0.735 (0.002) 0.718 (0.009) 0.628 (0.002) 0.035 (0.000) 0.992 (0.000)
[0.735-0.736] [0.715-0.720] [0.628-0.629] [0.035-0.036] [0.992-0.993]
a
Abbreviations: ADHD, attention-deficit/hyperactivity disorder; AUC, area under the Each model was trained 50 times per diagnosis; subsequently, all performances of each
receiver operating characteristics curve; LGB, light gradient-boosting machine; NPV, model were listed and the mean was calculated.
negative predictive value; PPV, positive predictive value; RF, random forest; XGB,
extreme gradient boosting.

Figure 2. Area Under the Receiver Operating Characteristic (AUC) Curves of Target Diagnoses Prediction

A Attention-deficit/hyperactivity disorder prediction B Sleep problems prediction

1.0 1.0

0.8 0.8 Predictions for attention-deficit/hyperactivity disorder

(A) and sleep problems (B). The dashed line presents
the average performance of the machine learning
0.6 0.6 model. AUC (95% CI) findings for the ADHD
Sensitivity

Sensitivity

population were 0.739 (0.736-0.742) for random

forest, 0.789 (0.788-0.790) for extreme gradient
0.4 0.4 boosting (XGBoost), 0.791 (0.790-0.792) for light
gradient-boosting machine (LightGBM), and 0.798 for
Best model (LightGBM) best model (LightGBM). AUC (95% CI) findings for the
LightGBM
XGBoost
sleep problems population were 0.704 (0.701-0.707)
0.2 0.2
Random forest for random forest, 0.717 (0.714-0.719) for XGBoost,
0.726 (0.723-0.730) for LightGBM, and 0.737 for best
0 0
0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1.0 model (LightGBM). Each 95% CI was calculated by the
1 – Specificity 1 – Specificity list of AUC values from machine learning models.

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 8/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

47.0 [40.7] minutes; P < .001); duration of sedentary activity, metabolic equivalents (mean [SD],
405.9 [103.3] minutes; P < .001); sum of calories (mean [SD], 2026.2 [437.5] kcal; P = .35);
amplitude, steps (mean [SD]; 408.2 [206.2] steps; P = .10); and heart rate, SD (mean [SD], 15.3 [3.1]
bpm; P = .70). The detailed average importance of the features is reported in eTable 2 in
Supplement 1.

Discussion
This study aimed to predict ADHD and sleep problems in children based on wearable device data,
anthropometrics, sex, and K-SADS data from the ABCD study. To our knowledge, this study is the first
attempt to construct an ML model to predict the diagnosis of specific psychiatric disorders using
wearable data from the ABCD study. In previous studies, the prevalence of various sleep-related
problems increased in patients with ADHD31-33; children with ADHD showed longer light sleep and
had poorer quality of sleep than those in the control cohorts.34 In this study, the patients with ADHD
showed shorter sleep times and higher intensities of daytime activity compared with the controls.
The problems of attention, arousal, and sleep control of patients with ADHD share some common
features regarding neuroanatomical functions.33-35
Children report various types of sleep problems,36 and all participants classified as having sleep
problems in the ABCD study reported insomnia. However, children with sleep problems napped
longer, were more sedentary, and engaged in less vigorous activity than those in the control group.
This finding suggests that the sleep problem diagnoses show digital phenotypes that contradict
circadian and homeostatic physiologic processes.37

Figure 3. Summary of Shapley Additive Explanations Importance by the Best Prediction Model

A Attention-deficit/hyperactivity disorder B Sleep problems

Top 20 features Top 20 features

Sex Duration of being asleep during a nap, 30 s

MESOR, HR Duration of sedentary activity, METs
Maximum, HR Sum of calories
Daytime minimum, HR Amplitude, steps
Duration of being asleep, 60 s Standard deviation, HR
Duration of light activity, METs Difference between Harris and BMRs
Minimum, HR Sum of bedtime, steps
Average, HR Acrophase, steps
MESOR, steps IS, steps
Amplitude, steps Duration of light activity, METs
Sum of calories Sex
M10, steps Duration of vigorous activity, METs
Rate of light sleep stage, 30 s Daytime average, HR
Daytime average, HR MESOR, HR
Bedtime maximum, HR Goodness of fit, HR
Maximum, METs Goodness of fit, steps
Rate REM sleep stage, 30 s Maximum, HR
IS, steps MESOR, steps
Quality of sleep, 60 s Acrophase, HR
Duration of being asleep during a nap, 30 s Bedtime maximum, HR

0 0.25 0.50 0.75 1.00 1.25 0 0.50 1.00 1.50 2.00

Average importance on model output Average importance on model output

The lists of feature importance for attention-deficit/hyperactivity disorder (A) and sleep heart rate; IS, interdaily stability; M10, most active 10-hour period; MESOR, midline
problems (B) are arranged in descending order. The higher value (average importance) estimating statistics of rhythm; METs, metabolic equivalents; and REM, rapid eye
indicates that the feature affects the model output (probability) to be higher. BMR movement.
indicates basal metabolic rate; Harris, Harris-Benedict equation for calculating BMR; HR,

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 9/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

In this study, it was possible to confirm the diagnostic potential by using the processed data to
ML training by applying circadian analysis to various data acquired with a wearable device. In
addition, the ML analysis method enabled us to identify features that could predict mental
disorders.38-41 We found that many of the top-ranked features for ADHD prediction were variables
related to heart rate. Heart rate is one of the indicators that can estimate autonomic nervous system
activity42,43; an increase in mean heart rate reflects the relative dominance of the sympathetic
nervous system.44 The highest-ranked features for predicting sleep problems were variables related
to nap and activity patterns. The difference in feature importance patterns between the ADHD and
sleep problems groups helps to understand sleep problems based on the digital phenotypes and
provides hints for practical therapeutic intervention.
Considering the general use of wearable devices, missing values are inevitable. Compared with
other models, the extreme gradient boosting and LightGBM models are more useful in predicting
diagnoses because they can reflect such missing values. However, we should consider that the
precision is relatively low compared with other prediction performances, and there are many false-
positives. These results suggest that there may be limitations in advancing psychiatric diagnosis
performance centering on wearable data. Wearable devices, like passive digital phenotypes, have the
advantage of being able to automatically and invisibly secure data in children’s daily lives. Therefore,
the results of this study are most useful for the early detection of high-risk groups or disease
screening in daily life rather than the application to diagnose ADHD or sleep problems in the
medical field.
In previous ABCD studies, predictions using survey data were found to have relatively high
accuracy with an AUC of 0.80 or higher.22,45-47 However, while questionnaires directly evaluate the
characteristics of diseases and require intention and effort, wearable data can be a useful digital
health technology because they can provide information related to life patterns or physiologic
characteristics in daily life.48 In particular, because wearable data derive clinical value by processing
data based on circadian rhythm, they are more meaningful.49,50

Limitations
This study has limitations. First, there was a highly imbalanced class problem due to insufficient
prediction classes. Therefore, the model should be strengthened by securing additional cohorts.
Second, wearable device data do not guarantee accurate measurement values. Therefore, the
wearable data may contain errors before processing.51 For example, wearable devices tend to be
underestimated in a controlled environment and overestimated in a free environment.52 Third, the
diagnosis of mental disorders should be multifaceted based on expert knowledge and clinical
experience. Therefore, the ML model based on the diagnosis using the K-SADS may be limited.53
Fourth, wearable devices can measure some sleep-related variables directly; however,
overestimation must be considered when interpreting ML performance results. Nevertheless, the
circadian characteristic feature data were useful for predicting ADHD and sleep problems in
children.41,54-57

Conclusions
In this diagnostic study of ADHD and sleep problems in children, we were able to develop a predictive
ML model that appears to be applicable for early detection or screening using digital phenotypes.
Generating and processing circadian rhythm–based features and using ML models for predicting
ADHD and sleep problems is beneficial for early intervention and prevention in daily life. The
accumulation of clinical and digital phenotyping data will enable future scholars to make advanced
prediction models and develop useful criteria for the early assessment and intervention of psychiatric
disorders in children.

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 10/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

ARTICLE INFORMATION
Accepted for Publication: January 31, 2023.
Published: March 17, 2023. doi:10.1001/jamanetworkopen.2023.3502
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2023 Kim WP
et al. JAMA Network Open.
Corresponding Author: Chul-Hyun Cho, MD, PhD, Department of Psychiatry, Korea University College of
Medicine, 73, Goryeodae-ro, Seongbuk-gu, Seoul, 02841, South Korea (david0203@gmail.com).
Author Affiliations: LumanLab Inc, R&D Center, Seoul, South Korea (W.-P. Kim, Lim); Department of Psychiatry,
Chungnam National University Sejong Hospital, Sejong, South Korea (H.-J. Kim); Department of Biotechnology and
Bioinformatics, Korea University, Sejong, South Korea (Pack); Department of Psychiatry, Korea University College
of Medicine, Seoul, South Korea (Cho, Lee); Department of Biomedical Informatics, Korea University College of
Medicine, Seoul, South Korea (Cho); Chronobiology Institute, Korea University, Seoul, South Korea (Cho, Lee).
Author Contributions: Dr Cho had full access to all the data in the study and Mr W. Kim and Dr Lim take
responsibility for the integrity of the data and the accuracy of the data analysis. Mr W. Kim and Dr H. Kim
contributed equally to this work.
Concept and design: W.-P. Kim, Pack, Lim, Cho, Lee.
Acquisition, analysis, or interpretation of data: W.-P. Kim, H.-J. Kim, Lim, Cho.
Drafting of the manuscript: W.-P. Kim, Lim, Cho.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: W.-P. Kim, H.-J. Kim, Lim, Cho.
Obtained funding: Pack, Lim, Cho.
Administrative, technical, or material support: W.-P. Kim, Lim, Cho.
Supervision: H. Kim, Lim, Cho, Lee.
Conflict of Interest Disclosures: None reported.
Funding/Support: This work was supported by a National Research Foundation (NRF) of Korea grant funded by
the Ministry of Science and Information and Communications Technology, Government of Korea (NRF-
2020R1C1C1007463 and NRF-2021R1A5A8032895), and Information and Communications Technology and
Future Planning for Convergent Research in the Development Program for R&D Convergence over Science and
Technology Liberal Arts (NRF-2022M3C1B6080866) (Dr Cho).
Role of the Funder/Sponsor: The funding organizations had no role in the design and conduct of the study;
collection, management, analysis, and interpretation of the data; preparation, review, or approval of the
manuscript; and decision to submit the manuscript for publication.
Disclaimer: This article reflects the views of the authors and may not reflect the opinions or views of the National
Institutes of Health or the ABCD study consortium investigators. The ABCD consortium investigators designed
and implemented the study and/or provided data but did not necessarily participate in the analysis or writing of
this report.
Data Sharing Statement: See Supplement 2.
Additional Information: A full list of supporters is available at https://abcdstudy.org/?s=nIH+collaborators. A
listing of participating sites and a complete listing of the study investigators can be found at https://abcdstudy.
org/principal-investigators.html. The Adolescent Brain Cognitive Development (ABCD) study was supported by
the National Institutes of Health and additional federal partners under award numbers U01DA041022,
U01DA041025, U01DA041028, U01DA041048, U01DA041089, U01DA041093, U01DA041106, U01DA041117,
U01DA041120, U01DA041134, U01DA041148, U01DA041156, U01DA041174, U24DA041123, and U24DA041147.

REFERENCES
1. World Health Organization. The World Health Report 2001: mental health: new understanding, new hope. 2001.
Accessed January 13, 2023. https://apps.who.int/iris/handle/10665/42390
2. American Psychological Association. Neurodevelopmental Disorders: DSM-5 Selections. American Psychiatric
Publications; 2015.
3. Kieling R, Rohde LA. ADHD in children and adults: diagnosis and prognosis. Curr Top Behav Neurosci. 2012;
9:1-16.
4. Sonuga-Barke EJ, Koerting J, Smith E, McCann DC, Thompson M. Early detection and intervention for attention-
deficit/hyperactivity disorder. Expert Rev Neurother. 2011;11(4):557-563. doi:10.1586/ern.11.39

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 11/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

5. Polanczyk G, de Lima MS, Horta BL, Biederman J, Rohde LA. The worldwide prevalence of ADHD: a systematic
review and metaregression analysis. Am J Psychiatry. 2007;164(6):942-948. doi:10.1176/ajp.2007.164.6.942
6. Stein MA, Mendelsohn J, Obermeyer WH, Amromin J, Benca R. Sleep and behavior problems in school-aged
children. Pediatrics. 2001;107(4):E60. doi:10.1542/peds.107.4.e60
7. Spencer TJ. ADHD and comorbidity in childhood. J Clin Psychiatry. 2006;67(suppl 8):27-31.
8. DuPaul GJ, Langberg JM. Educational Impairments in Children With ADHD. The Guilford Press; 2015.
9. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children,
Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common
diagnoses. J Am Acad Child Adolesc Psychiatry. 2000;39(1):28-38. doi:10.1097/00004583-200001000-00014
10. Kaufman J, Birmaher B, Brent D, et al. Schedule for Affective Disorders and Schizophrenia for School-Age
Children—Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc
Psychiatry. 1997;36(7):980-988. doi:10.1097/00004583-199707000-00021
11. Lilienfeld SO, Treadway MT. Clashing diagnostic approaches: DSM-ICD versus RDoC. Annu Rev Clin Psychol.
2016;12:435-463. doi:10.1146/annurev-clinpsy-021815-093122
12. Jain SH, Powers BW, Hawkins JB, Brownstein JS. The digital phenotype. Nat Biotechnol. 2015;33(5):462-463.
doi:10.1038/nbt.3223
13. Shah RN, Berry OO. The rise of venture capital investing in mental health. JAMA Psychiatry. 2020;78(4):
351-352. doi:10.1001/jamapsychiatry.2020.2847
14. Torous J, Kiang MV, Lorme J, Onnela J-P. New tools for new research in psychiatry: a scalable and customizable
platform to empower data driven smartphone research. JMIR Ment Health. 2016;3(2):e16. doi:10.2196/
mental.5165
15. Ometov A, Shubina V, Klus L, et al. A survey on wearable technology: history, state-of-the-art and current
challenges. Comput Netw. 2021;193:108074. doi:10.1016/j.comnet.2021.108074
16. Miralles I, Granell C, Díaz-Sanahuja L, et al. Smartphone apps for the treatment of mental disorders: systematic
review. JMIR Mhealth Uhealth. 2020;8(4):e14897. doi:10.2196/14897
17. Torous J, Onnela JP, Keshavan M. New dimensions and new tools to realize the potential of RDoC: digital
phenotyping via smartphones and connected devices. Transl Psychiatry. 2017;7(3):e1053-e1053. doi:10.1038/tp.
2017.25
18. Karcher NR, Barch DM. The ABCD study: understanding the development of risk for mental and physical health
outcomes. Neuropsychopharmacology. 2021;46(1):131-142. doi:10.1038/s41386-020-0736-6
19. Chaarani B, Hahn S, Allgaier N, et al; ABCD Consortium. Baseline brain function in the preadolescents of the
ABCD study. Nat Neurosci. 2021;24(8):1176-1186. doi:10.1038/s41593-021-00867-9
20. Bernanke J, Luna A, Chang L, Bruno E, Dworkin J, Posner J. Structural brain measures among children with and
without ADHD in the Adolescent Brain and Cognitive Development study cohort: a cross-sectional US
population-based study. Lancet Psychiatry. 2022;9(3):222-231. doi:10.1016/S2215-0366(21)00505-8
21. Gray JC, Schvey NA, Tanofsky-Kraff M. Demographic, psychological, behavioral, and cognitive correlates of
BMI in youth: findings from the Adolescent Brain Cognitive Development (ABCD) study. Psychol Med. 2020;50
(9):1539-1547. doi:10.1017/S0033291719001545
22. Michelini G, Barch DM, Tian Y, Watson D, Klein DN, Kotov R. Delineating and validating higher-order
dimensions of psychopathology in the Adolescent Brain Cognitive Development (ABCD) study. Transl Psychiatry.
2019;9(1):261. doi:10.1038/s41398-019-0593-4
23. ABCD Study. Study sites. Accessed February 6, 2023. https://abcdstudy.org/study-sites/
24. Mutak A. Cosinor2: extended tools for cosinor analysis of rhythms. R package, version 02. October 15, 2018.
Accessed February 6, 2023. https://cran.r-project.org/web/packages/cosinor2/index.html
25. Geoffroy PA, Etain B, Bellivier F. More comprehensive models are needed to understand how relative
amplitude might affect wellbeing and risk of mood disorders. Lancet Psychiatry. 2018;5(9):697. doi:10.1016/
S2215-0366(18)30254-2
26. Van Someren EJ, Swaab DF, Colenda CC, Cohen W, McCall WV, Rosenquist PB. Bright light therapy: improved
sensitivity to its effects on rest-activity rhythms in Alzheimer patients by application of nonparametric methods.
Chronobiol Int. 1999;16(4):505-518. doi:10.3109/07420529908998724
27. Schubach M, Re M, Robinson PN, Valentini G. Imbalance-aware machine learning for predicting rare and
common disease-associated non-coding variants. Sci Rep. 2017;7(1):2959. doi:10.1038/s41598-017-03011-5
28. Lundberg SM, Nair B, Vavilala MS, et al. Explainable machine-learning predictions for the prevention of
hypoxaemia during surgery. Nat Biomed Eng. 2018;2(10):749-760. doi:10.1038/s41551-018-0304-0

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 12/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

29. Lundberg SM, Erion G, Chen H, et al. From local explanations to global understanding with explainable AI for
trees. Nat Mach Intell. 2020;2(1):56-67. doi:10.1038/s42256-019-0138-9
30. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. Accessed February 12, 2023. https://
papers.nips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf
31. Melegari MG, Vittori E, Mallia L, et al. Actigraphic sleep pattern of preschoolers with ADHD. J Atten Disord.
2020;24(4):611-624. doi:10.1177/1087054716672336
32. van der Heijden KB, Smits MG, Gunning WB. Sleep-related disorders in ADHD: a review. Clin Pediatr (Phila).
2005;44(3):201-210. doi:10.1177/000992280504400303
33. Konofal E, Lecendreux M, Cortese S. Sleep and ADHD. Sleep Med. 2010;11(7):652-658. doi:10.1016/j.sleep.
2010.02.012
34. Díaz-Román A, Hita-Yáñez E, Buela-Casal G. Sleep characteristics in children with attention deficit
hyperactivity disorder: systematic review and meta-analyses. J Clin Sleep Med. 2016;12(5):747-756. doi:10.5664/
jcsm.5810
35. Owens JA. Sleep disorders and attention-deficit/hyperactivity disorder. Curr Psychiatry Rep. 2008;10(5):
439-444. doi:10.1007/s11920-008-0070-x
36. Carter KA, Hathaway NE, Lettieri CF. Common sleep disorders in children. Am Fam Physician. 2014;89(5):
368-377.
37. Borbély AA, Daan S, Wirz-Justice A, Deboer T. The two-process model of sleep regulation: a reappraisal. J Sleep
Res. 2016;25(2):131-143. doi:10.1111/jsr.12371
38. Cho C-H, Lee T, Kim M-G, In HP, Kim L, Lee HJ. Mood prediction of patients with mood disorders by machine
learning using passive digital phenotypes based on the circadian rhythm: prospective observational cohort study.
J Med internet Res. 2019;21(4):e11029. doi:10.2196/11029
39. Duda M, Ma R, Haber N, Wall DP. Use of machine learning for behavioral distinction of autism and ADHD. Transl
Psychiatry. 2016;6(2):e732-e732. doi:10.1038/tp.2015.221
40. Barnett I, Torous J, Staples P, Keshavan M, Onnela J-P. Beyond smartphones and sensors: choosing
appropriate statistical methods for the analysis of longitudinal data. J Am Med Inform Assoc. 2018;25(12):
1669-1674. doi:10.1093/jamia/ocy121
41. Lee H-J, Cho C-H, Lee T, et al. Prediction of impending mood episode recurrence using real-time digital
phenotypes in major depression and bipolar disorders in South Korea: a prospective nationwide cohort study.
Psychol Med. 2022;1-9. Published online September 23, 2022. doi:10.1017/S0033291722002847
42. Scheer FA, Kalsbeek A, Buijs RM. Cardiovascular control by the suprachiasmatic nucleus: neural and
neuroendocrine mechanisms in human and rat. Biol Chem. 2003;384(5):697-709. doi:10.1515/BC.2003.078
43. Oldehinkel AJ, Verhulst FC, Ormel J. Low heart rate: a marker of stress resilience. the TRAILS study. Biol
Psychiatry. 2008;63(12):1141-1146. doi:10.1016/j.biopsych.2007.12.006
44. Tonhajzerova I, Ondrejka I, Adamik P, et al. Changes in the cardiac autonomic regulation in children with
attention deficit hyperactivity disorder (ADHD). Indian J Med Res. 2009;130(1):44-50.
45. Goldstone A, Javitz HS, Claudatos SA, et al. Sleep disturbance predicts depression symptoms in early
adolescence: initial findings from the adolescent brain cognitive development study. J Adolesc Health. 2020;66
(5):567-574. doi:10.1016/j.jadohealth.2019.12.005
46. Fors PQ, Barch DM. Differential relationships of child anxiety and depression to child report and parent report
of electronic media use. Child Psychiatry Hum Dev. 2019;50(6):907-917. doi:10.1007/s10578-019-00892-7
47. Funkhouser CJ, Chacko AA, Correa KA, Kaiser AJE, Shankman SA. Unique longitudinal relationships between
symptoms of psychopathology in youth: a cross-lagged panel network analysis in the ABCD study. J Child Psychol
Psychiatry. 2021;62(2):184-194. doi:10.1111/jcpp.13256
48. Nisenson M, Lin V, Gansner M. Digital phenotyping in child and adolescent psychiatry: a perspective. Harv Rev
Psychiatry. 2021;29(6):401-408. doi:10.1097/HRP.0000000000000310
49. Lee HA, Lee HJ, Moon JH, et al. Comparison of wearable activity tracker with actigraphy for sleep evaluation
and circadian rest-activity rhythm measurement in healthy young adults.Psychiatry Investig. 2017;14(2):179-185.
doi:10.4306/pi.2017.14.2.179
50. Cho CH, Lee HJ. Applying circadian rhythm concepts in digital healthcare. Chronobiol Med. 2021;3(1):1-3. doi:10.
33069/cim.2021.0006
51. Jeong S, Seo JY, Jeon S, et al. Circadian rhythm of heart rate assessed by wearable devices tends to correlate
with the circadian rhythm of salivary cortisol concentration in healthy young adults. Chronobiol Med. 2020;2(3):
109-114. doi:10.33069/cim.2020.0022

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 13/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023

 JAMA Network Open | Psychiatry Machine Learning–Based Prediction of ADHD and Sleep Problems Among Children

52. Feehan LM, Geldman J, Sayre EC, et al. Accuracy of Fitbit devices: systematic review and narrative syntheses
of quantitative data. JMIR Mhealth Uhealth. 2018;6(8):e10527. doi:10.2196/10527
53. Dukart J, Weis S, Genon S, Eickhoff SB. Towards increasing the clinical applicability of machine learning
biomarkers in psychiatry. Nat Hum Behav. 2021;5(4):431-432. doi:10.1038/s41562-021-01085-w
54. Arns M, Kooij JJS, Coogan AN. Identification and management of circadian rhythm sleep disorders as a
transdiagnostic feature in child and adolescent psychiatry. J Am Acad Child Adolesc Psychiatry. 2021;60(9):
1085-1095. doi:10.1016/j.jaac.2020.12.035
55. Zerón-Rugerio MF, Carpio-Arias TV, Ferreira-García E, et al. ADHD subtypes are associated differently with
circadian rhythms of motor activity, sleep disturbances, and body mass index in children and adolescents: a case-
control study. Eur Child Adolesc Psychiatry. 2021;30(12):1917-1927. doi:10.1007/s00787-020-01659-5
56. Coogan AN, Baird AL, Popa-Wagner A, Thome J. Circadian rhythms and attention deficit hyperactivity
disorder: the what, the when and the why. Prog Neuropsychopharmacol Biol Psychiatry. 2016;67:74-81. doi:10.
1016/j.pnpbp.2016.01.006
57. Bondopadhyay U, Diaz-Orueta U, Coogan AN. A systematic review of sleep and circadian rhythms in children
with attention deficit hyperactivity disorder. J Atten Disord. 2022;26(2):149-224. doi:10.1177/1087054720978556

SUPPLEMENT 1.
eMethods 1. Study Population Inclusion and Exclusion Criteria
eMethods 2. Process of Making Wearable Dataset
eMethods 3. Process of Making Training Dataset
eTable 1. Basic Characteristics and Demographics of the ABCD Study Participants
eTable 2. Top 10 Shapley Additive Explanations (SHAP) Importance Summary
eAppendix. Equations

SUPPLEMENT 2.
Data Sharing Statement

JAMA Network Open. 2023;6(3):e233502. doi:10.1001/jamanetworkopen.2023.3502 (Reprinted) March 17, 2023 14/14

Downloaded From: https://jamanetwork.com/ on 11/14/2023