International Journal of Psychophysiology 116 (2017) 32–44

Contents lists available at ScienceDirect

International Journal of Psychophysiology

journal homepage: www.elsevier.com/locate/ijpsycho

Game-based combined cognitive and neurofeedback training using Focus

Pocus reduces symptom severity in children with diagnosed AD/HD and
subclinical AD/HD
Stuart J. Johnstone a,b,⁎, Steven J. Roodenrys a,b, Kirsten Johnson a, Rebecca Bonfield a, Susan J. Bennett a,b
a
School of Psychology, University of Wollongong, Wollongong, NSW, Australia
b
Early Start Research Institute, University of Wollongong, Wollongong, NSW, Australia

a r t i c l e i n f o a b s t r a c t

Article history: Previous studies report reductions in symptom severity after combined working memory (WM) and inhibitory
Received 23 May 2016 control (IC) training in children with AD/HD. Based on theoretical accounts of the role of arousal/attention mod-
Received in revised form 21 February 2017 ulation problems in AD/HD, the current study examined the efficacy of combined WM, IC, and neurofeedback
Accepted 24 February 2017
training in children with AD/HD and subclinical AD/HD. Using a randomized waitlist control design, 85 children
Available online 28 February 2017
were randomly allocated to a training or waitlist condition and completed pre- and post-training assessments of
Keywords:
overt behavior, trained and untrained cognitive task performance, and resting and task-related EEG activity. The
AD/HD training group completed twenty-five sessions of training using Focus Pocus software at home over a 7 to 8-week
Training period. Trainees improved at the trained tasks, while enjoyment and engagement declined across sessions. After
Inhibitory control training, AD/HD symptom severity was reduced in the AD/HD and subclinical groups according to parents, and in
Working memory the former group only according to blinded teachers and significant-others. There were minor improvements in
Neurofeedback two of six near-transfer tasks, and evidence of far-transfer of training effects in four of five far-transfer tasks. Fron-
EEG tal region changes indicated normalization of atypical EEG features with reduced delta and increased alpha activ-
ity. It is concluded that technology developments provide an interesting a vehicle for delivering interventions and
that, while further research is needed, combined WM, IC, and neurofeedback training can reduce AD/HD symp-
tom severity in children with AD/HD and may also be beneficial to children with subclinical AD/HD.
© 2017 Elsevier B.V. All rights reserved.

1. Introduction the electroencephalogram, or EEG). Electrodes on the scalp measure

Children with Attention-Deficit/Hyperactivity Disorder (AD/HD)
show developmentally inappropriate levels of hyperactive, impulsive,
and inattentive behavior (American Psychiatric Association, 2013).
Two non-pharmacological and technology-based approaches to reduc-
ing symptoms are cognitive training and neurofeedback training. Cogni-
tive training aims to improve particular psychological abilities (e.g.,
attention, memory) with practice via purpose-designed computer soft-
ware often in the form of games. These games must balance engage-
ment/fun (which assists with training compliance) with challenge/
learning, and typically include performance feedback with task difficul-
ty level varied according to performance. Neurofeedback training aims
to promote awareness and control of psychological “state” factors that
can be measured non-invasively through brain electrical activity (i.e.
⁎ Corresponding author at: School of Psychology, University of Wollongong,
Wollongong, NSW, Australia.
E-mail addresses: sjohnsto@uow.edu.au (S.J. Johnstone), steven@uow.edu.au
(S.J. Roodenrys), kej436@uowmail.edu.au (K. Johnson), rb983@uowmail.edu.au
(R. Bonfield), sbennett@uow.edu.au (S.J. Bennett).
the EEG with online processing of the raw signal allowing simplified
feedback about that ongoing activity to be provided to the user (e.g.,
current level of attention displayed numerically, in a bar graph, or as
an engaging visual display).
Recent meta-analytic studies in the cognitive training (Cortese et al.,
2015; Rapport et al., 2013; Robinson et al., 2014; Rutledge et al., 2012;
Sonuga-Barke et al., 2013; Sonuga-Barke et al., 2014) and
neurofeedback (NF) training areas (Arns et al., 2009; Arns et al., 2013;
Holtmann et al., 2014; Lofthouse et al., 2012; Micoulaud-Franchi et al.,
2014) report mixed outcomes following both approaches and point to
the need for improvement in study designs, control conditions, and
sample sizes. The vast majority of studies are single-component, in
that they investigate the efficacy of an intervention made up of only
one “active ingredient” (e.g. diet, working memory training, attention
training, slow-cortical potential NF training). Given the heterogeneous
nature of AD/HD, multicomponent approaches may result in increased
efficacy. Interventions that target multiple training components may re-
sult in enhanced outcomes and optimize the transfer of effects from
cognitive deficits to symptoms, and indeed stronger effects have been
reported for multi-component training (Cortese et al., 2015). Previous

http://dx.doi.org/10.1016/j.ijpsycho.2017.02.015
0167-8760/© 2017 Elsevier B.V. All rights reserved.
 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
multi-component training studies in younger children (4–5 years)
targeting inhibitory control, working memory, attention, visuo-spatial
abilities, planning, and motor skills (Halperin et al., 2013), and atten-
tion, working memory and impulsive behaviors (Re et al., 2015), have
reported encouraging outcomes. Studies examining multi-component
training in older children with AD/HD are limited. One study examined
working memory, inhibition, and cognitive flexibility training using a
double-blind placebo controlled design, and reported improvement on
visuospatial short-term memory and working memory (WM) in a
“full-active” condition and improved inhibition and interference control
in both the full-active and partially-active (i.e. primarily training inhibi-
tion and cognitive-flexibility) conditions, but no improvements for
teacher- or parent-rated AD/HD behaviors (Dovis et al., 2015).
Our training approach aims to reduce the symptoms of AD/HD by
improving processes that underpin the symptoms. It targets multiple
components; fundamental cognitive processes such as WM and inhibi-
tory control (IC), and psychological state factors such as attention and
relaxation via NF (Johnstone, 2013), and hence is termed
“neurocognitive” training. Our earlier cognitive training studies exam-
ined two of these components (WM and IC) and reported positive influ-
ences on AD/HD symptoms (Johnstone et al., 2012b; Johnstone et al.,
2010). That work built on findings showing that training cognitive pro-
cesses in those with below par ability can result in improved day-to-day
behavior; e.g. WM training for AD/HD (Holmes et al., 2010; Klingberg et
al., 2005); IC training for behavioral control issues, such as alcohol con-
sumption and eating behavior (Allom et al., 2016). Here we examine a
multi-component approach to AD/HD symptom reduction which is sup-
ported by the cognitive energetic model (CEM) of AD/HD (Sergeant,
2000). The CEM highlights the role of state-regulation dysfunction hin-
dering efficient engagement of computational/cognitive processes and
executive functions. According to this model, increased control over
psychological/arousal states should provide the necessary foundation
for effective engagement and use of cognitive processes, such as inhibi-
tion and working memory, to guide behavior. Our training targets (i.e.
psychological state and cognitive processes) were chosen based on evi-
dence supporting them as core issues for the majority of children with
AD/HD and the dynamic interplay between these factors suggested by
the CEM, with empirical support (Sergeant, 2005). Although, like all
models of AD/HD, the CEM model is based on the correlations between
cognitive functions and AD/HD symptoms, intervention studies can pro-
vide evidence for the causal nature of the relationship. Given the
established relationship between deficits in levels of the CEM and AD/
HD symptoms, our training approach targets multiple levels of the
model to improve abilities with subsequent behavioral implications. In-
deed, NF training itself may have an influence at several levels; “the
neurophysiological (enhancement of regulation capability of different
EEG parameters), neuropsychological (executive functions), and the
cognitive-behavioral (e.g. enhanced self-regulation by positive rein-
forcement of goal-directed behavior)”, with training effects likely due
to a combination of these variables (Albrecht et al., 2015, p. 5). These au-
thors argue that cognitive deficits (e.g. preparation, response inhibition)
and resting-state abnormalities may be improved with NF training, and
also that developing self-regulation skills may help with motivational
problems, another problem area for children with AD/HD (Sonuga-
Barke, 2011). Our group has recently published case-study data of mul-
tiple children supporting this combined WM/IC/NF approach in children
with AD/HD in China (Jiang and Johnstone, 2015). The current study
seeks to extend previous research by considering the efficacy of the
neurocognitive training approach (i.e. combined cognitive and NF train-
ing) using a randomized controlled design in a large sample.
In addition to expected improvement in the training tasks, we were
interested in the effect of the training on tasks involving similar process-
es but sharing few surface features. Thus, near-transfer (i.e. extension of
training-related improvements to related, but not trained, tasks) was
examined via the auditory Go-Nogo, auditory oddball, visual Counting
Span, verbal Digit Span, and active EEG tasks, while far-transfer (i.e.
33
generalization to domains that are considerably different from the train-
ing task) (Barnett and Ceci, 2002) was examined via behavior rating
scales and resting EEG tasks. Resting EEG is considered far-transfer as
this ability/process is not trained by the neurocognitive training ap-
proach, but is a measure that reliably differentiates children with AD/
HD from controls (Barry et al., 2003) and shows sensitivity to behavioral
change brought about by various treatments (Johnstone et al., 2012b;
Jiang and Johnstone, 2015).
In addition to children with diagnosed and clinically significant AD/
HD, we wanted to examine the efficacy of the intervention for those
with milder symptoms or sub-threshold AD/HD. These children display
mild to moderate attentive and/or hyperactive-impulsive problems but
do not meet diagnostic criteria for AD/HD, with a prevalence rate in
non-referred samples estimated to be 30–50% (Althoff et al., 2006;
Kóbor et al., 2012). These mild cases have problems in their day-to-
day lives and may need therapeutic help (Kóbor et al., 2012). Non-phar-
macological interventions are of particular importance to this group as
they are unlikely to be recommended a pharmacological treatment
and early intervention may reduce the impact of the mild symptoms
and/or prevent further functional decline.
For children in the training condition, it was predicted that there will
be (a) improved performance on the training tasks, and (b) stable en-
joyment and engagement across sessions and behavioral improvement
across sessions, as indicated by within training evaluations by parents -
in both the AD/HD and subclinical groups. For training outcomes, it was
predicted that both the AD/HD and subclinical groups in the training
condition compared to the waitlist control condition would show (c)
improvement in AD/HD symptoms post-training as indicated by the
Conners 3rd Edition Parent and ADHD-RS scales, (d) improvement in
general behavior as indicated by the Child Behavior Checklist, (e) im-
provement in cognitive task performance on lab-based tasks involving
similar processes to the training tasks, and (f) EEG normalization - in
the child AD/HD context, increased fast wave and decreased slow
wave activity - as indicated by resting EEG tasks.
2. Methods
2.1. Participants
Parents of 132 potential participants expressed interest in the study,
with 25 excluded based on strict exclusion criteria, as outlined below. A
total of 107 participants were randomized and completed the initial as-
sessment session, with 22 (10 from the Training condition and 12 from
the Waitlist condition; M = 9.42 years) ceasing participation due to
slow training rate, change in family circumstances, illness, or other com-
mitments. There were no significant differences between the 22 chil-
dren who dropped out and the 85 children who completed the
training on the full-scale or subscale scores of the WIAT-II, Conners 3-
P, or CBCL. The final sample consisted of 85 children; 44 children with
a previous (within previous 24 months) professional diagnosis by a clin-
ical psychologist or child psychiatrist of AD/HD based on DSM-IV criteria
(22 in Training condition, 22 in Waitlist condition; 31 male; M =
9.81 years; range 7.3–12.8 years) and 41 children without a diagnosis
but displaying similar behavior, termed the Subclinical group (22 in
Training condition, 19 in Waitlist condition; 33 male; M = 9.53 years;
range 7.4–12.6 years). Children in the Subclinical group scored in the
borderline range on both the Inattention and Hyperactivity/Impulsivity
subscales of Conners 3-P. Children were randomly assigned to a Waitlist
control (WL) or Training (TR) condition, based on a random permuta-
tion calculator (http://www.randomizer.org). Fig. 1 shows the CON-
SORT flow diagram. Twenty-five children from the AD/HD group were
taking medication as treatment for the disorder, with 9 in the TR condi-
tion (3 Ritalin, 3 Dexamphetamine; 2 Concerta, 1 multiple medications)
and 16 in the WL condition (6 Ritalin, 4 Dexamphetamine; 1 Strattera, 5
Concerta). None of the children in the Subclinical group were on any
similar or other significant medications.
34 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
Fig. 1. CONSORT flow diagram. Stage-wise representation of the progress of participants through the RCT. T1 = time 1, T2 = time 2.
Participants were excluded if they had suffered epileptic seizures,
periods of unconsciousness, serious head injuries, had a significant psy-
chological disorder, and/or displayed lower than expected academic
abilities as measured by the WIAT-II or did not display behaviors similar
to AD/HD. Participants were recruited from the Illawarra and southern
Sydney region via advertisements in primary school newsletters, emails
to various groups (e.g. university staff, psychology students, private psy-
chologists), promotional flyers placed in university psychology clinic,
and a local newspaper article. The University of Wollongong (UOW)
and Illawarra Shoalhaven Local Health District Health and Medical
Human Research Ethics Committee (HE12/420), and State Educational
Research Approval Process, approved the research protocol prior to
the commencement of data collection. The research was conducted in
accordance with the approved guidelines, and each participant's par-
ent(s) gave written informed consent.
2.2. Materials
2.2.1. Behavior rating instruments
The ADHD Rating Scale IV (ADHD-RS) was used to assess AD/HD
symptom severity (DuPaul et al., 1998), has sound psychometric prop-
erties (Zhang et al., 2005), and is closely related to AD/HD DSM-IV diag-
nostic criteria. Our modified ADHD-RS consisted of 18 items, drawn
from the symptom listings in DSM-IV (American Psychiatric
Association, 1994), with frequency ratings made on a 5-point Likert
scale (0 ‘Never’ to 4 ‘Always’), and a score range of 0–72. This instru-
ment was completed by three sources (two blinded) at pre-training
and post-training.
The 43-item Conners 3rd Edition Parent School Form (Conners 3-P)
measured characteristic behavior as perceived by a parent. The Conners
3-P is a widely used measure for assessing parental reports of problem-
atic behaviors and other concerns for children 6–18 years (Conners,
2008), and has high internal consistency (α = 0.89) and test-retest re-
liability (r = 0.86) (Conners, 2008).
The Child Behavior Checklist 6–18 years (CBCL) was used to measure
emotional and behavioral problems as rated by their parent(s). The
CBCL has high test-retest reliability (r = 0.95), sound internal consisten-
cy (all subscales N α = 0.78), strong content validity (p b 0.001 all sub-
scales), and both the AD/HD and Attention problems subscales of the
CBCL correlate highly with the DSM-IV checklist for AD/HD symptoms
(r = 0.80) (Achenbach and Rescorla, 2001).
The Wechsler's Individual Achievement Test 2nd edition Australian
Abbreviated (WIAT-II) was employed to assess achievement in Word
Reading, Numerical Operations, and Spelling. The WIAT-II has been
shown to hold high internal consistency (total composite corrected
r = 0.97) and test–retest stability (total composite corrected r =
0.97), with moderate to high construct- and criterion-validity
(Wechsler, 2007).
2.2.2. Experimental tasks
A battery of tasks to measure cognitive abilities and resting/active
EEG were administered at the pre- and post-training sessions. All exper-
imental tasks were presented on a 17″ computer monitor located 1 m
from the child at eye level. An auditory Go-Nogo (10 practice trials,
100 experimental trials, 30% Nogo stimuli, inter-stimulus-interval
1250 ms, “Press to frequent Go tone”), and auditory oddball task (10
practice trials, 100 experimental trials, 70% Non-target stimuli, inter-
stimulus-interval 2150 ms, “Press to infrequent target tone”) were com-
pleted, followed by a visual Counting Span (Engle et al., 1999) and audi-
tory Digit Span task. There were four EEG tasks. The resting eyes-open
(“fixate on smiley face on-screen”) and eyes-closed tasks (“close eyes
until told to open them”) lasted for 2 min each, while the focus (“con-
centrate as hard as possible on the smiley face”) and relax (“keep your
eyes open and try to relax”) tasks lasted for 1 min each.
2.2.3. EEG recording (pre- and post-training)
A 14-channel wireless EEG headset device (Emotiv EPOC®
Neuroheadset) was used to record continuous scalp EEG at 128 Hz
from 12 sites (AF3, F7, F3, FC5, T7, P7, P8, T8, FC6, F4, F8, and AF4) of
the 10–20 system at the pre- and post-training assessment sessions.
The device was modified to provide stimulus markers for ERP analysis,
which will be reported elsewhere, sacrificing the O1 and O2 channels
(Badcock et al., 2013). The device has been validated in adults and chil-
dren and produces auditory ERP components highly comparable to re-
search-grade EEG systems in adults (Badcock et al., 2013) and
children (Badcock et al., 2015).
 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
2.2.4. Data quantification
Raw EEG data from the EPOC device was divided into 2 s epochs
which were baseline corrected using Scan Edit software (Scan 4.5,
Compumedics NeuroScan). Epochs containing values exceeding
±100 μV were automatically rejected, and remaining epochs were visu-
ally inspected for any movement and/or muscle artifact or noisy chan-
nels – if identified, these were removed from future processing. These
epochs were Fourier transformed using a 10% Welch window, to obtain
absolute spectral power in four frequency bands: delta (0.5–3.75 Hz),
theta (3.75–7.5 Hz), alpha (7.5–12.5 Hz) and beta (12.5–25 Hz). Rela-
tive power was calculated by dividing each band into the total of the
four bands. Four regions (Left Frontal: AF3, F3, FC5, F7; Right Frontal:
AF4, F4, F8, FC6; Left Temporal–Parietal: T7, P7; Right Temporal-
Parietal: T8, P8) were formed for analysis.
2.3. Procedure
Recruitment strategies resulted in parents of potential participants
downloading an information pack from a university website. Those in-
terested completed an initial contact form via a secure online survey,
and were contacted to determine the participant's suitability. Following
verbal consent, parents were informed of the condition to which their
child had been randomly assigned and an initial assessment session ar-
ranged. Two researchers were present at each testing session taking
place in a quiet single-purpose laboratory at the university or in a
quiet location at the participant's home.
2.3.1. Pre-training assessment (Time 1)
Each participant was individually assessed during a session taking
1.5 h. Participants taking medication for AD/HD were asked to abstain
for 24 h prior to the session. The procedure was explained to the par-
ent/caregiver and participant and informed written consent was obtain-
ed. The parent/caregiver completed a demographics sheet, the CBCL,
Conners 3-P, and ADHD-RS. Two additional ADHD-RS forms were pro-
vided for the child's classroom teacher and a “significant-other” (indi-
vidual with regular contact with the child – typically a grandparent,
aunt/uncle, neighbor), with a coversheet instructing parents not to re-
veal information about the study and/or condition allocation. Thus, all
efforts were made to ensure that the teacher and significant-other
were blind to condition membership.
The WIAT-II, Digit Span, and Counting Span tasks were completed,
followed by fitting the EEG headset (positioned in accordance with
the 10–20 electrode system). Adequate electrode impedance, as indicat-
ed by the Emotiv Control Panel software (1.0.0.5 Premium), was obtain-
ed prior to recording. The Go-Nogo and oddball task were then
completed (counterbalanced order) followed by the four EEG tasks
(counterbalanced order), with different presentation orders at the
pre- and post-assessment sessions.
2.3.2. Computerized training
During the pre-training session for the TR condition, or post-training
session for the WL condition, the training hardware and software were
installed on the participant's computer, and demonstrated to the partic-
ipant and their parent(s). Participants were asked to complete 25 ses-
sions at home over a period of 6 to 8 weeks at a frequency of 3 to 4
sessions/week, consistent with previous studies (Dovis et al., 2015;
Jiang and Johnstone, 2015; Johnstone et al., 2012b; Johnstone et al.,
2010). Each training session took approximately 20 min to complete, and
took place in a designated quiet location in the participant's home -
typically, the child's bedroom or a study - to minimize distractions.
2.3.2.1. Hardware. The portable, dry-sensor, “Mindwave” EEG device
consisted of microchips, embedded firmware, a 10 mm active electrode,
and ear-clip reference ground electrode (ThinkGear, Neurosky, San Jose,
California, USA) contained within a headset (MindWave, Neurosky, San
Jose, California, USA). EEG was recorded continuously from site Fp1 at
35
256 Hz, with on-board conversion of the raw signal from the time- to
the frequency-domain via a fast Fourier transform to allow calculation
of EEG power in the delta, theta, alpha, and beta frequency bands. Pro-
prietary algorithms calculated values representing two independent
psychological state dimensions of “Attention” (low to high; highly cor-
related with power in the beta EEG band) and “Relaxation” (tense to
calm; highly correlated with power in the alpha EEG band). These mea-
sures were presented as a value between 0 and 100, enabling the provi-
sion of generalized feedback about ongoing brain activity in a form
understood by children, and a robust, universal, index of ongoing EEG
activity that does not require individual calibration. An additional
index, termed “Zen”, was calculated in the software by averaging the at-
tention and relaxation indices. These indices were sent to the computer
or iOS device wirelessly via Bluetooth for use by the software.
The device measured EEG during the entire training session and was
used to (a) control game-play during NF games and (b) quantify atten-
tion level during WM and IC games. The device measured attention level
during the WM and IC games and categorized the game average as
low, medium, high, or very-high as a multiplier for game points
achieved (× 1, × 2, × 3, or × 4, respectively). The device constantly
monitored electrode impedance and provided an ongoing numerical
representation of its quality; if sub-standard impedance occurred at
any point (e.g. device was removed, or vigorous head movement) the
game was paused until acceptable impedance was achieved.
The device EEG shows sensitivity to psychological state variations
relevant to the NF training goals, i.e. high compared to low attention
and high compared to low relaxation (Johnstone et al., 2012a;
Johnstone et al., 2012b), is reliable and valid when compared with re-
search-grade equipment (Johnstone et al., 2012a), and shows good
test-retest reliability (Rogers et al., 2016).
2.3.2.2. Software. Focus Pocus is a themed training software package de-
veloped by Neurocognitive Solutions Pty Ltd. (Australia) using intellec-
tual property licensed from UOW. NF, WM, and IC training games are
embedded in a themed game environment, where the player is a “wiz-
ard in training” working to improve important wizard skills such as
broomstick racing, transformation, potion making, etc. Each training
session consisted of 14 mini-games presented in random order with
four WM, four IC, and six NF games. In each training session, two NF
games were driven by Attention, two by Relaxation, and two by Zen
feedback; in a random order. The WM games involved holding informa-
tion in memory with subsequent recall to complete an action; difficulty
level was adaptive, decreasing by 1 the number of items to be remem-
bered and recalled for less-than-perfect performance on the previous
WM game, and increasing by 1 the number of items to be remembered
and recalled for perfect performance on the previous WM game. The
lowest level of difficulty - the starting level - was 4 items. The IC
games were based on the Go/Nogo paradigm and involved a press/tap
response to frequently presented visual “Go” stimuli and withholding
of response to infrequent “Nogo” stimuli; difficulty level was adaptive,
increasing or decreasing the time available to inhibit the prepotent Go
response by 50 ms based on 1 or more commission errors or no com-
mission errors, respectively, in the previous IC game. The lowest level
of difficulty - the starting level - was 1000 ms. The NF games aimed to
promote awareness and control of brain activity with practice. During
NF games, device EEG was sent to the software via Bluetooth providing
the participant real-time feedback of Attention, Relaxation, or Zen level
as a percentage value on a bar graph and in an engaging visual form. For
example, the player's wizard character's levitation speed in a levitation
game was dependent on the level of relaxation above that particular
player/games current threshold level. The threshold level increased by
5% after successful completion of the previous level and decreased by
5% if the previous level was not successfully completed; the lowest
level of difficulty – the starting level - was 5% and the highest was
95%. For each of the 3 game types (and specifically for Focus, Relaxation,
and Zen within the NF games), level of difficulty was tracked and carried
36 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44

forward into subsequent training sessions. Each of the eight NF games
could be dependent on feedback of Attention, Relaxation, or Zen level,
giving twenty-four unique game types. Performance feedback occurred
after each game in the form of a star (0–5) rating system, and was tied to
accuracy and reaction time (RT) in the IC games, accuracy in the WM
games, and time above threshold in the NF games.
2.3.3. Post-training assessment (Time 2)
The post-training session took place 7–9 weeks after the initial test-
ing session. Participants were instructed to abstain from any stimulant
medication for 24 h prior to testing. All tasks were identical to Time 1,
excluding the WIAT. Parents completed the ADHD-RS, Conners 3-P
and CBCL. Parents of participants in the TR condition completed an eval-
uation of their child's engagement (e.g. “My child was engaged by the
training games”), enjoyment, and behavior during training sessions 1–
6, 7–12, 12–18, and 19–25 using a 5 point Likert scale from 1 Strongly
Disagree to 5 Strongly Agree. Additionally, the same classroom teacher
and significant-other were asked to complete the ADHD-RS again.
2.3.4. Statistical analysis
For behavior ratings and cognitive task performance variables, Con-
dition (TR, WL) and Group (AD/HD, Subclinical) differences at Time 1
were examined using ANOVAs. To assess the effect of training, ANOVAs
considered Condition and Group effects at Time 2, with Time 1 as a co-
variate. Relative EEG power in each frequency band was examined
using repeated measures ANOVAs with between-subjects factors of
Condition and Group and within-subjects factors of Time (1, 2), Sagittal
(frontal, tempo-parietal), and Lateral (left, right) for the EC, EO, Focus,
and Relax tasks separately. Polynomial planned contrasts for the Sagittal
factor compared frontal with tempo-parietal, and for the Lateral factor
compared left with right hemisphere, to assess the topographic distri-
bution of each component. For the TR condition only, parent evaluations
were analyzed using repeated measures ANOVAs (separately for Enjoy-
ment, Engagement, Improvement) with Time (1–6, 7–12, 13–18, 19–
25) as a within-subjects factor (with planned polynomial contrasts for
the Time factor). Planned comparisons were specified a priori and crit-
ical values were corrected using a Bonferroni correction, therefore,
post hoc alpha adjustments were not required (Tabachnick and Fidell,
1996). Eta squared (partial η2) was calculated as a measure of effect
size, with .01, .06, and .14 representing small, medium, and large effects,
respectively (Cohen, 1992). Only statistically significant or near-signifi-
cant results are reported.
3. Results
3.1. Group characteristics
3.1.1. Pre-training behavior ratings
Table 1 shows means and statistical effects for the WIAT-II and
Conners 3-P. The AD/HD group scored lower than the Subclinical
group on the WIAT-II Composite and Reading Comprehension sub-
scales. For the Conners 3-P, the AD/HD group scored higher than the
Subclinical Group on Inattention, Learning Problems, and Executive
Functioning.
Table 2 shows means and statistical effects for the CBCL and ADHD-
RS. For the CBCL, the AD/HD group scored higher than the Subclinical
group on Attention Problems, Aggressive Behavior and Externalizing,
while increased Thought Problems approached significance. A Condi-
tion main effect revealed that the WL condition scored higher than the
TR condition on the Withdrawn subscale. A Group × Condition interac-
tion indicated that Anxious/Depressed subscale scores were slightly
lower for the WL than TR condition for the Subclinical group, but higher
for WL than TR condition for the AD/HD group. Scores from the parent
ADHD-RS were higher in the AD/HD than Subclinical group, with no dif-
ferences present for the teacher and significant-other ratings.
3.1.2. Pre-training cognitive task performance
Digit Span (M = 5.14, SD = 0.97) and Counting Span (M = 4.10,
SD = 1.49) did not show any significant Group, Condition, or interaction
effects at Time 1. For the oddball task, RT to target stimuli (M = 552.98,
SD = 85.48 ms) and percentage of correct target responses (M = 93.48,
SD = 7.31%) did not show Group, Condition, or interaction effects. The
percentage of correct non-target responses was higher in the TR
(M = 95.3, SD = 5.2%) than WL (M = 91.6, SD = 8.7%) condition
(F[1,81] = 5.442, p = 0.022, partial η2 = 0.07). For the Go-Nogo task,
RT to Go stimuli (M = 489.19, SD = 80.10 ms) and percentage of correct
responses to both Go (M = 85.21, SD = 12.57%) and Nogo (M = 60.43,
SD = 21.34%) stimuli did not show effects.

Table 1
WIAT-II and Conners 3-P scores for each Group and Condition at Time 1, with Group effect statistics. WIAT-II scores are standard scores. T-scores are shown for Conners 3-P; T-scores N64
are considered to be in the clinical range, 57–63 are considered to be in the borderline clinical range, b57 are considered to be in the normal range.

Measure Condition Group Group effect

AD/HD Subclinical

WIAT-II TR 87.2 (13.0) 97.6 (13.6) 89.7 (13.7) vs. 97.5 (14.1), F(1,81) = 6.648, p = 0.012, partial η2 = 0.08
Composite WL 92.2 (14.2) 97.4 (15.0)
WIAT-II TR 91.1 (13.9) 100.9 (14.6) 93.3 (14.3) vs. 102.1 (13.7), F(1,81) = 8.446, p = 0.005, partial η2 = 0.09
Reading comprehension WL 95.5 (14.7) 103.4 (12.7)
WIAT-II TR 84.9 (19.0) 94.1 (12.7) Ns
Numerical operations WL 87.4 (15.5) 91.2 (17.7)
WIAT-II TR 88.2 (11.3) 98.2 (15.3) Ns
Spelling WL 95.5 (16.1) 96.7 (16.4)
Conners 3-P TR 80.8⁎ (8.4) 76.7⁎ (10.0) 79.8⁎ (9.1) vs. 74.9⁎ (10.1), F(1,81) = 5.913, p = 0.017, partial η2 = 0.07
Inattention WL 78.9⁎ (9.9) 72.8⁎ (10.1)
Conners 3-P TR 74.0⁎ (15.1) 74.8⁎ (15.5) Ns
Hyperactivity/impulsivity WL 78.2⁎ (13.7) 68.9⁎ (15.2)
Conners 3-P TR 70.8⁎ (12.9) 65.6⁎ (14.6) 69.8⁎ (12.2) vs. 63.2# (12.8), F(1,81) = 6.133, p = 0.015, partial η2 = 0.07
Learning problems WL 68.7⁎ (11.8) 60.4# (10.1)
Conners 3-P TR 73.3⁎ (12.8) 68.9⁎ (10.4) 71.8⁎ (11.2) vs. 66.9⁎ (11.7), F(1,81) = 4.105, p = 0.046, partial η2 = 0.05
Executive functioning WL 70.3⁎ (9.3) 64.7⁎ (12.9)
Conners 3-P TR 66.0⁎ (17.8) 58.2# (16.1) Ns
Aggression WL 64.8⁎ (17.5) 62.8# (16.0)
Conners 3-P TR 70.8⁎ (15.7) 70.3⁎ (17.3) Ns
Peer relations WL 76.4⁎ (17.5) 66.9⁎ (19.8)

Note: #Borderline, ⁎Clinical, Ns not significant.

 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44 37

Table 2
CBCL and ADHD-RS scores for each Group and Condition at Time 1. CBCL T-scores N70 are considered to be in the clinical range, 67–70 are considered to be in the borderline clinical range,
and b67 are considered to be in the normal range. ADHD-RS scores are presented as raw scores – the score range for this scale is 0 to 72.

Measure Cond. Group Group effect Condition effect G × C interaction

AD/HD Subclinical

CBCL TR 58.5 60.2 (8.9) Ns Ns F(1,80) = 4.191,

Anxious/depressed (8.2) p = 0.044, partial
WL 64.4 58.2 (7.8) η2 = 0.05
(10.3)
CBCL TR 55.7 54.4 (4.7) Ns 55.1 (5.8) vs. 58.8 (12.1), F(1,81) = 3.988, Ns
Withdrawn (6.7) p = 0.049, partial η2 = 0.05
WL 58.9 58.8
(9.8) (12.1)
CBCL TR 56.7 58.3 Ns Ns Ns
Somatic (6.0) (14.7)
complaints WL 61.1 58.4 (7.2)
(8.7)
CBCL TR 60.3 61.0 (8.4) Ns Ns Ns
Social problems (7.6)
WL 63.7 59.7 (9.0)
(8.1)
CBCL TR 63.6 59.9 (6.5) 64.8 (10.0) vs. 61.0 (7.7), F(1,81) = 3.464, Ns Ns
Thought problems (9.8) p = 0.066, partial η2 = 0.04
WL 65.9 62.3 (8.8)
(10.4)
CBCL TR 69.9# 68.5# 71.5⁎ (9.5) vs. 66.4 (10.2), F(1,81) = 6.020, Ns Ns
Attention (7.5) (10.2) p = 0.016, partial η2 = 0.07
problems WL 73.2⁎ 64.1
(11.1) (10.0)
CBCL TR 60.0 56.8 (7.8) Ns Ns Ns
Rule-breaking (9.1)
behavior WL 62.2 59.1 (7.1)
(8.6)
CBCL TR 65.3 60.3 (9.0) 66.6 (11.9) vs. 60.7 (9.2), F(1,81) = 6.111, Ns Ns
Aggressive (11.3) p = 0.016, partial η2 = 0.07
behavior WL 67.8# 61.2 (9.8)
(12.6)
CBCL TR 56.3 59.2 (9.5) Ns Ns Ns
Internalizing (9.8)
WL 62.6 57.0
(10.1) (12.0)
CBCL TR 62.6 57.9 63.4 (10.4) vs. 58.3 (10.9), F(1,81) = 4.769, Ns Ns
Externalizing (10.2) (10.4) p = 0.032, partial η2 = 0.06
WL 64.2 58.6
(10.7) (11.7)
ADHD-RS TR 41.7 38.6 43.4 (11.9) vs. 36.9 (13.1), F(1,81) = 5.400, Ns Ns
Parent (11.5) (12.6) p = 0.023, partial η2 = 0.07
WL 45.3 35.3
(12.3) (13.7)
ADHD-RS TR 26.3 32.9 Ns Ns Ns
Teacher (17.0) (14.7)
WL 22.7 27.3
(16.9) (13.2)
ADHD-RS TR 32.7 28.3 Ns Ns Ns
Significant-other (13.1) (14.4)
WL 37.1 33.6
(14.2) (15.7)

Note: Cond. Condition, #Borderline, ⁎Clinical, Ns not significant. Italicized effect descriptions indicate approaching significance.

3.1.3. Pre-training EEG
For the EC task, delta was frontally maximal, theta was equipotential
in the sagittal dimension, and alpha and beta were maximal in the
tempo-parietal region. For the EO and Focus tasks, delta was equipoten-
tial across regions, theta frontally maximal, and alpha and beta maximal
in the tempo-parietal region. For the EC, EO, and Focus tasks there were
no main effects or interactions for Group or Condition. For the Relax
task, alpha and beta were maximal in the tempo-parietal region. A
Sagittal × Group interaction (F[1,81] = 12.679, p = 0.002, partial
η2 = 0.36) indicated that relative delta had a frontal N tempo-parietal
distribution for the AD/HD group, but a tempo-parietal N frontal distri-
bution for the Subclinical group (due mainly to their increased tempo-
parietal activity). A Sagittal × Group interaction (F[1,81] = 8.762, p =
0.007, partial η2 = 0.28) indicated that relative theta had a tempo-
parietal N frontal distribution for the AD/HD group, but a frontal N
tempo-parietal distribution for the Subclinical group (due to their in-
creased activity frontally and reduced activity in the tempo-parietal
region).
3.2. Training data
The 25 sessions of training were completed in an average of
67.3 days, at an average rate of 2.65 sessions/week with no difference
between Groups.
3.2.1. Training effects on trained tasks
Game difficulty level (actual game level divided by the maximum
potential level [WM: 14, IC: 20, NF: 20] × 100) was used as a proxy of
38 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44

task performance and quantified as an average in sessions 1–5, 11–15,

and 21–25. Difficulty level increases were based on perfect performance
at the lower difficulty level. As difficulty increased, the child was re-
quired to put in more effort and ability to maintain the cognitive pro-
cesses and/or psychological state factors to complete the task at a high
level. As shown in Table 3, the difficulty level of the WM and IC tasks in-
creased from sessions 1–5 to 11–15 and plateaued to sessions 21–25.
The difficulty level of the Focus, Relax, and Zen NF tasks showed slight
decreases from sessions 1–5 to 21–25.

3.2.2. Within-training evaluations by parents

As shown in Fig. 2, behavioral improvement showed a Time main ef-
fect of borderline significance (F[3,38] = 2.63, p = 0.054) with planned
contrasts revealing a near-significant linear increase (F[1,40] = 3.783,
p = 0.060, partial η2 = 0.10) in behavioral improvement (sessions 1–
6: M = 4.3, SD = 1.1 vs. sessions 19–25: M = 3.7, SD = 1.34) and no in-
teraction with Group. Engagement showed a Time × Group interaction
(F[3,38] = 5.749, p = 0.003, partial η2 = 0.35) with planned contrasts
revealing a larger linear reduction (F[1,40] = 11.425, p = 0.000, partial
η2 = 0.31) in enjoyment for the AD/HD (sessions 1–6: M = 4.7, SD =
0.47 vs. sessions 19–25: M = 3.5, SD = 1.54) than Subclinical group
(sessions 1–6: M = 4.6, SD = 0.77 vs. sessions 19–25: M = 4.3, SD =
1.10). Enjoyment showed a near-significant Time main effect
(F[3,38] = 2.628, p = 0.067) with planned contrasts revealing a linear
reduction (F[1,40] = 6.535, p = 0.015, partial η2 = 0.16) in enjoyment
(sessions 1–6: M = 4.3, SD = 1.1 vs. sessions 19–25: M = 3.7, SD =
1.34) and no interaction with Group.

3.3. Pre- versus post-training comparisons

Each variable showing a significant Group effect at Time 1 was con-

sidered as a co-variate in the analyses reported below, and was not
found to alter the pattern of significance in the reported results.

3.3.1. Behavior ratings

For the subscales of the Conners 3-P, Condition main effects were
significant for Inattention (F[1,80] = 5.375, p = 0.023, partial η2 =
0.07), Hyperactive/Impulsive (F[1,80] = 9.571, p = 0.003, partial
η2 = 0.11), and Executive Functions (F[1,80] = 12.122, p = 0.001, par-
tial η2 = 0.14), with each showing lower scores at Time 2 for the TR
than WL condition, with Time 1 as a covariate (see Fig. 3). There were
no interactions with Group.
For the subscales of the CBCL, Condition main effects were significant
for Attention Problems (F[1,80] = 5.821, p = 0.018, partial η2 = 0.07),
Aggression (F[1,80] = 5.612, p = 0.020, partial η2 = 0.07), and Exter-
nalizing (F[1,80] = 10.127, p = 0.002, partial η2 = 0.12), with each
showing lower scores at Time 2 for the TR than WL condition, with
Time 1 as a covariate (see Fig. 3). There were no interactions with
Group.
As shown in Fig. 4, the parent ADHD-RS showed a Condition main ef-
Fig. 2. Within-training evaluations of (a) behavior, (b) training engagement, and (c)
fect (F[1,80] = 18.370, p = 0.000, partial η2 = 0.20) with lower scores training enjoyment by parents.
in the TR than WL condition at Time 2 with Time 1 as a covariate, and no

Table 3
Training effects on the difficulty level in the training tasks, across Group.

Task Session 1–5 Session 11–15 Session 21–25 Time effect Contrasts within Time

WM 20.36 (5.25) 25.71 (7.2) 26.07 (6.78) F(2,39) = 26.166, p = 0.000, partial η2 = 0.41 Linear F(1,40) = 54.044, p = 0.000, partial η2 = 0.59
Quadratic F(1,40) = 8.660, p = 0.006, partial η2 = 0.18
IC 14.62 (5.98) 23.0 (19.67) 22.62 (24.78) F(2,39) = 5.286, p = 0.007, partial η2 = 0.12 Linear F(1,40) = 4.860, p = 0.034, partial η2 = 0.12
Quadratic F(1,40) = 6.775, p = 0.013, partial η2 = 0.15
Focus 49.88 (8.24) 45.73 (5.48) 44.95 (5.33) F(2,39) = 7.689, p = 0.002, partial η2 = 0.29 Linear F(1,40) = 13.044, p = 0.001, partial η2 = 0.26
Relax 53.53 (5.50) 51.68 (3.53) 51.0 (3.37) F(2,39) = 3.383, p = 0.045, partial η2 = 0.16 Linear F(1,40) = 6.947, p = 0.012, partial η2 = 0.16
Zen 51.03 (4.73) 48.75 (3.43) 48.13 (3.62) F(2,39) = 6.966, p = 0.003, partial η2 = 0.27 Linear F[1,40] = 13.008, p = 0.001, partial η2 = 0.26

Note: WM working memory; IC inhibitory control.

 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
Fig. 3. Conners 3-P and CBCL effects. Significant Condition differences at Time 2 with Time 1 as a covariate for Conners 3-P (a) and CBCL (b).
interaction with Group. Teacher ADHD-RS showed a significant Group ×
Condition interaction (F[1,80] = 4.331, p = 0.046, partial η2 = 0.13)
with follow-up testing of the Condition effect in each Group separately
showing a significant reduction at Time 2 for the AD/HD (F[1,40] =
8.682, p = 0.009, partial η2 = 0.34) but not Subclinical group
(F[1,40] = 0.210, p = 0.656, partial η2 = 0.01). Significant-other
ADHD-RS showed a Group × Condition interaction (F[1,80] = 5.311,
p = 0.030, partial η2 = 0.18) with follow-up testing of the Condition ef-
fect in each Group separately showing a marginally significant reduc-
tion at Time 2 for the AD/HD (F[1,40] = 4.31, p = 0.057, partial η2 =
0.25) but not Subclinical group (F[1,40] = 0.540, p = 0.48, partial
η2 = 0.05).
3.3.2. Cognitive task performance
There were no effects of Group or Condition or interactions for the
Digit Span and Counting Span memory tasks, oddball task, and 2 of
the 3 Go-Nogo task variables. A near-significant Condition main effect
indicated that the percentage of correct Go stimuli was increased at
Time 2 for the TR condition (M = 88.31, SD = 15.46) compared to the
WL condition (M = 83.14, SD = 20.41) (F[1,80] = 3.483, p = 0.066,
partial η2 = 0.05), with Time 1 as a covariate.
3.3.3. EEG
For the EC task, a Time × Condition × Sagittal interaction (F[1,81] =
4.450, p = 0.048, partial η2 = 0.18) revealed a change from a frontal N
tempo-parietal distribution for relative delta at Time 1 to a tempo-
parietal N frontal distribution at Time 2 for the TR condition (due to a re-
duction in the frontal region and a large increase in the tempo-parietal
region), while the WL condition showed a similar frontal N tempo-pari-
etal distributions at Time 1 and 2. A similar near-significant interaction
(F[1,81] = 3.753, p = 0.067, partial η2 = 0.16) revealed that relative
alpha power had a tempo-parietal N frontal distribution at Time 1 that
was reduced in magnitude at Time 2 for the TR condition (due to an in-
crease in the frontal region and decrease in the tempo-parietal region),
39
while the WL condition showed a very similar tempo-parietal N frontal
topography at Time 1 and 2. These effects are shown in Fig. 5.
For the EO task, no significant Time × Condition interactions, or fur-
ther interactions with Group, Sagittal, or Lateral, were present.
For the Focus task, a Time × Condition × Sagittal interaction
(F[1,81] = 4.545, p = 0.050, partial η2 = 0.29) revealed a change
from a near equipotential (frontal = tempo-parietal) distribution of rel-
ative delta at Time 1 to a tempo-parietal N frontal distribution at Time 2
for the TR condition (due to a large reduction in the frontal region), but a
similar tempo-parietal N frontal distribution at Time 1 and 2 for the WL
condition (due to reduction of a similar magnitude in both regions). This
effect is shown in Fig. 4.
For the Relax task, no significant Time × Condition interactions, or
further interactions with Group, Sagittal, or Lateral, were present.
4. Discussion
The current study used a randomized waitlist control design and a
range of outcome measures to examine the potential efficacy of a
neurocognitive training approach for symptom reduction in children
with AD/HD and Subclinical AD/HD.
Group differences evident pre-training indicated that the AD/HD
group was more severely affected than the Subclinical group. The AD/
HD group displayed poorer overall academic achievement and reading
comprehension (WIAT-II), and more substantial issues with inattention,
learning problems, and executive functions (Conners 3-P), attention
problems, aggressive behavior, and externalizing behaviors (CBCL).
There were no group differences in cognitive task performance. EEG dif-
ferences were evident in the relax task, with the Subclinical group
showing increased tempo-parietal delta activity and increased theta ac-
tivity frontally and reduced activity in the temporal-parietal region. The
resting state EEG literature would suggest increased delta and theta ac-
tivity in an AD/HD group compared to a control group (Barry et al.,
40 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44

Fig. 4. ADHD-RS effects. Significant Condition or Group × Condition differences at Time 2 with Time 1 as a covariate for Parent ADHD-RS (a), teacher ADHD-RS (b) and significant-other
ADHD-RS (c).

2003), but this was a Subclinical group during an active task requiring
spontaneous (i.e. no guidance or feedback) relaxation.
Children in the TR condition showed substantial improvement from
the early- to mid-sessions on the trained WM and IC tasks, with a pla-
teau over the mid- to late-sessions. These findings are in line with our
previous study (Johnstone et al., 2012b), and extend those results to a
Subclinical group. The NF training tasks did not show across-session im-
provement according to level of difficulty. This is an unexpected result,
and may be a consequence of substantial improvement in the first five
sessions (game started at lowest level of difficulty, effectively 5%)
 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44 41

Fig. 5. Significant effects for the EEG tasks. Training versus Waitlist difference as a function of Time for (a) the EC resting task and (b) the Focus task.

resulting in an average difficulty level of above 50% for each Focus,
Relax, and Zen and maintenance of that moderate level of difficulty
across subsequent sessions. The within-training evaluations by parents
revealed that while they saw behavioral improvement across sessions
(which plateaued over the 3rd and 4th quarters), there were reductions
in engagement with the training (larger reduction for AD/HD than Sub-
clinical) and enjoyment of the training across sessions. The training soft-
ware was designed to be engaging and enjoyable for children aged 8–
13 years, with a fully integrated themed environment, variety in each
training session, and industry-standard gaming features such as un-
locks, achievements, and performance feedback. While the gamification
was assumed to be intrinsically motivating, our findings suggest that
the “fun factor” and motivational factors may need to be reconsidered.
A form of extrinsic motivation should be considered for future studies,
perhaps in the form of a token economy or personal best goal-setting,
as in other studies (Dovis et al., 2015). Further, taken together the
42 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
training performance and evaluations findings may suggest that 25 ses-
sions of training are excessive and affects motivation to train and devo-
tion of full energy and resources to training after a period of time.
Alternatively, it could also be argued that 25 sessions may not be
enough to give an observable effect on some of the tasks (Rapport et
al., 2013) (e.g. to transfer to the other cognitive tasks) given this training
approach targets three components (but see below on possible con-
found in the Time 2 session data). Assessing the impact of these factors
on engagement and outcomes should be a focus of future research.
The central training outcome measure was AD/HD symptom severi-
ty as rated by parents, teachers, and a significant-other. ADHD-RS re-
sults indicated that symptom severity was reduced after training, with
the reduction being of a similar magnitude for the AD/HD and Subclin-
ical groups according to parent ratings, and isolated to the AD/HD
group according to teacher and significant-other ratings. The AD/HD
group findings support our previous studies that examined combined
WM and IC training (Johnstone et al., 2012b; Johnstone et al., 2010)
and the neurocognitive approach used in the current study (Jiang and
Johnstone, 2015). Symptom severity reductions were specific to the
AD/HD group according to teachers and a significant-other who were
blinded to condition allocation. Non-blinded raters are likely to be influ-
enced by an expectation bias, and therefore their ratings are not as ob-
jective as the blinded ratings. These findings are similar to our
previous randomized studies in this area, with more specific and isolat-
ed training effects revealed by blinded or probably-blinded raters
(Johnstone et al., 2012b; Johnstone et al., 2010). Results from the
Conners 3-P and CBCL completed by parents revealed more specific be-
havioral/emotional changes post-training, and indicated significant im-
provements in inattention, hyperactivity-impulsivity, executive
functions (Conners 3-P), as well as attention problems, aggression,
and externalizing behaviors (CBCL). Similar CBCL effects were reported
by parents and teachers in our multiple case study (Jiang and Johnstone,
2015). These results indicate that the neurocognitive training approach
has had an influence on core behaviors that are particularly characteris-
tic of AD/HD in clinical or Subclinical form. Dovis et al. used a random-
ized double-blind placebo controlled design with multi-component
gamified training (visuospatial WM, inhibition, and cognitive-flexibili-
ty) and reported no improvements in parent-, teacher-, or child-rated
AD/HD behaviors, executive functions in daily life, motivational behav-
iors, or general problem behaviors (Dovis et al., 2015). These outcome
differences could result from many factors, including differences in ex-
perimental design, the cognitive components targeted for training, the
method and duration of training of each component.
The four cognitive tasks were included to examine the near-transfer
of training effects from the gamified (all visual) training tasks to tasks
involving similar processes but sharing few surface features. There
were no significant changes post-training for the Digit Span and
Counting Span memory tasks, or the attentional oddball task. Although
not statistically significant in most cases, condition means were in the
direction expected for performance measures in the Go-Nogo (e.g. the
speed-accuracy trade-off index, as per Johnstone et al., 2012b, was
more positive indicating slower more accurate responding for TR but
stable for WL condition) and oddball task (e.g. increased accuracy for
non-target stimuli in TR but not WL condition) after training across
groups. Other indices showed almost identical descriptive effects for
both the TR and WL conditions (e.g. stable accuracy to target stimuli
in the oddball task; reduced RT to Go stimuli in the Go-Nogo task;
minor increase in accuracy to Nogo stimuli in the Go-Nogo task). For
the inhibitory Go-Nogo task, the TR condition showed a near-significant
increase in accuracy for Go stimuli post-training compared to the WL
condition. After cognitive training, we have reported improvements in
accuracy of responding to attentional targets in the oddball task, reduc-
tions in RT to incongruent stimuli in the Flanker task - similar findings
have been reported using a Stroop task (Dovis et al., 2015) - and in-
creased Counting Span memory capacity in children with AD/HD
(Johnstone et al., 2012b). It is possible that the overall decline in
engagement in and enjoyment of training carried over to affect motiva-
tion and effort during Time 2 data collection for those in the TR group,
resulting in no improvement compared to Time 1.
Training related changes in EEG were isolated to the eyes-closed
resting and active Focus task. For the EC task, relative delta activity in-
creased in the temporal-parietal region and decreased in the frontal re-
gion post-training, and relative alpha activity decreased in the
temporal-parietal region and increased in the frontal region post-train-
ing for those in the TR condition (both AD/HD and Subclinical groups).
In the Focus task, relative delta activity decreased in the frontal region
post-training for the TR condition, for both the AD/HD and Subclinical
groups. We have previously reported increased relative delta power
(largest in the central region), decreased relative theta power in the
central region, and decreased relative alpha power globally after train-
ing (Johnstone et al., 2010). The topographically specific changes re-
ported here provide evidence of training related changes in EEG under
resting and task conditions, and the frontal region changes could be
interpreted as support for normalization of the atypical EEG features
in AD/HD with reduced delta and increased alpha activity (Barry et al.,
2003).
These results indicate minor improvements in two of the six near-
transfer measures (Go-Nogo and Focus EEG). The limited evidence of
near-transfer makes it difficult to assess what the underlying mecha-
nisms of improvement might be and leaves open the possibility of pla-
cebo-type factors – although the latter are not supported by the
teacher and significant-other ADHD-RS effects. It may be the case that
near-transfer is indeed limited, or that the near-transfer tasks were
not appropriate or optimal, but it is possible that the decline in engage-
ment in and enjoyment of training extended to affect the Time 2 data,
masking any real near-transfer effects. Far-transfer of training effects
was found for four (ADHD-RS, Conners 3-P, CBCL, EC resting EEG) of
the five far-transfer tasks. The blinded behavior ratings indicated that
children in the Subclinical group did not show reductions in symptom
severity, while the training-related improvement in the trained tasks
and changes in cognitive task performance (significant and descriptive)
and EEG were the same for both groups.
5. Limitations of the study
The results of the current study should be interpreted in light of cer-
tain limitations. There was no active control group, and therefore it is
possible that the training effects found here may be attributable to
Hawthorne/placebo effects – however, note that the blinded ratings
from teachers and significant-other argue against this. Secondly, it is
unresolved whether the training effects reported here last beyond the
training period, and if so, for how long. While two previous studies
provide evidence for cognitive (Johnstone et al., 2012b) and
neurocognitive (Jiang and Johnstone, 2015) training effects lasting
beyond the end of training, this should be investigated in future studies.
Thirdly, while children were not on any medications at the pre- and
post-training assessment sessions, 57% of children in the AD/HD group
were on their normal course of AD/HD treatment medication during
training, so there may be an interaction with medication for the training
effects. Ideally, subsequent studies could use medication naive children.
Fourthly, we did not determine if teachers and significant others
remained blind to condition allocation leaving open the possibility
that they were “unblinded” by the child or parent - this should be
addressed in future research. Finally, the children's engagement with
and enjoyment in the training declined across sessions. Future studies
should include a reward system to motivate children to improve and
play at higher levels of difficulty.
6. Conclusions
A pragmatic conclusion from the study relates to the efficacy of new
technology as a vehicle for delivering interventions. Although
 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
computerized cognitive training has been around for some time, recent
developments in innovative wireless EEG recording technologies have
opened up exciting opportunities for applied psychophysiological re-
search. This study demonstrates that it is possible to undertake
neurofeedback training effectively in a comfortable home environment,
and obtain reliable pre- and post-training data, with children as young
as 7 years of age.
The findings from the present study provide evidence for the efficacy
of the neurocognitive training approach for reducing symptom severity
in children with diagnosed AD/HD and Subclinical AD/HD, but the pat-
tern of results raises a number of questions that can only be addressed
through further research. Although the AD/HD symptoms were reduced
after training, and there were clear changes in EEG characteristics, these
were not accompanied by a systematic change in cognitive processing.
This suggests that cognitive factors alone cannot explain the results,
and other factors, such as motivation, may play a role. It may be that
the training has an impact on the motivation of the children in a general
sense, or it may be that emotion and motivation are as much a determi-
nant of transfer from training to other tasks/situations as are the cogni-
tive processes being trained. It is possible that behavior ratings show
more transfer from training than does the laboratory cognitive tasks be-
cause the training games are actually closer to real world behavior than
they are to the laboratory tasks as they include emotional elements such
as frustration and reward for “appropriate behavior” which the labora-
tory tasks do not.
Finally, the cognitive-energetic model that provided a theoretical
basis for the study offers another potential explanation. If the state
factors were relatively similar in the unfamiliar laboratory setting,
changes in inhibitory control may be harder to detect than changes
in behavior in everyday settings, where state factors will be much
more variable. The training regime targets both state energetic
factors and cognitive processes, on the basis that behavior is deter-
mined by the interaction of both. It is not possible to isolate the
effects of the NF and cognitive training in the present data, but equal-
ly, it may be that neither is effective, or as effective, in isolation. More
research is needed that, where possible, addresses the limitations of
the current study and extends the area to consider the unique contri-
butions of the different training components, and their interaction,
to training outcomes.
Declaration of interest
SJ and SR are co-inventors of intellectual property licensed by the
UOW to Neurocognitive Solutions Pty Ltd. and are entitled to a small
portion of royalties received by UOW in relation to the sale of any
product that uses the UOW intellectual property. This intellectual
property makes up a proportion of the intellectual property used in
Focus Pocus. KJ, RB, and SB declare that there are no actual or poten-
tial conflicts of interest including any financial, personal or other
relationships with other people or organizations within 3 years of
beginning the submitted work that could inappropriately influence,
or be perceived to influence, their work. The funding sources did
not have a role in study design, collection, analysis, and interpreta-
tion of data, writing of the manuscript, nor the decision to submit
the article for publication.
Acknowledgements
We thank the participants and families who volunteered for this
study. We are grateful to Melinda Hickey for assistance with the EEG da-
ta collection and processing advice, and Jamila Solomon, Stephanie
Carrigan, Emily Church, and Jacob Sargeant for their assistance with
the recruitment and assessment process. This study was funded by a
University of Wollongong University Research Committee Research
Partnerships Grant with Neurocognitive Solutions Pty Ltd. as the partner
organisation (F-APP-1.11.5).
43
References
Achenbach, T., Rescorla, L., 2001. Manual for the ASEBA School-age Forms & Profiles: An
Integrated System of Multi-informant Assessment Research Center for Children,
Youth, & Families/Achenbach System of Empirically Based Assessment.
Albrecht, B., Sandersleben, H.U., Gevensleben, H., Rothenberger, A., 2015. Pathophysiology
of ADHD and associated problems - starting points for NF interventions? Front. Hum.
Neurosci. 9, 359.
Allom, V., Mullan, B., Hagger, M., 2016. Does inhibitory control training improve health
behaviour? A meta-analysis. Health Psychol. Rev. 10 (2), 168–186.
Althoff, R.R., Copeland, W.E., Stanger, C., Derks, E.M., Todd, R.D., Neuman, R.J., Hudziak, J.J.,
et al., 2006. The latent class structure of ADHD is stable across informants. Twin Res.
Hum. Genet. 9, 507–522.
American Psychiatric Association, 1994. Diagnostic and Statistical Manual of Mental Dis-
orders. fourth ed. American Psychiatric Association, Washington DC.
American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental Dis-
orders. Author, Washington, DC.
Arns, M., de Ridder, S., Strehl, U., Breteler, M., Coenen, A., 2009. Efficacy of neurofeedback
treatment in ADHD: the effects on inattention, impulsivity and hyperactivity: a meta-
analysis. Clin. EEG Neurosci. 40, 180–189.
Arns, M., Heinrich, H., Strehl, U., 2013. Evaluation of neurofeedback in ADHD: the long and
winding road. Biol. Psychol. 95, 108–115.
Badcock, N.A., Mousikou, P., Mahajan, Y., de Lissa, P., Thie, J., McArthur, G., 2013. Valida-
tion of the Emotiv EPOC EEG gaming system for measuring research quality auditory
ERPs. PeerJ 1, e38.
Badcock, N.A., Preece, K.A., de Wit, B., Glenn, K., Fieder, N., Thie, J., McArthur, G., 2015. Val-
idation of the Emotiv EPOC EEG system for research quality auditory event-related
potentials in children. PeerJ 3, e907.
Barnett, S.M., Ceci, S.J., 2002. When and where do we apply what we learn?: a taxonomy
for far transfer. Psychol. Bull. 128, 612–637.
Barry, R.J., Clarke, A.R., Johnstone, S.J., 2003. A review of electrophysiology in attention-
deficit/hyperactivity disorder: I. Qualitative and quantitative electroencephalogra-
phy. Clinical Neurophysiology 114, 171–183.
Cohen, J., 1992. A power primer. Psychol. Bull. 112 (1), 155–159. http://dx.doi.org/10.
1037/0033-2909.112.1.155.
Conners, C.K., 2008. Conners 3rd Edition: Manual. Multi-Health Systems.
Cortese, S., Ferrin, M., Brandeis, D., Buitelaar, J., Daley, D., Dittmann, R.W., Holtmann, M.,
Santosh, P., Stevenson, J., Stringaris, A., Zuddas, A., Sonuga-Barke, E.J., Group, E.A.G,
2015. Cognitive training for attention-deficit/hyperactivity disorder: meta-analysis
of clinical and neuropsychological outcomes from randomized controlled trials.
J. Am. Acad. Child Adolesc. Psychiatry 54, 164–174.
Dovis, S., Van der Oord, S., Wiers, R.W., Prins, P.J.M., 2015. Improving executive functioning in
children with ADHD: training multiple executive functions within the context of a com-
puter game. A randomized double-blind placebo controlled trial. PLoS One 10, 1–30.
DuPaul, G.J., Power, T.J., Anastopoulos, A.D., Reid, R., 1998. ADHD Rating Scale-IV (for Chil-
dren and Adolescents) Checklists, Norms, and Clinical Interpretation. Guilford
Publications.
Engle, R.W., Tuholski, S.W., Laughlin, J.E., Conway, A.R.A., 1999. Working memory, short-
term memory, and general fluid intelligence: a latent-variable approach. J. Exp.
Psychol. Gen. 128, 309–331.
Halperin, J.M., Marks, D.J., Bedard, A.C.V., Chacko, A., Curchack, J.T., Yoon, C.A., 2013. Train-
ing executive, attention, and motor skills: a proof-of-concept study in preschool chil-
dren with ADHD. J. Atten. Disord. 17, 711–721.
Holmes, J., Gathercole, S.E., Place, M., Dunning, D.L., Hilton, K.A., Elliott, J.G., 2010. Working
memory deficits can be overcome: impacts of training and medication on working
memory in children with ADHD. Appl. Cogn. Psychol. 6, 827–836.
Holtmann, M., Sonuga-Barke, E., Cortese, S., Brandeis, D., 2014. Neurofeedback for ADHD:
a review of current evidence. Child Adolesc. Psychiatr. Clin. N. Am. 23, 789–806.
Jiang, H., Johnstone, S.J., 2015. A preliminary multiple case report of neurocognitive train-
ing for children with AD/HD in China. SAGE Open 1–13.
Johnstone, S.J., 2013. Computer gaming and ADHD: potential positive influences on be-
haviour. IEEE Technol. Soc. Mag. 32, 20–22.
Johnstone, S.J., Blackman, R.J., Bruggemann, J., 2012a. EEG from a single channel dry sen-
sor recording device. Clin. EEG Neurosci. 43, 112–120.
Johnstone, S.J., Roodenrys, S.J., Blackman, R., Johnston, E., Loveday, K., Mantz, S., Barratt,
M.F., 2012b. Neurocognitive training for children with and without AD/HD. ADHD At-
tention Deficit Hyperactivity Disorders. 4, pp. 11–23.
Johnstone, S.J., Roodenrys, S.J., Phillips, E., Watt, A.J., Mantz, S., 2010. A pilot study of com-
bined working memory and inhibition training for children with AD/HD. ADHD At-
tention Deficit Hyperactivity Disorders. 2, pp. 31–42.
Klingberg, T., Fernell, E., Olesen, P.J., Johnson, M., Gustafsson, P., Dahlström, K., Gillberg,
C.G., Forssberg, H., Westerberg, H., 2005. Computerized training of working memory
in children with ADHD - A randomized, controlled trial. J. Am. Acad. Child Adolesc.
Psychiatry 44, 177–186.
Kóbor, A., Takács, Á., Urbán, R., Csépe, V., 2012. The latent classes of subclinical ADHD
symptoms: convergences of multiple informant reports. Res. Dev. Disabil. 33,
1677–1689.
Lofthouse, N., Arnold, L.E., Hersch, S., Hurt, E., DeBeus, R., 2012. A review of neurofeedback
treatment for pediatric ADHD. J. Atten. Disord. 16, 351–372.
Micoulaud-Franchi, J., Geoffroy, P.A., Fond, G., Lopez, R., Bioulac, S., Philip, P., 2014. EEG
neurofeedback treatments in children with ADHD: an updated meta-analysis of ran-
domized controlled trials. Front. Hum. Neurosci. 8, 906.
Rapport, M.D., Orban, S.A., Kofler, M.J., Friedman, L.M., 2013. Do programs designed to
train working memory, other executive functions, and attention benefit children
with ADHD? A meta-analytic review of cognitive, academic, and behavioral out-
comes. Clin. Psychol. Rev. 33, 1237–1252.
44 S.J. Johnstone et al. / International Journal of Psychophysiology 116 (2017) 32–44
Re, A.M., Capodieci, A., Cornoldi, C., 2015. Effect of training focused on executive functions
(attention, inhibition, and working memory) in preschoolers exhibiting ADHD symp-
toms. Front. Psychol. 6, 1–9.
Robinson, K.E., Kaizar, E., Catroppa, C., Godfrey, C., Yeates, K.O., 2014. A systematic review
and meta-analysis of cognitive interventions for children with central nervous sys-
tem disorders and neurodevelopmental disorders. J. Pediatr. Psychol. 39, 846–865.
Rogers, J.M., Aminov, A., Wilson, P.H., Johnstone, S.J., 2016. Retest reliability of a single-
channel, wireless EEG device. Int. J. Psychophysiol. 106, 87–96.
Rutledge, K.J., van den Bos, K., McClure, S.M., Schweitzer, J.B., 2012. Training cognition in ADHD:
current findings, borrowed concepts, and future directions. Neurotherapeutics 9, 542–558.
Sergeant, J.A., 2000. The cognitive-energetic model: an empirical approach to attention-
deficit hyperactivity disorder. Neurosci. Biobehav. Rev. 24, 7–12.
Sergeant, J.A., 2005. Modeling attention-deficit/hyperactivity disorder: a critical appraisal
of the cognitive-energetic model. Biol. Psychiatry 57, 1248–1255.
Sonuga-Barke, E.J.S., 2011. Editorial: ADHD as a reinforcement disorder-moving from gen-
eral effects to identifying (six) specific models to test. J. Child Psychol. Psychiatry 52,
917–918.
Sonuga-Barke, E., Brandeis, D., Cortese, S., Daley, D., Ferrin, M., Holtmann, M., Stevenson, J.,
Danckaerts, M., van der Oord, S., Döpfner, M., Dittmann, R., Simonoff, E., Zuddas, A.,
Banaschewski, T., Buitelaar, J., Coghill, D., Hollis, C., Konofal, E., Lecendreux, M.,
Wong, I., Sergeant, J., Group, E.A.G, 2013. Nonpharmacological interventions for
ADHD: systematic review and meta-analyses of randomized controlled trials of die-
tary and psychological treatments. Am. J. Psychiatry 170, 275–289.
Sonuga-Barke, E., Brandeis, D., Holtmann, M., Cortese, S., 2014. Computer-based cognitive
training for ADHD: a review of current evidence. Child Adolesc. Psychiatr. Clin. N. Am.
23, 807–824.
Tabachnick, B.G., Fidell, L.S., 1996. Using Multivariate Statistics. third ed. HarperCollins,
New York.
Wechsler, D., 2007. Wechsler Individual Achievement Test Second Edition Australian
Standardised Edition Abbreviated Examiner's Manual. second ed. NCS Pearson Inc.,
Sydney.
Zhang, S., Faries, D.E., Vowles, M., Michelson, D., 2005. ADHD rating scale IV: psychometric
properties from a multinational study as a clinician-administered instrument. Int.
J. Methods Psychiatr. Res. 14, 186–201.