You are on page 1of 17

Child Neuropsychology, 15: 425–440, 2009

http://www.psypress.com/childneuropsych
ISSN: 0929-7049 print / 1744-4136 online
DOI: 10.1080/09297040802646983

ARE EXECUTIVE CONTROL FUNCTIONS RELATED


TO AUTISM SYMPTOMS IN HIGH-FUNCTIONING
CHILDREN?

Lauren Kenworthy,1 David O. Black,2 Bryan Harrison,1


Anne della Rosa,1 and Gregory L. Wallace2
1
Children’s Nat’l Medical Center, Rockville, Maryland, USA, 2National Institute of
Mental Health, Bethesda, Maryland, USA

Background: Linking autism symptoms to cognitive abilities can expand phenotypic


descriptions and facilitate investigations into the etiology and treatment of this multiplex
disorder. Executive dysfunction is one of several potential cognitive phenotypes in autism.
Method: Archival clinical data on 89 children diagnosed with Autism Spectrum Disorders
and administered a large neuropsychological battery were evaluated for relationships
between executive functioning and autism symptoms.
Results: Significant relationships between both laboratory tasks and behavior rating
scales of executive functions and autism symptoms were identified. Multiple regression
analyses revealed that measures of semantic fluency, divided auditory attention, and
behavioral regulation were significantly correlated with autism symptoms, even after
accounting for the variance from correlated “nuisance variables,” such as vocabulary
and age.
Conclusions: Executive dysfunction is related to all three clusters of behavioral symptoms
in Autism Spectrum Disorders.

Keywords: Autism; Executive control; Asperger Syndrome; Communication; Social;


Repetitive behavior; Fluency; Attention.

Executive dysfunction is a consistent, well-documented finding in school-age


children with Autism Spectrum Disorders (ASDs; see reviews: Hill, 2004; Pennington &
Ozonoff, 1996; Sergeant, Geurts, & Oosterlaan, 2002). Although it has not been demon-
strated as a causal factor in ASDs (Dawson, Meltzhoff, Osterling, & Rinaldi, 1998;
Dawson et al., 2002; Griffith, Pennington, Wehner, & Rogers, 1999; Liss et al., 2001;
Yerys, Hepburn, Pennington, & Rogers, 2007), it may be related to some of the behaviors
that define this complex, multi-determined group of disorders (Happé, Ronald, & Plomin,
2006). The present study investigates this possibility.
Developmental disorders, such as ASDs, attention deficit/hyperactivity disorder
(ADHD), and dyslexia, are behaviorally defined but commonly associated with brain

The authors thank Jennifer A. Silvers, Laura Case, and Jennifer L. Sokoloff for their work compiling
data, Alex Martin and Benjamin Yerys for helpful comments on the manuscript, and the many children with
ASD and their families who made this research possible.
Address correspondence to Dr. Lauren Kenworthy, Children’s Nat’l Medical Center, 14801 Physicians’
Lane, Rockville, MD 20850, USA. E-mail: lkenwort@cnmc.org

© 2009 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
426 L. KENWORTHY ET AL.

abnormalities (for review, see: Giedd, Shaw, Wallace, Gogtay, & Lenroot, 2006). Defini-
tion of specific neurodevelopmental disorders at the behavioral and biological levels has
been hampered by problems of variability, comorbidity, age-related differences, and
widely ranging severity. Frith (2001) and Pennington (2002) argue for the utility of
identifying specific cognitive deficits in developmental disorders that may arise from
brain abnormalities and drive behavioral symptoms. Exploring cognitive correlates of
behavioral symptoms in neurodevelopmental disorders offers several possible advantages.
Cognitive profiles may supplement behavioral phenotypes, or even represent intermediate
phenotypes, and provide important data in the search for genetic etiologies (Gottesman &
Gould, 2003). Cognitive phenotypes can also inform intervention by proposing possible
antecedents to behavior problems (Fisher & Happé, 2005; Riggs, Jahromi, Razza,
Dillworth-Bart, & Mueller, 2006).
Like other neurodevelopmental disorders, ASDs are defined by behaviors, such as
poor eye contact, limited imaginative play, repetitive movements, and restricted interests.
Multiple genes are implicated (Folstein & Rosen-Sheidley, 2001) in this highly heritable
disorder. Happé et al. (2006) use evidence from a population-based sample of twins to
argue that the behavioral signs of autism do not stem from a unitary deficit. Instead, the
triad of impairments (social, communication, and repetitive behaviors) in autism repre-
sents a dissociable set of primarily genetically driven factors. If so, the pursuit of specific
cognitive phenotypes that may relate to the discrete impairments in autism could increase
power to detect genetic effects (Payton, 2006). There are a number of candidates for cog-
nitive phenotypes in autism, including language (Kjelgaard & Tager-Flusberg, 2001),
theory of mind (Baron-Cohen, 1995), executive function (EF), and central coherence
(Happé & Frith, 2006). This paper investigates possible linkages between EF and autism
symptoms.
EF is a cognitive domain related to self-regulation and the orchestration of other
cognitive functions in order to achieve a goal. Findings of executive dysfunction are
robust in school-age children with ASDs (for reviews see Hill, 2004; Kenworthy, Yerys,
Anthony, & Wallace, 2008; Pennington & Ozonoff, 1996; Sergeant et al., 2002), who
show specific impairment with tasks that require set-shifting and other aspects of mental
flexibility (Gioia, Isquith, Kenworthy, & Barton, 2002; Hughes, Russell, & Robbins, 1994;
Rumsey & Hamburger, 1988). In addition, numerous studies have highlighted the inability
of children with ASD to plan and organize response patterns on tasks such as the Tower of
London and the Rey-Osterrieth Complex Figure (Hughes et al., 1994; Kenworthy et al.,
2005; Ozonoff, Pennington, & Rogers, 1991).
The extensive literature on EF and theory of mind (TOM), which can be considered
a proxy for social abilities, provides evidence for a link between social symptoms and EF
in ASD. For example, Pellicano (2007) found correlations between TOM and EF indepen-
dent of age and ability level in young children with ASD, supporting the identification of
EF as an important factor in the advancement of TOM. Joseph and Tager-Flusberg (2004)
demonstrated a similar relationship among school-age children with ASD. They report a
positive relationship between performance on a task requiring working memory and inhib-
itory control and TOM, which was independent of language and nonverbal ability. In an
intervention study, Fisher and Happé (2005) found that EF training contributed to
improvements in TOM and postulated an indirect “trickle-down” effect of EF training on
TOM performance. Taking a much more concrete approach to social abilities, we have
previously linked parent-reported EF deficits to weak adaptive social skills (Gilotty,
Kenworthy, Sirian, Black, & Wagner, 2002).
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 427

Other investigations have explored the link between EF and autism symptoms
directly. Several previous investigations have targeted the repetitive, inflexible behaviors/
interests component of the ASD symptom triad. Lopez, Lincoln, Ozonoff, and Lai (2005)
investigated the association between repetitive behaviors and multiple measures of EF in a
sample of adults with ASD. They documented a significant relationship between cognitive
flexibility and repetitive behaviors as measured by the Autism Diagnostic Interview (ADI;
Le Couteur et al., 1989) and the Autism Diagnostic Observation Schedule (ADOS; Lord,
Rutter, DiLavore, & Risi, 1999). South, Ozonoff, and McMahon (2007) also found an
association between cognitive flexibility and repetitive behaviors in a group of children
between 10 and 19 years of age.
Taking a more expansive view of possible EF-autism symptom correlates, Bishop
and Norbury (2005a, b) failed to find a relationship between inhibition and any of the
three components of the autism triad but did find a link between fluency performance and
communication symptoms in autism. Power to reject the null hypothesis may have been
limited in the inhibition study by small sample size (14 children with ASD). Joseph and
Tager-Flusberg (2004) investigated the relationship between ADOS symptom scores
and three aspects of EF: working memory, combined working memory and inhibition, and
tower task performance. Significant EF-autism symptom correlations were noted. The
relationship between social abilities and EF measures became nonsignificant when verbal
ability was partialled out of the correlation, however. Many children had language impair-
ments (mean naming standard score = 78), which, combined with the limited opportunity
to document repetitive behaviors during the ADOS, may have limited power to detect
relationships in this investigation. In summary, conflicting results presented in previous
investigations of the relationship between EF and autism symptoms may reflect limits in
sample size, and the scope of symptoms and EFs assessed, as well as the confounding
influence of language impairments.
The present study investigates the relationships between EF and autism symptoms
by using data from an archival clinical sample of children with ASD. We explore the rela-
tionship between a broad range of EF-related tasks tapping attention, fluency, flexibility,
inhibition, planning and working memory, and comprehensive measures of ASD symp-
toms in a relatively large sample of high-functioning children with ASD. As argued in Hill
and Bird (2006), use of a high-functioning sample reduces the chances of misattributing
general cognitive and language impairments to executive dysfunction. Because EF is a
broadly defined construct that may be unitary or a set of discrete, fractionable functions
(Burgess, Alderman, Evans, Emslie, & Wilson, 1998; Gioia, Isquith, Guy, & Kenworthy,
2000), this study samples performance on a wide range of purported tasks of EF, some
tapping multiple EF-related abilities (e.g., the Tower of London; Unterrainer & Owen,
2006) and others that tap more specific abilities, such as inhibition. Because many
standard laboratory measures of EF lack ecological validity, making EF hard to capture
outside of realistic, “true to life” settings (Bernstein & Waber, 1990; Hill & Bird, 2006), a
measure of everyday EF as reported by parents and teachers was also included. In order to
be inclusive regarding autism symptoms, scores from the ADI and ADOS were combined
to create composite measures of autism symptomatology based on both parent report and
clinician observation (Lopez et al., 2005). Although other investigators have used the ADI
or ADOS as individual measures of symptoms, we are persuaded by the arguments of
Lopez et al. and others that the ADI and ADOS measure different aspects of ASD
symptomatology and that combining them gives a more comprehensive and valid overall
measure of autism symptoms (see also Risi et al., 2006).
428 L. KENWORTHY ET AL.

Correlations and regression analyses were used to test the hypothesis that EF per-
formance explains a unique portion of variance in autism symptoms independent of ver-
bal and nonverbal abilities. Based on previous findings, we hypothesized that a
composite autism symptom measure would be significantly related to parent and teacher
reports of executive dysfunction in everyday settings. We also hypothesized that because
the tower task taps multiple EF domains and has been consistently impaired in ASD
groups in the past, it too would be related to autism symptoms. We further hypothesized
that category (semantic) fluency would be negatively correlated specifically with com-
munication symptoms based on earlier findings (Bishop and Norbury, 2005b) and a pre-
sumed relationship between verbal fluency and verbal communication in autism.
Although our measure of flexibility, the TEA-Ch Creature Counting test, has not to our
knowledge been previously explored in ASD, we hypothesized that it, like other previ-
ously investigated measures of flexibility, would be significantly related to repetitive
behavior/circumscribed interest symptoms. Regarding measures of sustained auditory
and divided auditory attention and autism symptoms, we had no formal hypothesis,
because this is a relatively understudied domain of EF in ASD. These tasks offer a pri-
mary benefit of increased ecological validity, and, based on anecdotal clinical experi-
ence, we considered it important to explore the relationship between social symptoms
and the ability to attend to multiple auditory inputs simultaneously.

METHOD
Participants
Participants included a clinically referred sample of 89 children (n = 74 [83%] male)
with an autism spectrum diagnosis (autism n = 34; Asperger Syndrome n = 32; pervasive
developmental disorder, not otherwise specified n = 23) evaluated by a multidisciplinary
team at the Center for Autism Spectrum Disorders (CASD) at Children’s National Medi-
cal Center in the Washington, D.C., metropolitan area. Each child received a comprehen-
sive multidisciplinary evaluation that included a detailed medical and developmental
history, an extensive neuropsychological battery, as well as administration of the Autism
Diagnostic Interview (ADI; Le Couteur et al., 1989) or the Autism Diagnostic Interview-
Revised (ADI-R; Lord, Rutter, & Le Couteur, 1994) and the Autism Diagnostic Observation
Schedule (ADOS; Lord et al., 1999) by a trained, research reliable clinician. All diagnoses
were based on expert clinical impression using Diagnostic and Statistical Manual of Mental
Disorders – Fourth Edition (DSM-IV; American Psychiatric Association, 1994) criteria and
results from the ADI/ADI-R, ADOS and developmental history. In addition to meeting
DSM diagnostic criteria by clinical impression, all participants also met criteria for an ASD
on the ADOS and/or ADI/ADI-R according to criteria established by the National Institutes
of Health Collaborative Programs for Excellence in Autism (CPEA; see Lainhart et al.,
2006). Because the ADI and ADOS do not have an algorithm for Asperger Syndrome,
Lainhart and colleagues developed criteria that include an individual in the broad autism
spectrum if they: meet the ADI cutoff for autism in the social domain and at least one
other domain or meet the ADOS cutoff for the combined social and communication score.
In order to include the full autism spectrum, these criteria are relatively inclusive and were
used in this study in addition to the criteria that all subjects meet DSM criteria for ASD in
the judgment of a team of clinicians experienced with the assessment and diagnosis of
individuals with ASD. All participants also had an IQ scaled score of 5 (i.e., standard
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 429

Table 1 Participant Characteristics.

N M SD Range

Chronological age 89 9.6 2.82 6–17


Hollingshead Socioeconomic status 69 19.1 8.11 11–44
Vocabularya 88 9.6 4.58 1–19
Block Designb 89 10.0 3.97 1–19
ADI Reciprocal Social 89 19.8 5.11 6–29
ADI Communication Verbal 89 15.8 4.13 3–23
ADI Repetitive Behavior 89 7.1 2.7 0–12
ADOS Communication 81 4.1 1.92 0–8
ADOS Social 81 8.1 3.22 2–14
ADOS Stereotyped 81 1.9 1.65 0–8
a
Based on Wechsler Vocabulary scores reported in scaled format (M = 10, SD = 3).
b
Based on Wechsler Block Design scores reported in scaled format (M=10, SD = 3).

score of 75) or higher on either the Wechsler Vocabulary or Block Design subtests. Demo-
graphic and sample characteristics are presented in Table 1.

Measures
Socioeconomic status. Socioeconomic status (SES) was based on the four-
factor Hollingshead Index of Social Position (Hollingshead, 1975), which utilizes marital
status, gender, education, and occupation to compute a household SES score. Lower SES
scores reflect greater SES status.

Diagnostic measures. The Autism Diagnostic Interview (ADI or ADI-R; Le


Couteur et al., 1989; Lord et al.1994) is a detailed parent or caregiver interview of
developmental history and autism symptoms. Scores are aggregated into symptom clusters
that correspond to DSM-IV criteria for a diagnosis of autism. The Autism Diagnostic
Observation Schedule (ADOS; Lord et al. 1999) is a structured play and conversational
interview that includes a series of social presses and other opportunities to elicit symptoms
of an ASD.
ADI and ADOS scores were aggregated into a composite score for each symptom
domain (communication, reciprocal social interaction, and restricted and repetitive behav-
iors). This was done by first converting domain raw scores of the ADI and ADOS into
standardized z-scores and then taking the mean of the ADI and ADOS z-scores to create
composite scores. This approach was taken to make the contribution of ADOS and ADI
scores comparable. Using raw scores in this calculation would have overrepresented the
ADI in the composite because ADI algorithm totals include more items, and thus more
possible points, than the ADOS algorithm totals. Unstandardized raw scores are presented
in Table 1 for the ADOS and ADI/ADI-R.

General cognitive ability. General cognitive ability was assessed using one of
three Wechsler Intelligence scales, the Wechsler Intelligence Scale for Children – 3rd
Edition (Wechsler, 1991), Wechsler Intelligence Scale for Children – 4th Edition
(Wechsler, 2003), or the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999).
In order to create comparable cross-test measures, verbal ability was estimated using the
430 L. KENWORTHY ET AL.

Vocabulary subtest and nonverbal ability was estimated using the Block Design subtest.
Vocabulary and Block Design performance is presented as norm-referenced scaled
scores (M = 10; SD = 3).

Executive function measures. All executive functioning measures are pre-


sented as age-corrected, norm-referenced standard scores calculated using the reference
tables provided for each instrument.

Behavior Rating Inventory of Executive Functions (BRIEF; Gioia et al., 2000). The
BRIEF is an informant report inventory of executive functioning in everyday situations. It
yields eight scales that are collapsed into two broad indices: the Behavioral Regulation
Index (BRI) and the Metacognition Index (MCI). BRI and MCI T-scores (M = 50; SD = 10)
from both parent and teacher report were standardized into z-scores and then aggregated to
create composite scores for the BRI and MCI. Higher scores indicate greater impairment;
T-scores ≥ 65 indicate clinically significant symptoms.

Test of Everyday Attention for Children (TEA-Ch; Manly, Robertson, Anderson, &
Nimmo-Smith, 1999). The TEA-Ch is a standardized assessment battery of various
aspects of attention and EFs in children including sustained attention, divided attention,
working memory/flexibility, and response inhibition. TEA-Ch subtests are reported as
scaled scores (M = 10; SD = 3). Four TEA-Ch subtests were used:

1. Score! is a measure of auditory sustained attention. It requires respondents to count the


number of beeps heard on a tape.
2. Score-DT assesses divided attention and working memory. It combines the sustained-
attention demand from Score! with an additional auditory component. Respondents
count the sounds while listening for an animal name spoken during a concurrent audio
news broadcast.
3. Creature Counting is a measure of flexibility (or switching) and working memory that
requires respondents to alternate between counting forward and counting backward
while following a path of targets.
4. Walk-Don’t Walk is a measure of sustained attention and prepotent response inhibition.
Respondents must attend to two auditory stimuli (i.e., beeps and beeps plus crashing
noise); the former prompts respondents to mark the corresponding tile of a path, while
the latter cues them to stop marking on the corresponding tile of the path. To receive
credit, the child must correctly mark the number of beeps heard. The temporal interval
between the auditory stimuli decreases as the task proceeds.

Tower of London-Drexel (TOL-DX) (Culbertson & Zillmer, 2000). TOL-DX measures


multiple EFs such as planning, inhibition, and working memory. It requires the subject to
work step-by-step to copy a pattern of beads on pegs using the least number of moves
possible. The total-moves score was analyzed as an omnibus measure of EF. Results
are reported as standard scores (M = 100; SD = 15).

Semantic fluency. Semantic Fluency taps language organization by asking the sub-
ject to orally name as many words as possible that belong to a certain category (e.g., foods
or animals). Two versions of this task were given: NEPSY®: A Developmental Neuropsy-
chological Evaluation (NEPSY; Korkman, Kirk, & Kemp, 1998) or the Delis-Kaplan
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 431

Executive Functioning System (D-KEFS; Delis, Kaplan, & Kramer, 2001), depending on
the age of the child. Results are reported as scaled scores (M = 10; SD = 3).

Data Analytic Plan


All data were first examined for outliers and to determine that statistical assump-
tions of parametric tests were not violated. Data were analyzed initially using bivariate
correlations between possible nuisance variables (age, Vocabulary, and Block Design
scores) and both EF measures and composite autism symptom domains. Statistically
related nuisance variables were included in subsequent analyses. Because this is a clinical
dataset and not all data points were available for all subjects, using statistical significance
levels to identify meaningfully related variables was deemed inappropriate. Instead, a
medium effect size or larger, corresponding to an r ≥ .24 by Cohen’s standards (Cohen,
1977), was used as entry criterion into subsequent analyses. Scatter plots between related
predictor and outcome variables were composed to examine the distribution of scores for
problems that might affect primary results, such as a bimodal distribution and/or outliers.
To understand the unique contribution of each EF measure, multiple regression
models were calculated with each autism symptom composite score as the outcome vari-
able and EF scores as predictors. Laboratory and informant report measures were analyzed
in separate regression models because of the qualitative differences in source and type of
data. Stepwise regressions, entering predictor variables (EF variables correlated at ≥ .24
with autism symptom scores) at step 1, and nuisance variables at step 2 were conducted.
This strategy was selected first to examine the relationship between EF and autism symp-
toms and then to examine how that relationship changed with the addition of covariates.
When there was just one predictor, both the covariates and the predictor were entered in
the same step since entering a single predictor in step 1 would be redundant with bivariate
correlations.

RESULTS
Preliminary Results and Descriptive Statistics
Inspection of predictor and outcome variables did not reveal any problems with
skewness, kurtosis, outliers, or the presence of a bimodal distribution. The frequency
distribution of scores revealed no floor or ceiling effects with one exception: 8 subjects,
or 20% of the sample, performed at the floor on the tower task. Inspection of scatter
plots of tower scores and autism symptom scores indicated that those subjects with low
scores had autism symptom scores that were evenly distributed above and below the
mean for this sample, however. In addition, t-tests comparing the 8 subjects at the floor
of tower measure to the remaining sample revealed no differences between those at the
floor and the rest of the sample on any of the three symptom domains. As shown in
Table 2, despite having scored in the average range on the Wechsler Vocabulary and
Block Design subtests, scores on laboratory measures of EF tended to be in the low
average range. The lowest scores were on measures of sustained attention (TEA-Ch
Score!), response inhibition (TEA-Ch Walk Don’t Walk), and when multiple EFs were
assessed simultaneously (TOL-DX). Similarly, mean scores on both the parent and
teacher report of problems in behavior regulation and metacognition were in the
clinically significant range (t ≥ 65).
432 L. KENWORTHY ET AL.

Table 2 Participant Executive Functioning Profile.

Executive Functioning Measure N M SD Range

TEA-Ch (scaled scores; M = 10; SD = 3)


Score! 60 6.6 3.43 1–15
Score DT 41 8.2 3.45 1–15
Creature Counting 53 8.9 3.76 3–14
Walk Don’t Walk 49 7.0 3.98 1–16
Category Fluency (scaled score) 32 9.5 3.89 1–17
Tower of London DX (std. score; M = 100; SD = 15) 41 83.4 20.07 59–122
BRIEF (T-scores; M = 50; SD = 10)
Parent Behavior Regulation 80 66.6 12.14 41–87
Parent Metacognition 76 69.0 10.39 41–89
Teacher Beh. Regulation 64 65.8 14.38 42–99
Teacher Metacognition 63 68.4 12.09 42–109

Table 3 Correlations between Covariates, Measures Of Executive Functioning and Autism Symptoms.

Communication Reciprocal Social Restricted &


N Symptoms Interaction Symptoms Repetitive Behaviors

Covariates
Age 89 .04 .25* −.24*
Vocabulary 88 −.33* −.19 −.01
Block Design 89 −.18 −.03 .05
Executive Functioning Measures
Score DT 41 −.43* −.48* −.10
Category Fluency 32 −.44* −.58* .06
Walk Don’t Walk 49 −.22 .03 −.34*
Score! 60 −.22 −.17 −.12
Creature Counting 33 −.12 −.23 −.13
Tower of London- DX 41 −.11 −.01 −.20
BRIEF Behavior Regulation Index 84 .32* .29* .30*
BRIEF Metacognition Index 82 .18 .24* .04

*r ≥ .24 and all statistically significant at p < .05.

As shown in Table 3, Vocabulary was significantly negatively related to autism commu-


nication symptoms and age was positively related to autism reciprocal social interaction symp-
toms and negatively related to restricted and repetitive behaviors. With regard to laboratory
measures of EF, Score DT and Category Fluency were negatively correlated with communica-
tion and reciprocal social interaction symptoms and Walk Don’t Walk was negatively corre-
lated with restricted and repetitive behaviors. The BRIEF BRI was related to all three autism
symptom domains and BRIEF MCI was related to reciprocal social interaction symptoms.

Primary Analyses
Communication symptoms. Two multiple regressions were conducted with
communication symptoms as the outcome variable, one with laboratory measures of EF and
one with informant report measures of EF. In the first regression, Score DT and Category Flu-
ency were entered in step 1 and Vocabulary as a covariate entered in step 2. Results shown in
Table 4 indicated that better performance on Category Fluency was associated with fewer
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 433

Table 4 Laboratory and Parent Report Measures of Executive Functioning Predict Communication Symptoms.

Variables in equation F/ΔFa R2/ΔR2b β t pc


Laboratory Measures
Step 1. Model Summary 6.57 .48 .01
Score DT −.18 −0.88 .39
Category Fluency -.63 -3.13 <.01
Step 2. Model Summary .81 .03 .39
Score DT −.11 −0.50 .63
Category Fluency -.56 -2.60 .02
Vocabulary −.21 −0.90 .39
Parent Report Measure
Model Summary 10.43 .21 <.01
BRIEF BRI .30 2.99 <.01
Vocabulary -.33 -3.26 <.01
a
F indicates F-value at step 1; ΔF indicates change in F from step 1 to step 2.
b
R indicates R2at step 1; ΔR2indicates change in R2 from step 1 to step 2.
c
Bold indicates p < .05.

autism communication symptoms after accounting for variance from Score DT and Vocabu-
lary. Score DT and Vocabulary did not contribute meaningfully to the model. The second
regression investigated the informant report measure. Both the BRIEF BRI and Vocabulary
were entered in the same step. Results indicated that informant report of problems with behav-
ior regulation was associated with more communication symptoms (Table 4). Higher Vocabu-
lary scores were associated with fewer communication symptoms in this model.

Reciprocal social interaction symptoms. Following the same model as


described above, two multiple regressions were conducted with social symptoms as the
outcome variable. In the first regression, Score DT and Category Fluency were entered in
step 1 and age at testing was entered in step 2. As shown in Table 5, significant results
indicated that better performance on both Score DT and Category Fluency was associated
with fewer social symptoms. Entering age in the model did not influence the results. In the
second regression both the BRIEF BRI and BRIEF MCI were included in the model and
age was entered at step 2. This model was also significant indicating that BRIEF MCI and
BRIEF BRI are related to autism social symptoms (Table 5). A review of the T-scores
related to each variable indicated that, after accounting for shared variance, neither was
significant although BRI approached significance. The ΔF score at step 2 was not signifi-
cant indicating that age at testing did not contribute to the model.

Restricted and repetitive behaviors. Table 6 displays the results of two multi-
ple regressions conducted with restricted and repetitive behaviors as the outcome variable. In
the first regression, both Walk Don’t Walk and age at testing were entered in the same step.
In contrast to the correlations described above, after accounting for variance explained by
age, Walk Don’t Walk no longer accounts for significant variance in restricted and repetitive
behaviors. In the second regression, both the BRIEF BRI and age at testing were entered in
the same step. Results indicated that both BRIEF BRI and age were significantly related to
restricted and repetitive behaviors such that children with greater behavior regulation prob-
lems exhibit more restricted and repetitive behaviors. In both models, results indicate older
children exhibit fewer restricted and repetitive behaviors.
434 L. KENWORTHY ET AL.

Table 5 Laboratory and Parent Report Measures of EF Predict Social Symptoms.

Variables in equation F/ΔFa Δ R2 b β t pc


Laboratory Measures
Step 1. Model Summary 17.06 .71 <.01
Score DT -.44 -2.96 .01
Category Fluency -.60 -4.02 <.01
Step 2. Model Summary .08 .01 .78
Score DT -.46 -2.77 .02
Category Fluency -.60 -3.82 <.01
Age at testing −.05 −0.29 .78
Parent Report Measures
Step 1. Model Summary 4.07 .09 .02
BRIEF MCI .12 .97 .34
BRIEF BRI .22 1.75 .08
Step 2. Model Summary 1.90 .02 .17
BRIEF MCI .07 .55 .58
BRIEF BRI .21 1.70 .09
Age at testing .16 1.38 .17
a
F indicates F-value at step 1; ΔF indicates change in F from step 1 to step 2.
b
R indicates R2 at step 1; ΔR2 indicates change in R2 from step 1 to step 2.
c
Bold indicates p < .05.

Table 6 Age and Parent Report Measures of EF Predict Restricted/Repetitive Symptoms.

Variables in equation F R2 β t pa
Laboratory Measures
Model Summary 6.17 .21 <.01
Walk Don’t Walk −.23 −1.70 .09
Age at testing -.33 -2.41 .02
Parent Report Measure
Model Summary 10.25 .20 <.01
BRIEF BRI .38 3.70 <.01
Age at testing -.34 -3.37 <.01
a
Bold indicates p < .05.

DISCUSSION
In this investigation we documented several associations between autism symptoms
and executive control functions. Multiple regression analyses revealed that measures of
semantic fluency, divided auditory attention, and behavioral regulation were significantly
correlated with autism symptoms, even after accounting for the variance from correlated
“nuisance variables,” such as vocabulary and age. Our first finding was that, consistent
with our prediction, better performance on semantic fluency was associated with fewer
autism communication symptoms, even after accounting for vocabulary, suggesting that,
in this sample, as reported previously, verbal fluency is related to the presentation of com-
munication symptoms.
Many previous investigators document verbal fluency deficits in individuals with
ASD and their relatives (Geurts, Verte, Oosterlaan, Roeyers, & Sergeant, 2004; Hughes,
Plumet, & Leboyer, 1999; Kleinhans, Akshoomoff, & Delis, 2005; Verte, Geurts, Roeyers,
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 435

Oosterlaan, & Sergeant, 2005). Bishop and Norbury (2005a) reported a linkage between
fluency and autism-related communication symptoms. They argued that pragmatic lan-
guage deficits in autism may reflect deficits in generativity as well as social difficulties.
Indeed, both on fluency tasks and in conversation, individuals with ASD struggle to gen-
erate words that are relevant to a context or topic. Such difficulty could be reflected in
ADI and ADOS items that assess an individual’s capacity to engage in reciprocal conversa-
tions, as well as a tendency to rely on scripted or stereotypic speech, whether it be used
appropriately, such as when using a rote script to ask for help, or inappropriately, such as
when reciting lines from a favorite book or movie out of context. An alternative, or compli-
mentary, hypothesis is that the structural language deficits associated with high-functioning
autism and, in some cases, pervasive developmental disorder, not otherwise specified
adversely affect verbal fluency performance.
We also found that the Behavior Regulation Index (BRI) of the Behavior Rating
Inventory of Executive Function (BRIEF) predicted ASD-restricted and repetitive behav-
ior symptoms. The BRI is comprised of three subdomains that tap behaviors related to
inhibition, flexibility, and emotional control. Thus, like South et al. (2007), Lopez et al.
(2005), and Turner (1997), we link flexibility to autism-related restricted and repetitive
behavior symptoms. Both our findings and those of Lopez et al. conflict with Turner’s
hypothesis (1997) that generativity also plays a role in restricted and repetitive symptoms
in autism, however, as we found no link between fluency and restricted and repetitive
behavior symptoms.
Our third, and perhaps most significant, finding associates both semantic fluency
and auditory divided attention abilities with ASD social symptoms, a linkage that has not
been previously described. Indeed, Ozonoff et al. (2004) as well as Joseph and Tager-
Flusberg (2004) failed to find an association between EF and ASD social symptoms. This
discrepancy may be explained by the different measures of EF employed in each of these
studies. Our data are consistent with these investigations in finding no significant associa-
tion of performance on a tower or inhibition task with social symptoms. It is also possible
that differences in age (Ozonoff’s sample included adults) and sample size (Joseph’s sample
was somewhat smaller than ours) contributed to discrepant findings in these studies. One other
study (Bishop & Norbury, 2005a) found a trend toward significant correlations between
social autism symptoms and ideational fluency in a mixed group that included a subset of
children with ASD.
To our knowledge, ours is the first investigation of the relationship between auditory
divided attention (as measured by the TEA-Ch Score! DT task) and autism symptoms.
This task requires a child to simultaneously listen to a news broadcast for an animal name
and to count beeps, thus requiring dual attention and working memory in order to hold the
animal name, number of beeps counted, and rules of the task in mind. It is not unreason-
able that this ability to divide attention and working memory between two stimuli has
implications for social reciprocity and social interaction generally. Interacting with
another person, particularly another child, requires adept “on-line” processing and integra-
tion of multiple stimuli, such as facial expression, body language, and intonation (Klin,
Jones, Schultz, & Volkmar, 2003).
It is noteworthy that the EF measures on which participants scored most poorly did
not show the strongest relationship to symptoms of autism. We found low EF scores on
measures of response inhibition, sustained attention, and omnibus EF, but measures of
divided attention and semantic fluency, both with mean scores in the average range, were
significantly associated with autism social and communication symptoms. In this sample,
436 L. KENWORTHY ET AL.

EF performance covaries with autism symptoms most on tasks that are intact for the group
as a whole, indicating the importance of considering the heterogeneity within ASD sam-
ples, as well as between ASD and other disorders. Mundy, Henderson, Inge, and Coman
(2007) argue for a modifier model in autism that addresses heterogeneity in ASD by con-
sidering variables that may not be specific to autism but can alter expression of the disor-
der in significant ways, such as increasing social impairment. Our data is more consistent
with identification of divided attention and fluency as possible modifier variables than as
intermediate phenotypes in autism.
It is also important to note that, as has been reported elsewhere, many of the EF
laboratory measures used in this investigation were not related to autism symptoms, even
when performance on the measure was impaired. For example, performance on the tower
was impaired in this study when compared to normative data (see Table 2), but not related
to autism symptoms. Poor performance on the tower in ASD may reflect factors that are
not related to symptom expression. On the other hand, failure to find relationships could
reflect limitations in standard laboratory EF tasks such as the tower. Tower tasks are
reported to have weak test-retest reliability (Bishop, Aamodt-Leeper, Creswell, McGurk,
& Skuse, 2001; Lemay, Bedard, Rouleau, & Tremblay, 2004); although Culbertson and
Zillmer (2000) report moderate to high test-retest reliability on the Tower of London-DX
move score, which was used in this investigation.
In addition, Hill and Bird (2006) criticize standard measures of EF that fail to cap-
ture deficits in high-functioning individuals with ASD for their lack of ecological validity.
The BRIEF and the Test of Everyday Attention for Children Score! DT subtest, both used
in this study, are designed for increased ecological validity. By virtue of their reliance on
everyday activities, however, they tap multiple abilities (e.g., language, processing/motor
output speed) and may fail to isolate discrete EFs. This confound may be unavoidable in
the EF domain. Bernstein and Waber (2007) argue that what we call EF relies not on func-
tional modules but on functional neural networks that develop in the context of experi-
ence. This observation appears particularly relevant to autism, which has defied modular
explanation at the genetic, neuroanatomical, neurofunctional, and behavioral levels and is
increasingly understood as a disorder of distributed networks in the brain (Muller, 2007).
At the same time, it is important to acknowledge that reification of “executive functions”
may be premature, or even misguided, as we attempt to capture an illusive construct that is
not easily parsed into discrete components.
Several limitations must be acknowledged in this study. Reliance on archival clin-
ical data prevented comprehensive assessment of all subjects using the same metrics,
thus reducing power to detect potentially meaningful relationships. Nor were the strict
controls provided in an experimental setting, such as random task order, imposed.
Because there is no control group, we cannot know whether these EF-autism-symptom
links are specific to ASD or predictive in a wider population. Indeed, future research
may address this question by attempting to link EF with subclinical traits associated
with autism in typical controls, and children with other developmental disorders, such as
ADHD and language impairment. In addition, the EF abilities identified here as predic-
tive of autism symptoms shared verbal mediation. Although we limited our sample to
high-functioning children with ASD and, when appropriate, covaried the influence of an
expressive vocabulary formulation task (a proxy for language ability), we may not have
controlled adequately for these shared verbal demands. Finally, the restriction of our
sample to high-functioning children limits the relevance of these findings to lower func-
tioning children with ASD.
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 437

This study contributes to the scientific literature by expanding the question of EF’s
role in autism beyond repetitive behaviors to social and communication symptoms.
Although EF was originally conceptualized to account for rigidity, perseveration, and other
nonsocial deficits in ASD, this study indicates that some aspects of executive dysfunction
may exacerbate symptoms in all three components of the triad of impairment. In the
search for intervention targets, ameliorating executive dysfunction may serve to improve
not only nonsocial deficits (as originally conceptualized) but also social and communica-
tion difficulties.

Original manuscript received February 8, 2008


Revised manuscript accepted November 20, 2008
First published online January 27, 2009

REFERENCES
Baron-Cohen, S. (1995). Mindblindness: An essay on autism and theory of mind. Cambridge, MA:
MIT Press.
Bernstein, J. H., & Waber, D. P. (1990). Developmental neuropsychological assessment: The
systemic approach. In A. A. Boulton, G. B. Baker, & M. Hiscock (Eds.), Neuropsychology:
Neuromethods (Vol. 17, pp. 311–371). Clifton, NJ: Humana.
Bernstein, J. H., & Waber, D. P. (2007). Executive capacities from a developmental perspective. In
L. Meltzer (Ed.), Executive function in education: From theory to practice (pp. 39–54).
New York: Guilford Press.
Bishop, D. V. M., Aamodt-Leeper, G., Creswell, C., McGurk, R., & Skuse, D. H. (2001). Individual
differences in cognitive planning on the Tower of Hanoi task: Neuropsychological maturity or
measurement error? Journal of Child Psychology and Psychiatry and Allied Disciplines, 42,
551–556.
Bishop, D. V. M., & Norbury, C. F. (2005a). Executive functions in children with communication
impairments, in relation to autistic symptomatology. 1: Generativity. Autism, 9, 7–27.
Bishop, D. V. M., & Norbury, C. F. (2005b). Executive functions in children with communication
impairments, in relation to autistic symptomatology. 2: Response inhibition. Autism, 9, 29–43.
Burgess, P. W., Alderman, N., Evans, J., Emslie, H., & Wilson, B. A. (1998). The ecological validity of
tests of executive function. Journal of the International Neuropsychological Society, 4, 547–558.
Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press.
Culbertson, W. C., & Zillmer, E. A. (2000). Tower of London—Drexel University. North
Tonawanda, NY: Multi-Health Systems, Inc.
Dawson, G., Meltzhoff, A. N., Osterling, J., & Rinaldi, J. (1998). Neuropsychological correlates of
early symptoms of autism. Child Development, 69, 1276–1285.
Dawson, G., Munson, J., Estes, A., Osterling, J., McPartland, J., Toth, K., et al. (2002). Neurocogni-
tive function and joint attention in young children with autism spectrum disorder versus
developmental delay. Child Development, 73, 345–358.
Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis–Kaplan executive function system. San
Antonio: Psychological Corp.
Fisher, N., & Happé, F. (2005). A training study of theory of mind and executive function in children with
autistic spectrum disorders. Journal of Autism and Developmental Disorders, 35, 757–771.
Folstein, S. E., & Rosen-Sheidley, B. (2001). Genetics of autism: Complex aetiology for a heteroge-
neous disorder. Nature Reviews Genetics, 2, 943–955.
Frith, U. (2001). Mind blindness and the brain in autism. Neuron, 32, 969–979.
Geurts, H. M., Verte, S., Oosterlaan, J., Roeyers, H., & Sergeant, J. A. (2004). How specific are
executive functioning deficits in attention deficit hyperactivity disorder and autism? Journal of
Child Psychology and Psychiatry, 45, 836–854.
438 L. KENWORTHY ET AL.

Giedd, J., Shaw, P., Wallace, G., Gogtay, N., & Lenroot, R. K. (2006). Anatomic brain imaging
studies of normal and abnormal brain development in children and adolescents. In D. Cicchetti
& D. J. Cohen (Eds.), Developmental psychopathology: Vol. 2. Developmental neuroscience
(2nd ed., pp. 127–196). New York: John Wiley & Sons.
Gilotty, L., Kenworthy, L., Sirian, L., Black, D. O., & Wagner, A. E. (2002). Adaptive skills and
executive function in autism spectrum disorders. Child Neuropsychology, 8, 241–248.
Gioia, G. A., Isquith, P. K., Guy, S., & Kenworthy, L. (2000). BRIEF: Behavior Rating Inventory of
Executive Function. Odessa, FL: Psychological Assessment Resources.
Gioia, G. A., Isquith, P. K., Kenworthy, L., & Barton, R. M. (2002). Profiles of everyday executive
function in acquired and developmental disorders. Child Neuropsychology, 8, 121–137.
Gottesman, I. I., & Gould, T. D. (2003). The endophenotype concept in psychiatry: Etymology and
strategic intentions. American Journal of Psychiatry, 160, 636–645.
Griffith, E. M., Pennington, B. F., Wehner, E. A., & Rogers, S. J. (1999). Executive functions in
young children with autism. Child Development, 70, 817–832.
Happé, F., & Frith, U. (2006). The weak coherence account: Detailed-focused cognitive style in
autism spectrum disorders. Journal of Autism and Developmental Disorders, 36, 5–25.
Happé, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism.
Nature Neuroscience, 9, 1218–1220.
Hill, E. L. (2004). Evaluating the theory of executive dysfunction in autism. Developmental Review,
24, 189–233.
Hill, E. L., & Bird, C. M. (2006). Executive processes in Asperger syndrome: Patterns of perfor-
mance in a multiple case series. Neuropsychologia, 44, 2822–2835.
Hollingshead, A. B. (1975). Four factor index of social status. New Haven: Yale University.
Hughes, C., Plumet, M. H., & Leboyer, M. (1999). Towards a cogntive phenotype for autism:
Increased prevalence of executive dysfunction and superior span amongst siblings of children
with autism. Journal of Child Psychology and Psychiatry and Allied Disciplines, 40, 705–718.
Hughes, C., Russell, J., & Robbins, T. W. (1994). Evidence for executive dysfunction in autism.
Neuropsychologia, 32, 477–492.
Joseph, R. M., & Tager-Flusberg, H. (2004). The relationship of theory of mind and executive functions
to symptom type and severity with autism.Developmental and Psychopathology, 16, 137–155.
Kenworthy, L. E., Black, D. O., Wallace, G. L., Ahulvalia, T., Wagner, A. E., & Sirian, L. M.
(2005). Disorganization: The forgotten executive dysfunction in high-functioning autism
(HFA) spectrum disorders. Developmental Neuropsychology, 28, 809–827.
Kenworthy, L., Yerys, B. E., Anthony, L. G., & Wallace, G. L. (2008). Understanding executive
control in autism spectrum disorders in the lab and in the real world. Neuropsychology Review,
18, 320–338.
Kjelgaard, M. M., & Tager-Flusberg, H. (2001). An investigation of language impairment in autism:
Implications for genetic subgroups. Language and Cognitive Processes, 16, 287–308.
Kleinhans, N., Akshoomoff, N., & Delis, D. C. (2005). Executive functions in autism and
Asperger’s disorder: Flexibility, fluency, and inhibition. Developmental Neuropsychology, 27,
379–401.
Klin, A., Jones, W., Schultz, R. T., & Volkmar, F. R. (2003). The enactive mind – From actions to
cognition: Lessons from autism. Philosophical Transactions of the Royal Society, Biological
Sciences, 358, 345–360.
Korkman, S. L., Kirk, U., & Kemp, M. (1998). NEPSY: A developmental neuropsychological
assessment. San Antonio, TX: Harcourt Brace.
Lainhart J. E., Bigler E. D., Bocian, M., Coon, H., Dinh, E., Dawson, G., et al. (2006). Head circum-
ference and height in autism: A study by the Collaborative Program of Excellence in Autism.
American Journal of Medical Genetics, 140, 2257–2274.
Le Couteur, A., Rutter, M., Lord, C., Rios, P., Robertson, S., Holdgrafer, M., et al. (1989). Autism
diagnostic interview: A standardized investigator-based instrument. Journal of Autism and
Developmental Disorders, 19, 363–387.
EXECUTIVE CONTROL FUNCTIONS AND AUTISM SYMPTOMS 439

Lemay, S., Bedard, M. A., Rouleau, I., & Tremblay, P. L. G. (2004). Practice effect and test-retest
reliability of attentional and executive tests in middle-aged to elderly subjects. Clinical Neu-
ropsychologist, 18, 284–302.
Liss, M., Fein, D., Allen, D., Dunn, M., Feinstein, C., Morris, R., et al. (2001). Executive function-
ing in high-functioning children with autism. Journal of Child Psychology and Psychiatry and
Allied Disciplines, 42, 261–270.
Lord, C., Rutter, M., DiLavore, P., & Risi, S. (1999). Autism diagnostic observation schedule-WPS
edition. Los Angles, CA: Western Psychological Services.
Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised
version of a diagnostic interview for caregivers of individuals with possible pervasive develop-
mental disorders. Journal of Autism and Developmental Disorders, 24, 659–685.
Lopez, B. R., Lincoln, A. J., Ozonoff, S., & Lai, Z. (2005). Examining the relationship between
executive functions and restricted, repetitive symptoms of autistic disorder. Journal of Autism
and Developmental Disorders, 35, 445–460.
Manly, T., Robertson, I., Anderson, V., & Nimmo-Smith, I. (1999). The test of everyday attention
for children. London: Thames Valley Test Company.
Muller, R. A. (2007). The study of autism as a distributed disorder. Mental Retardation and
Developmental Disabilities Research Reviews, 13, 85–95.
Mundy, P. C., Henderson, H. A., Inge, A. P., & Coman, D. C. (2007). The modifier model of autism
and social development in higher functioning children. Research and Practice for Persons with
Severe Disabilities, 32, 124–139.
Ozonoff, S., Coon, H., Dawson, G., Joseph, R. M., Klin, A., McMahon, W. M., et al. (2004).
Performance on Cambridge Neuropsychological Test Automated Battery subtests sensitive
to frontal lobe function in people with autistic disorder: Evidence from the collaborative Pro-
grams of Excellence in Autism Network. Journal of Autism and Developmental Disorders,
34, 139–150.
Ozonoff, S., Pennington, B. F., & Rogers, S. J. (1991). Executive function deficits in high-functioning
autistic individuals: Relationship to theory of mind. Journal of Child Psychology and Psychiatry
and Allied Disciplines, 32, 1081–1105.
Payton, A. (2006). Investigating cognitive genetics and its implications for treatment of cognitive
deficit. Genes, Brain & Behavior, 5, 44–53.
Pellicano, E. (2007). Links between theory of mind and executive function in young children with
autism: Clues to developmental primacy. Developmental Psychology, 43, 974–990.
Pennington, B. F. (2002). The development of psychopathology: Nature and nurture. New York:
Guilford Press.
Pennington, B. F., & Ozonoff, S. (1996). Executive functions and developmental psychopathology.
Journal of Child Psychology and Psychiatry and Allied Disciplines, 37, 51–87.
Riggs, N. R., Jahromi, L. B., Razza, R. P., Dillworth-Bart, J. E., & Mueller, U. (2006). Executive
function and the promotion of social-emotional competence. Journal of Applied Developmental
Psychology, 27, 300–309.
Risi, S., Lord, C., Gotham, K., Corsello, C., Chrysler, C., Szatmari, P., et al. (2006). Combining
information from multiple sources in the diagnosis of autism spectrum disorders. Journal of the
American Academy of Child and Adolescent Psychiatry, 45(9), 1094–1103.
Rumsey, J. M., & Hamburger, S. D. (1988). Neuropsychological findings in high-functioning men with
infantile autism, residual state. Journal of Clinical & Experimental Neuropsychology, 10, 201–221.
Sergeant, J. A., Geurts, H., & Oosterlaan, J. (2002). How specific is a deficit of executive function-
ing for attention-deficit/hyperactivity disorder? Behavioural Brain Research, 130, 3–28.
South, M., Ozonoff, S., & McMahon, W. M. (2007). The relationship between executive function-
ing, central coherence, and repetitive behaviors in the high-functioning autism spectrum.
Autism, 11, 437–451.
Turner, M. (1997). Towards an executive dysfunction account of repetitive behavior in autism. In
J. Russell (Ed.), Autism as an executive disorder. Oxford, UK: Oxford University Press.
440 L. KENWORTHY ET AL.

Unterrainer, J. M., & Owen A. M. (2006). Planning and problem solving: From neuropsychology to
functional neuroimaging. Journal of Physiology, 99, 308–317.
Verte, S., Geurts, H. M., Roeyers, H., Oosterlaan, J., & Sergeant, J. A. (2005). Executive functioning in
children with autism and Tourette syndrome. Development and Psychopathology, 17, 415–445.
Wechsler, D. (1991). Wechsler Intelligence Scale for Children-Third Edition (WISC-III). San Antonio,
TX: The Psychological Corporation.
Wechsler, D. (1999). Wechsler Abbreviated Intelligence Scale (WASI). San Antonio, TX: Psychological
Corporation.
Wechsler, D. (2003). Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV). San Antonio,
TX: Psychological Corporation.
Yerys, B. E., Hepburn, S. L., Pennington, B. F., & Rogers, S. J. (2007). Executive function in
preschoolers with autism: Evidence consistent with secondary deficit. Journal of Autism and
Developmental Disorders, 37, 1068–1079.

You might also like