Brain Injury

ISSN: 0269-9052 (Print) 1362-301X (Online) Journal homepage: https://www.tandfonline.com/loi/ibij20

Psychometric properties of the awareness

questionnaire, patient competency rating scale
and Dysexecutive Questionnaire in patients with
acquired brain injury

Danique Hellebrekers, Ieke Winkens, Suzanne Kruiper & Caroline Van

Heugten

To cite this article: Danique Hellebrekers, Ieke Winkens, Suzanne Kruiper & Caroline Van
Heugten (2017) Psychometric properties of the awareness questionnaire, patient competency
rating scale and Dysexecutive Questionnaire in patients with acquired brain injury, Brain Injury,
31:11, 1469-1478, DOI: 10.1080/02699052.2017.1377350

To link to this article: https://doi.org/10.1080/02699052.2017.1377350

© 2017 Taylor & Francis Group, LLC Published online: 05 Oct 2017.

Submit your article to this journal Article views: 1220

View related articles View Crossmark data

Citing articles: 2 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ibij20
BRAIN INJURY
2017, VOL. 31, NO. 11, 1469–1478
https://doi.org/10.1080/02699052.2017.1377350

Psychometric properties of the awareness questionnaire, patient competency rating

scale and Dysexecutive Questionnaire in patients with acquired brain injury
Danique Hellebrekersa, Ieke Winkensd, Suzanne Kruiperc, and Caroline Van Heugtena,b,c
a
Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University Medical Center, Maastricht, the
Netherlands; bDepartment of Neuropsychology and Psychopharmacology, Faculty of Psychology and Neuroscience, Maastricht University,
Maastricht, the Netherlands; cRehabilitation Research Centre, Reade, Amsterdam, the Netherlands; dLimburg Brain Injury Center, Maastricht
University, Maastricht, the Netherlands

ABSTRACT KEYWORDS
Objective: Lack of self-awareness of impairments is common after acquired brain injury (ABI). We Acquired brain injury; self-
evaluated the psychometric properties of three commonly used instruments for measuring self-aware- awareness; psychometric
ness: Awareness Questionnaire (AQ), Patient Competency Rating Scale (PCRS) and Dysexecutive ques- properties; awareness
questionnaire; patient
tionnaire (DEX).
competency rating scale;
Method: We recruited 105 patients with ABI and their relatives. We determined feasibility, responsive-
Dysexecutive questionnaire
ness, test- retest reliability, internal consistency, and construct validity of AQ, PCRS and DEX.
Results: No floor or ceiling effects were present. Total scale scores showed sufficient responsiveness:
effect sizes were moderate to large (.57–.85); test-retest reliability was sufficient for patient forms (.75-
.88) but mixed for relative forms (.60–.66). Internal consistency was good (.80-.89). Construct validity
results confirmed a three-factor structured AQ and a four-factor structured PCRS. A two-facture structure
was found for DEX patient forms; a three-factor structure was found for relative forms.
Conclusion: Overall, the total scale scores of patient forms of all questionnaires demonstrated sufficient
psychometric properties. Psychometric properties of subscales are questionable. We could not replicate
the factor structures of AQ, PCRS and DEX: the items within subscales differed with previous findings.
Additional research into the test-retest, inter-rater reliability and responsiveness of relative and clinician
forms is required.

Introduction and usefulness in clinical practice: the Self-Awareness of

Deficits Interview (SADI), Patient Competency Rating scale
Lack of self-awareness of cognitive, behavioural and emo-
(PCRS) and the Awareness Questionnaire (AQ).
tional impairments has been reported as one of the most
The PCRS and AQ are both used to measure the degree
disabling consequences of acquired brain injury (ABI) (1).
of self-awareness in several domains. The PCRS focuses on
The pathogenesis of this lack of awareness is unclear (2).
patients’ competencies in the present time. Factor analyses
However, it is often, associated with dysfunctioning of the
revealed four sub-scales: activities of daily living (8 items),
frontal-executive system. After brain injury, patients often
interpersonal (7 items), emotional (7 items) and cognitive
have to deal with a lack of knowledge about their functioning,
functioning (8 items) (11). Questions about these compe-
and an inability to anticipate difficulties and recognize errors
tencies are given to patients and to relatives or clinicians
in their performance in activities of daily living (3,4). The
who know the patient well (12). Discrepancies between the
incidence of this lack of awareness after ABI ranges from 30 to
patient ratings and relative or clinician ratings are indica-
97% and depends on several factors such as type of assessment
tive of impairments in the patient’s self-awareness. Various
instrument, severity of injury and time since injury (5–7).
studies showed that the PCRS is sensitive to differences in
These brain injury related awareness deficits negatively influ-
patient, family and clinician perspectives of a patient’s
ence employment outcomes and rehabilitation processes (8,9).
functioning (13). Test-retest reliability coefficients are high
Several theoretical models (e.g. hierarchical, descriptive,
and range between 0.92–0.97 (5,14). Internal consistency
executive, frontal and psychological) described impairments
coefficients (Cronbach’s alpha) are good (>.90) (15).
in self-awareness in brain injury patients, but there is no
Overall, the PCRS has good reliability, validity and respon-
consensus on a model (10).
siveness, but reliability information of the different PCRS
Many methods and instruments are used to assess aware-
forms (e.g. relative and clinician) is not available (2). The
ness of deficits; there is no gold standard. In their systematic
AQ focusses on differences between patients’ current and
review, Smeets et al.(2). Identified 39 existing assessment
pre-injury functioning. Factor analyses revealed three sub-
instruments. Three were recommended for use in research
scales: motor/sensory (four items), cognitive (seven items)
and clinical settings based on their psychometric properties

CONTACT Caroline van Heugten caroline.vanheugten@maastrichtuniversity.nl Maastricht University, Department of Neuropsychology and
Psychopharmacology, Faculty of Psychology and Neuroscience, P.O. Box 616, 6200 MD Maastricht, the Netherlands.
© 2017 Taylor & Francis Group, LLC
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/),
which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
1470 D. HELLEBREKERS ET AL.
and behavioural/affective (six items) (13,16). Information
about the patient’s functioning is given by patients and
relatives or clinicians. Again, discrepancies between patient
ratings and relative or clinician ratings are indicative of
impairments in the patient’s self-awareness. Reliability ana-
lyses showed high internal consistency coefficients
(Cronbach’s alpha; 0.88) for both patient and relative rat-
ings (13). However, information about test-retest reliability
is still lacking (2). In summary, both the AQ and PCRS are
studied extensively, but some information on psychometric
properties is lacking (2).
We also included that the Dysexecutive syndrome question-
naire (DEX) in our study. The DEX is a discrepancy measure
too. Although it was not originally developed to measure
awareness, it is often used for this purpose. In clinical practice,
it is used to assess behaviour that is mediated by executive
functioning e.g. decision-making, impulsivity, social appropri-
ateness, and planning for the future (17). Factor analyses
revealed four subscales: initiating and sustaining actions (8
items), impulse control and sequencing of heard information
(5 items), psychophysical and mental excitability (4 items) and
social conventions (4 items) (18,19). Bodenburg and Dopslaff
found high internal consistency coefficients (Cronbach’s alpha;
0.85) (17). Besides these studies, little information on the psy-
chometric properties of the DEX is available.
To measure deficits in self-awareness after ABI, a gold
standard is needed. The present study further assessed and
replicated the psychometric properties of the AQ, PCRS and
DEX. For each questionnaire feasibility, responsiveness, test-
retest reliability, internal consistency, and construct validity
(factor analyses) was measured.
Methods
Participants
Participants were patients diagnosed with ABI who were
enrolled in the Intensive NeuroRehabilitation (INR) programme
of the rehabilitation centre Reade (Amsterdam, the Netherlands)
between August, 2008 and December, 2013. The INR pro-
gramme is an outpatient group intervention, based on a holistic,
therapeutic milieu approach described by Ben-Yishay (20).
Patients are assigned to the INR programme by rehabilitation
professionals, outpatient support services, and physicians.
Further details of the intervention are explained elsewhere
(21). For each patient, a relative or friend who could give reliable
information about the patient’s (current) functioning were also
recruited for the study. Patients were eligible for participation in
the study if they were (1) between 18 and 65 years old, (2)
diagnosed with non-progressive acquired brain injury (e.g.
stroke or traumatic brain injury) documented by medical/surgi-
cal records and/or brain imaging, (3) at least 12 months post-
injury, and (4) had some demonstrable knowledge about the
consequences of their injury on the basis of clinical judgment.
Participants were excluded in case of: (1) language deficits that
interfered with communication; (2) behavioural or psychiatric
disorders that interfered with group functioning (e.g., aggression
or extreme disinhibition); (3) inadequate proficiency in Dutch;
or (4) substance abuse.
Study design and procedure
The current study is part of an on-going longitudinal cohort
study on the outcomes of the INR intervention programme
for patients with ABI (21). Current study had a retrospective
design, whilst previously collected data of this longitudinal
study were used. For the current study the assessment of
participants at four time points were used: (T0) first baseline
measurement, (T1) second baseline measurement, approxi-
mately 5 weeks after T0 and 4 weeks before the start of the
INR programme, (T2) post-treatment measurement was about
13 weeks after T1 and no more than 4 weeks after the end of
the programme, and (T3) follow-up measurement, between 48
and 59 weeks after T2. This last measurement was approxi-
mately 1 year after the end of the programme. The first base-
line T0 was added only temporarily as part of a study into the
effectiveness of INR (21). That is why we used T1 as baseline
measurement. The collected data consisted of neuropsycholo-
gical assessments and questionnaires, which are standard clin-
ical procedures in the INR programme. Participants
completed the questionnaires at the rehabilitation centre (T0,
T1, T2) or at home (T3). Questionnaires that were completed
at home (T3) were returned by mail. Before the INR interven-
tion programme started, participants received information
about the programme and the use of their clinical data for
scientific purposes. All participants provided written informed
consent for the anonymous use of their data. A review pro-
cedure by a medical ethics committee was not required for
this type of data collection.
Measurements
Demographic and brain injury-related information
At baseline, patient characteristics (age, gender and educational
level) and brain injury-related information (time since injury,
cause of injury and initial severity of injury) were extracted from
the patients’ files. Level of education was classified according to a
standardized Dutch system (22) on a eight-point scale. Level of
education was dichotomized into two categories: low education
(classifications 1–4) and high education (classifications 5–8).
Injury severity was calculated with the Glasgow Coma Scale
(GCS 13–15 = mild, GCS 9–12 = moderate, GCS 3–8 = severe)
(23), the Post Traumatic Amnesia scale (duration of
PTA < 1 hour = mild, 1 to 24 hours = moderate,
>24 hours = severe) (23) and the Loss Of Consciousness scale
(duration of LOC<30 minutes = mild, 30 minutes to
24 hours = moderate; >24 hours = severe) (23). Characteristics
of relatives (gender and relationship with the patient) were
determined by questionnaires at time point T1.
Measurement of self-awareness
Awareness questionnaire (AQ)
The AQ (16) is a 17-item self-report questionnaire used to
assess impaired self-awareness after brain injury. Patients rate
their current performance relative to their performance before
brain injury on a variety of everyday motor/sensory, cognitive
and behavioural/affective activities. A 5-point Likert scale
from 1 (much worse) to 5 (much better) is used. The self-

ratings of patients are compared to the ratings of their rela-
tives or clinicians. The discrepancy score between these rat-
ings is a measure of self-awareness. Positive discrepancies
indicate over-estimation, negative discrepancies indicate
underestimation of functioning.
Patient competency rating scale (PCRS)
The PCRS (24) is a 30-item questionnaire. The PCRS evalu-
ates performance in activities of daily living, cognitive func-
tioning, interpersonal functioning and emotional regulation.
All items are rated on a 5-point Likert scale from 1 (cannot
do) to 5 (can do with ease). The PCRS has three forms; one is
completed by the patient, one by a relative and one by a
clinician. A discrepancy score is calculated between the
patient’s ratings and the relatives’ or clinicians’ ratings.
Positive discrepancies indicate over-estimation, and negative
discrepancies indicate underestimation of functioning.
The Dysexecutive Questionnaire (DEX) (25) is a 20-item
self-report questionnaire designed to measure everyday signs
of behavioural symptoms resulting from a dysexecutive syn-
drome (26). Items are rated on a 5-point Likert scale from 0
(never) to 4 (very often). Ratings consist of self-ratings and
informant ratings. A discrepancy score between the patient’s
rating and the informant’s rating is calculated. Positive dis-
crepancies indicate over-estimation of executive functioning,
negative discrepancies indicate underestimation of executive
functioning.
Statistical analyses
To analyse the psychometric properties of the AQ, PCRS and
DEX, the outcomes of patients’ and relatives’ forms were used.
We used the scores of patients and relatives separately instead
of using the discrepancy scores. Since, the discrepancy scores
are difference scores, they are difficult to interpret and do not
reflect the actual properties of the instrument itself.
Descriptive statistics were used to display frequency and
means of demographic and injury-related parameters.
Feasibility
We used descriptive statistics to indicate core distributions
(mean, median, standard deviation, range, floor and ceiling
effects, percentage of missing items, skewness) of patients’ and
relatives’ scores on the three questionnaires at time point T1.
Floor and ceiling effects were considered present if more than
15% of the patients had the highest or the lowest possible
score (27). Data were discarded if more than 25% items were
missing (28). If the number of missing items for any scale was
within the allowed range, the total scale score was imputed by
extrapolating the total score of the items available ((total
score/#completed items)*total#items on scale).
Responsiveness
Repeated measure analyses with Bonferroni correction was
applied to evaluate responsiveness by comparing the mean
scores of the AQ, PCRS and DEX over time. Effect sizes
(Cohen’s d) were calculated to compare the strength of the
effects observed (29). Effect sizes were expected to be high
between pre-treatment (T1) and post-treatment (T2)
BRAIN INJURY 1471
measurement. No differences were expected between baseline
measurements (T0–T1) or between post-treatment and follow-
up measurement (T2–T3). We defined the effect sizes as fol-
lows: 0.2–0.5 = small, 0.5–0.8 = medium and 0. 8 and
above = large (30). Before carrying out actual analyses, under-
lying assumptions were checked. Normality distributions of
the residuals were examined by visual inspection of histo-
grams and normal probability plots.
Test-retest reliability
To determine test-retest reliability, total scale scores and sub-
scale scores on time points T0 and T1 were used. No change
was expected between the baselines (T0 and T1). Test-retest
reliability was considered sufficient if the correlation coeffi-
cient (Spearman) was >0.8, moderate if between 0.7 and 0.8,
and insufficient if <0.7 (31).
Internal consistency
For all questionnaires, Cronbach’s alphas were calculated for
patient and relative responses on time point T1. Cronbach’s
alpha coefficient was considered as good if alpha was  :80,
moderate if between .70–.80 moderate, acceptable if between
.60 and .70, and poor if <. 60 (31).
Construct validity
The AQ, PCRS and DEX questionnaires of both patients and
relatives were evaluated by means of factor analyses. All items
were submitted to principal component analyses with varimax
rotation. To modify a viable model, factor loadings needed to
be higher than >0.4. To confirm the factors, maximum like-
lihood tests were used.
All statistical analyses were carried out using SPSS version
22.0 for MAC OS X. Results were considered significant
if p < .05.
Results
Participants
Between August, 2008 and December, 2013, 105 patients and
105 relatives participated in the INR intervention programme
and were included in the present study. Information from the
three questionnaires was available only for a smaller group of
84–95 patients and 80–94 relatives. Demographic and brain-
injury related characteristics of the 105 patients are presented
in Table 1. Initial severity determined by GCS was available
for 19 patients. Results showed a mean GCS score of 8
(SD = 4.9). The majority of these patients (10%) had severe
injury, 5% moderate injury and 4% mild injury. Initial severity
determined by PTA was available for 18 patients. A mean
PTA score of 10.2 (SD = 18.5) was found in this sample. A
mean LOC score of 16.2 (SD = 21.5) was calculated based on
the individual scores of 38 patients. Of the relatives, 29.2%
were male. The majority of relatives were the patients’ part-
ners (46.7%), 11.4% were brothers/sisters, 24.8% were one of
the parents, 1.9% were children, 3.8% were friends, and 1.9%
were ex-partners.
1472 D. HELLEBREKERS ET AL.

Table 1. Demographic characteristics of patients at T1. score was seen between the pre- and post-treatment measure-
T1 ment (T1– T2) on the AQ (p = .053) and PCRS (p = .054). The
Variables (n = 105) pre-and post-treatment measurement of the DEX was not sig-
Demographic characteristics nificant (p = .076). However, effect sizes for changes in self-
Male (%)
Patients 61 (58.1) awareness and awareness of dysexecutive complaints ranged
Mean age in years (SD) 41.7 (12.1) from moderate to large between pre-and post-treatment (T1–
Educational level at least high school (%) 47 (44.8)
Years post injury 7.5 (8.5) T2; See Table 3). Relatives’ scores were not significantly different
Type of brain injury (%) between baselines (T0–T1), pre- and post-treatment (T1–T2) or
TBI 45 (42.9)
Vascular 24 (22.9)
between post-treatment and follow up measurement
Tumour 13 (12.4) (T2–T3). However, moderate to large effect sizes were seen
Anoxia 6 (5.7) between pre- and post-treatment measurement (T1–T2).
Other* 7 (6.7)
Note: * = Multiple types (n = 4), inflammation (n = 2), intoxication (n = 1),
colloid cyst (n = 1), damage caused by radiation for non-Hodgkin lymphoma Test-retest reliability
(n = 1), epilepsy- status epilepticus (n = 1) and surgery (n = 1). Test-retest reliability analyses of patient forms found suffi-
TBI = traumatic brain injury.
cient and moderate correlation coefficients for AQ, PCRS and
DEX total scales (.80; .75; .88 respectively; Table 4a).
Psychometric properties Correlation coefficients of the subscales ranged between insuf-
ficient and moderate (.66; .79; .60 respectively; Table 4a).
Feasibility Test- retest reliability analyses of relative forms showed insuf-
Tables 2a and 2b present score distributions (mean, median, ficient and moderate correlation coefficients of all total scales
SD, range, skewness) of the total scale scores and subscale (ranges .60–.79; Table 4b). Correlation coefficients of the
scores of AQ, PCRS and DEX among patients and relatives at subscales ranged between insufficient and sufficient (ranges
time point T1. No floor and ceiling effects were found on any .61- .84; Table 4b).
questionnaire; only maximally five participants scored the
lowest or the highest possible score (<15%). With respect to Internal consistency
the missing values, no data from AQ, PCRS and DEX instru- Cronbach’s alpha’s of the AQ, PCRS and DEX at the pre-
ments were discarded. treatment measurement (T1) was good for patients’ and rela-
tives’ scores (ranges .80-.89; Tables 4a and 4b).
Responsiveness
Patients’ scores between baseline measurements (T0–T1) and Construct validity
between the post-treatment and follow-up measurement (T2– Patient forms. Principal component analysis of the AQ ques-
T3) showed no significant differences (See Table 3). A trend tionnaire showed that a factor structure retaining 12 items

Table 2a. Score distributions of AQ, PCRS and DEX among patients at T1 (n = 105).
Senso/Motor
Statistic AQ Cognition Behaviour functioning PCRS ADL Interpersonal Cognition Emotional DEX
Items 17 7 6 4 30 8 7 8 7 20
Range, scale 18–90 30–150 0–80
Score, mean ± SD 38.5 ± 7.3 13.5 ± 3.4 12.7 ±3.3 10.0 ± 2.2 105.9 ± 14.1 33.1 ± 4.8 26.1 ± 4.8 26.3 ±5.0 20.4 ± 3.9 33. ± 12.5
Range of scores 19.1–57.4 7.0–22.0 6.0–22.0 4.0–18.0 75.0–141.0 17.0–40.0 14.0–40.0 16.0–38.0 10.0–30.0 8.0–60.0
Median 39.3 13.0 13.0 10.0 107.0 34.0 26.0 26.0 20.5 33.0
Interquartile scores 10.3 5.0 4.5 3.0 20.8 6.6 7.0 6.0 5.0 20.0
Missing items (%) 13.3 13.3 13.3 11.4 20.0 18.1 20.0 20.0 20.0 9.5
Skewness, mean ± SD .1 ±.3 .5 ±3 .3 ±3 .3 ±3 −.06 ±.3 −.1±4.8 .2 ± 3 .1± 3 .0± 3 .0 ± 3
% with lowest score 1.0 1.9 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
% with highest score 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
AQ = Awareness Questionnaire; PCRS = Patient Competency Rating Scale; DEX = Dysexecutive Questionnaire

Table 2b. Score distributions of AQ, PCRS and DEX among relatives at T1 (n = 105).
Senso/Motor
Statistics AQ Cognition Behaviour functioning PCRS ADL Interpersonal Cognition Emotional DEX
Items 17 7 6 4 30 8 7 8 7 20
Range, scale 18–90 30–150 0–80
Score, mean ± SD 36.5 ± 6.6 13.2 ±3.2 11.1 ±2.6 9.9 ±2.0 103.8 ± 12.4 31.9 ± 4.9 25.7 ± 4.4 26.7 ± 4.4 20.0 ± 33.7 ± 12.7
3.7
Range of scores 22.3–57.2 7.0–21.0 6.0–18.0 5.0–14.0 80.0–137.0 20.0–40.0 16.5–38.0 16.0–35.0 11.0–30.0 3.0–62.0
Median 36.1 13.0 11.0 10.0 106.0 33.0 25.0 27.0 19.5 33.0
Interquartile scores 9.6 4.0 3.5 3.0 19.0 8.7 6.5 5.9 5.0 18.0
Missing items (%) 20 19 19 14.3 23.8 21.9 23.8 23.8 23.8 19.0
Skewness, mean ± SD .0 ±.3 .3 ± 3 .0 ± 3 −.6 ± 3 −.2 ±.3 −.6 ±.3 .1 ± 3 −.4± 3 .3± 3 .2± 3
% with lowest score 1.9 1.9 4.8 1.9 2.9 1.0 1.0 1.0 1.0 1.0
% with highest score 1.0 1.9 1.0 1.0 1.0 1.0 1.0 2.9 1.0 1.9
AQ = Awareness Questionnaire; PCRS = Patient Competency Rating Scale; DEX = Dysexecutive Questionnaire
 BRAIN INJURY 1473

Table 3. Mean scores of AQ, PCRS and DEX of patients and relatives.
T0-T1 T1-T2 T2-T3
Mean difference Mean difference Mean difference
Patientsc T0 T1 T2 T3 (d) (d) (d)
Patient Competency Rating Scale, PCRS (range 33 84 79 50
75–147, n)
Mean PCRS (SD) 106.8 (13.6) 105.9 (14.1) 115.9 (13.1) 116.4 (13.8) 0.90 (.06) −10 (.73) −0.5 (.04)
Awareness Questionnaire, AQ (range 19–73, n) 32 91 74 51
Mean AQ (SD) 37.8 (6.3) 38.5 (7.3) 43.3 (9.5) 42.2 (8.2) −0.70 (.10) −4.80 (.57) 1.10 (.12)
Dysexecutive Questionnaire, DEX (range 1–60, n) 33 95 93 56
Mean DEX (SD) 34.2 (12.3) 33.1 (12.5) 23.3 (10.4) 23.7 (12.1) 1.10 (.10) 9.80 (.85) −0.4 (.04)
Relatives a
T0 T1 T2 T3
Patient Competency Rating Scale, PCRS (range 28 80 73 44
75–145, n)
Mean PCRS (SD)b 100.3 (14.0) 103.8 (12.4) 110.0 (13.7) 112.6 (13.6) −3.50 (.10) −6.20 (.47) −2.60 (.06)
Awareness Questionnaire, AQ (range 22–90, n) 27 85 69 43
Mean AQ (SD)b 37.2 (8.0) 36.5 (6.6) 42.9 (11.3) 43.6 (11.8) 0.70 (.26) −6.40 (.69) −0.70 (.19)
Dysexecutive Questionnaire, DEX (range 2–71, n) 29 94 85 52
Mean DEX (SD)b 35.3 (13.8) 33.7 (12.7) 27.7 (12.5) 27.3 (13.4) 1.60 (.12) 6.0 (.48) 0.40 (.03)
Discrepancy scores d
T0 T1 T2 T3
Mean PCRS discrepancy score (SD) (range −25–45) 5.5 (15.0) 1.7 (12.8) 5.1 (13.3) 5.3 (11.5)
Mean AQ discrepancy score (SD) (range −38–35) 0.4 (8.6) 1.6 (7.4) 0.4 (12.1) −2.0 (10.6)
Mean DEX discrepancy score (SD) (range −43–34) −1.3 (15.9) −1.3 (13.3) −4.0 (13.0) −4.1 (11.4)
a
Relatives were brother, sister, parent, child or friend
b
Relative’s scores for patient.
c
Patient scores for themselves
d
Patient score minus relative’s score
PCRS = Patient Competency Scale; AQ = Awareness Questionnaire; DEX = Dysexecutive Questionnaire; Discrepancy score = Patient score minus relatives score;
T0 = baseline, T1 = pre-treatment, T2 = post-treatment, T3 = follow-up; d = Cohen’s d

Table 4b. Test-retest reliability of AQ, PCRS and DEX among relatives.
Table 4a. Test-retest reliability of AQ, PCRS and DEX among patients.
T0-T1
T0-T1
Variables r Cronbach’s α
Variables r Cronbach’s α
AQ .66** .82
AQ .80** .80 AQ_Cognition .82**
AQ_Cognition .67** AQ_Behaviour .74**
AQ_Behaviour .74** AQ_Senso/Motor .61**
AQ_Senso/Motor .59** PCRS .79** .87
PCRS .75** .89 PCRS_ADL .69**
PCRS_ADL .78* PCRS_Interpersonal .72**
PCRS_Interpersonal .75* PCRS_Emotional .84**
PCRS_Emotional .76** PCRS_Cognition .84**
PCRS_Cognition .63** DEX .60** .89
DEX .88** .89
NOTE: Correlation coefficient is Spearman’s correlation.
NOTE: Correlation coefficient is Spearman’s correlation. *. Correlation is significant at the 0.05 level (2-tailed).
*. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).
**. Correlation is significant at the 0.01 level (2-tailed). AQ = Awareness Questionnaire; AQ_Cognition = Awareness Questionnaire_Cognition;
AQ = Awareness Questionnaire; AQ_Cognition = Awareness AQ_Behavior = Awareness Questionnaire_Behaviour; AQ_Senso/
Questionnaire_Cognition; AQ_Behavior = Awareness Questionnaire_Behaviour; motor = Awareness Questionnaire_Senso/Motorfunctioning; PCRS = Patient
AQ_Senso/motor = Awareness Questionnaire_Senso/Motorfunctioning; Competency Rating Scale; PCRS_ADL = Patient Competency Rating Scale_
PCRS = Patient Competency Rating Scale; PCRS_ADL = Patient Competency Activities of Daily Living; PCRS_Interpersonal = Patient Competency Rating Scale_
Rating Scale_ Activities of Daily Living; PCRS_Intpersonal = Patient Competency Interpersonal; PCRS_Emotional = Patient Competency Rating Scale_ Emotional;
Rating Scale_ Interpersonal; PCRS_Emotional = Patient Competency Rating Scale_ PCRS_Cognition = Patient Competency Rating Scale_ Cognition;
Emotional; PCRS_Cognition = Patient Competency Rating Scale_ Cognition; DEX = Dysexecutive Questionnaire; T0 = baseline measurement; T1 = pre-treatment
DEX = Dysexecutive Questionnaire; T0 = baseline measurement; T1 = pre-treatment measurement
measurement

factor. Items for each factor, factor loadings and internal

accounted for 50.9% of the variance. Data were most ade-
quately described by a three-factor solution. Three factors
represented cognitive (four items), behavioural/affective (five
items) and senso/motorfunctioning (four items). In the alter-
native viable model, items 4, 10, 15, and 17 were excluded.
Item 4 (performance on thinking and memory skill tests) was
excluded based on functional considerations. Item 10 (keep-
ing up with the time and date)
failed to load >0
4 on at least one factor. Item 15 (ability
to organize) and item 17 (ability to adjust emotionally) cross-
loaded on the behavioural and senso/motor functioning
consistency scores of the sub-scales (Cronbach’s alpha) are
displayed in Table 5a. The internal consistency of the mod-
ified AQ scale after removing these items remained
high (α = .71).
Principal component analysis of the PCRS showed that a
factor structure retaining 22 items accounted for 56.2% of
the variance. Data were most adequately described by a
four-factor solution. The four factors symbolized activities
of daily living (four items), interpersonal aspects (seven
items), cognitive aspects (seven items) and emotional func-
tioning (four items). In order to modify a viable alternative
1474 D. HELLEBREKERS ET AL.

Table 5a. Factor structure of AQ among patients. Table 6a. Factor structure of patient competency rating scale among patients.
Factor Items Factor Loading
Items Loading Factor 1: ADL (α = .81)
Factor 1: Cognitive functioning (α = .70) 1. Preparing meals 0.66
5. How well can you do the things you want to do in life now as 0.62 2. Dressing 0.72
compared to before your injury? 4. Washing the dishes 0.85
11. How well can you concentrate now as compared to before 0.75 5. Doing the laundry 0.82
your injury? Factor 2: Interpersonal (α = .79)
12. How well can you express your thoughts to others now as 0.58 8. Starting group conversations 0.62
compared to before your injury? 15. Help when confused 0.56
13. How good is your memory for recent events now as 0.77 16. Adjusting to change 0.45
compared to before your injury? 20. Acting appropriately 0.72
Factor 2: Behavioural functioning (α = .72) 21. Showing affection 0.62
22. Group activities 0.40
1. How good is your ability to live independently now as 0.70 30. Controlling laughter 0.54
compared to before your injury? Factor 3: Cognitive (α = .85)
2. How good is your ability to manage money now as compared 0.60 7. Keeping appointments on time 0.68
to before your injury? 10. Remembering dinner 0.77
3. How well do you get along with people now as compared to 0.61 11. Remembering names 0.63
12. Remembering schedule 0.82
before your injury? 13. Remembering things have to do 0.72
14. How good are you at planning things now as compared to 0.71 24. Scheduling daily activities 0.50
before your injury? 26. Meeting daily responsibilities 0.68
16. How well can you keep your feelings in control now as 0.74 Factor 4: Emotional (α = .59)
compared to before your injury? 17. Handling arguments 0.77
Factor 3: Senso/motor functioning (α = .62) 18. Accepting criticism 0.43
6. How well are you able to see now as compared to before 0.53 25. Understanding new instructions 0.60
your injury? 27. Controlling temper 0.55
7. How well can you hear now as compared to before your 0.51
injury? Note. Only factor loadings over 0.4 are included. Items 3, 6, 9, 14, 19, 23, 28 and
8. How well can you move your arms and legs now as compared 0.78 29 were excluded.
to before your injury?
9. How good is your coordination now as compared to before 0.79
your injury?
Note. Only factor loadings over 0.4 are included. Items 4, 10, 15 and 17 were factor. Item 14 (driving if had to) and item 19 (controlling
excluded. crying) failed to load <0
4. Item 23 (ability to recognise
when something she/he says or does upset someone else)
loaded on the ADL factor. Item 9 (staying involved), item
Table 5b. Factor structure of AQ among relatives. 28 (keeping from being depressed) and item 29 (controlling
Factor emotions) are split across factors on both the cognitive and
Items Loading
emotional factor. Table 6a illustrates the sub-scales with
Factor 1: Cognitive functioning (α = .71)
10. How good are you at keeping up with the time and date 0.50 corresponding factors, factor loadings and internal consis-
and where you are now as compared to before your injury? tency scores (Cronbach’s alpha). The internal consistency of
14. How good are you at planning things now as compared to 0.88
before your injury?
the modified PCRS scale after removing these items
15. How well organized are you now as compared to before 0.88 remained high (α = .88).
your injury? Principal component factor analysis of DEX resulted in a
Factor 2: Behavioural functioning (α = .67)
3. How well do you get along with people now as compared to 0.70 two-structure model of 10 items. Items accounted for 46.8% of
before your injury? the variance. In order to modify a viable alternative model, items
12. How well can you express your thoughts to others now as 0.57 3, 6, 8, 9, 10, 13, 14, 15, 17 and 18 were excluded. Item 3
compared to before your injury?
16. How well can you keep your feelings in control now as 0.74 (confabulation), item 9 (disinhibition), item 13 (lack of concern)
compared to before your injury? cross-loaded (> .040) on factor 2 and 3. Item 6 (temporal
17. How well adjusted emotionally are you now as compared to 0.74
before your injury?
sequencing problems) split across factors 1 and 3. Item 10
Factor 3: Senso/Motor functioning (α = .69) cross-loaded (>.040) on factors 1, 2 and 3. Item 8 (apathy and
5. How well can you do the things you want to do in life now as 0.58 lack of drive) and item 17 (knowledge-response dissociation)
compared to before your injury?
6. How well are you able to see now as compared to before 0.57 cross-loaded (>.040) on factors 2 and 4. Item 14 (perseveration)
your injury? split across on factors 1 and 4. Item 15 (restlessness-hyperkin-
8. How well can you move your arms and legs now as compared 0.76 esis) and item 18 (distractibility) cross-loaded (>.040) on factors
to before your injury?
9. How good is your coordination now as compared to before 0.81 1 and 2. Table 7a illustrates the sub-scales with corresponding
your injury? factors, factor loadings and internal consistency scores
11. How well can you concentrate now as compared to before 0.47
your injury?
(Cronbach’s alpha). The internal consistency of the modified
Note. Only factor loadings over 0.4 are included. Items 1,2,4,7,10 were excluded.
DEX scale after removing these items remained good (α = .77).

Relative forms. Principal component analysis of the AQ

model, items 3, 6, 9, 14, 19, 23, 28 and 29 were excluded. showed that a factor structure retaining 12 items accounted
Item 3 (ability to take care of personal hygiene) cross- for 48.8% of the variance. Data were most adequately
loaded (>.040) on the ADL and interpersonal factor. Item described by a three-factor solution. Three factors represented
6 (ability to manage finances) loaded on the emotional cognitive (three items), behavioural/affective (four items) and
 BRAIN INJURY 1475

Table 6b. Factor structure of patient competency rating scale among relatives. Table 7b. Factor structure of Dysexecutive Questionnaire among relatives.
Items Factor Loading Factor
Factor 1: ADL (α = .83) Items Loading
1. Preparing meals 0.75 Factor 1 (α = .80)
2. Dressing 0.62 1. Problems in understanding people meanings 0.48
3. Personal hygiene 0.69 2. Acting without thinking 0.67
4. Washing the dishes 0.78 7. Difficult realizing the extent of problems and unrealistic 0.69
5. Doing the laundry 0.82 about future
24. Scheduling daily activities 0.40 9. Doing or saying embarrassing things in company of ethers 0.74
26. Meeting daily responsibilities 0.65 11. Trouble making decisions 0.41
Factor 2: Interpersonal (α = .73) 13. Unconcerned about how I should behave in certain 0.75
8. Starting group conversations 0.81 situations
15. Help when confused 0.62 20. Unaware or unconcerned about others feeling about my .075
21. Showing affection 0.66 behaviour
22. Group activities 0.73 Factor 2 (α = .68)
Factor 3: Cognitive (α = .78)
10. Remembering dinner 0.76 14. Hard to stop repeating saying or doing things once started 0.66
11. Remembering names 0.73 15. Being very restless and can’t sit still for any length of time 0.81
12. Remembering schedule 0.84 16. Getting over-excited about things, which can be a bit over 0.79
13. Remembering things have to do 0.79 the top
23. Scheduling daily activities 0.50 Factor 3 (α = .68)
Factor 4: Emotional (α = .74) 4. Difficult thinking ahead or planning future 0.79
16. Handling arguments 0.56 6. Get events mixed up with each other and get confused about 0.48
18. Accepting criticism 0.55 the right order
19. Controlling crying 0.64 19. Trouble making decisions 0.86
27. Controlling temper 0.48
28. Keeping from being depressed 0.76 Note. Only factor loadings over 0.4 are included. Items 3, 5, 8, 10, 12, 17 and 18
29. Controlling emotions 0.80 were excluded
Note. Only factor loadings over 0.4 are included. Items 6, 7, 9, 14, 17, 20, 25 and
30 were excluded
Data were most adequately described by a four-factor solution.
The four factors symbolized activities of daily living (seven items),
interpersonal aspects (four items), cognitive aspects (five items)
Table 7a. Factor structure of Dysexecutive Questionnaire among patients. and emotional functioning (six items). In order to modify a viable
Factor alternative model, items 6, 7, 9, 14, 17, 20, 25 and 30 were excluded.
Items Loading Item 6 (ability to manage finances), item 9 (staying involved), item
Factor 1 (α = .70) 14 (driving if had to), item 20 (acting appropriately), item 25
4. Difficult thinking ahead or planning future 0.63
7. Difficult realizing the extent of problems and unrealistic 0.51 (understanding new instructions) and item 30 (controlling laugh-
about future ter) failed to load >0
4. Item 7 (keeping appointments on time)
11. Difficult to show emotions 0.69
16. Difficult to stop doing something even if I know I shouldn’t 0.53 cross-loaded on the ADL and cognitive factor. Item 17 (handling
19. Trouble making decisions 0.65 arguments) split across the emotional and interpersonal. Table 6b
20. Unaware or unconcerned about others feeling about my 0.63 illustrates the sub- scales with corresponding factors, factor load-
behaviour
Factor 2 (α = .73) ings and internal consistency scores (Cronbach’s alpha). The
1. Problems in understanding people meanings 0.59 internal consistency of the modified PCRS scale after removing
2. Acting without thinking 0.62
5. Getting over-excited about things, which can be a bit over 0.83
these items remained high (α = .83).
the top Principal component analysis of the DEX revealed a factor
12. Losing temper at slightest 0.83 structure retaining 13 items that accounted for 53.5% of the
Note. Only factor loadings over 0.4 are included. Items 3, 6, 8, 9, 10, 13, 14, 15, variance. Items 3, 5, 8, 10, 12, 17 and 18 were excluded. Item 3
17 and 18 were excluded
(confabulation), item 10 (disturbed impulse control), item 17
(knowledge-response dissociation) cross-loaded (>.040) on fac-
tor 1 and 2. Item 5 (euphoria), item 8 (apathy and lack of drive)
and item 12 (aggression) cross-loaded (>.040) on factor 1 and 3.
senso/motor functioning (five items). In the alternative viable
Item 18 (distractibility) failed to load >0
4. Table 7b illustrates
model, items 1, 2, 4, 7 and 13 were excluded. Item 1 (ability to
the sub- scales with corresponding factors, factor loadings and
live independently), item 2 (ability to manage money) and
internal consistency scores (Cronbach’s alpha). The internal
item 4 (performance on thinking and memory skill tests)
consistency of the modified DEX scale after removing these
cross-loaded (>.040) on the cognitive and behavioural func-
tioning factor. Item 7 (ability to hear) and item 13 (ability to items remained high (α = .82).
remember recent events) failed to load >0
4 on at least one
factor. Items for each factor, factor loadings and internal
consistency scores of the sub-scales (Cronbach’s alpha) are Discussion
displayed in Table 5b. The internal consistency of the mod- The aim of the current study was to assess the psychometric
ified AQ scale after removing these items remained high properties of the AQ, PCRS and DEX in a group of patients
(α = .75). with ABI and their relatives. The feasibility, floor and ceiling
Principal component analysis of the PCRS showed that a factor effects, responsiveness, test-retest reliability, internal consis-
structure retaining 22 items accounted for 47.7% of the variance. tency, and construct validity of all three questionnaires were
1476 D. HELLEBREKERS ET AL.
investigated. Overall, results demonstrated good psychometric
properties of the total scores, which were similar to those of
earlier studies (5,14–16,18). Results concerning feasibility of
all three questionnaires showed no missing items, no floor
and ceiling effects and acceptable skewness values. Results of
responsiveness analyses showed moderate to large effect sizes
between pre and post-treatment measurement for both
patients and relatives forms.
Regarding the AQ, overall results showed that patient
and relative forms are reliable to assess self-awareness in
patients by comparing current functioning with functioning
before the brain injury(2). Results of test-retest reliability
and internal consistency on patient forms were high and
similar to those of earlier studies that investigated patients
with TBI (13). However, test-retest reliability outcomes of
relative forms were insufficient to moderate. These current
outcomes cannot be compared to previous studies, since
test-retest results for the relative forms were not studied
before (2). However, our current findings could be
explained by the important role of relatives in the INR
intervention. It is desirable that they participate in the
INR intervention, to become more aware of the ABI con-
sequences for patients. This may cause differences on AQ
questionnaire scores and affect test-retest reliability.
Additionally, environmental factors (i.e. life-events) of rela-
tives may provoke different scores on time-point T0 com-
pared to T1.
For the internal consistency, current outcomes for relatives
forms are good and in accordance with previous studies by
Jarman et al. and Sherer et al. (3,16) Finally, the current study
confirmed the three-factor solution of the AQ, with the
expected factors reflecting cognition, behaviour/affective and
physical (motor/sensor). These were previously defined by
Sherer et al.(16). However, we found some differences in
item loadings within these factors. Concerning the AQ rela-
tive forms, we found a three-factor solution similar to that for
the patient forms, with similar characteristics. However, dur-
ing this analysis some items cross-loaded differently than the
items for patient forms. The internal consistency of the mod-
ified AQ relative form was high, but seem to be lower than
that reported by Sherer er al (16).
The inconsistency regarding construct validity between the
current study and previous findings might be caused by
differences in the study population: (1) type of injury, and
(2) time of data collection. Sherer et al.(16). focused on (1)
TBI instead on various types of ABI, and (2) patients and
relatives were investigated approximately between 10 and
21 months. Overall, it can be concluded that AQ relative
forms also measure three factors reflecting cognitive, beha-
viour/effective and physical functioning.
Regarding the PCRS, overall results showed that patient
and relative forms are reliable self-awareness measures to
assess patient competencies in the present time (2). Our
study showed that the results of test-retest reliability of patient
and relative forms were moderate. Internal consistencies of
both forms were high, which is comparable to the findings of
Smeets et al.(15). Our outcomes are not in accordance with
those of Prigatano et al.(14). They reported higher test-retest
reliability outcomes for patient and relative forms (5).
However, Prigatano et al. evaluated these forms in a study
population of patients with solely TBI. Additionally, our study
has confirmed the four-factor structure model for patient
forms. The expected factors symbolized activities of daily
living, interpersonal and emotional aspects and cognition
(11). However, several items loaded differently compared to
previous findings (11). These inconsistencies between our
outcomes and those of Leathem et al.(11). might be due to
differences (1) in age, (2) in time of data collection, and (3)
type of injury. The study by Leathem et al. (1) focused on
young participants with a mean age of 34 years, (2) used data
collected approximately 26 months after the brain injury, and
(3) focused on TBI instead on ABI.
The principal component analysis of relative forms showed
a similar four-structure model with similar factors as the
analysis of the patient forms. No other previous findings of
principal component analysis of relative forms are known.
Overall, it can be concluded that relative forms of the PCRS
measure four factors that symbolize activities of daily living,
interpersonal aspects, emotional aspects and cognition (11).
Regarding the DEX, the psychometric properties differed
somewhat compared to previous findings. In the current
study, we found that test-retest reliabilities were sufficient in
patient forms and insufficient in relative forms. Due to the
limited research in psychometric properties of the DEX, test-
retest analysis cannot be compared with the previous findings.
In the current study, the internal consistency outcomes of
patient and relative forms were high and are comparable to
the previous study of Bodenburg and Dopslaff (18).
Surprisingly, we could not confirm the four-factor structure
of the DEX for the patient form that was previously reported
by Bodenburg and Dopslaff (18) and Mooney et al.(19).
Bodenburg and Dopslaff (18) identified factors reflecting initi-
ating & sustaining actions (factor 1), impulse control &
sequencing of heard information (factor 2), psychophysical
& mental excitability (factor 3), and social conventions (factor
4) (17). In the current study we found a two-factor structure
model. The two factors might symbolize initiating & sustain-
ing actions (factor 1) and impulse control & sequencing of
heard information (factor 2) reported by Bodenburg and
Dopslaff (18). We excluded several items of the DEX patient
forms because they cross-loaded on two factors. It is possible
that the excluded items represented one of the four factors
reported in the study of Bodenburg and Dopslaff (18). They
did not exclude items based on statistical and functional
considerations, which might explain the differences with our
findings. Furthermore, we found a three-factor structure for
the relative forms. Items within the first factor could charac-
terize factor 1, but also factor 4. For the second factor, items
symbolize factor 2 of Bodenburg and Dopslaff (18). We
thought that the items in our third factor did not reflect one
of the four factors previously found by Bodenburg and
Dopslaff (18). In our opinion these items could characterize
a new factor ‘planning & decision-making’ (18).
Unfortunately, our construct validity results cannot be com-
pared with previous findings due to the lack of information
on these findings.
Due to the large sample size of the current study, we have
added important and valuable information about

psychometric properties of patient and relative forms of the
AQ, PCRS and DEX. Smeets et al.(2). previously reported that
little information was available on responsiveness and test-
retest reliability in patient and relative forms of AQ and
PCRS.
Nevertheless, the mixed results of the construct validity
should be interpreted cautiously due to the following limita-
tions. Items were excluded based on both statistical and func-
tional considerations. This means that items were excluded if
they loaded on two factors and not loaded properly, according
to the meaning of the relevant subscale. Another limitation is
that the current study included patients with ABI, whereas
previous studies focused on TBI. It is known that type of
injury effects the level of self-awareness problems. Thereby,
difference in aetiologies may have affected our results on
problems in self-awareness (2,6,15). Additionally, another
limitation is that information was collected for clinical pur-
poses instead for assessing psychometric properties.
Therefore, the current retrospective study could only evaluate
the available data of the questionnaires, while some patient
and relative information was lacking. As a result, some infor-
mation on psychometric properties is still lacking: the inter-
rater reliability of AQ and DEX forms, more comparable
information of test–retest reliability of AQ and PCRS relatives
and clinician forms, and internal consistencies of AQ, PCRS
and DEX clinician forms could not be derived from the
available data.
Conclusion
The overall scores of the AQ, PCRS and DEX have acceptable
psychometric properties. However, the classification of the
items within the subscales is questionable; the current study
could not replicate the factor structures of the instruments.
The choice for an instrument to measure a patient’s awareness
of deficits after ABI depends on what and how one wants to
measure. Both the AQ and PCRS assess awareness of deficits
in various domains. However, only the domains of the AQ
can be interpreted separately. Another important difference is
that the AQ focuses more on current functioning and com-
pares it with functioning before brain injury, whereas the
PCRS asks about a patient competencies only in the present
time (2). The DEX can be a reliable measure to assess aware-
ness in executive behaviour, but is less commonly used to
measure awareness of deficits in other types of cognitive and
physical domains after ABI. Although, additional research
into the quality (test-retest, interrater reliability and respon-
siveness) of relative and clinician forms of these instruments
is necessary, their quantitative nature makes them suitable for
research. Moreover, we recommend the use of these ques-
tionnaires to measure a patient’s self-awareness after ABI in
clinical practice.
Acknowledgments
We would like to thank all participating patients of the Rehabilitation
Research Centre, Reade, Amsterdam, the Netherlands.
BRAIN INJURY 1477
Potential Conflicts of interest
None declared
References
1. Port A, Willmott C, Charlton J. Self-awareness following trau-
matic brain injury and implications for rehabilitation. Brain Inj.
2002;16:277–89. doi:10.1080/02699050110103274.
2. Smeets SM, Ponds RW, Verhey FR, van Heugten CM.
Psychometric properties and feasibility of instruments used to
assess awareness of deficits after acquired brain injury: a systema-
tic review. J Head Trauma Rehabil. 2012;27:433–42. doi:10.1097/
HTR.0b013e3182242f98.
3. Jarman RF, Vavrik J, Walton PD. Metacognitive and frontal lobe
processes: at the interface of cognitive psychology and neuropsy-
chology. Genetic, social, and general psychology monographs.
1995; 121: 153–210.
4. Toglia J, Kirk U. Understanding awareness deficits following brain
injury. NeuroRehabilitation. 2000;15:57–70.
5. Fleming JM, Strong J, Ashton R. Self-awareness of deficits in
adults with traumatic brain injury: how best to measure? Brain
Inj. 1996;10:1–16. doi:10.1080/026990596124674.
6. Fleming JM, Winnington HT, McGillivray AJ, Tatarevic BA,
Ownsworth TL. The development of self-awareness and relation-
ship to emotional functioning during early community reintegra-
tion after traumatic brain injury. Brain Impairment. 2006;7:83–94.
doi:10.1375/brim.7.2.83.
7. Prigatano G, Altman I. Impaired awareness of behavioral limita-
tions after traumatic brain injury. Arch Phys Med Rehabil.
1990;71:1058–64.
8. Ownsworth T, Fleming J. The relative importance of metacogni-
tive skills, emotional status, and executive function in psychoso-
cial adjustment following acquired brain injury. J Head Trauma
Rehabil. 2005;20:315–32. doi:10.1097/00001199-200507000-00004.
9. Ownsworth TL, Oei TP. Depression after traumatic brain injury:
conceptualization and treatment considerations. Brain Inj.
1998;12:735–51. doi:10.1080/026990598122133.
10. Bach LJ, David AS. Self-awareness after acquired and traumatic
brain injury. Neuropsychol Rehabil. 2006;16:397–414.
doi:10.1080/09602010500412830.
11. Leathem JM, Murphy LJ, Flett RA. Self-and informant-ratings on
the patient competency rating scale in patients with traumatic
brain injury. J Clin Exp Neuropsychol. 1998;20:694–705.
doi:10.1076/jcen.20.5.694.1122.
12. Sandhaug M, Andelic N, Berntsen SA, Seiler S, Mygland A. Self
and near relative ratings of functional level one year after trau-
matic brain injury. Disabil Rehabil. 2012;34:904–09. doi:10.3109/
09638288.2011.626484.
13. Sherer M, Hart T, Nick TG. Measurement of impaired self-aware-
ness after traumatic brain injury: a comparison of the patient
competency rating scale and the awareness questionnaire. Brain
Inj. 2003;17:25–37. doi:10.1080/0269905021000010113.
14. Prigatano GP, Altman IM, O’brien KP. Behavioral limitations that
traumatic-brain-injured patients tend to underestimate. Clin
Neuropsychol. 1990;4:163–76. doi:10.1080/13854049008401509.
15. Smeets SM, Ponds RW, Wolters Gregório G, Pouwels CG,
Visscher AJ, Winkens I, van Heugten CM. Impaired awareness
of deficits in individuals with neuropsychiatric symptoms after
acquired brain injury: associations with treatment motivation and
depressive symptoms. Neuropsychology. 2014;28:717.
doi:10.1037/neu0000068.
16. Sherer M, Bergloff P, Boake C, Jr W H, Levin E. The Awareness
Questionnaire: factor structure and internal consistency. Brain Inj.
1998;12:63–68. doi:10.1080/026990598122863.
17. Jackson J, Ely EW, Morey MC, Anderson VM, Denne LB, Clune J,
Siebert CS, Archer KR, Torres R, Janz D, et al. Cognitive and
physical rehabilitation of ICU survivors: results of the RETURN
randomized, controlled pilot investigation. Crit Care Med.
2012;40:1088. doi:10.1097/CCM.0b013e3182373115.
1478 D. HELLEBREKERS ET AL.
18. Bodenburg S, Dopslaff N. The Dysexecutive Questionnaire
advanced: item and test score characteristics, 4-factor solution,
and severity classification. J Nerv Ment Dis. 2008;196:75–78.
doi:10.1097/NMD.0b013e31815faa2b.
19. Mooney B, Walmsley C, McFarland K. Factor analysis of the self-
report Dysexecutive (DEX-S) Questionnaire. Appl Neuropsychol.
2006;13:12–18. doi:10.1207/s15324826an1301_2.
20. Ben-Yishay Y. Postacute Neuropsychological Rehabilitation. In:
Christensen AL, Uzzell BP (eds) International Handbook of
Neuropsychological Rehabilitation. In: Critical Issues in
Neuropsychology: Boston, MASpringer; 2000.
21. Holleman M, Vink M, Nijland R, Schmand B. Effects of inten-
sive neuropsychological rehabilitation for acquired brain injury.
Neuropsychol Rehabil. 2016:1–14. doi:10.1080/09602011.20
16.1210013.
22. Bie SE. Standaardvragen 1987: voorstellen voor uniformering van
vraagstellingen naar achtergrondkenmerken in interviews: Leiden:
VOI, Stuurgroep Dataverzameling; 1987.
23. Hannay H, Howieson D, Loring D, Fischer J, Lezak M.
Neuropathology for neuropsychologists. Neuropsychol Assess.
2004;4:157–94.
24. Prigatano GP. Neuropsychological rehabilitation after brain
injury. Baltimore, MD: Johns Hopkins Univ Pr; 1986.
25. Burgess P, Alderman N, Wilson B, Evans J, Emslie H. The dysex-
ecutive questionnaire. Behavioural assessment of the dysexecutive
syndrome. Bury St Edmunds, UK: Thames Valley Test Company;
1996.
26. Amieva H, Phillips L, Della Sala S. Behavioral dysexecutive symp-
toms in normal aging. Brain Cogn. 2003;53:129–32. doi:10.1016/
S0278-2626(03)00094-0.
27. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL,
Dekker J, Bouter LM, de Vet HC. Quality criteria were pro-
posed for measurement properties of health status question-
naires. J Clin Epidemiol. 2007;60:34–42. doi:10.1016/j.jclin
epi.2006.03.012.
28. Boosman H, van Heugten CM, Winkens I, Smeets SM, Visser-
Meily JM. Further validation of the Motivation for Traumatic
Brain Injury Rehabilitation Questionnaire (MOT-Q) in patients
with acquired brain injury. Neuropsychol Rehabil. 2016;26:87–
102. doi:10.1080/09602011.2014.1001409.
29. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for
assessing responsiveness: a critical review and recommendations. J
Clin Epidemiol. 2000;53:459–68. doi:10.1016/S0895-4356(99)
00206-1.
30. Cohen J. A power primer. Psychol Bull. 1992;112:155.
doi:10.1037/0033-2909.112.1.155.
31. Visser-Meily JA, Post MW, Riphagen II, Lindeman E. Measures
used to assess burden among caregivers of stroke patients: a
review. Clin Rehabil. 2004;18:601–23. doi:10.1191/026921
5504cr776oa.