NEUROPSYCHOLOGICAL REHABILITATION

2007, 17 (3), 273– 292

An investigation of the differential effect of

self-generation to improve learning and memory
in multiple sclerosis and traumatic brain injury

Amanda O’Brien, Nancy Chiaravalloti, Juan Carlos Arango-

Lasprilla, Jeannie Lengenfelder, and John DeLuca
Kessler Medical Rehabilitation Research and Education Corporation,
Neuropsychology and Neuroscience Laboratory, West Orange, NJ, USA; and
University of Medicine and Dentistry of New Jersey, New Jersey Medical School,
Department of Physical Medicine and Rehabilitation, Newark, NJ, USA

The generation effect (GE) is a phenomenon in which material that is produced

by an individual is learned and remembered better than information that is
provided to that individual. The current study examined the potential benefits
of self-generation on learning and memory in individuals with traumatic
brain injury (TBI) and multiple sclerosis (MS). The impact of cognitive impair-
ment on the benefits of self-generation was also examined. Subjects consisted
of 18 individuals with TBI and 31 individuals with clinically definite MS.
Both the TBI and MS groups recalled significantly more words in the self-
generated condition versus the provided condition. Those impaired in the
domains of working memory, episodic memory, or executive functioning
demonstrated a significant benefit from self-generation (all ps , .05). Further-
more, although individuals with impairments in multiple cognitive domains
recalled fewer words overall compared to those with no or one impaired
cognitive domain, this group demonstrated a large effect size in the difference
in recall for generated versus provided words.
Results demonstrate that people with cognitive impairments can benefit from
self-generation to improve learning and memory. Future research should focus

Correspondence should be sent to: John DeLuca, Ph.D., KMRREC Neuropsychology and
Neuroscience Laboratory, 300 Executive Drive, Suite 10, West Orange, NJ 07052, USA.
E-mail: jdeluca@kmrrec.org
This work was funded in part by the National Multiple Sclerosis Society Grant PP0752 and
the Henry H. Kessler Foundation.

# 2007 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
http://www.psypress.com/neurorehab DOI:10.1080/09602010600751160
274 O’BRIEN ET AL.

on how to amplify the benefit of the GE for impaired groups, apply it to

everyday functional tasks, and sustain its effect over time.

INTRODUCTION
The generation effect (GE) is a phenomenon in which people demonstrate
better recall and recognition when material is produced by them compared
to simply being provided to them. This effect was first examined by Slamecka
and Graf (1978) who reported the superiority of self-generation in learning,
now referred to as the “generation effect”. Improved recall with self-generated
versus provided learning has been found across a variety of assessment
conditions, including cued and uncued recognition, free and cued recall, and
confidence ratings in recognition (McElroy & Slamecka, 1982). Since the
initial studies, the generation effect has been shown to be a robust pheno-
menon in various research studies with healthy adults of different ages over
the past 26 years (e.g., Begg, Vinski, Frankovich, & Holgate, 1991; Clark,
1995; Gardiner & Rowley, 1984; Payne, Neely, & Burns, 1986; Pesta,
Sanders, & Murphy, 1999; Taconnat & Isingrini, 2004).
Despite the documented benefits of self-generation on recall and recog-
nition among healthy individuals, there is limited research on the benefits
of the generation effect in neurologically impaired populations. Studies of
the generation effect in neurological populations have been conducted
primarily in samples of people with dementia, mostly of the Alzheimer’s
type (e.g., Barrett, Crucian, Schwartz, & Heilman, 2000; Mitchell, Hunt, &
Schmitt, 1986; Souliez et al., 1996). This work has shown that people with
dementia can benefit from self-generation of information, although the
degree of benefit has been shown to vary by the severity and nature of cog-
nitive impairments. More specifically, there is support for the benefit of the
generation effect in samples of people with mild and probable Alzheimer’s
disease (Barrett et al., 2000; Lipinska, Backman, Mantyla, & Viitanen,
1994; Multhaup & Balota, 1997; Souliez et al., 1996), but not in samples
of people with more severe Alzheimer’s or frontal-subcortical dementias
(Barrett et al., 2000; Dick & Kean, 1989; Mitchell et al., 1986; Multhaup
& Balota, 1997). It has been hypothesised that people with more severe
dementia may gain little from self-generation due to disruptions of semantic
networks (Hodges, 1994). Despite this work, little is known about how the
generation effect is impacted by severity or type of cognitive impairment in
neurological populations other than dementia.
It is well known that persons with multiple sclerosis (MS) or traumatic brain
injury (TBI) demonstrate significant impairments in acquisition of new
information (DeLuca, Barbieri-Berger, & Johnson, 1994; DeLuca et al.,
1998, 2000; Thornton & Raz, 1997; Vanderploeg, Crowell, & Curtiss, 2001).
 LEARNING AND MEMORY IN MS AND TBI 275

For example, in a series of experiments, persons with MS, TBI, and healthy
controls were equated for the amount of information initially acquired on a
list-learning task, by reaching a common learning criterion. Despite taking
significantly more trials to reach the learning criterion in the MS and TBI
groups, recall and recognition performance did not significantly differ from
that of healthy controls at 30-minute and 90-minute delays and even up to
1 week after learning (DeLuca et al., 1994, 1998, 2000). These and other
data clearly demonstrate that the primary deficit observed among persons
with MS and TBI is in the initial acquisition of information and not in
retrieval from long-term storage. As such, techniques that serve to maximise
learning and strengthen the acquisition of new information, such as the self-
generation of information, may prove effective in improving recall in persons
with MS and TBI.
There is preliminary evidence demonstrating the effectiveness of self-
generation in improving learning and memory in persons with MS and
TBI. Chiaravalloti and DeLuca (2002) found that self-generation of verbal
information in participants with MS significantly improved performance on
measures of both recall and recognition. Similar results were observed in
TBI by Lengenfelder, Chiaravalloti, and DeLuca (2002) who also reported
improved recall and recognition performance in the self-generated versus
provided conditions. However, it is unclear whether these two clinical
samples differ in their degree of benefit from self-generation techniques.
Additionally, how type or severity of cognitive impairment may impact
the benefits of self-generation in MS or TBI remains unclear. Cognitive
domains that have been consistently asserted as areas of common cognitive
impairment for MS (e.g., Gaudino, Chiaravalloti, DeLuca, & Diamond,
2001; Lengenfelder, Chiaravalloti, & DeLuca, 2003) and TBI (e.g., McKinlay
& Watkiss, 1999; Olver, Ponsford, & Curran, 1996) will be investigated in
the current study. These include working memory, episodic memory, and
executive functioning. Furthermore, previous work has demonstrated a
correlation between these cognitive domains and recall for generated
information (Chiaravalloti & DeLuca, 2002).
The first objective of the present study was to compare and contrast the
benefits of self-generation on recall between persons with TBI and MS.
Although it has been documented that both these groups demonstrate some
benefit from self-generation, it is unclear if they experience a differential
benefit due to the nature of the neurological characteristics involved in
these two different samples. Additionally, it is not clear if the beneficial
effect of self-generation will be evident at longer-term follow up or if individ-
uals with TBI and MS will differ in how this benefit is sustained over time.
The second objective was to clarify how the nature and severity of cognitive
impairments may impact the ability to benefit from self-generation in persons
with MS and TBI. The current study will investigate how cognitive
276 O’BRIEN ET AL.

impairment in specific domains (working memory, episodic memory, execu-

tive functioning) impacts the generation effect.

METHOD

Participants
Participants consisted of 18 individuals with TBI, 31 individuals diagnosed
with clinically definite MS as per the criteria of Poser et al. (1983), and 20
age- and education-matched healthy control participants, all of whom were
between the ages of 18 and 55. MS participants were at least 1 month post
their most recent exacerbation and free from corticosteroid use. The mean
time since diagnosis was 127.8 months (SD ¼ 100.2) and the overall severity
of physical disease, as measured by the Ambulation Index, was mild
(M ¼ 1.85, SD ¼ 2.00, range 0– 6). TBI participants were at least 6 months
post-injury and had a history of moderate to severe TBI, which was confirmed
either through Glasgow Coma Scale (GCS) score, positive loss of conscious-
ness or positive computed tomography (CT) or magnetic resonance imaging
(MRI) findings. For all participants, exclusion criteria included any history of
alcohol or drug abuse; major psychiatric disturbance; neurological disease or
injury prior or subsequent to current diagnosis; or presence of aphasia. All
subjects spoke fluent English.
Demographic characteristics of the sample are presented in Table 1. As
expected, the distribution of gender was significantly different between the
two diagnostic groups, the MS sample was 83.9% female and the TBI
sample 27.8% female (x2 ¼ 15.89, p , .01). The mean age of the total
sample was 42.6 years (SD ¼ 10.03) and average years of education were
14.8 (SD ¼ 2.10). There was a significant difference between the MS
(M ¼ 15.26, SD ¼ 2.20) and TBI (M ¼ 13.78, SD ¼ 1.80) groups for
average years of education, F(2, 68) ¼ 3.13, p , .05. However, there were
no significant group differences for estimated premorbid intelligence, as

TABLE 1
Demographic information

MS TBI HC

Age 45.42 (8.35) 42.44 (10.20) 38.40 (11.21)

Education (years) 15.26 (2.22) 13.78 (1.80) 15.00 (1.95)
Estimated premorbid intelligence 100.90 (11.56) 98.50 (10.60) 103.15 (9.29)
(WRAT-3 scaled score)
Percent female 83.9% 27.8% 50.0%


Difference between MS and TBI groups at p , .05.
 LEARNING AND MEMORY IN MS AND TBI 277

measured by the Wide Range Achievement Test– 3, reading subtest (WRAT-

3, Wilkinson, 1993). Generation effect data from the MS participants in the
current study were previously published by Chiaravalloti and DeLuca
(2002). Generation effect data from the TBI participants were previously pre-
sented by Lengenfelder et al. (2002). The current study will use these data to:
(1) compare the benefits of the generation effect between TBI and MS sub-
jects, and (2) investigate how type and severity of cognitive impairments
impact the generation effect.

Materials

Neuropsychological assessments

Digit Span. The Digit Span subtest of the Wechsler Adult Intelligence
Scale – Revised (WAIS-R) consists of random number sequences of increasing
length that an examiner reads aloud at a set rate (1 number per second). In the
“forward” portion of the task, the participant is instructed to repeat the number
sequence in the same order as it is heard. In the “backwards” portion of this
task, the participant is instructed to repeat the numbers in the reverse order
from what the examiner reads. The Digit Span task assesses attention and
working memory abilities. Digit Span Backwards score was examined in the
context of complex working memory for the current study (Wechsler, 1981).

Paced Auditory Serial Addition Test (PASAT) (Brittain et al., 1991). In

the PASAT, a series of numbers, ranging from 1 to 9, are presented orally
by a standardised audiotape. Participants are instructed to add each number
that they hear to the number that immediately preceded it and to orally
report the sum. The PASAT consists of four trials, each with varying digit
presentation rates (2.4, 2.0, 1.6, and 1.2 seconds). There are a total of 50
digits presented in each trial, resulting in 49 total responses. The PASAT
total correct responses across trials was used for analyses in the current study.

Stroop Color-Word Test (Golden, 1978). The Stroop Color-Word Test

consists of three trials in which participants (1) read a series of words that
are colour names, (2) name the colour in which a block of xs is printed,
and (3) name the colour of the ink that the names of different colours are
printed in (i.e., the word “green” may be printed in red ink, a correct response
would be to name the colour “red”). This final trial requires the participant to
inhibit the more automatic response of reading the word in order to properly
respond and give the name of the ink colour. In the current study, participants
were given 45 seconds per trial to generate as many correct responses as poss-
ible using the paper (as opposed to computer) version of the task. The Stroop
Color-Word total score was used for the current study (Golden, 1978).
278 O’BRIEN ET AL.

Oral Trailmaking Test – B (TMT-B) (Ricker & Axelrod). The oral TMT-B
is an adapted version of the widely used written TMT. In the oral TMT-B,
participants are asked to alternate numbers and letters in ascending and alpha-
betical order, respectively, as quickly as possible. The oral version of this task
was used in order to eliminate the potential effects of manual-motor symp-
toms on performance. The TMT-B is used to assess executive functioning
abilities including sequencing, and shifting set (Ricker, Axelrod, &
Houtler, 1996). Total time to complete the trailmaking task was used as the
outcome variable, compared to a normed sample.

Controlled Oral Word Association Test (COWAT) (Spreen & Strauss,

1991). The COWAT is a test of verbal fluency and initiation. Subjects are
given a letter of the alphabet and allowed 1 minute to name as many items
as they can that begin with that letter (excluding proper nouns, numbers, and
the same word with a different ending). Three trials, each with a different
letter (C, F, and L) are used. The total score is the number of correct responses
across the three trials.

Wide Range Achievement Test –3, Reading subtest (WRAT-3) (Wilkinson,

1993). The WRAT-3 reading subtest requires participants to read aloud a
list of 75 words. This task is widely used to assess premorbid intellectual abil-
ities, as it was in this study. Total words correctly read are compared to vali-
dated norms to generate a WRAT reading score.

Wechsler Memory Scale– Revised (WMS-R) (Wechsler, 1987), Logical

Memory I and II. In the WMS-R Logical Memory I and II, two paragraph-
length stories are read aloud to the participant. Immediately following each
story, the participant must recall as much of each story as possible. Both
stories are read once, with recall trials after each reading. Logical Memory I
is the immediate recall of the stories. For Logical Memory II, the subject is
again asked to recall the stories after a 30-minute delay.

Generation effect protocol

An adaptation of the generation effect (GE) task from Multhaup and Balota
(Mitchell et al., 1986) was utilised for the current study. In this adapted
version, 32 sentences were presented individually on separate pages. In the
“generated” condition 16 sentences were presented with the last word
missing, as indicated by a blank line. In the “provided” condition an additional
set of 16 complete sentences were presented in which the last word was under-
lined. Participants were instructed that they would see sentences on individual
pages and would be asked to read the sentence aloud. They were also told that
they would either (1) fill in the blank at the end of the sentence with an
 LEARNING AND MEMORY IN MS AND TBI 279

appropriate word or (2) simply read the sentence including an underlined

word. Generated and provided sentences were alternated randomly. Partici-
pants were presented with the sentences one at a time and the order of the sen-
tences was the same for all participants. After the target stimuli were
presented, participants completed a distractor task in which they were asked
to circle all even numbers on a given page. Following the distractor task, par-
ticipants’ recall of both generated and provided words was tested immediately,
after a 30-minute delay, and after a 1-week delay (the latter via telephone).

Procedure
Prior to recruiting participants, the research protocol was approved by the
institutional review board. Healthy control participants were recruited
through advertisements posted in the community. MS participants were
recruited through advertisements distributed and posted at local support
groups and clinics. TBI participants were recruited through advertisements
within the inpatient and outpatient departments of a rehabilitation facility.
Additionally, former study participants who had given prior consent for
future contact were telephoned and invited to participate.
Upon initial telephone contact, participants were screened for inclusion/
exclusion criteria. If they met the inclusion criteria described previously,
they were scheduled for an initial interview and testing session. Prior to
beginning the study, all participants signed an IRB-approved consent form
and participants’ questions were answered. The initial interview and testing
session lasted approximately 2 –3 hours and included administration of stan-
dard neuropsychological tests as well as the testing protocol to examine the
presence of a generation effect (GE). Approximately one week after this
initial session, participants were contacted by phone to assess verbal recall
of items presented within the GE protocol. All participants received compen-
sation for their participation.

Neuropsychological domains
In order to determine if a specific neuropsychological impairment affected
one’s ability to benefit from self-generation, three cognitive domain scores
were created from the neuropsychological test data. Domain scores were
created for complex working memory, episodic memory, and executive func-
tioning. These cognitive domain scores were investigated for two reasons.
First, they have been consistently asserted as areas of common cognitive
impairment for MS (e.g., Gaudino et al., 2001; Lengenfelder et al., 2003)
and TBI (e.g., McKinlay & Watkiss, 1999; Olver et al., 1996). Second,
previous work has demonstrated a correlation between these domains and
recall for generated information (Chiaravalloti & DeLuca, 2002). Tests
within each cognitive domain were chosen based on categorisation of the
280 O’BRIEN ET AL.

neuropsychological skills they assess, as defined by Lezak, Howieson, and

Loring (2004). More specifically, WAIS-III Digit Span Backwards and
Paced Auditory Serial Addition Test are designated by Lezak et al. (2004)
as tests of information processing and working memory. Logical Memory I
and II are noted as valid and reliable tests of verbal memory (Lezak et al.,
2004). Finally, several tests were incorporated into the executive functioning
domain to cover the areas that Lezak et al. define as components of executive
functioning; volition, planning, purposive action, and effective performance.
The COWAT, a test of fluency (volition/initiation), the Stroop Color-Word
test (inhibiting competing responses/self-regulation) and the oral adaptation
of the Trailmaking B test (shifting set) were chosen to assess comp-
rehensively executive functioning skills. Cognitive domain scores were
created by averaging the z-scores of relevant tests for each domain, following
the methodology used in previous studies with neurological subjects
(Chiaravalloti, DeLuca, Moore, & Ricker, 2005; Gaudino et al., 2001).
Each participant (in the TBI and MS groups) was categorised as either
“impaired” or “not impaired”’ in each of the three domains, based on score
comparisons to the healthy control group. Participants were categorised as
impaired in any domain for cognitive domain z-score less than – 1.00 in
order to include those with mild impairments in the “impairment” groups.
This more liberal definition of impairment was used as no research has
previously examined how cognitive impairment in MS or TBI impacts the
generation effect. As such, the decision was made to include a wide range
of impairment, including those with borderline or mild impairment. As dis-
cussed in Lezak et al. (2004) and Spreen and Strauss (1998) a test score of
1 SD below the mean is considered to represent an impairment to many exam-
iners and is often labelled “borderline abnormal” or “probably impaired”.
Creation of each domain score was as follows:

1. Complex working memory: A domain score for complex working

memory was created by averaging the z-scores for Digit Span Back-
wards and the PASAT total score.
2. Episodic memory: This cognitive domain score was created by averaging
the z-scores on Logical Memory I and II (WAIS-R; Wechsler, 1987).
3. Executive functioning:This cognitive domain score was created by aver-
aging z-scores, for the Stroop Color-Word Test, Trails B, and COWAT.

There were no significant differences in the distribution of TBI and MS partici-

pants in the impaired or non-impaired groups for the domains of complex
working memory (x2 1.27, p . .05) and executive functioning (x2 ¼ 0.33,
p . .05; see Table 2). There were, however, significantly more TBI participants
in the impaired memory category (55.6% of persons with TBI demonstrated
 LEARNING AND MEMORY IN MS AND TBI 281

TABLE 2
Number of TBI and MS participants in impaired and not impaired groups across
cognitive domains

Working memory Episodic memory Executive functioning

Impaired Not impaired Impaired Not impaired Impaired Not impaired

TBI 11 5 10 8 2 16
MS 16 15 2 29 2 29

memory impairment) compared to MS participants (6.5% of persons with MS

demonstrated memory impairment), as may be expected (x2 ¼ 14.85, p , .05)
(see Table 2).
Data analysis
A 2  2  3 mixed design ANOVA with diagnostic group (MS versus TBI) as
the between group factor and time (immediate recall, 30-minute delayed recall
and 1-week delayed recall) and condition (generated versus provided) as the
within group factors, was conducted with the number of words correctly recalled
as the dependent variable to examine differences of the generation effect
between diagnostic groups. To investigate the impact of cognitive impairment
on the ability to benefit from self-generation, a 2  2 mixed design ANOVA
was conducted with cognitive functioning (not impaired versus impaired) as
the between group factor and condition (generated versus provided) as the
within group factor, and the number of words recalled correctly as the dependent
variable. This second model was run separately for each cognitive domain
(complex working memory, episodic memory, and executive functioning).
To examine if cognitive impairment in one domain versus cognitive
impairment in multiple domains (a measure of severity of cognitive impair-
ment) impacts the ability to benefit from self-generation, a 3  2 mixed
design ANOVA was performed with number of cognitively impaired
domains as the between group factor (no cognitive impairment, one impaired
cognitive domain, two or more impaired cognitive domains) and condition
(generated versus provided) as the within group factor. Number of words
recalled was the dependent variable.

RESULTS

The impact of diagnosis and time on the generation effect

An analysis to compare differences in the ability of individuals with MS and
TBI to benefit from self-generation across time demonstrated a significant
282 O’BRIEN ET AL.

beneficial effect of self-generation for both groups, representing a large effect

size, as measured by partial eta-squared (Green and Salkind, 2003) [main
effect of condition: F(1, 47) ¼ 15.62, p , .001, h2p ¼ .25; see Figure 1].
The MS group demonstrated a 49% increase in recall from the provided
(M ¼ 1.84, SD ¼ 1.44) to generated condition (M ¼ 2.75, SD ¼ 1.68) and
the TBI group demonstrated a 43% increase in recall from the provided
(M ¼ 0.66, SD ¼ 0.92) to the self-generated (M ¼ 1.15, SD ¼ 1.25) con-
dition. However, although both groups benefited from self-generation, they
did not differ significantly in this benefit [group  condition interaction,
F(1, 47) ¼ 1.50, p . .05]. There was a significant interaction between time
and condition [time  condition interaction: F(2, 47) ¼ 8.07, p , .01 (see
Figure 2)], such that there was a significant advantage for recall of self-
generated words during immediate and 30-minute delayed recalls, but not
at the one week delayed recall across groups.

Cognitive impairment and the ability to benefit from the

generation effect
Cognitively impaired versus not impaired participants in each cognitive
domain were compared in order to investigate any differential benefit of self-
generation based on a specific type of cognitive impairment. As the initial
analysis of the generation effect (reported above) indicated that no benefit of
self-generation was present at the 1-week delayed recall, this time point was
eliminated from further analyses. (See Table 3 for a summary of recall in
generated and provided conditions across cognitive domains.)
Participants who were not impaired in the domain of complex working
memory demonstrated a trend towards greater benefit from self-generation

Figure 1. Mean number of words recalled in the generated and provided conditions for the MS and
TBI groups.
 LEARNING AND MEMORY IN MS AND TBI 283

Figure 2. Interaction of time and condition on recall of words.

than those with an impairment of complex working memory, condition x

impairment interaction: F(1, 45) ¼ 2.91, p ¼ .08 (see Figure 3). Calculations
of effect size indicate that those with no impairment in complex working
memory demonstrated a large effect size (Cohen’s d ¼ .90) and those with
an impairment in complex working memory demonstrated a medium effect
size (Cohen’s d ¼ .48) when comparing the difference in provided versus
generated words recalled (See Table 3).
There was no significant interaction between condition (generation versus
provided) and episodic memory impairment, meaning participants with and
without an impairment in episodic memory did not differ to a significant
degree in their benefit from self-generation [condition  impairment inter-
action: F(1, 47) ¼ 0.32, p . .05]. Indeed, calculations of effect size indicate
that both those with and without an impairment in episodic memory demon-
strated a large effect size with regards to self-generation (Cohen’s d ¼ .82 and
.75, respectively). There was no significant interaction between condition and
executive functioning impairment, meaning participants with or without
impairments in executive functioning did not differ significantly in their
benefit from self-generation [condition  impairment interaction: F(1,
47) ¼ 0.03, p . .05 (see Table 3)]. Both groups with and without impairments

TABLE 3
Summary of mean number of words recalled in generated and provided conditions
across cognitive domains for non-impaired and impaired persons

Working memory Episodic memory Executive functioning

Number of Not Not Not

words recalled impaired Impaired impaired Impaired impaired Impaired

Provided M (SD) 2.20 (1.51) 1.38 (1.52) 2.15 (1.54) 0.42 (0.65) 1.78 (1.56) 1.13 (1.65)
Generated M (SD) 3.65 (1.98) 2.10 (1.49) 3.33 (1.85) 1.09 (1.02) 2.86 (1.91) 1.88 (2.27)
284 O’BRIEN ET AL.

Figure 3. The Interaction of complex working memory impairment and generated or provided
condition.

in executive functioning demonstrated a medium effect size of self-generation

(Cohen’s d ¼ .38 and .48, respectively).

Multiple impairments
To determine the impact of more diffuse impairments, an analysis was con-
ducted comparing participants with no cognitive impairments, those with a
cognitive impairment in one domain and those with cognitive impairments
in multiple domains (i.e., two or three; see Figure 4). Those with impairments
in two or more cognitive domains recalled significantly fewer words overall
than those with no impairments and those with only one cognitively impaired
domain [main effect of number of cognitively impaired domains: F(2,
44) ¼ 10.05, p , .01]. As expected, there was a main effect of condition
[generated vs. provided: F(1, 44) ¼ 18.45, p , .001]. The interaction
between condition and number of cognitively impaired domains was not stat-
istically significant, F(2, 44) ¼ 1.49, p ¼ .24. For those with no cognitive
impairments, the effect size in the difference between provided and generated
words was large (Cohen’s d ¼ .98). For those with one cognitive impairment,
the effect size was medium (Cohen’s d ¼ .47). Finally, although overall recall
of words was low for those with two or more cognitively impaired domains,
this group demonstrated a large effect size in the difference between provided
and generated words recalled (Cohen’s d ¼ .81).

DISCUSSION
The purpose of the current study was to elucidate the role of clinical group
membership as well as presence and degree of cognitive impairment on one’s
ability to benefit from self-generation in subsequent memory performance. In
 LEARNING AND MEMORY IN MS AND TBI 285

Figure 4. Words recalled by number of cognitively impaired domains.

a direct comparison of individuals with TBI and MS, both groups benefited from
self-generation of information, and the magnitude of benefit did not differ sig-
nificantly between the groups. More specifically, despite the fact that overall
recall was greater in the MS versus TBI group, both groups benefited signifi-
cantly from self-generation. However, across groups, although the benefit of
using self-generation to improve recall was observed immediately and 30
minutes after learning, it was not sustained at 1 week, (see below).
Previous research has demonstrated that persons with MS and TBI have
particular difficulty in learning new information, rather than in retrieval
from long-term storage (DeLuca et al., 1994, 1998, 2000). As such, interven-
tions geared toward improving the acquisition of information may prove par-
ticularly useful in such populations. Specifically, the generation effect
appears to be beneficial in that it requires people to participate more actively
in the learning process and encode information in a more active, distinct, and
in-depth manner than when information is simply provided (e.g., Barrett
et al., 2000; Begg, Snider, Foley, & Goddard, 1989; Hunt & McDaniel,
1993; Steffens & Erdfelder, 1998).
Given the known heterogeneity in persons with both TBI and MS, one’s
style of learning may impact the overall benefit from self-generation of
information. For example, in persons with TBI, four different types of “learn-
ing style” have been observed: active, disorganised, passive, and deficient
(Deshpande et al., 1996) Each of these styles reflects strengths or weaknesses
in various aspects of the learning and memory process. Researchers assert that
the process by which information is learned interacts with one’s learning
style, significantly impacting acquisition and therefore later recall. Therefore,
286 O’BRIEN ET AL.

it is possible that individuals with a primary retrieval failure may benefit

little from active encoding strategies such as self-generation (e.g., Duchnick,
Vanderploeg, & Curtiss, 2002; Eustache et al., 1999; Pollmann, Haupt,
Romero, & Kurz, 1993). Future studies should be geared towards the inter-
action of learning style with the strategy used to improve learning, with an
eye towards tailoring cognitive interventions to the specific impairments of
individual patients.
Although significant gains in recall were seen immediately and at 30
minutes as a result of self-generation, these gains were not sustained at a
1-week delay. There are several potential explanations for this lack of a
sustained generation effect at 1 week. Perhaps the most salient explanation
concerns the relevancy of the to-be-learned material. It is well known
that recall is better for information that is more personally meaningful or
salient (e.g., Bower & Gilligan, 1979; Craik & Tulving, 1975; McMurray
& McIntyre, 1981; Schallert, 1976; Seamon & Murray, 1976; Westmacott
& Moscovitch, 2003). Because of the increased depth of processing for
relevant versus less relevant information, recall for relevant information is
sustained for a greater period of time (e.g., Craik & Tulving, 1975).
Because the present study simply required the recall of lists of words, the
lack of sustained generation effect may be more related to the lack of rele-
vancy in learning and recall of a lab-based, word-list task. This hypothesis
would suggest that increased relevancy of the to-be-learned information
would result in a more sustained generation effect. Preliminary work in our
laboratory using more relevant and more functionally based tasks support
this contention (Goverover, Chiaravalloti, Johnston, & DeLuca, 2005).
Specifically, on two functional tasks with inherent relevance (i.e., meal prep-
aration and financial management), the beneficial effects of self-generation in
persons with MS were indeed sustained at 1 week post-learning, and in some
tasks, recall at 1 week demonstrated the largest effect of self-generation.
Additionally, less relevant material may not receive as much time or effort
during the attentional processes, diminishing the strength of encoding (e.g.,
Challis, Velichkovsky, & Craik, 1996; Craik & Lockhart, 1972; Craik &
Tulving, 1975). Further, there is strong support in the literature that infor-
mation that is self-referent in some manner (e.g., personally relevant) to an
individual will result in greater elaboration and organisation of the material
during encoding and significantly greater recall compared to information
that does not have this same personal relevance (e.g., Klein & Kihlstrom,
1986; Klein & Loftus, 1988; Symons & Johnson, 1997).
A second and related potential explanation for the lack of a sustained gen-
eration effect in the current study comes from the literature on false memories
and is related to the distinctiveness of a stimulus (e.g., Brainerd, Reyna, &
Kneer, 1995; Howe, 1998; Hunt, 2003). This work suggests that more distinct
information receives more attention, is processed actively, and therefore
 LEARNING AND MEMORY IN MS AND TBI 287

results in a stronger and more sustainable memory trace. As distinctiveness

decreases, retroactive interference is thought to increase and it becomes
more challenging to access the non-distinct piece of information. Lab-
based cognitive rehabilitation research typically does not include target infor-
mation that is particularly distinctive (i.e., list of random words). Therefore,
application of self-generation with tasks that are more salient or distinct as
well as more personally relevant may potentially increase the potential
long-term benefit of self-generation in improving recall and recognition.
To examine the potential influence of impairment in targeted cognitive
areas known to be associated with MS and TBI, three different cognitive
domains (working memory, episodic memory, and executive functioning)
were analysed to specifically examine whether impairments in each of
these domains would differentially influence the potential benefits of self-
generation. The results showed that both individuals with and without
impairment in each of the domains benefited to the same degree from self-
generation in improving memory performance. Similarly, in a different
analysis designed to investigate the impact of overall severity of cognitive
impairment (collapsed across domains), once again subjects benefited
equally from self-generation. Specifically, when comparing persons with no
cognitive impairments, those impaired in one cognitive domain, and those
with multiple domains of cognitive impairment, there was no significant
interaction between the number of cognitive impairments and magnitude of
benefit from self-generation. Therefore, even though the group with multiple
cognitive impairments recalled significantly fewer words, subjects with and
without cognitive impairment benefited equally from self-generation.
In contrast to the current findings with TBI and MS subjects, the dementia
literature suggests that as cognitive impairments become more severe, self-
generation becomes less useful in improving learning (e.g., Barrett et al.,
2000; Dick & Kean, 1989; Souliez et al., 1996). One explanation for this
difference across populations is that there is a significant decline in semantic
processes observed in severe dementia (Mitchell et al., 1986), and that com-
promised semantic processes are detrimental to the benefits of self-generation
(Barrett et al., 2000). In contrast, semantic processing is generally typically
preserved in both MS (Andrade et al., 2003; Arnett et al., 1997; Kenealy,
Beaumont, Lintern, & Murrell, 2002; Lokken et al., 1999; Rao et al., 1993;
Thornton, Raz, & Tucker, 2002) and TBI (Chobor & Schweiger, 1998;
Glisky & Delaney, 1996; Hough, Pierce, Difilippo, & Pabst, 1997; Rosenthal
& Ricker, 2000; Vanderploeg et al., 2001), which likely explains the relative
preservation of the generation effect in these two neurological conditions.
Another potential explanation for why those with multiple cognitive impair-
ments were still able to benefit significantly from self-generation in the
present study is that the structure and more active encoding process required
in self-generation may be particularly helpful for people with somewhat more
288 O’BRIEN ET AL.

diffuse cognitive problems. Recent research in MS lends support to the idea

that some people with greater cognitive difficulties may benefit more than
those with mild difficulties by utilising specific cognitive strategies to
improve learning and memory (Chiaravalloti et al., 2005).
The current study contributes to a small but growing body of literature that
supports self-generation as a valuable technique to improve information
acquisition, hence resulting in improved recall and recognition in persons
with brain damage. The current results also suggest that persons with cogni-
tive involvement across several domains may show particular benefit from
self-generation training techniques. Despite these positive results, the
present study has limitations which must be kept in mind. First, in the
current study, severity was defined as having impairments in multiple cogni-
tive domains. It is of course conceivable that patients may have mild impair-
ments across several domains or have very severe impairment in a targeted
area (e.g., amnestic syndrome). This methodology was utilised to assess the
impact of severity because there were few severely impaired individuals in
the current sample limiting the range of potential cognitive impairment.
Therefore, a proxy measure of severity was utilised for this study. Future
studies using other methodologies to define severity of cognitive impairment
and including a broader range of severity of impairment are needed to test
the magnitude and generalisability of the current findings. Second, partici-
pants in the current study were not specifically selected for impairments in
the acquisition of information as has been done in at least one previous
study (Chiaravalloti et al., 2005). It is possible that including only subjects
with impairments targeted to difficulties in information acquisition may
alter the pattern of results observed in the current study. Lastly, although par-
ticipants with multiple domains of cognitive impairment did demonstrate a
large effect size in benefiting from self-generation, the absolute number of
words recalled by this group was still low overall. Similarly, in the TBI
group, recall of words in both the provided and generated conditions was
low (even though this group did demonstrate a significant benefit from
self-generation). With performance at such low levels, it is possible that
the statistical ability to detect group differences was limited and therefore
replication of this work is needed. Additionally, the functional impact of
the increase in recall seen with the use of self-generation should be the
focus of future research. The challenge for cognitive rehabilitation research
designed to examine techniques that improve specific cognitive constructs
(e.g., learning and memory) is not only in demonstrating efficacy, but in appli-
cability. Therefore, future studies must be geared toward harnessing the
power of techniques such as self-generation and finding ways to incorporate
these techniques effectively in everyday clinical practice to promote greater
independence and improved functioning in various activities of daily living
for persons with TBI, MS, and other neurological disorders.
 LEARNING AND MEMORY IN MS AND TBI 289

REFERENCES

Andrade, V. M., Oliveira, M. G., Miranda, M. C., Oliveira, A. S., Oliveira, E. M., & Bueno, O. F.
(2003). Semantic relations and repetition of items enhance the free recall of words by mul-
tiple sclerosis patients. Journal of Clinical and Experimental Neuropsychology, 25(8),
1070–1078.
Arnett, P. A., Rao, S. M., Grafman, J., Bernardin, L., Luchetta, T., Binder, J. R., & Lobeck, L.
(1997). Executive functions in multiple sclerosis: An analysis of temporal ordering, seman-
tic encoding, and planning abilities. Neuropsychology, 11(4), 535 –544.
Barrett, A. M., Crucian, G. P., Schwartz, R. L., & Heilman, K. M. (2000). Testing memory
for self-generated items in dementia: Method makes a difference. Neurology, 54(6),
1258 – 1264.
Begg, I., Snider, A., Foley, F., & Goddard, R. (1989). The generation effect is no artifact:
Generation makes words distinctive. Journal of Experimental Psychology: Learning,
Memory and Cognition, 15, 977 –989.
Begg, I., Vinski, E., Frankovich, L., & Holgate, B. (1991). Generating makes words memorable,
but so does effective reading. Memory and Cognition, 19(5), 487 – 497.
Bower, G., & Gilligan, S. (1979). Remembering information related to one’s self. Journal of
Research in Personality, 13(4), 420 –432.
Brainerd, C. J., Reyna, V. F., & Kneer, R. (1995). False recognition reversal: When similarity is
distinctive. Journal of Memory and Language, 34, 157 –185.
Brittain, J. L., La Marche, J. A., Reeder, K. P., Roth, D. L., & Boll, T. J. (1991). Effects of age
and IQ on Paced Auditory Serial Addition Task (PASAT) performance. Clinical Neuro-
psychologist, 5, 163 –175.
Challis, B. H., Velichkovsky, B. M., & Craik, F. I. M. (1996). Levels-of-processing effects on a
variety of memory tasks: New findings and theoretical implications. Consciousness and
Cognition, 5(1/2), 142 – 164.
Chiaravalloti, N. D., & Deluca, J. (2002). Self-generation as a means of maximizing learning in
multiple sclerosis: An application of the generation effect. Archives of Physical Medicine
and Rehabilitation, 83(8), 1070 – 1079.
Chiaravalloti, N. D., DeLuca, J., Moore, N. B., & Ricker, J. H. (2005). Treating learning impair-
ments improves memory performance in multiple sclerosis: A randomized clinical trial.
Multiple Sclerosis, 11(1), 58– 68.
Chobor, K. L., & Schweiger, A. (1998). Processing of lexical ambiguity in patients with
traumatic brain injury. Journal of Neurolinguistics, 11(1/2), 119 – 136.
Clark, S. E. (1995). The generation effect and the modeling of associations in memory. Memory
and Cognition, 23(4), 442 – 455.
Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory
research. Journal of Verbal Learning and Verbal Behavior, 11, 671 – 684.
Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic
memory. Journal of Experimental Psychology-General, 104, 268 – 294.
DeLuca, J., Barbieri-Berger, S., & Johnson, S. K. (1994). The nature of memory impairments
in multiple sclerosis: Acquisition versus retrieval. Journal of Clinical and Experimental
Neuropsychology, 16(2), 183 –189.
DeLuca, J., Gaudino, E. A., Diamond, B. J., Christodoulou, C., & Engel, R. A. (1998).
Acquisition and storage deficits in multiple sclerosis. Journal of Clinical and Experimental
Neuropsychology, 20(3), 376 –390.
DeLuca, J., Schultheis, M. T., Madigan, N. K., Christodoulou, C., & Averill, A. (2000).
Acquisition versus retrieval deficits in traumatic brain injury: Implications for memory
rehabilitation. Archives of Physical Medicine and Rehabilitation, 81(10), 1327 – 1333.
290 O’BRIEN ET AL.

Deshpande, S. A., Millis, S. R., Reeder, K. P., Fuerst, D., & Ricker, J. H. (1996). Verbal learning
subtypes in traumatic brain injury: A replication. Journal of Clinical and Experimental
Neuropsychology, 18(6), 836 – 842.
Dick, M. B., & Kean, M. L. (1989). Memory for internally generated words in Alzheimer-
type dementia: Breakdown in encoding and semantic memory. Brain and Cognition, 9(1),
88– 108.
Duchnick, J. J., Vanderploeg, R. D., & Curtiss, G. (2002). Identifying retrieval problems using
the California Verbal Learning Test. Journal of Clinical and Experimental Neuropsychol-
ogy, 24(6), 840 – 851.
Eustache, F., Desgranges, B., Laville, P., Guillery, B., Lalevee, C., Schaeffer, S., de la Sayette, V.,
Iglesias, S., Baron, J. C., & Viader, F. (1999). Episodic memory in transient global amnesia:
Encoding, storage, or retrieval deficit? Journal of Neurology, Neurosurgery and Psychiatry,
66(2), 148–154.
Gardiner, J. M., & Rowley, J. M. (1984). The generation effect with numbers rather than words.
Memory and Cognition, 12(5), 443 – 445.
Gaudino, E. A., Chiaravalloti, N. D., DeLuca, J., & Diamond, B. J. (2001). A comparison of
memory performance in relapsing-remitting, primary progressive and secondary progress-
ive, multiple sclerosis. Neuropsychiatry, Neuropsychology and Behavioral Neurology,
14(1), 32 –44.
Glisky, E. L., & Delaney, S. M. (1996). Implicit memory and new semantic learning in posttrau-
matic amnesia. Journal of Head Trauma Rehabilitation, 11(2), 31 –42.
Golden, C. J. (1978). Stroop Color and Word Test. Chicago: Stoelting.
Goverover, Y., Chiaravalloti, N., Johnston, M., & DeLuca, J. (2005). Self-generation to improv-
ing learning of everyday functional tasks in multiple sclerosis and traumatic brain injury.
Journal of the International Neuropsychological Society, 11 (Suppl. 1), 49.
Green, S. B. & Salkind, N. J. (2003). Using SPSS for Windows and Macintosh: Analyzing and
understanding data (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Hodges, J. R. (1994). Exploring disorders of semantic memory (pp. 77– 94). New York: Plenum
Press.
Hough, M. S., Pierce, R. S., Difilippo, M., & Pabst, M. J. (1997). Access and organization of
goal-derived categories after traumatic brain injury. Brain Injury, 11(11), 801 –814.
Howe, M. L. (1998). When distinctiveness fails, false memories prevail. Journal of Experimen-
tal Child Psychology, 71, 170 –177.
Hunt, R. R. (2003). Two contributions of distinctive processing to accurate memories. Journal
of Language and Memory, 48, 811 – 825.
Hunt, R. R., & McDaniel, M. A. (1993). The enigma of organization and distinctiveness.
Journal of Memory and Language, 32, 421 – 445.
Kenealy, P. M., Beaumont, J. G., Lintern, T. C., & Murrell, R. C. (2002). Autobiographical
memory in advanced multiple sclerosis: Assessment of episodic and personal semantic
memory across three time spans. Journal of the International Neuropsychological
Society, 8(6), 855 – 860.
Klein, S. B., & Kilhstrom, J. F. (1986). Elaboration, organization, and the self-reference effect
in memory. Journal of Experimental Psychology: General, 115, 26– 38.
Klein, S. B., & Loftus, J. (1988). The nature of self-referent encoding: The contribution of elabora-
tive and organizational processes. Journal of Personality and Social Psychology, 55, 5–11.
Lengenfelder, J., Chiaravalloti, N. D., & DeLuca J. (2002). Examining the generation effect in
individuals with traumatic brain injury. Journal of the International Neuropsychological
Society, 8, 173.
Lengenfelder, J., Chiaravalloti, N. D., & DeLuca, J. (2003). The generation effect: Comparisons
between MS and TBI to improve new learning. Journal of the International Neuropsycho-
logical Society, 9, 320.
 LEARNING AND MEMORY IN MS AND TBI 291

Lezak, M.D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological assessment

(4th ed.). New York: Oxford University Press.
Lipinska, B., Backman, L., Mantyla, T., & Viitanen, M. (1994). Effectiveness of self-generated
cues in early Alzheimer’s disease. Journal of Experimental and Clinical Neuropsychology,
16(6), 809 – 819.
Lokken, K., Ferraro, F. R., Petros, T., Bergloff, P., Thompson, S., & Teetzen, M. (1999). The
effect of importance level, delay, and rate of forgetting on prose recall in multiple sclerosis.
Applied Neuropsychology, 6(3), 147 –153.
McElroy, L. A., & Slamecka N. J. (1982). Memorial consequences of generating nonwords:
Implications for semantic-memory interpretations of the generation effect. Journal of
Verbal Learning and Verbal Behavior, 21, 249 – 259.
McKinlay, W., & Watkiss, A. J. (1999). Cognitive and behavioral effect of brain injury. Reha-
bilitation of the adult and child with traumatic brain injury (3rd ed., pp. 74– 86). Philadel-
phia: FA Davis.
McMurray, D., & McIntyre, J. S. (1981). Depth of processing: Is there a semantic continuum?
Australian Journal of Psychology, 33(2), 197 –213.
Mitchell, D. B., Hunt, R. R., & Schmitt, F. A. (1986). The generation effect and reality moni-
toring: Evidence from dementia and normal aging. Journal of Gerontology, 41(1), 79– 84.
Multhaup, K. S., & Balota, D. A. (1997). Generation effects and source memory in healthy
older adults and in adults with dementia of the Alzheimer type. Neuropsychology,
11(3), 382 – 391.
Olver, J. H., Ponsford, J. L., & Curran, C. A. (1996). Outcome following traumatic brain injury:
A comparison between 2 and 5 years after injury. Brain Injury, 10(11), 841 –848.
Payne, D. G., Neely, J. H., & Burns, D. J. (1986). The generation effect: Further tests of the
lexical activation hypothesis. Memory and Cognition, 14(3), 246 – 252.
Pesta, B. J., Sanders, R. E., & Murphy, M. D. (1999). A beautiful day in the neighborhood: What
factors determine the generation effect for simple multiplication problems? Memory and
Cognition, 27(1), 106 – 115.
Pollmann, S., Haupt, M., Romero, B., & Kurz, A. (1993). Is impaired recall in dementia of
the Alzheimer’s type a consequence of a contextual retrieval deficit? Dementia, 4(2),
102 – 108.
Poser, C. M., Paty, D. W., Schinberg, L., McDonald, W. I., Davis, F. A., Ebers, G., et al. (1983).
New diagnostic criteria for multiple sclerosis. Annals of Neurology, 13, 227 –231.
Rao, S. M., Grafman, J., DiGiulio, D., Mittenberg, W., Bernardin, L., Leo, G. J., et al. (1993).
Memory dysfunction in multiple sclerosis: Its relation to working memory, semantic encod-
ing, and implicit learning. Neuropsychology, 7(3), 364 –374.
Ricker, J. H., & Axelrod, B. N. (1994). Analysis of an oral paradigm of the Trail Making Test.
Assessment, 1, 47 –51.
Ricker, J. H., Axelrod, B. N., & Houtler, B. D. (1996). Clinical validation of the oral Trail
Making Test. Neuropsychiatry, Neuropsychology and Behavioral Neurology, 9, 50 –53.
Rosenthal, M., & Ricker, J. (2000). Traumatic brain injury. In R. Frank, & T. Elliott (Eds.),
Handbook of rehabilitation psychology (pp. 49 –74). Washington, DC: American
Psychological Association.
Schallert, D. L. (1976). Improving memory for prose: The relationship between depth of pro-
cessing and context. Journal of Verbal Learning and Verbal Behavior, 15(6), 621 – 632.
Seamon, J., & Murray, P. (1976). Depth of processing in recall and recognition memory: Differ-
ential effects of stimulus meaningfulness and serial position. Journal of Experimental
Psychology: Human Learning and Memory, 2(6), 680 – 687.
Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon.
Journal of Experimental Psychology: Human Learning and Memory, 4(6), 592 – 604.
292 O’BRIEN ET AL.

Souliez, L., Pasquier, F., Lebert, F., Leconte, P., & Petit, H. (1996). Generation effect in short
term verbal and visuospatial memory: Comparisons between dementia of the Alzheimer
type and dementia of the frontal lobe type. Cortex, 32, 347 – 356.
Spreen, O., & Strauss, E. (1991). A compendium of neuropsychological tests: Administration,
norms, and commentary. New York: Oxford University Press.
Spreen, O., & Strauss, E. (1998). A compendium of neuropsychological tests: Administration,
norms, and commentary (2nd ed.). New York: Oxford University Press.
Steffens, M. C., & Erdfelder, E. (1998). Determinants of positive and negative generation
effects in free recall. Quarterly Journal of Experimental Psychology, 51(4), 705 – 733.
Symons, C. S., & Johnson, B. T. (1997). The self-reference effect in memory: A meta-analysis.
Psychological Bulletin, 121, 371 –394.
Taconnat, L., & Isingrini, M. (2004). Cognitive operations in the generation effect on a recall
test: Role of aging and divided attention. Journal of Experimental Psychology, Learning,
Memory and Cognition, 30(4), 827 – 837.
Thornton, A. E., & Raz, N. (1997). Memory impairment in multiple sclerosis: A quantitative
review. Neuropsychology, 11(3), 357 –366.
Thornton, A. E., Raz, N., & Tucker, K. A. (2002). Memory in multiple sclerosis: Contextual
encoding deficits. Journal of the International Neuropsychological Society, 8(3), 395 – 409.
Vanderploeg, R. D., Crowell, T. A., & Curtiss, G. (2001). Verbal learning and memory deficits
in traumatic brain injury: Encoding, consolidation, and retrieval. Journal of Clinical and
Experimental Neuropsychology, 23(2), 185 – 195.
Wechsler, D. (1981). Wechsler Adult Intelligence Scale – Revised Manual. New York:
Harcourt Brace Jovanovich.
Wechsler, D. (1987). Wechsler Memory Scale – Revised. San Antonio, TX: The Psychological
Corporation.
Westmacott, R., & Moscovitch, M. (2003). The contribution of autobiographical significance to
semantic memory. Memory and Cognition, 31(5), 761 –774.
Wilkinson, G. S. (1993). Wide Range Achievement Test – 3, Administration Manual. Wilming-
ton, DE: Wide Range.

Manuscript received September 2005

Revised manuscript received March 2006