The Clinical Neuropsychologist

ISSN: 1385-4046 (Print) 1744-4144 (Online) Journal homepage: http://www.tandfonline.com/loi/ntcn20

Cross-cultural Adaptation, Reliability, and Validity

of the BICAMS in Brazil

Carina T. Spedo, Seth E. Frndak, Vanessa D. Marques, Maria P. Foss, Danilo

A. Pereira, Lucas de F. Carvalho, Carlos T. Guerreiro, Rodrigo M. Conde,
Tatiana Fusco, Ana J. Pereira, Silvana B. Gaino, Ricardo B. Garcia, Ralph H. B.
Benedict & Amilton A. Barreira

To cite this article: Carina T. Spedo, Seth E. Frndak, Vanessa D. Marques, Maria P. Foss, Danilo
A. Pereira, Lucas de F. Carvalho, Carlos T. Guerreiro, Rodrigo M. Conde, Tatiana Fusco, Ana
J. Pereira, Silvana B. Gaino, Ricardo B. Garcia, Ralph H. B. Benedict & Amilton A. Barreira
(2015): Cross-cultural Adaptation, Reliability, and Validity of the BICAMS in Brazil, The Clinical
Neuropsychologist, DOI: 10.1080/13854046.2015.1093173

To link to this article: http://dx.doi.org/10.1080/13854046.2015.1093173

Published online: 29 Oct 2015.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=ntcn20

Download by: [The Am Academy of Clinical Neuro ] Date: 04 November 2015, At: 05:53
 The Clinical Neuropsychologist, 2015
http://dx.doi.org/10.1080/13854046.2015.1093173

Cross-cultural Adaptation, Reliability, and Validity of the

BICAMS in Brazil

Carina T. Spedo1, Seth E. Frndak2, Vanessa D. Marques1,

Maria P. Foss3, Danilo A. Pereira4, Lucas de F. Carvalho5,
Carlos T. Guerreiro1, Rodrigo M. Conde1, Tatiana Fusco1,
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Ana J. Pereira1, Silvana B. Gaino6, Ricardo B. Garcia3,

Ralph H. B. Benedict2, and Amilton A. Barreira1
1
Department of Neuroscience and Behavior Sciences, Ribeirão Preto Medical School, University
of São Paulo (FMRP-USP), São Paulo, Brazil
2
Departments of Neurology and Psychiatry, School of Medicine and Biomedical Sciences,
University at Buffalo, State University of New York, Buffalo, NY, USA
3
Faculty of Philosophy, Languages and Literature, and Human Sciences (FFLCH) of the
University of São Paulo, São Paulo, Brazil
4
Brazilian Institute of Neuropsychology and Cognitive Sciences (IBNeuro), Federal District,
São Paulo, Brazil
5
Department of Psychology, University of São Francisco (USF), São Paulo, Brazil
6
Department of Psychology, Federal University of the Recôncavo Baiano (UFRB), Centro, Cruz
das Almas, Brazil

Objective: To investigate the reliability and validity of a Brazilian-Portuguese adaptation of the

Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Method: A Brazilian
sample of 58 multiple sclerosis (MS) patients and 58 healthy controls (HC) were administered the
Brazilian-Portuguese BICAMS test battery, comprising the Symbol Digit Modalities Test
(SDMT), California Verbal Learning Test Second Edition (CVLT2), and the Brief Visuospatial
Memory Test Revised (BVMTR). Mean differences between groups on BICAMS tests were
assessed using analysis of covariance (ANCOVA), controlling for age, gender, education, anxiety,
and depression. Test–retest data were obtained from 49 of the MS patients, two weeks after the
initial assessment. Results: The MS patient group scored significantly lower on all BICAMS tests
(CVLT2 F1,110 = 28.99, p < .001; BVMTR F1,110 = 7.77, p < .01; SDMT F1,110 = 21.09,
p < .001). Mixed-factor ANCOVAs tested differences in learning curves across trials for CVLT2
and BVMTR. HCs had significantly steeper learning curves on both CVLT2 (F1,111 = 10.82,
p < .01) and BVMTR (F1,110 = 7.816, p < .01). These findings support diagnostic validity of the
Brazilian-Portuguese adaptation. Test–retest reliability was satisfactory for SDMT, CVLT2, and
BVMTR (.86, .84, and .77, respectively). Conclusion: The results suggest that this Brazilian ver-
sion of the BICAMS will be a valid and reliable measure once complete normative data become
available.

Keywords: BICAMS; Multiple Sclerosis; Cultural Adaptation; Cognitive Assessment; Neuropsychological

Testing.

Address correspondence to: Amilton A. Barreira, Department of Neuroscience and Behavior Sciences,
Ribeirão Preto Medical School, Av Bandeirantes 3900, CEP: 14049-900 Ribeirão Preto, São Paulo, Brazil.
E-mail: aabarrei@fmrp.com
(Received 16 March 2015; accepted 7 September 2015)

© 2015 Taylor & Francis

 2 CARINA T. SPEDO ET AL.

INTRODUCTION
Over 40% of multiple sclerosis (MS) patients suffer from cognitive dysfunction
(Benedict, Bruce et al., 2006; Chiaravalloti & DeLuca, 2008; Guimaraes & Sa, 2012;
Rao, Leo, Bernardin, & Unverzagt, 1991) with impairments most often manifesting in
decreased processing speed, poor learning, and memory and, less frequently, executive
dysfunction (Chiaravalloti & DeLuca, 2008; Guimaraes & Sa, 2012). Cognitively dis-
abled patients are more likely to be unemployed, and report fewer extracurricular and
social activities. (Kushwaha, Suri, Gupta, & Bala, 2009; Strober et al., 2012) Appropri-
ate test measures for identification of cognitive impairment (CI), and decline, are impor-
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

tant for the clinical management of this disease. CI is not a diagnostic criterion and is
therefore not typically included in the diagnostic work-up for the disease. Ideally, all
MS patients would be routinely evaluated and/or monitored for CI, with similar mea-
sures being employed across specialty care centers.
Clinical evaluation of cognition can be costly and time-consuming, with standards
for cognitive assessment varying between providers. The Brief International Cognitive
Assessment for Multiple Sclerosis (BICAMS) is a collection of tests chosen by an inter-
national panel to standardize and facilitate routine monitoring of cognition in MS
patients (Langdon et al., 2012). Three neuropsychological tests, the California Verbal
Learning Test Second Edition (CVLT2), Brief Visuospatial Memory Test Revised
(BVMTR), and the Symbol Digit Modalities Test (SDMT), comprise the battery. As
only the learning trials of the CVLT2 and BVMTR are employed, BICAMS can usually
be administered in about 15 min. The overall aim of the BICAMS initiative was to
adapt neuropsychological testing to many languages and cultures (Benedict, Amato
et al., 2012), and validations in some non-English settings have been published
(Dusankova, Kalincik, Havrdova, & Benedict, 2012; Eshaghi et al., 2012; Goretti et al.,
2014). These data typically find that the BICAMS is reliable in a test–retest design, dif-
ferentiating well patients from healthy controls (HC), and relate to the external factors
such as employment status. The BICAMS tests were selected because prior studies in
the US had similarly supported their reliability and validity (Benedict, Cookfair et al.,
2006; Benedict et al., 2005).
In this study, we set out to examine the reliability and validity of the BICAMS
following translation into Portuguese, and utilization at an MS Center in Sao Paulo,
Brazil. After a painstaking process of translation, back translation, and analysis of word
frequency (described later), the BICAMS was administered to MS patients and HC
volunteers.

METHODS
Participants
We recruited 58 MS patients (MS) and 58 healthy control volunteers (HC), as
monetary compensation for behavioral research is not permitted in Brazil. Controls
were recruited from community sources affiliated with the MS Center considering the
mean age of MS patients (41.2 ± 12.2). The 58 HCs were selected out of a larger ongo-
ing sampling of Brazilian control subjects (n = 105) intended for future normative pur-
poses. These 58 HCs were selected to match the MS sample on demographic
 COGNITION MS IN BRAZIL 3

characteristics. MS patients attending annual assessments were randomly selected from

the Neuromuscular and Autoimmunity Diseases Clinic. All participants had no history
of neurological disease other than MS and no prior psychiatric illness other than depres-
sion emerging after the onset of MS. The MS diagnoses were based on the revised
McDonald criteria (Polman et al., 2011) and all patients had relapsing–remitting course
(Lublin et al., 2014). MS patients were free of relapses for at least 30 days prior to
study entry, and none were being treated with corticosteroids. All participants were
18 years or older, fluent in Portuguese, and provided informed consent to all proce-
dures. Those with history of substance or alcohol dependence were excluded from the
study. The Medical School of Ribeirão Preto Ethics Committee, University of São
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Paulo, approved all research procedures for this study.

Assessment procedures
During the clinical visit, MS patients underwent a neurological exam with results
quantified in accordance with the expanded disability status scale (EDSS) (Kurtzke,
1983) conducted by a board-certified neurologist. The neuropsychological exam was
performed by a neuropsychologist or supervised student trainee. The two sub-scales of
the Hospital Anxiety and Depression Scale (HADS) (Hermann, 1997) were also
administered as a marker of major depression and generalized anxiety disorder. Scores
of 0–7 indicate normal levels of anxiety and depression; 8–10 indicate borderline abnor-
mal anxiety, and depression levels 11–21 suggest abnormal levels of anxiety and
depression. The SDMT, CVLT2, and BVMTR were administered in the same testing
session, in accordance with commonly used, manual-based instructions, using the stan-
dard order (Benedict, Amato et al., 2012). In brief, the SDMT (Smith, 1982) presents a
key of numbers paired with symbols, followed by row of symbols to which the subject
provides the correct numbers as rapidly as possible. Following earlier validation
research (Rao et al., 1991; Strober et al., 2009) and consensus opinion papers (Benedict
et al., 2002), only the oral response version of the SDMT was used. The CVLT2 (Delis,
Kramer, Kaplan, & Ober, 2000) is a commonly used test of auditory/verbal learning
and memory, used successfully in many MS studies (Benedict, 2005; Chiaravalloti,
Moore, Nikelshpur, & DeLuca, 2013; Strober et al., 2009). The five learning trials are
summed to obtain a total CVLT2 score. The BVMTR (Benedict, Schretlen, Groninger,
Dobraski, & Shpritz, 1996) is a visual/spatial memory test wherein subjects view a
stimulus card with six figures for 10 s, and then render the figures from memory. There
are three learning trials, which are summed to produce the final BVMTR score. Scores
reflect memory for the figures as well as their location. As with the other BICAMS met-
rics, there is good evidence that the BVMTR is reliable and valid when used with MS
patients (Benedict, 2005; Strober et al., 2009). Only the learning trials of the CVLT2
and BVMTR were administered, in accordance with BICAMS opinion paper (Langdon
et al., 2012).
Reliability analysis utilized a sub-sample of 49 MS patients were tested two
weeks following the first-study visit, using the same forms of the CVLT2, BVMTR,
and SDMT. Nine patients did not come in for follow-up testing due to the extreme
traveling distance. These patients were similar to the rest of the sample on age
(41.0 ± 15.10), education (12.0 ± 3.46), and EDSS score (Median: 3.5; Range: 7.0).
 4 CARINA T. SPEDO ET AL.

Translation and cultural adaptation

We followed the published consensus opinion guidelines for new language valida-
tion of BICAMS (Benedict, Amato et al., 2012), taking into consideration steps recom-
mended for cultural adaptation of psychometric instruments (Beaton, Bombardier,
Guillemin, & Ferraz, 2000; Guillemin, Bombardier, & Beaton, 1993; Herdman,
Fox-Rushby, & Badia, 1998; Reichenheim & Moraes, 2007). While the translation of
patient instructions for the SDMT and BVMTR were straightforward, the CVLT2
required extensive work in order to accomplish a valid translation of the stimulus word
list. The administration procedures were translated and back-translated by one translator
and the first author.
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Translation of the CVLT2 stimuli involved three independent translators, fluent in

both English and Portuguese. Differences among the three translated versions of the
word list were resolved and this preliminary version was piloted in 15 MS patients and
10 healthy volunteers. We identified several problems with the task, including a ceiling
effect in controls. Further investigation was undertaken in accordance with the CVLT2
test manual (Delis et al., 2000). The word list was thereby modified to retain the integrity
of the semantic categories while increasing task difficulty. A sample of 323 healthy per-
sons between 18 and 65 years, from North, Southeast, and Southern regions of Brazil,
were interviewed. Each subject was given 36 categories (e.g., vegetables, animals, trans-
portation, and furniture) and asked to list as many words in each category as possible in
30 s. Answers were collected and the frequencies of words in each category were calcu-
lated. Next, the top four words in each category identified as exceptionally common,
were replaced with another word that was considered less common. Differences between
the original list, the first consensus version and the new CVLT2 lists were discussed and
a final consensus version was obtained. This revised CVLT2 version was administered
to 10 MS patients and 10 healthy volunteers in a second pilot, revealing a level of per-
formance more in keeping with published CVLT2 data (Stegen et al., 2009).

Statistical analysis
Between-group differences in demographic statistics were tested using t-test.
Test–retest reliability was assessed using Pearson’s intra-class correlation coefficient
with Bonferroni correction. Diagnostic validity was assessed by determining the statisti-
cal significance of each outcome on the MS/control comparison by the analysis of
covariance (ANCOVA), accounting for covariates significantly correlated with
BICAMS tests. Effect sizes (Cohen’s d) were calculated. Differences in learning rates
were analyzed using repeated measures ANOVA. Pearson’s correlation coefficients with
Bonferroni correction were used to investigate structural validity among the three
BICAMS tests for the MS patients, HCs, and total sample separately.

RESULTS
The group comparisons for demographic and BICAMS metrics are presented in
Table 1. MS patients were well matched to HCs on demographics. Median EDSS score
for MS was 4.25 (range: 0–7.5) and the average disease duration was roughly eight
 COGNITION MS IN BRAZIL 5

Table 1. Demographic and clinical data for MS patients and controls

Cross-sectional MS Patients (n = 58) Controls (n = 58) Mean t- p- variance Effect

sample Mean ± SD (Min. Max.) ± SD (Min. Max.) test value explained size

% Female 69% 55.2%

Age (yrs.) 41.2 ± 12.2 (18 to 65) 40.3 ± 11.9 (18 to 65) .40 .689 .002 .053
Education (yrs.) 12.7 ± 5.2 (3 to 28) 12.5 ± 3.6 (4 to 22) .25 .803 .001 .033
HADS-D 6.9 ± 3.4 (0 to 14) 5.6 ± 2.9 (0 to 14) 2.22 .028 .201 .411
HADS-A 7.7 ± 5.2 (0 to 21) 7.1 ± 3.0 (1 to 18) .72 .474 .070 .141
EDSS 4.2 ± 2 (0 to 7.5) – – – – –
Disease duration 8.3 ± 6.6 (.2 to 28) – – – – –
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

(yrs.)
SDMT 35.9 ± 16.1 (4 to 64) 47.5 ± 13 (16 to 73) 4.28 <.001 .274 .524
CVLT2 42.1 ± 12.4 (20 to 68) 53.4 ± 10.8 (29 to 74) 5.21 <.001 .381 .618
BVMTR 19.9 ± 8.6 (4 to 36) 23.8 ± 7.7 (5 to 36) 2.52 .013 .105 .327

Notes: Mean ± the standard deviation shown where possible; CVLT2 = California Verbal Learning Test
Second Edition; BVMTR = Brief Visuospatial Memory Test Revised; SDMT = Symbol Digit Modalities Test;
HADS-D = Hospital Anxiety and Depression Scale (Depression); HADS-A = Hospital Anxiety and Depres-
sion Scale (Anxiety); EDSS = Expanded Disability Status Scale.

years. MS patients reported significantly higher HADS-D scores, with a moderate effect
size, but were not different from HCs on HADS-A. As expected, the MS group per-
formed more poorly on each cognitive test with large effect sizes.
The test–retest reliability findings are presented in Table 2. This MS sub-sample
included 49 patients with an average age of 38.1 ± 10.2 years, and mostly a high school
education (13.3 ± 3.7 years). The reliability coefficients were good to excellent, .86 for
the SDMT, .84 for CVLT, and .77 for BVMTR, all p < .001.
Age was significantly negatively correlated with all BICAMS tests for MS
patients (CVLT2 r = −.30, p < .05; BVMTR r = −.29, p < .05; SDMT r = −.30,
p < .05) and on BVMTR and SDMT for the control group (BVMTR r = −.34, p < .01;
SDMT r = −.49, p < .01). Likewise, education was positively correlated with these out-
comes in MS patients (CVLT2 r = .18, p < .05; BVMTR r = .27, p < .05; SDMT
r = .29, p < .05) and on CVLT2 and SDMT for HCs (CVLT2 r = .37, p < .01; SDMT

Table 2. Test–retest (intra-rater intraclass) reliability (n = 49) fixed-raters

(n = 49) Mean ± SD (Min. Max.) ICC (95% C.I.) p-value Internal consistency

Age 38.1 ± 10.2 (24 to 65)

Education 13.3 ± 3.7 (4 to 26)
SDMT (test) 50.0 ± 11.3 (19 to 73) .86 (.77 to .92) p < .001 Excellent
SDMT (retest) 53.2 ± 11.4 (21 to 75)
CVLT-2 (test) 56.2 ± 9.0 (33 to 74) .84 (.73 to .91) p < .001 Excellent
CVLT-2 (retest) 61.1 ± 9.2 (39 to 78)
BVMT-R (test) 24.7 ± 7.2 (7 to 36) .77 (.73 to .91) p < .001 Good
BVMT-R (retest) 29.7 ± 6.3 (8 to 36)

Notes: ICC = intraclass correlation; CVLT2 = California Verbal Learning Test Second Edition;
BVMTR = Brief Visuospatial Memory Test Revised; SDMT = Symbol Digit Modalities Test.
 6 CARINA T. SPEDO ET AL.

r = .49, p < .01). Controlling for age, education, HADS-D, and HADS-A scores in
ANCOVA did not eliminate the significance of the group effect on any of the BICAMS
tests (CVLT2 F1,110 = 28.99, p < .001; BVMTR F1,110 = 7.77, p < .01; SDMT
F1,110 = 21.09, p < .001).
Mixed-factor ANCOVAs were used to model the learning curves of MS patients
and HCs on CVLT2 and BVMTR (Figures 1 and 2). Covariates of age, education,
HADS-D, and HADS-A were included in the model. On the CVLT2, HCs performed
significantly better than MS patients across trials (F1,110 = 29.03, p < .001), with HCs
having a significantly steeper learning curve (F1,111 = 10.82, p < .01). Similar results
were obtained for the BVMTR, with both the group effect (F1,110 = 7.816, p < .01), and
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

the interaction effect of time and group (learning curve) demonstrating significance
(F1,111 = 7.747, p < .01).
Correlations between the BICAMS scores in the MS and HC groups are presented
in Table 3. The correlations were uniformly significant, and of medium strength,
suggesting an expected degree of shared variability but not overlapping constructs.

DISCUSSION
The purpose of the present paper was to adapt the recently proposed BICAMS
to the Brazilian culture and Portuguese language, and then to investigate its psycho-
metric properties. In Brazil, there is a scarcity of psychometric tools for routine mon-
itoring of cognition in MS. The BICAMS committee proposed that the tests could be
translated and validated in other countries (Benedict, Amato et al., 2012; Langdon
et al., 2012), and recognizing the need for such an instrument, embarked upon this
validation study.

Figure 1. Learning curves for the CVLT2. In each case, the MS group underperforms relative to HCs. The
group by time interaction is statistically significant for the CVLT2, where the slope of the learning curve is
flatter in the patient group.
 COGNITION MS IN BRAZIL 7
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Figure 2. Learning curves for the BVMTR. As in the CVLT2, the MS group underperforms relative to HCs,
with a significant group by time interaction.

Table 3. Correlations among the BICAMS scores

MS patients Healthy controls Total sample

SDMT CVLT2 SDMT CVLT2 SDMT CVLT2

CVLT2 .496* – .628* – .622* –

BVMTR .688* .590* .540* .542* .649* .598*

Notes: CVLT2 = California Verbal Learning Test Second Edition; BVMTR = Brief Visuospatial Memory
Test Revised; SDMT = Symbol Digit Modalities Test.
*p < .001.

The results show that the Brazilian BICAMS is reliable in MS, at least when the
same forms of the tests are administered. Of course, in the real setting, alternate forms
will be needed, but equivalent forms were not yet developed for the CVLT2 and we
chose not to differentially treat this outcome in the test–retest study design, repeating
forms for all three BICAMS measures. In clinical practice, MS patients, when stable,
are commonly seen on an annual or semi-annual basis, and alternate forms will be pre-
ferred. Soon, clinicians may apply some of these tests in the setting of a relapse, as
demonstrated in two recent papers (Benedict et al., 2014; Pardini et al., 2014). Equal,
alternate versions of the SDMT (Benedict, Smerbeck et al., 2012) and BVMTR
(Benedict et al., 1996) are available for MS monitoring and the stimuli can be used in
Brazil, as the tests are not language dependent. We are presently working to develop an
alternate word list for the CVLT2.
The validity of the Brazilian BICAMS was also supported in this study. All three
tests significantly differentiated the MS patients and HC, and the cognitive measures
were correlated with age, education, and each other in degrees consistent with prior
work in neuropsychology. Controlling for age, education, and mood disorder did not
erase the significance of the group effect. The effect sizes are generally in keeping with
 8 CARINA T. SPEDO ET AL.

prior publications although in this present study the BVMTR showed a slightly lower
effect than SDMT and CVLT2, whereas in US data both BVMTR and SDMT have
been more sensitive (Benedict, Cookfair et al., 2006; Strober et al., 2009). Whether or
not this has anything to do with psychometric testing in a different culture remains to
be seen. The slope of the CVLT2 learning curve is known to be flatter in MS than HC
(Stegen et al., 2009), and this same metric is affected by the behavioral memory train-
ing in MS (Chiaravalloti et al., 2013). The same finding was obtained here, in that the
CVLT2 group by trial interaction was significant, showing greater gain in HC. It is
noteworthy that the same effect was seen in the BVMTR data.
This Brazilian adaptation of BICAMS represents the fourth such effort coming to
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

publication. Previously, the BICAMS was translated into Persian and Czech versions,
with results paralleling those reported herein (Dusankova et al., 2012; Eshaghi et al.,
2012). In Italy (Goretti et al., 2014), the measure has been translated and normed, with
further MS research underway. In the future, it may be possible to follow the regres-
sion-based norms approach (Heaton, Avitable, Grant, & Matthews, 1999; Norman,
Evans, Miller, & Heaton, 2000; Testa, Winicki, Pearlson, Gordon, & Schretlen, 2009)
to enhance the study of cross-cultural differences in cognitive test performance. We
have applied this technique to a larger battery of tests in MS (Parmenter, Testa,
Schretlen, Weinstock-Guttman, & Benedict, 2010). With larger samples, it would be
possible to model BICAMS performance to include country or origin, language, and
other factors. Such work is underway by our group. Similar efforts could be important
to the future use of BICAMS in multinational studies—in this paper, we found that
CVLT2 performance was below the level of our US data (American: 57.01 ± 10.64 vs.
Brazilian: 48.80 ± 10.56) as well as for the SDMT (American: 50.24 ± 11.89 vs.
Brazilian: 46.28 ± 8.99).
While the BICAMS tests identified a statistically significant effect at the group
level, the effect sizes were medium, not large, such that the precision at the individ-
ual level is less than perfect. A more comprehensive battery approach will be needed
in cases of mild MS-associated CI. Of course, in the real setting, alternate forms will
be needed, but equivalent forms were not yet developed for the CVLT2, and we
chose not to differentially treat this outcome in the test–retest study design, repeating
forms for all three BICAMS measures. As a result, marked practice effects were
observed in this study, as has been shown elsewhere in English-speaking samples
when the same test forms are repeated (Benedict, 2005). In clinical practice, MS
patients, when stable, are commonly seen on an annual or semi-annual basis, and
alternate forms will be preferred. Our study is limited in sample size and one should
bear in mind that we did not investigate longitudinal changes in BICAMS or the cor-
respondence with meaningful outcomes such as employment. We suspect that the
predictive validity will be good, as was the case elsewhere (Morrow et al., 2010),
but more research is needed within Brazil, as is planned by our group. In addition,
we intend to recruit samples from other regions of Brazil, as this study was largely
conducted in and near São Paulo state.
These concerns notwithstanding, we have answered the call for cross-cultural
BICAMS validation. These preliminary data suggest that these tests are reliable and
valid in Brazil, suggesting that they may augment existing approaches to monitoring
the neurological status of MS patients.
 COGNITION MS IN BRAZIL 9

DISCLOSURE STATEMENT
Dr. Benedict receives royalties from Physiological Assessment Resources Inc.

REFERENCES

Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process
of cross-cultural adaptation of self-report measures. Spine, 25, 3186-3191.
Benedict, R. H. B. (2005). Effects of using same vs. alternate form memory tests in short-interval,
repeated assessment in multiple sclerosis. Journal of the International Neuropsychological
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Society, 11, 727–736.

Benedict, R. H. B., Amato, M. P., Boringa, J., Brochet, B., Foley, F., Fredrikson, S., … Langdon,
D. (2012). Brief international cognitive assessment for MS (BICAMS): International stan-
dards for validation. BMC Neurology, 12, 12–19. doi:10.1186/1471-2377-12-55
Benedict, R. H. B., Bruce, J. M., Dwyer, M. G., Abdelrahman, N., Hussein, S., Weinstock-
Guttman, B., … Zivadinov, R. (2006). Neocortical atrophy, third ventricular width, and
cognitive dysfunction in multiple sclerosis. Archives of Neurology, 63, 1301–1306.
Benedict, R. H. B., Cookfair, D., Gavett, R., Gunther, M., Munschauer, F., Garg, N., &
Weinstock-Guttman, B. (2006). Validity of the minimal assessment of cognitive function in
multiple sclerosis (MACFIMS). Journal of the International Neuropsychological Society, 12,
549–558.
Benedict, R. H. B., Fischer, J. S., Archibald, C. J., Arnett, P. A., Beatty, W. W., Bobholz, J., …
Munschauer, F. E. (2002). Minimal neuropsychological assessment of MS patients: A
consensus approach. The Clinical Neuropsychologist (Neuropsychology, Development and
Cognition: Section D), 16, 381–397.
Benedict, R. H. B., Morrow, S., Rodgers, J., Hojnacki, D., Bucello, M. A., Zivadinov, R., &
Weinstock-Guttman, B. (2014). Characterizing cognitive function during relapse in multiple
sclerosis. Multiple Sclerosis, 20, 1745–1752. doi:10.1177/1352458514533229
Benedict, R. H. B., Schretlen, D., Groninger, L., Dobraski, M., & Shpritz, B. (1996). Revision of
the brief visuospatial memory test: Studies of normal performance, reliability, and validity.
Psychological Assessment, 8, 145–153.
Benedict, R. H. B., Smerbeck, A., Parikh, R., Rodgers, J., Cadavid, D., & Erlanger, D. (2012).
Reliability and equivalence of alternate forms for the Symbol Digit Modalities Test: Implica-
tions for multiple sclerosis clinical trials. Multiple Sclerosis, 18, 1320–1325. doi:10.1177/
1352458511435717
Benedict, R. H. B., Wahlig, E., Bakshi, R., Fishman, I., Munschauer, F., Zivadinov, R., &
Weinstock-Guttman, B. (2005). Predicting quality of life in multiple sclerosis: Accounting
for physical disability, fatigue, cognition, mood disorder, personality, and behavior change.
Journal of the Neurological Sciences, 231, 29–34.
Chiaravalloti, N. D., & DeLuca, J. (2008). Cognitive impairment in multiple sclerosis. The Lancet
Neurology, 7, 1139–1151. doi:10.1016/S1474-4422(08)70259-X
Chiaravalloti, N. D., Moore, N. B., Nikelshpur, O. M., & DeLuca, J. (2013). An RCT to treat
learning impairment in multiple sclerosis: The MEMREHAB trial. Neurology, 81,
2066–2072. doi:10.1212/01.wnl.0000437295.97946.a8
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). Califorina verbal learning test –
2nd edition.. San Antonio, TX: The Psychological Corporation.
Dusankova, J. B., Kalincik, T., Havrdova, E., & Benedict, R. H. (2012). Cross cultural validation
of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS) and the
brief international cognitive assessment for multiple sclerosis (BICAMS). The Clinical
Neuropsychologist, 26, 1186–1200. doi:10.1080/13854046.2012.725101
 10 CARINA T. SPEDO ET AL.

Eshaghi, A., Riyahi-Alam, S., Roostaei, T., Haeri, G., Aghsaei, A., Aidi, M. R., … Sahraian, M.
A. (2012). Validity and reliability of a Persian translation of the minimal assessment of
cognitive function in multiple sclerosis (MACFIMS). The Clinical Neuropsychologist, 26,
975–984. doi:10.1080/13854046.2012.694912
Goretti, B., Niccolai, C., Hakiki, B., Sturchio, A., Falautano, M., Eleonora, M., … Amato, M.
(2014). The brief international cognitive assessment for multiple sclerosis (BICAMS):
Normative values with gender, age and education corrections in the Italian population. BMC
Neurology, 14, 1585. doi:10.1186/s12883-014-0171-6
Guillemin, F., Bombardier, C., & Beaton, D. (1993). Cross-cultural adaptation of health-related
quality of life measures: Literature review and proposed guidelines. Journal of Clinical
Epidemiology, 46, 1417–1432. doi:10.1016/0895-4356(93)90142-N
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Guimaraes, J., & Sa, M. J. (2012). Cognitive dysfunction in multiple sclerosis. Front Neurol, 3
(74), 1–7. doi:10.3389/fneur.2012.00074
Heaton, R. K., Avitable, N., Grant, I., & Matthews, C. G. (1999). Further crossvalidation of
regression-based neuropsychological norms with an update for the Boston naming test.
Journal of Clinical and Experimental Neuropsychology (Neuropsychology, Development and
Cognition: Section A), 21, 572–582. doi:10.1076/jcen.21.4.572.882
Herdman, M., Fox-Rushby, J., & Badia, X. (1998). A model of equivalence in the cultural
adaptation of HRQoL instruments: The universalist approach. Quality of Life Research, 7,
323–335. doi:10.1023/A:1024985930536
Hermann, C. (1997). International experiences with the hospital anxiety and depression scale – a
review of validation data and clinical results. Journal of Psychosomatic Research, 42,
17–41.
Kurtzke, J. F. (1983). Rating neurologic impairment in multiple sclerosis: An expanded disability
status scale (EDSS). Annals of Neurology, 13, 227–231.
Kushwaha, S., Suri, P., Gupta, M., & Bala, K. (2009). PO10-TU-64 Study of cognition and
behavioural dysfunction in multiple sclerosis and correlation with MRI brain. Journal of the
Neurological Sciences, 285, Supplement 1, S213. doi:10.1016/S0022-510X(09)70813-5
Langdon, D. W., Amato, M. P., Boringa, J., Brochet, B., Foley, F., Fredrikson, S., … Benedict,
R. H. (2012). Recommendations for a brief international cognitive assessment for multiple
sclerosis (BICAMS). Multiple Sclerosis Journal, 18, 891–898. doi:10.1177/1352458511
431076
Lublin, F. D., Reingold, S. C., Cohen, J. A., Cutter, G. R., Sorensen, P. S., Thompson, A. J., …
Polman, C. H. (2014). Defining the clinical course of multiple sclerosis: The 2013 revisions.
Neurology, 83, 278–286. doi:10.1212/WNL.0000000000000560
Morrow, S. A., Drake, A., Zivadinov, R., Munschauer, F., Weinstock-Guttman, B., & Benedict,
R. H. (2010). Predicting loss of employment over three years in multiple sclerosis: Clinically
meaningful cognitive decline. The Clinical Neuropsychologist, 24, 1131–1145.
doi:926682619[pii]10.1080/13854046.2010.511272
Norman, M. A., Evans, J. D., Miller, W. S., & Heaton, R. K. (2000). Demographically corrected
norms for the California verbal learning test. Journal of Clinical and Experimental
Neuropsychology (Neuropsychology, Development and Cognition: Section A), 22, 80–94.
doi:10.1076/1380-3395(200002)22:1;1-8;FT080
Pardini, M., Uccelli, A., Grafman, J., Yaldizli, O., Mancardi, G., & Roccatagliata, L. (2014).
Isolated cognitive relapses in multiple sclerosis. Journal of Neurology Neurosurgery
Psychiatry, 85, 1035–1037. doi:10.1136/jnnp-2013-307275
Parmenter, B. A., Testa, S. M., Schretlen, D. J., Weinstock-Guttman, B., & Benedict, R. H. B.
(2010). The utility of regression-based norms in interpreting the minimal assessment of
cognitive function in multiple sclerosis (MACFIMS). Journal of the International Neuropsy-
chological Society, 16, 6–16. doi:S1355617709990750[pii]10.1017/S1355617709990750
 COGNITION MS IN BRAZIL 11

Polman, C. H., Reingold, S. C., Banwell, B., Clanet, M., Cohen, J. A., Filippi, M., … Wolinsky,
J. S. (2011). Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald crite-
ria. Annals of Neurology, 69, 292–302. doi:10.1002/ana.22366
Rao, S. M., Leo, G. J., Bernardin, L., & Unverzagt, F. (1991). Cognitive dysfunction in multiple
sclerosis. I. Frequency, patterns, and prediction. Neurology, 41, 685–691.
Reichenheim, M. E., & Moraes, C. L. (2007). Operacionalização de adaptação transcultural de
instrumentos de aferição usados em epidemiologia Operationalizing the cross-cultural adap-
tation of epidemological measurement instruments. Revista de Saúde Pública, 41, 665–673.
Smith, A. (1982). Symbol digit modalities test: Manual. Los Angeles, CA: Western Psychological
Services.
Stegen, S., Stepanov, I., Cookfair, D., Schwartz, E., Hojnacki, D., Weinstock-Guttman, B., &
Downloaded by [The Am Academy of Clinical Neuro ] at 05:53 04 November 2015

Benedict, R. H. (2009). Validity of the California verbal learning test-II in multiple sclerosis.
Clinical Neuropsychologist, 24(2), 1–14. doi:917239750[pii]10.1080/13854040903266910
Strober, L. B., Christodoulou, C., Benedict, R. H., Westervelt, H. J., Melville, P., Scherl, W. F.,
… Krupp, L. B. (2012). Unemployment in multiple sclerosis: The contribution of personality
and disease. Multiple Sclerosis Journal, 18, 647–653. doi:10.1177/1352458511426735
Strober, L., Englert, J., Munschauer, F., Weinstock-Guttman, B., Rao, S., & Benedict, R. H.
(2009). Sensitivity of conventional memory tests in multiple sclerosis: Comparing the Rao
Brief Repeatable Neuropsychological Battery and the Minimal Assessment of Cognitive
Function in MS. Multiple Sclerosis, 15, 1077–1084. doi:1352458509106615[pii]10.1177/
1352458509106615
Testa, S. M., Winicki, J. M., Pearlson, G. D., Gordon, B., & Schretlen, D. J. (2009). Accounting
for estimated IQ in neuropsychological test performance with regression-based techniques.
Journal of the International Neuropsychological Society, 15, 1012–1022. doi:10.1017/
S1355617709990713