Applied Neuropsychology: Adult

ISSN: 2327-9095 (Print) 2327-9109 (Online) Journal homepage: http://www.tandfonline.com/loi/hapn21

BICAMS in the Argentine population: Relationship

with clinical and sociodemographic variables

Sandra Vanotti, A. Smerbeck, María B. Eizaguirre, Maria L. Saladino, Ralph R.

H. Benedict & Fernando J. Caceres

To cite this article: Sandra Vanotti, A. Smerbeck, María B. Eizaguirre, Maria L. Saladino,
Ralph R. H. Benedict & Fernando J. Caceres (2017): BICAMS in the Argentine population:
Relationship with clinical and sociodemographic variables, Applied Neuropsychology: Adult, DOI:
10.1080/23279095.2017.1323751

To link to this article: http://dx.doi.org/10.1080/23279095.2017.1323751

Published online: 23 May 2017.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=hapn21

Download by: [The UC San Diego Library] Date: 24 May 2017, At: 14:09
APPLIED NEUROPSYCHOLOGY: ADULT
https://doi.org/10.1080/23279095.2017.1323751

none defined

BICAMS in the Argentine population: Relationship with clinical and

sociodemographic variables
Sandra Vanottia, A. Smerbeckb, María B. Eizaguirrea, Maria L. Saladinoa, Ralph R. H. Benedictc and
Fernando J. Caceresa
a
Multiple Sclerosis Clinic, INEBA – Neurosciences Institute of Buenos Aires, Buenos Aires, Argentina; bDepartment of Psychology, Rochester
Institute of Technology, Rochester, New York, USA; cSUNY at Buffalo School of Medicine, Neurology, Buffalo, New York, USA

ABSTRACT KEYWORDS
The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) was developed to BICAMS; cognition;
provide valid assessment of cognitive impairment in multiple sclerosis (MS). The relationship depression; employment
between clinical and social variables and cognitive disorders has been extensively studied, but status; fatigue; multiple
sclerosis
primarily in developed countries with a focus on other cognitive measures or batteries. The
objectives of this study were to analyze the relationship between the BICAMS data and key clinical
and sociodemographic variables in the Argentine MS population. A total of 50 MS patients were
administered the Argentinean BICAMS Battery, comprised of the Symbol Digit Modalities Test
(SDMT), the California Verbal Learning Test I (CVLT I), and the Brief Visuospatial Memory Test
Revised (BVMTR). Disease progression, fatigue, depression, self-reported and informant report
cognitive status, and employment status were assessed. Disease progression and employment
status were most strongly associated with overall BICAMS performance (�2 effect size values
ranging from .302 to .624, all comparisons statistically significant). Informant rating of patient
cognition, age, depression, disease duration, fatigue, and work hours were significantly associated
as well. Gender, years of education, and patient-reported cognitive status were nonpredictive.
A similar pattern was seen at the individual test level, with more variables related to the SDMT and
CVLT I than BVMTR. BICAMS is strongly associated with overall disease progression and
employment status.

Introduction assessment of cognitive impairment in MS that can be

used across a range of language and cultural groups. It
Cognitive impairment occurs in roughly 40–60% of
is comprised of the Symbol Digit Modalities Test
multiple sclerosis (MS) patients (Benedict et al., 2006).
(SDMT), the California Verbal Learning Test II (CVLT
In Argentina, a national multicenter survey called
II), and the Brief Visuospatial Memory Test Revised
RECONEM (National Survey of Cognitive Impairment
(BVMTR) to assess processing speed, verbal memory,
in Multiple Sclerosis in Spanish) found a cognitive
and visual memory respectively (Benedict et al., 2012).
impairment rate of 46% among Argentine MS patients,
International validations are currently underway. At
consistent with the literature (Caceres, Vanotti, Rao, &
the present time, normative data exists for a range of
Workgroup, 2011). The relationship between clinical
nations including the Czech Republic (Dusankova,
variables and cognitive impairment in MS has been
Kalincik, Havrdova, & Benedict, 2012), Italy (Goretti
extensively studied, but this research has particularly
et al., 2014), Hungary (Sandi et al., 2015), Ireland
focused on the two most widely administered batteries,
(O’Connell, Langdon, Tubridy, Hutchinson, &
the Brief Repeatable Battery of Neuropsychological tests
McGuigan, 2015), Brazil (Spedo et al., 2015), Canada
(BRB-N) (Rao, 1990) and the Minimal Assessment of
(Walker et al., 2016) and Argentina (Vanotti, Smerbeck,
Cognitive Function in MS (MACFIMS) (Benedict
Benedict, & Caceres, 2016). Similar studies are being
et al., 2006).
carried out in other countries.
Early detection of cognitive impairment is essential
In some reviews, cognitive impairment has been
to improve MS patients’ quality of life (Glanz et al.,
shown to be unrelated to the duration of the disease
2009). The Brief International Cognitive Assessment
(Chiaravalloti & DeLuca, 2008; Langdon, 2011; Patti,
for MS (BICAMS) initiative was developed by an expert
2009), but larger and longitudinal studies have shown
committee to provide practitioners with a quick, valid

CONTACT Sandra Vanotti svanotti@ineba.net Guardia Vieja 4435-C1192AAW, Ciudad Autonoma de Buenos Aires-Argentina.
© 2017 Taylor & Francis Group, LLC
2 S. VANOTTI ET AL.
significant associations with duration (Amato, Ponziani,
Siracusa, & Sorbi, 2001; Achiron et al., 2005). Similarly,
SDMT performance has been shown to deteriorate over
time across a 3-year longitudinal study (Amato et al.,
2010). In the validation of the MACFIMS and BICAMS
batteries, Dusankova et al. (2012) found that patients
with cognitive impairment had a longer disease dur-
ation. Some authors have found an association between
the degree of physical disability and the presence of cog-
nitive impairment in MS patients (Lynch, Parmenter, &
Denney, 2005), In particular, researchers have found the
SDMT to be related to the Expanded Disability Status
Scale (EDSS) (Kurtzke, 1983), which measures overall
disease progression, but with an emphasis on sensori-
motor function (Caceres et al., 2011; Niino et al.,
2014). Other authors note that a small but significant
proportion of MS patients appear to have cognitive
impairment in the absence of physical disability (Amato
et al., 2010). Greater variation has been found in the
relationship between fatigue and cognitive decline, with
some researchers failing to find a meaningful relation-
ship (Morrow et al., 2009) (Jougleux-Vie et al., 2014),
while others report a significant association (Andreasen,
Spliid, Andersen, & Jakobsen, 2010; Niino et al., 2014).
Regarding the BICAMS specifically, Sandi et al. (2015)
reported significant associations between fatigue and
the three tests that compose the battery.
The relationship between cognitive impairment and
depression remains complicated, as the direction of
effect is often unclear (Hildebrandt & Eling, 2014). The
effects of depression on cognition can be wide-ranging,
however, with research suggesting that MS patients with
moderate to severe depression show deficits in speed of
information processing, working memory, and executive
functions (Siegert & Abernethy, 2005).
Prior studies have found a significant relationship
between cognitive performance as measured by direct
assessment and the functional complaints of patients
and their informants (Benedict et al., 2004; O’Brien
et al., 2007). The most widely used self-report instru-
ment for neuropsychological functioning in MS patients
is the Multiple Sclerosis Neuropsychological Question-
naire (MSNQ), developed by Benedict et al. (2004). In
initial research, both versions of MSNQ - the one that
is completed by the patient and the one that is
completed by the informant - were significantly corre-
lated with cognitive functioning. However, the patient
version was more strongly correlated with self-reported
depression than with direct assessments of cognition
(Benedict et al., 2004). Concerning the relationship
between BICAMS or its subtests and MSNQ measures,
discrepant results are described. Some authors found
strong associations between the measure of processing
speed on the BICAMS and the self-report MSNQ
(O’Brien et al., 2007), (Benedict et al., 2008), (Akbar,
Honarmand, & Feinstein, 2011), while other studies
report no relationship between either MSNQ and any
BICAMS tests (Walker et al., 2016).
One of the major concerns of patients with MS is loss
of employment, as this disease affects people of working
age with personal and familial economic responsibilities
(Cores, Vanotti, Burin, Politis, & Villa, 2014). In the
literature, both cross-sectional (Honarmand, Akbar,
Kou, & Feinstein, 2011; Krokavcova et al., 2010)
(Benedict, Rodgers, Emmert, Kininger, & Weinstock-
Guttman, 2014) and longitudinal studies (Amato
et al., 2001) report that cognitive impairment is a risk
factor for loss of employment. Some of the investiga-
tions mentioned above have administered the BICAMS
tests, but this was done in the context of other batteries
such as the MACFIMS battery (Benedict et al., 2006;
Morrow et al., 2009; Strober et al., 2012) and the
BRB-N (Honarmand et al., 2011). The relationship
between employment status and the BICAMS battery
specifically has only been examined by a single study
that nonetheless found that BVMTR performance was
predictive of vocational status (Walker et al., 2016).
The BICAMS was developed as a validated measure of
cognitive dysfunction in MS patients that can be used in
routine clinical practice. Whereas prior batteries required
more extensive testing and specialized skills or materials,
making them impractical for use outside of specialist
centers (Langdon et al., 2012), the BICAMS tests were
selected to minimize the burdens of assessment. Its
association with clinical and social variables, including
self-perceived cognitive decline, should be studied.
Although considerable prior research exists on the topic,
none of the above-mentioned studies examined these
variables in an Argentine or other Latin American
population. The objectives of the present study were: to
analyze the relationship between BICAMS data and
clinical variables, such as disease duration, physical
disability, fatigue, and depression as they manifest in
the Argentine MS population; to explore the relationship
between BICAMS performance and perceived cognitive
difficulties; and to study the relationship between the
three BICAMS tests and key employment variables.
Materials and methods
Study design and participants
Fifty MS patients were recruited from an MS Clinic in
Buenos Aires, Argentina. All participants who attended
the neurological consultation and who met the inclusion
and exclusion criteria were invited to participate. The
 APPLIED NEUROPSYCHOLOGY: ADULT 3

study was approved by the local Ethics Committee and application to Spanish-speaking adults (S. Vanotti et al.,
an independent institutional review board of INEBA 2016). The procedure began with the Rao (oral-only)
Institution. All participants provided signed informed adaptation of the SDMT (Smith, 1982), in which a series
consent. of nine abstract geometric symbols were displayed on a
Inclusion criteria for the MS group were a diagnosis of standard sheet of paper, each paired with a single digit
clinically-defined MS (Polman et al., 2011), 18–60 years in a key at the top of the page, followed by several rows
old, fluency in Spanish, and ability to provide informed of unpaired symbols. Over a period of 90 seconds, parti-
consent for all procedures. Exclusion criteria were cipants were asked to speak aloud the digit associated
current or previous neurological disorder other than with each symbol as rapidly as possible. There was a
MS, history of psychotic disorder, current psychiatric dis- single outcome measure: the number of correct answers
order related to mood, personality, or behavior changes obtained over the 90-second time span. The official
following the onset of MS, a medical condition that might BICAMS battery would next include the CVLT II (Delis,
affect cognition, a history of developmental disorder, a Kramer, Kaplan, & Ober, 2000), but for our purposes the
history of substance or alcohol dependence, current sub- Spanish version of the CVLT I (Artiola I Fortuny, Romo,
stance abuse, motor or sensory disability that might Heaton, & Pardee, 1998) was employed as it was an
interfere with test performance, and having experienced already-validated translation that could provide an
a relapse and/or corticosteroid pulse in the previous equivalent assessment. This test required the examiner
4 weeks of assessment. In a prior study, these patients’ to read a list of 16 words, which the examinee then tried
cognitive function was shown to differ significantly from to repeat in any order. This procedure was then repeated
that of healthy controls (SDMT p < .001; CVLT I four more times for a total of five learning trials
p < .001; BVMTR p ¼ .017; Vanotti et al., 2016). altogether. The outcome measure was the total number
Table 1 shows the participants’ demographic data and of recalled items over the five learning trials, thus yielding
raw score ranges on the BICAMS Battery. a range of scores from 0 to 80. Finally, the BVMTR was
administered. In this task, a page displaying six abstract
Tests and procedures designs was held in the patient’s view for ten seconds.
The display was then removed from view and parti-
Cognitive status was assessed using the three-test cipants were asked to draw the stimuli with pencil and
BICAMS Battery, which has been extensively validated paper. Each drawing received a score of 0–2 points,
cross-culturally (Benedict et al., 2012), including specific representing location and accuracy. Thus, each trial score
ranged from 0 to 12. There were three learning trials, and
Table 1. Demographic, predictor, and outcome scores. the primary outcome measure was the total number of
MS Patients n ¼ 50 points earned over the three learning trials, with a
Mean SD Range theoretical range of 0 to 36. Form 1 was used for the test
Gender (Female) 37 (74%) procedure (Benedict, 1997).
Age (years) 43.42 10.17 19–60 Perceived cognition function was assessed with the
Education level (years) 14.86 2.78 7–19
EDSS–median 2.25 3.29 2.55 0.0–8.0 Argentinean adaptation of the Multiple Sclerosis
MSFC-z þ.32 1.55 4.34– þ 2.69 Neuropsychological Questionnaire (MSNQ) (Benedict
BDI FS 3.30 3.04 0–12
FSS 38.08 15.72 11–63 et al., 2004), (Vanotti, Benedict, Acion, Caceres, &
Disease duration 13.06 9.08 1–40 Workgroup, 2009). This questionnaire is a single page
MSNQ Patient 19.48 13.61 3–51
MSNQ Informant 16.18 12.58 0–48 of 15 statements describing problems encountered by
Employment status (employed) 30 (60%) patients with MS in the course of daily life, involving pro-
Work hours (per day) 4.06 3.83 0–9
Clinical Forms:
cessing speed, dual task processing, attention, memory,
Relapsing-Remitting 39 executive function, and psychosocial conduct. Patients
Secondary Progressive 9 rate themselves (MSNQ-Patient – MSNQ-P) on a 4-point
Primary Progressive 2
BICAMS Battery: scale ranging from 0 (problem not encountered) to 4
SDMT 45.14 16.07 10–91 (severe, occurs very frequently). The problem list is
CVLT 50.92 12.37 25–76
BVMT-R 20.70 7.74 3–35 simply rephrased for an observer rating or informant-
Notes. MS ¼ Multiple Sclerosis; EDSS ¼ Expanded Disability Status Scale; report form (MSNQ-Informant- MSNQ-I), which can
MSFC ¼ Multiple Sclerosis Functional Composite; BICAMS ¼ Brief be completed by anyone who knows the patient well
International Cognitive Assessment for Multiple Sclerosis; SDM ¼ Symbol
Digit Modalities Test; CVLT ¼ California Verbal Learning Test; BVMT-R ¼ and sees him or her in a range of situations, typically a
Brief Visuospatial Memory Test Revised; MSFC-z ¼ Multiple Sclerosis spouse or other adult family member.
Functional Composite z score; BDI FS ¼ Beck Depression Inventory Fast
Screen; FSS ¼ Fatigue Severity Scale; MSNQ ¼ Multiple Sclerosis
Neurological disability was assessed with the
Neuropsychological Questionnaire; SD ¼ Standard Deviation. clinician-rated Expanded Disability Status Scale (EDSS)
4 S. VANOTTI ET AL.

(Kurtzke, 1983) and the Multiple Sclerosis Functional individually via multivariate regression analysis, run with
Composite (MSFC) (Fischer, Rudick, Cutter, Reingold, all three BICAMS raw scores (SDMT, CVLT I total
& National MS Society Clinical Outcomes Assessment learning, and BVMTR total learning) as a composite
Task Force, 1999). The MSFC was administered requir- dependent variable. Note: each predictor was examined
ing patients to walk 25 feet as fast as possible, complete separately; in no case was a predictor assessed while
a nine-hole peg placement task with each hand, and controlling for another predictor. This also allows for
complete a demanding measure of executive function- the interpretation of each predictor as it relates to overall
ing (Paced Auditory Serial Addition Test (PASAT) at BICAMS performance. In each case, the F statistic
a 3.0 s interstimulus interval). Scores on these three for Wilks’ Λ was assessed for statistical significance
tasks were combined to yield a composite score in (variablewise a level of .05). The g2 value serves as an
accordance with the published algorithm (Cutter et al., estimate of the proportion of variance attributable to
1999). the predictor, and was examined as a measure of effect
The Beck Depression Inventory – Fast Screen for size. Thus, these values indicate the magnitude of the
Medical Patients (BDI FS) (Beck, Steer, & Brown, relationship between the independent and dependent
2000), validated in MS (Benedict et al., 2004), was admi- variables. Miles and Shevlin (2013) recommend that a
nistered to quantify depression symptoms. The Fatigue g2 value of 0.02 be considered small, a value of 0.13 be
Severity Scale (FSS) (Krupp, LaRocca, Muir-Nash, & considered medium, and a value of 0.26 be considered
Steinberg, 1989) was used to evaluate the degree of large (Miles & Shevlin 2013). These values are reported
self-reported fatigue. Both self-report surveys were in Table 2. Only variables found to have a statistically sig-
administered in Spanish. For all survey report scales nificant multivariate association with the three BICAMS
(MSNQ, BDI FS, FSS) higher scores indicate more scores were retained for additional univariate analysis in
severe symptomatology. which each BICAMS test was considered as a separate
Work status was initially coded in three categories: dependent variable. If an analysis-wise Bonferroni cor-
full and part time employment, unemployment or rection were to be applied, an alpha level of .004 would
retired, and homemakers and students. However, due be used for each individual variable, thus excluding age,
to a small overall sample size, this left very few patients BDI Fast Screen, and disease duration from further
in each group. As such, the data were recoded dichoto- analysis. All three of these variables have g2 values in
mously as unemployed specifically due to disability or the medium range and as such have potential to be clini-
not, a variable called Employment status. This is poten- cally significant. They were thus retained for univariate
tially overly restrictive as some individuals may report analysis. Gender, years of education, and patient MSNQ
not working for a reason other than disability (e.g., pre- score were not significantly predictive and were dropped
fer to be at home with family), when in fact disability from further analysis. Univariate analysis was the
played a role in their lack of employment. However, this conducted with all retained variables, comparing the
categorization avoids the problem of assuming that all variable to each BICAMS score individually (see Table 3).
unemployed MS patients are unemployed due to
disability. These data were reported by patients and
confirmed via informant interview. Patients also reported Table 2. Multivariate association between BICAMS and
mean hours worked per day. The BDI-FS, the FSS and the predictor variables.
Predictor Wilks’ Λ F p-value g2
work status rating were completed by neuropsycho-
Gender 0.982 0.282 0.838 .018
logists. Testing was completed over two days. On the first Age 0.796 3.940 .0140* .204
day, the neurology specialist recruited the patient, Education 0.902 1.657 0.189 .098
EDSS 0.644 8.473 <.001*** .356
assessed eligibility, and established informed consent. MSFC-z 0.376 25.425 <.001*** .624
Afterward, the neurologist completed the neurological BDI FS 0.801 3.807 .016* .199
examination, EDSS, and MSFC. On the second visit FSS 0.819 10.734 <.001*** .181
Disease duration 0.833 3.068 .037* .167
(no more than 10 days after the first), the participant MSNQ Patient 0.869 1.786 0.163 .104
was seen by the neuropsychologist who administered MSNQ Informant 0.702 6.353 .001** .298
Employment status 0.698 6.487 <.001*** .302
the BICAMS battery, the MSNQ (patient and informant), Work hours 0.841 8.874 <.001*** .159
the BDI-FS, FSS, and work status rating. Note. Raw p-values are provided for each individual comparison. If correction
for multiple comparisons is applied across the table, an a level of .004
would be applied. BICAMS ¼ Brief International Cognitive Assessment for
Statistical analysis Multiple Sclerosis; EDSS ¼ Expanded Disability Status Scale; MSFC-z ¼
Multiple Sclerosis Functional Composite z score; BDI FS ¼ Beck
Depression Inventory Fast Screen; FSS ¼ Fatigue Severity Scale; MSNQ ¼
For the core analysis, to reduce the overall number of Multiple Sclerosis Neuropsychological Questionnaire.
comparisons, potential correlates were first examined *p < .05. **p < .01. ***p < .001.
 APPLIED NEUROPSYCHOLOGY: ADULT 5

Table 3. Univariate association between BICAMS and predictor variables.

SDMT CVLT I BVMTR
2 2
Predictor F p g F p g F p g2
Gender dropped from analysis
Age 8.674 .005** 0.153 7.769 .008** 0.139 9.56 .003** 0.166
Education dropped from analysis
EDSS 20.366 <.001*** 0.298 12.514 .001** 0.207 19.646 <.001*** 0.290
MSFC-z 58.114 <.001*** 0.548 43.866 <.001*** 0.477 30.944 <.001*** 0.392
BDI FS 4.664 .036* 0.089 11.892 .001** 0.199 3.040 0.088 0.060
FSS 12.069 .001** 0.075 22.615 <.001*** 0.133 0.251 0.617 0.002
Disease duration 7.547 .008** 0.136 8.194 .006** 0.146 4.569 0.038* 0.087
MSNQ Patient dropped from analysis
MSNQ Informant 15.468 <.001*** 0.248 12.416 .001** 0.209 2.555 0.117 0.209
Employment status 7.485 .009** 0.137 19.988 <.001*** 0.298 3.107 0.084 0.062
Work hours 21.946 <.001*** 0.133 21.143 <.001*** 0.129 9.585 0.002** 0.063
Note. Raw p-values are provided for each individual comparison. If correction for multiple comparisons is applied across the table, an a level of .002 would be
applied. BICAMS ¼ Brief International Cognitive Assessment for Multiple Sclerosis; SDMT ¼ Symbol Digit Modalities Test; CVLT I ¼ California Verbal Learning
Test; BVMTR ¼ Brief Visuospatial Memory Test Revised; EDSS ¼ Expanded Disability Status Scale; MSFC ¼ Multiple Sclerosis Functional Composite z score; BDI
FS ¼ Beck Depression Inventory Fast Screen; FSS ¼ Fatigue Severity Scale; MSNQ ¼ Multiple Sclerosis Neuropsychological Questionnaire.
*p < .05. **p < .01. ***p < .001.

Table 4. Zero-order correlations with the tests of BICAMS (MSNQ-I; p ¼ .001) was also strongly predictive of
battery. overall BICAMS performance, yielding a g2 value of
SDMT CVLT I BVMTR .289, also considered large. Statistically significant asso-
Gender .068 .059 .037 ciations and medium g2 effect sizes were found for age
Age .391** .373** .408**
Education .275 .251 .283*
(p ¼ .014), the BDI-FS (p ¼ .016), disease duration
EDSS .546** .455** .539** (p ¼ .037), FSS (p < .001), and work hours (p < .001).
MSFC-z .740** .691** .626** Gender (p ¼ .838), level of education (p ¼ .189), and
BDI FS .298* .244 .446**
FSS .221 .350* .006 patient-reported MSNQ (p ¼ .163) were found to have
Disease duration .369** .382** .295* no statistically significant relationship to BICAMS
MSNQ Patient .292* .294* .158
MSNQ Informant .498** .457** .227 performance.
Employment status .371** .546** .249 For all variables other than gender, education, and
Work hours .440** .487** .358*
patient MSNQ, univariate analyses were conducted to
Notes. BICAMS ¼ Brief International Cognitive Assessment for Multiple
Sclerosis; SDMT ¼ Symbol Digit Modalities Test; CVLT I ¼ California assess the relationships between the variable and each
Verbal Learning Test; BVMTR ¼ Brief Visuospatial Memory Test Revised; of the three BICAMS tests. The results are reported in
EDSS ¼ Expanded Disability Status Scale; MSFC ¼ Multiple Sclerosis
Functional Composite z score; BDI FS ¼ Beck Depression Inventory Fast Table 3. Unadjusted p-values are reported, with statisti-
Screen; FSS ¼ Fatigue Severity Scale; MSNQ ¼ Multiple Sclerosis cal significance noted at an alpha level of .05. As such,
Neuropsychological Questionnaire.
*p < .05. **p < .01.
all predictor variables are statistically significantly
related to the SDMT and CVLT I, while only age,
MSFC, EDSS, disease duration, and work hours are
Lastly, to aid in comparability with other studies, significantly related to the BVMTR.
zero-order correlations were provided to estimate rela- If the Bonferroni correction were applied to the uni-
tionships between clinical variables and the three tests variate analysis as a whole, an alpha level of .002 would
of the BICAMS battery (Table 4). be applied to each test. In this case, the age and disease
duration variables would no longer be significantly
related to any BICAMS test; the BDI-FS and employ-
Results
ment status would no longer be significantly predictive
The majority of predictor variables showed a statistically of the SDMT; and the work hour variable would no
significant multivariate relationship with the BICAMS, longer be predictive of the BVMTR. These associations
as seen in Table 2. The variables with the strongest are weaker than those that would be retained, as
association (statistically significant and largest effect indicated by their lower g2 values. All predictions that
size) to BICAMS performance in multivariate analysis yielded a g2 value of .200 or greater are statistically
were those that directly assessed the patient’s current significant even when applying the stringent Bonferroni
capabilities: MSFC (p < .001), EDSS (p < .001), and correction.
employment status (p < .001). These variables yielded The strongest predictions, those that exceed Miles
g2 values ranging from .302 to .624, indicating a large and Shevlin’s (2013) large effect size standard, were
effect size. Informant rating of the patient’s functioning found in relating the MSFC to each of the three tests,
6 S. VANOTTI ET AL.
the EDSS to the SDMT and BVMTR, and employment
status to the CVLT I. In the case of the EDSS, moderate
association was found with the CVLT I. The employ-
ment status variable was more complicated, yielding a
large g2 when related to the CVLT I, a medium g2 value
in predicting the SDMT, and a nonsignificant associ-
ation with the BVMTR.
Moderate association (.13 <g2 < .26) was found
between age and all three tests. MSNQ-Informant
showed a moderate association to the SDMT and a
strong association to the CVLT I, but no association
the BVMTR. The BDI FS had a moderate relationship
to the CVLT I, a small relationship with the SDMT,
and a statistically insignificant relationship to the
BVMTR. Disease duration, like work hours, yielded a
medium g2 value in association with the SDMT and
CVLT I, but a small value in prediction of the BVMTR.
Fatigue also showed a medium-strength relationship
with the SDMT and CVLT I, but no association with
the BVMTR.
As previously noted, zero-order correlation results
have been reproduced in Table 4 to aid in comparability
with future studies.
Discussion
Multiple sclerosis is a complex disorder with a wide
range of interacting symptoms. While cognitive impair-
ment is not universal among MS patients, it is relatively
common and has a significant impact on a range of
functional variables. In developing countries like Argen-
tina, the relationships between functional variables and
cognitive performance cannot be assumed to be the
same as in more prosperous countries. For example,
physical — rather than cognitive — functioning may
be more strongly related to employment in an economy
that relies heavily on physical labor, whereas the reverse
may be true in the service or technology sectors of the
global workforce. This analysis both establishes the pat-
tern of association between cognitive and functional
outcomes as well as demonstrates the validity of the
BICAMS as a useful assessment in Argentina.
Both clinical and social variables were found to be
associated with the three BICAMS tasks, as well as
informants’ perception of patients’ cognitive difficulties.
BICAMS performance was most strongly associated
with the EDSS – a measure of physical disability and
disease progression — the MSFC, a measure of motor
and cognitive disability — and employment status.
These three variables each in some sense represent the
current total burden of disease. As such, it is unsurpris-
ing that they are strongly associated with the BICAMS.
A moderate association was found between the
BICAMS battery and other clinical variables such as
informant perception of functioning, disease duration,
fatigue, depression, and hours worked per day. As raw
scores — rather than age-adjusted — were used for
the analysis, the expected association was found with
age as well. Gender, level of education, and patient
perceived cognitive disability were not related to
BICAMS results.
When considering individual BICAMS tests, a simi-
lar overall pattern was found. The variables which were
constituted as the strongest predictors of performance
in BICAMS were the MSFC on each of the three
BICAMS tests, EDSS on SDMT and BVMTR, and
employment status on CVLT I. This contrast between
the EDSS and employment status variables may reflect
the different roles played by the specific BICAMS tests.
The EDSS measures disease progression and the
BVMTR is highly sensitive to the specific type of
impairment commonly found in MS patients. Employ-
ment status is a practical measure and visuospatial
memory may be of less relevance many jobs, though
there was a marginal finding relating work hours to
the BVMTR. Moderate associations were found between
the EDSS and CVLT I, as well as between employment
status and the SDMT. A moderate association was
found between age and each of the three battery tests.
Perception of cognitive difficulties reported by an
informant and fatigue yielded moderate association
with SDMT and CVLT I, but not with BVMTR. The
informant MSNQ findings may reflect the fact that slo-
wed processing speed and verbal memory deficits may
be more readily apparent in daily living activities than
visual memory impairments. Working hours was
moderately associated with the SDMT, but weakly
associated with both memory tests. Depression was
moderately associated with verbal memory and weakly
associated with processing speed.
Disease duration moderately statistically predicted
SDMT and CVLT I performance, while weakly predict-
ing BVMTR performance. This is consistent with the
work of Dusankova et al, (2012), who found a relation-
ship between disease duration and diminished BICAMS
performance and Niino et al. (2014) who found disease
duration to be predictive of SDMT score (Niino et al.,
2014). On the other hand, Giedraitienė, Kizlaitienė,
and Kaubrys (2015), have not found an association
between cognitive impairment on testing and years of
disease (Giedraitienė et al., 2015).
With regard to physical disability, our findings are
consistent with previous studies. Giedraitienė et al.
(2015) found that the level of physical disability as
measured by EDSS was a significant predictor of the
severity of cognitive impairment all three BICAMS tests,

consistent with our findings (Giedraitienė et al., 2015).
Niino et al. (2014) also found an association between
physical disability and processing speed (Niino et al.,
2014).
A moderate relationship between self-reported
fatigue and performance on tests of processing speed
and verbal memory was found in our study. Sandi
et al. (2015) found an association between self-reported
fatigue (using the Fatigue Impact Scale) and all three
BICAMS tests (Sandi et al., 2015). Other studies have
shown that fatigue impacts delayed verbal recall, but
not immediate learning (Hildebrandt & Eling, 2014).
Since the BICAMS uses the learning trials only, we
did not replicate that finding.
This study has found a moderate association between
self-reported depressive symptoms and verbal memory
as well as a weak association with processing speed.
This finding is in line with results described by
Hildebrandtl and Eling (2014), who found associations
between depression and diminished performance in
the learning stage of the CVLT (Hildebrandt & Eling,
2014), a finding replicated by others (Jougleux-Vie
et al., 2014; Niino et al., 2014). Niino et al., (2014) also
described a much stronger relationship between
depression and the SDMT than was found in our study
(Niino et al., 2014).
Patient reports of neurocognitive deficits via the
MSNQ were not found to be related to the BICAMS
tests, while the MSNQ informant form was moderately
predictive of both the SDMT and CVLT I. Prior
research has shown that the MSNQ-I is associated with
the SDMT (Benedict et al., 2004, 2008; O’Brien et al.,
2007). The discrepancy between patient and informant
forms of the MSNQ has been found previously in this
population with the MSNQ-I showing substantially
stronger relationships with cognitive factors than
MSNQ-P (Vanotti et al., 2009). This is likely due to
the well-documented phenomenon of patient self-report
measures varying in response to the patient’s present
emotional state (Benedict et al., 2017).
Finally, a strong association between employment
status and verbal memory was found, while the associ-
ation between employment status and processing speed
was moderate. Similar findings have been reported by
Honan et al. (2015) and Strober et al. (2012), though
Benedict et al. (2016) and Walker et al., (2016) have
found an association with visuospatial memory as well
(Benedict et al., 2016) (Walker et al., 2016). Our study
found a weaker pattern of associations with work hours
– although all three tests were statistically significantly
associated, the effect sizes were smaller than for employ-
ment status. Patients’ vocational status may be more
complex than simply “employed” or “unemployed”
APPLIED NEUROPSYCHOLOGY: ADULT 7
and the use of a continuous hours-worked variable
may detect more variability than a dichotomized
employment status criterion. Prior research has shown
a relationship between impaired processing speed and
a reduction in working hours (Jongen et al., 2014),
consistent with our findings.
Conclusion
The BICAMS is a short, easily administered battery of
three neuropsychological tests which was designed to
be maximally sensitive to the cognitive impairments
seen in MS patients while minimizing administration
demands (Langdon et al., 2012). Our findings suggest
that it is associated with a range of variables, most
prominently physical disability, overall disease
progression, and employment status, in addition to
age, disease duration, fatigue, depression, and informant
perception of impairment. Prior research has shown
relationships between MS-related cognitive impairment
and a range of clinical and psychosocial variables,
though this literature has mainly focused on patients
in developed countries, and has not examined the
unique needs of Argentine MS patients (Lynch et al.,
2005) (Niino et al., 2014). For this reason, this study
adds to the body of literature supporting the validity
of the BICAMS, specifically in Argentina, as well as
enhances the interpretability of Argentinian BICAMS
results.
With regard to limitations, our analysis, of course,
only includes diagnosed patients who attended an MS
clinic, and is therefore not entirely representative of
the full range of Argentinian MS patients (i.e., no
representation of those who lack access to adequate
diagnostic resources). In addition, while our sample
was sizable given the low rates of MS in Argentina, it
was relatively small in a strict statistical sense. While
we took steps in statistical analysis to control for the
increased variability of small samples, we hope that
future research will re-examine these results in larger
samples.
The current study, with its clinical and sociodemo-
graphic approaches, provides a better understanding
of Latin American MS patients in relation to their cog-
nitive difficulties, with important implications in both
clinical settings and future clinical trials, in addition
to further validating the use of the BICAMS in
Argentina. We believe that these results can facilitate
better and comprehensive management of MS patients,
particularly in the neurological clinical setting. Neuro-
logists are often the first to raise concerns about a
patient’s cognitive functioning despite the fact that
routine neurological assessment lacks sensitivity in
8 S. VANOTTI ET AL.
identifying cognitive impairment (Romero, Shammi, &
Feinstein, 2015). Furthermore, even if routine neuro-
logical care is able to identify cognitive impairment,
many patients in Argentina and other Latin American
nations do not have access to high-quality medical care
including consultation with neurologists who specialize
in MS. By demonstrating the validity of a brief, easily
administered battery, we hope to enhance neuropsycho-
logical assessment of MS patients in Latin America. We
also hope to make clinicians aware of the constellation
of features most likely associated with cognitive impair-
ment in Latin American MS patients to improve patient
management. While the present study does not allow us
to draw conclusions about a causal relationship between
the variables, the tendency of observable features to
co-occur is still clinically useful. For example, if a
clinician observes that a patient’s MSFC has declined
significantly, and the patient’s spouse reports cognitive
decline, our data not only suggest that this patient
should be referred for cognitive evaluation, but also that
the patient, if employed, is at elevated risk of an
unwanted change in occupational status. Finally,
currently there are no publications examining the
relationship between clinical or social variables with
cognitive impairment, as measured by the tests of
BICAMS Battery, in Argentina or other countries in
Latin America. We hope that this paper provides a
valuable cross-cultural perspective.
References
Achiron, A., Polliack, M., Rao, S. M., Barak, Y., Lavie, M.,
Appelboim, N., & Harel, Y. (2005). Cognitive patterns
and progression in multiple sclerosis: construction and
validation of percentile curves. Journal of Neurology,
Neurosurgery & Psychiatry, 76(5), 744–749. doi:10.1136/
jnnp.2004.045518
Akbar, N., Honarmand, K., & Feinstein, A. (2011). Self-
assessment of cognition in multiple sclerosis: The role of
personality and anxiety. Cognitive and Behavioral Neurology,
24(3), 115–121. doi:10.1097/WNN.0b013e31822a20ae
Amato, M. P., Ponziani, G., Siracusa, G., & Sorbi, S. (2001).
Cognitive dysfunction in early-onset multiple sclerosis: A
reappraisal after 10 years. Archives of Neurology, 58(10),
1602–1606. doi:10.1001/archneur.58.10.1602
Amato, M. P., Portaccio, E., Goretti, B., Zipoli, V., Iudice, A.,
Pina, D. D., … Group, T. S. (2010). Relevance of cognitive
deterioration in early relapsing-remitting MS: A 3-year fol-
low-up study. Multiple Sclerosis Journal, 16(12), 1474–1482.
doi:10.1177/1352458510380089
Andreasen, A. K., Spliid, P. E., Andersen, H., & Jakobsen, J.
(2010). Fatigue and processing speed are related in multiple
sclerosis. European Journal of Neurology, 17(2), 212–218.
doi:10.1111/j.1468–1331.2009.02776.x
Artiola I Fortuny, L., Romo, D. H., Heaton, R. K., & Pardee,
R. E., III. (Ed.) (1998). Manual de normas y procedimientos
para la Batería Neuropsicológica en español [Manual of
norms and procedures for the Neuropsychological Battery
in Spanish]. New York, NY: Psychology Press.
Beck, A. T., Steer, R. A., & Brown, G. K. (2000). BDI-Fast
Screen for medical patients: Manual. San Antonio, TX:
Psychological Corporation.
Benedict, R. H. (1997). Brief Visuospatial Memory Test -
Revised: Professional manual. Odessa, FL: Psychological
Assessment Resources, Inc.
Benedict, R. H., Amato, M. P., Boringa, J., Brochet, B., Foley,
F., Fredrikson, S., … Langdon, D. (2012). Brief inter-
national cognitive assessment for MS (BICAMS): Inter-
national standards for validation. BMC Neurology, 12, 55.
doi:10.1186/1471-2377-12-55
Benedict, R. H., Cookfair, D., Gavett, R., Gunther, M.,
Munschauer, F., Garg, N., & Weinstock-Guttman, B.
(2006). Validity of the minimal assessment of cognitive
function in multiple sclerosis (MACFIMS). Journal of the
International Neuropsychological Society, 12(4), 549–558.
Benedict, R. H., Cox, D., Thompson, L. L., Foley, F.,
Weinstock-Guttman, B., & Munschauer, F. (2004). Reliable
screening for neuropsychological impairment in multiple
sclerosis. Multiple Sclerosis, 10(6), 675–678. doi:10.1191/
1352458504ms1098oa
Benedict, R. H., DeLuca, J., Phillips, G., LaRocca, N., Hudson,
L. D., Rudick, R. & Multiple Sclerosis Outcome Assess-
ments Consortium. (2017) Validity of the Symbol Digit
Modalities Test as a cognition performance outcome
measure for multiple sclerosis. Multiple Sclerosis Journal,
23(5), 721–733. doi:10.1177/1352458517690821
Benedict, R. H., Drake, A. S., Irwin, L. N., Frndak, S. E.,
Kunker, K. A., Khan, A. L., … Weinstock-Guttman, B.
(2016). Benchmarks of meaningful impairment on the
MSFC and BICAMS. Multiple Sclerosis Journal, 22,
1874–1882. doi:10.1177/1352458516633517
Benedict, R. H., Duquin, J., Jurgensen, S., Rudick, R., Feitcher,
J., Munschauer, F., … Weinstock-Guttman, B. (2008).
Repeated assessment of neuropsychological deficits in
multiple sclerosis using the Symbol Digit Modalities Test
and the MS Neuropsychological Screening Questionnaire.
Multiple Sclerosis Journal, 14(7), 940–946. doi:10.1177/
1352458508090923
Benedict, R. H., Rodgers, J. D., Emmert, N., Kininger, R., &
Weinstock-Guttman, B. (2014). Negative work events and
accommodations in employed multiple sclerosis patients.
Multiple Sclerosis Journal, 20(1), 116–119. doi:10.1177/
1352458513494492
Caceres, F., Vanotti, S., Rao, S., & Workgroup, R. (2011).
Epidemiological characteristics of cognitive impairment of
multiple sclerosis patients in a Latin American country.
Journal of Clinical and Experimental Neuropsychology,
33(10), 1094–1098. doi:10.1080/13803395.2011.603690
Chiaravalloti, N. D., & DeLuca, J. (2008). Cognitive impair-
ment in multiple sclerosis. The Lancet Neurology, 7(12),
1139–1151. doi:10.1016/S1474-4422(08)70259-X
Cores, E. V., Vanotti, S., Burin, D. I., Politis, D. G., & Villa, A.
(2014). Factores asociados con la situación laboral de
pacientes con esclerosis múltiple [Factors associated with
the work situation of patients with multiple sclerosis].
Reviews Neurology, 58(04), 0175–0183.
Cutter, G. R., Baier, M. L., Rudick, R. A., Cookfair, D. L.,
Fischer, J. S., Petkau, J., … Willoughby, E. (1999).

Development of a multiple sclerosis functional composite
as a clinical trial outcome measure. Brain, 122(5), 871–
882. doi:10.1093/brain/122.5.871
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000).
California Verbal Learning Test manual (2nd ed.).
San Antonio, TX: Psychological Corporation.
Dusankova, J. B., Kalincik, T., Havrdova, E., & Benedict, R. H.
(2012). Cross cultural validation of the minimal assessment
of cognitive function in multiple sclerosis (MACFIMS) and
the brief international cognitive assessment for multiple
sclerosis (BICAMS). The Clinical Neuropsychologist, 26(7),
1186–1200. doi:10.1080/13854046.2012.725101
Fischer, J. S., Rudick, R. A., Cutter, G. R., Reingold, S. C. &
National MS Society Clinical Outcomes Assessment Task
Force. (1999). The multiple sclerosis functional composite
measure (MSFC): An integrated approach to MS clinical
outcome assessment. Multiple Sclerosis Journal, 5(4),
244–250. doi:10.1177/135245859900500409
Giedraitienė, N., Kizlaitienė, R., & Kaubrys, G. (2015). The
BICAMS battery for assessment of lithuanian-speaking
multiple sclerosis patients: Relationship with age, edu-
cation, disease disability, and duration. Medical Science
Monitor, 21, 3853–3859. doi:10.12659/msm.896571
Glanz, B. I., Healy, B. C., Rintell, D. J., Jaffin, S. K., Bakshi, R.,
& Weiner, H. L. (2009). The association between cognitive
impairment and quality of life in patients with early mul-
tiple sclerosis. Journal of Neurology Sciences, 290, 75–79.
doi:10.1016/j.jns.2009.11.004
Goretti, B., Niccolai, C., Hakiki, B., Sturchio, A., Falautano,
M., Minacapelli, E., … Amato, M. P. (2014). The brief
international cognitive assessment for multiple sclerosis
(BICAMS): Normative values with gender, age and
education corrections in the Italian population. BMC
Neurology, 14, 171. doi:10.1186/s12883-014-0171-6
Hildebrandt, H., & Eling, P. (2014). A longitudinal study on
fatigue, depression, and their relation to neurocognition
in multiple sclerosis. Journal of Clinical and Experimental
Neuropsychology, 36(4), 410–417. doi:10.1080/13803395.
2014.903900
Honan, C. A., Brown, R. F., & Batchelor, J. (2015). Perceived
cognitive difficulties and cognitive test performance as pre-
dictors of employment outcomes in people with multiple
sclerosis. Journal of the International Neuropsychological
Society, 21, 156–168. doi:10.1017/S1355617715000053
Honarmand, K., Akbar, N., Kou, N., & Feinstein, A. (2011).
Predicting employment status in multiple sclerosis patients:
The utility of the MS functional composite. Journal of
Neurology, 258(2), 244–249. doi:10.1007/s00415-010-
5736-8
Jongen, P. J., Wesnes, K., van Geel, B., Pop, P., Sanders, E.,
Schrijver, H. … and the COGNISEC study group. (2014).
Relationship between working hours and power of atten-
tion, memory, fatigue, depression and self-efficacy one year
after diagnosis of clinically isolated syndrome and relapsing
remitting multiple sclerosis. PLoS ONE, 9(5), e96444.
doi:10.1371/journal.pone.0096444
Jougleux-Vie, C., Duhin, E., Deken, V., Outteryck, O.,
Vermersch, P., & Zéphir, H. (2014). Does fatigue complaint
reflect memory impairment in multiple sclerosis? Multiple
Sclerosis International, 2014, 692468. doi:10.1155/2014/692468
Krokavcova, M., Nagyova, I., Van Dijk, J. P., Rosenberger, J.,
Gavelova, M., Middel, B., … Groothoff, J. W. (2010).
APPLIED NEUROPSYCHOLOGY: ADULT 9
Self-rated health and employment status in patients with
multiple sclerosis. Disability and Rehabilitation, 32(21),
1742–1748. doi:10.3109/09638281003734334
Krupp, L. B., LaRocca, N. G., Muir-Nash, J., & Steinberg,
A. D. (1989). The fatigue severity scale: Application to
patients with multiple sclerosis and systemic lupus
erythematosus. Archives of Neurology, 46(10), 1121–1123.
doi:10.1001/archneur.1989.00520460115022
Kurtzke, J. F. (1983). Rating neurologic impairment in
multiple sclerosis: An expanded disability status scale
(EDSS). Annals of Neurology, 13, 227–231.
Langdon, D. W. (2011). Cognition in multiple sclerosis.
Current Opinion in Neurology, 24, 244–249. doi:10.1097/
WCO.0b013e328346a43b
Langdon, D. W., Amato, M. P., Boringa, J., Brochet, B., Foley,
F., Fredrikson, S., … Benedict, R. H. (2012). Recommenda-
tions for a brief international cognitive assessment for
multiple sclerosis (BICAMS). Multiple Sclerosis Journal,
18(6), 891–898. doi:10.1177/1352458511431076
Lynch, S. G., Parmenter, B. A., & Denney, D. R. (2005). The
association between cognitive impairment and physical
disability in multiple sclerosis. Multiple Sclerosis Journal,
11(4), 469–476. doi:10.1191/1352458505ms1182oa
Miles, J., & Shevlin, M. (2013). Applying regression and
correlation. A guide for students and researchers. London,
UK: Sage.
Morrow, S. A., Weinstock-Guttman, B., Munschauer, F. E.,
Hojnacki, D., & Benedict, R. H. (2009). Subjective fatigue
is not associated with cognitive impairment in multiple
sclerosis: Cross-sectional and longitudinal analysis.
Multiple Sclerosis, 15(8), 998–1005. doi:10.1177/
1352458509106213
Niino, M., Mifune, N., Kohriyama, T., Mori, M., Ohashi, T.,
Kawachi, I., … Kikuchi, S. (2014). Apathy/depression, but
not subjective fatigue, is related with cognitive dysfunction
in patients with multiple sclerosis. BMC Neurology, 14(1),
1–8. doi:10.1186/1471-2377-14-3
O’Brien, A., Gaudino-Goering, E., Shawaryn, M., Komaroff,
E., Moore, N. B., & DeLuca, J. (2007). Relationship of the
multiple sclerosis neuropsychological questionnaire
(MSNQ) to functional, emotional, and neuropsychological
outcomes. Archives of Clinical Neuropsychology, 22(8),
933–948. doi:10.1016/j.acn.2007.07.002
O’Connell, K., Langdon, D., Tubridy, N., Hutchinson, M., &
McGuigan, C. (2015). A preliminary validation of the brief
international cognitive assessment for multiple sclerosis
(BICAMS) tool in an Irish population with multiple scler-
osis (MS). Multiple Sclerosis and Related Disorders, 4(6),
521–525. doi:10.1016/j.msard.2015.07.012
Patti, F. (2009). Cognitive impairment in multiple sclerosis.
Multiple Sclerosis Journal, 15(1), 2–8. doi:10.1177/
1352458508096684
Polman, C. H., Reingold, S. C., Banwell, B., Clanet, M., Cohen,
J. A., Filippi, M., … Wolinsky, J. S. (2011). Diagnostic cri-
teria for multiple sclerosis: 2010 Revisions to the McDonald
criteria. Annals of Neurology, 69(2), 292–302. doi:10.1002/
ana.22366
Rao, S. M. the Cognitive Function Study Group of the
National Multiple Sclerosis Society. (1990). A manual for
the Brief Repeatable Battery of Neuropsychological Tests in
Multiple Sclerosis. Milwaukee, WI: Medical College of
Wisconsin.
10 S. VANOTTI ET AL.
Romero, K., Shammi, P., & Feinstein, A. (2015). Neurologists’
accuracy in predicting cognitive impairment in multiples
sclerosis. Multiple Sclerosis and Related Disorders, 4,
291–295.
Sandi, D., Rudisch, T., Fuvesi, J., Fricska-Nagy, Z., Huszka, H.,
Biernacki, T., … Bencsik, K. (2015). The Hungarian vali-
dation of the brief international cognitive assessment for
multiple sclerosis (BICAMS) battery and the correlation
of cognitive impairment with fatigue and quality of life.
Multiple Sclerosis and Related Disorders, 4(6), 499–504.
doi:10.1016/j.msard.2015.07.006
Siegert, R. J., & Abernethy, D. A. (2005). Depression in
multiple sclerosis: A review. Journal of Neurology,
Neurosurgery & Psychiatry, 76, 469–475. doi:10.1136/
jnnp.2004.054635
Smith, A. (1982). Symbol Digit Modalities Test: Manual. Los
Angeles, CA: Western Psychological Services.
Spedo, C. T., Frndak, S. E., Marques, V. D., Foss, M. P.,
Pereira, D. A., Carvalho Lde, F., … Barreira, A. A. (2015).
Cross-cultural Adaptation, Reliability, and Validity of the
BICAMS in Brazil. The Clinical Neuropsychologist,
29(6), 836–846. doi:10.1080/13854046.2015.1093173
Strober, L. B., Christodoulou, C., Benedict, R. H., Westervelt,
H. J., Melville, P., Scherl, W. F., … Krupp, L. B. (2012).
Unemployment in multiple sclerosis: The contribution of
personality and disease. Multiple Sclerosis Journal, 18(5),
647–653. doi:10.1177/1352458511426735
Vanotti, S., Benedict, R. H., Acion, L., Caceres, F., &
Workgroup, V. (2009). Validation of the multiple sclerosis
neuropsychological screening questionnaire in Argentina.
Multiple Sclerosis, 15(2), 244–250. doi:10.1177/
1352458508097924
Vanotti, S., Smerbeck, A., Benedict, R. H. B., & Caceres, F.
(2016). A new assessment tool for patients with multiple
sclerosis from Spanish-speaking countries: Validation of
the brief international cognitive assessment for MS
(BICAMS) in Argentina. The Clinical Neuropsychologist,
30, 1023–1031. doi:10.1080/13854046.2016.1184317
Walker, L. A., Osman, L., Berard, J. A., Rees, L. M., Freedman,
M. S., MacLean, H., & Cousineau, D. (2016). Brief inter-
national cognitive assessment for multiple sclerosis
(BICAMS): Canadian contribution to the international
validation project. Journal of the Neurological Sciences,
362, 147–152. doi:10.1016/j.jns.2016.01.040