You are on page 1of 6

Feasibility of Telecognitive Assessment

in Dementia
C. Munro Cullum
Myron F. Weiner
Helena R. Gehrmann
Linda S. Hynan
University of Texas Southwestern Medical Center at Dallas

Videoconferencing (VC) technology has been used successfully to provide psychiatric


services to patients in rural and otherwise underserved settings. VC-based diagnostic
interviewing has shown good agreement with conventional face-to-face diagnosis of
dementia in several investigations, but extension of this technology to neurocognitive
assessment has received little attention. To this end, the authors administered a brief bat-
tery of common neuropsychological tests via VC technology (telecognitive) and tradi-
tional face-to-face methods to 14 older persons with mild cognitive impairment (MCI)
and 19 persons with mild to moderate Alzheimer’s disease (AD). Highly similar test
scores were obtained when participants were tested in-person or via VC. Telecognitive
assessment appears to be a valid means to conduct neuropsychological evaluation of
older adults with cognitive impairment. Furthermore, continued development of VC tech-
nology has implications for expanding neuropsychological assessment options in under-
served populations.

Keywords: neuropsychological testing; cognition; dementia; videoconferencing; telemed-


icine; telecognitive assessment

Neuropsychological testing is used to detect, charac- is high satisfaction with VC-based psychiatric interviews
terize, and quantify cognitive impairment in elders and is (Hilty et al., 2002; Kobak, 2004). On the other hand, little
an integral component of dementia evaluations (Cullum, has been done to explore VC technology in the assess-
Paulman, Koss, Chapman, & Lacritz, 2003). However, ment of elders’ cognitive functioning. A literature review
the use of neuropsychological testing in some settings revealed only four publications that included elders, and
has been limited by the need for face-to-face contact. none included more than a few persons with adequately
Significant advances have been made in telemedicine in diagnosed dementia.
the first decade of its existence, and although still not Montani et al. (1997) administered the Mini-Mental
widely used, a growing number of studies suggest its State Exam (MMSE; Folstein, Folstein, & McHugh, 1975)
applicability and validity in psychiatry (Frueh et al., and the Clock-Drawing Test (Tuokko, Hadjistavropolulos,
2000; Hilty, Luo, Morache, Marcelo, & Nesbitt, 2002) Miller, Horton, & Beattie, 1995) to 14 medical inpatients
and geropsychiatry in particular (Tang, Chiu, Woo, (age 83 to 95) comparing face-to-face with VC examina-
Hjelm, & Hui, 2001). Good agreement between in-person tion. Tests were administered a week apart in random
and VC-based clinical diagnosis of dementia has been order. Mean MMSE scores were similar, 23.7 (face-to-
reported (Lee et al., 2000; Shores et al., 2004) and there face) versus 22.2 (VC), r = .95, p < .003. For Clock

This article was supported in part by NIA 2P30 AG12300. Please address correspondence to C. Munro Cullum, PhD, 5323 Harry
Hines Blvd., Dallas, TX 75390-9044; e-mail: munro.cullum@utsouthwestern.edu.
Assessment, Volume 13, No. 4, December 2006 385-390
DOI: 10.1177/1073191106289065
© 2006 Sage Publications

Downloaded from asm.sagepub.com at UCSF LIBRARY & CKM on April 6, 2015


386 ASSESSMENT

Drawing, the mean scores were similar but the correla- METHOD
tion was nonsignificant (r = .55).
Ball and Puffett (1998) administered the cognitive Participants
portion (CAMCOG) of the CAMDEX (Roth, Huppert,
Tym, & Mountjoy, 1988) comparing face-to-face with During March and April 2005, we recruited 14
VC administration to eight elders from an outpatient individuals with MCI (Petersen et al., 1999) and 19 indi-
psychogeriatric service tested 1 week apart. Ages ranged viduals with possible (n = 1) or probable (n = 18) AD
from 65 to 84; diagnoses included depression, schizo- (McKhann et al., 1984). Diagnosis was made via con-
phrenia, and dementia. It was noted that visual quality sensus (involving psychiatry, neurology, and neuropsy-
of test stimuli was good using VC-based presentation, chology) using standard criteria as noted and prior to
and agreement rates were good for orientation (r = .83) and participation in the current study. Diagnostic evaluations
memory (r = .84) but poor for calculation (r = .10) and included neuropsychological testing with instruments
abstract thought tasks (r = 0.41). Menon et al. (2001) other than those used in this investigation. Participants
used VC to administer the Short Portable Mental Status with AD were required to have an accompanying study
Questionnaire (SPMSQ; Pfeiffer, 1975) to a convenience partner (in all cases, this was a spouse or an adult child)
sample of persons admitted to a VA Geriatric Evaluation who could be with them in the VC testing room to
and Management Unit using videophones. Twelve partic- decrease potential anxiety associated with the unfamiliar
ipants underwent both face-to-face and VC interviews. environment and to assist the examiner if the patient
There were similar cognitive and depression scores became confused or uncooperative. We excluded per-
across conditions, although few quantitative details were sons who were illiterate, had profound hearing loss or
provided. Ratings of participant satisfaction with the severe visual impairment, or whose cognitive incapacity
interview procedures across conditions were similar. precluded informed consent. The MCI sample consisted
More recently, Hildebrand, Chow, Williams, Nelson, of nine men and five women. The AD sample was com-
and Wass (2004) administered a battery of neurocognitive posed of 18 participants with a diagnosis of probable AD
tests to a group of 29 cognitively intact volunteers older and 1 with possible AD and included 13 men and 6
than age 60. Participants were tested face-to-face and by women. All participants were fluent in English, and all
VC. Two test batteries were assembled using nine stan- but 1 was Caucasian; 3 participants had mild hearing
dardized measures and alternate forms, including tests of impairment. Because the primary goal of this study was
verbal learning, verbal fluency, vocabulary, visuospatial to demonstrate the feasibility of VC-based neuropsycho-
reasoning, attention, and clock drawing. Similar mean test logical testing in older participants with cognitive
scores (i.e., within 0.5-2.0 points) were obtained under each impairment, and to keep other test conditions stable,
condition on most measures, although a slight order effect optimize patient compliance, and minimize anxiety in
was seen on six of the nine tests, with greater variability in the novel VC environment, the same examiner tested the
scores found among the participants who underwent VC- same participants in each condition. Test condition (in-
based testing first. A survey of participants revealed satis- person or VC) and test forms were counterbalanced
faction with VC testing, although a slight majority (44%) across participants and tests were administered in the
indicated a preference for face-to-face testing. same order.
Little is known about the efficacy of VC as a medium
for neuropsychological assessment of cognitively impaired Procedure
elders and no studies have systematically examined
VC-based neurocognitive assessment in patients with a Approval for this study was obtained from the
clinical diagnosis of Mild Cognitive Impairment (MCI; Institutional Review Board at The University of Texas
Petersen et al., 1999) or Alzheimer’s disease (AD). With Southwestern Medical Center and written consent was
the rapid growth of the elderly segment of our popula- obtained from all participants prior to participation.
tion, there will be an increasing need for neuropsycho- Prospective participants were told that we were doing an
logical evaluations for both diagnosis and monitoring of experiment to see if cognitive testing could be done by
treatment. If telecognitive assessment is valid, reliable, television/video. The examiner gave each participant a
practical, and cost efficient, it may significantly advance brief introduction about the VC testing procedure at the
the diagnosis and treatment of underserved elders with outset and explained that the examiner would appear on
known or suspected cognitive impairment. screen but would be readily available in person should
We hypothesized that VC and face-to-face testing would they require assistance. Participants were seated and
yield consistent results across neurocognitive measures viewed the examiner and test materials on a 19-in. color
using a brief battery of commonly used tests to evaluate monitor. Study partners were asked to remain seated in
patients with MCI and AD. the background behind the patient to provide reassurance

Downloaded from asm.sagepub.com at UCSF LIBRARY & CKM on April 6, 2015


Cullum et al. / TELECOGNITIVE ASSESSMENT 387

TABLE 1 Equipment
Demographics and MMSE Scores
MCI + AD Sample (N = 33)
An H.323 PC-based Videoconferencing System was
set up in two nonadjacent rooms using a Polycom iPower
M SD Range 680 Series videoconferencing system (2 units). The 680
Age (years) 73.3 6.9 51–84
provided high quality audio and full motion video. Visual
Education (years) 15.1 2.7 8–20 stimuli for testing were displayed by a flatbed graphics
MMSE (face-to-face) 26.1 3.3 15–30 camera with integrated lighting, 12X zoom, autofocus, and
MMSE (VC) 26.3 3.7 15–30 iris (Canon Video Visualizer RE 650 MK II). Participants
NOTE: MMSE = Mini-Mental State Exam; MCI = mild cognitive viewed the examiner on a 19-in. color monitor and a
impairment; AD = Alzheimer’s disease; VC = videoconferencing. 29-in. monitor was used to view the participant. Because
we were able to adjust the audio volume of the monitor,
hearing-impaired individuals did not report or appear to
or assistance understanding test instructions if needed. have difficulty. The cost of this equipment, including
Testing took place in back-to-back sessions, each lasting both stations, was approximately $12,000.
45 to 50 min, with a brief (i.e., 5 to 10 min) break in
between. Examiners were two trained psychometrists
who also operated the VC equipment during each testing RESULTS
session.
Criteria for the selection of tests administered included The MCI and AD groups were similar in age, t(31) =
(a) brevity, (b) common use in the clinical evaluation of .37, p = .72, and education, t(31) = 1.3, p = .21; they dif-
older adults with cognitive impairment, (c) availability of fered significantly on face-to-face MMSE, t(22.99) =
alternate forms (where possible), (d) adequate test-retest 4.09, p < .001, and telecognitive MMSE, t(21.73) = 3.31,
reliability (Lezak, Howieson, & Loring, 2004), (e) suit- p = .003. For the purpose of this study as a demonstration
ability for VC-based administration and response record- of feasibility of telecognitive assessment, the MCI and
ing, and (f) assessment of a variety of cognitive skills AD groups were combined and their general characteris-
important in dementia evaluations (global cognitive func- tics are presented in Table 1.
tioning, verbal memory, attention, verbal fluency, and Paired samples statistics were performed for each
naming). Measures included the MMSE, Hopkins Verbal of the tests comparing in-person with telecognitive test-
Learning Test–Revised (HVLT-R; Benedict, Schretlen, ing across both groups combined. Because the Clock
Groninger, & Brandt, 1998), Clock Drawing Test Drawing test (Goodglass & Kaplan, 1983) has a range of
(Goodglass & Kaplan, 1983), Digit Span (Randolph, 1 to 3, Cohen’s Kappa and percentage of agreement were
1998), Category Fluency (fruits and vegetables; Spreen & used to examine reliability. For the other continuous
Strauss, 1991), letter fluency (FAS and CFL versions; measures, reliability across the two testing formats was
Benton, Hamsher, & Sivan, 1994), and 15-item versions of determined by examining the results for the Intraclass
the Boston Naming Test (BNT; Mack, Freed, Williams, & Correlation Coefficient, a measure of reliability agree-
Henderson, 1992). With the exception of the MMSE and ment, and the Bradley-Blackwood Procedure, a test that
Clock Drawing, the other tests had an available alternate simultaneously compares the means and variances of
form. We made the following modifications to adapt for the two measurements (Bartko, 1994). The pattern for
repeated testing and VC administration: The MMSE items good reliability using these two statistics is a significant
were the same in both testing conditions except for three- Intraclass Correlation Coefficient (indicating high agree-
word recall, which used different stimuli in each condition. ment between measurements) greater than .60 and a non-
For the HVLT-R, Forms 1 and 4 were selected due to their significant Bradley-Blackwood Procedure (the test-retest
high correlation. Four primary HVLT-R scores were measurements have similar means and variances). If
obtained: total words learned throughout trials 1 to 3, the Bradley-Blackwood Procedure was significant, the
delayed recall, percentage retention (Delayed Recall / Trial paired t test and the Pitman test were used to evaluate
3 × 100), and a recognition discrimination index (True where the bias between the two measurements occurred
Positives – False Positives). We used short form BNT ver- (inequality of means and variances). Analyses were per-
sions 3 and 4, which show the highest correlation (Mack formed using Statistical Package for the Social Sciences
et al., 1992). The visuospatial measures (MMSE pen- (SPSS) version 13 and due to the proof-of-concept
tagons and Clock Drawing) were scored via the television nature of the study design, the significance level for
monitor by asking participants to hold up their paper in reporting statistical results was set at .01 to adjust for
front of the camera when they were finished. multiple comparisons. Table 2 presents a comparison of

Downloaded from asm.sagepub.com at UCSF LIBRARY & CKM on April 6, 2015


388 ASSESSMENT

TABLE 2
Means, Standard Deviations, and Correlations for Cognitive Tests
Administered Via VC and Face-to-face
Pearson Bradley-Blackwood
Telecognitive Face-to-Face Correlation* Procedure

Measure M SD M SD r Intraclass Correlation p**

MMSE 26.27 3.67 26.09 3.29 .89 .88 .36


HVLT-R total recall 18.91 7.66 19.00 7.58 .77 .77 .99
HVLT-R delayed recall 3.21 3.91 4.39 5.07 .63 .61 .10
HVLT-R retention % 33.39 36.46 41.85 40.92 .55 .54 .33
HVLT-R recognition 8.03 3.84 8.00 3.64 .68 .68 .92
Digit span total 10.30 2.64 10.30 2.02 .81 .78 .05
Category fluency 9.67 3.39 9.18 3.28 .58 .58 .65
Letter fluency 32.64 13.78 33.97 12.53 .83 .83 .40
BNT (15 item) 12.30 2.62 12.48 2.84 .88 .87 .49

NOTE: VC = videoconferencing; MMSE = Mini-Mental State Exam; HVLT-R = Hopkins Verbal Learning Test–Revised; BNT = Boston Naming Test.
*p < .001. **two-tailed p.

neurocognitive test results between the two testing and face-to-face testing conditions. Thus, for most mea-
conditions. sures, similar results were obtained when participants
There were similar mean scores on HVLT-R Total were tested in-person or by VC. Examiners also solicited
Recall and Recognition, category and letter fluency, and feedback from participants about their testing experiences
BNT. Slightly higher means in the face-to-face condition following testing sessions. No participant expressed any
were seen on HVLT-R Delayed Recall. One of the larger difficulties or concerns regarding telecognitive testing,
(but statistically nonsignificant) differences was seen on although several indicated a preference for personal
HVLT-R Retention, with in-person testing associated contact.
with a higher mean score. Large standard deviations were
seen on this measure in both conditions, which was not
unexpected because the retention score is computed from DISCUSSION
immediate and delayed recall scores, thereby inflating
measurement error beyond the individual scores alone Scores between VC-based and face-to-face testing
and making it more difficult to achieve high stability esti- were highly similar across tests administered, supporting
mates. Scores on Clock Drawing (which has a very lim- our hypothesis of good agreement between testing
ited range of scores) had equal medians and Digit Span modalities in addition to good agreement between the
had identical mean scores for both testing groups. alternate test forms used. Overall, 9 of the 10 cognitive
The measures that showed significantly high (> .60) test scores were found to have both high associations and
ICC scores were MMSE, HVLT-R Total Recall, HVLT-R little bias between testing conditions. Measures of verbal
Delayed Recall, HVLT-R Recognition, Digit Span Total, learning and memory, simple attention, letter and cate-
Letter Fluency, and short form BNT. Although HVLT-R gory fluency, and confrontation naming showed excellent
Retention and Category Fluency scores were considered agreement between telecognitive and face-to-face testing.
significant, these measures did not meet the criterion of Despite a strong correlation, the verbal percentage reten-
.60 or higher. Kappa for the Clock Drawing scores was tion score on the HVLT-R showed substantial variability
moderate (κ = 0.48, p < .0001), with 75.8% (25/33) in each test session, suggesting that this score may not
agreement. be as reliable as the other memory indices. Additional
To examine the potential bias (unequal means and/or reminders for examinees to memorize the words for later
variances) for these two testing methods, the Bradley- recall might help decrease the variability on this task in
Blackwood Procedure was used. All tests were found to future studies. The finding of somewhat higher mean ver-
have simultaneously nonsignificant means and variances, bal retention scores in the face-to-face condition is inter-
suggesting highly similar results across testing condi- esting and merits further investigation, although it is
tions. Fully eight of the nine test scores (continuous mea- possible that the personal support of having the examiner
sures) had high associations and little bias between VC present in the room may somehow have a beneficial effect

Downloaded from asm.sagepub.com at UCSF LIBRARY & CKM on April 6, 2015


Cullum et al. / TELECOGNITIVE ASSESSMENT 389

on effort/performance on this arguably more challenging REFERENCES


task. The lack of statistical agreement between VC and
face-to-face administration of the Clock Drawing task is Ball, C., & Puffett, A. (1998). The assessment of cognitive function in
also noteworthy but likely due to the limited variance in the elderly using videoconferencing. Journal of Telemedicine and
the present sample. Telecare, 4(Suppl. 1), 36-38.
Bartko, J. J. (1994). Measures of agreement: A single procedure.
This study is limited by its inclusion of well-educated Statistics in Medicine, 13, 737-745.
patients who were familiar with neuropsychological testing. Benedict, R. H. B., Schretlen, D., Groninger, L., & Brandt, J. (1998).
None had profound receptive or expressive aphasia, as Hopkins Verbal Learning Test–Revised: Normative data and analysis
of inter-form and test-retest reliability. Clinical Neuropsychologist,
might be found in stroke patients, and all participants but 12, 43-55.
one were Caucasian. As noted, participants were tested by Benton, A. L., Hamsher, K. D., & Sivan, A. B. (1994). Multilingual
the same examiner in both conditions, which may have Aphasia Examination (3rd ed.). Iowa City, IA: AJA Associates.
(Distributed by Psychological Assessment Resources, Odessa, FL)
inflated test-retest correlations. As a pilot study, however, Cullum, C. M., Paulman, R. G., Koss, E., Chapman, S. B., & Lacritz, L.
the primary goal was to demonstrate the feasibility of VC- (2003). Evaluation of cognitive functions. In M. F. Weiner &
based cognitive testing, and we wanted to keep as many A. M. Lipton (Eds.), The dementias: Diagnosis, treatment, and
research (3rd ed., pp. 285-320). Washington, DC: American
testing conditions as stable as possible. Our concern about Psychiatric Publishing.
the potential effect of novelty of the VC environment on Folstein, M. F., Folstein, S. E., & McHugh, P. (1975). Mini-mental
cognitively impaired participants led us to have a study part- state: A practical method for grading the cognitive state of patients
for the clinician. Journal of Psychiatric Research, 12, 189-198.
ner sit in with them during testing. In practice, all partici- Frueh, B. C., Deitsch, S. E., Santos, A. B., Gold, P. B., Johnson, M. R.,
pants seemed to adapt readily to the VC-based interactions Meisler, N., et al. (2000). Procedural and methodological issues in
after initial introduction, and it is probably not necessary to telepsychiatry research and program development. Psychiatric
Services, 51, 522-1527.
have a caregiver present during VC testing for most partici- Goodglass, H., & Kaplan, E. (1983). Boston Diagnostic Aphasia
pants with mild to moderate AD. The use of VC technology Examination. Philadelphia: Lea & Febiger. (Distributed by
has previously been limited by the quality of sound and pic- Psychological Assessment Resources, Odessa, FL)
Hildebrand, R., Chow, H., Williams, C., Nelson, M., & Wass, P. (2004).
ture, but we did not find this to be an issue with the equip- Feasibility of neuropsychological testing of older adults via video-
ment we employed. We do not know what effect conference: Implications for assessing the capacity for independent
long-distance transmission of VC technology might have on living. Journal of Telemedicine and Telecare, 10, 130-134.
Hilty, D. M., Luo, J. S., Morache, C., Marcelo, D. A., & Nesbitt, T. S.
sound and picture, but we are currently investigating this. It (2002). Telepsychiatry: An overview for psychiatrists. CNS Drugs,
also should be noted that telecognitive assessment is limited 16, 527-548.
to tasks that do not require direct participant/examiner Kobak, K. A. (2004). A comparison of face-to-face and videoconfer-
ence administration of the Hamilton Depression Rating Scale.
interaction with test materials (e.g. Wechsler Adult Journal of Telemedicine and Telecare, 10, 231-235.
Intelligence Scale–Third Edition [WAIS-III] Block Design, Lee, J. H., Kim, J. H., Jhoo, J. H., Lee, K. U., Kim, K. W., Lee, D. Y.,
Wisconsin Card Sorting Test), although administration of a et al. (2000). A telemedicine system as a care modality for demen-
tia patients in Korea. Alzheimer Disease and Related Disorders, 14,
number of other popular measures should be feasible (e.g. 94-101.
Raven’s Matrices, WAIS-III Matrix Reasoning, Judgment Lezak, M., Howieson, D., & Loring, D. (2004). Neuropsychological
of Line Orientation). Future incorporation of computer- assessment (4th ed.). New York: Oxford University Press.
Mack, W. J., Freed, D. M., Williams, B. W., & Henderson, V. W. (1992).
based assessment procedures in conjunction with VC inter- Boston naming test: Shortened versions for use in Alzheimer’s dis-
action also is promising, particularly as this technology ease. Journals of Gerontology, 47, 154-158.
becomes less expensive and more widely available. McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., &
Stadlan, E. M. (1984). Clinical diagnosis of Alzheimer’s disease:
Report of the NINCDS-ADRDA work group under the auspices of
Department of Health and Human Services Task Force on
CONCLUSIONS Alzheimer’s Disease. Neurology, 34, 939-944.
Menon, A. S., Kondapavalru, P., Krishna, P., Chrismer, J. B., Raskin, A.,
Hebel, R. J., et al. (2001). Evaluation of a low cost videophone sys-
Neuropsychological tests administered by VC tech- tem in the assessment of depressive symptoms and cognitive func-
nology were successfully completed in all cases, and tion in elderly medically ill veterans. Journal of Nervous and
Mental Disease, 189, 399-401.
there was close agreement with test findings when admin- Montani, C., Billaud, N., Tyrrell, J., Fluchaire, I., Malterre, C.,
istered face-to-face in persons with MCI and mild to Lauvemay, N., et al. (1997). Psychological impact of a remote psy-
moderate AD. In addition, telecognitive testing was well chometric consultation with hospitalized elderly people. Journal of
Telemedicine and Telecare, 3, 140-145.
tolerated by all participants. This suggests that telecognitive Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J., Tangalos, E. G.,
assessment of persons with suspected or diagnosed & Kokmen, E. (1999). Mild cognitive impairment: clinical charac-
dementia is feasible, reliable, and merits further develop- terization and outcome. Archives of Neurology, 56, 303-308.
Pfeiffer, E. (1975). A short portable mental status questionnaire for
ment because it holds particular promise in rural and oth- the assessment of brain deficit in elderly patients. Journal of the
erwise underserved populations. American Geriatrics Society, 25, 433-441.

Downloaded from asm.sagepub.com at UCSF LIBRARY & CKM on April 6, 2015


390 ASSESSMENT

Randolph, C. (1998). RBANS manual: Repeatable Battery for the neurocognitive assessment and differential diagnosis of demen-
Assessment of Neuropsychological Status. San Antonio, TX: The tia, memory dysfunction, and neuropsychiatric disorders.
Psychological Corporation.
Roth, M., Huppert, F. A., Tym, E., & Mountjoy, C. Q. (1988).
CAMDEX: The Cambridge Examination for Mental Disorders of Myron F. Weiner is a geriatric psychiatrist. He is professor of
the Elderly. Cambridge, UK: Cambridge University Press. psychiatry and neurology at The University of Texas
Shores, M. M., Ryan-Dykes, P., Williams, R. M., Mamerto, B., Sadak, T., Southwestern Medical Center at Dallas. His clinical and
Pascualy, M., et al. (2004). Identifying undiagnosed dementia in research interests are in improving the diagnosis, management,
residential care veterans: Comparing telemedicine to in-person clin-
ical examination. International Journal of Geriatric Psychiatry, 19, and treatment of persons with dementing illness.
101-108.
Spreen, O., & Strauss, E. (1991). A compendium of neuropsychological Helena R. Gehrmann is a psychological assistant in the
tests. New York: Oxford University Press. Alzheimer's Disease Center at The University of Texas
Tang, W. K., Chiu, H., Woo, J., Hjelm, M., & Hui, E. (2001).
Telepsychiatry in psychogeriatric service: A pilot study.
Southwestern Medical Center at Dallas. Her research interests
International Journal of Geriatric Psychiatry, 16, 88-93. include dementia and neuropsychological assessment.
Tuokko, H., Hadjistavropolulos, T., Miller, J., Horton, A., & Beattie, B. L.
(1995). The Clock Test. Toronto, Canada: Multi-Health Systems. Linda S. Hynan is associate professor of biostatistics and psy-
chiatry at The University of Texas Southwestern Medical
C. Munro Cullum is a neuropsychologist and professor of psy- Center at Dallas. Her primary research interests include assess-
chiatry and neurology at The University of Texas Southwestern ing psychometric properties of tests, modeling behavioral and
Medical Center at Dallas. His primary research interests include biological processes, and medical decision making.

Downloaded from asm.sagepub.com at UCSF LIBRARY & CKM on April 6, 2015

You might also like