Expanded Disability Status Scale Calculator for Handheld Personal Digital Assistant: Reliability Study

Clyde E. Markowitz, MD; Mary D. Hughes, MD; Daniel D. Mikol, MD, PhD; Lisa Shi, PhD; MerriKay Oleen-Burkey, PhD; Douglas R. Denney, PhD; on behalf of the EDSS Calculator Study Group Through a standard neurological examination, the Expanded Disability Status Scale (EDSS) score is traditionally determined by a pen-and-paper method. The objectives of this study were to compare the interrater/method reliability of EDSS scores obtained through the pen-and-paper method with those obtained with the EDSS Calculator, evaluate test-retest reliability of the EDSS Calculator, and examine the validity of EDSS Calculator scores through correlation with the Ambulation Index (AI). During a single office visit, 62 subjects with multiple sclerosis (MS) from six study centers had disability levels evaluated by separate raters using the EDSS calculator and pen-and-paper methods in random order. Seven days later, subjects returned for reevaluation with the EDSS Calculator; AI was also evaluated. Mean EDSS scores for the sample were 3.5 2.2 with the calculator and 3.4 2.0 (range 08) with the traditional method. Interrater reliability between scores with the two methods was high ( = 0.84; confidence interval [CI], 0.740.94). Test-retest reliability with the calculator was also high ( = 0.93; CI, 0.860.996). Correlation between disability scores from the EDSS Calculator and AI was 0.73 (P < .001). Relative to the pen-and paper method, the EDSS Calculator was found to be highly reliable and provide valid results when measuring disability in MS patients. Int J MS Care. 2008;10:3339.
umerous assessment scales have been developed over the past several decades to measure the impact and progression of multiple sclerosis (MS). 1 Some have focused specifically on ambulation by having clinicians measure speed and use of assistance (eg, the Ambulation Index [AI]) or refinements such as differentiating stages of cane use (eg, disease steps), whereas others have been designed as patient-reported indices of symptoms and disability (eg, Symptom Inventory, performance scales, PatientDetermined Disease Steps).2-5 By far the most widely used method for assessing neurological disability in clinical trials has been the clinician-rated Kurtzke functional system and Expanded Disability Status Scale
From the MS Center, University of Pennsylvania, Philadelphia, PA (CEM); Department of Neurology, Medical College of Georgia, Augusta, GA (MDH); Department of Neurology, University of Michigan, Ann Arbor, MI (DDM); Information Solutions and Outcomes Research, Teva Neuroscience, Inc., Kansas City, MO (LS; MOB); and Department of Psychology, University of Kansas, Lawrence, KS (DRD), USA.

(EDSS), which measures disability in several domains in addition to ambulation.1 Developed as a 10-step scale and subsequently expanded to include half steps,6-8 EDSS has been the primary method used to rate disability in phase 3 clinical trials of therapies for MS.9-14 It has also been used in natural history studies to determine long-term disability progression and in studies correlating disability with other markers of disease activity (eg, brain atrophy and cytokine activity).15-18 Its near-universal familiarity among neurologists conducting MS research in academic settings around the world is one of the main strengths of EDSS in the research arena. 19 Despite widespread international adoption, however, EDSS has several limitations; eg, researchers have raised questions about the strength of its psychometric properties and have reported difficulties related to administration and scoring.20 EDSS is a traditional pen-and-paper assessment tool of some degree of complexity. Seven functional systems (pyramidal, cerebellar, brain stem, sensory, bowel and

International Journal of MS Care

33

Markowitz et al.

bladder, visual, and cerebral or mental) plus other functions are rated in a standard neurological examination with ordinal scales ranging from 0 to 5 or 6.6,8 The results are recorded on printed worksheets. These observations are used together with information concerning gait and need for assistive devices to derive the EDSS score on an ordinal scale ranging from 0 (normal neurological examination) to 10 (death from MS) in 0.5-point increments. The scale includes a set of predefined rules for calculating the EDSS score from the functional system findings, as outlined by Kurtzke.8 Although EDSS scoring time varies, the neurological examination of functional systems and subsequent determination of the EDSS score generally take between 15 and 30 minutes.21 The EDSS Calculator program was developed to offer MS clinics a means of consistently recording patients functional system scores, which are then used by the software program to compute EDSS scores. The software was designed for handheld personal digital assistants (PDAs) using the Palm or Windows operating system. The software can also be used on desktop or laptop computers to serve as a backup for the resulting database. The EDSS Calculator program enables the examiner to enter the functional system scores directly into the PDA. The predefined set of rules originally established in Kurtzkes 1983 article has been incorporated into the software. The calculator program computes the EDSS score and records it in a database stored on the PDA; it is also readily transferable to any personal computer. A report of the functional system scores and the EDSS score can also be printed for the patients chart. We designed the current study to determine the interrater/method reliability of the EDSS Calculator program relative to the standard pen-and-paper method of EDSS determination. We also examined the test-retest reliabili-

ty of the EDSS Calculator program when used by the same rater to assess patients on two different occasions separated by ~1 week. A secondary purpose was to perform a preliminary assessment of the validity of the EDSS Calculator for measuring MS-related disability by comparing it with findings on the AI for the same patient.

Materials and Methods

This was a multicenter study of patients with a clinically definite diagnosis of MS who were not currently experiencing exacerbations of disease. The protocol and informed consent were approved by the ethics committees or institutional review boards of the participating research sites. Men and women at least 18 years old were eligible for enrollment if they had a diagnosis of MS and EDSS scores between 1 and 9. Patients were excluded if they could not be evaluated with the EDSS, had significant comorbid illnesses that may have affected the EDSS rating, or had any condition that, in the investigators judgment, would interfere with study participation. Concomitant medication use was allowed during the study. Participants could be withdrawn from the study for reasons including serious adverse events, noncompliance with the protocol or other protocol violations, and loss to follow-up. The study consisted of two clinic visits (Figure 1). At the first visit, written informed consent was obtained from all eligible patients who were not currently experiencing an exacerbation of disease. Patients who met these criteria became study subjects and were randomized in a 1:1 manner to an initial disability assessment with either the EDSS Calculator program or the traditional pen-and-paper method. All EDSS Calculator assessments were performed with a handheld

Figure 1. Study design Note: Both study visits included monitoring for serious adverse events.

International Journal of MS Care

34

EDSS Calculator for Handheld PDA

device. After this initial assessment, the subjects were evaluated by a separate rater with the opposite EDSS assessment method. The order of these two evaluations was counterbalanced across subjects. The subjects were given an appointment to return for a second clinic visit after 7 days. (A window of 2 days was permitted for scheduling this second visit.) At the second visit, subjects were reexamined to verify that they were still free of exacerbations. The same clinician who had assessed disability with the EDSS Calculator at the first visit performed the disability assessment using the EDSS Calculator program at the second visit. This clinician also administered the 25-Foot Timed Walk to assess disability with AI. All six of the primary grading investigators for this study were neurologists; two were MS specialists, and the others were general neurologists who manage MS patients in their neurology practices. Most performed EDSS assessments daily, and all had experience with a handheld device and were willing to use it in conjunction with patient care. Coinvestigators could be neurologists or nurse practitioners who also had experience with handheld devices and performed EDSS assessments at least occasionally. The coinvestigators were mostly general neurologists, although a few were neurology subspecialists, in other than MS (eg, neuroophthalmology, movement disorders, pediatric neurology), and at one study site, the rater was a nurse practitioner. Among all investigators, experience with neurological disability assessment ranged from ~7 to 25 years, although for some, EDSS assessment was not a routine part of current practice. For this reason, a prestudy site initiation training program by a clinical research associate was implemented that covered EDSS assessment in general and the use of the PDA and its EDSS program. Administration of EDSS was reviewed by having all grading clinicians view a training CD that included three complete neurological examinations on video, six variations, and several special problems. The grading clinicians assessed the neurological status, functional systems, and EDSS for each of three cases. Administration and coding of AI were also reviewed. In addition, grading clinicians were trained on the use of one of the two handheld PDAs used in this study: the Palm Zire with the Palm operating system or the IPAQ h1935 Pocket PC with the Windows CE operating system. Each study center was assigned only one of the

two PDA models, and the EDSS Calculator program performed the same on each model. During training on the PDA, an EDSS Calculator User Training Manual and Quick Reference Guide were introduced and given to each study center. The full training program took ~2 hours per person at each center. The primary objective of this study was to compare the interrater/method reliability of EDSS disability scores obtained with the EDSS Calculator with the traditional pen-and-paper method. Interrater/method reliability was assessed with the statistic, where 1 represents perfect agreement and 0 represents agreement no better than would be obtained by chance.22,23 A 95% confidence interval (CI) was constructed for the statistic. Agreement between the two methods of EDSS evaluation was also assessed with the intraclass correlation coefficient (ICC).24 Categorical agreement was defined as the percentage agreement with no difference, a difference of 0.5 point (1 EDSS step), or a difference of 1.0 point (2 EDSS steps). Secondary objectives of this study were to evaluate the test-retest reliability of the EDSS Calculator program in a handheld device when used by the same rater with the same patient on two occasions 1 week apart and to examine the validity of EDSS scores obtained with the EDSS Calculator program through correlation with the AI. For test-retest reliability, the same analyses were used as for interrater/method reliability. Because EDSS scores are heavily influenced by ambulation, AI was also determined for each subject to serve as a preliminary basis for assessing the validity of the EDSS Calculator program. AI uses the time and level of assistance required to walk a marked 25-foot course to rate the individuals disability on a scale of 0 (asymptomatic and fully active) to 10 (bedridden).2,21 Like the EDSS itself, AI has been used to measure disability in population-based surveys and clinical trials.25,26 The correlation coefficient between the EDSS Calculator program score and AI was determined with Spearmans rank-order correlation. Our initial predictions of interrater reliability for purposes of designing an adequately powered study were based on findings by Sharrack and colleagues,27 who reported an interrater reliability of 0.65 with repeated use of the pen-and-paper EDSS. The assumption used for the sample size was that the correlation between the two EDSS assessments in this study would be at least 0.68. The determination was made that sample size

International Journal of MS Care

35

Markowitz et al.

(2.0) when obtained with the pen-and-paper EDSS assessment. For both methods, the median score was 3.0, ranging from 0 to 8. Results Sixty subjects returned the following week for the A total of 62 subjects at six study sites were enrolled second clinic visit; none was experiencing an exacerbaand randomized into the study. The subjects ranged in age from 20 to 63 (mean 43.8 10.4 [SD]) years. tion of symptoms at the time. For these subjects, Forty-six (74%) of the subjects were women. The sub- indicated an extraordinarily high level of test-retest relitype of MS was not recorded for one of the subjects, ability for the EDSS Calculator program ( = 0.93; but of the remaining sample, 51 (82%) had relapsing- 95% CI, 0.860.99; P < .0001). The distribution of remitting, 5 (8%) had primary progressive, and 5 (8%) differences in scores between the two assessments is shown in Figure 3. Exact agreement between the two had secondary progressive MS. EDSS scores occurred for 70% of the subjects, with the The value reflecting the level of agreement between EDSS scores determined via the EDSS Calcula- scores falling within 0.5 EDSS point for 83% and tor program versus the pen-and-paper method was within 1.0 EDSS point for 92% of the sample. The 0.84 (95% CI, 0.740.94). Values of > 0.80 indicate ICC was 0.92, with the 95% CI ranging from 0.87 to near-perfect agreement. The level of agreement was 0.95. Noteworthy outliers included two cases whose highly significant (P < .0001). The distribution of differences between EDSS scores EDSS scores with the EDSS Calculator program and obtained with the two methods is shown in Figure 2. the pen-and-paper method differed by 3.0 points and Forty-five percent of the subjects had exact agreement, one whose scores differed by 4.5 points. The 4.5-point and the scores fell within 0.5 EDSS point for 69% and difference occurred when an EDSS score of 5.5 was within 1.0 EDSS point for 84% of the sample. The obtained with the pen-and-paper method and the ICC was 0.86 (95% CI, 0.780.91). Across the full EDSS Calculator method produced a score of just 1.0, sample, the mean (SD) EDSS score was 3.5 (2.2) when suggesting a possible error in the entry of functional obtained with the EDSS Calculator program and 3.4 system ratings into the handheld device. This was affirmed a week later, when the EDSS Calculator program administered by 50 the same rater produced an EDSS score of 5.5 for this subject, matching the pen-and-paper method of the pre40 vious week. For the 3.0-point discrepancies, the functional system ratings 30 for each of the subjects led to higher EDSS scores with the EDSS CalculaEDSS Calculator < EDSS traditional EDSS Calculator > EDSS traditional tor program than the pen-and-paper 20 method of assessment. In one case, the rater agreed with his rating using the 10 EDSS Calculator program a week later. In the other case, the rater disagreed with his earlier rating and 0 agreed with the pen-and-paper method 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0 2.5 3.0 result of the prior week. In both of EDSS score difference (Calculator minus traditional) these instances, the discrepancies probably stemmed from differences in the Figure 2. Score differences between EDSS Calculator and subjects presentations during the two traditional EDSS evaluations examinations that occurred during the Note: = 0.84 (CI, 0.740.94; P < .0001). first visit.
Patients (%)

needed to be in the range of 2655 to provide a lower limit for a 95% CI of at least 0.410.50.

International Journal of MS Care

36

EDSS Calculator for Handheld PDA

Spearman rank-order correlations between the scores determined with the EDSS Calculator program and both the elapsed time and the AI based on the 25-Foot Timed Walk could be computed for 58 subjects. The correlation coefficients were 0.42 (P = .001) and 0.73 (P < .001), respectively. Two of the 60 subjects who reported for the second visit of the study did not complete the 25-Foot Timed Walk.

70

60

50

Patients (%)

40 Visit 2 < Visit 1 30 Visit 2 > Visit 1

20

Discussion
10 The primary finding of this study was the high interrater/method reliabili0 ty between the EDSS scores obtained 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 with the EDSS Calculator program in a EDSS score difference handheld device and those obtained (Visit 2 minus visit 1) with the pen-and-paper method. The Figure 3. Score difference between first and second EDSS statistic of 0.84 indicated almost perfect Calculator evaluations agreement between the two methods of Note: = 0.93 (CI, 0.860.99; P < .0001). assessment. Test-retest reliability with the EDSS Calculator program was similarly impressive, with = 0.93 and the observers pro- the information on EDSS mistakes is anecdotal, making ducing identical EDSS scores on the two assessments in an estimate of how often such errors occur difficult. 70% of the subjects. The consistently close agreement However, note that in the case of 3 (9%) of 34 subjects between both the methods of EDSS determination and with disagreements between the two methods, the functhe scores derived from the EDSS Calculator program tional system scores were identical, yet the resulting on two occasions demonstrated the feasibility of convert- EDSS scores were different. Such a result could not ing the EDSS assessment into a semiautomated process occur were it not for errors of the second or third type aided by a handheld device. described above. Sharrack and colleagues27 reported an interrater reliaThe functional systems workup with the EDSS Calbility of 0.65 for the pen-and-paper EDSS; other groups culator program is conducted in the same manner as have reported even lower levels of interobserver agree- with the pen-and-paper method. Therefore, the EDSS ment, ranging from 10 to 69%.19,28-31 Clinicians may Calculator does not address the potential for errors arrive at an inaccurate EDSS score for various reasons. resulting from the raters lack of neurological expertise First, errors in the detection and recording of symptoms and training. The need to devote substantial time to during the neurological examination may lead to inaccu- EDSS training has been raised as a limitation of the ratrate functional system scores, which affect the EDSS cal- ing scale,32,33 and lack of adequate training has been cited culation. Second, clinicians may miscalculate the EDSS as an underlying cause of low interrater reliability.1 The score despite an accurate assessment of the functional clinicians competence at evaluating the functional syssystems. Third, clinicians may misapply the predefined tems is an important issue that must be addressed rules for combining these functional system components regardless of the method used. In this study, EDSS traininto the overall EDSS score. Although not rectifying the ing was conducted at all participating research sites, and first source of error, the EDSS Calculator program elimi- videotaped review materials were available as an ongoing nates the possibility that an accurate assessment of the educational resource. The time required for additional functional systems could nevertheless result in an inaccu- training on the use of the PDA itself was minimal and rate EDSS score for the second or third reasons. Much of was accomplished during the study initiation visit for

International Journal of MS Care

37

Markowitz et al.

each site. This should be reassuring, especially for clinicians who may be reluctant to incorporate computerbased technologies into their everyday practice routines. The adequacy of the EDSS itself as a tool to measure MS-related disability has elicited considerable debate among MS researchers,20,34 and alternative disability scales have been introduced with purportedly more robust psychometric properties.35 In this study, the correlation between the EDSS Calculator and AI was determined to be 0.73. Other studies have reported correlations between the AI and the traditional pen-and-paper EDSS ranging from 0.55 to 0.79.36,37 Although a widely used measure of disability in population-based surveys and clinical trials,25,26 AI is certainly not considered the gold standard against which to establish the validity of alternative disability measures. Further study of the validity of the EDSS Calculator program is thus warranted, but its validity may prove to be equivalent to that of the traditional EDSS. It may in fact be slightly better, in that reliability is one of the properties of a measure that limits its validity, and use of the EDSS Calculator program appears to improve the reliability of the EDSS. This article is believed to be the first pertaining to the use of a computerized tool for assessing MS-related disability; however, the benefits of using electronic versions of rating scales have been demonstrated in other areas of medicine, including pain management,38,39 behavioral disorders,40 and asthma.41 For example, Cook et al.39 compared paper and electronic versions of the ShortForm McGill Pain Questionnaire and Pain Disability Index. With both instruments, both the written and electronic formats were shown to have good psychometric properties, and scores obtained with the two formats were highly correlated. In conclusion, the EDSS Calculator program is an innovative tool contributing to the ease and consistency of EDSS determination. The high level of interrater and test-retest reliability demonstrated in this study supports its use as a substitute for the traditional pen-and-paper version of the EDSS in research settings and clinical practice. Computerized storage of and access to EDSS information may enable clinics to more easily consider patients EDSS over time, thereby helping to identify disability trends at the patient level, for patient populations overall, or for various treated segments of those populations.

References
1. Amato MP, Ponziani G. Quantification of impairment in MS: discussion of the scales in use. Mult Scler. 1999;5:216219. 2. Hauser SL, Dawson DM, Lehrich JR, et al. Intensive immunosuppression in progressive multiple sclerosis: a randomized, three-arm study of high-dose intravenous cyclophosphamide, plasma exchange, and ACTH. N Engl J Med. 1983;308:173180. 3. Hohol MJ, Orav EJ, Weiner HL. Disease steps in multiple sclerosis: a simple approach to evaluate disease progression. Neurology. 1995;45:251255. 4. Schwartz CE, Vollmer T, Lee H, North American Research Consortium on Multiple Sclerosis Outcomes Study Group. Reliability and validity of two self-report measures of impairment and disability for MS. Neurology. 1999;52:6370. 5. Hohol MJ. Disease steps in multiple sclerosis: a longitudinal study comparing disease steps and EDSS to evaluate disease progression. Mult Scler. 1999;5:349354. 6. Kurtzke JF. A new scale for evaluating disability in multiple sclerosis. Neurology. 1955;5:580583. 7. Kurtzke JF. Further notes on disability evaluation in multiple sclerosis, with scale modifications. Neurology. 1965;15:654661. 8. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983;33:1444 1452. 9. IFNB Multiple Sclerosis Study Group. Interferon beta-1b is effective in relapsing-remitting multiple sclerosis: I clinical results of a multicenter, randomized, double-blind, placebo-controlled trial. Neurology. 1993;43:655661. 10. Jacobs LD, Cookfair DL, Rudick RA, et al. Intramuscular interferon beta1a for disease progression in relapsing multiple sclerosis. Ann Neurol. 1996;39:285294. 11. Johnson KP, Brooks BR, Cohen JA, et al. Extended use of glatiramer acetate (Copaxone) is well tolerated and maintains its clinical effect on multiple sclerosis relapse rate and degree of disability: Copolymer 1 Multiple Sclerosis Study Group. Neurology. 1998;50:701708. 12. PRISMS (Prevention of Relapses and Disability by Interferon beta-1a Subcutaneously in Multiple Sclerosis) Study Group. Randomised double-blind placebo-controlled study of interferon beta-1a in relapsing/remitting multiple sclerosis. Lancet. 1998;352(9139):14981504. 13. Polman CH, OConnor PW, Havrdova E, et al. A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis. N Engl J Med. 2006;354:899910. 14. Rudick RA, Stuart WH, Calabresi PA, et al. Natalizumab plus interferon beta-1a for relapsing multiple sclerosis. N Engl J Med. 2006; 354:911923. 15. Confavreux C, Vukusic S, Moreau T, Adeleine P. Relapses and progression of disability in multiple sclerosis. N Engl J Med. 2000; 343:14301438. 16. Pittock SJ, Mayr WT, McClelland RL, et al. Change in MS-related disability in a population-based cohort: a 10-year follow-up study. Neurology. 2004;62:5159. 17. Fisher E, Rudick RA, Cutter G, et al. Relationship between brain atrophy and disability: an 8-year follow-up study of multiple sclerosis patients. Mult Scler. 2000;6:373377. 18. Rentzos M, Cambouri C, Rombos A, et al. IL-15 is elevated in serum and cerebrospinal fluid of patients with multiple sclerosis. J Neurol Sci. 2006;241:2529. 19. Amato MP, Fratiglioni L, Groppi C, Siracusa G, Amaducci L. Interrater reliability in assessing functional systems and disability on the Kurtzke scale in multiple sclerosis. Arch Neurol. 1988;45:746748. 20. Hobart J, Freeman J, Thompson A. Kurtzke scales revisited: the application of psychometric properties to clinical intuition. Brain. 2000; 123:10271040. 21. National Multiple Sclerosis Society. Measures for use in clinical studies of MS. Available at: http://www.nationalmssociety.org/ MUCS_ AI.asp. Accessed 27 July 2007. 22. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Measurement. 1960;20:3746. 23. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159174.

International Journal of MS Care

38

EDSS Calculator for Handheld PDA

24. Norman GR, Streiner DL. Principal components and factor analysis. In: Norman GR, Streiner DL. Biostatistics. St. Louis, MO: Mosby; 1993: 129142. 25. Goodin DS. Survey of multiple sclerosis in northern California: the Northern California MS Study Group. Mult Scler. 1999;5:7888. 26. Visser LH, Beekman R, Tijssen CC, et al. A randomized, double-blind, placebo-controlled pilot study of i.v. immune globulins in combination with i.v. methylprednisolone in the treatment of relapses in patients with MS. Mult Scler. 2004;10:8991. 27. Sharrack B, Hughes RA, Soudain S, Dunn G. The psychometric properties of clinical rating scales used in multiple sclerosis. Brain. 1999;122(1):141159. 28. Noseworthy JH, Vandervoort MK, Wong CJ, Ebers GC. Interrater variability with the Expanded Disability Status Scale (EDSS) and Functional Systems (FS) in a multiple sclerosis clinical trial. Neurology. 1990;40:971975. 29. Verdier-Taillefer MH, Zuber M, Lyon-Caen O, et al. Observer disagreement in rating neurologic impairment in multiple sclerosis: facts and consequences. Eur Neurol. 1991;31:117119. 30. Francis DA, Bain P, Swan AV, Hughes RA. An assessment of disability rating scales used in multiple sclerosis. Arch Neurol. 1991;48:299 301. 31. Goodkin DE, Cookfair D, Wende K, et al. Inter- and intrarater scoring agreement using grades 1.0 to 3.5 of the Kurtzke Expanded Disability Status Scale (EDSS): Multiple Sclerosis Collaborative Research Group. Neurology. 1992;42:859863. 32. Herndon RM. Handbook of Neurologic Rating Scales. New York: Demos Vermande; 1997. 33. Lechner-Scott J, Huber S, Kappos L. Expanded Disability Status Scale (EDSS) training for MS-multicenter-trials [abstract]. J Neurol. 1997; 244(suppl 3):S25. 34. Whitaker JN, McFarland HF, Rudge P, Reingold SC. Outcomes assessment in multiple sclerosis clinical trials: a critical analysis. Mult Scler. 1995;1:3747. 35. Cutter GR, Baier ML, Rudick RA, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain. 1999;122:871882. 36. Ruiz-Pena JL, Pinero P, Sellers G, et al. Magnetic resonance spectroscopy of normal appearing white matter in early relapsing-remitting multiple sclerosis: correlations between disability and spectroscopy. BMC Neurol. 2004;4:8. 37. Vaithianathar L, Tench CR, Morgan PS, Constantinescu CS. Magnetic resonance imaging of the cervical spinal cord in MS. J Neurol. 2003; 250(3):307315. 38. Jamison RN, Gracely RH, Raymond SA, et al. Comparative study of electronic vs. paper VAS ratings: a randomized, crossover trial using healthy volunteers. Pain. 2002;99:341347. 39. Cook AJ, Roberts DA, Henderson MD, et al. Electronic pain questionnaires: a randomized, crossover comparison with paper questionnaires for chronic pain assessment. Pain. 2004;110:3017. 40. Van Winkle LC, Chastain DC, Hamill-Ruth RJ. Behavioral Dyscontrol Scale-Electronic Version: first examination of reliability, validity, and incremental utility. Clin Neuropsychol. 2005;9:426. 41. Verschelden P. Compliance with and accuracy of daily self-assessment of peak expiratory flows (PEF) in asthmatic subjects over a three month period. Eur Respir J. 1996;9:880885.

Acknowledgments: We acknowledge the members of the EDSS Calculator Study Group: Bruce Cleeremans, MD, Marshall Grossman, MD, and Michael Cummings, MD, NervePro Research, Irvine, CA; Mariko Kita, MD, Michael Elliott, MD, and Lynne Taylor, MD, Benaroya Research Institute at Virginia Mason, Seattle, WA; Richard Shubin, MD, and Jai-Hyon Rho, MD, Neuro-Therapeutics, Inc., Pasadena, CA; Clyde Markowitz, MD, and Ruth Brobst, MSN, CRNP, MS Center at University of Pennsylvania, Philadelphia, PA; Mary Hughes, MD, and Elizabeth Sekul, MD, Department of Neurology, Medical College of Georgia, Augusta, GA; and Daniel Mikol, MD, PhD, and Wayne Cornblath, MD, Department of Neurology, University of Michigan, Ann Arbor, MI. Additionally, Harold Schombert, JD, Thought Leaders Medical Communications, Inc., and Pippa Loupe, PhD, Scientific Communications, Teva Neuroscience, Inc., assisted in developing this manuscript. No patents or royalties are associated with the EDSS Calculator program. Dr. Oleen-Burkey and Dr. Shi are employed by Teva Neuroscience. Dr. Mikol has received consulting fees and speakers honoraria from Biogen Idec, Serono, and Teva Neuroscience. Dr. Denney has received consulting fees and grant support from Teva Neuroscience. No conflicts of interest have been submitted by Drs. Markowitz and Hughes.

International Journal of MS Care

39