• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • CommentGo Back
Download
 
Reliability and validity of the Edinburgh Visual Gait Score forcerebral palsy when used by inexperienced observers
A.M.L. Ong
a
, S.J. Hillman
b
, J.E. Robb
c,
*
a
College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, Scotland, United Kingdom
b
 National Centre for Training and Education in Prosthetics and Orthotics, University of Strathclyde,Glasgow, Scotland, United Kingdom
c
 Anderson Gait Analysis Laboratory, Royal Hospital for Sick Children, Department of Orthopaedic Surgery,Sciennes Road, EH9 1LF Edinburgh, Scotland, United Kingdom
Received 4 December 2007; received in revised form 19 January 2008; accepted 19 January 2008
Abstract
The Edinburgh Visual Gait Score (EVGS) for cerebral palsy has been validated for observer reliability and validity for observersexperienced in gait analysis. This study investigated the reliability and validity of the EVGS for observers inexperienced in gait analysis. Sixmedical students used the score to analyse videotapes from the original study by Read et al. [Read HS, Hazlewood ME, Hillman SJ, PrescottRJ, Robb JE. Edinburgh visual gait score for use in cerebral palsy. J Pediatr Orthop 2003;23:296–301]. These were viewed on two separateoccasions to provide inter- and intra-observer reliability, and the results of the numerical items were compared to those from three-dimensional (3D) gait analyses for validity. Observer agreement was tested using Coefficient of Repeatability (CoR), percentage of completeagreement and the kappa statistic. The CoR for inter-observer agreement for inexperienced observers was 5.99/5.07 (Session 1/Session 2)compared to 4.60/3.95 (Session 1/Session 2) for experienced observers. The CoR for intra-observer agreement for inexperienced observerswas5.15comparedto4.21forexperiencedobservers.Therewascompleteagreementfor52%ofthe10numericalitemswith3D-gaitanalysisdata for inexperienced observers compared to 64% for experienced observers. Ranking of reliability of individual items was similar betweenthe two groups and was generally best for events occurring at the foot and ankle. Observations of gait events by the inexperienced observersusing the EVGS were reasonably reliable but not very accurate when compared to experienced observers and 3D-gait analysis.
#
2008 Elsevier B.V. All rights reserved.
Keywords:
Gait; Cerebral palsy; Visual assessment
1. Introduction
Although three-dimensional gait analysis (3DGA) hasbecome widely accepted as the gold standard for evaluatingcomplex locomotor disorders it is not widely availableworldwide. In 2003 Read et al.[1]produced the EdinburghVisual Gait Score (EVGS) for cerebral palsy usingexperienced observers. The Score showed good intra-observer and inter-observer reliability and almost two-thirdsof the observations correlated with instrumented 3DGAdata, signifying good validity. Subsequently the Score wascompared to other measures (Gillette Gait Index (GGI),Gillette Functional Assessment Questionnaire and dimen-sionless speed) and showed significant correlations, espe-cially with the GGI[2].One problem in studies of gait is defining observerexperience and how to quantify this. The purpose of thisstudy was to investigate intra- and inter-observer reliabilityand validity of the EVGS when used by observers who werenot specifically experienced in clinical gait analysis. Ourhypotheses were firstly that the inexperienced observerswould be less reliable than experienced observers andsecondly that there would be a discernible learning effectand improvement of scores between the first and secondviewings.
www.elsevier.com/locate/gaitpost
 Available online at www.sciencedirect.com
Gait & Posture 28 (2008) 323–326* Corresponding author. Tel.: +44 131 536 0834; fax: +44 131 536 0852.
E-mail address:
j.e.robb@btinternet.com(J.E. Robb).0966-6362/$ – see front matter
#
2008 Elsevier B.V. All rights reserved.doi:10.1016/j.gaitpost.2008.01.008
 
2. Materials and methods
2.1. Subjects
The same video recordings of the five subjects used in Readetal’s.[1]studywereviewedbysix4thyearundergraduatemedicalstudents. These observers had no expertise in the visual assessmentof gait or cerebral palsy management but all had the same exposureto undergraduate locomotor system and neurology teaching. Toensure consistency all six received a standardised briefing aboutnormalgaitkinematics andanexplanationofhowtousetheEVGS.The observers then used the EVGS to assess each limb of thesubjects on the video recordings on two occasions, which wereseparated by an interval of 2 weeks. The subjects on the videotapeswere assessed in a different order at the second viewing. Theobservers were allowed to view the videotape in slow motion orfreeze-frame and no limit was placed on the time taken to view thesubjects’ gait.
2.2. Statistical analyses
The results from the two sessions were compared to calculateintra-observer reliability, using the Bland and Altman’s Coefficientof Repeatability (CoR)[3]. Inter-observer reliability for the 17items of the EVGS was examined in two ways: the percentage of complete agreement between observers and the kappa statistic.Grading of the kappa scores were carried out according to Landisand Koch[4].The results of the 10 numerical items from the EVGS werecompared to 3DGA data, considered to be the gold standard, tovalidate the observers’ scores. Complete agreement was noted if the joint angle ranges scored by the observers were within thedefined limits of the 3DGA values. The percentage of completeagreement was then calculated for each of the ten numericalitems. These results from inexperienced observers were thencompared with the results of the experienced observers fromRead et al.[1].
3. Results
3.1. Intra-observer reliability
The mean CoR for intra-observer agreement for theinexperienced observers (1–6) was 5.15 (range 3.79–7.27)compared to a mean CoR of 4.21 (range 3.29–5.93) forexperienced observers (A–E) from the study of Read et al.[1](Table 1).
3.2. Inter-observer reliability
The CoR for inter-observer agreement for inexperiencedobservers was 5.99/5.07 (Session 1/Session 2) compared to4.60/3.95 (Session 1/Session 2) for experienced observers.
3.3. Inter-observer reliability for individual items of theEVGS 
Inexperienced observers were generally less reliable thanexperienced observers for the 17 items in the EVGS.However, the ranking of reliability of individual items wassimilar between the two groups and was generally best forevents occurring at the foot and ankle (Table 2). The mostreliable item was initial contact which, for inexperiencedobservers, showed a kappa of 0.69 and percentageagreement of 80%, while experienced observers showed akappa of 0.94, and percentage of complete agreement of 96%(Table 3). Acomparison ofinter-observer reliabilityforindividual items between inexperienced and experiencedobservers using kappa scores is shown inTable 2. Thecomparison of inter-observer reliability for individual itemsbetween inexperienced and experienced observers usingpercentage of complete agreement is shown inTable 3.
3.4. Validity with 3DGA
There was complete agreement for 52% of the 10numerical items with 3DGA data for inexperiencedobservers compared to 64% for experienced observers(Table 4). The inexperienced observers showed similarlevels of accuracy, ranging from 46.5% complete agreementfortheleastaccurateobserverto55.5%forthemostaccurateobserver. There was also no notable difference in accuracybetween the first and second sessions, with 50.5% meanaccuracy for the first session and 53.3% mean accuracy forthe second session.
4. Discussion
Inexperienced observers achieved moderate intra-obser-ver reliability (mean CoR 5.15) compared with a mean of CoR 4.21 for experienced observers.The CoR represents therange within which 95% of the differences between twomeasures of the same quantity will be expected to lie. Thus adifference of less than or equal to approximately five wouldbe expected when the same inexperienced observer, usingthe EVGS, repeats an observation of a patient on twoseparate occasions. The CoR of inexperienced Observer 2was much higher than those for the other observers. If Observer 2’s results were considered as an outlier andexcluded, the CoR of the inexperienced observers would besimilar to those of the experienced observers, signifyingcomparable reliability. The reason why this observer wasless reliable is not clear but the CoR for the remaining
 A.M.L. Ong et al./Gait & Posture 28 (2008) 323–326 
324Table 1Intra-observer reliabilityInexperienced observers CoR Experienced observers CoRObserver 1 4.90 Observer A 3.63Observer 2 7.27 Observer B 4.20Observer 3 5.93 Observer C 5.93Observer 4 4.77 Observer D 3.29Observer 5 3.79 Observer E 4.00Observer 6 4.24Mean 5.15 Mean 4.21
 
inexperienced observers suggests that specific knowledge of gait was not the dominant factor in making reliableobservations.Inter-observer reliability was also moderate for theinexperienced observers (CoR of 5.99/5.07, Session 1/ Session 2) compared to 4.60/3.95 (Session 1/Session 2) forexperienced observers. Thus, a difference of less than six forinexperienced observers could be expected when a differentobserver repeats an observation of the same patient. Theseresults show that inexperienced observers were reasonablyreliable when observing gait compared to experiencedobservers. The higher reliability of values for individualintra-observer reliability compared with inter-observerreliability suggests that in clinical practice, when the scoreis to be used for comparison within or between subjects, itwould be preferable for these measurements to be made bythe same observer.Both inexperienced and experienced observers showed asimilarrankingforreliabilityoftheitemsoftheScore.Sixof the eight most reliable items for both sets of observers werefor events at the ankle and foot, while there were morevariations with more proximal observations (hip, pelvis, andtrunk). The three least reliable items were also the same forboth sets of observers.Comparison with instrumented 3DGA data showedcomplete agreement for 52% of the ten numerical items
 A.M.L. Ong et al./Gait & Posture 28 (2008) 323–326 
325Table 2Comparison of inter-observer reliability for individual items between inexperienced and experienced observers using kappa scoresRank Inexperienced observers Experienced observersScore item Kappa Strength Score item Kappa Strength1 Initial contact 0.69 Substantial Initial contact 0.94 Almost perfect2 Foot rotation 0.46 Moderate Knee progression angle 0.72 Substantial3 Hindfoot varus/valgus 0.45 Moderate Heel lift 0.62 Substantial4 Knee extn in terminal swing 0.44 Moderate Max ankle dorsi in swing 0.51 Moderate5 Max ankle dorsi in stance 0.41 Moderate Hindfoot varus/valgus 0.43 Moderate6 Heel lift 0.35 Fair Max ankle dorsi in stance 0.38 Fair7 Pelvic rotation in midstance 0.28 Fair Foot rotation 0.36 Fair8 Max ankle dorsi in swing 0.26 Fair Peak hip extn in stance 0.33 Fair9 Peak hip extn in stance 0.22 Fair Max lateral shift of trunk 0.33 Fair10 Knee progression angle 0.22 Fair Peak knee extn in stance 0.33 Fair11 Peak knee extn in stance 0.20 Slight Knee extn in terminal swing 0.32 Fair12 Peak knee flxn in swing 0.14 Slight Peak sagittal trunk position 0.28 Fair13 Peak sagittal trunk position 0.10 Slight Peak knee flxn in swing 0.27 Fair14 Max lateral shift of trunk 0.08 Slight Pelvic rotation 0.25 Fair15 Max pelvic obliquity in midstance 0.07 Slight Clearance in swing 0.22 Fair16 Peak hip flxn in swing 0.06 Slight Max pelvic obliquity in midstance 0.20 Slight17 Clearance in swing
À
0.08 Poor Peak hip flxn in swing 0.17 SlightMax = maximum, dorsi = dorsiflexion, flxn = flexion, extn = extension.Table 3Comparison of inter-observer reliability for individual items betweeninexperienced and experienced observers using percentage of completeagreementScore item Inexperiencedobservers %agreementExperiencedobservers %agreementInitial contact 80.0 96.0Heel lift 59.3 77.0Max ankle dorsi in stance 64.7 66.0Hindfoot varus/valgus 66.7 72.0Foot rotation 68.7 66.0Clearance in swing 63.3 75.0Max ankle dorsi in swing 64.7 75.0Knee progression angle 50.0 82.0Peak knee extn in stance 64.0 70.0Knee extn in terminal swing 73.3 55.0Peak knee flxn in swing 55.3 62.0Peak hip extn in stance 69.3 65.0Peak hip flxn in swing 65.3 75.0Max pelvic obliquity in midstance 58.7 64.0Pelvic rotation 65.3 60.0Peak sagittal trunk position 56.0 69.0Max lateral shift of trunk 62.7 68.0Mean 64.0 70.0Table 4Comparison of the 10 numerical items of the EVGS with instrumented gaitanalysisScore item Inexperiencedobservers %agreementExperiencedobservers %agreement(3) Max ankle dorsi in stance 52 69(5) Foot rotation 56 53(7) Max ankle dorsi in swing 56 83(9) Peak knee extn in stance 36 47(10) Knee extn in terminal swing 62 55(11) Peak knee flxn in swing 29 63(12) Peak hip extn in stance 62 75(13) Peak hip flxn in swing 58 6814) Max pelvic obliquity in midstance 44 55(15) Pelvic rotation 65 69Mean 52 64Max = maximum, dorsi = dorsiflexion, flxn = flexion, extn = extension.
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
You must be to leave a comment.
Submit
Characters: ...