Available online at www.sciencedirect.

com

Gait & Posture 28 (2008) 170–174

www.elsevier.com/locate/gaitpost

Technical note

Reliability of a method for analyzing three-dimensional

knee kinematics during gait
David R. Labbe a,b,*, Nicola Hagemeister a,b, Mathieu Tremblay a,c, Jacques de Guise a,b
a
Laboratoire de recherche en imagerie et orthopédie, Centre de recherche, Centre hospitalier de l’Université de
Montréal (CHUM), Montréal, Canada
b
École de technologie supérieure, Montréal, Canada
c
Université de Montréal, Montréal, Canada
Received 16 October 2006; received in revised form 19 October 2007; accepted 2 November 2007

Abstract

Background: Different attachment systems have been proposed in an effort to reduce skin movement artifacts when recording knee bone
movement during gait. One such system, called exoskeleton, has shown promising accuracy but little is known concerning its reliability. The
objective of this study was to determine the intra- and inter-observer reliability of this attachment system for recording 3D knee kinematics
during gait.
Methods: Two separate studies were conducted. The intra-observer study involved one observer who reinstalled the exoskeleton on 15
healthy subjects and recorded gait kinematics four times for each subject. The inter-observer study also involved 15 healthy subjects and for
each of these subjects, three observers reinstalled and recorded gait kinetics three times in randomized order.
Findings: In the intra-observer setting, ICC values were 0.92, 0.94 and 0.88 for knee flexion/extension, abduction/adduction and internal/
external tibial rotation, respectively. In the inter-observer setting, the corresponding values were 0.94, 0.92 and 0.89.
Interpretation: The high ICC values found indicate very high reliability of the exoskeleton for recording 3D knee kinematics despite
reinstallation. Moreover, the values between both settings are very similar which indicates that reliability is independent of the observer who
performs the installation. Therefore, evaluations may be carried out by several different clinicians without impacting reliability.
# 2007 Elsevier B.V. All rights reserved.

Keywords: Knee kinematics; Functional method; Validation; Reliability; Gait; Marker attachment system

1. Introduction remains skin displacement artifacts. To remedy this

Different pathologies such as osteoarthritis (OA) and
anterior cruciate ligament (ACL) rupture have been shown
to alter knee kinetics and kinematics during gait [1,2]; this
implies that gait analysis may be used to distinguish between
different pathologies, measure their evolution and evaluate
the value of different treatments, etc. However, a major
obstacle when analyzing three-dimensional knee kinematics
* Corresponding author at: Laboratoire de recherche en imagerie et
orthopédie, Centre de recherche, CHUM, Hôpital Notre-Dame, Pavillon
J.A. de Sève, 1st floor, Room Y-1614, 1560 rue Sherbrooke est, Montréal,
Qué., Canada H2L 4M1. Tel.: +1 514 890 8000x28723;
fax: +1 514 412 7785.
E-mail address: david.labbe.1@ens.etsmtl.ca (D.R. Labbe).
problem, Sati et al. [3] proposed a system for fixing markers
to the bones of the lower extremity in a semi-rigid manner
which is composed of a femoral component, called harness,
and a tibial component.
Using this system, called exoskeleton, the movement of
markers relative to underlying bones is greatly reduced as
shown by fluoroscopy [4]. Reliability of this exoskeleton
during repetitive movements of non-weight-bearing flexion/
extension has been found to be good [5]. However, reliability
when the exoskeleton is removed and reinstalled by one or
more observers has yet to be determined.
The main objective of this study was therefore to evaluate
the reliability of this knee joint analysis system in both intra-
and inter-observer settings.

0966-6362/$ – see front matter # 2007 Elsevier B.V. All rights reserved.
doi:10.1016/j.gaitpost.2007.11.002
 D.R. Labbe et al. / Gait & Posture 28 (2008) 170–174 171

Table 1
Particularities of intra- and inter-observer studies
Intra-observer study Inter-observer study
Number of subjects 15 (12 males, 3 females) 15 (9 males, 6 females)
Age of subjects 30.46 years (21–40 years) 27.13 years (23–26 years)
Kinematics recording device Infrared 3D position sensor system (Optotrak, Northern Digital Inc., 3D digital optical system (Vicon, Oxford
Waterloo, Canada) Metrics Ltd., Oxford, England) at 120 Hz
Kinetics recording device Two force platforms integrated into a treadmill (ADAL-COP, TECMachine, France) allowing for the measurement of
3D ground reaction forces

2. Materials and methods
Two separate studies were conducted in order to verify the
reproducibility of the exoskeleton. The first evaluated the intra-
observer reproducibility and the second, the inter-observer repro-
ducibility; both studies are presented here. Table 1 presents the
particularities of each of them.
All subjects gave their written consent by signing forms
approved by the institutional ethics committees.
3. Knee analysis
In order to follow the movement of the knee bones, the
exoskeleton was installed on the lower limb of the dominant
side. The tibial component consists of a rigid plate which
was placed over the medial face of the tibia just below the
tuberosity (Fig. 1a). This plate was held tightly against the
limb using two Velcro straps placed at each end. The femoral
component (harness) has two orthoplasts which insert of the
medial and lateral faces of the knee area. The lateral
orthoplast inserts between the biceps femoris and the
iliotibial band while the medial one inserts between the
sartorius tendon and the vastus medialis. These orthoplasts
lay atop the femoral epicondyles and are connected by a
rigid arch which applies inward pressure allowing them to
stay in place (Fig. 1a). To the medial orthoplast is attached a
rigid plate with a proximal Velcro strap that attaches around
the thigh to avoid rotation about the pressure points. A third
orthoplast applies stabilizing pressure directly over the
medial femoral epicondyle.
For the purpose of the calibration method, markers were
also fixed directly to the skin over the malleoli (Fig. 1b) and
a marker-mounted belt was tightly apposed around the waist,
directly over the iliac spines (Fig. 1b).
4. Experimental protocol
Each subject was first fitted with a pair of sandals which
were identical for all subjects in order to eliminate variations

Fig. 1. (a) The exoskeleton installed on the right lower limb of a subject. (b) The exoskeleton, pelvic belt and reflective markers installed on a subject.
172 D.R. Labbe et al. / Gait & Posture 28 (2008) 170–174

due to footwear. He/she then walked on the treadmill for a evaluation of reliability. The point of maximum knee flexion
10 min warm-up period at the end of which his/her was also considered. For each of these points, an intra-class
comfortable walking speed was self-determined. correlation coefficient (ICC) was calculated. These coeffi-
cients were averaged, resulting in a mean ICC value for each
4.1. Intra-observer kinematic parameter for both the intra- and inter-observer
settings. The following equations were used:
A single observer participated in this study; he reinstalled
the exoskeleton and performed the calibration method five MSid  MSe
ICCintra-observer ¼ (1)
times on each of the 15 subjects. Established comfortable MSe
walking speeds ranged from 1.0 to 1.3 m/s for all subjects;
three 20 s recordings of gait kinetics and kinematics were MSid  MSe
taken. The recordings were followed by the calibration ICCinter-observer ¼ (2)
MSid
method [6], which was performed with the treadmill
immobilized. The exoskeleton and all the markers were
then removed and the process was repeated four more times Table 2
using the same predetermined walking speed. Intra-observer reliability values
Point Flex/ext Abd/add Int/ext rot.
4.2. Inter-observer 1
ICC 0.93 0.93 0.90
For this study, three observers performed the installation S.E.M. (8) 0.6 0.4 0.8
and calibration of the exoskeleton. The protocol was 2
identical to that of the intra-observer study with the ICC 0.96 0.93 0.79
exception of the number of observers and the number of S.E.M. (8) 0.4 0.5 1.0
reinstallations performed by each of them. Here, the 3
observers reinstalled the exoskeleton twice each in a ICC 0.94 0.93 0.78
predetermined randomized order, for a total of six S.E.M. (8) 0.5 0.5 0.9
installations. The established comfortable walking speeds 4
ranged from 0.92 to 1.22 m/s for all subjects. ICC 0.91 0.94 0.91
S.E.M. (8) 0.8 0.4 0.6
5
5. Data analysis ICC 0.84 0.96 0.91
S.E.M. (8) 1.7 0.6 0.7
Data analysis was performed in the same manner for 6
both the intra- and inter-observer settings. The raw kinetic ICC 0.93 0.94 0.85
S.E.M. (8) 0.6 0.4 0.9
data obtained were filtered using singular spectral analysis
(SSA) [7] and combined with the functional postural (FP) 7
calibration method [6] data using the method described by ICC 0.95 0.94 0.79
S.E.M. (8) 0.6 0.5 1.0
Grood and Suntay [8]. Each set was composed of the three
rotation angles at the knee during gait. These data were 8
ICC 0.90 0.95 0.93
then divided into distinct gait cycles using foot-ground
S.E.M. (8) 0.9 0.4 0.5
contact as the cut off point, and each cycle was then
normalized to 100 sample points representing 100% of 9
ICC 0.93 0.94 0.88
gait cycle. Curves that exceeded two standard deviations S.E.M. (8) 0.6 0.4 0.8
were excluded from further data analysis as they were
10
judged to be the result of a misstep or of bad cycle division.
ICC 0.96 0.94 0.88
Gait cycle data were grouped by installation for further S.E.M. (8) 0.4 0.5 0.8
analysis.
11
ICC 0.84 0.94 0.96
5.1. Reliability S.E.M. (8) 1.3 0.5 0.4
12
To evaluate reliability, 12 points of particular interest ICC 0.97 0.95 0.94
were studied. These points were taken from those proposed S.E.M. (8) 0.3 0.9 0.8
by Chao et al. [9]; they represent local extremums of kinetic
data curves. For each single gait cycle, these points were Mean
ICC 0.92 0.94 0.88
identified on the kinetic data curves and the kinematic value
S.E.M. (8) 0.7 0.5 0.8
found at the corresponding sample point was used for the
 D.R. Labbe et al. / Gait & Posture 28 (2008) 170–174 173

Table 3 6. Results
Inter-observer reliability values
Point Flex/ext Abd/add Int/ext rot. 6.1. Intra-observer
1
ICC 0.94 0.94 0.91 In the intra-observer study, mean ICC values ranged from
S.E.M. (8) 0.4 0.2 0.6
0.88 to 0.94 for knee flexion/extension, abduction/adduction
2 and internal/external tibial rotation. The corresponding
ICC 0.96 0.94 0.89 S.E.M. values for these parameters were found to range from
S.E.M. (8) 0.4 0.4 0.7 0.58 to 0.88. Mean ICC and S.E.M. values across all points
3 are shown in Table 2.
ICC 0.97 0.91 0.92
S.E.M. (8) 0.3 0.3 0.5 6.2. Inter-observer
4
ICC 0.90 0.85 0.84 The data from 12 of the initial 15 subjects was analyzed;
S.E.M. (8) 0.8 0.4 0.7 incomplete optical recordings for three of the subjects
rendered their kinematic data unusable for the current study.
5
ICC 0.92 0.96 0.87 The inter-observer study yielded ICC values ranging from
S.E.M. (8) 0.8 0.5 0.9 0.89 to 0.94 for knee flexion/extension, abduction/adduction
and internal/external tibial rotation. The S.E.M. values
6
varied from 0.48 to 0.78 for those same parameters (Table 3).
ICC 0.94 0.94 0.91
S.E.M. (8) 0.5 0.2 0.7

7 7. Discussion and conclusion

ICC 0.95 0.94 0.90
S.E.M. (8) 0.4 0.3 0.7
Our results show that the system is highly reproducible in
8 both settings. In addition, the differences between the ICC
ICC 0.92 0.91 0.82 values of both settings are very small. Therefore, as far as
S.E.M. (8) 0.8 0.4 0.8
reproducibility is concerned, it makes no difference whether
9 the exoskeleton is reinstalled by a single observer or by
ICC 0.94 0.94 0.91 many different observers. We believe that this finding is
S.E.M. (8) 0.5 0.2 0.7
mainly explained by two factors, the first of which is the
10 design of the harness; it will only stay attached to the limb if
ICC 0.94 0.93 0.90 it is properly positioned. Otherwise, it will fall into place
S.E.M. (8) 0.5 0.4 0.7 when the subject begins to walk or will simply fall off.
11 Secondly, the FP calibration method, which we used, aims to
ICC 0.94 0.77 0.88 limit observer influence by determining joint centers in a
S.E.M. (8) 0.5 0.5 0.6 functional manner [6].
12
Many different authors have investigated the accuracy of
ICC 0.97 0.98 0.98 proposed marker attachment systems [3,4,10,11]. Reliability
S.E.M. (8) 0.3 0.4 0.3 studies have been less common in the literature with some
authors omitting this important step of validation process.
Mean
ICC 0.94 0.92 0.89 Amongst those studies that have focused on reliability, the
S.E.M. (8) 0.5 0.4 0.7 use of different statistical methods impedes direct compar-
ison of results. Kadaba et al. [12] found high reliability for
all three knee rotations within a single day with no
reinstallation of the skin markers. Between day reliability
was poor for knee abduction/adduction and internal/external
where MSid and MSe are respectively subject variability and tibial rotation. Growney et al. [13] found similar results
measurement error (mean square error). using a set of 21 reflective markers although they found
In order to obtain values that can be more readily used to slightly poorer reliability for abduction/adduction and tibial
interpret results, the standard error of measurement (S.E.M.) rotation, both within day and between day. In both previous
was computed from the ICC value for each of the studies, the poor between-day reliability of rotations in the
aforementioned points of interest. The S.E.M. values were frontal and transverse planes was due to a combination of
averaged across all 12 points resulting in a mean S.E.M. marker reinstallation error and physiological variability.
value for each kinematic parameter in both intra- and inter- Our results are in agreement with previous studies which
observer settings. have reported higher reliability for knee rotations in the
174 D.R. Labbe et al. / Gait & Posture 28 (2008) 170–174
sagittal plane than in the frontal and transverse planes.
However, in our case reliability in the frontal and transverse
planes was higher that those reported in the literature.
Studies which included many observers [14,15] found little
or no difference between observers, as did ours. The biggest
factors of variability for all studies seem to be reinstallation
of the markers or marker attachment system and day to day
physiological changes.
In 2005, Hagemeister et al. [6] conducted a study with the
objective of evaluating the intra- and inter-observer
reproducibility of the FP calibration method. In this study,
the authors used the same exoskeleton as we did to fix
markers to the lower limb but no reinstallation of the
attachment system was performed between trials thus the
reliability results reported by Hagemeister apply only to the
calibration method. Because each reinstallation must be
followed by a calibration phase, our results also include the
reproducibility of the calibration method. Thus, the only
additional source of error in our study is that the exoskeleton
is removed and reinstalled for each trial. Given the small
difference between our values and those of Hagemeister
which were obtained from a similar sample and in similar
conditions, it seems that variability between installations,
albeit very small, is caused more by the repetition of the
calibration method than by the actual physical reinstallation
of the exoskeleton.
Overall, our results show that reinstallation of the
exoskeleton has little effect on its reliability; the next step is
to verify its accuracy during gait. The evaluation of the
exoskeleton’s accuracy has been verified during active knee
flexion/extension [4] but also needs to be verified during gait
as load bearing and foot-ground impact may alter the results.
This will therefore likely be the next step in the validation of
the exoskeleton.
Acknowledgements
The authors would like to thank NSERC, CIHR, FQRNT
and Canada Research Chair in 3D Imaging and Biomedical
Engineering for funding.
Conflict of interest
There are two patents pending which are related to
methods used in this work and held by some of the co-
authors. The first is related to the knee analysis system as a
whole and the second is related to the calibration method
which we used.
1. de Guise J.A., Yahia L’H., Duval N., Godbout B., Koller
A., Sati M., Hagemeister N., Parent G., El Maach I. A
System for the Analysis of 3D Kinematics of the Knee.
Provisional patent : USA (# US 10/111 922) and
Europe (# EP 009 725 193), November 2000.
Exploitation rights granted to Solutions YD3.
2. El Maach I., de Guise J.A., Yahia L’H., Hagemeister N.,
Duval N., Koller A., St-Onge N., Anguelov B., Parent G.
Method of Calibration for the Representation of Knee
Kinematics and Harness for Use Therewith. Deposited in
the USA (no. 14604-3USPR), November 2001/ PCT
(international, # CA02/01916), Europe (# 02787249.8 ),
Australia, Canada. Exploitation rights granted to
EMOVI.
None of the authors have other potential financial conflict
of interest.
References
[1] Lafuente R, Belda JM, Sanchez-Lacuesta J, Soler C, Poveda R, Prat J.
Quantitative assessment of gait deviation: contribution to the objective
measurement of disability. Gait Posture 2000;11(3):191–8.
[2] Ferber R, Osternig LR, Woollacott MH, Wasielewski NJ, Lee JH. Gait
mechanics in chronic ACL deficiency and subsequent repair. Clin
Biomech (Bristol Avon) 2002;17(4):274–85.
[3] Sati M, de Guise JA, Larouche S, Drouin G. Improving in vivo knee
kinematic measurements: application to prosthetic ligament analysis.
Knee 1996;3(4):179–90.
[4] Ganjikia S, Duval N, Yahia LH, de Guise J. Three-dimensional knee
analyzer validation by simple fluoroscopic study. Knee 2000;7(4):
221–31.
[5] Hagemeister N, Yahia LH, Duval N, de Guise J. In vivo reproducibility
of a new non-invasive diagnostic tool for three-dimensional knee
evaluation. Knee 1999;6:175–81.
[6] Hagemeister N, Parent G, Van de Putte M, St-Onge N, Duval N, de
Guise J. A reproducible method for studying three-dimensional knee
kinematics. J Biomech 2005;38(9):1926–31.
[7] Alonso FJ, Castillo JM, Pintado P. Application of singular spectrum
analysis to the smoothing of raw kinematic signals. J Biomech
2005;38(5):1085–92.
[8] Grood ES, Suntay WJ. A joint coordinate system for the clinical
description of three-dimensional motions: application to the knee. J
Biomech Eng 1983;105(2):136–44.
[9] Chao EY, Laughman RK, Schneider E, Stauffer RN. Normative data of
knee joint motion and ground reaction forces in adult level walking. J
Biomech 1983;16(3):219–33.
[10] Cappozzo A, Catani F, Croce UD, Leardini A. Position and orientation
in space of bones during movement: anatomical frame definition and
determination. Clin Biomech (Bristol Avon) 1995;10(4):171–8.
[11] Reinschmidt C, van den Bogert AJ, Nigg BM, Lundberg A, Murphy N.
Effect of skin movement on the analysis of skeletal knee joint motion
during running. J Biomech 1997;30(7):729–32.
[12] Kadaba MP, Ramakrishnan HK, Wootten ME, Gainey J, Gorton G,
Cochran GV. Repeatability of kinematic, kinetic, and electromyo-
graphic data in normal adult gait. J Orthop Res 1989;7(6):849–60.
[13] Growney E, Meglan D, Johnson M, Cahalan T, An K-N. Repeated
measures of adult normal walking using a video tracking system. Gait
Posture 1997;6:147–62.
[14] Besier TF, Sturnieks DL, Alderson JA, Lloyd DG. Repeatability of gait
data using a functional hip joint centre and a mean helical knee axis. J
Biomech 2003;36(8):1159–68.
[15] Mills PM, Morrison S, Lloyd DG, Barrett RS. Repeatability of 3D gait
kinematics obtained from an electromagnetic tracking system during
treadmill locomotion. J Biomech 2007;40(7):1504–11.