Reliability and Construct Validity of a Practical Evaluation Manipulation Rubric

Background: High velocity low amplitude thrust manipulations apply physical forces to a joint,
over a short period of time, to increase range of motion or decrease pain. Manual therapy rubrics
are essential in assessment of joint manipulation. This study proposed to determine the interrater
reliability and construct validity of a practical rubric.

Methods: Twenty 2nd year DPT students were assessed on performance of distraction
manipulations of the atlanto-occipital, coxofemoral, and talocrural joints by two professors. A
novel rubric was used to assess body mechanics, preload, amplitude and speed of force, and
overall technique. These professors simultaneously graded students in each category utilizing a
likert scale from 0-3. Percent agreement between evaluators was calculated for individual items
and interrater reliability overall score. Construct validity of the rubric was assessed by comparing
professors’ scores of force and rate parameters to true force production utilizing a force sensor.

Results: The rubric demonstrated acceptable, good and poor internal consistency for the
coxofemoral (𝞪 = .793) atlanto-occipital (𝞪 =.80) and talocrural (𝞪 =.545) manipulations
respectively. There was a significant (p<.05) correlation in overall scores between examiners for
atlanto-occipital, coxofemoral, and talocrural joint with a good interrater reliability, ICC(3,1) = .
875, .872 and .878 respectively. Individual item percent agreement between raters ranged from
35% (amplitude) to 100% agreement (safety). Force sensor preload and peak force
measurements for all manipulations were not significantly associated (p>.05) with respective
examiner ratings on the rubric. Force sensor rate of force measurements for atlanto-occipital and
coxofemoral were not significantly associated (p>.05) with examiner ratings on the rubric.
Students with higher talocrural rates of velocity on the BTE were more likely to have higher
ranks on the velocity aspect of the rubric τb = .34, p<.05.

Conclusion: This study supported good reliability and poor to good internal consistency for the
rubric. Construct validity for force parameters was not established. The rubric serves as a reliable
tool for uniform grading and assessment of students’ manipulation performance.Graders may
benefit from further training to improve their ability to evaluate force and rate parameters, thus
increasing validity of the rubric.

References:
1. Boissonnault W, Bryan JM. Thrust joint manipulation clinical education opportunities for
professional degree physical therapy students. J Orthop Sports Phys Ther.
2005;35(7):416-423.
Reliability and Construct Validity of a Practical Evaluation Manipulation Rubric

2. Fisher BE, Piraino A, Lee YY, et al. The Effect of Velocity of Joint Mobilization on
Corticospinal Excitability in Individuals With a History of Ankle Sprain. J Orthop Sports
Phys Ther. 2016;46(7):562-570.
3. Puentedura EJ, Landers MR, Cleland JA, Mintken PE, Huijbregts P, Fernandez-de-Las-
Penas C. Thoracic spine thrust manipulation versus cervical spine thrust manipulation in
patients with acute neck pain: a randomized clinical trial. J Orthop Sports Phys Ther.
2011;41(4):208-220.
4. Estebanez-de-Miguel E, Fortun-Agud M, Jimenez-Del-Barrio S, Caudevilla-Polo S,
Bueno-Gracia E, Tricas-Moreno JM. Comparison of high, medium and low mobilization
forces for increasing range of motion in patients with hip osteoarthritis: A randomized
controlled trial. Musculoskelet Sci Pract. 2018;36:81-86.
5. Ceballos-Laita L, Estebanez-de-Miguel E, Martin-Nieto G, Bueno-Gracia E, Fortun-
Agud M, Jimenez-Del-Barrio S. Effects of non-pharmacological conservative treatment
on pain, range of motion and physical function in patients with mild to moderate hip
osteoarthritis. A systematic review. Complement Ther Med. 2019;42:214-222.
6. Chapman PD, Stomski NJ, Losco B, Walker BF. The simulated early learning of cervical
spine manipulation technique utilising mannequins. Chiropr Man Therap. 2015;23:23.
7. Cuesta-Vargas AI, Williams J. Inertial sensor real-time feedback enhances the learning
of cervical spine manipulation: a prospective study. BMC Med Educ. 2014;14:120.
8. Wise CH, Schenk RJ, Lattanzi JB. A model for teaching and learning spinal thrust
manipulation and its effect on participant confidence in technique performance. J Man
Manip Ther. 2016;24(3):141-150.
9. Lateef F. Simulation-based learning: Just like the real thing. J Emerg Trauma Shock.
2010;3(4):348-352.