Reliability: Consistency or Differentiating Among Subjects? Paul Stratford PHYS THER. 1989; 69:299-300.

The online version of this article, along with updated information and services, can be found online at: http://ptjournal.apta.org/content/69/4/299.2 Collections This article, along with others on similar topics, appears in the following collection(s): Statistics To submit an e-Letter on this article, click here or click on "Submit a response" in the right-hand menu under "Responses" in the online version of this article. Sign up here to receive free e-mail alerts

e-Letters

E-mail alerts

Downloaded from http://ptjournal.apta.org/ at St James's Hospital on August 17, 2012

Letters to the Editor

Letters submitted in response to articles published in Physical Therapy and on issues relevant to physical therapy research, practice, management, or theory will be considered for publication by the Editor. TENS: For Lease or Sale?
To the Editor: In reference to the article by Michael Nolan in the November 1988 issue (pages 1694-1698), I strongly disagree with its statement that transcutaneous electrical nerve stimulation units should not be sold, only leased. The article also supports the use of TENS exclusively in physical therapy facilities. This is definitely an excessively limited and costly way to give a patient pain relief. Also, leasing has potential for abuse because medical equipment companies can make excessive profits. They recoup their expenditures in three months or so and then go on to make much more than the unit is worth. In acute care, for short periods of use, leasing may be acceptable. In chronic pain care, leasing is definitely an abuse. A TENS unit is meant to be worn and used as needed. The potential for harm with a TENS unit is so minute that this whole article is arguable. Medications and surgery, which are "alternatives" for pain care, are far more harmful and costly than is TENS. The "profit-motivated practice of some health care professionals and corporations selling TENS" is far greater in leasing than in selling. A good physical therapist and a good medical supply company will continue to provide support and instruction to patients. I personally think TENS units could be sold without a prescription once people know about them and know how to handle them. The cost would definitely go down. Believe me, patients who don't get relief from TENS would never buy a unit because the wires are indeed a bother. The one-month trial required by Medicare and most insurers I agree with the possible solutions presented in the article; research, paid for by TENS manufacturers, is an excellent idea. We would all enjoy more detailed information about TENS. I also agree that there may be "unauthorized use" by family members, but a TENS unit is far less harmful than the abuse of medications. I hope to see more opinions about the use of TENS. Pauline E Fauser, PT 614 Loveville Rd C4A Hockessin, DE 19707 negates a sale to patients who won't benefit from TENS.

the variance attributed to differences among subjects divided by the total variance.3 A recurring dilemma for readers of physical therapy journals is that authors and editors have frequently contributed to publications that have not differentiated clearly between consistency and the ability to differentiate among subjectsboth known as reliability. For example, a recurring scenario has authors who are apparently interested in consistency using the intraclass correlation coefficient (ICC) as their index of reliability.4 Unfortunately, the ICC is a numerical representation of classical test theory's version of reliability, and it does not directly portray consistency. To illustrate this conflict, consider the contrived data sets, analysis-of-variance tables, and two indexes of reliability presented in the Figure. These data represent two repetitions of knee extensor torque values collected on three sets of five patients. An examination of the data reveals that the among-patient strength values differ the greatest in Data Set 3 (Patient 1, 154 N.m; Patient 2, 20 N.m) and the least in Data Set 2 (Patient 5, 146 N.m; Patient 4, 143 N.m). Likewise, the ICC's representation of reliability is the greatest in Data Set 3 and the lowest in Data Set 2. Of additional interest are the range scores, which reflect the within-patient torque differences (ie, the consistency between repetitions). These scores are identical for Data Sets 1 and 2 and the greatest for Data Set 3. This phenomenon is demonstrated by the standard error of measurement (SEM)*5 leaving one to conclude that the consistency between repetitions is

Reliability: Consistency or Differentiating Among Subjects?

To the Editor: Because reliability is a prerequisite for validity, it is of interest to physical therapists. But what is reliability? Mitchell states, "A reliable instrument is one with small errors of measurement, one that shows stability, consistency, and dependability of scores for individuals on the trait, characteristic or behavior being assessed." 1(p377) This view is concordant wih the opinion offered by Rothstein: "Reliability is basically the consistency of a measurement." 2(p5) There is, however, a second definition of reliability, that offered by classical test theory, that is somewhat different from the statements expressed above. This theory depicts reliability as being a ratio of variances; specifically, it is

*For a single measurement, the SEM is equal to the square root of the error variance term. It is also equal to where t2 is the total variance and R is the ICC. Physical Therapy/Volume 69, Number 4/April 1989
Downloaded from http://ptjournal.apta.org/ at St James's Hospital on August 17, 2012

299/59

represented by the SEM. Of further interest is that the ICC is a unitless quantity, whereas the units for SEM are identical to those of the original measurement. Because neither the ICC nor the SEM are surrogate measures for the other, I suggest that authors be required to indicate whether they are interested in consistency, the ability to differentiate among subjects, or both and that they be compelled to present a numerical index that reflects their purpose. Paul Stratford Teaching Master Dept of Physiotherapy Mohawk College Chedoke Campus Sanatorium Rd PO Box 2034 Hamilton, Ontario, Canada L8N 3T2

Data Set 1
Repetition Patient 1 2 3 4 5 s 1 144 126 139 118 151 135.6 13.4 2 142 129 138 120 149 135.6 11.3 Range 2 3 1 2 2

Analysis of Variance for Data Set 1

Source Among patients Error Total df 4 5 9 SS 1223.4 11.0 1234.4 MS 305.9 2.2 S2 151.9 2.2 154.1 R .986 1.48 SEM

References
1 Mitchell SK: Inter-observer agreement: Reliability and generalizability of data collected in observational studies. Psychol Bull 86:376390, 1979 2 Rothstein JM (ed): Measurement in Physical Therapy: Clinics in Physical Therapy. New York, NY, Churchill Livingstone Inc, 1985, vol 7 3 NunnallyJC: Psychometric Theory, ed 2. New York, NY, McGraw-Hill Book Co, 1978 4 Griffin JW: Differences in elbow flexion torque measured concentrically, eccentrically, and isometrically. Phys Ther 67:1205-1208, 1987 5 Verducci FM: Measurement Concepts in Physical Education. St Louis, MO, C V Mosby Co, 1980

Data Set 2
Repetition Patient 1 2 3 4 5 s 1 144 143 145 143 146 144.2 1.3 2 142 146 144 145 144 144.2 1.5 Range 2 3 1 2 2

Analysis of Variance for Data Set 2

Source Among patients Error Total df 4 5 9 SS 4.6 11.0 15.6 MS 1.2 2.2 S2 0 2.2 2.2 R .0 1.48 SEM

Study Presentation Questioned

To the Editor. I am writing in regard to the article entitled "Changes in Innominate Tilt After Manipulation of the Sacroiliac Joint in Patients with Low Back Pain" (68:13591363, 1988). The authors of this study seem quite knowledgeable about the material presented. I was very impressed with what was stated, but question certain areas of their study. One of the purposes of the study was to propose a method to detect sacroiliac joint dysfunction. The authors did propose a method simply by combining four clinical tests already available. From a diagnostic viewpoint, a clinician should use all available tests and information possible, including radiographs and a case history in diagnosing a patient's condition.

Data Set 3
Repetition Patient 1
2

1 154 20 135 90 50 89.8 56.2

2 148 29 133 96 44 90 52.6

Range 6 9 2 6 6

3 4 5 s

Analysis of Variance for Data Set 3

Source Among patients Error Total df 4 5 9 SS 23610.4 96.5 23706.9 MS 5902.6 19.3 S2 2941.7 19.3 2961.0 R .993 4.39 SEM

Figure. Illustrative data sets (in newton-meters) and analysis of variance. Physical Therapy/Volume 69, Number 4/April 1989

60/300

Downloaded from http://ptjournal.apta.org/ at St James's Hospital on August 17, 2012

Reliability: Consistency or Differentiating Among Subjects? Paul Stratford PHYS THER. 1989; 69:299-300.

Subscription Information

http://ptjournal.apta.org/subscriptions/

Permissions and Reprints http://ptjournal.apta.org/site/misc/terms.xhtml Information for Authors http://ptjournal.apta.org/site/misc/ifora.xhtml

Downloaded from http://ptjournal.apta.org/ at St James's Hospital on August 17, 2012