# Welcome back

## Find a book, put up your feet, stay awhile

Sign in with Facebook

Sorry, we are unable to log you in via Facebook at this time. Please try again later.

or

Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more

Download

Standard view

Full view

of .

Look up keyword

Like this

Share on social networks

1Activity

×

0 of .

Results for: No results containing your search query

P. 1

STATISTICAL TECHNIQUES FOR COMPARING MEASURERS AND
METHODS OF MEASUREMENT: A CRITICAL REVIEWRatings: (0)|Views: 22|Likes: 0

Published by jp134711

John Ludbrook. Clin Exp Pharm and Phys, (2002) 29, 527-536

John Ludbrook. Clin Exp Pharm and Phys, (2002) 29, 527-536

See more

See less

https://www.scribd.com/doc/156971203/STATISTICAL-TECHNIQUES-FOR-COMPARING-MEASURERS-AND-METHODS-OF-MEASUREMENT-A-CRITICAL-REVIEW

11/11/2013

text

original

SUMMARY

1.Clinical and experimental pharmacologists and physio-logists often wish to compare two methods of measurement, ortwo measurers.2.Biostatisticians insist that what should be sought is notagreement between methods ormeasurers, but disagreement orbias.3.If measurements have been made on a continuous scale,the main choice is between the Altman–Bland method of differ-ences and least products regression analysis. It is argued thatalthough the formeris relatively simple to execute, it does notdistinguish adequately between ﬁxed and proportional bias.Least products regression analysis, although more difﬁcult toexecute, does achieve this goal. There is almost universal agree-ment among biostatisticians that the Pearson product–momentcorrelation coefﬁcient (

r

) is valueless as a test forbias.4.If measurements have been made on a categorical scale,unordered orordered, the most popularmethod of analysis isto use the kappa statistic. If the categories are unordered, theunweighted kappa statistic (K) is appropriate. If the categoriesare ordered, as they are in most rating scales in clinical, psycho-logical and epidemiological research, the weighted kappastatistic (K

w

) is preferable. But K

w

corresponds to the intraclasscorrelation coefﬁcient, which, like

r

forcontinuous variables, isincapable of detecting bias. Simple techniques fordetecting biasin the case of ordered categorical variables are described andcommended to investigators.Key words: categorical variables, continuous variables,correlation, ﬁxed bias, kappa statistic, least products regressionanalysis, limits of agreement, log-linearmodelling, McNemartest, method of differences, proportional bias.

INTRODUCTION

Biomedical investigators often wish to compare two methods of measurement, usually to compare a new method with an establishedone. It is an important prerequisite for such studies that the sameindividual must make the measurements or ratings. Alternatively,investigators may wish to compare the performances of twomeasurers or raters who are using the same method of measurement,or to evaluate the repeatability of measurements made by thesame observer.In clinical and laboratory biomedical science, measurements of a variable are usually made on a continuous scale. Examples of thisare measurements of blood pressure, blood gases, lung function andplasma concentrations of a variety of endogenous or exogenoussubstances. In contrast, categorical scales are used to describe orscore attributes by epidemiologists, social scientists, clinicians and,occasionally, laboratory scientists. These categorical scales may beunordered, as in the description of eye colour or taste, or they maybe ordered, as in rating scales for cardiovascular functional status,operative (anaesthetic) risk, severity of stroke and any number of

ad hoc

scales. Somewhere in between are quasi-continuous scales,such as indices of disability, quality of life and so forth, when therange of the scales can be between 20 and 42.Whatever the scale of measurement, there is an unusual consensusamong biostatisticians that the goal of making these comparisonsshould not be to demonstrate agreement, but to detect disagreementor bias. However, biostatisticians disagree, sometimes sharply, onhow best to achieve this goal. The purpose of the present review isto describe some of the statistical techniques that can be used todetect bias and to evaluate them critically.

CONTINUOUS VARIABLES

This heading is shorthand for variables that are measured on acontinuous, or interval, scale. These measurements include distance,weight, concentration, pressure, velocity, temperature, age and soforth. This section deals with three techniques for comparingmethods of measurement: (i) regression analysis; (ii) the methodof differences; and (iii) correlation.

Detecting bias by regression analysis

As a rule, the values obtained by one method of measurement arelinearly related to those obtained by another. It seems to followlogically, therefore, that linear regression analysis would be auseful tool for comparing methods of measurement. But whatsort of linear regression analysis?The familiar form is least squares of

y

regression analysis,commonly known as ordinary least squares (OLS) regression. Thisis what is provided by most computer statistical programs. But, forcomparing two methods of measurement, this is the wrong modelon two counts. First, under statistical theory, it is an assumption of

SPECIALARTICLE

STATISTICALTECHNIQUES FOR COMPARING MEASURERS ANDMETHODS OFMEASUREMENT: ACRITICALREVIEW

John Ludbrook

The University of Melbourne, Parkville, Victoria, Australia

Correspondence: Dr John Ludbrook, 563 Canning Street, Carlton North,Victoria 3054, Australia. Email: johnludbrook@bigpond.comReceived 15 December 2001; revision 18 February 2002; accepted20 February 2002.

Clinical and Experimental Pharmacology and Physiology

(2002)

29

, 527–536

528

J Ludbrook

OLS regression that whereas the values of the

y

variable (predicted)are attended by random error, the values of the

x

variable (predictor)are ﬁxed in advance and without random error. It is obvious thatthis assumption is rarely, if ever, fulﬁlled in method comparisonstudies. Second, by convention, in OLS regression the values of the

y

variable are regarded as ‘true’, the ‘gold standard’or the ‘bench-mark’. Yet, when two methods are compared, it is usual thatneither can be regarded as a benchmark. In this context, it is aproperty of OLS regression analysis that the line resulting fromminimizing the sums of the squares of the deviations of the

y

values from the line and that resulting from minimizing the sumsof the squares of the deviations of the

x

values from the line, aredistinctly and sometimes markedly different.

1

There is an alternative to the OLS regression model. It isordinary least products (OLP) regression analysis, whose proper-ties and method of execution have been described in detail else-where.

1

Least products regression analysis allows for both the

y

andthe

x

values to be attended by random error. It depends on mini-mizing the sum of the products of the deviations of both

x

and

y

values from the estimated regression line. It requires no judgementthat the

y

variable or the

x

variable provides ‘true’or ‘benchmark’values. Instead, it provides a technique for interchanging two meth-

Fig.1

Data points and ordinary least products regression lines for four hypothetical comparisons of Methods Aand B for measuring systolic bloodpresssure. (a) No bias; (b) proportional bias; (c) ﬁxed bias; (d) proportional and ﬁxed bias. Data points are given in Appendix 1. Details are given inTable1. Regression model: E(A)=

a

+

b

(B). Coefﬁcients for the four regression lines are given in Table1. Proportional bias: 95% conﬁdence interval(CI) for slope (

b

) does not include 1. Fixed bias: 95% CI for intercept (

a

) does not include 0. In all four cases, Pearson’s

r

=0.939.

Review of statistical methods

529

ods, or calibrating one against the other. It detects proportional andﬁxed bias (or both) between two methods.There is a drawback to OLPregression analysis: whereas OLSregression analysis can be executed by a handheld calculator or evenwith a pencil and paper, OLPregression analysis is an iterative pro-cess that requires a computer program that can execute non-linearregression analysis by way of a loss function.

1

One assumption thatunderlies OLP(and OLS) regression analysis is that the scatter of values around the regression line is constant over the whole rangeof

y

(and

x

) values. In biological work, it is common that thescatter increases with the level of

y

(and

x

). In this circumstance,weighted least products (WLP) regression analysis can be used.

1

Butthis, too, is catered for by most computer statistics programs.

Detecting bias by the method of differences

The notion of examining the differences between the results of two methods of measurement was conceived by Altman and Bland

2

and has recently been elaborated by the same authors.

3

Theirproposal is that the differences should be plotted against themeans. If the OLS regression line ﬁtted to the plot has a slope(

b

) that differs ‘signiﬁcantly’from 0, then it is argued that propor-tional bias exists. If the mean value for the difference (

d

–) differs‘signiﬁcantly’from 0 on the basis of a one-sample

t

-test, then it isargued that there is ﬁxed (or, in their words,

2,3

relative) bias. Altmanhas been a strong and persistent advocate for making inferencesin medical research by using estimation by means of conﬁdenceintervals (CI) rather than using

P

values resulting from hypothesistesting.

4

The CI technique can be applied to the analysis of differences.

3

The slope of the regression of differences on means is a satis-factory method for detecting proportional bias. Although OLSregression analysis is used by Altman and Bland to estimate slope,this can be defended by the argument that differences must alwaysbe the dependent (predicted) variable, means never. There is, how-ever, a serious ﬂaw in detecting ﬁxed bias by the method of differ-ences.

1

It is that if there is proportional bias (

b

϶

0), then the meandifference (

d

–) will almost inevitably deviate from zero. Thus, thereis the risk that ﬁxed bias will be overdiagnosed. The exception tothis is if proportional bias is in one direction (e.g.

b

>0) and ﬁxedbias in the opposite direction. Then

d

–may be close to 0 and ﬁxedbias will be underdiagnosed.

Quantifying disagreement between methods of measurement

Bland and Altman make the important point that the mere absenceof bias often does not provide sufﬁcient information to allow a judge-ment that one method can, or cannot, be substituted for the other.

3

The ability to make this judgement may be very important inclinical practice. They suggest that a useful aid to making this judgement is to calculate what they call the 95% limits of agree-ment. Their formula for this is:

d

–±

z

2

␣

s

d

[1]where

d

–is the mean difference between methods,

s

d

is the standarddeviation of the difference between methods and

z

2

␣

is thestandardized normal deviate corresponding to two-sided

P

=0.05(1.960).Abetter, somewhat more conservative, formulation is safer in thecase of small samples, for instance

n

<100.

5

It is:

d

–±

t

n–1,2

␣

s

d

Ί

(1+1/n)[2]where

d

–is the mean difference between methods,

s

d

is the standarddeviation of the difference between methods,

t

n–1,2

␣

is the value of

t

corresponding to two-sided

P

=0.05 for d.f.=n–1 and

Ί

(1+1/n)is an adjustment for small sample size.Equations 1 and 2 are more commonly known as 95% tolerancelimits for the population.

5

That is, it is predicted that 95% of values in the parent population of differences will fall withinthese limits.

Hypothetical example of a continuous variable

Imagine that four different studies have been conducted in whichone indirect method for measuring systolic blood pressure has beencompared with another. One method (B) is common to all studies.The other four methods are coded A1, A2, A3 and A4. The dataresulting from these studies, each in 26 subjects, are provided inAppendix 1.These four studies were analysed ﬁrst by ordinary least products(OLP) regression analysis.

1

The outcomes are given in Fig.1 andTable1. They are described as indicating no bias, proportional bias,ﬁxed bias or both proportional and ﬁxed bias on the basis of the 95% CI attached to the OLPregression coefﬁcients (Fig.1;Table1).The four hypothetical studies were then re-analysed by theAltman–Bland method of differences.

2,3

The results of theseanalyses are presented in Fig.2 and Tables2,3. Proportional biasis indicated if the slope (

b

A.B

) of the OLS regression of differenceson means differs ‘signiﬁcantly’from 0 (

P

Յ

0.05) or, equivalently,if the 95% CI for

b

A.B

does not include 0. Fixed bias is indicatedif the difference (

d

–) differs ‘signiﬁcantly’from 0 (

P

Յ

0.05) byone-sample

t

-test or, equivalently, if the 95% CI for

d

–does notinclude 0. In Fig.1b and Table1b, OLPregression analysis indicatesthat there is proportional, but not ﬁxed, bias. However, in Fig.2b

Table1

Outcome of analyses by ordinary least products regressionProportional

ra

95% CI

b

95% CIProportional biasFixed bias(a) A1B0.939–8.8–32.6, 15.01.0560.900, 1.211NoNo(b) A2B0.939–7.0–26.0, 12.00.8440.720, 0.969YesNo(c) A3B0.93924.20.4, 48.01.0560.900, 1.211NoYes(d) A4B0.93919.40.4, 38.40.8440.720, 0.969YesYesFor data used in regressions, see Appendix 1.

r

, product–moment correlation coefﬁcient.

a

,

b

, coefﬁcients in ordinary least products regression model E(A)=

a

+

b

(B);

a

, A(

y

axis) intercept;

b

, slope;proportional bias, if 95% conﬁdence interval (CI) for

b

does not include 1; ﬁxed bias, if 95% CI for

a

does not include 0.See Fig.1 for graphical display.

- Read and print without ads
- Download to keep your version
- Edit, email or read offline

© Copyright 2015 Scribd Inc.

Language

Choose the language in which you want to experience Scribd:

Sign in with Facebook

Sorry, we are unable to log you in via Facebook at this time. Please try again later.

or

Password Reset Email Sent

Join with Facebook

Sorry, we are unable to log you in via Facebook at this time. Please try again later.

or

By joining, you agree to our

read free for two weeks

Unlimited access to more than

one million books

one million books

Personalized recommendations

based on books you love

based on books you love

Syncing across all your devices

Join with Facebook

or Join with emailSorry, we are unable to log you in via Facebook at this time. Please try again later.

Already a member? Sign in.

By joining, you agree to our

to download

Unlimited access to more than

one million books

one million books

Personalized recommendations

based on books you love

based on books you love

Syncing across all your devices

Continue with Facebook

Sign inJoin with emailSorry, we are unable to log you in via Facebook at this time. Please try again later.

By joining, you agree to our

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

CANCEL

OK

You've been reading!

NO, THANKS

OK

scribd