Scribd
Upload
75 views1 page

Understanding the AC1 Statistic in Depth

This chapter introduces two alternatives to the Kappa statistic for measuring inter-rater reliability: the AC1 and Aickin's α. The AC1 was developed by Gwet to overcome issues with Kappa, particularly the paradox problem. The chapter discusses the probabilistic models underlying AC1 and Aickin's α and presents the formulas for calculating each. It also introduces Gwet's AC2, an extension of AC1 to ordinal and interval ratings data.

Uploaded by

Suci Si Icus
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
75 views1 page

Understanding the AC1 Statistic in Depth

This chapter introduces two alternatives to the Kappa statistic for measuring inter-rater reliability: the AC1 and Aickin's α. The AC1 was developed by Gwet to overcome issues with Kappa, particularly the paradox problem. The chapter discusses the probabilistic models underlying AC1 and Aickin's α and presents the formulas for calculating each. It also introduces Gwet's AC2, an extension of AC1 to ordinal and interval ratings data.

Uploaded by

Suci Si Icus
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd



CHAPTER

4

The AC1 Statistic


OBJECTIVE
This chapter presents two alternatives to Kappa named the AC1 and
Aickins proposed by Gwet (2008a) and Aickin (1990) respectively,
the focus being on AC1 . Both coefficients were developed to overcome
the paradox problem, which has surrounded Kappa with controversies.
The probabilistic models underlying these 2 coefficients are discussed as
well. Also introduced is Gwets AC2 , the extension of the AC1 to ordinal
and interval ratings

CONTENTS
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.2 Gwets AC1 and Aickins Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.2.1 AC1 Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.2.2 Aickins Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3 Aickins Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
I Some Remarks on Aickins Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.1 Aickins Probability Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.2 Estimating from a Subject Sample . . . . . . . . . . . . . . . . . . . . . 67
4.4 Gwets Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.4.1 The Probabilistic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.4.2 Quantifying the Probability P(R) of Selecting an
H-Subject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
I The AC1 Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5 Estimating AC1 for 3 Raters or More, from a Sample
of n Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
I The AC1 Statistic for Multiple Raters . . . . . . . . . . . . . . . . . . . . . . . 73
4.6 AC2 Coefficient for Ordinal and Interval Data . . . . . . . . . . . . . . . . . . 76
4.6.1 AC2 for Interval Data and 2 Raters . . . . . . . . . . . . . . . . . . . . . . 77
4.6.2 AC2 for Interval Data and Multiple Raters . . . . . . . . . . . . . . . . 78
4.7 Weighting the AC1 Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
I AC2 for 2 Raters as a Weighted AC1 . . . . . . . . . . . . . . . . . . . . . . . . . 81
I AC2 for Multiple Raters as a Weighted AC1 . . . . . . . . . . . . . . . . . 81
4.8 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

- 59 -

You might also like