Original Article

Acta Radiologica
0(0) 1–6
An independent inter- and intra-observer ! The Foundation Acta Radiologica
2021
agreement assessment of the Article reuse guidelines:
sagepub.com/journals-permissions
Eno classification of sacroiliac DOI: 10.1177/02841851211029082
journals.sagepub.com/home/acr

joint degeneration

Julio Urrutia1 , Gaston Camino-Willhuber2, Nelson Astur3,

Marcelo Valacco4, Matias Borensztein5, Osvaldo Velan5 and
Pablo Cikutovic6

Abstract
Background: Recently, a scoring system to grade sacroiliac joint (SIJ) degeneration using computed tomography (CT)
scans was described. No independent evaluation has determined the inter- and intra-observer agreement using this
scheme.
Purpose: To perform an independent inter- and intra-observer agreement assessment using the Eno classification and
determining gas in the SIJ.
Material and Methods: We studied 64 patients aged
60 years who were evaluated with abdominal and pelvic
computed tomography scans. Six physicians (three orthopaedic spine surgeons and three musculoskeletal radiologists)
assessed axial images to grade SIJ degeneration into grade 0 (normal), grade 1 (mild degeneration), grade 2 (significant
degeneration), and grade 3 (ankylosis). We also evaluated the agreement assessing the presence of gas in the SIJ. After a
four-week interval, all cases were presented in a random sequence for repeat assessment. We determined the agree-
ment using the kappa (j) or weighted kappa coefficient (wj).
Results: The inter-observer agreement was moderate (wj ¼ 0.50 [0.44–0.56]), without differences among surgeons
(wj ¼ 0.53 [0.45–0.61]) and radiologists (wj ¼ 0.49 [0.42–0.57]). The agreement evaluating the presence of gas was also
moderate (j ¼ 0.45 [0.35–0.54]), but radiologists obtained better agreement (j ¼ 0.61 [0.48–0.72]) than surgeons
(j ¼ 0.29 [0.18–0.39]). The intra-observer agreement using the classification was substantial (wj ¼ 0.79 [0.76–0.82]),
without differences comparing surgeons (wj ¼ 0.75 [0.70–0.80]) and radiologists (wj ¼ 0.83 [0.79–0.87]). The intra-
rater agreement evaluating gas was substantial (j ¼ 0.77 [0.72–0.82]), without differences between surgeons (j ¼ 0.71
[0.63–0.78]) and radiologists (j ¼ 0.84 [0.78–0.90]).
Conclusion: Given the only moderate agreement obtained using the Eno classification, it does not seem adequate to be
used in clinical practice or in research.

Keywords
Sacroiliac joint, joint degeneration, agreement study, intra-articular gas
Date received: 2 February 2021; accepted: 11 June 2021

1
Department of Orthopaedic Surgery, School of Medicine, Pontificia
6
Universidad Catolica de Chile, Santiago, Chile Department of Radiology, School of Medicine, Pontificia Universidad
2
Institute of Orthopedics “Carlos E. Ottolenghi,” Hospital Italiano de Catolica de Chile, Santiago, Chile
Buenos Aires, Buenos Aires, Argentina
3
Santa Casa de Misericordia de S~ao Paulo, Sao Paulo, Brazil and Hospital Corresponding author:
Israelita Albert Einstein, Morumbi, Sao Paulo, Brazil Julio Urrutia, Department of Orthopaedic Surgery, School of Medicine,
4
Hospital Churruca Visca, Buenos Aires, Argentina Pontificia Universidad Catolica de Chile Diagonal Paraguay 362, Santiago,
5
Department of Radiology, Hospital Italiano de Buenos Aires, Buenos Chile.
Aires, Argentina Email: jurrutia@med.puc.cl
2 Acta Radiologica 0(0)
Introduction
Sacroiliac joint (SIJ) degeneration has been considered
a potential cause of low back pain, but the identifica-
tion of symptomatic SIJ is difficult, and a clear associ-
ation between imaging findings and symptoms has not
been established. Some studies have revealed a high
prevalence of SIJ degeneration in adults (1–4), with
an increasing frequency in older individuals, which
may represent a normal aging process of this joint.
While articular degeneration has been extensively eval-
uated for limb joints, SIJ degeneration has received less
attention.
Recently, Eno et al. (1) described a scoring system to
grade SIJ degeneration using computed tomography
(CT) scans. Such a grading scheme considers four cat-
egories: grade 0 ¼ a normal SIJ; grade 1 ¼ mild degen-
erative changes with mild subchondral sclerosis,
minimal osteophyte formation, and subtle joint-space
narrowing; grade 2 ¼ significant degeneration with
large bridging osteophyte formation but without anky-
losis; and grade 3 ¼ SIJ ankylosis. As any scoring
system, this scheme should allow communication
among physicians using it, but it should also help for
decision-making in individual patients, and aid in
research. To achieve those objectives, the grading
system should be easy to apply, comprehensive, and
reproducible among different observers and by the
same observer on different occasions (5). Although
this classification has already been in use in clinical
practice and research (3), no independent evaluation
has been performed to determine the inter- and intra-
observer agreement using this scheme.
Given that clinicians and radiologists can use the
Eno’s grading system (1), an independent inter- and
intra-observer agreement assessment using this grading
scheme should be done by a multicenter panel of clini-
cians and radiologists to determine the classification’s
real value. The aim of the present study was to perform
an independent inter- and intra-observer agreement
evaluation of the system proposed by Eno et al., includ-
ing orthopaedic spine surgeons (OSS) and musculoskel-
etal radiologists (MSR) as assessors. The secondary
aim of the present study, given that the presence of
gas in the SIJ (SIJ vacuum phenomenon [SJVP]) is fre-
quently described in imaging reports evaluating the SIJ,
was to evaluate whether the presence of SJVP was
related to degenerative changes of the SIJ. In addition,
we determined the agreement assessing the presence of
SJVP.
Material and Methods
Institutional review board approval was obtained to
conduct this study; informed consent was waived as
this was a retrospective study with no risk to
participants.
We studied 64 patients aged 60 years and over who
were examined with abdominal and pelvic computed
tomography (CT) scans at a tertiary care university
hospital. Each CT scan was obtained with a multide-
tector CT scanner (GE, Milwaukee, WI, USA). The
CT images were requested for a variety of reasons
that were not related to the spine or the SIJ, including
fever, suspicion of an abdominal or pelvic malignancy
or infection, urolithiasis, and examination of malignan-
cies under treatment. One OSS, who later did not par-
ticipate in the classification phase of this work, selected
the cases from a large database of patients to include
patients with all grades of SIJ degeneration as defined
by the Eno’s scheme. The exclusion criteria were the
presence of a sacral or iliac fracture or tumor, a known
diagnosis of inflammatory disorders, or the presence of
instrumentation in the lumbosacral spine.
The CT scans were evaluated using the Impax
Web3000 program (Agfa-Gevaert, Mortsel, Belgium),
by six physicians from three different centers in three
countries: three fellowship-trained OSS and three
fellowship-trained MSR, all with at least 10 years of
experience as sub-specialist. Axial images were evaluat-
ed on both sides using bone contrast windows to grade
SIJ degeneration according to Eno’s grading system
into Grade 0 (normal joint), Grade 1 (mild degenera-
tive changes with mild subchondral sclerosis, minimal
osteophyte formation, and subtle joint space narrow-
ing), Grade 2 (significant degeneration with large
bridging osteophyte formation but without ankylosis),
and Grade 3 (sacroiliac joint ankylosis) (1) (Fig. 1).
Each SIJ was scored separately; therefore, we evaluated
128 SIJ. Specifically, we looked at the antero-inferior
portion of the joint to determine SIJ degeneration. We
also evaluated the agreement evaluating the presence of
SJVP. At the time of evaluation, the assessors were
blinded to the patients’ clinical and personal data.
The evaluators were trained in this classification
system through an online session to discuss it and to
clarify doubts before performing the assessments;
therefore, we could standardize the evaluation process.
Additionally, they were provided with the original arti-
cle by Eno et al. (1) to solve any doubt at the time of
evaluation.
Sample size estimation was performed with R soft-
ware (The R Project for Statistical Computing, Vienna,
Austria). We used a confidence interval (CI) approach
to sample size estimation for inter-observer agreement
studies with multiple raters as reported by Rotondi
et al. (6). Using a lower limit of 0.61 and an upper
limit of 0.80 (an expected substantial agreement), for
six evaluators and a 95% CI, we calculated that the
required sample size was 64 cases.
Urrutia et al.
Fig. 1. Classification of sacroiliac joint degeneration: (a) Grade 0: normal; (b) Grade 1: mild degenerative changes with mild
subchondral sclerosis, minimal osteophyte formation, and subtle joint space narrowing; (c) Grade 2: significant degeneration with large
bridging osteophyte formation but without ankylosis; and (d) Grade 3: sacroiliac joint ankylosis.
Statistical analyses were conducted using Stata sta-
tistical software, version 13.0 (Stata Corp., College
Station, TX, USA). Considering the classification
described by Eno et al. is an ordinal variable, we
used the weighted kappa statistics (wj) for two-way
agreements. Weighted kappa allows measuring agree-
ment with multiple response levels when not all dis-
agreements are equally important; weight was set
linearly. For the presence of SJVP, we used Cohen’s
kappa (j). Inter-observer agreement was determined
by comparing the initial read of all the assessors.
Intra-observer agreement was calculated by comparing
the same evaluator’s reads between two assessments of
the same patients. The two assessments were separated
by a four-week interval and presented in a random
sequence to avoid recall bias.
Levels of agreement for j and wj were determined
as proposed by Landis et al. (7), as follows: j values
0.00–0.20 ¼ slight agreement; 0.21–0.40 ¼ fair agree-
ment; 0.41–0.60 ¼ moderate agreement; 0.61–0.80 ¼
substantial agreement; and 0.81–1.00 ¼ almost perfect
agreement. All agreements are expressed with 95% CIs.
Additionally, we used the chi-square test to compare
gas prevalence in the SIJ among different SIJ degener-
ation grades.
3
Results
We evaluated 64 patients (27 women; mean
age ¼ 72.34  9.36 years). We performed a total of
768 initial evaluations (128 SIJ by six evaluators); of
these initial evaluations, 107 (14%) were grade 0, 321
(42%) grade 1, 190 (25%) grade 2, and 150 (19%)
grade 3 of the Eno grading system. The evaluators
rated the same grade of SIJ degeneration in both
sides in 82% of the patients. The presence of gas was
observed in 210 of the 768 initial evaluations (27%) of
the evaluated joints. Side-by-side agreement for the
presence of gas was 89%. The presence of gas was
identified significantly more frequently in SIJ rated as
grades 1 and 2 (39% and 33%, respectively) than in SIJ
rated as grades 0 and 3 (12% and 6%, respectively)
(Table 1).
The inter-observer agreement among the six evalua-
tors was moderate (wj ¼ 0.50; 95% CI ¼ 0.44–0.56),
without difference comparing agreement of OSS
(wj ¼ 0.53; 95% CI ¼ 0.45–0.61) and MSR
(wj ¼ 0.49; 95% CI ¼ 0.42–0.57). The agreement eval-
uating the presence of SJVP was also moderate among
observers (j ¼ 0.45; 95% CI ¼ 0.35–0.54), but was sig-
nificantly better for MSR, who obtained a substantial
4 Acta Radiologica 0(0)
Table 1. Presence of gas and sacroiliac joint degeneration
Gas
Eno grade
Absent Present
Grade 0 94 (88) 13 (12)
Grade 1 196 (61) 125 (39)
Grade 2 127 (67) 63 (33)
Grade 3 141 (94) 9 (6)
Values are given as n (%) unless otherwise indicated.
*Significant difference (P < 0.05) in the pairwise comparison of gas frequency between grades.
agreement (j ¼ 0.61; 95% CI ¼ 0.48–0.72), than among
OSS, who had a fair agreement (j ¼ 0.29; 95%
CI ¼ 0.18–0.39).
The intra-rater agreement was substantial
(wj ¼ 0.79; 95% CI ¼ 0.76–0.82). We found no differ-
ences in the intra-observer agreement obtained by OSS
(wj ¼ 0.75; 95% CI ¼ 0.70–0.80) and the intra-observer
agreement reached by MSR (wj ¼ 0.83; 95%
CI ¼ 0.79–0.87). The observation of SJVP also had a
substantial intra-rater agreement (j ¼ 0.77; 95%
CI ¼ 0.72–0.82), with no significant difference between
OSS (j ¼ 0.71; 95% CI ¼ 0.63–0.78) and MSR
(j ¼ 0.84; 95% CI ¼ 0.78–0.90).
Discussion
SIJ degeneration is a common finding in the aging pop-
ulation, but most patients are asymptomatic (1,2).
Nevertheless, SIJ degeneration can be a cause of low
back pain in some patients, and adequate grading of
this degenerative process is required for a correct diag-
nosis. While several imaging tools may provide visual-
ization of the SIJ, CT scan is a widely accepted method
to evaluate its degeneration (1,2,4). Surprisingly, there
is little consensus on the best instrument to grade SIJ
degeneration. Recently, Eno et al. (1) developed a hier-
archical classification to evaluate SIJ degeneration, but
no previous studies have evaluated the agreement using
this grading scheme by different observers, and by the
same observer on separate occasions.
Our study showed a moderate inter-observer agree-
ment and a substantial intra-observer agreement using
this classification by a multicenter, independent panel
of assessors, including OSS and MSR. Like any grad-
ing system, this scheme should facilitate communica-
tion among physicians, standardize research
terminology, and help to guide decision-making in indi-
vidual patients. However, it should be acknowledged
that any attempt to grade a continuous process (such as
SIJ degeneration) into discrete grades (as the Eno clas-
sification does) is limited because, with no objective
boundaries, no classification scheme can perfectly
determine if a specific joint should be classified into a
P value when compared with other grades
Grade 0 Grade 1 Grade 2 Grade 3
<0.01* <0.01* 0.08
<0.01* 0.191 <0.01*
<0.01* 0.191 <0.01*
0.08 <0.01* <0.01*
precise category, or in the next one. Additionally,
ankylosis was not well defined by Eno et al. (1), repre-
senting another limitation of this classification that can
reduce the agreement using it. In fact, Eno et al. did not
clarify whether anterior ankylosis (such as in cases with
diffuse idiopathic skeletal hyperostosis), or an enthe-
seal ankylosis posterior to the SIJ, are also part of
the Grade 3.
In the present study, we decided to evaluate a vari-
able not included in the original classification: the pres-
ence of gas in the SIJ. The SVJP is a commonly
observed phenomenon in degenerated SIJ (8–10); sim-
ilar to using the Eno grading scheme, our panel of
assessors had a moderate inter-observer agreement
and a substantial intra-observer agreement evaluating
the presence of SJVP. Of note, the MSR showed a sig-
nificantly better inter-observer agreement than OSS
assessing the presence of SIJ gas. We believe such a
difference may be explained because MRS frequently
seek gas in multiple joints throughout the body in dif-
ferent clinical conditions (10). However, given our
results, we consider that evaluating the presence of
SVJP does not add value to the Eno classification.
Interestingly, the presence of gas was significantly
more frequent in joints exhibiting grade 1 or 2 degen-
eration than in normal joints (grade 0), or in ankylosed
joints (grade 3), as shown in Table 1. This finding could
be explained because SJVP seems to be a finding of
degenerated SIJ, but it should tend to disappear if a
joint becomes completely ankylosed (11). However,
similar to other characteristics of SIJ degeneration,
there is no clear association of SJVP and symptoms.
It has been widely described that independent stud-
ies generally exhibit a lower inter-observer agreement
than the agreement obtained by the original panel that
developed a classification (12,13). Even though the
original description of the classification was performed
by a group of orthopaedic surgeons (1), we decided to
perform this study with a panel of OSS and MSR, since
such a classification may be useful in the day-to-day
practice of clinicians and radiologists. The moderate
inter-observer agreement using this classification
scheme by the entire panel, and by each subgroup of
Urrutia et al.
specialists, does not reach the j value of 0.55 proposed
as the minimal inter-observer agreement level for a
classification scheme to be useful in clinical practice
(14). However, the panel describing this classification
did not assess their agreement using it; therefore, we
cannot establish a comparison with the authors
describing the scheme.
We obtained a substantial intra-observer agreement
in this study; this result can be explained because the
intra-observer agreement depends on the individual
interpretation of the grading system by each assessor,
which reflects either a consistently correct or a consis-
tently erroneous evaluation by an observer, indepen-
dent of agreement with other assessors. Therefore, a
substantial intra-observer agreement alone is not
enough to support using this classification unless it
demonstrates a better agreement when used by differ-
ent observers.
Our cohort included patients 60 years, and older
who were evaluated with abdomen and pelvic CT
scans independently of the presence of spinal symp-
toms. We decided to include patients in such an age
range to have enough cases with advanced SIJ degen-
eration; although evaluating older patients may be con-
sidered a limitation of our study, 56% of our
evaluations were joints classified as grade 0 and 1.
Therefore, we could obtain enough representation of
all grades of degeneration.
Recently, there has been an increased interest in dif-
ferent invasive treatments to treat presumed sacroiliac
joint pathology, from injection techniques to surgical
options (15). The diagnosis is usually based on symp-
toms that may be non-specific, physical examination,
and degenerative changes observed on CT scans.
However, the association between pain and SIJ degen-
eration on CT scans has not been established.
Comparable to lumbar disc degenerative findings
(16,17), SIJ degeneration is present in many asymptom-
atic subjects and increases with advancing age (1,2,4).
Consequently, caution is required when attributing
symptoms to degeneration observed in CT scans; there-
fore, the clinical use of this classification has yet to be
proven. However, alike the Pfirrmann classification of
disc degeneration, this classification is sometimes used
in imaging reports and in research; hence, an indepen-
dent evaluation of this classification’s capacity to allow
communication among physicians was required.
It has been proposed that a new classification
requires a three-step validation process (18): (i) a defi-
nition of categories by experts; (ii) a multicenter agree-
ment study performed by a representative group of
future users of the classification; and (iii) a prospective
clinical study to evaluate its clinical usefulness. The
present study contributes to this scheme’s validation
process with a multicenter, multi-specialist agreement
5
evaluation performed by potential users of this
classification.
In conclusion, given an only moderate agreement
observed by our panel using this classification, we
believe it does not seem adequate to be used in clinical
practice or in research; only further prospective studies
will reveal if its use is valuable in clinical practice.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
ORCID iD
Julio Urrutia https://orcid.org/0000-0003-1757-1144
References
1. Eno JJ, Boone CR, Bellino MJ, et al. The prevalence of
sacroiliac joint degeneration in asymptomatic adults.
J Bone Joint Surg Am 2015;97:932–936.
2. Vogler JB 3rd, Brown WH, Helms CA, et al. The normal
sacroiliac joint: a CT study of asymptomatic patients.
Radiology 1984;151:433–437.
3. Sato Y, Kashiwabara K, Taniguchi Y, et al. Associated
factors for and progression rate of sacroiliac joint degen-
eration in subjects undergoing comprehensive medical
checkups. Eur Spine J 2020;29:579–585.
4. Shibata Y, Shirai Y, Miyamoto M. The aging process in
the sacroiliac joint: helical computed tomography analy-
sis. J Orthop Sci 2002;7:12–18.
5. Urrutia J, Besa P, Campos M, et al. The Pfirrmann clas-
sification of lumbar intervertebral disc degeneration: an
independent inter- and intra-observer agreement assess-
ment. Eur Spine J 2016; 25: 2728–2733.
6. Rotondi MA, Donner A. A confidence interval approach
to sample size estimation for interobserver agreement
studies with multiple raters and outcomes. J Clin
Epidemiol 2012;65:778–784.
7. Landis JR, Koch GG. The measurement of observer agree-
ment for categorical data. Biometrics 1977;33:159–174.
8. Lo SS, Atceken Z, Carone M, et al. Sacroiliac joint
vacuum phenomenon–underreported finding. Clin
Imaging 2011;35:465–469.
9. Takata Y, Higashino K, Morimoto M, et al. Vacuum
phenomenon of the sacroiliac joint: correlation with sac-
ropelvic morphology. Asian Spine J 2016;10:762–766.
10. Gohil I, Vilensky JA, Weber EC. Vacuum phenomenon:
Clinical relevance. Clin Anat 2014;27:455–462.
11. Backlund J, Clewett Dahl E, Skorpil M. Is CT indicated
in diagnosing sacroiliac joint degeneration? Clin Radiol
2017;72:693.e9–e13.
12. Urrutia J, Zamora T, Yurac R, et al. An independent
interobserver reliability and intraobserver reproducibility
6 Acta Radiologica 0(0)
evaluation of the new AOSpine Thoracolumbar Spine
Injury Classification System. Spine (Phila Pa 1976)
2015; 40 :E54–58.
13. Urrutia J, Zamora T, Yurac R, et al. An Independent
Inter- and Intraobserver Agreement Evaluation of the
AOSpine Subaxial Cervical Spine Injury Classification
System. Spine (Phila Pa 1976) 2017; 42: 298–303.
14. Sanders R. The problem with apples and oranges.
J Orthop Trauma 1997;11:465–466.
15. Ou-Yang DC, York PJ, Kleck CJ, et al. Diagnosis and
management of sacroiliac joint dysfunction. J Bone Joint
Surg Am 2017;99:2027–2036.
16. Boden SD, Davis DO, Dina TS, et al. Abnormal
magnetic-resonance scans of the lumbar spine in asymp-
tomatic subjects. A prospective investigation. J Bone
Joint Surg Am 1990;72:403–408.
17. Jensen MC, Brant-Zawadzki MN, Obuchowski N, et al.
Magnetic resonance imaging of the lumbar spine in
people without back pain. N Engl J Med 1994;331:69–73.
18. Audige L, Bhandari M, Hanson B, et al. A concept for
the validation of fracture classifications. J Orthop
Trauma 2005;19:401–406.