Title - Inter and Intra Observer Agreement in Measuring Urolithiasis Density On Non Enhanced Computed Tomography

Journal of Endourology
Page 1 of 22
© Mary Ann Liebert, Inc.

DOI: 10.1089/end.2019.0811
1
Title: Inter‐ and Intra‐observer agreement in Measuring Urolithiasis
Density on Non‐Enhanced Computed Tomography
This paper has been peer‐reviewed and accepted for publication, but has yet to undergo copyediting and proof correction. The final published version may differ from this proof.
Running title: The Reliability of Measuring Urolithiasis Density

Inter‐ and Intra‐observer agreement in Measuring Urolithiasis Density on Non‐Enhanced Computed Tomography (DOI: 10.1089/end.2019.0811)
Authors:
G.W. Disselhorst, M.D.1 (gwdisselhorst@gmail.com)
J.J. Reimerink, M.D. Ph.D.2 (j.reimerink@olvg.nl)
Downloaded by GRIFFITH UNIVERSITY from www.liebertpub.com at 02/14/20. For personal use only.
R.K. van der Vijgh, M.D.1 (rvijgh@gmail.com)
R. Sandkuyl, M.D.1 (rutgersandkuyl@gmail.com)
P.C. Kauer M.D.1 (p.c.kauer@olvg.nl)
R. Reinhard, M.D.2 (r.reinhard@olvg.nl)
M.M. Tjeenk Willink, M.D.2 (mmtjeenkwillink@gmail.com)
S. Jensch M.D. Ph.D.2 (s.jensch@olvg.nl)
E.P. van Haarst, M.D. Ph.D.1 (e.vanhaarst@olvg.nl)
Affiliation:
1. Department of Urology, OLVG, Amsterdam
2. Department of Radiology, OLVG, Amsterdam


Keywords:
Tomography.
G.W. Disselhorst, M.D.
Corresponding author:
1061 AE Amsterdam, The Netherlands.
T +31638132541; gwdisselhorst@gmail.com
Department of Urology, OLVG, Jan Tooropstraat 164,
Stone Density; Hounsfield Units; Observer Variation; Urinary Calculi; X‐Ray Computed
2
Page 2 of 22
Page 3 of 22

3
Abstract
Introduction: Treatment choice for urolithiasis is partially based on measuring stone
density in Hounsfield Units (HU) on Non‐Enhanced Computed Tomography (NECT). Inter‐
observer variability in these measurements could have treatment consequences. This
study aims to assess the observer agreement of measuring HU and whether the use of a
protocol leads to a better agreement.
Materials and Methods: We retrospectively included 155 consecutive NECTs of patients
with stones ≥4 mm. Five observers (two radiologists, one urologist, one urology resident,
and one radiology resident) assessed all anonymized NECTs four times in randomized
order.
HU was measured without instruction (rounds 1;2) and subsequently using two protocols
(A;B, rounds 3;4). Protocols comprised of using bone setting, using zoom, and measuring
HU without the penumbra, in either three (A) or one (B) axial plane. The inter‐ and intra‐
observer agreement were evaluated using the Intraclass Correlation Coefficient (ICC).
Results: Inter‐observer agreement on HU measurement without protocol was: ICC=0.84
(CI‐0.79‐0.87). Agreement diminished with protocol A to ICC=0.62 (CI‐0.37‐0.76) and
improved with protocol B to ICC=0.90 (CI‐0.86‐0.92). Intra‐observer agreement without
protocol was ICC=0.87, with protocol A ICC=0.87, and with protocol B ICC=0.93. The
biggest improvement was seen for urologists’ agreement from no protocol to protocol B,
where ICC improved from 0.81 (CI‐0.70‐0.87) to 0.91 (CI‐0.84‐0.94).
Conclusions: Observer agreement of HU measurement of urolithiasis without protocol is
already good but using zoom, bone setting, and measuring in a representative plane is
recommended. This protocol results in higher agreement, especially among urologists.
Measuring in 3 axial planes does not increase agreement.
Page 4 of 22

4
1. Introduction
The worldwide incidence and prevalence of urolithiasis has increased over the last few
decades.1 Extracorporeal Shock Wave Lithotripsy (SWL) is one of the preferred treatment
options for small renal and ureteral stones, due to its acceptable success rate and low
complication rate.2.
Several patient and stone characteristics have been studied to predict the chance of
successful disintegration after SWL. Stone size, location, density, and volume, and skin‐to‐
stone distance have shown to be correlated with the outcome of SWL.3,4,13–20,5–12 The gold
standard used to detect urolithiasis is Non‐Enhanced Computed Tomography (NECT) with
high sensitivity and specificity.21
The association of stone density before the procedure with success of SWL has been
studied with varying results. The density of a stone is assessed by measuring the
Hounsfield Units (HU) on NECT. Reported HU‐thresholds for successful SWL vary between
593 and 1000 HU.3,6–15,19 The American (AUA) and European (EAU) Urology guidelines on
urolithiasis recommend to use a different treatment than SWL for stones of 1000 HU or
more.2,22 Inter‐observer variation is present, which may lead to a different treatment and,
thus, have clinical consequences for the patient.
The aim of this study is to assess the inter‐ and intra‐observer agreement of measuring
HU value in urolithiasis and whether using a protocol will improve inter‐ and intra‐observer
agreement.
2. Materials and Methods
We retrospectively identified all consecutive low‐dose NECT scans made of patients
suspected to have urolithiasis in our institution between 01‐01‐2014 and 08‐08‐2016
(n=1006). Patients were identified from the Picture Archiving and Communication System
(PACS) using specific radiology department treatment codes.
Two of the authors who did not participate in the measurements reviewed all CT scans
and included 160 scans where a renal or ureteral stone with an axial diameter of 4 mm or
greater was present. The inclusion criteria were set predominantly to incorporate stones
Page 5 of 22

5
that are suitable for SWL and whose chance for spontaneous passage was low. The stone
had to be measurable on at least two axial slices in bone setting on NECT. CT scans
containing smaller or no stones, multiple stones close to each other, stones next to a
ureteral stent, or staghorn stones were excluded.
See addendum 1 for specific information about the CT scanner and imaging techniques
being used. In order to guarantee blinded assessment, all patient data was removed from
the scans. The local ethics board committee approved this retrospective diagnostic study
(ACWO: 16‐039). No informed consent was deemed necessary because of the
retrospective use of anonymized NECT scans.
Measuring protocol
Five observers independently assessed all 160 CT scans four times in four separate
rounds. In all rounds, all observers measured HU and diameter of the stones. To reflect
clinical practise, the group of observers consisted of two abdominal radiologists, one
urologist specialized in endourology, a senior urology resident, and a senior radiology
resident.
In rounds one and two, all observers were asked to perform measurements as normal,
without instructions in advance. After the initial two rounds, the observers were informed
of two measurement protocols during a consensus meeting. In rounds three and four, all
observers measured HU and diameter using these two predetermined measuring
protocols. The protocols were designed by the authors who did not participate in the
measurements.
The first protocol (A) was based on the measuring protocol described by El‐Nahas et al.
(2007), because the results of that study informed the 1000 HU threshold used in several
guidelines.2,14,22 The protocol comprised measuring HU in bone setting, using zoom (at
least 300%) and drawing a region of interest (ROI) just smaller than the stone on three
different axial planes (cranial, middle, and caudal).
The second “modified” protocol (B) instructed observers to report the single most
representative HU value and plane, keeping the other measuring techniques identical.
Page 6 of 22

6
All measurements were performed on a dedicated diagnostic radiology workstation
with high‐resolution monitors. All scans were randomized between rounds. A minimum
one‐week interval was used to minimize the risk of recall bias.
Outcome measures
The inter‐observer agreement on HU measurement between all five observers, as well
as the intra‐observer agreement for each observer between their own repeated
measurements, were evaluated using the Intraclass Correlation Coefficient (ICC, two way
random, absolute agreement, single measures). For interpreting the ICC, reference values
of Koo and Li were used.23
Inter‐ and intra‐observer agreement was calculated from the first two rounds (no
protocol) and compared to agreement of the last two rounds (protocol A; B) to detect
differences after adhering to a measuring protocol. For protocol A, the mean HU value of
the three different measurements in three planes was used.
Power‐analysis, Statistics
An a priori power analysis was performed to estimate the number of scans needed to
significantly detect an improvement in ICC of 10% (R package: ICC.Sample.Size). The
expected improvement of 10% was chosen as we deemed this effect to be clinically
relevant to introduce a new protocol in clinical practice. The α‐error level was fixed at
0.05; the power was set at 80%; at least two ratings per subject. The optimum sample size
was calculated to be 149 CT scans in order to significantly detect a difference of 10% in the
ICC (p0=0.7, p=0.8). For certainty reasons, 11 extra scans were included, making up a total
of 160 CT scans.
All statistical analyses were performed using R (version 3.3.2, R Foundation, Vienna,
Austria).
3. Results
Baseline characteristics
Due to incomplete data provided by the observers, five CT scans were excluded from
analysis. Characteristics of 155 stones of 143 patients are presented in Table 1.
Page 7 of 22

7
Inter‐observer Agreement
Table 2 shows the inter‐observer agreement, as well as data divided for radiologists and
urologists.
The inter‐observer agreement on measuring HU value without the use of a measuring
protocol was good (ICC=0.84). The inter‐observer agreement was moderate using protocol
A (ICC=0.62), but was excellent using protocol B (ICC=0.90). Urologists, especially, showed
a higher agreement using protocol B compared to measurement without protocol.
Intra‐observer Agreement
In Table 3 the intra‐observer agreement is shown. The intra‐observer agreement for HU
measurements without measuring protocol was good (ICC=0.87). Protocol A did not lead
to a different intra‐observer agreement. The intra‐observer agreement using protocol B
was excellent (ICC=0.93).
4. Discussion
The association of HU value and successfulness of SWL has been reported in several
studies.3,4,6–17,19,20 The measured HU value is therefore an important parameter and a good
pre‐operative predictor of stone density. However, the majority of these studies do not
report uniform instructions on the method of measuring HU.
The aim of this study was to assess the current agreement of measured HU value in
urolithiasis and whether using a measuring‐protocol could improve agreement. A good
inter‐observer agreement when measuring HU without protocol (ICC=0.84) was observed,
but using protocol B (using zoom, using bone window, and measuring the stone without
measuring the penumbra in a single representative axial plane) improved agreement to
excellent (ICC=0.90). Measuring in three axial planes did not lead to a better agreement in
this study (ICC=0.62) (protocol A).
A previous comparable study by Okhunov et al. (2012), also investigated the inter‐
observer agreement of the measurement of stone density (HU).24 Four observers (two
urologists and two urology residents), reviewing 58 scans, were instructed to measure HU
Page 8 of 22

8
“drawing a line surrounding as much of the stone as possible while minimizing the
adjacent soft tissue”. Given current clinical practices, we believe that most observers do
not measure HU by following these specific instructions. The results in our study of
measurement without following a protocol might be a better reflection of agreement in
current clinical practice. The measuring protocol from Okhunov et al. has the most
resemblance to our protocol B but without the explicit use of bone setting. In their study,
an ICC of 0.91 (CI‐0.87‐0.94) was observed, comparable to the agreement using protocol B
in our study.
The HU threshold of 1000 used in the AUA and EAU guidelines is based on the study by
El‐Nahas et al. (2007).2,14,22 In this study, HU is measured by drawing a ROI on three
different axial planes. We chose this protocol as a point of comparison to the normal mode
of measurement (without protocol) because El‐Nahas et al. could be considered a
landmark study. Another argument for measuring different axial planes is that a mean HU
of these three planes might be more reliable, as stones are often irregular in shape and
composition, reflecting in a variation in HU values. Nonetheless, because measuring on
different planes is more time‐consuming, it is important to know if this does result in a
higher agreement.
Contrary to our initial expectations, we found a lower agreement using protocol A
compared to agreement without a protocol. A possible explanation might be that some of
the stones included were too small to measure in three axial planes. In sub‐analysis, the
agreement of the most cranial and caudal measurement was done separately, showing
even worse agreement (Table 4). This suggests that the lower agreement found with
protocol A is due to these cranial and caudal measurements. This might be even more
pronounced in smaller stones because of the difficulty to choose three axial planes.
One can imagine that zooming and bone setting increases the ability to distinguish the
actual stone from the penumbra around it. This will, in turn, lead to more precise placing
of the region of interest, ultimately leading to a more accurate measurement and
agreement. The alternative protocol with the same measuring method was also evaluated
(protocol B) because we anticipated that some of the stones would be difficult to measure
Page 9 of 22

9
in three planes. The observed higher agreement resulting from using protocol B is most
likely a better representation of the increase in accuracy of measurements following the
instructions.
The intra‐observer variability was less affected by the use of protocol compared to the
inter‐observer variability. The highest increase of agreement was seen for the urologists
and for the radiology resident. As radiologists have more experience in assessing scans it
seems likely that this reflects in the higher agreement seen among them, independent of
the method of measurement. Our results show that especially urologists benefit from
following a protocol, leading to a more uniform measurement with better agreement
throughout the entire group of observers. This is a relevant finding as a lot of clinicians
may measure HU themselves before choosing a treatment, and might therefore be the
group with the largest clinical consequences.
This study was designed to justify changing measurement methods to three planes, and
we postulated that an improvement of 0.10 in ICC would validate such a change. As such,
our study was powered to distinguish this significant improvement. Our results show that
for the entire group of observers, the agreement was already quite high and only
improved a total of 0.07 point ICC. However, the literature suggests that when important
decisions are made based on the ICC value, this value should be at least 0.90 to 0.95.25,26
This is achieved when using protocol B, but not without following a protocol. One could
argue that even though the difference is less than 0.10, it is still relevant to validate
measuring according to our proposed protocol (B).
It is important to note that the HU value, as measured with single‐energy CT, is an
approximation of the actual stone density. In our hospital, we use a single‐energy scanning
protocol, as opposed to newer dual‐energy protocols, to predict stone composition and
density.27 Although it might not be a perfect correlation, the guidelines are based on stone
density as assessed with single‐energy HU values.
Furthermore, patient characteristics such as body‐mass‐index could influence scanning
parameters, tube voltage, signal‐to‐contrast and noise‐to‐contrast ratio, and possibly HU
value.28 In our study, all patients were scanned using the same protocol and fixed voltage,
Page 10 of 22

10
minimizing these variations. Only the amount of noise could slightly influence the spread
of HU measurements.29 However, these variations are innate to a consecutive population
and also occur in the clinical setting.
To assess agreement on SWL suitability additional analysis using Kappa statistics was
carried out. The continuous variable HU was rescaled to a binary: <1000 HU; ≥1000 HU.
This cut‐off point is used in the current guidelines to decide to perform SWL (<1000 HU) or
not. As with the ICC, agreement was calculated per round. The inter‐ and intra‐observer
agreement on SWL suitability showed comparable results to the ICC (Table 5).
It is important to note that, besides HU, there is an inter‐observer variability in stone
size measurement. Inter‐observer agreement ICC for the measurement of stone size was
0.82 (CI‐0.71‐0.88), which was comparable to results seen in the study of Okhunov: 0.80
(CI‐0.73‐0.83).24
Our study has some limitations. Firstly, the selection of scans we made was based on
the presence of stones possibly eligible for SWL. However, no clinical data was used so not
all patients were actually scheduled for treatment. This might have resulted in a selection‐
bias and a slightly population difference from patients scheduled for treatment for which
the decision on treatment is actually influenced by HU measurements.
Secondly, by assessing 160 CT scans in a study environment, a study‐bias might be
introduced. The knowledge that results will be tested might lead observers to measure
differently. Moreover, more accurate measurements in the last round may reflect a
practise effect. We assume that this is of minimal consequence as each observer had
several years of experience in reviewing CT scans.
Finally, using protocol A, one of the observers (#3) continuously scored differently than
the other observers with all stones below HU=1000 and could not be compared properly.
As an additional check, analyses were also conducted by excluding observer 3. Only
differences in protocol A were observed: ICC=0.62 and ICC=0.83, with and without
observer 3 respectively. No differences were observed between measurement without a
protocol and protocol B.
Page 11 of 22

11
For future research, it would be interesting to assess if the proposed method of HU
measurement will improve patient selection for SWL and, thereby, treatment outcome.
Additionally, as there will always be inter‐observer variability it would be interesting to
further explore and evaluate the possibilities of measuring HU using a software tool that
can measure the total stone. Another possible advancement would be to better quantify
stone variation. There is a variation in density throughout the stone. This variability
influences the success of SWL and, thus, the treatment choice for the individual patient.30
However, for current clinical practise, we advise measurements to be standardized using
zoom, bone setting, and measuring within the penumbra. Our results show the highest
agreement using these predefined measuring methods and should therefore become
common practice.
5. Conclusions
In conclusion, observer agreement of HU measurement of urolithiasis without protocol
is already good but improves with the use of a protocol. We advise that renal and ureteral
stones be measured using bone setting, a 300% zoom, and a ROI just smaller than the
stone on the most representative axial plane for the most reliable measurement of HU.
This protocol leads to a better inter‐observer group agreement and the improvement is
biggest among urologists. Measuring in three axial planes did not lead to a better
agreement in our study.
6. Author Disclosure Statement
No competing financial interests exist.
Page 12 of 22

12
References
1. Romero V, Akpinar H, Assimos DG. Kidney Stones: A Global Picture of Prevalence,
Incidence, and Associated Risk Factors. Rev Urol. 2010;12(2‐3):e86‐96.
2. Türk C, Neisius A, Petrik A, Seitz C, Skolarikos A, Thomas K. EAU Guidelines on
Urolithiasis. Eur Assoc Urol. 2018:1‐82.
3. Wang LJ, Wong Y‐C, Chuang C‐K, et al. Predictions of outcomes of renal stones after
extracorporeal shock wave lithotripsy from stone characteristics determined by
unenhanced helical computed tomography: a multivariate analysis. Eur Radiol.
2005;15:2238‐2243.
4. Yoshida S, Hayashi T, Ikeda J, et al. Role of volume and attenuation value histogram
of urinary stone on noncontrast helical computed tomography as predictor of
fragility by extracorporeal shock wave lithotripsy. J Urol. 2006;68:33‐37.
5. Bandi G, Meiners RJ, Pickhardt PJ, Nakada SY. Stone measurement by volumetric
three‐dimensional computed tomography for predicting the outcome after
extracorporeal shock wave lithotripsy. BJU Int. 2009;103:524‐528.
6. Foda K, Abdeldaeim H, Youssif M, Assem A. Calculating the Number of Shock Waves,
Expulsion Time, and Optimum Stone Parameters Based on Noncontrast
Computerized Tomography Characteristics. Urology. 2013;82:1026‐1031.
7. Gupta NP, Ansari MS, Kesarvani P, Kapoor A, Mukhopadhyay S. Role of computed
tomography with no contrast medium enhancement in predicting the outcome of
extracorporeal shock wave lithotripsy for urinary calculi. BJU Int. 2005;95:1285‐
1288.
8. Ng CF, Siu DY‐W, Wong A, Goggins W, Chan ES, Wong K‐T. Development of a Scoring
System From Noncontrast Computerized Tomography Measurements to Improve
the Selection of Upper Ureteral Stone for Extracorporeal Shock Wave Lithotripsy. J
Urol. 2009;181:1151‐1157.

Page 13 of 22

13
9. Pareek G, Armenakas NA, Fracchia JA. Hounsfield Units on Computerized
Tomography Predict Stone‐Free Rates After Extracorporeal Shock Wave Lithotripsy.
J Urol. 2003;169:1679‐1681.
10. Massoud AM, Abdelbary AM, Al‐Dessoukey AA, Moussa AS, Zayed AS, Mahmmoud
O. The success of extracorporeal shock‐wave lithotripsy based on the stone‐
attenuation value from non‐contrast computed tomography. Arab J Urol.
2014;12:155‐161.
11. Wiesenthal JD, Ghiculete D, John D’A Honey R, Pace KT. Evaluating the importance
of mean stone density and skin‐to‐stone distance in predicting successful shock
wave lithotripsy of renal and ureteric calculi. Urol Res. 2010;38:307‐313.
12. Joseph P, Mandal AK, Singh SK, Mandal P, Sankhwar SN, Sharma SK. Computerized
tomography attenuation value of renal calculus: can it predict successful
fragmentation of the calculus by extracorporeal shock wave lithotripsy? A
preliminary study. J Urol. 2002;167:1968‐1971.
13. Ouzaid I, Al‐qahtani S, Dominique S, et al. A 970 Hounsfield units (HU) threshold of
kidney stone density on non‐contrast computed tomography (NCCT) improves
patients’ selection for extracorporeal shockwave lithotripsy (ESWL): Evidence from a
prospective study. BJU Int. 2012;110:E438‐E442.
14. El‐Nahas AR, El‐Assmy AM, Mansour O, Sheir KZ. A Prospective Multivariate Analysis
of Factors Predicting Stone Disintegration by Extracorporeal Shock Wave Lithotripsy:
The Value of High‐Resolution Noncontrast Computed Tomography. Eur Urol.
2007;51:1688‐1694.
15. Perks AE, Schuler TD, Lee J, et al. Stone Attenuation and Skin‐to‐Stone Distance on
Computed Tomography Predicts for Stone Fragmentation by Shock Wave
Lithotripsy. J Urol. 2008;72(4):765‐769.
16. Celik S, Bozkurt O, Kaya FG, et al. Evaluation of computed tomography findings for
success prediction after extracorporeal shock wave lithotripsy for urinary tract stone
disease. Int Urol Nephrol. 2015;47:69‐73.
Page 14 of 22

14
17. Kacker R, Zhao L, Macejko A, et al. Radiographic Parameters on Noncontrast
Computerized Tomography Predictive of Shock Wave Lithotripsy Success. J Urol.
2008;179:1866‐1871.
18. Choi JW, Song PH, Kim HT. Urolithiasis Predictive Factors of the Outcome of
Extracorporeal Shockwave Lithotripsy for Ureteral Stones. Korean J Urol.
2012;53:424‐430.
19. Tarawneh E, Zeyad A, Hani A, et al. Factors Affecting Urinary Calculi Treatment by
Extracorporeal Shock Wave Lithotripsy. Saudi J Kidney Dis Transplant.
2010;21(4):660‐665.
20. Lee H, Yang Y‐H, Lee Y‐L, et al. Noncontrast computed tomography factors that
predict the renal stone outcome after shock wave lithotripsy. Clin Imaging.
2015;39:845‐850.
21. Yilmaz S, Sindel T, Arslan G, et al. Renal colic: comparison of spiral CT, US and IVU in
the detection of ureteral calculi. Eur Radiol. 1998;8:212‐217.
22. Assimos DG, Krambeck A, Miller NL, et al. Surgical Management of Stones: American
Urological Association (AUA)/ Endourological Society Guideline. Am Urol Assoc.
2016:1‐50.
23. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation
Coefficients for Reliability Research. J Chiropr Med. 2016;15:155‐163.
24. Okhunov Z, Helmy M, Perez‐Lansac A, et al. Interobserver Reliability and
Reproducibility of S.T.O.N.E. Nephrolithometry for renal calculi. J Endourol.
2013;27:1303‐1306.
25. Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing
Practice. Lippincott Williams & Wilkins; 2008.
26. Nunnally JC, Bernstein IH. Psychometric Theory. McGraw‐Hill, New York; 1994.

Page 15 of 22

15
27. Wisenbaugh E, Paden R, Silva A, Humphreys M. Dual‐energy vs Conventional
Computed Tomography in Determining Stone Composition. J Urol. 2014;83(6):1243‐
1247.
28. Cropp RJ, Seslija P, Tso D, Thakur Y. Scanner and kVp dependence of measured CT
numbers in the ACR CT phantom. J Appl Clin Med Phys. 2013;14(6):338‐349.
29. Alsyouf M, Smith D, Olgin G, et al. Comparing Stone Attenuation in Low‐ and
Conventional‐Dose Noncontrast Computed Tomography. J Endourol.
2014;28(6):704‐707.
30. Yamashita S, Kohjimoto Y, Iguchi T, et al. Variation Coefficient of Stone Density: A
Novel Predictor of the Outcome of Extracorporeal Shockwave Lithotripsy. J
Endourol. 2017;31(4):384‐390.


Abbreviations used:
K = Kappa coefficient
HU = Hounsfield Units
CI = Confidence Interval
ROI = Region Of Interest

NVU = Dutch Association of Urology
ICC = Intraclass Correlation Coefficient
AUA = American Urological Association
EAU = European Association of Urology
SWL = extracorporeal Shock Wave Lithotripsy
NECT = Non‐Enhanced Computed Tomography
16
Page 16 of 22
Page 17 of 22

17
Table 1: Demographic characteristics.
Characteristics N (%) or median (IQR)
Age 51 (23.5)
Sex: Male / Female 105 (73%) / 38 (27%)
Stone size in mm 6.8 (3.3)
Stone location: Kidney / Proximal ureter / Distal 87 (56%) / 32 (21%) / 36 (23%)

ureter
Stone Side: Right / Left 79 (51%) / 76 (49%)
Amount of stones: Single / Multiple 115 (74%) / 40 (26%)
Primary / Post‐Treatment 119 (77%) / 36 (23%)
Description: N=number; IQR=interquartile range; mm=millimeter.

Page 18 of 22

18
Table 2: Inter‐observer agreement ICC (CI).
All Radiologists Urologists

No protocol
Mean .84 (.79‐.87) .90 (.88‐.91) .81 (.70‐.87)
R1 .81 (.76‐.86) .88 (.84‐.91) .73 (.55‐.83)
R2 .86 (.78‐.91) .91 (.89‐.93) .90 (.85‐.94)

Protocol A
Mean .62 (.37‐.76) .51 (.08‐.75) .77 (.71‐.81)
R3 .60 (.33‐.75) .48 (.05‐.73) .71 (.60‐ .78)
R4 .65 (.38‐.79) .55 (.11‐.77) .83 (.78‐.87)

Protocol B
Mean .90 (.86‐.92) .92 (.89‐.93) .91 (.84‐.94)
R3 .90 (.85‐.93) .91 (.84‐.94) .88 (.75‐.93)
R4 .90 (.84‐.93) .93 (.91‐.95) .94 (.90‐.96)
Description: ICC= Intraclass Correlation Coefficient;
CI= Confidence interval; R1=round 1; R2=round 2;
R3=round 3; R4=round 4.

Page 19 of 22

19
Table 3. Intra‐observer agreement ICC (CI)
All Rad1 Rad2 RRad3 Uro4 RUro5

No
.87 .92 (.86‐.95) .91 (.87‐.93) .90 (.86‐.93) .78 (.59‐.87) .87 (.82‐.90)
protocol
Protocol A .87 .90 (.85‐.94) .90 (.76‐.95) .90 (.86‐.93) .76 (.66‐.83) .88 (.84‐.91)
Protocol B .93 .90 (.86‐.93) .91 (.84‐.94) .96 (.94‐.97) .92 (.87‐.95) .94 (.92‐.96)
Description: ICC=Intraclass Correlation Coefficient; CI=Confidence interval;
Rad1‐2=radiologist; RRad3=resident in radiology; Uro4: urologist;
RUro5=resident in urology.


Medial
Cranial
Caudal
Protocol A

CI=Confidence interval.
.62 (.37‐.76)
.26 (.12‐.40)
.38 (.21‐.53)
.88 (.84‐.91)
Inter‐observer
for protocol A, subdivided by different planes.
.59
.87
.68
.91
Description: ICC=Intraclass Correlation Coefficient;
Intra‐observer
Table 4. Inter‐ and intra‐observer agreement ICC (CI)
20
Page 20 of 22
Page 21 of 22

21
Table 5: Observer agreement Kappa
5a: Inter‐observer agreement Kappa
All Radiologists Urologists
No
.68 .76 .66
protocol
Protocol A .40 .24 .52

Protocol B .75 .77 .78

5b: Intra‐observer agreement Kappa
All Rad1 Rad2 RRad3 Uro4 RUro5
No
.74 .84 .76 .74 .60 .77
protocol
Protocol A .68 .75 .70 NA .56 .71
Protocol B .81 .76 .73 .83 .87 .87
Description: Rad1‐2=radiologist; RRad3=resident in radiology; Uro4: urologist;
RUro5=resident in urology; NA=not available.

Page 22 of 22

22
Supplementary data
Addendum 1: Specifications CT scanner and imaging techniques.
All included CT scans were made according to standard urolithiasis protocol with a 64‐slice
scanner (General Electric Medical systems CT model Discovery CT750 HD), with a set kV of
100. Dose modulation was performed based on the scout imaging with a maximum of 350
mA. Slice thickness was 2.5 mm with an increment of 1.25 mm. Iterative reconstruction of
50% was performed using GEs ASiR reconstruction technique. Coronal slices of 3 mm were
reconstructed from the axial imaging. All images were transmitted to the picture archiving
and communication system SECTRA Health Care Server v16.2.32.2422 (Sectra Imtec ABm
Linköping, Sweden).

Title - Inter and Intra Observer Agreement in Measuring Urolithiasis Density On Non Enhanced Computed Tomography

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Title - Inter and Intra Observer Agreement in Measuring Urolithiasis Density On Non Enhanced Computed Tomography

Uploaded by

Copyright:

Available Formats

Journal of Endourology

Stone location: Kidney / Proximal ureter / Distal 87 (56%) / 32 (21%) / 36 (23%)

All Radiologists Urologists

R2 .86 (.78‐.91) .91 (.89‐.93) .90 (.85‐.94)

R4 .65 (.38‐.79) .55 (.11‐.77) .83 (.78‐.87)

R4 .90 (.84‐.93) .93 (.91‐.95) .94 (.90‐.96)

All Rad1 Rad2 RRad3 Uro4 RUro5

Protocol B .93 .90 (.86‐.93) .91 (.84‐.94) .96 (.94‐.97) .92 (.87‐.95) .94 (.92‐.96)

All Radiologists Urologists

Protocol A .40 .24 .52

Protocol B .81 .76 .73 .83 .87 .87

You might also like