Professional Documents
Culture Documents
lent agreement.14
Percent of cases
20
Results 15
10
The 20 cases were graded by 32 participants; 16 attend-
ing urologists and 16 PGTs in post-graduate years 3 to 7. 5
Table 1. Inter-group agreement between PGTs and Table 1. Inter-group agreement between PGTs and
attending urologists attending urologists (cont’d)
Case RCCS Attending p Agreement Case RCCS Attending p Agreement
PGTs PGTs
No. Grade urologists value between groups No. Grade urologists value between groups
II 10 (62.5) 7 (43.75) I 0 (00) 1 (6.25)
1 0.48 81.3
III a 6 (37.5) 9 (56.25) II 13 (81.25) 7 (43.75)
III a 0 (00) 1 (6.25) 13 III a 2 (12.5) 7 (43.75) 0.06 56.5
2 1.00 93.8
III b 16 (100) 15 (93.75) III b 1 (6.25) 0 (00)
IV b 1 (6.25) 0 (00) IV a 0 (00) 1 (6.25)
3 1.00 93.8
V 15 (93.75) 16 (100) II 16 (100) 14 (87.5)
I 2 (12.5) 0 (00) 14 III a 0 (00) 1 (6.25) 0.23 87.5
4 0.48 87.5
III a 14 (87.5) 16 (100) IV b 0 (00) 1 (6.25)
II 0 (00) 3 (18.75) I 12 (75) 4 (25)
5 IV a 15 (93.75) 12 (75) 0.11 81.3 II 4 (25) 8 (50)
15 0.01 50.0
IV b 1 (6.25) 1 (6.25) III a 0 (00) 2 (12.5)
III a 14 (87.5) 15 (93.75) IV a 0 (00) 2 (12.5)
6 III b 2 (12.5) 0 (00) 0.12 87.5 I 1 (6.25) 1 (6.25)
IV a 0 (00) 1 (6.25) II 10 (62.5) 6 (37.5)
7 II 16 (100) 16 (100) 1.00 100 16 III a 0 (00) 3 (18.75) 0.26 30.0
III a 0 (00) 4 (25) III b 2 (12.5) 2 (12.5)
8 III b 15 (93.75) 12 (75) 0.03 68.8 IV a 3 (18.75) 4 (25)
IV a 1 (6.25) 0 (00) 17 V 16 (100) 16 (100) 1.00 100
I 15 (93.75) 13 (81.25) III a 11 (68.75) 9 (56.25)
9 0.60 87.5
II 1 (6.25) 3 (18.75) 18 III b 5 (31.25) 6 (37.5) 0.43 68.8
I 0 (00) 1 (6.25) IV a 0 (00) 1 (6.25)
10 1.00 93.8
II 16 (100) 15 (93.75) 19 I 16 (100) 16 (100) 1.00 100
I 14 (87.5) 8 (50) II 0 (00) 1 (6.25)
11 0.05 62.5
III a 2 (12.5) 8 (50) 20 IV a 15 (93.75) 12 (75) 0.25 75.0
III a 0 (00) 1 (6.25) IV b 1 (6.25) 3 (18.75)
12 III b 15 (93.75) 14 (87.5) 0.49 93.8 PGTs: post-graduate trainees; RCCS: Revised Clavien-Dindo Classification System.
IV a 1 (6.25) 1 (6.25)
PGTs: post-graduate trainees; RCCS: Revised Clavien-Dindo Classification System.
use of this RCCS to report complications in the published
Discussion literature. Moreover, the European Urological Association
highly recommended the use of this classification system
The RCCS has been validated and adapted as a standardized to standardize the report of complications in the urologic
method of reporting complications across various surgical literature.15 However, subjective interpretation of postop-
sub-specialties.3 There has been an exponential rise in the erative complications may result in inter-rater variability.11
In academic centres, Quality Assurance case presentations
are prepared by PGTs. Therefore, it was hypothesized that
PGTs Attending urologists
PGTs may be more comfortable in using the RCCS when
70
61.3 compared with attending urologists. On the other hand,
60 54.4 PGTs, when compared with attending urologists, may assign
50 45.6 a different RCCS grade since they have less experience in
Percent grades
Table 2. Inter-rater agreement among post-graduate Table 3. Fleiss’ kappa for inter-rater reliability among post-
trainees, attending urologists and all respondents graduate trainees, attending urologists and all respondents
Attending Attending
PGTs All respondents PGTs All respondents
urologists urologists
1 0.50 0.48 0.48 1 0.41 0.39 0.40
2 1.00 0.88 0.94 2 1.00 0.85 0.93
3 0.88 1.00 0.94 3 0.85 1.00 0.93
4 0.77 1.00 0.86 4 0.73 1.00 0.88
5 0.88 0.58 0.72 5 0.85 0.50 0.67
6 0.77 0.88 0.82 6 0.73 0.85 0.79
7 1.00 1.00 1.00 7 1.00 1.00 1.00
8 0.88 0.60 0.72 8 0.85 0.53 0.67
9 0.88 0.68 0.77 9 0.85 0.62 0.74
10 1.00 0.88 0.94 10 1.00 0.85 0.93
11 0.77 0.47 0.56 11 0.73 0.38 0.48
12 0.88 0.76 0.82 12 0.85 0.72 0.79
13 0.66 0.35 0.46 13 0.60 0.24 0.36
14 1.00 0.76 0.88 14 1.00 0.72 0.86
15 0.60 0.30 0.38 15 0.53 0.18 0.28
16 0.41 0.21 0.30 16 0.31 0.08 0.19
17 1.00 1.00 1.00 17 1.00 1.00 1.00
18 0.54 0.43 0.49 18 0.47 0.33 0.41
19 1.00 1.00 1.00 19 1.00 1.00 1.00
20 0.88 0.58 0.72 20 0.85 0.50 0.67
Overall 0.82 0.69 0.75 Overall 0.79 0.64 0.71
PGTs: post-graduate trainees. PGTs: post-graduate trainees.
complications post-hip preservation surgery. Excellent reli- RCCS in that grades III and IV have subdivisions that reduce
ability of this system was shown where the inter-rater and inter-rater agreement.10,12
intra-rater agreement rates were 86% and 89%, respect- There were no significant differences between both groups
ively.16 However, the authors modified the RCCS to remove in terms of overall grading of complications (p = 0.12). In
the sub-divisions in grades III and IV. two cases, the grades given by PGTs were significantly differ-
For cases with less than 80% agreement (cases 8, 13, ent from that given by attending urologists (p ≤ 0.03) (Table
15, 16, 18, 20, except for 11), there were more than two 1). In case 15, PGTs gave a significantly lower score (grade I
grades assigned mainly in the sub-divisions of grades III and and II by 75% and 25%, respectively) than attending urolo-
IV (Table 1). Part of this variation in grading these cases gists (grade I, II, and over in 25, 50, and 25%, respectively)
could be due to subjective interpretation of the different sub- (p = 0.01). However, the reverse was seen in case 8 where
classifications of grades III and IV, as has been previously 94% of PGTs had graded the complication significantly
mentioned.12,17 This is where subjective interpretation influ- higher than attending urologists (p = 0.03). Therefore, no
ences the different subdivisions in grades III and IV. This is clear pattern emerged in overall grading between PGTs and
because interventional procedures may be performed under attending urologists.
local anesthesia in one centre, and under general anesthesia When compared with attending urologists, PGTs showed
in another (grade III-a vs. grade III-b). Furthermore, there may a trend towards a higher overall agreement rate (0.82 vs.
be different institutional policies regarding intensive care 0.69, p = 0.68) and a higher overall Fleiss’ kappa value for
unit (ICU) admissions for observing postoperative intubated inter-rater reliability (0.79 vs. 0.64). Although these differ-
patients with sepsis (grade III-b vs. IV).10 This was the reason ences were not significant, it could perhaps be explained
for the discrepancy in grading case 16, where the patient by the fact that PGTs are responsible in preparing Quality
with suspected bladder-perforation post-ureteroscopy was Assurance cases and they may be more familiar with the
observed intubated in the ICU (Table 1). When grades III RCCS than attending urologists.
and IV subgroups were combined into one group each, the The main limitation of the present study is the relatively
concordance rates improved among PGTs (0.86, k = 0.83), small number of respondents (16 PGTs and 16 attending
attending urologists (0.77, k = 0.71) and the overall group urologists). Given the relatively small size of urology depart-
(0.80, k = 0.76). This highlights one of the limitations of the ments when compared with other surgical specialties, such
5. Constantinides CA, Tyritzis SI, Skolarikos A, et al. Short- and long-term complications of open radical
as general surgery and orthopedics, the number of respond- prostatectomy according to the Clavien Classification system. BJU Int 2009;103:336-40. http://dx.doi.
ents in the present study is reasonable. Nonetheless, this is org/10.1111/j.1464-410X.2008.08080.x
the first study in its kind in assessing inter-rater variability 6. Murphy DG, Kerger M, Crowe H, et al. Operative Details and Oncological and Functional Outcome of
among urology PGTs and attending urologists. It is possible Robotic-Assisted Laparoscopic Radical Prostatectomy: 400 cases with a Minimum of 12 Months Follow-up.
that when the same survey is administered to a larger cohort Eur Urol 2009;55:1358-66. http://dx.doi.org/10.1016/j.eururo.2008.12.035
7. Hennus PM, Kroeze SG, Bosch R, et al. Impact of comorbidity on complications after nephrectomy:
of urology PGTs and attending urologists, the inter-rater use of the Clavien Classification of Surgical Complications. BJU Int 2012;110:682-7. http://dx.doi.
agreements rates may improve. org/10.1111/j.1464-410X.2011.10889.x
8. Tefekli A, Ali Karadag M, Tepeler K, et al. Classification of Percutaneous Nephrolithotomy Complications
Using the Modified Clavien Grading System: Looking for a Standard. Eur Urol 2008;53:184-90. http://
Conclusion dx.doi.org/10.1016/j.eururo.2007.06.049
9. De La Rosette JJ, Zuazu JR, Tsakiris P, et al. Prognostic Factors and Percutaneous Nephrolithotomy
There was good overall agreement among PGTs and Morbidity: A Multivariate Analysis of a Contemporary Series Using the Clavien Classification. J Urol
attending urologists in application of the RCCS in urology. 2008;180:2489-93. http://dx.doi.org/10.1016/j.juro.2008.08.025
Therefore, it is appropriate for PGTs to complete the Quality 10. Rassweiler JJ, Rassweiler M, Michel M. Classification of Complications: Is the Clavien-Dindo Classification
Assurance meeting reports. the Gold Standard? Eur Urol 2012;62:256-8. http://dx.doi.org/10.1016/j.eururo.2012.04.028
11. Morgan M, Smith N, Thomas K, et al. Is Clavien the new standard for reporting urological complications?
BJU Int 2009;104:434-6. http://dx.doi.org/10.1111/j.1464-410X.2009.08516.x
Competing interests: None declared.
12. De la Rosette JJMCH, Opondo D, Daels FPJ, et al., on behalf of the CROES PCNL Study Group.
Categorisation of complications and validation of the Clavien score for percutaneous nephrolithotomy.
Eur Urol 2012;62:256-8. http://dx.doi.org/10.1016/j.eururo.2012.03.055
This paper has been peer-reviewed. 13. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971;76:378-82. http://
dx.doi.org/10.1037/h0031619
14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics
1977;33:159-74. http://dx.doi.org/10.2307/2529310
References 15. Mitropoulos D, Artibani W, Graefen M, et al. Reporting and Grading of Complications after Urologic Surgical
Procedures: An ad hoc EAU Guidelines Panel Assessment and Recommendations. Eur Urol 2012;61:341-9.
1. Clavien PA, Sanabria JR, Strasberg SM. Proposed classification of complications of surgery with examples http://dx.doi.org/10.1016/j.eururo.2011.10.033
of utility in cholecystectomy. Surgery 1992;111:518-26. 16. Sink EL, Leunig M, Zaltz I, et al. Reliability of a Complication Classification System for Orthopedic Surgery.
2. Dindo D, Demartines N, Clavien PA. Classification of Surgical Complications. A New Proposal with Evaluation Clin Orthop Relat Res 2012;470:2220-6. http://dx.doi.org/10.1007/s11999-012-2343-2
in a Cohort of 6336 Patients and Results of a Survey. Ann Surg 2004;240:205-13. http://dx.doi. 17. Hruza M, Weiss HO, Pini G, et al. Complications in 2200 consecutive laparoscopic prostatectomies:
org/10.1097/01.sla.0000133083.54934.ae standardized evaluation and analysis of learning curves. Eur Urol 2010;58:733-41. http://dx.doi.
3. Clavien PA, Barkun J, de Oliveira ML, et al. The Clavien-Dindo classification of surgical complications: five- org/10.1016/j.eururo.2010.08.024
year experience. Ann Surg 2009;250:187-96. http://dx.doi.org/10.1097/SLA.0b013e3181b13ca2
4. Donat SM, Shabsigh A, Savage C, et al. Potential impact of postoperative early complications on the Correspondence: Dr. Sero Andonian, Assistant Professor of Urology, Royal Victoria Hospital, McGill
timing of adjuvant chemotherapy in patients undergoing radical cystectomy: a high-volume Tertiary Cancer
University Health Centre, 687 avenue des Pins Ouest, Suite S6.92, Montreal, QC H3A 1A1; fax:
Center Experience. Eur Urol 2009;55:177-85. http://dx.doi.org/10.1016/j.eururo.2008.07.018
514-843-1552; sero.andonian@muhc.mcgill.ca