European Journal of Cardio-thoracic Surgery 35 (2009) 751—759

www.elsevier.com/locate/ejcts

Surgeon performance index: tool for assessment of individual surgical

quality in total quality management
Martin Hartrumpf, Thomas Claus, Michael Erb, Johannes M. Albes *
Department of Cardiovascular Surgery, Heart Center Brandenburg, Ladeburger Strasse 17, 16321 Bernau, Germany

Received 27 May 2008; received in revised form 30 September 2008; accepted 2 December 2008; Available online 9 March 2009

Abstract
Background: The surgeon’s individual performance is a key component of total quality management (TQM) in cardiac surgery. Early mortality as
well as postoperative complications can be stratified in order to develop a surgeon performance index (SPI). Material and methods: In three
consecutive annual periods (3703 patients) data of board-certified cardiac surgeons were compared. Risk-adjustment of early mortality and
postoperative complications was performed by logistical EuroSCORE (logES). Early mortality (EM), early rethoracotomy for bleeding (ReTh),
sternal rewiring for instability (ReWr), and mediastinitis (Med) were assessed. ReTh, ReWr, and Med were weighted according to empiric data:
(ReTh ! 2; ReWr ! 1; Med ! 3). Surgeon performance index was computed as follows: SPI = (EM/logES + [((ReTh/logES) ! 2) + ((ReWr/
logES) ! 1) + ((Med/logES) ! 3)]/6)/2. Ideal SPI was considered "1. SPI of the respective previous period was handed out to each surgeon
and discussed by means of a structured dialogue. Results: Patients from each period were allocated to 11 cardiac surgeons. Overall logES of the
three periods were 6.6%, 9.1%, and 11.2% respectively; EM 5.7%, 6.6%, 5.6%; ReTh 5.8%, 7.3%, 10.9%; ReWr 2.4%, 1.9%, 1.4%; and Med 0.9%, 1.8%,
1.8%. SPI showed a mean of 0.71, 0.56, and 0.49. Conclusion: Comorbidity increased between periods 1 and 3 significantly whereas early
mortality remained rather stable. SPI indicated improvement of the performance of the individual surgeon and a decrease of range and mean of
the overall performance. SPI is therefore an effective tool to assess individual surgical quality and serves as an instrument for human resource
management and development. Sustainable positive effects on overall performance can be expected.
# 2009 European Association for Cardio-Thoracic Surgery. Published by Elsevier B.V. All rights reserved.

Keywords: Cardiac surgery; Complications; Health economics; Total quality management; Individual surgical quality

1. Introduction analysis of individual treatment quality. Risk-adjustment,

Total quality management (TQM) has gained increasing
impact in current economically oriented health care systems
as a tool for strategic hospital development satisfying the
needs of the stakeholders [1]. In order to guarantee adequate
treatment quality as well as standards of care several
countries have already established quality control and public
reporting measures by law [2]. Thus, genuine benchmarking
has found its way into the health care environment [3].
Surprisingly, individual treatment quality has not been
addressed thus far in a truly systematic fashion although
treatment quality is markedly dependent on the individual
physician particularly in the surgical field. Robust data
concerning individual treatment quality, however, are likely
to become increasingly relevant for quality management
purposes as well as human resource management [4]. Early
mortality (EM) as well as typical complications of the
respective types of interventions may be utilized for an
* Corresponding author. Tel.: +49 3338 694510; fax: +49 3338 694544.
E-mail address: j.albes@immanuel.de (J.M. Albes).
1010-7940/$ — see front matter # 2009 European Association for Cardio-Thoracic Surgery. Published by Elsevier B.V. All rights reserved.
doi:10.1016/j.ejcts.2008.12.006
however, is mandatory in order to adjust for the individual
comorbidity of the patient the respective surgeon is taking
care of. In a cardiac surgery department a risk-adjusted
individual treatment quality ratio: surgeon performance
index (SPI) was therefore invented and assessed for all board-
certified cardiac surgeons over three consecutive annual
periods in order to evaluate the feasibility of such a tool for
identifying quality deficits, improving overall quality, and
optimizing human resource management.
2. Material and methods
In a single institution results of three consecutive years
with a total of 3703 cardiosurgical procedures were analyzed
and broken down into the results of each individual board-
certified cardiac surgeon. Data from patients operated upon
within the resident-training program were not used. Cases
were allocated by the program organizer on a day-to-day
basis in order to achieve a homogenous caseload for all
board-certified staff members and to match severity of the
individual case with the experience of the respective
752 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759
surgeon. Patients did not choose the individual surgeon on a
regular basis. However, in a few instances we complied with
a respective wish. All board-certified staff members
performed the entire procedure including opening and
closure of the sternum themselves while management of the
superficial layers was performed by residents in training for
educational purposes. The entirety of the data was used to
assess the individual treatment quality of the respective
surgeon and an individual quality ratio was developed.
Acceptance of these results by the involved staff members
was discussed after the respective period with the head of
the department using a structured dialogue within a staff
appraisal. Comparison of the results of the consecutive
annual periods then served for a revision of the effects of
individual quality assessment. All data were kept strictly
confidential but were disclosed to all board-certified
surgeons of the cardiac surgery department.
Statistical assessment of individual quality was performed
utilizing data from the operative report, the epicrisis, as well
as meta-data from the hospital’s information system of three
consecutive years (periods 1, 2, and 3). Items were: number
of procedures of the respective surgeon; EuroSCORE,
logistical EuroSCORE (logES), perioperative complications:
early mortality (30-day mortality or ‘never left the
hospital’), early rethoracotomy for hemorrhage (more than
200 ml/h in 2 consecutive hours or cumulative amount of
more than 1000 ml in the first 6 postoperative hours);
sternum instability (visible movement of sternal edges)
necessitating sternum rewiring, mediastinitis (Med) (as
defined by presence of local and systemic clinical findings,
inflammatory laboratory parameters, and positive culture
from the mediastinum). Surgeons were grouped according to
their experience (seniority: 1 = board-certified junior sur-
geon; 2 = staff member; 3 = senior staff member or head of
department).
3. Algorithm for assessment of ‘surgeon performance
index’ (SPI)
The following items were used for assessment of the
surgeon performance index: Risk-adjusted early mortality:
early mortality (%) divided by logistical EuroSCORE (%) as well
as rethoracotomy (ReTh; %), rewiring (ReWr; %), and
mediastinitis (%) divided by logistical EuroSCORE. The three
latter complications were empirically weighted according to
their impact on convalescence: rethoracotomy times factor
2; rewiring times factor 1; mediastinitis times factor 3. Risk-
adjusted early mortality as well as risk-adjusted complica-
tions were condensed to the following formula yielding a
ratio:
ðEM=logESÞ þ ðððReTh=logESÞ ! 2Þ
þ ððReWr=logESÞ ! 1Þ þ ððMed=logESÞ ! 3ÞÞ=6
SPI ¼
2
which can be mathematically rearranged to
ð1=2ÞEM þ ð1=6ÞReTh þ ð1=12ÞReWr þ ð1=4ÞMed
SPI ¼
log ES
or
6EM þ 3Med þ 2ReTh þ ReWr
SPI ¼
12 logES
Results of the respective period were handed out to all
involved surgeons. The head of the department was
integrated with his individual results in the program.
Thereafter, a structured dialogue was performed with each
member of the group and the head of the department. In the
following periods data were again handed out and were
statistically compared with the previous results.
Statistical analysis was performed using SPSS statistical
package for Windows (Version 15, SPSS Inc., Chicago, IL,
USA). Numeric variables with normal distribution were
analyzed by means of one-way ANOVA with Tukey-HSD
post-hoc comparisons. Non-normally distributed data were
analyzed by means of Kruskal—Wallis and Mann—Whitney
tests. Categorical variables were analyzed using X2-test. Data
are shown as mean, mean ' SD, or mean percentages.
Significance was assumed if p was <0.05.
4. Results
4.1. Distribution of procedures
The changes in the distribution of the procedures
performed during the three consecutive periods reflected
the currently observed shift in the industrialized nations
showing continuous decrease of isolated CABG-procedures
accompanied by a steady increase of valve procedures,
combined valve and bypass surgery, and other more complex
procedures such as aortic surgery or triple valve surgery with
or without concomitant bypass surgery (Table 1).
4.2. General ratios of operative performance
Overall early mortality was 5.7% in period 1, 6.6% in period
2, and 5.6% in period 3. The differences were not statistically
significant. Logistical EuroSCORE significantly rose from 6.6%
in period 1 over 9.1% in period 2 to 11.2% in period 3
indicating an increase of comorbidity of the patients,
severity of their preoperative condition as well as procedural
complexity. Risk-adjusted early mortality (EM/logES) sig-
nificantly decreased from 0.94 over 0.53 to 0.34. Number of
early rethoracotomies significantly increased from 5.8% over
7.3% to 10.9%. Sternal rewiring decreased consecutively from
2.4% in period 1 over 1.9% in period 2 to 1.4% in period 3
Table 1
Distribution of procedures (%).
Procedure Period 1 Period 2 Period 3
Isolated CABG 69.0 68.8 57.8
Isolated AVR 9.2 8.3 11.4
Isolated MVR 2.9 2.9 3.6
AVR + CABG 8.2 7.8 9.4
MVR + CABG 1.6 2.8 2.9
Other 9.1 9.5 14.9
CABG: aorto-coronary bypass graft; AVR: aortic valve replacement; MVR:
mitral valve replacement/reconstruction.
 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759 753

without gaining statistical significance. Mediastinitis rose

non-significantly from 0.9% to 1.8% between period 1 and
period 2 and remained constant in period 3 with 1.8%.
Comparison of the three periods exhibited a reduction of SPI
mean and range, which, however, did not gain statistical
significance (Table 2).

4.3. Individual results

Individual results demonstrated the expected differences

concerning the number of procedures between the staff
members. Senior staff-surgeons performed more procedures
per year than junior staff-surgeons whereas residents carried
out rather small numbers. Two surgeons left the department
early in period 3 (Table 3). Fig. 1. Surgeon performance index (SPI). Individual surgeons anonymized.

4.3.1. Surgeon performance index: SPI

In period 1 two surgeons exceeded the desired value of "1
while three surgeons demonstrated values in the vicinity of
1.0 (B: 0.87; G: 0.94; K: 0.76). In periods 2 and 3, however,
none of the surgeons exhibited an SPI of more than 1 and only
three surgeons (D, H, J) demonstrated values between 0.7
and 1.0. In period 3 all surgeons were able to maintain an SPI
below 0.7. Particularly younger surgeons demonstrated a
positive development of SPI toward values markedly falling
below 1. Also senior surgeons with a high percentage of high-
risk patients demonstrated a reduction of their individual SPI
towards values below 1 from period 1 to period 3 although an
increase of the logistical EuroSCORE was simultaneously
observed (Fig. 1; Table 3).
4.3.2. Logistical EuroSCORE
Logistical EuroSCORE of a respective surgeon increased
with seniority and experience. The overall increase of
logistical EuroSCORE from period 1 to period 3 corresponded
with a significant increase of the individual logistical
EuroSCORE of almost all surgeons (Table 4).
4.3.3. Early mortality
Early mortality of the individual surgeon varied signifi-
cantly between 0% and 8.9% in period 1, 0% and 18% in period
2, and 0% and 10.6% in period 3. Whereas the differences
were non-significant in the majority of the surgeons involved
two surgeons showed a significant increase between periods 1
and 2 (Table 5). Risk-adjustment of early mortality by means
of logistical EuroSCORE resulted in a leveling (Table 3).
Notable differences between individual surgeons, however,
remained.
4.4. Complications
Rethoracotomy: All except one surgeon demonstrated an
increase of early rethoracotomy between periods 1 and 3.
Three surgeons exhibited a significant increase between
periods 1 and 3 or periods 2 and 3 respectively. Rewiring:
None of the surgeons demonstrated significant changes in the
number of rewiring. One surgeon showed a high percentage
of rewiring in period 1. The total number of procedures of this
respective surgeon at that particular time was, however,
small (n = 19). Mediastinitis: Four surgeons showed a
percentage of mediastinitis above 3%. However, a significant
difference was only observed in the surgical oeuvre of one
surgeon exhibiting a significant increase of the number of
patients with mediastinitis between periods 1 and 2 followed
by a decrease below 1% in period 3 (Table 6).
4.4.1. Structured dialogue
Results of the structured dialogue could only be partially
quantified. The majority of the surgeons considered SPI an

Table 2
General statistical results.

Variable Period 1 Period 2 Period 3 p Test

EM 5.7% 6.6% 5.6% n.s. X2-test

LogES 6.6% 9.1% 0.000 Tukey-HSD
6.6% 11.2% 0.000 Tukey-HSD
9.1% 11.2% 0.000 Tukey-HSD
EM/logES 0.94 ' 0.51 0.53 ' 0.35 0.034 Mann—Whitney
0.94 ' 0.51 0.46 ' 0.24 0.015 Mann—Whitney
0.53 ' 0.35 0.46 ' 0.24 n.s. Mann—Whitney
ReTh 5.8% 7.3% 0.297 X2-test
5.8% 10.9% 0.000 X2-test
7.3% 10.9% 0.002 X2-test
ReWr 2.4% 1.9% 1.4% n.s. X2-test
Med 0.9% 1.8% 1.8% n.s. X2-test
Mean SPI 0.71 0.56 0.49 n.s. Kruskal—Wallis

EM: early mortality (%). LogES: logistical EuroSCORE (%); ReTh: early rethoracotomy for hemorrhage (%); ReWr: sternal rewiring for instability (%); Med: mediastinitis
(%); EM/logES: risk-adjusted early mortality; mean SPI: mean surgeon performance index.
754 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759

Table 3
Individual results of periods 1, 2, and 3.

Surgeon Seniority n ES logES EM% FL/logES ReTh% ReWr% Med% SPI

Period 1
A 3 126 7.85 15.01 7.94 0.53 8.73 0.79 0.00 0.37
B 2 111 4.08 3.96 5.41 1.37 4.50 0.00 0.00 0.87
C 3 112 5.55 8.78 7.14 0.81 6.25 0.00 0.00 0.53
D 1 19 4.05 3.78 0.00 0.00 0.00 10.53 0.00 0.23
E 1 33 3.58 3.87 6.06 1.57 9.09 0.00 3.03 1.37
F 3 118 4.51 5.42 3.39 0.63 4.24 4.24 0.00 0.51
G 2 114 4.66 5.42 7.02 1.30 6.14 1.75 1.75 0.94
H 3 158 5.65 7.69 6.33 0.82 1.90 1.90 0.63 0.49
I 3 226 4.91 6.90 5.75 0.83 7.96 1.33 0.44 0.64
J 3 179 5.97 6.40 8.94 1.40 6.15 4.47 3.91 1.07
K 2 172 4.38 5.52 4.65 0.84 9.30 1.74 0.58 0.76

Means/sums 1368 5.02 6.61 5.69 0.92 5.84 2.43 0.94 0.71

Period 2
A 3 122 7.67 15.14 10.66 0.70 9.84 1.64 0.82 0.48
B 2 93 4.26 4.59 2.15 0.47 2.15 1.08 3.23 0.51
C 3 132 6.35 10.39 4.55 0.44 3.79 0.76 0.76 0.30
D 1 52 4.82 4.56 1.92 0.42 7.69 1.92 3.85 0.74
E 1 50 4.30 3.99 0.00 0.00 10.00 2.00 0.00 0.46
F 3 100 4.85 6.14 3.00 0.49 5.00 3.00 5.00 0.62
G 2 100 5.77 7.64 2.00 0.26 8.00 3.00 2.00 0.40
H 3 108 7.13 11.47 13.89 1.21 8.33 1.85 0.93 0.76
I 3 166 6.44 12.19 9.64 0.79 5.42 1.20 1.20 0.50
J 3 147 7.94 14.81 18.37 1.24 8.84 1.36 1.36 0.75
K 2 146 5.84 8.92 6.85 0.77 11.64 3.42 0.68 0.65

Means/sums 1216 5.94 9.08 6.64 0.62 7.34 1.93 1.80 0.56

Period 3
A 3 147 7.70 15.79 10.20 0.65 11.56 1.36 1.36 0.47
B 2 89 5.02 7.06 2.25 0.32 8.99 1.12 3.37 0.50
C 3 152 7.16 10.94 4.61 0.42 13.82 1.32 0.00 0.43
D 1 75 5.77 7.88 6.67 0.85 8.00 1.33 2.67 0.69
E 1 72 4.04 4.03 0.00 0.00 13.89 0.00 1.39 0.66
F 3 131 6.22 10.57 3.82 0.36 12.21 3.05 0.76 0.42
G 2 137 6.51 11.10 5.11 0.46 5.11 0.00 3.65 0.39
I 3 175 7.98 15.66 6.86 0.44 12.00 1.71 0.57 0.36
J 3 141 8.21 17.43 10.64 0.61 12.77 2.84 2.13 0.47

Means/sums 1119 6.51 11.16 5.57 0.46 10.93 1.42 1.77 0.49

Surgeon (anonymized); seniority: 1 = board-certified junior surgeon; 2 = staff member; 3 = senior staff member or head of department; n: number of procedures; ES:
EuroSCORE; logES: logistical EuroSCORE (%); EM: early mortality (%); EM/logES: risk-adjusted early mortality; ReTh: early rethoracotomy for bleeding (%); ReWr:
rewiring for sternum instability (%); Med: mediastinitis (%); SPI: surgeon performance index.

Table 4
Individual statistical results: logistical EuroSCORE.

Surgeon Period 1 Period 2 Period 3 p (ANOVA) p (post-hoc Tukey)

A 15.01 15.14 15.79 0.923

B 3.96 4.59 7.06 0.001 0.001 Period 1 versus 2
C 8.78 10.39 10.94 0.328
D 3.78 4.56 7.88 0.011 0.027 Period 2 versus 3
E 3.87 3.99 4.03 0.983
F 5.42 6.14 10.57 0.001 0.001 Period 1 versus 3
0.001 Period 2 versus 3
G 5.42 7.64 11.10 0.000 0.000 Period 1 versus 3
0.049 Period 2 versus 3
H 7.69 11.47 0.003 (t-test)
I 6.90 12.19 15.66 0.000 0.002 Period 1 versus 2
0.000 Period 1 versus 3
J 6.40 14.81 17.43 0.000 0.000 Period 1 versus 2
0.000 Period 1 versus 3
K 5.52 8.92 0.000 (t-test)
 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759 755

Table 5 5. Discussion
Individual statistical results: early mortality (%).

Surgeon Period 1 Period 2 Period 3 p (X2-test) The most important values in health care are expertise
A 7.94 10.66 10.20 n.s.
and skills of personnel [5]. Professional competence as well
B 5.41 2.15 2.25 n.s. as humane care guarantees the success of medical therapy.
C 7.14 4.55 4.61 n.s. One, however, has to differentiate between individual and
D 0.00 1.92 6.67 n.s. collective quality. In recent years, collective quality has
E 6.06 0.00 0.00 n.s.
gained increasing importance within a total quality manage-
F 3.39 3.00 3.82 n.s
G 7.02 2.00 5.11 n.s. ment concept resulting in various internal and external
H 6.33 13.89 (0.063 Period 1 vs 2) measures [6]. Broad acknowledgment of individual quality,
I 5.75 9.64 6.86 n.s. however, has not yet been achieved. Although individual
J 8.94 18.37 10.64 0.019 Period 1 versus 2 quality appears to remain in a rather shadowy existence for
(0.091 Period 2 vs 3)
K 4.65 6.85 n.s.
the time being it is of utmost interest when looking at total
quality issues. In this regard physicians are particularly under
surveillance inasmuch as they perform the core service in
appropriate tool for improvement of the department’s hospitals. Individual physician’s quality is not only important
surgical quality. However, few surgeons fully agreed with a inside the hospital but is also momentous concerning the
positive impact of the SPI on their respective individual hospital’s reputation in the respective environment [7].
quality. Younger surgeons more often accepted critical While quality is being readily recognized by the patient as
assessment of their individual quality while the more senior one of the core criteria it has only been recently realized,
colleagues expressed some reservations towards an honest however, that individual physician’s quality requires proper
discussion of below-average individual results. Younger assessment, controlling, and support [8—10]. In the econo-
surgeons expected an increase of more demanding cases in mized health care system quality benchmarking of hospitals
the future while older surgeons were satisfied with the status already exists [3]. Only those institutions being able to cope
quo and rather wished a more evenly distributed workload. effectively with the business environment in terms of
comparable quality will maintain their market position.
Successfully established total quality management may even
Table 6
Individual statistical results: complications. elicit a unique selling proposition paving the way for
leadership in a particular medical field [1].
Surgeon Period 1 Period 2 Period 3 p (X2-test)
Quality improvement has significant impact on cost-
Rethoracotomy (%) efficacy in hospitals. Absence of complications alone reduces
A 8.73 9.84 11.56 n.s. factor-costs such as blood- and blood-product substitution or
B 4.50 2.15 8.99 n.s.
hospitalization time [11]. Furthermore, quality improvement
C 6.25 3.79 13.82 0.007 Period 2 versus 3
D 0.00 7.69 8.00 n.s. takes the pressure of many hospital processes and pathways
E 9.09 10.00 13.89 n.s. by alleviating organization and scheduling of surgical
F 4.24 5.00 12.21 0.042 Period 1 versus 3 procedures in the operating theatres or by maintaining
G 6.14 8.00 5.11 n.s.
intensive care unit capacity. Process optimization is there-
H 1.90 8.33 0.029
I 7.96 5.42 12.00 n.s. fore not only a prerequisite for quality improvement but also
J 6.15 8.84 12.77 n.s. one of the desired consequences [12]. Ideally, total quality
K 9.30 11.64 n.s. management eventually results in substantial process
Rewiring (%) optimization in the sense of a sustainable value chain
A 0.79 1.64 1.36 n.s. according to Porter [13]. Individual quality has entered the
B 0.00 1.08 1.12 n.s. literature as early as 1985 [14]. In this publication by Slogoff
C 0.00 0.76 1.32 n.s. and Keats complications were assigned to several individual
D 10.53 1.92 1.33 n.s.
E 0.00 2.00 0.00 n.s. anesthesiologists demonstrating that lack of experience
F 4.24 3.00 3.05 n.s. resulted in an increase of adverse events during anesthesia.
G 1.75 3.00 0.00 n.s. This approach, however, disappeared in the literature for
H 1.90 1.85 n.s more than 20 years almost entirely but reappeared recently
I 1.33 1.20 1.71 n.s.
in publications dealing with individual quality regarding
J 4.47 1.36 2.84 n.s.
K 1.74 3.42 n.s. number and severity of cardiosurgical complications [12].
First approaches focused on the maintenance of cardiosurgi-
Mediastinitis (%)
A 0.00 0.82 1.36 n.s. cal quality in resident-training situations [15]. Other authors
B 0.00 3.23 3.37 n.s. addressed the issue of cardiosurgical quality depending on
C 0.00 0.76 0.00 n.s. sleep deprivation [16]. Recently, the cumulative sum of
D 0.00 3.85 2.67 n.s. failure (CUSUM) approach pointed to the influence of
E 3.03 0.00 1.39 n.s.
experience, i.e. number of procedures performed, on
F 0.00 5.00 0.76 0.045 Period 1 versus 2
G 1.75 2.00 3.65 n.s. procedural quality in particularly demanding minimally
H 0.63 0.93 n.s. invasive cardiosurgical procedures. While it became quite
I 0.44 1.20 0.57 n.s. clear that experience correlates with surgical success and
J 3.91 1.36 2.13 n.s.
absence of adverse effects the issue of the influence of the
K 0.58 0.68 n.s.
patient’s comorbidity has not yet been addressed sufficiently
756 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759
[17]. In light of the demographic changes risk-adjustment of
cardiosurgical procedures, however, is mandatory in order to
develop robust and fair data for valid benchmarking.
Recently, ‘Cardiac-Surgery-Quality-Report-Cards’ document-
ing the individual early mortality of a respective surgeon
have been inaugurated in Anglo-American countries for the
purpose of improving transparency for the patient to choose
the doctor he or she feels most comfortable with [7—9].
However, early mortality alone does not represent the true
individual quality as many more aspects such as perioperative
complications or long-term patency rates, quality of life, and
absence of re-operations are important. Furthermore,
focusing on early mortality alone may provoke a ‘raisin
pecking’ of the surgeon refraining from operating upon high-
risk patients. Consequently, patients who are mostly in
demand of surgery would be driven from the ‘market’. In
economic terms, this represents a moral hazard as a
consequence of asymmetrical information in the principal
(patient)/agent (physician) relationship [18]. Differences in
individual quality are a reality as surgical skills depend on
individual stress resistance, dexterity, or intellectual cap-
ability [10,17,19]. Thus, they can only be ameliorated but not
be entirely eliminated.
Another important aspect for assessment of individual
performance is human resource management and develop-
ment [4]. Maintaining high quality in a cardiosurgical
department over a sustainable period requires constant
adjustment of personnel. Residents need to be trained,
senior staff members experience an evolutionary develop-
ment towards more and more complex procedures, fluctua-
tion requires acquisition of new staff members on the junior
as well as the senior level, and finally individual deficits need
to be detected early before they generate dire conse-
quences. In all these instances, robust data concerning the
individual quality greatly facilitates those constant changes.
Individual quality in cardiosurgical procedures can clearly
be addressed utilizing a variety of short-term parameters
such as early mortality, rethoracotomy for bleeding, sternum
rewiring for instability, or mediastinitis but also long-term
parameters such as quality of life, bypass patency rate,
absence of re-operations, or the patient’s satisfaction with
the individual surgeon’s care. Risk-adjustment, however,
must be considered. As a consequence, a large bundle of data
needs to be addressed when looking at individual quality
resulting in difficulties to achieve a true comparability, i.e. a
fair benchmarking [20]. In the economic world, ratios have
been developed in order to facilitate perception of financial
or procedural performance issues [13]. Thus, it may be
prudent to define a ratio for individual surgical quality too.
We therefore invented the ratio: SPI summarizing and
condensing the most-relevant early parameters in a risk-
adjusted fashion. Early mortality can already be properly
risk-adjusted by using evidence-based scoring systems. The
logistical EuroSCORE provides a percentage of the individual
risk to die early [21]. The percentage of the patients
operated upon by a respective surgeon who died early can
therefore readily be divided by the logistical EuroSCORE
yielding a dimensionless ratio: risk-adjusted early mortality.
Ideally, risk-adjusted early mortality lies around 1 (observed
EM; %/predicted EM; %). Less than 1 indicates an above-
average performance while values >1 show a below-average
performance. Early complications have not yet been
addressed in such a systematic risk-adjusted fashion.
However, evidence exists concerning the impact of early
rethoracotomy, sternum rewiring, and mediastinitis regard-
ing their respective impact on patient outcome. They can
therefore be weighted empirically. Sternum instability
appears to exert only minor adverse consequences while
early rethoracotomy already enhances the risk to die early
[22]. Mediastinitis, however, presents a rather severe
problem significantly contributing to an adverse outcome
[23]. Predicted percentages for the development of a
respective complication do not yet exist. According to a
recent study by Paul and colleagues, however, the EuroSCORE
can be used to predict the risk for wound infections [24]. For
the time being, risk-adjustment may therefore be performed
using the logistical EuroSCORE. The formula utilized in this
study is designed to normalize the sum of risk-adjusted early
mortality and the risk-adjusted respective complications
towards a value of 1 in order to maintain a readily perceptible
threshold. In the future, the empirical weighting may then be
replaced by an evidence-based risk-adjustment.
In this study we could show that individual parameters of
quality are helpful to identify below-average performances
as an internal benchmarking tool. An aggregated ratio of
weighted key-performance parameters can be readily
obtained by means of a surgeon performance index. SPI
can clearly elucidate the individual surgeon’s performance
on a fair basis. We could also demonstrate that this tool finds
acceptance among the surgical staff when kept confidential.
More important, however, is the impact of SPI on quality
improvement. It could be demonstrated that by using this
instrument individual and overall performance could be
optimized. Finally, SPI also served the purpose of human
resource management elucidating the individual surgical
development of the junior staff members over three annual
periods. SPI is meant as a constructive and objective tool.
However, it may sow undue rivalry if not properly
introduced, accompanied, and mediated by the head of
the department.
Assessment of several consecutive periods can result in a
reduction of variance and improvement of overall perfor-
mance as shown in Fig. 2. After several years, it can be
expected that mean SPI and SPI-variance will both approach
marginal values. Individual below-average performance,
however, can still be traced. In a holistic total quality
Fig. 2. Observed course of the surgeon performance index (SPI).
 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759
management concept the ratio SPI can be incorporated into
the ‘balanced scorecard’ (BSC) system, which comprises a
bundle of interdependent, harmonized targets supported by
ratios of different strategic levels such as innovative level,
process level, customer level, and financial level thereby
serving as an instrument for implementation of the TQM-
concept in health care institutions [25]. Inaugurated in the
early 1980s by Kaplan and Norton the balanced scorecard had
soon been successfully adopted by health care institutions
and various non-profit organizations. Consequently, it has
been implemented by an increasing number of hospitals
worldwide demonstrating an improvement of added value by
enhancing overall quality [5].
6. Limitations of the study
Our study only utilizes three consecutive periods.
Furthermore, results of only 11 surgeons (9 surgeons in
period 3) were analyzed. A long-term assessment on a larger
scale, however, may yield higher variances and less positive
effects. A true risk-adjusted incorporation of the three
complications was not yet possible. Consequently, the
desired (‘ideal’) threshold level cannot be exactly ‘1’. The
current ratios as well as the SPI as an aggregate of early
indicators only reflect short-term quality. As mentioned
above, additional long-term parameters such as bypass
patency rate, late mortality, absence of re-operations or
quality of life may be necessary for further developmental
steps towards a true short- and long-term total quality
instrument. The surgical development of the junior surgeons
alone may have resulted in an improvement of their
performance as evolution of surgical skills is faster in the
early course of training. In contrast, several senior surgeons
also exhibited quality improvement indicating the value of
the SPI as a valid and robust instrument for quality
management. In our study we saw an increase of the
necessity of early rethoracotomies in the consecutive three-
year course although general quality appeared to improve
according to the analyzed data and ratios derived thereof.
This particular phenomenon indicating an increase of early
bleeding problems may be the consequence of implementa-
tion of a new bed-side coagulation monitoring system in our
department between periods 1 and 2 in order to reduce the
amount of blood and coagulation product substitution for
cost-efficacy purposes [12]. Although this strategy turned out
to be successful the more restrictive use of blood and
coagulation products may have contributed to the non-
desired number of early rethoracotomies of almost 11% in
period 3.
7. Conclusion
Individual quality ratios such as SPI are useful tools for
individual quality assessment and also serve the purpose of
human resource management. In combination with other
quality measures it can evolve into a promising component of
total quality management. It is our underlying intention,
however, not to publicize individual surgeon-specific quality
data but instead enhance the performance of the entire
757
group by means of optimization of the individual member’s
quality in an internal peer-review process. As a consequence,
adverse patient selection can be avoided.
In the future, SPI may be expanded towards long-term
results such as quality of life, long-term patency rate, and
absence of re-operations. SPI may also be transferred to
other surgical or even interventional fields. In cardiology, for
instance, typical complications of an intervention may be
compiled, risk-adjusted and condensed to an interventionist
performance index (IPI).
References
[1] Filardo G, Nicewander D, Hamilton C, Herrin J, Galimbretti P, Tietze M,
McBride S, Gunderson J, Haydar Z, Williams J, Ballard DJ. A hospital-
randomized controlled trial of an educational quality improvement
intervention in rural and small community hospitals in Texas following
implementation of information technology. Am J Med Qual
2007;22(6):418—27.
[2] Tong BC, Harpole Jr DH. Audit, quality control, and performance in
thoracic surgery: a North American perspective. Thorac Surg Clin
2007;17(August (3)):379—86.
[3] Meehan TJ, Stedman TJ, Neuendorf KE, Francisco ID, Neilson MG. Bench-
marking Australia’s mental health services: is it possible and useful? Aust
Health Rev 2007;31(4):623—7.
[4] Harris C, Cortvriend P, Hyde P. Human resource management and per-
formance in healthcare organisations. J Health Organ Manage 2007;21(4—
5):448—59.
[5] Vallejo P, Saura RM, Sunol R, Kazandjian V, Urena V, Mauri J. A proposed
adaptation of the EFQM fundamental concepts of excellence to health
care based on the PATH framework. Int J Qual Health Care 2006;8(5):327—
35.
[6] Lindberg E, Rosenqvist U. Implementing TQM in the health care service.
Int J Health Care Qual Assur Inc Leadersh Health Serv 2005;18(4—5):370—
84.
[7] Neil DA, Clarke S, Oakley JG. Public reporting of individual surgeon
performance information: United Kingdom developments and Australian
issues. Med J Aust 2004;181(5):266—8.
[8] Glance LG, Dick A, Osler TM, Li Y, Mukamel DB. Impact of changing the
statistical methodology on hospital and surgeon ranking: the case of the
New York State cardiac surgery report card. Med Care 2006;44(4):311—9.
[9] Mukamel DB, Weimer DL, Zwanziger J, Gorthy SF, Mushlin AI. Quality
report cards. Selection of cardiac surgeons and racial disparities: a study
of the publication of the New York State Cardiac Surgery Reports. Inquiry
2004—2005 Winter;41(4):435—46.
[10] Treasure T, Valencia O, Sherlaw-Johnson C, Gallivan S. Surgical perfor-
mance measurement. Health Care Manage Sci 2002;5(4):243—8.
[11] Spalding GJ, Hartrumpf M, Sierig T, Oesberg N, Kirschke CG, Albes JM.
Cost reduction of perioperative coagulation management in cardiac
surgery: value of ‘‘bedside’’ thromboelastography (ROTEM). Eur J Car-
diothorac Surg 2007;31(6):1052—7.
[12] Krabatsch T, Artashes N, Hetzer R. Identification of center-specific risk
factors for complications in sternal wound healing as an essential part of
internal quality management. Thorac Cardiovasc Surg 2007;55(Suppl.
1):S69.
[13] Porter ME. Clusters and the new economics. Harv Bus Rev 1998;76(7):77—
90.
[14] Slogoff S, Keats AS. Does perioperative myocardial ischemia lead to
postoperative myocardial infarction? Anesthesiology 1985;62:107—14.
[15] Baskett RJ, Buth KJ, Legaré JF, Hassan A, Hancock Friesen C, Hirsch GM,
Ross DB, Sullivan JA. Is it safe to train residents to perform cardiac
surgery? Ann Thorac Surg 2002;74(4):1043—8 [discussion 1048—9].
[16] Ellman PI, Kron IL, Alvis JS, Tache-Leon C, Maxey TS, Reece TB, Peeler BB,
Kern JA, Tribble CG. Acute sleep deprivation in the thoracic surgical
resident does not affect operative outcomes. Ann Thorac Surg
2005;80(1):60—4 [discussion 64—5].
[17] Holzhey DM, Jacobs S, Walther T, Mochalski M, Mohr FW, Falk V. Cumu-
lative sum failure analysis for eight surgeons performing minimally
invasive direct coronary artery bypass. J Thorac Cardiovasc Surg
2007;134:663—9.
758 M. Hartrumpf et al. / European Journal of Cardio-thoracic Surgery 35 (2009) 751—759
[18] Goddard M, Mannion R, Smith P. Enhancing performance in health care: a
theoretical perspective on agency and the role of information. Health
Econ 2000;9(2):95—107.
[19] Carey JS, Danielsen B, Gold JP, Rossiter SJ. Procedure rates and outcomes
of coronary revascularization procedures in California and New York. J
Thorac Cardiovasc Surg 2005;129(6):1276—82 [Comment in: J Thorac
Cardiovasc Surg 2005;129(6):1223—5].
[20] Grant SW, Grayson AD, Jackson M, Au J, Fabry BM, Grotte G, Jones M,
Bridgewater B. Does the choice of risk-adjustment model influence the
outcome of surgeon-specific mortality analysis? A retrospective analysis
of 14637 patients under 31 surgeons. Heart 2008;94:1044—9.
[21] Roques F, Michel P, Goldstone AR, Nashef SA. The logistic EuroSCORE. Eur
Heart J 2003;24(9):881—2.
Editorial comment
Quality measurement in adult cardiac surgery: a challenge
Keywords: Cardiac surgery; Quality control; Outcome measures
The manuscript presented by Hartrumpf et al. concerns
a current but sensitive topic [1]. In contrast with others
reporting and promoting quality assessment at unit level
[2—4], the article highlights the surgeon’s individual
performance, expressed by the Surgeon Performance Index
(SPI). In this respect, the article is progressive, presenting
public data, good and bad, without distinction. The authors
thus take up a vulnerable position, which takes courage.
However, despite the efforts and the good intention, the
article contains several pitfalls. It is important to recognize
these pitfalls, especially in the case of quality control.
Otherwise, misuse of articles such as this will undermine the
importance and power of quality assessment.
1. Variables
The authors use early mortality, early rethoracotomy for
bleeding, sternal rewiring for instability and mediastinitis as
variables for their assessment and use the EuroSCORE for risk
adjustment [5]. Firstly, why not use variables which have an
accepted association with quality of care, as presented by
several quality improvement organizations [2—4]? Secondly,
the used variables differ from the definitions given in
internationally used systems, thus making benchmarking
impossible. Moreover, the used definitions are free to
interpretation. Sternum instability is defined as visible
movement of sternal edges, necessitating sternum rewiring.
This definition contains two weak points. For one, ‘visible
movement’, while everybody knows patients with a sternal
dehiscence without visible movements. Secondly, ‘necessi-
tating sternum rewiring’ which is not an event but a
therapeutic decision. Therefore it is biased. The same
goes for early rethoracotomy. As defined by the authors, it
is a therapeutic decision, but what about patients, with
[22] Toumpoulis IK, Anagnostopoulos CE, Derose Jr JJ, Swistel DG. The impact
of deep sternal wound infection on long-term survival after coronary
artery bypass grafting. Chest 2005;127(2):464—71.
[23] Ottino G, De Paulis R, Pansini S, Rocca G, Tallone MV, Comoglio C, Costa P,
Orzan F, Morea M. Major sternal wound infection after open-heart
surgery: a multivariate analysis of risk factors in 2.579 consecutive
operative procedures. Ann Thorac Surg 1987;44(2):173—9.
[24] Paul M, Raz A, Leibovici L, Madar H, Holinger R, Rubinovitch B. Sternal
wound infection after coronary artery bypass graft surgery: validation
of existing risk scores. J Thorac Cardiovasc Surg 2007;133(2):397—
403.
[25] Kaplan RS, Norton DP. The balanced scorecard measures that drive
performance. Harv Business Rev 1992;70(1):71—9.
minimal blood loss undergoing a rethoracotomy for cardiac
tamponade?
2. Mortality
It is important that when mortality is used, theoretically,
the whole early phase, about 6 months postoperative, must
be considered [6]. This means that an active follow-up is
necessary [6]. Because of difficulties with follow-up, most
centers use their hospital mortality. With the early discharge
policy most patients leave the cardiac surgical center within
10 days postoperative. This means that, even using the 30-
day mortality, as the authors did, an active follow-up is
needed. In order to have a good idea about the value of the
used rate of mortality in the evaluation, the method of
follow-up and the results must be presented.
3. Patient population
Besides clearly defined variables, quality control requires
a good description of the patient population. It is preferable
to focus on programs; at this moment it is generally accepted
that quality reports should focus on CABG, the most
commonly performed standardized cardiac procedure, with
several well-defined outcome, process and structure vari-
ables. It is even accepted that results of CABG give a good
idea of the overall adult cardiac surgery performance.
However, it is the authors’ choice to include valve surgery in
their studied population. Despite this decision, some
agreements must be clear. What about high-risk patients?
It is known that an additive EuroSCORE > 10 is not precise and
therefore it is probably better not to include the, generally
few, patients with a score > 10 in the assessment [7].