International Journal for Quality in Health Care 2005; Volume 17, Number 4: pp. 287–292 10.

1093/intqhc/mzi040
Advance Access Publication: 26 April 2005

Measuring the quality of hospital

tuberculosis services: a prospective
study in four Zimbabwe hospitals
CHARLES HONGORO1,2, BARBARA McPAKE2 AND PETER VICKERMAN2
1
Aurum Health Research, Economics and Health Systems, Johannesburg, Gauteng, South Africa, 2London School
of Hygiene and Tropical Medicine, Public Health and Policy, London, UK.

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

Abstract
Objective. To show how the use of a prospective approach to measuring the quality of services for a specific diagnosis can
generate useful information for improving the quality of services in environments with limited information technology and data.
Design. Tracer approach focusing on intensive treatment of tuberculosis in hospital. The study was conducted in Zimbabwe in
1999. Local tuberculosis management guidelines were first translated into explicit quality assessment criteria and a panel of
public health experts assisted in weighting different factors (structural and process) of the criteria. Factor weightings were
based on both local knowledge and experience, and potential contribution of a factor to the likelihood of a positive outcome.
A total of 138 patients was recruited into the study cohort at admission and followed up to discharge. An assessment of what
was done to and for the patient was made for the entire hospitalization episode using explicit criteria. Comparisons were made
between actual and maximum performance scores.
Setting. The study was conducted at four regional referral hospitals. The hospitals serve at least six secondary hospitals, and
several public and private primary care facilities. The hospitals have a dual role as they also provide secondary care to their
immediate catchment population.
Results. Notable quality gaps are observed between actual and maximum quality levels in all four hospitals although the size of
the gap differed significantly. Variation in the quality of services between the hospitals is explained by distinguishable differ-
ences in structural and process aspects of tuberculosis management.
Conclusions. It is feasible to conduct prospective quality assessment in developing countries with minimal disruption of routine
activities. The study also showed that prospective exploration of health care quality for a specific diagnosis can provide insights
into hospital-level quality issues. Such information is useful for monitoring and improving the quality of hospital services in
general.
Keywords: quality of services, tuberculosis, Zimbabwe

Health systems around the world are being reformed. Pressure
for change derives from the need to improve access, effi-
ciency, effectiveness and quality of health services. How-
ever, it remains unclear what the impact of these reforms
has been on quality of services [1]. To understand this
requires facing the challenge of assessing the quality of
health services in developing countries where information
technology and systems for data collection are still in their
embryonic stage.
The difficulties of measuring quality of health services corre-
spond to those of defining it. The measurement method used
depends on how quality is defined and what is feasible given
the prevailing circumstances. This study sought to define and
measure quality in a manner that was both sensitive to the con-
text, and capable of exposing areas responsive to improvement.
Many have grappled with the conceptualization and opera-
tionalization of the quality of care concept [2–6]. Attempts to
be comprehensive have often led to quality definitions that
are difficult to measure in practice. The study adopted the
Institute of Medicine [7] definition that: ‘Quality of care is the
degree to which health services for individuals and popula-
tions increase the likelihood of desired outcomes and are con-
sistent with current professional knowledge’ because this
definition is focused and could be translated in the study context.
Donabedian’s [3] structure–process–outcome paradigm is
commonly used in quality assessment studies. In developing
countries, quality assessment studies have tended to focus on
structural and process aspects of quality [8,9] because of the
problems of data availability and reliability. Data tend to be
collected for mostly administrative purposes. Furthermore,

Address reprint requests to Charles Hongoro, E-mail: charles.hongoro@lshtm.ac.uk,hongoro@hotmail.com

International Journal for Quality in Health Care vol. 17 no. 4

© The Author 2005. Published by Oxford University Press on behalf of International Society for Quality in Health Care; all rights reserved 287
C. Hongoro et al.
these studies have relied on retrospective data the quality and
completeness of which is often questionable, and patient
satisfaction surveys. Little attention has been given to assess-
ing the quality of in-patient services, and even less to using
prospective approaches. In this study an attempt was made to
correct for data deficiencies in estimating quality of in-patient
services by using prospective patient-specific methods.
Use of specific tracer diseases to explore quality issues in
health care is not new [10]. Use of tracer diseases in combina-
tion with explicit management criteria might provide an
understanding for improving hospital services quality.
Approaches available for setting explicit quality criteria
include review of literature, panel of experts [11], and use of
consensus building methods like the Delphi technique
[12,13]. When using a panel of experts, it is important to
ensure that the panel is constituted from practitioners working
in the system.
There are two broad ways of measuring quality perform-
ance: (i) categorization of care into qualitative divisions such
as ‘excellent’, ‘good’, ‘fair’, or ‘poor’; and (ii) use of quantitative
weighted scores for specified aspects of care. The weight
should reflect the relative importance of each factor to the
total quality score. A panel of experts or literature review is
used to assign the scores and weights. Providers are then
ranked according to the total scores, and also against an
expected yardstick quality score [14]. A provider performance
index (PPI) is an example of such a total score [12,13].
However, the use of numerical scores faces two major
challenges: (i) medical care sometimes has an all or nothing
aspect which the component numerical scores might not
reflect; and (ii) the manner in which different treatment com-
ponents are weighted to arrive at a total score remains subjective.
In this study, a local panel of experts assisted in designing the
weighting system.
Methods
Choice of tracers and case definitions
The study was carried out in four tertiary hospitals in Zimbabwe
in 1999. The hospitals are regional referral centres with
150–235 beds. Tuberculosis was selected for the study
because it is a major cause of hospital admissions and deaths.
In 1997, 3965 tuberculosis cases were hospitalized (7% of
total admissions), and 438 patients died (20% of all hospital
deaths) in one province [15]. The study was supported by the
Ministry of Health and Child Welfare who provided national
tuberculosis management guidelines.
The case definition for tuberculosis was described accord-
ing to site and sputum status or history. Using the former,
sputum-positive tuberculosis refers to a patient with two con-
secutive positive sputum smear examinations or with one
positive smear examination and a chest X-ray suggestive of
tuberculosis. Historically, a new case was defined as a patient
who had never received a full course of tuberculosis treat-
ment or who had received treatment for less than 1 month
[16]. The treatment protocol for such patients was Category 1
288
high priority: 2 months of intensive treatment followed by a
continuation phase of 4 months. The drug regimens based on
WHO guidelines are given in the Essential Drug List and
Standard Treatment Guidelines for Zimbabwe [17]. For
practical purposes, only the principal diagnosis was consid-
ered in recruiting patients. Ninety per cent of all admitted
tuberculosis cases were estimated to be HIV positive [15],
which meant that cases were likely to be homogeneous in that
respect. Co-morbidities tend to increase hospital stay, but the
presence of home-based care programmes for chronic ill-
nesses at these hospitals might have reduced the effect on
length of stay.
Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020
Setting quality criteria and case recruitment
Explicit normative quality criteria for tuberculosis manage-
ment were developed from the national management guidelines
with the help of a selected panel of regional experts: three
public health physicians and two senior nursing officers.
National guidelines formed the core to which other clinically
relevant but non-clinical process and structural factors were
added. The criteria included diagnostic, therapeutic manage-
ment, patient and environmental hygiene, documentation,
and nursing variables. Each quality component was assigned a
score weighted according to its relative importance in increas-
ing the likelihood of the desired positive outcome. Process
aspects of quality were entirely patient specific. Organiza-
tional and policy-related process factors were excluded
because of physical constraints.
For each study site, two research assistants (health account-
ants) with nursing experience were recruited for patient
recruitment and follow-up. They were trained in the application
of the quality criteria, followed by trial runs on at least two in-
patient cases each. Training was essential for standardizing
data abstraction and ratings. Patients with symptoms of
tuberculosis were conveniently recruited from hospital outpa-
tient and casualty departments. Once recruited, they were
visited every day, and as and when necessitated by critical
moments in their treatment process. The assistant recorded
what was done to and for the patient. Data were collected from
patient charts and interviews (where possible), and ward staff
(on non-sensitive aspects or verification of patient records).
A total of 138 cases was recruited from four hospitals.
Suspected tuberculosis patients that were admitted for pre-
sumptive treatment and/or investigative purposes, and were
later on either referred to a specialized infectious disease hos-
pital (if positive) or discharged (if negative), were excluded
from the analysis. Informed consent was obtained from the
patients or their guardians before recruitment in the study.
Patient codes were used to ensure confidentiality and protec-
tion of people’s medical data. The study obtained ethical
approval from the Medical Research Council of Zimbabwe as
part of a larger study on hospital performance.
Data were coded by hospital and patient. Data were com-
puterized and analysed using SPSS. A total of 39 quality var-
iables (including background characteristics) was created,
and each variable entry represented a weighted score (Tables 1
and 2). A total weighted score was computed for each
 Quality of hospital TB services

n ing possible similarities in case-severity. However, high levels

case/patient: ∑ Xij , where Xij is the weighted score for each of self-referral (30–51%) reflect defective referral systems.
i
quality variable i for each patient j and n is the number of The hospitals had significantly different average lengths of
applicable variables per patient. The number of applicable stay. Case fatality rates were relatively high probably because
variables depended on whether the overall score was being of high levels of HIV infection.
calculated or whether the score by quality cluster, process or Table 2 shows a translation of the explicit quality criteria
structural, was being calculated. These weighted scores were for tuberculosis into a weighted schema as advised by the
used to calculate the average weighted score per patient/case panel of experts. The weights reflect the importance of a fac-
in each hospital: tor to both local quality expectations and prognosis, and were
used to establish maximum overall and cluster scores. The
 N n 
 PPI measures the extent to which hospital care providers per-
 ∑ ∑ Xij /N  , formed in accordance with explicit criteria.
 j=1i=1  The overall quality scores for all the hospitals are shown in

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

where N is the total number of cases/patients in a particular
hospital. The actual scores were compared against the max-
imum scores, calculated using maximum scores derived from
the explicit criteria for tuberculosis management. By calculat-
ing subcategory or cluster scores it was possible to examine
which factors influenced the overall score or quality perform-
ance. A PPI was also calculated by dividing the observed
scores (AS) by the maximum scores (MS), and measures the
extent to which hospital care providers performed according
to explicit criteria, and takes values from 0 to 1. A value of 1
means total conformity with the quality criteria and zero
reflects total non-conformity. Statistical comparisons of quality
scores across hospitals were made using chi-squared and
analysis of variance (ANOVA).
Results
The general characteristics of the study cohort are shown in
Table 1. The mean age of patients ranged from 34 to 37 years,
and no significant differences were found in sex mix. No
marked differences in the proportion of referred and unre-
ferred cases were observed across the study hospitals suggest-
Table 3. None of the hospitals fully met the desired standard
of service. Hospital 2 had the highest score whilst Hospital 3
had the lowest. Actual scores exceeded 64% of the maximum
overall score. However, a shortfall of up to 36% for a fairly
manageable disease such as tuberculosis raises serious ques-
tions about the quality of service. The overall scores differed
significantly between the hospitals (P < 0.001) signifying
dissimilarities in tuberculosis protocol compliance levels
between hospitals. The overall difference between maximum
and observed overall scores might be explained by the rela-
tively large discrepancies that exist between optimum and
actual structural quality scores (rather than between actual
and optimum process scores). To enhance interpretation of
the overall scores, it is necessary to look at the component
parts separately.
Attempts to interpret structural quality scores should
recognize the dearth of knowledge on the relationship
between structure and process. All but Hospital 4 fell
within the upper quarter of the maximum score even
though statistically significant differences were observed
between them (P < 0.001). This means that the basic infra-
structure to potentially provide services of comparable quality
was available. The low score for Hospital 4 is explained by
poor ratings in the following factors: availability of drugs,

Table 1 Sample description

Hospital/variable 1 2 3 4 P-value
(n = 31) (n = 41) (n = 37) (n = 29)
....................................................... .....................................................................................................................................................................
1
Mean patient age, years (range) 34 (18–61) 33 (5–36) 26 (1–56) 39 (21–77) 0.003
Patient sex, n (%)
Female 15 (48) 14 (35) 16 (44) 11 (38) 0.5272
Male 16 (52) 26 (65) 20 (56) 18 (62)
Referral status, n (%)
Referred 16 (52) 20 (49) 26 (70) 19 (68) 0.1522
Self-referred 15 (48) 21 (51) 11 (30) 9 (32)
Length of stay(days) 6.1 6.5 9.0 10.3 0.0091
3
Case fatality rate, n (%) 4 (13) 4 (10) 6 (16) 2 (7)

Changes in sample sizes across factors is attributable to missing values.

1
ANOVA test.
2
Pearson χ2.
3
No statistical test because of small numbers.

289
C. Hongoro et al.

Table 2 Criteria for calculation of maximum weighted mean presence of floor beds (overcrowding), inadequate micro-
scores for pulmonary tuberculosis scopes, and poor building condition. For Hospital 2, the
low score is explained by inadequate hand-washing facilities
Criteria Weighting Overall score in the wards and functional equipment, and the presence of
............................................................................................................ floor beds. The high score for Hospital 1 (>70% of the
A Process factors maximum score, 25) was due to better drug availability,
Time of contact with 1 1 working equipment, and the absence of floor beds. Hospi-
staff since arrival at facility tal 3 had adequate space, and water and sanitation facilities.
Laboratory 1 Significant differences in process quality scores were
Sputum collection 1 observed (P < 0.001). Hospital 4 had the highest process
Collection in open air/ 1 score (25/31) whereas Hospital 2 had superior ratings for the
well-ventilated room majority of process aspects except for supervision of sputum
Specimen collected under 1 collection, direct observation of treatment, and patient pri-

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

supervision of competent vacy compared with the remaining two hospitals. This means
person that Hospitals 2 and 4 provided relatively better diagnostic,
Laboratory turnaround 1 clinical, and nursing services for in-patients. What is striking
time is that Hospital 4 had the lowest structural quality score but
X-ray taken 1 the highest process score. Hospital 1 had a relatively low pro-
X-ray turnaround time 1 cess quality rating even though its PPI surpassed 69%. Low
Weighing of patient1 1 process quality ratings could be linked to hospital clinical
Total laboratory score 1 7 practices (relatively low protocol compliance—frequency and
Intensive phase treatment2 observation of treatment), patient and environmental
Choice of drug 3 hygiene. Despite the observed statistical differences in the
combination: quality scores, at the aggregate level, the PPIs indicate
HRZE or HRZS relatively high levels of treatment compliance across all four
Drug dosage by weight 3 hospitals
Treatment directly 1
observed
Patient privacy 1 Conclusions
Total treatment score 8
Nursing services Measuring the quality of hospital services is difficult, and
Personal hygiene 2 more so in contexts where information technology is poor.
Elimination 2 Even where data exist, they are of poor quality because the
Nutrition 6 rationale for collection is usually administrative and not
Environmental hygiene 3 linked, for instance, to improving patient management or
Patient records 2 quality assurance activities. The prospective approach used in
Total nursing score 15 this study might have directly or indirectly introduced a Haw-
Total possible process score 31 thorne effect. We sought to minimize this by avoiding direct
B Structural factors observation of treatment, for instance, the administration of
Availability of tuberculosis 8 an injection, and instead checked (through records and
drugs patient interviews) at timely intervals whether an injection
Availability of toilets 1 was given or not. The use of two research assistants at each
Cleanliness of toilets 1 study site for data collection allowed for cross-checking of
Availability of safe water 1 data quality, and it enhanced patient recruitment. No compar-
Availability of lighting 1 isons were made to assess consistency. Such comparisons
Availability of hand-washing 1 could have improved the quality of the results. Overall, the
facilities study showed that prospective assessment of the quality of
Availability of floor beds 1 tuberculosis services in hospitals was feasible and allows for
Availability of microscopes 1 an examination of both patient-specific and non-patient-spe-
Condition of microscopes 2 cific aspects of hospital services.
Condition of buildings 2 At the patient level, it was clear that the management of
Availability of staff 6 tuberculosis services in all hospitals did not meet the local and
(nurse and doctors) international standards because of structural and process defi-
Total possible structural score 25 ciencies. Hospital 4 had two wards (male and female) with
Total weighted score 56 low but similar structural scores. Although Hospital 4 had a
poor structure for delivering tuberculosis services it had rela-
1 tively better process scores attributable to better nursing serv-
Not necessarily a laboratory activity but a necessary precursor to
effective treatment. ices, environmental hygiene and diagnostic response. The

290
 Quality of hospital TB services

Table 3 Tuberculosis mean weighted quality scores across hospitals

Hospitals/quality 1 2 3 4 Maximum P-value1

scores (n = 31), n (SD) (n = 41), n (SD) (n = 37), n (SD) (n = 29), n (SD) score
............................................ ................................................................................................................................................................ ..................
2
Structural score 17.6 (1.05) 16.1 (0.69) 16.4 (0.79) 14.0 (0) 25 <0.001
Process score 21.3 (2.18) 24.0 (3.05) 19.7 (1.71) 25.4 (1.70) 31 <0.001
Overall score 38.9 (2.12 40.1 (3.35) 36.1 (1.88) 39.4 (1.70) 56 <0.001
PPI 69.5 (0.04) 71.6 (0.06) 64.4 (0.03) 70.3 (0.03) 1 <0.001
1
ANOVA test.
2
The score was similar for both the female and male wards where the cases were assessed.

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

hospital had the lowest case fatality rate, of 7%. Critical fac-
tors like patient counselling, direct observation of treatment,
record-keeping, and patient hygiene were often ignored by
hospital staff in hospitals which had better process quality
scores.
Explanations of quality differences between hospitals
that account for the differences in scores allow for con-
sideration of the question of whether differences judged
statistically significant have clinical significance. The rela-
tively small absolute size of the differences in scores might
suggest not, but the presence of drug availability and facilit-
ies for basic hygiene among those factors causing differ-
ences suggests that differences may in fact be important for
outcomes.
The absence of adequate facilities and human resources to
deliver quality hospital services, particularly for a high-burden
disease such as tuberculosis in such a context, might not
reflect successes in non-facility-based directly observed treat-
ment (DOTs) programmes but poor funding and coordina-
tion of resources. This situation has been worsened by the
high HIV/AIDS prevalence amongst adults (24%). Intensive
phase treatment is a key component of the national tubercu-
losis programme and forms the basis for the DOTs pro-
gramme. Therefore, the quality of initial treatment for
tuberculosis has implications for the success of the national
programme.
At the hospital level, prospective analysis of the quality of
tuberculosis services allowed for an in-depth exploration of
key hospital services essential for other pathologies such as
radiography and laboratory (diagnostic) services, treatment,
nursing services, and environmental hygiene. Such informa-
tion is crucial in identifying areas of weakness and con-
straints in the hospital system—a prerequisite for any quality
improvement programme. Tracers are used in this study as
diagnostic tools for quality of hospital services, and it is clear
from the results that interventions may generate notable
quality improvements by focusing on structure and process
aspects. The underlying assumption is that appropriate struc-
tures and processes increase the likelihood of positive out-
comes. The foregoing is true for both developed and
developing countries [18,19]. Nevertheless, the need is
greater in developing countries where resources and informa-
tion infrastructure cannot generate outcome measures of
quality.
References
1. Heiby JR, A crisis in quality. Int J Qual Health Care 2002; 14
(suppl.): 5.
2. Donabedian A. Evaluating the quality of medical care. Milbank
Mem Fund Q 1966; 44 (suppl.): 166–206.
3. Donabedian A. The Definition of Quality and Approaches to its Assess-
ment. Ann Arbor, MI: Health Administration Press, 1980.
4. Shortbridge MH. Quality of medical care in an outpatient set-
ting. Med Care 1974; 12: 283–300.
5. Black N. Quality assurance of medical care. J Public Health Med
1990; 12: 97–104.
6. Bitran R. Efficiency and quality in the public and private sectors
in Senegal. Health Policy Plann 1995; 10: 271–283
7. Lohr KN, Institute of Medicine. Medicare: A Strategy for Quality Assur-
ance. Vol. I. Washington, DC: National Academies Press, 1990.
8. De Geyndt W. Managing the Quality of Health Care in Developing
Countries. World Bank Technical Paper No. 258. Washington,
DC: World Bank, 1995.
9. Gilson L, Magomi M, Mkangaa E. The structural quality of
Tanzanian primary health facilities. Bull World Health Organ 1995;
73: 105–114.
10. Kessner DM, Snow C, Singer J. Assessing health quality—the
case of tracers. New Engl J Med 1973; 288: 189–194.
11. Mainz J. Defining and classifying clinical indicators for quality
improvement. Int J Qual Health Care 2003; 15: 523–530.
12. Payne BC, Lyons TF, Neuhaus E et al. Method of evaluating and
improving ambulatory medical care. Health Serv Res 1984; 19: 219–245.
13. Mates S, Sidel VW. Quality assessment by process and outcome
methods: evaluation of emergency room care of asthmatic
adults. Am J Public Health 1981; 71: 687–693.
14. Montoya-Aguilar C. Measuring the performance of hospitals
and health centres. World Health Organization, District Health
Systems, Division of Strengthening of Health Services, WHO/
SHS/DHS/94.2, 1994.
15. Government of the Republic of Zimbabwe, Ministry of Health
and Child Welfare. Benchmark report, 1998. MOHCW, 1998.
16. Government of Republic of Zimbabwe, Ministry of Health and
Child Welfare. Zimbabwe Tuberculosis Control Programme Manual.
MOHCW, 1994.

291
C. Hongoro et al.

17. Government of Republic of Zimbabwe, Ministry of Health and
Child Welfare. Essential Drug List of Zimbabwe and Standard Treatment
Guidelines for Zimbabwe. MOHCW, 1994.
18. Hostein A, Plaschke H, Schleiker et al. Structural and process
quality in the management of diabetic emergencies in Germany.
Int J Qual Health Care 2002; 14: 33–38.
19. Urassa DP, Carlstedt A, Nystrom LG. et al. Quality assessment
of the antenatal program for anaemia in rural Tanzania. Int J
Qual Health Care 2002; 14: 441–448.
20. Government of Republic of Zimbabwe, Ministry of Health and
Child Welfare. Revised Standard Treatment Guidelines on Tuberculosis.
MOHCW, 1998.
21. Heidermann EG. The contemporary use of standards in health
care. Geneva: World Health Organization, 1993.
Accepted for publication 6 March 2005

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

292