You are on page 1of 15

international journal of health planning and management

Int J Health Plann Mgmt 2004; 19: 303–317.

Published online in Wiley InterScience ( DOI: 10.1002/hpm.766

Organization and delivery of primary health

care services in Petrópolis, Brazil
James Macinko1*, Celia Almeida2, Eliane dos Santos Oliveira2,
and Paulo Klingelhoefer de Sá3
Department of Nutrition, Food Studies, and Public Health, Steinhardt School of Education,
New York University, USA
Department of Health Administration and Planning, National School of Public
Health/Oswaldo Cruz Foundation, Rio de Janeiro, Brazil
Faculty of Medicine of Petrópolis, Municipal Secretariat of Health, Petrópolis, Brazil

The objective of the study was to adapt and apply an instrument to measure the organizational
features of the primary care system in the municipality of Petrópolis. The study compared the
performance of the new Family Health Program (Programa Saúde da Famı́lia or PSF) with
traditional primary care facilities using data from facility surveys and key informant inter-
views. The main results include: (a) the methodology was capable of distinguishing between
the two types of primary care services in the municipality; (b) the PSF clinics scored higher
on most dimensions of primary care, although in some areas the traditional health units had
equivalent scores; and (c) data obtained from interviewing key informants was generally com-
patible with that obtained by conducting facility surveys. The results suggests that in spite of
making important advances in primary care, the municipality of Petrópolis continues to face
several challenges including the need to improve access, enforce the gatekeeper role of
primary care, and improve the coordination and community orientation of both types of
primary care services. The methodology could be used to set objectives and monitor progress
towards improving the organization and delivery of primary care in Petrópolis and elsewhere.
Copyright # 2004 John Wiley & Sons, Ltd.

key words: primary health care; health care reform; family health program; Brazil


Since the end of the 1980s the health system in Brazil has been undergoing a process
of health sector reform, the landmark of which was the new constitution approved in
1988. The constitution institutionalized universal social rights, including the right to
health, which then became a citizen’s right and the State’s obligation and responsi-
bility, and created the Brazilian national health system (Sistema Único de Saúde or

* Correspondence to: Dr J. Macinko, Assistant Professor of Public Health, Department of Nutrition,

Food Studies, and Public Health, Steinhardt School of Education, New York University, 35 West 4th
Street, 12th Floor, New York, NY 10012-1172, USA. E-mail:
Contract/grant sponsors:Fulbright Commission; Pan American Health Organization.

Copyright # 2004 John Wiley & Sons, Ltd.


SUS). The antecedents date back to the 1970s, when a political movement advo-
cated and actively worked for a new health system that could surmount the historical
segmentation of access to health care. This social and political movement and the
reform process itself coincided with a combination of economic crises, important
political changes (transition to democracy) and a significant revitalization of both
organized civil society and the political party system. These factors had an impor-
tant influence on the government’s policy agenda and its handling of social and eco-
nomic issues. Implementation of health sector reform was strongly influenced by
these conditions and fraught with contradictions and conflicts due to both domestic
and international political and economic conditions, as well as health sector con-
straints and the priorities of State reform (Almeida et al., 1999).
Throughout the Latin American region, health sector reforms have primarily had
the aim of streamlining healthcare financing and decentralizing authority for plan-
ning and implementation, and have increasingly incorporated other objectives such
as improving the quality of care and enhancing equity (Almeida et al., 1999). These
objectives have developed against the backdrop of widespread poverty, increasing
inequality, social unrest and environmental deterioration (Londoño and Szekely,
1997; Szekely, 2001). However, unlike the health sector reform process of the past
decade in other countries within the Latin-American region, health sector reform in
Brazil was at first not included on the government agenda, either as a demand or as a
result of adjustment policies. On the contrary, the political and organizational inno-
vation proposed for the Brazilian health system ran counter to these policies,
because it was based on ideas that had guided the organization of health systems
in post-war Europe and was strongly influenced by the philosophy of Keynesian
welfare social rights and of the State as service provider (Almeida, 1999, 2000,
2002a, 2002b).
Certain dimensions of the decentralization process acquired specific characteris-
tics in the health area and formed the axis of SUS implementation. Although
changes were made in the rights and responsibilities of the different levels of gov-
ernment and in the rules for resource allocation, important links with the central
level were maintained. Decentralization was implemented by the Basic Operating
Norms (Normas Operacionais Básicas—NOBs in 1991, 1992, 1993 and 1996)
and more recently by the Operating Norms for Health Care (Norma Operacional
da Assistência à Saúde-NOAS) in 2001 and 2002. Local government has since
become a stronger service provider and interregional differences in service supply
have diminished. The intensity of these changes, however, differed greatly from one
region to another, as well as within the same region or even within the same state
(Barros et al., 1996; Lucchese, 1996; Almeida et al., 1999).
In response to these challenges the Brazilian government developed several
special programs such as the community health workers (Programa de Agentes
Comunitários de Saúde or PACS) in 1991, and the family health program (Pro-
grama de Saúde da Famı́lia or PSF) since 1994 to bolster the provision of primary
health care services. At the same time, major changes have been introduced in terms
of health policy meant to improve primary health care. For example, the NOB laid
out the responsibilities of different levels of government in the provision of
health care and the government developed a national system of per capita funding

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.
for primary health care known as the Basic Health Floor (Piso de Atenção Básica or
As one of the main program and policy responses to growing health inequities
within Brazil, the Family Health Program has witnessed considerable growth. It
has expanded from 900 teams in 900 municipalities in 1996, to about 5000 teams
in 1870 municipalities in 1999, to over 10 000 teams in more than 3000 of Brazil’s
5561 municipalities in 2000 (Aguiar, 1998; MS, 2000). The PSF program is com-
posed of interdisciplinary health teams based in a health center with a geographi-
cally defined catchment area and in most places includes a team of community
health workers. Innovative characteristics of the PSF program include its emphasis
on reorganizing primary care to become the gatekeeper of the health system, inte-
grating primary care with other social services, changing the focus of care from the
individual to the family, and developing community participation and outreach
methods such as regular visits to each household in a given geographic district.
In spite of the considerable investment in the Family Health Program, to date,
there has been little research into the extent to which these innovative features have
actually changed the way Brazilians receive primary care. The vast majority of
existing evaluations focus on program financing, human resource needs, types of
municipalities most likely to adopt the PSF, and characteristics influencing the
adoption of the program at the local level (Vianna and Pierantoni, 2002; Svitone
et al., 2000; Escorel et al., 2002; Medina and Aquino, 2002; Ministério da Saúde,
1999; Canesqul and de Oliveira, 2002).
The study builds upon experiences using the Primary Care Assessment Tool
(PCAT), an instrument that previously has been used to evaluate primary care ser-
vices in the United States and Canada (Starfield, 1998, 2000; Shi et al., 2001).
Research in the USA and in European countries has suggested that strong primary
care systems are associated with lower costs, higher patient satisfaction and better
quality of care (Starfield, 1994; Starfield and Shi, 2002). There is mounting evidence
that they are also associated with better population health, at least in wealthy indus-
trialized nations (Macinko et al., 2003; Starfield, 1992; Starfield, 1998; Boerma and
Fleming, 1998; Bindman et al., 1996; Bunker, 2001; Casanova and Colomer, 1996).
Nevertheless, there is surprisingly little evidence of the impact of primary care on
population health in developing countries (Hill et al., 2000).
In our early discussions, municipal health authorities indicated their desire for a
rapid and valid way of measuring changes in primary health care center organization
and service delivery. The need for a rapid monitoring and evaluation tool was par-
ticularly important as the PSF program is growing by approximately 25% per year.
For this reason, we developed the pilot stage of the project that employed key infor-
mants as a way of streamlining the data collection process. This study presents
results from a rapid assessment of the organization of primary care services in Pet-
rópolis, Brazil. It aims to characterize essential features of the primary health care
system, test whether these features differ between the traditional and the newly
reformed PSF primary health care centers, and assess the extent to which key deci-
sion-makers are cognizant of these differences.
In this paper, we use ‘primary care’ rather than ‘primary health care’ to refer to the
most basic level of health care provided in the health system. This is in accordance

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

with the term ‘atenção básica’ (basic care) used in Brazil. We use the term ‘clinics’
to refer to the most basic level of health service that provides primary care. The Por-
tuguese translation of the word ‘clinics’ has a different connotation in Brazil. Thus,
readers familiar with the Brazilian health system should read the word ‘clinics’ as
‘unidades’ (health units) or ‘centros’ (health centers) and not ‘clı́nicas’.


Primary care measures

This study used a functional definition of primary care since primary care definitions
such as those developed by the Brazilian Ministry of Health—‘the sum of indivi-
dual and community-based actions located at the first level of health care and
oriented towards health promotion, disease prevention, treatment, and rehabilita-
tion’ (MS 1999; authors’ translation)—do not readily lend themselves to measure-
ment of the degree of attainment of components related to primary care. A
functional definition of primary care includes structural elements such as accessibil-
ity, range of services provided, definition of a patient population and continuity of
care. It also includes process elements such as receipt of health services and health
problem recognition. All four major domains of primary care—first contact care,
continuity (sometimes called longitudinality to capture its person-focused aspects
over time), comprehensiveness, and coordination of care—can be assessed by
examining these structural and process elements of a health services system.
It is in this spirit that Starfield developed the following functional definition of
primary care as ‘that level of a health system that provides entry into the system
for all new needs and problems, provides person-focused (not disease-oriented) care
over time, provides care for all but very uncommon or unusual conditions, and coor-
dinates or integrates care provided elsewhere or by others’ (Starfield, 1998: 8).
It is possible to combine the health services-oriented model of Starfield with a
broader conception of primary health care that reflects the community and multisec-
torial nature of primary care as practised in the developing world. This conceptua-
lization additionally allows for the synthesis of the categories proposed by Starfield
with the community-oriented approach suggested by Alma Ata.
Based on an analysis of Starfield’s (1998) work, and literature on Health for All
by the Year 2000 (WHO, 1978), a preliminary list of the main attributes of primary
care systems would include:
 Accessibility—the presence (or absence) of financial, organizational, and/or
structural barriers to receiving primary care;
 First contact—the extent to which primary care serves as the entry point to
other levels of care (in non-emergency situations);
 Longitudinality—whether primary care is continuous (person-focused) over
 Comprehensiveness—the extent to which all essential services needed to
provide for the majority of population health needs are offered at primary care

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.
 Coordination—the extent to which primary care facilitates patients’ care
between levels and with other important social services and sectors;
 Family-focused—the extent to which primary care considers the patient within
the wider context of his or her familial environment;
 Community orientation—how well primary care responds to community
needs, promotes community participation in health, and is involved in intersec-
toral actions designed to promote health;
 Provider characteristics—the type and extent of training of primary care

In order to capture these primary care dimensions, the study utilized a version of
the Johns Hopkins University Primary Care Assessment Tool (PCAT). The PCAT is
a valid and reliable tool for measuring dimensions of primary care and includes ver-
sions for use with either health providers (facility-level survey) or clients (Shi et al.,
2001). The facility-level survey was translated from English to Portuguese, back-
translated, and extensively pre-tested. Additional questions were added and several
sections (such as the list of essential primary care services provided) were adapted to
better reflect the primary care context of Brazil.
Two versions of the questionnaire were prepared. The facility survey was directed
toward health providers and questions refer to the health clinic in which they work.
The second version utilized the same questions but phrased them so that responses
would describe health managers’ understanding of the entire primary care system
that they oversee.

The municipality of Petrópolis is located in the state of Rio de Janeiro in the South-
Eastern portion of Brazil. Table 1 presents basic demographic and health data on
the municipality. It was chosen to test the methodology because of its history of
development of an acceptable level of health service coverage, its role as an early
adopter of the family health program, and the persistence of numerous traditional

Table 1. Characteristics of Petrópolis, 2002

Indicator Petrópolis Rio de Janeiro
State (average)

Area (km2) 811 43 696

Population 286 348 14 724 479
Per capita income (in Brazilian Reais) 7588 9788
Infant mortality rate (per 1000 live births) 23.20 19.74
Low birth weight births (%) 10.34 8.58
Live births to mothers with 4 or more prenatal visits (%) 90.53 89.67
Mortality from cerebrovascular diseases (pr 100 000) 79.57 60.23
Family health program coverage (% of population) 14.7 12.9

Source: Instituto Brasileiro de Geografia e Estatistica (IBGE) 2002.

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

clinics functioning alongside family health clinics. Petrópolis is considered a mid-

sized municipality with a population close to 300 000. Per capita income is slightly
lower than the average for the state of Rio de Janeiro, and health indicators are, on
average, slightly worse than the average for the state of Rio de Janeiro. The family
health program currently covers about 15% of the population, traditional health
clinics cover up to an additional 60% of the population for primary care, and the
private sector covers the remaining proportion.
The facility survey was administered by a trained interviewer to health provi-
ders (physician, chief nurse, or both) in a sample of reformed (PSF) and traditional
clinics. The sample included all functioning clinics that had been in operation
for at least 6 months and that reported delivering a minimum of 1000 ambulatory
care procedures during the past year. The final selection represented 80% of all
government primary care clinics in the municipality of Petrópolis and included
23 out of a total of 26 PSF and 10 out of a total of 15 traditional clinics. The num-
ber of ambulatory care procedures was verified using the Brazilian Ministry
of Health’s on-line database called SIASUS (Sistema de Informação Ambulatorial
do SUS).
Key informants were selected from supervisors and municipal health authorities
in Petrópolis. Informants were selected based on their experience working in the
health system in Petrópolis (minimum of 5 years), their professional qualifications
(physician or nurse), and their current role as supervisor or manager of the entire
network of PSF or traditional clinics. Two informants were interviewed regarding
the performance traditional health clinics, two were asked about the PSF clinics, and
one was asked about the entire public primary care system.

The questionnaire was designed with 100 questions; each pertaining a specific
dimension of primary care. Interviewers responded to a likert-type scale for each
question with values ranging from 0 (never) to 5 (always). Scores were then
summed for each question, and summed across each of the eight dimensions of pri-
mary care. Differences in scores between types of clinic (PSF vs traditional clinics)
were tested using the Chi-square (2) test. Differences were considered significant
at the p < 0.05 level. This limit is considered conservative since the overall sample
size was small (n ¼ 33) (Fisher and Van Belle, 1993).
Several mechanisms were utilized to verify the validity of responses. First, key
informants were carefully chosen based on their current work position, number of
years working in Petrópolis, and their professional qualifications. Second, each
facility survey respondent performed a self-evaluation of their confidence in the
answers they provided. This technique helped to identify nurses and physicians as
the preferred sources of information on facilities, since tests with community health
workers and nurse assistants indicated a much lower level of confidence in their
ability to provide data on the overall organization and delivery of primary care ser-
vices in their own clinic. Finally, secondary data were utilized in order to perform a
process of triangulation to confirm overall findings derived from facility surveys and
key informant interviews (Gomes Vı́ctoria et al., 2000).

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

Tables 2a and 2b present results of the facility surveys by individual indicator. The
first column represents the mean and standard deviation for the sum of individual
responses at the PSF clinics. The second column presents results for the traditional
clinics. The final column presents the results of the chi-square test of difference
between the PSF and traditional clinic scores.
In terms of access, there was a significant difference between PSF clinics and tra-
ditional clinics in the area of weekend access ( p < 0.05). Some traditional clinics
have weekend hours while no PSF clinics do. Neither type of clinic charged users
for visits, and PSF and traditional clinics reported having adequate stocks of med-
icines and supplies only some of the time. Both types of clinics reported that most of
the time patients should be able to get an appointment for non-urgent care within
24 h and should not have to wait more than 30 min at the clinic before being seen
by a medical professional.
The PSF clinics score slightly higher than the traditional clinics in terms of first
contact, but this difference was not significant.
Longitudinality differed between PSF and traditional clinics in two important
ways: nearly all PSF clinics have a geographically defined population while
traditional clinics do not and professionals report that in PSF clinics patients
always have enough time to discuss their doubts with their provider, whereas in tra-
ditional clinics this is reported to happen only some of the time ( p < 0.05). Other
aspects of longitudinality did not significantly differ between the two types of
Comprehensiveness of care data showed that PSF clinics were more likely to pro-
vide prenatal care, treatment for tuberculosis and counselling on domestic violence
than traditional clinics ( p < 0.05). PSF clinics more frequently provided health
education and counselling on alcohol and tobacco use than did traditional clinics
(if statistical significance is set at the p < 0.1 level). Both types of clinics provided
most types of primary care almost always, however, both clinic types were deficient
in the provision of minor surgeries, dental health, health education and treatment of
minor mental health problems.
Both PSF and traditional clinics show average levels of coordination. PSF clinics
were superior to traditional clinics in terms of having written treatment protocols,
guidelines for referral processes, and discussing referral options with the client
( p < 0.05). Neither type of clinic experienced periodic audits of medical records.
PSF clinics appear to be more family-focused than traditional ones. PSF clinics
were more likely to organize records by family rather than individual, and more
likely to allow the family to be present during the medical consultation ( p < 0.06).
Community orientation differs between clinics in terms of PSF clinics more likely
to have conducted surveys of community health, more likely to make home visits
and more likely to have engaged in intersectorial collaboration than traditional
clinics ( p < 0.05).
In terms of health personnel, PSF clinics are more likely to have had physicians
specifically trained in primary care, and a team trained in understanding the cultural
diversity of the community in which they work ( p < 0.05).

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

Table 2a. Primary care indicators

Indicator PSF clinics Traditional 2 test for
(n ¼ 23) clinics (n ¼ 10) difference
Meana SD Meana SD

Adequate medication 3.17 1.190 2.60 1.170 0.222
Adequate equipment 3.26 1.210 3.00 1.410 0.924
Co-pay 5.00 1.040 5.00 0.000 0.503
Get appointment in <24 hours 3.04 1.610 2.60 1.650 0.097
Open weekends 0.00 0.000 1.00 2.100 0.027
Open 1 night after 6 pm 0.04 0.660 0.70 0.421 0.372
Telephone to make appointments 4.69 0.208 3.90 1.630 0.156
After hours telephone 0.13 0.457 1.00 2.100 0.344
Waiting time <30 minutes 2.77 1.230 3.10 1.720 0.134
First contact (gatekeeper) 3.43 1.720 2.20 1.980 0.333
Able to see the same provider 3.52 1.120 4.40 0.966 0.127
Geographically defined population 4.78 0.518 2.33 2.290 0.002
Discuss doubts with professional 4.48 1.080 3.00 1.563 0.033
Appointment long enough 4.56 0.662 4.20 1.135 0.271
Consistent use of medical records 5.00 0.000 5.00 0.000 n/a
Provider knows which medications you use 4.34 0.787 4.40 0.699 0.823
Provider knows if you can’t buy medications 3.96 1.065 3.60 1.349 0.488
Vaccinations for children 4.56 0.945 4.90 0.316 0.697
Child health services 4.95 0.208 5.00 0.000 0.503
Adult health services 4.95 0.208 5.00 0.000 0.503
Elderly health services 5.00 0.000 5.00 0.000 n/a
Prenatal care 5.00 0.000 4.22 1.641 0.015
Family planning 4.35 0.884 4.12 1.356 0.243
Treatment for STDs 4.74 0.751 4.75 0.463 0.332
Tuberculosis 2.87 2.221 0.13 0.353 0.031
Endemic diseases 3.91 1.649 2.89 2.260 0.231
Epidemic diseases 4.22 1.412 2.75 2.052 0.164
Chronic diseases 4.83 0.650 4.00 1.772 0.121
Diabetes 4.91 0.288 5.00 0.000 0.361
Hypertension 4.91 0.288 5.00 0.000 0.336
Minor injuries 4.95 0.208 4.90 0.316 0.532
Small surgeries 0.52 1.162 1.22 1.986 0.368
Alcohol and tobacco counselling 4.56 1.199 2.89 2.204 0.079
Minor mental health 3.13 1.841 2.55 2.242 0.664
Nutrition 3.04 1.894 4.25 1.752 0.171
Health education 2.83 1.800 1.00 1.322 0.061
Domestic violence 2.91 1.564 1.89 1.833 0.021
Home accident prevention 3.35 1.921 3.78 1.922 0.566
Dental health promotion 1.65 2.248 3.22 2.438 0.694
Scores range from 0 (never) to 5 (always).

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.
Table 2b. Primary care indicators
Indicator PSF clinics Traditional 2 test for
(n ¼ 23) clinics (n ¼ 10) difference
Meana SD Meana SD

Treatment protocols present 4.39 1.033 3.22 2.166 0.045
Clinic-held child health records 4.74 1.053 5.00 0.000 0.629
Client-health child health records 4.95 0.208 4.80 0.421 0.151
Clinic-held records of pregnant women 5.00 0.000 4.44 1.666 0.104
Client-held records of pregnant women 4.91 0.288 4.33 1.658 0.145
Guidelines for information transfers 3.70 1.550 4.78 0.440 0.002
Guidelines for referrals 4.40 1.196 3.56 1.013 0.105
Discuss referral options w/ client 4.78 0.518 4.10 1.523 0.003
Make referral appointments 4.78 0.518 2.80 2.097 0.123
Provide written referral information 4.82 0.576 4.50 0.971 0.110
Receive results of specialist referral 2.48 1.274 2.40 1.897 0.466
Protocols for laboratory tests 4.82 0.576 4.56 1.013 0.457
Discuss lab results in primary care 4.22 1.782 4.80 0.632 0.253
Clients informed of lab results 4.30 1.717 4.30 1.636 0.425
Review of need for referral 3.35 1.897 1.89 2.420 0.180
Clients see their medical records 2.30 2.119 3.00 2.507 0.437
Medical records always available 5.00 0.000 4.90 0.316 0.124
Periodic medical record audits 0.56 1.036 0.43 0.534 0.293
Family focus
Medical records organized by family 4.39 1.616 0.00 0.000 0.000
Provider asks about family risks 4.26 0.963 4.44 0.726 0.836
Family can be present during exam 4.52 0.790 3.70 0.948 0.061
Social risk factors assessed in exam 4.82 0.387 4.50 0.707 0.193
Community orientation
Community satisfaction surveys 1.74 1.321 1.40 1.776 0.510
Community health surveys 2.69 1.819 1.20 1.813 0.052
Community representation 3.27 2.142 1.44 2.185 0.112
School health services 3.00 1.834 1.78 1.855 0.366
Home visits 4.78 0.850 1.55 2.068 0.001
Intersectorial collaboration 3.56 1.561 2.50 2.000 0.018
Autonomy to change services 3.78 1.622 1.78 1.922 0.113
Health personnel
Clinic has  1 physician present 4.95 0.208 5.00 0.000 0.503
Nurses substitute for physicians 3.95 1.521 3.80 1.751 0.171
Physicians trained in primary care 4.69 0.702 3.12 2.230 0.023
Others trained in primary care 4.69 0.558 3.87 1.642 0.108
Team trained in cultural diversity 4.30 1.222 1.88 2.267 0.006
Scores range from 0 (never) to 5 (always).

Table 3 compares the overall scores between PSF and traditional clinics. Scores
are calculated as an unweighted average of all indicators in each dimension. Both
types of clinics scored lower than expected average levels of access (an average
score is considered to be 2.5 out of 5 points), average levels of first contact care
and slightly higher than average levels of comprehensiveness and coordination.
PSF clinics had significantly higher levels of longitudinality, family focus and

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

Table 3. Primary care scores

Summed scale Items Scale PSF clinics Traditional 2 test for
# reliability (n ¼ 23) clinics (n ¼ 10) difference
Cronbach  p-value
Mean SD Mean SD

Access 10 0.65 2.18 0.337 2.27 0.744 0.262

First contact 1 n/a 3.43 1.730 2.20 1.990 0.333
Longitudinality 6 0.48 4.39 0.388 3.87 0.536 0.050
Comprehensiveness 23 0.77 3.98 0.409 3.70 0.615 0.365
Coordination 19 0.46 3.73 0.472 3.54 0.256 0.498
Family focus 4 0.57 4.50 0.617 3.13 0.463 0.009
Community orientation 7 0.80 3.26 0.980 1.54 1.290 0.249
Professional training 5 0.59 4.52 0.407 3.60 1.040 0.012

professional training than traditional clinics ( p < 0.05). PSF clinics also showed
stronger community orientation than traditional clinics although this difference
was not statistically significant.
Table 3 presents summary scores for each dimension of primary care. Scales vary
in terms of their reliability scores. Dimensions such as access, comprehensiveness
and community orientation have scores close to 0.70, indicating that the scale for
each of these dimensions could be considered minimally reliable. The other dimen-
sions have scores below 0.60, indicating that the items comprising these scales may
be measuring more than one concept.
Table 4 shows a comparison between facility survey results and those obtained
from key informant interviews. As a general observation, nearly all scores obtained
from key informants are lower than those derived from the facilities themselves.
Overall, the key informants’ evaluations agreed with the facility surveys 62.5%
of the time for the PSF clinics and 87.5% of the time for traditional clinics. In terms
of access, key informants significantly differed from facility surveys in both PSF
and traditional clinics. PSF key informants also differed from facility surveys in
their assessment of longitudinality and coordination ( p < 0.05).

Table 4. Agreement between facility survey scores and key informant ratings
Scale PSF PSF Agreement Traditional Traditional Agreement
facility informant facility informant

Access 2.2 1.3 Noa 2.3 1.2 Noa

First contact 3.4 3.0 Yes 2.2 3.5 Yes
Longitudinality 4.4 3.6 Noa 3.9 3.6 Yes
Comprehensiveness 3.9 3.9 Yes 3.7 3.6 Yes
Coordination 3.7 3.0 Noa 3.5 3.3 Yes
Family focus 4.5 4.7 Yes 3.1 2.9 Yes
Community 3.2 3.1 Yes 1.5 1.9 Yes
Professional 4.5 4.6 Yes 3.6 4.2 Yes
Difference between scores statistically significant (2 test for difference p < 0.05).

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

The contemporary discussion on evaluation of health systems describes the impor-

tance of defining the limits of the health system in order to focus on features that
are amenable to intervention by health authorities (PAHO, 2001). However, given
the global tendency toward decentralization of health and other social services, a
wide variety of government, non-governmental and private sector provider are
offering an increasing proportion of health services worldwide (WHO, 2001).
The multiplicity of healthcare providers combined with the competing definitions
of the scope and role of primary care complicates the task of defining the appropriate
scope and role of the health system. For example, the literature on primary care
defines it at times as a level of the health care system, a specific set of clinical
services, or a strategy for addressing the fundamental determinants of health
(Vuori, 1985). The tool presented here focuses on the organizational and functional
features inherent in any mode of primary care provision. It sought to assess primary
care not only as a direct provider of basic health services but also as that of a catalyst
(but not necessarily the sole or even lead instigator) of broader human development
The results suggest the utility of a rapid assessment methodology for measuring
the essential dimensions of primary care. The questionnaire was capable of measur-
ing specific indicators of primary care quality, distinguishing between the two types
of clinics, and providing feedback to managers on each primary care dimension. As
expected, reformed (PSF) clinics performed better overall than the traditional pri-
mary care clinics, although there was important variation among PSF clinics and
between PSF and traditional clinics.
The results also suggest that the use of health managers as key informants pro-
vided important complementary information and aided in validation of survey find-
ings. Health managers and facility surveys rated clinics similarly in 62.5% of cases
in PSF clinics and in 87.5% of cases among traditional clinics. Given the increased
scrutiny of PSF clinics as a policy and programmatic innovation, one would expect
that there would be wider agreement in terms of PSF clinic performance. The higher
level of variation in PSF scores is especially surprising given the fact that the PSF
program contains a comprehensive health management information systems (the
Primary Care Information System known as the Sistema de Informação da Atenção
Básica or SIAB) as well as a specialized municipal-level monitoring and evaluation
system (known as the Primary Care Pact or Pacto de Atenção Básica). The SIAB, in
particular, provides clinic, regional and municipal-wide reports of all procedures
performed and all diagnoses made in each PSF clinic. Previous evaluations of the
PSF program have documented that local health authorities do not routinely use
health information systems in day-to-day operations, an observation supported by
the evidence presented here (Escorel et al., 2002).
PSF clinics did not earn uniformly higher scores than traditional clinics in all
primary care dimensions. These discrepancies can be explained partially by the
historical development of primary care in Petrópolis. First, the PSF is a relatively
new phenomenon in Petrópolis, having begun with only a few pilot clinics in
1997. Currently, the PSF is in a process of rapid expansion and, at the time of the

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

study, some PSF clinics had been in operation for only 6 months. This might explain
the lower level of agreement between scores obtained directly from PSF clinics and
those obtained by interviewing municipal health managers. Perhaps more impor-
tantly, the process of primary care reform in Petrópolis—as opposed to many other
Brazilian municipalities—has also included investment in the system of traditional
clinics. The results presented here show that, in addition to the development of the
PSF program, Petrópolis has developed a parallel network of traditional clinics that
are reasonably organized and comprehensive.
Overall scores for access, gatekeeping and community orientation were low for
both types of clinics. This result is likely to reflect the fact that although primary
care reform in Petrópolis has included development of new financing mechanisms
to expand the PSF program, structural conditions within the overall health system
(SUS) have not uniformly benefited from these reforms. Both PSF and traditional
clinics still rely on the same system of referral and procurement, for example. This
implies that in order to improve the impact of health reform in Petrópolis, simply
expanding the number of PSF clinics will not be enough: systematic changes in the
way the patients are referred between levels of care will be also be necessary.
Additional results merit comment. The fact that it is more likely that a client
would see the same provider in traditional rather than PSF clinics is surprising given
that the PSF clinics are staffed by only one physician and one nurse, while tradi-
tional clinics usually have multiple physicians rotating through the clinic over the
course of the week. Discussion with clinic staff revealed that within PSF clinics,
nursing staff tend to have more defined roles than in traditional clinics. Thus, clients
presenting to PSF clinics will sometimes have their health problem attended by nur-
sing staff instead of the physician.
Regarding the percent of the clientele that is geographically defined, nearly 50%
of traditional clinics reported that some of their population was geographically
defined—even though traditional clinics do not have any defined catchment area.
Conversations with clinic staff revealed that traditional clinics also attend to a
sizeable proportion of clients who are members of communities that are served
by the PSF program. Clinic staff claim that, from time to time, PSF clients also
attend traditional clinics because some traditional clinics have more convenient
hours, are closer to workplaces, or have shorter waiting times than PSF clinics.
In terms of longitudinality, the fact that PSF clinics earned higher scores in this
dimension is likely to be due to the fact that PSF providers seem better able to allow
clients to clarify their doubts during the consultation. Nevertheless, providers in
both types of clinics claim that they give sufficient time during consultations. It
could be that the focus on family orientation in PSF clinics, their location within
defined communities, and the on-going training of PSF providers have improved
PSF providers’ rapport with their clients.
The PSF demonstrated higher levels of coordination than traditional clinics.
These practices would be expected to be similar between both clinic types since they
are each referring clients to the same set of specialists. The improved performance
of PSF providers is most likely related to on-going training provided to all PSF staff.
This in-service training is not provided to traditional clinic staff. In addition, the
Secretary of Health provides PSF clinics with additional management and logistic

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.
support that is not available in equal measure for traditional clinics. One recommen-
dation from discussion with municipal authorities is that some of the benefits of the
PSF program, such as professional training and additional logistical support, could
be applied in traditional clinics as well.
It is also important to identify a practice that is commonplace in both clinics: the
use of the medical chart in every medical consultation and the practice of taking
medical histories that include enumeration of all medication the patient may be tak-
ing. This is in accordance with the generally high level of medical training in Brazil.
Finally, it is important to point out that the instrument was applied as part of a
rapid assessment methodology. In general, interviews were conducted in under
20 minutes, suggesting that the instrument could be used as part of ongoing moni-
toring and evaluation plans. Presentation of results to municipal health authorities
and managers of PSF and traditional clinics resulted in a rich discussion specific to
attainment of quality primary care in both types of clinicsparticularly the possibility
of sharing best practices from both PSF and traditional clinics and piloting these
practices across all clinic types. The results of the instrument facilitated the devel-
opment of specific, measurable objectives for quality improvement rather than more
general discussions of perceived performance problems and resource shortages
which have characterized previous discussions.
This study has several important limitations. First, the sample size was small
(n ¼ 33) thus the power to detect significant differences was limited. However, those
differences that were detected are likely to be important in practice. Small sample
size is also likely to have influenced our ability to adequately test the psychometric
properties of the instrument.
Another limitation is the fact that PSF clinics are constantly being added to
Petrópolis and that at least three of the PSF clinics assessed had been in operation
for only 6 months. It is likely that this contributed to the variation in performance
ratings as well as the lower percentage of agreement between key informants and
facility surveys in PSF settings. The traditional clinics have a longer history of
operation in the municipality. For this reason, key informants may have a better idea
of the organizational and operational features of these clinics.
In spite its limitations, the Petrópolis experience provided insight into the nature of
health reform in Brazil. Instead of the idea of the PSF program replacing the
traditional system of primary care, in Petrópolis it seems a new model of hybrid
systems is beginning to take place. This hybrid appears due to the historical devel-
opment of primary care in the municipality, fiscal constraints which have forced local
decision-makers to try to obtain the most from all available resources, and perceived
political difficulties associated with discontinuing the system of traditional clinics
which, although they are often maligned as being of poor quality, are viewed by most
citizens as a fundamental right and the responsibility of government.


This study was partially funded by the Fulbright Commission and the Pan American
Health Organization. Additional support was provided by the Department of Health

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.

Administration and Planning at the National School of Public Health/FIOCRUZ in

Rio de Janeiro, and the Office of the Municipal Secretariat of Health of Petrópolis,
Brazil. The authors wish to thank Barbara Starfield for helpful comments on an ear-
lier draft of this manuscript.


Aguiar DS. 1998. A Saúde da Famı́lia no Sistema Único de Saúde: Um novo Paradigma?
Dissertação de Mestrado. Fundação Oswaldo Cruz/Escola Nacional de Saúde Pública,
Almeida CM. 1999. Reforma do Estado e reforma de sistemas de saúde: Experiências
internacionais e tendências de mudança. Ciência & Saúde Coletiva 4: 263–286.
Almeida CM. 2000. Saúde e eqüidade nas reformas contemporneas. Saúde em Debate 24:
Almeida C. 2002a. Eqüidade e reforma setorial na América Latina: um debate necessário.
Cadernos de Saúde Pública 18(Suplemento 2002): 23–36.
Almeida C. 2002b. Health systems reform and equity in Latin America and the Caribbean:
lessons from the 1980s and 1990s. Cad. Saúde Pública, jul./ago 18: 905–925.
Almeida CM, Travassos C, Porto S, Labra ME. 1999. Health reform in Brazil: a case on
inequity. Intern J Health Serv 30: 1.
Barros ME, Piola SF, Vianna SM. 1996. Polı́tica de Saúde no Brasil: Diagnóstico e
perspectivas. Texto para discussão 401. IPEA: Brası́lia.
Bindman A, Grumbach K, Osmond D, Vranizan K, Stewart A. 1996. Primary care and receipt
of preventive services. J Gen Intern Med 11: 269–276.
Boerma W, Fleming D. 1998. The Role of General Practice in Primary Health Care. The
Stationery Office/World Health Organization Regional Office for Europe: Norwich.
Bunker JP. 2001. The role of medical care in contributing the health improvements within
societies. Int J Epidemiol 30:1260–1263.
Casanova C, Colomer C. 1996. Pediatric hospitalization due to ambulatory care-sensitive
conditions in Valencia (Spain). Int J Qual Health Care 8: 51–59.
Canesqul AM, de Oliveira AM. 2002. Saúde da Famı́lia: modelos internacionais e estratégia
brasileira. Em O Sistema Único de Saúde em Dez Anos de Desafio. Negri B, Viana AL
(organizadores). Sobravime, Cealog: São Paolo.
Escorel S. (Coord.); Giovanella L, Mendonça MH, Magalhães R, Senna MCM. 2002.
Avaliação da Implementação do Programa Saúde da Famı́lia em Dez Grandes Centros
Urbanos. Sı́ntese dos Principais Resultados. Brasil. Ministério da Saúde, Secretaria de
Polı́ticas de Saúde. Departamento de Atenção Básica/DAB.
Fisher L, Van Belle G. 1993. Biostatistics: A Methodology for the Health Sciences. John
Wiley: New York.
Gomes Vı́toria C, Riva Knauth D, de Nazareth Agra Hassen M. 2000. Pesquisa Qualitativa
em Saúde. Tomo Editorial: Porto Alegre.
Hill AG, Macleod WB, Joof D, et al. 2000. Decline of childhood mortality in children in
rural Gambia: the influence of village-level primary health care. Tropical Med Int Health 5:
Londoño JL, Székely M. 1997. Persistent Poverty and Excess Inequality: Latin America,
1970–1995. Working Paper Series 357. Inter-American Development Bank: Washington,
Lucchese P. 1996. Descentralização do financiamento e gestão da assistência à saúde no
Brasil: A implementação do Sistema Único de Saúde, 1990/1995. Planejamento e Polı́ticas
Públicas 14. IPEA: Brası́lia.
Macinko J, Starfield B, Shi L. 2003. The contribution of primary care systems to health
outcomes within organization for economic cooperation and development (OECD)
countries, 1970–1998. Health Services Research 38: 819–854.

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.
Medina MG, Aquino R. 2002. Avaliando o Programa de Saúde da Familia. Em Os Sinais
Vermelhos do PSF, MF de Sousa (organizadora). Editora Hucitec: São Paulo; 135–151.
Ministério da Saúde. 1999. Manual para a Organização da Atenção Básica. Ministério da
Saúde, Sectária de Assistência à Saúde: Brası́lia.
Or Z. 2001. Exploring the Effects of Health Care on Mortality Across OECD Countries.
Organization for Economic Cooperation and Development (OECD): Paris.
Pan American Health Organization (PAHO). 2001. Regional Consultation of the Americas on
Health Systems Performance Assessment (background paper for the regional consultation).
Final Report. WHO: PAHO. 2001. Health System Performance Assessment and
Improvement in the Regions of the Americas. PAHO/WHO: Washington, DC.
Shi L, Starfield B, Xu J. 2001. Validating the adult primary care assessment tool. J Family
Practice 50: 161–175.
Starfield B. 1992. Primary Care: Concept, Evaluation, and Policy. Oxford University Press:
Starfield B. 1994. Primary care: is it essential? Lancet 344: 1129–1133.
Starfield B. 1998. Primary Care: Balancing Health Needs, Services and Technology. Oxford
University Press: London.
Starfield B. 2000. (Ed. Español; primeira edição inglês 1998), Atención Primaria—
Equilibrio entre Necesidades de Salud, Servicios y Tecnologı́a. Fundación Jordi Gol i
Gurina/SCMFIC/Masson: Barcelona.
Starfield B. 2003. (Ed. Em português; primeira edição em inglês 1998). Atenção Primária:
Equilı́brio entre necessidades de saúde, serviços e tecnologia. UNESCO: Brası́lia.
Starfield B, Shi L. 2002. Policy relevant determinants of health: an international perspective.
Health Policy 60(3): 201–216.
Svitone E, Garfield R, et al. 2000. Primary health care lessons from the Northeast of Brazil.
Pan American Journal of Public Health 7(5): 293–302.
Székely M. 2001. The 1990s in Latin America: Another Decade of Persistent Inequality, but
with Somewhat Lower Poverty. Working Paper 454. Inter-American Development Bank:
Washington, DC.
Vianna AL, Pierantoni CR. 2002. Indicadores de Monitoramento da Implementação do PSF
em Grandes Centros Urbanos. Relatório Sı́ntese e Relatório Metodológico (mimeo).
Vuori H. 1985. The role of schools of public health in the development of primary health care.
Health Policy 4: 221–230.
Walt G. 1994. Health Policy—An Introduction to Process and Power. Witwatersrand
University Press: Johannesburg, New Jersey and London.
WHO. 1978. Declaration of Alma Ata. World Health Organization: Geneva.
WHO. 2000. World Health Report 2000. World Health Organization: Geneva.
WHO. 2001. Macroeconomics and Health: Investing in Health for Economic Development.
Commission on Macroeconomics and Health—Final Report. WHO: Geneva.

Copyright # 2004 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2004; 19: 303–317.