You are on page 1of 5

 RESEARCH 

A Conceptual Framework to Measure Performance


of the Public Health System
| Arden Handler, DrPH, Michele Issel, PhD, and Bernard Turnock, MD, MPH

During the past decade, increasing attention


Objectives. This article describes a unifying conceptual framework for the public health system as a
has been focused on performance measure-
ment in the delivery of medical care. This at- way to facilitate the measurement of public health system performance.
tention has centered on the various relation- Methods. A conceptual framework for the public health system was developed on the basis of the
ships between organizational structure, work of Donabedian and a conceptual model previously developed by Bernard Turnock and Arden
clinical practices, and patient outcomes, with Handler.
the strong recognition that the practice of Results. The conceptual framework consists of 5 components that can be considered in relationship
medicine should be evidence based. The to each other: macro context, mission, structural capacity, processes, and outcomes.Although the avail-
movement toward evidence-based medicine ability of measures for each of these components varies, the framework can be used to examine the
has been accompanied by a research agenda performance of public health systems as well as that of agencies and programs.
supported by public agencies such as the Conclusions. A conceptual framework that explicates the relationships among the various compo-
Agency for Healthcare Research and Quality
nents of the public health system is an essential step toward providing a science base for the study of
as well as efforts toward performance mea-
public health system performance. (Am J Public Health. 2001;91:1235–1239)
surement supported by accrediting bodies
such as the National Committee for Quality
Assurance and the Joint Commission on Ac- system began to use this core function frame- of Donabedian,12 which links structure,
creditation of Healthcare Organizations. work to conceptualize the practice of public processes, outputs, and outcomes in a model
The activities of these accrediting bodies, health and to assess aspects of public health for quality assessment and systems monitor-
other private research institutes, and the fed- performance.2–10 These efforts, however, were ing. Bernard Turnock and Arden Handler ad-
eral government have not led to a unified of limited value for several reasons, including vanced a similar model as the basis for ex-
conceptual framework for assessing medical their focus on only one aspect of public health amining public health system performance
care system performance per se. However, system performance, the key processes associ- during the mid-1990s.13 In their earlier ef-
health services researchers who focus on the ated with public health practice. With one no- fort, these authors examined the history of
performance of the medical care delivery sys- table exception,11 they were also largely fo- attempts to measure public health perform-
tem understand that their efforts are part of a cused at one level of the public health system, ance in the United States and concluded that
larger strategy to enhance the quality of med- local public health performance. Most impor- these past efforts lacked an adequate concep-
ical care and thus improve individual patient tant, without a conceptual framework that de- tual framework for defining the public health
outcomes. scribed the components of the public health system.
Unfortunately, there has been no parallel system, there were few attempts to under- The framework proposed in this article was
movement, research agenda, or conceptual stand the effects of external forces on the developed in conjunction with an expert
framework to allow for an examination of the overall public health system or its subsystems panel as well as the Public Health Practice
performance of the public health delivery sys- or to examine the relationships among the dif- Program Office of the Centers for Disease
tem and the relationship between the practice ferent system components. Control and Prevention (CDC). Figure 1 de-
of public health and population outcomes. To provide a science base for the study of picts the main components included in the
This lack of a focus on public health system public health system performance, it is neces- proposed framework. As shown in the figure,
performance has stemmed partly from a lack sary to articulate a conceptual framework that the public health system includes 4 compo-
of consensus on how to operationalize the explicates the various components of the pub- nents: mission, structural capacity, processes,
mission of public health. During the 1990s, lic health system and the relationships be- and outcomes. These system components are
however, the public health community moved tween them. In this article, we propose such a affected by a fifth component, the macro con-
to redefine the operational aspects of its mis- framework. text.
sion in light of the Institute of Medicine’s Fu- Measuring public health system perform-
ture of Public Health report, which described OVERVIEW ance—the extent to which the system
the broad functions of public health as assess- achieves its mission—requires the ability to
ment, policy development, and assurance.1 The proposed conceptual framework for measure each of the components of the sys-
Researchers and practitioners interested in the public health system as a foundation for tem and their relationships with each other.
the science base of the public health delivery measuring performance is based on the work Although each component is described and

August 2001, Vol 91, No. 8 | American Journal of Public Health Handler et al. | Peer Reviewed | Research Articles | 1235
 RESEARCH 

discussion of issues related to measurement is


provided for each component.

FRAMEWORK COMPONENTS

Macro Context
Macro context represents the supra-system
level and the milieu that directly or indirectly
affects the existence and functioning of the
public health system. It incorporates phenom-
ena such as the social, political, and economic
forces operating in the overall society (e.g.,
the national economy at any given point in
time); the extent of demand and need for
public health services within the population;
social values and preferences for the products
of the public health system (e.g., clean water);
and forces external to the public health sys-
tem that exert pressure on it to function in
particular ways (e.g., the medical delivery sys-
tem, technologic advances, and the nature of
federal–state–local relationships).
Inclusion of the macro context in the
FIGURE 1—Conceptual framework of the public health system (PHS) as a basis for model demonstrates that the public health
measuring system performance. system is engaged in a dynamic relationship
with a host of factors external to its own mis-
sion and purpose. The macro context can af-
discussed separately, the public health sys- which the system operates. If the mission fect the performance of the public health sys-
tem is assumed to be an open system with and functions of the public health system tem through its impact on the system’s
relationships that lead to interaction and mu- are to be achieved, the appropriate struc- mission (e.g., changes in the medical care sys-
tual adjustments among the components. tural capacity (e.g., human and information tem may affect how public health defines its
Likewise, numerous feedback loops are as- resources) must be in place. The resources role), on capacity (e.g., only a limited amount
sumed to exist between components; most of and relationships that constitute this capac- of fiscal or human resources may be available
these loops are not explicated here, in order ity are used to carry out the processes of for the public health system), on processes
to keep the framework as parsimonious as public health, those that identify and priori- (e.g., technologic advances may affect the effi-
possible. tize population health needs and determine cacy of interventions), and on outcomes (e.g.,
This framework can be used as the basis how they will be addressed, as well as those the relevance of particular health status out-
for measurement of the performance of the that represent the outputs of these more comes is dependent on social values and need
public health system as a whole (the variety fundamental processes, public health serv- at any point in time).
of agencies and organizations engaged in the ices, policies, and interventions. These sys- To date, questions about the context in
practice of public health) or of a specific pub- tem processes constitute public health prac- which the public health system operates, as
lic health organization. It can be applied at tice. The ultimate results of public health well as its impact on system components and
multiple levels to examine the national public practice are system outcomes, typically relationship to system performance, have not
health system, state and local public health measured as improvements in population been well formulated. However, researchers
systems, or community public health systems. health status. and practitioners interested in understanding
While the model can be applied to examine Descriptions of the components of the con- the impact of the social, economic, and politi-
the performance of a specific public health in- ceptual framework and the relationships cal milieu on public health system perform-
tervention or program, the focus here is on among them are provided subsequently. ance will probably be able to obtain measures
more complex systems. While there are clearly a multitude of re- of specific macro context variables from a va-
As shown in Figure 1, the mission, struc- search- or practice-based questions that can riety of public and private sector data
tural capacity, processes, and outcomes of be addressed with this framework, these are sources. A host of possible questions and
the public health system are affected by the not offered here. In meaningful instances, measures exist; however, for many of the
social, economic, and political milieu in however, some examples are included, and a macro context constructs of interest (e.g., soci-

1236 | Research Articles | Peer Reviewed | Handler et al. American Journal of Public Health | August 2001, Vol 91, No. 8
 RESEARCH 

etal values), there currently may be insuffi- are among the most useful single sources of • Develop policies and plans that support
cient measures or no measures at all. information about the structural capacity of individual and community health efforts.
local public health systems. Similar informa- • Enforce laws and regulations that protect
Mission tion had been available on state public health health and ensure safety.
The mission of the public health system in- agencies until the mid-1990s through the As- • Link people with needed personal health
cludes its goals at any point in time and how, sociation of State and Territorial Health Offi- services and ensure the provision of health
at the conceptual level, these goals are opera- cials reporting system, operated by the Public care otherwise unavailable.
tionalized. At the beginning of the 21st cen- Health Foundation. • Ensure a competent public health and
tury, the mission of public health is to ensure The Lewin Group17 compiled an extensive personal health care workforce.
conditions in which people can be healthy.1 inventory of data sources related to obtaining •Evaluate the effectiveness, accessibility,
This mission is conceptualized as being car- information on public health infrastructure. and quality of personal and population-based
ried out through the performance of the core While this inventory demonstrates that there health services.
functions of assessment, policy development, is no single, complete source of data on the • Conduct research to produce new in-
and assurance. These functions have been de- structural capacity of the public health sys- sights and innovative solutions to health
fined and described in various ways since tem, the conceptual model presented here problems.
they were characterized in the Institute of provides an opportunity to identify a coherent
Medicine report; however, they have come to set of questions in order to draw upon exist- These essential public health services can
represent the general ways in which public ing data sets and begin to systematically gen- be viewed as partly cyclic. The cycle begins
health problems are identified and addressed erate knowledge about structural capacity vis- with the identification and investigation of
through organized, collective efforts. à-vis other system components. These efforts health problems. These initial processes, in
Measuring the “mission” of the public may lead to a demand for the creation of conjunction with the process of mobilizing
health system as distinct from its other com- more complete and consistent measures of and educating communities, lead to the devel-
ponents may be possible. One could imagine the structural capacity of the public health opment of policies and plans for interven-
an examination of the impact of changes in system and may also assist practitioners in tions. Through the activities of a competent
the public health mission during the 20th identifying areas of capacity that require workforce, these policies and plans are trans-
century on system capacity or processes with strengthening. lated into the outputs or interventions of the
“time” as a surrogate for mission. Likewise, if public health system, the enforcement of reg-
Processes
the aim is to examine mission or purpose ulations and laws, and the development of
The practice of public health can be
across systems (e.g., international compar- other interventions and services to which in-
thought of in terms of the key processes
isons), it may be possible to operationalize dividuals and populations are linked. Al-
through which practitioners seek to identify,
whether the mission of a particular public though research can contribute at several
address, and prioritize community or popula-
health system is “population based” or fo- points in this cycle, evaluation creates the
tionwide health problems and resources and
cused on “personal health services.” feedback loop from the public health system’s
the outputs of these more fundamental
outcomes to planning. However, the results of
processes, public health’s interventions, poli-
Structural Capacity evaluation activities clearly add to the re-
cies, regulations, programs, and services. The
The structural capacity of the public health search findings in any particular area.
processes of public health are those that iden-
system is the cumulative resources and rela- It is very likely that there are alternatives
tify and address health problems as well as
tionships necessary to carry out the important to the feedback loops described here. This
the programs and services consistent with
processes of public health. Structural capacity description represents only a portion of the
mandates and community priorities. At the
includes the following elements: information relationships that might be explicated and po-
beginning of the 21st century, the processes
resources, organizational resources, physical tentially considered by those interested in the
of public health are expressed as “essential
resources, human resources, and fiscal re- role of public health practice in public health
public health services”18 and represent the
sources. More detailed descriptions of the ele- system performance.
core of public health practice. These essential
ments of structural capacity can be found in Historically, the majority of efforts to mea-
services are as follows:
earlier work by Turnock.14 sure public health practice have been focused
Measures of the structural capacity of the • Monitor health status to identify commu- on the measurement of exposure to categori-
public health system exist in many forms and nity health problems. cal public health interventions (outputs). Over
are available from many sources. The Na- • Diagnose and investigate health problems the past decade, however, with the explica-
tional Association of County and City Health and health hazards in the community. tion of public health’s core functions through
Officials has published several national pro- • Inform, educate, and empower people the essential public health services frame-
files15,16 of local health departments, with an about health issues. work, there have been several efforts to de-
extensive assessment of public health infra- • Mobilize community partnerships to iden- velop generic measures of public health prac-
structure currently under development. These tify and solve health problems. tice that have gone beyond the focus on

August 2001, Vol 91, No. 8 | American Journal of Public Health Handler et al. | Peer Reviewed | Research Articles | 1237
 RESEARCH 

categorical interventions. Increasingly, the Outcomes and its potential or actual impact on 1 or
unit of measurement for public health prac- Ideally, carrying out the system’s planning more health status outcomes (e.g., immuniza-
tice is shifting from the categorical program to and policy development processes generates tions and infectious diseases, prenatal care
the community and organization. Because interventions (outputs) intended to improve and adverse pregnancy outcomes). However,
public health practice is more than the sum of health status, the bottom line of the public very little of this work has also linked public
categorical programs, efforts to measure its health system. These immediate and long- health system outcomes to public health sys-
processes must transcend programs even as it term changes experienced by individuals, tem processes such as assessment and plan-
includes them. families, communities, providers, and popula- ning or to the structural capacity of the sys-
In collaboration with staff from the CDC’s tions are the system’s outcomes, the cumula- tem (e.g., human resources or information
Public Health Practice Program Office, re- tive result of the interaction of the public resources).
searchers based at the University of Illinois at health system’s structural capacity and Likewise, research on generic public health
Chicago and the University of North Carolina processes, given the macro context and the practice2–9,11,19 has primarily focused solely on
developed and tested a variety of measures of system’s mission and purpose. Outcomes can the performance of public health practice
public health practice performance. These ef- be used to provide information about the sys- (processes) rather than on the relationships
forts sought to answer questions about per- tem’s overall performance, including its effi- between practice performance and other sys-
formance of core function–related processes ciency, effectiveness, and ability to achieve tem components such as structural capacity.
by local health agencies within the communi- equity between populations.21 For example, whereas some researchers have
ties they serve,3–9 resulting in the develop- Measurement of the structural capacity of examined expenditures with respect to essen-
ment of 20 consensus measures of core func- the public health system (e.g., dollars spent, tial public health services,23–28 they have not
tion–related local public health performance number of adequately trained personnel) focused on the relationship between these ex-
based on field tests conducted between 1991 and even the processes of public health penditures and actual public health practice
and 1995.19 might be undertaken with a relatively lim- performance. However, others have at-
The CDC is developing a more extensive ited set of measures. It is more difficult to tempted to examine the relationship between
set of performance measures for state and imagine using a limited set of measures to public health practice and structural capac-
local public health practice as part of the Na- assess system outcomes, particularly be- ity29,30 and, in one instance, between public
tional Public Health Performance Standards cause each unique intervention or output health practice and aspects of the macro con-
Program. These performance standards will may be linked to a multitude of outcomes.22 text as well.30 Only 1 report has examined
be included in revisions to the Assessment To guide the assessment of public health the relationship between process performance
Protocol for Excellence in Public Health20 as system performance with respect to out- and measures of community health status,
a new self-assessment and capacity-building comes, the nation has established national and it revealed no consistent relationship be-
tool for community public health systems, health objectives every decade since 1990. tween process performance and outcomes.31
Mobilizing for Action through Planning and For most but not all of these objectives, ade- It is likely that the explication of a concep-
Partnerships (MAPP). These performance quate surveillance systems (e.g., vital tual framework for the public health system
measures may also be useful as part of a vol- records) are in place that allow easy access as the basis for measuring public health per-
untary national accreditation program for to data to track changes in outcomes over formance will encourage researchers to exam-
state and local public health organizations. time. If these outcome measures are linked ine relationships between the different com-
Likewise, these national performance mea- to information on the capacity or generic ponents of the model. Similarly, such a
sures have the potential to provide researchers processes of the public health system, re- framework could support performance man-
as well as practitioners with the first nationally searchers and practitioners may then begin agement and improvement efforts in the prac-
agreed-upon indicators of public health prac- to develop a better understanding of the tice sector. Over the past decade, state and
tice performance. However, the measures’ po- particular contribution of the public health local public health improvement plans have
tential in regard to answering questions about system to changes in health status beyond struggled to consider how the effects of en-
public health practice performance will de- the benefit typically derived from an evalua- hanced resources and relationships can be
pend on the prevalence and timeliness of their tion of a specific public health program or measured and linked to the performance of
implementation. If their use is widespread (or intervention. public health processes and, ultimately, out-
even mandatory), and if data are collected at comes. As a result, efforts in the practice com-
regular intervals, there may finally be a na- RELATIONSHIPS BETWEEN munity have promoted rebuilding the public
tionally agreed-upon set of measures that will COMPONENTS health infrastructure (e.g., Health Alert Net-
allow comparisons in public health practice work funding), organizing state and local pub-
performance over time and will enable exami- To date, the majority of published research lic health practice around the essential public
nation of the relationship of public health on public health system performance has fo- health services framework (e.g., the National
processes to structural capacity and outcomes, cused on the implementation of a specific cat- Public Health Performance Standards Initia-
as well as mission. egorical public health intervention (output) tive), and achieving common health objectives

1238 | Research Articles | Peer Reviewed | Handler et al. American Journal of Public Health | August 2001, Vol 91, No. 8
 RESEARCH 

(e.g., Healthy People 2010). Although these Contributors 12. Donabedian A. Explorations in Quality Assessment
activities are often conducted simultaneously All of the authors contributed to the development of and Monitoring: The Definition of Quality and Ap-
the conceptual model. A. Handler had major responsi- proaches to Its Assessment. Vol 1. Ann Arbor, Mich:
in the practice community, their links and in- bility for preparation and writing of the paper. M. Issel Health Administration Press; 1980.
terrelationships have never been explicitly ac- and B. Turnock contributed specific sections and pro- 13. Turnock BJ, Handler AS. From measuring to im-
knowledged. Both the CDC Public Health vided editorial comments on all drafts. proving public health practice. Annu Rev Public Health.
1997;18:261–282.
Practice Program Office and the National
Turning Point Program, a major initiative to Acknowledgments 14. Turnock BJ. Public Health: What It Is and How It
This project was funded by Association of Schools of Works. Gaithersburg, Md: Aspen Publications; 1997.
reform public health systems sponsored by Public Health/Centers for Disease Control and Preven- 15. 1990 National Profile of Local Health Departments.
the Robert Wood Johnson Foundation and tion/Agency for Toxic Substances and Disease Registry Washington, DC: National Association of County
cooperative agreement S755-18/19. Health Officials; 1992.
the Kellogg Foundation, have recognized the
We would like to thank Edward L. Baker, MD, 16. 1992–1993 National Profile of Local Health De-
importance of using a common framework for MPH, and Paul Halverson, DrPH, of the Public Health partments. Washington, DC: National Association of
research and performance management to Practice Program Office, Centers for Disease Control County and City Health Officials; 1995.
and Prevention, for their support of this effort. We
enhance the science base of modern public 17. Lewin Group. Strategies for Obtaining Public Health
would also like to thank Stephanie Bailey, MD,
health practice. Data at the Federal, State and Local Levels. Fairfax, Va:
MSHSA, Ron Bialek, MPP, Arthur Chen, MD, Kristine
US Dept of Health and Human Services; 1997.
Gebbie, RN, DrPH, Bernard Guyer, MD, MPH, C.
Arden Miller, MD, F. Douglas Scutchfield, MD, James 18. Harrell JA, Baker EL. The essential services of
FUTURE OF THE FRAMEWORK Studnicki, ScD, Jack Thompson, PhD, and William Wa- public health. Leadership Public Health. 1994;3(3):
ters Jr, PhD, for their input and insights. 27–30.
Researchers as well as practitioners inter- 19. Turnock BJ, Handler AS, Miller CA. Core func-
tion-related local public health performance. J Public
ested in the public health delivery system References Health Manage Pract. 1998;4(5):26–32.
have been slowly generating a body of work 1. Institute of Medicine. The Future of Public Health.
Washington, DC: National Academy Press; 1988. 20. Assessment Protocol for Excellence in Public Health.
that examines aspects of public health system Washington, DC: National Association of County
2. Scutchfield FD, Hiltabiddle SE, Rawding N, Vio-
performance. However, these efforts have Health Officials; 1991.
lante T. Compliance with the recommendations of the
been undertaken without a unified concep- Institute of Medicine report, The Future of Public Health: 21. Aday LA, Begley CB, Larson DR, Slater CH. Eval-
a survey of local health departments. J Public Health uating the Medical Care System: Effectiveness, Efficiency,
tual framework and without an agreed-upon and Equity. Ann Arbor, Mich: Health Administration
Policy. 1997;18:155–166.
set of measures. As we move further into the Press; 1993.
3. Handler AS, Turnock BJ, Hall W, et al. A strategy
21st century and endeavor to improve the for measuring local public health practice. Am J Prev 22. Institute of Medicine. Improving Health in the Com-
Med. 1995;11(suppl 6):29–35. munity: A Role for Performance Monitoring. Washington,
performance of the public health system and
DC: National Academy Press; 1997.
its ability to respond effectively, it is essential 4. Miller CA, Moore KS, Richards TB, Monk JD. A
proposed method for assessing the performance of 23. Barry MA, Centra L, Pratt E, Brown CK, Giordano
that public health researchers and practition- local public health functions and practices. Am J Public L. Where Do the Dollars Go? Measuring Local Public
ers undertake their efforts with an agreed- Health. 1994;84:1743–1749. Health Expenditures. Washington, DC: Public Health
Foundation; 1998.
upon framework that specifies the compo- 5. Miller CA, Moore KS, Richards TB, McKaig C. A
screening survey to assess local public health perform- 24. Dillenberg J, Atchison C, Baumgartner E, et al. Es-
nents of the public health system and how sential expenditures for essential public health serv-
ance. Public Health Rep. 1994;109:659–664.
these components interact. ices—selected states, fiscal year 1995. MMWR Morb
6. Richards TB, Rogers JJ, Christenson GM, Miller
The conceptual framework described here Mortal Wkly Rep. 1997;48:150–152.
CA, Gatewood DD, Taylor MS. Assessing public health
practice: application of ten core function measures of 25. Eilbert K, Barry M, Bialek R, Garufi M. Measuring
can guide the development of strategies and
community health in six states. Am J Prev Med. 1995; Expenditures for Essential Public Health Services. Wash-
research tools for monitoring public health ington, DC: Public Health Foundation; 1996.
11(suppl 6):36–40.
system performance and the generation and 7. Richards TB, Rogers JJ, Christenson GM, Miller 26. Eilbert KW, Barry M, Bialek R, et al. Public health
funding of research and other efforts de- CA, Taylor MS, Cooper AD. Evaluating local public expenditures: developing estimates for improved policy
health performance at a community level on a making. J Public Health Manage Pract. 1997;3(3):1–9.
signed to make system improvements. The
statewide basis. J Public Health Manage Pract. 1995; 27. Public Health Foundation. Measuring state expen-
model will allow public health researchers, 1(4):70–83. ditures for core public health functions. Am J Prev Med.
practitioners, and policymakers to more ef- 8. Turnock BJ, Handler AS, Hall W, Potsic S, Nalluri 1995;11(suppl 6):58–73.
fectively examine the relationship between R, Vaughn EH. Local health department effectiveness 28. Studnicki J, Steverson B, Blais HN, Goley E,
in addressing the core functions of public health. Public Richards TB, Thornton JN. Analyzing organizational
the practice of public health and population
Health Rep. 1994;109:653–658. practices in local health departments. Public Health Rep.
outcomes and will contribute to the develop- 1994;109:485–490.
9. Turnock BJ, Handler AS, Hall W, Lenihan DP,
ment of a science base for the public health Vaughn EH. Capacity-building influences on Illinois 29. Handler AS, Turnock BJ. Local health department
system. local health departments. J Public Health Manage Pract. effectiveness in addressing the core functions of public
1995;1(3):50–58. health: essential ingredients. J Public Health Policy.
10. Healthy People 2000: National Health Promotion 1996;17:460–483.

About the Authors and Disease Prevention Objectives. Washington, DC: US 30. Mays G. Organization of the public health deliv-
The authors are with the School of Public Health, Univer- Department of Health and Human Services; 1991. ery system. In: Novick L, Mays G, eds. Public Health
sity of Illinois, Chicago. DHHS publication PHS 91-50212. Administration: Principles for Population-Based Manage-
Requests for reprints should be sent to Arden Handler, 11. Mays GP, Halverson PK, Miller CA. Assessing the ment. Gaithersburg, Md: Aspen Publishers; 2001.
DrPH, University of Illinois School of Public Health, 1603 performance of local public health systems: a survey of 31. Schenck SE, Miller CA, Richards TB. Public health
W Taylor, Chicago, IL 60612 (e-mail: handler@uic.edu). state health agency efforts. J Public Health Manage performance related to selected health status and risk
This article was accepted August 23, 2000. Pract. 1998;4(4):63–78. measures. Am J Prev Med. 1995;11(suppl 6):55–57.

August 2001, Vol 91, No. 8 | American Journal of Public Health Handler et al. | Peer Reviewed | Research Articles | 1239

You might also like