RESEARCH
Challenges of Outcomes Research for Nurse Practitioners
Eileen Breslin, RNC, PhD
Margaret Burns, RN, PhD, CFNP
Patricia Moores, RN, PhD
Purpose
To describe the numerous methodological
challenges nurse practitioners (NPs) face in
designing and conducting outcomes research
and provide practical tips to implement an
outcomes study within an institution.
Data Sources
Review of world wide scientific literature on
outcomes research.
Conclusions
Nurse practitioners must be aware of the chal-
lenges of conducting outcomes research.
Challenges associated with variable definition,
designing outcomes studies, use of data sets,
and instrument development and selection
must be understood prior to undertaking an
outcome study
Implications for Practice
Valuable studies have laid a foundation for evi-
dence of quality care provision by NPs. Now is
the time to measure patient outcomes of the
NP care longitudinally over significant periods
of time.
Key Words
Outcomes research.
Authors
Eileen Breslin, RNC, PhD, is Dean and
Professor at the University of Massachusetts,
School of Nursing, Amherst, Mass. Margaret
Burns, RN, PhD, CFNP, is Executive Director
for Women's, Children's & Emergency
Services at Elliot Hospital, in Manchester,
N.H. Patricia Moores, RN, PhD, is an
Associate Professor of Adult Health at Beth-El
College of Nursing, Colorado Springs, Colo.
Contact Dr. Breslin by e-mail at breslin@nurs-
ing.umass.edu
138
Nurse practitioners (NPs) strive to make a difference in the health status
of the patients they serve. Traditionally, the effectiveness of NP interventions
was determined through judgment (did the patient get better) as opposed to
carefully planned measurement. Today, it is necessary to document through
scientific evidence whether or not NP care is effective, rigorous, efficient and
satisfactory to the patient (Jennings, 1998). This article examines the
methodological challenges that face NPs in designing outcomes research and
provides information to facilitate implementation of a research project with-
in a health care institution.
METHODOLOGICAL CHALLENGES
Specific judgments associated with methodological challenges in outcomes
research routinely occur in designing outcomes research. Nurse practitioners
face multiple challenges in implementing outcomes research within practice
settings. Jennings (1991) provides a useful framework to focus this discussion
in four specific areas: variable parameters, design issues, data sets, and instru-
ment development. Each of these methodological challenges will be discussed
in detail.
Variable Definition
When defining the variables for an outcome study, the following must be
considered: outcomes, interventions, and structure. Outcomes are, as
implied, the result or effect of care provided. Interventions include those
aspects of care for which outcomes are sought. Structure includes the context
in which the interventions and outcomes occur.
Issues raised specific to outcomes are threefold. The first challenge is that of
perception: outcomes according to whom? Patients? Providers? Families?
Organizations? Accrediting Agencies? All will have a different perspective. One
must be clear at the onset whose perspective the outcome measure will reflect.
Hegyvary (1991) proposes four categories useful for outcome assessment
measures that take into account the multiple perspectives of providers, con-
sumers, and purchasers. The first assessment is clinical, the patient's response
to medical and nursing interventions. The second assessment focuses on func-
tion, specifically, the maintenance and improvement of physical functioning.
Financial outcomes, the third assessment outcome, focuses on the most effi-
cient use of resources. The last assessment, the perceptual, examines the
patients’ satisfaction with their own outcomes, the care they received and the
providers of that care.
The second issue is outcomes as reflected by what? A vast number of indi-
cators can be used to reflect outcomes. Historically, the common indicators
were such parameters as mortality, morbidity, and incidence of complications.
Traditional outcomes were referred to as the five Ds: disability, disease,
VOLUME 14, ISSUE 3, MARCH 2002
death, discomfort, and dissatisfaction (Lohr, 1988). The newest
D to be added is dollars (Flanagan, 1995).
The nursing profession identifies more inclusive measures,
such as behavioral aspects, quality of life and functional status, as
more encompassing indicators that reflect outcomes. Mitchell,
Heinrich, Moritz, and Hinshaw (1997a) indicated that out-
comes of nursing care may be the following: achievement of
appropriate self care; demonstration of health-promoting behav-
iors; health-related quality of life; patient perception of being
well cared for; and symptom management to criterion. Clearly
these outcomes go beyond traditional measures of mortality and
morbidity. An outcomes assessment focuses on measuring these
factors, monitoring patients over time, and giving providers
feedback about results to help them optimize patient care.
Specific example of outcomes measures may be such factors as
blood pressure, pulse, number of emergency room visits, smok-
ing cessation rates, and functional scores of activities of daily liv-
ing (USDHHS, 1992).
The third issue related to outcomes is outcomes to what
extent? A simple assessment would be whether or not the out-
come was achieved. In reality, patients are complex. Rarely do
patients give simple yes or no answers to questions. Careful
attention to the sensitivity of measurement is essential. The sen-
sitivity of measurement instruments should reflect the smallest
amount of change in a variable that can be detected or measured
precisely (Strickland, 1997). Stewart and Archibold (1992,
1993) advocate that this criterion should predominate the selec-
tion of all outcomes measurements. Holzemer and Henry (1999)
advocate continuing identification and development of measure-
ment instruments for therapeutic outcomes responsive to nurs-
ing interventions.
Although discussed second, NPs must identify the interven-
tions for which outcomes will be sought before attempting to
identify measurable outcomes. With respect to interventions,
there is a pressing need to standardize operational definitions of
a nursing intervention. Debate exists as to what actually consti-
tutes a nursing intervention with Bulechek and McCloskey
(1992) espousing that a nursing intervention must be an
autonomous action and Carpentino (1993) challenging that not
all interventions are autonomous, in fact, some are delegated.
Regardless, the need to agree on standard definitions for inter-
ventions is undeniable. Strickland (1997) states there is a critical
need for the intervention to be scientifically based and carefully
operationalized. One must be clear about the purpose and goals
of the intervention. The underlying conceptual model provides
support for variable selection and expected outcomes. When an
NP uses a specific set of clinical recommendations to guide care,
this framework identifies the interventions.
Efforts to create a standardized language for nursing treat-
ments, otherwise known as interventions and outcomes, is being
done at the Center for Nursing Classification at the University
of Iowa. The intervention classification system identifies 486
interventions performed by nurses, organized into 30 classes and
domains. The nursing outcomes classification system describes
260 outcome variables that are responsive to nursing interven-
tions and may be used by nurses and other disciplines. The clas-
JOURNAL OF THE AMERICAN ACADEMY OF NURSE PRACTITIONERS
sification system includes corresponding definitions, measures,
indicators, and references (Iowa Outcomes Project, 2000).
Having identified potential outcomes to be measured and the
interventions on which they are based, the third variable para-
meter relates to the structure variable. This variable requires con-
sideration of where the care takes place and who are the
providers of care. Specifically, the type of setting in which care is
rendered may have an effect on care. This is an important con-
sideration for providers who care for patients in multiple settings
or who compare outcomes measured in different settings. Are
there similarities and differences in like care settings that impact
outcomes? Or, does care differ dependent upon different care
settings, such as nonprofit or for profit organizations? In addi-
tion, the dynamics occurring within the care setting can also
make a difference. For instance in a landmark study, Knaus,
Draper, Wagner and Zimmerman (1986) demonstrated that
outcomes for intensive care patients more strongly relate to staff
interaction than any other factors. The influence of structural
variables, such as an organizations' influence on patient out-
comes, is not well understood nor well defined (Mitchell,
Heinrich, Moritz, & Hinshaw, 1997b).
Another dimension of the structural variables relates to the
care provider. Recently, a randomized clinical trial examined the
variable of providers in ambulatory care setting comparing
physician and NP primary care and found patient outcomes
were comparable (Mundinger et al., 2000). There are many con-
siderations specific to providers. Competence of the care
provider may affect patient care outcomes. In designing studies,
how can one differentiate between the novice and expert
provider? Is differentiation of the educational preparation of the
provider necessary? Additionally, since NPs function within a
multi disciplinary health care team, how can one measure the
NP's specific contributions to patient outcomes?
Design Issues
The next critical issue is that of design. Study design refers to
the overall structure of the study. There are a number of method-
ological challenges to consider when designing the study, which
center on the issue of control, particularly with respect to quasi-
experimental and experimental designs. Most NPs will be
designing exploratory or quasi-experimental studies. The
amount of control the researcher has over variables being studied
varies from very little in exploratory studies to a great deal in
experimental designs. Two basic concepts, internal and external
validity, are central to understanding of control.
Internal validity may be defined as the extent to which results
of a study can actually be attributed to the action of the inde-
pendent variable, in this case the intervention or care, and not
something else. Quantitative study designs should consider how
much control one plans to have over the experimental variables.
Are all patients able to keep their appointments? Are patients
able to take their medications as prescribed? Internal validity
may be impacted by the reliability of the data collection instru-
ments. Reliability refers to the consistency, stability and repeata-
bility of the data collection instrument. Ideally, when designing
the study, a reliable instrument will be chosen that does not
139
respond to chance or environmental factors and produces
responses that can be replicated regardless of who administers
the instrument. Suggestions for locating reliable instruments are
discussed later.
External validity refers to the degree the findings are gen-
eralizable to the target population. Therefore, it is important
in designing the study to consider whether the sample is rep-
resentative of the population of interest (Brink, & Wood,
2000). When planning outcomes research, the population of
interest must be determined. For instance, if measuring out-
comes of care provided in a clinic for indigent patients diag-
nosed with congestive heart failure, the sample (and out-
come) would not be representative of all patients diagnosed
with a chronic illness.
Frequently, NPs will perform exploratory or qualitative out-
come studies, rather than research projects planned as strictly
controlled experimental designs. Similar concepts may be used
to evaluate exploratory, qualitative designs where rigor, not con-
trol, is the primary issue. In qualitative research, Lincoln and
Guba (1985) propose criteria that address the notion of rigor
within naturalistic settings useful for exploratory studies. These
criteria include credibility, similar to internal validity; transfer-
ability, similar to external validity; and dependability, similar to
reliability. Credibility refers to the truth-value test in that recon-
structions are credible to the constructors of this reality, the
study participants. Transferability questions if the findings fit
within other contexts. Dependability is similar to reliability. To
assess dependability, Guba (1981) proposed an inquiry audit
that examines both the process by which the study is conducted
and the product of the study so that a second researcher could
draw comparable conclusions.
Jennings (1991) advocates that the nursing profession use
designs from both qualitative and quantitative perspectives in
developing outcomes research studies. Qualitative designs may
provide a clear sense of the dynamics of patient outcomes and
quantitative designs assist with explicating both indirect and
direct variable relationships to outcomes. Qualitative methods
beg causality but allow the researcher to capture the reality of
outcomes from an inductive perspective. Quantitative designs
need to allow for clinical complexity. Triangulation of designs
may be a way of allowing for such complexity.
Triangulation is a research design that combines "different
theoretical perspectives, different data sources, different investi-
gators, or different methods within a single study" (Mitchell,
1986, pp 19). Triangulation is based on a navigational term in
which a point is determined by taking readings from two differ-
ent sites. Combining qualitative and quantitative methods con-
stitutes methodological triangulation.
Reasons one may chose to use triangulation are to compen-
sate for the weakness that might be found in one method, cor-
rect for bias, and provide balance. Other reasons include (a) pro-
viding a mechanism for combining qualitative and quantitative
data within one study, (b) increasing internal and external valid-
ity, (c) examining traits, not just the method, (d) using multiple
methods, and (e) increasing the richness or density of the con-
cept under study.
140
Data Sets
The third challenge Jennings (1991) raises encompasses the
use of data sets. Developing meaningful data sets from infor-
mation that can be retrieved is no small undertaking. Data sets
that record nursing interventions must be developed and
implemented. Achieving consensus about the development
and choice of these data elements is very time consuming. If
NP outcome research is to have broad generalizability, the
issue of standardized nomenclature is paramount as well as the
need for development of data systems across health care deliv-
ery systems.
Given the trend towards decreasing hospital admissions and
increasing the management of patient care in ambulatory set-
tings, NPs are now responsible for a diversity of roles, which
includes management of patents with acute needs and long-
term chronic illness, in addition to routine primary care.
Hastings and Muir-Nash (1989) developed a taxonomy that
may be useful to describe and classify professional nursing activ-
ities in ambulatory nursing practice. The principle that guided
the placement and sorting of activities was the focus or phase of
nursing care. The resulting taxonomy has nine responsibility
areas: (a) health assessment status, (b) planning, (c) patient
counseling and support, (d) patient education, (e) therapeutic
care, (f ) communication, (g) documentation, (h) normative
care, and (i) non patient-centered care. This study may provide
a beginning framework for NPs in ambulatory settings by pro-
viding a validated vocabulary with which to build a data set.
More work on this taxonomy is necessary given the complexity
of NP care.
Mark and Burleson (1995), in a random sample of 20 hospi-
tals, examined the availability and consistency of five patient
outcome indicators. The outcome indicators consisted of med-
ication administration errors, patient falls, occurrence of new
decubitus ulcers, nosocomial infections, and unplanned read-
mission to the hospital. The study results indicated that only two
outcome indicators, the medication errors and patient falls, were
consistently collected by the responding hospitals. Nurse practi-
tioners in acute care settings are challenged to develop indicators
sensitive to patient outcomes.
In evaluating data sets, one also needs to consider what con-
stitutes data and what influences data sets. Multiple factors can
influence the reliability and validity of data collected in patient
records. Aaronson and Burman (1994) examined the use of
health records in research. They outlined a number of factors
that affect the reliability and validity of a study. Factors include
the clinical competence and expertise of the clinician, patient
cooperation and competence, the type of provider, where the
care takes place, various situational factors, and the type of data.
They also address the fact that some providers will change their
questions based on their perceptions of the patient and that, at
times, sensitive issues are infrequently documented in the health
record.
Separate issues also arise when one extracts data from the
health record. Extracted data is influenced by coder training, the
amount of interpretation the coder must make, and the level of
coding requirement.
VOLUME 14, ISSUE 3, MARCH 2002
Instrument development and selection
Instrument development, the last methodological challenge,
is fundamental to measurement of all variables. Measurement is
the process of translating reality into numbers (Knapp, 1985).
Measurement error must be reduced to the greatest possible
extent. Instrument development requires a complex process of
tasks to ensure validity and reliability. For novice researchers, a
number of resources exist that may be used in the selection of
instruments that have already undergone this process. The
Health and Psychosocial Instruments (HAPI) CD-ROM
Behavioral Measurement Database Services is available. This
valuable resource provides concise information about an instru-
ment of interest. The full record will report the title, author, and
source of the instrument plus the number of questions contained
in the instrument, a brief abstract discussing the instrument and
whether the reliability and validity of the instrument is reported.
Additional resources are Standards for Educational and
Psychological Testing AERA, APA, & NCME Joint Committee
(1985), which establishes a set of standards and guidelines for
measurement of human behavior; Instruments for Nursing
Research (Frank-Stromberg, & Olsen, 1998), and Measuring
Health: A guide to rating scales and questionnaires (McDowell,
& Newell, 1987) which also provide tests, scales and instruments
available for variable measurement. For NPs providing primary
care, Tools for Primary Care Research (Stewart, 1992) may also
provide information of value.
If using instruments from organizations and institutes, fees
for use and analyses of data may be charged. If an instrument is
obtained through the public domain, one should consult with
the author prior to use.
In selecting instruments for inclusion in a study, Harris and
Warren (1995) suggest six criteria for assessing tools. The six fac-
tors are applicability, practicality, comprehensiveness, reliability,
validity, and responsiveness. Applicability, the first factor, con-
siders the purpose of the instrument. The instrument's purpose
may be to (a) discriminate between subjects at a point in time,
(b) evaluate changes within subjects over time, (c) predict future
outcomes, (d) screen for problems, or (e) assess quality of care.
The next factor is practicality. The instrument should include
outcomes important to the patient. The tool should be short and
easy to administer. The questions should be easy to understand
and culturally relevant. Scores should reflect the condition sever-
ity and be discriminative. The information obtained should be
clinically useful.
Comprehensiveness focuses on the completeness of assess-
ment. Lack of conceptual clarity in the operationalization of
variables has been a criticism of nursing studies in the past. The
notion of comprehensiveness is particularly important if one
wants to make comparisons across studies.
Reliability refers to how closely the data obtained from the
instrument relate to the individual's true score. Consultation
with experienced researchers would be appropriate in the selec-
tion of reliable instruments.
Validity refers to whether an instrument measures what it says
it measures. Berk (1990) discusses the current trend to view the
three traditional components of validity (content, criterion-relat-
JOURNAL OF THE AMERICAN ACADEMY OF NURSE PRACTITIONERS
ed, and construct) in a more unified conceptualization which
emphasizes the data gathering process and methods to obtain the
kind of evidence essential to answer validity questions. Those
who develop the instruments concentrate their efforts on the
inferences to be made with the scores and the evidence necessary
to support the inferences.
The last factor, responsiveness, is the extent to which an
instrument is able to detect a minimally important difference
within a subject over time. Stewart and Archibold (1992, 1993)
have referred to this as "sensitivity to change" to describe this
instrument characteristic and present a case that this is the most
important criterion to consider in selecting an instrument to
evaluate the effects of an intervention.
A final note should be made addressing the need for ensuring
that if an instrument is chosen, that it be truly culturally appro-
priate for the population to be studied. Given that NPs often
work with the underserved and culturally diverse populations,
pilot testing of all instruments for cultural appropriateness is
warranted.
NURSE PRACTITIONER LITERATURE
Within the past 15 years there has been increased attention
on studies exploring the effectiveness of outcomes of care by
NPs. Earlier studies focused on the provider of care and its effect
and the quality of care (U.S. Congress, Office of Technology
Assessment, 1986; Crosby, Ventura, & Feldman, 1987; Brown,
& Grimes, 1995; Mundinger et al., 2000). These initial valuable
studies laid the foundation for the evidence of quality care pro-
vision by NPs. Now is the time to measure patient outcomes of
the NP care longitudinally over significant periods of time.
Attention is being drawn in the literature for advance practice
nurses to become more involved in outcomes assessment and
research (Kleinpell-Nowell, & Weine, 1999). In one review arti-
cle, Stone (1994) assesses quality of care studies related to NPs
and succinctly summarizes some of the methodological chal-
lenges found in those studies. Stone divides the quality of care
issues into two spheres. The first set of issues involves technical
care, which focuses on the diagnostic and the therapeutic com-
ponents of care. The second sphere includes the art of care,
which focuses on the interpersonal manner and behavior in car-
ing for and communicating with the patient. Regarding limita-
tions of previous outcome studies, she cites small sample sizes
without power analysis, focus on short-term outcomes, the use
of nonrandomized study populations, the application of single
evaluation criteria, and the use of incomplete and nonstandard-
ized medical records. Stone identified the following biases evi-
dent in conducting the studies: NPs and educators doing the
research on NPs; patients of NPs not being as sick as patients
cared for by other providers; comparison of NPs with new house
staff; and a focus only on the technical medical tasks. She calls
for future researchers to be aware of the methodological flaws
evident in prior studies.
Future research should be designed to include summary sta-
tistics suitable for future analysis and be grounded within a the-
141
oretical perspective. Additionally, outcomes research opportuni-
ties have yet to be fully explored at the community level, focus-
ing on health of communities. Nurse practitioners can be invalu-
able in assisting community health information systems cultur-
ally attuned to communities they serve.
IMPLEMENTATION OF A RESEARCH PROJECT WITHIN
INSTITUTION
Many NPs provide care within larger health care institutions
or systems. Practical hints adapted from the Joint Commission
(1994) will assist in the implementation of outcomes research
within a health care institution. Seven critical elements need to
be taken into consideration: organization, management of study,
financing, education, ethical and legal concerns, interinstitu-
tional issues and demonstrating value. Even for NPs who work
in smaller practice settings, this guidance identifies many issues
to be considered in planning outcomes research.
The organization issues that must be considered are support
from administration, piggybacking on existing capabilities and
related activities, and the leadership and vision to embark on
such a project. Nurse practitioners are uniquely qualified within
an organization to recognize, act, and evaluate the effectiveness
of patient care.
Issues relevant to the management of the study center on the
effective implementation of the study. Starting with a small
clearly defined study is helpful. Instruments should be reliable,
valid, and easy to administer; existing measurement tools should
be used whenever possible. Study protocols should create a min-
imum of disruption within the clinical setting and specify when
and how the data should be collected. Data should be linked
whenever possible to existing management systems, such as
billing systems, utilization of services, and patient profile sys-
tems.
Financing considerations include the initial start up of the
project, technology required for the study (such as a computer),
and the ongoing daily operation. The bottom line is to budget
for all of the anticipated expenses. Hidden costs may be related
factors such as permission to use measurement tools, unantici-
pated staff time, and computer time.
The importance of education, training, and awareness of the
study cannot be overemphasized. Everyone who might be affect-
ed by or involved in the study must be identified and apprised of
the study. Specific staff skills may be developed in the process of
conducting the study.
Ethical and legal concerns need to be considered. There
should be investigation as to how the conduct of outcomes stud-
ies will be handled within the organization, and who approves of
the conduct of the study. Each institution is likely to require
slightly different application formats; these should be identified
and followed, and approval received, prior to starting any phase
of the project. Researchers should obtain informed consent prior
to enrolling participants in any study, using the approved for-
mat.
142
Interinstitutional issues are raised when the resources within
institutions are insufficient to conduct the study. Building link-
ages with other researchers in university settings might facilitate
maximum use of resources. The pooling of data from multiple
sites offers an opportunity to compare results across sites.
Resources such as data management systems should be evaluat-
ed carefully.
The last issue, demonstrating value, is probably the most
important. As previously discussed, conducting outcomes stud-
ies are of value to patients, policy makers, purchasers, adminis-
trators, and NPs. Obstacles, such as fears relating to how the data
may be used, cost of development, and overall indifference, may
be difficult to overcome. Nurse practitioners must exert the lead-
ership and vision to ensure that outcomes studies are conducted
that advance the science and art of nursing. This fulfills the con-
tract nursing has with society as outlined in the American
Nurses’ Association (1996) Social Policy Statement.
Quality improvement activities, research utilization, and
research conduct are interrelated and independent processes that
influence optimal patient outcomes. Patient outcomes research
represents the opportunity to demonstrate how specific practices
contribute to patient outcomes and, in turn, will add to the sci-
entific base for practice.
CONCLUSION
Nurse practitioners who design and implement outcomes
research experience a myriad of methodological challenges dur-
ing this process. This article presents an overview of some of
these challenges and suggests methods to avoid some of the pit-
falls. The review of NP literature regarding evaluative studies of
outcomes research confirms the need for more studies in this
area. The practical hints for conducting outcomes research pro-
vide a guide for the novice researcher in implementing a study.
REFERENCES
Aaronson, L., & Burman, M. (1994). Focus on psychometrics use of health
records in research: Reliability and validity issues. Research in Nursing &
Health, 17, 67-73.
AERA, APA & NCME Joint Committee. (1985). Standards for educational and
psychological testing. Washington, DC: American Psychological Association.
American Nurses' Association. (1996). Nursing: A social policy statement.
Kansas City, MO: American Nurses' Association.
Berk, R.A. (1990). Importance of expert judgment in content-related validity evi-
dence. Western Journal of Nursing Research, 12(5), 659-671.
Brink, P. & Wood, M. (2001). Basic steps in planning nursing research: From
question to proposal (5th ed.). Boston, MA: Jones and Bartlett.
Brown, S.A., & Grimes, D.E. (1995). A meta-analysis of nurse practitioners and
nursed midwives in primary care. Nursing Research, 44(6), 332-339.
Bulechek, G.M., & McCloskey, J.C. (1992). Nursing interventions: Essential
nursing treatments. Philadelphia: W. B. Saunders Company.
Carpentino, L.J. (1993). Nursing diagnosis: Application to clinical practice (5th
ed.). Philadelphia: J. B. Lippincott Co.
Crosby, F., Ventura, M.R., & Feldman, M.J. (1987). Future research recommen-
VOLUME 14, ISSUE 3, MARCH 2002
 dations for establishing NP effectiveness. Nurse Practitioner, 12(1), 75-76.
Flanagan, A. (1995). Ambulatory outcomes management. American Academy of
Ambulatory Care Nursing Viewpoint, 17(2), 1, 6.
Frank-Stromborg, M., & Olsen, S. (1998). Instruments for clinical health care
research. Boston, MA: Jones & Bartlett.
Guba, E. (1981). Criteria for assessing the trustworthiness of naturalistic
inquiries. Educational Communication and Technology Journal, 29, 75-92.
Harris, M.R., & Warren, J.J. (1995). Patient outcomes: Assessment issues for the
CNS. Clinical Nurse Specialist, 9(2), 82-86.
Hastings, C., & Muir-Nash, J. (1989). Validation of a taxonomy of ambulatory
nursing practice. Nursing Economics, 7, 142-149.
Hegyvary, S.T. (1991). Issues in outcomes research. Journal of Nursing Quality
Assurance, 5(2), 1-6.
Holzemer, W.L., Henry, S.B. (1999). Therapeutic outcomes sensitive to nursing.
Handbook of Clinical Nursing Research. Hinshaw AS et al. pp 185-98.
Iowa Outcomes Project. (2000). In M. Johnson, M. Maas, & S. Moorhead (Eds).
Nursing Outcomes Classification(NOC; 2nd ed). St. Louis: Mosby.
Jennings, B.M. (1998). A Mandate to Increase Rigor in Outcomes Management
Studies. Research in Nursing & Health, 21, 473-474.
Jennings, B.M. (1991). Patient outcomes research: Seizing the opportunity.
Advances in Nursing Science, 14(2), 59-72.
Joint Commission on Accreditation of Healthcare Organizations. (1994). A guide
to establishing programs for assessing outcomes in clinical settings.
Oakbrook Terrace, IL: Author.
Kleinpell-Nowell, R., & Weiner, T. (1999). Measuring Advanced Practice Nursing
Outcomes. AACN Clinical Issues, 10(3), 356-368.
Knapp, T.R. (1985). Validity, reliability, and neither. Nursing Research, 34(3),
189-191.
Knaus, W., Draper, E., Wagner, D., & Zimmerman, S. (1986). An evaluation of
outcome from intensive care in major medical centers. Annals of Internal
Medicine, 104(3), 410-418.
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills: Sage
Publications.
Lohr, L. (1988, March/April). Outcome measurement: Concepts and questions.
Inquiry, 17(2), 37-50.
Mark, B.A., & Burleson, D. L. (1995). Measurement of patient outcomes, data
JOURNAL OF THE AMERICAN ACADEMY OF NURSE PRACTITIONERS
availability and consistency across hospitals. JONA, 25(4), 52-59.
McDowell, I., & Newell, C. (1987). Measuring health: A guide to rating scales
and questionnaires. New York: Oxford University Press.
Mitchell, E. (1986). Multiple triangulation: A methodology in nursing science.
Advances in Nursing Science, 8(3), 18-26.
Mitchell, P., & Shortell, S. (1997. Adverse Outcomes and Variations in
Organization of Care Delivery. Medical Care, 35(11), 19-32.
Mitchell, P., Heinrich, J., Moritz, P., & Hinshaw, A. (1997a). Measurement into
Practice: Summary and Recommendations. Medical Care, 35(11), 124-127.
Mitchell, P., Heinrich, J., Moritz, P., & Hinshaw, A. (1997b). Outcome Measures
and Care Delivery Systems: Introduction and Purposes of Conference.
Medical Care, 35(11), 1-5.
Mundinger, M., Kane, R., Lenz, E., Totten, A., Tsai, W., Cleary, P., Friedwald, W.,
Siu, A., & Shelanski, M. (2000). Primary Care Outcomes in Patients Treated
by Nurse Practitioners or Physicians. Journal of the American Medical
Association, 283(1), 59-68.
Stewart, M. (Ed.). (1992). Tools for primary care research. Thousand Oaks, CA:
Sage Publications, Inc.
Stewart, B., & Archibold, P. (1992). Focus on psychometrics nursing intervention
studies require outcome measures that are sensitive to change: Part one.
Research in Nursing & Health, 15, 447-481.
Stewart, B., & Archbold, P. (1993). Focus on psychometrics nursing intervention
studies require outcome measures that are sensitive to change: Part two.
Research in Nursing & Health, 16, 77-81.
Stone, P. (1994). Nurse practitioners' research review: Quality of care. Nurse
Practitioner, June, 17, 21,27.
Strickland, O. (1997). Challenges in Measuring Patient Outcomes Nursing
Clinics of North America, 32(3), 495-511.
United States Congress, Office of Technology Assessment. (1986). Nurse
Practitioners, Physicians Assistants, and Certified Nurse-Midwives: A Policy
Analysis. Washington, DC: U.S. Government Printing Office, Case Study 37.
U.S. Department of Health and Human Services (1992). Patient outcomes
research: Examining the effectiveness of nursing practice, proceedings of
the state of the science conference (NIH Publication No. 93-3411).
Bethesda, MD: National Institute for Nursing Research.
143