You are on page 1of 10

Available online at www.sciencedirect.


Nurs Outlook xxx (2017) 1e10

Twenty years of staffing, practice environment, and

outcomes research in military nursing
Patricia A. Patrician, PhD, RN, FAANa,*, Lori A. Loan, PhD, RN, FAANa,
Mary S. McCarthy, PhD, RN, FAANb, Pauline Swiger, RN, MSN, CNLc,
Sara Breckenridge-Sproat, PhD, RNd, Laura Ruse Brosch, PhD, RNe,
Bonnie Mowinski Jennings, PhD, RN, FAANf
Department of Family, Community Health & Systems, University of Alabama at Birmingham, School of Nursing, Birmingham, AL
Madigan Army Medical Center, Tacoma, WA
University of Alabama at Birmingham, School of Nursing, Birmingham, AL
Regional Health Command Europe, Sembach, Germany
Office of Research Protections (ORP) Headquarters, US Army Medical Research and Materiel Command, Fort Detrick, MD
Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, GA

article info abstract

Article history: Background: Two decades ago, findings from an Institute of Medicine (IOM) report
Received 29 March 2017 sparked the urgent need for evidence supporting relationships between nurse
Revised 28 June 2017 staffing and patient outcomes.
Accepted 29 June 2017 Purpose: This article provides an overview of nurse staffing, practice environment,
and patient outcomes research, with an emphasis on findings from military
studies. Lessons learned also are enumerated.
Keywords: Method: This study is a review of the entire Military Nursing Outcomes Database
Nursing sensitive indicators (MilNOD) program of research.
Outcomes Discussion: The MilNOD, in combination with evidence from other health care
Practice environment studies, provides nurses and leaders with information about the associations
Staffing between staffing, patient outcomes, and the professional practice environment
of nursing in the military. Leaders, therefore, have useful empirical evidence to
make data-driven decisions. The MilNOD studies are the basis for the current
Army nursing dashboard, and care delivery framework, called the Patent Car-
ingTouch System.
Conclusion: Future research is needed to identify ideal staffing based on workload
demands, and provide leaders with factors to consider when operationalizing
staffing recommendations.
Cite this article: Patrician, P. A., Loan, L. A., McCarthy, M. S., Swiger, P., Breckenridge-Sproat, S., Brosch, L.
R., & Jennings, B. M. (2017, -). Twenty years of staffing, practice environment, and outcomes research in
military nursing. Nursing Outlook, -(-), 1-10.

* Corresponding author: Patricia A. Patrician, Department of Family, Community Health, & Systems, University of Alabama at Bir-
mingham School of Nursing, NB 324, 1720 Second Avenue South, Birmingham, AL 35294-1210.
E-mail address: (P.A. Patrician).
0029-6554/$ - see front matter 2017 Elsevier Inc. All rights reserved.
2 Nurs Outlook xxx (2017) 1e10

Introduction journey to explore the associations among nurse

staffing, nursing practice environment, and patient
safety in military hospitals. This article provides an
In the two decades since the release of the 1996 Insti- overview of staffing, nursing practice environment,
tute of Medicine (IOM) report that underscored the lack and patient outcomes research in the military over the
of empirical evidence about the contributions of nurse past two decades, introduces the lessons learned from
staffing to the quality of patient care (Wunderlich, this body of research, and identifies future directions
Sloan, & Davis, 1996), researchers have attempted to for this line of inquiry related to health care in the
document empirically the contribution of nursing to military.
patient care outcomes in a variety of settings. A
compelling body of evidence suggests that staffing is
associated with patient outcomes in clinically mean- The Case for Military Nursing-Specific
ingful ways (Kane, Shamliyan, Mueller, Duval, & Wilt, Research
2007; Lang, Hodge, Olson, Romano, & Kravitz, 2004;
Page, 2004). Higher nurse to patient ratios, a higher
proportion of registered nurses (RNs), and higher total Distinctive Features of Nursing Staff in Military
nursing care hours have been linked to lower patient Hospitals
mortality (Aiken, Clarke, Sloane, Sochalski, & Silber,
2002; Kane et al., 2007; Lang et al., 2004; Needleman, Although many of the findings about nurse staffing and
Buerhaus, Meattke, Stewart, & Zelevinsky, 2002), the nursing practice environment discovered from
decreased length of stay, and a lower likelihood of studies conducted in civilian hospitals are applicable to
patient complications, such as nosocomial infections military nursing, there are important differences be-
and pressure injury (Kovner & Gergen, 1998; Lang et al., tween civilian and military health care. Similar to the
2004; Needleman et al., 2002). civilian sector, nursing personnel working in military
The research on adverse events such as medication hospitals may be RNs, licensed practical or vocational
errors and patient falls as they relate to nurse staffing nurses (LPNs and LVNs), or nursing assistants (NAs).
is less conclusive. Lang et al. (2004) conducted a sys- Less than half of the RNs and LPN/LVNs are active duty
tematic review of the nurse staffingepatient outcomes military officers or enlisted personnel. Department of
research from 1980 to 2003 and did not find evidence Defense (DoD)/federal civilians account for most
attributing nurse staffing to a number of patient safety workforce, with additional nurses provided by contract
outcomes, including falls, medication errors, and pa- and military reserve nursing personnel. Compared
tient injuries. Other investigators reported insignifi- with civilian hospital nurse staffing models, military
cant associations between staffing patterns and falls staffing models typically include a lower proportion of
(Bolton et al., 2007; Lake & Cheung, 2006), medication RNs and a higher proportion of LPNs/LVNs, and more
errors (Cho, Ketefian, Barkauskas, & Smith, 2003; Lang assistive personnel, such as medics, corpsmen, and
et al., 2004; Mark & Belyea, 2009), or both (Mark, NAs. Findings have suggested that the military rank
Hughes, Belyea, Bacon, Chang, & Jones, 2008). structure may facilitate more positive working re-
Conversely, other researchers found higher risk of falls lationships between nurses and physicians in military
with fewer nursing care hours per day and a lower health care settings compared with civilian settings
proportion of RNs (Dunton, Gajewski, Taunton, & (Foley, Kee, Minick, Harvey, & Jennings, 2002; Patrician
Moore, 2004) and increased medication errors with et al., 2010b).
lower staffing levels (Blegen & Vaughan, 1998). Compared with their federal civilian counterparts
The context of hospital care, or the practice envi- and nurses working in civilian hospitals, military RNs
ronment, also has been found to influence outcomes. are younger, the proportion of men is greater, and
The nursing practice environment is defined as the there is more ethnic diversity (Patrician et al., 2010b,
conditions under which nurses work and includes 2011a, 2011b). More military RNs have bachelors,
consideration of leadership, resource availability, and masters, and doctoral degrees as well as advanced and
collaboration with physicians (Lake, 2002). There is specialty training (e.g., critical care nursing, operating
ample empirical evidence that favorable professional room nursing) than civilian nurses (Spratley, Johnson,
nursing practice environments are associated with low Sochalski, Fritz, & Spencer, 2000). Although federal
levels of nurse burnout, higher job satisfaction, less civilian nurses may choose to move into identified
nurse turnover, and more positive patient outcomes to leadership positions, leadership education and skills
include lower mortality and higher satisfaction (Aiken are essential for military nurses because they are ex-
& Sloane, 1997; Aiken, Clarke, Sloane, Lake, & Cheney, pected to assume leadership roles and advance in rank.
2008; Aiken, Sloane, Lake, Sochalski, & Weber, 1999; Military courses and competitive selection to attend
Kazanjian, Green, Wong, & Reid, 2005; Patrician, civilian universities for advanced degrees are typical
Shang, & Lake, 2010b). avenues for higher education.
In parallel with this body of evidence from the Although baccalaureate education is associated
civilian sector, military nurse researchers began a with better patient outcomes (Aiken, Clarke, Cheung,
Nurs Outlook xxx (2017) 1e10 3

Sloane, & Silber, 2003), so is nursing experience environments on nurses job satisfaction, emotional
(Blegen, Vaughn, & Goode, 2001). In the military, as in exhaustion, intent to leave the military, and ratings of
civilian settings, new graduates with less nursing care quality was conducted by Patrician et al. (2010b).
experience are more likely to provide care at the These investigators collected data from all 23 army
bedside. hospitals. On average, nurses working in these military
Another unique characteristic of military nurses is settings rated their practice environment as favorable,
that each nurse assumes a dual role: one as a nurse and reflecting average scores that represented ratings be-
another as a military service member. Military training tween those reported for Magnet and nonmagnet
requirements may remove nursing personnel from hospitals on the Practice Environment Scale. Patrician
their assigned patient care unit for an hour here and et al. (2010a, 2010b) also found that more poorly rated
there or for many months at a time, creating a more military nursing practice environments were associ-
dynamic work environment as staff is augmented or ated with nursing job dissatisfaction, intent to leave,
changed (Breckenridge-Sproat, Johantgen & Patrician, emotional exhaustion, and poor quality of care ratings.
2011). Military deployments also displace nursing
personnel from their assigned role. Moreover, manda-
tory relocation of military personnel approximately Beginnings of a Military Program of Research
every 3 years has significant implications on the defi- on Staffing, Practice Environment, and
nition and measurement of staff turnover. Outcomes

Distinctive Features of the Military Health System

Linking Nursing Care Quality to Patient Outcomes
Although civilian hospitals report data to external
stakeholders such as the Centers for Medicare and Studies performed by military nurse researchers
Medicare Services, American Hospital Association, in- following the 1996 IOM report (Wunderlich et al., 1996)
surance providers, and other entities, military hospi- demonstrate an ongoing commitment to investigate
tals generally have not participated in large public associations between nursing care quality and patient
database collection initiatives. The lack of standard- outcomes. These early studies parallel efforts by the
ized and accessible military health data is a challenge Agency for Healthcare Research and Quality (then the
for investigators who are trying to study nurse staffing Agency for Health Care Policy and Research) promoting
and patient outcomes. Primary data collection for patient outcomes research (Mitchell & Durenberger,
nurse staffing and patient outcome studies often is cost 1990) and the American Nurses Associations (ANA)
prohibitive. The need for a database containing mili- work to create a nursing care report card for use in
tary data related to nurse staffing and patient out- acute care settings (ANA, 1995), which ultimately
comes was a catalyst to creating the Military Nursing evolved into the National Database of Nursing Quality
Outcomes Database (MilNOD). Indicators (Montalvo, 2007).
Military nurse researchers interested in patient
outcomes saw the opportunity to align military
Early Studies Reflecting the Military Nursing research with the ANA report card efforts, motivating
Practice Environment them to seek funding from the TriService Nursing
Research Program (TSNRP) to assess the feasibility of
collecting various data elements from existing military
Before research focused on nurse staffing and patient data sources related to patient outcomes. The first was
outcomes, military nurse scientists began to explore a pilot study designed to determine the feasibility of
the culture and dynamics of the nursing practice collecting the ANA nursing quality indicators at one
environment within the military health care system. army medical center (Jennings, Loan, DePaul, Brosch, &
These studies are sparse, and the findings are incon- Hildreth, 2001; Loan, Jennings, Brosch, DePaul, &
sistent. For instance, in the latter part of the 1990s, Hildreth, 2003). From a sample of 5,082 patient re-
investigators conducted a study at a single military cords on five inpatient units, nursing care quality
medical center; they found poor nurseephysician scores were calculated by evaluating compliance with
communication and low autonomy in military nursing patient education, skin care, patient safety, and central
practice environments (Anderson, Maloney, Oliver, line management standards of care. Nurse satisfaction,
Brown, & Hardy, 1996; Maloney, Anderson, Gladd, based on survey responses from 73 nursing staff
Brown, & Hardy, 1996). By contrast, Foley et al. (2002), members, also was assessed. Along with demon-
who studied two different military hospitals, found strating that ANA nursing quality indicators could be
nursing practice environments characterized by high collected using existing data, analyses found that
autonomy, control over practice, clinical expertise staffing mix (RNs, LPNs, and unlicensed staff) and total
among the nursing staff, and collegiality with nursing care hours per patient day were positively
physicians. correlated at a significant level with various measures,
The first systemwide investigation within army including quality scores (Jennings et al., 2001; Loan
hospitals examining the effect of nursing practice et al., 2003).
4 Nurs Outlook xxx (2017) 1e10

A New Approach to Collecting and Analyzing Nurse expand the army database and ANOD processes by
Staffing and Patient Safety Data implementing the new MilNOD at seven inpatient
hospital sites across the army, navy, and air force. The
The second foundational study funded by TSNRP was second aim was to sustain a high-quality database to
known as the Army Nursing Outcomes Database support evidence-based clinical and administrative
(ANOD) demonstration project, undertaken in collabo- decision making as well as to conduct research related
ration with experts from the California Nursing Out- to nurse staffing effectiveness and patient safety. The
comes Coalition (CalNOC), now known as the MilNOD project included database development, data
Cooperative Alliance for Nursing Outcomes (CALNOC) specifications, and a series of studies derived from
(Aydin et al., 2004; Brosch, 2005). The ANOD project had MilNOD data.
many features that were instrumental to future military Needleman et al. (2002) noted we need to better
nurse staffing studies. First, ANOD was designed to use understand the factors influencing both staffing
data collection, analysis, and reporting techniques that levels and mix of personnel in hospitals (p. 1045).
would yield a more granular assessment of the rela- The MilNOD afforded a way to derive a better un-
tionship between staffing and patient outcomes. Sec- derstanding of nurse staffing in military hospitals. It
ond, ANOD investigators wished to determine if data is the bedrock of the database that continues to
collection could be standardized on 24 clinical nursing enable military nurse leaders and military nurse re-
units from two large army medical centers. Believing searchers to trace and analyze daily variations in
that assessment of staffing at the shift and unit levels staffing and examine the effect of staffing on patient
and the real-time occurrences of safety events might safety and outcomes. Unique to the MilNOD approach
better serve to detect meaningful associations, the use was the decision to use the unit level for data
of aggregated hospital-level data was replaced with collection and analysis. Civilian outcomes re-
unit-level data collection for every 8-hr shift. A third searchers advocated a movement away from focusing
product from the ANOD study was developing pro- only on hospital-level data and toward focusing on
cesses for producing staffing and outcome reports at the unit-level data (Mitchell & Shortell, 1997). Unit-level
unit, section, and hospital levels and delivering these to data better support operational groups, such as de-
hospital nurse leaders who also were trained about how partments and patient care units, that assume ulti-
to use them for decision making. Collecting and storing mate responsibility for outcomes (Whitman, Kim,
data to allow for future secondary analysis of staffing, Davidson, Wolf, & Wang, 2002). MilNOD in-
patient and nurse outcomes, and the practice environ- vestigators believed the opportunity to advance
ment became a reality. measurement precision resided in the ability to cap-
A fourth consideration by ANOD investigators was ture valid and reliable nurse staffing and measures of
to examine the feasibility and usefulness of collecting clinical workload, along with daily patient care out-
nurse staffing and patient and nurse outcome indicator comes at the patients bedside.
data using the CalNOC methodologies (Aydin et al., Good data are essential to guide clinical and
2004; Donaldson, Brown, Aydin, & Bolton, 2001). As administrative decision making and support rigorous
part of the ANOD project, a first of its kind, shift-level research. Thus, the initial MilNOD work focused on
relational database was constructed. This distinction developing processes to ensure sustainment of high-
is noteworthy; the aim was to construct and test a quality data collection. Early efforts focused on
system to collect data more closely synchronized to the rigorous assessment of the completeness, accuracy,
timing of adverse events, such as falls. and quality of data collected for each indicator in the
Results from the ANOD project demonstrated, and database. Along with the indicators that were previ-
army nursing leaders concurred, that it was feasible to ously collected in the ANOD, for example, falls and
implement a standardized, shift-level nurse staffing and pressure ulcer prevalence, the investigators expanded
patient outcome measurement methodology. Moreover, the MilNOD to include two additional indicatorsd
the reports generated with the ANOD data enabled nurse nurse-committed medication errors and nursing staff
leaders to compare nursing care quality to internal and needlestick injuriesdand two additional potential
external benchmarks, on a unit-by-unit, section-by- explanatory variablesdpatient turnover and patient
section, or hospital-by-hospital basis (Brosch, 2005). acuity. The investigators developed data definitions
and collection methodologies for these indicators and
The MilNOD Studies Along with developing the database, the early
MilNOD work also provided direct benefits to nurse
leaders at participating hospitals. For example, chief
Proof of Concept for Data Quality, Usefulness, and nurse executives received quarterly MilNOD
Generalizability reports reflecting nurse staffing effectiveness and
patient safety. These data and reports satisfied
The MilNOD project began in 2002 (Loan, Brosch, The Joint Commission staffing effectiveness
McCarthy, & Patrician, 2005). The first aim was to requirements.
Nurs Outlook xxx (2017) 1e10 5

Relationships Between Staffing, Outcomes, and the needlestick injuries. Additional details of the shift-
Practice Environment level staffing and adverse events are reported else-
where (Patrician et al., 2011a; Patrician, Pryor, Fridman,
During the next 4 years, 2003 to 2006, six additional & Loan, 2011b).
military hospitals were added to the study for a total
of 13, and analyses were conducted to explore the Practice Environment Findings
associations between nurse staffing as well as patient There was a strong association between the PES-NWI
and nurse outcomes at the shift level. Nurse staffing, and nurse job satisfaction. MilNOD investigators
patient census, patient turnover, and patient acuity found differences between the PES-NWI responses of
indicators were collected from each unit and entered RNs, LPNs, and NAs, with NAs tending to score much
into a spreadsheet that was submitted monthly to the higher than RNs and LPNs. For this reason, an RN and
central MilNOD database. Patient falls, nurse- LPN-only measure of PES-NWI was recalculated and
committed medication errors, and needlestick in- aggregated to the unit level and used for all analyses. In
juries were obtained from hospital and occupational this sample of military hospitals, job satisfaction was
health reports. Pressure injury prevalence and re- strongly associated with total nursing care hours per
straint use data were collected by conducting bian- patient shift, even when controlling for the effects of
nual prevalence surveys at each of the participating the PES-NWI scores.
hospitals. Nursing personnel completed annual sur-
veys about nurse education, experience, and job Nursing Personnel Job Satisfaction
satisfaction. The annual survey also included Nurse job satisfaction was dependent more on ones
administration of the Practice Environment Scale of position in the unit rather than on the staffing or pa-
the Nursing Work Index (PES-NWI; Lake, 2002). tient turnover rates. Military LPNs and military unli-
Annual patient surveys included satisfaction with censed personnel were the least satisfied with their
pain management, patient education, and care as jobs, and DoD civilians were most satisfied. These
measured by The Patient Satisfaction with Nursing findings may be explained by the additional duties the
Care Questionnaire (Jacox, Bausell, & Mahrenholz, military are required to perform, such as working extra
1997). A summary of findings from the 13 military hours when civilian nurses call in sick. Theoretically,
hospitals that participated in the MilNOD during the 4 the military are supposed to be available 24 hr a day,
years follows. 7 days a week.

Safety Improvements Over Time The Practice Environment and Workload Intensity:
On average, statistically significant improvements Testing Mediation and Moderation Effects
were demonstrated in fall rates, nurse-committed
medication error rates, and hospital-acquired pres- The most recent study was a secondary analysis of
sure injury (HAPI; National Pressure Ulcer Advisory MilNOD data using 111,500 shifts from 57 units in 13
Panel, 2016). Data collection became routine, reports hospitals and 1,586 surveys from nurses who worked in
were being used by hospital nurse leadership to these hospitals. The focus of these efforts was based on
monitor patient safety and staffing effectiveness, and examining workload intensity, which was conceptu-
several best practices were adopted by all hospitals, alized as patient turbulence and patient acuity. Patient
such as using the Braden scale (Bergstrom, Braden, turbulence was measured by the admissions, dis-
Laguzza, & Holman, 1987) to predict patient pressure charges, and transfers (ADT) that occurred on a shift.
injury risk. Investigators calculated an ADT index based on the
patient census at the beginning of the shift. As an
Shift-Level Analyses example, an ADT of 0.50 meant that 50% of the patients
The shift-level analyses were conducted using were transferred or discharged during a shift and
Bayesian hierarchical logistic modeling techniques to replaced by patients who were admitted during that
model the odds of one or more adverse event(s) (i.e., shift. A daily measure of patient acuity from the mili-
separate analyses for falls, falls with injury, nurse- tarys Workload Management System for Nursing was
committed medication errors with and without harm, also included (Wolgast, Taylor, Garcia, & Watkins,
and needlestick injuries) occurring on a shift. As ex- 2011). These data were used to examine how work-
pected, with most adverse events, lower RN skill mix load intensity and the quality of the nursing practice
and/or lower levels of total nurse staffing were asso- environment (i.e., scores on the PES-NWI) influenced
ciated with the adverse events. Patient acuity was the relationship of staffing on patient outcomes using
higher when falls and nurse-committed medication mediation and moderation analysis. In this study, the
errors occurred. Nurse staff occupational categories investigators also evaluated the staffing effect on HAPI
(military, civilian, military reserve, or contract) showed using prevalence data from different time points
that civilian staff was a proxy for higher experience before the date the HAPI was discovered. Higher LPN
levels. Higher levels of experienced staff on a shift were care hours 3 days before the prevalence assessment
associated with fewer falls, medication errors, and were associated with lower HAPI rates. HAPI was
6 Nurs Outlook xxx (2017) 1e10

significantly lower with higher nursing care hours evaluations of these data that are related to nurse
worked by LPNs (Patrician et al., 2017). These associa- staffing are used to formulate and justify staffing
tions were not detected when the LPN staffing was models and inform day-to-day decisions, such as
aggregated to higher levels (i.e., to the week and considering trade-offs between patient outcomes and
month). Workload intensity, measured by the ADT nursing requirements when adequate staffing is in
index, was associated with nearly all adverse events, question.
but patient acuity was not. The ADT index partially
mediated the effect of LPN hours on HAPI.
When assessing whether staffing had different ef- Research-Related Lessons Learned
fects on outcomes, based on practice environment
(PES-NWI) scores, there was only one noteworthy
interaction. In medical and surgical units, better The research team has learned several important les-
practice environments and higher total nursing care sons over the course of this work. These lessons
hours were both associated with fewer falls. There pertain to operational definitions, levels of measure-
was a significant interaction of practice environment ment, model specification, and lessons about imple-
scores with total nursing care hours per patient per mentation of measurement strategies to aid in quality
shift and patient falls, indicating that staffing has a improvement work.
stronger effect on falls in better nursing practice First, we cannot influence what we do not measure.
environments. We have shown significant improvements in measures
tracked in the 13 MilNOD hospitals during the period of
active data collection. We believe that comparing the
Moving Beyond MilNOD data among units and hospitals and sharing best
practices are what really work to improve care. We
cannot corroborate this belief, however, because it was
The MilNOD research findings provided military nurses beyond the scope of our studies to measure the in-
and leaders with information about staffing effective- terventions that took place to change practice. We did
ness and the professional practice of nursing in the track the use of evidence-based practices to prevent
military. Leaders were informed and able to make de- pressure injury and saw change over time. At the start
cisions based on high quality and useful data. of our work, only 3 of the 13 hospitals consistently used
In addition, the MilNOD studies were the basis for a pressure injury risk assessment tool. By the end of
the current armywide nursing dashboard, an impor- the first year of the study, 100% of the hospitals were
tant component of the armys care delivery framework using the Braden scale.
called the Patent CaringTouch System (PCTS) The second lesson is that definitions matter. Before
(Breckenridge-Sproat et al., 2015). The goals of the PCTS the MilNOD series of studies, the military hospitals
are to inspire exceptional nursing care by supporting used various operational definitions of adverse events
improved patient outcomes, reducing variations in to include falls. If we cannot agree on standardizing
practice within the Army Health System, and nomenclature, we have no basis for comparisons.
decreasing nursing turnover (Horoho, 2011). To address This is a critical consideration if the nursing profes-
low scores on the Nursing Participation in Hospital sion is to progress in developing nurse-sensitive
Affairs scale of the PES-NWI noted in MilNOD research indicators.
findings, the PCTS incorporates unit-, hospital-, and The third lesson pertains to the unit of analysis.
system-level nursing practice councils, an aspect of From the beginning, ANOD and the MilNOD were
shared governance that has empowered all nursing developed to collect and analyze data at the shift level.
personnel on the team to participate in professional This choice was predicated on the belief that it is
and hospital improvements. important to evaluate the effects of staffing and
Most metrics selected for use in the PCTS were adverse events at the closest time point to the event, as
based on those used in the MilNOD projects (Loan, with discrete adverse events tied to a shift (e.g., falls,
Patrician, & McCarthy, 2011; Patrician, Loan, medication administration errors). We found only one
McCarthy, Brosch, & Davey, 2010a; Patrician et al., other study in which the investigators measured
2011a, 2011b). Nurses in senior leadership positions staffing at the shift level. Needleman et al. (2011) found
and nurse leaders at the bedside use these metrics to increased inpatient mortality associated with shifts
identify early signals of increased or decreased staffing where the staffing levels were below what was rec-
effectiveness and evaluate unit-, hospital-, or army- ommended by the staffing planning methodology.
level changes in nursing structure or process by They also found increased patient turnover (ADT)
monitoring associated outcomes. The research find- associated with increased patient mortality. Thus,
ings along with routinely collected longitudinal data shift-level data are superior to data aggregated at the
are used as a basis for quantifying success of new hospital level if the goal is to understand these discrete
improvement efforts and fostering accountability for phenomena.
the nursing care quality at all levels (Breckenridge- Through the studies based on the MilNOD, we have
Sproat et al., 2015). Findings from deep-dive learned that patient acuity is not a good measure of
Nurs Outlook xxx (2017) 1e10 7

nursing workload. This may be, in part, because acuity demands associated with medication administration
is used to plan staffing in the military hospitals. A more are complex, begin before, and last far beyond the
precise measure of acuity, used in the study by scheduled administration time (Jennings, Sandelowski,
Needleman et al. (2011), may be a ratio of required & Mark, 2011). Validation of existing and newly
nursing care hours to actual care hours. The mea- emerging workload systems, therefore, needs to
surement of workload intensity as it affects the need continue, as the business of nursing is complex and
for additional staffing has been elusive in the field of demanding (Swiger, Vance, & Patrician, 2016). The
health services research in nursing; military research validation of these workload measurement systems can
shows that ADT is superior to census alone in only occur in light of the research identifying variables
measuring staffing effectiveness. that impact nursing workload. For example, workload
Fourth, modeling some of our rare events, such as generated from organizational demands and the effects
falls and medication errors, is extremely difficult of patient turnover are often left out of measurement
because of the analytic problems caused by the systems (Jennings, Sandelowski, & Higgins, 2013;
numerous shifts where these rare events do not occur. Krichbaum et al., 2007; Myny et al., 2012; Park, Dunton,
Rare events complicate statistical analyses and & Blegen, 2016) and may confound the results of staff-
require different types of approaches, including Pois- ing and outcomes studies when left unaccounted.
son and negative binomial models (Loan & Patrician, Second, the use of contemporary statistical methods
2005). and the study of big data generated from electronic
Fifth, we need to incorporate implementation sci- health records may advance this area of research.
ence methods to effect improvements in care (Tabak, Mining large data sets may provide opportunities to
Khoong, Chambers, & Brownson, 2012). Implementa- identify associations between staffing and outcomes
tion science offers ideas about strategies for that were previously unknown (Topaz & Pruinelli,
improving nursing-sensitive patient outcomes. 2017). The capture of data as an event happens may
Although we shared results quarterly with nursing also provide insights that exceed those generated from
leaders at each site, we do not have a good account of the shift-level data seen in the MilNOD studies.
whether (or how) they used the data to make changes. Next, measurement instruments would benefit from
We used some outreach methods, such as newslet- modernization. For example, the PES-NWI was devel-
ters, to share best practices to assist with knowledge oped in the early 2000s with samples collected during
building; these methods were helpful but difficult to the original Magnet hospital studies of the 1980s (Lake,
sustain. 2002). The PES-NWI is a common covariate used in
staffing research, and although this measure has un-
dergone modifications for use in specific settings, the
The Future of Military Nursing Staffing, overall instrument remains unchanged. The context in
Practice Environment, and Outcomes which nursing care is delivered and the complexity of
Research that care, however, has changed substantially since the
late 1980s (Ebright, 2010; Ebright, Patterson, Chalko, &
Render, 2003; Myny et al., 2011; Suter et al., 2012).
Although many studies, including those discussed in this Further development and refinement of the PES-NWI
article, demonstrate the positive effects of more staffing, would ensure we are not missing important factors
there is still insufficient evidence to determine the ideal that contribute to the delivery of high-quality care, yet
staffing needed to ensure the delivery of high-quality remain unmeasured by the current instrument.
care (Griffiths et al., 2016). Most staffing and outcomes Finally, learning about measures of individual nurse
research aggregates data to the hospital level; minimal performance and effectiveness may also be important
research is conducted at the unit or shift levels. In addi- in identifying the ideal number and skill mix of nursing
tion, because of perceived and actual methodological staff required to provide quality care. Similar to the
restrictions and the presence of unmeasured or poorly way performance and compliance is tracked for inde-
measured variables, the results of staffing research are pendent licensed providers, such as physicians and
difficult to translate into specific practice recommenda- nurse practitioners, individualized measures of per-
tions (Griffiths et al., 2016). One goal of future military formance could lead to a direct measure of nurse
staffing research may be focused on identifying the ideal effectiveness. Measures commonly studied to deter-
number of staff and best skill mix to maintain high- mine individual nurse effectiveness, such as educa-
quality care. Several steps could help achieve this goal. tion, experience, and licensure, are often used as
First, existing workload measurement systems indirect measures of nursing competence (Griffiths
should be revalidated. Workload measures such as et al., 2016). More concrete measures of individual
nursing care hours per patient day and total nursing competence and performance, such as identification of
care hours per patient day, which are used frequently to deteriorating patient status and taking appropriate
determine staffing needs, could be incorrect or biased follow-on actions, could be tracked electronically. The
toward underestimating the actual need for nursing goal of collecting individual performance measures
care (Twigg, Gelder, & Myers, 2015). For example, the would not be to place blame but instead to identify top
8 Nurs Outlook xxx (2017) 1e10

performers and support underperformers with addi- Aiken, L. H., Sloane, D. M., Lake, E. T., Sochalski, J., & Weber, A. L.
tional resources. This philosophy is in line with high (1999). Organization and outcomes of inpatient AIDS care.
Medical Care, 37, 760e772.
reliability organizations that gather and display quality
American Nurses Association (ANA). (1995). Nursing quality
measures, reward staff for reaching and sustaining
indicators: Recommended definitions. Washington, DC: American
quality goals, and work to identify unsafe conditions Nurses Publishing.
before patients are harmed (Griffith, 2015). Anderson, F. D., Maloney, J. P., Oliver, D. L., Brown, D. L., &
Hardy, M. A. (1996). Nurse-physician communication:
Perceptions of nurses at an Army medical center. Military
Medicine, 161, 411e415.
Conclusions Aydin, C. E., Bolton, L. B., Donaldson, N., Brown, D. S., Buffum, M.,
Elashoff, J. D., & Sandhu, M. (2004). Creating and analyzing a
statewide nursing quality measurement database. Journal of
Nursing Scholarship, 36, 371e378.
Although the understanding of nurse staffing as it re- Bergstrom, N., Braden, B. J., Laguzza, A., & Holman, V. (1987). The
lates to quality care has advanced over the past 20 Braden scale for predicting pressure sore risk. Nursing
years, the complexity of the care environment and the Research, 36(4), 205e210.
patients themselves leave much to be learned about Blegen, M. A., & Vaughn, T. (1998). A multisite study of nurse
ideal nurse staffing. The program of research repre- staffing and patient occurrences. Nursing Economics, 16,
sented by the MilNOD, its predecessors, and the armys 196e203.
Blegen, M. A., Vaughn, T., & Goode, C. J. (2001). Nurse experience
Patient CaringTouch System illustrate how military
and education: Effect on quality of care. Journal of Nursing
nurses and funding by the TSNRP have ensured mili- Administration, 31, 33e39.
tary beneficiaries benefit from military unique knowl- Bolton, L. B., Aydin, C. E., Donaldson, N., Brown, D. S., Sandhu, M.,
edge and evidence-based practice. Not only has Fridman, M., & Aronow, H. U. (2007). Mandated nurse staffing
MilNOD ensured that the military kept pace with in- ratios in California: A comparison of staffing and nursing-
vestigations of nurse staffing and quality care in the sensitive outcomes pre- and post-regulation. Policy, Politics, &
Nursing Practice, 8, 238e250.
civilian sector, but studies based on MilNOD data are
Breckenridge-Sproat, S., Freeman, J. J., Belew, D. L., Loan, L. A.,
able to show the benefit of shift-level datada largely McCarthy, M. S., & Patrician, P. A. (2015). A nursing care redesign
unprecedented accomplishment. framework for system wide improvement: The Patient
CaringTouch System. Clinical Nursing Studies, 3(4), 72e78.
Breckenridge-Sproat, S., Johantgen, M., & Patrician, P. A. (2011).
Influence of unit-level staffing on medication errors and falls
Acknowledgments in military hospitals. Western Journal of Nursing Research, 34,
Brosch, L. R. (2005). Establishing a military nursing outcomes
database. Final report N02eP06). Bethesda, MD: TriService
This project was funded by the TSNRP, Uniformed
Nursing Research Program.
Services University of the Health Sciences (grants Cho, S. H., Ketefian, S., Barkauskas, V. H., & Smith, D. G. (2003).
N97-011, N01-016, N02-P06, N03-P07, N10-C01, and The effects of nurse staffing on adverse events, morbidity,
N13-P13); however, the information or content and mortality, and medical costs. Nursing Research, 52, 71e79.
conclusions neither do necessarily represent the offi- Donaldson, N. E., Brown, D. S., Aydin, C. E., & Bolton, L. B. (2001).
cial position or policy of nor should any official Nurse staffing in California hospitals 1998-2000: Findings from
endorsement be inferred by the TSNRP, Uniformed the California nursing outcomes coalition database project. Policy,
Politics & Nursing Practice, 2(1), 20e29.
Services University of the Health Sciences, the
Dunton, N., Gajewski, B., Taunton, R. L., & Moore, J. (2004). Nurse
Department of Defense, or the U.S. government. staffing and patient falls on acute care hospital units. Nursing
Outlook, 52, 53e59.
Ebright, P. R. (2010). The complex work of RNs: Implications for
healthy work environments. The Online Journal of Issues in
references Nursing, 15(1), Manuscript 4.
Ebright, P. R., Patterson, E. S., Chalko, B. A., & Render, M. L. (2003).
Understanding the complexity of registered nurse work in
Aiken, L. H., Clarke, S. P., Cheung, R. B., Sloane, D. M., & Silber, J. acute care settings. Journal of Nursing Administration, 33,
H. (2003). Educational levels of hospital nurses and surgical 630e638.
patient mortality. JAMA, 290, 1617e1623. Foley, B. J., Kee, C. C., Minick, P., Harvey, S. S., & Jennings, B. M.
Aiken, L. H., Clarke, S. P., Sloane, D. M., Lake, E. T., & Cheney, T. (2002). Characteristics of nurses and hospital work
(2008). Effects of hospital care environment on patient environments that foster satisfaction and clinical expertise.
mortality and nurse outcomes. Journal of Nursing Journal of Nursing Administration, 32, 273e282.
Administration, 38, 223e229. Griffith, J. R. (2015). Understanding high-reliability organizations:
Aiken, L. H., Clarke, S. P., Sloane, D. M., Sochalski, J., & Silber, J. H. Are Baldrige recipients models? Journal of Healthcare
(2002). Hospital nurse staffing and patient mortality, nurse Management, 60, 44e61.
burnout, and job dissatisfaction. JAMA, 288, 1987e1993. Griffiths, P., Ball, J., Drennan, J., DallOra, C., Jones, J., Maruotti, A.,
Aiken, L. H., & Sloane, D. M. (1997). Effects of organizational ., Simon, M. (2016). Nurse staffing and patient outcomes:
innovations in AIDS care on burnout among urban hospital Strengths and limitations of the evidence to inform policy and
nurses. Work and Occupations, 24, 453e477. practice. A review and discussion paper based on evidence
Nurs Outlook xxx (2017) 1e10 9

reviewed for the National Institute for Health and Care Mitchell, G. J., & Durenberger, D. (1990). Promoting value in health
Excellence Safe Staffing guideline development. International care: The new Agency for Health Care Policy and Research.
Journal of Nursing Studies, 6, 213e225. Academic Medicine, 65, 204e205.
Horoho, P. D. (2011). Army nursing: Transforming for a new Mitchell, P. H., & Shortell, S. M. (1997). Adverse outcomes and
century of caring. US Army Medical Department Journal, 4e9. variations in organization of care delivery. Medical Care, 35(11
Jacox, A., Bausell, B., & Mahrenholz, D. (1997). Patient satisfaction Suppl), S19eS32.
with nursing care in hospitals. Outcomes Management for Montalvo, I. (2007). The National Database of Nursing Quality
Nursing Practice, 1(1), 20e28. Indicators. Online Journal of Issues on Nursing, 3(2). Retrieved
Jennings, B. M., Loan, L. A., DePaul, D., Brosch, L. R., & Hildreth, P. from
(2001). Lessons learned while collecting ANA indicator data. ANAMarketplace/ANAPeriodicals/OJIN/TableofContents/
Journal of Nursing Administration, 31, 121e129. Volume122007/No3Sept07/NursingQualityIndicators.html
Jennings, B. M., Sandelowski, M., & Higgins, M. K. (2013). Turning Myny, D., Van Goubergen, D., Gobert, M., Vanderwee, K., Van
over patient turnover: An ethnographic study of admissions, Hecke, A., & Defloor, T. (2011). Non-direct patient care factors
discharges, and transfers. Research in Nursing & Health, 36, influencing nursing workload: A review of the literature.
554e566. Journal of Advanced Nursing, 67, 2109e2129.
Jennings, B. M., Sandelowski, M., & Mark, B. (2011). The nurses Myny, D., Van Hecke, A., De Bacquer, D., Verhaeghe, S., Gobert, M.,
medication day. Qualitative Health Research, 21, 1441e1451. Defloor, T., & Van Goubergen, D. (2012). Determining a set of
Kane, R. L., Shamliyan, T. A., Mueller, C., Duval, S., & Wilt, T. J. measurable and relevant factors affecting nursing workload in
(2007). The association of registered nurse staffing levels and the acute care hospital setting: A cross-sectional study.
patient outcomes: Systematic review and meta-analysis. International Journal of Nursing Studies, 49, 427e436.
Medical Care, 45, 1195e1204. National Pressure Ulcer Advisory Panel (2016). National Pressure
Kazanjian, A., Green, C., Wong, T., & Reid, R. (2005). Effect of the Advisory Panel announces a change in terminology from pressure ulcer to
hospital nursing environment on patient mortality: A pressure injury and updates the stages of pressure injury. Press release.
systematic review. Journal of Health Service Research & Policy, 10, Retrieved from
111e117. advisory-panel-npuap-announces-a-change-in-terminology-
Kovner, C., & Gergen, P. J. (1998). Nurse staffing levels and adverse from-pressure-ulcer-to-pressure-injury-and-updates-the-sta
events following surgery in U.S. hospitals. Image: Journal of ges-of-pressure-injury/
Nursing Scholarship, 30, 315e321. Needleman, J., Buerhaus, P., Meattke, S., Stewart, M., &
Krichbaum, K., Diemert, C., Jacox, L., Jones, A., Koenig, P., Zelevinsky, K. (2002). Nurse-staffing levels and the quality of
Mueller, C., & Disch, J. (2007). Complexity compression: Nurses care in hospitals. The New England Journal of Medicine, 346,
under fire. Nursing Forum, 42(2), 86e94. 1715e1722.
Lake, E. T. (2002). Development of the practice environment scale Needleman, J., Buerhaus, P., Pankratz, V. S., Leibson, C. L.,
of the Nursing Work Index. Research in Nursing & Health, 25, Stevens, S. R., & Harris, M. (2011). Nurse staffing and inpatient
176e188. hospital mortality. The New England Journal of Medicine, 364,
Lake, E. T., & Cheung, R. B. (2006). Are patient falls and pressure 1037e1045.
ulcers sensitive to nurse staffing? Western Journal of Nursing Page, A. (2004). In Keeping patients safe: Transforming the work
Research, 28(6), 1e24. environment of nurses. Washington, DC: National Academy
Lang, T. A., Hodge, M., Olson, V., Romano, P. S., & Kravitz, R. L. Press.
(2004). Nurse-patient ratios: A systematic review on the effects Park, S. H., Dunton, N., & Blegen, M. A. (2016). Comparison of unit-
of nurse staffing on patient, nurse employee, and hospital level patient turnover measures in acute care hospital
outcomes. Journal of Nursing Administration, 34, 326e337. settings. Research in Nursing & Health, 39, 197e203.
Loan, L. A., Brosch, L. R., McCarthy, M. S., & Patrician, P. A. (2005). Patrician, P. A., Loan, L., McCarthy, M., Brosch, L. R., & Davey, K. S.
Designing and implementing a national database depicting (2010a). Towards evidence-based management: Creating an
quality of nursing care and staffing effectiveness. Army Medical informative database of nursing-sensitive indicators. Journal of
Department Journal, 50e58. Nursing Scholarship, 42, 358e366.
Loan, L. A., Jennings, B. M., Brosch, L. R., DePaul, D., & Hildreth, P. Patrician, P. A., Loan, L., McCarthy, M., Fridman, M.,
(2003). Indicators of nursing care quality: Findings from a pilot Donaldson, N., Bingham, M., & Brosch, L. R. (2011a). The
study. Outcomes Management for Nursing Practice, 7(2), 51e58. association of shift-level nurse staffing with adverse patient
Loan L., & Patrician, P. A. (2005). Nursing quality data: Characteristics events. The Journal of Nursing Administration, 41, 64e70.
of the data and analytical challengesdReport from the 2004 Patrician, P. A., McCarthy, M. S., Swiger, P., Raju, D., Breckenridge-
National Nursing Quality Databases Analytical Conference. Paper Sproat, S., Su, X., Randall, K. H., & Loan, L. A. (2017).
presentation at the National Nursing Quality Databases 2005 Association of hospital-acquired pressure injury with
National Conference, San Francisco, CA temporal variations in staffing in military hospitals. Research in
Loan, L. A., Patrician, P. A., & McCarthy, M. (2011). Participation in Nursing & Health, 40(2), 111e119.
a national nursing outcomes database: Monitoring outcomes Patrician, P. A., Pryor, E., Fridman, M., & Loan, L. (2011b).
over time. Nursing Administration Quarterly, 35(1), 72e81. Needlestick injuries among nursing staff: Association with
Maloney, J. P., Anderson, F. D., Gladd, D. L., Brown, D. L., & shift level staffing. American Journal of Infection Control, 39,
Hardy, M. A. (1996). Evaluation and comparison of health care 477e482.
work environment scale in military settings. Military Medicine, Patrician, P. A., Shang, J., & Lake, E. T. (2010b). Organizational
161, 284e289. determinants of quality care and nurse job outcomes among
Mark, B. A., & Belyea, M. (2009). Nurse staffing and medication Army Medical Department registered nurses. Research in
errors: Cross-sectional or longitudinal relationships? Research Nursing & Health, 33, 99e110.
in Nursing & Health, 32, 18e30. Spratley, E., Johnson, A., Sochalski, J., Fritz, M., & Spencer, W.
Mark, B. A., Hughes, L. C., Belyea, M., Bacon, C. T., Chang, Y. K., & (2000). The registered nurse population: Findings from the National
Jones, C. A. (2008). Exploring organizational context and Sample Survey of Registered Nurses. Washington, DC: US
structure as predictors of medication errors and patient falls. Department of Health and Human Services, Health Resources
Journal of Patient Safety, 4, 66e77. and Service Administration.
10 Nurs Outlook xxx (2017) 1e10

Suter, E., Deutschlander, S., Mickelson, G., Nurani, Z., Lait, J., Twigg, D. E., Gelder, L., & Myers, H. (2015). The impact of
Harrison, L., ., Grymonpre, R. (2012). Can interprofessional understaffed shifts on nurse-sensitive outcomes. Journal of
collaboration provide health human resources solutions? A Advanced Nursing, 71, 1564e1572.
knowledge synthesis. Journal of Interprofessional Care, 26, 261e268. Whitman, G. R., Kim, Y., Davidson, L. J., Wolf, G. A., &
Swiger, P., Vance, D. E., & Patrician, P. A. (2016). Nursing workload Wang, S. L. (2002). The impact of staffing on patient
in the acute care setting: A concept analysis of nursing outcomes across specialty units. Journal of Nursing
workload. Nursing Outlook, 64, 244e254. Administration, 32, 633e639.
Tabak, R. G., Khoong, E. C., Chambers, D., & Brownson, R. C. Wolgast, K. A., Taylor, K., Garcia, D., & Watkins, M. (2011).
(2012). Bridging research & practice: Models for dissemination Measuring nursing essential contributions to quality patient
and implementation research. American Journal of Preventive care outcomes. The United States Army Medical Department
Medicine, 43, 337e350. Journal, 74e81.
Topaz, M., & Pruinelli, L. (2017). Big data and nursing: Implications Wunderlich, G. S., Sloan, F., & Davis, C. K. (1996). Nursing staff in
for the future. Studies in Health Technology and Informatics, 232, hospitals and nursing homes: Is it adequate? Washington, DC:
165e171. National Academy Press.