Practical Challenges of Systems Thinking and Modeling in Public Health

Wiiiiam M, Trochim, PhD, Derek A, Cabrera, iVlA, Bobby Milstein, iVIPH, Richard S, Gallagher, BS, and Scott J, Leischow, PhD

Modem public health practice encompasses a complex, loosely coupled system' of actors including govemmental entities at the intemational, national, regional, and local levels; a diverse conglomeration of nongovemmental organizations (such as foundations, advocacy and specicil interest groups, coalitions and partnerships, for-profit and nonprofit medical systems, and businesses); and citizens in the public at large. The broad array of threats to well-being, ranging from obesity and tobacco use to violence and infectious diseases, can be most aptly portrayed from a complex and adaptive system perspective. Systems thinking and modeling are broad classes of intellectual endeavors that are being incorporated increasingly into contemporary public health. Research has proven both the general potential of systems thinking^"" and applications in specific areas,'^"^* Empirical studies related to complex systems have appeared of late in notable medical journals, including ihe Joumal ofthe American Medical Association, Lancet, and the New England Joumal of Medidne?^'^^ The authors of an Institute of Medicine report. Crossing the Quality Chasm: A New Health System for the 21st Century,"^''''^'^^ used a systems perspective to delineate 10 "simple mles to guide the redesign of the health care system" and described the entire health care system as a complex adaptive system:
A health care system can be defined as a set of connected or interdependent parts or agentsincluding caregivers and patients—bound by a common purpose and acting on tiieir knowiedge. Heaith care is complex because of the great number of interconnections within and among smail care systems.'"'''''"

Objectives. Awareness of and support for systems thinking and modeling in the public health field are growing, yet there are many practical challenges to implementation. We sought to identify and describe these challenges from the perspectives of practicing public health professionals. Methods. A systems-based methodology, concept mapping, was used in a study of 133 participants from 2 systems-based public health initiatives (the Initiative for the Study and Implementation of Systems and the Syndemics Prevention Network), This method identified 100 key challenges to implementation of systems thinking and modeling in public health work. Results. The project resulted In a map identifying 8 categories of challenges and the dynamic interactions among them. Conclusions. Implementation by public health professionals of the 8 simple rules we derived from the clusters in the map identified here will help to address challenges and improve the organization of systems that protect the public's health, (Am J Public Health. 2006;96:538-546, doi:10,2105/AJPH,2005,066001)

over the past decades, particularly infieldssuch as system dynamics and complexity theory. It is relatively easy to identify examples of public health issues that can be understood accurately only by examining the complex and dynamic part-and-whole interactions that make up systems. For instance, consider the area of tobacco control. At the policy level, it is reasonable to argue that the 1964 surgeon general's report on smoking^^ had profound effects on the policy debate with consequences for smoking prevalence and consumption to this day. The report itself was the product of a complex series of events that led to its production. In tum, it set off a cascade of events and changes. It is virtually impossible to determine the effects of that important event in isolation, as a part that is separable from the whole. For instance, the report was most likely an important catalyst in creating a public policy climate that enabled the litigation that led to the Tobacco Settlement Agreement several decades later, to increased taxation of cigarettes by states, to legal restrictions on smoking in public places, and to tobacco counteradvertising. By the same token, the report may have led to unanticipated "negative" consequences by spurring the tobacco industry to adapt its

Systems thinking encompasses and is consonant with ecological models^^""** familiar to public health practitioners, including the ideas of human ecology, population health, and the social determinants of public health. But it goes beyond these models, incorporating advances

product marketing, lobbying, and public relations and perhaps indirectly contributed to the creation offi-ontgroups and covert efforts to undermine tobacco control research,""* There are countless other examples. Smokers may react to fears about the harmful effects of smoking by switching to so-called "light" cigarettes (i,e,, low tar and nicotine formulations as determined by machine smoking tests). Yet, despite their manufacturers' claims, "light" cigarettes may actually increase the prevalence of a more lethal type of lung cancer, probably because of the way their ventilated filters alter the physical act of smoking,^'"*^ Or consider the way the industry responds to youth advertising restrictions. Billboard advertising was outlawed by the Master Settlement Agreement, In response, the tobacco companies increased advertising and promotion in retail stores, effectively saturating the retail environment with product images,'*'' Thus, banning billboard advertising has led to even more children being exposed to pro-tobacco messages as they go about their daily Uves, Even the cigarette itself is a complex system, a highly engineered product designed for extreme elasticity of delivery.''^ And tobacco control as a public endeavor can be viewed

538 I Research and Practice i Peer Reviewed i Trochim et al.

American Journal of Public Health | March 2006, Vol 96, No, 3

heterarchical. we consider 2 common misconceptions about systems thinking that are important to an understanding of these ideas and metaphors. In addition. Second. nonlinear systems and chaos^^. the rules typically instruct the agent to activate the EMS system by "calling 911" and then administer basic care until professional help arrives. and ecology'*"*'. but it is also something very modem.^''^* focuses on understanding systems of this type."*'' which are now part of the public vernacular. 3 | American Journal of Public Health Trochim et al. we provide an overview of public health initiatives that are exploring and using systems thinking and modeling. control theory^*. Most systems in the public health arena eire complex in that they consist of many interacting stakeholders with often different and competing interests. and organizations that are engaged infightingtobacco use. biology. networks of adaptive agents. the EMT initiates the involvement of the next set of agents by Dynamics. small world phenomena®^"*''. autopoiesis^^"^''."'"'''" and another description suggests that a working definition of a complex system is "one whose properties are not fully explained by an understanding of its component parts. and it is not easily summarized. a common theme is change."""''"' Simple rules. These roles and rules are graduated in a linked chain from first responder to emergency room doctor to rehabilitation specialist For a citizen in first responder role. with its own rich history dating back to Isaac Nevrton. fractal self-similarity'"^. ^^^''.' Finally. Despite the growing cognizance of and support for "systems thinking" in public health. have their source in the study of chaotic systems. First. boundciry conditions. For example.^^ Terms such as the "butterfly effect"*'** and the "tipping point. SYSTEMS THINKING AND MODELING Systems thinking is a general conceptual orientation concerned with the interrelationships between parts and their relationships to a functioning whole. often understood within the context of an even greater whole.'^•'^•''^•'**' We sought to identify and describe the challenges that must be addressed by public health leaders in implementing effective systems-based approaches. and set. firefighter. one description of a complex adaptive system is independent variables following simple local rules leading to emergent complexity. information theory"^. each agent has a role to play and a simple set of rules to follow. feedback. decision and game theory". universality. Most defmitions of such systems include some notion of the relationship between the emergent or unpredictable behavior of a system and autonomous agents self-organizing by simple rules. we present the results of an initial empirical study in which we used a systems-based method (concept mapping) with a self-selected group of public health professionals in an attempt to identify the chcillenges facing those who support systems thinking and modeling in public health. as a complement to experimentation and theory. We offer 2 organizing ideas (dynamics and complexity) and 2 infiuential metaphors {mechanical and biological) that can help us understand this daunting array. In chaos. No.''^ The depth eind breadth of systems science can be bewildering. It illustrates well how a complex adaptive system is made up of various independent agents following simple rules and interacting locally with other independent agents in the system. we discuss systems thinking and modeling and their most recent developments. government agencies. It is ancient in origin and fcimiliar to us all. \ Peer Reviewed | Research and Practice | 539 . Next in the chain. Such systems are not controlled centrally. Agents in these networks must constantly adapt to the actions of others and to a changing environment that is in turn affected by the actions of the agents themselves. silo effects^'. Consider just a few of the topics associated with contemporary systems thinking: causal feedback''*. stock-flow structures and open and closed systems'*®. particularly the Initiative for the Study and Implementation of Systems (ISIS). as in all other areas of public health work. Dynamics and Complexity Complexity. and emergence are hallmarks of complex adaptive systems. and the need for systems thinking and modeling is ubiquitous. and when we try to puzzle out the dynamics of the economy. A good example of a complex adaptive system that is familiar to virtually everyone is the emergency medical services (EMS) system that comes into play in some way in virtually every medical or health emergency that an individual experiences. and self-organizing systems^°'^'. In tum. We engage in a type of systems thinking in our everyday lives when we contemplate the complex interactions of our relationships with families and fiiends. or the study of complex adaptive systems. Systems modeling is a methodological tradition that involves the use of formal models or simulations as explicit aids to increase our understanding of complex systems and improve the effectiveness of our actions within them. or emergency medical technician (EMT) plays a different role and follows a different rule set with primary responsibility for advanced field care and transport. In the EMS. implementation of effective systems approaches remains challenging. complex adaptive systems'^"^®. they are self-organizing. general systems cybernetics'''''^'. In tobacco control. the professional rescuer. Computational modeling and simulation. Here we address 3 major topics.mimm hm mmmi as a set of systems challenges regarding how we might best balance the complex configurations of individuals.=^'^'*'''*2'«3'*^'«'' The vastness of the literature alone can be overwhelming. "a deterministic system exhibits aperiodic behavior that depends sensitively on the initial conditions. decentralized. evolution. centralized. phase transitions. thereby rendering longterm prediction impossible. or changes in even more complicated ways. changes in repeating cycles. seeding. hierarchical. are hallmarks of recent systems thinking and the systems sciences. provides a vocabulary and a methodology for understanding these changes. power laws."^^''''" This concept is just one of the many useful ideas in the field of dynamics. and network theoiy. systems problems are legion. computational simulation''^™. Vol 96. celluleir automata®''. particularly as one first is introduced to its underlying principles and formulations. and renormalizadon^^'^". March 2006. system dynamics'^"^'. A field called dynamics. graph. Whether a system settles into a state of equilibrium. self-organization. when we organize in our communities or workplaces. Complexity theory. But systems thinking also encompasses some of the most advanced and sophisticated recent work in contemporary science.

2 misconceptions about the systems approach need to be addressed. More scholars are studying and writing on the topic. or atomistic and framed by mechanical. Toward that end. systems epistemology "shares the same scientific attitude" with scientific or analytic thinking. we take the EMS system for granted. knowledge management and informatics. the emergency room connects with the agents of hospital inpatient care. Although the vast array of participating govemmental. or federal-level emergency disaster response system has not yet evolved sufficient capacity to deal with large-scale catastrophes. According to Von Bertalanffy. Studies of systems. tobacco control. and it is difficult to remember a time when it did not exist to help individuals in an emergency. Tens of thousands of people are trained in various roles by a wide variety of entities ranging from medical and public health schools to nonprofit organizations such as the Red Cross. noted: "It is important to remember the embedded nature of systems. management sciences. ISIS supported efforts that resulted in practical examples of systems ideas in public health contexts: development of a system dynamics model for characterizing the complex state of In addition to the 2 broad organizing ideas just discussed.g. infiuenced by both types of metaphors. But the recent horrific hurricane in New Orleans and surrounding areasremindsus that a comparable regional-. instead. biological.'"^ A good example is ISIS. more research is emphasizing a systems view. and accessible. behave more like machines than like biological organisms. but some systems. in turn often composed of subsystems that are themselves evolving and adapting to the environment. such as in bioengineering. This network of thinkers considered some core questions: How can the flow in botk directions between research and practice be optimized? How can systems structure and function be best characterized to be useful to the public health community? Which approaches can be used for better understanding and optimization of networks? Through which strategies do information and knowledge become the currency for change? The ISIS team concluded that systems thinking in public health cannot be encompassed by a single discipline or even a single approach to "systems thinking" (e. and these agents in tum join subsequently with posthospital rehabilitation. No. What is considered a holistic. 3 ."" Consequently. and precursors to systems thinking. public health. have led to many significant scientific discoveries. 540 I Research and Practice | Peer Reviewed | Trochim et al. systems are construed as machines made up of parts or subsystems that interact in complex ways to produce certain characteristic behaviors. The purpose of ISIS is to explore whether systems thinking can serve as a foundation for more effective public health efforts to combat tobacco use.^^ it is not a centrally and hierarchically controlled system. First. systems view is considered a reductionist view when the boundaries of the system are redrawn. it does not replace or reject it. from clinicians to policymakers. and reductionist and holist thinking and methods into a federation of approaches to systems thinking and modeling. is to value systems thinking as a guiding approach.'°^ The ISIS team also recognized that the complexity and breadth of systems thinking may be dismissed as being too complicated. American Journai of Pubiic Heaith | IVlarch 2006. There are also mixes of the metaphors. particularly in the face of countervailing forces such as the efforts of the tobacco industry. most of the techniques used for systems thinking and modeling are rooted in mathematics as well as the physical. systems thinking is not a rejection of traditional scientific views that are linear.' This fallacy probably stems from popular literature portraying systems thinking as "soft" or in opposition to scientific or analytic thinking. medical.communicating with the emergency room. CURRENT SYSTEMS THINKING EFFORTS IN PUBLIC HEALTH The field of public health is adapting to the evolution of systems thinking and its accompanying modeling approaches.^^ Although systems thinking inherently is not either mechanistic or biological. system dynamics). In the biological metaphor. The biological metaphor appears to be increasingly prevalent. it consists of a transdisciplinary integration of public health approaches that strive to understand and reconcile linear and nonlinear. and nonprofit organizations involved in the EMS system have coordinated their efforts in various ways for several decades. reductionist.. manageable.^^ Although individual agents may be aware of the existence of the broader system. mechanistic. particular phenomena may be aptly characterized by one or the other metaphor or by some combination of the 2 metaphors. In the mechanical metaphor. barriers to. Mechanical and Biological Systems even complex and nonlinear systems. and they have been used to conduct some of the most rigorous and sophisticated experiments ever devised. their training concentrates on their specific part in it and how it connects with adjacent parts—the role they play and the rules they follow—leading to a system that can adapt to a great variety of individual medical emergencies. A second misconception is that systems thinking lacks scientific rigor. state-. a project of the Tobacco Control Research Branch of the US National Cancer Institute.''"'"'''*"''^"'^''^'' In a study focusing on the enablers of. If needed. Misconceptions Finally. systems are living and evolving entities. it must be practical. and ambitious attempts are under way to focus practitioners on improving overall system performance. ISIS brought together a transdisciplinary group of leaders in fields such as system dynamics. or temporal metaphors. and social sciences. and health policy to develop afi-ameworkfor systems action. systems thinking encompasses and includes reductionism. in which cells are thought of as tiny biological machines. Davidz et al. except in rural and more isolated communities in the United States. spatial. Today. network analysis. it is useful to distinguish 2 metaphors for systems that are both prevalent and infiuentiar«''°'"'^''-'=: systems as mechanical and systems as biological."^^'"''"^' Contrary to popular claims. Voi 96. qualitative and quantitative.^^'"'''^''' Systems thinkers achieve a holistic view of complex phenomena«0'^°'^'''^=^'''^'»o precisely because they approach the study of relationships as a distinct and legitimate form of inquiry. If the public health community.

The resulting maps showed each of the statements. Most of these individuals are practicing public health professionals (e." the Community-University Partnerships Initiative"'' sponsored by the W. K. 133 participants visited the Web page. systems-based view of financing. and the cluster map showing the 8-cluster solution. researchers.285. Here we describe each cluster briefiy."^ and the efforts of the World Health Organization's Commission on Social Determinants of Health. No. | Peer Reviewed | Research and Practice | 541 . Figure 1 shows the final map with the cluster labels arrived at through a consensus process that involved a subgroup of the participants."'' the Healthy Cities movement. ISIS is hardly the only effort to assess the value of systems thinking and modeling in public health work. a significant percentage also identified themselves as having special expertise and training in systems thinking and modeling.. with more similar ones located nearer each other."* the Turning Point Program. participants brainstormed or free listed a large set of statements addressing an agreed-upon focus statement for the project. and the ISIS list included approximately 60 participants from Canada.300. Vol 96. and 54 completed the ratings. collaborations.""""^ Examples of other relevcint efforts include a major Institute of Medicine report. The cluster labeled "Expeind Cross-Category Funding" consisted of 10 statements primarily related to financial issues. creation of a map of the social network of tobacco control organizations. (More details on the methods and results of this project are available from the first author on request.) Concept iVIapping cluster analysis'^*'^^ methods. with a range from 0. listed by cluster.tobacco use and its control.352. moving from highest to lowest in cluster average importance rating as listed in Table 1. ISIS supported actual networks for global tobacco research and reduction of harm from tobacco and produced a monograph' summarizing the 2-year effort and serving as a road map for future approaches to systems thinking in pubUc health. policymakers). Concept mapping is a systems method that enables a group to describe its ideas on any topic'^' and represent these ideas visually in a map.'^' The method has been used in a wide range offields."'' the Community-Campus Partnerships for Health. Australia."' the efforts of the Institute for Healthcare Improvement. and partnerships that span traditional disciplines and boundaries and value diverse perspectives."^ the Partnership for the Public's Health.. In a study of the reliability of concept mapping. The Syndemics Prevention Network included more than 300 members from 11 countries. the point map showing each statement..g. The stress value in the present analysis was 0. teams. for each cluster. Initial interpretation of the maps followed the general process described by Trochim. the community-based participatory research efforts sponsored jointly by the Agency for Healthcare Research and Quality and the W. program managers. Trochim''"' reported an average stress value across 33 projects of 0. however. 56 psirticipants completed the sorting task. Kellogg Foundation. The Syndemics Prevention Network.'^^including health services research'^^•'^'' and public health. and the United States. The cluster labeled "Support Dynamic and Diverse Networks" contained 8 statements about encouraging networks. The general procedure for concept mapping has been described in detciil by Trochim. Table 1 lists. 3 | American Journal of Public Health Trochim et al. These statements challenged traditional funding categories and explicitly encouraged a more integrative. and a knowledge management map to guide the use of information in tobacco control. 2004.155 to 0.'^' RESULTS In the brainstorming phase. Each generated statement completed the following focus prompt: "One specific practical challenge that needs to be addressed to encourage and support effective systems thinking and modeling in public health work is. methodological. with a lower stress value indicating a better fit. AIDS) expands the conceptual. An 8-cluster solution was selected as the one that preserved the most detail and yielded substantively interpretable clusters of statements. violence. the associated practical dilemmas and challenges need to be identified and addressed.'°''""'^ supported by the Centers for Disease Control and Prevention. The stress value is the usual statistic reported in multidimensional scaling analyses to indicate goodness of fit.'^" If systems thinking cind modeling are to be successfully integrated into public health practice. METHODS Participants Invitations to participate in the project were sent to 359 individuals who were on the e-mail distribution lists of the Syndemics Prevention Network and the ISIS project as of December 15. K. studies how recognition of mutually reinforcing health problems (substance abuse. To this end. In the sorting and rating phase. These data were assessed in a sequence of multivariate statistical cUiEilyses that included multidimensional scaling*^*"^' cind hierardiicsil March 2006. and moral dimensions of public health work. The key materials used in the interpretation of the results included the statements produced through brainstorming. a concept mapping project to promote better understanding of how to integrate research and practice. Each participant was invited to sort these statements into groups of similar ones and rate each statement in terms of its relative importance as a challenge that must be addressed to encourage and support systems thinking and modeling in public health. Kellogg Foundation. we conducted an initial study with systems thinkers (individuals involved in ISIS and the Syndemics Prevention Network) in which we used a systems methodologystructured concept mapping—to describe the challenges of systems thinking and modeling in public health.. the 3 challenges to implementing systems thinking that were assigned the highest average importance ratings. All participation was via the Web.'^^ To accomplish this project. In addition. This group seeks to leam how innovative ways of thinking about health as a system— along with the methodological techniques they inspire—lead to more effective and ethical action. and illustrated how the statements were grouped into clusters." The group generated 318 statements that were synthesized and edited to a final set of 100 statements used during the remainder of the project.

and each term in the cluster name modifies the others. when interacting with Explore Systems Paradigms and Perspectives. In the first case. In systems jargon. including issues of bureaucracy. or change). In many cases.. and evaluation.. The "Use Systems Measures and Models" cluster (10 statements) related to the creation of methods and tools for systems-based measurement and data collection. Examples are training and education. and evaluation. Each cluster resembles a semi-autonomous agent functioning in a highly integrated system.Support Dynamic and Diverse Networks Use Systems Measures and Models Foster Systems Planning and Evaluation Inspire Integrative Learning Show Potential of Systems Approaches Explore Systems Paradigms and Perspectives Expand Cross-Category Funding Utilize System Incentives Instead. they affect each other.g. For example. The label for each cluster of challenges was carefully reverse checked with the cluster statements to ensure that they were adequately represented. When one cluster interacts with another. Thus. clusters can be viewed as simple rules that encourage emergence and adaptation. and development of systems tools and approaches. networks.researchinto what works) are represented. "Utilize Systems Incentives" (14 statements) emphasized the need to address political and sodal factors that influence use of systems thinking. action. American Journal of Public Health | March 2006. systems-based measurement and modeling are used to inform decisionmaking and action. their activity converges on the central cluster. dissemination and diffusion of systems thinking and approaches. use of interactive and Web-based resources. In the second case. the statements suggest 2 challenges to systems thinking in public health: that planning and evaluation are not yet sufficiently systemic and that planning should be continuous and adaptive. the clusters can be thought of as interacting conceptual or thematic agents that can influence other duster agents when coupled. Finally.g. can be considered central both graphically and conceptually. jurisdictional conflicts among organizations. the map can be interpreted as a collection of interacting "cluster agents" each affecting the other. "Inspire Integrative Learning" included the most statements (23). planning. 542 I Research and Practice | Peer Reviewed | Trochim et al. job difficulty. Show Potential of Systems Approaches. reflecting the fact that the topics contained in this cluster were most frequently brainstormed by participants. we offered 2 broad organizing ideas that help to make sense of the often bewildering and diverse landscape of systems thinking: Systems are dynamic and systems are complex. apprehensions about job loss. "Explore Systems Paradigms and Perspectives" included 15 statements addressing values and implications of a potential paradigm change involved in systems thinking and the influence of such thinking on the perspectives people bring to public health work. The cluster in the center of the map (Figure 1). the modifying effect of "systems" on "planning" in the Foster Systems Planning and Evaluation cluster is not merely semantic. where assessment and dissemination (e. use and evaluation of systems methods and models. As the clusters in the exterior ring interact in various ways. the disseminated content in tum excites new activity in the exterior "ring" of dusters. "Foster Systems Planning and Evaluation" (9 statements) suggested the integration of systems thinking and modeling into traditional planning and evaluation. single words in the cluster name represent entire fields of inquiry such as integrative learning. with constant feedback among planning. From the perspective of complex systems. it reflects a perspective that differs dramatically from traditional planning in which planning precedes action and evaluation follows in a rational and linear fashion. learning is more formal and adaptive. and factors in academic environments that limit adoption or use of systems approaches. In terms of dynamics. Vol 96. 3 . learning is centered on systems paradigms and perspectives at a conceptual or epistemological level. No. and a broad emphasis on understanding the area of systems thinking and modeling. DISCUSSION The map developed in this project depicts 8 distinct clusters of practical challenges that need to be addressed to encourage and support effective systems thinking and modeling in public health. it might be thought of as a central attractor in the d)mamic cycle of the overall map. Each is considered briefiy. people's fears (e. the Support Dynamic and Diverse Networks cluster interacts with Expand Cross-Category Funding to create FIGURE 1-Elght-cluster concept map of practical challenges that need to be addressed to encourage and support effective systems thinking and modeling in pubiic heaith work. The 11 statements in the "Show Potential of Systems Approaches" cluster described the value and impact of systems thinking and modeling. In another exeimple. the Inspire Integrative Learning cluster takes on a slightly different meaning thcin when it is considered in connection with Use Systems Measurement and Modeling. The Figure 1 map can be viewed through the lens of systems thinking itself Earlier. From the perspective of dynamic systems.

such as syndemics. In contrast. complex. the clusters can be thought of as combinations interacting in triples or larger groupings..06 Rating diverse and dynamic networks of monetary flows and feedback loops. reducing health care errors. which was closer in nature to a focus group or expert panel than to a sample survey.54 3. The sample consisted of self-selected participants invited from the e-mail lists of 2 prominent initiatives for systems thinking in public hecilth. This study involved seyeral limitations. which holds that simple local rules lead to emergent complexity. researchers. reductionism. With respect to complexity.70 3.25) Rigorous research that demonstrates the value of systems thinking. disease-focused frameworks) Getting government and public health officials at state and federal levels to appreciate the value of community-based approaches and highlight citizenship and local governance in public health Show Potential of Systems Approaches (average: 3.39) Development of methods and tools that encourage systems approaches in research and evaluation Develop new evaluation approaches that will help demonstrate the value of systems approaches. constructivist. No. multidimensional. methods. with just 8 simple clusters. 3. which contains statements relating to research. positivism.. In addition.g.13 4. a diverse and complex number of possible innovative ideas can result. politicians' ignorance of how their systems work.. ecological. needs.35 3.20) Integrate organizational planning and evaluation functions around a systems approach Apply systems thinking to physical and mental health problems affecting individuals. so the generalizability of our results is limited to similar groups. education..g. and communities throughout the human life cycle Develop a unified mission/vision across sectors (e. state. the map might lead us to ask what a dynamic network of best practices in systems thinking and modeling would look like.72 3. and resources Sustain multidisciplinary teams from a broad range of health and science backgrounds and thinking (e.TABLE 1-Participants' Ratings of Practical Chaiienges to Effective Systems Thiniting and iViodeling in Public Heaith Cluster and Challenge Expand Cross-Categoiy Funding (average: 3. behavioral health) and between layers (e. eliminating health and health care disparities) Foster Systems Planning and Evaluation (average: 3. generalizability was not a major purpose of this study. Shown are the 3 challenges in each cluster with the highest average importance ratings. public employee unions that avoid employee accountability.09 4. and nonlinear frameworks) Recognize the limitations of the dominant paradigm in public health (e.54 3. with differing combinations and juxtapositions of clusters suggesting specific novel practices.74 3. systems-oriented organizations and networks will natunilly emerge in the public health system.g. HIV.93 3.76 3.78 3. The especially intriguing hypothesis from a systems point of view is that if multiple independent agents (individuals and organizations) follow the 8 rules represented by the cluster names and are provided with appropriate feedback about what is occurring as a result. and best practices.. public health.56 3. 3 | American Journal of Public Health Trochim et al. innovative notions about supporting diverse networks of researchers and practitioners emerge. the medical model.38) Identify and disseminate examples of "best practices" or "what works" in systems thinking inside and outside of public health A critical mass of practitioners who are able to approach public health from a nonlinear perspective Training and education in systems research techniques for public health professionals Explore Systems Paradigms and Perspectives (average: 3. participatory. Each cluster label is worded as a simple object-action "rule" that can be used by public health professionals to encourage and manage a complex and adaptive system.g.65 3.50) Encourage collaborations between researchers and practitioners by clarifying the link of systems thinking and modeling to eveiyday practice in public health Use participatory action approaches to partner with communities to co-define public health problems. families. logic models. Each label completes the prompt that was the focus of this project and constitutes a practical challenge to be addressed.65 3. civil service systems that encourage stagnation) Note. is combined with Support Dynamic and Diverse Networks. public safety.74 4.86) Identify and develop funding sources that will encourage systems approaches to public health Develop funding for demonstration projects that validate systems approaches to public health Increase funding for transdisciplinary and interagency collaborative projects with a systems focus Support Dynamic and Diverse Networks (average: 3. Increasing in complexity.57 4. research/practice) Use Systems Measures and Models (average: 3. the map can be viewed from the perspective of the theory of complex adaptive systems.g.05) Provide incentives that encourage systems thinking Reduce the overemphasis on immediate positive program impacts by taking a longer term view Address issues of politics and bureaucracy that hinder systems thinking (e. bridging the quality chasm. Vol 96. objectivism. community) regarding the systems approach Utilize System Incentives (average: 3.g. tobacco. national.50 3. and research Set priorities by analyzing system-wide issues rather than simply ranking by disease burden or attributable risk Connect systems thinking and modeling to the series of recent Institute of Medicine reports (e.g... The cluster-as-agent interpretation generates 56 possible pairings.g.93 3. | Peer Reviewed | Research and Practice | 543 .46 3. linear causality. program-focused.72 3.00 3.63 3.26) Recognize the importance of a systems paradigm to public health (e. However. assets. This interpretation is dynamic because. when Show Potential of Systems Approaches. For example.74 3. approaches. systemic. obesity) as possible tipping points for early examples of systems thinking and modeling Inspire Integrative Learning (average: 3. challenges. Brainstorming occurred during late December and March 2006. holistic. this was an unfunded project accomplished under considerable time constraints that may have limited participation. adaptive. deductive/inductive.. in public health Identify priority public health issues (e.

3. 7. 2001. Nurs Res. DC: US Ofiice of Health Information and Health Promotion. Advances in Health Care Organization Theory. San Francisco. Complexity. complexity and wellbeing. BMJ [serial online]. and they offer the promise that more effective public health systems will consequently emerge. The entire project was accomplished within 2 months. Physicians and decisions: a simple rule for increasing connections in hospitals. The results reported here. Cabrera. Should health education abandon attitude change strategies: perspectives from recent research. First. Wyltenbach ME. Leischow contributed to the section on current systems thinking efforts in public health. Glouberman S. BMJ. Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like? Toronto. To help address this limitation. the Netherlands: Elsevier Science Ltd. Complexity Science and Health Care Management Amsterdam. Health Care Manage Rev.25: 109-115. 9. simulations) to enhancing leaming capacity (development of centers and of electronic materials). 1970. Leischow was with the National Cancer Institute. NY. About the Authors William M. The number of clusters and the diversity of their themes now serve as a check against planning and capacity-building initiatives that might otherwise be scoped too narrowly or too abstractly. Vol 96. Ways of understanding. Accessed November 27. DC: Institute of Medicine. Savage G. Gallagher is with Gallagher and Associates. Washington. a map that can be used to navigate them. Cabrera is with the Department of Education. Anderson RA. com/cgi/content/full/323/7315/O/a. who contrihuted to and revised the article. Med Encounter. 5. Available at: http://bmj. Crossing the Quality Chasm: A New Heatth System for the 21st Century. Green LW. to distribute detailed results of this project and encourage further interaction and discussion of these issues and the resulting concept map. Trochim is with Ihe Department of Policy Analysis and Management. McDaniel RR. discussed the research at all phases. Any group or organization can examine the clusters. Cometl University. Zimmerman B. Managing health care organizations: where professionalism meets complexity science. Ashmos DP. researchers.t:24-48. W. McDaniel R. Washington. 2001. Ithaca. D. and the general public face these challenges. Richard S. 2002. 2000. we can now embark on more productive multistakeholder dialogues and think together about precisely what kinds of supports are needed if an authentic system orientation is to thrive in public health agencies. a significant aspect of having done the concept mapping is that it helps to establish an appropriate and vndely shared boundary for thinking about the many issues in question. Health care organizations as complex adaptive systems. Health Care Manage Rev. Begun JW. • study. 14. Equipped with the practical insights from the concept mapping. Trochim originated the study and collaborated with and initiatives such as the Syndemics Prevention Network and ISIS can be viewed from a systems perspective as dynamically interacting components in the growing awareness and support of systems thinking and modeling in public health. Anonymous. major reports such as Crossing the Quality Chasm. NY. 2001. Committee on Assuring the Health of the Public in the 21st Century. leadership. The map constitutes a structure and a set of rules that different agents can use depending on their roles in the public health system. Wilson T. References 1. 52:12-21. This article was accepted September 19. Dooley K. and R. PhD. or the statements contained within them. Trochim. Md. Finally. particularly at a relatively early stage in our efforts to build institutional support for a systems orientation. 11. McDaniel RR. and management in healthcare organisations. Nursing homes as complex adaptive systems. 8. Phillips TS. Derek A. In: Mick SS. 2003. McGinnis M. we created a Web site (available at: http://www. Scott f. 2. Zimmerman B. W. Fottler M.25: 83-92. B. 249 MVR Hall. 10. Many of the limitations of this study would be best ameliorated through independent replications and studies involving the use of alternative methods for identiiying practical challenges to support effective systems thinking and modeling in public health work. The Future ofthe Public's Health in the 21st Century. Human Participant Protection This study was approved by the institutional review board of Cornell University. 2001:323. and detennine the degree to which they constitute or suggest actions they might take to address practical challenges to systems thinking and modeling in pubHc health. policymakers. 2005. the concept map results provide a basis for subsequent action by various actors in public health agencies (and beyond). Gallagher collaborated on the conceptualization and structure of the article. 2000. Such simulations would make it possible for the field of public health to leam more dynamically about the effects of systems thinking and modeling and to cinticipate better where funding resources might be most effectively allocated to encourage systems evolution. Milstein and S. Calif: Jossey-Bass. Because of time and technology constraints. DHHS publication PHS 81 50169. Washington. Cabrera in implementing the 544 I Research and Practice | Peer Reviewed | Trochim et al. Strategies for Promoting Health for Specific Populations.bmjjoumals. This study provides an initial identification of the challenges. Ga. NY 14853 (e-mail: wmtl @comell. Cabrera contributed much ofthe content on systems thinking. Policy Analysis and Management.16:17-19. including everything from potential curriculum topics (participatory methods. nonlinear dynamics. eds. 13. 6. greaterthanthesum. Green LW. 1981. Duchon D. Trochim. Requests for reprints should be sent to William M.A.M.A. and participated in the interpretation of results. American Journal of Public Health I March 2006. Atlanta. 2002. Suchman AL. Anderson RA. 2005. It has practical utility in terms of helping organizational practitioners. eds. At the time ofthe study. Health Educ Monogr. and a set of 8 simple rules for facing these challenges and moving toward effective implementation of systems approaches in public health efforts. the map provides a conceptual model that serves as a basis for developing role play simulations that can enable public health organizations to try out different actions and explore in a controlled context how their adoption could potentially change the outcomes of public health efforts. 2002: 253-288. Ontario. Greenhalgh T. 323:746-749. Plsek PE. 12. Cornell University. 2002. J. Cornell University. 3 . Bethesda. PIsek PE. Institute of Medicine. Issel LM. it was not feasible to engage a broad cross section of the entire participant group in an interpretation of the map results or to engage the broader public health community directly. 2001.early January and conflicted with busy holiday seasons. Driebe DJ. Blair J. Ithaca. The challenge of complexity in health care. Ithaca.323:625-628. McDaniel RRJ. No. DC: National Academy Press. Devereaux M. Montes H. Bobby Mibtein is with the Centers for Disease Control and Prevention. This project is significant for several reasons. Contributors This article was a collaborative effort of all of the authors. the results clearly point to a need for education and leaming in systems thinking and modeling. Second. Linearity. BMJ.S. Third. 4.M. Canada: Commission on the Future of Health Care in Canada.A.

Scientific Advisory Council 2004. for Laws of Self-Organization and Complexity. Goldberger AL.353:1390-1396. Simulation for the Social 28. Gilbert GN. Complexity. New York. Multiple organ dysfunction syndrome: exploring the paradigm of complex nonlinear systems. Herzog A. Pelletier KR. t6. In nature. 2nd ed. NJ: Princeton University Press. 2000. No. 1961. Philadelphia.12:141-157 76. Morgenstem 0. 24:1107-1116. Watdrop MM.//lAM. World Health Organization. Young JF. 1998. Stacey RD. Reading.camagazine. Chaos and the limits of modem medicine. 72. Wolfram S. 1944. New York. New York. Cote M. Tob Control. Develtions. Dissipative and aging: potential applications of fractals and chaos Structures and Spontaneous Social Order. and emerging at Deaconess Bitlings Clinic. 2005. Lindberg C. 48. The history and status of general systems. 1995. 14th ed. Buchman TG. March 2002. Uncoupling biological osciilators: a complementary hypothesis concerning multiple organ dysfunction syndrome. Complexity theory in organization NY: Springer. 1992. Amaral LAN. 27. Griffin D. Let's call it plectics. Life at the edge of chaos—health care applications of complexity science. Mass: Perseus Books. Lipsitz LA. 54. Mass: MIT Press. Germany: KG Saur. 2000. 70. Feedback Thought in Social Science Dallas. Weibel ER. 1990. NY: Scribner.54:5284-5295. 37 Healthy People 2010. Pa: University of 29. 36. Hidden Order: How Adaptation Builds Complexity. Lindberg C. Science. Gell-Mann M. Forrester JW. Heart-rate recovery immediately after exPennsylvania Press. Fottler M. Goldberger AL. Lancet 1996. 1996. 2000. 1972:423. 74. Schmidt G. 1996. June 1994. 3 I American Journal of Public Health Trochim et al. Shannon C. Emergence: From Chaos to Order. ed. Wynder EL. System dynamics modeling: tools for learning in a complex world. Termites. http:// 2003:12:140-147 www. llie role of smoking Theory. 42. Mass: Shambala. In: Zeleny M. 69. Cybernetics. AmJ Physiol. Maturana HR. Lessons From Complexity Science for Health Care Leaders.28:2193-2200. Roadmaps: A guide to leaming system dynamics. Von Bertalanffy L. Cambridge. Emergence: The Connected Lives of Ants. New York. the Netherlands: D.26(3):20-25. NY: Elsevier 45. Mass: Addison-Wesley. Yeomans JM. Goldberger AL. Plsek P. Syme SL. 2003. Balfour JL. A New Kind of Science. NY: Oxford Unipremature beats as a predictor of mortality after acute versity Press tnc. htm. ing? New York.Publishing Co. 38. Crit Care Med. 1980. Washington. 1991 . Holland JH. 75. Conn: Appleton & Lange. 41. 73. 1996. 2000. and Software. Pa: Open University Press. tn: Blair J. England: Oxford University Press Inc.1:5. Francois C. Maturana HR. Welfare. 2001. 25. Calif Manage Rev. Sontag ED. Circulation. www/roadmaps. Shaw P. 2nd ed. CA Magaence with tobacco control programs. Am J Public Health.27:379-423. 2001:30: 234-244. Theory of Games and Economic Behavior Princeton. Physidans and decisions: a simple rule for increasing connections in hospitals. | Peer Reviewed | Research and Practice | 545 .com/index. 34. 1999.102:1239-1244. 56. Cambridge. Concord. Sterman JD. Leaming through system dynamics as preparation for the 21st century. animals that stop and start NY: Oxford University Press Inc. and complexity at the bedside. Autopoiesis: reproduction. Westview Press. 22. Maturana HR. AutopoiesLs and Cognition: The Realization ofthe Living. 2000. 2003. McDaniel R. 43:8-25. ed. Holland J. Fractal variability versus pathologic periodicity: complexity loss and stereotypy in disease. Dordrecht. et al. Tex: VHA Inc.t5. Complexity: The Emerging Science at mittee to the Surgeon General ofthe Public Heatth Service. and Engineering. 1998. 1992. Heart-rate turbulence after ventricular and Selection in Evolution. New York. 1993. zine [serial online]. science: seizing the promise or becoming a fad? Emergence. Nonlinear Dynamics and Chaos: With Applications to Physics. DC: US Dept of Health. Smoking and lung can63.html. Education. Beyond toxicity: human health and the natural environment. Muitifractality in human heartbeat dynamics. Vol. Reading.267:1806-1809. 65. The ecology of work and heatth: research and policy directions for the promotion for employee health. demerging. 39. 1992. Stillman F. Cole C. 2001. ercise as a predictor of mortality. 35. A matter of trust and respect.261: 361-369. ed. Savage G. The Origins of Order: Self-Chganization 32. Complexity and Management: Fad or Radicat Challenge to Systems Think. International Encyclopedia of Systems for informing machine-smoking protocols.//IMA 1997:278:1399-1400. Kauffman SA. Systems Thinking and Dynamic Modeling Conference for K-12 Education. 2001:11-36. Pennisi E. Trochim W. Public Health and Preventive Medicine. Richardson GP. fractals. The origin of disease and health. New York. Chemistry. Reading. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Wakefield MA. 1948. 47. Goldstein J. Advances in Health Care Management. myocardiai infarction./Fam Pract. 1969. 2000. NY: Wiley-Interscience. In: Klir J. 1994. Richardson GP. Bell Syst TechJ. 64. 26. 46:369-376. Gell-Mann M. A mathematical theory of communication. 2000. Brains. Sterman J. Nicolas V. Business Dynamics: Systems Thinking 341:1351-1357. and Systems Theory. Hoffrnann 1. Non-linear dynamics for clinicians: chaos theory. 57. England: Abacus. Ashmos DP. Accessed March 18. and Schuster.6: 18-32. Varela FJ. heredity 33. 23. Goodwin JS. 1998. von Neumann J. 46. McGraw-Hill/Irwin. Crit Care Med 2000. The Quark and the Jaguar: Adventures in the Simple and the Complex. 21. Health Care Manage Rev. Fielding JE. cer: scientific challenges and opportunities. Chaos and fractals in human physiology. Chaloupka FJ. Accessed November 27.23:137-158. 1950-1995. Health Educ Q. Troitzsch KG. West BJ. Reidel Publishing Co. Industrial Dynamics. 1999. win the race. Lancet 1999. Lindberg C. Biology. 43. Schmitt C. Varela FJ. 2001. Report to the and Cybernetics. Merry M. Fractal geometry: a design principle for living organisms. Baskin K. Goldstein J. Cities. 58. Champaign. Johnson S.399:461-465.40:543-561. 1994.25:109-n5. 24. Dardik I.262: 42-49. Kauffman SA. Colo: theory to senescence.288:83-85. Terry YM. Goldberger AL. Stokols D. 68. Greenwich. on Tobacco Regulation. Complexity science and health care management. The changing cigarette. 18. 1994. N EnglJ Med. Paper presented at. 19. et al. Mass: MIT Press. The Tree of Knowledge: The Biological Roots of Human Understanding. heart waves: the single solution to heart rate variability and ischemic preconditioning. Dekker J. Vol 96. Hoffmann D. Fmmkin H. McDaniel RD. Zimmerman B. 55. Low heart rate variability in a 2-minute rhythm strip predicts risk of coronary heart disease and mortality from several causes. Conn: JAt Press.347:1312-1314. McKelvey B. Clark PI. Problems for the future of system dynamics. March 2006. Munich. Autopoiesis. Ivanov PC. Philadelphia. 31. Edgeware: Scientist. Statistical Mechanics of Phase Transi40. 1997. 1998:795. Trends in General Systems 44. Duchon D. 1992. III: Changes at the point-of-sale for tobacco following the Wolfram Media. Seely A. Dean J. the Edge of Order and Chaos. Cancer Res. 1964. Understanding change in primary care practice using complexity theory. 1998:24(l):6-20. 51. 53. 307-364. New York. 49. Forrester JW. et al. Crow RS. Sci Am. 2002. Strogatz SH. NY: Routledge. eds. Available at. 2. and Modeling for a Complex World. NY: 30. NY: Simon & Washington. Rigney DR. London. Smoking and Health: Report ofthe Advisory Com59. 71. 50. Perspect Biol Med 1997. Social determinants of disease. Gotdberger AL. / Toxicol Environ Health. Merging. Mass: Addison-Wesley. In: Wallace RB. Folsom AR. 1991. Oxford. opment of a model of the tobacco industry's interfer61.cfm/ci_id/6798/la_id/l. Miller W.50: 62. Syst Dynamics Rev. Clark P. Nature. Stamford. 52. Boulder. 1999:1:5-32. Hoffmann D. Forrester JW. VI. AmJ Prev Med. Resnick M. Available at: http://web. 1999 tobacco billboard ban. Godin P]. Mass. Loss of 'comptexity' and evolution. Boston. 1981:48-80. 2003:1-23 67. Turtles. Physician Executive. topography in assessing human smoking and its utility 66. 20. 2002. 1997. At Home in the Universe: The Search 1999. DC: US Dept of Health and Human Services. 92:937-939. Djordjevic MJ. Frontier Perspect. Physician Executive. 1995. New York. and Traffic Jams: Explorations in Massively Parallel Microworlds.

Available at. Gleick J. 103. Reliability of concept mapping. 3 . Spotlight on Syndemics. About the Cam79. 2nd ed. Free Inquiry Creative Sodol. Accessed November 24. 2nd ed. 1977. Health Promotion Pract. Yardely. Tex. 108. Intemational Healthy Cities Eoundation. Vol 96. Chicago. University of Chicago Press. 83. NY. 1966. England: Elamingo. Generations of suffering: experiences of a treatment program for substance abuse during pregnancy. On being autonomous.healthycities. 1992. Singer M. 1997. Predictability: does the flap of a butterfly's wings in Brazil set off a tornado in Texas? Paper presented at: Annnal Meeting of the American Association for the Advancement of Science. Soc Sd Med. 2000. 107.//www. Available at. Calif: Paramount. Syst Dynamics Rev. Soc Sd Med. 81. Accidental Death and Disability: The Neglected Disease of Modem Society. 1997. San Ecology Soc [serial online]. 95. New York. Appleby N. 2005. Richmond B. New York. Calif. ed. Plenum Press. In: Encyclopedia of Evaluation. Batterham R. NY: Cornell University. arXiv. 115. Conn: Praeger. 404-405. Trochim W An introduction to concept mapping for planning and evaluation.3:222-234. Annual Conference of the American Evaluation Association. NY: Hyperion. Methodology. conceptualizing the SAVA syndemic. Syndemics and public health. System Intervention: Philosophy.oi^/Pnbs/ YouthED/Pub591. Emeryville. Depla M. 78. Chaos: Making a New Science. ed.10(2):6. 2000. Kniskal JB. 123. 1969. Available at: http://www. Report 924156198X.99-110. Enablers. Systems thinking. 2003. 117. Trochim W Concept mapping: soft science or hard art? Eval Program Plann. Medical Anthropology and the World System. Applied Systems Research. Institute for Healthcare Improvement. Mill Valley Calif: Mill Valley School District. NJ: Princeton University Press. ecologyandsociety. Jackson S. 92. NY: Viking. 94.24. 1994. Community-Campus Partnerships for Health (CCPH). 2004. Accessed September 15. Patterns of knowledge: knowledge as a complex. 122. 546 I Research and Practice i Peer Reviewed i Trochim et al. 2005. Institute of Medicine. Princeton. Creating partnerships. Newbury Park. Milstein B. Emirbayer M. 127. 2002. Calif: Sage Publications. 97 Von Bertalanfly determinants/en. 1978. WK Kellogg Eoundation. Singer M.Everitt B. Lorenz E. 1999. Watts DJ. National Academy of Sciences.who. Brandon. About us. Syndemics. 90. 1996. 2nd ed. November 3-6. Development. Westport. Accessed November 24. Amf Sodol 1997. evolutionary system. NY: Braziller. Commission on Social Determinants of Health. Bacterial burritos. The World Health Report 2000: Health Systems: Improving Performance. Pa: Staywell. Dallas. New York. NY: Wiley-Interscience. Seattle. 2003. 110. In: Klir G. Davidz HL. Switzerland: World Health Organization. 112. 105. Available at: http://www. Six Degrees: The Science ofa Connected Age. 2004:5:8-19. Boston.pdf. Behind the Wizard's Curtain: An Integration Environment for a System of Systems. critical thinking skills for the 1990s and beyond. No. World Health 10/iss2/art6. 104. / Health Care Poor Underserved. Small Worlds: The Dynamics of Networks Between Order and Randomness. 2003. Paper presented at. Accessed December 19. What is system dynamics? Available at: http://www. Singer M. Cabrera DA. 2002. 2005. Vt: Solomon Press. Partnership for the Public's Health. Gladwell M. Available at: http://www. Wash: University of Washington. NY: Doubleday. Social Dimensions of Life. NY.12:1-16. 113. NY: Academic Press Inc. Geneva. The Tuming Point: A Transformative Vision for an Ecological Age [audiotape]. Mindwalk [videotape]. Valera EJ. Severens P. Atlanta. 2003:1. Med Anthropol Q. Ga: Centers for Disease Control and Prevention. New York. and Practice. Hollywood.77. Manifesto for a relational sociology. Rhodes DH. 2004.//www. 1999. The History and Status of General Systems Theory. a case of AIDS. Cognitive. Multidimensional Scaling. 2004. New Dimensions Eoundation. Applications. Von Bertalanfly L. 1972. Midgley G. 2003. December 1979. 1992. In: Miller R. 125. global country partners address social determinants of health. NY: Kluwer Academic. 85. Milstein B. the role of communitybased participatory research. 86.ihi. Southem D. Trochim W. The structure and function of complex networks. Susser 1. 2001. University of Washington. 98. Capra F. 2005. 1993. Malsch M.54:1225-1241. New York. Beverly Hills. 1999. 2004. CPft/AEDfor the Professional Rescuer. Watts DJ. General System Theory: Founda2002. Mass. 93. Available at: http://www. Davison ML. 109. Beliefs on coping with illness: a consumer's perspective. Eval Program Plann. 129. Cluster Analysis. March 25. a powerful tool for improving the well-being of families and neighborhoods. Barriers. 124. DeRidder D. What is the Healthy Cities movement? Available at: http:// www. Newman MEJ.cfm?id=52. Washington. Available at:. Tuming Point National Program Office. Clair S. Pressler V. Krygiel news_story. 82. 111. a touch of violence. http. London. 1989. 2000. Washington. 100. 99. 91. Watts DJ. 2005. mixed metaphors and a man in the middle: a scientific life at the crossroads of biology. Integration and implementation sciences: building a new specialization. New York. and Precursors to Systems Thinking Development: The Urgent Need for More Information. NY: John Wiley & Sons Inc. New York. Accessed November 24. US Dept of Health and Human Services. 2002. Strogatz SH.htm. Accessed November 24. 84.17. 1987 89. Calif: Enhanced Audio Systems. reconceptualizing disease in bio-social context. 2005. 2002. 120. New Paradigm Thinking [audiotape]. Accessed October 5. Halsted Press. Available at. 87. Bammer G. American Journal of Public Health | March 2006. Cluster Analysis for Applications. New York. Trochim W. Baer HA. the lessons 101. 2003. 121. 1973. The Tipping Point: How Little Things Can Make a Big Difference. 80. DC. The Structure and Dynamics of SmallWorld Systems [dissertation]. 1989:12:87-110. and bridge. Accessed March 18. an educational imperative. New York. Lewin R. of natural history for systems theory. Anderberg MR. 114. tions. 106. MIT Press. Singer M. 102. Calif: Sage Publications.html. 130. 1993. Multidimensional Scaling. A dose of drugs. ahrq.9:113-134.77-85. Erom knowledge to action. 1997. New York. DC: Institute for National Strategic Studies. Partnerships. Cabrera DA. Wood B. Wish coll/about/overview. http. from Silos to Systems: Using Performance Management to Improve the Public Health. Accessed April 20. Capra E The Hidden Connections: Integrating the Hidden Connections Among the Biological.wkkf. 118.partnershipph. 96. Capra E The Web of Life: A New Synthesis of Mind and Matter. 1993. 1980. et al. Setting objectives for community and systems change: an application of concept mapping for planning a statewide health improvement initiative. American Red Cross. Mass: Little Brown & Co. 128. http://www. Nightingale DJ. III.genomics. 119. Accessed November 24. Snipes C. 2005. 2005.comell. 126. New York: WW Norton & Co.423-441. From the Parts to the Whole: System's Thinking in Ecology and Education.systemdynamics. Complexity: Life at the Edge of Chaos. chemistry and engineering. System Dynamics Society. Creating Leaming Communities [online edition]. Sync: The Emerging Science of Spontaneous Order. 1991. DC. improving health. Washington.44:553-559. Construction of a GP integration model. 88.103:281-317. 116.

Sign up to vote on this title
UsefulNot useful