2016 Book InformationModelsAndSustainabi

Public Administration and Information Technology 20
Jing Zhang
Luis Felipe Luna-Reyes
Theresa A. Pardo
Djoko Sigit Sayogo Editors
Information,
Models, and
Sustainability
Policy Informatics in the Age of Big Data
and Open Government
Public Administration
and Information Technology
Volume 20
Series Editor
Christopher G. Reddick, San Antonio, USA
More information about this series at http://www.springer.com/series/10796

Jing Zhang • Luis Felipe Luna-Reyes
Theresa A. Pardo • Djoko S. Sayogo
Editors
Information, Models,
and Sustainability
Policy Informatics in the Age of Big Data
and Open Government
Editors
Jing Zhang Luis Felipe Luna-Reyes
Clark University, Worcester, MA, USA University at Albany, Albany, NY, USA
Theresa A. Pardo Djoko S. Sayogo

University at Albany, Albany, NY, USA University at Albany, Albany, NY, USA
Public Administration and Information Technology

ISBN 978-3-319-25437-1 ISBN 978-3-319-25439-5 (eBook)
DOI 10.1007/978-3-319-25439-5
Library of Congress Control Number: 2015957953
Springer Cham Heidelberg New York Dordrecht London

© Springer International Publishing Switzerland 2016
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, express or implied, with respect to the material contained herein or for any errors
or omissions that may have been made.
Printed on acid-free paper
Springer International Publishing AG Switzerland is part of Springer Science+Business Media

(www.springer.com)
Acknowledgements
We gratefully acknowledge financial support from the National Science Foundation

(Grant # 0955935) and the Mexican Consejo Nacional de Ciencia y Tecnología
(National Council on Science and Technology, CONACYT, Grant #s84082, 117118
and 133670), through which we have been able to collaborate for the last 4 years on
I-Choose, a multi-institutional, multinational research project focused on the
critical issue of full-information product pricing.
This book is one of the products of the I-Choose project. We wish to thank all the
members of the I-Choose team for their unwavering collegiality and enthusiasm for
all things on I-Choose. Thank you to David Andersen, Deborah Andersen, Francois
Duhamel, Isis Gutierrez-Martinez, Jana Hrdinova, Holly Jarman, Joanne Luciano,
Sergio Picazo-Vela, Giri Tayi, Xing Tan, and Andrew Whitmore for working with
us and each other to collaboratively advance the I-Choose research agenda in such
a rewarding and enjoyable way. We are especially indebted to Jana Hrdinova, who
in addition to becoming a subject matter expert, provided tireless support for the
project, organized face-to-face meetings, and coordinated the online communica-
tions of the team, among many other critical tasks. Special thanks go to David
Andersen who provides the vision and intellectual leadership and helped the team
remain focused over the course of these 4 years, and still. This project has received
valuable support from a number of students, Weijia Ran, Nicde Paula, Haixin Liu,
and James Michaelis. We are grateful to them for their assistance.
We also wish to acknowledge the support from our institutions, the Graduate
School of Management and George Perkins Marsh Institute at Clark University, the
Universidad de Las Americas-Puebla, and the Center for Technology in Government,
Rockefeller College, and the College of Engineering and Applied Sciences at the
University at Albany, State University of New York. The in-kind and financial sup-
port from these institutions made it possible for us to build on the work of the
I-Choose team and to continue our productive collaboration.
v
Contents
Information, Policy, and Sustainability: The Role

of Information Technology in the Age of Big
Data and Open Government .......................................................................... 1
Jing Zhang, Luis F. Luna-Reyes, and Theresa A. Pardo
“Reading the Minds” for Quantitative Sustainability:
Assessing Stakeholder Mental Models via Probabilistic
Text Analysis .................................................................................................... 21
Matteo Convertino, Rafael Munoz-Carpena, and Carolina Murcia
Environmental Performance or Productivity Loss? .................................... 39
Shital Sharma
Supply-Chain Transparency and Governance Systems:
Market Penetration of the I-Choose System................................................. 67
Weijia Ran, Holly Jarman, Luis F. Luna-Reyes, Jing Zhang,
Deborah Andersen, Giri Tayi, Djoko S. Sayogo,
Joanne Luciano, Theresa A. Pardo, and David Andersen
Green Government Procurement: Decision-Making
with Rough Set, TOPSIS, and VIKOR Methodologies ............................... 93
Chunguang Bai and Joseph Sarkis
Enhancing the Usability of Climate Information
and Models Through Stakeholder Engagement........................................... 121
Elizabeth Allen and Jennie C. Stephens
Do Open Data Initiatives Promote and Sustain
Transparency? A Comparative Analysis
of Open Budget Portals in Developing Countries ........................................ 137
Jyldyz Kasymova, Marco Aurelio Marques Ferreira,
and Suzanne J. Piotrowski
vii
viii Contents
On Sustaining Sustainability: The Case of Implementing

Decisions Based on Policies for a Sustainable
Future via Tablets in a Board of a Swedish
Housing Corporation ...................................................................................... 157
Jenny Eriksson Lundström and Mats Edenius
About the Editors
Luis Felipe Luna-Reyes is an Associate Professor of Informatics at the University

at Albany, NY. He holds a Ph.D. in Information Science from the University at
Albany, and he is also a member of the Mexican National Research System. His
research focuses on electronic government and on modeling collaboration processes
in the development of information technologies across functional and organiza-
tional boundaries. His research interests are related to areas such as interorganiza-
tional collaboration, information sharing, success of government-wide Web sites,
and information policy to promote economic exchange in the NAFTA region. He is
the author or co-author of articles published in Government Information Quarterly,
European Journal of Information Systems, Information Polity, Gestión y Política
Pública, and System Dynamics Review, among others.
Theresa A. Pardo is the Director of the Center for Technology in Government,

University at Albany, State University of New York (SUNY) and a Research
Associate Professor at the Rockefeller College of Public Affairs and Policy and the
College of Computing and Information. Dr. Pardo serves as Open NY Policy
Adviser to New York State’s Governor Andrew Cuomo, as a member of the National
Advisory Committee for the U.S. Environmental Protection Agency and as President
of the Digital Government Society. Her work has been funded by the U.S. National
Science Foundation (NSF), the Social Science and Humanities Research Council of
Canada (SSHRC), the Library of Congress, the World Bank, the United Nations,
Microsoft, SAP, and numerous governments at all levels, among others.
Jing Zhang is an Associate Professor of Management at Clark University. Her

research interests focus on interorganizational information and knowledge sharing,
organizational impact of technology and innovation, and the role played by infor-
mation technology to support sustainable consumption and sustainable supply
chain. Her work is supported by National Science Foundation and Mosakowski
ix
x About the Editors
Institute of Public Enterprises. Her publication appears in European Journal of

Information Systems, Government Information Quarterly, Public Performance and
Management Review, Information Technology and Management, among others.
Professor Zhang holds a Ph.D. in Information Science from the University at
Albany, State University of New York.
About the Authors
Elizabeth Allen is a Ph.D. student in the School of the Environment at Washington

State University. Her research interests include climate change communication,
public engagement in science, the relationships between scientific research and
policy decision-making, and environmental modeling.
Deborah Lines Andersen is Associate Professor of Information Studies and

Informatics at the University at Albany. Her research centers on library policy, espe-
cially public libraries internationally, and on information transfer across supply
chains. She teaches in the areas of library science, information policy, and research
design and statistics.
David F. Andersen is Distinguished Service Professor of Public Administration,

Public Policy, and Information Science at the Rockefeller College, University at
Albany. His work centers on applying system dynamics, systems thinking, and
information technology approaches to problems in the public, not-for-profit, and
private sectors, especially using group modeling approaches. He has served as a
technical consultant to public and not-for-profit agencies in the federal, state, and
local sectors as well as corporate clients in North America and Europe. Most
recently, he has been working on information standards for bringing sustainable,
fair labor, and environmental-friendly products to market in the NAFTA region as
part of his Carlos Rico Fulbright Award for 2010–2011.
Matteo Convertino, Ph.D., P.Eng. is Assistant Professor in the School of Public

Health−Division of Environmental Health Sciences at the University of Minnesota-
Twin Cities. Dr. Convertino got his B.Sc., M.Sc., and Ph.D. in Civil and
Environmental Engineering from the University of Padova, Italy. Dr. Convertino is
the PI of HumNat. The transdisciplinary mission of HumNat is to develop Predictive
Complex Systems Multiscale Models as Design Technology (Physical-based
Theoretical and Computational Models) for Biomedical, Biological, Socio-
Behavioral, Environmental, and Clinical Research for population health. Current
focus is on “One Health” and intelligent systems for the diagnosis, etiognosis, and
xi
xii About the Authors
prognosis of diseases and complex systems in general. Dr. Convertino is also adjunct
professor in the Department of Systems Engineering and member of the Center for
Systemic Risk Analysis focused on animal, human, and ecosystem health hosted by
the College of Veterinary Medicine. Dr. Convertino is a resident fellow of the
Institute on the Environment, Institute for Engineering in Medicine, and of the
Institute for Advanced Study at the University of Minnesota. Dr. Convertino also
holds an adjunct professor of Biodiversity Conservation and Ecosystem Health at
the Institute of Forest Ecology, Environment and Protection, Chinese Academy of
Forestry. His work is focused on the analysis, modeling, and management of com-
plex natural–human systems for designing population health. Web site: http://www.
tc.umn.edu/~matteoc/Welcome.html
Marco Aurelio M. Ferreira is Coordinator of the Graduate Program at Public

Administration in the Federal University of Vicosa (UFV)—Brazil. Hi received his
Ph.D. in Applied Economics (2005) at UFV—Brazil. He is a member of the
Brazilian Academy of Administration (ANPAD), Brazilian Society for Public
Administration (SBAP), Public Management Research Association (PMRA/USA),
and American Society for Public Administration (ASPA/USA). He is also a research
member of the National Council of Science and Technology of Brazil (CNPq) and
a research fellow at the Institute of Public Policy and Sustainable Development
(IPPDS/Brazil). From 2010 to 2011, he was a Visiting Professor at the School of
Public Affairs and Administration—SPAA/Rutgers University (USA). His research
interests lie in the broad area of public administration, with a particular focus on
public policies, transparency and open government, performance in public sector,
and social programs and federalism.
Holly Jarman is a political scientist who studies the effects of market regulation,
particularly crossborder regulation and international trade and investment agree-
ments on health and social policies. Within this broad topic, her publications address
questions related to the relationship between tobacco control regulation and the
world trading system, the regulation of crossborder health markets in the EU, cross-
border collaborations to promote innovative public health and environmental regu-
lation, and the impact of crossborder patient mobility on global health governance.
She received her Ph.D. in Political Science from the London School of Economics
and Political Science (LSE) for comparative research into the use of non-trade
issues, including environmental protections and labor standards, as weapons and
incentives in EU and US trade policy.
Joanne Luciano is the President and Founder of Predictive Medicine, Inc. a

Boston-based research and development consultancy, Visiting Research Associate
at the University of California at Irvine, Visiting Research Scientist, Rensselaer
Polytechnic Institute. She holds a Ph.D. in Cognitive and Neural Systems from
Boston University and a Bachelors and Masters in Computer Science. Luciano is a
pioneer in the use of computational methods in psychiatry and in the application of
semantic technologies. As an internationally recognized multidisciplinary scientist,
About the Authors xiii
her research focuses on computational enabling technologies and the reusability of

research artifacts. Dr. Luciano has played a leading role in several consortia, initia-
tives, and standards enabling eScience and holds two patents. She founded the field
of Health Web Science, a subfield of Web Science in her book Health Web Science
(2014) that studies the role of the Web in all things related to health. Luciano’s work
has appeared in several leading peer-reviewed journals.
Luis Felipe Luna-Reyes is a professor of business at the Universidad de las

Américas Puebla in México. He holds a Ph.D. in information science from the
University at Albany. Luna-Reyes is also a member of the Mexican National
Research System. His research focuses on electronic government and on modeling
collaboration processes in the development of information technologies across
functional and organizational boundaries.
Rafael Munoz-Carpena, Ph.D. is a Professor in the Agricultural and Biological

Engineering Department at the University of Florida, Gainesville. He has been
Associate and Assistant Professor in the same department since 2000. Before that
he was Tenured Researcher at the Canary Islands Agricultural Research Institute,
Spain, and Adjunct Professor at the University of La Laguna, Spain. Dr. Munoz-
Carpena got his Ph.D. in Biological and Agricultural Engineering from North
Carolina State University. Munoz-Carpena’s program at the Department of
Agricultural and Biological Engineering is to conduct Research and Teaching that
focus in surface/groundwater hydrology and water quality. Research emphasis is on
hydrologic processes and includes water management and hydrologic field and
modeling studies of contaminant transport, transformation and fate processes, and
of water-related agricultural production practices, which will improve the compati-
bility of agriculture with the surrounding natural ecosystem and quickly expanding
urban population in Florida and elsewhere. More recently, he is combining hydro-
logical modeling to large-scale environmental modeling coupled to global sensitiv-
ity and uncertainty analyses to analyze complex systems and predict future
trajectories over space and time. Munoz-Carpena’s projects about shorebird vulner-
ability to sea-level rise and the Amazon tropical forest resilience, respectively,
funded by SERDP and NSF are worth mentioning.
Carolina Murcia is currently the Science Director for the Organization for
Tropical Studies at Duke University. Dr. Murcia is also Courtesy Assistant Professor
in the Department of Biology at the University of Florida, Gainesville. Dr. Murcia
is also a Consultant at the Center for International Forestry Research, Bogor,
Indonesia. Dr. Murcia received degrees in Biology and Zoology from the Universidad
del Valle, Cali, Colombia, and the University of Florida, Gainesville, Florida, USA..
Dr. Murcia has extensive research in the field of tropical ecology where she investi-
gated natural and human-generated dynamics of tropical forests. Dr. Murcia has a
deep interest in translating science into actions via strong collaborations with stake-
holders involved in the science and management of ecosystems. Dr. Murcia is
xiv About the Authors
currently involved in Costa Rica tropical rainforest and wetland ecosystem works.
Web site: http://www.ots.ac.cr
Theresa A. Pardo is the Director of the Center for Technology in Government,

University at Albany, State University of New York (SUNY) and a Research
Associate Professor at the Rockefeller College of Public Affairs and Policy and the
College of Computing and Information. Dr. Pardo serves as Open NY Policy
Adviser to New York State’s Governor Andrew Cuomo, as a member of the National
Advisory Committee for the U.S. Environmental Protection Agency and as President
of the Digital Government Society. Her work has been funded by the U.S. National
Science Foundation (NSF), the Social Science and Humanities Research Council of
Canada (SSHRC), the Library of Congress, the World Bank, the United Nations,
Microsoft, SAP, and numerous governments at all levels, among others.
Suzanne J. Piotrowski is an Associate Professor of Public Affairs and

Administration at Rutgers University-Newark. Professor Piotrowski is the chair of
the American Society for Public Administration’s Ethics section. She founded and
moderates the International Transparency and Secrecy Research Network listserv.
She currently serves as the independent assessor of the US national action plan for
the Open Government Partnership. Dr. Piotrowski’s research focuses on nonmission-
based values in public administration, including administrative transparency and
ethics. She authored the book Governmental Transparency in the Path of
Administrative Reform, the State of University of New York Press (2007). In 2010,
Lexington Books published Piotrowski’s latest volume Governmental Transparency
and Secrecy: Linking Literature and Contemporary Debate. Professor Piotrowski
has written widely on public management, accountability, and transparency issues
including over 30 book chapters, journal articles, case studies, and encyclopedia
articles.
Weijia Ran is a Ph.D. candidate in Informatics at the University at Albany. Her

research interests include sociotechnical systems, analytic-based decision making,
modeling and simulation, and sustainability.
Djoko Sigit Sayogo is senior lecturer in the Department of Economics and

Business at the University of Muhammadiyah Malang (UMM), Indonesia.
Additionally, Djoko is the Director at the Center for Economics, Business and
Entrepreneurship Development at the Department of Economics, University of
Muhammadiyah Malang. He is currently a Postdoctoral Associate at the Center for
Technology in Government. He is currently researching the critical factors affecting
the creation and implementation of a food traceability system that would support the
goals and aims of food safety and economic development policies. He was working
on the CTG team that aims to develop a proof of concept for global information
systems using Semantic Web technologies based on OWL ontology, called I-Choose.
He is a former Fulbright Presidential Scholar from Indonesia.
About the Authors xv
Jennie C. Stephens is an Associate Professor and the Blittersdorf Professor of

Sustainability Science and Policy at the University of Vermont. She has a joint
appointment with the Rubenstein School of Environment and Natural Resources
and the College of Engineering and Mathematical Sciences. Professor Stephens’
research, teaching, and community engagement focuses on sociopolitical aspects
of energy technology innovation, electricity system change, climate change com-
munication, and facilitating social learning in the transition from fossil-fuel to
renewables-based energy systems. She has contributed to understanding the social
dynamics of wind power, carbon capture and storage, and smart grid, and brings
experience in stakeholder engagement and communication among experts, practi-
tioners, academics, and the public. Professor Stephens was previously on the fac-
ulty at Clark University (2005–2014), and she did postdoctoral research at Harvard’s
Kennedy School (2002–2005). She earned her Ph.D. (2002) at Caltech in
Environmental Science and Engineering and her B.A. (1997) at Harvard in
Environmental Science and Public Policy.
Giri Kumar Tayi is a Professor of Management Science and Information Systems

at the State University of New York at Albany. He obtained his Ph.D. from Carnegie
Mellon University, and his research and teaching interests are interdisciplinary and
span the fields of Information Systems, Operations Management, and Operations
Research. His current research streams include Modeling of Supply Chains (Reverse
Logistics, Product Recovery and Return Architectures), E-Commerce and
Marketing, Information Sharing in Supply Chains, Economics of Information
Systems (Information Security, Software Sample Design, Pricing of Information
Goods), Modeling of Online Communities (Creative Commons, Digital Copyright
Policies), Geographically Distributed Software Development (Process Choice,
Software Quality), IS Project Outsourcing (Risk Management, Engagement
Models), Data Quality and Data Mining, Models and Algorithms for Mobile
Computing Environments, Online Auctions, and Grid Computing.
Jing Zhang is an Associate Professor of Management at Clark University. Her

research interests focus on interorganizational information and knowledge sharing,
organizational impact of technology and innovation, and the role played by infor-
mation technology to support sustainable consumption and sustainable supply
chain. Her work is supported by National Science Foundation and Mosakowski
Institute of Public Enterprises. Professor Zhang holds a Ph.D. in Information
Science from the University at Albany, State University of New York.
Information, Policy, and Sustainability:
The Role of Information Technology
in the Age of Big Data and Open Government
Jing Zhang, Luis F. Luna-Reyes, and Theresa A. Pardo
Abstract Sustainability has become an important focus for government, civil soci-
ety, and the corporate community worldwide (United Nations Environment
Programme 2011). Growing interest in addressing environmental deterioration and
associated social inequality and economic challenges is shifting focus to this impor-
tant issue. The lack of fresh water and arable land, extreme weather, rising cost of
relying on fossil fuels, and poverty and regional instability are drawing attention to
the need for government intervention and policy instruments that encourage the
development of sustainable alternatives.
Introduction
Sustainability has become an important focus for government, civil society, and the
corporate community worldwide (United Nations Environment Programme 2011).
Growing interest in addressing environmental deterioration and associated social
inequality and economic challenges is shifting focus to this important issue. The
lack of fresh water and arable land, extreme weather, rising cost of relying on fossil
fuels, and poverty and regional instability are drawing attention to the need for gov-
ernment intervention and policy instruments that encourage the development of sus-
tainable alternatives.
This growing attention on sustainability is directly reflected in the United Nation’s
Millennium Development Goals (UN General Assembly 2000). Set in year 2000 by
world leaders, the MDGs constitute a series of targets with a deadline of 2015, rang-
ing from eradicating extreme poverty, achieving universal primary education, to
ensuring environmental sustainability. On environment protection, the development
J. Zhang (*)
Clark University, 950 Main St, Worcester, MA 01610, USA
e-mail: jizhang@clarku.edu
L.F. Luna-Reyes • T.A. Pardo
University at Albany, Albany, NY, USA
e-mail: lluna-reyes@albany.edu; tpardo@ctg.albany.edu
© Springer International Publishing Switzerland 2016 1

J. Zhang et al. (eds.), Information, Models, and Sustainability,
Public Administration and Information Technology 20,
DOI 10.1007/978-3-319-25439-5_1
2 J. Zhang et al.
goals particularly committed to the adoption of a new ethics of conservation and

stewardship in all environmental actions, with the purpose to counter the threats of
irredeemable environmental damage by human activities and to curb the unsustain-
able exploitation of nature resources. These goals direct attention to areas such as
emissions reduction in greenhouse gases, conservation and sustainable development
of forests, preservation of biological diversity, providing adequate and equitable
access to water resources and reduction of natural and manmade disasters.
Sustainability can be defined broadly or narrowly, and the definition is context
dependent, strongly influenced by the perspectives or domains in which it is applied
(Brown et al. 1987). Initially emerged in The Ecologist’s (1972) A Blueprint for
Survival, sustainability and sustainable development have been defined in the per-
spective of Biology, Economy, Sociology, Urban Planning, and Environmental eth-
ics (Basiago 1995). Although the definitions vary because of differences in
disciplinary focus and methods, a commonly accepted definition from the Report of
the World Commission on Environment and Development refers to sustainable
development as “development that meets the needs of the present without compro-
mising the ability of future generations to meet their own needs” (Brundtland 1987).
Governments can play very important roles in facilitating sustainable development
through better public policies (Dovers 2005). Public investments can be directed
toward establishing incentives for renewable energy, energy efficiency, sustainable
agriculture, and land and water conservation, or toward leveling the field for sustain-
able alternatives by phasing out the subsidies directed to unsustainable production and
development. In addition, policies can be made to encourage private investment and
public–private collaboration in innovation and technological advancement. Moreover,
policy incentives for innovations and education development enable nations and
regions to lead and capitalize on new technological and economic opportunities.
More importantly, government has been using regulatory and pricing mecha-
nisms to direct resources to support environmental sustainability and solving envi-
ronmental problems. For example, two types of policy instruments have been used
in curbing environmental pollutants. One, characterized as a command-and-control
approach, intends to bring down the level of emission through setting emission per-
formance standards. This standard or technology forcing approach is effective in
certain contexts, but has been criticized for high levels of cost, inflexibility, and
performance uncertainty (Stavins 2007). The effectiveness of such instruments
often relies on multiple factors such as the feasibility of targets and an agency’s
credibility to enforce standards (Gerald and Lave 2005). The second one is a market
approach that targets emissions through adding a price to the environmental exter-
nalities. This can be in the form of emission tax or a cap-and-trade system (Jaffe
et al. 2005). In recent years, cap-and-trade systems have received growing support
and are being strongly advocated as the better approach and the one more likely to
be adopted in the United States (Stavins 2007). One key argument for this approach
is that the costs of achieving emission reduction to those affected can be lower as the
distribution of the reduction tends to gravitate toward the sources where emission is
least costly to reduce, while giving firms incentives to reduce emissions (Stavins
2007). This type of market approach is also more flexible given that the firms have
options to pay for the allowances if targeting toward emission reduction is too costly
or difficult for the short term.
Information, Policy, and Sustainability… 3
A similar role that can be played by government is to use information as a policy

instrument for the development for sustainable consumption (Huq and Wheeler 1993;
Robinson et al. 2009). Traditional production models dominate because externalities
detrimental to social and environmental sustainability are largely unaccounted for by
existing pricing mechanisms. Government policy in regulating prices, certification, or
information can send correction signals to the market and play key roles in establish-
ing a favorable condition for sustainable products (Sayogo et al. 2014).
Delivering on the many promises of government interventions grounded in or
enabled by such policies hinge, to a great extent, on the quality of those policies.
Such policy decisions related to investments in sustainable development are situated
in complex environments and involve large numbers of heterogeneous stakeholders
(Helbig et al. 2012b). Understanding enabling conditions and trade-offs between
long-term benefits and short-term impacts, or between environmental sustainability
and economic progress is challenging, in large part due to historical gaps between
how and when information and information technology became a part of the policy
informing, development, and evaluation process.
These gaps, are however, according to Dawes and Janssen (2013), closing. The
availability of large quantities of data, growth in computing power, and advanced
analysis and presentation tools are changing the relationship between information
and information technology and the policy-making processes. This change is giving
rise to the field of policy informatics, which involves groups of policy makers and
other stakeholders in using models, data, and other technical tools to analyze prob-
lems and policy alternatives. Policy informatics is especially suitable to the domain
of sustainability: understanding the interactions of job growth and investing in envi-
ronmental sustainability under certain regulatory framework, projecting the effec-
tiveness of sustainable agricultural regime, monitoring climate change, and
understanding the impact of emerging technologies on capability to understand and
pursue policy priorities, among other complex issues.
Concomitant with these new opportunities are new challenges, such as engaging
stakeholders, interpretation of results, understanding the limitation of models and
analytical tools (Dawes and Janssen 2013). The sections below outline some of the
latest thinking on the building blocks, or pillars, of policy informatics and the chap-
ters themselves provide the reader with new insights into their use. The book reflects
on emerging trends, developments, and challenges of sustainability policies and
information technology and puts forward new and valuable insights to both research
and practice communities.
Policy Informatics
Applying mathematical models and empirical data to solve complex problems is not
new. In fact, using quantitative models to support the public policy-making process
promoted the development of the area of Policy Modeling in the 1970s (Ruiz
Estrada and Yap 2013). This traditional view of policy modeling emphasizes under-
standing causes and effects of policy choices and has been dominated in the last 40
years by an economics orientation (Ruiz Estrada 2011). New approaches to policy
4 J. Zhang et al.
modeling, such as policy informatics, made possible through new and emerging
technologies, and the concomitant innovations in data capture, management, and
use have enabled a move from this traditional perspective into a broader understand-
ing of causes and effects of policy problems using a broad range of modeling and
analytical techniques (Puron-Cid et al. 2012, 2014). Further, advances in modeling
and visualization techniques have made it possible to gain new insights into the
importance of including groups of policy makers and other stakeholders in using
models, data, and other technical tools to analyze problems and policy alternatives.
Throughout the last decade, we have witnessed a renewed interest in policy model-
ing (or better, policy informatics) with a broader approach (Barrett et al. 2011;
Johnston and Kim 2011; Sonntagbauer et al. 2014; Dawes et al. 2014). Further,
beyond recognition of the importance of modeling and empirical data, contempo-
rary policy informatics promotes stakeholder involvement as well as interdisciplin-
ary approaches to policy analysis (Cockerill et al. 2009; Eden et al. 2009; Klievink
and Janssen 2010; Ackermann et al. 2011).
This renewed interest in policy informatics has been promoted, at least partially,
by two independent movements: open government and open data, and the develop-
ment of technologies that enable the use of traditional, and the creation of novel,
analytical techniques (Puron-Cid et al. 2012, 2014). New imperatives for more open
governments, including increased transparency, accountability, and engagement,
are driving investments in initiatives designed to make government data increas-
ingly “open.” To fully leverage these investments in open data, governments and
other stakeholders are turning to policy informatics as the means to gain expected
public value from the newly available data. In this sense, policy informatics relies
on four main pillars: analytical methods, data, technology tools that facilitate the
use of methods, and the engagement of stakeholders.
The increasing variety of modeling techniques, analytical tools, and data resources
creates both new opportunity for those engaged or interested in policy informatics and
new complexity. Designing a particular policy modeling and analysis effort requires a
deep level of understanding of the policy question, the available data resources, and
the tools and techniques available to both gain new insight and to ensure that new
insight is consumable to stakeholders and policy makers as part of the policy-making
process. The sections below and the chapters that follow are designed to reduce this
complexity by helping to fill the gaps in what academics and practitioners know about
policy informatics. The sections on analytical methods and techniques, data and open
data, information technologies, and stakeholder involvement outline some of the latest
thinking about the building blocks, or pillars, of policy informatics and the chapters
themselves provide the reader with new insights into their use.
Analytical Methods and Techniques
A model, in general, is a conceptual representation of a problem, and it helps policy

makers and other stakeholders structure the inquiry process (Bryson et al. 2004;
Pidd 2010). In many cases, policy analysis requires the use of many different
modeling tools and techniques depending on the problem at hand, and in fact, using
different modeling techniques with the same problem may lead to different policy
options (Andersen 1980). Regardless of the modeling approach, policy analysts
need to have a deep understanding of a variety of model types and modeling tech-
niques, the basic assumptions underlying each and their respective strengths and
limitations. Without this deep knowledge, policy analysts may run the risk of bias-
ing the analysis process.
Researchers and practitioners in the fields of economics, mathematics, opera-
tional research and systems analysis, among other disciplines, have, over the years,
developed many types of qualitative and quantitative models (Sterman 2000; Jackson
2003; Bryson et al. 2004; Pidd 2010). Most of these models and techniques are usu-
ally classified according to their purpose or according to the modeling technique
(see Fig. 1). In terms of their purpose, models can be either descriptive, prescriptive
or designed to explore scenarios. In an analysis of 30 years of research in the Journal
of Policy Modeling, Ruiz Estrada and Yap (2013) found that only 9 % of the pub-
lished papers in the time period of 1979–2012 followed a descriptive approach, only
1 % of the papers in the same sample used a what-if approach, and the rest of them
followed a prescriptive approach to modeling. In terms of the technique used to
model development, it is common to classify qualitative models as mental, prose,
and flow models. There are many different ways to classify mathematical models, in
Fig. 1 A simple classification of models (Adapted from Puron-Cid et al. 2014)

6 J. Zhang et al.
terms of their approach to probabilistic events (stochastic vs. deterministic), their

approach to time (static vs. dynamic or discrete vs. continuous), their approach to
the nature of the causality (linear vs. nonlinear), and their approach to find or explore
policy alternatives (analytical vs. simulation). Quantitative models represent a little
bit more than half (55 %) of the research in policy modeling, and static models con-
stitute 85 % of the papers published in the aforementioned journal (Ruiz Estrada and
Yap 2013). Simulation approaches, however, have been of increasing interest among
policy makers and researchers (Janssen and Cresswell 2005; Klievink and Janssen
2010; Kim et al. 2011), particularly in working with groups of policy makers in col-
laborative modeling (Cockerill et al. 2009; Eden et al. 2009; Ackermann et al. 2011;
Richardson et al. 2015).
Data and Open Data
While making government data open and available creates a host of new opportuni-
ties across governments and sectors, using these data for policy informatics presents
many real challenges (Janssen et al. 2012). For example, the data required for mod-
eling most policy problems needs to be integrated from multiple, and disparate
sources (Pardo et al. 2008; Barrett et al. 2011; Maheshwari and Janssen 2014).
Further, organizations and people think about, manage, use, and benefit from
government data in a wide variety of ways (Dawes et al. 2004a; Dawes and Helbig
2010; Klischewski 2012; Harrison et al. 2012). Those involved in policy informatics
must understand the context within which data of interest has been created and that
within which it will be used (Dawes and Helbig 2010; Klischewski 2012; Helbig
et al. 2012a). That is to say, to use data that has been captured, managed, and used
as part of a government business process in policy modeling often requires that data
to be reorganized and often cleansed and consolidated in ways that make it relevant
or usable to that modeling effort. For example, Helbig and her colleagues (2012a)
discuss how the release of restaurant inspection data to the public required several
adaptations and transformations before it was useful for citizens and others as part
of their restaurant choice-making processes. A stakeholder analysis can build appre-
ciation of how different people might use and benefit from the insights created
through different uses of data.
Zuiderwijk and her colleagues (2014) suggest that there are four basic elements
to increasing the potential of open data initiatives to create new public value: (1)
data, (2) tools for searching, (3) tools for data visualization, and (4) mechanisms
that users can use to share interpretations, discuss the data and give feedback to the
data producer. Additionally, they establish as desirable characteristics of open data
initiatives, showcasing examples of ways of using data, a quality management pro-
gram, and the publication of many types of meta-data. Finally, for available data to
be useful in the context of policy analysis, it requires individuals with the skills and
techniques to handle them. Dawes (2010) proposes developing specific stewardship
for open data initiatives, which relates to skills, knowledge, and practices required
to make government data a strategic resource inside and outside government.
Information Technologies
The use of information technologies in the context of public policy analysis is a

critical component of policy informatics (Johnston and Kim 2011). For example,
the use of IT makes the collection, access and use of government data by different
stakeholders possible (Noveck 2009; Dawes and Helbig 2010; Barrett et al. 2011;
Lampe et al. 2011; Klischewski 2012; Maheshwari and Janssen 2014). The avail-
ability of technology-based tools, sometimes using graphical user interfaces, for
example, facilitates stakeholder engagement in the model development process
(Zagonel et al. 2004). The increasingly vast array of technological tools available
for policy analysis can be categorized generally into four types: (1) quantitative
tools, (2) qualitative tools, (3) policy-modeling tools, and (4) Web 2.0 applications
(Puron-Cid et al. 2012).
The quantitative tools involve a combination of the advances in mathematical,
statistical, and computational power that have emerged in the policy analysis arena.
These tools comprise the vast multidisciplinary contribution of analytical comput-
ing sophistication from economics, psychology, sociology, management, and politi-
cal science, among others. Today policy makers, analysts, interest groups, and even
citizens have access to mathematical and statistical software packages and tools to
process data. Some are highly sophisticated instruments and others are user-friendly
applications; and these quantitative software tools provide quasi-intuitive methods
to analyze and describe data and to infer associations among the critical indicators
of the related public policy problem. Main tools being used for statistical data anal-
ysis and visualization include but are not limited to SPSS, SAS, R, or Python. Some
of these tools are proprietary, such as SPSS or SAS, but some others are open source
such as R and Python. In fact, although open source tools require more advanced
programming skills, they are attracting communities of analysts sharing tools and
code for quantitative analysis in software code repositories such as Github (http://
github.com).
The qualitative tools combine various technologies to process information con-
tent from unstructured and diverse material such as documents, interview tran-
scripts, survey responses, audio, videos, pictures, forms, reports, or media clips.
These tools assist in the examination of focus group results, in-depth interviews,
content analysis, text analysis, ethnography, evaluation, and semiotics, among other
techniques. Some examples of these software tools are NVivo, Atlas.ti, and
Ethnograph.
The policy-modeling tools apply highly sophisticated computing advances to
simplified abstractions of complex realities developed for representation, analysis,
or simulation (Pidd 2010). Many approaches have been developed over time to
model complex policy problems, and many software tools have been developed to
8 J. Zhang et al.
work in each of these approaches. Some approaches that have been applied to policy
modeling include, but are not limited to Social Network Analysis (Rethemeyer
2007), Agent-Based Modeling (Kim et al. 2011), System Dynamics (Richardson
et al. 2015), and Discrete Event Simulation (Janssen and Cresswell 2005). These
high-level modeling approaches have the main objective of developing new insights
into the main leverage points or bottlenecks in the policy area.
Finally, Web 2.0 tools constitute social media and other applications that allow
policy makers to increase the reach of the policy discussion or crowdsource public
policy analysis (Barrett et al. 2011; Charalabidis et al. 2012; Lampe et al. 2014).
The academic literature in public administration and digital government is still
scarce in terms of exploring crowdsourcing, although there are examples in areas
such as rule-making (Schlosberg et al. 2007) or emergency response (Spiro et al.
2014). Many examples of crowdsourcing approaches for better understanding pub-
lic problems exist already in the nonacademic literature. For instance, Foodborne
Chicago (https://www.foodbornechicago.org/) is a web-based tool to report food
poisoning episodes. The Chicago Department of Public Health is using such tools
to better protect restaurant customers by identifying inspection violations through
patterns and trends in restaurant inspection data.
Stakeholder Involvement
Stakeholder involvement in the construction and analysis of models to explore pol-

icy is the final pillar of policy informatics. The particular type of stakeholder
engagement found to facilitate policy modeling finds its roots in decision confer-
ences and groups decision support systems (Nunamaker 1989; Rohrbaugh 1992).
Experiences in running this type of facilitated meetings show that there are two
main components in the process: dividing the facilitation task in different roles and
organizing the conversations around small facilitated tasks or scripts (Richardson
and Andersen 1995; Andersen and Richardson 1997; Ackermann et al. 2011;
Richardson et al. 2015).
In terms of group facilitation, research shows that engaging stakeholders in
developing models for policy analysis requires the facilitation task to be divided
into a team of people, given that it is complicated for a single person to take care of
the group processes at the same time to think in a modeling the policy problem. In
this way, the particular approach promoted at the University at Albany, State
University at New York divides the task in the roles of facilitator and modeler/
reflector (Richardson and Andersen 1995; Dawes et al. 2004b; Richardson et al.
2015). The facilitator is mainly in charge of group processes, making sure that the
model represents all stakeholder points of view. The modeler/reflector, on the other
hand, is mostly in charge of developing a useful and robust model to facilitate pol-
icy analysis. Both roles need to be in close communication, balancing the tension
between having a model that constitutes a robust representation of the problem and
at the same time is useful to be a conversational tool, helping stakeholders to better
understand the dependencies among their main areas of responsibility when facing
a complex problem (Black and Andersen 2012).
The second key component in facilitating conversations among stakeholder
groups is designing modeling sessions that promote both the presentation of diverse
points of view (divergent in nature), and agreement among different players (con-
vergent in nature). Such short activities have been called scripts (Andersen and
Richardson 1997), and can be considered as a set of best practices in designing groups
sessions and stakeholder involvement (Ackermann et al. 2011). While different
modeling communities gather and organize their experiences in different ways, the
system dynamics community, as one particular modeling community, uses a shared
repository called Scriptapedia (Hovmand et al. 2013). This set of best practices
constitutes a valuable tool to organize and facilitate stakeholder groups (Hovmand
et al. 2012) and to support the development of the policy informatics community
more generally.
Policy Informatics and Sustainability
The use of mathematical models to better understand sustainability can be traced

back several decades. One of the first efforts in using formal modeling to understand
sustainability was the book “World Dynamics,” written by Jay Forrester in 1971
(Forrester 1973). The book describes a simulation model that offers a simplified
version of the world, and assumes that there are limits to population growth. The
objective of the model was to promote debate on the future and sustainability of our
species. In fact, the model presented in the book suggested a collapse in world popu-
lation, promoted by limits on renewable resources and pollution resulting from the
high standards of living, particularly in developed countries. At the time, the work
actually caused significant controversy and debate in academia but also between
citizens and other stakeholders. Main points in the debate involved main assump-
tions and data captured in the model. Unfortunately, the debate had no impact on
global policies for reducing greenhouse gas emissions or promoting any alternative
solution to the energy production problem.
In recent years, increases in temperature and other changes in the environment
have renewed this debate and newly proposed mechanisms to reduce carbon emis-
sions and improve sustainability are now part of the debate. Quantitative dynamic
models are also playing a role in facilitating the conversation and the policy analysis
process. An example of this renewed interest comes from one of the original actors
involved in the debate about sustainability back in the 1970s. In his recent book,
Jorgen Randers (2012) introduces a robust model that uses current data and a variety
of modeling techniques to better understand trends for sustainability in the next 40
years. Randers points out in his book that we are late in reacting to the main prob-
lems presented in the original World Dynamics book and therefore, he argues, we
must act fast.
10 J. Zhang et al.
Another interesting example of the use of policy informatics in the area of sus-
tainability is the C-Roads project, which is a project of Climate Interactive (http://
www.climateinteractive.org/) in collaboration with researchers at the MIT Sloan
School and Ventana Systems. C-Roads is a simulation model describing the effects
of the reduction of greenhouse gases in the atmosphere on climate change. EN-Roads
is an extension to the C-Roads model that includes more specifically the way in
which different energy policies impact gas emissions and ultimately global warm-
ing. These models are being used in both creating awareness about the problem and
designing better policies for sustainability. The United Nations Framework
Convention on Climate negotiators have used the model to assess proposals for
reducing greenhouse emissions.
Increasing the impact of policy informatics in understanding the problem of sus-
tainability requires the continuous improvement of tools, models, data, and facilita-
tion techniques. This book looks to contribute to this debate by offering a set of
insights in each of these policy informatics pillars.
Chapters on Information, Policy, and Sustainability
The seven chapters of this book, beyond this editorial chapter, each put forward a
unique perspective and approach to questions of sustainability. Across the chapters,
the authors from China, the United States, Sweden, Mexico, and Brazil and repre-
senting a number of disciplines, provide a diversity of considerations of sustainabil-
ity and introduce and examine a wide range of tools and techniques related to
modeling, understanding, and achieving sustainable strategies and systems. The
book begins with a chapter focused on the use of modeling to engage stakeholders
and quantify their preferences as part of the policy priority setting processes; the
next two focus on the use of modeling to build new understanding of the relation-
ships between the entities and actors that influence sustainability of the respective
systems studied. Chapter four provides a consideration of the mechanisms needed
and in use to make climate science data and models more usable to decision-makers.
Chapter five introduces a set of tools designed to support the efforts of government
to improve the complex process of green procurement as part of an environment and
social sustainability agenda. The final two chapters provide examinations of the use
and impact of particular technologies such as web portals and mobile devices, in
executing policy priorities such as transparency and paperwork reduction.
In the first chapter of the collection, Reading the Minds for Quantitative
Sustainability: Assessing Stakeholder Mental Models via Probabilistic Text
Analysis, Matteo Convertino, Rafael Munoz-Carpena, and Carolina Murcia look to
mental modeling as a useful method to quantify stakeholders’ preferences for eco-
system management by representing them as a visual socio-cognitive network that
can be analyzed with equivalent tools used to analyze complex biological and tech-
nological systems. Management of complex ecosystems, they note, is a difficult
process that involves multiple factors and stakeholders with various objectives and
preferences. The authors present a case study of using mental modeling to assess
stakeholders’ preferences for the management of a large-scale watershed in Costa
Rica, the Tempisque-Bebedero-Palo Verde ecosystem (TBPV). Academics and rep-
resentatives of non-profit organizations (NGOs), the key decision-makers for TBPV
infrastructure projects, participated in this study by attending a 4-day workshop that
generated 85 research questions. These research questions along with the institu-
tional origin of their authors helped identify stakeholder preferences, which were
then used to: evaluate trade-offs among ecosystem objectives; identify the prob-
lem’s most important aspects; and developmental models used for quantitative anal-
ysis. The study found that academic stakeholders are more focused on the current
and past dynamics of natural and human processes, while NGO stakeholders are
focused on the future socio-legal aspects of ecosystem management. The chapter
demonstrates that a decision-analysis-based mental modeling approach allows
transparent and participatory decision-making concerning ecosystem management
and facilitate the identification and balance of trade-offs among stakeholder groups.
In their chapter, Supply-Chain Transparency and Governance Systems: Market
Penetration of the I-Choose System Weijia Ran, Holly Jarman, Luis F. Luna-Reyes,
Jing Zhang, Deborah Andersen, Giri Tayi, Djoko S. Sayogo, Joanne Luciano,
Theresa Pardo, and David Andersen examine the impacts of the factors influencing
the market diffusion of technical innovations that promote sustainable consumption
through a case analysis using system dynamics group modeling and simulation. In
their chapter, they address the growth of consumer markets for sustainable products
and related requirements for information about products’ utilities, qualities, and
environmental and social impacts of production processes. Technical innovations
that facilitate the disclosure and use of such trustworthy and quality information,
they argue, can help consumers make more informed purchasing decisions and can
promote transparent supply chain and sustainable consumption and production.
However, the success of these technical innovations, the authors found, is influ-
enced by a variety of nontechnical factors. The case studied is I-Choose, an initia-
tive that engaged a variety of supply chain stakeholders to implement a prototype
system that makes consumer information about how, where, by whom, and under
what conditions coffee was produced available and usable. The simulation experi-
ments revealed that the market resists “take-off” of such technical innovations
unless external financial support can be found for marketing and infrastructure.
Marketing budgets drive how fast users adopt a system, and without external spon-
sorship the market for a system collapses in the short run. However, it was also
found that the market sustainability of a technical innovation such as I-Choose is a
function of the quality and trustworthiness of the consumer information it provides,
which is a function of the system’s governance reflected in information complete-
ness, openness, relevance, and reliability.
Environmental Performance or Productivity Loss? The third chapter, provided by
Shital Sharma, addresses this question through an examination of the link between
environmental regulation, plant-level productive efficiency, and environmental effi-
ciency in two U.S. manufacturing industries (oil refineries and pulp and paper mills)
during the 1970s and the 1990s. Proponents of environmental sustainability and
12 J. Zhang et al.
improved environmental outcomes, Sharma notes, have pointed out that the inverse
relationship between environmental regulation and productivity is inevitable due to
the traditional methods used to measure productivity. This has, according to Sharma,
led to the introduction of methods to measure productivity that account for the
reduction in emissions due to the diversion of resources from production to emission
reduction. The chapter focuses on determining how the output loss due to compli-
ance with environmental regulation changes when environmental performance is
accounted for in the measurement of productive efficiency. Sharma finds that plants
suffer a reduction in productivity as they are forced to devote resources on abate-
ment due to environmental regulations restricting emissions. The inclusion of envi-
ronmental outcomes in these measurements reduces such costs to plants. Sharma
notes, however, the efficiencies in emission reduction do not compensate for reduc-
tions in productive activities. This implies that a win-win outcome from regulations
may not exist for plants and that an increase in efficiency in abatement practices is
needed for such outcomes. The paper also compares the costs of regulation in terms
of reduced productivity to its benefits in terms of reduced mortality. Results from
this comparison suggest that the benefits of regulations are far greater than its costs
and that environmental regulations make economic sense and can be an effective
form of government policy in pursuing an environmentally sustainable future.
In chapter five entitled, Green Government Procurement: Decision Making with
Rough Set, TOPSIS, and VIKOR Methodologies, Chunguang Bai, and Joseph Sarkis
introduce a set of tools and techniques for improving capability for Green
Procurement. Green government procurement (GGP) is a popular method used by
government entities to help green industries through market mechanisms. The
selection of green suppliers, according to Bai and Sarkis, is a complex process
involving environmental and social sustainability metrics, thousands of potential
suppliers of a broad variety of products and services, and group decision-making. In
addition, governmental entities may require and acquire substantial supplier perfor-
mance data that needs to be filtered and evaluated. The complexities and magni-
tudes for GGP, they argue, make the supplier selection process a major undertaking.
Thus, government entities, according to Bai and Sarkis, need tools to aid GGP
decision-making. This chapter introduces a series of tools within a broader method-
ology to help meet these practical requirements. These tools are meant to filter out
decision factors and aggregate decision-maker inputs. They include a Rough Set
tool to help reduce the number of factors for consideration in a relatively complex
decision environment and TOPSIS and VIKOR tools for aggregation and decision
support to help rank and evaluate performance of suppliers and products. Using
illustrative data, Bai and Sarkis discuss the methodological application and practical
implications for the implementation of these tools and methodology for GGP and
their possible use in other public and private sector environments.
In Enhancing the Usability of Climate Information and Models through
Stakeholder Engagement, Elizabeth Allen and Jennie Stephens recognized that the
impacts of climate instability intensify and environmental scientists and earth sys-
tem modelers are rapidly advancing our understanding of the complex dynamics of
climate change. They argue, however, although large quantities of high-quality cli-
mate data and climate science information are now publicly available, there is a
limit to the usability of this information to policy makers and other nonacademic
decision-makers. Novel approaches are needed, they further argue, to increase the
usability of climate science information and to enhance its influence in decision-
making. In their chapter, Allen and Stephens report on a regional climate change
impact modeling project called BioEarth, which is attempting to bridge the climate
science usability gap. The authors consider the lessons learned in the course of
BioEarth’s stakeholder engagement efforts and reflect on how those insights may
contribute to other initiatives seeking to develop and communicate climate science
information. In particular, Allen and Stephens consider the Regional Climate Hubs,
recently established by the US Department of Agriculture. These Hubs are envi-
sioned as internet-based centers that will deliver climate change data and analyses
to farmers, ranchers, and forest landowners. This chapter contributes to the emerg-
ing field of policy informatics by exploring mechanisms by which climate science
data and models can be made more usable to decision-makers.
Chapter “Do Open Data Initiatives Promote and Sustain Transparency? A
Comparative Analysis of Open Budget Portals in Developing Countries” contrib-
uted by Jyldyz Kasymova, Marco Aurelio Marques Ferreira and Suzanne
J. Piotrowski raise questions about the effectiveness of information technology in
promoting and sustaining transparency through a comparative analysis of online
budget portals in Brazil and Kyrgyzstan. In Do Open Data Initiatives Promote and
Sustain Transparency? A Comparative Analysis of Open Budget Portals in
Developing Countries, the authors argue that information technology is an impor-
tant tool to improve government transparency and facilitate good governance, citi-
zen engagement, and accountability. They argue that in many developing countries
government transparency is limited or completely lacking. The comparative analy-
sis of the selected portals examines the impact of those portals on advancing and
sustaining government transparency. Using data collected through interviews with
public administrators, archival information, and an analysis of the portals them-
selves, the authors use Meijer’s (2013) transparency theoretical framework to assess
the relationship between budget portals and government transparency. The study
finds that both Brazil and Kyrgyzstan face similar challenges promoting online dis-
closure, including a lack of infrastructural development and a limited use of their
portals by the public. The centralized online budget disclosures, the authors argue,
have not promoted a sense of transparency in the broader public. However, the
authors did fine that the media has effectively used the information in the portals to
strengthen reporting on budgeting issues, leading to wider public discourse on bud-
geting in both countries. The authors highlight the specific roles played by the Open
Budget Index and international players in advancing and sustaining budget transpar-
ency in these countries.
In the final chapter of the book, On Sustaining Sustainability—the Case of
Implementing Decisions based on Policies for a Sustainable Future via Tablets in a
Board of a Swedish Housing Corporation, Jenny Eriksson Lundström, and Mats
Edenius examine how a large Swedish municipal board’s decision to reduce paper
use was implemented by introducing tablets in one of its sub-boards. Government
14 J. Zhang et al.
organizations, the authors note, are adopting handheld devices such as smartphones
and tablets as a means to address sustainability concerns with an underlying assump-
tion that policies promoting the use of handheld devices will result in a more sus-
tainable future by supporting the shift from paper records to digital information.
Therefore, the authors argue, it is essential to have a deeper understanding of how
the adoption and use of handheld devices actually can support sustainability poli-
cies. Lundstom and Edenius used Melville’s (2010) Belief-Action-Outcome frame-
work as their theoretical lens to examine how the implementation of the
municipality’s sustainability policy and the introduction of mobile technology
affected the board’s work processes and its sustainability performance. The authors
used data collected through interviews and observations of the sub-board’s opera-
tions to explore the decision to implement a sustainability policy through the intro-
duction of mobile technology. They studied the introduction of those technologies
and how they served as a catalyst to establish new business practices which prompted
changes in the board’s work processes and raised issues of data security and privacy.
The authors found that the introduction of mobile technology and new business
practices failed to reach most targets established by the board’s sustainability policy
including reducing the use of paper. The study, the authors argue, demonstrates how
important it is for decision-makers to consider the broader social and organizational
context when developing organizational policies designed to support sustainability.
The papers provide the reader with an historical perspective on policy modeling
and across the chapters, new insight into the twenty-first century instantiation of the
concept, in the form of policy informatics. Through attention to modeling, data,
technology, and people, the chapters provide further support for the argument that
policy informatics, and its use to build new understanding of the interactions in
systems, is ideally suited to sustainability investment decision-making. The chap-
ters by Sharma and Bai and Sarkis address organizational investment on technology,
data, and modeling. Sharma tests the impact of abatement policy by applying a set
of analytical tools to explore the link between environment regulation, plant-level
productive efficiency, and environmental efficiency and determines that forced
investments in abatement result in a reduction in productivity. In Bai and Sarkis’
consideration of the case where governments are opting into green procurement
strategies but unprepared to navigate the complexity of the process, policy informat-
ics is understood in terms of technology and data. To reduce this complexity, Bai
and Sarkis introduce a new set of tools to help government decision-makers who are
opting into green government procurement to navigate the complex process of green
supplier selection.
Convertino and Weija et al. contribute chapters that put stakeholders at the center
of their instantiation of policy informatics. The authors place stakeholders at the
center of the modeling process and see the data provided by such stakeholders as
key to developing understanding of the dynamics of the systems of interest. These
chapters contribute to new understanding of policy informatics in particular in terms
of people and modeling.
Allen and Stephens’ contribution bridges the gap between the more tool oriented
policy informatics of Sharma and Bai and Sarkis and the more stakeholder oriented
efforts of Convertino and Weija et al. Allen and Stephens’ work leans toward the
data and people aspects of policy informatics by focusing primarily on the success
of efforts to deliver usable and consumable data to decision-makers. This chapter
contributes to new understanding of policy informatics and its relevance to sustain-
ability through its focus on communication of insights found in data to
decision-makers.
Kasymova et al. and Lundstrom and Edenius draw on the broadest understanding
of economic and politic sustainability by raising questions about the impact of
technology-based strategies for sustaining policy priorities such as transparency, in
the case of Brazil, and for operationalizing a government policy to enhance sustain-
ability in Sweden. In these two chapters, policy informatics is understood in terms
of technology and people. The chapters each examined a particular situation in
which governments were executing on a policy priority and, in both cases, people
and what they experienced and learned and how those learnings would be captured
and used to guide future investments was at the center.
Taken together, the papers highlight the challenges of building understanding of
enabling conditions for successful investments in sustainable development. They
provide robust contributions to the consideration of the complex environments
within which such investments must take place and the wide range of stakeholders
who must be engaged as part of the investment analysis and decision processes.
Contribution of This Book
This book is a first of its kind in terms of its focus on connecting the discussion of
policy informatics, data and information technology with the issue of sustainability.
The chapters, taken together, provide a novel consideration of how new and emerg-
ing technologies have made is possible for groups of people to use vast amounts of
data newly available to support the policy-making process. They provide a unique
consideration of how emerging technologies are resulting in a vast new array of
tools and techniques to use that data to inform public policy questions, including
questions about sustainability and sustainable systems. This book postulates valu-
able viewpoints on the impact of advancements in computer and communication
technology in transforming the role of government to facilitate a market of sustain-
able development that recognizes perfect information through smart public policies
and provides a timely response to the paucity of relevant research in recent develop-
ments in open government and open government data, including policies that
encourage sharing of data and information to the public such as open data and smart
disclosure policy.
Policy makers and practitioners around the world are recognizing the opportu-
nity that data and emerging technologies represent in terms of creating the insight
needed to improve policy making and implementation processes. This collection of
chapters is primarily geared to support their efforts. As such, the primary audience
for this book is researchers and policy makers seeking to change the landscape by
16 J. Zhang et al.
using policy informatics as a tool to apply to the problems of sustainability. This is

a growing and vibrant community that increasingly understands the potential of new
technologies and the world’s growing investments in data as a public asset. This
community is seeking ways for innovative tools and techniques to be developed and
used in the increasingly complex public policy-making process. Whether operating
at the global level on the Post 2015 Sustainable Development goals, issues of cli-
mate change and creating environmental policies focused on sustainable systems or
working at the national level to understand the levers of sustainable systems required
for transparency and accountability, or at the local level on problems of data con-
sumption, this book is geared to supporting a range of discussions anchored in sus-
tainability, policy informatics, and information technology and data.
Two communities in particular; the sustainability community and the policy
informatics community are served by this book. Across these communities, the
book will have particular interest to researchers, students, and those working in
policy making and program design and implementation roles in government and
non-profit organizations. Researchers and students in environmental policy studies,
environmental management, ecological economics, information science with focus
on information technology in government, economic development with focus on
sustainability issues, and public administration scholars with interests in sustain-
ability, open data and information technology, in particular, will benefit. Government
officials and professionals working for international organizations with a focus on
social inequality, environmental degradation, and economic development and for
non-governmental organization with focus area on social inequality, environmental
degradation and economic development, public officials responsible for developing
policies for environmental sustainability will also find the book particularly relevant
and useful.
Acknowledgments This book was supported in part the U.S. National Science Foundation, Grant
# 0955935 and the Mexican Consejo Nacional de Ciencia y Tecnología (National Council on
Science and Technology, CONACYT) Grant # 133670. We wish to acknowledge Alan Kowlowitz,
Government Fellow at the Center for Technology in Government, University at Albany, State
University of New York, for his help in producing this chapter.
References
Ackermann F, Andersen DF, Eden C, Richardson GP (2011) ScriptsMap: a tool for designing
multi-method policy-making workshops. Omega 39:427–434. doi:10.1016/j.
omega.2010.09.008
Andersen DF (1980) How differences in analytic paradigms can lead to differences in policy con-
clusions. In: Randers J (ed) Elements of the system dynamics method. Productivity Press,
Cambridge, pp 61–75
Andersen DF, Richardson GP (1997) Scripts for group model building. Syst Dyn Rev
13:107–129
Barrett CL, Eubank S, Marathe A et al (2011) Information integration to support model-based
policy informatics. Innov J 16:1–19
Basiago AD (1995) Methods of defining “sustainability”. Sustain Dev 3:109–119

Black LJ, Andersen DF (2012) Using visual representations as boundary objects to resolve conflict
in collaborative model-building approaches. Syst Res Behav Sci 29:194–208
Brown BJ, Hanson ME, Liverman DM, Merideth RW Jr (1987) Global sustainability: toward defi-
nition. Environ Manage 11:713–719
Brundtland GH (1987) Report of the World Commission on environment and development: our
common future. United Nations, New York
Bryson JM, Ackermann F, Eden C, Finn CB (2004) Visible thinking: unlocking causal mapping for
practical business results, 1st edn. Wiley, West Sussex
Charalabidis Y, Triantafillou A, Karkaletsis V, Loukis E (2012) Public policy formulation through
non moderated crowdsourcing in social media. In: Tambouris E, Macintosh A, Sæbø Ø (eds)
Electronic participation. Springer, Berlin, pp 156–169
Cockerill K, Daniel L, Malczynski L, Tidwell V (2009) A fresh look at a policy sciences methodol-
ogy: collaborative modeling for more effective policy. Policy Sci 42:211–225. doi:10.1007/
s11077-009-9080-8
Dawes SS (2010) Stewardship and usefulness: policy principles for information-based transpar-
ency. Gov Inf Q 27:377–383. doi:10.1016/j.giq.2010.07.001
Dawes SS, Helbig N (2010) Information strategies for open government: challenges and prospects
for deriving public value from government transparency. In: Wimmer MA, Chappelet J-L,
Janssen M, Scholl HJ (eds) Electronic Government: 9th IFIP WG 8.5 International Conference,
EGOV 2010, Lausanne, Switzerland, August/September 2010: Proceedings. Springer, Berlin,
pp 50–60
Dawes SS, Janssen M (2013) Policy informatics: addressing complex problems with rich data,
computational tools, and stakeholder engagement. ACM, New York, pp 251–253
Dawes SS, Pardo TA, Cresswell AM (2004a) Designing electronic government information access
programs. Gov Inf Q 21:3–23
Dawes SS, Pardo TA, Simon S et al (2004b) Making smart IT choices: understanding value and
risk in government IT investments. Center for Technology in Government, University at
Albany, SUNY, Albany
Dawes SS, Helbig N, Nampoothiri S (2014) Workshop report: exploring the integration of data-
intensive analytical skills in public affairs education. Center for Technology in Government,
University at Albany, SUNY, Albany
Dovers S (2005) Environment and sustainability policy: creation, implementation, evaluation.
Federation, Sydney
Eden C, Ackermann F, Bryson JM et al (2009) Integrating modes of policy analysis and strategic
management practice: requisite elements and dilemmas. J Oper Res Soc 60:2–13. doi:10.1057/
palgrave.jors.2602575
Forrester JW (1973) World dynamics, 2nd edn. Pegasus Communications, Waltham
Gerald D, Lave LB (2005) Implementing technology-forcing policies: the 1970 clean Air Act
amendments and the introduction of advanced automotive emissions controls in the United
States. Technol Forecast Soc Change 72:761–778
Harrison TM, Pardo TA, Cook M (2012) Creating open government ecosystems: a research and
development agenda. Future Internet 4:900–928. doi:10.3390/fi4040900
Helbig N, Cresswell AM, Burke BG, Luna-Reyes LF (2012a) The dynamics of opening govern-
ment data. Center for Technology in Government, University at Albany, SUNY, Albany
Helbig N, Nakashima M, Dawes SS (2012b) Understanding the value and limits of government
information in policy informatics: a preliminary exploration. In: Proceedings of the 13th annual
international conference on Digital Government Research. ACM, New York, pp 291–293
Hovmand PS, Andersen DF, Rouwette E et al (2012) Group model-building “scripts” as a collab-
orative planning tool. Syst Res Behav Sci 29:179–193
Hovmand PS, Rouwette EAJA, Andersen DF et al (2013) Scriptapedia 4.0.6
Huq M, Wheeler D (1993) Pollution reduction without formal regulation: evidence from
Bangladesh. World Bank, Washington, DC
18 J. Zhang et al.
Jackson MC (2003) Systems thinking: creative holism for managers, 1st edn. Wiley, Chichester
Jaffe AB, Newell RG, Stavins RN (2005) A tale of two market failures: technology and environ-
mental policy. Ecol Econ 54:164–174
Janssen M, Cresswell AM (2005) An enterprise application integration methodology for
e-government. J Enterp Inf Manag 18:531–547. doi:10.1108/17410390510623990
Janssen M, Charalabidis Y, Zuiderwijk A (2012) Benefits, adoption barriers and myths of open
data and open government. Inf Syst Manag 29:258–268. doi:10.1080/10580530.2012.716740
Johnston E, Kim Y (2011) Introduction to the special issue on policy informatics. Innov J 16:1–4
Kim Y, Johnston EW, Kang HS (2011) A computational approach to managing performance
dynamics in networked governance systems. Public Perform Manag Rev 34:580–597.
doi:10.2753/PMR1530-9576340407
Klievink B, Janssen M (2010) Simulation gaming as a social development instrument: dealing with
complex problems. Inf Polity Int J Gov Democr Inf Age 15:153–165. doi:10.3233/
IP-2010-0204
Klischewski R (2012) Identifying informational needs for open government: the case of Egypt.
IEEE Computer Society, Washington, DC, pp 2482–2490
Lampe C, LaRose R, Steinfield C, DeMaagd K (2011) Inherent barriers to the use of social media
for public policy informatics. Innov J 16:1–17
Lampe C, Zube P, Lee J et al (2014) Crowdsourcing civility: a natural experiment examining the
effects of distributed moderation in online forums. Gov Inf Q 31:317–326. doi:10.1016/j.
giq.2013.11.005
Maheshwari D, Janssen M (2014) Reconceptualizing measuring, benchmarking for improving
interoperability in smart ecosystems: the effect of ubiquitous data and crowdsourcing. Gov Inf
Q 31:S84–S92. doi:10.1016/j.giq.2014.01.009
Noveck BS (2009) Wiki government: how technology can make government better, democracy
stronger, and citizens more powerful. Brooking Institution, Washington, DC
Nunamaker JF Jr (1989) Experience with and future challenges in GDSS (group decision support
systems): preface. Decis Support Syst 5:115–118
Pardo TA, Gil-Garcia JR, Luna-Reyes LF (2008) Collaborative governance and cross-boundary
information sharing: envisioning a networked and IT-enabled public administration. Lake Placid
Pidd M (2010) Tools for thinking: modelling in management science, 3rd edn. Wiley, Chichester
Puron-Cid G, Gil-Garcia JR, Luna-Reyes LF (2012) IT-enabled policy analysis: new technologies,
sophisticated analysis and open data for better government decisions. In: Proceedings of the
13th annual international conference on Digital Government Research. ACM, New York,
pp 97–106
Puron-Cid G, Gil-Garcia JR, Luna-Reyes LF (2014) Opportunities and challenges of policy infor-
matics: tackling complex problems through the combination of open data, technology and ana-
lytics. Centro de Investigación y Docencia Económica CIDE, Aguascalientes
Randers J (2012) 2052: a global forecast for the next forty years. Chelsea Green, White River
Junction
Rethemeyer RK (2007) Policymaking in the Age of internet: is the internet tending to make policy
networks more or less inclusive? J Public Adm Res Theory 17:259–284
Richardson GP, Andersen DF (1995) Teamwork in group model building. Syst Dyn Rev
11:113–137
Richardson GP, Andersen DF, Luna-Reyes LF (2015) Joining minds: group system dynamics
modeling to create public value. In: Bryson JM, Crosby BC, Bloomberg L (eds) Valuing public
value. Georgetown University Press, Washington, DC
Robinson DG, Yu H, Zeller WP, Felten EW (2009) Government data and the invisible hand. Yale
J Law Technol 11:160
Rohrbaugh J (1992) Cognitive challenges and collective accomplishments. In: Bostrom RWR,
Kinney ST (eds) Computer augmented teamwork: a guided tour. Van Nostrand Reinhold,
New York, pp 299–324
Ruiz Estrada MA (2011) Policy modeling: definition, classification and evaluation. J Policy Model
33:523–536. doi:10.1016/j.jpolmod.2011.02.003
Ruiz Estrada MA, Yap SF (2013) The origins and evolution of policy modeling. J Policy Model
35:170–182. doi:10.1016/j.jpolmod.2011.12.003
Sayogo DS, Zhang J, Luna-Reyes L et al (2014) Challenges and requirements for developing data
architecture supporting integration of sustainable supply chains. Inf Technol Manag
16(1):5–18
Schlosberg D, Zavestoski S, Shulman SW (2007) Democracy and E-rulemaking: web-based tech-
nologies, participation, and the potential for deliberation. J Inf Technol Polit 4:37–55
Sonntagbauer P, Nazemi K, Sonntagbauer S et al (2014) Handbook of research on advanced ICT
integration for governance and policy modeling. IGI Global, Hershey
Spiro ES, Johnson B, Fitzhugh S et al (2014) Warning tweets: serial transmission of messages dur-
ing the warning phase of a disaster event. Inf Commun Soc 17:765–787
Stavins, R. N. (2007). A US Cap and Trade System to Address Global Climate Change, Research
Working Paper 07-052, Kennedy School of Government, Harvard University
Sterman JD (2000) Business dynamics: systems thinking and modeling for a complex world.
Irwin/McGraw-Hill, Boston
UN General Assembly (2000) United Nations Millennium Declaration: resolution. UN, New York
United Nations Environment Programme (2011) Towards a green economy: pathways to sustain-
able development and poverty eradication. United Nations Environment Programme, Nairobi
Zagonel AA, Rohrbaugh J, Richardson GP, Andersen DF (2004) Using simulation models to
address “what if” questions about welfare reform. J Policy Anal Manage 23:890–901
Zuiderwijk A, Janssen M, Davis C (2014) Innovation with open data: essential elements of open
data ecosystems. Inf Polity Int J Gov Democr Inf Age 19:17–33. doi:10.3233/IP-140329
“Reading the Minds” for Quantitative
Sustainability: Assessing Stakeholder Mental
Models via Probabilistic Text Analysis
Matteo Convertino, Rafael Munoz-Carpena, and Carolina Murcia
Abstract Management of complex ecosystems is a difficult process that involves

multiple factors and stakeholders. In most cases, the interactions of these factors
and stakeholders’ trade-offs are not considered quantitatively in the design and
management of ecosystems. To address these issues mental modeling is useful for
eliciting stakeholder objectives and preferences in order to evaluate preliminary
knowledge about structure and function of complex ecosystems. This is advanta-
geous for ecosystem analysis, modeling, and management.
Here, we provide an assessment of stakeholder preferences and mental models for
the case study of a large-scale watershed in Costa Rica composed by two rivers
basins and one wetland. Trade-offs are related to water management in relation to
ecological, agricultural, energy, and tourism endpoints that are affected by potential
sets of dam and canal configurations. We apply and further develop a network-based
model for stakeholder text analysis. Our innovation is the introduction of Network
Complexity (NC) as a metric to characterize the inferred influenced diagram and to
monitor the variation of such diagram under perturbations in socio-ecological factors
provided by stakeholders and related to ecosystems. Perturbation effects—innova-
tively determined and assessed by global sensitivity and uncertainty analyses may
reveal fundamental factor importance and interactions of ecosystem factors and eco-
system resilience. The derived influence diagram can be considered as the mental
M. Convertino (*)
Division of Environmental Health Sciences, School of Public Health, University of Minnesota
Twin-Cities, Minneapolis, MN, USA
Institute for Engineering in Medicine and Public Health Informatics Programs,
University of Minnesota Twin-Cities, Minneapolis, MN, USA
Institute on the Environment, University of Minnesota Twin-Cities, Minneapolis, MN, USA
e-mail: matteoc@umn.edu
R. Munoz-Carpena
Department of Agricultural and Biological Engineering, University of Florida,
Gainesville, FL, USA
C. Murcia
Organization for Tropical Studies at Duke University, Durham, NC, USA

DOI 10.1007/978-3-319-25439-5_2
22 M. Convertino et al.
model of stakeholders because its construction is based on the direct elicitation of

stakeholder preferences and objective in ad hoc workshops organized for this study.
The decision analysis-based mental modeling approach allows a transparent and
participatory decision-making concerning ecosystem management. The approach
facilitates the identification and balance of trade-offs among stakeholder groups.
Thus, it is coherent with the sustainability paradigm that includes social factors into
the analysis, design, and management of complex systems.
Introduction
Mental Models
Complex networks are graphs that inform about the physical or functional connec-
tions among components of a system and among systems. Mental models are repre-
sentation of the reality of complex systems—for example, ecosystems—based on
stakeholders knowledge of systems. Mental models are often represented in the form
of networks (undirected or directed) whose features can be analyzed with equivalent
tools used for complex networks found in biological and technological systems.
Thus, mental models can considered as socio-cognitive network of stakeholders.
Mental models are useful in natural resource management for quantifying pre-
liminary evidence from data and stakeholder preferences. Mental models are widely
used for any complex systems, and they can be modeled used systems dynamics
model (Kim 2009; Kim and Andersen 2012) or combined system dynamics and
decision science models (Convertino et al. 2012, 2013; Convertino and Valverde
2013). Preferences are beliefs about a set criteria related to the problem at hand.
Thus, preferences possibly reveal subjective probabilities of criteria’s weights used
to evaluate the alternative solutions against each other for the problem investigated.
Such preferences reveal a mental model of stakeholders and can shift the decision-
making process when they vary. Thus, preferences are not fixed in time, nor among
stakeholder groups, and can be leveraged in order to change decision-making
(Reynolds et al. 2010; Walker et al. 2006).
Jones et al. (2011) and Wood et al. (2012a, b, c) reviewed a variety of elicitation
methods for identifying and describing stakeholders’ mental models that have been
successfully deployed in a variety of natural resource management (NRM) con-
texts. These methods are broadly categorized into direct and indirect methods.
Direct elicitation methods are those where stakeholders work in conjunction with
an analyst to describe and produce a graphical representation of the model in an
iterative and interactive fashion. Indirect elicitation methods are those where a
research team utilizes textual information from interviews, web sites, and other
documents to extract a graphical model via content analysis and/or the help of ad
hoc-designed computer programs. These models and programs can really help
policy via informatics by translating qualitative conversations of stakeholders into
Mental Modeling Complex Ecosystems 23
numbers that can be used effectively for policy-relevant decision-making. Such

models can detect individual and group preferences of stakeholders, thus empha-
sizing differential needs and contrasts to balance. Thinking about a government
organization dealing with a complex problem where social, environmental, and
economic criteria can collide against each other—in a sustainability perspective—
the proposed model can be useful to provide a balanced solution (Morgan et al.
2002; Wood et al. 2012a). Stakeholder weights for social, environmental, and eco-
nomic criteria can be used to balance criteria value for the selection of the optimal
highest scoring alternatives, thus providing a balanced objective and subjective
solution for the system considered. Specifically, here we propose an indirect elici-
tation method based on directly elicited data for an ecosystem management prob-
lem in Costa Rica where multiple solutions and criteria are evaluated. The case
study is just for illustrative purpose of a model that can be applied in any settings,
thus we prefer to keep the generality of the discussion in order to emphasize the
utility of the model for broader complex systems.
The Costa Rican Case Study
We provide a post hoc preliminary analysis and synthesis of the research questions
formulated at the Palo Verde research workshop in Costa Rica about the manage-
ment of the Tempisque–Bebedero–Palo Verde ecosystem (TBPV, hereafter). The
participating stakeholders belong to two kinds of institutions: academic and non-
profit private organizations (NGOs). These stakeholders were considered after
selection of key people involved in the decision-making of the water-related infra-
structure project in the TBPV. Other stakeholder may exist but they were not
involved in such study; however, it is very important in to include, if possible, any
stakeholder involved in the ecosystem management. Such stakeholders have been
identified by the Organization for Tropical Studies that is currently monitoring the
ecosystem and developing a network of scientists to tackle the ecosystem problems
in this area.
By analyzing the research questions and their institutional origin we: (i) evaluate
trade-offs among the objectives of the problem and researcher preferences; (ii)
screen the most important factors of the problem; and (iii) elicit mental models for
possible development of a probabilistic decision model and for guidance of a more
sophisticated modeling effort. For instance, research questions can be used to build
a utility function that considers the important factors composing the objective of the
problem, and mental models can be used to evaluate land-management policies rel-
evant to the TBPV. Hence, ultimately the analysis and quantification of stakehold-
ers’ information is useful for environmental management and for socio-cognitive
research related to individual and collective response of stakeholders facing
multicriteria decision problems (Morgan et al. 2002; Linkov et al. 2012; Sparrevik
et al. 2011; Wood et al. 2012a).
Materials and Methods
The Tempisque–Bebedero–Palo Verde Ecosystem
The 5404 km2 Costa Rican Tempisque basin extends from the Tilarán and Guanacaste
Mountains (in the NE) to the Gulf of Nicoya (SW) (Fig. 1). The Tempisque river and
its tributaries, flow into the northern Gulf of Nicoya, Pacific Ocean. The basin outlet
forms the Palo Verde wetland (PV), protected by the Palo Verde National Park and
internationally recognized by the Ramsar Convention (http://ramsar.org). The Bebedero
basin extends in the east area of the Tempisque basin. The region is a mixture of tropi-
cal dry forest and dry-with-transition to moist forest life zones (Bolaños and Watson
1999); hence, water is a limiting factor for both natural and human systems.
The basin was not significantly transformed until the 20th century, when forest
gave way to pasturelands and cattle ranches. In the 1970s, a pivotal hydrological
change occurred when the government created a large-scale irrigation district
(Fig. 1), funded by the International Development Bank, to provide agricultural
land to low-income Costa Ricans and food security (rice, beans, and sugar) for the
country. The irrigation district receives water (30–65 m3/s) transferred from the
Caribbean versant at Lake Arenal, and then directed to a hydroelectrical dam that
generates 12 % of the country’s electrical power (Fig. 1). Upon discharging neat the
Fig. 1 Tempisque–Bebedero–Palo Verde ecosystem

town of Cañas, the water feeds a network of channels that spread through the middle
Tempisque basin irrigating 44,000 ha of agricultural lands (Jiménez et al. 2001). It
eventually flows through the lower wetland-dominated basin, and into the Pacific at
the Gulf of Nicoya. This large addition of water has transformed the middle and
lower sections of the Tempisque basin. It modified the hydrology and thus the phys-
ical environment of both natural and human systems, and allowed the establishment
of a new and extensive land use (agriculture and aquaculture). It also changed the
TBPV dynamics by altering how the different ecosystems interact with each other.
Direct links are now established through the transport and transfer of energy and
materials among them, such as sediments, species propagules, agrochemicals, and
other pollutants. While the extent of water contamination and its impact on human
health are unknown, some manifestations are already visible in the protected wet-
lands in the lower basin (Daniels and Cumming 2008).
The wetlands of the lower Tempisque basin historically supported regionally
important populations of waterbirds, including ducks, ibises, wood storks (Mycteria
americana), and regionally endangered Jabiru storks (Jabiru mycteria); hence, the
site’s recognition as a Ramsar Wetland of International Importance. However,
beginning in the late 1980s, there has been a large reduction (>90 %) in the numbers
of aquatic birds supported. Temporal observations and experimental evidence sug-
gest that this change was precipitated by a vegetative regime shift, towards massive
overdominance of cattails (Typha domingensis), resulting in stands so dense that
birds cannot land or feed.
While the current situation already yields many environmental, socioeconomic,
and institutional problems, these conditions are likely to be exacerbated by climatic
variability and change. The 2007 Intergovernmental Panel on Climate Change
(IPCC) report indicates strong consensus among climate models for increasing tem-
perature and decreasing precipitation for much of Pacific Central America. Unless
greenhouse gas trends change, average temperatures are expected to increase 2–6 °C
in the region, possibly with more extreme hydro-climatological events. Wet season
precipitation is expected to decrease as much as 27 % with associated drier soils and
loss of water storage for irrigation, hydropower production, and protected wetlands.
Dry season river flow is also expected to decrease due to reduced cloud cover on the
mountain ridges. These changes may unfold in as little as two decades, with a trend
towards increasing aridity already evident in NW Costa Rica (Birkel and Demuth
2006). Recent and regional scale weather patterns appear to be consistent with long
term and global climate scenarios that portend severe impacts on agriculture, biodi-
versity, and land use (Murcia et al. 2012).
The gradual destruction of the Palo Verde wetland (Fig. 2), the Sardinal conflict,
pressures for new irrigation development and the proposed Rio Piedras Dam and its
unintended consequences, illustrate how individualistic actions to acquire water have
created a dysfunctional and unsustainable water system. They emphasize the urgency
of conducting an integrative analysis of the situation that informs strategies to reach
a consensual agreement on new policy that simultaneously considers all stakeholders
interests. This is the motivation for which in this study different management actions
are preliminary investigated by evaluating stakeholder preferences, trade-offs, and
Fig. 2 Stakeholder preferences for topic (top plot) and time categories (bottom plot) related to the
workshop questions
objectives for the TBPV ecosystem. The topic categories are: climate behavior (1),
climate on vegetation (2), governance (3), human impact on natural systems (4),
human impacts of natural systems on climate/production (5), impacts on water sys-
tem behavior (6), natural system behavior (7), use/decision processes (8), water sys-
tem behavior (9), natural system sustainability (10), water sustainability (11),
incentive mechanisms (12), governance structure (13), institutions (14), laws and
policy (15), stakeholders (16), and water use (17). The time categories are: historic
trends (1), current status (2), expected behavior (3). G = 0.62 and 0.25 for the two
frequency distributions of topic and time categories of NGOs and Academics, respec-
tively. The G-test value is proportional to the Kullback–Leibler divergence of the two
distributions that are compared; thus, the higher G the more dissimilar are the two
distributions that are compared. The frequency distributions are normalized consid-
ering the different number of stakeholders in stakeholder groups.
Palo Verde Workshop
Objectives
A 4-day workshop was held at the OTS Palo Verde Biological Field Station, Costa
Rica, from April 24 to April 27, 2012. The goals of this workshop were to: (1) for-
mulate and refine compelling research questions and hypotheses on water sustain-
ability and climate for ensuing research collaborative proposals and infer stakeholder
preferences; (2) define the teams that will prepare research proposals; (3) identify
funding sources; and (4) agree on the mechanisms for communication and integra-
tion among the different working teams.
Participants included 20 researchers from the four participating US universities and
organizations (UF, ASU, Columbia, and OTS) and five Costa Rican collaborators from
UCR, ITCR, MarViva, Texas A&M’s Soltis Center, and ProDesarrollo Internacional.
Activities
On the first day of the workshop, participants were taken on a day-long field trip in
the Tempisque River Basin so that they could get to know the basin first-hand and
become familiar with key components of this system, in terms of water management
and water use. Driving out of the Palo Verde field station, participants were shown
rice and sugarcane fields, the two major agricultural crops grown in the Tempisque
Basin. They were then taken to the Sandillal hydropower generating station, man-
aged by the Costa Rican Electricity Institute (ICE), where they were given an hour-
long tour by ICE staff. This is one of two power generating stations below the
Arenal Dam, and once passed through the station; this water is diverted into the two
primary irrigation canals of the Tempisque Basin at the Miguel Pablo Dengo
Diverting Station, the third stop of our tour. The group was then taken to a large
tilapia fish farm and given a tour of the facilities by its staff, where each phase of
production was explained in detail. Finally, in the afternoon, they were driven to the
Pacific coast (Playas del Coco) to get a feel of the tourism industry (a major compo-
nent in terms of water use in the region). On the way to the coast, they stopped at the
La Guardia gauging station, the only gauging station in the Tempisque River. The
group then returned to the Palo Verde field station in the evening, better prepared to
discuss water management challenges faced in the region the following morning.
From Wednesday to Friday (April 26–28 2012), the group stayed at the Palo
Verde field station. The days consisted of multiple plenary and group break-out ses-
sions to address the goals of the workshop. These included presentations on the
analysis of current conditions in the basin, discussions on the elements of a social
model and defining the long-term goals of the project. Participants also identified
research questions, discussed potential funding sources as well as possible stake-
holder engagement strategies.
Workshop Outputs
Participants were asked to submit (individually or in small disciplinary groups) an

unlimited number of research questions to address one or more of the three main
objectives identified above. A total of 85 questions on a diversity of topics, from
hydrology and natural ecosystems, to social and governance issues were proposed
(these are available at http://www.tc.umn.edu/~matteoc/). These were mapped to the
specific goals of the project, and will form the core of the research grant proposals that
will be generated by this group. They are also mapped to the five thematic groups
(listed above), indicating where additional intergroup coordination will be required. In
the next section, we present a post hoc preliminary analysis based on these questions
to explore further gaps in the project’s research framework and trends of thought.
Elicitation of Stakeholder Preferences
Stakeholder preferences have been analyzed as a function of two categories (i.e.,

topic and time categories) assigned by stakeholders to the questions formulated dur-
ing the workshop. The topic categories are: climate behavior (1), climate on vegeta-
tion (2), governance (3), human impact on natural systems (4), human impacts of
natural systems on climate/production (5), impacts on water system behavior (6),
natural system behavior (7), use/decision processes (8), water system behavior (9),
natural system sustainability (10), water sustainability (11), incentive mechanisms
(12), governance structure (13), institutions (14), laws and policy (15), stakeholders
(16), water use (17). The time categories are: historic trends (1), current status (2),
expected behavior (3).
The selection of a topic or time class independently of the questions represents a
preference formulation of stakeholders. All stakeholders (with the exception of
stakeholders from governmental organizations) selected topic and time classes for
the same set of questions.
Considering the selection of stakeholders, we assessed the frequency distribu-
tions of selected topic and time classes for academics and NGOs. In our case, the
number of stakeholders for each stakeholder group is different; however, this is a
common situation that does not affect the intercomparison of stakeholder groups’
preferences. More participation was observed for academics.
Semantic Network Extraction Model
The inference of the semantic network from a text is useful for assessing the poten-
tial mental model of stakeholders for a given problem. Aggregated mental models
of stakeholders can be useful for the construction of probabilistic decision networks
(i.e., influence diagrams) for the evaluation of policy options through the integra-
tions of data, decisions, and model predictions. Here, we use a textual analysis for
all questions formulated in the Palo Verde workshop. Thus, the analysis is consider-
ing all questions for all the goals assembled together.
The text mining functionality of the model provides support for creating term
maps based on a corpus of a text. In this case, the corpus is composed by all
workshop questions together. A term map is a two-dimensional map in which
terms are located in such a way that the distance between two terms can be inter-
preted as an indication of the relatedness of the terms. In general, the smaller the
distance between two terms, the stronger the terms are related to each other. The
relatedness of terms is determined based on co-occurrences in documents or in
the same text analyzed; this means that two closed nodes (terms) are mentioned
closely in the text.
To create a term map based on a corpus of text, the model distinguishes the fol-
lowing steps:
1. Identification of noun phrases. The approach that we take is similar to what is
reported in papers available in the literature (Van Eck et al. 2010a). We first per-
form part-of-speech tagging (i.e., identification of verbs, nouns, adjectives). The
Apache OpenNLP toolkit (http://incubator.apache.org/opennlp/) is used for this
purpose. We then use a linguistic filter to identify noun phrases. The filter selects
all word sequences that consist exclusively of nouns and adjectives and that end
with a noun (e.g., change, basin, but not variability of climate and highly critical
areas). Finally, we convert plural noun phrases into singular ones.
2. Selection of the most relevant noun phrases. The selected noun phrases are
referred to as terms. The essence of the technique for selecting the most relevant
noun phrase is as follows. For each noun phrase, the distribution of (second-
order) co-occurrences over all noun phrases is determined. This distribution is
compared with the overall distribution of co-occurrences over noun phrases. The
larger the difference between the two distributions (measured using the Kullback–
Leibler distance), the higher the relevance of a noun phrase. Intuitively, the idea
is that noun phrases with a low relevance (or noun phrases with a general mean-
ing), such as change, basin, and new method in this case study, have a more or
less equal distribution of their (second-order) co-occurrences. On the other hand,
noun phrases with a high relevance (or noun phrases with a specific meaning),
such as variability of climate and highly critical areas, have a distribution of
their (second-order) co-occurrences that is significantly biased towards certain
other noun phrases. Hence, it is assumed that in a co-occurrence network noun
phrases with a high relevance are grouped together into clusters. Each cluster
may be seen as a topic. The criterion for a noun phrase to be incl uded in the lexicon
was that a fragment of the noun phrase (e.g., “Basin level”) occurs at least three
times in the text.
3. Mapping and clustering of the terms. We use the unified framework for mapping
and clustering defined in Van Eck et al. (2010b), and in Waltman et al. (2010).
Mapping and clustering are complementary to each other. Mapping is used to
obtain a fairly detailed picture of the structure of a semantic network; while clus-
tering is used to obtain a fairly detailed picture of the clusters of topics in a
semantic network. Note that the clusters are determined by a statistical technique
and not by an a priori delineation of topics. Naturally, it is hoped that the cluster-
ing technique leads to recognizable topics, but it has to be explicitly investigated
whether this is actually the case.
4 . Visualization of the mapping and clustering results. The model can “zoom” and
scroll on a term map, and it allows term search functionality to support a detailed
examination of a term map. Other relevant network variables can be calculated
and represented.
Influence Diagram Construction
The direction of each arrow is assessed by the analysis of the sequence of terms in
the text. The point of the arrow is directed from each term to the most frequent term
in the text that appears after the term considered. The analysis is repeated for each
combination of the most important terms (Table 1). The frequency of the repetition
of the term “A” after the term “B” in the text (all questions together) is calculated.
Thus, the arrow is drawn from A to B. In this way, potential casual relationships
among factors of the system can be assessed.
Network Complexity as Objective Function
The Network Complexity (NC) is introduced here as a metric to characterize the

inferred influenced diagram. NC is used as a metric to monitor the variation of the
influenced diagram under changes in the text provided by the stakeholders. Thus, in
absence of an objective function defined by the stakeholders—in which the nodes of
the influence diagrams are the criteria and the weights are their relative frequency—
we measure the stability of the influence diagram with a topological metric that
characterize its overall structure; a variation in NC reflects a variation of any objec-
tive function. Yet, variations of NC are dictated by variations of stakeholder prefer-
ences. This metric is defined as that is the ratio between network connectivity (A is
the number of connected nodes where V is the total number of nodes) and length (L
is the length of each link that is here defined as the number of words that separate
the connected words i, j in the stakeholder text, where Z is the total number of links).
Table 1 Mapping of all Term Occurrences Relevance

questions formulated in the
Basin level 3 2.14
workshop
Prediction 4 2.09
Current pattern 3 1.81
Institution 7 1.62
Local scale 3 1.56
Sustainability 3 1.56
Approach 4 1.47
Enso 4 1.43
Large-scale atmospheric 4 1.44
driver
Water use 6 1.34
Phase 6 1.25
Local precipitation 6 1.14
Future climate scenario 4 1.01
Statistical relationship 4 0.95
Asadas 3 0.83
Watershed 4 0.79
Tempisque basin 6 0.60
Palo verde wetland 7 0.56
(Pvw)
Water 5 0.52
Hydrology 3 0.48
Impact 6 0.46
Groundwater 3 0.41
Climate change 8 0.33
Change 15 0.28
Terms are listed with their occurrence and relevance.
Low and high relevance terms are noun phrases with a
general and with a specific meaning, respectively. The
occurrence is related to the frequency of terms
NC varies between 0 and 1. Networks with high complexity are characterized by

both high node-node connectedness and small node-node separation. This definition
of NC is equivalent to the definition of network complexity of Bonchev and Buck
(2005). Variations in stakeholder preferences are induced by global sensitivity and
uncertainty analyses assuming a uniform distribution with a standard deviation of
±20 % around the average frequency.
V V
A
ååa
i =1 j =1
ij
NC = = Z Z
L
åål
i =1 j =1
ij

Global Sensitivity and Uncertainty Analyses
The goal of the sensitivity analysis is to identify which variables (or input factors)
in the influence diagram have the highest effect on NC, thus to measure the relative
importance of the variables that constitute any potential objective function that
depends on NC. For the global sensitivity analysis, we adopted the Morris method
(Morris 1991) for screening variable importance by varying all the variables simul-
taneously. The Morris method is composed of individually randomized variable
designed models. Each variable may assume a discrete number of values that are
selected randomly within an allocated range of variation. Then, the Morris method
calculates the importance of each variable and the interaction of each variable with
all the others for NC. The former is the mean of the elementary effect μ* (i.e., the
local derivative of output NC, with respect to input factors for values sampled at
each level of factor Xi in the k-dimensional inputs space), and the latter is the varia-
tion of the mean elementary effect, σ. The elementary effect di(x) for factor Xi is
defined as
y ( x1 ,¼, xi -1 , xi + D, xi +1 , xk ) - y ( x )
di ( x ) =
D
where xi + ∆ is the perturbed value of xi; k is the number of factors, i = 1, …, k.
di(x) is considering the ratio between the variation of the output, y = NC, and the
variation of the input factors, xi. The resulting probability distribution of the
elementary effects of factor Xi is characterized with its mean μ* (absolute val-
ues) and standard deviation σ. Although elementary effects are local measures,
the method, is considered global, as the final measure μ* is obtained by averag-
ing the elementary effects which eliminates the need to consider the specific
points at which they are computed (Saltelli et al. 2005). The higher μ* the higher
the absolute importance of each variable for NC. The number of simulations, N,
required to perform the Morris analysis is given as N = r(k + 1), where r is the
sampling size for each trajectory (r = 10 produces satisfactory results (Saltelli
et al. 2005)). The variables with μ* values close to zero can be considered as
negligible ones. The variables with the largest value of μ* are the most important
variables. However, the value of this measure for a given variable does not pro-
vide any quantitative information on its own and needs to be interpreted qualita-
tively, i.e., relatively to other variables values. The meaning of σ can be
interpreted as follows: if the value for σ is high for a variable, Xi, the elementary
effects relative to this variable are implied to be substantially different from each
other. In other words, the choice of the point in the input space at which an ele-
mentary effect is calculated strongly affects its value. Conversely, a low σ value
for a variable implies that the values for the elementary effects are relatively
consistent, and that the effect is almost independent of the values for the other
input variables (i.e., no interaction). The uncertainty analysis is performed by
assigning a probability distribution to each variable. Thus, the calculation of NC

is repeated via Monte Carlo simulations of the textual analysis model for sets of
values of each variable according to their distribution. In the uncertainty analy-
sis, the probability of occurrence of each variable value is related to the assigned
distribution.
Results and Discussion
The first and easiest analysis to be performed is the analysis of the frequency distri-
bution of the choices made by stakeholders during the workshop. Figure 2a shows
that academics have a predominant preference towards climate behavior (topic 1),
human impacts on natural systems (4), impacts on water system behavior (6), water
sustainability (11), and institutions (14). NGOs and other private organizations have
a very different set of preferences; namely, these preferences are about use/decision
processes (8), law and policy (15), and water use (17). However, NGOs consider
important also the topics 4, 10, 11, 13, and 16 as academics.
As for the time component of the ecosystem management, in Fig. 2b it appears
that NGOs and private organizations strongly believe in investing more analysis and
plans for the future of the TBPV ecosystem (time class 3) rather than investing in
analysis about the past. Academics consider the present situation more important
than the future, even though the difference is minimal. Academics believe more
than NGOs in analysis about the past of the TBPV ecosystem. These preferences
about the future of the basin can prioritize research and development activities, and
increase efforts towards solutions with different time horizons in terms of their
potential effects. Overall, from the workshop questions it seems there is an agree-
ment towards a strategic planning for the future.
The term map derived from textual analysis (Methods) is shown in Fig. 3. The
total terms are 317 and after the minimum threshold for the occurrences (equal to 3)
the total terms are 29. Table 3 shows the occurrence of the 29 terms in the text ana-
lyzed (all questions together) and their relevance. Low relevance is for the terms
with general meaning and vice versa. Thus, relevance should not be confused with
absolute importance of the word in the problem at hand. The overall importance of
each term that can be a factor of the environmental problem considered is captured
by the occurrence of each term.
Terms that are located close to each other in the map often occur together in
the same text, while terms that are located far away from each other do not or
almost not occur together. In general, terms in the center of the map co-occur
with many different terms and are therefore related to various topics. In contrast,
terms at the edges of the map tend to co-occur only with a small number of other
terms. Terms at the edges therefore often belong to relatively isolated fields. The
color of a term indicates the cluster to which the term has been assigned, and the
size of a term indicates the frequency with which the term occurs in the editorials.
Fig. 3 Inferred mental model from textual analysis. The visualized network does not take in
account the sequence of words in workshop’s questions. Node distance and color are related to the
relevance and occurrence in Table 1, respectively
The color of an item is determined by the score of the item, where by default
colors range from blue (score of 0) to green (score of 1) to red (score of 2). The
size of a cluster in the map is influenced by many factors (e.g., the number of
terms in the cluster, the frequency of occurrence of the terms, and the strength
with which the terms are related to each other) and therefore does not have a
straightforward interpretation. The density of an area in the map is determined
by the number of terms in the area and by the frequency with which the terms
occur in the text.
Of great importance is the potential influence diagram shown in Fig. 4. The
influence diagram is built from the term network in Fig. 3 by considering the
occurrence and frequency of pairs of terms in the text. The width of a link in the
influence diagram is proportional to the frequency of the pair of terms that are
connected. The distribution of nodes (terms) is random, so the length of links has
no meaning. The direction of the arrow is related to the sequential appearance of
terms in the text. In this case, the inferred causal relationships are related to all
stakeholders. In fact, we investigated the text of answers of all stakeholders for
the questions formulated during the workshop. Hence, the semantic network in
Fig. 4 can be used as an influence diagram for preliminary modeling of the TBPV
ecosystem problem. This can be done after assigning the marginal and conditional
probability distribution functions.
Fig. 4 Probabilistic decision network for the TBPV ecosystem problem. The influence diagram is
assessed by considering the terms/factors of the TBPV ecosystem workshop and the order in which
they occur in the text. In general, the smaller the distance between two terms, the stronger the terms
are related to each other
Fig. 5 Mental models of academics and NGOs. The influence diagram is assessed by considering the terms/
factors of the TBPV for the two different groups of stakeholders: NGO and private, and academics
Fig. 6 Network complexity and global sensitivity and uncertainty analyses. Network Complexity
(NC) (upper plot) is a metric to characterize the inferred influenced diagram and to monitor the
variation of such diagram under perturbations in socio-ecological factors provided by stakeholders.
Perturbation effects are assessed by global sensitivity and uncertainty analyses (GSUA) that reveal
factor importance and interaction of ecosystem factors for NC as output variable (bottom plot)
Conclusions
The consideration of stakeholder preferences, objectives, and mental models is a

worthwhile effort for the analysis and management of complex ecosystems. We
show that by realizing workshops of stakeholders it is possible to use workshop
material to infer stakeholder preferences and their knowledge about the ecosystem
for which different management strategies are evaluated. Certainly, mental model-
ing is a costly effort, but the payoff of this effort is also related to engage
stakeholder since the very beginning of the planning process that makes easier
future communication of results, request of feedbacks and/or more data, and build-
ing community capacity. Community capacity is also education of stakeholders to
unknown problem and trade-offs among ecosystem factors and needs. Moreover,
mental modeling and textual analysis of workshop products facilitate model con-
structions and the integration of models of different research groups. We emphasize
that ecosystem and society should coexist and the proposed direct and indirect men-
tal modeling effort is a preliminary tool to enhance this linkage. In the Costa Rica
case study, we show that academic stakeholders are more focused on the current and
past dynamics of natural and human processes, while NGO stakeholders are focused
on the future socio-legal aspects of ecosystem management. Mental modeling is a
way to unify these parts which need to be address together necessarily for the sus-
tainability of ecosystems.
Acknowledgment Funding from the NSF U.S.-Costa Rican Workshop: “Interdisciplinary work-
group on water sustainability in the Tempisque Basin, Palo Verde NP, Costa Rica” (April 2012) are
gratefully acknowledged.
References
Birkel C, Demuth S (2006) Drought in Costa Rica: temporal and spatial behaviour, trends and the
relationship to atmospheric circulation patterns. In: Demuth S, Gustard A, Planos E, Seatena F,
Servat E (eds) Climate variability and change: hydrological impacts. IAHS Press, Wallingford,
pp 338–343
Bolaños R, Watson V (1999) Mapa ecológico de Costa Rica 1:200,000: Según el sistema de zonas
de vida de Holdridge. Centro Científico Tropical, San José
Bonchev D, Buck GA (2005) Quantitative measures of network complexity. In Bonchev D,
Rouvray D (eds) Complexity in chemistry, biology, and ecology, Chapter 5. pp 191–235. http://
dx.doi.org/10.1007/0-387-25871-x_5. Accessed 6 June 2013
Convertino M, Valverde LJ Jr (2013) Portfolio decision analysis framework for value-focused
ecosystem management. PLoS One 8(6):e65056. doi:10.1371/journal.pone.0065056
Convertino M, Baker K, Lu C, Vogel JT, Suedel B, Linkov I (2013) Use of multi-criteria decision
analysis to guide metrics selection for ecosystem restorations (PDI-1541). Ecol Indic 26:76–86
Convertino M et al (2012) Decision evaluation in complex risk network systems (DecernsSDSS)
demo and Black River case study. doi:10.1594/PANGAEA.776746
Daniels AE, Cumming GS (2008) Conversion or conservation? Understanding wetland change in
northwest Costa Rica. Ecol Appl 18, pp. 49–63
Jiménez JA, Jiménez EG, Mateo-Vega J (2001) Perspectives for the Integrated Management of the
Tempisque River Basin, Costa Rica. Organization for Tropical Studies: San José, C.R.
Jones NA, Ross H, Lynam T, Perez P, Leitch A (2011) Mental models: an interdisciplinary synthe-
sis of theory and methods. Ecol Soc 16(1):46
Kim H, Andersen DF (2012) Building confidence in causal maps generated from purposive text
data: mapping transcripts of the Federal Reserve. Syst Dyn Rev 28(4):311–328. doi:10.1002/
sdr.1480, Published online 17 October 2012 in Wiley Online Library (wileyonlinelibrary.com)
Kim H (2009) In search of a mental model-like concept for group-level modeling. Syst Dyn Rev
25(3):207–223. doi:10.1002/sdr.422, Published online 21 August 2009 in Wiley InterScience
(www.interscience.wiley.com). Copyright © 2009 John Wiley & Sons, Ltd
Linkov I, Cormier S, Gold J, Satterstrom FK, Bridges T (2012) Using our brains to develop better
policy. Risk Anal 32(3):374–380. doi:10.1111/j.1539-6924.2011.01683.x
Morgan MG, Fischoff B, Bostrom A, Atman CJ (2002) Risk communication: a mental models
approach. Cambridge University Press, Cambridge
Morris M (1991) Factorial sampling plans for preliminary computational experiments. Technomet-
rics 33(2), 161–174
Murcia C, Losos E, Muñoz-Carpena R, Albertin AR, Graham WD, Huffaker R, Kiker GA, Waylen
PR. 2012. Collaborative Research Catalyzing New International Collaborations:
Interdisciplinary workgroup on water sustainability in the Tempisque Basin, NW Costa Rica—
End of year one report. National Science Foundation
Reynolds TW, Bostrom A, Read D, Morgan MG (2010) Now what do people know about global
climate change? Survey studies of educated laypeople. Risk Anal 30(10):1520–1538
Saltelli A, Ratto M, Tarantola S, Campolongo F (2005) Sensitivity analysis for chemical models.
Chemical Reviews 105(7), 2811–2828. doi:10.1021/cr040659d
Sparrevik M, Barton DN, Oen AM, Sehkar NU, Linkov I (2011) Use of multicriteria involvement
processes to enhance transparency and stakeholder participation at Bergen Harbor, Norway.
Integr Environ Assess Manag 7(3):414–425. doi:10.1002/ieam.182
Van Eck NJ, Waltman L, Noyons ECM, Buter RK (2010a) Automatic term identification for bib-
liometric mapping. Scientometrics 82(3):581–596
Van Eck NJ, Waltman L, Dekker R, Van den Berg J (2010b) A comparison of two techniques for
bibliometric mapping: multidimensional scaling and VOS. J Am Soc Inf Sci Technol
61(12):2405–2416
Walker B, Gunderson L, Kinzig A, Folke C, Carpenter S, Schultz L (2006) A handful of heuristics
and some propositions for understanding resilience in social-ecological systems. Ecol Soc
11(1):13
Waltman L, Van Eck NJ, Noyons ECM (2010) A unified approach to mapping and clustering of
bibliometric networks. J Inform 4(4):629–635
Wood MD, Bostrom A, Bridges T, Linkov I (2012a) Cognitive mapping tools: review and risk
management needs. Risk Anal 32(8):1333–1348, Early View
Wood MD, Kovacs D, Bostrom A, Bridges T, Linkov I (2012b) Flood risk management: US Army
Corps of Engineers and layperson perceptions. Risk Anal 32(8):1349–1368, Early View
Wood MD, Bostrom A, Convertino M, Kovacs D, Linkov I (2012c) A moment of mental model clar-
ity: response to Jones et al. 2011. Ecol Soc 17(4):7, http://dx.doi.org/10.5751/ES-05122-170407
Environmental Performance or Productivity
Loss?
Shital Sharma
Abstract The principle of sustainable growth and development has influenced

much of the environmental policies that have come into existence in the last three
decades in the USA. Since the establishment of the U.S. Environment Protection
Agency (EPA) in 1970, the regulations put in place have been increasingly stringent
on the environmental standards they set over time. For instance, the Clean Air Act
introduced in 1963 and the Clean Water Act founded in 1948 have been amended
multiple times, each time tightening the control on the type and amount of emission
allowed into the environment. This has no doubt accrued much benefit to the society
within the USA in the form of reduced morbidity, increased recreational opportu-
nity, cleaner living environment, increased ecosystem vitality, and possible increased
land values (Palmer et al. 1995). Such outcomes are essential for an environmen-
tally sustainable future and are also the cornerstone of a society that holds itself
accountable to future generations. However, at the same time concerns have also
been raised as to whether these benefits are worth the cost of such regulations. In
addition to the direct costs of pollution abatement, proponents of this view have
blamed stifled economic growth, decline of labor and capital productivity, as well as
loss of jobs on such increasingly stringent environmental regulations.
Introduction
The principle of sustainable growth and development has influenced much of the
environmental policies that have come into existence in the last three decades in the
USA. Since the establishment of the U.S. Environment Protection Agency (EPA) in
1970, the regulations put in place have been increasingly stringent on the environ-
mental standards they set over time. For instance, the Clean Air Act introduced in
1963 and the Clean Water Act founded in 1948 have been amended multiple times,
each time tightening the control on the type and amount of emission allowed into
the environment. This has no doubt accrued much benefit to the society within the
S. Sharma (*)
Clark University, Worcester, MA, USA
e-mail: shsharma@clarku.edu

DOI 10.1007/978-3-319-25439-5_3
40 S. Sharma
USA in the form of reduced morbidity, increased recreational opportunity, cleaner

living environment, increased ecosystem vitality, and possible increased land values
(Palmer et al. 1995). Such outcomes are essential for an environmentally sustain-
able future and are also the cornerstone of a society that holds itself accountable to
future generations. However, at the same time concerns have also been raised as to
whether these benefits are worth the cost of such regulations. In addition to the
direct costs of pollution abatement, proponents of this view have blamed stifled
economic growth, decline of labor and capital productivity, as well as loss of jobs
on such increasingly stringent environmental regulations.
Such concerns have led many studies to examine the costs of pollution abatement
over the years finding varying degree of impacts on productivity. While research
like Denison (1979) that uses abatement cost survey data and studies like Gray
(1987) and Barbera and McConnell (1986) that use industry level data have found
some effect of regulation on productivity, studies using plant-level data like Gollop
and Roberts (1983), Gray and Shadbegian (1995), Shadbegian and Gray (2005),
Färe et al. (1989), and Boyd and McClelland (1999) have found even greater
declines in productivity due to regulations. Brännlund et al. (1995) was among the
initial empirical studies that explored the effect of regulation on profitability of
plants using Data Envelopment Analysis (DEA).1 Some other contributions to the
literature using this method include Boyd and McClelland (1999), Färe et al. (2007),
Shadbegian and Gray (2006), and Aiken et al. (2009) all of which have found vary-
ing degrees of effect of regulation on productivity.
Proponents of environmental sustainability and improved environmental
outcomes have also pointed out that this inverse relationship between environmen-
tal regulation and productivity is inevitable due to the traditional methods used to
measure productivity (Repetto et al. 1997). This has led to introduction of methods
to measure productivity that account for the reduction in emissions that occur due
to the diversion of resources from production to emission reduction. Some of such
methods that account for environmental performance are the additive inverse
approach (Koopmans 1951; Berg et al. 1992), the multiplicative inverse approach
(Golany and Roll 1989; Knox Lovell et al. 1995), the measure of hyperbolic
efficiency that uses the weak disposability assumption (Färe et al. 1989), and the
directional distance function approach (Chung et al. 1997). A need to account for
such environmental outcomes is necessary for multiple reasons. First, this allows us
to measure the efficiency of plants in their ability to reduce emissions when faced
with environmental regulations. Second, this also allows us to observe if the
resources being diverted from production are being as efficiently used as they would
otherwise be if regulations were not in place. Since one of the goals of environmen-
1
Data Envelopment Analysis (DEA) was proposed by Farrell, M.J. 1957. “The Measurement of
Productive Efficiency.” Journal of the Royal Statistical Society. Series A (General), 120(3), 253–
90. and later operationalized by Charnes, A.; WW Cooper and E. Rhodes. 1978. “Measuring the
Efficiency of DMU.” European Journal of Operational Research, 2(6), 429–44. Variable returns to
scale to the estimation method was later added by Banker, R.D.; A. Charnes and W.W. Cooper.
1984. “Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment
Analysis.” Management science, 1078–92.
Environmental Performance or Productivity Loss? 41
tal regulations is to push the society towards an environmentally sustainable future,

obtaining these measures allows us to gauge the effectiveness of such regulations in
achieving their objective.
The main focus of this paper is to study the link between environmental regula-
tion, plant-level productive efficiency, and environmental efficiency in two US man-
ufacturing industries: oil refineries and pulp and paper mills using DEA models.
The paper specifically investigates how the measure of output loss arising out of a
need to comply with environmental regulation changes when environmental perfor-
mance is accounted for in the measurement of productive efficiency. Data on emis-
sions and abated emissions are used in different frontier models to study outcomes
on plant productivity when emissions are regulated and unregulated.
The paper uses plant-level data on emission of solid waste and abated emission of
certain pollutants, Particulate Matter (PM) and Sulfur Dioxide (SO2) between 1974
and 1982 from the U.S. Environmental Protection Agency’s (EPA) Pollution
Abatement Costs and Expenditure (PACE) survey. This provides information on
environmental efficiency in the early stages of environmental regulation.2 In addition
to using data on initial years of regulation in the USA, the paper uses additional data
from 1994 to 2000 on the pulp and paper and oil refining industries obtained from the
Annual Survey of Manufactures (ASM) and Census of Manufactures (CMF), allow-
ing a comparison of changes in productivity in response to regulation over time.
Finally, measures of water pollution from the Permit Compliance System (PCS)
namely Biological Oxygen Demand (BOD) and Total Suspended Solids (TSS) and
measures of solid waste emission from EPA’s Toxic Release Inventory (TRI) are also
obtained for the period 1994–2000. Analysis of productivity in existence of regulation
using this data sheds light on how the effect of regulation on productivity behaves over
a longer period of time. It also allows us to extend the general results of the research
beyond the first few years in which PACE data is being used for the analysis.
The traditional measure of productivity loss is measured using the weak dispos-
ability assumption which ensures that a cost of reduction in the desirable output y is
imposed due to the reduction of the undesirable output z by the parameter θ.
Introduction of a constraint on the amount of pollutant emitted will indicate the pro-
ductivity loss arising out of environmental regulations here. This traditional method is
then modified to study how the measure of productivity can change when environmen-
tal concerns are central to research. Productivity loss is calculated when performance
on reduction of emissions is taken into account in addition to production of output.
This empirical paper contributes to the extant literature by studying the effects of
regulation on plant-level productivity during the 1970s and 1990s. The paper finds
that plants suffer a reduction in productivity as plants are forced to devote resources
on abatement due to restrictions on emissions from environmental regulation.
Though the inclusion of environmental outcomes reduces such costs to plants, they
2
Environmental Regulations have been around since the 1940s but the bulk of regulations we see
today such as The Clean Air Act and The Clean Water Act were introduced and enforced starting
around 1970s when EPA was established. This brings the data used in this paper closer to the time
when the costs of regulations started rising considerably making the study pertinent.
42 S. Sharma
are not observed to be as efficient in emission reduction as they are in productive

activities. This implies that a win–win outcome from regulations may not exist for
plants and that an increase in efficiency in abatement practices is needed for such
win–win outcomes to exist. Efficiency increases could arise in time from plants
learning to adjust to regulations, plants innovating in their compliance efforts, and
transfer of information and technology on best practices in complying with regula-
tions from the most efficient plants to their less efficient peers. The paper also makes
a comparison of the costs of regulation in terms of reduced productivity to its ben-
efits in terms of reduced mortality. Results from this comparison suggest that the
benefits of regulations are far greater than its costs. This implies that environmental
regulations make economic sense and can be an effective form of government pol-
icy in the pursuit of an environmentally sustainable future.
The rest of the research is organized as follows. The next section reviews the
literature and outlines the possible contributions this research makes. This is fol-
lowed by the methods section that details the empirical methods of estimation used
in this analysis. The subsequent sections describe the data, present the empirical
results, and finally conclude with a discussion of the results.
Literature
In the past few decades, many academic studies have examined the regulation pro-
ductivity relationship. Denison (1979) uses a growth accounting approach and finds
that the average annual impact of regulation on productivity ranges from 0.05 per-
centage points to 0.22 percentage points over the period 1967–1978. Norsworthy
et al. (1979) finds that a decline in non-abatement capital formation leads to produc-
tivity decline. A time series regression is used by Christainsen and Haveman (1981)
to indicate that 12–21 % of the decline in productivity growth of labor can be
explained using regulations for the period from 1973 to 1977. Gollop and Roberts
(1983) finds that production costs considerably increased due to the restrictions
placed on emissions. Gray (1987) uses Occupational Safety and Health
Administration (OSHA) and EPA information on regulation enforcement from the
entire manufacturing sector to find that about 30 % of the productivity slowdown
can be attributed to regulations. Similar effects have also been found by Barbera and
McConnell (1986). Plant-level study of the impact of carried out by Gray and
Shadbegian (1993) indicate that plant productivity levels as well as productivity
growth rates are lower for plants that are regulated as opposed to those that are
not regulated. An extension of this study by Gray and Shadbegian (1995) finds
smaller effects of regulation on productivity than previously obtained. Papers using
DEA methodology to study this relationship like Brännlund et al. (1995), Boyd and
McClelland (1999), Färe et al. (2007), and Aiken et al. (2009) also find similar
effects of regulation on productivity at varying degrees.
While the methods used by papers like Brännlund et al. (1995) succeed in study-
ing the impact of regulation on productive efficiency, their model allows for an
unlimited increase in the undesirable outputs. Such models do not consider the envi-
ronmental outcome arising out of the diversion of resources from production efforts
to efforts on abatement of pollution. Proponents of environmental sustainability
argue that if reduction in emission as a result of compliance efforts is measured as a
desirable outcome for the plants, the final measure of output loss due to regulation
could change.
One of the models used for such an analysis is the hyperbolic efficiency model.
It was developed and first used by Färe et al. (1989). An analysis of paper mills in
the USA in 1976 by the paper suggests that when firms are credited for reduction in
emissions, the ranking of the efficient firms change quite significantly suggesting
that disregarding the reduction of emissions in efficiency analysis could potentially
distort the results of any analysis. The paper also finds a sizeable impact of regulation
on the output of the firms in the sample. Similar results are also found by Boyd and
McClelland (1999). Zofio and Prieto (2001) uses data on the manufacturing indus-
tries of 14 OECD countries to look at how successfully the hyperbolic models can
measure environmental performance and efficiency. Results indicate that in cases
where the weak disposability of the undesirable outputs is binding, the hyperbolic
model does show differences in efficiency levels due to regulation. An analysis of
such differences for several industries over a longer period of time could shed light
on how environmental performance might have “replaced” the “traditional” produc-
tivity when the latter is seen to decrease due to regulation.
There have been many more attempts at modeling DEA problems with undesir-
able outputs using this idea of crediting firms for a reduction in their emissions. For
instance, the inverse multiplicative model and the additive inverse model represent
some of such attempts. Scheel (2001) sheds light on their drawbacks and presents a
model of its own. The “non-separating” approach introduced in this paper indicates
that the firms would be considered much less efficient if the decision of increasing
desirable output and the decision to decrease undesirable output were analyzed
together rather than separately.
A final approach that has been widely used for efficiency analysis which includes
undesirable outputs is the directional distance function approach initially introduced
by Chung et al. (1997). This approach allows for the measurement of a Malmquist-
type productivity index that can be decomposed into efficiency change and techno-
logical change such that performance of firms can be studied over a period of time.
This method has also been recently used to study environmental performance or
eco-efficiency by Zhang et al. (2011), Wang et al. (2013), and Halkos and Tzeremes
(2013). A similar model has also been used by Ball et al. (2005) to study biases in
the “traditional” productivity measures when environmental performance is not
considered. A panel state-by-year data from the US agricultural sector is used to
determine that the traditional measure of productivity is biased upwards when the
production of undesirable outputs is increasing and biased downward when the pro-
duction of the same is decreasing. The study is done using cost data rather than the
usual quantity data that is used for such DEA analysis.
Following the argument by Porter (1991) and Porter (1996), some studies have
tried to empirically test whether there could be a win–win outcome from environ-
mental regulations. While studies like Telle and Larsson (2007) find significant
win–win results, others like Cordeiro and Sarkis (1997) and Hart and Ahuja (1996)
44 S. Sharma
find a win–lose result arising out of environmental regulations. This paper contrib-
utes to the literature with a plant-level empirical test of the win–win possibilities for
pulp and paper mills and oil refineries in the USA.
Method of Estimation
Traditionally, the most basic form of productivity or efficiency measurement has

been the ratio of output to input. Though this basic approach allows us to understand
the idea of efficiency measurement, it does have its drawbacks. First, the model
struggles to include multiple inputs and outputs. The model runs into additional
problems when the inputs being used or outputs being produced are measured in
varying units. A need for a better model was fulfilled by Farrell (1957) identifying
that “… it is important to know how far a given industry can be expected to increase
its output by simply increasing its efficiency, without absorbing further resources.”
The efficiency of plants can be measured using distance functions.
The output distance function on the output set P(X) can be defined as:
ì æqö ü
d 0 ( x,q ) = min íd : ç ÷Î P ( X ) ý where the output set P(x) represents the set of all
î è d ø þ
output vectors q that can be produced using input vector x. If the output belongs to
the production possibility set, i.e., q Î P ( x ) , then d 0 ( x,q ) £ 1. The distance will
equal unity if q belongs to the frontier of the production possibility set. Using this
notion of output distance function, technical efficiency can be defined as: TE = d 0 .
The input distance function on the input set L(q) can be defined as:
ì æxö ü
d 0 ( x,q ) = max ía : ç ÷Î L ( q ) ý where the input set L(q) represents the set of all
î è ø a þ
input vectors x that can produce the output vector q. If the input belongs to the input
set, i.e., x Î L ( q ) , then d 0 ( x,q ) ³ 1. The distance will equal unity if q belongs to the
frontier of the input set. Using this notion of input distance function, technical effi-
ciency can be defined as: TE = d 0 .
The input and output approaches described above assume that the production
function for the producer is known. In most of the cases, the production function is
either not known or too complicated to calculate. In such cases, a nonparametric
piecewise linear and convex frontier can be created using points from the data such
that no plant lies above or to the right of the frontier (The condition is reversed in the
input approach). In this manner, the frontier envelops all the data points present in the
dataset such that the frontier can be compared to any other plant within the dataset.
To explain the Farrell efficiency model (Farrell 1957), assume there are n plants
with plant j (j = 1, …, n) that uses p inputs xij (i = 1, …, p) and produces q outputs yrj
(r = 1, …, q). If the weights or prices of the inputs and outputs i and r are known to
be vi and ur, respectively, then the conventional efficiency measure e can be calcu-
åur yrj
lated as: e = r . The Farrell measure that defined the weights as costs for
åvi xij
i
inputs and prices for outputs were a bit restrictive since cost and price data is not
always available to researchers. It was Charnes et al. (1978) that operationalized the
model which later came to be known as DEA. The Charnes et al. (1978) model
(CCR) calculated the technical efficiency for firm a by means of the following frac-
tional programming problem:
ea = max åu y / åv x
r
r ra
i
i ia
s.t. åu y - åv x
r
r rj
i
i ij £ 0, "j
ur , vi ³ e , "r , j
where ε is a non-Archimedean value to restrict the weights to be strictly positive. To
further develop the model, some transformations in the variables need to be made.
Let m r = tur and n i = tvi where t = ( åi vi xia ) . Using the theory of fractional pro-
-1
gramming, the problem above (11) can be now transformed into the following linear
programming problem:
ea = max åm y
r
r ra
s.t. ån x = 1
i
i ia
åm y - ån x
r
r rj
i
i ij £ 0, "j
m r ,n i ³ e , "r , j
The problem can also be stated by duality of the linear programming problem as
follows:
æ ö
ea = min q a - e ç åsr+ + åsi- ÷
è r i ø
s.t. ål j xij + si- = q a xia , i = 1, ¼, p
j
ål y
j
j rj + sr+ = yra , r = 1, ¼, q
l j , si- , sr+ ³ 0, "i, r , j

q a unconstrained
where θa is the measure of efficiency or inefficiency for firm a.
This CCR model has been worked on and developed by many researchers since
1978 to make it applicable to a wide variety of cases. For instance, the method has been
used to measure efficiency in the banking industry (Asmild et al. 2004; Havrylchyk
2006; Rezitis 2006; Staub et al. 2010; Fethi and Pasiouras 2010; Barth et al. 2013), in
the health care industry (Hollingsworth et al. 1999; Hollingsworth 2003, 2008; De
Nicola et al. 2012; Lin et al. 2013), in the electric utilities sector (Sueyoshi and Goto
2001; Arocena 2008; Nemoto and Goto 2003; Pacudan and De Guzman 2002; Vaninsky
46 S. Sharma
2006), in the transportation sector (Nolan et al. 2002; Karlaftis 2004; Markovits-
Somogyi 2011; Shen et al. 2012), and in the police force (Gorman and Ruggiero 2008;
Thanassoulis 1995; Verschelde and Rogge 2012) among many more.
One of the advantages of DEA is that it allows for operational friction and does
not rely on a theoretical production function to obtain a measure of efficiency. It
also provides an easier way to measure efficiency in a multi-output and multi-input
case, especially when inclusion of undesirable outputs is necessary as in this
research paper. However, critiques have pointed to the influence of measurement
errors or imprecise data on the results as a drawback of the approach. A discussion
of the advantages and disadvantages of using DEA as opposed to using regression
methods to measure efficiency is provided by Thanassoulis (1993) and Bowlin et al.
(1984). A comparison with other multi-criteria decision-making methods is also
provided by Velasquez and Hester (2013).
This research employs the DEA method to study the relationship between envi-
ronmental regulations, productivity, and environmental performance. The DEA
method constructs a frontier using the linear combination of the most efficient pro-
duction points in the data and then measures inefficiency of other production points
as the distance from these points to the frontier. Such distance from the frontier to
each production point is measured using various different types of distance func-
tions, the most common of which is the Shephard’s distance function (Shephard
et al. 1970). Among the range of distance function approaches, the directional dis-
tance function approach (Chung et al. 1997) is used here.
For the problems considered in this research, the following general definition of
the production technology is used in describing all estimation models. Assume that
there are n firms with firm j (j = 1, …, n) that uses p inputs xij (i = 1, …, p), produces
q desirable outputs yrj (r = 1, …, q), and t undesirable outputs zsj (s = 1, …, t).
The production technology can then be modeled using the output set P(x) denoted
as:
P ( x ) = {( y,z ) : x can produce ( y,z )}

The following assumptions are made:
if ( y,z ) Î P ( x ) and y˘ £ y Þ ( y˘ , z ) Î P ( x )
if ( y,z ) Î P ( x ) and 0 £ q £ 1, then (q y, q z ) Î P ( x )

if ( y ,z ) Î P ( x ) and z = 0, then y = 0

The first assumption imposes strong disposability of the desirable output such that
the output y can be disposed of at no cost to the plant. The second assumption
imposes the weak disposability of the undesirable output such that a reduction in z
by the factor θ comes at a cost of a reduction in y of the same factor θ. The final
assumption enforces null-jointness on the production of the two outputs y and z.
This means that a firm cannot produce the desirable output y without producing the
undesirable output z. The directional distance function for output set P(x) can be
{ }
defined as: D0d ( x, y, z : g ) = sup b : ( ( y,z ) + b g ) Î P ( x ) . Note that this distance
function scales the desirable and undesirable outputs according to the direction vec-
tor g allowing more flexibility to researchers in how they want to measure
efficiency.
Given this production technology, the weak disposability assumption imposes a
cost of loss of output when the emissions are restricted. The evaluation of such
“traditional” productivity loss is made by calculating the following two efficiency
measures at constant returns to scale. The linear programming problem in (1)
assumes that there is no restriction on the undesirable output z. This model as such
allows plants to emit as much pollutants as they want.
E1 = max ba
s.t. ål x
j
j ij £ xia , i = 1, ¼, p
(1)
ål y
j
j rj ³ (1 + b a ) yra , r = 1, ¼, q
l j ³ 0, " j

E 2 = max ba
s.t. ål x
j
j ij £ xia , i = 1, ¼, p
ål y
j
j rj ³ (1 + b a ) yra , r = 1, ¼, q (2)
ål z
j
j sj = zsa , s = 1, ¼, t
l j ³ 0, " j

The problem in (2) on the other hand imposes a restriction on the amount of emis-
sion. This restriction implies that the amount of pollutants the plants emit is exactly
equal to the amount that they are required to emit under regulation. This restriction
is also associated with a weak disposability of the two outputs.
The ratio of the productivity measures obtained from (1) and (2) can be used to
calculate the cost of pollution abatement in terms of output loss. To bring the esti-
mation in line with Shephard’s distance functions,3 we calculate the following:
æ 1 ö
ç ÷
1 + E1 ø
Output loss = 1 - è (3)
æ 1 ö
ç ÷
è 1 + E3 ø
3
The relationship between efficiency scores from the directional distance function approach and
æ 1 ö
the Shephard’s distance function approach is D0 = ç d ÷
as presented in Färe, R. and
è 1 + D0 ø
S. Grosskopf. 2000. “Theory and Application of Directional Distance Functions.” Journal of
Productivity Analysis, 13(2), 93–103.
48 S. Sharma
If there is no restriction imposed on emission by regulations, the efficiency scores

obtained from the two models will be equal and the output loss function will be equal
to 0. On the other hand, if the additional binding restriction on model (2) yields a dif-
ferent result, this indicates some effect of regulations on productive efficiency. The
measure of output loss is greater than 0 and represents a measure of the output lost due
to the imposition of regulation. For instance, if the measure of output loss is 0.11, then
the amount of output lost due to regulation would be about 11 % of current outputs.
This traditional measure of productivity loss, however, does not regard the reduc-
tion in emissions due to compliance with regulations as a desirable output. In doing
so, the loss measure obtained does not reflect any information on the environmental
performance of plants. A modification of the previous linear programming problem
is presented next. Just as the previous models measure how far-off plants are from
their best performing peers in terms of production of desirable outputs, this model
measures the performance of plants compared to their best performing peers in
terms of production of both the desirable and the undesirable outputs.
First consider (1) and (2). In (1), there are no restrictions imposed on emissions and
efficiency measured by the coefficient β reveals the degree to which output can be
increased without increasing the use of inputs. The use of the weak disposability
assumption is made in (2) and β here measures the extent to which output can be
increased when there is a constraint on the amount of pollutants that can be emitted.
Now consider (4). The coefficient βa is present in both the restriction for the
desirable output y and the restriction for the undesirable output z. The difference is
that while the (1 + βa) on the desirable output restriction measures the extent to
which the production of desirable output can be expanded by (βay) without an
increase in inputs, the (1 − βa) on the undesirable output restriction measures the
extent to which the production of undesirable output can be contracted by (βaz)
without any increase in the use of inputs.4 Thus, this estimation method not only
measures where plants are in terms of production of output but also where they are
in terms of reduction of emission compared to their best performing peers.
E 3 = max ba
s.t. ål x
j
j ij £ xia , i = 1, ¼, p
ål y
j
j rj ³ (1 + b a ) yra , r = 1, ¼, q (4)
ål z
j
j sj = (1 - b a ) zsa , s = 1, ¼, t
l j ³ 0, " j

In a general sense, the directional distance function approach allows measurement
of such potential expansions and contractions in any direction. The direction of (y,
−z) is chosen to conform to the measures of efficiency obtained using Shephard’s
distance functions. A direction of (1, −1) could also be chosen which would allow
4
An analogous model can also be used for abated emissions but is not presented here for brevity. I
leave it upon the reader to adjust the model here for such a purpose.
us to interpret the results in terms of the units in which the desirable and the undesir-
able outputs are measured.
The basic premise in (4) is that if environmental concerns are central to a
research, then the measurement of “traditional” productivity which accounts for the
potential to increase outputs produced using a constant amount of inputs is not
adequate. If a plant is using inputs to produce both desirable outputs y and reduction
in undesirable outputs z, then both have to be accounted for in the measurement of
productivity. Increases in abatement inputs might lead to what appears to be a pro-
ductivity loss just because reduction in emissions, which is an output from the use
of abatement inputs, is not accounted for in productivity measurement. This can be
adjusted by measuring where plants are in terms of their level of emissions com-
pared to their most efficient peers in addition to measuring traditional productive
efficiency. Efficiency scores obtained from (4) can be compared to those obtained
from (1) to study if they are different and if the ranking of the plants change signifi-
cantly when efficiency measurement accounts for plants’ performance in terms of
emissions in addition to production of output.
A new measure of gain in outputs can then be calculated as:
æ 1 ö
ç ÷
1 + E5 ø
Output gain = è (5)
æ 1 ö
ç ÷
è 1 + E3 ø
If the efficiency scores obtained from (1) are equal to the scores obtained from (4),
then this indicates that there are no gains when plants’ performance on emissions are
accounted for in efficiency measurement and the gain function in (5) will be equal to
1. If on the other hand the two scores are different, such difference implies gains from
accounting for plants’ performance in terms of emissions. Since these gains are not
included while measuring “traditional” productivity and output loss, they can now be
adjusted to the output loss arising out of regulation initially measured from (3).
A more intuitive approach to the estimation methods presented above follows
next and is illustrated in Fig. 1. Let us define the production set as P(x) such that
P(x) is the set of all outputs (y, z) that can be produced using the input vector x
where y is the desirable output and z is the undesirable output.
The following assumptions are made:
1. Strong disposability of the desirable output y. For instance, if production point S
is possible, then production point S1 is also possible without the plant incurring
a cost of reduction in Y.
2. Weak disposability of the outputs. For instance, if production of the undesirable
output Z is to be reduced by the factor θ, then the desirable output should also be
reduced by the same factor to impose a cost of reduction in emissions. This is
shown by production points S and θS.
3. Reaching the production point S2 from the production point S is not possible
because of weak disposability. (i.e., Z cannot be reduced with an increase in Y or
Y has to decrease with a decrease in Z.)
4. Possibility of inaction is also shown in the figure. This means that if Y = 0, then Z = 0.
50 S. Sharma
B X T C
Y S2 W
S
U
θS V
A
P(x)
D
S1
g
O M N E Z
Z
Fig. 1 Directional distance function
Construction of P(x) provides a frontier OABCDE where each point on the fron-
tier is the maximum amount of desirable output Y given different amounts of Z and
constant inputs with input vector X. Let us now assume that the unregulated equilib-
rium for a particular plant is at point S. The efficiency Ea for this plant is measured
towards the frontier in the direction of the output using the distance between S and
X.5 The efficiency scores like Ea range from 0 to 1 with 1 being most efficient and 0
being least efficient.
Now, if plants become regulated such that their emission level is restricted at Z ,
then the plants will be forced to produce at the production point θS. This is because
of the weak disposability assumption that requires the desirable output to be scaled
back by the same factor as the undesirable output Z. The efficiency Eb of the plant
can then be measured using the distance between θS and W. Here, Ea > Eb (or
SX < θSW) making the unregulated production point S more efficient than θS. The
efficiency loss arising out of regulation can be measured by calculating the differ-
ence between Ea and Eb.
However, the production point S is not observed and no data is available for the
unregulated regime. Data is only available for the regulated production point
θS. Therefore, to observe the efficiency loss arising out of regulation, the restriction
imposed on the undesirable output at Z is lifted. Removing this restriction, a differ-
ent production point V is obtained where the level of output is the same as that of
the point θS and the input vector X remains the same but the amount of emission is
higher. This is done using linear programming such that the distance between V and
the frontier is maximized. A measurement of efficiency can then be made using the
5
To be precise, efficiency is measured as the ratio SX/MX but for ease, the distance from points θS
and V to the frontier can be understood as the efficiency of these production points due to the com-
mon denominator VN and qSZ . Thus, the cost of regulation can be graphically interpreted as the
difference between the distances of points θS and V to the frontier.
distance VT = Ec. Had the plant not been regulated, this is how efficient the plant
would have been.
Note that Eb > Ec. This means that the plant is considered much more inefficient
when it is producing the same output Y with the same inputs X when emission is not
regulated. The loss arising out of the regulation can then be measured as the ratio of
Ec to Eb. This is what is calculated by (3).
The goal of the environmental regulation though is to reduce emissions and not
restrict the production of the desirable output Y. Considering the production point
θS again, it can be observed that the distance from the point θS to W is different than
the distance from the same point to U.6 Since U and W are both on the frontier, the
question then is—“Can the efficiency be measured in the direction g towards U,
where g is (y, −z), rather than in the direction Y towards W?”
This change in direction of measurement of efficiency allows researchers to
answer not just where plants are compared to their efficient peers in terms of pro-
duction of output but also where plants are compared to their efficient peers in terms
of reducing emissions. If the plants are judged to be more efficient when distance
between θS and U is considered compared to distance between θS and W, then this
would mean that plants do not lose as much efficiency because of regulations that
restrict emissions. On the other hand, if plants are less efficient when this new direc-
tion of measuring efficiency is considered, then the losses from regulations would
be greater than initially measured using the distance between θS and W. This is
what is calculated in (5). If plants are found to be more efficient when distance from
θS to U is considered compared to W, then this provides a “gain” that can be adjusted
to the initial loss measured as the ratio of Ec to Eb.
Data
This study uses data from various sources to estimate the regulation–productivity
relationship. Due to the nonparametric nature of the study and the estimation meth-
ods used, information on plant-level outputs, inputs, as well as amount of pollutants
emitted by each plant is required. Since information from varied facets of a plant’s
production is required, the study uses different samples for each estimation method
on each industry. The ASM and CMF from the U.S. Census Bureau are the main
source of the plant output and input information. Data on abatement expenditures
are obtained from the Environmental Protection Agency’s (EPA) PACEs Survey.
Data on emissions and abated emissions for earlier periods is obtained from the
PACE survey as well. A separate set of emissions data for a later period is obtained
from EPA’s PCS and Toxic Release Inventory (TRI) database. Finally, data from
6
Though θSW is larger than θSU in the construction above, this might not always be true in reality.
The point here is in measuring efficiency towards a chosen direction that makes more sense rather
than minimizing the efficiency loss out of regulation.
52 S. Sharma
Gray and Shadbegian (2004) is used to calculate the benefits accruing from reduc-
tion in particulate emissions.
The study includes plants from two industries in this analysis—pulp and paper
mills and oil refineries. These two industries are among the heaviest emitters of both
air and water pollutants in the U.S. Pulp and paper mills are large emitters of air
pollutants like particulate matter (PM) and sulfur dioxide (SO2), and water pollut-
ants like BOD and total suspended solids (TSS) which are generated chiefly during
the pulping process. Similarly, oil refineries also emit a large amount of pollutants
chiefly during the process of catalytic cracking and cooling. Such high intensity of
pollution means that the effects of regulation can be observed more clearly.
Data on outputs and inputs for all years for both pulp and paper mills and oil
refineries are obtained from the ASM and the CMF. Output (y) is measured using
the total value of shipments, which is adjusted for inventories and work in progress.
The vector of inputs (x) contains four inputs: labor (L), materials (M), energy (E),
and capital (K). Labor input is measured as total production hours. Materials input
is represented by the sum of dollar expenditure on materials, resale and contract
work. Energy input is the sum of dollar expenditure on fuel and purchased electric-
ity. The measure of plant-specific real capital stock is based on the standard per-
petual inventory method (PIM) that accounts for new investments as well as
disposals and depreciation for each year, applied to the ASM and CMF data on new
investment for each plant.7 All the variables used for inputs and outputs except for
labor are measured in 1987 dollars.
Data on emissions is obtained from various sources for various periods. Data on
solid waste emissions is obtained from PACE survey for the period from 1974 to
1982. This emission is measured in tons per year. Measures of water pollution from
the PCS namely BOD and total suspended solids (TSS) and measures of solid waste
emission from EPA’s Toxic Release Inventory (TRI) measured in tons per year are
obtained for the period 1994–2000. These emissions data are then combined with
the input and output information from ASM and CMF for the estimation method
presented in (1) and (3). In addition, the PACE survey also contains data on abated
emission of Particulate Matter (PM) and Sulfur dioxide (SO2), from 1974 to 1982.
In order to compare the cost of regulation to the benefits derived from it for the pulp
and paper mills, benefits data from Gray and Shadbegian (2004) is used. This data
measures the benefits of reduction in particulate matter pollution using an air dis-
persion model, SLIM-3, which calculates the impacts of pollution from each plant
to the population surrounding it. In this manner, a plant-specific marginal benefit
measure is derived for reductions in particulate matter at each plant.
One additional assumption that is made in the DEA approach is that the technol-
ogy available to all plants under analysis is the same. This is done to ensure that
plants at the frontier and plants below it are comparable. If the plants under the
frontier do not have access to technology being used by the plants at the frontier, a
comparison of the two is not going to yield a meaningful result. To confirm that this
7
I would like to thank John Haltiwanger for making his version of these data available to other
researchers.
assumption is maintained, only those plants are considered in the samples that have
access to the same technology. In case of the paper industry, paper mills that incor-
porate a pulping process are included in the sample. In the case of oil refineries,
those that include catalytic cracking are included in the sample. In addition to ensur-
ing that the plants in our sample are using the same technology, this sample con-
struction also focuses on those technologies that are considered to be more polluting
than others.
Outliers are a major concern for DEA analysis especially when they enter the
frontier against which all other plants are compared. The method proposed in
Wilson (1993) is used to detect such production points that have a very low proba-
bility of occurrence. No outliers were found in all the samples used in the analysis
for this study.
Summary statistics for the variables used in the analysis are presented in Tables 1,
2, and 3. All variables are reported in millions of 1987 dollars except for the produc-
tion workers which is measured in thousands of hours. Table 1 provides summary
statistics for the economic variables used in one of the analyses for the pulp and
paper mills and oil refineries, respectively. Other samples using emissions and
abated emissions data are very similar to the ones presented in Table 1 and thus are
excluded here.
Table 1 Summary statistics

Pulp and paper mills (68 obs. per year) Oil refineries (34 obs. per year)
Mean in SD in % Growth in mean Mean in SD in % Growth in
1974 1974 1974–1994 1974 1974 mean 1974–1994
Y 215.02 96.57 48.68 1175.36 759.62 62.49
K 87.22 45.19 181.98 245.18 135.01 148.15
L 1.60 0.98 −2.50 1.31 0.82 9.16
M 102.56 45.72 43.80 1149.92 682.04 42.73
E 19.82 13.02 16.04 26.36 24.33 130.24
PAOC 2.45 3.66 153.47 11.32 11.44 226.33
PACE 6.42 9.22 −61.06 13.19 14.57 135.33
Summary statistics are presented for 1974 with average growth for the 20 years under analysis.
Detailed year by year summary statistics tables for all variables are available upon request from the
author. All following results of the analysis including the figures and tables are provided for alter-
nate years for brevity. Results for years that have been omitted are very similar to the ones included
in the tables. A detailed year-by-year table of all results is available upon request from the author
Variable definitions
Y = Total value of shipments − output (millions of dollars)
K = Capital stock (millions of dollars)
L = Production worker hours (thousands of hours)
M = Cost of raw materials (millions of dollars)
E = Value of energy consumed (millions of dollars)
PAOC = Pollution abatement operating cost (millions of dollars)
PACE = Pollution abatement capital expenditure (millions of dollars)
54 S. Sharma
Table 2 Summary statistics for pulp and paper mills emission

% Growth in mean Number of
Mean in 1974 SD in 1974 1974–1982 observations
Solid waste 37.36 (58.42) 0.78 37
Abated PM 26.83 (38.95) 0.59 67
Table 3 Summary statistics for other pollutants

% Growth in mean Number of
Mean in 1994 SD in 1994 1994–1998 observations
Paper BOD 3147.91 (3152.75) −0.38 81
Paper TSS 7496.15 (16000.00) −0.51 81
Paper TRI 1151.58 (856.04) 0.06 81
Oil BOD 170.53 (270.34) −0.24 31
Oil TSS 270.64 (431.88) −0.31 31
All emissions are reported in millions of tons
It can be observed from Table 1 that while the values for output—Y, capital—K,
energy—E, and materials—M grew over the sample period, labor—L did not. A
similar upward trend can also be observed for pollution abatement operating costs
(PAOC). Pollution abatement capital expenditure (PACE) seems to have declined
relative to the large abatement capital investments required in the early 1970s. For
instance, installing a smokestack scrubber is not a regular annual expense for a plant
but a one-time investment.
Tables 2 and 3 provide summary statistics for the pollutants emitted and the
amounts of pollutants abated in millions of tons.8 It can be observed from Table 2
that the amount of abated emission of particulate matter for the sample period has
increased. This increase along with increase in abatement spending from previous
tables indicates some effect of regulation on emissions and abatement. This is fur-
ther enforced by figures in Table 3 that report the decrease in amounts of emission
of other pollutants in both industries in thousands of tons. The emission of solid
waste in Table 2, however, is increasing. Such increases do not suggest that this
emission is not affected by regulation. It could be argued that the figures might have
been higher had the regulation not restricted the emission of such solid waste. In
addition, disposal of solid waste was not as heavily regulated as air or water emis-
sion in the initial years as long as the solid waste was disposed of properly.
According to Table 3, the emission of BOD and total suspended solids (TSS) for
the paper industry has both decreased over the sample period. Similarly, BOD and
TSS for oil refineries also show a decline. Waste measured from toxic release inven-
tory (TRI) for pulp and paper mills, however, has remained stable over the period.
8
While the analysis was conducted for both Paper and Oil Industries using abated PM, abated SO2,
and solid waste emissions from the PACE survey, results, and summary statistics are only provided
for emissions and abated PM for the pulp and paper mills. All omitted results and summary statis-
tics are qualitatively very similar.
Results
The existence of regulation that requires plants to spend on abatement and restricts
emission of pollutants gives rise to reduced productivity and outputs for the regu-
lated plants. However, such measure of “traditional” productivity loss does not
account for the fact that when plants spend on abatement, the objective of such
spending is not in fact to produce more desirable outputs but to reduce the undesir-
able ones. This change in the objective of the plant leads to a change in outcomes
that are often not measured by traditional efficiency analysis. While such methods
account for the reduced outputs arising out of restricted emissions and reduced
inputs, they do not account for the decrease in the undesirable outputs that arise out
of an increase in the abatement inputs. This leads to results that indicate heavy pro-
ductivity losses from compliance with regulations.
It is thus necessary, if environmental concerns are imperative in a research, to
measure how efficient a plant is in the direction of the undesirable outputs as well.
This also allows us to be more responsive to our needs to be sustainable in viewing
plants’ performance. If a plant emits less than most of its peers and including this
outcome in the analysis results in the plant being considered more efficient, the
“traditional” measure of productivity and output loss might be overestimating the
loss arising out of regulation. If on the other hand, the results obtained from such
analysis do not show a difference in efficiency scores compared to the previous
analyses, it can be concluded that the inclusion of an “environmental objective”
within the analysis of how plants are performing compared to their peers in terms of
emissions does not matter for efficiency and productivity measurements. Results of
such analysis are presented below.
Table 4 presents results from the sample of pulp and paper mills that uses data on
solid waste emissions. The average efficiency scores using both models are presented
in the first two columns. In most cases, the efficiency scores under the regulated and
the unregulated regime seem to be very similar. In addition, some years have a lower
efficiency score for (4) than for (2) suggesting that on average, accounting for perfor-
mance on emission reduction makes plants more inefficient. This could be because
for these particular years, the plants compared to their most efficient peers did worse
off in terms of emission reduction than in other years. These peculiar results may also
be explained using the lax regulatory restrictions placed on the disposal of solid waste
compared to the restrictions on emission of other hazardous pollutants. Such results
also suggest that for this sample, the cost of regulation in terms of lost output is not
significantly overestimated when emission reduction is also taken into account.
Table 5 presents the results from the various samples with emissions data in the pulp
and paper industry. Results indicate that there is a statistically significant difference
between the efficiency scores obtained from the two models in most of the years for all
pollutants. It is also clear from the results that efficiency scores obtained from (4) are
higher than the ones from model (2). This indicates that accounting for plants’ perfor-
mance on emission reduction in addition to production of output makes plants more
efficient compared to just accounting for production of outputs given constant inputs.
Table 4 Mean efficiency scores in regulated regime for solid waste emissions (pulp and paper
mills)
Year Eff. model A Eff. model B SR test
1974 0.87 0.87 0.01
(0.13) (0.13)
1975 0.81 0.81 0.27
(0.16) (0.15)
1976 0.88 0.87 0.04
(0.13) (0.13)
1977 0.87 0.87 0.49
(0.12) (0.12)
1978 0.88 0.89 0.19
(0.12) (0.11)
1979 0.92 0.92 0.76
(0.10) (0.09)
1980 0.88 0.88 0.16
(0.13) (0.13)
1981 0.91 0.91 0.33
(0.10) (0.09)
1982 0.94 0.94 0.82
(0.10) (0.09)
Model A: mean efficiency scores for regulated regime without crediting plants for emission
reduction
Model B: mean efficiency scores for regulated regime after crediting plants for emission
reduction
Mean values are reported with standard deviations in parenthesis
Table 5 Mean efficiency scores in regulated regime for other emissions (pulp and paper mills)
BOD BOD TSS TSS TRI TRI SR
Year (M.A) (M.B) SR test (M.A) (M.B) SR test (M.A) (M.B) test
1994 0.89 0.90 0.00 0.84 0.84 0.04 0.89 0.90 0.00
(0.12) (0.11) (0.14) (0.13) (0.12) (0.10)
1995 0.91 0.92 0.00 0.89 0.90 0.00 0.89 0.90 0.00
(0.11) (0.10) (0.13) (0.11) (0.13) (0.11)
1996 0.90 0.91 0.00 0.90 0.91 0.00 0.90 0.90 0.00
(0.11) (0.10) (0.11) (0.10) (0.12) (0.11)
1997 0.84 0.86 0.00 0.83 0.84 0.00 0.85 0.86 0.00
(0.14) (0.12) (0.14) (0.13) (0.14) (0.13)
1998 0.86 0.87 0.00 0.86 0.87 0.00 0.84 0.85 0.00
(0.12) (0.11) (0.12) (0.11) (0.14) (0.12)
1999 0.85 0.86 0.02 0.85 0.86 0.11 0.87 0.88 0.00
(0.12) (0.11) (0.12) (0.11) (0.11) (0.10)
2000 0.83 0.84 0.30 0.84 0.85 0.05 0.86 0.87 0.00
(0.13) (0.13) (0.13) (0.12) (0.12) (0.11)
M.A: mean efficiency scores for regulated regime without crediting plants for emission
reduction
M.B: mean efficiency scores for regulated regime after crediting plants for emission reduction
It could be posited that plants that did not do well under the conditions set in
problem (2) did well under the conditions set in problem (4) because of the tradeoff
between emission reduction and output. That is, if a plant abates more and produces
less output, measuring performance in terms of emission reduction along with out-
put will naturally render it more efficient than just measuring efficiency in terms of
output. However, even for plants that abate the same amount of pollutants and pro-
duce the same amount of outputs using equal amount of inputs, their efficiency in
the use of abatement spending places them at different points below the frontier
thereby allowing a measure of efficiency in the direction of emission reduction. This
indicates that changing the direction in which efficiency is measured matters.
Results from the samples that use emissions data from the oil refineries and the
sample using abated particulate matter data are also very similar. All of these results
indicate that the including emission reduction in the efficiency analysis renders the
plants more efficient. This also implies that the output loss calculated initially as a
measure of the cost of regulation in model (2) might be overestimated if environ-
mental outcomes are to be included in measures of efficiency.
The significant difference between the efficiency scores obtained from (2) and
(4) points out a critical difference between the two estimation methods. In order to
include this difference in the measure of output loss the gain obtained from (5) is
used. This gain signifies the improved performance of plants when the direction of
measurement of efficiency is altered to include the environmental objective in effi-
ciency measurements. A final measure of loss of output due to regulations is then
calculated by adjusting for the gains from the traditional measure of output loss.
Results for all samples are presented in Tables 6, 7, 8, 9, and 10.
As can be observed from Tables 6, 7, 8, 9, and 10, the first (loss) column on each
section of the result shows the average loss arising out of environmental regulation
using the traditional productivity and output loss measure. The second column
shows the gains in efficiency that is observed when efficiency is measured in the
direction of the undesirable output in addition to the desirable output. The third
column refers to the average loss measured when efficiency gain from environmen-
tal performance is adjusted to the traditional output loss measure.
Based on these results, a distinction can be made on the effect of regulation
between the two time periods under analysis. Tables 6 and 10 show results for the
pulp and paper mills from an earlier period in the 1970s. These results indicate that
including environmental outcomes in efficiency analysis does not lead to much gain
in efficiency for plants on average. Results from the 1990s on the other hand show
that plants do experience a gain by including environmental outcomes while mea-
suring efficiency.
These results can be attributed to several possible factors. First, compliance with
environmental regulation requires a large amount of initial capital investment and a
change in the production process. Since the 1970s is when the EPA became more
active in enforcement of regulations, all plants may not have had a chance to make
such changes. Second, plants may be inefficient in environmental performance ini-
tially because they need time to learn to adjust to a regulatory environment. Plants
adjust in time as they learn to use their abatement resources more efficiently. Third,
58 S. Sharma
Table 6 Mean loss from regulation adjusted for emission reduction for solid waste emissions—
(pulp and paper mills)
Loss Gain Adjusted loss SR test
1974 4.28 −0.40 4.67 0.01
(5.63) (1.99) (6.19)
1975 5.24 0.13 5.11 0.36
(7.07) (4.47) (9.12)
1976 6.09 −1.13 7.22 0.04
(9.50) (7.61) (15.26)
1977 6.68 −0.33 7.01 0.45
(8.28) (7.10) (13.10)
1978 6.87 0.83 6.04 0.19
(7.47) (4.14) (9.02)
1979 2.35 0.54 1.81 0.75
(3.30) (1.81) (3.85)
1980 4.39 −0.50 4.89 0.16
(7.58) (5.87) (12.14)
1981 3.89 0.40 3.49 0.32
(6.70) (2.30) (7.22)
1982 3.30 0.46 2.84 0.82
(5.30) (1.49) (5.11)
Table 7 Mean loss from regulation adjusted for emission reduction for other emissions (pulp and
paper mills)
Paper BOD Paper TSS
Loss Gain Adj. loss SR test Loss Gain Adj. loss SR test
1994 8.77 1.31 7.46 0.00 3.47 1.07 2.40 0.04
(10.0) (3.1) (10.4) (6.1) (4.1) (7.3)
1995 9.99 0.89 9.10 0.00 7.95 1.41 6.54 0.00
(9.5) (5.3) (12.4) (8.1) (2.4) (8.4)
1996 3.97 0.87 3.11 0.00 4.03 0.86 3.17 0.00
(5.9) (1.7) (5.8) (6.0) (2.0) (6.0)
1997 5.50 2.10 3.41 0.00 4.10 2.03 2.06 0.00
(6.9) (3.5) (7.5) (6.1) (3.9) (6.6)
1998 10.33 1.50 8.83 0.00 10.02 1.35 8.67 0.00
(10.8) (3.0) (11.0) (9.9) (2.2) (9.6)
1999 4.20 1.17 3.03 0.02 4.30 0.79 3.51 0.11
(6.5) (3.0) (6.8) (6.5) (3.1) (7.3)
2000 4.91 0.95 3.96 0.31 6.14 1.33 4.82 0.05
(8.2) (4.6) (9.5) (9.4) (4.5) (10.3)
Table 8 Mean loss from regulation adjusted for emission reduction for other emissions (pulp and
paper mills)
Paper TRI
Loss Gain Adj. loss SR test
8.92 1.67 7.26 0.00
(8.7) (2.1) (8.9)
7.67 1.73 5.94 0.00
(8.4) (3.1) (9.1)
3.39 0.66 2.73 0.00
(4.1) (1.4) (4.2)
6.56 1.28 5.28 0.00
(6.9) (2.7) (7.9)
7.70 2.21 5.48 0.00
(8.3) (4.6) (10.0)
6.65 0.80 5.85 0.00
(7.6) (3.9) (9.4)
8.13 0.88 7.25 0.00
(8.8) (6.3) (13.0)
Table 9 Mean loss from regulation adjusted for emission reduction for other emissions (oil
refineries)
Oil BOD Oil TSS
Years Loss Gain Adjusted loss SR test Loss Gain Adjusted loss SR test
1994 5.53 0.79 4.74 0.87 7.89 2.37 5.52 0.00
(7.59) (3.43) (8.14) (10.43) (4.19) (11.41)
1995 1.68 0.47 1.22 0.00 5.70 0.90 4.79 0.00
(2.79) (2.02) (3.69) (7.11) (1.70) (7.51)
1996 2.44 0.26 2.18 0.07 4.24 0.98 3.26 0.00
(3.22) (1.99) (4.21) (5.38) (2.03) (6.13)
1997 4.35 0.18 4.17 0.81 4.43 0.83 3.61 0.06
(7.07) (1.31) (7.34) (5.80) (1.65) (6.03)
1998 4.26 0.16 4.11 0.01 5.13 0.36 4.77 0.03
(5.22) (1.72) (5.37) (7.24) (0.81) (6.98)
1999 5.98 2.05 3.93 0.00 6.82 2.06 4.76 0.00
(8.01) (2.81) (7.82) (6.46) (3.19) (6.82)
plants may also have benefited from the explosion of information technology and
the advent of internet during the 1990s. This would have no doubt allowed a
smoother access to information on regulations and compliance requirements. This
ease of access to information may have also helped in the transfer of technologies
and production processes between plants allowing them to become more efficient in
their abatement efforts.
60 S. Sharma
Table 10 Mean loss from regulation adjusted for emission reduction for abated PM—PACE (pulp
and paper mills)
Loss Gain Adjusted loss SR test
1974 3.38 0.49 2.89 0.34
(4.89) (2.61) (5.25)
1975 10.23 −1.03 11.26 0.00
(10.52) (2.95) (11.52)
1976 7.51 0.40 7.11 0.03
(8.28) (5.79) (11.16)
1977 5.06 0.39 4.68 0.84
(6.70) (2.59) (7.58)
1978 4.36 0.36 4.00 0.22
(7.77) (2.73) (8.71)
1979 3.93 0.32 3.61 0.20
(5.49) (1.95) (5.96)
1980 4.12 0.10 4.02 0.61
(6.80) (2.02) (7.80)
1981 3.64 0.53 3.11 0.11
(4.73) (2.76) (5.82)
1982 3.44 0.53 2.92 0.03
(5.54) (3.21) (5.63)
It can also be observed from the results that the final loss figures on all tables are
still significantly high despite adjustments made for the gains due to environmental
performance. This implies that plants need to become more efficient in their produc-
tion as well as in their abatement processes to reduce such losses arising out of
environmental regulations. This may be achieved in time from innovation in pro-
duction and abatement technology and a transfer of information and technology
from the most efficient plants to their less efficient peers. Such results could also
suggest that the costs of regulations outweigh the benefits and that regulation is
overburdening plants. Could this be true?
The simple answer is no. The gains measured above only account for the perfor-
mance of plants in terms of the amount of pollutants emitted or amount of pollutants
abated in addition to production of output. In doing so, it helps plants identify where
they stand compared to their peers not just in terms of the use of productive inputs
but also in terms of their use of abatement inputs by comparing the respective
outcomes. Such identification could lead plants to seek more efficient ways of using
this input for greater emission reductions.
However, the benefit of environmental regulation to the larger society comes in
the form of reduced mortality, reduced morbidity, better ecosystem vitality, and a
better environment for a sustainable future for human beings. The benefit from
reduced mortality is measured here using data from Gray and Shadbegian (2004) is
used. This data measures the benefits of reduction in particulate matter pollution
Table 11 Benefits and losses Year Avg. loss (mil. $) Avg. ben (mil. $)
from regulation abated
1974 5.95 14,400
PM—PACE (pulp and paper
mills) (10.50) (25,100)
1976 12.34 17,700
(15.83) (23,200)
1978 8.67 32,800
(17.80) (79,300)
1980 7.36 39,400
(13.99) (47,500)
1982 5.76 48,200
(9.21) (69,700)
Mean values are reported with standard devia-
tions in parenthesis
Average measures refer to the average loss or
benefit per plant each year
using an air dispersion model, SLIM-3,9 which calculates the impacts of pollution
from each plant to the population surrounding it.
It can be clearly seen from Table 11 that the benefits that accrue to the society
from reduction in PM10 emissions are far greater (by two orders of magnitude)
compared to the costs that the plants incur because of such regulations restricting
the emission of PM10 in each year of the analysis. Though caution has been advised
in the interpretation of the cost of regulation as only a lower bound and though there
are many other costs of regulation such as costs of enforcement to the regulators,
these costs cannot offset the benefits that people get from such regulations. Further,
the benefits presented above only account for reductions in mortality. Other benefits
such as those from reduced morbidity and improved ecosystem vitality would only
expand the total benefits that accrue due to environmental regulations.
In summary, this section confirms that firms could benefit by recognizing that
changing the direction of efficiency measurement allows firms to measure plants’
performance on differing objectives of production and pollution control and abate-
ment at the same time. Researchers could also benefit from such methods of effi-
ciency measurement when environmental objectives are important to a research. In
addition, the section also establishes that while there are costs to plants from envi-
ronmental regulation, the benefits that accrue to the society from such regulation are
much larger than these costs. This provides ample justification for the existence of
such regulations and why they are important. The next section will now look at the
relationship between regulation and productivity relation over time and decompose
the productivity change measure into efficiency change and technological change to
explore the regulation–technology relationship.
9
For more information on SLIM-3 model, refer to Gray, W.B. and R.J. Shadbegian. 2004.
“‘Optimal’pollution Abatement—Whose Benefits Matter, and How Much?” Journal of
Environmental Economics and Management, 47(3), 510–34.
62 S. Sharma
Conclusion
Environmental regulations and their enforcement place a burden on plants to com-

ply with those regulations. Since the establishment of the U.S. Environment
Protection Agency (EPA) in 1970, many regulations like the Clean Air Act and the
Clean Water Act have been introduced and amended multiple times to make the
regulatory environment more stringent over time. This increasing stringency trans-
lates into higher costs for plants that face the responsibility of complying with such
regulations and reducing their emissions.
An environmental production model was constructed which provides a frame-
work for specifying a production technology that allows joint production of desir-
able and undesirable outputs. This model assumed that when a regulation is
introduced, plants are required to limit or bring down their emission of harmful
pollutants to a certain level. Since the constraints in the model require plants to emit
a specific amount of pollutants and no more, the model assumed that inputs associ-
ated with abatement activities “crowd out” inputs used in the production of the
desirable output on a one-for-one basis. Results indicate that plants do suffer from
reduced productivity because of environmental regulations. These losses ranged
between about 2 % to about 10 % of output which agrees with results obtained ear-
lier from the resource diversion model.
These results also agree with the wider literature that investigates the loss in pro-
ductivity due to environmental regulations in that it does find a large amount of output
loss for plants that have to spend on abatement or are restricted to emit harmful pol-
lutants. The magnitude of the effect of regulation on output, however, varies across
the multitude of papers that examine this issue. One thing to note here is that the
analysis is conducted by looking at the effect of the entirety of the regulatory regime.
As such this research does not look at the effect of one specific regulation but assumes
that all regulations are already in place and studies the cost of such regulation on pro-
ductivity. Since each frontier is constructed based on contemporary peers in the same
year, the results show efficiency scores for each plant based on the regulations it faces
in that year. Thus, a change in regulations from 1 year to the next is accounted for by
the use of efficient peers from the same year in the construction of the frontier.
Though results show that plants experience output losses due to regulation, the
“traditional” measure of productivity and output loss might overestimate the loss
arising out of regulation if a reduction in emissions is not considered within the effi-
ciency estimation models. Manufacturing plants might experience a win–win result
from environmental regulations if environmental performance is also factored into
efficiency measurements. Therefore, this research examined the effect of including
the environmental performance of plants in efficiency measurement. Results indi-
cate that changing the direction of efficiency measurement does bring about changes
in the measure of cost of regulation for plants. When reduction in emission is seen
as a “good” outcome, the productivity loss arising out of regulation becomes smaller
than when it is not included. This implies that firms could benefit from the inclusion
of an “environmental objective” in productivity measurements by recognizing that
altering the direction of efficiency measurement allows firms to measure plants’
performance on differing objectives of production and pollution control or abate-

ment at the same time. This allows firms to be more responsive to needs to be sus-
tainable. Researchers could also benefit from such methods of efficiency measurement
when environmental objectives and sustainability are important to a research.
A win–win scenario for the plants, however, is not observed from the results. On
the contrary, a win–lose scenario is observed suggesting that while plants are able
to increase their efficiency by accounting for reduction in emissions, environmental
regulations still result in lost output. These results thus agree more with those found
by Hart and Ahuja (1996) and Cordeiro and Sarkis (1997) compared to those found
by Telle and Larsson (2007).
An encouraging sign though is that results suggest that plants have become more
efficient in their abatement practices over time. This could be due to multiple rea-
sons such as capital investments made towards abatement, plants learning to adjust
to regulations over time, and development of information technology systems
allowing for smoother transfer of information on compliance requirements and
abatement processes and technologies. Such efficiency in abatement efforts could
also be aided by the government with a conducive environment for transfer of infor-
mation on best practices in abatement. This would not only lead to gains in effi-
ciency for manufacturing plants but also lead to the development of sustainable
practices benefiting the society.
Lastly, though plants face a cost of regulation in terms of lost output and
decreased productivity, results also indicate that the benefits of these environmental
regulations in terms of reduced mortality and morbidity far outweigh these costs. It
thus makes economic sense to impose such regulations since doing away with them
would mean a greater loss for the wider society.
Acknowledgments I am very grateful to my advisor Wayne Gray for his continuous advice, sup-
port, and encouragement during this research. I am indebted to my committee members Junfu
Zhang and Chih Ming Tan for providing valuable inputs. I also greatly benefitted from discussions
with Wang Jin during the course of this research. Finally, I thank numerous conference and semi-
nar participants for helpful comments and suggestions.
Disclaimer Any opinions and conclusions expressed herein are those of the author and do not
necessarily represent the views of the U.S. Census Bureau. All results have been reviewed to
ensure that no confidential information is disclosed.
References
Aiken DV, Färe R, Grosskopf S, Pasurka CA (2009) Pollution abatement and productivity growth:
evidence from Germany, Japan, the Netherlands, and the United States. Environ Resour Econ
44(1):11–28
Arocena P (2008) Cost and quality gains from diversification and vertical integration in the elec-
tricity industry: a DEA approach. Energy Econ 30(1):39–58
Asmild M, Paradi JC, Aggarwall V, Schaffnit C (2004) Combining DEA window analysis with the
Malmquist index approach in a study of the Canadian Banking Industry. J Prod Anal
21(1):67–89
64 S. Sharma
Ball E, Fare R, Grosskopf S, Zaim O (2005) Accounting for externalities in the measurement of
productivity growth: the Malmquist cost productivity measure. Struct Change Econ Dyn
16(3):374–394
Banker RD, Charnes A, Cooper WW (1984) Some models for estimating technical and scale inef-
ficiencies in data envelopment analysis. Manage Sci 30(9):1078–1092
Barbera AJ, McConnell VD (1986) Effects of pollution control on industry productivity: a factor
demand approach. J Ind Econ 35:161–172
Barth JR, Lin C, Ma Y, Seade J, Song FM (2013) Do bank regulation, supervision and monitoring
enhance or impede bank efficiency? J Bank Financ 37(8):2879–2892
Berg SA, Forsund FR, Jansen ES (1992) Malmquist indices of productivity growth during the
deregulation of Norwegian banking, 1980-89. Scand J Econ 94:S211–S228
Bowlin WF, Charnes A, Cooper WW, Sherman HD (1984) Data envelopment analysis and regres-
sion approaches to efficiency estimation and evaluation. Ann Oper Res 2(1):113–138
Boyd GA, McClelland JD (1999) The impact of environmental constraints on productivity
improvement in integrated paper plants. J Environ Econ Manage 38(2):121–142
Brännlund R, Färe R, Grosskopf S (1995) Environmental regulation and profitability: an applica-
tion to Swedish pulp and paper mills. Environ Resour Econ 6(1):23–36
Charnes A, Cooper WW, Rhodes E (1978) Measuring the efficiency of DMU. Eur J Oper Res
2(6):429–444
Christainsen GB, Haveman RH (1981) Public regulations and the slowdown in productivity
growth. Am Econ Rev 71(2):320–325
Chung YH, Färe R, Grosskopf S (1997) Productivity and undesirable outputs: a directional dis-
tance function approach. J Environ Manage 51:229–240
Cordeiro JJ, Sarkis J (1997) Environmental proactivism and firm performance: evidence from
security analyst earnings forecasts. Bus Strategy Environ 6(2):104–114
De Nicola A, Gitto S, Mancuso P (2012) Uncover the predictive structure of healthcare efficiency
applying a bootstrapped data envelopment analysis. Expert Syst Appl 39(12):10495–10499
Denison EF (1979) Accounting for slower economic growth: the United States in the 1970’s.
Brookings Institution Press, Washington
Färe R, Grosskopf S (2000) Theory and application of directional distance functions. J Prod Anal
13(2):93–103
Färe R, Grosskopf S, Lovell CAK, Pasurka C (1989) Multilateral productivity comparisons when
some outputs are undesirable: a nonparametric approach. Rev Econ Stat 71(1):90–98
Färe R, Grosskopf S, Pasurka CA (2007) Pollution abatement activities and traditional productiv-
ity. Ecol Econ 62(3–4):673–682
Farrell MJ (1957) The measurement of productive efficiency. J R Stat Soc Ser A 120(3):253–290
Fethi MD, Pasiouras F (2010) Assessing bank efficiency and performance with operational
research and artificial intelligence techniques: a survey. Eur J Oper Res 204(2):189–198
Golany B, Roll Y (1989) An application procedure for DEA. Omega 17(3):237–250
Gollop FM, Roberts MJ (1983) Environmental regulations and productivity growth: the case of
fossil-fueled electric power generation. J Polit Econ 91(4):654–674
Gorman MF, Ruggiero J (2008) Evaluating US state police performance using data envelopment
analysis. Int J Prod Econ 113(2):1031–1037
Gray WB (1987) The cost of regulation: OSHA, EPA and the productivity slowdown. Am Econ
Rev 77(5):998–1006
Gray WB, Shadbegian RJ (1993) Environmental regulation and manufacturing productivity at the
plant level. National Bureau of Economic Research, Cambridge
Gray WB, Shadbegian RJ (1995) Pollution abatement costs, regulation, and plant-level productiv-
ity. National Bureau of Economic Research, Cambridge
Gray WB, Shadbegian RJ (2004) ‘Optimal’ pollution abatement—whose benefits matter, and how
much? J Environ Econ Manage 47(3):510–534
Halkos GE, Tzeremes NG (2013) A conditional directional distance function approach for measuring
regional environmental efficiency: evidence from UK regions. Eur J Oper Res 227(1):182–189
Hart SL, Ahuja G (1996) Does it pay to be green? An empirical examination of the relationship
between emission reduction and firm performance. Bus Strategy Environ 5(1):30–37
Havrylchyk O (2006) Efficiency of the Polish banking industry: foreign versus domestic banks.
J Bank Financ 30(7):1975–1996
Hollingsworth B (2003) Non-parametric and parametric applications measuring efficiency in
health care. Health Care Manag Sci 6(4):203–218
Hollingsworth B (2008) The measurement of efficiency and productivity of health care delivery.
Health Econ 17(10):1107–1128
Hollingsworth B, Dawson PJ, Maniadakis N (1999) Efficiency measurement of health care: a
review of non‐parametric methods and applications. Health Care Manag Sci 2(3):161–172
Karlaftis MG (2004) A DEA approach for evaluating the efficiency and effectiveness of urban
transit systems. Eur J Oper Res 152(2):354–364
Knox Lovell CA, Pastor JT, Turner JA (1995) Measuring macroeconomic performance in the OECD:
a comparison of European and non-European countries. Eur J Oper Res 87(3):507–518
Koopmans TC (1951) Analysis of production as an efficient combination of activities. Act Anal
Prod Alloc 13:33–37
Lin R-C, Mustafa Y, Pasupathy KS (2013) Multi-objective simulation optimization using data
envelopment analysis and genetic algorithm: specific application to determining optimal
resource levels in surgical services. Omega 41(5):881–892
Markovits-Somogyi R (2011) Measuring efficiency in transport: the state of the art of applying
data envelopment analysis. Transport 26(1):11–19
Nemoto J, Goto M (2003) Measurement of dynamic efficiency in production: an application of
data envelopment analysis to Japanese electric utilities. J Prod Anal 19(2):191–210
Nolan JF, Ritchie PC, Rowcroft JE (2002) Identifying and measuring public policy goals: ISTEA
and the US bus transit industry. J Econ Behav Organ 48(3):291–304
Norsworthy JR, Harper MJ, Kunze K (1979) The slowdown in productivity growth: analysis of
some contributing factors. Brookings Pap Econ Act 1979(2):387–421
Pacudan R, De Guzman E (2002) Impact of energy efficiency policy to productive efficiency of
electricity distribution industry in the Philippines. Energy Econ 24(1):41–54
Palmer K, Oates WE, Portney PR (1995) Tightening environmental standards: the benefit-cost or
the no-cost paradigm? J Econ Perspect 9(4):119–132
Porter M (1991) America’s green strategy. Sci Am 264(4):168
Porter M (1996) America’s green strategy. Business and the environment. Earthscan, London,
pp 33–35
Repetto R, Rothman D, Faeth P, Austin D (1997) Has environmental protection really reduced
productivity growth? Challenge 40(1):46–57
Rezitis AN (2006) Productivity growth in the Greek banking industry: a non-parametric approach.
J Appl Econ 9(1):119–138
Scheel H (2001) Undesirable outputs in efficiency valuations. Eur J Oper Res 132(2):400–410
Shadbegian RJ, Gray WB (2005) Pollution abatement expenditures and plant-level productivity: a
production function approach. Ecol Econ 54(2–3):196–208
Shadbegian RJ, Gray WB (2006) Assessing multi-dimensional performance: environmental and
economic outcomes. J Prod Anal 26(3):213–234
Shen Y, Hermans E, Brijs T, Wets G, Vanhoof K (2012) Road safety risk evaluation and target set-
ting using data envelopment analysis and its extensions. Accid Anal Prev 48:430–441
Shephard RW, Gale D, Kuhn HW (1970) Theory of cost and production functions. Princeton
University Press, Princeton
Staub RB, da Silva e Souza G, Tabak BM (2010) Evolution of bank efficiency in Brazil: a DEA
approach. Eur J Oper Res 202(1):204–213
Sueyoshi T, Goto M (2001) Slack-adjusted DEA for time series analysis: performance measure-
ment of Japanese electric power generation industry in 1984–1993. Eur J Oper Res
133(2):232–259
66 S. Sharma
Telle K, Larsson J (2007) Do environmental regulations hamper productivity growth? How

accounting for improvements of plants’ environmental performance can change the conclu-
sion. Ecol Econ 61(2):438–445
Thanassoulis E (1993) A comparison of regression analysis and data envelopment analysis as
alternative methods for performance assessments. J Oper Res Soc 44(11):1129–1144
Thanassoulis E (1995) Assessing police forces in England and Wales using data envelopment
analysis. Eur J Oper Res 87(3):641–657
Vaninsky A (2006) Efficiency of electric power generation in the United States: analysis and fore-
cast based on data envelopment analysis. Energy Econ 28(3):326–338
Velasquez M, Hester PT (2013) An analysis of multi-criteria decision making methods. Int J Oper
Res 10:56–66
Verschelde M, Rogge N (2012) An environment-adjusted evaluation of citizen satisfaction with
local police effectiveness: evidence from a conditional Data Envelopment Analysis approach.
Eur J Oper Res 223(1):214–225
Wang H, Zhou P, Zhou DQ (2013) Scenario-based energy efficiency and productivity in China: a
non-radial directional distance function analysis. Energy Econ 40:795–803
Wilson PW (1993) Detecting outliers in deterministic nonparametric frontier models with multiple
outputs. J Bus Econ Stat 11(3):319–323
Zhang C, Liu H, Bressers H, Buchanan KS (2011) Productivity growth and environmental
regulations-accounting for undesirable outputs: analysis of China’s thirty provincial regions
using the Malmquist–Luenberger index. Ecol Econ 70(12):2369–2379
Zofio JL, Prieto AM (2001) Environmental efficiency and regulatory standards: the case of CO2
emissions from OECD industries. Resour Energy Econ 23(1):63–83
Supply-Chain Transparency and Governance
Systems: Market Penetration of the I-Choose
System
Weijia Ran, Holly Jarman, Luis F. Luna-Reyes, Jing Zhang,

Deborah Andersen, Giri Tayi, Djoko S. Sayogo, Joanne Luciano,
Theresa A. Pardo, and David Andersen
Abstract In this chapter, we explore the impacts of key characteristics of Supply

Chain Governance Systems in the development and diffusion of technology innova-
tions that promote supply chain transparency and sustainable consumption and pro-
duction. The model presented in this chapter was developed following group model
building methods. Our simulation experiments reveal that the market resists “take-
off” unless external financial support can be found. Additionally, “take-off” dynam-
ics of the system are dominated by marketing budgets and external support for
infrastructure. Marketing budgets drive how fast users adopt the system, and with-
out external sponsorship of system, the final market collapses. Finally, the quality of
governance—reflected in information completeness, openness, relevance and reli-
ability, and the resultant trustworthiness of information determines final sustainable
market share.
W. Ran (*) • L.F. Luna-Reyes • D. Andersen • G. Tayi • D. Andersen

University at Albany, Albany, NY, USA
e-mail: wran@albany.edu; luisf.luna@udlap.mx; dla@albany.edu;
gtayi@albany.edu; david.andersen@albany.edu
H. Jarman
University of Michigan, Ann Arbor, MI, USA
e-mail: hjarman@umich.edu
J. Zhang
Clark University, Worcester, MA, USA
e-mail: jizhang@clarku.edu
D.S. Sayogo • T.A. Pardo
Center for Technology in Government, University at Albany, Albany, NY, USA
e-mail: dsayogo@ctg.albany.edu; tpardo@ctg.albany.edu
J. Luciano
Rensselaer Polytechnic Institute, Troy, NY, USA
e-mail: jluciano@gmail.com

DOI 10.1007/978-3-319-25439-5_4
68 W. Ran et al.
Introduction
The study presented in this chapter is at the intersection of multiple disciplines

including information technology, sustainability, and policy informatics. We used a
computer-based simulation approach to investigate factors that are important for the
market success of information-technology innovations that promote sustainable
consumption. From our modeling and simulation efforts emerges a theory of market
penetration dynamics of a large-scale socio-technical system to promote sustainable
consumption through building more transparent supply chains. Our study results
provide insights for policy making and governance in sustainable development.
Consumption is the driving force of production activities and even for human
development. Production consumes resources, and its products and by-products
changes the natural environment. The evolution of the way of production leads to
the specialization of roles and the division of labor, which is an underlying force of
social differentiation and inequity. Smart consumption and production can contrib-
ute to sustainable development, sustaining a natural ecosystem with a high level of
biodiversity and abundant resources that retains environmental and economic vitali-
ties, along with a diverse, equal, and democratic society that values the quality of
life (Basiago 1995). In free commodity markets, consumption and production activ-
ities are activated by purchase decisions made by consumers. Consumers’ prefer-
ences and choices direct what producers produce and how they produce. With the
increasing awareness of sustainability issues, more and more consumers are inclined
to buy and willing to pay a higher price for products that were produced in an envi-
ronmentally, economically, and socially sustainable way (Bray et al. 2011; Watts
and Wyner 2011). This potential for extra profit induces the producer’s opportunis-
tic “green-washing” behavior to claim that a product is more sustainably produced
than it really is (Kirchhoff 2000). “Green-washing” jeopardizes consumer trust, hin-
ders the flourishing of markets for sustainable products, and negatively impacts on
sustainable development.
The growth of markets for sustainable products needs the disclosure of informa-
tion about products’ utilities, qualities, and environmental and social impacts of
production processes to attract consumers. This motivates and provides opportuni-
ties for producers, independent third parties, or entrepreneurs to develop innova-
tions that facilitate the disclosure and use of trustworthy and quality information
that can help consumers to make more informed purchasing decisions. The success-
ful adoption and diffusion of these innovations will promote transparent supply
chain and sustainable consumption and production. Nevertheless, the success of
innovations in the market is not guaranteed, and is likely to be influenced by not
only technical but also a variety of nontechnical factors such as investment deci-
sions, consumer trust, and governance.
Our study goal is to examine impacts of a range of factors on the market diffu-
sion process of innovations that promote sustainable consumption. Understanding
which economic, social, and institutional factors are crucial to the market success of
these innovations as well as how these factors are influencing the market penetration
Supply-Chain Transparency and Governance Systems… 69
dynamics can help to facilitate sustainable consumption and production. These are
questions which we wanted to answer in this study. To this end, we performed a case
analysis using a system dynamics group model building and simulation approach.
The case under study is an initiative called I-Choose that aims to create a socio-
technical system to facilitate information sharing and interoperability among stake-
holders in the supply chain. Our current simulation experiments reveal that the
market resists “take-off” unless external financial support can be found. Additionally,
“take-off” dynamics of the system are dominated by marketing budgets and exter-
nal support for infrastructure. Marketing budgets drive how fast users adopt the
system, and without external sponsorship of system, the final market collapses.
Finally, the quality of governance—reflected in information completeness, open-
ness, relevance and reliability, and the resultant trustworthiness of information
determines final sustainable market share.
The chapter is organized in six sections, including this brief introduction. The
second section of the chapter includes a description of our current research efforts
developing a system to support ethical consumption and transparent supply chains.
The third section includes a literature review on the relevance of governance sys-
tems for the development of such systems. The fourth section is a description of the
methods that we followed to develop the current preliminary model. The fifth sec-
tion includes an introduction of existing adoption and diffusion models, a descrip-
tion of our simulation-model structure and simulation experiments. We end the
chapter with some final remarks and future research.
The I-Choose System
I-Choose project is supported in part by US-NSF (Grant No. IIS-0540069),

CONACYT-Mexico (Grant No. 133670), and the Canadian and COMEXUS
Fulbright Commissions.1 Its mission is to help diminish information asymmetries in
free markets and promote ethical production and consumption. Examples of similar
efforts include “fair trade,” “sustainable,” “green,” “locally produced,” “fair wage,”
or “organic (for food)” production processes. I-Choose project furthers existing
efforts by creating a scalable socio-technical system to facilitate information shar-
ing and interoperability among stakeholders in the supply chain (Luna-Reyes et al.
2011, 2012b).
The socio-technical system includes three components: a set of data standards to
share information across the supply chain, a set of Application Programming
Interface (API) standards to make it possible for developers and other interested
groups to create specific applications to make this information usable by regular
consumers, and a governance system, which will be in charge of creating and modi-
fying the standards over time. This socio-technical system is called I-Choose sys-
tem (Luna-Reyes et al. 2012a, b).
1
More information about I-Choose can be found at http://www.ctg.albany.edu/projects/ichoose.
70 W. Ran et al.
I-Choose project research team collaborated with a variety of stakeholders and

implemented a prototype of this socio-technical system through a case study of cof-
fee grown and sold in the North American Free Trade Area (NAFTA). This proto-
type is an information system which we refer to as a Full Information Product Pricing
(FIPP) system. An FIPP system makes information about how, where, by whom, and
under what conditions a particular type of coffee was produced available and usable
to consumers, and helps them make ethical decisions with regard of the social and
environmental impacts of the coffee they drink (Luna-Reyes et al. 2012a, b).
Literature Review
The theoretical ground of our simulation model is related to many different research
areas, including research in measuring the value of information and information
technologies as well as the distribution of this value along the supply chain (Elofson
and Robinson 2007; Malhotra et al. 2005; Wang et al. 2006), the role of transpar-
ency in improving performance in supply chains (Bayat et al. 2011; Davis et al.
2010), the role of supply chain transparency in building more sustainable supply
chains (Davis et al. 2010; Goleman 2009), the role of information agents in improv-
ing buying decisions along the supply chain (Komiak and Benbasat 2006; Nissen
and Sengupta 2006; Sathiyamoorthy et al. 2010; Xiao and Benbasat 2007), market-
ing research in motivations for ethical consumption (Carrington et al. 2010; Kim
et al. 2010; Punj and Moore 2007), product labeling (Beales et al. 1981; Caswell
and Padberg 1992), as well as the key role of trust in the development of applica-
tions and systems to promote transparency and improved consumption decisions
(Komiak and Benbasat 2006, 2008; Ladd 2010). We believe that our modeling
efforts are going to benefit from knowledge developed in all these different areas,
and also it has the potential to contribute to all of them. In this section, we review
the relevant literature and identify key issues associated with the performance of
FIPP systems. Together with empirical data, this review serves as the basis of our
simulation-model-building effort.
The Governance of FIPP Systems
Governance, in the simplest definition, is the process of governing: of steering a

system, organization, or society towards certain desirable outcomes (Rosenau 1995,
2000). Our project focuses on how FIPP systems are and might be governed—i.e.,
how interorganizational relationships are formed to promote competition that is
based on non-price product characteristics. This is distinct from referring to the
internal governance of a corporation or organization. Use of the term governance, in
our definition, therefore implies collaboration of various kinds between firms,
non-governmental organizations, and governments in order to steer the system

towards certain, mutually agreed upon, outcomes.
But it is not enough to just have “governance.” The quality of that governance
must somehow be assessed. The term “good governance” is frequently used in the
context of states, corporations, and international organizations to refer to a desirable
set of principles on which organizations should deliver. So what constitutes, or
should constitute, “good governance” among FIPP systems? How might FIPP sys-
tems be governed in order to maximize the benefits of such systems to producers,
retailers, certifiers, or consumers?
There exists a great diversity of opinion with regard to what constitutes “good”
governance. Fortunately, a review of the scholarly literature reveals a consensus
around three core principles (Key works include Kooiman 1993; UNDP 1994;
Woods 1999; CEC 2001; Kaufmann and Kraay 2002; de Búrca and Scott 2006;
Abbott and Snidal 2009; Sabel and Zeitlin 2010; Héritier and Lehmkuhl 2008;
Mossialos et al. 2010; World Bank Group 2012; Vermeulen et al. 2012; Greer et al.
2015). Despite slightly different wording, the concepts of accountability, transpar-
ency, and participation are most frequently viewed by organizations and scholars as
essential to good governance. By extrapolating from these core governance princi-
ples, we can gain some idea of how they can be harnessed to produce trustworthy
information and therefore to support FIPP systems.
In the most basic sense, accountability refers to the requirement for those with
authority to answer for their decisions and actions. It refers not only to the sharing
of key information such as reporting or accounting requirements, but extends
beyond that to providing clarity regarding power relationships. Strong accountabil-
ity within an organization requires that all participants know who is accountable, for
what, and to whom (Woods 1999, p 44). In FIPP systems, accountability can be
used to increase trust. Accountability increases trust in the system because decision-
making procedures are clear, and are followed consistently, and because decisions
can be challenged. When the system is accountable, we can expect increases in the
reliability of the data because participants within the system have channels they can
use to challenge the validity of product information and the way it is interpreted.
Transparency refers to the extent to which organizations and individuals outside
a system or institution can access information about its decisions and actions. Strong
transparency requires not just that information on decision-making procedures be
available, but that the resources spent to access the information should be reason-
able (OECD 2002, 2003). The principle of transparency frequently underpins
accountability because the ability for participants and outsiders to see how the sys-
tem works can enhance their ability to challenge decisions made by the system. In
FIPP systems, transparency can increase trust by increasing the openness of the
system and its data. Transparency of the disclosure process increases trust in the
data because it can be accessed and checked independently.
Participation refers to the extent to which relevant stakeholders are willing and
able to participate in the system. Strong participation goes beyond merely sharing
information with stakeholders, allowing them to take part in decision-making and
setting guiding principles, rules, or standards. Participation underpins accountability
72 W. Ran et al.
by supporting avenues which allow outsiders to challenge decisions. In FIPP systems,

participation breeds trust in the system among participants. Strong participation
increases the relevance and completeness of the data by increasing the chance that a
stable consensus will be reached regarding such issues as what data should be dis-
closed and which standards should be adopted.
Consumer Trust
When additional non-observable information about products is provided, products

become “credence goods” (Nadai 1999). Many consumers are willing to pay a
higher price for credence goods, and this induces producer opportunistic behavior
(Karl and Orwat 2000). An example of this opportunistic behavior is “green-
washing”—the producer claims a product is more environmental-friendly than it
really is (Kirchhoff 2000). Green-washing results in consumers’ reluctance in buy-
ing credence goods. In order to make consumers willing to pay a higher price for
credence goods, the key is to gain consumer trust towards the information (Amstel
et al. 2008). Rogers (1983, 1995) suggested product adoption and diffusion is a
function of the willingness to use. As far as FIPP systems are concerned, we hypoth-
esize the consumer’s willingness to trust is a significant factor that determines the
willingness to use.
Trust influences consumer behavior and is an important concept in the buyer–
seller relationship (Milne and Boza 1999). In the buyer–seller relationship, trust is
defined as perceived credibility and benevolence of a target of trust (Doney and
Cannon 1997; Kumar et al. 1995; Larzelere and Huston 1980). The credibility refers
to the buyer’s perceived credibility of the seller. The benevolence refers to the sell-
er’s interest in the buyer’s interests. Marketing research on trust focuses on trust to
supplier firms and trust to salespeople (Doney and Cannon 1997).
In adoption and diffusion studies on innovations similar to FIPP systems, e.g.,
online recommendation agencies, relationship-aggregation services, and online
shopping, consumer trust has been included in many research theoretical frame-
works and been found as an important factor to the adoption and diffusion process.
For example, in studies on web recommendation agencies conducted by Wang and
Benbasat have found (2005, 2005, 2007), study results showed that consumer trust
influences consumer attitude towards recommendation agencies and the consumer’s
intention to use. In Pavlou and Fygenson’s study (2006) on e-commerce adoption,
the researchers built a theoretical framework based on the Technology Acceptance
Model and the Theory of Planned Behavior and included the concept of consumer
trust into the framework. They considered trust an antecedent of attitude and per-
ceived behavioral controls. Ruyter et al. (2001) studied the adoption of the inte-
grated customer online support service using the theory of Diffusion of Innovations.
Besides traditional factors such as relative advantage and consumer intention, they
also have considered perceived service quality, perceived risk, and consumer trust.
In e-commerce, consumer trust has been found as a key factor for the success of
business (Gefen et al. 2003; McKnight et al. 2002). On the other hand, a low level
of consumer trust has been found as a great barrier for e-commerce to fully realize
its potential (Urban et al. 2000).
Information Quality and Trustworthiness
Quality is one of the important factors that determine the perceived usefulness
(Boyd and Richerson 1985; Henrich 2001), and the perceived usefulness directly
influences the consumer’s attitude towards a product (Davis et al. 1989). In FIPP
systems, the product is the information carrying unobservable product attributes.
Given the distinctive nature of information, the usefulness and quality of informa-
tion does not mean the same as the usefulness and quality of other commodities.
Although our review has not found a unified definition of information usefulness or
quality, we have found in many studies, trustworthiness is considered the most sig-
nificant feature of good information. The trustworthiness of information has differ-
ent meanings under various contexts, and can be determined by or evaluated from
different aspects. The following describes two frameworks that are most relevant to
our study and can be applied to operationalize and evaluate the trustworthiness of
information provided by FIPP systems in a simulation model.
Amstel et al. (2008) evaluated the trustworthiness of eco-labels from four aspects.
They are the completeness of information, i.e., if the information is sufficient; the
usefulness of information, i.e., if the information about impacts and consequences
is available (Nyborg 1999); the presence of recommendations, and the compliance
with the rule of law. Consistent with the Theory of Planned Behavior, the usefulness
of information in this framework addresses the importance of the outcome of action
(Ajzen 2002). The rule of law refers to the principles of separation of powers,
democracy, and legal equity (Van Schooten-Van der Meer 1997). This a means to
diminish the power asymmetry in the seller–buyer relationship caused by informa-
tion asymmetry. Sellers who possess full and complete information are the stronger
party in this relationship. Consumers who do not have complete information are the
weaker party. The principle of separation of powers ensures the objectivity of the
information. For example, if an eco-label follows the principle of separation of pow-
ers, the party who sets eco-label standards (standardization body), the party who
controls issuing certifications (certification body), the party who audits the stan-
dardization and certification bodies (accreditation body), and people who carry out
these standards (producers) should be separate and independent parties. This sepa-
ration enhances the objectiveness of the information carried by the eco-label (De
Graaff 1995). The principle of democracy improves the trustworthiness of informa-
tion by encouraging participation. If more parties are involved in the process of
producing the information, e.g., setting eco-label standards, then the information
will be more trustworthy. The principle of legal equity means the information about
standards and the compliance with these standards should be transparent and actions
of actors in the product supply chain should be traceable.
74 W. Ran et al.
The second relevant and useful framework for improving the trustworthiness of
information is the framework developed by Wang and Benbasat (2007) based on
their study that investigated how to improve consumer trust in online recommenda-
tion agencies. Their study shows that the explanation facility is important for the
success of recommendation agencies, since the explanation facility can help to build
consumer trust by answering: how the final recommendations are reached; why the
recommendation agency asks consumer certain questions; and what trade-offs are.
According to Wang and Benbasat, the “how” question diminishes the information
asymmetry and improves consumer competence; “why” explains recommendation
agency’s intentions and shows its benevolence; “trade-off explanation” exhibits a
sense of justice, honesty, and objectivity, which makes consumers more willing to
trust (Mayer et al. 1995; Wang and Benbasat 2007).
Summary
Extrapolating from this discussion, how should a potential FIPP system be

governed? We can say that an FIPP system should have three key governance prin-
ciples which support and enhance each other: accountability, transparency, and par-
ticipation. And that these key principles should support four key characteristics of
the information within the system: completeness, reliability, openness, and rele-
vance. The information disclosed should be as complete as possible, meaning both
that it should tell us as much about the supply chain as possible and that the number
of missing entries should be minimized. It should be as reliable as possible, mean-
ing that it should accurately reflect the realities of the product supply chain. It should
be as open as possible to allow verification and promote its use and reuse. And it
should be as relevant as possible to promote further disclosure as well as consumer
use of the system.
In light of the above discussion, we hypothesize that the key to successful market
penetration of I-Choose system is a governance system that supports complete, reli-
able, open, and relevant data in FIPP systems and which can eventually promotes
consumer trust in product information.
Methods
Simulation methods have been recognized as useful ways to build and test theories
in the social sciences (Davis et al. 2007; Hanneman 1987, 1995). System Dynamics
in particular has been associated with other qualitative theory-building methods as
a powerful way of developing robust dynamic theories for social phenomena
(Andersen et al. 2012; Black et al. 2004; Kim and Andersen 2012; Kopainsky and
Luna-Reyes 2008; Luna-Reyes and Andersen 2003). In fact, building small simula-
tion models has been recognized as a way to incorporate current knowledge about a
system in order to better understand complex relationships among variables, and

refining our understanding of basic theories to continue with empirical research
(Davis et al. 2007; Ghaffarzadegan and Andersen 2012). There are in the literature
many different examples on ways in which simulation models have been used to
refine theories (Black et al. 2004; Ghaffarzadegan and Andersen 2012; Luna-Reyes
and Gil-Garcia 2011; Zagonel et al. 2004). Moreover, given the interdisciplinary
nature of our work, we have chosen group model building as a way to integrate in a
single model a diversity of points of view, using the system dynamics model as a
boundary object in this collaborative theory-building process (Black and Andersen
2012). Group model building obtains insightful model structure and extends the
ownership of the model by involving a group of people in the model conceptualiza-
tion and model formulation (Andersen and Richardson 1997; Richardson and
Andersen 1995; Vennix 1996).
Overall, our research progressed through three methodological phases: (1) A
large-scale meeting with stakeholders in the I-Choose supply chain to elicit key
system concepts to be modeled, (2) A smaller scale and more formal group model
building project to draw model maps, involving only team researchers who had
been present at the larger stakeholder meetings, (3) The creation of a running small
model based on the previous two steps. Each of these steps is described in more
detail below.
Stakeholder involvement at various stages of system development is regarded as
a key success factor in system development and implementation (Robey and Farrow
1982). However, most information systems literature only considers internal stake-
holders, largely overlooking the influence of external stakeholders to system devel-
opment (Pouloudi 1999). In order to create a prototype of the standards necessary to
share supply chain information and a prototype of a governance system to support
these standards, the I-Choose project has created a network of stakeholders associ-
ated with the coffee supply chain including both internal and external stakeholders
(Zhang et al. 2012).
We had a workshop with these stakeholders in a two-day meeting in August
2011. The goals of the workshop were to understand who the main stakeholders of
a system like I-Choose were, and the key issues to be considered in the develop-
ment of I-Choose. The workshop involved a series of brainstorming and discussion
sessions that have informed our modeling process (Zhang et al. 2012). We also
conducted a series of follow-up interviews with these stakeholders. The objective
of these interviews was to further our empirical knowledge about the socio-techni-
cal I-Choose system to be modeled and simulated. The interview data was used to
derive model assumptions and calibrate parameter values in our later model-
building effort.
Similar to other theory-building efforts (Luna-Reyes et al. 2006), the second
stage of the model involved a group-model-building exercise involving researchers
from multiple disciplines. This Group Model Building Efforts consisted in a 3-h
meeting to elicit key variables, key model assumptions, boundary considerations,
key reference modes, and an initial model structure (Luna-Reyes et al. 2013).
76 W. Ran et al.
Finally, we followed standard system dynamics practices to develop the simula-

tion model (Richardson and Pugh 1981; Sterman 2000). We used Vensim simula-
tion software to build the model. As in many other simulation projects, the iterative
process has resulted on several model versions. The one presented in the model is a
parsimonious theory of market penetration.
Model Structure and Behavior
In this section of the chapter, we introduce existing adoption and diffusion models,
describe the structure of our simulation model, and report simulation experiments.
Existing Adoption and Diffusion Models
Classic diffusion theories explain the adoption and diffusion process as the result of
aggregated imitation behaviors triggered by social interactions and communica-
tions. The majority of potential adopters are risk-averse, and they make adoption
decisions under uncertainty through the evaluation of risks and benefits of adoption.
The left small portion of potential adopters who are adventurous and innovative
becomes pioneers and early adopters. Through communications and social interac-
tions, the early adopters’ behavior influences the risk–benefit evaluations of other
potential adopters. Gradually, more and more potential adopters start accepting the
technology under influences such as the word of mouth or marketing. A representa-
tive of this view of adoption and diffusion phenomena is the famous Bass Model
(1969). The key hypothesis of the Base Model is that the probability that an initial
purchase will be made at a certain time is a linear function of the previous buyers.
The bass model addresses the importance of early adopters and other adopters’
imitation behavior. The pressure operating on imitators increases as the number of
previous buyers increase.
Besides imitation behaviors, existing theories also attribute adoption and diffu-
sion phenomena to adopters’ rational economic choices. Potential adopters make
their adoption decisions based on the cost–benefit evaluation. If the consumption
benefit exceeds the price, the product will be adopted, while the consumption ben-
efit is determined by the product’s utility and performance. This point of view is
reflected in the Diffusion of Innovation theory (Rogers 1995), the Technology
Acceptance Model (Davis et al. 1989), the Theory of Planned behavior (Ajzen
1991), and the Expectation Confirmation Theory (Oliver 1993). These theories and
models suggest the adoption decision is a result of the consumer’s intention of use.
The intention of use is influenced by the consumer’s attitude towards the product,
which, in turn, is determined by a range of internal and external factors. Internal
factors refer to the consumer’s and product’s characteristics, such as the consumer’s
self-efficacy and the product’s usefulness. External factors refer to environmental
factors such as social influence or communications.
Most diffusion simulation models conceptualize the adoption and diffusion phe-
nomenon as the result of both consumers’ imitation and rational behaviors (Dattee
et al. 2007; Dattee and Weil 2005; Homer 1987; Weil and Utterback 2005); so is our
simulation model. We built our simulation model and made model assumptions
based on existing models in diffusion literature and empirical data collected in the
I-Choose project. The model in its current form includes the universe of producers
who could potentially contribute information to the system as well as consumers
who select to use the information to make better retail purchase decisions. Producer
interest in the system is assumed to be influenced by a consideration of the costs and
benefits implied in joining the initiative (Ladd 2010; Malhotra et al. 2005).
Consumers are assumed to become active users when there are many producers and
suppliers contributing information to the system, and when the information pro-
vided by the system is both trustworthy and of high quality (Sathiyamoorthy et al.
2010; Xiao and Benbasat 2007). The model makes a series of assumptions about
how an open governance structure can contribute both to high quality and trustwor-
thy data.
Model Structure
Figure 1 is an abstracted view of our model structure, which illustrates the main
causal loops that we now believe to be operating in the system. There are two major
reinforcing loops that dominate the system behavior. The increase of the number of
producers and the governance openness and trustworthiness of data will lead to the
growth of green consumers using the system. More system users will cause higher
benefits to costs ratio, which will result the increase of the number of producers
providing data to the system. In the meantime, more producers will bring more
resources to support the system, reduce net costs of the system and lead to higher
benefits to costs ratio of the system.
These two loops can work in both directions. Once the system is over a key “tip-
ping point” where there are BOTH enough consumers to make it profitable for a
producer to join the system AND there are enough producers using the system to
provide consumers high enough utility to bother using the system, then a take-off in
both consumer and producer adoption of the system will take place. On the other
hand without enough consumers using the system, it is not cost-effective for the
marginal producer to join the system. And without producers adding their data to
the system, consumers find little utility in using the overall system. In these cases,
the reinforcing loops create a trap that prevents the system from taking off. Even in
the presence of a technically perfect prototype system with a governance system
that would insure absolute consumer trust in the information in the system, overall
grown will not occur as the system remains locked down the trap produced by these
two positive loops.
78 W. Ran et al.
Governance Openess
and Trust Worthiness of
Data
+ +
Utility of System for
Green Consumers
Producers Providing
Data to the System +
Green Consumers
+ Using the System
R
R
+
Benefits to Costs
Resources to + +
Support the System + - Financial Benefits
to Producers
Costs of the
System
Fig. 1 Main causal loops—Forces influencing system take off
Model Description
Of course, the final reduced form model is somewhat more complicated than the
high-level view provided in Fig. 1. Our latest version of the model has two main
sectors, Growth of Green Market and Cost–Benefit Evaluation. Key variables and
causal relationships among variables in each sector are presented in Figs. 2 and. 3,
respectively. As shown in Fig. 2, we mainly focus on green consumer behaviors at
the current stage of model development. We conceptualize three different subgroups
in green consumer population, green consumers who are using the I-Choose system,
those who are not using the system, and those who are former but not current system
users. The basic model sector structure is an adaption of the classic diffusion model.
There are two paths by which non-system users would become system users. One is
through the influence of word of mouth, and the other is through marketing. The
degree of the influence of word of mouth depends on system attractiveness and the
number of existing system users, while the degree of the influence of marketing is
determined by available marketing budgets and the number of green consumers
who are not using the system.
Importantly, retention of current users of the system depends critically on
“System Attractiveness,” which in turn is ultimately a function of the “Market
Share” of the producers supplying information to the system and the long-term
“Quality of the Information in the System” (see Fig. 2). That is, consumers can be
induced to first try out the system via one of two dynamic mechanisms (word of
mouth or direct marketing efforts), but another dynamic relating to overall quality
of the system will drive whether or not they stay with the system in the long run.
Governance Completeness Governance Openness

Relevance Reliability Transparency
+
+ Quality of
Market Share of
the System Information in the
System
New System Users +
+ + +
from Word of Mouth System
per Year System Capacity
+ Attractiveness
+
+ -
Green Consumers + Former Users of
Green Consumers Not
Using the System the System
Using the System New System System Users
Users per Year Leaving per Year
+ Marketing Budget
+ per Year
New System Users from +
Marketing per Year
Fig. 2 Key variables and causal relationships in model sector I—Growth of Green Market
- Producer
Total System Unregistration Rate
Cost per Year
-
Benefit Cost Ratio + + -
+ +
+
Total Number of Producers
Total Benefit per Producers Producers Unregistered per Year Unregistered
Year registered in Producers
+ + the system
Marketing Cost
per Year Producers
registration per year
+ +
Green Consumers Fixed System Building
and Maintaining Cost per -
Using the System +
Year Market Share of
the System
Benefit Collected from
per Green Consumer per
Year + Producer
Registration Rate
Fig. 3 Key variables and causal relationships in model sector II—Cost–benefit evaluation
As shown in Fig. 2, the key variable “System Attractiveness” is a combined

effect of quality of information in I-Choose system and the market share of the sys-
tem. The quality of information is determined by three system characteristics, the
“System Capacity,” the “Governance Completeness, Relevance, and Reliability,”
and the “Governance Openness and Transparency.” The last two variables reflect
key governance principles derived from our former theorizing and discussion. The
market share is determined by producer behaviors, their willingness to register as a
certificated producer in the I-Choose system and disclose full product information
to consumers. In our current model, producers’ willingness to use the system
80 W. Ran et al.
depends on the benefit–cost evaluation of the system, which is in turn determined by

the number of green consumers who are using the system and the total cost of build-
ing, maintaining, and marketing the system (see Fig. 3 for details).
Simulation Experiments
Through the iterative model-refinement process and extensive experiments, we have

learned that model behaviors are greatly shaped and confined by factors in three
dimensions: (1) The information quality, which is the reflection of governance fac-
tors in the model as well as the overall and final system capacity. These are key
determinants of whether or not consumers continue to use the system. (2) System
fixed costs per year which drive the overall benefit-cost ratio that controls whether
or not new producers choose to join the I-Choose system. (3) The variables that
reflect marketing efforts. Variables associated with marketing impact on producer
decisions to join I-Choose because marketing costs are part of the benefit–cost cal-
culation that drive new producer acquisition. In turn, marketing also promotes ini-
tial uptake of I-Choose by consumers, helping to push the system towards a
consumer-driven “take-off” point. Hence, marketing variables impact both on con-
sumer and producer market share.
Therefore, we conducted a series of experiments to explore the influence of
factors in these three dimensions. Figures 4 and 5 illustrate two sets of experimental
results designed to shed insight on the dynamics that are driving the model. In this
Green consumers using the system

23 23 23
30,000 23
23
2 1 1 1 1
1
22,500
2
Person
6 6 6 6 6 6
15,000 6 1
4 4
3 4 4 4
4
7500
6 4
1
2 4 3
3
0 1 5 5 5 5 5 5 5 5
7 7 7 7 7 7 7
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Time (Year)
Green consumers using the system : High information quality (0.8) Marketing efforts (5000 person per year) Fixed costs ($1) 1 1
Green consumers using the system : Highest information quality (1) Marketing efforts (10000 person per year) Fixed costs ($1) 2 2
Green consumers using the system : Highest information quality (1) Marketing efforts (3000 person per year) Fixed costs ($1) 3 3
Green consumers using the system : Medium information quality (0.6) Marketing efforts (5000 person per year) Fixed costs ($1) 4 4
Green consumers using the system : Medium-high information quality (0.7) Marketing efforts (1000 person per year) Fixed costs ($1) 5 5
Green consumers using the system : Medium-high information quality (0.7) Marketing efforts (7000 person per year) Fixed costs ($1) 6 6
Green consumers using the system : Very low information quality (0.04) Marketing efforts (100 person per year) Fixed costs ($1) 7 7
Fig. 4 Information quality drives final equilibrium (market share)

Green consumers using the system

20,000
15,000
Person
10,000
2 2 6
6 6
5000 2 2 6 2 6 2 6 2 6 2 6
1 4 1 4 1 4 1 4 1 4 1 4 1 4
4
3 3 3 3 3 3 3
3
0 5 5 5 5 5 5 5 5
1 7 7 7 7 7 7 7
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Time (Year)
Green consumers using the system : High information quality (0.8) Marketing effort (5000 person per year) Fixed costs ($1million) 1
Green consumers using the system : Highest information quality (1) Marketing effort (20000 person per year) Fixed costs ($1million) 2
Green consumers using the system : Highest information quality (1) Marketing effort (3000 person per year) Fixed costs ($1million) 3 3
Green consumers using the system : Medium-high information quality (0.6) Marketing effort (5000 person per year) Fixed costs ($1million) 4
Green consumers using the system : Medium-high information quality (0.7) Marketing effort (1000 person per year) Fixed costs ($1million) 5
Green consumers using the system : Medium-high information quality (0.7) Marketing effort (7000 person per year) Fixed costs ($1million) 6 6
Green consumers using the system : Very low information quality (0.04) Marketing efforts (100 person per year) Fixed costs ($1million) 7 7
Fig. 5 Final market share collapses without external subsidy of system costs
reduced form model, there are a total of 30,000 potential green consumers in the
system being supplied by a total of 100 producers.
In Fig. 4, we can see that when system costs are low enough to neglect (one dol-
lar in this set of simulations), the amount of marketing efforts determines how
quickly the number of green consumers using the system grows (the system starts to
take off), but the final system equilibrium, the market share of the I-Choose system,
is determined by the level of quality of information provided by the system. In all of
these runs, the operational costs of operating the system are low enough so that
producers will ultimately opt into the system if and when the customer base grows.
Initial growth is promoted by the marketing budget and support by a word-of-mouth
campaign explaining why in these runs the market takes off. However, long-term
equilibrium is determined not by consumers’ initial decision to try out the I-Choose
system, rather by their longer term decision to stay with the system (to not quit
using the system). For this set of decisions, the model assumes that long-term infor-
mation quality is the dominant factor. The model further assumes that governance
variables drive information quality.
Figure 5 illustrates the combined effect of factors in all three dimensions. The
effects of information quality and marketing factors remain the same, but with
regular system costs (one million dollars in this set of simulations) and without
external subsidy of the costs, final market share collapses eventually. In Fig. 4, final
the final equilibrium level for green consumers was dominated by consumer deci-
sions to stay in the system. The highest possible information quality (Information
Quality = 1) yielded the largest retention of users and hence in equilibrium all of the
82 W. Ran et al.
30,000 eligible green consumers eventually became system users. In Fig. 5, even
with the highest possible quality of information, the final equilibrium is much lower,
not because of information quality, but rather because participation by producers is
lower. Producer participation is lower because the benefit–cost calculation that
drives consumer signing up is lower and prevents all producers from joining the
system. The initial push from marketing coupled with word-of-mouth effect is suf-
ficient to support a transient growth in the customer base, but when enough produc-
ers fail to come online the overall system overshoots and eventually falls back to an
equilibrium that is jointly determined by the quality of the system as perceived by
consumers and the benefit–cost ratio of the system as perceived by the producers.
Discussion of Overall Dynamics in the Model
Our simulation experiments show that take-off dynamics of the I-Choose system are
dominated by marketing budgets and investment support for system costs. At the
beginning stage of the diffusion process, the number of green consumers using the
system resists growing unless external financial support can be found to pay for
marketing and to cover system costs, although in the long run, after a take-off in
both consumer and producer adoption of the system, marketing, and system costs
will be well covered by accelerated growth in financial benefits as indicated by our
recent research findings. On the other hand, the quality of governance and the resul-
tant trustworthiness of information drives final sustainable market share. So we
come to the conclusion that there appears to be a mutual dependence between pri-
vate support for market development and the openness of governance structures
regulating green product information systems such as I-Choose. Ultimately, the
economic success of I-Choose-like systems is determined both by the dynamics of
take-off (dominated by private investment and marketing) and the final market share
of the system (related to long-term information quality assumed in our model to be
an indicator of governance structures). Direct private investment in infrastructure
and in marketing is critical to market take-off, and an open governance structure is
a key determinant of final market share of the information commons.
Theoretical Implications
Our simulation results show that marketing effort and initial investment are crucial
to overcome the strong forces of reinforcing loops that resist the growth of consum-
ers at the beginning stage of diffusion, and a governance system that promotes con-
sumer trust can maintain a high level of market penetration in the final equilibrium
and sustains commercial success. These findings are consistent with findings from
previous research but advances by explaining when and how key forces and their
interactions contribute to the diffusion process. In addition, our study addresses the
importance of marketing strategy and governance to successful market take-off and

penetration. These supply-side nontechnical processes and factors are often
neglected in previous diffusion studies (Miller and Garnsey 2000).
Previous studies investigated forces that drive the diffusion process. Many stud-
ies have found that media advertising and market communication provide consum-
ers supply-side positive signals about products and are an important force that drives
the diffusion process (Bass et al. 1994; Krishnan et al. 1999, 2000). A different
perspective on diffusion addresses the significant role of social influence in the dif-
fusion process. Studies from this perspective believe consumers choose to use a new
product in pursuit of conformity of group behavior. A most common example of
social influence is word of mouth (Attewell 1992), which has been found as a strong
force in spreading new products (Krishnan et al. 2000). Some researchers also sug-
gest the formation of “critical mass” in the diffusion process is tremendously crucial
for the market take-off (Markus 1987). If the size of the user base is not large
enough, then the technology will be abandoned. Our findings have unified these
views and explained their linkages. Initial investment provides sufficient marketing
budget to accumulate a big enough consumer basis. With the help of word of mouth,
this consumer basis is able to develop into the critical mass that leads to the accel-
eration of the market penetration. A good governance system guarantees consumer
trust and sustains the high level of market penetration in the long run.
Implications for Practitioners
Consumer Trust is a key issue in FIPP information systems. Our earliest case-study
research into relatively small-scale systems indicated that establishing a trusted
relationship between consumers and producers is a key feature of all successful
FIPP systems. Simply put, if a consumer is going to pay a premium for a product
based on an expanded information package, then the customer needs to be able to
trust the content of that information package. Most producers and distributors in
small to moderate scale FIPP systems pay a lot of attention to creating and sustain-
ing trust, according to our interview data. So the question arises of how do large-
scale FIPP systems, spanning multiple producing and distribution organizations,
both create and sustain consumer trust? This question is especially challenging
when one realizes that those who produce the information package (producers and
distributors) most likely have a strong financial interest in the content of the package
as well as many opportunities to bias or even falsify the information package. Some
of our interviews, especially in Latin America indicated that in many instances,
government-certified programs fared no better in terms of citizen trust (presumably
due to lack of high performance standards or even the perception of graft and/or
corruption) (Luna-Reyes et al. 2012a). Therefore, good governance of FIPP systems
is required to promote trust in the system and its data. Our simulation results dem-
onstrate the governance that can promote consumer trust is critical to the sustaining
market success of FIPP systems. Third party certifiers, consumer advocates, and
84 W. Ran et al.
government regulators are the stakeholders who must design and insure this trusted
governance structure.
Our findings suggest that the initial investment is crucial for the market take-off
of FIPP systems since it provides financial resources to support system construction
and marketing effort that helps to accumulate a big enough consumer base for con-
tinuing growth in the market share. In our recent research development, we have
also observed a significant time lag between when the initial investment is made and
when the return on that investment starts to grow. Private firms are likely to be
daunted by the high risk of big investment in FIPP systems. The lagged investment
return may give business decision-makers wrong signals, make them doubt a suffi-
cient investment return, and be blind to the high-reward potential of the investment
in FIPP systems. Even if private firms have realized financial benefits in the long
run, they might not be powerful enough to involve all stakeholders in the supply
chain, afford the big investment in building the infrastructure, and insure a gover-
nance structure that promotes consumer trust. Therefore, support from the public
sector, either in the form of financial assistance or favorable regulations or policies,
should be provided to help launching I-Choose-like enterprises that will contribute
to sustainable consumption and production and the larger common good.
Conclusion and Future Work
The larger I-Choose project within which this pilot simulation study is nested is an
exploration of how a particular type of web-based technology, i.e., semantic web tech-
nologies based on OWL ontologies, can create an information system to produce a
kind of open source product information about non-observable product traits to a wide
range of consumers. We are seeking to create a pilot FIPP system. If we assume that
an FIPP system such as I-Choose is totally feasible from a technical point of view, this
study explores what other conditions must exist in markets and in governance systems
for such a system to achieve market share and become a commercial success.
We have found that a deep partnership between commercial producers and sup-
pliers in the supply chain and others involved in the governance of such a system
will be critical to commercial viability. In our simulator, producers and suppliers are
necessary to provide marketing push and to support the development of system
infrastructure. Third party certifiers, consumer advocates, and government regula-
tors are the stakeholders who must design and insure a trusted governance struc-
ture—who must craft and monitor the “rules of the road,” who must stay on top of
how these systems are being governed. Without trust arising from governance cou-
pled with market share driven by private investment FIPP product information sys-
tems such as the I-Choose system will not become a commercial reality even if they
are technically feasible.
The reduced form model described and analyzed in this chapter depends criti-
cally on a number of structural assumptions that need to be the subject of future
research. Several of these key structural assumptions are briefly discussed below.
Business Model Assumes that Producers Pay All System Costs Our simulations
assume that both the marketing costs and the infrastructure development costs of an
I-Choose-like system are to be borne by producers. When total benefits fall short of
total costs, producers will not join the system. In another business model, system
costs or some share of marketing costs could be somehow shared in a broader com-
mons supported by the government or a broader alliance of Green producers.
Different Dynamics for Short-Term versus Long-Term System Adoption A key
assumption of our model is that word of mouth and marketing first draw consumers
into using the I-Choose system but that other longer term factors associated with
quality and trustworthiness of the information retain customer loyalty. These
assumptions explain much of the differences between transient take-off and long-
term equilibrium dynamics.
Scaling Effects on Fully Investigates Our model assumes a specific hypothetical
scale—100,000 consumers in total, 30,000 potential green consumers, only 100
producers, specific benefit and cost figures, and so on. Varying these specific para-
metric values will deflect typical dynamics in a base run and in policy runs.
Governance is Highly Related to Quality of Information This overall research
effort began with an effort to investigate the impact of various governance regimes
on overall market growth. As we moved towards the reduced form model, many of
the original co-flows and possible dynamics associated with governance were made
more and more simple. The core remaining linkage between a complex discussion
of governance motivating this work and the formal simulation runs in the reduced
form is the (quite plausible) set of assumptions that (a) consumers’ decisions to
remain in the system are dominated by overall quality of information in the system,
and (b) governance issues will be important determinants of trustworthiness of
information which are equivalent to information quality. While these assumptions
are certainly plausible, they have not been empirically proven and tested.
Part of the Supply Chain and Market Structure is Missing In order to capture the
basic and most fundamental dynamics in the I-Choose system, our reduced form
model ignores many complications that need to be considered. More stakeholders
other than just producers need to voluntarily supply information to a Green product
information system such as I-Choose. Heterogeneity exists among potential adopt-
ers in terms of their economic status, values, and inclinations towards sustainable
consumption; therefore, potential adopters should consist of subpopulations, and
transformations between subpopulations are possible as the market and social envi-
ronments change. Market competition is not conceptualized in our model while in
diffusion research it is usually treated as an essential contributor to the dynamics of
innovation. A typical diffusion process starts with a market full of evenly matched
competitors and will eventually develop into a market dominated by one or a few
firms. Given the extensive involvement of a large number of stakeholders in the sup-
ply chain and the huge investment to launch the I-Choose system, we argue it is very
unlikely there will be many evenly matched competitors in the market under the
context of our study. Nevertheless, we will conceptualize market competition in our
future simulation models and investigate its impact on the diffusion dynamics.
86 W. Ran et al.
Appendix: Model Equations
Sector I: Growth of Green Market
Average green contacts per person per year = 365 Units: Person/Person/Year
Chance of recommending = 2/365 Units: Dmnl
Chance of successfully recommending upon recommending = WITH LOOKUP (
Effective system attractiveness,([(0,0)-(1,1)], (0,0), (0.16,0.02), (0.16,0.02),
(0.27,0.02), (0.36,0.02),(0.43,0.04),(0.51,0.06),(0.68,0.14),(0.88,0.30),(1,0.
5))) Units: Dmnl
Effective system attractiveness = System attractiveness * System attractiveness
enabler + System attractiveness CONSTANT * (1-System attractiveness enabler)
Units: Dmnl
Former Users of System = INTEG (System users leaving per year-System users
reconsider using the system, 0) Units: Person
Fraction of LOHAS = 0.3 Units: Dmnl
Fraction of system users leaving = WITH LOOKUP (
Effective system attractiveness, ([(0,0)-(1,1)], (0,1), (0.05,0.97), (0.15,0.96), (0.22,
0.94),(0.33,0.87),(0.49,0.49),(0.64,0.22),(0.81,0.03), (1,0))) Units: 1/Year
Fraction of system users leaving per year = Fraction of system users leaving/Time to
leave Units: 1/Year
Green consumers can be reached by the marketing effort each year = Green consum-
ers not using the system/Minimum time to reach a consumer not using the system
Units: Person/Year
Green consumers not using the system = INTEG (System users reconsider using the
system-New system users per year, Total green consumers) Units: Person
Green consumers using the system = INTEG (New system users per year-System
users leaving per year, 0) Units: Person
Max possible marketing number per year CONSTANT = 100 Units: Person/Year
Minimum time to reach a consumer not using the system = 0.5 Units: Year
New system users from marketing per year = MIN (Green consumers can be reached
by the marketing effort each year, Max possible marketing number per year
CONSTANT) Units: Person/Year
New system users from word of mouth per year = “Non system-users’ green contacts
per year” * Probability of successfully recommending the system * Probability
that a contact is using the system Units: Person/Year
New system users per year = New system users from word of mouth per year + New
system users from marketing per year Units: Person/Year
“Non system-users’ green contacts per year” = Average green contacts per person
per year * Green consumers not using the system Units: Person/Year
Probability of successfully recommending the system = Chance of successfully rec-
ommending upon recommending * Chance of recommending Units: Dmnl
Probability that a contact is using the system = Green consumers using the system/
Total green consumers Units: Dmnl
System attractiveness = System utility Units: Dmnl

System attractiveness CONSTANT = 1 Units: Dmnl
System attractiveness enabler = 1 Units: Dmnl
System users leaving per year = Green consumers using the system * Fraction of
system users leaving per year Units: Person/Year
System users reconsider using the system = Former Users of System/Time to change
mind Units: Person/Year
Time for registration = 1 Units: Year
Time to change mind = 4 Units: Year
Time to leave = 1 Units: Year
Total consumers = 100,000 Units: Person
Total green consumers = Total consumers*Fraction of LOHAS Units: Person
Sector II: Cost–Benefit Evaluation
Benefit per system user per year = 16 * 2 Units: Dollar/Person/Year

“Benefit–cost ratio” = Total system benefit per year/Total system cost per year
Units: Dmnl
Fixed cost per year CONSTANT = 1 Units: Dollar/Year
Fraction of producers to be registered = WITH LOOKUP (“Benefit–cost ratio”,
([(0,0)-(1.5,1)], (0,0.0001), (0.5,0.01), (0.8,0.1), (1,0.2), (1.1,0.8), (1.2,0.9), (1.3,1)))
Units: Dmnl
Fraction of producers to be unregistered = WITH LOOKUP (“Benefit–cost ratio”,
([(0,0)-(1.5,1)], (0,1), (0.155294,0.960854), (0.310588,0.911032), (0.5,0.8), (1,0.1),
(1.1,0))) Units: Dmnl
Governance completeness relevance reliability = 0.2 Units: Dmnl
“Governance openness & transparency” = 0.2 Units: Dmnl
Green consumers using the system = INTEG (New system users per year-System
users leaving per year, 0) Units: Person
Marginal cost per producer per year = 1000 + 200 Units: Dollar/Producer/Year
“Market share of the system—scale” = Producers registered in the system/Total pro-
ducers Units: Dmnl
Marketing cost per person = 20 Units: Dollar/Person
Marketing cost per year = Marketing cost per person * New system users from mar-
keting per year Units: Dollar/Year
New system users from marketing per year = MIN (Green consumers can be reached
by the marketing effort each year, Max possible marketing number per year
CONSTANT) Units: Person/Year
Producers registered in the system = INTEG (Producers registration per year-
Producers unregistered per year, 0) Units: Producer
Producers registration per year = Unregistered producers * Fraction of producers to
be registered/Time for registration Units: Producer/Year
88 W. Ran et al.
Producers unregistered per year = Producers registered in the system * Fraction of

producers to be unregistered/Time for unregistration Units: Producer/Year
Products initially registered = Initiators * Products per producer Units: Product
“Quality of information in the system—scale—CONSTANT” = 0.04 Units: Dmnl
“Quality of information in the system—scale” = System capacity CONSTANT *
Governance completeness relevance reliability * “Governance openness &
transparency”/100 Units: Dmnl
System attractiveness = System utility Units: Dmnl
System capacity CONSTANT = 100 Units: CapacityProcess
System utility = “Market share of the system—scale”*“Quality of information in
the system—scale—CONSTANT” Units: Dmnl
Time for registration = 1 Units: Year
Time for unregistration = 2 Units: Year
Total producers = 100 Units: Producer
Total system benefit per year = Benefit per system user per year * Green consumers
using the system Units: Dollar
Total system cost per year = Fixed cost per year CONSTANT + Marginal cost per
producer per year*Producers registered in the system + Marketing cost per year
Units: Dollar/Year
Unregistered producers = INTEG (Producers unregistered per year-Producers reg-
istration per year, Total producers) Units: Producer
References
Abbott K, Snidal D (2009) The governance triangle: regulatory standards, institutions, and the
shadow of the state. In: Mattli W, Woods N (eds) The politics of global regulation. Princeton
University Press, Princeton, pp 44–88
Ajzen I (1991) The Theory of Planned Behavior. Organizational Behavior and Human Decision
Processes, 50:179–211
Ajzen I (2002) Constructing a TPB questionnaire: conceptual and methodological considerations.
http://www-unix.oit.umass.edu/~aizen/tpb.html
Andersen DF, Richardson GP (1997) Scripts for group model building. Syst Dyn Rev 13(2):
107–129 (Wiley)
Andersen DL, Luna-Reyes LF, Diker VG, Black L, Rich E, Andersen DF (2012) The disconfirma-
tory interview as a strategy for the assessment of system dynamics models. Syst Dyn Rev
28(3):255–275. doi:10.1002/sdr.1479
Amstel M v, Driessen P, Glasbergen P (2008) Eco-labeling and information asymmetry: a com-
parison of five eco-labels in the Netherlands. J Clean Prod 16(3):14
Attewell P (1992) Technology diffusion and organizational learning: the case of business comput-
ing. Organ Sci 3:1–19
Basiago AD (1995) Methods of defining ‘sustainability’. Sustain Dev 3:109–119
Bass FM (1969) A new product growth model for consumer durables. Manag Sci 15:215–227
Bass FM, Krishnan TV, Jain DC (1994) Why the bass model fits without decision variables. Mark
Sci 13:203–223
Bayat A, Sundararajan S, Gustafson HR, Zimmers EW (2011) Sustainability driven supply chains.
Ind Eng 43(8):26–31
Beales H, Craswell R, Salop SC (1981) The efficient regulation of consumer information. J Law
Econ 24(3):491–539
Black LJ, Andersen DF (2012) Using visual representations as boundary objects to resolve conflict
in collaborative model-building approaches. Syst Res Behav Sci 29(2):194–208
Black LJ, Carlile PR, Repenning NP (2004) A dynamic theory of expertise and occupational
boundaries in new technology implementation: building on barley’s study of CT scanning.
Adm Sci Q 49(4):572–607
Boyd R, Richerson PJ (1985) Culture and the evolutionary process. University of Chicago Press,
Chicago
Bray J, Johns N, Kilburn D (2011) An exploratory study into the factors impeding ethical con-
sumption. J Bus Ethics 98(4):597–608. doi:10.1007/s10551-010-0640-9
de Búrca G, Scott J (eds) (2006) Law and new governance in the EU and the US. Hart, Portland
Carrington M, Neville B, Whitwell G (2010) Why ethical consumers don’t walk their talk: towards
a framework for understanding the gap between the ethical purchase intentions and actual buy-
ing behaviour of ethically minded consumers. J Bus Ethics 97(1):139–158. doi:10.1007/
s10551-010-0501-6
Caswell JA, Padberg DI (1992) Toward a more comprehensive theory of food labels. Am J Agric
Econ 74(2):460–468
Commission of the European Communities (CEC) (2001) European governance: a white paper.
COM (2001) 428 final, 25 July, Brussels
Dattee B, FitzPatrick D, Weil HB (2007) The dynamics of technological substitutions. In:
Proceedings of the 25th international conference of the system dynamics society
Dattee B, Weil HB (2005) Dynamics of social factors in technological substitutions. Massachusetts
Institute of Technology, Sloan School of Management (Working Paper 4599-05)
Davis C, Nikolic I, Dijkema GPJ (2010) Industrial ecology 2.0. J Ind Ecol 14(5):707–726.
doi:10.1111/j.1530-9290.2010.00281.x
Davis FD, Bagozzi RP, Warshaw PR (1989) User acceptance of computer technology: a compari-
son of two theoretical models. Manag Sci 35(8):982–1003
Davis JP, Eisenhardt K, Bingham CB (2007) Developing theory through simulation methods. Acad
Manage Rev 32(2):480–499
De Graaff V (1995) Private certification in a governance context, an assessment towards commu-
nicative governance. Eburon, Delft
Doney PM, Cannon JP (1997) An examination of the nature of trust in buyer-seller relationships.
J Mark 61:35–51
Elofson G, Robinson WN (2007) Collective customer collaboration impacts on supply-chain per-
formance. Int J Prod Res 45(11):2567–2594. doi:10.1080/00207540601020528
Ghaffarzadegan N, Andersen DF (2012) Modeling behavioral complexities of warning issuance
for domestic security: a simulation approach to develop public management theories. Int Public
Manag J 15(3):337–363. doi:10.1080/10967494.2012.725566
Gefen D, Karahanna E, Straub DW (2003) Trust and TAM in online shopping: an integrated
model. MIS Q 27(1):51–90
Goleman D (2009) Ecological intelligence: how knowing the hidden impacts of what we buy can
change everything. Broadway Books, New York
Greer SL, Wismar M, Figueras J (2015) Strengthening governance for health systems. European
Observatory on Health Systems and Policies, Brussels
Hanneman RA (1987) Computer-assisted theory-building. Sage, Newbury Park
Hanneman RA (1995) Simulation modeling and theoretical analysis in sociology. Sociol Perspect
38(4):457–462
Henrich J (2001) Cultural transmission and the diffusion of innovations: adoption dynamics indi-
cate that biased cultural transmission is the predominant force in behavioral change. Am
Anthropol 103:992–1013
Héritier A, Lehmkuhl D (2008) The shadow of hierarchy and new modes of governance. J Public
Policy 28(1):1–17
90 W. Ran et al.
Homer J (1987) A diffusion model with application to evolving medical technologies. Technol
Forecast Soc Chang 31(3):197–218
Karl H, Orwat C (2000) Economic aspects of environmental labeling. In: Folmer H, Tietenberg T
(eds) The international yearbook of environmental and resource economics. Edward Elgar,
Cheltenham
Kaufmann D, Kraay A (2002) Growth without governance. Policy Research Working Paper 2928,
World Bank
Kim G-S, Lee G, Park K (2010) A cross-national investigation on how ethical consumers build
loyalty toward fair trade brands. J Bus Ethics 96(4):589–611. doi:10.1007/
s10551-010-0486-1
Kim H, Andersen DF (2012) Building confidence in causal maps generated from purposive text
data: mapping transcripts of the Federal Reserve. Syst Dyn Rev 28(4):311–328
Kirchhoff S (2000) Green business and blue angel, a model of voluntary overcompliance with
asymmetric information. Environ Resour Econ 15:403–420
Komiak SYX, Benbasat I (2006) The effects of personalization and familiarity on trust and adop-
tion of recommendation agents. MIS Q 30(4):941–960
Komiak SYX, Benbasat I (2008) A two-process view of trust and distrust building in recommenda-
tion agents: a process-tracing study. J Assoc Inf Syst 9(12):727–747
Kooiman J (1993) Social-political governance. In: Kooiman J (ed) Modern governance. Sage,
London
Kopainsky B, Luna-Reyes LF (2008) Closing the loop: promoting synergies with other theory
building approaches to improve system dynamics practice. Syst Res Behav Sci
25(4):471–486
Krishnan TV, Bass FM, Jain DC (1999) Optimal pricing strategy for new products. Manag Sci
45:1650–1663
Krishnan TV, Bass FM, Kumar V (2000) Impact of a late entrant on the diffusion of a new product.
J Mark Res 37:269–278
Kumar N, Scheer LK, Steenkamp J-BEM (1995) The effects of perceived interdependence on
dealer attitudes. J Mark Res 32:348–356
Ladd S (2010) Corporate “greening”: Good for the soul, but is it good for the bottom line?
Financial Executive, 26(2):36–41
Larzelere RE, Huston TL (1980) The dyadic trust scale: toward understanding interpersonal trust
in close relationships. J Marriage Fam 42(3):595–604
Luna-Reyes LF, Andersen DF, Andersen DL, Derrick D, Jarman H (2012a) Full information prod-
uct pricing (FIPP) regimes: policy implications for US-Mexico sustainable commerce. Center
for Technology in Government, Albany
Luna-Reyes LF, Gil-Garcia JR (2011) Using institutional theory and dynamic simulation to under-
stand complex e-Government phenomena. Gov Inf Q 28(3):329–345. doi:10.1016/j.
giq.2010.08.007
Luna-Reyes LF, Martinez-Moyano IJ, Pardo TA, Cresswell AM, Andersen DF, Richardson GP
(2006) Anatomy of a group model-building intervention: building dynamic theory from case
study research. Syst Dyn Rev 22(4):291–320. doi:10.1002/sdr.349
Luna-Reyes LF, Andersen DL (2003) Collecting and analyzing qualitative data for system dynam-
ics: methods and models. Syst Dyn Rev 19(4):271–296. doi:10.1002/sdr.280
Luna-Reyes LF, Ran W, Jarman H, Zhang J, Andersen DL, Tayi GK, Sayogo DS, Luciano J,
Andersen DF (2013) Group model building to support interdisciplinary theory building. Center
for Technology in Government, Albany
Luna-Reyes L, Sayogo DS, Zhang J, Pardo T, Tayi GK, Hrdinova J et al (2012) Beyond open
government: ontologies and data architectures to support ethical consumption. In: Proceedings
of the 6th international conference on theory and practice of electronic governance (ICEGOV),
Albany
Luna-Reyes LF, Zhang J, Whitmore A et al (2011) Full information product pricing: an informa-
tion strategy for harnessing consumer choice to create a more sustainable world. Commun
Assoc Inf Syst 34(1):637–654
Malhotra A, Gosain S, El Sawy OA (2005) Absorptive capacity configurations in supply chains:

gearing for partner-enabled market knowledge creation. MIS Q 29(1):145–187
Markus ML (1987) Toward a ‘critical mass’ theory of interactive media: universal access, interde-
pendence and diffusion. Commun Res 14:491–511
Mayer RC, Davis JH, Schoorman FD (1995) An integrative model of organizational trust. Acad
Manage Rev 20(3):709–734
McKnight DH, Choudhury V, Kacmar C (2002) Developing and validating trust measures for
e-commerce: an integrative typology. Inf Syst Res 13(3):334–359
Miller D, Garnsey E (2000) Entrepreneurs and technology diffusion: how diffusion research can
benefit from a greater understanding of entrepreneurship. Technol Soc 22:445–465
Milne GR, Boza M-E (1999) Trust and concern in consumers’ perceptions of marketing informa-
tion management practices. J Int Mark 13(1):5–24
Mossialos E, Permanand G, Baeten R, Hervey TK (2010) Health systems governance in Europe:
the role of EU law and policy. Cambridge University Press, Cambridge
Nadai A (1999) Conditions for the development of a product ecolabel. Eur Environ 9:202–211
Nissen ME, Sengupta K (2006) Incorporating software agents into supply chains: experimental
investigation with a procurement task. MIS Q 30(1):145–166
Nyborg K (1999) Informational aspect of environment policy deserves more attention: comment
on the paper by Frey. J Consum Policy 22:419–427
OECD (2002) Foreign direct investment for development- maximising benefits, minimising costs.
OECD Secretariat
OECD (2003) Public sector transparency and international investment policy. Directorate for
Financial, Fiscal and Enterprise Affairs
Oliver RL (1993) Cognitive, affective, and attribute bases of the satisfaction response. J Consum
Res 20:418–430
Pavlou PA, Fygenson M (2006) Understanding and predicting electronic commerce adoption: an
extension of the theory of planned behavior. MIS Q 30(1):115–143
Pouloudi A (1999) Aspects of the stakeholder concept and their implications for information sys-
tems development. In: Proceedings of the 32nd annual Hawaii international conference, IEEE
Computer Society, Hawaii, p 17
Punj GN, Moore R (2007) Smart versus knowledgeable online recommendation agents. J Interact
Mark 21(4):46–60. doi:10.1002/dir.20089
Richardson GP, Andersen DF (1995) Teamwork in group model building. Syst Dyn Rev
11(2):113–137
Richardson GP, Pugh AL (1981) Introduction to system dynamics modelling with dynamo.
Productivity, Cambridge
Robey D, Farrow D (1982) User involvement in information system development: a conflict model
and empirical test. Manag Sci 28:73–85
Rogers EM (1983) Diffusion of innovations, 3rd edn. Free, New York
Rogers EM (1995) Diffusion of innovations, 4th edn. Free, New York
Rosenau J (1995) Global governance in the twenty-first century. Glob Gov 1(1):13–43
Rosenau PV (ed) (2000) Public-private policy partnerships. MIT, Cambridge
Ruyter K d, Wetzels M, Kleijnen M (2001) Customer adoption of e-service: an experimental study.
Int J Serv Ind Manag 12(2):184–207
Sabel CF, Zeitlin J (eds) (2010) Experimentalist governance in the European Union: towards a new
architecture. Oxford University Press, Oxford
Sathiyamoorthy E, Iyenger N, Ramachandran V (2010) Agent based trust management framework
in distributed e-business environment. Int J Comput Sci Inf Technol 2:14–28
Sterman JD (2000) Business dynamics: systems thinking and modeling for a complex world.
Irwin/McGraw-Hill, Boston
Urban GL, Sultan F, Qualls WJ (2000) Placing trust at the center of your internet strategy. Sloan
Manage Rev 42(1):39–48
92 W. Ran et al.
Van Schooten-Van der Meer H (1997) Formation rule in the rule of law. A study to the legitimization
and the operation of the right and the modern alternative forms of regulation. University Press,
Twente
Vennix JAM (1996) Group model building: facilitating team learning using system dynamics, 1st
edn. Wiley, Chichester
Vermeulen WJV, Uitenboogaart Y, Pesqueira LDL, Metselaar J, Kok MTJ (2012) Roles of govern-
ments in multi-actor sustainable supply chain governance systems and the effectiveness of their
interventions: an exploratory study. Environmental Assessment Agency (PBL), Bilthoven
Wang ETG, Tai JCF, Wei H-L (2006) A virtual integration theory of improved supply-chain per-
formance. J Manag Inf Syst 23(2):41–64
Wang W, Benbasat I (2005) Trust in and adoption of online recommendation agents. J Assoc Inf
Syst 6(3):72–101
Wang W, Benbasat I (2007) Recommendation agents for electronic commerce: effects of explana-
tion facilities on trusting beliefs. J Manag Inf Syst 23(4):217–246
Watts S, Wyner G (2011) Designing and theorizing the adoption of mobile technology-mediated
ethical consumption tools. Inf Technol People 24(3):257–280. doi:10.1108/09593841111158374
Weil HB, Utterback JM (2005) The Dynamics of Innovative Industries. In Proceedings of the 23rd
International Conference of the System Dynamics Society. Boston, MA
Woods N (1999) Good governance in international organizations. Glob Gov 5:39–61
Xiao B, Benbasat I (2007) E-commerce product recommendation agents: use, characteristics and
impact. MIS Q 31(1):137–209
Zagonel AA, Rohrbaugh J, Richardson GP, Andersen DF (2004) Using simulation models to
address “what if” questions about welfare reform. J Policy Anal Manage 23(4):890–901
Zhang J, Sayogo DS, Luna-Reyes LF, Jarman H, Tan X, Andersen DL, Andersen DF (2012) Issues
and requirements for developing data architecture supporting integration of sustainable con-
sumption and sustainable supply chains. Center for Technology in Government, Albany
Green Government Procurement:
Decision-Making with Rough Set,
TOPSIS, and VIKOR Methodologies
Chunguang Bai and Joseph Sarkis
Abstract Public and private organizations have started to respond to various stake-
holder and market pressures to improve their environmental and social sustainabil-
ity performance. Government agencies represent one of the most pertinent
stakeholders. Government stakeholder pressures to encourage greater organiza-
tional sustainability include coercive measures such as penalties, fines, and removal
of license to operate if organizations are unable to meet specific regulatory require-
ments. Yet, noncoercive approaches are also available to government agencies and
regulators for encouraging the greening of organizations and markets.
Introduction
Public and private organizations have started to respond to various stakeholder and
market pressures to improve their environmental and social sustainability perfor-
mance. Government agencies represent one of the most pertinent stakeholders.
Government stakeholder pressures to encourage greater organizational sustainabil-
ity include coercive measures such as penalties, fines, and removal of license to
operate if organizations are unable to meet specific regulatory requirements. Yet,
noncoercive approaches are also available to government agencies and regulators
for encouraging the greening of organizations and markets.
The pollution prevention act encouraged government agencies to help develop
noncoercive measures such as benchmarking and information sharing as tools to
help private and public organizations become greener. For example, the US govern-
ment’s 33/50 program was a voluntary, noncoercive, program to help organizations
C. Bai (*)
School of Management Science and Engineering, Dongbei University of Finance
and Economics, Jianshan Street 217, Dalian 116025, P.R. China
e-mail: chunguang.bai@dufe.edu.cn
J. Sarkis
School of Business, Worcester Polytechnic Institute, 100 Institute Road,
Worcester, MA 01609-2280, USA
e-mail: jsarkis@WPI.edu

DOI 10.1007/978-3-319-25439-5_5
94 C. Bai and J. Sarkis
improve their pollution prevention practices and go beyond compliance to govern-

ment regulations (Arora and Cason 1995). Another voluntary approach was through
an information-based regulatory requirement such as the toxics release inventory
(TRI) program. This program required organizations to gather and publicly release
information from a listing of hazardous materials. The only requirement was the
release of this information by organizations. But, releasing this information to the
public had the potential outcome of hurting the image and reputation of many orga-
nizations, especially those that released the largest quantities of hazardous materials
(Norberg-Bohm 1999; Deltas et al. 2014). Many organizations then responded by
reducing their emissions.
One other popular method by governmental bodies to help green industry and
product/service markets is through market mechanisms such as green (sustainable)
procurement programs (Marron 1997). Green government procurement (GGP) is a
program designed to purchase and contract with green firms and vendors and
focuses more on the “carrot” rather than “stick” approach to greening organizations.
There is a significant international effort for GGP (e.g., Ho et al. 2010; Michelsen
and de Boer 2009; Preuss 2009; Zhu et al. 2013), and may occur at local, national,
or international government agency levels. Investigating and understanding GGP’s
processes, practices, and approaches can be helpful at a global level and is not just
a localized concern.
A critical aspect to GGP is the identification and selection of appropriate ven-
dors based on greening and/or social metrics and not just business criteria. The
research in general green supplier selection has recognized the complexity of sup-
plier selection when environmental and social sustainability metrics are to be
included in the decision process (Bai and Sarkis 2010a; Govindan et al. 2015).
Government agencies may have to deal with thousands, if not millions, of potential
suppliers of a broad variety of products and services. These additional complexi-
ties and magnitudes for GGP make the supplier selection process a major under-
taking, depending on the size of the contracts. Usually, these contracts and decisions
are not completed by individuals but may require a group decision. Thus, tools to
help aid in this complex and multi-decision maker environment can be helpful to
governmental agencies.
In addition, governmental agencies may require and acquire substantial supplier
performance data comprising many fields and dimensions. This big data set will
need to be filtered and evaluated, similar to data mining, to determine the most per-
tinent and informative attributes. This filtration and evaluation will be critical for
effective and efficient application of the multiple decision maker, multiple criteria
decision approaches.
To help meet these practical requirements, we introduce a series of tools within
a broader methodology. The tools in this chapter are meant filter out decision factors
and aggregate decision maker inputs. Using illustrative data, the focus will be on the
methodological application contributions in this chapter. Practical implications for
the implementation of these tools and methodology, especially given the GGP envi-
ronment, are also discussed. The techniques and methodology are extensible to
other environments, public and private organizations.
Green Government Procurement… 95
Contrasting GGP and Corporate Green Supplier Selection
The majority of green procurement initiatives studies have focused on private orga-
nizations (McMurray et al. 2014). Green supply chain management in the private
setting has been synonymous with gaining competitive advantage through improv-
ing profitability of organizations (Zhu et al. 2012). Public, governmental agencies,
in response to their social welfare mission, have implemented environmental pro-
curement projects to further support the greening of various industries and commu-
nities (Zhu et al. 2013; Dou et al. 2014). Although there are similarities, it is
sometimes difficult to translate private organization procurement strategies and
modes into the government procurement activities, where certain rules and regula-
tions bound their decision processes and approaches (Mosgaard et al. 2013).
The analysis of both GGP and corporate green supplier selection differences indi-
cate what might be critical factors in establishing successful GGP initiatives. First is
the difference of the role: GGP is important to provide leadership through internal
changes to procurement policies, procedures, and contract award criteria for supply
and services contracts (CECNA/FWI 2003; Day 2005). GGP plays a crucial role
because the government as the single most important customer has a significant influ-
ence over the supply base (New et al. 2002). In this regard, there is a prevalent view
that “public authorities must act as ‘leaders’ in the process of changes in consumption
towards greener products” (Kunzlik 2003). The second difference is the size of pur-
chases: The state is usually a large-scale consumer, and government procurements are
often relatively much larger in terms of revenue. GGP of goods and services expendi-
tures range from 8 to 25 % of Gross Domestic Product (GDP) for Organization for
Economic Co-Operation and Development (OECD) member countries (OECD 2006).
This number falls in the middle, estimated at 16 %, for the European Union (EU) (EC
2004). Third is a difference in the procurement goal. GGP is utilized in order to meet
the needs of public service and public service activities while reducing damage to the
environment. The main purpose of private procurement is for profit.
Overall, two key differences have emerged between public and private sector
responses to environmental challenges: (a) the effect of regulation on procurement
practice and (b) the use of green supply approaches for other than immediate com-
mercial purposes (New et al. 2002). Another difference is that private organizational
green criteria are permitted to be more flexible, where the organizations can define
their own greenness definition. The government, on the other hand, must incorpo-
rate regulation into procurement criteria (New et al. 2002).
To facilitate the GGP process, the central government should be prepared to
develop GGP indicators, criteria, and guidance. Once such tools have been devel-
oped, additional government agencies are likely to seek official certification as a
new promotion method (Geng and Doberstein 2008). The fourth difference is the
process of green procurement: private organizations use what control procedures
they deem appropriate which is open to considerable flexibility. The public sector,
as custodians of public money, must perform traceability and structured procedures
in GGP so that all potential suppliers are treated fairly.
Although differences exist, there are also many similarities. The expressed purpose
is to green products and materials in the operations and practices of organizations,
public or private, as the most critical aspects of GGP and green procurement in general.
Also, in most GGP contexts the most obvious similarity with corporate green supplier
selection is the reliance on multiple decision criteria, formal procedures, and mecha-
nisms such as bid tendering, competitive negotiation, and group decision-making.
The tools available to aid green procurement in both environments may be inter-
changeable. Given the many constraints and considerations, flexible decision sup-
port tools with multiple dimensions for consideration, and ease of use, can prove
valuable. In this chapter, one such group of tools is introduced with a focus on
GGP. The tools will now be introduced, and their illustrative applications and direc-
tions for further research and development are summarized. We begin with a back-
ground the three major tools of rough set theory, TOPSIS, and VIKOR.
ough Set and TOPSIS and VIKOR Background

R
and Notation
A variety of tools and techniques have been developed for green supplier and product
selection and purchase (see Govindan et al. 2015; Brandenburg et al. 2014 for over-
view surveys of green supplier selection and analytical modeling). The number of
tools, and their variety, is relatively sparse for green procurement and supply chain
management, especially when compared to the number of tools and applications for
basic supplier selection and supply chain management decisions (Seuring 2013).
Thus, in this portion of the chapter we introduce tools that have been rarely used
together for any purpose, much less green procurement. The rough set tool helps with
reducing the number of factors for consideration in this relatively complex decision
environment, while the other two tools are aggregation and decision support tools to
help rank and evaluate performance of suppliers and products. We begin with intro-
ducing Rough Set methodology (theory) and then introduce the TOPSIS and VIKOR
multiple criteria decision-making (MCDM) tools, respectively.
Rough Set Theory
Rough set theory (Pawlak 1982) has been applied as a data-mining technique to
help evaluate large sets of data. It is a valuable tool for policy informatics, especially
in the domain of sustainability. It is a nonparametric method that can classify objects
into similarity classes containing objects that are indiscernible with respect to previ-
ous occurrences and knowledge. It has been utilized for such diverse applications as
investigating marketing data (Shyng et al. 2007), justification for green information
technology (Bai and Sarkis 2013b), reverse logistics (Bai and Sarkis 2013a), and
more recently for sustainable supply chain and operations management concerns
(Bai and Sarkis 2010a, b, 2014).
Rough set can integrate both tangible and intangible information and can select use-
ful factors from a given information system. In the methodology presented here, we
utilize rough set to reduce factors to be integrated into a multi-attribute decision-making
(MADM) or MCDM set of models, with specific emphasis on GPP. Attribute reduction
through rough set techniques attempts to retain discernibility of original object factors
from a larger universe of factors (Liang et al. 2012). Heuristic attribute reduction algo-
rithms have been developed in rough set theory to overcome the difficulty of being
computationally very expensive as with other available methods (such as entropy and
regression), especially in cases with large-scale data sets of high dimensions (Liang
et al. 2006). Thus, an advantage is its capability to more efficiently utilize data for
decision-making. In practical research, the empirical survey usually is difficult to col-
lect all the date, so that it has to use some of the incomplete date to estimate the totality.
Rough set approaches can effectively evaluate incomplete and intangible information
(Bai and Sarkis 2011). Not only can it be used on its own as a tool, but can be integrated
with other tools to arrive at solutions in an efficient manner. Unlike tools such as regres-
sion, its nonparametric characteristics allow for greater flexibility.
Some definitions that help to explain rough set are now introduced.
Definition 1 Let U be the universe and let R be an equivalence relation on U. For
any subset X Î U , the pair T = (U ,R ) is called an approximation space. The two
subsets, regions of a set are:

{
RX = x Î U | [ x ]R Í X } (1)

{
RX = x Î U | [ x ]R ∩ X ¹ f } (2)
R-lower (1) and R-upper (2) are approximations of X, respectively. Lower

approximations describe the domain objects which definitely belong to the subset of
interest. Upper approximations describe objects which may possibly belong to the
subset of interest. Approximation vagueness is usually defined by precise values of
lower and upper approximations.
The difference between the upper and the lower approximations constitutes a bound-
ary region for the vague set. Hence, rough set theory expresses vagueness by employ-
ing a boundary region of a set. The R-boundary region of X is represented by (3).
BN R ( X ) = RX - RX (3)

If the boundary region of a set is empty ( BN R ( X ) = 0 ) , it means that the set is
crisp, otherwise the set is rough (inexact). In many real world applications, the
boundary regions are not always so crisp. BN R ( X ) > 0 provides a rough set for
evaluation.
POSR ( X ) = RX is used to denote the R-positive region of X (represented by the
blackened cells in Fig. 1). NEG R ( X ) = U - RX is used to denote the R-negative
region of X (which is represented by the white cells in Fig. 1). The cells in Fig. 1
represent objects to be evaluated, white cells are considered to be outside the rough
set, black cells are definitely within the rough set. Grey cells in Fig. 1 may or may not
Fig. 1 A graphical The rough set

RX
representation of a rough
set environment
BN R (X )
RX
fit within our set. The process for the rough set approach to identify various sets is
defined within the detailed steps of the illustrative example. The MCDA techniques,
TOPSIS and VIKOR, are now initially introduced.
The TOPSIS Method
The TOPSIS method, a ranking technique for order preference by similarity to an

ideal solution, takes into consideration how an object performs on the basis of mul-
tiple criteria. TOPSIS seeks to rank units based on a shorter distance from the ideal
solution and a larger distance from the negative-ideal solution, the nadir point
(Hwang and Yoon 1981; Chen et al. 1992). This method has been widely applied in
the literature (Chen and Tzeng 2004; Opricovic and Tzeng 2004; Krohling and
Campanharo 2011; Bai et al. 2014).
The ideal solution is a solution that maximizes beneficial criteria, criteria which
improve as they increase in value, and minimizes unfavorable criteria, criteria which
improve as they decrease in value. The negative ideal solution maximizes the unfa-
vorable criteria and minimizes the beneficial criteria. Additional definitions for this
methodology are now present to further set the foundation.
Definition 2 Let S = (U, C, V, f ) be an “information system” where U is the uni-
verse, and C is decision factor sets for U; V = ∪ Va indicates the factor range of
aÎ A
factor a; f : U ´ C ® V is an information function, that is for " x Î U if a Î A

then f ( x,a ) Î Va .
The TOPSIS method can be expressed using the following steps:
1. Normalize the decision matrix U = ( xij ) using (4):
n´ m
xij
vij = , i = 1,¼, n; j = 1,¼, m (4)
n
åx 2
kj
k =1

( )
2. Determine the ideal S + and nadir (negative-ideal) S - solutions. ( )
{
S + = v1+ ,¼,vm+ }
{(max v )( )}
(5)
= ij j Î I , min vij j Î J ,
i i

{
S - = v1- ,¼,vm- }
{(min v )( )}
(6)
= ij j Î I , max vij j Î J ,
i i

where I is associated with benefit criteria, and J is associated with negative criteria.
3. Calculate the separation measures using the n-dimensional Euclidian space dis-
tance. The separation of each alternative from the ideal solution is defined by:
å(v )
2
mi+ = ij - v +j , i = 1,¼, n. (7)
j =1

Similarly, the separation from the nadir solution is defined by:
å(v )
2
mi- = ij - v -j , i = 1,¼, n. (8)
j =1

4. Calculate the relative closeness to the ideal solution Ti. The relative closeness of
the alternative Si with respect to S + is defined as
mi-
Ti = (9)

mi+ + mi-
5. Rank the preference order. The larger the value of Ti, the better the alternative Si.
The best alternative is the one with the greatest relative closeness to the ideal
solution. Alternatives can be ranked in decreasing order using this index
(Opricovic and Tzeng 2004).
The VIKOR Method
The VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) method was

developed for multi-criteria optimization and compromise solutions of complex sys-
tems (Opricovic and Tzeng 2002, 2004). It is a discrete alternative multiple criteria
ranking and selection approach, and determines compromise solutions for a problem
with conflicting criteria. Compromise solutions can aid decision makers reach improved
final decisions. Here, the compromise ranking is a feasible solution which is the “clos-
est” to the ideal alternative, and a compromise means an agreement established by
mutual concessions (Opricovic and Tzeng 2007).
The multi-criteria measure for compromise ranking is developed from the Lp-metric
used as an aggregating function in a compromise programming method (Yu 1973).
Development of the VIKOR method starts with the following form of the Lp-metric:
1/ p
ìï m püï
( ) ( )
L p,i = íå éë w j f j+ - fij / f j+ - f j- ùû ý , 1 £ p £ ¥; i = 1,¼, n. (10)
îï j =1 ïþ
where i is an alternative, for alternative i, the rating of the jth criteria is denoted by
fij; m is the number of criteria. Within the VIKOR method L1,i [as Si in (11)] and L¥,i
[as Ri in (12)] are used to formulate the ranking measure.
ïì m ïü
( ) ( )
Si = L p =1,i = íå éë w j f j+ - fij / f j+ - f j- ùû ý , i = 1,¼, n (11)
îï j =1 ïþ
( ) ( )
Ri = L p =¥,i = max éë w j f j+ - fij / f j+ - f j- ùû ,
j
j = 1,¼, m, i = 1,¼, n (12)

Both the VIKOR method and the TOPSIS method are based on an aggregating
function representing “closeness to the ideal” which originates in the compromise
programming method. These two methods introduce different forms of aggregating
function for ranking and different kinds of normalization to eliminate the units of
criterion function (Opricovic and Tzeng 2004). The VIKOR method uses linear
normalization and the TOPSIS method uses vector normalization. For aggregating
functions, the VIKOR method introduces an aggregating function representing the
distance from the ideal solution, considering the relative importance of all criteria,
and a balance between total and individual satisfaction. The TOPSIS method intro-
duces an aggregating function including the distances from the ideal point and from
the nadir point without considering their relative importance. However, the refer-
ence point could be a major concern in decision-making, and to be as close as pos-
sible to the ideal is the rationale of human choice (Opricovic and Tzeng 2004).
In the proposed methodology in this chapter, TOPSIS will be used initially for
single decision maker evaluation, and later VIKOR will be used for group decision
maker ranking, which are determined from TOPSIS. TOPSIS provides an intuitive
reaction for each decision maker evaluation for every supplier, and does not consider
conflicting attributes. Later, for all decision makers ranking values by TOPSIS, we use
VIKOR to rank green vendors and consider conflicting decision maker evaluation and
identify compromise solutions.
Triangular Fuzzy Numbers
To capture the real world uncertainties associated with managing green procure-
ment (governmental or otherwise), the use of fuzzy numbers will be introduced. A
fuzzy number is a convex fuzzy set characterized by a given interval of real
mx (x )
0 xl xm xu x
~
Fig. 2 A triangular fuzzy number x
numbers, each with a grade of membership between 0 and 1. The most commonly
used fuzzy numbers are triangular fuzzy numbers. We now briefly introduce some
basic definitions of the triangular fuzzy number function.
~
Definition 3 A triangular fuzzy number x can be defined by a triplet value (xl, xm, xu).
The membership function is defined as depicted in Fig. 2 (Dubois and Prade 1980),
ì( x - xl ) / ( xm - xl ) , xl £ x < x m
ï
ï1, x = xm
m x~ ( x ) = í (13)
ï( xu - x ) / ( xu - xm ) , x m < x £ xu
ïî0,, otherwise

~
where xl £ xm £ xu , and xl and xu are the lower and upper bounds of x , respectively.
~
xm is the mean of x .
~
Obviously, if xl = xm = xu , then the triangular fuzzy number x is reduced to a
real number. Conversely, real numbers may be easily rewritten as triangular fuzzy
numbers. The triangular fuzzy number can be flexible and represent various seman-
tics of uncertainty (Li 2012). The triangular fuzzy number is based on a three-value
judgment: the minimum possible value xl, the most likely value xm, and the maxi-
mum possible value xu.
Definition 4 Let the distance measure of two triangular fuzzy numbers ( x = xl1 ,x1m ,xu1
~1
( )
~2
and x = (x ,x
2
l
2
m
2
,x
u ) ) be the Minkowski space distance represented by (14):
)
1

æ ~1 ~ 2 ö
è ø êë
((
L ç x , x ÷ = é1 / 3 xl1 - xl2 ) +(x
p
1
m -x 2
m ) +(x
p
1
u -x 2
u )
p
ùp
úû

(14)
where p is some exponential power, in our illustrative example p = 2 (quadratic power).
An illustrative example application is now introduced that brings together the
various techniques and characteristics within the GGP decision environment.
An Illustrative Application
The fuzzy decision table for GGP is introduced in this section. Assume that a data-
base of suppliers exists (a fuzzy table) by some government agency. This fuzzy
æ ~ö
decision table is defined by T = ç U, A , V, f ÷ , where U = {GS1 ,...,GSn } is a set of
è ø
n alternative green suppliers called the universe. A = {a1 ,...,am } is a set of m attri-
~
butes for the suppliers. Where the f is a grey description function used to define
the values V.
For this illustrative case, U = {GSi , i = 1,¼, 30} (i.e., 30 government suppliers)
with nine attributes A = {a j , i = 1,¼, 9} each. The attributes represent the three
triple-bottom-line factors for sustainability. An example set of attributes for GGP
are shown in Table 1.
For the case illustration, it assumed that four decision makers D = {dk , k = 1,¼, 4} ,
exist.
The multi-stage and multi-step procedure within the context of a GGP is illustra-
tive application is now introduced. There are three stages and 12 steps in the
methodology to arrive at our final selection and/or ranking of green suppliers.
Further details of the illustrative application are defined below.
Table 1 Listing of potential attributes for green government procurement decisions

Environmental attributes Economic attributes Social attributes
Energy sources save in The price of green products Customer satisfaction
providing products
Waste production in providing Availability of spare parts Working conditions: labor
product and repair services standard, health, and safety
Reuse and recoverable of Durability, adaptability, Operation in a safe manner
products compatibility of products
Toxic-free and low chemical Quality management Comply with labor laws
content of products
Advanced of environmental Delivery time Donates to philanthropic
management system organizations
Comply with various Technical capabilities Volunteers at/for local charities
environmental regulations
Compulsory use of Innovativeness capabilities Organization’s/Council’s/ public
environmental labels image
Bad environmental records or Contribute to the modernization
reports of suppliers and international competitiveness
of local industry
Sources: Bai and Sarkis (2010a), McMurray et al. (2014), Michelsen and de Boer (2009), Nissinen
et al. (2009), Parikka-Alhola (2008), Walker and Brammer (2009), and Zhu et al. (2013)
tage 1: Reducing Attributes and Determining Core Attribute

S
Weights
In this stage, we focus on the use of rough set theory to deduce the reduced set of
attributes and determining core attribute weights. This set reduction will help man-
agers more easily comprehend the most information bearing factors and lessen
effort with the use of the other stages of the process.
tep 1: Determine Performance Levels of Suppliers on Various

S
Sustainability Factors
From the team of decision makers attribute values of suppliers need to be deter-
mined for each of the sustainability attributes. The team members assign textual
perceptual scores ranging from very poor to very good for each supplier and their
attributes. The seven level scale used in this study is shown in Table 2. A fuzzy scale
score ~v that will be assigned to each supplier (i) by each decision maker (k) for each
attribute (j) for each respective scale level is also defined.
The textual assignments for the case example are shown in Table 3. In this step, the
decision makers evaluate the 30 suppliers on each of the nine sustainability attributes.
Step 2: Determine Information Content for Each Attribute
This step determines how the level of information content across each attribute (aj)
using the (15) (Liang et al. 2006):
K n
1
I ( Atr ∪ a j ) = 1 - åå X i
k
(15)
| U |2 k =1 i =1
In (15), I ( Atr ∪ a j ) is the information content1 of conditional attribute a j Ï Atr.
Atr is the previous reduct set and changes in every cycle of the methodology. In the
initialization step, the core conditional attribute set in the reduct set is Atr = Æ . |U| is
the cardinality of the universe (120 in the example: 30 suppliers × 4 decision makers).
Table 2 The scale Scale ~v

of attribute ratings ~v
Very poor (VP) (0, 0, 0.1)
Poor (P) (0, 0.1, 0.3)
Somewhat fair (SF) (0.1, 0.3, 0.5)
Fair (F) (0.3, 0.5, 0.7)
Somewhat good (SG) (0.5, 0.7, 0.9)
Good (G) (0.7, 0.9, 1)
Very good (VG) (0.9, 1, 1)
1
This term has also been defined as information entropy of a system (Liang and Shi 2004).
Table 3 Evaluation of suppliers on sustainability attributes by decision makers
Decision maker 1 Decision maker 2 Decision maker 3 Decision maker 4
Suppliers Ev1 Ev2 Ev3 Ec1 Ec2 Ec3 So1 So2 So3 Ev1 Ev2 Ev3 Ec1 Ec2 Ec3 So1 So2 So3 Ev1 Ev2 Ev3 Ec1 Ec2 Ec3 So1 So2 So3 Ev1 Ev2 Ev3 Ec1 Ec2 Ec3 So1 So2 So3
Supplier 1 VG SG SF SG G SF F F F VG G F SG SG F F F F G SG SF SG SG P SF SF SF VG G SF SG SG SF SF F SF
Supplier 2 SF P G P SF VG SG G P F VP G VP F VG SG G VP SF VP G VP SF VG SG SG VP SF VP G VP F VG SG SG P
Supplier 3 VG VG SF G G G F SG P VG VG F G G G F SG P G VG SF G G G SF SG P G VG SF G G G SF SG P
Supplier 4 VG G P G F G G F VG VG G P G F G G F VG VG G P G SG G G F G VG G P G F G G F VG
Supplier 5 G VG G P G VG SF SF SF G VG G P G G F F F G VG G P G G P P P G VG G P G G SF P P
Supplier 6 P F F F P P P VP VP P F F F P P P VP VP P F SG SG P P P VP VP P F F SG P P P VP VP
Supplier 7 G SG SF SG SG P SF SF SF VG G SF SG G SF SF F F VG SG SF SG SG SF F F SF VG G F SG SG F F F F
Supplier 8 G F SF P G P VG SG SG G SG SF SF G P G F G G F F P G SF G F SG G F F F G F G F SG
Supplier 9 F G SG G SF VP SF VG G F VG F G SF VP P VG G F G F SG P VP P VG G F G F G F VP F VG G
Supplier 10 G SG SF SG SG P SF SF SF VG G SF SG G SF F F F VG SG SF SG SG SF SF F SF VG G F SG SG F F F F
Supplier 11 VP G G SF SG VP VP SF G VP G G F G VP VP F G VP G G SF G VP VP P G VP G G F G VP P SF SG
Supplier 12 G G G G F G G G VG G G G G F G G G VG G G G G SG G G G VG G G SG G F G G G VG
Supplier 13 P G VP P G SF P F VP P G VP VP G F P F VP P G VP P G P VP SG VP P G P P G P P F VP
Supplier 14 P P G G SF G F P P P P G SG F G F VP F P P G SG SF G F VP P P P G SG SF VG SG VP SF
Supplier 15 F SG P P P VG P G G F SG P P VP VG F G G F SG P P P VG P G G F SG P P P VG SF G SG
Supplier 16 VG SG VP G SG SG VG G G VG SG VP G SG SG VG G G VG SG VP G SG SG VG G G VG SG P G G SG VG G G
Supplier 17 SG F F VP P G G P VP SG F F VP VP G G VP VP SG F F VP VP G G P VP F F SF VP VP G G P P
Supplier 18 SF G SG P P G VG G VP F G SG F P G VG G VP P G F P P G VG G VP P SG SG SF VP G VG G VP
Supplier 19 P VP VP P P G P G G P VP VP VP P G F G G P VP VP VP P G P G G P VP P P P G SF G G
Supplier 20 G P VP VP G G P SF G G P VP VP G G P F G G P VP VP G G P P G G P VP VP G G P SF G
Supplier 21 VP G VG VP G SF P G F VP G VG P G F VP SG F VP G G P G P VP SG F VP G G P G SF VP F F
Supplier 22 F VP P VP G G SG VG SF SG VP P VP G G SG VG F SG VP P VP G G SG VG SF G VP SF P G G SG VG P
Supplier 23 G SG F P G SF G F SG G F SF F G P VG SG SG G F SF P G P G F SG G F F SF G F G F G
Supplier 24 VP F P VG F VP G SG SF VP F P VG F VP G SG F VP F P VG SG P G SG F P F P VG F VP G SG SF
Supplier 25 G SF SG F G G P F F VG F F SG G SG P F F VG SF F SG SG SG P F F VG SF F SG SG G P F SF
Supplier 26 P P SF VG F SG G SF G P P F VG F G G F G P P P G F G G SF G P P SF VG SG G G SF G
Supplier 27 VP P G P G G VG G G P P G F G G G G G P P G P G G G G G P P G SF G G G G G
Supplier 28 G SG F P G SF G F SG G F F F G F G F SG G F SF P G P G SG SG G F SF SF G P VG F G
Supplier 29 G P VG P SG SF VP SF SF SG P G P F F VP F F SG P G P F P VP P SF SG P G VP F P VP P SF
Supplier 30 P G P SF P F SG P F F G P F P SG SG F F SF G P SF P SG SG P F F G P SF SF SG SG SF F
| Xik | is the number of suppliers for any decision maker evaluation with the same attri-
bute levels across conditional attribute(s) Atr ∪ a j for a supplier i and decision maker k.
As an example, supplier 01 is “VG” for decision maker 1. There are 21 suppliers
with the same value for different decision makers, thus | X 01 1
|= 22 . Thus, using (15)
for the original set of conditional attribute a1 is
2666
I ( a1 ) = 1 - = 0.815
| 120 |2
Step 3: Determine the Information Significance of a Conditional Attribute
For this step, the information content on the null core conditional attribute set (ini-
tially for the core conditional attribute set Atr has no attributes assigned to it) is
defined. That is: I ( Æ ) = 0 .
To calculate the information significance of a conditional attribute j(aj) (16) is used.
Sig ( a j ) = I ( Atr ∪ a j ) - I ( Atr ) (16)

For example, the significance for the Ev1 conditional attribute can be calculated as:
Sig ( Ev1) = I ( Æ ∪ Ev1) - I ( Æ )

= 0.815 - 0
= 0.815

Step 4: Select and Update Core Conditional Attribute Set and Reduct
This step requires selecting the conditional attribute aj that satisfies (17):
j
(
max Sig ( a j ) ) (17)
To update the core conditional attribute set Atr, the following rule is applied:
( )
If max Sig ( a j ) > e , where ε is a positive infinitesimal real number used to
j
control the convergence, then Atr ∪ a j Þ Atr . We then return to step 2 with a new
core conditional attribute set Atr. Otherwise if max j
( )
Sig ( a j ) £ e , we stop and the
final reduct set and core conditional attribute set is Atr.
For the illustrative example e = 0.001 . For So1, Sig ( So1) = 0.831 > 0.001 , so
Atr = Atr ∪ So1 = {So1} . We then return to Step 2.
After a number of iterations the final set Atr is {So1, Ec1, So2, Ev3, Ec2}. The
reduced decision table is shown in Table 4.
Table 4 Evaluation of suppliers on reduced attributes by decision makers
Suppliers Ev3 Ec1 Ec2 So1 So2 Ev3 Ec1 Ec2 So1 So2 Ev3 Ec1 Ec2 So1 So2 Ev3 Ec1 Ec2 So1 So2
Supplier 1 SF SG G F F F SG SG F F SF SG SG SF SF SF SG SG SF F
Supplier 2 G P SF SG G G VP F SG G G VP SF SG SG G VP F SG SG
Supplier 3 SF G G F SG F G G F SG SF G G SF SG SF G G SF SG
Supplier 4 P G F G F P G F G F P G SG G F P G F G F
Supplier 5 G P G SF SF G P G F F G P G P P G P G SF P
Supplier 6 F F P P VP F F P P VP SG SG P P VP F SG P P VP
Supplier 7 SF SG SG SF SF SF SG G SF F SF SG SG F F F SG SG F F
Green Government Procurement…
Supplier 8 SF P G VG SG SF SF G G F F P G G F F F G G F
Supplier 9 SG G SF SF VG F G SF P VG F SG P P VG F G F F VG
Supplier 10 SF SG SG SF SF SF SG G F F SF SG SG SF F F SG SG F F
Supplier 11 G SF SG VP SF G F G VP F G SF G VP P G F G P SF
Supplier 12 G G F G G G G F G G G G SG G G SG G F G G
Supplier 13 VP P G P F VP VP G P F VP P G VP SG P P G P F
Supplier 14 G G SF F P G SG F F VP G SG SF F VP G SG SF SG VP
Supplier 15 P P P P G P P VP F G P P P P G P P P SF G
Supplier 16 VP G SG VG G VP G SG VG G VP G SG VG G P G G VG G
Supplier 17 F VP P G P F VP VP G VP F VP VP G P SF VP VP G P
Supplier 18 SG P P VG G SG F P VG G F P P VG G SG SF VP VG G
Supplier 19 VP P P P G VP VP P F G VP VP P P G P P P SF G
Supplier 20 VP VP G P SF VP VP G P F VP VP G P P VP VP G P SF
Supplier 21 VG VP G P G VG P G VP SG G P G VP SG G P G VP F
Supplier 22 P VP G SG VG P VP G SG VG P VP G SG VG SF P G SG VG
Supplier 23 F P G G F SF F G VG SG SF P G G F F SF G G F
(continued)
107
Table 4 (continued)
108

Suppliers Ev3 Ec1 Ec2 So1 So2 Ev3 Ec1 Ec2 So1 So2 Ev3 Ec1 Ec2 So1 So2 Ev3 Ec1 Ec2 So1 So2
Supplier 24 P VG F G SG P VG F G SG P VG SG G SG P VG F G SG
Supplier 25 SG F G P F F SG G P F F SG SG P F F SG SG P F
Supplier 26 SF VG F G SF F VG F G F P G F G SF SF VG SG G SF
Supplier 27 G P G VG G G F G G G G P G G G G SF G G G
Supplier 28 F P G G F F F G G F SF P G G SG SF SF G VG F
Supplier 29 VG P SG VP SF G P F VP F G P F VP P G VP F VP P
Supplier 30 P SF P SG P P F P SG F P SF P SG P P SF SF SG SF
C. Bai and J. Sarkis
Table 5 The weight of core Core Information

attributes attributes content Weight
Ev3 0.819 0.203
Ec1 0.828 0.205
Ec2 0.749 0.185
So1 0.831 0.206
So2 0.811 0.201
Step 5: Determine the Core Attribute Importance Weight wj
The importance weight for each core attribute j (wj) is now determined using (18).
I (aj )
wj = (18)
å I (a )
jÎAtr
j

The aggregated weight value meets the condition:
åw
jÎAtr
j =1 (19)

The final adjusted attribute importance weight values are shown in Table 5.
tage 2: Evaluating Suppliers Utilizing TOPSIS for Each

S
Decision Maker
tep 6: Determine the Core Final Attribute Value by Adjusting
S
with the Importance Weight
Considering the weights of each attribute, the weighted normalized decision matrix can
be computed by multiplying the importance weights of the evaluation attribute and the
fuzzy values in the normalized decision matrix. This step is completed with (20):
k æ kij kij kij ö

wvijk = w j ´ v˜ ij = ç w j ´ v˜ l , w j ´ v˜m , w j ´ v˜ u ÷ " k Î K , j Î Attr, i Î n (20)
è ø
For the green supplier 01, attribute 3 (Ev3) for the decision maker 1 the
∼1 ∼1
adjusted fuzzy value is: wv13 = w3 ×v 13 = (0.1 × 0.203, 0.3 × 0.203, 0.5 × 0.203)
=(0.0203, 0.0609, 0.1015).
The overall adjusted aggregate attribute scores results with the decision maker 01
for each supplier is presented in Table 6.
Table 6 Combined weight scores of green suppliers for decision maker 01

Decision maker 1
Suppliers Ev3 Ec1 Ec2 So1 So2
Supplier 1 (0.0203, (0.1025, (0.1295, (0.0618, 0.103, (0.0603,
0.0609, 0.1015) 0.1435, 0.1845) 0.1665, 0.185) 0.1442) 0.1005, 0.1407)
Supplier 2 (0.1421, (0, 0.0205, (0.0185, (0.103, 0.1442, (0.1407,
0.1827, 0.203) 0.0615) 0.0555, 0.0925) 0.1854) 0.1809, 0.201)
Supplier 3 (0.0203, (0.1435, (0.1295, (0.0618, 0.103, (0.1005,
0.0609, 0.1015) 0.1845, 0.205) 0.1665, 0.185) 0.1442) 0.1407, 0.1809)
Supplier 4 (0, 0.0203, (0.1435, (0.0555, (0.1442, (0.0603,
0.0609) 0.1845, 0.205) 0.0925, 0.1295) 0.1854, 0.206) 0.1005, 0.1407)
Supplier 5 (0.1421, (0, 0.0205, (0.1295, (0.0206, (0.0201,
0.1827, 0.203) 0.0615) 0.1665, 0.185) 0.0618, 0.103) 0.0603, 0.1005)
Supplier 6 (0.0609, (0.0615, (0, 0.0185, (0, 0.0206, (0, 0, 0.0201)
0.1015, 0.1421) 0.1025, 0.1435) 0.0555) 0.0618)
Supplier 7 (0.0203, (0.1025, (0.0925, (0.0206, (0.0201,
0.0609, 0.1015) 0.1435, 0.1845) 0.1295, 0.1665) 0.0618, 0.103) 0.0603, 0.1005)
Supplier 8 (0.0203, (0, 0.0205, (0.1295, (0.1854, 0.206, (0.1005,
0.0609, 0.1015) 0.0615) 0.1665, 0.185) 0.206) 0.1407, 0.1809)
Supplier 9 (0.1015, (0.1435, (0.0185, (0.0206, (0.1809, 0.201,
0.1421, 0.1827) 0.1845, 0.205) 0.0555, 0.0925) 0.0618, 0.103) 0.201)
Supplier 10 (0.0203, (0.1025, (0.0925, (0.0206, (0.0201,
0.0609, 0.1015) 0.1435, 0.1845) 0.1295, 0.1665) 0.0618, 0.103) 0.0603, 0.1005)
Supplier 11 (0.1421, (0.0205, (0.0925, (0, 0, 0.0206) (0.0201,
0.1827, 0.203) 0.0615, 0.1025) 0.1295, 0.1665) 0.0603, 0.1005)
Supplier 12 (0.1421, (0.1435, (0.0555, (0.1442, (0.1407,
0.1827, 0.203) 0.1845, 0.205) 0.0925, 0.1295) 0.1854, 0.206) 0.1809, 0.201)
Supplier 13 (0, 0, 0.0203) (0, 0.0205, (0.1295, (0, 0.0206, (0.0603,
0.0615) 0.1665, 0.185) 0.0618) 0.1005, 0.1407)
Supplier 14 (0.1421, (0.1435, (0.0185, (0.0618, 0.103, (0, 0.0201,
0.1827, 0.203) 0.1845, 0.205) 0.0555, 0.0925) 0.1442) 0.0603)
Supplier 15 (0, 0.0203, (0, 0.0205, (0, 0.0185, (0, 0.0206, (0.1407,
0.0609) 0.0615) 0.0555) 0.0618) 0.1809, 0.201)
Supplier 16 (0, 0, 0.0203) (0.1435, (0.0925, (0.1854, 0.206, (0.1407,
0.1845, 0.205) 0.1295, 0.1665) 0.206) 0.1809, 0.201)
Supplier 17 (0.0609, (0, 0, 0.0205) (0, 0.0185, (0.1442, (0, 0.0201,
0.1015, 0.1421) 0.0555) 0.1854, 0.206) 0.0603)
Supplier 18 (0.1015, (0, 0.0205, (0, 0.0185, (0.1854, 0.206, (0.1407,
0.1421, 0.1827) 0.0615) 0.0555) 0.206) 0.1809, 0.201)
Supplier 19 (0, 0, 0.0203) (0, 0.0205, (0, 0.0185, (0, 0.0206, (0.1407,
0.0615) 0.0555) 0.0618) 0.1809, 0.201)
Supplier 20 (0, 0, 0.0203) (0, 0, 0.0205) (0.1295, (0, 0.0206, (0.0201,
0.1665, 0.185) 0.0618) 0.0603, 0.1005)
Supplier 21 (0.1827, 0.203, (0, 0, 0.0205) (0.1295, (0, 0.0206, (0.1407,
0.203) 0.1665, 0.185) 0.0618) 0.1809, 0.201)
Supplier 22 (0, 0.0203, (0, 0, 0.0205) (0.1295, (0.103, 0.1442, (0.1809, 0.201,
0.0609) 0.1665, 0.185) 0.1854) 0.201)
(continued)
Table 6 (continued)
Decision maker 1
Suppliers Ev3 Ec1 Ec2 So1 So2
Supplier 23 (0.0609, (0, 0.0205, (0.1295, (0.1442, (0.0603,
0.1015, 0.1421) 0.0615) 0.1665, 0.185) 0.1854, 0.206) 0.1005, 0.1407)
Supplier 24 (0, 0.0203, (0.1845, 0.205, (0.0555, (0.1442, (0.1005,
0.0609) 0.205) 0.0925, 0.1295) 0.1854, 0.206) 0.1407, 0.1809)
Supplier 25 (0.1015, (0.0615, (0.1295, (0, 0.0206, (0.0603,
0.1421, 0.1827) 0.1025, 0.1435) 0.1665, 0.185) 0.0618) 0.1005, 0.1407)
Supplier 26 (0.0203, (0.1845, 0.205, (0.0555, (0.1442, (0.0201,
0.0609, 0.1015) 0.205) 0.0925, 0.1295) 0.1854, 0.206) 0.0603, 0.1005)
Supplier 27 (0.1421, (0, 0.0205, (0.1295, (0.1854, 0.206, (0.1407,
0.1827, 0.203) 0.0615) 0.1665, 0.185) 0.206) 0.1809, 0.201)
Supplier 28 (0.0609, (0, 0.0205, (0.1295, (0.1442, (0.0603,
0.1015, 0.1421) 0.0615) 0.1665, 0.185) 0.1854, 0.206) 0.1005, 0.1407)
Supplier 29 (0.1827, 0.203, (0, 0.0205, (0.0925, (0, 0, 0.0206) (0.0201,
0.203) 0.0615) 0.1295, 0.1665) 0.0603, 0.1005)
Supplier 30 (0, 0.0203, (0.0205, (0, 0.0185, (0.103, 0.1442, (0, 0.0201,
0.0609) 0.0615, 0.1025) 0.0555) 0.1854) 0.0603)
Step 7: Determine the Ideal and Nadir Solution
First, the most “ideal” reference solution Sk+ ( wv ) for decision maker k is determined
by selecting the maximum value from amongst each of the attributes using (21).

{ ( ) ( )
Sk+ ( wv ) = max wvik1 ,max wvik2 ,¼,max wvim
k
( )} (21)
Second, the most “nadir” reference solution Sk- ( wv ) for decision maker k is
determined by selecting the minimum value from amongst each of the attributes
using (22).

{ ( ) ( )
Sk- ( wv ) = min wvik1 ,min wvik2 ,¼,min wvim
k
( )} (22)
Using (21)–(22) for this illustrative problem, two sub-steps will be completed.
First, the most “ideal” reference green suppliers S1+ for the decision maker 1 is
determined to be:
ïì( 0.1827,0.203,0.203 ) , ( 0.1845,0.205,0.205 ) , ( 0.1295,0.1665,0.185 ) , üï

S1+ = í ý
ï
î ( 0.1854,0.206,0.206 ) , ( 0.1809,0.201,0.201) ïþ
Second, the most “nadir” reference green supplier alternative S1- for the decision
maker 1 is determined as:
S1- = {( 0,0,0.0203 ) , ( 0,0,0.0205 ) , ( 0,0.0185,0.0555 ) ,

( 0,0,0.0206 ) , ( 0,0,0.0201)}
Step 8: Calculate the n-Dimensional Distance for Separation Distance
Based on the fuzzy numbers distance (14) and the TOPSIS separation measure (7)–
(8), new separation measures are defined for an alternative object and “ideal” (23)
and nadir (24) alternative.
mik + = å L ( S ( j ) , S ( j ))
jÎAtr
+
k i
k
(23)

mik - = å L ( S ( j ) , S ( j ))
jÎAtr
-
k i
k
(24)

For the illustrative example, an example calculation for m01
1+
is shown using (23).
1+
m01 = å L ( S ( j ) , S ( j ) ) = 1.140
jÎ Atr
+
1j
+
1
1
01

The solutions for the alternatives’ separation distances from the ideal point are
presented in Table 7.
Step 9: Calculate the Relative Closeness to the Ideal Solution

+
The relative closeness of the alternative Sik with respect to Sk is calculated using
(9). The relative closeness coefficient helps for rank ordering of all alternatives,
allowing the decision makers to select the most feasible alternative. A larger for Ti
value represents a more superior alternative.
Using (9), the final comparative distances Tik are shown in Table 7. An example
calculation for the first supplier and the decision maker 1 is presented here:
1-
m01 0.7186
T011 = 1+ 1-
= = 0.538
m01 + m01 0.6164 + 0.7186
After calculating the Tik for each decision maker k, we can form the relative-
closeness matrix and the results are shown in Table 8.
tage 3: Ranking Suppliers Utilizing VIKOR for all Decision

S
Makers
Step 10: Evaluate and Assign the Importance Level for Each
Decision Maker
The importance of each decision maker and their input into the decision is defined
by dk. For the four decision makers with ( k Î K and K = 4), we have the following
importance levels, respectively: d 1 = 0.4, d 2 = 0.3, d 3 = 0.2, d 4 = 0.1 .
Table 7 The relative closeness of green suppliers for decision maker 01
Suppliers mi1+ mi1- Ti1

Supplier 1 1.140 1.549 0.576
Supplier 2 1.113 1.576 0.586
Supplier 3 0.917 1.772 0.659
Supplier 4 1.116 1.573 0.585
Supplier 5 1.387 1.302 0.484
Supplier 6 2.056 0.633 0.235
Supplier 7 1.477 1.213 0.451
Supplier 8 1.079 1.610 0.599
Supplier 9 0.950 1.739 0.647
Supplier 10 1.477 1.213 0.451
Supplier 11 1.542 1.147 0.426
Supplier 12 0.448 2.241 0.833
Supplier 13 1.877 0.812 0.302
Supplier 14 1.228 1.461 0.543
Supplier 15 2.002 0.687 0.255
Supplier 16 0.783 1.906 0.709
Supplier 17 1.830 0.860 0.320
Supplier 18 1.142 1.547 0.575
Supplier 19 2.063 0.626 0.233
Supplier 20 2.059 0.630 0.234
Supplier 21 1.149 1.540 0.573
Supplier 22 1.246 1.443 0.537
Supplier 23 1.140 1.549 0.576
Supplier 24 0.934 1.755 0.653
Supplier 25 1.246 1.443 0.537
Supplier 26 1.073 1.616 0.601
Supplier 27 0.634 2.055 0.764
Supplier 28 1.140 1.549 0.576
Supplier 29 1.584 1.105 0.411
Supplier 30 1.992 0.697 0.259
Step 11: Identify Group Positive Ideal and Group Nadir Solutions
First, the group positive ideal solution f + ( T ) is determined by selecting the maxi-
mum value from amongst each of the decision makers using (25):

{ ( )
f + ( T ) = max Ti1 ,¼,max Ti k ( )} (25)
Second, the nadir reference solution f - ( T ) is determined by selecting the mini-

mum value from amongst each of the decision makers using (26):
Table 8 The relative closeness to the ideal solution for each decision maker k
Suppliers Decision maker 1 Decision maker 2 Decision maker 3 Decision maker 4
Supplier 1 0.538 0.548 0.486 0.509
Supplier 2 0.543 0.557 0.524 0.545
Supplier 3 0.582 0.61 0.569 0.569
Supplier 4 0.543 0.547 0.576 0.554
Supplier 5 0.492 0.542 0.465 0.484
Supplier 6 0.357 0.367 0.422 0.397
Supplier 7 0.475 0.52 0.532 0.555
Supplier 8 0.55 0.539 0.549 0.594
Supplier 9 0.575 0.536 0.506 0.61
Supplier 10 0.475 0.543 0.509 0.555
Supplier 11 0.463 0.53 0.472 0.526
Supplier 12 0.691 0.694 0.747 0.679
Supplier 13 0.397 0.392 0.421 0.421
Supplier 14 0.522 0.517 0.502 0.525
Supplier 15 0.369 0.414 0.382 0.404
Supplier 16 0.609 0.613 0.622 0.659
Supplier 17 0.407 0.391 0.407 0.381
Supplier 18 0.538 0.585 0.526 0.558
Supplier 19 0.355 0.4 0.354 0.404
Supplier 20 0.356 0.392 0.345 0.369
Supplier 21 0.536 0.522 0.517 0.495
Supplier 22 0.518 0.523 0.529 0.56
Supplier 23 0.538 0.598 0.526 0.569
Supplier 24 0.578 0.582 0.614 0.59
Supplier 25 0.518 0.523 0.512 0.512
Supplier 26 0.551 0.603 0.531 0.584
Supplier 27 0.643 0.687 0.642 0.668
Supplier 28 0.538 0.586 0.549 0.558
Supplier 29 0.455 0.452 0.413 0.4
Supplier 30 0.372 0.449 0.384 0.425

{ ( )
f - ( T ) = min Ti1 ,¼,min Ti k ( )} (26)
Using (25)–(26) for this illustrative problem, the group positive ideal solution
and nadir reference solutions are:
f + ( T ) = {0.691, 0.694, 0.747, 0.679}

f - ( T ) = {0.355, 0.367, 0.345, 0.369}

Step 12: Compute the Group Utility Si and the Maximal Regret Ri Using

(27)–(28)
( ) (
Si = ådk fk+ - fik / fk+ - fk- ) (27)
k =1

( ( ) (
Ri = max dk fk+ - fij / fk+ - fk- )) (28)
where Si and Ri show the mean of group utility and maximal regret, respectively.
The group utility is emphasized in the case of p = 1. The importance of maximal
regret rises as the value of parameter p increases when p = ∞.
Step 13: Compute the Index Values Qi Using (29)
( ) ( ) (
Qi = v Si - S + / S - - S + + (1 - v ) Ri - R + / R - - R + ) ( ) (29)
where S + = min i Si , S - = max i Si , R + = min i Ri , R - = max i Ri , and v is intro-

duced as a weight for maximum group utility, whereas 1 − v is the weight of the
individual regret.
The values of Si, Ri, and Qi are calculated for all suppliers, are shown in
Table 9. Supplier ranks, sorting by the values Si, Ri, and Qi, are also shown in
Table 9.
Step 14: Propose a Compromise Solution
We propose as a compromise solution the supplier (A(1)), which is the best ranked
by the measure Q (minimum) when the following two conditions are satisfied:
C1. Acceptable advantage:
Q ( A ( 2 ) ) - Q ( A (1) ) ³ DQ, (30)

where A(2) is the alternative positioned second in the ranking list by Q; DQ = 1/(U − 1).
C2. Acceptable stability in decision making:
The alternative A(1) must also be the best ranked by S and/or R.
This compromise solution is stable within a decision-making process, which
could be the strategy of maximum group utility (when v > 0.5 is needed), or “by
consensus” v ≈ 0.5, or “with veto” (v < 0.5). Here, v is the weight of the decision-
making strategy that provides maximum group utility for the majority of criteria.
Table 9 The supplier ranks by the values Si, Ri, and Qi

Suppliers Si Ranking Ri Ranking Qi Ranking
Supplier 01 0.429 14 0.455 11 0.442 14
Supplier 02 0.391 13 0.44 9 0.415 13
Supplier 03 0.284 4 0.326 4 0.305 4
Supplier 04 0.374 12 0.441 10 0.408 11
Supplier 05 0.497 21 0.592 19 0.545 19
Supplier 06 0.815 28 0.995 28 0.905 28
Supplier 07 0.483 20 0.642 21 0.562 21
Supplier 08 0.374 11 0.42 8 0.397 8
Supplier 09 0.364 9 0.361 6 0.363 6
Supplier 10 0.475 19 0.642 20 0.558 20
Supplier 11 0.521 22 0.679 22 0.6 22
Supplier 12 0 1 0 1 0 1
Supplier 13 0.748 25 0.876 25 0.812 25
Supplier 14 0.459 18 0.504 16 0.481 17
Supplier 15 0.78 27 0.958 27 0.869 27
Supplier 16 0.206 3 0.243 3 0.224 3
Supplier 17 0.756 26 0.847 24 0.801 24
Supplier 18 0.369 10 0.456 14 0.413 12
Supplier 19 0.817 29 1 30 0.909 29
Supplier 20 0.836 30 0.997 29 0.917 30
Supplier 21 0.442 16 0.46 15 0.451 15
Supplier 22 0.436 15 0.514 18 0.475 16
Supplier 23 0.356 7 0.455 12 0.405 9
Supplier 24 0.284 5 0.336 5 0.31 5
Supplier 25 0.457 17 0.514 17 0.485 18
Supplier 26 0.333 6 0.417 7 0.375 7
Supplier 27 0.102 2 0.143 2 0.122 2
Supplier 28 0.359 8 0.455 13 0.407 10
Supplier 29 0.651 23 0.702 23 0.676 23
Supplier 30 0.743 24 0.951 26 0.847 26
If one of the conditions is not satisfied, then a set of compromise solutions is

proposed, which consists of:
• Alternatives A(1) and A(2) if only the condition C2 is not satisfied, or
• Alternatives A(1), A(2), …, A(M) if the condition C1 is not satisfied; A(M) is
determined by the relation Q ( A ( M ) ) - Q ( A (1) ) < DQ for maximum M (the
positions of these alternatives are “in closeness”).
Ranking the suppliers by the VIKOR method gives, as a compromise solution for
the value v = 0.5, supplier 12. In addition, conditions C1 and C2 are satisfied as this
alternative is also the best ranked by S and R, and Q(A(27)) − Q(A(12)) ≥ DQ.
Discussion and Conclusion
The concern of green vendor selection is an important aspect to green procurement.

Although the methodology presented here can have broader application, the use of
it for GPP is clearly evident.
Tools such as these can be used beyond just the selection of specific suppliers,
but also products and product families. For example in California, their green pro-
curement system focused on developing and ranking product characteristics and
their green characteristics when seeking to develop contracts for procurement
(Swanson et al. 2005). For example, previous government purchasing data can be
used to produce a ranked list of product classes which then can be used for prioriti-
zation and selection purposes. Part of this ranking approach may utilize measures
such as likelihood or probability of making an environmental improvement impact.
More recent work on GPP has defined various processes that agencies will go
through to meet specific regulations and policies (Amann et al. 2014). These pro-
cesses include investigating whether certain GPP performance metrics, that are
typically regulated, are to be met. For example, whether policy goals are met at the
tendering phase, inclusion in offers, and general goals after delivery are met. There
are various international, national, and local regulatory policies and goals that would
need to be met. Incorporating some of these metrics into tools presented here can
prove beneficial for decision makers.
Many rules and regulations also exist, where aggregation from multiple govern-
mental agencies would need to be involved in GGP to help manage these dispersed
rules and regulations. The use of the multiple decision maker format allows for dif-
ferent agencies to effectively evaluate and aggregate the best green suppliers. Thus,
application across agencies can be practical in this situation, where differing agency
importance valuations can be integrated to find the best overall suppliers.
The tools, as presented in this chapter, need to be further evaluated in a practical
setting. The acquisition of data for particular attributes is not a trivial matter and
may require the development of systems to acquire such data. Many times, manag-
ers and decision makers would like to know how the approaches work. Alternatively,
very large amounts of data may exist with government agencies. The technique is
valuable in that the rough set technique is valuable for data mining and identifying
the most important and information attributes before application of the multiple
criteria decision approaches.
The decision methodology in this chapter uses TOPSIS in evaluating suppliers for
each decision maker and VIKOR in ranking suppliers utilizing all decision makers. In
the second stage, each decision maker evaluates suppliers based on their own unified
understanding of GGP strategy and objectives. The main goal at this stage is to rank the
supplier base on the principle that the optimal point should have the shortest distance
from the positive ideal solution (more profit) and the farthest from the negative ideal
solution (avoids the most risk). In the third stage, all decision makers evaluate suppliers
based on their different understanding of GGP strategy and objectives. The main goal
in the third stage is to consider and integrate different understanding of decision makers
or managers and provide a balance between total and individual satisfaction. VIKOR is
good at determining a compromise solution, which could be accepted by the decision
makers because it provides a maximum “group utility” of the “majority” and a mini-
mum of the individual regret of the “opponent” (Tzeng et al. 2005).
Both the TOPSIS and VIKOR methods are suitable for evaluating similar prob-
lems, provide excellent results close to reality, and support superior analysis (Chu
et al. 2007). Inverting the use of these two methods or just using VIKOR or TOPSIS
in both stages could be another viable decision-making method. Our selection of
TOPSIS first and VIKOR second is a preference based on our initial thoughts on
individual and group relationships, but joint and mixed order may also be accept-
able and left for future investigation. Although TOPSIS and VIKOR are intuitive,
the rough set approach may require significant explanation to help managers under-
stand the process. Thus, acceptance of the approach by decision makers may take
some convincing. Overall, the development of a decision support tool to help in the
process will make it easier for acceptance.
Thus, there is significant direction for future research and application. This chap-
ter only seeks to introduce this multi-stage, multi-method approach. Its application
to GGP, and general green procurement, is made evident.
Acknowledgments This work is supported by the National Natural Science Foundation of China
Project (71102090, 71472031); Program for Liaoning Excellent Talents in University (WJQ2014029).
References
Amann M, Roehrich J, Eßig M, Harland C (2014) Driving sustainable supply chain management
in the public sector: the importance of public procurement in the European Union. Supply
Chain Manage 19(3):351–366
Arora S, Cason TN (1995) An experiment in voluntary environmental regulation: participation in
EPA’s 33/50 program. J Environ Econ Manage 28(3):271–286
Bai C, Sarkis J (2010a) Integrating sustainability into supplier selection with grey system and
rough set methodologies. Int J Prod Econ 124(1):252–264
Bai C, Sarkis J (2010b) Green supplier development: analytical evaluation using rough set theory.
J Clean Prod 18(12):1200–1210
Bai C, Sarkis J (2011) Evaluating supplier development programs with a grey based rough set
methodology. Expert Syst Appl 38(11):13505–13517
Bai C, Sarkis J (2013a) Flexibility in reverse logistics: a framework and evaluation approach.
J Clean Prod 47:306–318
Bai C, Sarkis J (2013b) Green information technology strategic justification and evaluation. Inf
Syst Front 15(5):831–847
Bai C, Sarkis J (2014) Determining and applying sustainable supplier key performance indicators.
Supply Chain Manage 19(3):275–291
Bai C, Dhavale D, Sarkis J (2014) Integrating Fuzzy C-Means and TOPSIS for performance evalu-
ation: an application and comparative analysis. Expert Syst Appl 41(9):4186–4196
Brandenburg M, Govindan K, Sarkis J, Seuring S (2014) Quantitative models for sustainable sup-
ply chain management: developments and directions. Eur J Oper Res 233(2):299–312
CECNA/FWI (2003) Green procurement: good environmental stories for North Americans.
Commission for Environmental Cooperation of North American/Five Winds International. http://
www.resourcesaver.org/file/toolmanager/CustomO16C45F42803.pdf. Accessed 10 Oct 2006
Chen MF, Tzeng GH (2004) Combining grey relation and TOPSIS concepts for selecting an expa-
triate host country. Math Comput Model 40(13):1473–1490
Chen SJJ, Hwang CL, Beckmann MJ, Krelle W (1992) Fuzzy multiple attribute decision making:
methods and applications. Springer, New York
Chu MT, Shyu J, Tzeng GH, Khosla R (2007) Comparison among three analytical methods for
knowledge communities group-decision analysis. Expert Syst Appl 33(4):1011–1024
Day C (2005) Buying green: the crucial role of public authorities. Local Environ 10(2):201–209
Deltas G, Harrington DR, Khanna M (2014) Green management and the nature of pollution pre-
vention innovation. Appl Econ 46(5):465–482
Dou Y, Sarkis J, Bai C, Dou Y, Sarkis J, Bai C (2014) Government green procurement: a Fuzzy-
DEMATEL analysis of barriers. In: Supply chain management under fuzziness, vol 333.
Springer, Berlin, pp 567–589
Dubois D, Prade H (1980) Systems of linear fuzzy constraints. Fuzzy Set Syst 3(1):37–48
European Commission (EC) (2004) Buying green!—a handbook on environmental public procure-
ment. Office for Official Publications of the European Communities, Luxembourg. http://ec.
europa.eu/environment/gpp/buying_handbook_en.htm
Geng Y, Doberstein B (2008) Developing the circular economy in China: challenges and opportu-
nities for achieving ‘leapfrog development’. Int J Sustain Dev World Ecol 15(3):231–239
Govindan K, Rajendran S, Sarkis J, Murugesan P (2015) Multi criteria decision making approaches
for green supplier evaluation and selection: a literature review. J Clean Prod 98:66–83
Ho LW, Dickinson NM, Chan G (2010) Green procurement in the Asian public sector and the
Hong Kong private sector. Nat Resour Forum 34(1):24–38
Hwang CL, Yoon K (1981) Multiple attribute decision making: methods and applications: a state
of the art survey. Springer, New York
Krohling RA, Campanharo VC (2011) Fuzzy TOPSIS for group decision making: a case study for
accidents with oil spill in the sea. Expert Syst Appl 38(4):4190–4197
Kunzlik P (2003) International procurement regimes and the scope for the inclusion of environ-
mental factors in public procurement. In: OECD (ed) The environmental performance of public
procurement, p 157
Li DF (2012) A fast approach to compute fuzzy values of matrix games with payoffs of triangular
fuzzy numbers. Eur J Oper Res 223(2):421–429
Liang J, Shi Z (2004) The information entropy, rough entropy and knowledge granulation in rough
set theory. Int J Uncertainty Fuzziness Knowl Based Syst 12(1):37–46
Liang J, Shi Z, Li D (2006) Information entropy, rough entropy and knowledge granulation in
incomplete information systems. Int J Gen Syst 35(6):641–654
Liang J, Wang F, Dang C, Qian Y (2012) An efficient rough feature selection algorithm with a
multi-granulation view. Int J Approx Reason 53(6):912–926
Marron DB (1997) Buying green: government procurement as an instrument of environmental
policy. Public Finan Rev 25(3):285–305
McMurray AJ, Islam MM, Siwar C, Fien J (2014) Sustainable procurement in Malaysian organiza-
tions: practices, barriers and opportunities. J Purch Supply Manage 20(3):195–207
Michelsen O, de Boer L (2009) Green procurement in Norway: a survey of practices at the munici-
pal and county level. J Environ Manage 91(1):160–167
Mosgaard M, Riisgaard H, Huulgaard RD (2013) Greening non-product-related procurement—
when policy meets reality. J Clean Prod 39:137–145
New S, Green K, Morton B (2002) An analysis of private versus public sector responses to the
environmental challenges of the supply chain. J Public Procurement 2(1):93–105
Nissinen A, Parikka-Alhola K, Rita H (2009) Environmental criteria in the public purchases above
the EU threshold values by three Nordic countries: 2003 and 2005. Ecol Econ 68(6):1838–1849
Norberg-Bohm V (1999) Stimulating ‘green’ technological innovation: an analysis of alternative
policy mechanisms. Policy Sci 32(1):13–38
OECD (2006) OECD Factbook. OECD, Paris
Opricovic S, Tzeng GH (2002) Multicriteria planning of post‐earthquake sustainable reconstruc-
tion. Comput Aided Civil Infrastruct Eng 17(3):211–220
Opricovic S, Tzeng GH (2004) Compromise solution by MCDM methods: a comparative analysis

of VIKOR and TOPSIS. Eur J Oper Res 156(2):445–455
Opricovic S, Tzeng GH (2007) Extended VIKOR method in comparison with outranking methods.
Eur J Oper Res 178(2):514–529
Parikka-Alhola K (2008) Promoting environmentally sound furniture by green public procure-
ment. Ecol Econ 68(1):472–485
Pawlak Z (1982) Rough sets. Int J Comput Inf Sci 11(5):341–356
Preuss L (2009) Addressing sustainable development through public procurement: the case of
local government. Supply Chain Manage 14(3):213–223
Seuring S (2013) A review of modeling approaches for sustainable supply chain management.
Decis Support Syst 54(4):1513–1520
Shyng JY, Wang FK, Tzeng GH, Wu KS (2007) Rough set theory in analyzing the attributes of
combination values for the insurance market. Expert Syst Appl 32(1):56–64
Swanson M, Weissman A, Davis G, Socolof ML, Davis K (2005) Developing priorities for greener
state government purchasing: a California case study. J Clean Prod 13(7):669–677
Tzeng GH, Lin CW, Opricovic S (2005) Multi-criteria analysis of alternative-fuel buses for public
transportation. Energy Policy 33(11):1373–1383
Walker H, Brammer S (2009) Sustainable procurement in the United Kingdom public sector.
Supply Chain Manage 14(2):128–137
Yu PL (1973) A class of solutions for group decision problems. Manage Sci 19(8):936–946
Zhu Q, Sarkis J, Lai KH (2012) Green supply chain management innovation diffusion and its
relationship to organizational improvement: an ecological modernization perspective. J Eng
Technol Manage 29(1):168–185
Zhu Q, Geng Y, Sarkis J (2013) Motivating green public procurement in China: an individual level per-
spective. J Environ Manage 126:85–95
Enhancing the Usability of Climate
Information and Models Through Stakeholder
Engagement
Elizabeth Allen and Jennie C. Stephens
Abstract As the impacts of climate change intensify, environmental scientists and

earth system modelers are rapidly advancing our understanding of the complex
dynamics of climate change. Although large quantities of high-quality climate data
and climate science information are now publicly available, the usability of this
information to policy-makers and other nonacademic decision-makers is limited. It
is widely recognized that novel approaches to increasing the usability of climate
science information and enhancing its influence in decision-making are needed.
This chapter reports on a regional climate change impacts modeling project called
BioEarth funded by the United States Department of Agriculture (USDA) and based
at Washington State University, which is attempting to bridge the gap climate sci-
ence usability gap. We consider the lessons learned in the course of BioEarth’s
stakeholder engagement efforts and reflect on how those insights may contribute to
other initiatives seeking to develop and communicate useable climate science infor-
mation. Lessons from BioEarth may be applicable to USDA’s recently established
Regional Climate Hubs, which are envisioned as internet-based centers that will
deliver climate change data and analyses to farmers, ranchers, and forest landown-
ers. This research contributes to the emerging field of policy informatics by explor-
ing mechanisms by which climate science data and models can become more usable
to decision-makers.
E. Allen (*)
Washington State University, Pullman, WA, USA
e-mail: lizb.allen@gmail.com
J.C. Stephens
University of Vermont, Burlington, VT, USA
e-mail: jstephe1@uvm.edu

DOI 10.1007/978-3-319-25439-5_6
122 E. Allen and J.C. Stephens
Introduction
As climate change is recognized as an inevitable and growing threat to food sys-

tems, water supply, infrastructure, and public health, governments around the world
are faced with the challenge of how to harness information and models to prepare
for impacts and reduce the multiple risks of climate change (IPCC 2014). The past
two decades has seen a rapid rise in production of climate change science, yet it is
well recognized that climate science information is underutilized in decision-
making (Weaver et al. 2013). Acknowledgement of the need to overcome the so-
called climate science usability gap (Lemos et al. 2012) has resulted in the
development of an array of novel government-supported strategies for enhancing
the relevance of climate science information and models (McNie 2007; Weaver
et al. 2013).
The research presented here contributes to initiatives designed to support societal
engagement with climate science by reviewing insights from academic researchers
and nonacademic decision-making stakeholders participating in a climate modeling
research project, BioEarth, based at Washington State University. BioEarth is a
5-year integrated earth system modeling project focusing on the Columbia River
Basin in the Pacific Northwest with an explicit emphasis on engaging directly
throughout the modeling process with regional policy-makers and other stakeholders
who may be able to use model results to inform their decision-making (Adam et al.
2014). BioEarth can be viewed as a project that takes a bottom-up approach to pro-
ducing useable climate science information relevant to the forestry and agriculture
sectors in the Pacific Northwest. This project involves 18 principal investigators at
multiple institutions who are working to integrate process-based models that provide
detailed information on specific subsystems including land, air, water, and econom-
ics. BioEarth is one of multiple regional-scale earth system models (EaSMs) funded
by the United States Department of Agriculture (USDA) to generate usable informa-
tion for natural resource decision-makers. As such, the project is representative of
the value that federal funding agencies increasingly place on integrated modeling
approaches and explicit stakeholder engagement processes to ensure buy-in to
enhance the salience, credibility, and legitimacy of model results.
In this chapter, we consider lessons learned from the BioEarth project as they
relate to other initiatives seeking to provide useable climate science information to
policy-makers and resource management decision-makers at the regional level.
Specifically, the recently established Regional Climate Hubs are discussed as an
example of a program that could benefit from incorporating findings from the
BioEarth project related to communication and stakeholder engagement. USDA’s
Regional Climate Hubs are envisioned as entities that will collaboratively provide
data, research findings, tools, and forecasts, i.e., integrated information for the
agricultural and forestry sectors, with particular attention to meeting the needs of
underserved populations, including rural communities and tribes (USDA 2014).
We juxtapose analysis of the BioEarth regional integrated modeling project with
consideration of the newly established Regional Climate Hubs because both
Enhancing the Usability of Climate Information and Models Through Stakeholder… 123
initiatives seek to operate within the emerging paradigm of enhanced communica-

tion and closer collaboration between climate scientists and professionals who
make decisions on the basis of technical/scientific information and “useable” sci-
ence. These two initiatives are dissimilar in their scope, breadth of the intended
audience, and decision-making process through which they were envisioned,
designed, and implemented, but they share the core objective of enhancing usability
of climate science. The major distinction between these initiatives is that the
BioEarth project was developed in response to a USDA National Institute for Food
and Agriculture (NIFA) and National Science Foundation call for proposals (grant
number 2011-67003-30346) for interdisciplinary integrated environmental model-
ing research, and the Regional Climate Hubs are an initiative explicitly seeking to
improve information dissemination, which was called for by the executive branch of
the US government as a component of President Obama’s Global Climate Change
Initiative. The stated goal of the Regional Climate Change Hubs is to “deliver tai-
lored, science-based knowledge to farmers, ranchers, and forest landowners to help
them understand and prepare for the impacts of climate change” (Whitehouse.gov
2014). In the case of BioEarth, the stakeholder engagement approach is one feature
of the proposal that made it unique and attractive to the funding agency (the USDA),
but the research project was primarily selected for funding based on the scientific
strength of its goal to link hydrologic, atmospheric, and terrestrial models and assess
system changes and feedbacks under future climatic and socioeconomic change.
Based on findings from communication and stakeholder engagement research con-
ducted within the BioEarth project, this chapter presents lessons about effective prac-
tices and potential challenges for initiatives seeking to develop and provide useable
climate science information to decision-makers. Given the daunting scale and com-
plexity of climate science information, this exploration of initiatives designed to
enhance the usability of climate science data and modeling is a novel contribution to
the emerging field of policy informatics which includes research and practice focused
on supporting policy decision-making based on data, scientific evidence and modeling,
and creating, managing, and evaluating systems for policy development, analysis, and
implementation (LaValle et al. 2013). Enhancing the usability of climate data is becom-
ing a critical societal challenge, where policy informatics offers valuable insights.
The chapter begins with a review of the literature about interdisciplinary science
as it relates to the goal of useable information for decision-makers, focusing on (1)
characteristics of information and (2) characteristics of stakeholder engagement
approaches that contribute to the production and use of decision-relevant scientific
knowledge. The methods section details the process of collecting and analyzing
information about the perceptions and information needs of the non-scientist stake-
holders involved in the BioEarth project. Specific recommendations and insights
from stakeholders are presented in the results section. The concluding discussion
considers lessons learned during BioEarth’s stakeholder engagement efforts as they
may inform and contribute to the implementation of Regional Climate Hubs and
other initiatives that seek to engage decision-makers in the production of climate
science knowledge and promote the use of earth systems model outputs and scien-
tific information in decision-making.
Literature Review: Interdisciplinary Science, Big Data,

and the Need for Stakeholder Engagement
Complex environmental change questions and resource management decisions,

which initiatives such as the BioEarth Project and the Regional Climate Hubs seek
to inform, require input from diverse interdisciplinary teams (Adam et al. 2014;
USDA 2014). Climate change impacts research connects biological sciences, atmo-
spheric sciences, hydrology, engineering, economics, social sciences, and commu-
nication studies. BioEarth and Regional Climate Hubs are representative of a
growing body of large interdisciplinary climate change projects working at the con-
fluence of multiple disciplines. The developing interdisciplinary approach stands in
contrast to traditional reductionist science (Lélé and Norgaard 2005; Pennington
et al. 2013) In interdisciplinary research, the focus is shifted away from analyzing
strictly defined inputs and toward synthesizing bodies of knowledge to produce
socially relevant outputs (Meinke 2006; Pennington et al. 2013).
Traditional reductionist approaches to problem definition are one central reason
why potential end-users of climate science often fail to incorporate state-of-the-
science information in their decision-making or deem climate information unusable
or irrelevant for their specific needs (Cash and Buizer 2005; Lemos et al. 2012;
Weaver et al. 2013). Cash and Buizer (2005) suggest that the effective translation of
climate science information into on-the-ground action requires three essential com-
ponents: salience (perceived relevance of the information), credibility (perceived
technical quality of the information), and legitimacy (perceived objectivity of the
information-sharing process). Within the climate science research community,
awareness of the need to ensure and highlight the salience, credibility, and legiti-
macy of climate science information is expanding (McNie 2007; Allen et al. 2013a,
b). BioEarth’s stakeholder engagement strategy and the Regional Climate Hubs ini-
tiative are both illustrative of this expansion of understanding in the research
community.
The complexity, uncertainty, and large quantities of climate science data pose
perpetual challenges for usability. As climate science researchers and communica-
tors attempt to improve the usability of the data, they face many challenges includ-
ing: (1) sources of the climate change problem are diverse and disparate in time and
space; (2) the scientific data is complex and uncertain, with linkages and feedbacks
that are often not fully understood; and (3) powerful political and economic inter-
ests may interfere with the production and dissemination to the policy-making
sphere of climate science knowledge (Moser 2010; Weaver et al. 2013). To address
these challenges, modeling efforts designed to generate useable climate science
information must be able to assess where vulnerably exists in a range of projected
conditions, and then explore decisions that perform well across a range of possible
futures (Lemos et al. 2012; Convertino et al. 2013). This is in contrast to the conven-
tional assumption that scientists can supply data and models for decision-makers to
“predict, then act.” The traditional reductionist approach attempts to determine the
most likely future conditions to inform the design of appropriate management plans,
investments, and policies (Lemos et al. 2012; Weaver et al. 2013). A fundamental
challenge with this “predict, then act” system is that it places unrealistic demands
on climate impacts modeling. The newer more robust modeling framework that
assesses vulnerabilities and provides insights on a range of different futures seeks to
better understand and better represent how socio-ecological systems work. Models
focus on identifying where management plans, investments, or policies may fail—
accounting for complexity and uncertainty in both earth systems and human
decision-making (Weaver et al. 2013). The new vulnerability assessment approach
depends on intensive engagement and collaboration with decision-makers from an
early stage in problem definition and research program design (Hegger et al. 2012;
Weaver et al. 2013; Pennington et al. 2013). Establishing close working relation-
ships with the stakeholders who may be able to apply current climate science infor-
mation is essential to overcome shortcomings of the traditional approach (Weaver
et al. 2013).
Literature about stakeholder identification and stakeholder engagement in envi-
ronmental science draws from many fields including business and organizational
theory, political science, sociology, and anthropology. In organizational theory,
stakeholders are defined as individuals or groups who can affect, or are affected by,
the actions and results of a specific organization, initiatives, policies, or projects
(Harrison and Freeman 1999). In climate information initiatives reported on in this
chapter, stakeholders are considered any individuals or organizations that might
have interest or an ability to use climate data and models in their decision-making.
In management literature, stakeholder theory is centered on the idea that institutions
should focus on meeting a broader set of interests than simply amassing profits for
shareholders. Benefits and impacts on other constituents (stakeholders) should also
be considered in decision-making. In the context of considering stakeholder theory
in research, this translates to focusing on transmitting relevant knowledge and infor-
mation to a broader set of interests than the traditional academic audience. The key
principle of stakeholder engagement is that organizations seek to understand,
respect, and meet the needs of parties who have a stake in the actions and outcomes
of the organization (Plaza-Úbeda et al. 2010). Involving stakeholders in the produc-
tion of knowledge can be seen as an ethical requirement of businesses or institutions
(Harrison and Freeman 1999). Importantly, in the field of environmental science,
and specifically climate change science, working closely with stakeholders outside
the academic community can be understood as a strategic necessity to ensure that
consideration of up-to-date scientific information about climate change impacts and
risks is taken into account in decisions about land and resource management and
social policy (Plaza-Úbeda et al. 2010; Hegger et al. 2012).
Stakeholders have been classified in three broad categories: internal, external,
and distal (Sirgy 2002). In the case of an environmental science research initiative,
internal stakeholders include program administrators, principal investigators, grad-
uate students, collaborating researchers, and science advisors. External stakeholders
include all individuals and groups who may be able to use the climate science data.
Distal stakeholders are traditionally defined as groups with competing or conflicting
interests to the internal stakeholder groups (Sirgy 2002). In the case of research and
science communication initiatives, groups with directly competing interests may

not be readily identifiable. Members of the general public or interest groups without
a direct relationship to the research initiative may be considered, or may self-
identify, as “non-stakeholders” although it is important to understand that even par-
ties who do not see themselves as having a direct stake in the outcome of a research
or science communication initiative may still have an economic, health, or social
well-being interest in the larger environmental change issue (Harrison and Freeman
1999; Plaza-Úbeda et al. 2010). An alternate definition and typology of stakehold-
ers is based on the concept that in order to be a stakeholder, the group or individual
must possess one or more of the following relationship attributes: power, legitimacy,
and urgency (Mitchell et al. 1997). Given the all-encompassing impacts of climate
change, it could be argued that everyone and every organization have some legiti-
macy and power relationship with climate science information.
In addition to grappling with the question of how to define stakeholders, institu-
tions seeking to develop and communicate useable climate science data, informa-
tion, and decision-making tools must invest time and resources in establishing a
plan for how stakeholders will be engaged (McNie 2012). Potential roles of stake-
holder in research are varied and can include any of the following:
• Identifying research questions
• Sharing values, preferences, expectations, and perceptions of risk
• Providing quantitative data or local expertise
• Commenting on research concepts, drafts, and results
• Learning from the research process
• Integrating research findings into a decision-making processes (Bucchi and
Neresini 2008)
Policy informatics is a field of research and a community of practice focused on:
(1) supporting policy decision-making that is based on data, scientific evidence, and
modeling; and (2) creating, managing, and evaluating systems for policy develop-
ment, analysis, and implementation (LaValle et al. 2013). Research focused on the
relationship between scientific data, modeling tools, and policy analysis and imple-
mentation is emerging contributing to the field of policy informatics (Niemeijer
2002; Pielke 2007; Convertino et al. 2013). Policy informatics includes tools, mod-
els, and simulations to help individuals and groups deliberate and evaluate policy
decisions that can be informed by large amounts of data (LaValle et al. 2013). There
is a practical challenge of reconciling the supply of scientific information with
users’ demands (McNie 2007; Sarewitz and Pielke 2007). Oftentimes, decision-
makers have specific information needs that are unmet, or they may not be aware of
the existence of potentially useful information. In the climate science arena, there is
a great need for continuing development of climate data and models that represent
large amounts of complex and uncertain information in a way that might be salient
and relevant to policy-making and other decision-making. Arguably, the greatest
needs surrounding climate data do not concern the quantity or quality of informa-
tion, but rather the organization, representation, communication, and accessibility
of that information.
Research Methods
The BioEarth research team includes a working group focused on communication

that facilitates engagement between the academic scientists and nonacademic stake-
holders and also conducts research on these interactions. This section describes how
nonacademic stakeholders were recruited to participate in the BioEarth project and
how the stakeholder engagement workshops were structured. The methods for gath-
ering information from BioEarth researchers and stakeholders about their percep-
tions relating to characteristics of useable climate science information and
characteristics of effective engagement and outreach approaches are detailed.
The stakeholder engagement approach utilized in BioEarth was designed based
on a history of cooperative agricultural and forestry extension programs that com-
municate university research to land managers and decision-makers (Bull et al.
2004). Building on work done by the Washington State University Center for
Sustaining Agriculture and Natural Resources (CSANR), BioEarth’s communica-
tion approach was designed to be bidirectional, allowing the environmental model-
ing team to learn from stakeholders about concerns and information needs while
developing decision-relevant model outputs. BioEarth researchers developed a plan
for a series of six issue-based workshops to learn from regional natural resource
managers. The project’s communication working group has conducted five of those
workshops to date; two workshops in 2013 focused on carbon and nitrogen manage-
ment and water supply, and three workshops in 2014 focused on rangeland manage-
ment, forestry, and atmospheric pollution. Using existing contacts, internet research,
and fellow stakeholders’ recommendations, the communication working group
identified and invited 384 stakeholders from state, federal, tribal, and local govern-
ment agencies, private industry, non-governmental organizations, and research
institutions. Ultimately, 87 stakeholders participated in a series of five workshops.
The workshop format consists of a brief presentation about BioEarth’s modeling
approach followed by facilitated round-table discussion. Digital response “clickers”
are used to allow real-time visualization of perceptions in the room and establish a
starting point for conversation about concerns and information needs.
With facilitation from experienced extension faculty on the communication
team, BioEarth environmental modelers engage directly with stakeholders with a
goal of mutual learning; the modelers learn from the stakeholders and the stake-
holders learn from the modelers. Feedback and recommendations provided by
stakeholders have been published in a series of Workshop Summary Reports avail-
able on the BioEarth website.1 Specific recommendations from the stakeholders
have been tabulated in spreadsheets that research team members use to formulate
and prioritize model development plans. Before and after each of the workshops,
participating stakeholders completed surveys consisting of multiple choice and
open-ended questions. Survey responses (84 pre-workshop surveys and 41 post-
workshop surveys) and comments recorded during workshops are the basis of
1
BioEarth workshop summary report, available at: http://www.cereo.wsu.edu/bioearth/publica-
tions.html
reflections and recommendations analyzed in this report. Responses were recorded

in Excel spreadsheets and coded based on emergent themes. NVivo qualitative anal-
ysis software is being used for further analysis of stakeholders’ written responses
and the detailed notes recorded by BioEarth communication working group
researchers at workshops.
Results: BioEarth Stakeholders’ Reflections

and Recommendations
Stakeholder advisory workshop discussions in the BioEarth project generated a

wealth of information about the types of data that might be useful to decision-makers
who are concerned with Pacific Northwest water the Pacific Northwest water supply,
carbon and nitrogen cycling, atmospheric issues, and rangeland and forest manage-
ment. Gaps in understanding related to the feedbacks between these systems in the
context of climate change also surfaced. A total of 87 stakeholders from outside the
BioEarth research team participated in advisory workshops. A majority of the indi-
viduals who participated in these workshops expressed enthusiasm and eagerness to
continue to be involved in the BioEarth model development process. Recognition of
the BioEarth project’s complexity and ambitious scope was frequent among partici-
pating stakeholders. At the same time that positive overall reactions to the research
initiative were frequently expressed, approximately 75 % of the 41 stakeholders who
completed a post-workshop evaluation survey expressed the opinion that the project
entailed significant challenges, which they were concerned may not be surmountable
within the scope of BioEarth. Stakeholders cited project complexity, funding limits,
and time limits as the primary factors that may present challenges for the research
team and limit the project outputs’ usability and relevance to decision-makers.
Participants demonstrated near-unanimous appreciation for the opportunity to con-
tribute to the BioEarth project. One representative from a non-governmental organi-
zation expressed, “stakeholder involvement from the early stages of research question
formulation generates buy-in and confidence in results,” and stakeholders noted fre-
quently that having even more early involvement in research question formulation
would be favorable.
Our results suggest that future communication with stakeholders as part of the
BioEarth project should include multiple opportunities for reflection, refinement,
and revisiting of linked process-based models, for example, providing sample
outputs that people can respond to. Seven of the stakeholders who completed a post-
workshop survey specifically stated that demonstrating sample model outputs in a
tangible way could help decision-makers understand the scope and scale of what is
possible, and comments during workshops reinforce this understanding that having
results to respond to and ask questions of is essential. Stakeholders urged environ-
mental modelers who are presenting their work to continue to develop their ability
to present technical concepts visually, using minimal jargon and acronyms. Along
with this feedback, four participants expressed that within interdisciplinary teams,
including experienced science communicators is essential for overall project
success. One stakeholder noted, “researchers should continue to make good use of
University Extension programs in the region—this is often where capacity lies in
getting information out to people who can use it.”
Survey responses and comments during discussion also demonstrate that presen-
tations to stakeholders are most effective when there are explicit linkages between
research questions and land or resource management decisions. Two participants
perceived a disconnection between the integrated modeling approach and separat-
ing workshops based on topic (i.e., cross-specialty dialogue is also important).
When prompted to describe which factors enhanced their learning and enjoyment of
workshops, stakeholders frequently cited experienced workshop facilitation,
thoughtful discussion questions, and having had the opportunity to see other stake-
holders’ responses to multiple-choice questions.
Reflecting on the project’s communication and outreach approaches, different
stakeholder groups have diverse levels of experience with environmental models
and climate science; their information needs vary accordingly. Responses to a dis-
cussion question posed at workshops suggest that fact sheets and research summa-
ries that clearly outline conclusions are more likely to be read and talked about than
long articles. The majority of participating stakeholders expressed that visual data,
including graphs, maps, and conceptual models are often easier to interpret than
dense text or equations and numbers. Among BioEarth stakeholders who completed
post-workshop surveys, approximately 70 % expressed strong interest in seeing
regional climate science information available via an easily navigable website with
the capacity for stakeholders to interact directly with researchers. There is demon-
strated interest in tools that would enable decision-makers to ask questions and
request clarification directly from scientists. Webinars are also recognized as valu-
able forums for learning and collaboration, but a majority of participants stated that
in-person workshops with environmental modelers are not interchangeable with
online presentations; an in-person workshop format enables a level of dialogue and
mutual understanding that cannot be replicated via a website or video conference.
BioEarth stakeholders also shared feedback about specific characteristics of cli-
mate science data and models that enhance the relevance, accessibility, and utility
of that information. One stakeholder summarized the general attitude of many when
they stated that “in order for models to be trusted in decision-making contexts, on-
the-ground monitoring is essential to track model accuracy.” However, it was also
stated that for government agencies, using model projections to make management
decisions opens up challenging ethical questions of how accurately models represent
different areas and processes. A finding from discussions as well as written survey
responses is that synthesis and clear interpretation of experimental data and models
are frequently of greater value to users than full access to data that contributed to a
given analysis.
Within environmental model-based research, which has a goal of understanding
complex environmental change and human-earth systems feedback processes, a
paradox exists. To adequately represent the diversity and complexity of the dynam-
ics of earth systems, the research must involve people with diverse expertise. Yet as
the complexity of model outputs increases, the pool of individuals who can interpret
those findings decreases. A high level of technical expertise is necessary to understand
complex models and to promote the application or usability of that research to deci-
sion-making. Many decision-makers who participated expressed that they would
welcome more opportunities to learn about the science of environmental modeling
and to further interact with BioEarth researchers to deepen their understanding of
model structure, limitations and capacity, or possible outputs.
Discussion: Implications for Initiatives Seeking to Produce

and Communicate Useable Climate Science
Stakeholders’ recommendations and reflections shared during BioEarth project

workshops suggest important considerations for other research programs. The dis-
cussion section is divided into three sections; we begin with a consideration of the
need for research teams to define shared goals and develop communication compe-
tencies (section “The Need to Define Goals and Develop Communication
Competencies”); next, we present a value chain describing stages in the production
of useable climate science information (section “Defining a Useable Climate
Science Value Chain”), finally lessons from the BioEarth project are discussed in
terms of how they could inform the national Regional Climate Hubs program (sec-
tion “Applying Lessons from a Regional Project to a National Program”).
The Need to Define Goals and Develop Communication

Competencies
Stakeholders in industry, government agencies, and advocacy organizations often

state that increased access to climate science data would be of benefit to their
decision-making. However, we find that there is a disconnect between the kind of
information that academic research institutions seek to produce and the kind of
information that will be of practical use for land-owners’ and resource managers’
decision-making (Allen et al. 2013a, b). Synthesis and clear interpretation of experi-
mental data and models may be of considerably greater value to users than would be
full access to data that contributed to a given analysis. It is essential for institutions
seeking to provide useable information to be closely attuned to their audience’s
information needs.
A key lesson from BioEarth is that researchers and science communicators must
clearly conceptualize joint goals at the outset of a useable climate science project.
Often, involvement in interdisciplinary stakeholder-focused research requires that team
members to develop new vocabularies and new skills. For example, communication
researchers will need to learn about process-based modeling or modelers may need to
devote time and energy to developing skills and experience related to stakeholder
engagement. Research scientists who are less familiar with policy-making processes
and decision-makers’ needs may need training and opportunities for learning that will
enhance their understanding of how scientific information is digested and applied.

Based on feedback received in the course of the BioEarth project, we find that if poten-
tial benefits to stakeholders can be made explicit from the outset of a project, stakehold-
ers have more clarity of purpose in their engagement. Potential benefits may be different
for different stakeholders, but sending a clear signal to stakeholders that their interests
and questions are being responded to improve the likelihood the results of scientific
inquiry will be applied in decision-making contexts. Certain information needs or use-
able model outputs are highly specific to different kinds of stakeholders, and those
kinds of outputs and analyses cannot be produced in the absence of close working
relationships with the stakeholders who may use that information and the model devel-
opers. What constitutes meaningful climate science information will certainly vary
depending on the defined end-user, and there may be regional variability as to the most
pressing information gaps and information needs of policy decision-makers.
Defining a Useable Climate Science Value Chain
The value chain concept, which has origins in business and management literature,
describes a series of activities that create and enhance value. Eventually, these activ-
ities culminate in an output, which the organization delivers (Porter 1998). The
value chain concept has been applied to many specific contexts, including knowl-
edge management (Lee and Yang 2000) and big data development and management
(Miller and Mork 2013). Here, we adapt the value chain concept to describe a
framework for developing and synthesizing complex, data-intensive climate sci-
ence information where usability for decision-makers is the end goal (Fig. 1).
Processes that add value, as indicated by BioEarth project stakeholders, are sum-
marized and displayed in a value chain diagram that other research institutions and
initiatives can refer to and adapt to suit specific project attributes. Steps of knowl-
edge production, information integration, and application of environmental change
Knowledge Production Information Integration Application
Define project Implement Synthesis: Analysis: Reporting: Communicating for

scope: project: -Incorporate -Run scenarios, -Tailor outputs and Decision Support:
-Establish research -Facilitate stakeholder input produce model/ data to audience -Publicize tools and
team stakeholder -Clarify terms research outputs needs outputs
-Identify and engage engagement -Integrate models, -Assess -Develop visual and -Refine tools for
stakeholders -Assess define scenarios uncertainties graphical stakeholders, e.g.
-Formulate research information needs -Develop tools/ -Evaluate representations visualization,
question(s) -Collaborate system for data results/ outputs -Webinars, classes. simulations and
-Data management across disciplines management publications, online gaming
plan tools, Extension environments
Continual project evaluation and revisiting of goals and stakeholders’ needs.
Fig. 1 The useable climate science value chain (adapted from Miller and Mork 2013)
research products are defined. This value chain is not simply a linear process that
produces a defined output. Rather, it is more accurately understood as an iterative
cycle because there is a need for continual evaluation and engagement with stake-
holders to adapt the research approach and generate relevant outputs.
Applying Lessons from a Regional Project to a National Program
An example of a larger initiative that may be able to incorporate lessons and best
practices suggested by the BioEarth regional modeling project is USDA’s Regional
Climate Hubs program. This recently established initiative has the ambitious vision
of accessible, credible, salient, and legitimate climate science, primarily internet-
based information, targeted for agriculture, rangeland, and forestry decision-makers.
In February 2014, United States Secretary of Agriculture Tom Vilsack announced
the seven regional information centers and three sub-hub locations charged with
delivering information to support adaptation to climate change and weather vari-
ability (USDA 2014). Climate hubs will be located at existing Agricultural Research
Service (ARS) or Forest Service locations, and leadership in each regional hub will
be provided by the ARS, the Forest Service and the Natural Resource Conservation
Service (NRCS) will jointly provide leadership in each Regional Climate Hub
(USDA 2014). According to a USDA fact sheet outlining the initiative, partners will
include: universities, including extension departments; USDA researchers, pro-
grams, and field offices; private sector farm groups; state, local, and regional gov-
ernments; tribes; National Oceanic and Atmospheric Administration and Department
of the Interior regional climate change experts; and non-profits (USDA 2014).
As of December 2014, the structural elements needed to support the Regional
Climate Hubs are still under development. This will necessitate allocation of funds,
human resources, and development of transdisciplinary science communication
expertise. Salience of climate science requires that modelers and researchers look
beyond disciplinary boundaries and explore interactions between science and policy
decision-making, and between the environment and socioeconomic forces (Meinke
et al. 2005; Sarewitz and Pielke 2007). Close communication and engagement with
diverse stakeholders can improve the incorporation of socioeconomic variables into
future change scenario development. Credibility emerges from social relationships
between climate scientists and those affected by climate risk. Science derives cred-
ibility not just from its technical precision, but also from engagement processes
(Meinke et al. 2005). Establishing legitimacy depends on making processes of
knowledge development and sharing transparent and equitable (Meinke et al. 2005).
There are potential tensions and challenges associated with the design of the
Regional Climate Hubs. This program was established by the highest level of lead-
ership within USDA as a component of the Obama administration’s National
Strategy on Climate Change. The decision came without a commitment of new
funding, and hub leaders were selected with minimal engagement in the planning
and decision-making process. Programmatic decisions made by a high level of lead-
ership in a federal agency may be perceived by those on the ground as lacking clar-
ity in terms of specific responsibilities and defined, measurable objectives (Sabatier
1986). Observing the general history of top-down program design, there is some
risk that the burden of program implementation will fall on agency staff, who are
already constrained by limits on available funding and other job demands. While
USDA staff and partners will be responsive to the directives from the leadership in
the agency, there is also concern that work developing Climate Hubs in the man-
dated manner will take place at the expense of bottom-up, locally specific approaches
to providing decision-makers with relevant and useable climate science informa-
tion. BioEarth stakeholders’ feedback indicates that bottom-up approaches that
engage decision-makers directly from an early stage in program design and provide
ample opportunities for viewing sample model outputs and data are the surest path-
way toward useable climate science information for decision-makers.
It is important to note that analyses and tools that have already been developed in
collaboration with stakeholder groups may provide some of the initial content for the
Regional Climate Hubs. Established working relationships with stakeholders in diverse
sectors may lay the groundwork for an audience or user community that will continue
to interact with the regional hub for useable tools. The BioEarth project, for example,
may directly integrate and overlap with the emerging Pacific Northwest Climate Hub.
Conclusion
Climate change will dramatically affect our food, water, energy, and infrastructure
systems, so building understanding of feedbacks, trade-offs, and drivers of change
is of paramount importance (Miller et al. 2013). Moreover, ensuring that informa-
tion is accessible and readily utilizable by policy decision-makers and land and
resource managers is essential (Weaver et al. 2013). Regionally focused research on
climate change impacts on natural resource management needs to be a top priority
(Convertino et al. 2013; Hegger et al. 2012). Washington State University’s BioEarth
project and the newly established USDA Regional Climate Hubs are well posi-
tioned to carry out and communicate much of this vital work.
In order to maximize usability and effectiveness of knowledge and information,
stakeholders’ perspectives and information needs must be incorporated into knowl-
edge production and knowledge dissemination initiatives. While BioEarth’s focus is
specifically on supporting development of a regional integrated earth systems model
that is relevant to farmers, foresters and natural resource managers in Oregon,
Washington, and Idaho, the lessons learned from round-table discussions at stake-
holder workshops and from in depth pre- and post-workshop surveys are applicable
beyond the temporal and geographic scope of the research project. Detailed feed-
back from a diverse group of the term natural resource managers encompasses this
group and natural resource decision-makers in BioEarth may contribute to envi-
sioning approaches for other initiatives seeking to create and promote the use of
science data and models for decision-makers.
The potential for transdisciplinary sustainability science research projects to

improve the understanding of regional human–environment interactions and inform
decisions is large. We find that overcoming multiple challenges related to commu-
nication across disciplinary divides and between researchers in academia and stake-
holders in government, industry, and non-profit organizations requires learning new
approaches to research and communication. These new approaches may include all
of the following: working in the early phase of research question development to
identify stakeholders’ information needs; developing shared vocabularies and new
forums for translating and communicating knowledge; and working closely with
stakeholder groups to increase organizational capacity to apply research findings.
Acknowledgments This research is funded from the Department of Agriculture, National

Institute of Food and Agriculture grant number 2011-67003-30346. The authors would like to
thank the stakeholders who have contributed invaluable insights to the BioEarth project and have
engaged in critically important work addressing regional environmental challenges.
References
Adam J, Stephens J, Chung S, Brady M, Evans R, Kruger C, Lamb B, Liu M, Stöckle C, Vaughan
J, Chen Y, Guenther A, Harrison J, Kalyanaraman A, Leung F, Leung L, Perleberg A, Tague C,
Yoder J, Hamlet A, Nijssen B, Chinnayakanahalli K, Choate M, Jiang X, Nelson R, Yoon J,
Yorgey G, Zhu J, Allen E, Anderson S, Malek K, Nergui T, Poinsatte J, Rajagopalan K, Reyes
J (2014) BioEarth: a regional biosphere-relevant earth system model to inform agricultural and
natural resource management decisions. In: ASA, CSSA and SSSA Annual Meetings on
Climatic Change, Cincinnati, OH, pp. 1–17.
Allen E, Kruger C, Leung FY, Stephens JC (2013a) Diverse perceptions of stakeholder engage-
ment within an environmental modeling research team. J Environ Stud Sci 3(3):343–356
Allen E, Kruger C, Leung FY, Stephens JC, Yorgey G (2013) BioEarth stakeholder advisory work-
shops synthesis report. Washington State University. http://www.cereo.wsu.edu/bioearth/pub-
lications.html. Accessed 17 May 2014
Bucchi M, Neresini F (2008) Science and public participation. The handbook of science and tech-
nology studies. MIT, Cambridge, pp 449–472
Bull NH, Cote LS, Warner PD, McKinnie MR (2004) Is extension relevant for the 21st century.
J Ext 42(6):1–6
Cash D, Buizer J (2005) Knowledge-action systems for seasonal to interannual climate forecast-
ing: summary of a workshop, report to the roundtable on science and technology for sustain-
ability, policy and global affairs. The National Academies Press, Washington, DC, http://books.
nap.edu/catalog/11204.html
Convertino M, Foran CM, Keisler JM, Scarlett L, LoSchiavo A, Kiker GA, Linkov I (2013)
Enhanced adaptive management: integrating decision analysis, scenario analysis and environ-
mental modeling for the everglades. Sci Rep 3, 2922
Harrison JS, Freeman RE (1999) Stakeholders, social responsibility, and performance: empirical
evidence and theoretical perspectives. Acad Manage J 42(5):479–485
Hegger D, Lamers M, Van Zeijl-Rozema A, Dieperink C (2012) Conceptualising joint knowledge
production in regional climate change adaptation projects: success conditions and levers for
action. Environ Sci Pol 18:52–65
IPCC (2014) Climate change 2014 mitigation of climate change, intergovernmental panel on
climate change. https://www.ipcc.ch/report/ar5/wg3/. Accessed 17 May 2014
LaValle S, Lesser E, Shockley R, Hopkins MS, Kruschwitz N (2013) Big data, analytics and the
path from insights to value. MIT Sloan Manag Rev 62:4
Lee CC, Yang J (2000) Knowledge value chain. J Manag Dev 19(9):783–794
Lélé S, Norgaard RB (2005) Practicing interdisciplinarity. BioScience 55(11):967–975
Lemos MC, Kirchhoff CJ, Ramprasad V (2012) Narrowing the climate information usability gap.
Nat Clim Chang 2:789–794
Meinke, H., & Stone, R. C. (2005). Seasonal and inter-annual climate forecasting: the new tool for
increasing preparedness to climate variability and change in agricultural planning and opera-
tions. Climatic change, 70(1-2), 221–253.
Meinke, H., Nelson, R., Kokic, P., Stone, R., Selvaraju, R., & Baethgen, W. (2006). Actionable
climate knowledge: from analysis to synthesis. Climate Research, 33(1), 101.
McNie EC (2007) Reconciling the supply of scientific information with user demands: an analysis
of the problem and review of the literature. Environ Sci Pol 10(1):17–38
McNie EC (2012) Delivering climate services: organizational strategies and approaches for
producing useful climate-science information. Weather Clim Soc 5(1):14–26
Miller HG, Mork P (2013) From data to decisions: a value chain for big data. IT Prof 15(1):57–59
Miller M, Anderson M, Francis CA, Kruger C, Barford C, Park J, McCowng BH (2013) Critical
research needs for successful food systems adaptation to climate change. J Agric Food Syst
Community Dev 3(4):161–175
Mitchell RK, Agle BR, Wood DJ (1997) Toward a theory of stakeholder identification and salience:
defining the principle of who and what really counts. Acad Manage Rev 22(4):853–886
Moser SC (2010) Communicating climate change: history, challenges, process and future
directions. Wiley Interdiscip Rev Clim Chang 1(1):31–53
Niemeijer D (2002) Developing indicators for environmental policy: data-driven and theory-driven
approaches examined by example. Environ Sci Pol 5(2):91–103
Pennington DD, Simpson GL, McConnell MS, Fair JM, Baker RJ (2013) Transdisciplinary
research, transformative learning, and transformative science. BioScience 63(7):564–573
Pielke RA (2007) The honest broker: making sense of science in policy and politics. Cambridge
University Press, Cambridge, p 188
Plaza-Úbeda JA, de Burgos-Jiménez J, Carmona-Moreno E (2010) Measuring stakeholder
integration: knowledge, interaction and adaptational behavior dimensions. J Bus Ethics
93(3):419–442
Porter ME (1998) Clusters and the new economics of competition. Harv Bus Rev 76(6):77–90
Sabatier PA (1986) Top-down and bottom-up approaches to implementation research: a critical
analysis and suggested synthesis. J Public Policy 6(1):21–48
Sarewitz D, Pielke RA Jr (2007) The neglected heart of science policy: reconciling supply of and
demand for science. Environ Sci Policy 10(1):5–16
Sirgy MJ (2002) Measuring corporate performance by building on the stakeholders model of busi-
ness ethics. J Bus Ethics 35(3):143–162
United States Department of Agriculture (USDA) (2014) USDA regional climate hubs. http://
www.usda.gov/oce/climate_change/regional_hubs.htm. Accessed 17 May 2014
Weaver CP, Lempert RJ, Brown C, Hall JA, Revell D, Sarewitz D (2013) Improving the contribu-
tion of climate model information to decision making: the value and demands of robust
decision frameworks. Wiley Interdiscip Rev Clim Chang 4(1):39–60
Whitehouse.gov (2014) Climate change and President Obama’s action plan. http://www.
whitehouse.gov/climate-change. Accessed 17 May 2014
Do Open Data Initiatives Promote and Sustain
Transparency? A Comparative Analysis
of Open Budget Portals in Developing
Countries
Jyldyz Kasymova, Marco Aurelio Marques Ferreira,

and Suzanne J. Piotrowski
Abstract This research provides a comparative analysis of open budget portals in

the context of budget transparency. Specifically, we examine open data portals in the
Ministry of Planning, Budget, and Management in Brazil and the Finance Ministry
in Kyrgyzstan. We utilize interviews with public administrators, archival informa-
tion, and an analysis of the portals. Furthermore, we use Meijer’s 2013 transparency
theoretical framework to assess the relationship between budget portals and trans-
parency in Brazil and Kyrgyzstan. The paper finds that both jurisdictions face simi-
lar challenges promoting online disclosure, including a lack of infrastructural
development and a limited use of portals by the public. Although centralized online
budget disclosures have not promoted a sense of transparency across the broader
public, the media has used these disclosures effectively. This has led to a wider
public discourse on budgeting in both countries. The authors highlight the specific
roles played by the Open Budget Index and international players in advancing and
sustaining budget transparency in these countries.
J. Kasymova (*)
SUNY Buffalo State, Buffalo, NY, USA
e-mail: jkasymova@gmail.com
M.A.M. Ferreira
The Federal University of Vicosa (UFV), Vicosa, Brazil
e-mail: marcoaurelio@ufv.br
S.J. Piotrowski
Rutgers University, Newark, NY, USA
e-mail: spiotrow@andromeda.rutgers.edu

DOI 10.1007/978-3-319-25439-5_7
138 J. Kasymova et al.
Introduction
Since the 1990s, information technologies have been widely used in the delivery of
public services. Internet use, as a means of communication between citizens and
governments, has intensified around the world. Technology has become an impor-
tant tool to improve government transparency as well. The role of information tech-
nology as an enabler of good governance, citizen engagement, and improvement of
accountability was also stressed in various studies (Koliba et al. 2011; Wachhaus
2011). At the same time, the global financial crisis has contributed to an increased
interest in financial transparency. In general, limited transparency or lack of it is
common for many emerging developing countries. In this context, understanding
the effectiveness of technology in facilitating and sustaining transparency becomes
crucial both for practitioners and theoreticians.
This paper utilizes an exploratory approach and examines the implementation of
online budget information initiatives in Brazil and Kyrgyzstan and its impact on the
advancement and sustainability of government transparency. Specifically, the paper
addresses the following research question: How do open data portals promote and
sustain transparency in developing countries?
To answer this question, we use Brazil and Kyrgyzstan as our information sources.
We specifically selected these two countries because of their different levels of eco-
nomic and technological progress. That said, Brazil and Kyrgyzstan are comparable
jurisdictions; both countries have a history of engaging citizens in budgeting and
have recently started using technology to share and display budget information.
Individual countries use different narratives to promote transparency policies.
Hajer (1993) argues that the use of a specific narrative impacts the implementation
process of a particular policy (Hajer 1993: 44). In Kyrgyzstan, budget transparency
has been presented as a natural force that is inevitable in the era of advanced infor-
mation technologies. In the case of Brazil, the main narrative has been disclosure in
order to change the culture of secrecy and enhance democracy.
This chapter is presented in the following format. We first offer the analysis of exist-
ing studies on transparency and open data initiatives. We also explain our theoretical
framework. Second, we explain methods and our information collection procedures.
Third, a brief introduction to the origins of transparency policies and the implementa-
tion of open data portals for each country are provided. We conclude the chapter with
a comparative analysis of both countries using Meijer’s theoretical framework.
Transparency and Open Data Initiatives in Existing

Literature
Transparency is defined as “availability of information about an actor that allows

other actors to monitor the workings or performance of the first actor (Meijer 2013:
2).” Different studies emphasize various aspects of transparency. For example,
Do Open Data Initiatives Promote and Sustain Transparency… 139
Heald (2006) makes the distinction between process and event transparency.
Commentators also separate online transparency and other forms of transparency
(Grimmelikhuijsen and Welch 2012; Cuillier and Piotrowski 2009; Meijer 2014). It
is believed that values regarding transparency or secrecy are grounded in the history
of a given country, but are also influenced by new developments (Meijer 2013). In
the context of this research, we use Meijer’s definition of transparency. At the same
time, we evaluate transparency from a sustainability perspective. Sustainability as a
concept has different definitions (Basiago 1995). It is often referred to as, “making
decisions based on how they will affect not only our generation but also seven gen-
erations to come (Carstens 2010).” The origin of the concept of sustainability is
rooted in ecology, but it is often used in the context of sociology and human interac-
tions. Methodologically, sustainability consists of several components; however, for
the purpose of this research we focus on two of its components: futility and equity
(Basiago 1995). Accordingly, in the context of this project, we define “sustainable
transparency” as transparency that is continuous and applicable or useful for all
groups of a population in a given country.
The principle of proactive disclosure, which implies that information must be
publicly available prior to a public request, is crucial in achieving transparency in
government (OECD 2011). Proactive disclosure ensures that information seekers
receive immediate access to public information, avoiding the costs of filing a request
or engaging in administrative procedures (OECD 2011). However, the act of dis-
closing information is not sufficient. Commentators note that information disclo-
sure can have some empowering effect only in a participatory and pluralistic
political culture with a strong civil society and independent media, as well as when
policy design allows the reported information to become embedded in the users’
routines (Dingwerth and Eichinger 2010).
Budget transparency is an important aspect of government transparency initia-
tives. Budget transparency is particularly important for many developing countries,
such as Brazil and Kyrgyzstan. Previous studies of budget transparency have fre-
quently discussed the topic in the context of citizen participation in budgeting
(Wagle and Shah 2001; Santos 1998; Hordijk 2009). As Sintomer et al. (2008) have
pointed out, engaging residents in budgeting contributes to improving the commu-
nication between citizens, administrators, and political elites. Several studies have
found that face-to-face involvement in budgeting has a positive influence on citizen
empowerment (Sintomer et al. 2008). But despite its effectiveness, this format of
engagement is feasible only at local levels. Therefore, a limited number of studies
have examined citizen involvement at the central government level (Bastida and
Benito 2007). As governments around the world are becoming more involved in
communicating with residents online, it becomes imperative to evaluate the effects
of budget information disclosure and online engagement mechanisms at the central
level. In this respect, policy informatics as an area of research gains importance,
specifically in understanding how technology changes policy processes at the group
and individual levels (Krishnamuthy et al. 2014). Furthermore, diverse categories
and sources of information assist in addressing complex managerial and social
problems (Helbig et al. 2012), which is also critical for developing countries.
Information is the fundamental element of government transparency as well;

therefore, changes in the quality, sources, and presentation of government informa-
tion have a direct impact on transparency. The types of information disclosed need
to be critically evaluated. Presently, numerous “administrative data” have been
released for external use; however, it is being disclosed without the consideration of
the public mindset, which makes its format less useful to the majority of the popula-
tion (Helbig et al. 2012). Furthermore, a well-designed informatics system should
incorporate a framework to engage the population in a robust interactive dialogue
(Koliba et al. 2011), which is often not the case for government-led online platforms
(Lampe et al. 2011).
In 2013, Rios and his colleagues studied socioeconomic, political, and institu-
tional factors that facilitated online budget disclosures internationally. Using a sam-
ple of 93 countries, the authors found that, among other things, internet penetration
and the education level of citizens are the most influential factors in budget openness.
The authors also found that Anglo-Saxon and Nordic countries are more likely to
conduct online budget information disclosure practices than continental countries.
Despite abundant empirical evidence supporting the benefits of an open budget,
some scholars are skeptical about the limited impact of online open budget initia-
tives. Commentators have even put some consistent arguments against transparency
in the context of public budgets (Kolstad and Wiig 2009; Prat 2005; Dabla-Norris
et al. 2010; Gollwitzer 2010).
Furthermore, Norris and Reddick (2013) have demonstrated that although there
are many potential positive outcomes in implementing electronic government data,
many drawbacks exist, including the high cost of implementation. It is also argued
that low-income countries do not have enough capacity to deploy e-government
services equitably throughout their territories, which could lead to ICT haves and
have-nots (Basu 2004: 177). At the same time, donor organizations and countries
tend to advance electronic open data initiatives as development projects in low-
income countries. In this respect, Heeks (2003) highlights that most e-government
projects in developing countries tend to fail, with 35 % often classified as total fail-
ures. Heeks (2003) notes that design-reality and public-private gaps are common
reasons for failures in low-income countries.
According to Aladwani (2013), any research on online government initiatives
needs to take into account various cross-cultural differences between countries. The
author demonstrates that perceptions of the quality and functionality of national
government websites differ between countries. The main practical implication of
Aladwani’s research is that global studies of portals need to understand and measure
cultural differences in order to evaluate holistically the effectiveness of government
portals and online information disclosure practices across countries.
In 2013, Meijer developed a theoretical model to analyze transparency. Meijer
notes that any transparency initiative is heavily impacted by environmental condi-
tions and its framing in society. Meijer argued that transparency is a dynamic phe-
nomenon, and could be evaluated using a three-stage analysis process, including
strategic analysis, cognitive analysis, and institutional analysis (Please refer to
Table 1). The three-stage analysis applied here would mean examining the strategic
Table 1 Heuristic model for the social-political construction of government transparency

Strategic analysis How do power games influence the construction of government
transparency?
How do new forms of transparency influence power games?
Institutional analysis How do institutional rules influence the construction of government
transparency?
How do new forms of transparency influence institutional rules?
Source: Meijer (2013: 432)
players in the transparency process, the framework of transparency, and the various
institutional rules that prevent or promote transparency. We focus on the strategic
and institutional analysis developed by Meijer because it is the most appropriate for
this research. Using the recommended framework, we would like to discern how the
new political structure influences budget disclosure and the facilitation of portals,
and how online disclosures influence institutional rule in Brazil and Kyrgyzstan.
Methods
Design
This chapter uses an exploratory case design of two countries. According to Dubé
and Paré (2003), case research is often used for descriptive and exploratory purposes.
Furthermore, a qualitative approach was necessary to gain a better understanding of
websites’ implementation challenges and their impact on transparency. We relied on
semi-structured interviews as our primary source of data. Three public officials
working specifically on open budget portals were interviewed in both countries dur-
ing the months of August 2013 and September 2014. Each interview lasted any-
where from 30 min to 1 h. Given the nature of the research questions, interviewees
were selected through snowball sampling. Interviews were analyzed for developing
themes and commonalities and differences between the two cases. The interview
questions solicited responses related to origins of open budget initiatives, key play-
ers, current implementation processes, and challenges. The complete list of inter-
view questions is provided in the appendix. In addition to interviews, we collected
and analyzed government documents and evaluated websites in both jurisdictions.
Case Selection
In this research, we compare the initiation, implementation, and consequences of

online budget portals in Brazil and Kyrgyzstan, relative to transparency. Brazil and
Kyrgyzstan are comparable entities due to several factors. First, both jurisdictions
have similarities in economic development. Both countries face identical challenges,

including a history of limited transparency and income disparity. Both jurisdictions
have a recent history of online budget disclosure and both countries experienced
popular protests triggered by inadequate public expenditures within the last 5 years.
Both countries launched online government open data portals in 2012. Two national
open data portals were created in Brazil: the 2004 portal administered by the National
Controller and a 2012 portal managed by the Ministry of Planning. In the case of
Brazil, we focus on the 2012 portal. In the case of Kyrgyzstan, we examine the 2011
budget portal created by the Finance Ministry. As a result the focus of the paper is on
“http://dados.gov.br/” in Brazil and “https://budget.okmot.kg” in Kyrgyzstan. Please
refer to Table 2 for key socioeconomic indicators for the two cases.
In the following sections we present a brief introduction of transparency condi-
tions in both countries and the implementation of open budget data portals.
Table 2 Key indicators in Brazil and Kyrgyzstan

Brazil Kyrgyzstan
System of Presidential Parliamentarian
government
Websites evaluated http://dados.gov.br/ https://budget.okmot.kg
Websites created 2012 2011
Freedom of Law on Access to Law on Access to Information held by
Information Laws Information held by Federal State Bodies and Local Self-
Government (12527/2011) government bodies on the Kyrgyz
Republic (2006)
Open Budget Index 73 20
(100 is
maximum-best)
Global 0.6167 0.4879
E-government Index
(100 is maximum-
best) (%)
Number of internet 49.8 21.7
users (per 100
persons)
Corruption 42 24
Perception Index
(100 maximum-best)
Mobile phones (per 125 125
100 persons)
Per capita GDP 11,340 1160
(US$)
Adult literacy rate 90 99
(%)
Human 0.730 0.622
Development Index
Sources: The World Bank (http://data.worldbank.org/indicator/IT.NET.USER.P2), International
Budget Partnership, Right to Info (http://www.right2info.org/laws#section-19); Transparency
International
Following Dubé and Paré (2003) we provide a detailed description of the research
context in order to ensure the credibility and generalizability of the research results.
Analysis
Strategic Analysis of Transparency Conditions in Kyrgyzstan
After transitioning to democracy in the end of the 1990s, former Soviet republics
adopted various legal mechanisms to advance information access and its dissemina-
tion through mass media (Ramkumar and Petkova 2007; Zaharchenko 2009).
However, the access to information has remained a “paper” provision and has not
been fully enforced. As a result, a lack of government accountability has continued
to plague Kyrgyzstan. In 2010 the country ranked 164 out of 178 in the Transparency
International Corruption Perception Index (CPI). Corruption has been a daily factor
in the lives of the majority of the population. Its reach has extended to all citizens
within the state and it affected the entire public service (AETS Consortium 2011),
leading to antigovernment protests in 2005 (Shukuralieva 2012; UNDP 2011). The
newly elected government in 2010 adopted several initiatives to advance transparent
and open governance in such areas as energy and mining as well as public finance.
In 2011, a President’s Decree was signed which required the active participation of
civil society in monitoring the management and control of financial flows in the
energy sector (Otunbaeva 2011).
Budget transparency has become a priority of the finance ministry. In 2011, it
introduced a simplified procedure to design the national budget. The ministry also
emphasized transparency in public procurement and transitioning to a two-tier bud-
get system, which was expected to strengthen local governments and improve trans-
parency of vertical public resources distribution and tax collection (Tulundieva
2011). Finally, several constitutional amendments were adopted, according to which
the opposition party was entitled to head the budget committee within the parliament.
These reforms made the annual monitoring and review of the national budget more
competitive.
Since 2000, international agencies have implemented several initiatives to
involve civil society in the budget process (Toktakunov 2007). With the support of
the office of the Open Society Foundation, for example, key events during the bud-
get process have been broadcasted on national television and radio. The UK govern-
ment has also supported the establishment of budget transparency alliances between
NGOs and the Kyrgyzstan government (Toktakunov 2007). In 2012, the United
Nations Development Program (UNDP) in Kyrgyzstan presented a new project
called Support to National Budget Transparency budgeted at one million dollars.
The main purpose of the project was to make budget information accessible.
Since 2010 the newly elected government has used the narrative of “transparent
government” to differentiate itself from the previous administration. A budget por-
tal was developed as a part of an effort to foster a “culture of transparency” within
the finance ministry, which historically had a reputation of being less open given the
former Communist government tradition.
Using Meijer’s strategic analysis (Meijer 2013), we identified that both the govern-
ment and international donors were the key factors advancing and shaping the trans-
parency movement in public finance, including online transparency in Kyrgyzstan.
Implementation of Open Data Practices in Kyrgyzstan

and Institutional Complexities
Kyrgyzstan has inherited many features of the Soviet public administration, includ-
ing an opaqueness in managing public finances. In 2011, Kyrgyzstan received a
score of 20 out of a possible 100 on budget transparency by the Open Budget Index
Report, placing it among countries with scant or no information on budgeting. In
recent decades the country has implemented many tools to improve transparency,
including budget hearings at the local and national levels. Among several instru-
ments, the creation of web portals on the public budget stands out as the most prom-
ising in addressing the lack of budget transparency.
The Kyrgyz Finance Ministry initiated the Open Data Portal (www.budget.
okmet.kg), which is an automated system tracking the revenues and expenditures of
the Kyrgyz national and local governments and is managed by the state-owned com-
pany InfoSystem. Founded on April 7, 2009 by a Kyrgyz Government Decree,
InfoSystem is a small entity, which operates within the Kyrgyz Finance Ministry.
The overall purpose of the entity is to ensure the successful modernization of the
public treasury and finances in Kyrgyzstan. Other projects currently administered
by InfoSystem include the state portal of electronic government procurement, uni-
fied taxpayer’s card, the economic map portal, and the single population register.
InfoSystem operates with a staff totaling less than a hundred people.
Launched in 2012, the Open Budget Portal’s goal is the proactive disclosure of
budget information online. Along with budget information, it displays regularly
updated information on the internal and external debt of Kyrgyzstan. More than
70 % of portal users are located in Kyrgyzstan. Other users originate from the
neighboring countries (Interviewee K1). It is noted that the daily traffic of the portal
includes 300–400 unique users. Financial data is equally available for residents,
NGOs, and other interested parties. The most frequent users of the budget portal are
journalists, NGOs, government administrators, the national security personnel, and
some residents (Interviewee K1, K2). The portal provides financial information in
various formats and is free of charge. The information is accessible in multiple lan-
guages and the citizens’ budget and other budget presentations are regularly updated.
In 2013, the World Bank conducted a survey on the disclosure of open budget
data initiatives in 198 countries. According to its evaluation, it placed Kyrgyzstan in
a group of countries (including Canada) with acceptable levels of financial data. The
evaluators noted that in the category of low-income countries, Kyrgyzstan was the
only country with a dynamic and regularly updated website (Dener and Min 2013).
Although the country has advanced its open budget infrastructure, it needs an
educated citizenry to utilize proactively disclosed information (Interviewee K3). To
evaluate the level of residents’ budget literacy, in 2013 the International Legal
Foundation within the Soros Foundation conducted a survey in the northern region of
Kyrgyzstan to assess the level of knowledge and understanding of the budget process
among residents, public employees, and community organizations. The survey found
that 80 % of respondents were aware of budget processes, but that public finance
knowledge was limited. Members of local legislative councils were also inadequately
educated or trained with respect to public financial management (Kalkanov 2013).
The interviewees noted several challenges preventing the effective implementa-
tion and sustainability of online budget disclosures. The first problem involved a
lack of infrastructural development within the country. This issue is particularly
acute in rural parts of the country. The second problem relates to a meager level of
citizens’ interest in budget issues: “Given the current difficult socioeconomic envi-
ronment in the country, this type of technological advancement is of limited interest
for many people. Many users of our online resources are based in urban centers
(Interviewee K2).”
As in other low-income countries, the majority of residents of Kyrgyzstan have
an inadequate understanding of budgeting and skills for requesting and accessing
budget information online. Therefore, it is imperative that the government and other
interested partners encourage other forms of citizen involvement activities that will
eventually lead to wider information sharing on citizen rights and opportunities
(interviewee K2).
Limited internet access is another problem and the regional internet penetration
in Kyrgyzstan varies from 20.0 to 39.8 %, with the highest number of users concen-
trated in major urban areas. Actual internet subscriptions remained relatively low in
2011 (less than 2 % penetration), with the local population making extensive use of
public internet access areas, such as cybercafés (Research and Markets 2013).
In sum, using Meijer (2013) we identified several challenges or institutional
complexities that prevent the sustainability of transparency through online budget
disclosure in Kyrgyzstan. These complexities include inadequate infrastructural
development, such as internet access, especially in the rural areas, and the design-
reality gap, which implies that the current format of online budget information is
still technical and requires additional skills among residents to utilize information.
Strategic Analysis of Transparency in Brazil
Brazil has a long history of engaging citizens in decision making and has been at the
forefront in developing participatory budgetary decision making (Goldfrank and
Schneider 2006; Sintomer et al. 2008). During the recent decade, citizen engagement
with the internet has grown in Brazil. In 2010 the newly elected president, Dilma
Rousseff, made government transparency an important initiative of her administra-
tion. On assuming her position, President Rousseff facilitated the adoption of the
Freedom of Information Law and the launch of the Portal of Open Data. The main
impetus given by the government was a wish to change the culture of secrecy, which
dominated Brazilian public institutions as a result of a military regime that lasted two
decades, from 1964 to 1984. Further, President Rousseff created the Truth
Commission with the goal of disclosing information on crimes committed by the
military regime. These legal instruments were expected to facilitate access to public
documents, including classified military documents (Michener 2012).
The right to information access is enshrined in the Brazilian Federal Constitution
and the Universal Declaration of Human Rights. In order to enforce those rights, on
November 18, 2011, the Law on Access to Public Information (Law 12.527/2011)
was sanctioned, regulating access to data and information held by the Brazilian
government. This law states that information requested by citizens must follow
technological criteria aligned with the rules of open data and the international
democratization of information for enhancing citizen engagement. Brazil also
adopted two federal decrees (Decree 5.482/2005 and Ministerial Decree 140/2006),
which made it mandatory for all levels of government and public agencies to pub-
lish relevant and timely information on their web pages. The adoption of these
decrees was due to the pressure from activist groups and Brazil’s commitment to the
International Open Government project.
Due to the leading role of Brazil in global transparency initiatives, the country’s
success in these projects became a political necessity. A transparent budgeting pro-
cess is important not just for the public, but for the private sector, which has gained
influence in the policymaking process.
Brazil is the 12th best performing country among 100 nations assessed with
respect to budget openness (International Budget Partnership 2012). But, it scored
74th out of 100 in 2006 and 73 in 2012, which implies that in the last several years
its progress in budget openness has remained virtually unchanged.
The use of the existing open budget portal intensified during the time of popular
protests in 2013. According to the analysis of the Open Data Portal conducted from
August to September 2013—the period in which the main protests took place
around the country—27,323 visits were made and 86,727 pages were visited in the
portal, with an average of 3.17 pages per visit. About 23 % of the visitors returned
to the portal during a period of 30 days.
In sum, the key factors that facilitated the advancement of transparency in Brazil
were the newly elected government and its commitment to the International Open
Government project. These events have shaped the design of transparency, which
has been promoted from the top-down, chiefly from online disclosures.
Implementation of Online Budget Portals in Brazil

and Institutional Complexities
Launched in 2012, the Brazilian open data portal (http://dados.gov.br/) offers open
and easily accessible information on spending and government investments, strati-
fied by level of government, areas, and public policies. The portal’s main purpose is
to present budgetary information in a user-friendly format, translated into an acces-

sible “citizen language.” The website does not require personal identification. Both
measures are intended to facilitate citizen oversight of the federal budget execution.
On the expense side, the website works as a catalog of other public institutions’
information on: (1) the direct spending of national government agencies; (2) all
transfers to states, municipalities, and projects; and (3) transfers to citizens benefit-
ing from social programs and special policies.
A team of three civil servants affiliated with the Ministry of Planning manages
the portal. They are information technology analysts responsible for a sector called
the National Infrastructure Open Data (INDA). Although institutionally the portal is
described as an instrument promoting the better use of data, the portal is not cur-
rently designed to interact or encourage a dialogue with citizens. The portal operates
within the framework of the three Laws of Open Government Data. These three laws
are stated as the following: “(1) If it cannot be indexed, it does not exist; (2) If it is
not available in open and machine-readable format, it cannot engage; (3) If a legal
framework does not allow it to be repurposed, it does not empower (Eaves 2009).”
Online budget disclosures require a certain level of infrastructural development.
In 2012, the estimated average internet usage in Brazil reached 49.8 % (World
2014). However, as in other transitional countries, the digital divide is still wide in
Brazil.
Some commentators express their doubts about the possibility of transparency in
the context of public budgets and they believe that fiscal transparency could produce
incentives for governments to provide inaccurate budget information. In fact, bud-
get credibility remains a significant challenge in many developing countries,
including Brazil (Dabla-Norris et al. 2010; Gollwitzer 2010; Kolstad and Wiig
2009). Furthermore, commentators question the quality of information released.
According to Vieira (2007), open data portals in Brazil were not initially able to
ensure transparency because the government posted disorganized information.
The Institute of Social and Economic Studies (INESC) conducted research in
which it created a budget cycle transparency index to investigate the level of open
budget progress in Brazilian state capitals. The analysis of the index revealed that
more than 70 % of the cities studied scored below 60 out of 100 (INESC 2011). The
average score was 47. They found that cities tended to make available the Enacted
Budget such as the Multiannual Plan Law (PPA), the Budgetary Guidelines Law
(LDO), and the Annual Budgetary Law (LOA). Information regarding the budget
execution was the least available (INESC 2011). As a result, citizens are expected
to have sufficient financial knowledge to discern whether money is being spent well
and whether the government works efficiently (Ferreira et al. 2013; Kolstad and
Wiig 2009; Vieira 2007).
Although there is a feedback mechanism on the open budget sites, the channel of
communication is not effective. As a result, lack of engagement may discourage lay
citizens to use the website.
In sum, we identified three institutional complexities that prevent sustainability
of transparency through portals in Brazil. These are the digital divide, the low qual-
ity of information displayed online, and the lack of training for citizens so that they
might better understand budgeting and computer technologies.
Comparative Analysis and Discussion
Although differing in per capita income and other socioeconomic features, Brazil
and Kyrgyzstan are transitional countries with a large marginalized population.
Both countries have adopted Freedom of Information Laws and established demo-
cratically elected governments. The rate of internet penetration is also accelerating
in both countries. Meijer’s approach in identifying strategic complexities has helped
to identify and separate key factors that formed transparency policies in both coun-
tries as well as how policies were presented. We find that governments in both
countries implemented budget transparencies online and in a top-down manner, as
a response to demands from donors and as a way of improving international status
in global completion for budget transparency.
Also, Brazil and Kyrgyzstan had a series of popular protests triggered by inade-
quate allocations in public transportation and a lack of oversight in public utilities.
The events forced governments in both countries to accelerate, intentionally or not,
the process of transparency in budgeting and proactive information disclosure using
portals (please refer to Table 3 for details of a comparative analysis).
Using Meijer’s approach, our analysis of website implementation demonstrates
five similar institutional complexities that both countries face in promoting and
Table 3 Comparative analysis of two cases

Brazil Kyrgyzstan
Origins ● Newly elected government ● Popular protests in 2010
with the focus on transparency
● Membership in the Open ● Newly elected pro-transparency
Government Partnership President in 2010
Transparency ● Top-down implementation of ● Transforming information into
narratives and portals without the focus on a user-friendly format is a
strategic availability of user-friendly challenge
complexities information
● Competition for a better status ● Competition for a better rating
within the Open Government in the annual Open Budget
Partnership project Index report
● Top-down implementation of
budget transparency
Institutional ● Digital divide ● Digital divide
complexities ● The quality of data released ● Limited budget trainings
among residents
● Limited budget trainings ● Limited impact on improving
among residents citizens’ perception of budget
● Limited impact on improving transparency. Budget
citizens’ perception of budget transparency is beneficial for
transparency. Budget those who are better equipped
transparency is beneficial for to understand financial
those who are better equipped to documents
understand financial document
sustaining budget transparency through portals. These are a digital divide, income
disparity, a low quality in the information displayed, a lack of financial literacy
among residents, and a design-reality gap. Among all these challenges, the digital
divide stands out as the factor that most intensifies inequities in political decision
making (Cordella and Bonina 2012).
Given the existing differences in terms of information infrastructure access in
developing countries, some argue that electronic budget information could poten-
tially become biased, favoring educated and urban residents (Basu 2004: 177).
Therefore, theorists express concerns about the true impact of online information
disclosure through website portals.
In addition, commentators state that a well-educated society is crucial for sus-
taining fiscal transparency (Premchand 2001; Robinson 2006; De Renzio and
Krafchik 2007). Our interviews with administrators in Brazil and Kyrgyzstan reveal
the importance of budget literacy, because most of the available information requires
public finance knowledge—due to the fact that it is presented as “a catalog of raw
data.” Consequently, online budget disclosure still remains a guide to improve trans-
parency mechanisms rather than as an effective tool for citizens to take part in.
According to Alves and Heller (2013), this is also due to the top-down manner of
open data implementation. It promotes openness without authentic or sustainable
transparency and accountability.
There is a gap between financial resources and technological capacities across
state and municipal levels, which prevents the increase of budget information dis-
closures. According to INESC (2011), the federal government in Brazil should pro-
vide technical and financial assistance to municipalities to help them meet the legal
requirements for posting budget information online.
The creation of the portal has influenced institutional rules of transparency and
the transparency model adopted in Brazil. For example, the online budget informa-
tion on public spending on the World Cup and the Olympic Games was used by the
Brazilian local press, influencing an upsurge in protests in Brazil during 2013. The
news media divulged Olympic Game expenditures, contrasting them to mediocre
resource allocations for education, public transportation, and health. Against this
background, open budget data in Brazil becomes a double-edged sword. By promot-
ing transparency, the government can improve its image and engage citizens, but it
can also increase discontentment if government expenditure policies do not serve
the needs of citizens.
Administrators recognize the limitations in the current financial data. For exam-
ple it was stated that, “the displayed information in the raw material needs tools and
methods to transform it into user-friendly information. Thus, the profile of the main
user demands knowledge and skills to prepare and perform analysis, and use it.”
(Interviewee #B2).
Portals in Brazil and Kyrgyzstan provide a dense set of information, but just
releasing it to the public is not an end in itself. Open data needs to be simple and free
from manipulation, otherwise it will not promote sustainable transparency as
desired. This may also lead to openness without genuine transparency.
Developing the culture of data management through public administration

involves such essential goals as sustaining the commitment to publish, continually
updating published information, and presenting it in a user-friendly format. Pasquier
and Villeneuve (2007) noted that on numerous occasions public organizations were
willing to facilitate more transparency, but they were poorly equipped to make it
happen. For example, InfoSystem in Kyrgyzstan carried out several projects to illus-
trate budgetary dilemmas in infographic formats. But these were delivered in a lim-
ited number and required continuous maintenance. Cross-national information
provided through performance indicators could potentially instigate broader discus-
sion and greater citizen participation across the country.
Problems in the disclosure of unusable data are not as clear-cut in Kyrgyzstan as
they are in Brazil. In the case of Kyrgyzstan, the pressure of donor agencies as well
as the publicity surrounding the International Budgetary Partnership ratings have
had some positive impact and improved the quality of disclosure over the years. The
government in Brazil is also watchful of its Open Budget rating. Brazil has a vested
interest in promoting online budget transparency due to its leading role in the Open
Government Partnership Project.
Conclusion
The government-led Open Budget Portals were launched in 2012 in Brazil and in
2011 in Kyrgyzstan, but both countries are still facing numerous challenges in using
budget websites effectively. The external evaluation of the portal in Kyrgyzstan
demonstrated a sufficient level of public finance information disclosure, placing it
alongside such countries as Canada. However, the level of budget literacy and lim-
ited internet access among residents have resulted in inadequate use of the initia-
tives in Brazil and Kyrgyzstan, leading to questions regarding the sustainability of
budget transparency in both countries through online portals. On the other hand,
despite potential concerns over technological divides, some assert that developing
IT infrastructure is becoming more accessible for low-income families (Basu 2004).
Policy makers often focus on the technical dimensions of information technol-
ogy, but information itself should be treated critically. The quality of information
goes beyond simple accuracy, and includes things such as believability, objectivity,
and the reputation of sources (Helbig et al. 2012). Therefore, in addition to publish-
ing the budget through budget portals, it is imperative that governments make bud-
get information more understandable and usable for lay citizens. As the situation
currently stands, only wealthy and educated urban residents in both countries use
the open budget portals. In 2011, Lampe and colleagues evaluated the government-
run website www.AdvanceMichagan.gov, which was designed to collect citizen
feedback on services in Michigan communities. The authors found that contrary to
expectations, the website did not lead to active interaction and became underuti-
lized. As in our study, numerous barriers were noted, not limited to difficulty of use,
software design and lack of literacy among residents.
The context in which information is presented and its understandability are

important. In order to ensure the sense of transparency among the majority of the
population, budget information disclosures in developing countries should not be
promoted only electronically. Proactive budget information sharing should be com-
plementary to continuous face-to-face citizen engagement, citizen training in bud-
geting and technologies, and other budget literacy projects. This could lead to better
use of the portal by the majority.
Several positive externalities should be noted. The requirement for continuous
financial reporting had some positive impact in the form of improved managerial
and technological capabilities in finance departments at both local and federal levels
of government. Online budgeting also strengthened journalistic reporting on bud-
geting. In the case of Kyrgyzstan, it produced a separate group of journalists who
now specialize in reporting on public finances. Finally, portals played a critical role
during antigovernment demonstrations in both countries through their wide use by
the media. In other words, ICT has enabled new social capital and knowledge pro-
duction, which are critical for the democratic development of countries (Dawoody
2011).
As Meijer (2013) notes, transparency is a dynamic phenomenon amendable by
various forces. This argument is particularly relevant for developing countries, as
they have not reached a certain level of institutional stability. The existence of third
parties, such as the Open Budget Partnership positively influences governments’
efforts to improve budget transparency. Global open budget indicators encourage
regional competition among countries and facilitate public discussions on these
topics.
As with any study, this research has several limitations. We evaluated two budget
portals in two countries, which makes the findings less applicable generally. Second,
any future interviews conducted in this area should be wider and include commu-
nity members who use online portals frequently. Third, an additional set of inter-
view questions specifically focusing on transparency should evaluate the perception
of openness and quality of information experienced by portal users.
Appendix: Interview Questions
Causes
Could you please tell me who you are, what your role is in the web portal project,
and how long you have been working on it?
Could you please tell me about the history of this project, how it began, and what
motivated the ministry to launch the open data portal?
How does the financing for the project work? Who finances the project? How
many people are involved?
Current Process
Do you keep track of who and how many people use your web portal?
How would you characterize the primary users of your portal? Is it designed
primarily for the use of NGOs, administrators, or residents?
Do you have any statistics on users that you could share with us (average user,
etc.)?
How do you work with other ministries and local governments to distribute the
information on the portal?
Challenges
What are some challenges in promoting this portal?

How do residents perceive the portal? Do you think the average person living in
this country is aware of the portal’s existence?
How often do you update the portal and how useful do you think the portal is for
average residents?
Do you think average residents understand how to use the portal? Could you
elaborate on this?
How expensive is it to maintain the portal and what are future plans for it?
References
Aladwani A (2013) A cross-cultural comparison of Kuwaiti and British citizens’ views of

E-government interface quality. Gov Inf Q 30(1):74–86
Alves JA, Heller P (2013) Accountability from the top down? Brazil’s advances in budget transpar-
ency despite a lack of popular mobilization. In: Khagram S, Fung A, De Renzio P (eds) Open
budgets: the political economy of transparency, participation, and accountability. Brookings
Institution, Washington, pp 76–104
Basiago AD (1995) Methods of defining ‘sustainability.’. Sustain Dev 3(3):109–119
Bastida F, Benito B (2007) Central government budget practices and transparency: an international
comparison. Public Adm 85:667–716
Basu S (2004) E-government and developing countries: an overview. Int Rev Law Comput Technol
18(1):109–132. doi:10.1080/13600860410001674779
Carstens L (2010) Defining, inspiring, and implementing sustainability. Nat Civic Rev 99(3):11–16
CGU (2012) The Comptroller General. In Portuguese. http://www.cgu.gov.br/imprensa/
Noticias/2012/noticia10212.asp. Accessed 23 Nov 2013
AETS Consortium (2011) Support to preparation of new Rule of Law action in the Kyrgyz Republic.
http://eeas.europa.eu/delegations/kyrgyzstan/documents/press_corner/news2012/background_
note_on_transparency_an d_accountability_december_9_2011_qa_en.pdf. Accessed 2 May 2013
Cordella A, Bonina CM (2012) A public value perspective for ICT enabled public sector reforms:
a theoretical reflection. Gov Inf Q 29(4):512–520
Cuillier D, Piotrowski SJ (2009) Internet information-seeking and its relation to support for access
to government records. Gov Inf Q 26(3):441–449
Portal da Transparencia (2012) Portal of transparency of the federal government. In Portuguese.

http://www.portaldatransparencia.gov.br/servidores/. Accessed 11 Nov 2013
Dabla-Norris E, Allen R, Zanna L, Prakash T, Kvintradze E, Lledo V, Yackovlev I, Gollwitzer S
(2010) Budget institutions and fiscal performance in low-income countries’. IMF working
paper 10/80. International Monetary Fund, Washington
Davis J (1998) Access to and transmission of information: position of the media. In: Deckmyn V,
Thomson I (eds) Openness and transparency in the European Union. European Institute of
Public Administration, Maastricht, pp 121–126
Dawoody AR (2011) The global participant-observer emergence, challenges and opportunities.
Innov J 16(1):1–20
De Renzio P, Krafchik W (2007) Lessons from the field: the impact of civil society budget analysis
and advocacy in six countries’ practitioners’ guide. International Budget Project, Washington
Dener C, Min C-Y (2013) Financial management information systems and budget transparency:
do governments report on where the money goes? World Bank, Washington
Dingwerth K, Eichinger M (2010) Tamed transparency: how information disclosure under the
global reporting initiative fails to empower. Glob Environ Polit 10(3):74–96, August
Dubé L, Paré G (2003) Rigor in information systems positivist case research: current practices,
trends, and recommendations. MIS Q 27(4):597–636
Eaves D (2009) The three laws of open government data. http://eaves.ca/2009/09/30/three-law-of-
open-government-data/. Accessed 10 Sept 2014
Ferreira MAM, Olveira A, Silva AAP (2013) Monitoring of public policies in Brazil: a tool for
promoting transparency and accountability policy. In: Fourth congress GIGAPP—IUIOG,
Madrid, Spain. In Portuguese
Goldfrank B, Schneider A (2006) Competitive Institution building: the PT and participatory bud-
geting in Rio Grande do Sul. Lat Am Polit Soc 48:1–31
Gollwitzer S (2010) Budget institutions and fiscal performance in Africa. J Afr Econ
20(1):111–152
Grimmelikhuijsen SG, Welch EW (2012) Developing and testing a theoretical framework for
computer-mediated transparency of local governments. Public Adm Rev 72(4):562–571
Gurstein M (2011) Open data: empowering the empowered or effective data use for everyone?
First Monday 16(2) February. http://firstmonday.org/ojs/index.php/fm/article/view/3316/2764.
Accessed 12 June 2013
Hajer M (1993) Discourse coalitions and the institutionalization of practice: the case of acid rain
in Britain’. In: Fischer F, Forester J (eds) The argumentative turn in policy analysis and plan-
ning. Duke University Press, Durham, pp 43–67
Heeks R (2003) Most e-government-for-development projects fail: how can risks be reduced?
I-Government working paper series. University of Manchester, Institute for Development
Policy and Management, Manchester
Helbig N, Nakashima M, Dawes S (2012) Understanding the value and limits of government infor-
mation in policy informatics: a preliminary exploration. in: Proceedings of the 13th annual
international conference on digital government research, 4–7 June
Hordijk M (2009) Peru’s participatory budgeting: configurations of power, opportunities for
change. Open Urban Stud J 2:43–55
International Budget Partnership (2012) Open budget survey. International Budget Partnership,
Washington
Kalkanov K (2013) Nurturing the culture of openness in the budgetary process. E-Newsletter. http://
soros.kg/wp-content/uploads/2013/05/E-newsletter_-1_Soros_ru.pdf. Accessed 4 July 2013
Khagram S, Fung A, De Renzio P (2013) Open budgets: the political economy of transparency,
participation, and accountability. Brookings Institution, Washington
Koliba C, Zia A, Lee BH (2011) Governance informatics: managing the performance of inter-
organizational governance networks. Innov J 16(1):1–26
Kolstad I, Wiig A (2009) Is transparency the key to reducing corruption in resource-rich countries?
World Dev 37(3):521–532
Krishnamuthy R, Desouza K, Johnston E, Bhagwatwar A (2014) A glimpse into policy informat-

ics: the case of participatory platforms that generate synthetic empathy. Commun Assoc Inf
Syst 33(21):365–380
Lampe C, LaRose R, Steinfield C, DeMaagd K (2011) Inherent barriers to the use of social media
for public policy informatics. Innov J 16(1):1–17
Research and Markets (2013) Kyrgyzstan—telecoms, mobile, Internet, and forecast. Taylors Lane,
Dublin. http://www.researchandmarkets.com/research/4d0280/kyrgyzstan_telec. Accessed 30
July 2013
Meijer A (2013) Understanding the complex dynamics of transparency. Public Adm Rev
73(3):429–439
Meijer A (2014) Transparency. In: Bovens M, Goodin RE, Schillemans T (eds) The Oxford hand-
book of public accountability. Oxford University Press, Oxford, pp 507–524
Michener G (2012) Brazil’s open government shock treatment. Personal Democracy Media. http://
techpresident.com/news/wegov/22476/brazils-open-government-shock-treatment. Accessed
27 June 2014
OECD (2011) Proactive disclosure of information. In: Government at a glance 2011. OECD, Paris,
pp 142–144
Otunbaeva R (2011) Speech by the President of the Kyrgyz Republic. In: 5th EITI global confer-
ence. OECD Conference Center, Paris
Overseas Development Institute (ODI) (2007) Budget monitoring and policy influence. Briefing
paper 16, London
Pasquier M, Villeneuve JP (2007) Organizational barriers to transparency: a typology and analysis
or organizational behaviour tending to prevent or restrict access to information. Int Rev Adm
Sci 73(1):147–162
Pérez CC, Bolívar MPR, Hernández AML (2008) E-government process and incentives for online
public financial information. Online Inf Rev 32(3):379–400
Prat A (2005) The wrong kind of transparency. Am Econ Rev 95(3):862–877
Premchand A (2001) Fiscal transparency and accountability, idea and reality. Paper prepared for
the workshop on Financial Management and Accountability, Rome
Ramkumar V, Petkova E (2007) Transparency and environmental governance. In: Florini A (ed)
The right to know: transparency for an open world. Columbia University Press, New York,
pp 279–308
Ríos AM, Benito B, Bastida F (2013) Determinants of central government budget disclosure: an
international comparative analysis. J Comp Policy Anal 15(3):235–254
Robinson M (2006) Budget analysis and policy advocacy: the role of non-governmental public
action. IDS working paper 279. Institute of Development Studies, Brighton
Santos BDS (1998) Participatory budgeting in Porto Alegre: toward a redistributive democracy.
Polit Soc 26(4):461–519
Shukuralieva N (2012) The family in power: a new past for an old country. J Cent Asia Caucasian
Stud 7(13):30–56
Sintomer Y, Herzberg C, Rocke A (2008) Participatory budgeting in Europe: potentials and chal-
lenges. Int J Urban Reg Res 32(1):164–178
The Institute of Social and Economic Studies (INESC) (2011) Measuring subnational budget
transparency, participation, and accountability: Brazil. International Budget Partnership,
Washington. http://internationalbudget.org/wp-content/uploads/Measuring-Subnational-
Budget-Transparency-Participation-and-Accountability-Brazil_final.pdf. Accessed 29 July
2013
Toktakunov N (2007) Promoting transparency in the budget process in the Kyrgyz Republic.
E-newsletter. International Budget Partnership. http://internationalbudget.org/wp-content/
uploads/newsletter40.pdf. Accessed 28 Sept 2013
Tulundieva N (2011) The concept of managing the budget of the country. In Russian. Academy of
Management under the President of Kyrgyzstan, Bishkek
UNDP (2011) Kyrgyzstan’s energy sector: a poverty and social impact assessment. http://km.undp.
sk/uploads/public1/files/vulnerability/Senior%20Economist%20Web%20site/PSIA_Energy_
Kyrgyzstan.pdf. Accessed 22 Oct 2013
Vieira VLR (2007) Transparency portal awarded in 12 innovations contest in Federal Public
Management. In Portuguese. ENAP, Brasília
Wachhaus TA (2011) Governance as a framework to support informatics. Innov J 16(1):1–14
Wagle S, Shah P (2001) Participation in public expenditure systems. An issue paper. World Bank,
Washington, DC
World Bank (2014) Internet users. http://data.worldbank.org/indicator/IT.NET.USER.P2/coun-
tries. Accessed 23 Sept 2014
Zaharchenko T (2009) On the way to transparency: a comparative study on post-Soviet states and
the Aarhus convention. Woodrow Wilson International Center for Scholars, Washington
On Sustaining Sustainability: The Case
of Implementing Decisions Based on Policies
for a Sustainable Future via Tablets in a Board
of a Swedish Housing Corporation
Jenny Eriksson Lundström and Mats Edenius
Abstract Catering for sustainability concerns and enterprise mobility for adminis-
trations, handheld artifacts of Information and Communication Technology (e.g.,
tablets and smartphones) are making their way into organizational administration.
Often such change is the result of decisions aimed at implementing policies for a
more sustainable future, intending to support a shift from paper to digital. For this
reason, a deeper understanding of tablets as the means to enable robust and efficient
implementation of policies for sustainability is essential. This chapter examines
how a decision made by one of the largest municipal boards in Sweden on reducing
the use of paper in the municipality was implemented as an introduction of tablets
in one of the municipal sub-boards, the board of one of the municipally owned hous-
ing corporations (HC), and how the decision prompted changes in the board work
setup, raised issues concerning security and privacy of data while yet failing to
implement most of the targeted sustainability policies. We adopt the Belief-Action-
Outcome framework of Melville (MIS Q 31: 1–21, 2010) as our theoretical lens on
how the actual use of the tablets emanates from the social practices of the board, and
how the implementation of the sustainability policy of the municipality thus affected
the alignment of the board work and the organization’s policies on sustainability.
The empirical material consists of interviews and observations in the board of one
of the municipally owned housing corporations. The contribution includes an
account as to how a decision to implement policies for sustainability via new tech-
nology serve as a catalyst to establish and reproduce new setups and practices.
However, in order to adopt the new technology, it may also partly or completely
reinvent well-established practices without aligning it to the intended policies. Key
highlights are the importance for decision makers to consider a broader context and
to see the complexity of the practices of the organization, and the role this plays for
the implications of making policies on organizational changes sustainable.
J.E. Lundström (*) • M. Edenius

Department of Informatics and Media, Uppsala University, Uppsala, Sweden
e-mail: jenny.eriksson@im.uu.se

DOI 10.1007/978-3-319-25439-5_8
158 J.E. Lundström and M. Edenius
Introduction
Several manifestations of demands for a more sustainable world1 are visible in

green movements in both political (e.g., WCED 1987; The 1997 Kyoto protocol of
United Nations Framework Convention for Climate Change 2014) and individual
initiatives of grassroots (The Paperless Project retrieved March 13, 2014).
Contributing to a more sustainable world, information technology (IT) has become
ubiquitous in organizations that are trying to assume responsibility. For this reason,
mobility in the form of handheld devices such as tablets is rapidly introduced into
everyday practices in the administration of organizations. Often such change is the result
of decisions made by the top management of the organization who aims at enforc-
ing/implementing organizational policies that support ecological (and economical)
sustainability via a shift from paper to digital throughout the organization.
Unfortunately, this transformation is not a straightforward call. It is extensively
accepted that public policies, despite good intentions, often fail. Several explana-
tions have been put forward both by organizational theorists (Powell and di Maggio
1991) and Economists (Friedman 1982) but also by Information Systems theorists.
In addition, the notion of sustainability itself has been debated for focusing too nar-
rowly on environmental factors such as CO2-emission (Elkington 1997). For a more
comprehensive classification for measuring governmental and organizational sus-
tainability impact (cf. GRI-index, retrieved April 5 2014) we turn to the triple bot-
tom line (Elkington 1997) that includes economic, social as well as ecological
factors in how to make an organization sustainable. In conclusion, catering for
green initiatives by procuring information systems that are thought to implement
sustainability policies is usually not enough to make an organization’s overall sus-
tainability performance aligned to the sustainability policy.
The core issue of this book is to identify emerging trends, development, and
challenges of using IT in the process of informing, developing, and evaluating pol-
icy on sustainability. Hence the ambition of this chapter is to provide insights to the
prerequisites of this question by problematize IT itself. In this way we address a
foundational aspect of the challenges of using information technology in the pro-
cess of informing, developing, and evaluating policy on sustainability. What is
needed is an understanding of what various types of IT do to our social practices.
With this in place, the great potential of IT for informing, developing, and evaluat-
ing policy on sustainability may be more robustly realized.
As researched in the field of Information Systems, the implications of technol-
ogy on social practices are not necessarily straightforward (cf., e.g., Gherardi 2010;
Orlikowski and Scott 2008). In this vein, Vaast and Walsham (2005) argue that there
is a need of empirical investigations on how organized human activities manifest
themselves within new IT-environments/IT-implementations and what such change
1
A comprehensive definition of sustainability is presented by Brundtland (WCED 1987): “…
development that meets the needs of the present without compromising the ability of future gen-
erations to meet their own needs” (WCED 1987, p. 43).
On Sustaining Sustainability… 159
may involve. Given the dynamic nature of organizations and technologies, the
ongoing transformation of practices, i.e., organized activities involving IT-artifacts,
establishes whether IT can be useful in informing, developing, and evaluating
policies.
Summarizing research in sustainability Information Systems published in lead-
ing Information Systems (IS) outlets, in line with Melville (2010) and Bengtsson
and Ågerfalk (2011) concluded that there is a lack of studies of sustainability from
a general Information Systems (IS)-perspective. Similarly for introducing mobility
in organizations Sørensen and Landau (2013) argue that the implications of such
IS-related change are yet to be understood from a theoretical stance. Therefore,
despite the potential of IT, mobility and handheld devices for the facilitation of
informing, developing, and evaluating policies, we have not much knowledge about
how the outcome of IT and policies while implemented in administrative practices
without taking into account what do does, i.e., how introducing and using tablets in
the board work are to be aligned to already well-established practices as well as to
implement the intended policies. For this reason, further empirical investigations
based on reasonable theoretical assumptions are needed to extend our knowledge of
the implications on ICT-based sustainability performance in organizations.
Our case organization is a board of a publicly owned housing corporation in
Sweden (below HC). Together with 15 other municipally owned corporations, of
which eight concerns housing or real estate management, HC is one of the largest
municipally owned corporations. The board of HC is directly subordinate to the
municipal board. HC follows the municipality environmental program and thus, it
is committed to operating in an environmentally responsible manner and in a man-
ner that is consistent with sustainable development. Due to a decision to implement
policies for digital renewal via new technology such as tablets of the municipal
board in 2007 (as part of the Vision 2030—a World-Class City of the municipality
and its IT-program for digital renewal and green-IT), paper as the bearer of informa-
tion is replaced with tablets throughout the organization’s boards. Prior to this deci-
sion being carried out in HC, similar projects in sister organizations had been
conducted. Details of how the implementation was to be executed were largely left
to the organizations to decide. The choice of tablets was made based on existing
procurement contracts, and thus, neither the choice of hardware nor software was
made based on any requirements collected from the board of HC. A short introduc-
tion on how to use the tablet and software was given to the board members at the
first meeting after the board members had received their new tablets.
Our argumentation is based on the well-grounded BAO-framework of Melville
(2010). We study this particular decision of the municipal board to implement the
sustainability policy by the introduction of tablets to replace paper in the board
work, and how well the outcome is aligned to the sustainability policy of the munic-
ipality’s organizational structures and social systems. The organization, or rather
the activities in the organization, will be important entities to further analyze how IT
can be employed for policy implementation, as the use of IT for policies ultimately
depend on the organization and its activities. More specifically, we investigate how
board-members perceive of starting to use tablets in carrying out the activities of the
board and situated practices related to the implementation. We illuminate its impli-
cations for the existing setup and practices of the board. In the analysis we shed
light on the intimate relationship between belief and actions for promoting or
demoting sustainable outcomes. Furthermore, we argue that sustainability issues do
not lend themselves to be captured only as a snapshot, but by shedding light on dif-
ferent processes for how sustainability comes into being. In other words, how sus-
tainability becomes and more importantly, how it is to be sustained.
This line of inquiry is part of a larger study with the overall aim to take part of
the contemporary discourse about knowledge and understanding of change in social
practices. This perspective is also in line with the recommendations of Bengtsson
et al. (2012) who argue that “assuming responsibility for a sustainable future is a
social issue for which ecological impact is one example of social actions and deci-
sions (cf. Elkington 1997; Epstein and Roy 2001; van Marrewijk 2003; Salzmann
et al. 2005; Sharma and Henriques 2005)”.
The chapter is outlined as follows: In the following section we present the theo-
retical frame of our inquiry, the BAO-framework of Melville (2010), with a focus on
social and organizational outcomes of implementing policies of sustainability per-
formance in the board activities via IT. Then we present the method of the study,
including the research setting. The board meeting and its activities are described as
outcomes of beliefs and actions by presenting the perceptions of the board members
related to the implementation of the tablets and their impact of the actions on the
work of the board. This is followed by a discussion of the results on the alignment
of the decision of the municipal board to implement the sustainability policy and
different board members beliefs about using the tablets linked to a discussion of
principle about how the organization’s overall sustainability performance was
affected by IT. In the discussion we illustrate how the BAO-framework can be
extended with an analysis about what the introduction of the tablet “does” to the
activities and setup of the board work. By doing this we strike a blow for an analyti-
cal approach to study implementation of policies by means of IT, an approach that
entail processes about how policies for sustainability can be sustained. By doing
this we open for a further, and perhaps deeper, discussion about what both may
promote and demote how IT can be used in informing, developing, and evaluating
sustainability policies in an organization.
Board Work as Sustainability Performance of Organizations
Hill (1995) notices that there are few studies in the academic literature on how
boards conduct their work. Board meetings could be said to face three different, but
interrelated roles: First, the board members monitor executives; second, the board is
responsible to define, create, and implement a corporate strategic plan; third, the
board members represent the company and legitimize the business to its environ-
ment (Ruigrok et al. 2006). These different roles have been in focus for different
studies in the field with emphasis on the board meeting’s monitoring and strategic
role (Myers and Young 1997; McNulty and Pettigrew 1999; Ruigrok et al. 2006).
Within such more conventional approach, the analysis might be said to be based on
a meta-level, as, for example, the role of executive directors in the meetings, gender
equality, etc. (cf. Hill 1995).
For our case organization, the way the board members perceive enablers and bar-
riers to fulfilling the monitoring, strategic, operational as well as the legitimizing
role of the board directly have impact on the board work. As a result it affects eco-
nomic, social, and ecological activity in and for the municipality, the political par-
ties as well as the surrounding society and the inhabitants of the housing corporation,
making the mentioned recommendations of the literature (e.g., Bengtsson et al.
2012; Elkington 1997; Epstein and Roy 2001; van Marrewijk 2003; Salzmann et al.
2005; Sharma and Henriques 2005) to address sustainability as an outcome of social
actions and decisions well suited for our inquiry.
In this way, the activities of the board may be seen as catering for aspects of sus-
tainability of the organization in line with the triple bottom line and WCED recom-
mendations. As such impact may be discussed in terms of a normative scale to
illuminate its positive or negative nature (Dyllick and Hockerts 2002), the imple-
mentation of the decision of the municipal board to implement the sustainability
policy to introduce tablet to replace paper may be examined for its impact on how the
activities of the board was aligned to the sustainability policy of the municipality.
Analytical Lens
From this admittedly schematic presentation of board work as activities that could
promote or demote sustainability performance, we present the analytical lens of our
investigation. As previously stated, frameworks for the theoretical inquiry on IS and
sustainability are scarce, and to our knowledge no systematic approach for assess-
ing the sustainability impact of handheld devices on social and organizational levels
is presented. Consequently, in line with the WCED (1987) recommendations for
catering for a sustainable future, we apply an analytic lens that includes notions of
economic, ecological as well as social responsibilities in human actions in our
examination.
Based on Coleman (1986) and Melville (2010) presents the belief action out-
come (BAO)-framework (see Fig. 1) for the examination of the role of IT for a
variety of sustainability outcome types, including the social and organizational phe-
nomena that are causing individuals and organizations to adopt various systems for
energy management. Given the expressed intent of the model, it fits well with the
WCED definition of sustainability to which we adhere and the focus of our study.
In sum, the BAO framework provides a way of framing research questions intersecting
information systems and environmental sustainability in organizations, is compatible with
IS research diversity, and subsumes macro and micro perspectives found in the scholarly
and popular literature (Erdmann et al. 2004; Farrell and Oppenheim 2008; Romm 2002).
Melville (2010) p. 5.
Societal MACRO Behavior of

structure social system
4
1 5 3
Beliefs 2 Sustainability
about
actions
environment
MICRO
1'
5'
3'
Organizational 4' Behavior of

structure organization
MACRO
Fig. 1 The belief-action-outcome (BAO)-framework for the examination of the role of IT for a
variety of sustainability outcomes (Melville 2010)
As shown by Bengtsson and Ågerfalk (2011) any organizational change that is

implemented via an information system with the aim to accommodate a sustain-
ability initiative can cause changes in the sustainability performance of an organiza-
tion. Hence, we draw on the BAO-framework as our analytical lens for our empirical
material concerning how the organization’s overall sustainability performance may
be aligned to the sustainability policy of the municipality. It serves as the point of
departure for our discussion (however tentatively) about how it can be extended to
include a “becoming” perspective too.
For our purpose, to identify the impact of the decision of the municipal board to
implement the sustainability policy on organizational sustainability performance
and the nature of its impact on how the organization’s overall sustainability perfor-
mance may be aligned to the sustainability policy of the municipality, we address
the following associations of societal and organizational structures of Melville
(2010).
On the macro-to-micro level, comprising the formative influences of collections
of individuals such as organizations and societies on individual beliefs about the
environment, we find societal structures influencing individual beliefs about the
environment (1), and organizational structures influencing individual beliefs about
the environment (1′). Relevant on a micro-to-macro level, these beliefs manifest
themselves as actions (2), which in turn result in outcomes with impact on the per-
formance of the organization (3′) as well as the social system (3).
On a macro-to-macro level, groups of individuals such as organizations and soci-
eties, we find four types of associations: societal structure and behavior of social
systems (4), organizational structure and behavior of organization (4′), societal
structure and behavior of organization (5), and organizational structure and behav-
ior of social systems (5′). In this interpretation of the relationship of organizational
change, the beliefs and actions are expressed by individuals. However, as groups of
individuals form collective structures such as societies and organizations, also in
this, our, admittedly instrumental interpretation, we find a dual relationship con-
cerning the formation of beliefs and actions that excerpts influence on the individual
as well as the collective to which the individual belongs.
Melville’s theoretical framework (analytical lens) can of course be enacted in
different ways (see Melville 2010). We have chosen a quite instrumental way to put
it into practice. Our analysis has shed light on different statements of the members
of the board and some situated observations. The statements are interpreted as
beliefs guiding further thought and behavior of the members of the board. Well
aware of the difficulty of research in linking people’s perception (what they say) to
what they do and think (cf. Schuman and Johnson 1976) we put forward that the
respondents’ perception may be related firmly to action and outcomes.
Method
A single in-depth case study was adopted to obtain rich and naturalistic data. Hence,
we employ naturalistic observation as we study the subjects in their natural envi-
ronment. While frequently criticized for being dependent on a single case, this
approach is suitable, given the explorative character of the study and the impor-
tance of accumulation of theoretically grounded knowledge in this domain
(Sørensen and Landau 2013).
Empirical Setting
Housing Corporation in Sweden (HC) is a municipally owned housing corporation

in Sweden. Hence it is directly subordinate to the municipal board. It services one
of the largest municipalities in Sweden with publicly owned housing. In 2014,
49,000 tenants were living in flats provided and serviced by HC. The tenants are of
heterogeneous economic and social backgrounds. HC has 300 employees and a
yearly overturn of 1.9 billion SEK. HC follows the municipality’s environmental
programs. In this is committed to operating in an environmentally responsible man-
ner and in a manner that is consistent with sustainable development. The board
consists of seven regular members and seven deputy members, who all politically
appointed and represent their respective political parties. The political governance
of the municipality reflects the governance of the board. Usually, the board mem-
bers are very experienced in board work, as being appointed to this board is consid-
ered an important and prestigious political assignment. In addition, three official
representatives are elected into the board.
The main manifestations of the board work are the meetings and the preparations
of the meetings. The meeting is prepared by the CEO in close connection with the
chairman. The meeting agenda, which includes details of the date, time, venue, and
agenda items, and a board pack, containing minutes from the previous meeting,
reports, proposals, and the necessary documents that are needed for the discussion
at the meeting, are organized and it is traditionally sent out from the secretary of the
CEO with regular mail. Usually the secretary arranges the material in a chronologi-
cal order according to the agenda items. During the meeting the chairman chairs the
meeting according to the agenda and assigns the word to the members of the board
as they request it. Objections and comments are added to the minutes of the board
meeting. For certain issues on the agenda an official, the CEO or specially invited
speakers give presentations to the board. Sometimes an issue concerns sensitive
information, e.g., issues concerning tenants, economical or strategically important
issues. These documents are printed on papers that have a distinct green color, and
they are distributed at the meeting. The classified documents can either be left at the
table to be properly destroyed after the meeting or brought home by the member of
the board who then takes responsibility for ensuring that the data remains
uncompromised.
The decision of the municipal board to replace paper as the bearer of information
with tablets throughout the organizations boards was made in 2007 as part of imple-
menting the Vision 2030—a World-Class City of the municipality and its IT-policy
for digital renewal and green-IT. The policies state that IT should be seen as a tool
in the daily work of the municipality and used in a new and innovative way. Over
the next few years, the city committees and boards should make IT development in
the following five focus areas a priority:
1. Availability and service to citizens, businesses, and visitors.
2. Improved ability to compile, analyze, and reuse information.
3. Training and system development in order to improve and simplify the use of
existing IT support.
4. Collaboration and information sharing with other agencies and organizations.
5. Innovative business development using IT.
Of direct relevance to the use of IT of this study, i.e., the digitalization of board
packs and memoranda, the first, second, and fourth focus areas deal with IT for
implementing policies improving information, developing and evaluation of the
work of the organization. The third and fifth focus areas address the ability to master
the business support offered via IT in order to offer a high degree of availability, to
attract, recruit and retaining dedicated and competent employees with the intent to
increase the possibilities for business development, business innovation and effect
repatriation inherent in the offered IT-tools.
Prior to the introduction of tablets, the introduction of tablets in sister organiza-
tions of HC had been conducted. Some of the board members have also been part of
a board of a sister organization, and these board members hold firsthand experience
from these projects. The choice of tablets was made based on existing procurement
contracts, and hence, the choice of hardware and particular software was already
pre-made. Regardless of earlier experience or access to an existing tablet, all board

members received a new tablet, a preinstalled software package for accessing the
board pack, an email account and alias, and security codes. A short introduction on
how to use the tablet and software was given to the board members. Largely no
additional recommendations on how the implementation was to be executed accom-
panied the decision of the municipal board to implement the sustainability policy
via tablets. Moreover, not all codes needed for accessing the board pack were dis-
tributed with the tablet, which led to that the material couldn’t be readily accessed.
Also, at the first board meeting the tablets were collected for an update, which made
it difficult to use them during the board meeting.
Data Collection
In order to understand how the decision to implement policies for digital renewal
via new technology such as tablets was implemented and its consequences for the
board work, we developed a series of interview questions, and conducted in situ
interviews with the full board of HC. The regular members of the board are four
male and three females and three official representatives, two females and one male.
In addition, the board has seven deputy members, six males and one female. The
ages of the full board range from 25 to 70, with a median age of 57 and an average
age of 54. Regardless of role in the board, the members all are experienced com-
puter users in the sense that they are using or have been using computers for work
and for political assignments on a daily basis. Their earlier experiences of tablets
were very varying, ranging from no previous experience to having incorporated a
tablet into their everyday doings in leisure and work (i.e., both for the board activi-
ties as well as other work assignments unrelated to the board work).
The interviews ranged from 30 to 100 min. Each interview was digitally recorded
and later transcribed verbatim. In addition, notes were taken throughout each inter-
view. The interview questions were used as a guide for the conversation, rather than
as a strict question-and-answer tool. In this way, the interviewers were able to struc-
ture the conversation in a way that obtained the most relevant information about
how the respondent’s perceived the introduction of tablets. During the interview
with the various representatives, the discussions were substantially richer in content
than the following texts and summary depict. The perceptions of the board members
were our focus, but in order to gain a deeper understanding of their accounts, the
interviews were complemented with observations. To ensure trust in the board we
choose to conduct overt observations (cf., e.g., Creswell 2012; Adler and Adler
1994). To mitigate any effect the researcher had to the board members, triangulation
with practitioner-researcher observation by one of the board members was also
employed. In this way changes in the activities could be interpreted with more
confidence.
In Table 1 the methods for collecting the empirical material are presented. The
empirical material is collected during the period 2012–2014.
Table 1 Data generation methods

Method Description
Observation 3 years of Practitioner-Researcher participation in board meetings as a board
member prior to the introduction of the tablets, participant in the education
on the tablets, and ongoing
2 h of Complete-Observer participation at a board meeting approximately 12
months after the introduction of the tablets
Interviews 15 Semi-structured interviews of 30–100 min with 15 respondents. The
respondents consist of all regular board members and officials of the public
housing corporation
In parallel with the empirical fieldwork we also started to conduct a thematic

analysis (cf. Taylor and Bogdan 1998). Inspired by the technique of grounded theory
(Glaser and Strauss 1967; Strauss and Corbin 1998) we conducted an iterative textual
analysis of the interviews transcripts and field notes. This process was proceeded by
multiple readings and coding of the empirical material. Different data were identified
and put into different sub-themes of classified patterns in line with Constas (1992)
note that the interpretative approach should be considered as a “distinct point of
origination” related to our theoretical framework. Furthermore, by drawing on
Mellville’s BAO-framework (as a template for interpreting the empirical results)
of our study, we also strive for analytical generalization of the empirical material
(cf. Yin 2009).
Hence, the study is inspired by this way of regarding generalization and is
centered on a more intuitive, empirically grounded generalization that sees a harmo-
nious relationship between the reader’s experiences and the case study itself (Stake
1995). From such a standpoint, a case study is both the process of learning about the
case and the product of our learning. The intention of our case study has not primarily
been to solve the problem at stake, but to work with the situation that presents itself
in each case, to clarify and gain a better understanding (cf. Stake 1995; Hanson 1958).
Result
We present the results of the study organized according to the relations of individual
and collective beliefs and activities and collective outcomes for the organization and
societal structures of the BAO-framework (Melville 2010). Initially, the general per-
ceptions of the study are presented:
General Perceptions
Most respondents express that the impact of IT in general on the board work is very
positive.
I would say that in two to three years’ time, I believe it to be impossible not to conduct
board work in this way [by means of digital board packs] NF p. 14
Several of the board members perceive that the driving forces of this particular
decision of the municipal board to implement the sustainability policy are twofold,
environmental concerns to reduce the large quantities of paper used for the board
pack before each meeting as well as to simplify work for the administration. The
instant access to new and updated material for the meeting is seen as a positive
effect of the change.
We are doing this because, among other, so we could stop distribution of paper. … the sec-
retary have to spend a lot of time for making copies and sending them out… it will be more
effective and much better for her too, than to collect and keep the material. And to send out
the material comes with a cost… And available… the board members who travel… and for
this purpose it is great, I mean, they will get it regardless of where they are at current. They
get immediate access to it. JM p. 17.
I truly believe that this can result in a drastically reduction of the use of paper and a more
simple administration. CK p. 17
I thought it was great. Truly, because I have been concerned about this excessive use of
paper. BJ p. 4
The negative impact of the decision of the municipal board to implement the sus-
tainability policy that the respondents express is mainly concerned with the imple-
mentation of the new policy and lack of functionality in the tablets. Concerned with
the success of implementing the decision to implement policies via tablets, one of the
respondents states that initially he was doubtful to the initiative due to his previous
experience. In that case, people didn’t bring their tablet and a negative connotation
of the tablets was developed. But now he is positive as the software is much easier to
use and the paper version of the board pack is about to stop being distributed.
Deep concerns of the respondents refer to the many ways paper can be used in to
receive, give, or evaluate information and when refining or debating information
with others. In particular these concerns arise when discussing key aspects in how
the respondent perceive of as part of enacting their role as board member.
… the accumulated information on a paper or on a whiteboard, it is still unmatched when
you are to get a group of people to progress in an issue. If you just want to impose a decision
it is fine without, but if you want people to rally with you it is [unmatched]. JM p. 18
I believe that it is about that, in emails I can communicate to everyone and let them know
what I have worked on. What have you acted on? And exchange, a dialogue. And it cannot
be done with this tool. I cannot reply to everyone. I cannot communicate. I cannot even
reply the organization; at least I believe I cannot. MS p. 11
Other concerns that are expressed have to do with limitations in the functionality.
This concern several aspects of what the respondents indicate that they perceive as
their role and responsibility in the board work. As an example, writing in the pdfs to
add a comment to oppose a suggestion was not a function that could be demon-
strated by the one signing out the tablets.
They could not explain how to connect the notes to a document that already existed… [Not]
take separate notes… I want to merge them. BL p. 9
The limitations are also manifested as the lack of compatibility between the tab-
let and software that the respondent has already learned to work with.
I believe that in some cases I have used a ppt-presentation. In this case I cannot use the
tablet. I cannot write it myself, so I ask one of my co-workers to create the ppt-presentation,
so that I can give a presentation on the issues in question… JM p. 3
As stated above, the stipulated policy aim for digitalization of board packs and
memoranda with the intent to enable monitoring of quality and facilitating to make
wise decisions, while reducing the cost to produce decision making. Clearly, there is a
limitation in the use of the tablet compared to what the respondent perceives as neces-
sary to be able to freely conduct one of the main responsibilities of the board work.
Thus, the policy directive of the third focus area to increase the possibilities for busi-
ness development and effect repatriation inherent in the offered IT-tools by developing
the ability to master the business support offered via IT largely seems dependent on the
possibility to (attract, recruit and) retaining dedicated and competent employees.
With this illustration of the general reception of tablets as a means to implement
the policy and its relation to the beliefs and abilities of the employees of the organi-
zation, we have sketched a cursory view of the perceptions of the board and its rela-
tion to the sustainability policy. In the following we proceed to presenting the results
of our study according to the expressed relations of BAO (Melville 2010) which
highlights the macro and micro perspectives of the role of IT and organizational
sustainability performance.
1 Expectations in the societal structure result in beliefs with impact on the
activities
Several of the respondents express that their view of the tablets and their useful-
ness have been influenced by others. Sources of influences are adult children, col-
leagues at the work place or others that they meet while conducting their regular
work. Three of the respondents all state that when the decision was known, the
respondent’s beliefs were influenced by earlier encounters with experienced tablet
users in society.
I thought that it was an excellent idea. … I had seen people. Actually, I was on the verge to
buy one myself, because I have seen my children use these. It is an incredibly good inven-
tion. But before I got to purchasing one it was announced that all in the board was to get a
tablet. DD p. 3
Well, I was more curious as to how it would be to use it as I saw these possibilities from
how X [one of the other board members] uses the tablet. So I thought that this will be excit-
ing… BL p. 6
…I have friends that own tablets of both Swedish and International brands which have been
slightly different from each other. I have meet with my friends and asked them how it is and
so. If you read, in particular if you read more extensive texts. JN p. 6
Clearly, several of the board members express very positive beliefs on the tablets
and what it will be able to accomplish for their board work.
Despite that most respondents indicate that society has provided positive influ-
ence on how they perceive the tablets, one respondent shares previous experience
from the implementation of the policy in another part of the organization that
resulted in a negative impact on using the tablet:
It was almost as if you gave the impression that you were not prepared for the meeting if
you didn’t bring the paper pack as well. You looked as if you only had your tablet and
everyone else had the paper pack which made you look as you weren’t properly prepared.
JJ p. 5
Our results indicate that the social relations both within and outside the organiza-
tion are highly influencing the beliefs of how the tablets may be useful in the board
work.
1′ Expectations in the organizational structure result in beliefs with impact on
the activities
Most respondents express a firm attitude towards their role concerning legiti-
macy of the board work. Adhering to the legal ramifications is perceived as very
important.
We can never put ourselves in the position in which one of the board members hasn’t gotten
enough information so that a decision cannot be taken. Formally it is the case that such a
decision can be challenged and nullified, and thus the decision needs to be readdressed.
According to the Swedish corporate law all members of a board needs to be well informed
before the board makes its decision. CK p. 4
Several respondents express that they have not yet found their own way when it
comes to conducting their work via the tablets. Most however are positive and hope-
ful as to what can be done with the tablets once they have mastered the device and
software. To illustrate what they see possible to do with the tablet and when and
where the tablet can be used for board work, several of the respondents are drawing
on analogies to their use of other IT devices for accomplishing board work or use
them as the starting point for their beliefs:
Well, with the smart phone I could read emails and handle simple tasks while in the metro,
standing. I believe that the tablet I would bring out if I was seated for a longer trip, not dur-
ing rush hour. BI p. 8
Still, the legal ramifications are also seen as more rigid than the technology, mak-
ing the norms the outer boundaries of to what extent the municipal policy can be
implemented via the tablets.
Will not affect the board’s work very much … will not affect at the board work in the sense
that the board will lead a somehow completely different life compared to today. We will
have meetings, for that we must have. We cannot, we could do everything via the tablet but
it must not under the Municipal Act, so the board must meet… and then, all board members
need to sign … The difference is that the board pack will we have in our tablets. DD p. 18
Our results indicate that the organizational norms and use of other types of IT
devices for conducting board work is highly influencing the beliefs on the tablets
and the impact of tablets as means for implementing the sustainability policy.
3 Individual beliefs result in actions with impact on the functioning of the social
system
Overall the respondents seem to perceive that the change will bring a positive
and relaxed impact to the board.
Well, we have a very nice board and a good atmosphere and then of course when everyone
got their new tablets so it was even. Then everyone is happy, if you understand. So it is as
well, so nice and stuff. There’s very little prestige, people say happily that I understand
nothing and how should I do this. It contributes to a good atmosphere, I think, and will
surely do so in the future. JM p. 9
However, three respondents voice some more negative or ambiguous beliefs. As

an example, one respondent indicate that the overall board work is disturbed as the
new technology isn’t always more easy to browse and manage.
And it is so easy when you have paper, I think it is that you’ve got an overview. It’s just to
take the example of all those who have tablets and whatever it is called when they are to use
their calendar functions. I mean you have got to always sit and wait for them. MS p. 4
Still, also respondents that express a sense of resistance or barriers to adopt the
new technology based on differences between working with paper and working
with a tablet, also expresses a willingness to adapt on behalf of the other board
members.
… For my part I had gone just as good to keep the paper versions really. But of course, if
you sit in several committees and boards and councils and other, then it is very possible that
this is much more efficient. I cannot be sure about this because I’m not in the situa-
tion. … QV p. 7
According to the results, the functioning of the social system is impacted by

individual beliefs and resulting actions. Mainly, these concern the barriers of new
technology, but also the perceived positive potential of IT to support others in their
work.
3′ Individual beliefs results in actions with impact on the functioning of the
organization
Ensuring that HC was not the first organization to implement the tablets, one
respondent expressed very negative experiences with tablets replacing the paper
form of the board pack. This experience directly resulted in HC not being the first
organization in the municipality to implement the tablets.
But I have such a tremendously bad experience from trying to use tablets in the City
Planning Committee Context so I said if we’re going to introduce them in HC then they
should work. We cannot be a test bed, I’m not doing it, and then it was one of our sister
companies that went before, and I guess that they handled some of the problems as it has so
far worked well for us. CM p. 7
Interestingly, despite no extensive knowledge transfer between the board mem-

bers having occurred, the respondent expresses beliefs that some board members
are “very experienced” etc. with tablets. These beliefs are manifesting themselves
into almost a confirmation on what could be accomplished with the tablet in the
actual board work:
It has simplified, above all for those that already use, that have already used the tablet at the
board meeting. Both the CEO and several of the board members… Primarily as they do not
have to keep the high piles of paper in front of them and browse in. Instead of going through
a bunch of papers, with a couple of clicks you at once get what it is about. CM p. 18
Also other motivations (in line with other policies of the municipality) that have
to do with improving the functioning of the organization are visible for introducing
the tablets in the board:
If the board has a tablet and you work in a tablet so it also means that it is okay that I am
introducing it elsewhere in the company. So I can see eventually that all my service techni-
cians should have one, And then it’s so… Yes, I have learned from a wise colleague here in
our city that is that you must always ensure that the board gets it first, then you can have it
in the organization… It is a way to implement something in the organization. If the board
has it they will think that this is of course a great instrument…Because then they know how
good it is and how it… So, first, that it is positive for the board work, but also to then be able
to implement it in other parts of HC. JM p. 10
No, but I thought it was great for it is impossible to escape the fact that this is a way to
distribute the paper in the future and, I thought about it and it’s partially this research proj-
ect, that, that. Doing the same thing we did before, but to get it, to solve [? 43:50] in this
way , but I wonder can we use the tablet in a thousand other ways to our board of which I
know nothing about yet…especially if we can work in different ways in board work ahead.
Where is we do not yet, but it’s exciting to see. NF p. 15
The results indicate that there is an expectation from some board members that
the full board needs some time to adjust to the new material and the success of the
change dependent on this, hence, the change is done gradually.
Well, we cannot make an abrupt ban of paper, but we will gradually phase out the paper
documents CM p. 4
On the contrary, this is one of the concerns voiced by another member of the
board, that a complete stop to using paper is vital to the success of the change.
And then it has not worked well because people haven’t had the tablets with them at meet-
ings, it has not been, you’ve still got the documents in paper form so people have the habit
and continued to use the papers, etc., etc.. And it will not be sent out paper wise, if I under-
stand it right, but then it’s forcing people to learn it and then I am going, it will be used more
frequently. So it feels good. JJ p. 3
Yet others of the respondents have made up their mind to go their own way if
feasible routines are not presented. Often it has to do with resorting back to old
ways of working. But also for these respondents this is only seen as a last resort.
But it is of course possible to export then to one’s own email and then print it from your
computer. So it is always possible to do it like this… and that’s what I’ve decided. I think
it’s nice to have as little paper as possible and so. And then it’s a matter of habit. MS p. 4
The results indicate that the beliefs on the use of the tablets in the board work as
well as the way the implementation of the policy-based decision are carried out in
the organization are highly influenced by individual beliefs and the previous experi-
ences of the key board members.
4 Expectations in the societal structure result in actions with impact on the func-
tioning of the social system
One respondent remarks that it lies in the Swedish culture to have a technology
positive attitude:
The top management said that now we want to try to digitalize the information and thus we
are now to use tablets. Fine, the board said. We Swedes never question a new technology.
We take it as it comes. DD p. 16
The members of the board seem to sport an attitude that is very allowing and
supportive. The results indicate that some board members have taken the role to
ensure that this attitude is manifested and reproduced during meetings:
In particular X who do not mind to say that I understand nothing, and it’s very good because
then there is no need. Yes, then all can say yes, I do not understand either. It will be very
nice. JM p. 9
Clearly, this feeling of empowerment via the collectively expressed cultural atti-
tude and the not-knowing is viewed as important for the working climate of the
group. However, despite this positive climate, very few of the respondents state that
they feel confident with using the tablet after the introductory training to the tablets.
Almost all respondents express that on behalf of the group they were satisfied with
the training process, as most see a need for others to get training at the given level.
However, both newcomers to the tablets as well as experienced tablet users indicate
that the training was not spot-on for their needs.
4′ The organizational structure results in actions with impact on the functioning
of the organization
Despite the positive attitude towards the decision of the municipal board to
implement the sustainability policy and the tablets, the lack of alignment between
the functionality of the tablets and software and the activities of the board, i.e., the
requirements from the organization on the functionality of the tablets, seem to have
resulted in actions with impact on the functioning of the organization.
Yes, it was decided and it came to both the e-readers, or tablets, and then also our phones
that they have chosen that. And so, yes. You could think about, but there is no question that
I have had any influence on but it’s just. I can say, for me it’s a shortcoming that we will still
end up in. If I really should be working on something or write something, it’s clear that I do
not use the tablets. Because it’s impossible to write on it. I mean if I’m going to write. I can
answer emails and such, but if I must write something longer emails or documents, it is
clear that I do not use it. JM p. 5
Also the respondents indicate that routines concerning privacy of the data have
to be changed. Previously the respondents used paper with a distinct green color to
signal classified data. To provide such information via the tablet was recognized as
an important issue to handle, the reason being that digitizing the board pack means
that the sensitive information cannot be withdrawn.
… lot of questions as well, which I have to think about. This one actually has to think about,
if it would not require a code. Despite you may have logged into the computer you should
need to have a code when you open this [classified information]. JM p. 8
The results indicate that shortcomings of the tablet as compared to paper given
the existing routines and ways of working are identified by the board members but
probably not comprehensively addressed by the policy-makers.
5 Expectation in the societal structure results in actions with impact on the func-
tioning of the organization
Looking at the differences in expectations on tablets as compared to expectations
on paper, one of the respondents made the remark that, especially concerning con-
fidential issues, the board work ventures into new territory, in which no explicit
routines are available. They voice the concern that the popularity of the tablet leads
to the risk that the board pack is exposed to people outside of the board.
So this is indeed a device that is very popular. I think everyone let their kids get involved
with it and so there because it is so easy. Or rather, we have no control on it. What people
do with them … Then maybe one needs to consider that it can be opened by a number of
people who are probably not just the one who sits on the board and then it may be necessary
to have a special code. JM p. 8
Most board members state that they have installed several different apps for per-
sonal use and leisure. A couple of board members also indicate that people outside
of the organization have been allowed to use and install software on the tablets.
The results indicate that differences in the perceptions and expectations on who
can use a tablet as compared to paper may have impact on the functioning of the
organization.
5′ The organizational structure results in actions with impact on the functioning
of the social system
Several of the respondents sport a very positive attitude towards complying with
the decision to implement policies for digital renewal via new technology such as
tablets. They indicate a clear and firm willingness to change and they provide sev-
eral examples to how they are working with themselves to adapt to the new
technology.
Of course we will use the tablet now that we have decided that we will use the tablet and
made a huge investment MS p. 18
Now that we have decided that we will use the tablet, I will use it… I will not print the board
pack or in any way argue for any different way of working. BJ p. 13
I try to educate myself to using it, or that I will, use the tablet and read from it at board
meetings. JM p. 11
It annoys me sometimes that you cannot, [a particular character] are not there. Some char-
acters … Not familiar territory, this keyboard setup … Which has meant that when I send
emails from the tablet I do not write my name as I usually do, but in [another way that does
not require this special character] BL p. 13
Some respondents have also started using the tablet for other aspects of their life
that has impact on their other social relations.
The technology also simplifies the everyday life outside of the organization. Now I can
write to my wife in the X app, put it down now. Previously, she was always somewhere
online, or my children. Now I can write in the tool like this: Dinner is ready. MS p. 12
In this way, we find that both the board as a social collective as well as the social
structure outside of the board are impacted by the decision of the municipal board
to implement the sustainability policy.
Melville’s Framework Put into Immediate Practice
Several of the board members express very positive beliefs on the tablets and what
they will be able to accomplish for their board work. From the empirical material,
we observe that the actual way the tablets have been introduced in the board has
been highly influenced not only by the decision of the municipal board to imple-
ment the sustainability policy and other norms of the organization, but also by indi-
vidual beliefs and the previous experiences of the key board members. The parallel
use of paper and digital board pack is one example of these influences. In this way,
the organizational influence of the decision of the municipal board to implement the
sustainability policy is mainly produced via the chairman. This observation is also
in line with Melville’s cf. “… ways in which an organization divides its labor into
distinct tasks and achieves coordination among them” (Melville 2010, p. 6).
As recommended by Melville (2010) sustainability is not only with regards to
ecological sustainability but also needs to ensure that social aspects of sustainability
are advocated and catered for. In this vein, two respondents indicate that they per-
ceive that the decision to implement policies for digital renewal via new technology
such as tablets is beneficial also from a democracy perspective:
I think it’s a good thing, a good development and a tool. Yes, one think one should use. It’s
just that now look at the trials in progress at pre-school level where one and two year olds
get tablets as part of the teaching process, or, well, learning simply. So, no, I probably
thought it was natural to try … I was part of and made sure we got the Computers for politi-
cally elected, it was from a democracy work we had done, where we saw that there was a
need that people did not have computers at home. Printers and so on. And it was not equal
opportunities for our elected representatives. MS p. 8
No, but I thought it was good. Yeah … I did not know so much what a tablet really was. I
had barely held one. … No, I just think it’s a good tool. It was so before that we in municipal
got a computer, a small computer that you had at home and printers. We had it for 10 years,
then it was pulled four years ago, maybe, or three. Three maybe something like that. It was
withdrawn. We had to return them because the contracts were too expensive, and then they
took away the opportunity for many people. Yes, you could buy one yourself, but they took
away this opportunity. Therefore, I thought it was fine with the tablet for the sake of emails
too. To start with I didn’t send work related e-mails either via the cell phone, so it was dif-
ficult with them. NJ p. 12
In this way, for the immediate future, we find that both the sustainability perfor-
mance of the organizational structure as well as the social collective are affected by
the beliefs and actions of key individuals.
However, it should be noted that also other members of the board as well as
trusted relations outside of the organization exercise impact on the beliefs of the
respondents. As an example, the empirical material indicates that technology savvy
persons are both influencing the beliefs of how well the tablet will serve its purpose
and what is possible to achieve with the tablet. Similarly, trusted persons, mainly
family members, have also been allowed to use the tablets and in some cases been
allowed to make installations of apps on the tablets, thus, actualizing issues of secu-
rity and integrity of sensitive data, a vital part of social sustainability performance
for the tenants of HC. Moving from the macro-level effects of the implementation
of the decision of the municipal board to implement the sustainability policy to
micro-level effects, we find that these effects may be either positive or negative for
the individual, and thus warrant the conclusion that the effects of a decision of the
municipal board to implement the sustainability policy may be expressed as a
sustainability performance measurement of the overall organization (cf. Bengtsson
et al. 2012; Dyllick and Hockerts 2002).
How to Ensure That Sustainability Will Sustain?
Melville’s theoretical framework (analytical lens) can be enacted in different ways

(see Melville 2010). Our analysis has shed light on different statements by the mem-
bers of the board and some situated observations. The respondents’ statements are
interpreted as beliefs guiding their further thoughts and behavior.
Long Term Effects on the Organizational Sustainability

Performance
The starting point of Melville (2010) is the collective as the origin of belief.
According to Melville (2010) normative or cultural patterns determine the expecta-
tions that aid in producing and reproducing the relationships of these collectives. On
a similar note, Salzmann et al. (2005) argue that the complexity of organizational
sustainability performance is context dependent and vary depending on type of
industry, country and point in time. Hence, the board members partake in and relate
to both the social collective that “constructs” the board, and partake in and relate to
the social structuring of the board as part of the organizational structure.
On this note, from the empirical material we see that most respondents had a
very positive attitude to learning to use the tablet, even to the point that even though
issues or barriers were pointed out, they were seen to be possible to surpass, or even
seen as mandatory to surpass. The decision of the municipal board to implement the
sustainability policy can be viewed as part of the already implemented and accepted
policies of the organization, which manifest itself as impact to the “willingness” of
the collective to implement the decision of the municipal board to implement the
sustainability policy in the routines and activities of the board, as well as impact the
beliefs of the board members of the board as a social collective. In line with this
interpretation of Melville (2010), we found that already implemented and accepted
policies such as legal prerequisites for the board, as well as expectations from soci-
ety and organizational structures are translated into beliefs of the board members.
This is not surprising as this alignment to patterns of the societal and organizational
structures are directly translatable to the legitimacy of the board work as well as the
role of representative of their political party and thus political beliefs.
Useful to guide our investigation, we find that the framework of Melville and its
“societal structure” and “organizational structure” is equipped to give us valuable
snapshots. However, the structure of the model is by default quite inflexible. It high-
lights what is within and suppresses what is without, as technology for representa-
tion it makes states of affairs frozen. What happens if we regard peoples’ beliefs as
both an outcome and a medium of different perceptions and behavior (cf. Giddens
1984) and recognize that existence can occur only on the basis of impermanence?
This opens up for further questions not only how perceptions and behavior are sus-
tained by difference in depth and impact, but also reproduced (otherwise they could
neither be regarded as habits or as recurrent sustainable activity). This view partly
puts Melville’s framework in a different guise and transforms it to a springboard for
studying processes of “becoming” an implemented policy too. Paraphrasing Michel
Foucault, we might say that such approach opens up for a discussion not only that
people know what they do; or why they do what they do; but also what they don’t
know about what doing it does (cf. Dreyfus and Rabinow 1983, p. 187).
Exploring this more fluid perspective, we find that the empirical material indi-
cates that the shortcomings of the tablet as compared to paper are not clearly
addressed from the policy-makers. The lack of directives for implementation caters
for an ambiguity in what the tablet is and how to relate to it in the first place, thus
affecting the beliefs of the board members. As an illustration, several respondents
draw on analogies to use of more well-known technology to express and make sense
of the tablet at the point of introduction. Often referral is made to perceived benefits
or bad experience of other more experienced users. In a couple of other cases the
analogy is made to other domains of use, e.g., the use of tablets in pre-schools, to
justify why the tablets are useful and the decision of the municipal board to imple-
ment the sustainability policy is beneficial despite the high initial cost of providing
tablets to the board.
In a similar vein, the empirical material illuminates that initially a feeling of
empowerment via a collectively expressed cultural attitude of not-knowing during
the introduction of the tablets was present and it was viewed as important for the
initial working climate of the board. However, almost all respondents express that
they haven’t found their way with the tablet and they express that the offered train-
ing was not aligned to their needs. Hence, the outcome of the decision of the munic-
ipal board to implement the sustainability policy via tablets depend on an ongoing
process of what doing board work with digital board packs does in relation to the
social collective. In other words how well individual board members perceive the
tablets work for conducting the board work, and how these beliefs are translated
among the members of the board.
As a final illuminating example, we also find that the empirical material clearly
indicates a difference in perceptions of tablets and paper in society. As a conse-
quence, the routines of the board concerning privacy of the data are required to
change in order to promote social sustainability.2 The reason is that the popularity
of the tablet leads to the risk that the board pack is exposed to people outside of the
board. A digitized board pack means that the sensitive information cannot be with-
drawn. Preserving security and privacy in the digitized board pack require new rou-
tines and functionality. Currently this level of security is not offered in the
functionality or otherwise handled, thus rendering the organization’s overall sus-
tainability performance alignment to the sustainability policy of the municipality
dependent on the doings of the board.
So this is indeed a device that is very popular. I think everyone let their kids get involved
with it and so there because it is so easy. Or rather, we have no control of it. What people
do with the tablets. JM p. 8
Then maybe one needs to consider that it can be opened by a number of people who are
probably not just the one who sits on the board and then it may be necessary to have a spe-
cial code. JM p. 8
… a lot of questions as well, which I have to think about. This one actually has to think
about, if it would not require a code. Despite you may have logged into the computer you
should need to have a code when you open this [classified information]. JM p. 8
If we take into consideration this kind of process-oriented approach, and link it to

Melville’s framework, what can we say about the implementation of policies on digi-
tal renewal and sustainability? Our analytical approach carried by Melville’s frame-
work shows the importance of analyzing sustainability not only in static terms but in
terms of processes. Such an approach moves the perspective to beliefs and correlated
actions of certain depth and impact and how policies on sustainability can be retained
and become robust, only by means of making the outcome of IT useful in the social
setting. We have by several illustrations illuminated the potential value of such an
approach. Because what would be the value of sustainability if it doesn’t sustain?
Conclusion
This chapter examines how a decision to implement policies for digital renewal via
new technology such as tablets in a municipality was implemented as an introduc-
tion of tablets in one of the municipal boards, and how the decision to implement
policies for digital renewal via new technology such as tablets prompted changes in
2
From instruments that measure sustainability performance of organizations (eg. GRI-index,
GRIG4 Part2 Implementation-Manual, retrieved April 7 2014 p. 233) we know that protection of
customer privacy is a generally recognized goal in national regulations and organizational policies.
It serves as an indicator for organizational sustainability performance in the GRI-index. Non-
compliance is stated to indicate either inadequate internal management systems and procedures or
ineffective implementation.
the board work setup as well as raised issues concerning security and privacy of
data. Summing up our analysis, the contribution includes an account to how policies
for digital renewal via new technology such as tablets are aligned to a municipali-
ty’s IT policy (vision). We have, for example, shown that replacing paper as an
outcome of this policy may serve as a catalyst to establish and reproduce new setups
and practices in the board room setting not necessarily connected to previously
established policies and sustainability goals of the organization. Key highlights are
the importance for decision makers to understand the context and ongoing practices
related to a decision to implement policies for digital renewal via new technology
such as tablets, and the role this plays for how the sustainability of the results of
different policies becomes.
So, what impact did the introduction of tablets have on the policies of the Vision
2030—a World-Class City of the municipality and its IT—for digital renewal and
green-IT?
The policies stated that IT should be seen as a tool in the daily work of the
municipality and used in a new and innovative way. Unfortunately, few or any of the
policies have been manifest in the daily routines of the board. Some improvement
in compiling the information has been established. The board packs have been digi-
talized, however, a parallel paper version is still in use, hence not reducing the cost
of producing wise decisions. However, there seems to be some confusion on what
version to regard as the master set of the board pack.
Drawing on the existing IT support and procurement contracts, no adaption of
hardware or software to fit the needs of this particular board have been possible.
Thus, it can be suggested that simplification of the use of existing IT support has not
been achieved, instead quite the opposite. The respondents seem convinced that
there is potential for using the tablets for other means of collaboration and informa-
tion sharing, however no concrete examples have been seen in the empirical mate-
rial. As the tablet has been introduced as a mere analogue to the paper and, without
any other training, the possibilities for innovation have been limited. In this sense,
the sustainability index of WCED becomes even more important. Simply measur-
ing net increase in resources by CO2 index or use of power supplies cannot account
for sustainability, and as such, equally important as the understanding of the prereq-
uisites of successful use of IT, for the evaluation of IT in policy information, devel-
opment or evaluation for a sustainable future.
References
Adler PA, Adler P (1994) Observation techniques. In: Denzin NK, Lincoln YS (eds) Handbook of
qualitative research. Sage, Thousand Oaks, pp 377–392
Bengtsson F, Ågerfalk P (2011) Information technology as a change actant in sustainability inno-
vation: insights from Uppsala. J Strat Inf Syst 20(1):96–112
Bengtsson F, Eriksson Lundström J, Sjöström S, Hrastinski S, Ozan H (2012) Sustainability
impact of open innovation software. In: Proceedings of the XXIII ISPIM conference, action for
innovation: innovating from experience, Barcelona, Spain, 17–20 June 2012
Coleman JS (1986) Social theory, social research, and a theory of action. Am J Sociol 91:1309–1335
Constas MA (1992) Qualitative analysis as a public event: the documentation of category develop-
ment procedures. Am Educ Res J 29(2):253–266
Creswell JW (2012) Qualitative inquiry and research design: choosing among five approaches.
Sage, Los Angeles
Dreyfus H, Rabinow P (1983) Michel Foucault: beyond structuralism and hermeneutics. University
of Chicago Press, Chicago
Dyllick T, Hockerts K (2002) Beyond the business case for corporate sustainability. Bus Strategy
Environ 11:130–141
Elkington J (1997) Cannibals with forks. Capstone, Oxford
Epstein MJ, Roy M-J (2001) Sustainability in action: identifying and measuring the key perfor-
mance drivers. Long Range Plann 34(5):585–604
Erdmann L, Lorenz H, Goodman J, Arnfalk P (2004) The future impact of ICTs on environmental
sustainability, European Commission Joint Research Centre. ftp://ftp.jrc.es/pub/EURdoc/
eur21384en.pdf. In: Melville NP (2010) Information systems innovation for environmental
sustainability. MIS Q 31(1): 1–21
Farrell D, Oppenheim J (2008) The carbon productivity challenge: curbing climate change and
sustaining economic growth. McKinsey Global Institute. http://www.fypower.org/pdf/MGI_
Carbon_Productivity.pdf. In: Melville NP (2010) Information systems innovation for environ-
mental sustainability. MIS Q 31(1): 1–21
Friedman M (1982) Capitalism and freedom. University of Chicago Press, Chicago
Gherardi S (2010) Telemedicine: a practice-based approach to technology. Hum Relat 63(4):501–
524 [Originally published online 8 January 2010]
Giddens A (1984) The constitution of society: outline of the theory of structure. University of
California Press, Berkeley
Glaser B, Strauss A (1967) The discovery of grounded theory: strategies for qualitative research.
Aldine, Chicago
GRI-index (2014) Global Reporting Initiative-index. https://www.globalreporting.org. Accessed 5
Apr 2014
Hanson N (1958) Patterns of discovery: an inquiry into the foundation of science. Cambridge
University Press, Cambridge
Hill S (1995) The social organisation of boards of directors. Br J Sociol 46(2):245–278
McNulty T, Pettigrew A (1999) Strategies on the board. Organ Stud 20(1):47–74
Melville NP (2010) Information systems innovation for environmental sustainability. MIS Q
31(1):1–21
Myers MD, Young LW (1997) Hidden agendas, power and managerial assumptions in information
systems development: an ethnographic study. Inf Technol People 10(3):224–240
Orlikowski WJ, Scott SV (2008) Sociomateriality: challenging the separation of technology, work
and organization. Acad Manage Ann 2(1):433–474
Powell WW, di Maggio PJ (eds) (1991) The institutionalism in organizational analysis. University
of Chicago Press, Chicago
Romm J (2002) The internet and the new energy economy, resources, conservation, and recycling
36(3): 197–210. In: Melville NP (2010) Information systems innovation for environmental
sustainability. MIS Q 31(1): 1–21
Ruigrok W, Peck S, Board H (2006) Board characteristics and involvement in strategic decision
making: evidence from Swiss companies. J Manage Stud 43(5):1201–1226
Salzmann O, Steger U, Ionescu-Somer A (2005) The business case for corporate sustainability:
literature review and research options. Eur Manage J 23(1):27–36
Schuman H, Johnson MP (1976) Attitudes and behaviour. Annu Rev Sociol 2:161–207
Sharma S, Henriques I (2005) Stakeholder influences on sustainability practices in the Canadian
forest products industry. Strat Manage J 26:159–180
Sørensen C, Landau J (2013) We’ve got 99 problems, but a phone ain’t one: mobile ICT and aca-
demic agility in information systems research. In: Eseryel YM (ed) IFIP WG 8.2 OASIS
workshop
Stake RE (1995) The art of case study research. Sage, Thousand Oaks
Strauss A, Corbin JM (1998) Basics of qualitative research: techniques and procedures for devel-
oping grounded theory. Sage, Thousand Oaks
Taylor SJ, Bogdan R (1998) Introduction to qualitative research methods: a guidebook and
resource, 3rd edn. Wiley, New York
The 1997 Kyoto protocol of United Nations Framework Convention for Climate Change (2014)
https://unfccc.int/essential_background/kyoto_protocol/items/1678.php. Accessed 2 Apr 2014
The Paperless Project (2014) http://www.thepaperlessproject.com/. Accessed 13 Mar 2014
Vaast E, Walsham G (2005) Representation and actions: the transformation of work practices with
IT us. Inf Organ 15:65–89
van Marrewijk M (2003) Concepts and definitions of CSR and corporate sustainability: between
agency and communion. J Bus Ethics 44(2/3):95–105
WCED (1987) Our common future. Oxford University Press, Oxford
Yin RK (2009) Case study research: design and methods, 4th edn. Sage, Los Angeles

2016 Book InformationModelsAndSustainabi

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2016 Book InformationModelsAndSustainabi

Uploaded by

Copyright:

Available Formats

Public Administration and Information Technology 20

More information about this series at http://www.springer.com/series/10796

Theresa A. Pardo Djoko S. Sayogo

Public Administration and Information Technology

Library of Congress Control Number: 2015957953

Springer Cham Heidelberg New York Dordrecht London

Printed on acid-free paper

Springer International Publishing AG Switzerland is part of Springer Science+Business Media

We gratefully acknowledge financial support from the National Science Foundation

Information, Policy, and Sustainability: The Role

On Sustaining Sustainability: The Case of Implementing

Luis Felipe Luna-Reyes is an Associate Professor of Informatics at the University

Theresa A. Pardo is the Director of the Center for Technology in Government,

Jing Zhang is an Associate Professor of Management at Clark University. Her

Institute of Public Enterprises. Her publication appears in European Journal of

Elizabeth Allen is a Ph.D. student in the School of the Environment at Washington

Deborah Lines Andersen is Associate Professor of Information Studies and

David F. Andersen is Distinguished Service Professor of Public Administration,

Matteo Convertino, Ph.D., P.Eng. is Assistant Professor in the School of Public

Marco Aurelio M. Ferreira is Coordinator of the Graduate Program at Public

Joanne Luciano is the President and Founder of Predictive Medicine, Inc. a

her research focuses on computational enabling technologies and the reusability of

Luis Felipe Luna-Reyes is a professor of business at the Universidad de las

Rafael Munoz-Carpena, Ph.D. is a Professor in the Agricultural and Biological

Theresa A. Pardo is the Director of the Center for Technology in Government,

Suzanne J. Piotrowski is an Associate Professor of Public Affairs and

Weijia Ran is a Ph.D. candidate in Informatics at the University at Albany. Her

Djoko Sigit Sayogo is senior lecturer in the Department of Economics and

Jennie C. Stephens is an Associate Professor and the Blittersdorf Professor of

Giri Kumar Tayi is a Professor of Management Science and Information Systems

Jing Zhang is an Associate Professor of Management at Clark University. Her

Jing Zhang, Luis F. Luna-Reyes, and Theresa A. Pardo

© Springer International Publishing Switzerland 2016 1

goals particularly committed to the adoption of a new ethics of conservation and

A similar role that can be played by government is to use information as a policy

Analytical Methods and Techniques

A model, in general, is a conceptual representation of a problem, and it helps policy

Fig. 1 A simple classification of models (Adapted from Puron-Cid et al. 2014)

terms of their approach to probabilistic events (stochastic vs. deterministic), their

Data and Open Data

The use of information technologies in the context of public policy analysis is a

Stakeholder involvement in the construction and analysis of models to explore pol-

Policy Informatics and Sustainability

The use of mathematical models to better understand sustainability can be traced

Chapters on Information, Policy, and Sustainability

Contribution of This Book

using policy informatics as a tool to apply to the problems of sustainability. This is

Basiago AD (1995) Methods of defining “sustainability”. Sustain Dev 3:109–119

Matteo Convertino, Rafael Munoz-Carpena, and Carolina Murcia

Abstract Management of complex ecosystems is a difficult process that involves

© Springer International Publishing Switzerland 2016 21

model of stakeholders because its construction is based on the direct ­elicitation of

numbers that can be used effectively for policy-relevant decision-making. Such

The Costa Rican Case Study

The Tempisque–Bebedero–Palo Verde Ecosystem

Fig. 1 Tempisque–Bebedero–Palo Verde ecosystem

Palo Verde Workshop

Participants were asked to submit (individually or in small disciplinary groups) an

Elicitation of Stakeholder Preferences

Stakeholder preferences have been analyzed as a function of two categories (i.e.,

Semantic Network Extraction Model

Influence Diagram Construction

Network Complexity as Objective Function

model of stakeholders because its construction is based on the direct elicitation of

The Costa Rican Case Study

The Tempisque–Bebedero–Palo Verde Ecosystem

Palo Verde Workshop

Elicitation of Stakeholder Preferences

Semantic Network Extraction Model

Influence Diagram Construction

Network Complexity as Objective Function

Global Sensitivity and Uncertainty Analyses

Contrasting GGP and Corporate Green Supplier Selection

ough Set and TOPSIS and VIKOR Background

Rough Set Theory

The TOPSIS Method