Evaluation and Action Research Linnea L Rademaker Full Chapter

Evaluation and Action Research Linnea
L. Rademaker
Visit to download the full and correct content document:
https://ebookmass.com/product/evaluation-and-action-research-linnea-l-rademaker/
Evaluation and Action Research
Research to the Point Series
Evaluation and Action Research: An Integrated Framework to Promote Data Literacy and
Ethical Practices
Linnea L. Rademaker and Elena Y. Polush
Evaluation and Action
Research
An Integrated Framework to
Promote Data Literacy and
Ethical Practices
Linnea L. Rademaker and

Elena Y. Polush
1
1
Oxford University Press is a department of the University of Oxford. It furthers
the University’s objective of excellence in research, scholarship, and education
by publishing worldwide. Oxford is a registered trade mark of Oxford University
Press in the UK and certain other countries.
Published in the United States of America by Oxford University Press

198 Madison Avenue, New York, NY 10016, United States of America.
© Oxford University Press 2022
All rights reserved. No part of this publication may be reproduced, stored in

a retrieval system, or transmitted, in any form or by any means, without the
prior permission in writing of Oxford University Press, or as expressly permitted
by law, by license, or under terms agreed with the appropriate reproduction
rights organization. Inquiries concerning reproduction outside the scope of the
above should be sent to the Rights Department, Oxford University Press, at the
address above.
You must not circulate this work in any other form

and you must impose this same condition on any acquirer.
Library of Congress Cataloging-in-Publication Data

Names: Rademaker, Linnea L., editor. | Polush, Elena Y., editor.
Title: Evaluation and action research : an integrated framework to promote data
literacy and ethical practices / Linnea L. Rademaker and Elena Y. Polush.
Description: New York, NY : Oxford University Press, [2022] |
Series: Research to the point series | Includes bibliographical references and index.
Identifiers: LCCN 2021044056 (print) | LCCN 2021044057 (ebook) |
ISBN 9780197620823 (hardback) | ISBN 9780190921729 (paperback) |
ISBN 9780190921743 (epub) | ISBN 9780190921750
Subjects: LCSH: Social sciences—Research. | Action research.
Classification: LCC H62 .E93 2022 (print) |
LCC H62 (ebook) | DDC 001.4/2—dc23
LC record available at https://lccn.loc.gov/2021044056
LC ebook record available at https://lccn.loc.gov/2021044057
DOI: 10.1093/oso/9780197620823.001.0001
9 8 7 6 5 4 3 2 1
Paperback printed by Integrated Books International, United States of America
Hardback printed by Bridgeport National Bindery, Inc., United States of America
Contents
Preface vii
1. Evaluation/Action Research Integrated Framework: Participative

and Collaborative 1
2. Determining a Purpose and a Focus 14
3. Getting Down to the Design: Determining Examination Main

Questions and Appropriate Design 28
4. Qualitative Data Collection and Quantitative Data Collection 60
5. Ethical Challenges in Action Research and Evaluation: Reflecting on Tensions 78
Interlude: Preface to Data Analysis and Informing Data Literacy 89
6. Qualitative Data Analysis 94
7. Quantitative Data Analysis 119
8. Mixed Methods: Integrating Data Analyses to Gain Comprehensive

Understanding of Problems, Contexts, and Stakeholder Needs 140
9. Revisiting the Integrated Model: Focus on Facilitating Social Justice

and Equitable Societies 148
Glossary 161
References 165
Index 175
Preface
Within these opening pages, we share (although briefly) our professional paths into ac-
tion research and evaluation and how this book came about. By giving you insight into
our journey and thinking, we hope that you will be able to better understand our action
research and evaluation integrated model that we propose and present in Chapter 1. The
development of this model was driven by our passion for both modes of inquiry that could
be considered as a framework for organizational capacity building and data literacy. Our
journeys within and through action research and evaluation—and our journeys as scholars,
practitioners, and educators—have taught us that collaboration, critical stance, and multiple
perspectives are essential to creating knowledge for sustainable change in any organization.
This book is a journey to explore the forms of interconnectedness between research
and practice and how this interconnectedness is made possible by our engaging creativity,
pushing the boundaries, and discovering new possibilities. We view action research and
evaluation as the pursuit of understandings and the creation of new knowledge within the
quest for social good. We live in a complex world that requires integration, collaboration,
dialogue, and critical stance to make decisions grounded in relevant and actionable evi-
dence. To borrow from Donaldson’s (2001) view of evaluation as the blueprint of a better
future, we extend this idea to the concept of integrated frameworks, specifically action re-
search and evaluation, to promote inclusivity within our actions and inquiries.
Yet, we find from our experiences that barriers to the integration we describe continue
to flourish because of the increasing divide between research (knowledge created by a few)
and practice (practitioners, leaders, etc.). Within this divide, we find that knowledge claims
generated by practitioners are often discounted. This book, then, is written for the prac-
titioner who wants support in their efforts to (1) do social good; (2) generate new know-
ledge of use, value, meaning, and rigor; (3) use, understand, and implement research and
viii | P r e fa c e
evaluation–based studies and utilize their results for improvement, change, and actions;
and (4) collaborate with stakeholders for sustainability and organizational success. When
our efforts are responsive and responsible (Stake, 2004), we demonstrate that we are com-
mitted to ethics and moral obligations that are integral to our positionality in undertaking
action research and evaluation work.
Our intent is to explore the potential of action research and evaluation by integrating
and situating both modes of inquiry within practitioners’ quest for organizational improve-
ment. As educators and practitioners, our aspirations are for research and evaluation to
serve practitioners and also people internal to and with a particular problem/concern. We
want practitioners to consider and use research and evaluation as a tool to (1) generate rele-
vant and actionable evidence and (2) inform the process and appropriate decision-making.
We argue for the importance and legitimacy of practitioner-created knowledge. Action re-
search and evaluation are human inquiries (Reason, 1994). The nature of the underlying
processes is participative and positive change driven—doing action research and evaluation
with rather than on those involved and impacted. What we do as researchers and practi-
tioners impacts peoples’ lives!
How Has This Book Come About?

Our Positionality
Action research and evaluation are our passions. We both have served as evaluators and
researchers on a variety of projects working with diverse groups of stakeholders internal
and external to organizations. We both belong to the professional communities of action
researchers and evaluators. However, our professional paths have evolved with different
emphases. Linnea’s experiences primarily have been within the action research community
of scholars and practitioners, and Elena’s have been within the field of evaluation. Situated
more within each community, we think and speak as members of that community—that
is, an evaluator and an action researcher. Collaborating on various initiatives has enabled
us to share our views and languages; reflect on strengths and weaknesses of both modes of
inquiry; and explore the possibilities for integration to contribute to the practitioners’ abil-
ities to examine problems in practice, think of potential solutions, and generate and act on
evidence. We, especially, embrace these positions within the context of teaching leadership
and working with the leadership networks that bring various leaders to the table to address
complex problems collectively at the mega (societal) level, macro (organizational) level, and
the micro (departmental) level.
For example, within evaluation one of the debatable issues is related to developing
recommendations. That is, is it the evaluator’s role to provide recommendations? Certainly,
in practice, the answer to that question is contextual, and it depends on the stakeholders’
needs in initiating the evaluation. An evaluator can be internal (part of the organization)
or external (hired to conduct the evaluation). Even with this binary of internal/external,
we cannot remove ourselves from our work as an evaluator, and hence our job is to help
stakeholders understand and utilize evaluation methods to generate rigorous results and
P r e fa c e | ix
implications. As practitioners (e.g., a leader, a member of the organization, and an evalu-

ator), we dive into a situation of which we become a part. We develop relationships; learn;
provide and contribute to expertise; and become intimately engaged and involved with the
context and settings as we plan, design, and implement our studies. Our professional as-
pirations are to facilitate and support the use of the evaluation, needs assessment, and/or
action research–based findings.
Furthermore, within the recent and extended call for evaluation “to give greater
weight to the development of more equitable organizations, governments, and societies”
(Donaldson & Picciotto, 2016, p. vii), one of the questions that we kept asking was, Could
action research provide complementary perspectives as it relates to extending the evalu-
ators’ role proposing courses of action informed by our discoveries to inform appropriate
decisions, facilitate improvements, and contribute to equality and social justice? In 2017, we
presented a conceptual paper to the Action Research Special Interest Group of the American
Educational Research Association (AERA). In that paper, we detailed the need for “meth-
odological plurality” (Koro-Ljungberg, 2012). The AERA 2017 theme was “Knowledge to
Action,” which AERA described as “that in order to address complex problems that we
face, knowledge needs to be constructed (generated) from different perspectives (points
of view).” For our purposes, the points of view we speak to are those of people who may
have been marginalized from knowledge creation—the practitioners, those in the work-
places—facing and solving problems in context. Hence, we position our text for people
in an organization to use action research and evaluation as their tools in an informed and
meaningful way.
Our Perspectives on Action Research

and Evaluation
In the following narratives, we provide our response to questions we created to intro-
duce our perspectives and our guiding values and principles, which in turn constitute our
positionality:
• What is the meaning of evaluation and/or action research to me?

• Who am I as the evaluator/action researcher?
Linnea’s Meaning of Action Research: Empowerment
Action research to me is empowerment. When I can figure out what I need to do to

fix a problem or explore an issue, and I know that I can use rigorously obtained ev-
idence that I collected to support my exploration, then I know that I can also make
x | P r e fa c e
informed decisions about my practice from that work. I was introduced to action
research practice while I was a doctoral student. In my first job after finishing my
doctorate, I worked for a federal grant that included a teacher action research com-
ponent as part of the teacher professional development we provided and supported
with our partner schools. I immersed myself in the readings of Noffke, Reason and
Bradbury, McNiff and Whitehead, and others. I was fascinated with the connections
between action research and empowerment as described by Freire when he wrote
of participatory action research. I believed then, as I do now, that action research is
a venue through which we can empower ourselves through the creation of and dis-
semination of new knowledge that we created to support our own practice. I found
that much of what teachers continue to do is dictated by those in power (i.e., pol-
icymakers, school boards, school administration); and yet, teachers knew best their
own practice, classrooms, and students. Action research provided a way for teachers
to be empowered to speak up with evidence about “what works” with their students
and to have more of a voice in school, district, and state policy.
In 2008, I worked closely with two teachers as they created and implemented
an action research project in their classroom. I noted these teachers’ hesitancy about
their skills in “research” and tried to encourage, mentor, and provide research sup-
port that they could then use to implement their project. As these teachers grew
more empowered with the knowledge they created, they presented their results to
their school, their district, and their region. They not only empowered themselves,
but those around them. I also was hesitant—hesitant to force my ideas and know-
ledge creation on others.
In 2017, I was asked to write a chapter for a handbook on action research
and was given the chapter on Popular Education and Action Research. My work re-
searching for this writing included learning about the participatory, emancipatory,
and collaborative action research of Myles Horton, Orlando Fals Borda, Paulo Freire,
and many others across the globe. Action researchers are reclaiming their own power
through learning to create and disseminate knowledge about their contexts.
In the same manner, as a professor and professional research consultant, I want
to support other practitioners as they seek to find answers (that are grounded in
rigorously obtained evidence) to their questions that are inherent in their own prac-
tice contexts. My work in the professional world has included heavy involvement in
several action research organizations, including serving as the Program Chair for the
Action Research Special Interest Group of AERA and serving on the Executive Board
of the Action Research Network of the Americas.
P r e fa c e | xi
Elena’s Meaning of Evaluation: Quest for Good
When people outside professional circles ask me about what I do and I reply, “I am
an evaluator,” more often than not I sense a pause—inviting additional insights into
what I mean by this. In a sense, we do evaluate all the time. To me, evaluation is in-
separable from our everyday experiences. We do not ascribe a particular term as we
go about daily routines of our lives, necessarily. We just do it! Could it be a profession
as well?! To me, this is the beauty of being an evaluator. My personal and professional
lives are two in and as one. They shape and are shaped by each other.
The field of evaluation is always evolving, dynamic, and fluid. With and within its
processes I grow as well. At times, I think of evaluation as an ocean into which I dive
to engage, express, make sense, and reflect by learning from and with each other.
Evaluation is a human inquiry (Reason, 1994); as such, it shares a sense of com-
munity and belonging. The nature of its underlying process is participative—doing
evaluation with rather than on people. As a human endeavor, the conduct of evalua-
tion is critically reflexive. As fundamental issues periodically resurface in new forms,
they demand new attention as circumstances of our work change, and we face new
challenges (Smith, 2008, p. 3). Addressing those concerns and sharing experiences
cumulatively facilitates our collective, as the professional community, and individual
growth, as the members of this community. This participation also requires curiosity
and creativity to seek new possibilities and try new things and explore their use
within new contexts. To be creative is to engage our imagination, be open to possi-
bilities, and be tempted by their potentials.
The nature of evaluation is cross-disciplinary. To our professional space as eval-
uator–researchers and practitioners we come through different doors—diverse edu-
cation backgrounds and work experiences. Prior to becoming the evaluator, I worked
across various fields—that is, engineering, international development and affairs,
research, industrial relations/human resources, and education. To meaningfully con-
tinue my professional path, I was looking for a “home” that would enable me to
(1) integrate my various backgrounds by further enhancing my analytical and in-
terdisciplinary skills and (2) fulfill my aspirations for being a team player sharing
expertise and contributing to a collective effort to make a difference. Evaluation has
become that home for me! A home that has space for everyone to get involved in
being a part of the community. A home that recognizes and values differences; cap-
italizes on diversity of perspectives; gives voices—looks outward into the world of
human affairs and possibilities (Dimitriadis, 2012).
The participative perspective supports the aspiration for evaluation as social
betterment “driven by an analysis of the potential contribution that the evaluation
can make, in the particular circumstances, to the democratic processes that de-
fine and seek social betterment” (Mark, Henry, & Julnes, 2000, p. 12). I believe in
xii | P r e fa c e
evaluation as a quest for good, when good is the point of a continuous critical re-
flection on how this craft is used within the examination of, and shapes and is being
shaped by, what it is that we are evaluating, within the complexity of the dynamics
of interactions; the quest that is always emerging, in its making. To me, evaluation is
the inquiry grounded in the concept of goodness.
Within its quest for good, the essence of evaluation is to improve—always! That
is, evaluation is never to prove, which is, unfortunately, one of the continued miscon-
ceptions of evaluation. Furthermore, the three recognized virtues of evaluation are
(1) asking critical questions, (2) engaging in a continuous learning, and (3) building
personal and institutional capacity. We can only express them from and with others.
That is, evaluation facilitates engagements and deeper understanding of “reality”
co-constructed with those involved in and, equally important, impacted by the eval-
uation. Evaluation is a dialogical and critically reflective space.
Evaluation is done to initiate and contribute to positive action and change.
Hence, generating actionable evidence of meaning and value is at the core of the
evaluation conduct. The quest for good is the quest for social justice and inclusivity.
It demands taking positions. Speaking truth to power is a moral and ethical obliga-
tion and commitment that we espouse.
Evaluation is a journey. Who I am as the evaluator is an ongoing project. Yet, the
qualities that anchor my reflections and within which I make meanings of my experi-
ences are hope, beauty, and appreciation. Furthermore, working within today’s world
that is changing all the time, being an evaluator is being a systems thinker—that is,
“View matters from multiple perspectives, respect ecological interdependence of things,
and tell a story that connects what is being evaluated with the wider picture” (Ofir, 2016).
Reflecting on our positionalities, we saw the following key points of intersec-
tion between action research and evaluation meanings to us as human endeavors
(professions). Specifically, both inquiries are
• driven by passion and empowerment;

• centered in collaboration, participatory nature;
• grounded in ethics;
• constituting professional work identity;
• creating knowledge—learning from and with each other;
• enabling a sense of belonging and community building;
• facilitating capacity building;
• emphasizing cultural sensitivity;
• providing access to contexts and power in contexts; and
• struggling for legitimacy (randomized controlled trials vs. other designs).
Each of these points informed the development of a framework, introduced here and
in Chapter 1. We integrate action research and evaluation and illustrate the frame-
work throughout this text.
P r e fa c e | xiii
What Is the Need for an Integrated Model?

We position the textbook within the following needs:
1. We need an integration of action research and evaluation to examine possibilities for a

more engaged work and alternative views of knowledge in internally driven and context-
specific settings.
2. We need an integration so that problem-solving and decision-making processes in the
contexts are action and improvement focused.
3. We need this textbook to help all practitioners at all levels of the organization learn
data literacy; learn how to create knowledge; and engage in collaborative, data-informed
decision-making.
4. In both evaluation and action research, internal users must identify problems (chal-
lenges) to be examined and connect those problems to organizational improvements
within holistic and systems-oriented approaches. Our premise is that action research
and evaluation complement and complete each other by sharing many attributes, which
in turn enables this type of engagement.
5. There is a need for a textbook to address the problems of practice and to position practi-
tioners’ knowledge claims as legitimate and equitable within the quest for organizational
improvement, change management, and sustainability.
6. Leaders, particularly servant leaders, need to be able to facilitate (not direct) practitioner
learning and practitioner knowledge creation so that all can own the problems identi-
fied, the knowledge created, and the decisions made that were informed by their own
data in context. This role could be formal or informal. For example, an internal evaluator
who may be a member of the data assessment and evaluation unit or a member whose
title may not be a researcher or an evaluator could lead this effort within an organization.
Audience
One rationale for this book is to provide support to practitioner–leaders in organizations
who are trying to solve real problems and use data-informed decisions to do so. We believe
this is a critical need in this era of big data, multistakeholder organizations, and diversity
of representation. We also believe that there is currently an anti-scientific movement that
seeks to undermine the rigor and science behind decisions. We want to help practitioners
understand that they can create, understand, analyze, and use data to inform decisions and,
equally important, to facilitate practitioner research-driven processes and together engage
stakeholders in the ongoing work of organizational improvement. Collaboratively, leaders
and stakeholders can take ownership of problems and solutions.
A second rationale considers the growing demand for online graduate programs
and certificate programs for practitioner–leaders. We are seeing increasingly more applied
doctorates and certificate programs in research and evaluation, in which the need for data
xiv | P r e fa c e
literacy is clear. Most EdD programs include at most three research courses: a general qual-
itative course, a general quantitative course, and a specific design course. Many applied
doctorates are also promoting program evaluation and action Research as suitable designs
for applied dissertations. The same trends appear to be characteristic of various certificate
programs for practitioners. This book could be a bridge between research and practice to
help students make the connection between good leadership practice and good research
practice (as intertwined).
Finally, our textbook’s orientation on critical thinking, reflection, ethics, and equity
will enable students to extend their thinking about research designs within the vision of re-
search as a quest for social good and positive change. Our approach to bring action research
and evaluation into dialogue could encourage students, practitioners, and researchers/
evaluators to view research as a creative process. We position our content within the argu-
ment practitioners can and do create legitimate knowledge. We encourage practitioners and
learners to push established boundaries and explore what is possible—what the established
ways of doing enable and preclude us from seeing and why (Dimitriadis, 2012). This is the
essence of our engagement with inquiry to address problems/concerns that we face.
Book Organization
This book is organized into nine main chapters. Our writing is informed by a proposed ac-
tion research and evaluation integrated model that shapes how we think and go about (i.e.,
theoretical and methodological frameworks) studying what is of interest to us. This model
or paradigm serves as the foundation on which all other chapters’ contents are developed.
Hence, we suggest reading the chapters in order.
In Chapter 1, we present our integrated framework and rationale. We introduce ter-
minology related to action research and evaluation, and we explore tensions and connec-
tions between the two frameworks.
In Chapter 2, we explore the nature of programs and contexts and how we discover
and address problems or needs from within. We briefly address the idea of data literacy
using theory from both the action research and evaluation field to address a variety of prob-
lems. This information will assist the reader with identifying the gap or need and with
choosing an approach to explore that gap or need in the reader’s own context. We introduce
the logic model, a visual representation of a program, and show how a logic model can
assist with both problem identification and research approach. We emphasize the impor-
tance of (1) clear problem identification and (2) inclusion of stakeholders. We explain our
positionality with regard to problem identification and research approach, and we provide
vignettes from the field.
In Chapter 3, we explore the nature and components of various research designs and
supporting methods. We explore the nature of research problems in more depth and how
various designs can better answer specific problems. We caution practitioner–researchers
to deeply reflect on problem identification. Often, we can be persuaded to set aside our own
goals to address the priorities of those in power. Although at times both goals may coincide,
P r e fa c e | xv
at other times those goals may conflict, and we need to prioritize problems and needs that
are central to our context. Stakeholder inclusion (including those in power and those not in
power) can help ensure that all understand the interconnected and relational importance of
prioritizing needs and gaps. Rigorous, trustworthy designs that include the voices of all af-
fected can ensure the validity and utility of results. Last, we address the importance of being
transparent about practitioner–researcher positionality throughout the process.
In Chapter 4, we speak to the practical issues of data collection methods. We show
how a study’s research questions should be the driving force behind the choice of data col-
lection methods, and we explain the utility/supportive nature of a variety of qualitative,
quantitative, and mixed methods resources. We include our vignettes of practice in using
diverse methods.
In Chapter 5, we address ethical considerations from participant respect to institu-
tional review board (IRB) approval. We discuss procedural ethics and practice ethics. We
explore the nature of standards in the evaluation and action research field. We explain the
nature of the IRB as a historical, international, and federal entity designed to guide the
ethical treatment of humans within research. We connect the idea of ethics to our own
positionality and include vignettes from our own work.
Following Chapter 5, we provide a brief interlude as we prepare to address data
analysis.
In Chapter 6, we explore analyzing and interpreting quantitative data, connecting data
analysis to stakeholder inclusion, positionality, and ethics.
In Chapter 7, we explore analyzing and interpreting qualitative data, connecting data
analysis to stakeholder inclusion, positionality, and ethics.
In Chapter 8, we discuss the importance of returning to your purpose in a mixed
methods study (either predetermined or emergent) to explore ways that your mixed types
of data can help you examine your study purpose and questions.
Finally, in Chapter 9, we address the need to implement (utilize) and share the results
of our inquiry for the improvement of practice, reminding readers of main points we cov-
ered throughout the text. We also address the philosophical nature of the improvement of
contexts and of societies. We provide support for imagining societies that value continuous
improvement and value learning leaders and organizations that seek to improve organiza-
tions using data and new knowledge to support sustainable change.
Guiding Questions as an Invitation to

Pause and Reflect
Before you join us on the journey to reflect on your experiences and explore new possi-
bilities for action research and evaluation within your practice context, we invite you to
take a few minutes and capture in writing the meaning of action research and evaluation
to (1) you and (2) your organization or settings of your current work as a member of your
organization/community. Writing is a way of knowing and doing (Rossman & Rallis, 2017).
We then would like you to keep your initial reflective narrative as a living document to refer
xvi | P r e fa c e
back to and to engage with our shared ideas actively and critically. The intent of our writing
is not to prescribe but, rather, to describe our experiences and enable a space to dialogue
and to learn: What makes sense to you? What might surprise you? With what positions might
you agree? With what positions might you disagree? Why?
We invite you to read and reflect, working to make connections from our ideas and
examples and vignettes to your own work. We read to learn, understand, and experience.
We generate knowledge, see and relate to the world around us within a new light, and have
the ability to express ourselves differently. Learning is relational from within and through
others. In that vein, we provide the following questions for ongoing reflection as you read
this book, with additional critical questions at the end of each chapter:
1. How has your organization/institution/company/unit/team held you accountable?

2. What do you bring to the table of leadership and/or decision-making processes in your
current position?
3. How can you influence (impact) the culture and/or structure of your organization/insti-
tution/company/unit/team?
4. Where do opportunities for action research and evaluation, individually and combined
(integrated), lie for your organization currently and in the future?
5. What role do action research and evaluation play for organizational change? For
leadership?
1
Evaluation/Action Research
Integrated Framework
Participative and Collaborative
In this chapter, we introduce the idea of a new framework for research. We define our
framework as a model for practice that can be used by people within an organization—
leaders, colleagues, practitioner–researchers, and evaluators—to improve practice and
solve problems in context. This book is about providing support for the thousands of people
in organizations who are tasked with finding and solving problems, whether they are formal
internal evaluators or not. The book will enhance organizational capacity by helping them
understand and utilize this model. Our goal is to introduce this integrated framework in
a way that readers can see its connection to their settings and apply the underlying shared
positions and principles to their own work. Finally, the described model serves as the foun-
dation on which all other chapters’ contents are developed.
Chapter Learning Outcomes

• Develop working definitions of action research and evaluation focusing on data literacy
for an equitable society.
• Understand the role of action research and evaluation for practitioner-driven inquiries to
examine problems in context.
• Explore points of intersection between action research and evaluation.
• Learn about the proposed action research and evaluation integrated framework
(Figure 1.1).
• Reflect on the potential value and meaning of this model for your work context and
settings.
Evaluation and Action Research. Linnea L. Rademaker and Elena Y. Polush, Oxford University Press. © Oxford University Press 2022.
DOI: 10.1093/oso/9780197620823.003.0001
2 | E va l u at i o n and Action Research
Problems in
Context
Action Research
& Evaluation
Ethical Thinking
Theory and
Methods
Stakeholder
Inclusion for
Sustainable Change
and Improvements
Applicable, Relevant,
Actionable, Rigorous,
Trustworthy Responses
to Contextual Problems
F I GUR E 1.1 Integration of action research and evaluation.
Everyday practitioners are expected to use, collect, and make sense of data to help them
make decisions and to solve problems. At times, practitioners and leaders may have
hired external evaluators to conduct needs assessments, program evaluations, or as-
sist in other evaluation activities. However, we believe that in today’s world of budget
constraints and calls for efficiency, practitioners can and should undertake these ac-
tivities, either themselves or with other colleagues at work. We also believe that in
today’s data-driven climate, it is critical for organizations to develop evaluation and re-
search capacity to address complex problems using organizational data. Frequently
throughout this book, we use vignettes— short, descriptive illustrations—
in order to
provide concrete examples of ideas we introduce. In Vignette 1.1, you will meet Martha
and read about how she worked with external evaluators to build her organizational
capacity.
E va l u at i o n / A c t i o n R e s e a r c h I n t e g r at e d F r a m e w o r k | 3
Vignette 1.1: Learning to Evaluate My Context
Martha (a pseudonym) was the associate director of a small nonprofit organization.

Martha was in charge of implementing initiatives at her site, each of which was
funded by a national grant program. Martha and other employees wrote the grant
that the nonprofit received to fund these initiatives. Martha wanted to implement a
program evaluation but didn’t believe she had the expertise to do so. Martha con-
tacted an evaluation consulting firm to help her evaluate the effectiveness of these
initiatives in achieving their overall goals. Martha shared with the consultants that
she thought the initiatives weren’t as effective as they could be and that she believed
the staff wasn’t implementing them as designed. However, Martha’s grant budget
was not sufficient to hire an external evaluator. The consultants suggested that they
could serve as mentors to help Martha implement her own program evaluation.
In Vignette 1.1, you met Martha, the associate director of a nonprofit. Martha repre-
sents practitioner–researchers, like those we referenced in Figure 1.1. Like many who work
to direct the day-to-day activities of various nonprofits, Martha did not believe she had the
skills necessary to conduct her own program evaluation. Through her work with the con-
sultants, Martha implemented her own evaluation, with the consultants acting as mentors.
You will read more about Martha in upcoming chapters as you learn how research, evalua-
tion, and action research can be implemented by practitioners in everyday settings.
Now, let’s examine the components of our framework as outlined in Figure 1.1.
First, people in organizational settings frequently encounter problems that need solving.
They can identify the problems themselves by conducting a needs assessment or by ana-
lyzing a recurring issue or gap that an organization needs to address. The people who
discover these problems are what we term practitioner–researchers because often the
people most affected by problems must also solve those problems, or fill the need, or fix
the gap. Second, we believe both action research and evaluation are useful in helping
the practitioner–researcher solve these problems. Last, practitioner–researchers who
are intimately involved and affected by problems in context are also the most involved
stakeholders who can ensure the utilization of solutions discovered through system-
atic inquiry. We explore the proposed framework further throughout the chapter,
explaining how action research and evaluation can be integrated to serve the practi-
tioner–researcher in leading change.
Here, we want to take a moment to discuss the term research within the context of
organizational improvement. “Research” can be an intimidating idea to many people.
However, when we talk about organization research and practitioner research, we are re-
ferring to an investigation of human experience within a specific context. We conduct re-
search in order to learn new things; understand an issue/problem; or evaluate our decisions,
policies, or programs. We want to know if a new policy implemented in our department
will help the department be more productive and further the success of the organization.
Or, we want to investigate whether implementing a new procedure in a classroom can help
students achieve better learning outcomes. We believe that a new framework, an integrated
framework, can help more practitioners undertake research that is rigorous and that pro-
vides new knowledge and evidence to support their work.
Why Do We Need a New Framework?

We believe this is the time to have discussions about work and life and how we can use
research to support both. The essence of research is to contribute to betterment across
all domains within a particular society, as well as globally. Our writing is inspired by
the need for methodological plurality and creative, imaginary engagement with research
(M. Greene, 1998; Koro-Ljungberg, 2012) within practice (or context). We argue that the
concept of integration is a quality that distinguishes the way we conduct research and the
tools we use to support us in our research. In this chapter, we offer a dialogue on inte-
grating action research and evaluation to examine possibilities for a more engaged work
and alternative views of knowledge in internally driven and context-specific settings. In
action research, the internal is the practitioner; in evaluation, the internal is the internal
evaluator.
Integration in research is not new. For example, McKeown, Fortune, and Dupuis
(2016) explored the possibilities of utilizing an appreciative mode of inquiry within action
research. Action research and case study are both used to emphasize the uniqueness of a
program or organization in context. R. Smith (2016) explained that the methodological
plurality was a central premise of action research. Using Deleuze’s philosophy of art, in
which the use and mechanics of something are more important than its meaning, Smith
contended that when using art-making in action research, we can remain “open to new
methods” and we can avoid “the fixed nature of many traditional forms of research [which]
posits the potential for new ways of acting and thinking especially through artmaking”
(p. 43). In the evaluation field, Thomas and Parsons (2017) wrote about their divergent
approaches to evaluation—one a culturally responsive approach and the other a systems
theory approach—illustrating how they merged the two approaches in order to better
serve their clients in the evaluation of a science, technology, engineering, and mathematics
(STEM) education project. The authors mapped out the characteristics of each approach
and showed how the questions suggested by each approach could help shape a more robust
evaluation. Similarly, we contend that by integrating action research and evaluation, we can
embrace the strengths of each approach while acknowledging the challenges or limitations
of each. We hope that by integrating, we can resolve some of these challenges and limita-
tions. Integrating different strategies engages our creativity and imagination, allows us to
improve programs, and, equally important, facilitates our learning as we solve problems
within our contexts. In the next section, we offer some practical considerations about terms
that we use throughout this book.
Acknowledging Language as a Potential

Barrier: Working Toward Practicality and
Clear Explanations
As we work to clarify terms, procedures, designs, and methods, we acknowledge that lan-
guage and the jargon of the field may be a barrier to some readers. We provide definitions
of terms (and multiple definitions, where applicable) as needed, as well as a glossary. We
recognize that multiple perspectives within the field can result in multiple legitimate defin-
itions, so our choice reflects part of our positionality within a particular field. We encourage
readers to explore suggested resources (at the end of each chapter) to help develop their
own positionality and understanding of the tools of the field.
Defining Action Research

We synthesize a variety of definitions from scholar–practitioners, believing that action re-
search is a model for problem-solving in context, including taking action from the results
of our inquiry. Pine (2009) also asserted that action research was a paradigm—a model
through which practitioners could study their own contexts and practices, using a variety
of methods. The type of democratic change Pine described could only be achieved through
the collaboration of those who identify and are affected by problems in context. Reason
and Bradbury (2008) provided a rationale for contextualizing problems and described ac-
tion research as “an orientation to inquiry that seeks to create participative communities
of inquiry in which qualities of engagement, curiosity and question posing are brought to
bear on significant practical issues”(p. 1). Swantz (2008) described participatory action re-
search as democratic, political development. Swantz asserted that it is “crucial that research
is not separated from life” (p. 45), further confirming the need for understanding context.
Cochran-Smith and Lytle (2009) encouraged teacher research as a way to move knowledge
about education out of the academy and the policymakers and into the classroom, where
those who were affected by problems and policies could create the knowledge needed for
change. Carr and Kemmis (2009) explained critical action research as necessary in a “dem-
ocratic society committed to extending opportunities for all citizens collectively to shape
the future of their society” (pp. 77–78) by creating new knowledge.
Defining Evaluation
Our definition of evaluation is rooted in the work of several key theorists. Essentially, eval-
uation is a form of inquiry that seeks to use rigorous data collection to determine the value,
merit, or worth of an intervention, a solution, or a program. Although there are many def-
initions of evaluation that signify the diversity of its historical roots, conceptions, and fields
of practice, Newborn (2001) suggested Patton’s (1981) definition could be useful. Patton
(1981, as cited in Newborn, 2001) placed the focus on the essential qualities of evaluation;
specifically, evaluation is
the systematic collection of information about the activities, characteristics and

outcomes of programs, personnel and products for use by specific people to reduce
uncertainties, improve effectiveness, and make decisions with regard to what those
programs, personnel, or products are doing and affecting. (p. 7)
Evaluation is also defined from the points of inspiration, commitment, and professional re-
sponsibility and ethics that engaging with this craft brings. For example, Donaldson (2001)
spoke of evaluation as a helping profession. Schwandt (2002) viewed evaluation from the
point of moral discourse. Finally, Mark, Henry, and Julnes (2000) defined the purpose of
evaluation as serving social betterment. As shown in Figure 1.1, all of these aspects are
critical and are combined with the internal, connected nature of our work as practitioner–
researchers within our context.
Defining Data Literacy

We define data literacy as a person’s ability (i.e., knowledge, skills, and experiences) to read,
understand, create, analyze, and think critically about data to make informed decisions for
appropriate organizational actions.
Defining Organizational Capacity Building

We define organizational capacity building as having an organizational climate that supports
the development and use of the research and evaluation knowledge and skills.
Defining Equitable Societies

We define equitable societies as societies that are diverse in makeup and inclusive in decision-
making. Equitable societies value social justice for all members and make data-informed
decisions considering potential consequences for their members, and they also think criti-
cally and reflectively to make decisions that benefit all members.
Action Research and Evaluation:

Creating Actionable Results to Inform
Decisions
Our intent is to give leader–practitioner–researchers the integrated tools to support their
work in their context. Leaders must judge the merit and worth of programs and policies and
must solve problems in context. Leaders must use data (either that they collected or that
were collected by others within or outside their organization) in order to make decisions
about policies, programs, and people. Let’s take a look at each paradigm—action research
and evaluation—and see how they historically have been used by leaders and stakeholders
to help solve problems in context. We introduce each separately and then show how an in-
tegrated framework can strengthen decision-making efforts.
Action Research as Problem-Solving Paradigm

Action research is considered by many to be a practitioner-driven approach to solving
problems in context (Pine, 2009). Some have asserted that when practitioners or internal
evaluators implement action research in their own practical contexts, either rigor or the-
oretical framework may suffer due to the practitioner’s level of expertise (e.g., Coghlan &
Shani, 2014; Davison, Martinsons, & Ou, 2012). This perception of practitioner–researchers
as unable to make knowledge may stem from the ancient idea that only trained researchers/
scholars can create knowledge (Levin & Greenwood, 2008; McNiff & Whitehead, 2009).
We assert, however, that action research is an internally driven paradigm of organizational
improvement, much like internal program/organizational evaluation. The person who is at
the center of that context is the best choice to investigate and work to improve practices be-
cause insiders usually have a deep connection to the context—emotionally, professionally,
and financially. In addition, to support robust problem-solving in context, if we view action
research as a framework (a model for problem exploration), then we are not limited to only
qualitative research designs but can include quantitative designs and mixed methods de-
signs (Efron & Ravid, 2020; Ivankova, 2015; Mertler, 2017). Expanding available method
choices allows for a variety of approaches to problem exploration and problem-solving.
Also, when practitioner–researchers choose the design and method that best support the
problem at hand, then they can also work to show rigorous and applicable results.
Evaluation as Problem-Solving Approach

Dahler-Larsen (2012) stated that we live in the age of evaluation and described the growing
prevalence of evaluation practice in organizations across disciplines and throughout the
world (Schwandt, 2015). Dahler-Larsen detailed examples of evaluation from education to
nonprofits, healthcare, and business contexts. The author explained the diversity of the eval-
uation field, discussing auditing, accreditation, and certification—methods of assessment
and evaluation—and suggested that all could be subsumed under the field of evaluation as
examples of practices that included an “evaluand, values or criteria, knowledge or methods,
and use” (p. 11). Interestingly, these coincide with action research practices and definitions,
which require action or use. Mertler (2017) reiterated and explained the action research
stage of “developing an action plan” (p. 219) from the results of action research data col-
lection and analysis. Basically, we are saying, as others have said, ask yourself, “What do
I do now? How can I make my context better with this new knowledge?” (Ivankova, 2015;
Pine, 2009).
Scriven (2003) defined evaluation as a transdiscipline, which means evaluation as a
discipline naturally crosses the boundaries and is used by others in diverse disciplines and
that these disciplines serve the public as they are used by evaluators to improve programs
and services (pp. 19–20). Although evaluation is inherently a practical field designed to
be used to solve social problems, theories do guide our conduct and our design. Scriven
and others noted the various uses and purposes of evaluation (Fitzpatrick, Sanders, &
Worthen, 2011), such as “facilitating social justice” agendas (Donaldson & Picciotto, 2016,
p. vii) within local and/or global settings. Yet, as Weiss (1998) argued, what they all have in
common is “the notion of judging merit” (p. 3). In addition to merit questions, evaluators
also ask “Why?”;for example, “Why are these outcomes important?” and “Why did the pro-
gram staff choose this intervention?”
Action Research and Evaluation: “Action”

at the Center
Evaluation is a locus of action (Mel Mark, personal communication, July 7, 2016), which
means that the consequences of evaluation are important for practitioner–researchers to
reflect upon. Taking action and making decisions (or not taking action) as a result of the
evaluation are significant to the practitioner–researcher and to those who are invested
stakeholders of the program/problem evaluated (Mark & Henry, 2004; Patton, 2008).
Therefore, practitioner–researchers should first gain an in-depth understanding of what
is to be evaluated (i.e., program or process) in order to generate evidence of merit and use
of both the program and why there is a need to evaluate it. In other words, we need all in-
vested in the program to share their understanding of the program and what the needs are
in the evaluation. So, too, action research anchored in the word “action” implies a stance
of taking positions, actions, and making decisions based on what we find. “If there’s no ac-
tion, then it’s not action research” (Jean McNiff, ARNA, 2015, Toronto, Ontario, Canada).
Practitioner–researchers can use both modes of inquiry to help create rigorous, actionable
results.
Action Research and Evaluation Addressing

Relational Problems
Evaluation and action research are relational processes, which means that the
purpose, conduct, and outcomes of evaluation are dependent on the dynamics of
human interactions. Although evaluation may be focused on problem could be not
knowing whether a policy is effective. Or, we could institute a new program to
address student retention, but if we do not evaluate it, we have the problem of
not knowing if we improved student retention and not knowing if we should keep
the program (which may have had significant financial cost for wing if we should
keep the program (which may have had significant financial cost for the organiza-
tion). Hence, considering and critically reflecting on tensions, the complexity of
dynamics of relationships, and diversity of perspectives are paramount. Therefore,
in both evaluation and action research, internal users must identify the problem
to be examined and connect that problem to total system/organizational improve-
ment, including the human element. Collaborative inclusion of stakeholders al-
lows for greater ownership of the problem and the solution(s). Stakeholders select
methods that are appropriate to the problem and specific context and also the needs
of the organization that specifically address the identified problem. In Vignette 1.2,
Martha considers the human element as she works to evaluate programs funded by
a grant.
Vignette 1.2: Identifying the Actual Problem
As we return to examine Martha and her work at a nonprofit, we listen to Martha

describe the problem she identified as not understanding why the implemented
programs were failing to result in improved identified outcomes. The evaluation con-
sultant asked Martha if she was implementing all of the programs in the same way,
and she indicated that other staff members were responsible for varying elements
of implementation. Martha’s response led the consultants to ask further if she had
queried the staff to determine staff perceptions of (1) the grant purpose,(2) the grant
programs, and(3) staff importance to the success of the grant. Martha discovered
a disconnect between staff perceptions of inclusion or non-inclusion in the grant-
writing and implementation process and also miscommunication as to the impor-
tance and processes of the grant. Staff also indicated diverse ways of implementing
programs in the grant, which Martha had not considered, leading her to believe that
the programs were not the problem. Instead, she needed to address the problem of
staff communication and staff “buy-in” before assessing and evaluating the success
or failure of the programs.
The point we are making is that inclusivity is a complex construct. When we work to be
inclusive, we gather tensions, politics, and diverse values. Yet, inclusivity is at the core of
action research and evaluation. Some may doubt the value of inclusivity because they may
fear the inability of stakeholders to provide rigorous evidence or that stakeholders may have
conflicting viewpoints from those of the leaders or those who enacted programs and pol-
icies. However, the consequences of not including stakeholders are vast. Cousins and Earl
(1995) argued (and have continued to argue) that stakeholder inclusion and collaboration
are key to success in organizational improvement. As they wrote about educational eval-
uation, the authors claimed that a participatory approach is more “likely to be responsive
to local needs” (p. 9) and is “problem solving in professional work” (p. 10), which from a
systems thinking perspective can provide more informed decisions that are sustainable and
achieve total organizational buy-in (Stroh, 2015).
Points of Intersection and Tensions Between

Action Research and Evaluation
The essence of our proposed framework is integration. Integration is not just mixing
(Morse & Niehaus, 2009). We think of integration as the process of bringing the two modes
of inquiry into a dialogue—a constructive and critical dialogue. What do we mean by this?
It requires analytical thoughtfulness to examine compatibility (whether, indeed, the two
or more “things” can be brought together; see our previous discussion) and aspects of
complementarity.
TABLE 1.1 Points of Intersection and Tension in Integrating Action Research and Evaluation
Points of Intersection Action Research Evaluation Critical Questions as Opportunities for Integration
Purposes Community empowerment Origins include governmental Have we identified the significant problem to address?
(worker, teacher, self) accountability and efficiency Has anyone tried to co-opt identified purpose?
Problem-solving in context Internal/external judgment of Who is at the table and who is not at the table in terms of driving and
User-driven problem, value/merit/worth supporting identified priorities?
purpose identification Internal and external
stakeholder problem
identification
Nature Problems identified from Driven by the needs within a Are the mega needs (societal) designated as priority over macro and micro?
(essence): mega (society), within; solutions proposed particular organization (micro How are societal needs (mega) considered?
macro (overarching from within; problems or macro) How can all stakeholders understand the integral nature of all three?
organization), and investigated from within
micro (department, unit, (micro)
community, or program)
Theory informing purposes Questions initiated from Questions initiated from How does theory impact evaluation/AR identified purposes, design, and
and design implicit or explicit theory implicit or explicit theory implementation?
Program theory Theory implied, but not Logic models are visual How do research purposes impact theory (design and SBS theory) choices?
Evaluation design theory always at the forefront representations of program How do research purposes imply research design theory (i.e., quasi-
(i.e., appreciative inquiry, theory. experimental design is implied by looking at impact)?
critical participatory How does past research inform all of these choices?
evaluation, or
deliberative, democratic
evaluation, etc.)
Social and Behavioral
Sciences (SBS) theory
Orientation toward action Action is imperative. Action is imperative. How can we ensure the usage of my AR/evaluation results?
and improvement Cyclical nature of ongoing Utilization of evaluation and Do those in power want to co-opt results (take control, change outcomes
assessment of the results to improve practice is and meanings)?
problem and of new implied. What have we learned in this process (critically reflective stance)?
contextualized solutions
is implied.
Validity: What is valid AR? In AR, practitioners view In evaluation, evaluators and What is stakeholder credibility?
What is valid evaluation? validity as adding value and stakeholders contribute to the How can we balance stakeholder credibility and scientific credibility?
improvement to practice. concept of what is valid. How can we show the rigor/validity/trustworthiness of our work?
Design diversity Qualitative, quantitative, Qualitative, quantitative, mixed Do we privilege one research design or method over another?
mixed methods, methods, multimethod How can we ensure that we are using appropriate designs to generate
multimethod credible evidence?
How does our design fit with our purpose?
Participatory (stakeholder Practitioner-driven; Stakeholder (broadly defined) What are the intersecting relationships among stakeholders and among
inclusion) stakeholder inclusion; driven approaches mega, macro, and micro contexts within the organization?
mega, macro, micro Power dynamics can drive Who is at the table and who is not at the table?
contexts stakeholder inclusion. How do stakeholder relationships make research complex?
Ethics (actions) and moral Reflective ethics Reflective ethics What are the ethical considerations of our purpose, design, and
stance (positionality) Reflective practitioners Reflective evaluators procedures?
Reflecting on the Telling the story that data tell How are we considering impact to participants and stakeholders from our
consequences you vs. what stakeholders AR/evaluation and from the resulting actions?
Reflecting on stakeholders’ want to hear How does our positionality impact our design choices and implementation,
preferences Reflecting on the consequences as well as results dissemination?
Reflecting on stakeholders’
preferences
AR, action research; SBS, social and behavioral sciences.

Recall that the premise of our framework is to explore and build on the strengths of
both action research and evaluation. Table 1.1illustrates our thinking about integration. We
highlight points of intersection that can facilitate the integration of the two approaches. We
also highlight unique and shared tensions in the implementation of either approach. The
points we include here reflect the steps of action research and evaluation. Although each
aspect might have tensions, we critically interrogated those tensions and considered them
as opportunities for improvement of methods and outcomes. Our analysis is not meant to
be all-inclusive but, rather, illustrative. Examining the points of intersection illuminates
strengths that allow for integration. As you look at Table 1.1, having read our previous
rationale, we ask you to think about where you might have critical questions within your
organization that could benefit from the integrated approach.
Conclusion and Preparing for What

Comes Ahead
We have already shared two vignettes about Martha’s experience culled from our work
in order to illustrate specific points in this chapter. Because we are both practicing evalu-
ators and action researchers, we believe that our experiences enable us to engage with the
sites and the stakeholders to improve knowledge democracy and knowledge mobilization.
Dahler-Larsen (2012) argued that “the complexity of the unfolding social world is always
greater than our attempts to understand it” (p. ix). Our main argument is that there is much
to learn from practitioners, insiders who are best positioned to create knowledge in the
field, integrating knowledge about and our experiences with both modes of inquiry. We can
benefit from using practices and theoretical models from evaluation and the practitioner-
initiated cyclical improvement stance from collaborative action research and participatory
action research models. By being practical fields, action research and evaluation are not
technocratic but, rather, eclectic, dynamic, and constantly evolving by being open to possi-
bilities. Creativity and curiosity are central to our engagement (Patton, 1981).
As you read through the following chapters, we encourage you to think of integrating
the action research and evaluation framework into your own practice as you seek to im-
prove and to solve problems. We have provided critical questions at the end of this chapter
(and at the end of each of the following chapters) that will help you apply the concepts we
have addressed.
Critical Questions
1. What is your role as a practitioner (e.g., a leader, a member of your organization, an eval-
uator, a data analyst, an assessment specialist, or a researcher) in your context?
2. Is your context a microcontext (department or small element), a macro context (the or-
ganization as a whole), or a mega context (how the organization fits within society)?
3. How do you determine who the stakeholders are? What strategies would you use to in-
clude stakeholders in identifying a clear, prioritized problem in your context?
4. If you already have an identified problem, consider if your problem is more an action
research problem, an evaluation problem, or a combination of both? How might you
integrate the methods of both inquiries to improve practice (thinking forward toward
Chapters 2 and 3, in which we discuss integrating methods into the design)?
5. If you are still thinking about a purpose, how can you include stakeholders in the setting
to help identify and prioritize the needs of the organization?
Return to the narrative you wrote in the Preface in which you addressed the following
questions: What makes sense to you? What might surprise you? With what positions might
you agree? With what positions might you disagree? Why?
• How have your answers changed after reading this chapter?

• What would you add to your narrative?
• What would you take away?
Additional Chapter Resources

Defining action research: Several authors have attempted to define action research, but
most recently, Rowell, Polush, Riel, and Bruewer (2015) conducted a Delphi study
to garner action research experts’ vision of defining action research.
Defining, planning, and conducting evaluation: Better Evaluation (https://www.bette
revaluation.org/en) is an international collaboration- based site that provides
a wealth of useful and informative resources comprehensive of all phases of the
evaluation process and its essential steps with the focus on improving the evaluation
practice. Resources include theories and practical applications (i.e., methods and
approaches).
EvaluATE: EvaluATE is an invaluable open access resource aimed at educating about
and promoting evaluation. EvaluATE is the evaluation support center for the
National Science Foundation’s Advanced Technological Education (ATE; https://
www.evalu-ate.org/about). See, for example, the Center Comprehensive Evaluation
Quick Reference Guides at https://www.evalu-ate.org/quick-reference-guides.
Action Research Special Interest Group of the American Educational Research
Association: This special interest group within a large research organization is a
community of action researchers from throughout the world who share resources
and ideas (https://sites.google.com/site/aeraarsig).
Action Research Network of the Americas (ARNA): ARNA (https://arnawebsite.org)
is a collaborative action research organization that includes members from all
the Americas (North America, South America, and Central America) as well as
partners throughout the world.
American Evaluation Association (AEA): AEA 365 (https://aea365.org/blog/about) is
a dynamic blog-based site that highlights valuable tips, experiences, resources, and
lessons learned by evaluators throughout the world.
2
Determining a Purpose and

a Focus
How is it that we know so much? Yet, so little is done?

—Jean McNiff (personal communication, March 2013)
In this chapter, we examine the steps in defining and analyzing a problem and focusing our
examination on that problem in context. We utilize both action research and evaluation
frameworks to develop a logical path or illustration of how and why we will both identify
and address a problem or issue. We also explore the role that prior knowledge and con-
ducting needs assessment play in this process. We discuss needs assessment as a compre-
hensive strategy within evaluation and action research. We introduce the concepts of theory
of action and logic models to show the connection with the problem—that is, how theory
of action and logic models can (1) be used to identify the problem and shape its statement
and (2) serve as pathways to determine outcomes or impact and inform planning and im-
plementation of an intervention. We emphasize the importance of clear problem identifi-
cation and stakeholder inclusion. We conclude the chapter with our (action researcher and
evaluator) positionality pertaining to problem identification.
When we revisit the term positionality, which we discussed in the Preface, as we will in
this and each subsequent chapter, we believe that it is important to integrate reflection on our
positionality into every aspect of creating a study. Positionality refers to our stance (experi-
ences, values, and beliefs) and our relationship to the context (power structures) and how these
contribute to our choice of problem, our inclusion of stakeholders, our integration of methods,
and our pushing against the status quo. In a sense, it could be viewed as the process; that is,
positionality permeates every aspect of our work as practitioner–researchers (Figure 2.1). Two
stances that we take, as mentioned in the Preface, are empowerment and research “for so-
cial good.” Both are related to the evaluation and action research purposes (mega, macro, and
micro—which purposes evaluation and action research should address) and interests (power
constructs—whose interests evaluation and action research should serve) and how we view our
role(s) and the relationships that we build as internal practitioner–researchers (J. Greene, 2006).
DOI: 10.1093/oso/9780197620823.003.0002
Determining a Purpose and a F o c u s | 15
Finding problems in context

requires stakeholder collaboration
Problems in
Context
Action Research
& Evaluation
Ethical Thinking
Theory and
Methods
Stakeholder
Inclusion for
Sustainable Change
and Improvements
Applicable, Relevant,
Actionable, Rigorous,
Trustworthy Responses
to Contextual Problems
F I GUR E 2.1 Integration of action research and evaluation: Focusing on the problem.
Chapter Learning Outcomes

• Examine the connection between problem, purpose, and questions.
• Begin to develop a logic model representation of the problem and purpose (to be ex-
panded in Chapter 3 and 4).
• Engage with prior research to discover what is already known about the problem and
what is unknown.
• Draft a problem statement that will continually drive your purpose.
• Draft a purpose statement aligned with your problem statement that suggests design,
methods, and scope of your study.
• Connect your problem and purpose to positionality.
Any study is intentional and framed within certain perspectives, whether implicit or ex-
plicit. That is, our studies are driven by our purpose and main questions—intent. Figure 2.2
General Topic and Problematic Situation
Specific Problem/Need/Concern/Gap
in Context
Purpose of Examination
Main
Question(s)
and/or
Hypotheses
F I GUR E 2.2 From general topic/problem to focused research questions.
shows a standard problem statement development that works from a general topic, and a
general problem, funneling down to a specific problem (i.e., a gap in knowledge or a con-
textual need) that requires immediate attention and investigation. First, we examine the
problem development from three domains, which are the sources that help us funnel to the
specific problem. Then, we connect specific problems to developing a clear purpose for our
study. Looking ahead, we connect the purpose to the research questions in Chapter 3, in
which we show how research questions inform design, methods, data sources, and context.
Identifying a Problem in Our Context

In this section, we present three concise ways that we can work to develop a problem state-
ment that implies action from us. We may have been given a directive to examine a partic-
ular program, policy, or procedure, but we may not fully understand why we are conducting
this examination. As we explain later, by reading prior research, collaborating with col-
leagues and stakeholders, and through our creation of a logical model (a visual represen-
tation of what we know and do not know), we can better conceptualize our problem as
worthy of study and justify our proposed research to funders, those in power, and other
stakeholders and colleagues.
This is not to say that we must use all three domains of investigation when developing
a problem statement. For example, if a community action group has already identified a
problem and it asks us, as community members, to further investigate the problem, we may
still want to find relevant research literature to help us conceptualize and further define our
focus, but we will not need to conduct a needs assessment.
Vignette 2.1: Working for Equity Requires Inclusivity

and Stakeholder Engagement
Martha worked closely with her board of directors to implement all policies and pro-
cedures. The board brought a problem to Martha and her staff: to determine why the
participants in her organization did not mirror the demographics of the community.
Specifically, African American community members were underrepresented in the or-
ganization. Martha and her staff met with key community members to get an initial
understanding of why they utilized the organization or why they did not. In meeting
with these groups—community members, business organizations, and city council
members—Martha and her staff discovered that the problem was more nuanced than
just engaging with the nonprofit. By including stakeholders, they found that many
stakeholders held misconceptions about the purpose and services offered by the
organization. In addition, Martha and her staff examined other similar nonprofit or-
ganizations and what past researchers in the nonprofit field cited as issues related
to the use and support of similar nonprofits. Martha’s willingness to take time to en-
gage with multiple stakeholders broadened and deepened her understanding of the
problem. This work provided substantial justification to take back to the board for
further direction on a purpose for their research.
We return to Martha’s work on this policy after we introduce three domains that can help us
narrow down a problem to specific issues/concerns to examine.
Domain 1: Examining Prior Knowledge

In our work as educators, action researchers, and evaluators, we often utilize a review of
the research literature and/or relevant professional publications (e.g., reports and reviews)
prior to engaging in a study. This work can be considered as the development of a concep-
tual framework that will justify and guide the conduct of our study. The conceptual frame-
work is important because it helps us determine what is known and unknown about a topic
and also helps us strengthen our arguments about the need for the study. By this, we mean
that although past research may not exactly mirror our context and what happens in our
context, little of our daily work is completely new and unknown to those who do similar
work. So, by examining a research study by one or more authors in which they studied a
similar organization, with a similar structure and purpose, we may discover results that
could be meaningful for us, and we may also discover that they made recommendations for
future research that could speak to what we think might be a problem in our organization.
Prior authors may have done past research that we could replicate in our own context, thus
giving a new context for the prior research.
According to Rallis (2018), a conceptual framework is a “map of the world a researcher
intends to study” (p. 2). When we create a conceptual framework, we explain our concepts
to the reader and provide information about their origin, authors, and connection to our
context and problem. We use this framework to structure our investigation. Think about
how a scaffold is used to paint, construct, or otherwise work on large buildings. In the same
way, every behavioral issue is deeply embedded in human relationships, organizations, and
processes and procedures. We need to provide this framework that we create to the reader
so that they understand what we are doing. We use prior research to create our framework.
We may do this by looking at a psychological terminology or a set of sociobehavioral the-
ories that we want to use as a lens to examine our own setting.
As an example, consider Duffy, Wickersham-Fish, Rademaker, and Wetzler’s (2018)
study on doctoral mentoring. These authors created a framework borrowed from Rusbult,
Zembrodt, and Gunn’s (1982) framework from the psychological relationship literature
on romantic relationship deterioration. The authors applied that framework to doctoral
mentoring because in their experience, they found many similarities between mentor–
protégé relationship problems, as Rusbult et al. did when examining romantic relation-
ships. Although a doctoral mentoring relationship is not romantic, mentors and protégés
often find themselves in conflict and the mentor will struggle to maintain the relationship
and support the student.
Here, we see that Duffy et al. (2018) created a framework that would resonate with
readers—other doctoral mentors working with difficult student relationships. This frame-
work also helped to focus on the study. Researchers have studied doctoral mentoring rela-
tionships from a variety of perspectives, including how students learn, what students need
to advance in academic work environments, and the problems of unfinished doctoral work.
But Duffy et al. wanted to explore the nature of the relationship and what happens when the
relationship starts to deteriorate, specifically in online environments.
Domain 2: Conducting Needs Assessments

To ensure that we are focusing on the program-designed outcomes, we first want to check
that in its conception our program does address the key needs of, and is culturally relevant
to, intended recipients. We also want to ensure that our program delivers on its promises.
That is, we want to systematically investigate the effectiveness of our program in a way that
is tailored to its political and organizational environment and designed to inform action
aimed at improvement. In evaluation, this is an evaluative criterion. In action research, we
are seeking to examine what is occurring. Thus, our integrated framework allows investi-
gators to borrow from either framework to examine how needs are determined and if they
are being met.
Needs assessments are a critical component of the inclusion of stakeholders. A needs
assessment is the process of surveying or interviewing colleagues and stakeholders to get
a broader understanding of what is going on in the setting, particularly what is going well
and what needs improvement. I remember watching an episode of a television show in
which bosses of large conglomerates disguised themselves and went to work in a local sub-
sidiary or lower department of their companies. I remember watching several bosses, all
of whom started these large companies, be surprised at the expertise and awareness of the
detail needed at these micro (departmental) levels in order to make the company successful.
I also remember the bosses interacting with management at these micro levels, each of
whom indicated where specific needs impeded their progress in their departments, also
surprising the bosses. In systems theory, we learn how integral all parts are to the whole or-
ganization (Stroh, 2015). By conducting needs assessments with micro-level stakeholders,
we can determine specific problems that need addressing that affect the macro (within the
company) and mega organization (where the company connects to society). We may dis-
cover a problem through our own work, or the administration may ask us to investigate a
problem they believe is important. Regardless of who initiates the general problem explo-
ration, when we engage with stakeholders, we can develop a broader conceptualization of
both the problem and the needs within the context.
Often, working knowledge (practitioner-based) is trivialized as lacking broader con-
nection to the macro or mega context. An organizational leader might say to an employee
who identified a need, “But, you don’t understand the entire picture of the organization
and what my priorities are.” However, if we remember, from a systems perspective, that the
micro is essential to the whole, we can understand how this type of knowledge is essential
to our conceptualization of both the problem and the purpose. The organizational leader,
when engaged with stakeholders, can better understand how their own priorities might be
affected if they do not address the micro need.
In writing about how leaders should always be learning, Schein (2017) discussed the
most difficult aspect of being a learning leader:
The toughest problem for learning leaders is to come to terms with their own lack
of expertise and wisdom. Once we are in a leadership position, our own needs and
the expectations of others dictate that we should know the answer and be in control
of the situation. . . . The only way to build a learning culture that continues to learn
is for leaders themselves to realize that there is much that they do not know, and
to realize that they must teach others to accept that there is much that they do not
know. The learning task then becomes a shared responsibility and requires leaders
at all levels to build more personal, open, and trusting relationships with their
subordinates. (p. 347)
In applying this to problem identification and focus, and organizational needs, we assert
that when we engage with stakeholders, we are confirming that their practical knowledge in
the field is as important and relevant to the macro and mega context as leaders’ perceptions
and those of others in power.
Needs assessment can be thought of as a visual representation of what is (what is cur-
rently happening) versus what is the ideal. The discrepancy between the “what is” and the
“ideal” is the need (Kaufman & Guerra-López, 2013):
Formal needs assessment: A formal needs assessment might include an instrument

that is sent to all stakeholders to ask their perceptions about the critical nature of
problems you have pre-identified in order to determine if they concur or find other
needs that are pressing and must be addressed.
Informal needs assessment: Similar to Martha’s query among her staff; Martha went
directly to employees to get their perspective on issues.
Kaufman and Guerra-López provided a number of templates to help get you started in cre-
ating an effective needs assessment to get to the heart of the matter in your context.
Domain 3: Logic Models to Assist

in Problem Identification
A logic model is a visual representation of how a program, policy, or system operates within
the organization. When we depict the operational mechanisms in a logic model, we are
showing the intended relationships (“if . . . then”) between investments made in a pro-
gram, policy, initiative, or intervention and expected results for the targeted population(s)
or beneficiaries (Taylor-Powell & Henert, 2008, p. 3). When we create a logic model, this
helps us articulate our understandings of a situation, changes needed, people involved and
impacted, and actions required.
A logic model has components that describe the inputs; outputs (activities and par-
ticipants)—short-, mid-, and long-term outcomes; and, most important, connections or
links between these components to capture the underlying assumptions. A logic model is
a tool for organizational and community strategic planning, management, learning, im-
provement, and design. It is a collaborative tool. For the purpose of the problem identifi-
cation, the logic model allows us to (1) bring together a diverse group of stakeholders with
different views and often competing priorities and (2) facilitate a constructive dialogue.
A logic model allows users to unpack the program, policy, or system’s assumptions
and how those were initiated and determined to be a “best fit” for the context, situation,
and stakeholders. In other words, a logic model helps make implicit assumptions explicit.
Program assumptions constitute stakeholders’ system of beliefs that they have about a
problem in context and the situation and people involved. Assumptions shape our under-
standing of the situation. Assumptions are our whys: Why do we (stakeholders) believe in
the importance of a problem? What is a specific need or gap in knowledge in our context,
and why? A key point to make is that assumptions are never provable. They represent one’s
perspectives. Others may hold different perspectives. Hence, one needs to be (1) trans-
parent about why one believes in what one believes (What might have informed the person’s
perspectives and how they have been shaped?) and (2) inclusive of/sensitive to/considerate
of the perspectives of others. Looking ahead (to design), the articulated assumptions con-
stitute the theoretical perspectives that, in turn, inform our study’s question(s) (how we will
state our questions) and choices of methods (design).
Furthermore, one of the key components of logic models is determining external fac-
tors. External factors are certain events, circumstances, policies, or situations that could
contribute to a problem positively or adversely but that we cannot control. In designing a
F I GUR E 2.3 Logic model template depicting inputs and both short- range and long- range out-
puts. Source: Adapted from an open source template provided by the University of Wisconsin–Madison
Division of Extension. https://fyi.extension.wisc.edu/programdevelopment/logic-models/bibliography.
study, these external factors could be considered as limitations. Figure 2.3 is a template to
aid users in developing their own logic model.
In the following vignette, Martha develops a logic model to describe the problem at
hand and the root causes and to show specific concerns and potential solutions. Martha is
working with the issue attendance and use at her nonprofit. As she gathers information, she
utilizes the logic model to get to the heart of the issue.
Vignette 2.2: Integrating the Domains into a

Logic Model to Support Problem Identification and
Solution Identification
Martha started working with the logic model template. The problem —user demo-
graphics did not mirror the community demographics—was originally identified by
the board. After she and her staff interviewed community leaders and members,
she narrowed down the problem to a lack of knowledge about the organization. But
Martha had also looked up information on her specific type of organization and found
that prior researchers had noted that issues of access and equity were connected to
member knowledge and use of facilities and services in similar types of organiza-
tions. Martha began by focusing on the limited demographics of users and wrote
justifications for why diversity was crucial for her organization and how it needed to
explore whether access was preventing use and if diverse users were treated differ-
ently by staff (different requests for access based on who was asking—ethnicity, age,
gender, etc.). In addition, she wondered if the programs and services offered would
meet the needs (i.e., types of services and times of services) of the community they
represented. Martha began to see that the original problem was too vague, and the
issues were much more complex. She knew that in this small study she could not
address every problem, but she had to present to the board with evidence to show
how she and her staff were working to address one problem at a time. Martha in-
volved the whole staff in a collaborative meeting to complete the logic model of their
prioritized issue and what they wanted the outputs and outcomes to be inclusive of
the staff and stakeholders’ points of view. She and the staff decided to prioritize the
issue of meeting stakeholders’/users’ needs, which meant discovering what types of
services best served the community and when these services could be offered.
Integrating the Three Domains

As you may have noticed, each domain requires stakeholder engagement. This is not by acci-
dent. Identifying a problem to inform the development of a study problem statement within
an action research and evaluation framework is relational and values-based. Applying the
work of Friedman, Rothman, and Withers (2006), the focus should be on the process (How
will we will go about it? Who are the key stakeholders? What would it take? What tools and
strategies do we have?) and dynamics of interaction (What is taking place? What are we
learning? Why?), and not merely the outcome (i.e., problem identification). It is the process
that facilitates learning, and engagement with and ownership of a problem, which in turn,
we argue, could be extended to subsequent phases of a study’s design, implementation, and
results and their utilization to inform appropriate choices of actions and improvements.
It is difficult for us to give specific steps to defining and including stakeholders. But,
we do believe there are key approaches anchored in our positionalities. In action research, if
the practitioner is directly responsible for defining and solving a problem in their own prac-
tice, they may find that identifying key stakeholders is easy, particularly if the practitioner
is, for example, a classroom teacher. What they may not consider, because they are likely
at the bottom of the “power structure,” is that stakeholders include their students, parents,
and those in power. In evaluation, internal evaluators may be chosen by those in power and
may have the perception of having power over subordinates. So, the internal evaluator has
to deeply consider their own positionality and if and how they will push back against those
in power if the power goals subvert other stakeholders’ needs. In our integrated frame-
work, we recognize that stakeholder identification is critical but difficult in both scenarios.
Therefore, ideas such as critical reflexivity benefit both frameworks. We believe the fol-
lowing core ideas can help define and include stakeholders:
Being critical (critical reflexivity)

Whose goals are we addressing?
Under what circumstances?
Who is at the table and who is not, and why?
These questions allow us to challenge the social benefit and relevance of a program’s pre-
determined goals and outcomes and also the needs for and use of the research/evaluation
processes and findings. Once we have determined who is and who should be at the table
(stakeholders), we then work on how to be inclusive.
Being inclusive: Work directly with identified stakeholders to determine our study’s
problem and purpose and examine all of our assumptions/perspectives that frame
our decisions.
How will we generate evidence about our chosen problem and purpose?
How will we include stakeholders’ perspectives in our analysis and interpretations?
How will we ensure that stakeholders who may be in the minority are not
marginalized throughout the process?
Within action research and evaluation, the logic model helps one envisage and engage
in the change process. For Martha and her staff, the logic model helped them get informa-
tion on the general problem, which in turn helped them focus on the particular need to
address. They were able to document the significance of the chosen problem/need and to
present that evidence to the board and to stakeholders. Last, the logic model, as a collabora-
tive tool and process, helped them provide visibility to the community of their commitment
to serving all stakeholders. The outcome of using the logic model helped them create a clear,
concise problem statement, which was accepted by the board, and their work was funded
and supported. In the next section, we focus on writing a concise problem statement.
Writing the Problem Statement

We write problem statements to put together all of our prior work in investigating a ge-
neral problem, emphasize our specific focus, give rational and evidence of the need for the
problem, and provide a connection between what is known about a problem and what we
do not yet know (but need to know in order to be successful in our organization). We also
want to provide a logical connection to what potential solutions could help us do. In ge-
neral, we answer the following questions when we write a problem statement:
1. What is the general problem?

2. Are there specific issues within the general problem?
3. Who is affected by the general problem and specific issues?

4. What is the significance of the problem and issues to the health of the organization?
5. What is known and not known about this problem?
6. Are there potential solutions that could address the problem and how do we
know this?
Vignette 2.3: Martha Writes a Problem Statement
The following is the problem statement written by Martha and her staff after collab-
oration and background research:
Diverse members of the community are underrepresented in using the facility and
services of our organization. Specifically, we discovered issues contributing to un-
derrepresentation include: diversity and access; staff member cultural competen-
cies; services and availability of the organization; lack of marketing to targeted
communities. All of these issues significantly affect the health of the organization.
The organization is funded by the community, which implies that the organization
needs to serve all members adequately, and be known by all members. If we fail to
address these issues, the community could reduce their financial and physical sup-
port of the organization in the community, which would have an adverse impact
on the health of the community and on the sustainability of the organization. Prior
research noted that first steps in improving visibility for similar organizations are
to meet the needs of the stakeholders in terms of offerings and facility availability.
While all of the issues are important, we suggest addressing offerings and availa-
bility (services needed in the community and times of use) as a first step in solving
the problem of underrepresentation. Our suggestions for solutions are to begin
with a community needs assessment and to revisit our services and facilities to
make adjustments in what we offer.
How Do Problems and Purposes Connect? Moving from a

Problem to a Researchable Purpose
As Vignette 2.3 shows, Martha and her staff are planning to conduct a needs assessment and
to implement revisions (as applicable) to the organization’s services offerings and facility
availability. Their specific purpose could then be stated as follows:
The purpose of our investigation is to examine the needs of the community in the
organization services with a focus on the types of services and times of use. The
needs assessment results will inform (1) the potential changes and modifications in
the current services provided and (2) the development of the evaluation to assess the
effectiveness of the changes made as it relates to the diverse community members’
engagement and their perceptions of their experiences.
The developed problem and purpose statement will then guide Martha and her staff mem-
bers’ efforts to design a study within a stakeholder engagement framework that will be the
focus of our discussion in Chapter 3.
You may notice that the problem statement and purpose statement seem repetitive.
This is by design because the alignment helps the user focus their work and keep that focus.
When we are engaged in lengthy or even longitudinal work, having a brief statement of focus
for the problem and purpose to show continued alignment helps keep the study on track.
Connecting to Positionality
Although a broader stakeholder representation in the problem identification plays an im-
portant role within a study’s process, we believe that challenging the social relevance of
the stakeholders’ predetermined goals, speaking truth to power, and asking critical ques-
tions are important. We can ask, who determines the purpose of the evaluation? Whose
questions drive the evaluation? Who is at the table and who may not be, and why? Who is
impacting, and why? Who is benefiting and who may not be, and why? Answering these
questions is critical to our stance as action researchers and evaluators and for evaluation
for the social good and empowerment of all stakeholders (Donaldson & Picciotto, 2016).
Conclusion and Preparing for What

Comes Ahead
In this chapter, we focused on identifying and framing a study problem. We described how
it is important for the practitioner–researcher to collaboratively reflect on the nature of a
problem to determine if other issues are complicating the problem. We can do this with the
use of stakeholders, prior knowledge, needs assessments, and logic models as some of our es-
sential tools and approaches. We chose these approaches (among many available) because we
have found these are the most commonly used strategies and most easily accessible. Although
needs assessment is natural to action researchers, for evaluators it can be a separate domain.
Evaluators may rely on a few stakeholders or power needs; yet, in our integrated framework,
we may get a more applicable and specific focus through the use of needs. So, too, the logic
model, which may be more commonly seen in evaluations, can provide a clear representation
or path of the needs and the change required. The complementary nature of action research
and evaluation approaches allows us to utilize tools relevant to the situation, despite our pre-
determined perceptions of those tools. In Chapter 3, we move forward with thinking about
specific study questions to inform research designs implied by our problem and purpose.
Critical Questions: Reflect on Problem in Context

1. Stakeholder identification
• Who are the key stakeholders? Why? (Who is present? Who is missing?)
• What is our relationship with stakeholders?
• How are we going to “weigh” the stakeholders’ input problem identification?

• What strategies are we going to use to gain a good (comprehensive) understanding
about a problem and its context?
• How can we meaningfully engage stakeholders (community members) to determine a
problem? To create knowledge on which to act and to sustain change?
2. Problem identification
• What problem are we trying to solve? Why? What is at stake?
• How is the identified problem justified?
• Is the identified problem feasible?
• Can we agree on/understand the identified problem statement?
• What processes are in place?
• What is our current theory of action?
• Who is impacted? Why?
• What are the specific concerns?
3. Equity lens
• Whose perspectives drive the problem identification?
• Who is impacted?
• Who is not impacted? Why?
Return to the narrative you wrote after reading the Preface, in which you addressed
the following questions: What makes sense to you? What might surprise you? With what posi-
tions might you agree? With what positions might you disagree? Why?
• How have your answers changed after reading this chapter?

• What would you add to your narrative?
• What would you take away?
Additional Chapter Resources

Logic models: The University of Wisconsin–Madison Extension is a comprehensive
source to learn about and develop a logic model that also provides a variety of
examples and templates: https://fyi.extension.wisc.edu/programdevelopment/
logic-models.
Center for Collaborative Action Research: The Center for Collaborative Action
Research at Pepperdine Graduate School of Education and Psychology provides
collaborative action research resources for educators, researchers, and community
members, with an emphasis on evidence-based reasoning to solve problems in
contexts: https://www.actionresearchtutorials.org/.
Community Tool Box: The Community Tool Box is a useful and informative online
resource that supports communities’ work toward social change. It contains
resources, facilitation techniques, and toolkits: https://ctb.ku.edu/en/about. The
tool box has a section on needs assessment: https://ctb.ku.edu/en/table-of-conte

nts/assessment/assessing-community-needs-and-resources.
North Carolina State University blog: This blog focuses on conducting needs
assessments. It provides good explanations of needs assessments and guides to
conducting an assessment in your organization: https://www.ies.ncsu.edu/blog/
how-to-conduct-needs-assessment-part-1-what-is-it-and-why-do-it.
Kaufman, R., & Guerra- López, I. (2013). Needs assessment for organizational
success. Alexandria, VA: American Society for Training & Development.
ISBN:9781562868147.
McNiff, J. (2008). Planning, designing, and doing action research: https://www.actionr
esearch.net/writings/jack/jmplanar.pdf.
3
Getting Down to the Design

Determining Examination Main Questions and
Appropriate Design
Good studies always “live” to a great extent on the basis of the researchers’
personalities and the methods they use.
—Flick, von Kardoff, and Steinke, 2004.
In this chapter, we discuss the types and essential components of various designs. We ex-
plore the nature of research problems in more depth and the role of design as it relates
to (1) having confidence in collecting relevant data, (2) using appropriate procedures to
analyze data and to interpret results, and (3) generating credible evidence to address our
examination questions and problems. We caution practitioner–researchers to deeply reflect
on problem identification (see Chapter 2). There are two main reasons for this. In some
situations, priorities identified by a board of directors or school district board members or
emerging needs could take precedence over the existing needs or goals. Although at times
both areas of priorities may coincide, at other times they may conflict and/or compete for
time, resources, and expertise. In both scenarios, we need to consider and prioritize those
needs to be both (1) responsive to not only existing but also equally important, emerging
priorities and (2) responsible/accountable to stakeholders or community members. This is
why we emphasized the development of a clear problem and purpose statement in Chapter
2 so that you can continually refer back to it as you are working with stakeholders planning
a study to not only stay focused on agreed-upon priorities but also keep critically reflecting
on those priorities, ensure their relevance, and make adjustments or modifications if nec-
essary and feasible.
Second, how we think about the problem and, subsequently, determine a particular
aspect of it on which to focus our examination that, in turn, guides the development of a
study’s purpose and main question(s) is embedded in our perspectives about knowledge
or evidence (type of evidence). In other words, it is perspectives that support different
modes of inquiry or research (qualitative, quantitative, or mixed methods) and justify our
DOI: 10.1093/oso/9780197620823.003.0003
Another random document with
no related content on Scribd:
The following Romanza is simple and direct. One cannot fail to hear
the stormy motive of the first movement, however, in the
accompaniment figure of the second.[77] Also one may suspect the
movement to have been modelled pretty closely on the Cavatina in
the Beethoven quartet in B-flat major. The broken effects—von
Bülow called them sanglots entrecoupés in the piano sonata, opus
110—in the Beethoven work are copied rather closely in the Brahms.
The Scherzo and Trio are widely contrasted; the one being in shifting
harmony and 2/4 rhythm; the other plainly in F major and true
Viennese waltz rhythm. In the final allegro motives from the first
movement appear, so that the entire quartet is rather closely woven
into a whole.
Apart from the general traits of Brahms’ style one finds little to
comment upon. It is striking that Brahms, in nearly the same
measure as Beethoven, was able to express symphonic material,
that is material of the greatest force and dramatic power, in the form
of the quartet without destroying the nature of the smaller form. But
the Brahms quartets are by no means the unfathomable mysteries of
the last Beethoven quartets. They are comparable in general to the
Rasumowsky quartets.
There is scarcely need to speak of the quartet in A minor, opus 51,

No. 2, nor of that in B-flat major, opus 67, in detail. Brahms was
already master of his technique and in the short period between
writing the quartets opus 51 and the quartet opus 67, his manner of
expression hardly developed or changed. Kalbeck describes in detail
the significance of the chief motive, A-F-A2-E, in the A minor quartet.
The F-A2-E may be taken as initial letters of the motto Frei aber
einsam, which was of deep meaning both to Brahms and Joachim, to
the latter of whom Brahms would have liked to dedicate the quartet.
The four movements, Allegro non troppo, Andante moderato, Quasi
minuetto moderato, and Allegro non assai are vaguely related by
minute motives. The quartet in B-flat major is on the whole happy in
character, in noticeable contrast to the melancholy which pervades
that in A-minor.
There is not, either in the quartets of Schumann or those of Brahms,
any radical change from the so-called classical method. One is not
surprised to find in Schumann’s a concentration upon lyrical
moments rather than an organic development. This is the mark of
the romanticists. A thoughtful ear will detect the same underlying
lyricism in those of Brahms, though Brahms’ power of construction
passes wholly unchallenged. In the matter of harmony neither
composer is so modern as Schubert. Schumann, it is true, gives us
the first allegro of a quartet in A minor in the key of F major. This is
what one might call an external irregularity only. There are rhythmical
oddities in all Schumann’s music, and ever present evidence of a
complicated rhythmical system in Brahms’. These peculiarities are
represented in their quartets.
The quartets of able men like Robert Volkmann and Joachim Raff
are not less classical. There are three quartets of Raff’s which stand
a little out of the general path; one in form of a suite, one called Die
schöne Müllerin, and one in form of a canon. But in the main it may
be said that the string quartets of all German composers down to the
present day adhere closely to the model of the Rasumowsky
quartets, not only in form, but in general harmonic principles. We
must look to other countries for changes.
III
Among the very great quartets, that in D minor by César Franck
holds a foremost place. Vincent d’Indy remarks in his life of Franck
that the great quartets have been the work of mature genius. Franck
waited until his fifty-sixth year before attempting to write in the form.
He prepared himself specially by a year’s study of the quartets of
Haydn, Mozart, Beethoven, and even Brahms; and in 1889 began
work upon what was to prove one of his indisputable masterpieces.
The peculiarities of Franck’s style are striking and have been

discussed at some length elsewhere in this series. They are clearly
marked in the string quartet: the constant chromatic shifting of
harmonies, the intensive cultivation of short phrases, the polyphonic
skill, and the singular purity of thought that fills all his music with the
spirit of cathedrals. His workmanship is everywhere fine, and shows
at its best in the treatment of the four parts. The analogies which
have been suggested between him and the great Bach are at least a
little supported by the fact that Franck, like Bach, was influenced in
all his work by the organ. The great chords in the opening portions of
the quartet suggest organ music. Yet on the whole the style of the
quartet is perfectly adapted to the instruments for which it was
written.
The form is unusual. There is an opening section in D major, poco

lento, an indescribably full and glorious expression of the
fundamental musical thought of the entire work. It is complete in
itself, but is followed without pause by the first allegro, in D minor.
The allegro movement is regular in structure, except for the
recurrence of the theme of the introduction as foundation for the first
part of the development section, and again as coda. The first theme
recalls motives in the first movement of the pianoforte quintet in F
minor. There is a transitional theme in D minor (violoncello) which
plays a considerable part in this movement, and which later on is
metamorphosed and becomes a part of the second theme of the last
movement. The second theme of the first movement appears
regularly in F major (first violin).
The first part of the development section is, as already suggested, a

fugal treatment of the introductory motive. The tempo becomes piu
lento, so that we seem to be listening to a section of music
independent of the allegro. At the end of this fugal process the time
becomes again allegro and the development of the first and second
allegro themes, together with the transitional motive of the first
section, proceed regularly according to classical traditions. The
restatement is likewise regular; but the coda is built upon the
opening motif. Hence the movement as a whole presents the
interweaving of two quasi-independent movements, each very nearly
complete in itself, and each consistently developed through its own
proper course. In fact the three sections marked Piu lento could be
joined to each other with scarcely a change of note; and the sections
marked allegro likewise. The double scheme is carried out perfectly
to the very end of the movement, even the coda itself playing with
motives from both sections.
The Scherzo is in F-sharp minor, with a Trio in D major, delicate

throughout; and the Largo is in B major. Of the latter nothing can be
said in words that will represent the strange, devout exaltation of its
beauty.
The last movement brings us face to face with the structural

principles upon which Franck worked, and which are clear in the
violin sonata, the works for pianoforte solo, the pianoforte quintet,
and the symphony. The fragmentary introduction is a combination of
snatches of music yet to be made fully known, and reminiscences of
themes that have gone before: of the melody of the Largo; the
rhythmical figures of the Scherzo and the motive of the Trio; and
finally, as preparation for the last movement itself the violoncello
sings once more the motive of the first introduction, and is answered
by the first violin.
The Allegro molto begins after a pause. The first theme is given to
the viola, a theme that is almost note for note the theme we have just
had recalled to us. The entrance of the second theme is prepared by
many anticipations. The theme is in three broad clauses, more or
less widely separated from each other. The first of these is a
changed form of transitional motive from the first movement. It is
given out in sustained chords, a little slowly. The second clause
(violins in unison) follows shortly after the restoration of the original
tempo. This is considerably developed, dying away to a series of
chords on the motive of the first clause (originally from the first
movement). There is a powerful crescendo, and a dramatic stamping
of chords as announcement to the third clause of the second theme
(molto energico, first violin).
The development and restatement of this material follows the regular

course of the sonata form. The coda brings back the motives of the
Scherzo, and these, developed with the first theme (originally from
the first introduction), lead up to a sublime chant of the melody of the
Largo (in augmentation). A few measures, recitative built upon
phrases of the first theme, and a short Presto bring the work to full
completion.
The César Franck quartet is a great work, and it is a great quartet.

The material is symphonic, but it is finely divided among the four
instruments. There is rich sonority but no thickness. The lines of the
form are clear, and it is not surprising to find genuine polyphony in
the work of a man who, like Franck, possessed a technical skill that
was instinctive. One may only raise a question as to whether this
quartet is really a further development of the last Beethoven quartet,
if indeed it is in principle of structure akin to them. In the matter of
form it is strikingly different from the quartets of Schumann and
Brahms, but is it not equally different from those of Beethoven?
There is a more vital organization in the C-sharp minor quartet of
Beethoven than can be explained by the presence of the same
thematic material in all the movements.
The Flonzaley Quartet.
From a photograph
The entire work is in the nature of the development of a germinal
thought. This thought expresses itself in various forms; in the initial
fugue subject, in the gyrating theme of the second movement, in the
half-barbaric dance of the last. The quartet is, broadly speaking, a
series of variations, each outgrown from one before. The music
literally grows. In the quartet of Franck it progresses, and its various
themes are arranged. His method is nearer akin to the symphonic
poems of Liszt, or to the Symphonie Fantastique of Berlioz. The
affinities between the various movements of the C-sharp minor
quartet are subtle, indeed almost not to be proved but only felt. In the
quartet of César Franck, the relationships are evident and even
striking. This question of form, however, concerns all branches of
music, and is not peculiar to the quartet.
Among the many devoted pupils of César Franck one is

distinguished by, among other things, two excellent quartets. This is
Vincent d’Indy. The quartet in D major, opus 35, was composed in
1890, the second quartet, in E major, opus 45, in 1897. The second
reveals two characteristic features of d’Indy’s style: a use of folk-
melodies, together with a powerful intellectual command of the
principles of musical form. The cycle of four movements is
constructed upon a single motive which is printed as a motto at the
head of the score. The procedure recalls Schumann, particularly the
Sphinxes of the Carnaval. There is a slow introduction in which the
motive is made clear. An animated movement in sonata form then
follows, of which the opening measures (cello) are sprung from the
motive, and developed into a broad melody (first violin). After a lovely
second theme (G major, first violin, initiated by viola) there is a long
development of the motive and this first theme. In one section—très
calme—the motive appears augmented—now for viola, now for first
violin and at the same time violoncello (syncopated). In the next
section it is tossed about between the violins, over a repeated B
(violoncello). Suggestions of the returning theme are given in C-
sharp major (first violin) and in C-sharp minor (second violin). The
second theme returns, regularly, in E major (viola).
In the following movement the motive is given in a piquant dance-like
style (5-4). In the adagio (très lent) it forms the first notes of the chief
melody (first violin and viola in unison); and in the last movement is
reduced to an accompanying whirr, suggestive of the beginning of
the last movement of the pianoforte quintet of Franck. It is likewise in
the monotonous melody of the first violin, taken up by the 'cello, by
the two violins in unison and repeated with a mad sort of swing. Near
the end it is given a soft, gently songful character (first violin) in long
notes, while the viola continues softly the same motive on a different
degree of the scale and in a different rhythm.
There is an unfinished quartet in C minor, opus 35, by Ernest

Chausson, consisting of three movements. The development of the
first theme of the first out of the motive of the slow introduction is
worthy of notice. The scherzo is delicate, but the best of the work is
in the slow middle movement, with its calm interweaving of soft
voices over a drowsy figure, and its moments of enraptured song.
There is a strong classical element, however, in the quartet of César

Franck and even in d’Indy’s quartet in E major. Both, compared with
one of the later quartets of Beethoven, will appear more richly scored
and harmonically more highly colored than the older work. And yet,
in spite of the introduction of new ideas of form, the old ideas still are
at the basis of these works. This is because both composers have
adhered to the fundamental harmonic principles of the classics, the
principle of a tonic key, of a dominant key, of keys that are contrasted
with the tonic key. They have added to the heritage which passed
from Beethoven and Schubert, through Chopin and Wagner, to them;
but they have discarded no part of it, nor added to it except in kind.
The richness of their works, however, must signalize a further and
remarkable growth upon the ancient stock of Bach and Beethoven.
IV
In a great many Russian quartets the adherence to established
forms is even more evident. The three quartets of Tschaikowsky and
the two of Borodine may be taken as representative of what we must
now call the older Russian school. The well-known quartet in D
(opus 11) by the former follows the classical model step by step as to
the arrangement of themes and even the disposition of keys. And
though the later quartets, in F (opus 22) and in E-flat minor (opus 30,
written in memory of Ferdinand Laub [1832-1875], a famous violinist)
present wild and even harsh features, the ground plan of them is
essentially the classical plan. We have but to note in them a richer
and more highly colored harmony, and a few sonorous effects—the
muted beginning of the first part of the second movement in opus 11;
the pizzicato basso ostinato in the second part of the same
movement; the syncopated chords, the rolling accompaniment (cello
in the development section) in the first movement of opus 22; and
others.
It would, of course, be absurd to claim that the Tschaikowsky

quartets are classical in style, or in spirit. Their quality is most
intensely romantic. Rhythm, melody, and harmony have well-nigh a
barbaric guise in many places. Yet they represent but modifications
and alterations of a familiar plan. We have a new poem in a
language that has not yet developed beyond our knowledge of it. Of
the haunting beauty of these poems in music there is little need to
speak.
Borodine in regard to form is classical. The first movement of the

quartet in A is a masterpiece in clear construction. The exposition of
the principal allegro theme is as simple as Haydn. The second
theme follows regularly in E major. There is a development section
with a little fugato, and a restatement of the chief themes, both in the
tonic key. The first movement of the later quartet—in D major—is
similarly regular in structure. And there is scarcely any structural
oddity or newness in any of the subsequent movements. But
Borodine, like Tschaikowsky, has added a touch of new colors here
and there which mark an advance—at least technical—in handling
the instruments together. His style is remarkably clear throughout.
Note only the opening measures of the allegro. And it loses none of
its transparency when it expands to effects of great sonority, as in
the treatment of the second theme at the end of the development
section, and of the first theme later on in the restatement. The use of
harmonics in the Trio is almost unprecedented in quartet music.
The lovely effects in the slow movement of Tschaikowsky’s quartet in

D major, and these effects of Borodine’s, remain within the limits of
the quartet style. But they point most significantly towards an
orchestral treatment of the group which becomes the unconscious
aim of the majority of composers. It is difficult and perhaps absurd to
define a quartet style. Still a certain transparency and a fineness of
movement and drawing are peculiar to this combination alone; and it
may be said that when the volume of sound is thickened, and the
delicacy of movement coarsened; or when special tonal effects are
introduced which add color at the expense of line, then those
peculiar possibilities of the quartet are ignored. Hence music so
written may be called orchestral, though only by comparison, of
course, with the traditional quartet style, the outlines of which we
have chosen to fix upon the model of Mozart and Beethoven.
The later Russian composers have almost without exception aimed

at effects of sonority and color. For example there are five
Novellettes by Glazounoff, opus 15; one Alla Spagnola, full of
pizzicato, an Orientale, a Valse, and an All’Ungherese, all of which
are made up of effects of color and rhythm. There is a Quatuor
Slave, opus 26, the Mazurka of which is again wholly ‘effective.’ The
final movement—Une fête Slave—might far better be written for
orchestra. The earlier quartets, opus 1 and opus 10, are
inconspicuous.
Mention should be made of the quartet written in honor of the

publisher Belaieff, to which Rimsky-Korsakoff, Liadoff, Borodine and
Glazounoff each contributed a movement. The same men, except
Borodine, joined in another quartet called Jour de fête.
There are six quartets by Serge Taneieff, all carefully written but in
the main orchestral. The third (D minor) is perhaps best known, but
the fourth and fifth seem to me more significant. There are quartets
by Alexander Gretchaninoff, by A. Kopyloff, by Nikolas Sokoloff. Most
of the Russian composers have written one or two. Reinhold Glière,
among the more recent, has been successful. A quartet in G minor,
opus 20, was published in 1906. It shows some influence of the
modern French movement in the matter of harmony; but unlike the
recent French quartets, this is in most pronounced orchestral style. A
glance over the final movement, an Orientale, will serve to show how
completely the traditional quartet style may be supplemented by
effects of color and wild sonority. In Taneieff there is trace of the
older tradition; but elsewhere in the modern Russian quartets the
ancient style has disappeared.
V
The same tendency has become evident in the quartets of nearly all
nations. The Grieg quartet offers a striking example. Here is a work
which for lovers of Grieg must always have a special charm.
Nowhere does he speak more forcefully or more passionately. There
is a wild, almost a savage vitality in the whole work. But there is
hardly a trace of genuine quartet style in any movement. In the
statement of the first theme the viola, it is true, imitates the violin; but
the second violin and the cello carry on a wholly orchestral
accompaniment. The climax in this statement, and the measures
before the second theme almost cry aloud for the pounding force of
the piano, or the blare of trumpets and the shriek of piccolos. In fact
almost through the entire movement the style is solid, without
transparency and without flexibility of movement. The coda is the
most startlingly orchestral of all. Measure after measure of a tremolo
for the three upper instruments offers a harmonic background for the
cello. The tremolo by the way is to be played sul ponticello, yet
another orchestral manner. One cannot but recall the strange ending
of the E major movement in Beethoven’s quartet in C-sharp minor,
where, too, the instruments play sul ponticello, but each one
pursuing a clear course, adding a distinct thread to the diaphanous
network of sound. Surely in the hands of Grieg quartet music has
become a thing of wholly different face and meaning.
There have been magnificent quartets written in Bohemia. One by
Smetana is a great masterpiece. But here again we have the
orchestral style. The quartet—Aus meinem Leben—proved on this
account so distasteful to the Society of Chamber Music in Prague
that the players refused to undertake it. Smetana suspected,
however, that sheer technical difficulty rather than impropriety of
style was at the bottom of their refusal.[78] Whatever the reason may
have been, the work is supremely great. It seems to me there is no
question of impropriety or change of style here. Smetana set himself
to tell something of his life in music, and he chose the quartet
because the four instruments speak as it were intimately, as he
would himself speak in a circle of his friends about things which
caused him more suffering than he could bear. We have then not a
quartet, which is of all music the most abstract, or, if you will,
absolute; but an outpouring of emotions. This is not l’art pour l’art,
but almost a sublime agony of musical utterance.
As a quartet it stands unique—no piece of program music has

accomplished more successfully the object of its composer than this.
The first movement represents ‘love of music in my youth, a
predominating romanticism, the inexpressible yearning for something
which I could neither name nor clearly define, and also a sort of
portent of my future misfortune.’ The second movement brought
back memories of happy days when he wrote dance music for all the
countryside, and was himself an impassioned dancer. And there is a
slower section which tells of associations with the aristocracy. It is of
this section that the players of Prague chiefly complained. A Polka
rhythm runs through the whole movement. And after this
thoughtlessly gay passage, the third movement speaks of his love
for the woman who afterwards became his wife. The last movement
speaks of the recognition of the awakening national consciousness
in ‘our beautiful art,’ and his joy in furthering this until the day of his
terrible affliction (deafness). At this place the music, which has been
unrestrainedly light-hearted and joyful, suddenly stops. The cello
attacks a low C, the second violin and viola plunge into a shuddering
dark series of harmonies, and over this the first violin for more than
six measures holds a high, piercing E, symbolical of the chords, the
ceaseless humming of which in his ears foretold his deafness. After
this harrowing passage the music sinks sadly to the end with a
reminiscence of hopes of earlier years (a theme from the first
movement). No thematic or formal analysis can be necessary. The
work is intense with powerful emotion from the first note to the last,
and speaks with a directness that does not spare the listener thus
introduced into the very heart of an unhappy and desperate man.
The general orchestral style is noticeable at the beginning, and in the
fateful passage at the end. In the second section of the second
movement there is a phrase (viola) to be played quasi Tromba. This
is later taken up by the second violin, and still later by the first violin
and viola in octaves. The form is regular and clear-cut, the technical
skill of the highest order. There is a later quartet, in D minor, which is
irregular, fragmentary, explosive. The writing is here, too, orchestral.
There is an excess of frantic unison passages, of mad tremolo, as
there is also at the beginning of the last movement.
In the quartets of Dvořák the orchestral manner is not so evident, but

none of his quartets is emotionally so powerful as Smetana’s great
work. Dvořák brings the quartet back into its proper sphere. His
instinct for effects shows itself at the very beginning. Notice in his
first quartet—in D minor, opus 34, dedicated to Johannes Brahms—
the presentation of the second theme in the first movement: the
rolling figure for cello, the persistent figure for the viola which by
holding to its shape acquires an independent significance, and over
these the duet between first and second violin. The varied
accompaniment in the second movement is well worth study.
The whole first movement of the second quartet—E-flat major, opus

51—is perfectly adapted to the four string instruments. Every part
has an independence and a delicate free motion. The second
movement, a Dumka, is one of his masterpieces in chamber music,
and the following Romanze is almost its equal. The final movement
cannot but suggest Schumann. The third and fourth quartets, opus
61 and opus 80, lack the inspiration of the two earlier ones.
In our time we come to the famous quartet in F major, opus 96,
written in Spillville, Iowa, in June, 1893. One may call it the little
sister of the New World Symphony, which had been composed
shortly before in New York City. Like the bigger work it is founded
upon motives and themes which have characteristics common to the
music of the American negro. Some say these same characteristics
are common to music in Bohemia and Hungary, even to Scottish
music. Hence the discussion which has raged from time to time over
the New World Symphony, though the title of the symphony was of
Dvořák’s own choosing;[79] and the quartet, and the quintet which
followed it (opus 97) have likewise been made a bone of contention.
However, it must be granted by all alike that the quartet is one of the
most successful pieces of chamber music that has been written.
Nowhere does Dvořák’s style show to better advantage, and few, if
any quartets, are better adapted to the nature of the instruments for
which they were written.
Two later quartets, opus 105, in A-flat major, and opus 106, in G
major, do not compare favorably, at least from the point of view of
musical vitality, with the earlier works.
VI
Merely to mention the composers who have written string quartets
and to enumerate their works would fill a long chapter, and to little
avail. Haydn gave the quartet a considerable place among the forms
of musical composition. Mozart’s six quartets dedicated to Haydn are
almost unique as an expression of his genius not influenced by
external circumstances. The last Beethoven quartets are the final
and abstract account of that great man’s conclusions with life and his
art. Since the day of these three masters few composers have
brought to the form such a special intention. Few string quartets
since that day contain a full and special expression of the genius of
the men who composed them. We look to other forms for the
essentials of their contribution to the art of music. Indeed, among the
men who have been discussed in this chapter there are few whose
quartets are of real significance or of a merit that is equal to that of
their other works.
As to form there has been little radical change down to the time of
the recent composers who have abandoned deliberately all that it
was possible to abandon of classical tradition. Of them and their
work we shall speak presently. Schumann, Brahms, Tschaikowsky
and Borodine, Smetana and Dvořák, and even César Franck and
Vincent d’Indy have adhered closely to the classical model, varying it
and adding to it, but never discarding it.
In the matter of style and technique most of the advance has been
made in the direction of special effects, already described, and of
increased sonority. With the result that the ancient and traditional
quartet style has given way in most cases to an orchestral style, in
which effects are essentially massive and broad, which is a tapestry,
not a web of sound. Take, for example, three quartets by modern
composers of yesterday: that of Tschaikowsky in D, Smetana’s Aus
meinem Leben, and César Franck’s. If these are not the greatest
since Schubert they have at least few companions; and they
represent more than those of Brahms, we think, the development in
technique as well as the change in style that the century brought.
There are few pages in any one of them which do not show fine and
sensitive workmanship; but the tone of all three is unmistakably
orchestral, in the sense that it is massive, sensuous, and richly
sonorous.
It is then with some surprise that we find what at the present day we
call the modern movement expressed in three quartets which are as
conspicuous for delicate quartet style as for the modernness of their
forms and harmonies.
Debussy’s quartet was written comparatively early (1893), not more

than three or four years after Franck had completed his. It is not a
work of his first period, however, of the time when he was still a
disciple of Wagner. Rather it belongs to the second period of which
L’isle joyeuse, and Estampes, for piano, L’après-midi d’un faune, for
orchestra, and the opera ‘Pelleas and Melisande,’ are, with it,
representative works. It is written according to his own ideas of
harmony, explained elsewhere in this series, and hence may be
taken as the first quartet in which the classical tradition has been
radically altered if not wholly disregarded. For the forms of sonata,
symphony, and quartet were founded upon a system of harmony.
Musical material, however freely disposed, rested upon a basis of
key and contrasting keys common to all music of that era, the
passing of which seems now before us. The Debussy quartet is
constructed thematically in a way which in principle is old and
familiar, but upon a basis which transforms the work beyond
recognition of those to whom his harmonic series is not yet familiar.
There is little to be said of the plan of the work. The four movements
are constructed upon a single phrase. Men wrote suites that way in
the early seventeenth century. This phrase, in which there are two
motives, is given out at once by the first violin, solidly supported by
the other instruments. The movement is animé et très décidé. There
is an impassioned abandon to sound. Secondary motives are given
out: by the violin under which the three other instruments rise and fall
in chords that whirr like the wind; by the cello, the same wind of
harmony blowing high above. Then again the opening motives,
growing from soft to loud; and a new motive (first violin and viola in
tenths), over a monotonous twisting (second violin and cello in
sixths). Then comes a retard. One would expect a second theme
here. The harmony rests for a moment on F-sharp minor, and there
is a snatch of melody (first violin). But for those broad harmonic
sections of the sonata there is here no regard. The key flashes by.
The melody was but a clever change rung upon the opening phrase.
It comes again following an impetuous and agitated crescendo. Note
how after this the music rushes ever up and up, and with what a
whirling fall it sinks down almost to silence; how over a hushed triplet
figure on an imperfect fifth (A-flat—D, cello) it gains force again, and
the opening phrases recur, and something again of the secondary
motives. There is perfect order of all the material, an order hardly
differing from that of the classical sonata; but the harmonies melt
and flow, they have no stable line, they never broaden, never rest.
And so all seems new, and was, and still is new.
The second movement (assez vif et bien rythmé) is in the nature of a

scherzo. Four pizzicato chords begin, and then the viola gives out
the chief idea, an easily-recognized variant of the fundamental idea
announced at the beginning of the first movement. But this is used
first as a tenore ostinato (if one may speak of it so). It is repeated by
the viola fourteen times without variation; then five times by first
violin, and twice, dying away, by cello. Meanwhile the other
instruments are at something the same monotonous game. Nothing
is clear. There are cross-rhythms, broken phrases, a maze of odd
movements, independent of each other.
Then follows a passage of different character. The lower instruments

weave a network of faint sound, and the violin has a phrase, clearly
related to the fundamental motive, though greatly augmented. Then
the queer rioting chatter of the first part comes hack, all the
instruments pizzicato, the time 15/8.
The third movement (andantino, doucement expressif) presents the

motive (first violin) wrapped so to speak in a veil of melody and thus
disguised. The last movement, beginning slowly and working up to
frenzy, brings every sort of fragmentary suggestion of this motive. It
is particularly noticeable in augmentation (first violin) about the
middle of the movement; and this middle section is developed to a
tremendous climax at the height of which the first violin gives out the
whole phrase (avec passion et très contenu) in broad octaves. A
short coda (très vif) brings yet another transformation.
The style of the whole quartet is decidedly homophonic. There are

some measures, now and then passages of several measures, in
which there is only an harmonic effect; but for the most part there is
one instrument treated as the solo instrument; usually the first violin.
Page after page presents the familiar scheme of melody and
accompaniment. There is almost no trace of a polyphonic method,
none of conventional counterpoint, of fugal imitations.
Such devices were essential to the older quartet style.
Accompaniment figures were abominable in music which passed
through definite and long harmonic sections. Even the tremolo was
not often satisfactory, and, being indistinct, tended to make the style
orchestral. But here we have to do with a fluent harmony that is
almost never still, that does not settle, as it were, into well-defined
lakes of sound on which a theme may start forth with all sail set.
Hence the accompanying parts move with a free and wide motion.
The style is flexible and animated, and thoroughly suited to the
quartet.
The fineness of Debussy’s conceptions offers the key to the subtlety

of his technique. He handles the instruments with a touch the
delicacy of which has hardly been equalled. He has new things to
whisper. The whirring figures beginning in the thirteenth measure,
the triplet figures (in sixth) after another statement of the principal
motive, over which, or interlaced with which, there is a melody for
violin, followed strangely by the viola; the wide accompanying figures
for violin and cello in contrary motion, not long before the end of the
first movement; all these are effects proper, though somewhat new,
to the quartet style. The first section of the second movement is a
masterpiece of quartet writing. Each instrument is at odds with the
others. In listening one could hardly say how many different parts
were at work in the music. Nowhere has the pizzicato been used
with better effect. The second section of the same movement offers
a contrasting effect of vagueness and quiet. The slow movement is
newly beautiful, and the last movement dramatic. By the treatment of
the instruments the quartet may stand as a masterpiece, the most
conspicuous development properly in quartet technique since the
last quartets of Beethoven.
The quartet in F major by Maurice Ravel shows an instinct for the

instruments not less sensitive or delicate, and in a few places even
more bold. But the form of the work is more conventionally organized
than that of Debussy. There are distinct themes, regularly
constructed in four-measure phrases, and occurring regularly
according to established plans. The harmonies, however, are all
fluent, so that the sound of the work belies its close kinship to the
past.
And Ravel is a master of the quartet style. The opening measures

have a suave polyphonic movement. There is polyphony in the
treatment of the second theme as it is taken up by second violin and
woven with a counter-melody by the first. And when he is not
polyphonic he has the same subtlety of harmonic procedure that
distinguishes Debussy’s quartet. The beginning of the second
movement (assez vif—très rythmé) seems to me not so
extraordinary as the beginning of the second movement in
Debussy’s quartet, but it offers a brilliant example of the use of
pizzicato effects. The muted sections in the middle of this movement;
the accompaniment figures quasi arpa; the same sort of figures in
the following slow movement combined with pizzicato notes of the
cello; and the extraordinary figures in the 5/4 section of the last
movement, indeed all the last movement, are all signs of the new
development in a quartet style which is not an orchestral style.
Finally the quartet, opus 7, by Arnold Schönberg. The work was

composed in 1905. Among earlier works there are songs, a string
sextet, Verklärte Nacht, the Gurre-Lieder, for solo voices, chorus and
orchestra, and a symphonic poem, ‘Pelleas and Melisande.’ Later
works include a second string quartet (1907-8), five pieces for
orchestra, a monodrama, Erwartung, and a few pieces for pianoforte.
The Verklärte Nacht is a work of rich, sensuous beauty. At the head

of the score are printed lines from a poem by Richard Dehmals,
which are either utterly decadent or naïve. They are beautiful, too.
So prefaced, the sextet proves to be a symphonic poem, in which
the composer has chosen to confine himself to the limited
possibilities of tone color within the range of the six instruments.
There are two violins, two violas and two cellos. The harmonies are
richly varied and free, but not at all unfamiliar. The form is the
progressive form made possible by the system of leading or
characteristic motives. All follows the poem very closely. The
opening is depressed and gloomy. The repeated low D’s (second
cello and second viola) seem to suggest the lifeless tread of the man
and woman, going unhappily through the cold barren grove. The
sadly falling phrases (first viola, later with violins) are indicative of
their mood. After considerable development, which clearly stands for
the woman’s confession of sin and woe, comes a beautiful section in
E major which seems to reflect her dream that in motherhood she
should find happiness. This is roughly broken off. The situation
demands it. For having come with child by a strange man for whom
she had no love, she finds herself now walking with one whom she
would have greatly preferred. However, the man is generous, finds
that his love for her has made a child of him, and that he and she
and the babe unborn are to be transfigured by the strength of that
love. At the end, following this amorous exaltation, the music
broadens and gradually takes on an almost unearthly beauty.
Technically, as regards the treatment of the instruments, the sextet is

extraordinary. The additional cello and viola make it possible to
employ the pronounced color of the upper tones of these instruments
and at the same time reserve the resonant lower notes as a
foundation. Much use is made of harmonics, especially toward the
end, where full chords are given that ethereal quality so like a flute
that one may easily be misled into thinking wind instruments must
have joined in the ensemble.
The quartet is radically different. The sextet is emotionally rich and

vital; the quartet is in the first place a vast intellectual essay. There
are moments in the Adagio section, and toward the close, where
music speaks in common language thoughts which are noble and
inspired. For the most part, however, the quartet is in a language
which whatever may be its future is incomprehensible to many today.
One approaches it as through a new grammar. One must first seek
to master the logic behind it, both in the matter of its broad form and
in the idiom of its harmonies. There are many who feel this language
a sort of Esperanto, artificial, not to say factitious. There are more
and more who recognize naturalness and spontaneity in it.

Evaluation and Action Research Linnea L Rademaker Full Chapter

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Evaluation and Action Research Linnea L Rademaker Full Chapter

Uploaded by

Copyright:

Available Formats

Evaluation and Action Research Linnea

Linnea L. Rademaker and

Published in the United States of America by Oxford University Press

© Oxford University Press 2022

All rights reserved. No part of this publication may be reproduced, stored in

You must not circulate this work in any other form

Library of Congress Cataloging-in-Publication Data

1. Evaluation/​Action Research Integrated Framework: Participative

2. Determining a Purpose and a Focus  14

3. Getting Down to the Design: Determining Examination Main

4. Qualitative Data Collection and Quantitative Data Collection  60

5. Ethical Challenges in Action Research and Evaluation: Reflecting on Tensions  78

Interlude: Preface to Data Analysis and Informing Data Literacy  89

6. Qualitative Data Analysis  94

7. Quantitative Data Analysis  119

8. Mixed Methods: Integrating Data Analyses to Gain Comprehensive

9. Revisiting the Integrated Model: Focus on Facilitating Social Justice

How Has This Book Come About?

implications. As practitioners (e.g., a leader, a member of the organization, and an evalu-

Our Perspectives on Action Research

• What is the meaning of evaluation and/​or action research to me?

Linnea’s Meaning of Action Research: Empowerment

Action research to me is empowerment. When I can figure out what I need to do to

Elena’s Meaning of Evaluation: Quest for Good

• driven by passion and empowerment;

What Is the Need for an Integrated Model?

1. We need an integration of action research and evaluation to examine possibilities for a

Guiding Questions as an Invitation to

1. How has your organization/​institution/​company/​unit/​team held you accountable?

Chapter Learning Outcomes

F I GUR E 1.1 Integration of action research and evaluation.

Vignette 1.1: Learning to Evaluate My Context

Martha (a pseudonym) was the associate director of a small nonprofit organization.

Why Do We Need a New Framework?

Acknowledging Language as a Potential

Defining Action Research

the systematic collection of information about the activities, characteristics and

Defining Data Literacy

Defining Organizational Capacity Building

Defining Equitable Societies

Action Research and Evaluation:

Action Research as Problem-​Solving Paradigm

Evaluation as Problem-​Solving Approach

Action Research and Evaluation: “Action”

Action Research and Evaluation Addressing

Vignette 1.2: Identifying the Actual Problem

As we return to examine Martha and her work at a nonprofit, we listen to Martha

Points of Intersection and Tensions Between

AR, action research; SBS, social and behavioral sciences.

Conclusion and Preparing for What

• How have your answers changed after reading this chapter?

Additional Chapter Resources

Determining a Purpose and

How is it that we know so much? Yet, so little is done?

Finding problems in context

Chapter Learning Outcomes

General Topic and Problematic Situation

F I GUR E 2.2 From general topic/​problem to focused research questions.

Identifying a Problem in Our Context

Vignette 2.1: Working for Equity Requires Inclusivity

Domain 1: Examining Prior Knowledge

Domain 2: Conducting Needs Assessments

Formal needs assessment: A formal needs assessment might include an instrument

1. Evaluation/Action Research Integrated Framework: Participative

2. Determining a Purpose and a Focus 14

3. Getting Down to the Design: Determining Examination Main

4. Qualitative Data Collection and Quantitative Data Collection 60

5. Ethical Challenges in Action Research and Evaluation: Reflecting on Tensions 78

Interlude: Preface to Data Analysis and Informing Data Literacy 89

6. Qualitative Data Analysis 94

7. Quantitative Data Analysis 119

8. Mixed Methods: Integrating Data Analyses to Gain Comprehensive

9. Revisiting the Integrated Model: Focus on Facilitating Social Justice

• What is the meaning of evaluation and/or action research to me?

1. How has your organization/institution/company/unit/team held you accountable?

Action Research as Problem-Solving Paradigm

Evaluation as Problem-Solving Approach

F I GUR E 2.2 From general topic/problem to focused research questions.

tool box has a section on needs assessment: https://ctb.ku.edu/en/table-of-conte