MICHAEL QUINN PATTON (2002)

Basic research
to contribute to fundamental knowledge and
theory
Applied research
to illuminate a social concern
Evaluation research
Summative evaluation ---to determine
program effectiveness
Formative evaluation --- to improve program
Action research
to solve a specific problem
The purpose
knowledge for the sake of knowledge
to understand and explain
Within specific disciplines
physics; biology; psychology; economics; geography;
sociology
The most prestigious contribution to knowledge takes the
form of a theory that explains the phenomenon under
investigation
Works to generate new theories or test existing theories
Interested in formulating and testing theoritical construct
and its propositions that ideally generalize across time
and space
The finding are published in scholarly books, journals, and
disertations
Qualitative inquiry contributes to basic research through
inductive theory development --- “grounded theory”
Anthropology
What is the nature of culture?
How does culture emerge?
How is it transmitted?
What are the functions of culture?
Psychology
Why do individuals behave as they do?
How do human beings behave, think, feel, & know?
What is normal & abnormal in human development &
behavior?
Sociology
What holds groups & societies together?
How do various forms of social organization emerge & what
are their functions?
What are the structures & processes of human social
organizations?
Political science
What is the nature of power?
How is power organized, created, distributed & used?
Economics
How do societies & groups generates & distributed scarce
resources?
How are goods & services produced & distributed?
What is the nature of wealth?
Geography
What is the nature of & variations in the earth’s surface &
atmosphere?
How do various forms of life emerge in & relate to variations
in the earth?
What is the relationship between the physical
characteristics of an area & the activities that take place in
that area?
Biology
What is the nature of life?
What are variations in the forms of life?
How have life forms emerged & how do
they change?
Communications (tentatively)
What is the nature of symbols &
interaction?
How is symbol organized, created,
distributed & used in interaction?
Work on human and societal problems
The purpose
To contribute knowledge that will help people understand
the nature of a problem in order to intervene, thereby
allowing human beings to more effectivelly control their
environment
Offen guided by the findings, understandings, and
explanations of basic research
Conduct studies that test applications of basic theory and
disciplinary knowledge to real-world problems and
experiences
The results are published in specialis journals
Applied interdiciplinary fields are especially problem
oriented rather than discipline oriented
The findings typically are limited to a specific time, place,
and condition
Applied economic anthropology
How can the prosperous economy of an isolated,
small minority group be preserved when that
group encounters new competition from the
encroaching global economy?
Applied social psychology
How can a group within a large organization
develop cohesion & identity within the mission &
values of its parent structure & culture?
Applied political geography
How can people of previously isolated towns, each
with its own history of local governance, come
together to share power & engage in joint
decision making at a regional level?
Applied educational & organizational
development
How can students from different
neighborhoods with varied ethnic & racial
backgrounds be integrated in a new
magnet school?
Applied communication (tentatively)
How can public relations tools increase
awareness of organizational members to
support all the organization activities?
Examine and judge the processes and
outcomes aimed at attempted solutions
Study of programs, policies, personnel,
organizations, and products
Can be conducted on virtually any
explicit attempt to solve problems or
bring about planned change
 Summative evaluations
Serve the purpose of rendering an overall
judgment about effectiveness of a
program, policy, or product for the purpose
of saying that the evaluand (thing being
evaluated) is or is not effective and,
therefore, should or should not to be
continued, and has or does not have the
potential of being generalizable to other
situations
Qualitative data typically add depth, detail,
 Formative evaluations
Serve the purpose of improving a specific
program, policy, group of staff or product
Aim at forming (sharping) the thing being
studied
Rely heavily on process studies,
implementation evaluation, case studies,
and evaluability assessment
Formal design and the data are collected
and / or analyzed, at least in part, by
evaluator
Aims at solving specific problems within a program,
organization, or community
Explicitly and purposefully becomes part of the
change process by engaging the people in the
program or organization in studying their own
problems in order to solve those problems
Focus on specific programs at specific points in time
Design and data collection tend to be more informal
The people in the situation are often directly
involved in gathering the information and then
studying themselves, and the result are used
internally to attack specific problems within a
program, organization, or community
The findings of formative evaluation and
action research are seldom disseminated
beyond the immediate program or
organization within which the study takes
place
Publication and dissemination of findings are
more likely to be through briefings, staff
discussions, and oral communications
Basic research
What are variations in types of family & what functions do
those variations serve?
Applied research
What is the divorce rate among different kinds of families in
the US & what explain different rates of divorce among
different groups?
Summative evaluation
What is the overall effectiveness of a publicly funded
educational program that teaches family members
communication skills where the desired program outcomes
are enhanced communications among family members, a
greater sense of satisfaction with family life, effective
parenting practices, & reduced risk of divorce?
Formative evaluation
How can a program teaching family
communications skills be improved?
What are the program’s strengths & weakness?
Who do participants like & dislike?
Action research
A self-study in a particular organization (e.g.,
church, neighborhood center) to figure out what
activities would be attractive to families with
children of different ages to solve the problem of
low participation in family activities
TYPES PURPOSE FOCUS OF DESIRED DESIRED KEY PUBLICAT STANDA
OF RESEARC RESULT LEVEL OF ASSUMPTIO ION MODE RD FOR
RESEARC H ORGANIZAT NS JUDGING
H ION
BASIC Knowledge Questions Contributio Across time The world is Major Rigor of
RESEARC as an end deemed n to theory & space patterned; refereed research,
H in itself; important (ideal) those scholarly universalit
discovered by one’s patterns are journals in y&
truth discipline knowable & one’s verifiabilit
or personal explainable discipline, y of
intellectual scholarly theory
interest books
APPLIED Understand Questions Contributio Within as Human & Specialized Rigor &
RESEARC the nature deemed ns to general a societal academic theoretica
H & sources important theories time & space problems journals, l insight
of human by society hat can be as possible, can be applied into
& societal used to but clearly understood research problem
problems formulate limited & solved journals
problem- application with within
solving context knowledge disciplines,
programs interdiscipli
& nary
interventio problem-
ns focused
journals
TYPES OF PURPOSE FOCUS OF DESIRED DESIRED KEY PUBLICAT STANDA
RESEARC RESEARC RESULT LEVEL OF ASSUMPTIO ION MODE RD FOR
H H ORGANIZA NS JUDGING
TION

SUMMATIV Determine Goals of Judgment All What works Evaluation Generaliz

E effectivene interventio & intervention one place reports for ability to
EVALUATIO ss of n generaliza s with under program efforts &
N human tions similar goals specified funders & to other
interventio about conditions policymake programs
ns & effective should work rs, & policy
actions types of elsewhere specialized issues
(programs, interventi journals
policies, ons & the
personnel, conditions
products) under
which
FORMATIVE Improve an Strengths Recomme Limited to People can & Oral Usefulnes
those
interventio & ndations specific will use briefings; s to &
efforts are
EVALUATIO n: A weaknesse for setting information conference actual use
effective
N program, s of the improvem studied to improve s; internal by
policy, specific ents what they’re reports; intended
organizatio program, doing limited users in
n, or policy, circulation the
product product, or to similar setting
personnel programs, studied
being other
studied evaluators
TYPES PURPOSE FOCUS OF DESIRED DESIRED KEY PUBLICATI STANDAR
OF RESEARC RESULT LEVEL OF ASSUMPTI ON MODE D FOR
RESEAR H ORGANIZAT ONS JUDGING
CH ION

ACTION Solve Organizatio Immediate Here & now People in a Interperson Feelings
RESEARC problems n& action; setting can al about the
H in a community solving solve interactions process
program, problems problems as problems by among among
organizati quickly as studying research research
on, or possible themselves participants participants
communit ; informal , feasibility
y unpublishe of the
d solution
generated
A discussion of design strategies and trade-offs is
necessited by the fact that THERE ARE NO PERFECT
RESEARCH DESIGNS
Trade-offs:
Breadth vs Depth
No rule of thumb exists to tell a researcher precisely how to
focus a study
The extent to which a research or evaluation study is broad or
narrow depends on purpose, the resources available, the time
available, and the interests of those involved
There are not choices between good and bad but choices
among alternatives, all of which have merit
Units of analysis
The key issue in selecting and making decisions about the appropriate
unit of analysis is to decide what it is you want to be able to say
something about at the end of study
The logic and power of purposeful sampling
lie in selecting information-rich cases for
study in depth
Studying information-rich cases yields
insights and in-depth understanding rather
than empirical generalizations
Purposeful sampling is sometimes called
purposive or judgment sampling
Select information-rich cases strategically &
purposefully; specific type & number of cases
selected depends on study purposes &
Extreme or deviant case sampling
Learning from unusual manifestations of the
phenomenon of interest
Example: outstanding successes/ notable failures;
top of the class/ dropouts; exotic event; crises
Ethnomethodology
Intensity sampling
Information-rich cases that manifest the
phenomenon intensely, but not extremely
Example: good students/ poor students; above
average/ below average
Heuristic research
Maximun variation (heterogenity) sampling
Document unique or diverse variations that have emerged
in adapting to different conditions
Identify important common patterns that cut across
variations (cut through the noise of variation)
Aims at capturing and describing the central themes that
cut across a great deals of variation
A small sample of great diversity will yield two kinds of
findings:
High-quality, detailed descriptions of each cases, which are
useful for documenting uniquenesses
Important shared pattern that cut across cases and derive their
significance from having emerged out of heterogeneity
Homogeneous samples
Focus; reduce variation; simplify analysis; facilitate group
interviewing
The purpose is to describe some particular subgroup in
depth
FGD are based typically on homogeneous groups
 Typical case sampling
Illustrate or highlight what is typical, normal, average
The purpose is to describe and illustrate what is typical to
those unfamiliar with the setting – not to make generalized
statements about the experiences of all participants
The cases are selected with the cooperation of key
informants or using survey data, a demographic analysis of
average or other statistical data that provide a normal
distribution of characteristics from which to identify
Critical case sampling
Permits logical generalization & maximum
application of information to other cases because
if it’s true of this one case, it’s likely to be true of
all other cases
“if it happens there, it will happen anywhere”
“if that group is having problems, then we can be
sure all the groups are having problems”
A small or “N of 1” sample
To pick the site that would yield the most
information and have the greatest impact on the
development of knowledge
Snowball or choin sampling
Identify cases of interest from sampling people who know
people who know people who know what cases are
information rich, that is, good examples for study, good
interview participants
An approach for locating information-rich to informants or
critical cases
 Criterion sampling
The logic of criterion sampling is to review and study all
cases that meet some predetermined criterion of
importance
A strategy common in quality assurance efforts
Picking all cases that meet some criterion
Quality assurance
Example: all children abused in a treatment facility
Theory-based sampling, operational
construct sampling, and theoritical
sampling
Finding manifestations of a theoretical construct
of interest so as to elaborate & examine the
construct & its variations
A more conceptually oriented version of criterion
sampling
Operational construct sampling simply means that
one samples for study real-world examples (i.e.,
operational examples) of the constructs in which
one is interested
Theoritical sampling is what grounded theorists
Confirming and disconfirming cases
Elaborating & deepening initial analysis; seeking
exceptions; testing variation
In the early part of qualitative fieldwork, the
evaluator is exploring – gathering data and
watching for patterns to emerge.
Over time, the exploratory process gives way to
confirmatory fieldwork
The sample determines what the evaluator will
have something to say about – thus the
importance of sampling carefully and thoughtfully
Stratified purposeful sampling
Illustrate characteristics of particular subgroups of interest;
facilitate comparisons
Samples within samples
 Opportunistic or emergent sampling
Following new leads during fieldwork; taking advantage of the
unexpected; flexibility
On-the-spot decision about sampling to take advantage of new
opportunities during actual data collection
 Purposeful random sampling (small sample size)
Add credibility when potential purposeful sample is larger than one
can handle
Reduce bias within a purposeful category (not for generalizations
or representativeness)
In advande of knowledge of how outcomes would appear
The purpose is credibility, not representativeness
 Sampling politically important cases
Attract attention to the study (or avoid attracting undesired
attention by purposefully eliminating from the sample
politically sensitive cases)
Evaluation is inherently and inevitably political
 Convenience sampling
Do what’s easy to save time, money & effort
Poorest rationale; lowest credibility
Yields information-poor cases
While convenience and cost are real considerations, they
should be the last factors to be taken into account
Is neither purposeful nor strategic
Combination or mixed purposeful sampling
Triangulation; flexibility; meet multiple interests & needs
Representativeness: sample size a
function of population size & desired
confidence level
Simple random sample
Permit generalization from sample to the
population it represents
Stratified random & cluster
samples
Increase confidence in making
generalizations to particular subgroups
THERE ARE NO RULES FOR SAMPLE SIZE
IN QUALITATIVE INQUIRY
The validity, meaningfulness, and insights
generated from qualitative inquiry have more
to do with the information richness of the
cases selected and the observational/
analytical capabilities of the researcher than
with sample size
The redudancy is the primary criterion
Triangulation
Strengthens a study by combining methods
 Types of triangualtion
Data triangulation
the use of a variety of data sources in a study
Investigator triangulation
the use of several different researcher or evaluator
Theory triangulation
the use of multiple perspectives to interpret a single set
of data
Methodological triangulation
the use of multiple methods to study a single problem or
program
Understanding inconsistencies in findings across different kinds
of data can be illuminative
Finding such inconsistencies ought not be viewed as weakening
the credibility of results, but rather as offering opportunities for
deeper insight inti the relationship between inquiry approach
and the phenomenon under study
Triangulation within a qualitative inquiry strategy can be
attained by:
combining both interviewing and observations
mixing different types of purposeful samples
examining how competing theoritical perspectives inform a
particular analysis (eg, the trancendental phenomenology of
Husserl vs the hermeneutic phenomenology of Heidegger)
A study can also be designed to cut across inquiry approaches
and achieve triangulation by combining qualitative and
quantitative methods
Mixing Data, Design, and Analysis
Approach
The ideal-typical qualitative methods strategy is
made up of three parts
Qualitative data
A holistic-inductive design of naturalistic inquiry
Content or case analysis
The traditional hypothetico-deductive approach to
research, the ideal study would include
Quantitative data from
Experimental (or quasi-experimental) design
Statistical analysis
Mixing Data, Design, and Analysis
Approach
Mixed form
Experimental design - Qualitative data –
Statistical analysis
Experimental design – Qualitative data –
Content analysis
Naturalistic inquiry – Qualitative data –
Statistical analysis
Naturalistic inquiry – Quantitative data –
Statistical analysis
Mixing Data, Design, and Analysis
Approach
Guba and Lincoln (1988) have argued that:
The internal consistency and logic of each
approach, or paradigm, mitigates
(mengurangi/ meredakan) againts
methodological mixing of different inquiry
modes and data collection strategies
Mixing Data, Design, and Analysis Approach
Patton (2002; 1981; 1987a):
Mixing parts of different approaches is a matter of philosophical
and methodological controversy.
Yet, the practical mandate in evaluation to gather the most
relevant possible information for evaluation users outweighs (lebih
berat daripada) concerns about methodological purity based on
epistemological and philosophical arguments
The intellectual mandate to be open to what the world has to offer
surely includes methodological opennes.
In practice, it is altogether possible, as we have seen, to combine
approaches, and to do so creatively
The spirit of adaptability and creativity in designing studies is
aimed at being pragmatic, responsive to real-world conditions and,
when doing evaluations, to meeting stakeholders information
needs
Design and Methods Decisions
In qualitative inquiry, the problem of design poses a
paradox
Term design suggest a very specific blueprint, but
“design in the naturalistic sense ... means planning
for certain broad contingencies without, however,
indicating exactly what will be done in relation to
each (Lincoln and Guba, 1985)
A qualitative design needs to remain sufficiently
open and flexible to permit exploration of whatever
the phenomenon under study offers for inquiry
Qualitative designs continue to be emergent even
after data collection begins