Professional Documents
Culture Documents
Paper Number 36
June 2001
Social Analysis
Selected Tools and Techniques
Richard A. Krueger
Mary Anne Casey
Jonathan Donner
Stuart Kirsch
Jonathan N. Maack
First printing: June 2001
This publication was developed and produced by the Social Development Family of the World Bank.
The Environment, Rural Development, and Social Development Families are part of the Environmentally
and Socially Sustainable Development (ESSD) Network. The Social Development Family is made up of
World Bank staff working on social issues.
Papers in the Social Development series are not formal publications of the World Bank. They are
published informally and circulated to encourage discussion and comment within the development
community.
Acknowledgments iv
Contributors v
iii
Acknowledgments
iv
Contributors
v
vi
1. Introduction to the Issues
1
Social Analysis: Selected Tools and Techniques
Reliability No. of
of How many Prevalence analysts/ Strengths Limitations
informa- respon- Speed of of actors/ facilitators of of
Method tion dents? analysis Cost analysts required technique technique
Q-sorts High 10–100 Rapid Modest – Analyst- 1 person To force Not a
medium driven fully trained choices and research
Validation is Full (if actors in method- reveal method;
rapid, analysis already are ology, or 3 mental questions
because within 24 assembled (facilitator, models are pre-
actors are hrs. on site) analyst, and determined
on site. computer
analyst)
Focus Medium 6–8 per group, Medium Medium – Analyst- 4–5 To Does not
groups repeated 3–4 expensive driven depending understand build
Depends on times for each on no. of how people consensus
nos. and unit of repetitions think or feel
depth of analysis (i.e., Does not
interviewing 18–32 people) provide
empirical
reality
Scenario Not subject 10–25 Slow Expensive Analyst- 4–6 To identify Does not
analysis to reliability facilitated, key drivers identify next
testing, actor-driven and steps
because it indicators
is imagining Will not
futures To survey all
understand stakeholder
High complex interests
reliability in situations
identifying
drivers of To build
change consensus
Ethno- High micro- Depends on Slow Modest Actor-driven 1 person To Inferring
graphic level breadth of minimum; understand patterns
methods reliability in inquiry ideally, how people from
limited team of 3-5 think or feel individual
spheres in different data may
locations not be
Medium-low reliable
macro-level
reliability
blueprints for action. Each practitioner must (Berkeley and Los Angeles: University of California
refine them and adapt them to the context, so Press, 1984).
that they enhance the quality of the particular 2. Appreciative Inquiry is a technique that has
project or program. been used with great success to find local solutions,
locate excellence in organizational culture, and build
Table 1 sets out some of the characteristics of
coalitions on the basis of past success. The technique
the techniques presented in this volume. is fully explained in C. Elliott, Locating the Energy for
Change: An Introduction to Appreciative Inquiry
Notes (Winnipeg: International Institute for Sustainable
Development, 1999). It is also concisely explained in
1. Stakeholders are active social actors with three S. A. Hammond, The Thin Book of Appreciative Inquiry
key characteristics: (1) practical consciousness (Plano, Tex.: Thin Book Publishing Company, 1996),
derived from life experience; (2) their own theories so is not included as a separate chapter in this
about social relationships and power; and (3) volume. See also M. Grindle, “Divergent Cultures:
systems of reflective monitoring on how things When Public Organizations Perform Well in Develop-
work. See A. Giddens, The Constitution of Society ing Countries,” World Development 25 (4) (April 1997).
2
Introduction to the Issues
3
2. Designing and Conducting Focus
Group Interviews
Richard A. Krueger and Mary Anne Casey
Market researchers have used focus groups to to generate diverse ideas but not so many
search for ways to improve and market their participants that they do not have a chance
products to consumers since the 1950s. In the to share.
last 20 years, government agencies, nongovern- • Hold three or four separate focus groups for
mental organizations (NGOs), academics, and each type of participant one desires to use as
nonprofit organizations have started using a unit of analysis. For example, to compare
findings from focus group interviews to help how men and women view a particular
make decisions about their products and issue, one should conduct three or four
services. International public health organiza- groups with men and three or four groups
tions were quick to make use of focus groups, with women.
particularly in social marketing efforts. Educa- • Avoid power differentials among partici-
tional and environmental organizations also pants. All participants in a group should feel
have used the technique to listen to their comfortable talking with one another.
employees as well as to their current and
potential users. Create a Comfortable Environment
The techniques we share are based on what
we have learned while running focus groups • Hold focus groups in familiar or neutral
with nonprofit organizations. settings such as office buildings, libraries,
schools, homes, cafes, or community gather-
Characteristics of Focus Group Interviews ing spots.
• Seat people so that they can easily see one
A good focus group has the following characteris- another.
tics: carefully recruited participants, interacting • Interview people in their language. Do not
in a comfortable environment, led by a skillful use an interpreter in the group. Using an
moderator, followed by systematic analysis interpreter stilts the discussion and turns
and reporting. When conducting focus groups: the process into serial interviews rather
than a lively discussion among partici-
Carefully Recruit Participants pants. It is better to train someone who
knows the language to moderate the
• Invite individuals who have the characteris- groups than to have someone who knows
tics, experience, or knowledge needed to how to moderate work through an inter-
provide rich information on the topic. preter. After the group is over, if necessary,
• Limit the size of the group to six to eight translate the discussion back into the
people. It is desirable to have enough people analyst’s language.
4
Designing and Conducting Focus Group Interviews
• Record the discussion for analysis. Take field Roles in Conducting Focus Groups
notes and, if possible, make an audio tape
recording. Sometimes a laptop computer can There are different roles in a focus group study:
be used to make a transcription during the organizer (leads planning and developing
focus group. Explain why the focus group is questions), recruiter (invites participants),
being recorded and who will have access to moderator (leads the groups), assistant mod-
the data. erator (handles logistics and captures data),
and analyst/reporter (summarizes the data and
Choose a Skillful Moderator prepare reports).
One person can fill all the roles. However,
• Pick moderators who make people feel we prefer working with a team of four or five
comfortable and who are good at listening. people. Team members work together to
For groups of people who are used to being complete the study, but individuals may take
in powerful positions, pick a moderator who primary responsibility for certain tasks. For
can keep the group on track and control example, one may have multiple moderators
dominant participants. who speak different languages to better reach
• Use predetermined questions. The modera- the groups being addressed. Each of these
tor should be prepared to ask a set of ques- moderators would be responsible for the
tions designed to get the information needed outcome of his or her focus group.
by decisionmakers. Finally, although one person may be respon-
• Establish an open environment. It is the sible for organizing the study, the plan and
moderator’s job to create the feeling questions are usually stronger when several
that it will be safe to talk in the focus people are involved in their development.
group.
Planning
Record and Analyze Data Systematically
The main challenge during the planning stage
• Develop a systematic approach to recording is to design a study that will answer the key
and analyzing data. The analyst should be questions within the time and budget con-
sure that he or she can describe the process straints. The task manager and the team must
to others. be clear about the purpose of the study. The
• Use a verifiable process. This means the task manager must decide whether focus
analysis process should be documented and groups are the appropriate method. If focus
there should be evidence to support the groups are the best method, then the task
findings. Another researcher should be able manager must decide how many to do and to
to review all the data and see how the whom he or she wants to listen.
analyst arrived at the findings and conclu-
sions based on those data. First Steps with Focus Group Studies
A focus group is working well when partici- In developing and planning a focus study
pants begin to talk to one another and build on group, five key steps should be followed:
one other’s comments rather than continually
responding directly to the moderator. Ideally, 1. Decide whether focus groups are appropriate.
participants become engaged, and the focus Focus groups work particularly well for the
group becomes a forum for their own discus- following tasks:
sion. The moderator should begin to play a less • Understanding how people see needs and
central role as participants share experiences, assets in their lives and communities.
debate ideas, and offer opinions. Some groups • Understanding how people think or feel
arrive at this point quickly; others never about an issue, idea, behavior, product, or
reach it. service.
5
Richard A. Krueger and Mary Anne Casey
• Pilot testing ideas, reforms, or projects. 3. Decide what types of people to listen to the
Focus groups can be used to get reactions to (target audience(s)). What types of people have
plans before large amounts of money are the experiences or characteristics that will allow
spent in implementation. them to provide input on the study topic? The
• Evaluating how well programs or projects target audience may not consist of the most
are working and how they might be im- highly educated or the most influential people
proved. in an area, but its members have direct experi-
• Developing other research instruments, such ence with something about which the task
as surveys or case studies. manager wants to learn more. For example,
young people who drop out of school know a
Focus group interviews are not meant to be great deal about what it would take to keep
used as: other young people in school. Teachers, counse-
lors, and parents may give the task manager
• A process for getting people to come to different perspectives on the same issue.
consensus The task manager should consider the
• A way to teach knowledge or skills usefulness of listening to a wide variety of
• A test of knowledge or skills. people. These include elected officials, influen-
tial local figures, the people most affected by
If the task manager answers yes to any of the change, the people who must buy into the
the following questions, he or she likely will change before it can happen, and employees
need to consider other methods to use in (both frontline staff and management) of the
conjunction with, or instead of, focus group organizations that will implement programs or
interviews: services to support the change.
• Do you need statistical data? Focus groups 4. Get advice from the target audience(s). The
cannot provide statistical data to project to a task manager should find a few people similar
population. The number of people listened to to those whom he or she wants to invite to the
is too small to be statistically representative. focus groups and tell them about the study. The
• Will harm come to people who share their task manager can ask for their advice on
ideas in a group? Although the task manager several issues: Who can ask these kinds of
can guarantee that he or she will keep questions, that is, who should moderate the
information shared in the group confidential, focus groups? Where might the groups be
the moderator cannot promise that other held? What days or times would work well for
participants will do the same. If harm may people? How does one find people with these
come to people who openly share in the characteristics? What would it take to get
group, choose another method, such as people to come?
individual interviews.
• Are people polarized by this topic? It is 5. Put thoughts in writing. The task manager
difficult to conduct focus groups if people should develop a plan that includes the study’s
holding opposing views on controversial purpose, the number of groups, the potential
issues are in the same group. questions, a timeline, and a budget. This plan
• Is there a better, more efficient way to get the will clarify one’s thinking and provide a basis
information? for further discussions. Then this plan should
be shared with colleagues and their feedback
2. Clarify the purpose of the study. Team mem- invited.
bers may disagree about the information that
the study should produce and what should be Sampling and Number of Groups
done with the results. They should come to
agreement on a clear purpose for the study. The basic sampling strategy is to conduct three
This will make the entire process simpler. or four focus groups for each audience category
6
Designing and Conducting Focus Group Interviews
7
Richard A. Krueger and Mary Anne Casey
can be easily covered in five minutes. Key ques- researcher might ask students a question that
tions may be complex or include activities. A key he or she does not want to ask parents or
question can take 15 to 20 minutes to answer. teachers. Nevertheless, the researcher needs to
Once we have estimated times for each of the keep a core set of questions the same in each
questions, we add up the total to determine questioning route so that responses can be
whether we should add or delete questions. compared across audiences.
3. Send the draft questioning route to the team Phrasing and Sequencing Questions
for feedback. Ask the brainstorming team the
following questions: Will these questions get us Care is needed in the phrasing and sequencing
the information we need? What have we of the questions. Focus group questions are
missed? What can be deleted? Are these the distinctive. When developing questions, the
words participants would use? Does the flow researcher should keep in mind several guide-
make sense? Then the researcher can revise the lines:
questioning route based on feedback.
Use open-ended questions. Most of the ques-
It is important to remember that the same tions in the focus group are open-ended or non-
questions are asked in all the interviews with directive. These questions deliberately give the
the same audience. However, if separate participants as much latitude as possible for
groups are going to be conducted for different their responses. Be cautious of phrases such as
audiences, a slightly different questioning route “how satisfied” or “to what extent” because
might be used for each. For example, the these words limit the range of responses.
8
Designing and Conducting Focus Group Interviews
Use different types of questions. Five types of Focus the questions, sequencing them from
questions are asked in focus groups: (1) opening general to specific. As a rule, the questions tend
questions (answered by everyone), (2) introduc- to begin with general and broader topics and
tory questions, (3) transition questions (move move to more specific categories. For example,
participants to key questions), (4) key questions if a researcher were doing a focus group on a
(address one of the fundamental issues of the specific brand of cola, he or she might begin by
study), and (5) ending questions (get a final asking about beverages, then sodas, then colas,
viewpoint from participants on key topics). Not and finally about the specific brand.
all questions are the same. Some are designed
to get people talking. They are easy for every- Recruiting
one to answer. Others move participants to the
key questions that are the most important. At The researcher has the plan and knows the
least half of the focus group’s time should be questions that he or she is going to ask. The
spent on key questions. Ending questions help next challenge is to find the right people and
wrap up the discussion (box 2). get them to attend the focus groups.
The first step in recruiting participants is to
Use questions that get participants involved. identify as precisely as possible the characteris-
Participants can do more than talk. Ask them to tics of the target audience. A basic principle of
try a product or a task and talk about it. Or ask focus groups interviewing is that the researcher
them to draw a person who uses the program controls attendance. The researcher invites
or service that is the subject of the focus group. people because they meet the “screens” or
Have them rate different ideas. Remember to qualifications of the study devised in the
take into consideration reading and writing planning stages. Participants are selected and
abilities before using certain types of questions. invited because they have certain experiences
9
Richard A. Krueger and Mary Anne Casey
10
Designing and Conducting Focus Group Interviews
Soon after the person has agreed to participate, • Compliment (others value your insights).
send him or her a personalized letter. Do not use a • Honor (we value your opinions).
generic salutation such as “Dear friend.” This • Enjoyment (you will have a nice time).
letter should thank the person for agreeing to • Community (your participation will help the
participate and confirm the date, time, and community).
place of the focus group. In certain cases, the
recruitment and the meeting time are close Moderating
together in time. In these cases, the personal-
ized letter is not possible, but it may be helpful The challenge of moderating is making people
to give those agreeing to participate a one-page feel comfortable enough to share in a group
fact sheet with the details of the study. what they think and how they feel. Participants
must trust the moderator, the process, and the
Telephone or contact each person the day before sponsoring organization, and they must believe
the focus group to remind him or her of the group. that the results will be used in a positive way.
“I’m looking forward to seeing you tomorrow The moderator must know when to wait for
at….” more information and when to move on. The
moderator must be able to control dominant
Sometimes recruiting is done just before the speakers and encourage hesitant participants.
focus group is held. For example, the re- The moderator must respect the participants,
searcher might find the pool of participants at a listen to what they have to say, and thank them
festival or community event at which many for their views even when the moderator may
people have gathered. As people arrive at the personally disagree with those views.
event, a limited number can be invited to join
the discussion. If many qualified participants Moderator Skills
are present, the researcher might randomly
select those who will participate. In this sce- The moderator should have enough knowledge
nario, it would be a good idea to arrange for a about the topic to understand what the partici-
special place to hold the focus group where the pants are saying. He or she does not need to be
participants will not be disturbed by the an expert on the topic but should understand
surrounding events. common terms that will be used in the discus-
sion.
Getting People to Attend: Incentives It sometimes helps to have a moderator who
looks like the participants. This can make the
The researcher should think about what might participants more comfortable and give the
make it hard for people to attend and try to impression that “this person will understand
eliminate these things. If appropriate and what I have to say.” The researcher should
possible, provide transportation and consider things like gender, age, race, and
childcare. ethnicity. For some topics, these issues may not
The researcher should also think about what matter, but for other topics they are very
might entice people to participate and offer important. For example, women may be more
some or all of these things. He or she can ask a willing to talk about breastfeeding with a
few people who are similar to the target audi- woman than a man. As mentioned before, the
ence what it would take to get them to come. moderator should be fluent in the participants’
Incentives that have been used to encourage language.
people to participate include: Other things that the moderator should do
include:
• Money (we will pay you).
• Food (there will be something to eat). Be mentally prepared.
• Gifts (we have a gift for you). • Be alert and free from distractions. The
11
Richard A. Krueger and Mary Anne Casey
moderator should arrive early so he or she is • Small-talk topics should be easy to discuss.
relaxed and ready to listen. They can focus on what is happening locally
• Have the discipline to listen. Beginning (if it is off the topic of the focus group). If the
moderators are often delighted that people moderator is new to the area, he or she can
are talking and do not notice that the partici- ask participants about the weather, geogra-
pants are not really answering the questions. phy, transportation, places to eat, or their
As the moderator listens, he or she needs to families. The moderator’s job is to make
determine whether participants are really people feel welcome.
answering the question. If not, the modera-
tor should refocus their attention on it. Give a short and smooth introduction.
• Be familiar with the questioning route. • Welcome everyone.
Know which questions are the most impor- • Give an overview of the topic.
tant and which can be dropped if time is • Provide any ground rules for the discussion.
running out. • Ask the first question.
Work with an assistant moderator. Use pauses and probes to draw out more
• An assistant moderator improves the quality responses.
of the groups. • Be comfortable using a five-second pause.
• The assistant helps by handling details (for Beginning moderators are sometimes un-
example, taking care of refreshments, moni- comfortable with silence. However, pauses
toring recording equipment, integrate encourage people to add to the conversation.
latecomers into the group). • Use probes to get more detail. Generally,
• More importantly, the assistant helps ensure more detailed information is more useful.
the quality of the analysis by taking careful Consider using these questions and com-
notes, summarizing the discussion at the ments: “Would you explain further?” “Can
end, and acting as another set of eyes and you give an example?” “I don’t understand.”
ears for analysis. “Tell me more.”
12
Designing and Conducting Focus Group Interviews
• Save unplanned questions for the end. The recommended pattern for introducing
Sometimes new and unanticipated questions the group discussion includes a welcome, an
will occur to the moderator. There is a risk in overview of the topic, the ground rules, and the
asking these questions during the focus first question (box 3).
groups, as they might interrupt the sequence
of the planned questions and throw partici- Expectations of the Assistant Moderator
pants off topic. Save these questions and ask
them at the end of the focus group discus- It is helpful to have a second member of the
sion. research team present in the focus group. We
call this person the assistant moderator or the
Use subtle group control. reporter. Because there is so much going on in
• The moderator’s job is not to make sure the focus group the moderator often is not able
everyone speaks the same amount in a to lead the discussion, observe, and take notes
group. However, everyone should have the at the same time. The assistant helps with the
opportunity to share. Some people will have arrangements, takes careful field notes, and
more to say. If they are answering the ques- assists with the analysis.
tion and giving new and useful information, Assistant moderators typically take responsi-
let them continue. bility for the following:
• Control dominant talkers by thanking them
for their input and asking for others to share. • Equipment. Ensure that needed equipment is
Remind the group that it is important to hear available and working. This includes record-
from everyone. ers, microphone, tapes, and handouts.
• Call on quiet participants. They are often • Refreshments. Arrange for food (either com-
reflective thinkers and have wonderful plete meals or snacks) and beverages to be
things to offer. Invite them to share with available on time.
something like, “Maria, I don’t want to leave • The room. Arrange chairs and table and be
you out of the discussion. Is there something attentive to background noises that would
you would like to add?” affect the audio recording as well as room
temperature and lighting.
Conclude with a summary and final questions. • Welcoming participants as they arrive. Act as
• Summarize the key points of the discussion the hosts and make participants feel wel-
and ask for confirmation. Usually, the come and comfortable.
assistant moderator does this. (Do not • Sitting in a designated location. Sit outside the
summarize the entire focus group, but circle, opposite the moderator, and closest to
instead summarize three to five of the most the door, so that they can greet those arriv-
important points.) ing late, briefly telling them what has been
• Review the purpose and ask if anything has discussed so far, and find them a place to sit.
been missed. • Taking notes throughout the discussion. Cap-
• Thank the participants and conclude the ture the details of the group interaction in
session. their notes.
• Not participating in the discussion. Speak only
Beginning the Focus Group Discussion if invited to by the moderator. Control their
nonverbal actions no matter how strongly
The first few moments in a focus group discus- they feel about an issue.
sion are critical. In a brief time, the moderator • Asking questions when invited. At the end of
must create a thoughtful, receptive atmosphere; the discussion, ask questions of amplifica-
provide ground rules; and set the tone of the tion or clarification when invited to by the
discussion. Much of the success of group moderator.
interviewing can be attributed to creating an • Giving an oral summary. Provide a brief oral
open environment. summary (about three minutes) and invite
13
Richard A. Krueger and Mary Anne Casey
“Welcome! Thank you for joining us to talk about want to miss any of your comments. People often say
education. My name is Richard Krueger, and this is very helpful things in these discussions, and we
Peter Wyet. We are working with the Department of cannot write fast enough to get them all down. We
Education. We know that most teenage boys from this will not use any names in our reports, and we will
school district do not graduate from high school. We keep what you say confidential.
want to find out what can be done to help these young “I have about a dozen questions to ask. My job is
people stay in school. The Department of Education will to listen and to make sure everybody has a chance
use the information we get from these discussions to to say what he wants to say. You do not need to
design new programs specifically for boys. respond directly to me all the time. Feel free to
“You have been invited because you are all follow-up on something someone else says. You may
fathers of junior high school boys. Also, each of your want to add something or share a different experi-
sons has missed several days of school in the past ence. I will ask a question, and then you can feel free
month. We would like your thoughts on what might to have a conversation about it.
be causing this and what could be done to keep your “Well, let us begin. Let us find out some more
sons in school. We will also be talking to junior high about one another by going around the table. Please
age boys, mothers, teachers, and community tell us what grade your son is in and one of his
leaders. favorite things to do when he is not in school.”
“There are no wrong answers. We expect that you
will have differing points of view. That is okay. You do Note: After the first question is completed, it is helpful to
not have to agree. Please feel free to share your point say, “We will not be going around the circle anymore, so
of view even if it differs from what others have said. just feel free to jump into the conversation whenever you
“You have probably noticed the microphone. We want.” If the moderator continues to go around the table,
are tape-recording the session because we do not discussion becomes boring, and participants tune out.
• Memory is the most fallible. Memories of the The researcher’s choice of methods will
moderator and the assistant fade quickly depend on resources and circumstances. At
14
Designing and Conducting Focus Group Interviews
minimum, we recommend memory, field notes, only one person. This is possible only if the
and audio tape recording. analyst has a clear understanding of the pur-
pose of the study.
Analysis
Systematic Analysis Process
The analyst must take the data and find what is
meaningful to the purpose of the study, so that Focus group analysis should be systematic.
the results can be used. Not all studies require Analysts should develop a protocol that fol-
the same level of analysis. One of the skills that lows a predetermined and verifiable set of
beginning analysts must learn is to match the steps. There is no single “best” systematic
level of analysis to the problem at hand. Some process. Box 4 gives an example of a systematic
of the choices are listed in table 1. A complex analysis process that we have often used.
study in which the researchers are trying to Notice that the process begins while the first
understand how different types of people think focus group is still being conducted and contin-
or feel about a cultural practice may require ues past the last group.
transcript-based analysis. However, if research-
ers are trying to determine which of three sets Focus Group Analysis Tips
of educational materials is more appealing,
analysis based on notes may be all that is When analyzing focus group data, the analyst
required. No matter what level of analysis is needs to consider many different aspects of the
selected, breaking analysis into manageable focus group and its participants’ responses,
chunks keeps the material from becoming including words the participants use in the
overwhelming. The analyst must look for the discussion, context, internal consistency of the
major themes that cut across groups as well as participants’ views, frequency of comments,
a key insight that might have been shared by degree of agreement on a topic, intensity of
Oral or written Usually oral report Usually oral and Usually oral and Usually oral and
reports only. written report. written report. written report.
15
Richard A. Krueger and Mary Anne Casey
The analyst(s) should: • Listen to the tape, review field notes, and read
transcript if available.
1. Start while still in the group. • Use transcripts as the basis for the next
steps.
• Listen for vague or inconsistent comments and
probe for understanding. Option 2: Analysis without transcripts
• Consider asking each participant a final • Prepare summary of the individual focus group
preference question. in a question-by-question format with amplifying
• Offer an oral summary of key findings and quotes.
inquire if the summary is correct. • Share report with other researchers who were
present at the focus groups for verification.
2. Prepare for analysis Immediately after the • Use summaries as the basis for the next
focus group. steps.
• Draw a diagram of seating arrangement. 4. Analyze the series of focus groups within days
• Spot check tape recording to ensure the of the last group.
recorder picked up the discussion. If the tape
does not work, immediately take time to expand • Analyze groups within a category (for example,
written notes. Recreate as much of the discus- first analyze the groups of parents, then the
sion as possible. groups of teachers, then the groups of stu-
• Turn the tape recorder back on and record the dents).
observations of the moderator and assistant • Analyze groups across categories (for example,
moderator. compare and contrast the parent groups with the
• Discuss issues such as: What seemed to be the teacher and student groups).
key themes of this discussion? What was • Look for emerging themes by question and then
surprising? How did this group compare with overall.
prior groups? Do we need to change anything • Construct typologies or diagram the analysis if
before the next group? Note hunches, interpre- appropriate.
tations, and ideas. • Describe findings and use quotes to illustrate.
• Label and file field notes, tapes, and other
materials. 5. Prepare the report.
3. Analyze the individual group within hours of • Consider narrative style versus bulleted style.
completion of the focus group. • Use a few quotes to illustrate each important
point.
Analysts can proceed with or without transcripts. • Sequence could be either question by question
or by theme.
Option 1: Transcript-based analysis • Share report with other researcher(s) for
• Make back-up copy of tapes and send tape to verification of findings.
transcriber if necessary. • Revise and finalize report.
feeling toward a topic, specificity of re- Context. Participant responses were triggered
sponses, and “big ideas” that emerge from by a stimulus—a question asked by the mod-
the discussion. erator or a comment from another participant.
Examine the context by finding the triggering
Words. Think about both the actual words stimulus and then interpret the comment with
used by the participants and the meanings of its environment in mind. The response should
those words. Some words are powerful, color- be interpreted in light of both the preceding
ful, or very descriptive. Different participants discussion and the tone and intensity of the
will use different words and phrases, and the oral comment.
analyst will need to determine the degree of
similarity among these responses. Internal consistency. Participants in focus
16
Designing and Conducting Focus Group Interviews
groups sometimes change, and even reverse, weight than responses that are vague and
their positions after interaction with others. impersonal. To what degree can the respondent
This phenomenon rarely occurs in individual provide details when asked a follow-up probe?
interviews due to a lack of interaction from Greater attention is often placed on responses
other participants. When there is a shift in that are in the first person as opposed to those
opinion, the researcher typically traces the in the hypothetical third person. For example,
flow of the conversation to find clues that “I feel the new practice is important because I
might explain the change. What seems to have have used it and been satisfied” has more
prompted the change? weight than “These practices are good and
people in the area should use them.”
Frequency. Frequency is the measure of how
often a comment was made. Frequency alone Finding big ideas. One of the traps of analysis
does not tell us how many different people is focusing so much on the detail that the
made this particular comment. Indeed, the analyst misses the big ideas. Step back from the
same person may have made the same, or discussions by allowing extra time for big ideas
similar, comments 10 times within the course of to percolate. For example, after finishing the
a single discussion. Do not assume that fre- analysis, the researcher might set the report
quency is an indicator of importance. It is not aside for a brief period and then jot down the
necessarily true that items that are discussed three or four of the most important findings.
most often are most important. Assistant moderators or others skilled in
qualitative analysis might review the process
Extensiveness. Extensiveness is the measure and verify the big ideas.
of how many different people made a particu-
lar comment. This measure gives the analyst a The Old-Fashioned Analysis Strategy: Long Tables,
sense of the degree of agreement on a topic. Scissors, and Colored Marking Pens
Unfortunately, it is impossible to determine
extensiveness using only the transcript unless An analyst who has not analyzed focus group
names are attached to comments. If the analyst data before may want to try this strategy. It is
was present in the focus group, he or she will a concrete way of categorizing and “seeing”
have a sense of the degree of extensiveness, and the data. After the analyst understands this
this can be captured in the field notes. In focus method, it is easier to understand how this
group analysis, extensiveness is a more useful process can be accomplished using computer
concept than frequency. software. The equipment needed includes two
copies of all transcripts, scissors, tape, lots of
Intensity. Occasionally, participants talk room with long tables and possibly chart
about a topic with a special intensity or depth stands, large sheets of paper (flip charts,
of feeling. Participants will sometimes use newsprint paper), colored marking pens, and
words that connote intensity or tell the mod- stick-on notes. This analysis strategy has 10
erator directly about their strength of feeling. components:
Intensity may be difficult to spot with tran-
scripts alone because the voice tone, speed, and 1. Prepare the transcripts for analysis. The
emphasis on certain words are key to commu- analyst will save time and agony later if he or
nicating emotion. Individuals will differ in how she is careful in preparing the transcripts. Be
they display strength of feeling. Strong emotion sure they follow a consistent style. For ex-
may be evident in fast, excited speech in some ample, single-space comments and double-
people and in slow, deliberate speech in others. space between speakers. The moderator’s
Pay attention to what is said with passion or comments should be easily identifiable by
intensity. bolding, capitalizing, or underlining.
Specificity. Responses that are specific and 2. Make two copies of each transcript. One will
based on experiences should be given more be cut up, and the other one stays intact for
17
Richard A. Krueger and Mary Anne Casey
later reference. At this point, the analyst may will be several different points of view, and
want to follow a tip that will be useful at later the analyst can cluster the quotes around
stages of the analysis. Consider printing tran- these points of view. The quality and rel-
scripts on different colors of paper and color- evance of quotes will vary. In some groups,
coding by audience type or category. For the analyst may find that almost all of the
example, copy the comments of teenagers on quotes can be used, but in other groups, there
blue paper and those of parents on green paper. will be few usable quotes. Set the unused
In addition, use a marker to put one line down quotes aside for later consideration. If a
the right margin of each page of the first parent participant’s comments are really addressing
transcript, two lines down each page of the another question, tape the comment under the
second parent transcript, three lines down each question it addresses.
page of the third parent transcript. This way,
once the analyst has all of the transcripts cut 7. Move similar quotes into categories or “piles.”
up, he or she can easily see that a quote on As the analyst reads each quote, he or she
green paper with two lines down the side came needs to reflect on whether it is similar to or
from the second parent group. To take this one different from other quotes already assembled
step further, most word processing programs and put similar quotes together. If a quote
allow numbering each line. raises different issues or ideas, then create a
new category and a separate pile for this
3. Arrange transcripts in an order. The order information.
can be based on the sequence in which the
groups were conducted, but transcripts are 8. Write a statement about each question. Look
more likely to be arranged by category of over the quotes and prepare a paragraph that
participant or by the demographic screening describes responses to that question. A number
characteristics of participants (for example, of possibilities may occur. For example, the
users, non-users, and employees, or teens, analyst may be able to compare and contrast
young adults, and older adults). This arrange- differing categories. There may be a major
ment helps the analyst to be alert to changes theme and a minor theme. The analyst may
that may be occurring from one group to discuss the variability of the comments, or even
another. the passion or intensity of the comments.
Following the overview paragraph, several
4. Read all transcripts at one sitting. This quick additional paragraphs may be needed to
reading reminds the analyst of the entire scope describe subsets of views or to elaborate
of the focus groups and refreshes his or her selected topics. Compare and contrast how
memory of where information is located, what different audiences (for example, parents,
information is missing, and what information teachers, students) answered the question.
occurs in abundance. (If the analyst color-coded the transcripts,
then the colors easily help him or her “see”
5. Prepare large sheets of paper. Use a large how the different audiences answered the
sheet of paper for each question. Place the large questions.) When finished, go on to the next
sheets on chart stands, on a long table, or on the question.
floor. Write the question at the top of the sheet.
9. Take a break. Get away from the process for
6. Cut and tape. Read responses to the same a while. Refocus on the big picture. Think
question from all focus groups. Cut out rel- about what prompted the study. It is easy to get
evant quotes and tape them to the appropriate sidetracked into areas of minor importance. Be
place on the large sheet of paper. Look for open to alternative views. Be skeptical. Look
quotes that are descriptive and capture the over the pile of unused quotes. Think about the
essence of the conversation. Sometimes there big picture.
18
Designing and Conducting Focus Group Interviews
10. Prepare the report. Invite a research col- time will vary with the typist’s speed, the
league to look over the report and offer feed- quality of the tape recording, the length of the
back. session, the experience of the typist with focus
groups, and the complexity of the topic.
Transcribing Focus Group Interviews
Special Topics
In transcribing focus group interviews, keep
these suggestions in mind. Involving the Community in Focus Group Research
Use quality playback equipment. The typist In the past decade, a variation of focus group
should avoid tape players with small speakers research has emerged that engages the commu-
and awkward buttons. Earphones sometimes nity in the study. In these studies, the re-
provide greater sound clarity. Focus group searcher takes on a different role. Instead of
interview tapes always have background noise, being the outside expert coming in to conduct
and participants speak with different tones and the study, the researcher becomes the facilita-
voice levels. Therefore, these tapes will require tor, teacher, coach, and mentor. The researcher
concentration and the best quality playback works with, teaches, and guides one or more
equipment that can be obtained. If possible, use people from the community who, in turn,
equipment with a tape speed control and foot- recruit participants, conduct focus groups, and
operated rewinding. help with the various aspects of the focus
group study. These community members can
Minimize distractions. Type transcripts in a be volunteers but often are employed for the
place with minimal distractions or interrup- duration of the study.
tions. Many of these community-based studies
have been on topics relating to policy develop-
Identify moderator statements. Use bold print ment, community development, or prevention
for the moderator’s statements and questions. (drugs, teen use of alcohol, teen pregnancy,
If needed, type the name of each speaker violence). In these studies, the researcher
followed by his or her comment. (If the person trained and supervised local people who took
doing the analysis was not in the groups, it will on specific tasks in the focus group study. As
be helpful to have the names attached.) Single- mentioned earlier, if the researcher does not
space the comments and double-space between know the language of the target audience, he or
speakers. she should seek out a local person who is
trusted by the target audience and fluent in
Type comments word for word. People do not the local language. This person should receive
talk in complete sentences. Insert punctuation training and coaching from the researcher
where it seems appropriate. Avoid the tempta- and then be sent out to conduct the focus
tion to add or change the words, or to correct groups.
the grammar. If some of the words are unintel- Some tasks are easier to share with people in
ligible, type an ellipsis (“...”) to indicate that the community than others. Among the easier
words are missing from the transcript. tasks are:
Note special or unusual sounds that could help • Recruiting. Invite people to attend the focus
analysis. For example, note laughter, loud group (in person or on phone) with written
voices, or shouting in the transcript in paren- follow-up reminders.
thesis. • Moderating. Guide the discussion in several
focus groups using predetermined questions.
Allow sufficient time. Typically, it takes about • Recording. Take notes, listen to the focus
8 hours to transcribe a 60-minute tape. But the groups, and assist the moderator.
19
Richard A. Krueger and Mary Anne Casey
Other tasks can be more complicated and However, involving the community in focus
require more time, skill, and practice. These group research has some disadvantages. For
include: example, the studies may take more time
because the researcher must recruit, train, and
• Planning. Develop the overall strategy and supervise the team. In addition, the research
determine the time schedule, budget, and team members may interpret results differently
groups to be conducted. than would the expert researchers (although
• Developing questions. Determine the questions this could also be an advantage).
to be used for a focus group study. Involving the community in focus group
• Testing questions. Pilot test the focus group research often works best in situations in which
questions with a specific audience. local people are able to obtain information
• Transcribing. Type the results of a focus otherwise unavailable to researchers, help
group while listening to the audio tape. stretch scarce resources, increase the likelihood
• Reporting. Share written results and present that the results will be used, and develop new
an oral report with a specific audience. skills or insights about their community.
Still other tasks are among the most compli- Telephone Focus Groups
cated and difficult to share. These skills require
considerable time to develop and perform, and Focus group discussions can be conducted on
they include: the telephone. With a conference call, a mod-
erator can carry on a focus group discussion
• Analyzing. Prepare a three- to five-page with people scattered around the country. The
question-by-question summary of the focus primary advantage of a telephone focus group
group or analyze across the groups and is that it allows greater potential for partici-
write the report. pation. This is particularly true for busy
• Coordinating. Overall coordination of the people or people who are geographically
entire project must include all budgets, time- dispersed.
schedules, logistics, and equipment. The principle disadvantage of telephone
focus groups is that the researcher misses the
The researchers need to weigh what is nonverbal communication. Much is gained in
gained and what might be lost if the commu- focus groups by bringing people together and
nity becomes engaged in the study. Involving being able to observe the participants’ reac-
the community gives several major tions—head nodding, signs of boredom, smiles,
advantages: frowns, alertness, interest in the topic—all of
which are lost on the telephone. In effect, a
• Focus groups can be conducted in different telephone focus group will lack the richness of
languages and dialects by moderators who in-person focus groups.
are trusted by the participants. In comparison to in-person focus groups,
• Focus group participants (and especially the telephone focus groups are shorter and have
research team) feel a sense of ownership of fewer participants and questions. In most
the study. situations, two hours is too long to be on the
• Because the community members have telephone. We recommend one-hour telephone
muted the bias that might occur from the focus groups. Because we have less time, we
sponsoring organization, results can be more recruit only four to six people for a phone focus
authentic. group and limit the number of questions. We
• The results are deemed more believable and also send out the questions ahead of time,
trustworthy by local residents because they which seems to make the limited time more
were familiar with the people conducting the productive. The advance preparation helps the
study. participants know the direction in which we
• Local team members gain skills. are heading, know what they want to say, and
20
Designing and Conducting Focus Group Interviews
stick with us mentally, even when we are not situations. The focus group, which has gained
together physically. considerable popularity among Western devel-
Some have argued that telephone focus oped countries, has been found to have distinct
groups are less intimidating because partici- advantages in developing countries as well:
pants cannot see one another. As a result, there
may be greater clarity of language because of • Participants do not need to be literate.
the absence of eye contact, gestures, and • Focus groups do not depend on mail service
nonverbal signals. Participants cannot detect or telephone systems.
signals of approval or disapproval from other • The process is familiar in that it resembles
participants or the moderator. The difficulty is the decisionmaking process of many cul-
that the moderator has limited ability to detect tures. Having a small group of people talk
whether someone is “tuning out” of the conver- about a current topic of interest is a very
sation and paying more attention to reading basic interaction.
the newspaper, reviewing documents, or
working on a computer. In summary, the However, the task of conducting focus
moderator has no reliable indicator of the groups in developing country settings is beset
degree of attention that a participant is giving by a number of challenges that researchers
to the discussion. should consider.
If the researcher chooses to use telephone
focus groups as the discussion format, it is Power differentials. The first and most impor-
useful to keep several suggestions in mind (see tant challenge is the power differential between
box 5). focus group participants and the sponsor. The
sponsors of most international focus groups are
International Focus Groups organizations that control resources, people, or
power. This power differential in conjunction
Increasingly, focus group research is being with cultural differences has the potential to
conducted in cross-cultural and international create communication problems.
These power differentials almost always
cause difficulties in focus group research. For a
Box 5. Tips for Telephone Focus Groups focus group to work, participants must be able
to talk without feeling threatened or fearing
• Invite a small group of only four to six people. future reprisal. The researcher must set up the
• Send questions or discussion topics to the
focus group so that there is no obvious advan-
participants in advance of conducting the
focus group. Include any visuals, with each
tage or disadvantage to providing either
item labeled and in the order that they will be positive or negative information. The re-
used. searcher must clearly communicate that all
• Limit questions to five to eight. points of view are valued and appreciated. This
• If voices are not recognizable, ask partici- clear communication of openness begins with
pants to identify themselves before speaking. the first contacts with community leaders,
• Limit discussion to about 60 minutes.
continues as participants are recruited, and is
• Call on people who are not audibly participat-
ing.
repeated when the group begins. Researchers
• Consider asking participants to reflect for a should anticipate that they might hear points of
moment or to jot down something on a piece view that are critical of the sponsoring agency
of paper and then read the comments. It is or upsetting to themselves. The moderator
fine to include questions that ask participants must take care not to show emotional reactions
to rate or assign values to ideas or things. when participants offer negative views on a
Keep the scales simple and logical.
program. It is often advisable for the moderator
• Include ending questions such as the “all
things considered” or the “have we missed not to be a person who has traditionally repre-
anything?” question. sented the international organization, because
there is a tendency for participants to hold back
21
Richard A. Krueger and Mary Anne Casey
22
Designing and Conducting Focus Group Interviews
Focus groups can be a powerful learning tool. Krueger, R. A., and J. A. King. 1998. Involving
They can help organizations listen to their Community Members in Focus Groups. Thousand
Oaks, Cal.: Sage.
employees and to the people whom they serve.
They can be used to assess needs and assets, Merton, R. K., M. Fiske, and P. L. Kendall. 1990. The
develop social marketing efforts, pilot test Focused Interview. 2d ed. New York: The Free
ideas and products, and evaluate services or Press.
programs. However, researchers must carefully
recruit participants, create a comfortable Morgan, D. L. 1997. Focus Groups as Qualitative
environment so that people feel safe to talk, Research. Newbury Park, Cal.: Sage.
choose the right moderator for the group, and
Patton, M. Q. 1990. Qualitative Evaluation And
record and analyze the data systematically.
Research Methods. 2d ed. Newbury Park, Cal.:
Above all, the researchers must respect the Sage.
participants. The researchers must believe they
have something to learn from the participants Tesch, R. 1990. Qualitative Research: Analysis Types and
and be open to hearing what is shared. Software Tools. New York: Falmer.
23
3. Using Q-Sorts in Participatory Processes:
An Introduction to the Methodology
Jonathan C. Donner
24
Using Q-Sorts in Participatory Processes
this change process. It was administered and particular statement or concept versus another
analyzed at the start of a five-day offsite at does not need to be known or even hypoth-
Monitor’s headquarters.1 esized in advance. Neither do the elements
have to be mutually exclusive nor completely
Overview exhaustive of all the possible concepts that
could apply. The elements are assumed to be
After a brief overview, roughly sketching the simply a subset of the possible concepts that
methodology, this chapter will present four may be important to the issue at hand, just as
sections: construction, administration, data the participants may be considered a subset of
analysis, and interpretation. At each stage, the the possible stakeholders.
SDV data will be used to illustrate the steps Let us take a simple example to begin. If we
involved. wanted to know more about at what kind of
Q-methodology was developed by William restaurant people would like to eat on a Friday
Stevenson (1953) in the 1930s and has benefited after work, we could arrange a Q-sort. We
over the past couple of decades by the develop- would ask the umbrella question: “Which
ment of statistical packages that make its statements best describe a restaurant you’d like
factor-analysis calculations easier to perform. to visit on a Friday after work?” We would
For a detailed methodological and statistical populate the sort with 30-odd elements ranging
exploration of Q, refer to McKeown and from “costs less than $15 per person” to “is
Thomas (1988). near to my home” to “has a hip clientele.”
Q-method requires participants to prioritize Participants would be asked to sort the 30
a set of 20 to 50 elements or statements in order elements in ascending order from least to most
from least to most desirable. The statements are attractive. A completed sort may look like table
often presented as multiple possible answers to 1, with the numbers corresponding to each
a given umbrella question, such as: “What is an element written in the answer sheet as such.
attractive outcome?” or “What is important for Q requires each participant to assign an exact
this group to study?” Q-methodology does not number of elements to each potential value. In
require a large number of participants—even the example above, 2 elements were required at
one is worthy of review—and meaningful, each of the extremes (-4 and 4), while 6 were
discernible groups can be found with as few as required at the neutral point of 0. There are
a dozen participants. It is quite rare to adminis- usually an odd number of column values to
ter a Q-sort to more than 100 participants. allow for the neutral column. It is also common
Although the root of the Q is the prioritiza- to lump more statements in the middle.
tion of elements, one of its strengths is that the Each person uses his or her own subjective
pattern or logic that drives the weighting of a criteria to evaluate the relative attractiveness of
25
Jonathan C. Donner
each element. Participants’ logic (perspective) holders perceive the issue you are confronting.
is their own; but since you are asking the same Instead of hypothesizing relationships between
question, using the same elements, to be placed items in advance and testing that structure, the
in the same format, you can compare these researcher gleans the relationship between the
subjective perspectives with more rigor than items only once the sort has been completed
normal qualitative methods would allow. (McKeown and Thomas 1988). Along with the
The results of a small Q-sort like this example easy and novel administration and fast turn-
could be ready to analyze within an hour or so, around, these theoretical elements make Q
using the freeware program PQMethod.2 The especially good for participatory processes, in
output (discussed in more detail in the analysis which the participants and eventual beneficia-
section below) would tell us: ries of the analysis are one and the same
(Argyris, Putnam, and Smith 1985). A bibliogra-
• Which criteria or elements were rated at phy of Q-related materials, compiled by Steven
roughly the same level (either high, low, or R. Brown and available on Peter Schmolk’s
neutral) by most participants. We call these QMethod Page is an excellent place to assess
“consensus items.” the breadth of research topics that have been
• Which elements garnered a real split deci- addressed using Q.3
sion, meaning they were highly agreeable to In the case of the Social Development Fam-
some participants and disagreeable to ily, the multidimensionality of the issues
others. We call these contention items. surrounding of SDV’s strategy were clearly
• And, perhaps most interestingly, the analysis evident. SDV managers were concerned with
would identify distinct subgroups within the choices regarding their role as advocates in the
set of participants who share a similar Bank, the introduction of training, the ideal
pattern of responses (meaning members of location for SDV staff (field or in Washington,
the subgroup agreed on the contention D. C.), and their capacity to offer independent
items). Each subgroup can be said to share a products directly to clients. Depending on to
similar perspective or voice about the topic. whom we spoke, some or all of these issues
were salient—and their relative importance in
It is up to us to infer the meaning of these charting a course for SDV was not clear. The
outcomes. We will get to the details of that issue of how SDV managers imagined the
process in the interpretation section. optimal course for SDV’s development was
ripe for further exploration via a Q-sort.
Construction
Umbrella Question
To create a Q-sort, you need a research topic, an
umbrella question, and some elements. The umbrella question to a Q-sort should be
broad enough to hold all the statements
Research Topic underneath. Thus, something as simple as:
“Sort the statements from least attractive to
Although the Q can fit a variety of research most attractive” imposes no additional
topics—from restaurant choices to program constraint on the participants. If you prefer,
plans to beliefs about fairness and justice, it is or if your research goals require it, you can
especially well suited to situations in which a use the umbrella question to bound the
single “issue” is made out of subdimensions, conditions of the sort in some way. “Where
and in which you are not necessarily sure how would you like to go on a Friday night,” for
all these sub-dimensions fit together. Consider example, puts participants in a frame of mind
it more exploratory than confirmatory, more of to evaluate attributes of restaurants for a
an opener than a conclusion to a process of particular event. Fill-in-the-blank questions
social inquiry. Q is effective as a way to discern can also be used, such as “In order for this
the lay of the land in terms of the way stake- department to be most effective, it must
26
Using Q-Sorts in Participatory Processes
__________” (participants are then asked to Elements can be created in a variety of ways.
sort phases according to how well they com- A researcher can create all the elements “from
plete the sentence). The SDV example de- scratch” if he or she desires. Literature reviews
scribed here assumes that the same statements may be of assistance, or an external list may be
will be evaluated once by a range of individu- used. Another approach is to involve partici-
als, using a single umbrella question, to deter- pants in the element-generation stage. In
mine what distinct perspectives may exist on advance, a researcher can ask some or all
the issue. However, you may also choose to participants to submit a few elements for the
look at how one or two participants rank-order sort. If an explicit invitation for submission is
the same statements under a variety of condi- not possible or desirable, a researcher can still
tions. This can be achieved by varying the draw on elements generated by the stakeholder
umbrella question accordingly. community by pulling comments or phrases
In the case of the SDV Q-sort, we wanted from sources that stakeholders have previously
everyone to consider what an “effective” SDV produced (flyers, papers, books, headlines).
would look like in the future but also wanted Even quotes from interviews can make good
to ensure that each participant was considering elements. Keep in mind that elements do not
the same time horizon. So, our umbrella ques- need to be lengthy sentences. Pictures or
tion set the scene: images, simple phrases, or single words can
work. This may be especially helpful with less
After an especially long journey back to literate populations. Brown’s bibliography,
DC for a Social Development meeting, mentioned earlier, is a good place to review
you awake in the morning to find you’ve different approaches to element design.4
really overslept. It is December 2002. You If a researcher has a few ideas in mind
arrive at 1818 H St. and find your col- about dimensions that may be critical, he or
leagues to see what they’ve been up to. she can “seed” the elements with items that
As it turns out, they have made great represent different levels of the dimension
progress over the three years you were (Brown 1993).
slumbering. . . . There is no clear rule of thumb for the
Which of the following statements number of elements that should be included,
describe this new and improved Social but sorts with as few as 20 or as many as 60
Development Family, as it appears to you items are possible. One simply adjusts the
in 2002? Sort these statements from “least structure of the answer curve accordingly.
describes” to “best describes.” There are a few rules of thumb you can use
to craft a good set of elements. First, try to
Elements choose elements that mean different things.
Items that are nearly repeats of one another are
The most challenging part of designing a Q- confusing to participants, as are pairs of items
sort is not in defining the umbrella question, that are the exact inverse of each other (for
but rather in selecting the elements that the example, restaurant takes credit cards; restau-
question asks participants to sort. The best rant does not accept credit cards).
thing to keep in mind is that while no list of Next, try to avoid extreme elements, so good
statements is perfect, no list has to be. Since (or so repulsive) that everyone you sample
what is really of interest are the tacit, underly- could be expected to either agree (or disagree)
ing criteria and perceptions people use to with, to the exclusion of prioritizing other
consider an issue, the elements themselves are items. Instead, elements should be plausible
of secondary importance. Their job is to be competitors with one another, such that some
broad enough (and clear enough) to set these participants may be attracted to them and
tacit criteria and perceptions to work, and to others disinclined to choose them.
give the researcher insights about them once Finally, try to keep the elements parallel in
the sort is complete. style. Pick all sentences or all phrases. Avoid
27
Jonathan C. Donner
double negatives. Be clear. However you as small as possible while keeping the font
construct your list of elements, you may want legible and try to keep all the chips the same
to pre-test the items with a couple participants, size. Approximately 30 brief statements will fit
for clarity and general comparability. on an 8-1/2 x 11-inch page if the margins are
In the case of the SDV Q-sort, the Monitor set as wide as possible.6 Print them onto card
consultants initially developed the elements, stock or heavy paper, and use a paper cutter to
drawing on materials written by SDV. Then, a separate the elements from each other. Bind
team composed of SDV managers and the them together with an elastic band or put them
Monitor consultants reviewed the items, in an envelope. Double-check to make sure that
suggested revisions for clarity, and debated each set of chips is labeled consistently and
additions and subtractions until a list that was contains exactly one of each element. Duplicates
satisfactory to all was developed. Although no or missing items at this stage can ruin a sort.
pre-determined dimensions were specifically The guide strip helps participants structure
“seeded” into the sort, a list of sub-issues that the sort according to the pattern determined
the group felt SDV was facing was created, and in advance by the researcher. The guide strip
items were written to touch on them. See should be wide enough to accommodate the
appendix 1 for a list of the items selected. chips underneath. You can print out two
pages horizontally and tape them together to
Administration yield a strip about 18 inches wide and 3
inches high. The guide for the SDV Q-sort
With the elements prepared and tested, and the (22 elements) looked like the one in table 2.
umbrella question crafted, the administration The answer sheet can be a single page, with
becomes a matter of preparing the instrument the umbrella question on the top and the guide
and instructing the participants. strip (and answer spaces) underneath. It would
be a blank version of table 1, with the question
Preparing the Instrument written at the top. Make sure the number of
spaces matches the number and shape of the
The instrument has four components: (1) chips, elements as presented in the header. If you
(2) guide bar, (3) answer sheet, and (4) instruc- have any other tracking questions (group
tions. Since the sort should be tactile and affiliation, name, date administered, or any
iterative, it is difficult to avoid using (and other survey questions), you can place them on
creating!) little chips. There is PC software that the same page as the answer sheet. The sheet
can represent the sort on a screen using an we used for the SDV sort is attached as appen-
Internet browser,5 and it is possible to adminis- dix 2.
ter extremely brief sorts using a conventional Finally, it is helpful to include an instruction
survey instrument. However, basically, sheet for participants. Although spoken instruc-
nothing so far beats the cards. The rest of the tions are best, the backup is useful, especially
chapter assumes a “conventional” sorting as the number of simultaneous participants
technique. increases. The instructions we used for the SDV
The chips are what participants actually sort. sort are attached as appendix 3.
Write or type one element on each chip, and Enclose the guide strip, the chips, the answer
number the chips from 1 to X. Make the chips sheet, and the instructions in a folder, and you
28
Using Q-Sorts in Participatory Processes
are ready to turn to the participants and the like, until they arrive at a first pass on the
setting. sort, with the right number of statements
arranged under each column of the guide
Preparing the Setting and Instructing Participants strip.
5. Take a step back, review the sort, and make
You will need to consider the setting and the changes. Again, the process is intended to be
instructions you will deliver to participants. iterative (as participants consider the attrac-
The Q-sort can be administered in group or tiveness of each element, relative to the other
solo settings. In either case, participants need elements). Sometimes this second look will
a wide, flat workspace large enough to lay yield changes.
out the guide strip, sort the statements, 6. Record the sort. Participants can record their
visually review the entire sort (to make sorts by writing the number corresponding
adjustments), and then record their answers to each element in the space on the answer
without disturbing the sort. Sorts can be sheet that represents its place on the sort on
handled fairly quickly; allot 20 or 25 minutes the table. The answer sheet is a “snapshot”
for a sort of 30 statements. of the sort on the table.
The instructions should make clear to 7. Check the recorded sort. Have participants
participants the purpose of each of the four double-check their own entry for double-
components in the pack. If in a group admin- counts or blank spaces. They will not catch
istration, verbal instructions should be given every mistake but will catch a few.
immediately upon—if not before—distribu-
tion of the pack, since curious participants Be available for help as participants com-
will take things out of the pack and start plete the sort.
working with them as soon as they can. The As you collect the responses, try to run your
seven basic steps that participants should be eyes over each sort to check for mistakes and
instructed to follow are: blanks. Mistakes can be hard to remedy later,
and if one item is wrong, the participant’s
1. Count the chips. Make sure the proper num- whole sort is invalid. Once the sheets are
ber of chips are there. collected, you can take back the chips and
2. Review the question (the instructions or header. Carefully check the number of chips
context for the sort). Make it understood that before reusing them.
each chip represents one element that can be
interpreted as “more” or “less” important to Data Analysis
the question at hand.
3. Review the guide. See the shape into which Analysis can take as little as a couple of hours
the elements must be sorted. There is no or considerably longer, depending on the
impact on the order within columns, only number of iterations you wish to pursue and
between them. the degree of depth to which you want to work
4. With the question in mind, sort the items in a with the data. A normal statistical package with
multi-stage process. Most people cannot assign factor analysis capability is sufficient to per-
a number to each statement on the first pass. form the analysis, but a specialty program such
It is much easier to do a rough sort first, as PQMethod (mentioned earlier) makes things
categorizing items into three piles: less much easier.
important, neutral, and more important. It This section will present the minimal num-
does not matter how many statements go ber of steps necessary to yield a Q analysis. It
into each pile at first. If the participant then will presume and require little statistical
works to prioritize statements within each background and will be linked to the use of the
pile in turn, the sort will begin to emerge. free PC (DOS) program PQMethod. Research-
Participants should be encouraged to adjust ers interested in a more in-depth treatment of
statements on the fly, as many times as they the statistical underpinnings of Q, the various
29
Jonathan C. Donner
options and nuances of the method, and the 1. Launch PQMethod. It is free. Download it at
range of possibilities should examine McKeown the Q-method page (mentioned earlier). It is
and Thomas (1988), or Steven R. Brown’s available for PC (DOS), Mac, or Unix. These
primer on Q-methodology (1993). The latter instructions refer to PQMethod for PC (DOS).
appears in Operant Subjectivity, a journal dedi- Make sure to get the manual, as it will help
cated almost exclusively to Q-methodology. with the downloading and installation. This
Finally, this summary assumes a Q targeted at chapter is not a tutorial on DOS, nor is it a
multiple participants at a single administration. substitute for the PQMethod manual. What it
Slightly different methods would be used to will do is guide you through the steps with an
look at a single participant’s sorts under eye toward getting to the three key outputs as
different umbrella questions. easily as possible.
Starting with the end in mind, a Q-sort
creates three basic outputs for analysis and 2. Enter the statements and data. Once
discussion: PQMethod is running, you will be asked to
name the project for DOS. This is straightfor-
1. The distinct groups, or common “perspec- ward, as long as you do not mind saving to the
tives” that appear in the sample. Using a form “projects” folder within the PQMethod folder.
of factor analysis, the 10, 20, 30, or 50 partici- Otherwise, you will have to specify a path
pants who completed the sort are compressed name. Keep the study name to 8 characters. I
into 3, 4, 5, or more subgroups, each reflecting a chose SDVV2000 for this project. You then see
common pattern of responses. Each of these the main dialogue screen for PQMethod (table
subgroups can be portrayed with a “snapshot,” 3)—the one you will return to between each
summarizing the average sort of the partici- operation.
pants in that subgroup. It is useful to name Once you have named the project, the first
these subgroups with an evocative summary stage in the PQMethod analysis is to enter the
title. statements routine: STATES. These correspond
2. Contention elements are elements that to the elements you have used for the sort. You
distinguish one subgroup from another. The can type them by hand into the DOS editor,
farther apart the rankings of any two sub- specify a different text editor open automati-
groups on an item, the more contentious that cally, or tell the software to find a file. The third
element is likely to be. Identification of these option may be the easiest: prepare the file in
items can prompt excellent discussions in
participatory sessions.
3. Consensus elements are elements generally Table 3. Main Dialogue Screen for PQMethod
agreed on by members of most groups (every-
one agrees, or everyone disagrees). If the Current Project is ... c:/pqmethod/projects/sdv2000.
Choose the number of the routine you want to run
discussion of the contention items gets out of
and enter it.
hand, it is handy to have the consensus items in
your back pocket. 1 – STATES - Enter (or edit) the file of statements.
2 – QENTER - Enter Q-sorts (new or continued).
To generate these three outputs, there are six 3 – QCENT - Perform a Centroid factor analysis.
steps to be performed using PQMethod: 4 – QPCA - Perform a Principal Components factor
analysis.
5– QROTATE - Perform a manual rotation of the
1. Load and launch PQMethod. factors.
2. Enter the statements and data. 6– QVARIMAX - Perform a varimax rotation of the
3. Extract initial factors. factors.
4. Rotate the factors. 7– QANALYZE - Perform the final Q analysis of
5. Group participants. the rotated factors.
8– View project files sdv2000.*
6. Generate the data run(s).
X– Exit from PQMethod.
30
Using Q-Sorts in Participatory Processes
advance with Microsoft Word or Notepad and • These factors are rotated to arrive at clearer
save it as a text-only file (.txt). Then, rename it representations of distinct patterns of obser-
with the correct extension for PQMethod, vations.
replacing .txt with .sta. As you enter state-
ments, keep two things in mind: (1) limit There are slight differences between a factor
your statements to 60 characters each, and analysis (in which variables that exhibit similar
(2) make sure you enter the statements in the patterns are grouped) and Q-analysis (in which
correct order, since the program assigns num- people who exhibit similar patterns of re-
bers to each statement automatically. Separate sponses are grouped), but for details on this
statements with a simple hard return (line topic, please visit two texts that handle the
break). seemingly bottomless nuances of Q-statistics
Now you are ready to enter the data, using better than this author could: McKeown and
routine #2: QENTER. To build the data matrix, Thomas (1988) or (Brown 1993).
PQMethod will ask for four pieces of data: The first two phases of this triad of opera-
tions are completely automatic for beginners.
1. A title for the project Running routine #4 QPCA will derive the
2. The number of statements that were sorted correlations and then perform an initial factor
3. The leftmost and rightmost column values in analysis using the principal components
the sort (for example, -3 and 3) method. This will give you the raw building
4. The number of statements that appear under blocks for the evaluation step (rotation) that
each column (for example, 2, 3, 4, 4, 4, 3, 2). will follow. Skip routine #3 QCENT. QCENT is
an alternative method of extracting initial
Once the software has the shape and size of factor, which would run a factor analysis using
the curve, its process for guiding you through centroid extraction. The difference between the
data entry is fairly painless. Starting with the two approaches is negligible for most Q-sorts
identification number, PQMethod will ask you (McKeown and Thomas 1988).
for the numbers for each statement that ap- The main benefit of QPCA over QCENT is
pears under each column. If you mis-type, or if that QPCA gives you eigenvalues for each
your data are flawed, the system will not let factor, which can help you decide how many
you continue. You can also re-visit or edit sorts groups you eventually want to build. Eigenval-
you have previously entered. The system is not ues are a measure of the relative contribution of
perfect—there are many hard returns between a factor to the explanation of the total variance
you and a completed data matrix. However, in the correlation matrix. Factors with an
the system is better than alternatives in SPSS or eigenvalue greater than one explain more
Excel because it allows you to enter the data in variance than a single variable would. Thus,
the shape in which they were gathered. If you the maximum number of factors you would
gathered other information while doing the want to carry into the rotation step is equal to
sort (gender, age, attitude data), this system is the number of initial factors with eigenvalues
not as effective, but you can handle those greater than one. Table 4 displays the eigenval-
variables in Excel or any statistical package ues extracted by QPCA on the SDV data. In this
once the Q-sort is complete. case, the maximum number of factors to carry
into the next step would be 5 (although #5 is
3. Extract the initial factors. At the core of a Q- just barely above one).
sort is set of three procedures that look very
much like a factor analysis (Kim and Mueller 4. Rotate the factors. This is the first stage in
1978): which there is real flexibility in the process for
beginners, since the rotation stage allows you
• Correlations between observations (partici- to specify the number of factors (subgroups of
pants’ sorts) are derived. participants) you want to carry forward into
• Initial factors are extracted. the analysis. As mentioned above, the QPCA
31
Jonathan C. Donner
procedure yields eigenvalues for each initial mizing the variance between each of the
factor extracted. Using this information (dis- factors. (It adjusts the weights given to each
played in table 4), you should have identified element on each factor.) Launch the procedure
the maximum number of factors you will rotate and specify the number of factors you want to
(based on all factors with an eigenvalue greater rotate. The DOS version of PQMethod will ask
than one). However, you should not be guided you whether you want to use the newer graphi-
by the eigenvalues alone. It may be the case cal procedure. You should opt to do so.
that choosing fewer factors will yield sufficient Table 5 illustrates roughly what the output
dispersion to make the story clear. The more from the rotated factor solution will show.7
factors you have, the more fragmented the data Each factor is a column; each participant is a
will become. You will have to judge for your- row. The numeric value in each cell indicates
self whether a 3-, 4-, or 5- (or more) factor the strength of the participant’s “loading” onto
solution is most compelling. Thus, you will the factor. Loads can be as high as 1 (indicating
probably want to run the analysis more than perfect agreement), or as low as –1. In the case
once, starting with the smallest number factors of negative values, participants can be said to
you think you will accept, and ending at the disagree with the factor. So, in the case of the
maximum number you think you can rotate, SDV data, participant #3 (R3) loaded highly
based on the eigenvalues. (0.8) onto factor 1, and relatively strongly (-0.4)
PQMethod offers two procedures for onto factor 3. The more a participant loads
rotating factors. Routine #5, QROTATE, offers cleanly (disproportionately) onto a single
functionality to rotate factors by hand and by factor, the better that factor represents that
sight. This is a more advanced procedure, participant’s sort—and subjective perspective
which will not be covered in this chapter. on the issue at hand.
Skip routine #5: QROTATE.
The second procedure for rotating factors is
5. Group participants. The final aspect of the
#6: QVARIMAX. Commonly applied through-
rotation sequence is to assign participants to
out the social sciences, and likely to yield an
factors. This is the step at which you are creat-
acceptable result, this varimax procedure is the
ing the subgroups and establishing the founda-
one you should use. Varimax rotation clarifies
tion for the calculation of the distinct “voices”
the structure of your chosen factors by maxi-
present among your participants. The software
Cumulative
Eigenvalues As percentages percentages
32
Using Q-Sorts in Participatory Processes
Table 5. Example of Rotated Factor Solution: of the automatic pre-flagging on the left for the
SDV Data SDV data, and the manual edits I made on the
right.
Fac1 Fac2 Fac3 Fac4
In this case, the only manual change I made
R1 0.38 0.10 0.80 0.74 to the automatic pre-flagging was to switch the
R2 0.14 0.76 -0.14 0.30
affiliation of participant R15 from factor 2 to
R3 0.80 0.32 -0.40 -0.10
R4 -0.15 0.78 0.27 0.14 factor 4. Negative loads, although clear in
R5 -0.10 0.21 0.88 -0.50 theory, are difficult for participants to under-
R6 0.80 0.42 0.57 0.22 stand. Imagine this conversation: “Well . . . Bill,
R7 0.68 0.20 0.32 0.17 Bob, and Mary belong to subgroup 2. But as for
R8 0.73 -0.20 0.70 0.45 Joe, Joe’s defining characteristic is how little he
R9 0.29 -0.11 0.64 0.13
0.29 0.50 0.16 0.62
agrees with Bob, Bill, and Mary.” The 0.49 load
R10
R11 0.87 0.30 0.16 0.16 of R15 on group 4 is not great, but not bad
R12 0.13 0.30 0.42 0.50 either: the minimum threshold for loading on a
R13 0.30 -0.19 0.71 0.26 group is just about 0.50. The first tip, therefore,
R14 0.55 -0.60 0.20 0.58 in terms of how to manually adjust automatic
R15 -0.42 -0.67 0.60 0.49 pre-flags is to avoid assignment to groups
R16 -0.10 0.70 0.20 0.80
R17 0.12 0.17 -0.30 0.66
based on negative loads.
The second tip is to avoid or ignore people
who do not load cleanly onto a factor. For
can begin this process for you by performing example, in the SDV data, R14 sits at 0.55 on
an “automatic pre-flag.” All it will do is mark factor 1, but also at 0.58 at factor 4. I chose to
the cases that load cleanly onto a factor. After not assign that participant to a group. Again,
that, you may want to manually adjust the the more factors you run, the less likely it is
flags to compensate for close calls or other that you will encounter double- or triple-
nuances in your data. Table 6 shows the results loading and the cleaner your factors are likely
33
Jonathan C. Donner
to be. However, you will also have more • Correlations between each factor. This demon-
subgroups to discuss at the end, and fewer strates how similar each factor (subgroup) is
consensus statements left in the data. to each other factor.
• Normalized factor scores for each factor
6. Generate the data run(s). Once you have (subgroup). These tables go factor by factor,
assigned participants to subgroups, you are listing all the statements in descending order
ready to submit the prepared data back to the of ranked importance. You will use these
PQMethod software for analysis. Select routine pages to determine the overall character of a
#7, QANALYSE, and let it write the results to a subgroup’s perspective. The Z-scores show
list file. You can print it out or save it in how far from the overall mean (measured in
Microsoft Word, according to what you need. standard deviations) each item is for the
At this stage, it is easy to run two or three group.
different analyses at once, with different • Array of differences between factors. This block
numbers of factors and assignments to groups, of tables allows for per-item comparisons
and then print them all to compare side by between each factor.
side. At that point, you can close PQMethod. • Factor Q-Sort values for each statement. These
two snapshot pages give you the “voice”—
Interpretation average sort—for each factor group. For easy
interpretation, the Z-scores are translated
The PQMethod file you generate for each run back into the original scale used in the sort.
will contain a number of items and is likely to These are sorted in order of statement
be over 20 pages long. There is more detail than number, then by degree of agreement be-
you need as a beginner, but it is quite conve- tween the groups.
nient to have everything generated at once and • Factor characteristics. This reports the number
in one place. Items generated include the of defining variables (statistically distinct
following (items needed for a basic evaluation from other groups) and other indexes of how
of the results are marked with an asterisk): well the factor holds together.
• Distinguishing characteristics for each factor.
• Correlation matrix. This shows the correlation These tables highlight contention statements
between individual pairs of Q-sorts. that subgroup members have ranked signifi-
• Unrotated factor matrix. This will contain cantly differently from other subgroups
eight factors if you choose QPCA, plus the (higher or lower than overall average). Use
accompanying eigenvalues and the percent- these tables to help define the key differ-
age of variance each factor represents. ences among subgroups.
• Rotated factor matrix, with the number of • Consensus statements. These items do not
requested factors, the item loadings, and distinguish between any pair of sub-
percent of variance explained by each factor. groups.
(The format looks similar to table 5 above.
Unlike the display for the unrotated matrix, The first key output from the analysis—
this table will contain information only for profiles of the perspective of each distinct
the factors you selected (in this case, four)). subgroup—is derived using three sources: the
• Free distribution data results. The participant- factor Q-sort values, the normalized factor
by-participant summaries of the mean and scores, and the distinguishing characteristics of
standard deviations of their sort should be each factor. We will visit the contribution of
the same for each case and are therefore of each in turn.
little use. The Factor Q-Sort Values for each statement
• Rank statement totals for each factor. The provide a good snapshot of how each group
totals tell you how each factor (subgroup) ranked the items. Table 7 contains the example
ranked each statement. It can be useful, but from the SDV data. The values in the columns
the same information will be available in represent the “archetypal” pattern sort of the
clearer formats later in the analysis packet. participants assigned (flagged) to that factor.
34
Using Q-Sorts in Participatory Processes
From this Q-sort values page alone, you Now the profile is clear: items at the top of
could create “snapshots” of each group’s the table are ranked as more important by the
perspective on the issue. However, it is difficult group. Items at the bottom of the page are
to get a sense of the relative priorities of any ranked as less important.
one group without sorting for that group. So, The final piece of the profiling sequence is to
returning to output 7 (normalized factor scores) is isolate the elements in the sort that are distinc-
helpful. Table 8 contains the normalized factor tive for each group. Output 11 (distinguishing
scores for first subgroup in the SDV sort. characteristics for each factor) is helpful here.
35
Jonathan C. Donner
Table 9 illustrates the distinguishing character- networks) and item 6 (involved on every
istics for subgroup 1 in the SDV data. project) lower than average.
This table shows that the four participants in To create the summary profile of a sub-
subgroup 1 rate item 1 (skill mix), item 14 group, combine the three outputs discussed
(training for Bank staff) and item 12 (speed of above. Take the factor values for the subgroup
adjustment) higher than the average of the from table 7, put them in the order of the
other subgroups. They rate item 3 (NGO factor-specific sort from table 8, and flag the
36
Using Q-Sorts in Participatory Processes
Group 1 Statements
significantly different than Factor 1 Factor 2 Factor 3 Factor 4
overall mean @ p<0.05 (bold n=4 n=2 n=4 n=6
@<0.01)
Num Statement RNK Z RNK Z RNK Z RNK Z
SDV is comprised of people with
a mix of educational and
professional backgrounds with
1 2 0.87 0 0.00 -1 -0.46 -2 -0.74
management and client-interface
skills that match their technical
proficiencies.
SDV serves as a center for
14 training and development of 2 0.80 -3 -1.30 -2 -1.19 0 -0.08
other Bank staff on social issues.
SDV is able to quickly anticipate
12 and adjust its offerings based on 1 0.71 -2 -1.27 0 -0.05 -1 -0.63
the needs of client teams.
SDV manages a network of
external links to NGOs both in
developing and developing
3 nations, which enables SDV to 0 -0.36 2 1.27 -2 -1.06 -2 -0.98
play a role in coordinating
actions among multiple actors in
the development arena.
SDV is involved, through an
6 allocated staff member, on nearly -3 -1.50 1 0.65 -1 -0.60 1 0.16
every relevant Bank project.
significant variables from table 9 to tell the does not feel the need to be involved via an
story of the worldview of the four participants allocated staff member on “nearly every”
who you categorized into subgroup 1. Table 10 Bank project. (Mention the items the sub-
is an example of such a summary profile, again group ranked as defining low.)
using subgroup 1 from the SDV data. • Like many others, members of subgroup 1
We’re finally at the moment at which real see “world class research” and “stand-alone
interpretation can commence. Using this products” as less important. (Mention the
summary chart, a few patterns emerge: subgroup’s absolute lows, whether distin-
guishing or not.)
• Like others in the sample, members of
subgroup 1 want SDV to be a leading re- At this point, a researcher could show the
source within the Bank for knowledge about data chart above, the bullet points, or some
and solutions to “social issues” and to be combination. If there were demographic clues
involved in policy formation. (The highest to the composition of the group, there may
items are worth mentioning, even if they are be additional points of differentiation to
not unique.) discuss.
• Subgroup 1 is particularly interested in We often find that naming the group with a
developing a mixed skill-set (including brief, evocative, sometimes colorful name can
management skills) and in supplying train- help anchor the group in audience’s minds. On
ing on social issues to other Bank staff. (Note the other hand, it can be a great interactive
the defining high items, even if they are not exercise to have the participants name the
top-ranked.) group together. Often, they will have a person
• Subgroup 1 is relatively less interested than or an archetype in mind that helps this particu-
others in managing a network of NGOs and lar worldview come alive for them. It also
37
Jonathan C. Donner
begins the process of having the participants pants, the last clue that makes helps in the
take the perspective of another, as they try to naming and interpretation of this profile is
ascertain what the members of subgroup 1 other profiles. Seeing how subgroup 1 looks
were thinking as they prioritized the items. different from subgroups 2, 3, and 4 can make
This is a key benefit of the playback/participa- interpreting subgroup 1 a little easier. Tables
tory process. 11-13 show the profiles for the other three
Whether naming is done by the researcher subgroups, along with some explanatory
ahead of time, or in collaboration with partici- bullets.
38
Using Q-Sorts in Participatory Processes
• Like others in the sample, the two members of • Subgroup 2 is relatively less interested in
subgroup 2 want SDV to be a leading resource training non-Bank staff. Most interestingly,
within the Bank for knowledge about (but not they feel strongly that “constant innova-
necessarily solutions to) “social issues” and to tion” may be less important than other
be involved in policy formation. goals.
• Subgroup 2 is particularly interested in • Like some others, subgroup 2 is not as con-
going to the front lines and in offering its cerned with training Bank staff or with SDV’s
products directly to clients in countries. role in compliance reviews within the Bank.
39
Jonathan C. Donner
• Like others in the sample, the four members sion and providing solutions at the regional
of subgroup 3 want SDV to be a leading level. They are also relatively more inter-
resource within the Bank for solutions to ested in SDV’s role in producing world-class
(but not necessarily knowledge about) research.
“social issues” and to be involved in policy • Subgroup 3 is least interested in training
formation. non-Bank staff. They are also less interested
• Subgroup 3 is particularly interested in in getting involved on all Bank projects and
affecting change in the Bank through persua- in having stand-alone projects. Finally, they
40
Using Q-Sorts in Participatory Processes
are less interested in being a source for cross- subgroup 4 want SDV to be a leading re-
cutting knowledge for the Bank of social source within the Bank for solutions to (but
issues. not necessarily knowledge about) “social
• Like some others in the sample, members of issues” and to be involved in policy
subgroup 3 are not as concerned with SDV’s formation.
role in compliance reviews within the Bank • Subgroup 4 is particularly interested in
and in training Bank staff. challenging the Bank to retain its focus
• Like others in the sample, the six members of on poverty and to serve as the primary
41
Jonathan C. Donner
advocate within the Bank for the needs of uncover different perspectives concerning
countries’ most vulnerable populations. It SDV’s future. These data became one of the key
also rates “affecting the Bank by persuasion” inputs to the change process during the five-
higher than others in the sample. day offsite.
• Subgroup 4 is less interested in continually At times, you may want to downplay the
innovating and getting involved in all Bank exact characteristics of groups, and focus
projects, and in providing solutions at the instead on individual items. Contention and
region level. consensus items are easy to list and discuss.
• Like some others in the sample, members of Table 15 shows the items we found in the case
subgroup 4 are not as concerned with SDV of the SDV group.
delivering stand-alone projects, and in SDV’s Looking at the sorts on a per-item basis
being a source for world-class research. yields a slightly different view. Three items
were consensus high. SDV can most likely
Based on these patterns, summarized below agree that it should remain (or become) a locus
in table 14, the SDV team and the Monitor for policy formation on certain social issues
consultants arrived at the following labels— and should be a source for knowledge and
representing four distinct visions for SDV (in solutions about social issues. This approach
2002, after a long slumber…). Clearly, in a identifies actionable consensus items.
small group of 17 managers, who work In terms of contention items, there were lots
closely (and well) together, we were able to of items with a reasonable gap (4 of 6 possible
42
Using Q-Sorts in Participatory Processes
43
Jonathan C. Donner
points). The three items concerning advocacy Q helps plumb knotty, multidimensional
and challenging the Bank fell in this category. problems in interactive and participatory ways,
Whether SDV should be an internal training in a short time, with just a dozen or so partici-
center was of even greater contention, as was pants. It is a great tool to have in your tool kit.
whether SDV needs to be constantly innovating Give it a try!
(in terms of developing new products).
Notes
Conclusion
1. Thanks to Gloria Davis, Judith Edstrom, James
This chapter summarizes the results of one Edgerton, Anis Dani, and Ashraf Ghani, among
particular four-factor Q-sort solution using others, for leading the SDV team that was evaluating
SDV’s data. As with any Q-sort, there is a great its strategy, and to James, Ashraf, and Anis especially
deal to explore by varying the number of for their help designing the Q-sort itself. The SD data
factors and the assignment of participants to displayed below represent a slight change from the
groups. As a first step, however, this chapter analysis originally presented to the SD group, but
should be a helpful guide to the process of the basic composition of the groups and topics
conducting a Q-sort. It really is easy; the addressed were not affected by the re-analysis of
these data. The re-analysis was necessitated by the
detailed statistics are not a barrier, thanks to
choice to use PQMethod as the recommended
PQMethod. Q-methodology is a very powerful
software package in this chapter.
tool, not only for analysis and research but also
2. PQMethod and its manual can be downloaded
as a starting point for interventions and partici- at no charge at Peter Schmolk’s QMethod Page
patory exercises. It can bring into clarity a http://www.rz.unibw-muenchen.de/~p41bsmk/
small number of dimensions to an issue that qmethod. It is available for DOS/Windows (PC),
previously felt too messy or thorny to ap- MAC, and UNIX platforms. All commentary in this
proach. Similarly, it can enable groups who chapter refers to PQMethod for DOS 2.09.
seem divided into factions by political or social PQMethod was updated and maintained by Peter
differences to see some commonalities of Schmolk, based on a mainframe program by John
beliefs across factions, and some differences Atkinson. There are also other programs available.
See the Qmethod page.
within them—a key step in beginning to reach
3. The bibliography’s link is http://www.rz.unibw-
group consensus.
muenchen.de/~p41bsmk/qmethod/syllabus00.pdf.
Participants like the novel administration
4. The bibliography’s link is http://www.rz.unibw-
method, enjoy the apparent magic of the sorts, muenchen.de/~p41bsmk/qmethod/syllabus00.pdf.
and are always interested in seeing their 5. WebQ can be found at: http://www.rz.unibw-
opinions translated into patterns and quanti- muenchen.de/~p41bsmk/qmethod/webq.
fied. With skilled facilitation, the resulting 6. I cannot show you an example from the SDV
conversations can be lively, are rooted in data, sort since the margins would be outside what is
and help participants consider not only the acceptable for a chapter like this.
uniqueness of their own perspectives on an 7. Screen capture from the graphical procedure
issue, but also the perspective of others who within PQMethod is not possible, so I have created
participated in the sort. tables that approximate the outputs.
44
Using Q-Sorts in Participatory Processes
After an especially long journey back to DC for a Social Development meeting, you awake in the morning to
find you’ve really overslept. It is December 2002. You arrive at 1818 H St., N.W., and find your colleagues to
see what they’ve been up to. As it turns out, they have made great progress over the three years you were
slumbering….
Which of the following statements describe this new and improved Social Development Family, as it appears
to you in 2002?
q1 SDV is composed of people with a mix of educational and professional backgrounds with management
and client-interface skills that match their technical proficiencies.
q2 SDV is a leading resource within the Bank for easily applicable solutions to social issues facing country
teams, such as participatory methods or local capacity building.
q3 SDV manages a network of external links to NGOs both in developing and developing nations, which
enables SDV to play a role in coordinating actions among multiple actors in the development arena.
q4 SDV delivers stand-alone projects to country teams, complete with staff or connections to local staff.
q5 SDV is a provider of region-level solutions to problems; with a staff and budget that resides primarily with
the regions.
q6 SDV is involved, through an allocated staff member, on nearly every relevant Bank project.
q7 SDV has its own products that can be offered directly to clients in countries.
q8 SDV is respected and in demand within the Bank as a source for efficient and fair compliance reviews.
q9 SDV serves as the primary advocate within the Bank for the needs of countries' most vulnerable
populations.
q10 SDV affects change in the Bank by persuasion; its voice and recommendations are respected by Bank
audiences.
q11 SDV is the primary locus of policy formation for certain crosscutting social issues.
q12 SDV is able to quickly anticipate and adjust its offerings based on the needs of client teams.
q13 SDV is a leading resource within the Bank for knowledge on a broad range of crosscutting social issues.
q14 SDV serves as a center for training and development of other Bank staff on social issues.
q15 SDV challenges and pushes the Bank to maintain a focus on programs that reduce poverty.
q16 SDV is a driver of quality assurance for Bank projects.
q17 SDV is a source of world-class research in issues related to social development.
q18 SDV attracts and retains the most skilled social scientists working in development.
q19 SDV is a constant innovator; developing a series of new products and services to offer within the Bank.
q20 SDV serves as a center for training and development of non-Bank staff (government, other NGOs) on
social issues.
q21 SDV works on the front lines, allocating staff to designing and implementing projects on the ground in
developing countries.
q22 SDV serves as a clearinghouse for a core set of assessment tools that are used by the majority of Bank
project teams.
45
Jonathan C. Donner
After an especially long journey back to DC for a Social Development meeting, you awake in
the morning to find you’ve really overslept. It is December 2002. You arrive at 1818 H St., N.W.,
and find your colleagues to see what they’ve been up to. As it turns out, they have made great
progress over the three years you were slumbering. . . .
Which of the following statements describe this new and improved Social Development Family,
as it appears to you in 2002?
Once you have finished your sort, write the number on each of your chips in the corresponding space below.
-3 -2 -1 0 1 2 3
Least Most
Important Neutral Important
Once you have completed the Q-sort, please double-check that there is one number in each cell and no
duplicates.
46
Using Q-Sorts in Participatory Processes
Detailed Instructions (We will review this with you in person, as well.)
1. Count the blue chips. You should have 22 statements and one “header” with numbers ranging
from -3 to 3. Alert one of the Monitor team if you are missing any chips.
2. Each statement on the chip concerns one element of a vision for SDV’s future.
After an especially long journey back to DC for a Social Development meeting, you awake in
the morning to find you’ve really overslept. It is December 2002. You arrive at 1818 H St., N.W.,
and find your colleagues to see what they’ve been up to. As it turns out, they have made great
progress over the three years you were slumbering. . . .
Which of the following statements describe this new and improved Social Development Family,
as it appears to you in 2002?
3. Look for a minute at the diagram on the next page and at the “header.” To complete the exer-
cise, you should organize the chips in the shape indicated by the diagram: 2 statements go
under the column marked “-3: least important”, 3 statements under “-2”, 3 under “-1”, 4 under
“0: neutral”, and so on. There is no difference among the importance of items as long as they
are placed in the same column.
4. Most people find it difficult to do this prioritization in one step. Instead, it is easier to start by
separating the statements into three piles: not important (or disagree), neutral, and important.
5. Once you have three general piles, focus first on the “important” pile. Leave the others aside
and simply try to organize the chips you considered important into the appropriate number of
slots for “1”, “2” and “3: most important.”
6. Now turn to the general pile you called “not important” and perform the same sort.
7. Next sort out any statements you left in the “neutral” pile in the beginning.
8. You should now have a complete sort that matches the number of statements listed on the
header. Feel free to look at the whole picture and make any changes you want.
9. When you are comfortable with the sort, write the NUMBER of each statement in the corre-
sponding slot on the answer sheet on the next page.
10. If you have any questions, ask one of the Monitor team for help.
47
Jonathan C. Donner
Respondents
Sort value 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
-3 4 17 6 19 20 20 3 6 4 6 4 3 20 17 7 1 7
-3 17 12 10 20 8 14 17 17 22 17 17 17 4 4 5 7 5
-2 16 14 17 18 14 1 20 3 7 16 9 20 17 6 11 17 18
-2 3 9 9 14 3 6 7 4 6 19 8 8 21 7 4 12 3
-2 8 8 21 8 16 15 9 7 14 20 6 22 16 18 22 4 4
-1 6 19 4 12 6 4 16 15 3 4 7 4 3 20 1 19 17
-1 5 16 7 22 1 3 8 19 20 12 10 6 1 19 3 18 14
-1 1 10 15 10 13 12 15 21 9 3 15 12 7 22 13 16 9
-1 19 5 12 4 22 8 6 22 12 7 16 5 8 3 16 14 12
0 21 20 18 1 12 18 21 5 16 18 5 18 22 5 2 21 8
0 18 18 14 2 18 10 14 8 17 13 3 10 15 14 21 3 20
0 12 7 1 9 15 5 5 16 13 10 21 13 14 15 20 8 19
0 7 1 8 17 9 7 18 18 21 1 20 14 9 16 14 20 2
1 14 4 3 16 21 16 4 1 15 14 22 1 12 9 19 22 10
1 20 22 16 6 7 21 19 9 2 21 13 21 13 10 18 11 22
1 9 6 20 15 19 19 10 10 18 11 18 11 18 12 17 5 1
1 10 15 5 3 4 17 13 12 8 5 19 7 6 21 8 13 6
2 22 13 2 21 2 13 1 11 1 15 1 19 5 11 10 15 21
2 11 2 22 5 17 2 11 14 19 8 12 15 2 13 6 10 16
2 2 11 19 13 5 22 12 20 10 22 14 16 11 8 12 6 13
3 13 3 11 7 11 9 2 2 11 2 11 2 19 2 9 9 15
3 15 21 13 11 10 11 22 13 5 9 2 9 10 1 15 2 11
48
Using Q-Sorts in Participatory Processes
Links
WebQ http://www.rz.unibw-muenchen.de/~p41bsmk/
qmethod/webq
References
Argyris, C., R. Putnam, and D. McLain Smith. 1985. Action Science: Concepts, Methods and Skills for Research and
Intervention. San Francisco, Ca.: Jossey-Bass.
Brown, S. R. 1980. Political Subjectivity: Applications of Q Methodology in Political Science. New Haven: Yale
University Press.
Donner, J. C. 1998. “Making Mental Models Explicit: Quantitative Techniques for Encouraging Change.” In
Monitor Company Seeds for Change Series.
Kim, J., and C. W. Mueller. 1978. Factor Analysis: Statistical Methods and Practical Issues. Beverly Hills, Ca.:
Sage.
McKeown, B., and D. Thomas. 1988. Q Methodology. Beverly Hills, Ca.: Sage.
Stephenson, W. 1953. The Study of Behavior: Q-technique and Its Methodology. Chicago: University of Chicago
Press.
49
4. Ethnographic Methods: Concepts
and Field Techniques
Stuart Kirsch
Poverty is not a certain small amount of goods, nor with two additional techniques. The first are
is it just a relation between means and ends; above the participatory research methods that are
all it is a relation between people. normally used in rapid appraisal exercises.
— Anthropologist Marshall Sahlins (1972:37) These methods, which emphasize local partici-
pation and making research results accessible
Poverty is much more than a lack of income. Pov- to community members, can be used to assist
erty also means [the poor] having no the poor by providing them with valuable
“voice” in influencing key decisions that knowledge and skills. The second recommen-
affect their lives, or representation in state dation is derived from the model of commu-
and national political institutions. nity-based research, in which the skills and
— World Bank (2000) resources of the researcher are put in service of
community goals. The resulting collaboration
can increase local understanding of, and
This chapter provides guidance on the applica- control over, the processes of development.
tion of qualitative methods developed by These methods can be used to stimulate the
anthropologists for ethnographic research to following innovations in development policy
World Bank project design, implementation, and practice:
and evaluation.1 It offers task managers practi-
cal guidelines for using ethnographic methods • Developing agendas that are responsive to
of participant-observation, qualitative and community needs
semistructured interviewing techniques, and • Integrating complementary knowledge bases
case study analyses. It also encourages task • Providing information and research tools to
managers to use ethnographic methods to local communities and encouraging them to
supplement quantitative data collected by the develop their own analyses.
World Bank.2
Ethnographic methods may be used at the While listening to the poor is an essential
level of local institutions as well as in processes strategy for analyzing the causes and conse-
that transcend local contexts. These methods quences of poverty and devising appropriate
can also be used to “study up,” to provide interventions, it poses new methodological
information about development projects and challenges for Bank task managers. Ethno-
the workings of capital. graphic methods, with their focus on local
This chapter recommends that task manag- knowledge and understanding, provide impor-
ers supplement ethnographic research methods tant analytical tools to help address the prob-
50
Ethnographic Methods: Concepts and Field Techniques
lems of poverty. Applied in conjunction with • Gender lines, for example, increasing
participatory research methods and the model women’s workloads or restricting their
of community-based research, these methods autonomy.
can provide the poor with new skills and • Generational lines, for example, providing
information that will help them to analyze, the younger generation with the means to
understand, and alter their predicaments. challenge the authority of their seniors.
This chapter is organized in the following • Class lines, for example, landowners may
sections: oppose land reforms favored by peasants, or
agro-industry may object to water diversifi-
1. Use of ethnographic research methods cation projects.
2. Key principles in conducting ethnographic • Linguistic lines, for example, creating barri-
research ers in the flow of critical information.
3. Limitations of ethnographic methods • Religious lines, for example, non-Catholics
4. Identification of the right data for one’s may resent support for Catholic community
needs organizations.
5. Appropriate interview strategy
6. How to conduct effective ethnographic Project Planning
interviews
7. How to organize an ethnographic account Ethnographic research can also assess local
8. The case study method responses to program assumptions, objectives,
9. How to combine ethnographic methods and techniques. If incorporated in the early
with participatory research analysis stages of planning, ethnographic research can
10. How to combine ethnographic methods have beneficial impact on decisionmaking and
with community-based research results. Successful applications of participant-
observation and other ethnographic methods in
1. Use of Ethnographic Research Methods development contexts include primary health
care planning in Nepal (Justice 1986), the
Ethnographic research methods may be pro- problems of involuntary displacement (Cernea
ductively employed by World Bank task 1993), and the evaluation of a World Bank-
managers in the following contexts: (a) prelimi- sponsored public housing project in Ecuador
nary research to assess differences within and (Salmen 1987).
among communities, especially the unequal Ethnographic investigation may be particu-
distribution of power, (b) project planning, and larly useful in identifying problems inherent in
(c) project evaluation. planning assumptions, as Gardner and Lewis
(1996:147-51) argued in their analysis of a
Assessing Difference within Communities project to construct a fish farm in rural South
Asia. The initial project was strongly biased
Existing rifts and fault lines in a community toward the introduction of new technology and
may well become impediments to successful failed to adequately investigate the social
project implementation if they are not clearly context. Yet, “by opening up avenues for
identified in advance. Ethnographic investiga- discussion with local people, and identifying
tion can reveal local sources of power and some of the potentially contradictory interests
inequality, including gendered differences in and needs of different classes and groups,
political participation, unbalanced land distri- better decisions [were] made about . . . what
bution, local alliances and class interests, and the project has to offer to specific categories
controls over information. Projects may also of person” (p. 150). By integrating local
inadvertently transform these differences knowledge and technologies in project
into new forms of inequality or exacerbate planning, the anthropologists also increased
existing inequalities. These differences may the acceptance—as well as the efficacy—of
occur along: the proposed project.
51
Stuart Kirsch
52
Ethnographic Methods: Concepts and Field Techniques
informants in ethnographic research differs continually undergo change. Thus, the prin-
significantly from the standards of objective ciples of reliability and generalizability must be
formality and distance that characterize the conceptualized differently by focusing on
relationships between other social scientists corroboration and consistency within ethno-
and the persons that they study. These relation- graphic accounts.
ships can be facilitated by explaining the Another way to overcome these limitations
objectives of the research project to community is by making use of complementary methods,
members and soliciting their input. Such two- including surveys and questionnaires. These
way communication is the cornerstone of methods may be applied after one has carried
ethnographic research and is an appropriate out preliminary ethnographic research to
starting point for any form of collaboration, establish the most important issues and ques-
including development initiatives. tions. Alternatively, ethnographic methods can
be used to collect and interpret data in selected
Include All of the Major Segments or Factions locales following the implementation of rapid
of the Community under Study assessment or participatory appraisal exercises.
Ethnographic research methods can also be
This may require meeting separately with used to supplement quantitative studies by
different groups of people, for example, with ground-truthing or spot-checking more gener-
youth, women, or political leaders. The views alized findings. They can also be used to
of laypersons must be considered in addition to interpret key research results.
those of local experts. This makes it possible to
map or track the distribution of local knowl- 4. Identification of the Right Data
edge, paying attention to divergent views as for One’s Needs
well as bottlenecks in the flow of information,
rather than to simply the dominant (or most Ethnographic research produces several kinds
common) understandings. of data. It is important to identify one’s needs
in advance.
Ensure That One’s Informants Will Not
Be Harmed by the Research • Methodological conclusions indicate how to
collect relevant data on a particular subject.
The most important tenets of the American For example, what variables will give one an
Anthropological Association’s code of ethics is efficient measure of household poverty in a
that the ethnographer: (1) is obligated to ensure particular community? Data collection can
that the research will not cause any harm to the then be replicated in other project areas to
subjects; (2) must respect local rights to pri- identify regional variation or to monitor
vacy; and (3) is required to make public the change over time.
results of his or her research (American Anthro- • Relational data consider how different factors
pological Association 1998; see also Association or variables are interconnected. For example,
of Social Anthropologists of the Common- how does the price of fuel affect the deci-
wealth 1999). sions that people make regarding the mar-
keting of produce?
3. Limitations of Ethnographic Methods • Sociological data examine who participates in
particular social processes and why. Such
The relatively small sample size of ethno- information may be critical in identifying the
graphic research projects means that these factors that inhibit participation in the
studies do not provide the same guarantees of formal economic sector as well as relative
generalizability as large, controlled studies. rates of success.
Furthermore, ethnographic research is not • Interpretive information considers the mean-
generally replicable, given differences between ings given to specific actions or behaviors by
researchers and the fact that research settings the actors themselves, providing an impor-
53
Stuart Kirsch
tant middle ground between raw data and In contrast, unstructured interviews allow
analytical models. the informants to shape the discussion, so that
• Explanatory data provide information on they direct the researcher toward the most
how a particular process works. For instance, important issues from their perspectives.
how are prices determined by competing Successful ethnographic research design should
individuals in a market? It is important to retain the capacity to surprise the researcher
note that on their own, ethnographic meth- with unexpected results. The contribution of
ods cannot determine causality, that is, that the researcher is in identifying the relevance of
one behavior is the cause or consequence of this information. Comparability and focus are
another. They can, however, be used to compromised with this interview strategy,
formulate hypotheses that may be scientifi- although this method has the potential to
cally tested. produce the most novel results. This type of
interview strategy is used in Appreciative
5. Appropriate Interview Strategy Inquiry and Scenario Analysis (see chapter 5).
54
Ethnographic Methods: Concepts and Field Techniques
become a better source of information. How- (b) interviewer strategies, (c) recognizing
ever, these benefits must be balanced by the cultural differences, (d) choosing the right
need to obtain a sufficiently large and diverse informants, (e) recording the information and
sample. (f) reporting the results.
For development contexts, semi-structured The interview itself should be relaxed and
interviewing that combines both structured comfortable, not confrontational. A neutral
and unstructured elements is generally recom- setting is desirable. An institutional setting
mended (Rietbergen-McCracken and Narayan may make the subject uncomfortable, whereas
1997a). Sanal Ece describes semi-structured if the setting is too relaxed, it may hinder
interviews as a “low-cost” and “rapid” way to productivity.
collect information “from individuals and
small groups,” recommending that they be Interviewer Strategies
kept “conversational enough to allow partici-
pants to introduce and discuss issues that they It is helpful to present oneself as informed, but
deem to be relevant.” Group interviews are an independent. The interviewer should avoid
efficient means of documenting a wide range talking too much, as an interview is not a
of views in a short time. This technique can be conversation, although some turn-taking, that
supplemented with one or more interviews is, sharing of examples from other research
with several carefully chosen individuals. contexts, may be warranted on occasion. This
These secondary interviews are particularly can also be helpful in terms of illustrating the
useful in drawing out the full implications kind of information that one seeks.
of the material presented in the group
interviews. Recognizing Cultural Differences
The best approach to interviewing is to
combine strategies. One suggestion is to begin The researcher should keep in mind that there
research on a new project by carrying out may be culturally appropriate or inappropriate
unstructured interviews that generate a list of strategies for asking appropriate questions, for
questions and topics to be pursued further. example, that Euro-American executives like to
These initial interviews can be very time- be asked about what they have accomplished,
consuming, but generally reveal unanticipated or that people are often reluctant to divulge
but critical variables. Subsequent interviews “private” information about their earnings.
should be started with a several short questions Seek clarification of what one does not under-
that elicit background information for com- stand, but do not restrict the responses of one’s
parative purposes. Next, several questions informants only to terms with which one is
(derived from the preliminary research) could already familiar. (A common mistake in inter-
be used to facilitate the transition to an open- viewing strategies is to ask informants about
ended interview. Unlike formal survey meth- their understanding of Euro-American catego-
ods, it is appropriate to use information from ries, for example, leasing of land, before inves-
previous interviews to guide the discussion tigating local property models).
and even to solicit commentary on this infor-
mation. Choosing the Right Informants
55
Stuart Kirsch
marketing, or the user of a particular service. • It can document the steps in a process, for
(Violation of this principle is a common cause example, the operation of a collective irriga-
of gender bias in research). In addition to being tion system.
knowledgeable on the subject, the informant • It can produce a holistic account of a process
must be willing to share this information and or an event, for example, how a particular
be able to communicate effectively. The re- commodity is produced, marketed, and
sponses of a good informant should also be consumed.
internally consistent. Rather than select infor- • It can examine how events are interpreted, for
mants on the basis of their impartiality, the example, how rural villagers view pollution
interviewer must learn to evaluate the interests caused by a mining company, in contrast to
and stakes of the persons participating in their scientific assessments of the damage down-
research. stream.
• Qualitative interviewing can bridge inter-
Recording the Information subjectivities, conveying a situation from the
insider’s perspective, including the use of
Semi-structured interviews generate more first-person narrative description and
information than it may be possible to record analysis.
by hand. Write notes as the conversation • Finally, qualitative interviewing can identify
develops, trying not to interrupt the flow of the variables and frame hypotheses for subsequent
discussion, although it may be necessary to quantitative research (Weiss 1994:9-11).
slow the pace. An interview should be directed
back on track when the subject speaks in 7. How to Organize an
general terms about things about which he or Ethnographic Account
she lacks first-hand knowledge. Many anthro-
pologists tape their interviews, although Ethnographic description relies on discerning
working with tapes and transcripts is very the “experience-near” concepts that people
labor-intensive. Leave time after each interview naturally and effortlessly use to define what
to write up the results while they are still fresh, they think, feel, or imagine, in contrast to the
including questions that one did not think to “experience-distant” categories that specialist
ask. observers use to advance their scientific or
practical objectives (Geertz 1983:57-58). Dis-
Reporting the Results of Qualitative Interviews crimination, poverty, and hunger are condi-
tions that people experience directly, whereas
The interview material may be difficult to social stratification, structural inequality, and
categorize, with the result that the analysis economic disincentives to production are
relies more on interpretation, summary, and examples of concepts formulated by analysts.
integration than quantification and correlation. For example, consider the presumably straight-
Findings are usually supported by quotations forward task of collecting economic data from
and case descriptions rather than statistical families in a rural community. It is critical that
summaries and tables (Weiss 1994:3). While research take local definitions, such as for
organizing interview data, keep the following “household,” into account. In a village census
organizational frameworks in mind: in Papua New Guinea, for example, one may
obtain very different answers to the simple
• Qualitative interviewing can produce detailed questions of “Whose lives in this house?” and
descriptions of events and processes. “Who sleeps in this house?” The answer to the
• It permits the integration of multiple perspec- first question includes all of those persons who
tives that no single individual can possess in have the right to live in a particular house, even
totality, for example, how the welfare system if they were born and reside elsewhere, whereas
works from the perspective of legislators, the second question will elicit a list of current
case workers, and recipients. occupants.
56
Ethnographic Methods: Concepts and Field Techniques
Here are some guidelines for organizing an the accessibility of microcredit schemes tar-
ethnographic account of a specific behavior or geted to individual women?
event: Ethnographers also employ a variety of
heuristic devices that assist them in organizing
• Analysis of the specific behavior or event complex information in a manner parallel to
starts with the identification of the behavior in the problem at hand. This list is suggestive
question as well as related questions about rather than exhaustive or exclusive:
the form(s) that it might take.
• Then one determines the frequency of the • Use specialized local vocabulary that sug-
behavior, how often it occurs, as well as its gests different concepts or categories.
magnitude, strength, or importance. • Structure the material in terms of the steps of
• One should also investigate the place of this the analysis.
practice within a larger sequence of events, • Organize the argument by following a
including its origins (looking backward in sequence of events.
time) and consequences (projecting forward • Use social hierarchy as a frame of reference,
in time), or in both directions through for example, from lower to upper class, or
longitudinal study (from beginning to end). from workers to management.
• Also important is the complementarity • Organize the analysis with reference to
of related events, whether symmetrical spatial organization.
(a change in one prompts a comparable • Track the process under study, for example,
change in the other) or asymmetrical the steps involved in agriculture, from
(the change in one prompts an equivalent preparing the soil to harvest.
change in the other, but in the opposite • Trace the social life of a particular object
direction). from raw material to manufacture to sale to
• Finally, there are the contributing or associated use to discard.
factors, the conditions under which the • Focus on life histories (of a person or a
behavior is more or less likely to occur. family) or social history (of an institution or
a period of time).
Consider a hypothetical study of women • Emphasize the boundaries and borders that
marketing produce from their gardens. One define groups and difference, or the criteria
must investigate how often this practice occurs, for entering and leaving a group.
and its economic significance to the partici-
pants (measured in terms of resources gained 8. The Case Study Method
as well as their effect on female autonomy) and
their households. It is also necessary to con- The case study method “involves the investiga-
sider which events encourage direct marketing tion of a relatively small number of naturally
by women, including questions of origins (for occurring (rather than researcher-created) cases”
instance, planting with surplus production in (Hammersley 1992:185). It follows the parties
mind) and consequences (whether women involved, including their interactions and reac-
control the money they earn), as well as longi- tions, as well as the consequences of the events
tudinal study for the long-term consequences for the people and their activities. Kane (1995:176)
of this behavior. What is the relationship presents a detailed and very useful example of
between these behaviors and other related how the case study method operates:
events? For example, how does the marketing
of agricultural produce affect women’s Suppose you have found that while
workloads, including child-care responsibili- most girls from poor rural backgrounds
ties? Finally, what factors assist women in grow up to live in poverty and to do the
marketing produce directly, for example, same work that their parents performed,
improving safety at the marketplace, the effects others become professionals, large-scale
of government subsidies or price supports, or entrepreneurs, political figures, and so on.
57
Stuart Kirsch
What made the difference in these special permit is now required to make
women’s lives? Case studies use almost all jaggery (unrefined sugar lump) by tradi-
the research techniques in the social tional methods rather than to give cane to
scientist’s tool kit. Let us say that you did the sugar factories. Peasants trying to
a survey of these women and found build a small dam by selling some of the
certain patterns emerging: they had a sand from a dried-up river running
dynamic headmistress at school, a sup- through their village had to engage in a
portive parent, were good at a particular four-year struggle with the state govern-
subject, took a particular approach toward ment to get the rights to prevent the sand
life, and so on. One factor is unlikely to be being auctioned off to contractors
the cause. This is what a case study is (Omvedt 1993: 119).
good for: showing how factors and cir-
cumstances come together over time. You Presumably the economic policies that
can select your case studies from among Omvedt describes were well intentioned,
women who seem to illustrate the pattern, although their impact on the poor may not
and build a picture of them. Interview the have been adequately understood when they
women, their families, their teachers, and were implemented. Using case study methods,
anyone else who seems relevant. Look at anthropologists can document the processes
documents such as their old school that create and perpetuate the cycle of poverty.
records. Not only can they give you
information, they can tell whether the 9. How to Combine Ethnographic Methods
timing was important. Was a particular with Participatory Research Analysis
headmistress present at a particular stage
in their schooling? Observe how the When ethnographic research methods are
women behave, their attitudes, how they combined with participatory techniques
go about things now. Do these give you (Chambers 1992, 1998; Rietbergen-McCracken
any insights into the qualities that have and Narayan 1997a), important information
helped them? Using material from all can be shared with community members (box
these techniques and sources, you try to 2). Mascarenhas (1992) and Chambers (1992)
show how these factors worked in real described a number of strategies to involve
life. community members in the research process.
Local research assistants and other community
The case study method focuses simulta- members can map primary resource use (land,
neously on structure and agency, or practice water, and forests) and other economic pat-
(van Velsen 1979). In other words, it takes into terns. These variables can be given historical
consideration both the structural—political, depth by constructing a time line of events that
organizational, legal, economic—factors in a can be applied to past resource use. This
society and individual choices, which combine information can be gathered in part by inter-
to produce particular outcomes. The concrete- viewing older people about changes to the
ness of the case study method makes it particu- landscape at different points in time. Seasonal-
larly relevant for research in applied settings. ity can be documented by diagramming
Returning to the question of poverty, con- patterns of rainfall, employment, income and
sider the following observation made by an expenditure, credit and debt, food and nutri-
Indian activist: tion, and disease. Value ranking techniques
may be constructively applied to the economic
In Maharashtra, where I live, farmers importance of plants and animals or different
are not permitted to follow traditional economic activities. Wealth and inequality may
methods of separating cotton lint from be documented by establishing the economic
seeds under the Cotton Monopoly Pur- position of community members. By participat-
chasing Scheme; in sugarcane areas a ing in the data-gathering, community members
58
Ethnographic Methods: Concepts and Field Techniques
59
Stuart Kirsch
60
Ethnographic Methods: Concepts and Field Techniques
Ece, S. “Semistructured Interviews.” Available at: Rietbergen-McCracken, J., and D. Narayan. 1997a.
<http://www-wds.worldbank.org/> “Participatory Rural Appraisal.” In Participatory
Ervin, A. M. 2000. Applied Anthropology: Tools and Tools and Techniques: A Resource Kit for Participation
Perspectives for Contemporary Practice. Boston: and Social Assessment. Module III. Washington,
Allyn and Bacon. D. C.: World Bank.
Gardner, K., and D. Lewis. 1996. Anthropology, _____. 1997b. “SARAR” [Self-esteem, associative
Development and the Post-Modern Challenge. strength, resourcefulness, action planning and
London: Pluto Press. responsibility]. In Participatory Tools and Tech-
Geertz, C. 1983. “‘From the Native’s Point of View’: niques: A Resource Kit for Participation and Social
On the Nature of Anthropological Understand- Assessment. Module IV. Washington, D. C.: World
ing.” Local Knowledge: Further Essays in Interpretive Bank.
Anthropology. New York: Basic Books, 55-70. Sahlins, M. 1972. Stone Age Economics. New York:
Hammersley, M. 1992. What’s Wrong with Ethnogra- Aldine Publishing Company.
phy? Methodological Explorations. New York: Salmen, L. F. 1987. Listen to the People: Participant-
Routledge. Observer Evaluation of Development Projects. New
Jackson, B. 1987. Fieldwork. Urbana: University of York: Oxford University Press for the World Bank.
Illinois Press. Schensul, J., ed. 1999. Ethnographer’s Toolkit. Seven-
Justice, J. 1986. Policies, Plans and People: Culture and volume series. Walnut Creek, Cal.: AltaMira Press.
Health Development in Nepal. Berkeley: University Sclove, R. E., M. L. Scammell, and B. Holland. 1998.
of California. Community-based Research in the United States: An
Kane, E. 1995. Seeing for Yourself: Research Handbook Introductory Reconnaissance. Amherst: The Loka
for Girls’ Education in Africa. Economic Develop- Institute. Also available at <http://www.loka.org>
ment Institute Learning Resources Series. Wash- van Velsen, J. 1979. “The Extend-Case Method and
ington, D. C.: World Bank. Situational Analysis.” In A. L. Epstein, ed. The Craft
Mascarenhas, J. 1992. “Participatory Rural Appraisal of Social Anthropology. New York: Pergamon Press.
and Participatory Learning Methods.” In N. S. Weiss, R. 1994. Learning from Strangers: The Art and
Scrimshaw and G. R. Gleason, eds. Rapid Assess- Method of Qualitative Interview Studies. New York:
ment Procedures. Boston: International Nutrition Free Press.
Foundation for Developing Countries (INFDC), World Bank. 2000. “‘Voices of the Poor’ New Study
306-21. Offers Unique Human Insight Into Living With
Narayan, D., with R. Patel, K. Scafft, A Rademacher, Poverty.” World Bank News Release no: 2000/
and S. Koch-Schulte. 2000. Voices of the Poor: Can 248/S on Narayan, D., with R. Patel, K. Scafft, A
Anyone Hear Us? New York: Oxford University Rademacher, and S. Koch-Schulte. 2000. Voices of
Press for the World Bank. the Poor: Can Anyone Hear Us? New York: Oxford
Omvedt, G. 1993. Letter to the editor. The Ecologist 23 University Press for the World Bank. Available at:
(3):119. < http://www.worldbank.org/poverty/voices/
Pelto, P. J., and G. H. Pelto. 1988. Anthropological reports.htm>
Research: The Structure of Inquiry. 2d ed. New York:
Cambridge University Press.
61
5. Scenario Analysis: A Tool
for Task Managers
Jonathan N. Maack
Scenario analysis has been used by the private Managing risk. Scenarios that look at future
sector for the last 25 years to manage risk and paths for a country help decisionmakers
develop robust strategic plans in the face of an manage risk and develop concrete contingency
uncertain future. Its success in helping firms plans and exit strategies. Describing how and
manage large capital investments and change why possible futures might occur enables
corporate strategy has made it a standard tool decisionmakers to reflect on how political,
of medium- to long-term strategic planning. social, and economic changes affect the opera-
Scenarios have helped public sector agencies tion and to plan accordingly.
plan for population growth and regional
development, state transportation investments, Building consensus for change. Scenarios are a
and the distribution of landfills. In the develop- logically rigorous, transparent means to give
ing world, scenarios have been used to high- stakeholder representatives a role in planning.
light the opportunities, risks, and trade-offs in Because a number of possible futures are
national policy debates. created, many perspectives can be included,
and the discussion does not revolve around the
Uses advocacy of fixed positions. All participants
take part in formulating the core elements of all
Scenario analysis is different from the high, scenarios, and later break into smaller groups
medium, and low cases for lending considered to write each of the four or five scenarios. This
as part of Country Assistance Strategy (CAS). process helps decisionmakers share their
High, medium, and low scenarios look at thinking about uncertainty and risk, develop
different rates of progress along a path that is mutually understood contingency plans, and
based on a single set of projections. Scenario defuse blame-casting when forces outside their
analysis focuses on the areas of greatest uncer- control lead to a change in strategic direction.
tainty for a country or an operation, systemati-
cally develops several plausible alternative Augment understanding about the future.
future environments in which the operation Scenarios are designed to bring up issues that
might be implemented, and determines how otherwise would not be considered by expos-
they would affect its success. This structured ing the underlying forces in a sector or region.
approach to thinking about the future could help They are most effective when dealing with big
World Bank country teams make strategic choices issues and strategic directions, rather than
about where and how to direct lending over the tactical decisions, and should not be used for
medium term in several ways: short-term planning.
62
Scenario Analysis: A Tool for Task Managers
Monitoring progress and scanning changes in and technological forces that will impact on a
the environment. Scenario analysis can help project or strategy. Scenarios are arrived at by a
establish indicators that create a framework to team composed of key decisionmakers, experts,
monitor the execution of a strategy. Decision- and stakeholder representatives during two or
makers then “scan” the external environment three one-day workshops held over a period of
for the agreed indicators that the country or weeks or months. Because they focus on areas
region is beginning to move toward a different of uncertainty and the potential for unexpected
scenario (or to a different stage along a scenario future discontinuities, scenarios provide a
path). The focus on key drivers of change perspective not captured through projections
makes explicit the assumptions underlying based on past data.
lending and speeds mobilization of resources
in response to changes in the external environ- Scenario analysis involves constructing or
ment. developing scenarios (steps 1-4 below), and
integrating the content of scenarios into decision-
Scenario analysis is a tool with a strong track making (steps 5-8 below). During the exercise
record and significant potential, but it needs to itself, it is important that both of these elements
be adapted and piloted before it can be more receive equal attention and that mutually
widely used in the Bank. This chapter describes acceptable triggers and measures of success be
a scenario process designed to be the corner- agreed as outcomes of the process.1
stone of a strategic plan. It also suggests ways
to use the key concepts of scenario thinking in 1. Define a focal issue or decision. Scenarios are
shorter exercises designed to help decision- best suited to looking at the future through the
makers minimize risk, address key uncertain- lens of a specific issue, such as the likely
ties, and more effectively share their thinking outcome of a national election leading to a
with stakeholders about a lending program transfer of power or a currency devaluation.
and the country’s future. Without this grounding, there is a danger that
they will be too general.
Process
2. Identify driving forces. These are the social,
Scenario analysis follows a systematic process economic, environmental, political, and techno-
to create a set of four to five plausible and logical factors that are most relevant to the
vividly contrasting narratives that describe focal issue. They should be prioritized by the
possible evolutions of key areas of uncertainty scenario team according to their level of pre-
(figure 1). These narratives, known as sce- dictability and importance in affecting the
narios, examine the social, political, economic, desired outcome.
Strategic
implications of
Legend : each scenario
Core scenario team
Full scenario team
63
Jonathan N. Maack
3. Write scenario plots. These are the stories formed on the basis of the scenarios. Use or
that explain how driving forces interact and modify existing systems to monitor progress
what effects they have on the operation or toward operational goals as well as changes in
strategic direction being discussed. the external environment.
64
Scenario Analysis: A Tool for Task Managers
In the public sector, scenarios have been used makers and key stakeholders agree on substan-
to guide large capital investments in transpor- tive messages about possibilities for future
tation, landfill development, and city planning change. The process helps create informal
(box 2). These scenarios deal with potential networks and understanding among the
changes in user behavior, the utility of capital participants that can lead them to become
investments to a mobile population, and the champions of the policy recommendations that
economic and social pressures likely to develop come out of scenario discussions. The second
on a national scale over time. Special emphasis function of public sector scenarios is to engage
is also placed on likely voter pressures, larger public opinion. The completed scenarios are
political and demographic trends, and ways to simplified and broadcast in the mass media,
ensure continued political support for projects explained at local meetings, and highlighted in
once they are underway. Scenarios can help leaflets and policy bulletins. Shaping public
create the best long-term strategies for contin- debate about issues of national concern is made
ued growth and flexibility for states and easier by the flexibility of the scenario method-
regions. ology. Scenarios are stories that bring together
the key elements of policy in a narrative that
Scenario Analysis for Long-Range can be understood by the public. Because each
National Strategy scenario can illustrate plausible changes along
many dimensions, discussion begins to focus
Significant scenario development also has been on the interactions among forces, rather than on
done in the public sector at the national policy specific issues.
level. Scenarios have been used to shape
agendas for change in Colombia, Guatemala, Adapting Scenario Analysis
and South Africa (box 3). In these countries, to World Bank Interventions
which were mired in turmoil or facing funda-
mental socioeconomic crises, leaders from In the World Bank, scenarios can provide
academia, business, trade unions, the military, assistance in formulating goals, designing
rebel groups, political parties, and peasant projects, building ownership, and monitoring
unions came together to discuss steps forward. progress. Scenarios are most valuable when
When used in the public sector, strategic they are brought in at the point in the project
scenarios have the goal of helping decision- cycle at which the overall direction of the
Box 2. Scenario Analysis for the Public Sector: New Jersey Department of Transportation
In the mid-1990s, the State of New Jersey used a experts, academicians, and activists. The scenarios
participatory scenario process to develop a vision for focused on economic growth and citizen attitudes
the future that would help it deal with intransigent (see box 7 for full scenario matrix). Special attention
transportation issues. The core decision was was paid to regional growth, because New Jersey is a
whether to focus on sustaining existing roads, major transportation corridor between the two large
expanding the highway system, or building more light metropolitan areas of New York and Philadelphia.
rail. To develop the scenarios, the scenario team The scenarios were widely publicized and
looked at the likelihood of regional and national sparked significant public debate. All of the sce-
economic growth, citizen priorities, environmental narios highlighted the importance of innovation and
pressures and political will. Research, which cooperation with the public sector and the need for
included a values survey of a cross-section of environmentally sound development. Through
citizens and businesses, studies of existing travel innovative public-private partnerships, the depart-
and commerce patterns, and a study of conventional ment of transportation leveraged federal funds for its
forecasts, was carried out as part of the process. In long-term plan of mixed road-rail development.
addition, consultations were held in communities Business practices were also changed to allow for
across the state. more citizen interaction.
The scenario team included state politicians,
department of transportation decisionmakers, Source: Adapted from Bonnet, 308-24
65
Jonathan N. Maack
In 1990, four distinct—and evocatively titled— deep economic slump that leaves the country in
scenarios for South Africa’s post-apartheid future worse condition than it was during the transition.
were developed during three 3-day workshops. The 4. Flight of the flamingos. A smooth transition
process was initiated by a local university and took followed by a move toward sustainable policies to
place during the heated national debates between increase growth and inclusion. This story, named
the National Party and the African National Con- after the slow take-offs and close flight patterns of
gress (ANC). The participants looked at the possible flamingos, shows steady economic growth, biparti-
actions of various stakeholders in the political san priority setting, and a greater increase in long-
process and came up with four scenarios. term benefits to the poor.
1. Ostrich. A closed transition process in which The scenarios were developed by economists,
conservative parties attempt to forge a nonrepre- businesspeople, politicians from all major parties,
sentative compromise government rather than academicians, and NGOs. They received heavy
reach a negotiated settlement. The outcomes of press coverage, and F. W. de Klerk, leader of the
this policy, named after the bird’s habit of sticking National Party, was quoted as saying that he was
its head in the sand in the face of danger, are “not an Ostrich.” The Icarus scenario, which critiqued
political and economic isolation and deepening unsustainable government spending, was endorsed
ethnic tensions. by key left-wing economists and comforted the local
2. Lame duck. A half-hearted and prolonged and international business community. The transition
transition of power that creates a coalition govern- government that eventually came into power was
ment stripped of any real authority. The outcomes of notable for its restrained economic policies and its
this scenario are a deteriorating political climate, success at using tested, sustainable methods to
lack of sufficient investment in economic redevelop- reach the poor. The scenario process was an
ment, and the risk of continued isolation. excellent way to broaden the involvement of stake-
3. Icarus. A populist national government takes holders in policy dialogue without creating chaos in
power and adopts an unsustainable policy of high the decisionmaking process.
government spending and taxation. The outcome
of this policy is short-term prosperity followed by a Source: Adapted from Kahane, 325-32.
intervention is being designed or significantly analysis make it well suited to this approach.
rethought. If it is meant to be the key strategic Scenarios are also valuable for monitoring the
framework for an operation, the scenario progress of both the client and the Bank toward
process should be integrated with the budget- the goals outlined in the CAS.
ing, negotiation, and implementation of a
project rather than treated as a stand-alone 2. Formulation of participatory national
exercise. strategies. The emphasis on participation
Such an application of scenario analysis makes scenarios useful for planning pro-
would add significant value in five strategic cesses that need dialogue and agreement
areas: among multiple stakeholders, such as the
Poverty Reduction Strategy Paper (PRSP)
1. Formulation of Country Assistance Strategies. and the Comprehensive Development Frame-
The CAS is the Bank’s business plan in a work (CDF). A country-driven scenario
nation, the closest analog to the corporate process is an excellent way to develop owner-
strategies around which scenario analysis ship and incorporate the political situation
techniques were developed. Scenarios can and relevant external forces in PRSP and CDF
increase the selectivity and realism of a CAS by documents. Because the scenario process can
focusing discussion on likely future opportuni- be led by the country, as it was in Colombia
ties and constraints. The technique has been and South Africa, the Bank can concentrate
presented in clinics for Bank task managers on on its comparative advantage: leveraging the
formulating CAS documents. The combination input of macroeconomic, political, and
of a focus on areas of key uncertainty and a sectoral experts to add depth and believabil-
strong participatory element in the scenario ity to the scenarios.
66
Scenario Analysis: A Tool for Task Managers
67
Jonathan N. Maack
There are no fixed rules for how to carry out a The first step of the scenario workshop is to
scenario analysis, but several stages occur in clarify the decision focus of the exercise. This
almost every case. The actual creation of should have been outlined to individuals when
scenarios takes place with a group in a work- they were invited to participate, but it is
shop. The organizers need to carry out a important for the group to work together to
number of activities before the first and last clarify the issue. As this discussion can become
workshops. The four broad stages to the contentious, it is best to state clearly within the
process follow. The specifics of each stage will first 10 minutes of the meeting the behavior
be discussed later in the chapter. An illustration expected during the meeting. Propose, or ask
of a sample process from start to finish can be the group to agree on, whether a facilitator is
found in appendix 2. needed, and, if so, who it will be; a system to
recognize speakers; a time limit on the length
1. Preparation. A number of tasks need to be of comments; and other basic procedures of the
carried out before the scenario workshops. They meeting.
include defining the scope of scenario use, Discussion of the decision focus forces all
choosing and interviewing the scenario team, participants to concentrate on the goals sought
and preparing background material for the by the development intervention rather than on
workshops. These are crucial steps in the process, the specific process that will be used to reach
because they help the task manager build confi- those goals. This separation of means from objec-
dence in the scenario exercise and gain an early tives is key to giving the group the conceptual
understanding of the issues that will be most space needed to come up with new approaches.
important to participants. The specifics of this Several key issues should be addressed when
process are detailed in appendix 1. deciding the decision focus:
2. Building the scenarios. This process can 1. Desired outcome of the intervention. This
occur in one workshop, or two held back to discussion should begin with input from the
back. Together the scenario team determines organizers of the event but then will be open to
the focus of the intervention, identifies and further discussion. The group may come up
prioritizes the key external factors that will with a different formulation of the desired
affect its success, sets them in a matrix to outcome than was originally brought to the
differentiate their effects, and writes simple table. This reformulation will guide the group’s
scenario plots. understanding of the forces that will affect it.
3. Investigating and writing the scenarios. This 2. Desired outcome of the scenario process. Once
exercise is carried out between workshops. The the outcome of the intervention is clarified, the
task manager and core team gather relevant role that scenarios will play should be ex-
qualitative and quantitative information to plained. Will they be used as a way to test
determine whether the assumptions made in existing strategies for riskiness, as a tool to
the scenarios are accurate. Decisionmakers formulate new strategies, or as a means to
from the scenario team should write more build a shared vision for governmental or
finished drafts of the scenarios. organizational reform? The organizers should
clarify how they envision the scenarios being
4. Using the scenarios to plan strategy. In this applied to strategy, and explain any internal
workshop, the scenario team goes over the business tools that will be used to put that
finished scenarios and looks at the implications strategy into practice. For example, if the goal
for the strategic decision being made. Specific is the creation of a medium-term framework to
external “signposts” of changes in the key guide the sequencing of a program of adjust-
forces are also identified at this stage. ment loans, the group needs to understand
68
Scenario Analysis: A Tool for Task Managers
how the program of loans works and where storming. The goal of this stage is to build a
scenarios fit in. good conceptual model of the forces that are
most relevant to the decision focus.
3. Time frame. The time frame will signifi- There are two large categories of driving
cantly affect the range of issues a scenario can forces. External forces are the social, economic,
address. The technological and economic shifts environmental, and political forces in society
that should be considered in a 20-year time that are relevant to the topic of the scenario
horizon may have no relevance for a 5-year discussion but are outside the participants’
plan. Scenarios should be conceived of as ways control. Organizational forces are the product of
to shape strategy within the medium term. The the actions of the Bank and other stakeholders
time frame should be long enough for signifi- and can be affected by the project. These driv-
cant changes to occur in the driving forces, ing forces will become the key foci of further
ideally 3 to 5 years. scenario development, and they can be identi-
fied intuitively by using the visual clustering
Key Factors method described in appendix 1 or separated
into a series of logical groups and analyzed.
After the decision focus has been discussed and
analyzed, the team should brainstorm key External Forces: The SEEPT (Social,
factors for the success of the strategy or opera- Economic,Environmental, Political and
tion. This session should be led by a person Technological) Framework
with experience in facilitation. Key factors for
project success can include: The simplest way to systematically address
external forces is to organize the issues identi-
• Specific events, for example, the passage of fied in the brainstorming sessions and prelimi-
legislation that gives legal status to NGO nary interviews in five broad categories:
microcredit providers
• General trends, for example, a decline in the 1. Social
cost of fertilizer 2. Economic
€ Improvements in service delivery measured 3. Environmental
by positive beneficiary feedback as well as 4. Political
number of persons served. 5. Technological.
69
Jonathan N. Maack
Table 1. Illustration of the SEEPT (Social, Economic, Environmental, Political and Technological)
Framework for Identifying External Forces
This is an illustrative SEEPT framework for a national-level scenario exercise. This chart can serve as a menu for scenario
teams or as a stand-alone example of the sort of questions that can be posed in each of the SEEPT categories. The Social
Indicators of Development, Country Political and Institutional Assessment information, United Nations Development
Programme (UNDP) Human Development Indicators, and the World Development Indicators are useful sources of data for
these domains.
Economic Macroeconomic GNP, balance of trade, rate of inflation, exchange rate; current and future
conditions relations with international financial markets, current debt levels;
governmental expenditures, deficits
Changes in the economic structure of nation (dependence on single export,
percentage of exports in finished goods), formation of new regional trading
blocs
Microeconomic Change in size, type and ownership of firms; formal and informal labor force
conditions structure by region; changes in economies of scale/structure of key
industries
Environmental Physical environment Air/water/land pollution trends and locations, environmental quality issues
(global warming)
Natural resources Energy prices and availability (likelihood/impact of an oil shock), raw
materials (rate of depletion, ease of access), land use (farming methods,
erosion levels), sustainability (strategic use of resources) regional
distribution of natural resources
Political Geopolitical Trends in international relations; relationship with other nations in region
(regional trading blocs, military alliances); levels of tension, conflict
(regional, international); trade and protectionism
70
Scenario Analysis: A Tool for Task Managers
the future, the group could create differentiated Apparent direction of these forces today. What
stories about how broad future trends and are the current trends in these forces and
developments could affect the operation. why? One of the most critical tasks for sce-
nario analysis is a good understanding of the
Organizational Forces causes of present trends and conditions.
Present trends are a good base from which to
Organizational forces are the specific actions extrapolate future interactions among driving
and general positions taken by actors (indi- forces.
viduals, agencies, and organizations) that affect
project success. These actors may include Relevance to the decision focus. What is the
government agencies, the private sector, magnitude of the impact of these forces on the
political parties, NGOs, and international future course of key factors for project success?
agencies. General trends in governmental Are reforms dependent on a political coalition
actions might include a move toward that draws its support from a constituency
privatization of state industries or trade liberal- whose influence is shrinking?
ization. As in the case of external forces, the
behavior of actors is interconnected. If the Future prospects. How much, in what ways,
government and trade unions work together to and how fast might these trends change in the
create an enabling environment for trade, future? At this stage of analysis it is integral
international businesses might begin to make that the scenario team differentiate among
serious investments or solidify their presence. trends that are:
The actions of agencies and organizations
should make sense within the economic and • Predictable. Certain things, such as current
political environment of a scenario. If organiza- demographic trends, are already known and
tional forces are used as key scenario drivers, will need to be taken into account in all
they should be consistent with past behavior in planning.
that organization. Radical changes of position • Unpredictable/uncertain. Market prices,
on critical issues should not be incorporated currency rate fluctuations, demand for
unless the rest of the scenario environment export goods, and changes in political values
makes such changes likely. are all unpredictable elements.
71
Jonathan N. Maack
• The number of chits that equal a “high, Creating a Scenario Matrix to Outline Four
”medium,” or “low” rating should be Distinct Scenarios
decided on by the group based on the
number of forces that the group has devised The results of the impact/uncertainty matrix
and the number of people in the group. are seldom conclusive. A number of forces will
cluster in the upper right hand corner (highly
The impact/uncertainty matrix (table 2) is an unpredictable/highly relevant). The group
illustration of how the ranking system can be needs to pick the two most relevant of these forces
used to narrow the list of forces to the most and set up a scenario matrix using them as the
relevant for differentiating scenarios. Scenario axes. The group will then work out the four
plots should be built around high-impact/low- permutations of these forces and build the
uncertainty issues (highly relevant issues with scenarios around them.
predictable future outcomes for which current The scenario matrix (box 4) ensures that each
planning must prepare) and high-impact/high- scenario will be different in a logical, nonran-
uncertainty issues (issues that could shape dom way, because the top scoring factors will
different futures which planning should take be key drivers in all scenarios. The scenario
into account). team discusses each permutation of the two
Table
Table 2. Example
2. Example of Impact/Uncertainty
of Impact/Uncertainty Matrix
Matrix
Degree of uncertainty
72
Scenario Analysis: A Tool for Task Managers
most relevant forces. The decisions made about • “Critical Scenario Drivers” from the Impact/
how these forces are likely to behave sets the Uncertainty Matrix. Each of these factors
tone for each of the four different scenarios. should play a role in at least one scenario.
This process enables the quick formulation of That role should work naturally with the
distinct, logical scenarios. Because the decision other forces in play in each scenario, adding
on the two axes of uncertainty is group-driven, the twist of giving decisionmakers a chance
it encourages group members to challenge to explore how a particularly positive or
one another and seek justification that a negative development might affect the
certain force is the best way to differentiate a operation/strategy as a whole.
scenario. Narrowing the discussion enables • Systems of interaction among forces. The
the group to explore ways in which these two numerous relevant forces and issues devel-
forces could develop and how these develop- oped by the group should be explored
ments can help focus the other forces around systematically in each of the different sce-
a distinct plot. narios. Changes in the two key axes of
uncertainty can be thought of as stones
What Makes a Good Scenario Plot? tossed in a pond: their aftereffects will ripple
through the host of other forces that also
The scenario matrix approach is designed to bear strongly on the decision focus.
cut through many complex issues and isolate
two sources of great uncertainty and great An easy way to start the process is to think
importance for the success and sustainability of of different static “end states” that could result
the work being planned. This process of reduc- from the interaction of two forces, and work
tion may leave members of the scenario team backwards to explore how forces would need
uncomfortable, because not all issues can be to interact to reach that point. Some of the key
effectively addressed in terms of these two key factors cited by the group earlier in the exercise
axes of uncertainty. Although these forces will will be precisely this sort of “end state.” It is
remain the skeleton of each future narrative, important to remember, however, that sce-
the next phase involves the addition of detail to narios are not descriptions of a static future
the stories and the inclusion of a larger number world described in great detail. They are
of driving forces. A good plot combines driving narratives about how events might unfold
forces in a dynamic, coherent story. between now and a future date given the ways
in which the forces that the scenario team has
Key Elements identified interact.
Scenarios should be:
The key elements of a good scenario plot are:
1. Plausible. The events in the story should be
• “Critical Planning Issues” from the Impact/ possible, and the narrative should be credible
Uncertainty Matrix. These are the compara- (the descriptions of what happened, and why
tive sureties in the future that are particu- and how it happened should be believable).
larly relevant to the decision focus. The
critical planning issues may concern specific 2. Distinctive. Each scenario should focus on a
aspects of the issue that the task team is different combination of the key forces. Sce-
studying and should have a role in all of the narios should be clearly differentiated in
stories about the future. Issues such as structure and in message, not variations on a
demographic trends should be implicit in all single theme. Multiple scenarios should be used
plots, but the role they play may be quite to explore how different permutations of the
different depending on how political, social, same key forces can yield very different worlds.
and economic factors affect issues such as
education, employment, emigration, and 3. Consistent. Each scenario should have a
consumption. strong internal logic. The goal of scenarios is to
73
Jonathan N. Maack
High Growth
Low Growth
74
Scenario Analysis: A Tool for Task Managers
explore the way that forces interact, and each • Winners and losers. These scenarios explore
action should have a reaction. Neither actors the future in terms of ascendant versus
nor external factors should completely over- declining organizations, nations, or social
turn the evidence of current trends and posi- forces. Examples include scenarios that look
tions unless logical explanations for those at trading blocs, focusing on dominant
changes are a central part of the plot. For groups and exploring how different nations
example, it is highly unlikely that there will be would fare in a climate of struggle and
low inflation and high growth, or that a regula- change.
tory agency that was formerly very strict will • Good news/bad news. These scenarios use
significantly loosen its requirements without large external factors, such as currency
some extenuating circumstances. devaluation, a massive increase in corrup-
tion, or breakdown of the rule of law, to
4. Relevant. Each scenario, and the entire set encourage rethinking existing strategies and
of scenarios taken as a whole, should contrib- priorities. These stories ask decisionmakers
ute specific insights into the future that relate “what if?” questions about the logic of their
to the decision focus chosen by the group. It is long-term plans in the face of key forces that
important for the author to return frequently to can move in unexpected directions.
the objectives of the exercise as she or he builds • States of change. This model is one of additive
the scenario and assesses which of the external change, a world in which a series of alter-
and institutional factors created in brainstorm- ations feed off one another to move society
ing should be explored. into a different mode. These alterations can
involve a series of innovations leading to
5. Creative. The emphasis on logic and rigor improvement, or a series of mistakes leading
in the process should not stamp out innovative to political gridlock and economic stagna-
thinking. Part of writing a scenario is enticing tion. Some models focus on structural
the reader into the scenario world. The author change in a nation (the ending of apartheid
can do this by telling his or her story through in South Africa), an industry (the introduc-
parables, citing examples, or using famous tion of software to produce online reports,
figures as stand-ins for complex ideas. removing the need for specialized printers),
or a sector (environmental movements spur
6. Challenging. Scenarios should challenge strong pressure to build public transporta-
conventional wisdom about the future. When tion and reduce pollution) and how the
thinking about the major sources of uncer- change will affect different actors and the
tainty, the author should try to explore alterna- viability of investments.
tives that will significantly alter the basis for • Cycles. Another way of looking at change is
the assumptions that go into the “official through cycles, both economic and political.
future.” It is often worthwhile to write a “wild In this model, the rise and fall of social
card” scenario that uses some of the key forces groups or stakeholders can be charted in
not listed in the scenario matrix. This gives the relation to the economic cycles of relevant
set of scenarios as a whole more breadth and industries or international trends.
strength. • Wild cards. Wild-card stories explore how
catalytic developments could completely
Scenario Plot Types reshape a society. For example, the develop-
ment of the internet and its effect on busi-
Scenario plots should differ from one another ness in the United States is a wild-card
without being wildly positive or negative. It is scenario that is still being played out. Other
important that they challenge assumptions examples might include the unexpected
while remaining balanced enough that they are discovery of oil reserves or other resources.
not dismissed out of hand. Here are some plot An example of this is Tuvalu, an impover-
“types” that can help create stories about the ished South Pacific island nation, which
future: discovered it had a unique asset. Tuvalu’s
75
Jonathan N. Maack
76
Scenario Analysis: A Tool for Task Managers
Box 5. Destino Colombia Scenario: “When the Sun Rises, We’ll See”
Weariness, laziness or inability to face problems are all justified by the phrase “When the sun
rises, we’ll see.” Darkness turns into a pretext for dreams and apathy, but the clear light of dawn,
rather than inspiring important decisions, simply gives rise to a new period of uncertainty. In the
face of the country’s crisis, this irrational confidence in unexpected, miraculous outcomes, this
recourse to halfway solutions, this generalized tendency to put off basic actions until later, have
combined to the point of becoming a collective alternative. Before its enormous and rapidly
growing problems, Colombia seems to be saying “When the sun rises, we’ll see.’”
— Quote from the Destino Colombia scenarios
As crises repeatedly rock the country, exhaustion seeps into the government and the populace. Continual
crises lead to a focus on solving problems day to day. The government wastes valuable time and energy
negotiating without a strategy and is captured by pressure groups including the wealthy, various regional
warlords, and corrupt officials. This “law of the strongest” leads to the extraction of the greatest costs from the
most successful industries, the growth of the drug trade, the rise of regional governments with their own
taxation systems, and the end of a national oil industry and economic independence.
As the situation deteriorates, the poor suffer the most. International organizations step in but are able to
provide only stopgap solutions. Ultimately, increasing human rights violations and economic disorganization
lead to the country’s becoming economically and politically isolated. This isolation leads to military threats to
the borders by neighboring countries and increasing levels of emigration. The scenario closes with the
comment that the disintegration was slow enough that people simply “became numb to the proliferation of
their misfortunes.”
77
Jonathan N. Maack
2. What are the implications of this world? Each current strategy of the Bank and the client. In
scenario provides a different world for the the case of a specific loan, this encourages the
intervention. This step involves a discussion of scenario team members to look at how what
the implications of each scenario for the central they have learned about likely impacts on the
strategic decisions that need to be made. A intervention fits into the national development
standard set of questions designed to apply to strategy.
these decisions should be created for this phase When scenarios are designed to create a
of discussion and used for each scenario. A framework for a series of loans, it is more
team might want to look at which ministries appropriate to compare the conclusions
are most affected by each scenario or which reached to the consensus about development in
geographic areas would experience the most the country. This consensus can be identified
change. through Country Assistance and/or Poverty
Reduction Strategies as well as through com-
3. What is the best strategy for this scenario? parisons with the national strategies of other
Each scenario presents risks and opportunities. donor agencies.
The team should focus on the broader strategic The team should look at the contingency
context and answer the following questions: plans suggested by the scenario team and
identify:
• What is the best strategy for dealing with this
situation? The scenario team should look first 1. Which alternatives seem to be suggested by a
at how the intervention can most effectively majority or all of the scenarios? These should be
move toward its desired outcome in this key parts of any strategic plan. They reflect an
future world. What general strategic position interaction of forces that is very relevant to the
should be taken? Should it be one of inter- intervention and that the scenario team feels is
vention in certain areas, or restraint to allow likely to occur.
the situation to run its course?
• What are the major opportunities and threats in this 2. Which alternatives challenge most strongly the
scenario? This may involve looking at how assumptions underlying the current strategy? The
different sectors will be affected by the changes team should look to these alternatives for
outlined in a scenario. It may also involve guidance in rethinking strategic orientation.
looking at the effects that different scenarios Even if the scenario from which they are drawn
have on specific social groups, identifying the does not come to pass, they highlight a blind
winners and losers in each story. spot in current plans. When making final
• If we know this scenario will come to pass, the strategic decisions, the scenario team should
Bank/client should (name one action)? It should look for creative ways to include these insights
stop doing (name one action)? These questions in the intervention.
concretize the discussion by relating it to the
specific set of actions that are being per- 3. Which alternatives are logical extensions of
formed (or proposed) as part of the interven- the current strategy? These alternatives give
tion. In addition, the scenario team should decisionmakers an idea of how alternate future
develop exit strategies for all or part of the developments could be leveraged to push
intervention based on each scenario. What forward an agenda or program that is already
situations will trigger the abandonment of a in place.
project or set of projects?
Each alternative strategy presented by the
By answering these questions, the scenario scenario team should be carefully considered
team has in effect developed a series of simple during the workshop. The final step in the
contingency plans for each potential future. exercise is deciding which strategic alternatives
The next phase of the process is to assess how should be adopted. Again, a simple set of
much these strategies have in common with the questions can serve as a guide:
78
Scenario Analysis: A Tool for Task Managers
1. What events would trigger each strategy? nomic changes. These indicators should be
What impact (positive or negative) would tracked throughout the life of the project and
those events have on the project? How effective compared to the scenarios developed at its
is the strategy at addressing these issues? outset to determine progress toward a given
future. Monitorable indicators can include
2. What is the evidence to support the assump- (1) economic figures, such as export and
tions underlying the strategic suggestion? What growth levels; (2) demographic trends, such
aspect of the scenario serves as the underpin- as rate of migration to geographic areas; and
ning of the strategy? (3) employment trends, such as the number
and type of jobs created in the formal sector.
3. Is it feasible for the organizations involved to • When choosing trends to monitor, scenario
execute the alternative? What would prevent team members should look at where they
them from being able to do so? can obtain data, how much it costs to pro-
duce that data, and how feasible it is to
Develop Monitorable Progress Indicators regularly track it. Another strategy is to look
toward Each Scenario at how data that is already collected would
be influenced by the driving forces of a
Once each scenario has been tied to a set of scenario. By tracing the likely impact of a
strategic actions, the team must develop a way scenario on an already monitored set of
to look at what is happening in the country and demographic or poverty indicators, the team
the world to determine which scenario is can ensure that the information will be
coming to pass. The changes described in recorded in a standardized and reliable way.
different plots (unified vs. disintegrated poli- Thinking about how each scenario would
ties, outbreaks of ethnic violence) do not occur translate into migration, poverty rates, or
overnight. The scenario team should identify other affected data helps make the world the
both indicators based on fixed events and scenario team describes more real.
indicators based on trends.
Make It a Reality
• Fixed events. Fixed events are pivotal inci-
dents that push groups into conflict or create The final stage of the scenario process is the
economic opportunity. They can be external dissemination of the message and its imple-
or internal events and may be time-sensitive mentation on the ground. These two goals are
(elections) or ongoing external processes fundamentally linked. One of the keys to
(currency fluctuations due to the collapse of successful implementation is giving the public
markets in Asia). The group should identify and key stakeholders (especially those who
at least four “turning points” for each sce- will be carrying out the strategy) a good
nario. Events such as oil prices reaching an understanding of the lessons learned from the
all-time high or a 20 percent decline in scenarios and the changes in policy that have
tourism can serve as signs that the world is been made as a result of them.
developing along the lines of the scenario. The basic steps are to:
One way to identify such turning points is to
invent newspaper headlines from the future Simplify scenarios and prepare them for dissemi-
that sum up crucial turning points or sym- nation. Reduce each scenario to a 1- to 2-page
bolic indications of the forces at work in the narrative that can be read or explained to an
scenario. As these events occur, it is impor- audience in 2 minutes. Use newspaper inserts,
tant to look at how their outcomes are related radio and television broadcasts, and videos
to the scenarios that have been created. shown during clinics and seminars to dissemi-
• Trends. Indicators of trends can include nate the scenarios. In Colombia, a video was
measurable changes in attitudes, demo- made with various team members explaining
graphic shifts (urbanization), and macroeco- their scenario with the aid of cartoons.
79
Jonathan N. Maack
Develop support for the scenarios. Once the The team also should rethink scenario plots in
scenarios have been written, it is essential to light of new developments (ideal at Mid-Term
involve larger groups of stakeholders to dis- Review) and adjust them so that they corre-
cuss them. During the dissemination period, spond to the most recent information.
the scenario team should build relationships
with local politicians, influential figures, and Common Pitfalls in Developing Scenarios
important civil society organizations. The team
can organize clinics and seminars for client Despite its usefulness, scenario analysis is not a
country officials, local political leaders, and panacea. Scenarios are a process, not a product.
other figures and use handouts and videos to To become truly effective, they need to be used
explain the scenarios and the strategy that and refined over time. This requires sustained
resulted from them. It is important to monitor commitment on the part of high-level
local and media responses to the scenarios and decisionmakers and a skilled and dedicated
adjust for them in presentations and seminars. scenario team. It also requires the government
to carefully develop and track indicators to
Build accountability systems. The scenarios help decisionmakers tie what is going on in the
should lead to the definition of clear goals and outside world to the scenarios.
pathways to alternate scenarios. It is important Scenario development is not easy. The
to involve the individuals who will generate process demands significant effort, thought,
the data in designing the implementation. The and creativity of the scenario team. To help the
team should develop incentives and mandates scenario team avoid mistakes made by others,
for data collection and reporting. Responsibili- common pitfalls are listed below.
ties should be clear, and the relationship
linking the data collected, the scenarios, and 1. Failure to gain the support of key decision-
strategy should be explained. For example, makers. To be credible, scenarios need to be
insights from scenarios should be tied to the integrated in the decisionmaking apparatus of
choice of baseline survey during the design the organization in which they will be imple-
phase and to the strategy for monitoring and mented. Without real buy-in at the top, the
evaluation during supervision. scenarios will not be implemented.
Implement. Clarify the expected results of the 2. Unrealistic goals and expectations. Scenarios
intervention and define a clear strategy. Re- do not produce action plans; they help
solve residual conflicts between previous ways decisionmakers envision what will happen.
of doing business and the current system. The methodology is not suited to addressing
Target priority areas for strategic resource specific tactical issues. It is meant to provide a
allocation. Provide guidance and training broad view of the uncertainties facing an
(including a background on the scenarios) to all intervention. Strategic decisions flow from this
of the line managers who will implement the understanding, but they are not a direct prod-
program. uct of the exercise.
Monitor progress and test results. Tie decisions 3. Failure to develop a clear map of the future with
about proceeding with the intervention to the monitorable indicators. It is essential that the team
indicators of progress toward certain scenarios. develop clear, monitorable milestones of progress
Scenario indicators should be a part of the Mid- toward the various scenarios. These milestones
Term Review process and should be examined should have a direct relationship to the goals and
during supervision missions to chart progress. planned outcomes of the intervention.
Re-examine environment and strategy. The 4. Scenarios that are not credible. Scenario work-
scenario team should review for major prob- shops are not brainstorming sessions. Scenarios
lems, adjust objectives based on observed must be based on solid quantitative as well as
results, and revise time tables and priorities. qualitative projections if they are to be credible to
80
Scenario Analysis: A Tool for Task Managers
those implementing them. Because scenarios do frames. Each scenario should be a smooth
not assign probabilities or project against current narrative that makes intuitive sense to the
trends, it is important to make sure that they are reader. The main aspects of the future should
based on strong research. be internally consistent; the outcomes postu-
lated for the two key uncertainties should be
5. Scenarios that are not tied to the planning able to coexist; and the actions of stakeholders
process. The indicators and thinking in sce- should be compatible with their interests.
narios must be directly built into the way that
an intervention is planned. Scenario indicators 10. Lack of diversity of inputs. If the scenario
should be closely monitored and associated team members are of homogeneous educa-
with explicit changes in strategy, including exit tional backgrounds and institutional affilia-
strategies. Similarly, scenarios should be tions, they will be much less likely to come up
related to the client’s and the Bank’s budget with innovative solutions. To build successful
and policy cycles. scenarios, the participation of a diverse group
of people is essential.
6. Not enough time to carry out the scenario
process. The process requires discipline and Conclusion
attention. It can be divided in two phases:
scenario building and relating scenarios to strategy. If scenario development is carried out with
Although these two phases may take place at commitment and purpose, it can be a powerful
separate times to conform with the Bank’s tool for improving development interventions.
project cycle and the client’s policy cycle, the Scenarios are useful tools for task managers
scenario team should have enough time to operating in environments in which long-term
think through the logic of the scenarios and thinking, flexibility, and the inclusion of stake-
ensure that they are properly researched. holders in decisionmaking are needed. Sce-
narios bring insight to the potential futures in
7. Inappropriate time frame and scope. Scenarios which an intervention will operate, what it will
that focus on current crises and existing prob- need to do to succeed in each future, and what
lem areas rather than looking at the interaction needs to be done to make that success possible.
of broader forces do not generate the kind of new By addressing rather than minimizing uncer-
thinking necessary to jump-start an agenda. tainty, scenarios spur innovative and robust
solutions. Because they are developed with a
8. Mistaking projections for scenarios. Projec- team of knowledgeable stakeholders, they are
tions are based on past data and often posit a an effective way to gain buy-in for strategies.
continuation, or slight variation, of current Furthermore, the research and discussion that
trends. Projections are not well suited to dealing accompanies scenario development promote
with the potential for significant discontinuities collaborative learning by both World Bank
brought on by external events. Scenarios are staff and the client. Ideally, all members of
designed to highlight “what if?” situations for the team walk away with a better under-
decisionmakers. Scenarios should be built standing of the interactions among the
around the forces that shape society. Trends are country’s social, economic, political, and
symptoms of these forces rather than their technological forces.
cause. These unexpected changes cannot be
predicted with a study of current trends. Note
9. Failure to tell a dynamic, internally consistent 1. The steps that follow owe a significant debt to
story. Scenarios should be movies, not still P. Schwartz, The Art of the Long View.
81
Jonathan N. Maack
Appendix 1. Preparation
The first step in the scenario analysis process is preliminary interviews, analyze the country
for the task manager to define the intervention situation, and ground-truth the assumptions in
with a counterpart in the client government. each scenario.
The task manager decides how scenario analy-
sis will be used in the project and works out a Assembling the Scenario Team:
general plan to incorporate it in the project A Preliminary Checklist
cycle. She or he then assembles the scenario
team in consultation with the client govern- Although it may not be possible to include
ment and local and international NGOs. Once multiple representatives from each group listed
the team is assembled, the task manager below, the best scenario teams are diverse.
conducts a series of one-on-one, preliminary Below is a list of individuals likely be part of a
interviews to determine the team members’ scenario team:
points of view regarding the intervention.
These interviews can help the guide the 1. Decisionmakers. The task manager and a
organizers of the exercise in assembling senior representative from the borrower gov-
useful background material before the first ernment should jointly lead the process and
workshop. write the final scenarios.
2. People with a thorough knowledge of the
Scenario Team World Bank and its role in the country. A represen-
tative from the Country Office or the task
In the broadest sense, the scenario team is the manager can fill this role.
group of leaders and experts who gather to 3. People with an understanding of the borrower
create four or five distinct narratives about the government and its role in the sector/field (when
future over the course of several workshops. appropriate). This includes senior staff at the
Each of the members gives his or her time, appropriate ministry and officials who are
energy, leadership experience, and technical interested in the scenario process and are active
expertise to the group for the duration of the in the field.
workshops. Team members should be chosen 4. Experts and specialists. This includes
based on their ability to represent distinct Bank, engineers/technical specialists, local
viewpoints on the issue being discussed, be it academicians, and other relevant experts
technical or political. Ideally, all also will be from other donor agencies or international
champions of the scenario process and generate NGOs.
support for its ideas among their colleagues 5. Line ministry staff and/or implementing
and communities. NGOs. These individuals will carry out the
There is also a smaller group within the operation if it is an investment loan. In a
scenario team that plays an active role before, strategy context, the net can be cast more
between, and after the series of workshops broadly, but there is nonetheless a clear need
during which the scenarios are created. This for practitioners to give input on the realities of
smaller group, known as the core scenario team, implementation.
should be led by key decisionmakers at the 6. Leaders from the private sector and from trade
Bank and in the country. The finished scenarios unions. In an investment loan, these representa-
need to be authored by, or in collaboration tives should come from the industry or field
with, the people who will be asked to lead the most closely related to the project. In a strategy
implementation of the intervention. Other context, a diverse group of dynamic business-
members of this smaller core team will conduct persons can be considered. It is important that
82
Scenario Analysis: A Tool for Task Managers
both local and international interests be repre- clarify what has been said. The challenge in this
sented when they will be equally affected by sort of interviewing is to remain unobtrusive
the intervention. and affect the interviewee’s train of thought as
7. Leaders from NGOs, activist groups, religious little as possible. The following list of open-
organizations, and beneficiary organizations. ended questions should help interviewers to
8. Politicians. Members of all relevant politi- move the conversation without directing it. All
cal parties should be included in strategy questions should be phrased so that they apply
discussions. In Colombia, politicians included primarily to the organizations/situations with
guerillas and revolutionaries. which the interviewee has the greatest interac-
tion/association.
83
Jonathan N. Maack
Visual Clustering to Analyze Interview Results group. The final stage consists of using the
“clustered” statements from each related group
Immediately after the interview, interviewers of interviews, which should not total more than
should sit down for at least 30 minutes to go 15 statements per group of interviews, as the
through their notes to identify important basis for a last clustering to form a final, overall
observations to be processed further. A simple map of interview responses.
guideline for identifying key points is: “Is the
view expressed relevant and significant in the Visual Clustering Process
context of the organization’s relationship with
beneficiaries and the external environment?” The core scenario team should write the state-
Issues that are related to internal processes of ments from the interviews on adhesive
the organization should be included only notecards and place them on a display surface
insofar as they affect the way that the inter- or a wall. Statements should be written in
viewee or organization will react to the outside large, heavy characters so that they can be read
world. Interviews normally yield from 40 to 60 from a distance and text should be limited to a
statements. Each statement should be reduced few (at most 8) words. As team members read
to a single line. the statements, they should be able to note
In the next stage of analysis, the interviewer patterns emerging. The idea is to organize the
combines the statements from all of the inter- separate comments into higher-level concepts
views to create logical cause-and-effect chains that can be related to one another.
of reasoning. At first, statements will seem like
random, unconnected thoughts, but as inter- 1. When clusters are finalized, team mem-
viewers read through them, patterns and bers should rephrase each one into a one-word
connections should emerge. Clustering, or statements that sum up the logic of the various
grouping these statements, serves three pur- statements brought together to form it. For
poses: (1) it forces the analysts to pay attention example, a key internal constraint to educa-
to the conceptual meaning of what has been tional planning is the lack of efficient disburse-
said; (2) it organizes the thoughts of a diverse ment mechanisms, which affects both teacher
group into more coherent stories; (3) it is an pay and procurement.
iterative process that promotes serial appraisal 2. Statements that do not fit naturally into
of the outlooks of each of the people inter- existing categories should be put aside and
viewed. This technique is best suited to cluster separate categories created for them later.
comments from a set of 5 to 10 interviews. It is 3. As the process progresses, team mem-
important to remember that with only 9 inter- bers begin to arrange the clusters hierarchi-
views, the core scenario team could be dealing cally according to higher level concepts. For
with as many as 450 statements. example, it may be possible to sum up 15
However, up to 25 persons can participate in statements about disbursement in a cluster,
a scenario exercise. The interviewer can simply which should then be organized under a
take the statements from several related inter- larger cluster focusing on, say, internal
views and cluster them separately. Say, for budgetary issues.
example, there are 20 interviews, and 5
interviewees are from the Bank, 6 are from At the end of the process, there should be
NGOs, 4 are from the client government, and five to six larger concepts and enough clusters
the other 5 have no particular affiliation. The to ensure that no cluster holds more than 15 or
statements of NGO representatives could be so statements. Statements should flow from
clustered separately, as could those of the other simple clusters to larger ideas.
related groups. Clustering enables the inter-
viewer to work with a manageable number of Assembling Background Information
statements. Each related group of interviewees
will then yield a smaller number of “clustered” The clustered results of the preliminary inter-
statements that synthesize the thinking of the view should give the core scenario team an
84
Scenario Analysis: A Tool for Task Managers
idea of the expertise of the group and the and composition, and population growth
information that team members are likely to rates.
need to make informed judgments when • Unpredictable, relevant forces. Unpredictable
building scenarios. If possible, the core team forces such as the price of oil, international
should draw together basic information on investment flows, aid levels, and the sustain-
relevant issues and present it to the group as ability of debt can significantly affect an
a packet. This background can help the intervention. Research into these topics is
group start off with the same basic reference more involved. Industry and academic
points. experts could be consulted, and historical
The collection of relevant information trends affecting these forces could be
regarding these forces should focus on the assembled to aid comparisons with the
country itself, its neighbors, trading partners, current situation.
and (when appropriate) regional trends. The • Sociopolitical information. A focused analysis of
general areas for preliminary data gathering the historical and political situation in the
are: country as it relates to the intervention should
be assembled for the benefit of outsiders. It
• Predictable, relevant forces. This information could become the focus of a brief discussion
might include short-term economic forecasts, during the workshop to enable people from
relevant demographic data, household type the affected areas to add detail.
85
Jonathan N. Maack
Preparation
· Assemble group that is representative and diverse
· Conduct interviews to determine mental models of participants
· Carry out preliminary research based on key issues revealed
· Schedule and arrange workshops
Build Scenarios
Workshop I Workshop II
· Determine decision focus (1 hr) · Rethink scenario logics (1-2 hrs)
· Identify key factors and driving forces (3-4 hrs) · Flesh out scenarios in plenary session (2 hrs)
· Prioritize forces using scenario matrix (2 hrs) · Break out in groups to finish scenarios (4 hrs)
· Write basic scenario plots (1 hr) · Present scenarios to groups and wrap up (1 hr)
Workshop III
· Correct, revise, and amend draft scenarios in plenary sessions (1 hr)
· Explore strategic implications of each individual scenario (3 hrs)
· Develop monitorable indicators for project success in each scenario (2 hrs)
· Highlight “trigger” events that will signal environment is moving toward one of the scenarios (2 hrs)
Implement Strategy
Disseminate scenarios Set up implementation strategy
· Simplify scenarios and prepare them · Build accountability and incentive systems
for dissemination · Implement changes on the ground (ongoing)
· Gain support from public and those who will
implement scenarios (ongoing)
87