Research Design

Training of Trainers:
Module 2 Methodology
Webex, May 2020
Research Design
Training of Trainers:
Module 2.1 Methodology
design (Research methods)
Webex, May 2020
 1. Overview of research methods

2. Distinction between quantitative &

Session qualitative research

Contents 3. Types & applicability of different

research methods

4. Q&A
1. Research
method overview
Research methodology

The methodology is an outline of the overall data collection and analysis strategy that will be
used to implement the research cycle

The methodology should:

 Be compatible with the preliminary data analysis plan
 Be designed in a way that ensures the intended scope of the research (i.e. objectives and
research questions) can be feasibly achieved to the required quality, given the time,
resources and access available

Designing a methodology involves three key components: Our focus for today! 
 Selecting the overall research method
 Selecting the appropriate data collection approach(es)
 Designing the sampling strategy
Categories of research methods
Research methods are broadly distinguished between the following categories:
Quantitative
 Measure prevalence of
issues, verify hypotheses
and establish causal
relations between
variables
 Large samples, structured
data collection, and
predominantly deductive
analysis
Qualitative
 Explore and discover
themes, develop theories,
rather than verify
hypotheses and measure
occurrences
 Smaller samples, semi-
structured data collection,
inductive analysis
Mixed Methods
 Combines both qualitative
and quantitative to (1)
collect and analyse both
types of data and (2) use
both approaches in
tandem
Deductive (quantitative) vs. inductive (qualitative) analysis approach
Selecting your research method

What factors to consider when choosing one research method over another?
 Overall applicability to meet research objectives
 Time i.e. key planning and decision-making milestones to inform
 Resources available
 Material resources
 Financial resources
 Human resources
 Access to population of interest
2. Quantitative
vs. Qualitative
research
Differences between quantitative & qualitative research

The distinction between quantitative and qualitative research is not always as clear-cut:

 Individual and household surveys

o Commonly associated with quantitative, large sample research
o Could also be used for a qualitative case study
 Key Informant interviews and community discussions
o Commonly associated with qualitative, semi-structured data collection & analysis
o Could also be used for quantitative data collection & analysis where time and resources do
not allow a large, representative sample
 Focus Group Discussions
o Perhaps the most distinctly qualitative research method, always administered using a semi-
structured data collection tool
o Often analysed using content analysis i.e. a somewhat quantitative approach counting the
number of times a theme or set of words appear with the discussion transcripts
o This content analysis can then inform the more in-depth qualitative analysis.
Differences between quantitative & qualitative research
Distinction between the two can be made based on the following three criteria:
Quantitative
1. Type of data collection Structured, close-ended data
collection tools
2. Type of analysis Measuring prevalence,
quantifying issues, and
primarily involves deductive
analysis
3. Type of sampling strategy Can use both probability or
non-probability sampling 
generalisation to the wider
population possible
Qualitative
Semi-structured (but not
unstructured) data collection tools
Exploratory, and primarily involves
inductive analysis
Non-probability sampling 
generalisation to the wider population
not possible
3. Types &
applicability of
different
research
methods
Types of research methods (1)

Category Type of research Description When to use this method

methods
Quantitative Structured, probability Structured, close-ended data To measure prevalence and make
sampling/ census collection; generalizable claims,
Quantitative analysis; To conduct deductive analysis
(relationship tests, verify hypothesis)
Data collected from a census or
through large samples, with To identify key factors that influence a
sample size calculated based on particular outcome or understand the best
probability theory predictors of a specific outcome

Quantitative Structured, non- Structured, close-ended data To measure prevalence (indicative only)
probability sampling collection; but contextual and/ or logistical
constraints do not allow for large,
Quantitative analysis;
repressentative samples
Can be small or large sizes; non-
To draw indicative inferences from a
probability sampling
sample to a population
Types of research methods (2)

Category Type of research Description When to use this method

methods
Qualitative Semi-structured, non- Semi-structured data collection; No measurement of prevalence or
probability sampling verification of hypothesis needed;
Qualitative analysis;
No or limited prior understanding of the
Relatively small sample sizes;
situation to be studied and the specific
non-probability sampling
variables to be assessed;
To conduct inductive analysis i.e.
explore and develop a theory or pattern
of meaning, based on experiences,
observations and perspectives of the
situation being studied
Mixed N/A Combines both qualitative and Quantitative or qualitative methods by
Methods quantitative methods, both in themselves inadequate to understand
terms of collecting and analyzing the research problem;
both types of data but also using
To use all methods possible to obtain an
both in tandem to enhance the
in-depth, comprehensive understanding
overall strength of the study
of the research problem.
The most powerful research method?

 Mixed methods research – if time, access, resources allow!

 Common misnomer that quantitative research is the strongest – not always!

 Not all issues need to explained in a quantifiable way
 Some issues are over-simplified if only explored in numeric terms
 In-depth explanation and contextualisation is useful

 Ultimately depends on the research objectives

Questions?
Research Design
Training of Trainers:
Module 2.2 Methodology design
(Data collection approaches)
Webex, May 2020
 1. Unit of measurement
2. Types of data collection approaches
(structured)
3. Types of data collection approaches (semi-

Session structured)
4. Types of data collection approaches (mixed
Contents methods)
5. Frequently Asked Questions (FAQs)
6. Overview of remote data collection
7. Q&A
8. Task for the week
Unit of
measurement
What is it?

 The unit that will be used to record,

measure and analyse observations/
information collected

 Examples?
 Individual
 Family
 Household
 Community/ group
 Town/ village
 Facility
 Cow
 Remember…

 Unit will impact the time, resources needed to collect and analyse information
 Unit will define the depth of information possible and scope of analysis

Household level
Individual level
Depth of information

o
s sible t ower
po l
It is e from a ent
t m
g g rega easure sehold
a of m ou ly
Community/Group level unit (e.g. h ut rare
a rd ) b
upw mmunity rsa
to co -ve
vice

Location level

Time / Cost / Access

Data collection
approaches:
Structured
1. The structured survey approach

 Information collected through an interview, a discussion, a conversation

 Using structured, close-ended data collection tools
 Collection of quantifiable information
 Cross-sectional or longitudinal

Types of data collection methods?

Household (HH) survey – collecting data at HH level, to understand experiences and characteristics of HHs within population of interest

Individual survey – collecting data at individual level, to help understand situation and characteristics of individuals within the population of
interest  can include some HH level indicators if needed

Key informant interview – collecting data at community. location or group level from a key informant (KIs) i.e. an individual whose informal/
formal position gives him specific knowledge about other people, processes, or events that is more extensive, detailed, or privileged than
other individuals in their group/ community/ location

Group discussion – collecting data at community, location or group level from a group of representatives e.g. KIs
 1. The structured survey approach- Applicability

When should you use this approach?

 To measure prevalence  provide a quantifiable, numeric description of
the trends, behaviours, experiences, attitudes or opinions of a population
 To generalize findings to a wider population  probability sample 
statistically representative information
 Need prevalence data, understanding of scale of crisis but probability
sampling not possible  non-probability sample  indicative information

Types of research cycles this approach is commonly used for?

Multi-sector needs assessments

In-depth thematic needs assessments e.g. WASH Cluster needs assessment

Longitudinal studies

Third party monitoring (impact evaluation, outcome monitoring, post-

distribution monitoring, etc.)
2. The structured experimental approach

What is it?
 Similar to survey approach
 But relies on experimental survey design 
control vs. treatment group

Types of data collection methods?

Household (HH) survey – collecting data at HH level, to understand experiences and
characteristics of HHs within population of interest

Individual survey – collecting data at individual level, to help understand situation and
characteristics of individuals within the population of interest  can include some HH level
indicators if needed
2. The structured experimental approach - Applicability

When should you use this approach?

To measure prevalence and evaluate the outcomes or impact of a medium to large-scale
intervention on the population of interest
 Generalize findings to a wider population  probability sample  statistically
representative information

Types of research cycles this approach is commonly used for?

 Outcome monitoring
 Impact evaluations
 Etc.
 3. The structured observation approach (Description)

What is it?
 Information collected through observation rather than
conversation
 Using structured, close-ended checklists to collect
quantifiable information
 Looking for specific object, behaviour or event against a
checklist e.g. Household using soap? Damage to health
center? Students participating in classroom?
 Can be used as part of experimental approach

Types of data collection methods?

 Participant observation – researcher participates in context
(e.g. anthropologists)
 Direct observation – researchers observes context (e.g.
psychologists or clinical research)

3. The structured observation approach - Applicability

When should you use this approach?

 Serves similar purpose as survey approach
 Depends on research objectives  observation vs.
conversation?

Types of research cycles this approach is commonly used for?

 Could be same as survey approach
 Could be same as experimental approach
Data collection
approaches:
Semi-
structured
4. The semi-structured discussion approach
What is it?
 Information collected through detailed, narrative interviews, group discussions
 Using semi-structured (NOT UNSTRUCTURED) data collection tools 
open-ended questions, probes
 Purposefully selected participants

Types of data collection methods?

Individual interview – collecting data at individual level, to help understand situation and characteristics of individuals within the population of
interest  can include some HH level indicators if needed

Key informant interview – collecting data at community. location or group level from a key informant (KIs) i.e. an individual whose informal/
formal position gives him specific knowledge about other people, processes, or events that is more extensive, detailed, or privileged than
other individuals in their group/ community/ location

Group discussion – collecting data at community, location or group level from a group of representatives e.g. Kis

Focus group discussion – bringing together people from similar backgrounds or experiences to discuss a specific topic of interest; data
collected at community, location or group level
4. The semi-structured discussion approach - Applicability

When should you use this approach?

 To gather detailed insights about the
experiences, perspectives of specific population
group or location
 To provide a qualitative description of the
experiences, trends, attitudes or opinions of a
population

Types of research cycles this approach is commonly

used for?
 In-depth assessments where there is limited
prior understanding of a situation e.g. access to
cash among refugees & migrants in Libya
 Participatory mapping exercises (mapping
FGDs or KI interviews)
‘Most Significant Change’ data collection technique

 A very specific type of participatory,

discussion-based data collection method
used for monitoring & evaluation
 Invites participants (through KI
interviews, individual interviews or FGDs)
to explain the most significant changes
brought about in their lives by a project
over a given period of time, in key
domains of change
 Useful for third party monitoring or impact
evaluation research cycles
‘Most Significant Change’ data collection technique
The stories, anecdotes you collect from beneficiaries/ project partners, broken
down by «domain» of interest

The stories, anecdotes you select to qualitatively analyse change per

«domain», in consultation with project team
5. The semi-structured observation approach
 Similar to structured observation approach
 But two key differences:
Structured observation
1. Differences in data collection Information collected using a
methods structured set of questions,
usually to identify specific object,
behaviour or event against a
checklist
2. Differences in purpose Provide a quantifiable, numeric
description of the trends,
behaviours, experiences, etc. of
a population
Semi-structured observation
Information collected based on a
short set of open-ended
questions for observations e.g.
movement patterns of refugees
in and out of camps during a
sustained period of time
Gather detailed insights about
the behaviours, experiences of a
specific population group or
location, and to understand, by
observation, how things are done
and what issues exist
Data collection
approaches:
Mixed Methods
 6. Sequential mixed methods data collection

 Method used to sequentially elaborate or expand on the findings of one type of research
method with another
2. Explanatory sequential approach
1. Exploratory sequential approach
Qualitative
Quantitative • Measure
• Identify coping prevalence of • Understand
strategies known coping and
• Measure strategies contextualize
prevalence of
identified observed
Quantitative trends in
coping
Qualitative strategies prevalence

3. The “ideal” sequential approach

Quantitative
• Identify coping • Understand and
strategies • Measure contextualize
prevalence of observed
identified trends in
coping prevalence
Qualitative strategies Qualitative
7. Concurrent mixed methods approach
 Method used to merge or converge the findings from different research methods collected at the same
time
 Alternative to sequential approach if time constraints  sequential better practice if time and
resources allow
 Concurrent mixed methods serves two key purposes:

Triangulation strategy Embedded strategy

 Primary
method:
Convergence quant
of information (What?
collected? Where?)

Key findings &

conclusions

Divergence of Secondary
information method:
collected? qual (How?
Why?)
Case study data collection technique

 Using a combination of different data

collection methods to zoom in to a
specific issue, area or group

 A component within a research cycle,

not a research cycle by itself

 Useful to collect detailed information on

an event, activity, process, group e.g.
zoom in to one specific type of
intervention in an area within a larger
DFID-funded humanitarian programme
Frequently
Asked
Questions
(FAQs)
FAQs (1)

 What is the difference between a key informant interview and an individual

interview? Isn’t the key informant also technically an individual?
 The differences lies in the unit of measurement  individual experiences
(individual interview) vs. community/ village/ institution experiences (KI interview)

 For semi-structured data collection, when is it recommended to use FGDs over KI

or individual interviews?
 This depends on two things
 Research objectives and type of information needed e.g. Variety of opinions
and experiences useful? Specific information needed from an expert? Topics
sensitive to discuss in group setting?
 Logistical constraints e.g. Large number of individuals to be reached within a
short timeframe?
FAQs (2)

 Is it possible to have two different units of measurement in the same questionnaire?

 Ideally, should be avoided, but there are some exceptions:
 Individual information within a household survey (e.g. child attendance roster)
 Household information within an individual survey (e.g. household size or income indicators)
 Individual information within a village/ community/ location level interview (e.g. KI’s displacement status
and experiences, if KI also part of the affected population)
 Household information within a village/ community/ location level interview (e.g. KI estimates # or % of
households affected by a specific situation in a village)

 What if my population of interest includes minors (i.e. individuals <18 years of age)? Can I
collect data from minors?
 Only if absolutely necessary to meet objectives of the research
 Only if required information cannot be collected from adult respondents e.g. parents or caregivers
 Ideally, only from respondents >15 years
 Only if the required protocols are being followed
 Will de discussed later in this training 
Questions?
What methods
to use if you
don’t have
access to the
population of
interest?
 What is remote data collection?

Remote data collection is a means of gathering data without a

physical presence in the data collection location and without direct, in-
person contact with the population of interest

When is it useful? 
When it is not possible to conduct in-person visits to the locations /
populations of interest because of reasons such as:
 Disease outbreak (e.g. COVID-19) 
 Time or resource constraints (e.g. not enough budget to hire
enumerators to cover all areas for face to face interviews)
 Access constraints due to:
 Security concerns
 COVID-19 travel restrictions
 Physical access barriers such as lack of infrastructure
 Severe weather conditions which limits travel
possibilities, etc.
 Etc.
Pros and cons of remote data collection

Pros Cons
Challenging and time consuming to
More time and resource efficient; if set up correctly (e.g. identifying
Planning efficiency necessary logistics already in place, respondents, organizing necessary
could be fairly straightforward to logistics, etc.), difficult to apply
deploy stratification in sampling; challenging
to monitor progress
Higher likelihood of low response
Easier to implement even with limited rates; limited means of verifying
Implementation efficiency time, access and resources (assuming responses/ data quality assurance;
planning and design is done robustly) more challenging to build trust with the
respondents; difficult to deploy long or
complicated questionnaires 
Difficult to have the “full picture” as it
could introduce potential sampling
Ensures maximum possible coverage
Coverage biases (e.g. based on phone network
of areas and population of interest
coverage) and results in exclusions/
despite access constraints 
oversight of certain population groups
or areas
 Some types of remote data collection methods (1)
2. REACH “Area of Knowledge” methodology (face-to-
1. Phone-based (individual, household, community level) face data collection in alternate location)

 Most relevant for: needs assessments, post distribution monitoring (PDMs),

humanitarian situation monitoring (HSM)
 Representative sampling could be possible
 Most relevant for: community-level needs assessments or HSM
 Representative sampling not relevant (requires identifying the
respondent most likely to have the required knowledge)

3. Internet-based data collection

 Tools include: social media, web-based surveys, online discussion

platforms, chatbots (WFP mVAM), etc.
 Most relevant for: community-level needs assessments or HSM (KI
interviews or group discussions), PDMs (individual perception
surveys)
 Representative sampling could be possible (but extremely difficult to
implement e.g. would need email address database and usually low
response rates)
 Some types of remote data collection methods (2)

4. Remote sensing 5. Secondary data review and “expert” consultations

 Only relevant if aim is to gain an understanding based on specific physical

characteristics of an area (e.g. agriculture and vegetation health analysis, shelter  Most relevant for: needs analysis or HSM
damage assessment, flood impact assessment, etc.)  Only feasible if relevant and «reliable» data sources already exist
 Representative sampling or even census could be possible

6. Paper form submissions

 Only applicable if respondents have no movement restrictions and are able

to send paper forms back through required means
 Logistically difficult, not the most time and resource efficient
 Most relevant for: community-level needs assessments or HSM (KI
interviews), PDMs (individual perception surveys)
 Representative sampling could be possible (but extremely difficult to
implement e.g. would need postal address database and expect very low
response rates)
Phone-based data collection example: Iraq UNHCR Cash
Assistance PDM (2017-now)
Project background
 Post-distribution
monitoring (PDM) of cash
assistance and core relief
items to refugees and
IDPs across Iraq
 Project began in 2016
and remains ongoing
 Data collected through
two call centres: Erbil and
Baghdad
 Household level data
collection, providing at
least a 90% confidence
level and 10% margin of
error at Governorate level
Why was it remote?
 To improve time and cost
efficiency, since most of
the data collected would
not be verifiable by
enumerators in the field
 Access to beneficiary
contact lists ensures time-
efficient data collection
 The project has a wide
geographical spread, so
the call centre allows for
rapid, far reaching data
collection
What worked well?
 A team of enumerators
have been well trained and
dedicated to this
assessment continuously
 Availability of anonymised,
comprehensive beneficiary
lists for sampling purposes
 Remote data collection
helps ensure data privacy
 Typically the call centre
remains functional,
regardless of changing
access constraints
Challenges?
 Building trust among
respondents
 Ensuring respondents
understand the role of this
assessment
 Potential for duplication as
beneficiary lists were at the
individual level while
sampling was at the
household level
 Space constraints within
the call centre during
multiple ongoing
assessments
AoK data collection example: 3-border HSM in Sahel
(December 2019- now)

Project background Why was it remote? What worked well? Challenges?

 Humanitarian Situation  To gather information  Once knowledge of  Reliability is not high and
Monitoring in ‘hard to about areas where population movements ability to verify validity of
reach areas’ of ‘3 border’ humanitarian access is low within a region is clear, data collected is low – it’s
area between Mali, Niger or unreliable easy to set up data indicative only
and Burkina Faso collection to ‘capture’
 To ensure supply of  KIs reporting on overall
information about
 Since November 2019 information about these situation at settlement
different areas
 Remote data collection areas is regular and not level can hide inequalities
contingent on access,  Ability to cover data
through face to face  While it is less
allowing for trends across a vast territory
interviews with KIs who challenging finding KIs
monitoring from a handful of static
travel between accessible from relevant geographic
bases.
and inaccessible areas  Less resource intensive – areas, it can be difficult to
good compromise to  Ability to monitor trends find a balance of KI
 Collect information about
gather indicative data in on situation in hard to profiles (men, women,
humanitarian situation in
complement to existing, reach areas and to age groups, vulnerable
each country / areas with
more robust data collection compare and contrast groups etc), impacting
same tool to allow for
systems between severity levels. comparative analysis.
comparability
Now available: SOPs for Data Collection during COVID 
Questions?
Next session?
Task for the
week
 Instructions

Take the research objectives & preliminary analysis plan you formulated last week and briefly
determine:
• Which overall research method would be most appropriate and why?
• Which data collection approach(es) would be most appropriate and why?
• It is up to you to decide whether you want to assume face-to-face data collection is
possible/ remote data collection is necessary in your scenario 
• Don’t go into sampling just yet, we will come back to that next week
• Is there likely to be any sensitive information collected? Is this suitable to the data collection
approach being discussed?
• What additional information do you need to make final decisions on the approaches?

We can discuss how this goes next week!