Professional Documents
Culture Documents
Certificate
Directly Applicable Action Point(s)
Follow-up Q&A with us for 3 months (via
trainer@alexanderbrookes.com )
Access to regular updates via our newsletter & knowledge
centre
Course Objectives
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
1
From research problem to research questions
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Examples of Survey data
Sampling
CBN Surveys
Emerging data collection platforms
Ethics
interest,
magnitude,
measurement of concepts,
level of expertise,
relevance,
availability of data,
ethical issues
Temporal horizones.
1 Population specification
2 Sampling
3 Selection
4 Non-response
5 Measurement 2
2
see Qualtrics
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Examples of Survey data
Sampling
CBN Surveys
Emerging data collection platforms
Ethics
Population specification
Sampling
Selection
Non-Response
Obtained sample differs from the original selected sample.
Example:
In some surveys, some respondents are inaccessible because they
are not at home for the initial call or call-backs. Others have
moved or are away from home for the period of the survey.
Not-at-home respondents are typically younger with no small
children, and have a much higher proportion of working wives than
households with someone at home. People who have moved or are
away for the survey period have a higher geographic mobility than
the average of the population. Thus, most surveys can anticipate
errors from non-contact of respondents. Online surveys seek to
avoid this error through e-mail distribution, thus eliminating
not-at-home. But in Africa the situation is complex if you
wanted to include the nomadic communities or rural areas.
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Examples of Survey data
Sampling
CBN Surveys
Emerging data collection platforms
Ethics
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
Types of data
Cross-sectional data
Time series data
Panel data
Big data
The World Bank has sponsored and helped design many panel
surveys. For example Nigeria LSMS panel data sets.
Class discussion
Collecting Panel Data in Developing countries: Does It Make
Sense? LSMS working paper (1986)[?]
Ongoing debate on open data
Big data: data sets that are granular, high frequency and/or
non-numeric.
Require high analytical techniques, governing arrangements,
and strategic objectives in central banks,
Recent financial and economic crisis has prompted the
demand for data by central banks
These data are high volume, often because they are reported
on a granular basis, that is, item-by-item, for example,
loan-by-loan or security-by-security.
These data are high velocity, because these data are frequently
updated and, at the limit, collected and analyzed in real-time;
These data are qualitatively various, meaning they are either
non-numeric, such as text or video, or they are extracted from
novel sources, such as social media, internet search records or
biometric sensors.
Common Practice
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
CBN Surveys
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
Induction
Figure 1: Essential
Kiplimocharacteristics
Araap Langat, PhD by ANALYSIS
design, OF
measurement, & analysis
SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling
Emerging data collection platforms
Ethics
Research Design
Qualitative Vs Quantitative
External Validity
External Validity
External Validity
Construct Validity
Construct Validity
The best research designs are those that can assure high levels
of internal and external validity.
Such designs would guard against spurious correlations, inspire
greater faith in the hypotheses testing, and ensure that the
results drawn from a small sample are generalizable to the
population at large.
Manipulation
Elimination
Inclusion
Statistical control
Randomization
Examples
Experimental Design
4
Example: administer the drug to subjects in the treatment group, but only
give a placebo to control group
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling
Emerging data collection platforms
Ethics
Experimental Design
Field Surveys
The limitations of this design are that the data might not have
been collected in a systematic or scientific manner and hence
unsuitable for scientific research, they may not adequately address
the research questions of interest to the researcher, and interval
validity is problematic if the temporal precedence between cause
and effect is unclear
Case Study
Focus Group
Action Research
Action Research
Ethnography
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
Selecting Participants?
The research question will dictate the type of participants
selected for the study
Also need to match the participants to the instrumentation
and methods
Definition of terms
Important Consideration
Representative Sample
Sampling Technique
Probability Sampling
Systematic Sampling
Stratified sampling
Cluster Sampling
Matched-Pairs Sampling
Multi-stage Sampling
Non-Probability Sampling
Convenience sampling;
Quota sampling;
Expert sampling;
Snowball sampling;
Sampling Distribution
Sampling Distribution
How do we know if the sample statistics are at least
reasonably close to the population parameters?
The concept of sampling distribution
Take three different random samples from a given population,
for each sample, you derived sample mean and standard
deviation.
For a sample which is representative of the population, the
three random samples will be identical (and equal to the
population parameter), and the variability in sample means
will be zero.
Hence, a sampling distribution is a frequency distribution of a
sample statistic (like sample mean) from a set of samples,
Commonly referenced frequency distribution is the distribution
of a response (observation) from a single sample.
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling Instruments and Measurements
Emerging data collection platforms
Ethics
Sampling Distribution
Confidence Interval
Biased Sample
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
Research instruments
Response formats
Question sequencing
Quantitative variables
Scales of Measurement
Level of measurement is the first decision to be made in
operationalizing a construct
Levels of measurement, also called rating scales, refer to the
values that an indicator can take (but says nothing about the
indicator itself).
For example, male and female (or M and F, or 1 and 2) are
two levels of the indicator gender.
According to psychologist Stanley Smith Stevens (1946),
there are four generic types of rating scales for scientific
measurements:
1 nominal,
2 ordinal,
3 interval, and
4 ratio scales
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling Instruments and Measurements
Emerging data collection platforms
Ethics
Nominal scales
Nominal
Ordinal scales
Ordinal Scales
Ordinal Scales
Interval Scales
Interval scales are those where the values measured are not
only rank-ordered, but are also equidistant from adjacent
attributes.
Example: The temperature scale (in Fahrenheit or Celsius),
where the difference between 10◦ and 20◦ is the same as that
between 10◦ and 20◦ Fahrenheit.
Likewise, if you have a scale that asks respondents annual
income using the following attributes (ranges): Ksh. 0 to
10,000, Ksh, 10,000 to 20,000, Ksh. 20,000 to 30,000, and so
forth, this is also an interval scale, because the mid-point of
each range (i.e., Ksh. 5,000, Ksh. 15,000, Ksh. 25,000, etc.)
are equidistant from each other.
Ratio
Ratio scales are those that have all the qualities of nominal,
ordinal, and interval scales, and in addition, also have a true
zero point (where the value zero implies lack or nonavailability
of the underlying construct).
Most measurement in the natural sciences and engineering,
such as mass, incline of a plane, and electric charge, employ
ratio scales, as are some social science variables such as age,
tenure in an organization, and firm size (measured as
employee count or gross revenues).
Example
Rating Scales
Binary
Binary scales are nominal scales consisting of binary items
Assume one of two possible values, such as yes or no, true or
false, and so on. Gender Male, Female,
Likert scale
Example
Example
Guttman scale
Example
Scaling
Indexes
Summary
Scale (or index) construction in social science research is a
complex process involving several key decisions. Some of
these decisions are:
1 Should you use a scale, index?
2 How do you plan to analyze the data?
3 What is your desired level of measurement (nominal, ordinal,
interval, or ratio) or rating scale?
4 How many scale attributes should you use (e.g., 1 to 10; 1 to
7; 3 to +3)?
5 Should you use an odd or even number of attributes (i.e., do
you wish to have neutral or mid-point value)?
6 How do you wish to label the scale attributes (especially for
semantic differential scales)?
7 Finally, what procedure would you use to generate the scale
items (e.g., Thurstone, Likert, or Guttman method) or index
components?
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling Instruments and Measurements
Emerging data collection platforms
Ethics
manner.
In order to be valid, a test must be reliable; but reliability
does not guarantee validity.
Reliability
Inter-rater reliability;
Test-retest reliability;
Split-half reliability;
Internal consistency reliability
Inter-rater reliability
5
Usually, this is assessed in a pilot study, and can be done in two ways,
depending on the level of measurement of the construct.
If the measure is categorical, a set of all categories is defined, raters check off
which category each observation falls in, and the percentage of agreement
between the raters is an estimate of inter-rater reliability. For instance, if there
are two raters rating 100 observations into one of three possible categories, and
their ratings match for 75% of the observations, then inter-rater reliability is
0.75. If the measure is interval or ratio scaled (e.g., classroom activity is being
measured once every 5 minutes by two raters on 1 to 7 response scale), then a
simple correlation between measures from the two raters can also serve as an
estimate of inter-rater reliability.
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling Instruments and Measurements
Emerging data collection platforms
Ethics
Test-retest reliability
Split-half reliability
Split-half reliability is a measure of consistency between two
halves of a construct measure.
For instance, if you have a ten-item measure of a given
construct, randomly split those ten items into two sets of five
(unequal halves are allowed if the total number of items is
odd), and administer the entire instrument to a sample of
respondents.
Then, calculate the total score for each half for each
respondent, and the correlation between the total scores in
each half is a measure of split-half reliability.
The longer is the instrument, the more likely it is that the two
halves of the measure will be similar (since random errors are
minimized as more items are added), and hence, this
technique tends to systematically overestimate the reliability
of longer instruments
Kiplimo Araap Langat, PhD ANALYSIS OF SURVEY DATA (SPSS/STATA)
Introduction
Research Design in Analysis of Survey Data
Sampling Instruments and Measurements
Emerging data collection platforms
Ethics
Cronbachs alpha
Cronbachs alpha
Composite scores
5(.346)
α= ) = .73
(1 + (5 − 1).346
Validity
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
PAPI,
CAPI,
WAPI,
CASI,
CAWI,
CATI,
TAPI,
TASI,
SAPI and
SASI.
Introduction to ODK
Course Outline
1 Introduction
Definitions
Examples of Survey data
CBN Surveys
3 Sampling
Instruments and Measurements
5 Ethics
Plagiarism
Group Work
The End