What is a survey?

● Surveys are a systematic method for gathering information from (a sample of) a population for the
purposes of constructing quantitative descriptors of characters and attributes of the larger
population of which the entities are members
● Components: Target population, sampling frame, sampling design, mode of data collection

What broad types are there?

● Face-to-face surveys ● Self-administered computer surveys (typically
● Telephone surveys online)
● Self-administered paper and pencil surveys

What is this method of data collection good for? What kinds of research questions could be
adequately addressed by data collected using this method?
● Understanding social problems and develop programs to address it (e.g. drug use)
● Measuring public attitude and opinion (e.g. election polls)
● Help gauge the representativeness of individual views and experiences (e.g. utilizing standard protocols
for the measurement of crime victimization across the US)
● Understand a specific group (e.g. marginalized group university enrollment)
● Market research (e.g. information about a product or customer satisfaction)
● Assessments of social performance (e.g. National assessment of educational progress)
● Observing progress or changes over time to identify trends and enable predictions (e.g. current
employment statistic program)
● Comparisons (e.g. between effectiveness of products)
● Provide an overview of the status of society or the economy (e.g. Census)

What is this method of data collection not so good for? What kinds of research questions could not be
adequately addressed by data collected using this method?
● Capturing complex or nuanced ● Sensitive topics
information ● Research that has time or money constraint
● Understanding/measuring behaviors or
actions

Issues of validity and reliability which should be considered with data collected through this method
Measurement:
Construct Validity: The extent to which the measure accurately assesses/reflects the construct.
Measurement Error: a departure from the true value of the measurement as applied to a sample unit and the
value provided.
Processing Error: people coding the response based on judgment.

Representation:

Sampling Error - This occurs when the characteristics of a sample differ from those of the population from
which it was drawn, due to random variation in the selection of the sample.
Nonresponse Error - This occurs when there is a difference between the statistics calculated based solely on
respondent data and those based on the data from the entire sample.
Nonresponse Bias: This occurs when those who opt out of the study are systematically different from people
who agree to participate in the study.
Adjustment Error - Adjustment involves weighing the obtained sample data to balance those cases over and
underrepresented (by known standards). The Adjustment error then occurs when there is a difference between
an adjusted statistic and the population parameter.
Coverage error refers to the error that arises from a failure to include some members of the population in the
sampling frame, resulting in a sample that is not representative of the target population.
Coverage bias, on the other hand, refers to the systematic differences between the characteristics of the
individuals who are included in the sample and those who are excluded, which can lead to biased estimates of
population parameters.
(Also refer to Week 5 on threats to external and internal validity)

Problems in Answering Survey Questions

● Failure to encode the information sought ● More or less deliberate misreporting
● Misinterpretation of the question ● Failure to follow instructions
● Forgetting and other memory problems

Issues of knowledge production or research ethics that are particularly prevalent in the use of this
method of data collection (Use examples from assigned readings)

Sensitive topics:
● Issues arise when a researcher does not adopt a trauma informed approach and fails to account for the
harm the survey could cause to participants e.g. national crime victimization survey enquiring about
traumatic past events
Researcher/interviewer positionality:
● Power dynamics between the interviewer/interviewee and researcher e.g. national survey of drug use and
health (NSDUH) asking participants to disclosure information on drug use

Intention of research (purpose and audience):

● Question framing can influence the results- why have you asked your questions in the way you have?
E.g. survey on whether Americans would agree to airstrike in Iran that would kill 100,000 or more
civilians, if it meant they could end a war and save the lives of 20,000 US troops could be used to
manufacture/inflate public support to violate international humanitarian law
● The act of expressing an opinion as a powerful socializing device - Researchers must be reflective on the
power you have as a researcher and uphold ethical obligation to ensure that any adverse effects on public
understanding be noted and minimized. E.g. in the Airstrikes case, can undermine public understanding
of what is permissible under international law
● Reciprocity: Who will benefit from the results? Researchers need to put moral value at the forefront.
Es.g in the Airstrikes case,could inadvertently provide elites with a road map to manufacturing consent
for war crimes making vulnerable populations at risk of political violence.