Survey Research

Week 5 10/7 & 10/9 Ch. 5

Survey Research
• Used to assess people’s thoughts, opinions, feelings • Pragmatic • Can have varying scope (broad/global or narrow/specific) • Correlational research
– Describe and predict

Characteristics of Surveys
• Predetermined questions
– Oral, written, computer-entered responses – Same questions given to all participants

• Representative Sampling
– Since same questions given, we can describe general attitudes of the sample/population – Can compare/contrast attitudes of different samples

Sampling in Survey Research
• Steps: • Determine population of interest • Decide who should respond
– Select representative sample – Help us generalize findings
• We want to be able to describe/predict to greater population, not just our respondents

– How representative our sample is depends on the extent to which it exhibits the same distribution of characteristics as the population

Sampling in Survey Research continued...
• Population: all cases of interest (all DePaul students) • Sampling Frame: list of all members of population of interest (registrar’s list) • Sample: subset of population drawn from sampling frame (50 students chosen for study) • Element: each member of a population (you)

Sampling Bias
• Bias is a major threat to representativeness • Selection Bias- when procedure used to select the sample results in over/underrepresentation of some segment of the population • Response Rate Bias- low/high response rate among particular participants- may be biased in some way (access, interest, educational level, etc)
– Becomes almost like a convenience sample

Sampling Approaches
• Nonprobability sampling- no guarantee that each element has some chance of being included in the sample
– Convenience Sampling- selecting respondents based on availability-- Intro Psych Courses, “person on the street,” call-in surveys

Sampling Approaches continued...
• Probability sampling- each element has a chance of being selected for the sample
– Superior to nonprobability sampling in ensuring a representative sample

• Two types:
– Simple Random Sampling
• Every element has an equal chance of being included in the sample • Systematic sampling- choose every kth element

– Stratified Random Sampling
• Population divided into subpopulations (strata)- random samples drawn from each strata • Can draw equal random samples from strata or

Survey Methods
• Mail Surveys
– Pros: self-administered, quick, eliminate observer bias, anonymity of respondents – Cons: no options for questions, low response rate (response bias)

• Personal Interviews
– Pros: greater flexibility in asking questions, higher response rate – Cons: costly, interviewer bias (need to remain neutral, do not write only selected portions of response)

Survey Methods continued...
• Telephone Interviews:
– Pros: 97% of households have phones (representative), better access to dangerous neighborhoods or people available only during evening, can be completed quickly, better supervision – Cons: chance of selection bias and interviewer bias, participants may respond differently to a “faceless voice,” cell phones (“on the go”), higher SES have multiple phone numbers, difficulty remembering response options, telemarketers

Survey Methods continued...
• Internet Surveys
– Pros: efficiency, cost, large/diverse samples, cross-cultural, chat rooms (special interest or support groups) – Cons: potential for selection bias (internet access) and response bias (lower response than mail/phone), no real way to generate a random sample of internet users, lack of control because researcher is not present, ethical issues (informed consent, protection from harm)

Survey Research Designs
• Cross-Sectional Design
– One or more samples drawn from population at one time – Description- characteristics of a population, differences between populations – Prediction- predicting certain variables using other variables (demographics)

Survey Research Designs continued...
• Successive Independent Samples
– Series of cross-sectional surveys conducted over time (successively) – Two key aspects:
• Same set of questions to all respondents • Different samples drawn from the same population

– Describe changes in population over time – Limitations
• Respondents do not necessarily “change their mind”different respondents • Noncomparable successive samples- changes in populations

Survey Research Designs continued...
• Longitudinal Design
– Same sample of respondents surveyed on numerous occasions over time – Can determine direction and strength of change over time for the individual – Possible to investigate reasons for attitude/behavior changes – Can assess effect of naturally occurring events (divorce, maturation) – Limitations
• Difficult to pinpoint exact reasons for change • Difficult to find participants to survey over time-- attrition
– Threat to representativeness

• Possibility for reactivity, desire to remain consistent, reluctant to report same problems/attitudes

• Most survey research uses questionnaires • Measure various things: • Demographic variables
– Accurate classification? – Use deliberate and careful approaches

• Preferences and Attitudes
– Self-report scales – Reliability and validity

Reliability in Self-Report Scales
• Reliability- consistency- yields consistent results each time administered
– Scale must be reliable to make predictions – Test-Retest reliability- administering same questionnaire to large sample at 2 times
• Use correlations between participants scores (want r to be at .80 or above)

– Reliability increases with larger number of observations (in this case, questions/items) – Test more reliable when administered to a diverse sample-- more variability on factor being studied – More reliable when testing situation free of distractions, clear instructions are provided

Validity in Self-Report Scales
• Validity- truthfulness- item measures what it intends to measure • Construct Validity- measures theoretical construct it was intended to measure
– Convergent validity- two measures are correlated (converge) when measuring a particular construct – Discriminant validity- measure discriminates between two different theoretical constructs constructs (intelligence and knowledge of a certain subject)

Constructing a Questionnaire
• Steps for Constructing a Questionnaire
– – What information should be sought?
– – – Predict likely results- will it answer your question? Survey method? Develop own or use an established scale? If using established, can skip later steps

What type of questionnaire should you use?

– – – –

Write first draft Reexamine and revise
– – Have friends and experts revise Under similar conditions, ask for feedback

Pretest questionnaire Edit questionnaire and specify procedures/instructions for its use

• Respondents believe wording of a question is obvious
– Interpret ambiguous/vague words based on presumptions

Guidelines for Effective Wording of Questions

• Respondent assumes researcher will ask them questions they can answer questions (even if there is no valid answer) • Vocabulary should be simple, direct, and familiar to ALL participants • Use short questions (<20 words)- all conditional info precedes key idea • No “double-barreled” questions (write separate questions) • No leading questions (leads respondent; mention all possible perspectives or none) or loaded questions (contain emotion-laden words)

Two general types of questions
– Free-response (open-ended)
– – – Respondents give own answers Allows for greater flexibility Needs extensive coding/responding

Guidelines for Effective Wording of Questions continued...

Closed (multiple choice)
– – – – – Answer choices provided by researcher Can be answered quickly/easily Easier to summarize/compare responses Reduce spontaneity and “respondent’s voice” May choose less-than-preferred response

Ordering of Questions
• Self-administered
– Start with interesting questions, end with demographics

• Personal/Phone Interview
– Start with demographics, move into harder questions

• Funnel questions- start with general, move to specifics • Filter questions- general questions that can allow respondents to skip unnecessary questions (mainly to be used for objective information-- “Are you a DePaul student?”)

Things to Keep in Mind about Survey Research
• Truthfulness of participants’ responses
– Reactivity, social-desirability
• Be aware of their existence!

• When two variables items are correlated we can make predictions for the variables/items
– “Correlation does not imply causation” – Causation could go either way – Spurious variables- third variable that accounts for relationship (may or may not be measured) – Mediating Variables (explains correlation between two variables) and Moderating Variables (affect direction/strength of correlation)-- found through Path Analysis

Example of Mediator Variable



Example of Moderator Variable
Stress Level Time Studying
100 90 80 70 60 50 40 30 20 10 0 Low Study High Study

Performance on Midterm

High Stress Low Stress

Midterm Exam Tuesday (10/14)
• Worth 50 points • 30 multiple choice/true-false/matching questions (1 point each) • 5 short answer (2 points each) • 2 long (paragraph) answer (5 points each) • Anything from notes and readings is fair game

Group Exercise (back-up): Get into Groups of 4
• Construct a questionnaire-- stress during exams/studying habits • Steps of constructing a questionnaire (information sought, type of questionnaire, draft, revise, pretest, edit and specify procedures) • Clearly worded, short questions- no double-barreled, leading, or loaded questions • Ordering of questions? Self-administered versus interview? • Closed/open-ended questions? • What type of sampling? Nonprobability (convenience) versus probability (simple random, stratified random)

Class Exercise for Thursday
• Fill out this form • Be sure to print out the “Thank You” page
– Worth 5 extra credit points, but MUST bring it to class on Thursday for credit

• We will go through ways of how to analyze data using survey forms

• Midterm Evaluation for ME • Please fill out this form, and be honest (I will not be able to see your responses) • Be sure to PRINT the “Thank You” page at the end for credit (this assignment will be worth 5 extra credit points- MUST be turned in during class time either Thursday or next Tuesday)

Sign up to vote on this title
UsefulNot useful