This action might not be possible to undo. Are you sure you want to continue?
R. D. Wimmer & J. R. Dominick (2003) This handout is prepared with copyright by Jonathan Zhu. Reproduction or distribution is prohibited without prior consent.
Table of Contents
I. The Research Process
1. Science and Research 2. Elements of Research 3. Research Ethics 4. Sampling
III. Data Analysis
10. Introduction to Statistics 11. Hypothesis Testing 12. Basic Statistical Procedures
IV. Research Applications*
13. Research in the Print Media 14. Research in the Electronic Media 15. Research in Advertising 16. Research in Public Relations 17. Research in Media Effects 18. Mass Media Research & Internet
II. Research Approaches
5. Qualitative Research Methods* 6. Content Analysis 7. Survey Research 8. Longitudinal Research 9. Experimental Research * Omitted from this course.
Ch 1. Science and Research
What is scientific research? Why is scientific research the only learning approach that allows for self-correction (i.e., falsification) of existing knowledge or new findings?
Basic characteristics of scientific research
How to conduct scientific research?
Basic steps of research procedure
objective. qualitative or quantitative empirical analysis of one or more variables 4 . controlled.What Is Scientific Research? Research is an attempt to discover something new Scientific research is an organized.
Methods of Knowing Four approaches for people to find answers to research questions: 1. 3. Method of tenacity (to follow or apply previously established traditions) Method of intuition (to apply common sense. 4. to take it for granted) Method of authority (to follow a trusted source) Method of scientific inquiry (to discover truth through objective evaluations of data) 5 . 2.
Characteristics of Scientific Method Scientific method has the following five basic characteristics. anecdotal) Predictive (vs. subjective) Empirical (vs. ad hoc) 6 . speculative) Systematic and cumulative (vs. as compared with other methods of knowing: Public (vs. private) Objective (vs.
3. 8. Replicate the study (when necessary) 7 . 5. 2. Analyze the data and the interpret the results.Scientific Research Procedures A typical research process consists of eight steps: 1. 6. Review existing. 4. Present the results in an appropriate form. Select a problem. 7. Collect relevant data. Develop hypothesis or research questions. Determine an appropriate methodology/research design. relevant research and theory.
Replication Steps of the Research Procedure Presentation of results Analysis & Interpretation of data Data collection Determination of appropriate methodology Statement of hypothesis or research questions Review of existing research & theory Selection of problem 8 .
3.Step 1: How to Select a Research Topic? Issues to consider when selecting a topic: 1. 4. 5. 8. 7. 6. 2. Is the topic too broad? Can the problem really be investigated? Can the data be analyzed? Is the problem significant? Can the results be generalized? What costs and time are involved? Is the planned approach appropriate to the project? Is there any potential harm to the subjects? 9 .
How to Review Existing Research & Theory? Sources to review: Academic journals Professional trade publications Popular magazines The Internet Focus on when reviewing: What type research has been done in the area? What has been found in previous studies? What suggestions do other researchers make for future search? What has not been investigated? What research methods were used in previous studies? How can the proposed study add to our knowledge of the area? 10 .Step 2.
hypothesis is preferred. function. (Zhu) 11 .Step 3. direction. especially when there is a rich body of existing research on the topic. or strength of the relationships. direction. How to State a Hypothesis/Research Question? Hypothesis is a formal statement about the nature. Research question is a less formal statement about relationships between variables without specifying the nature. and strength of testable relationship between variables. (Zhu) Between the two.
How to Determine the Right Methodology and Research Design? (Zhu) Basic research design: Between-subjects design (cross-sectional. 8) Mixed design (cross-sectional and longitudinal) Unit of analysis: Individuals (micro-level) Groups or communities Societies (macro-level) Research purpose: Seek internal validity: use experiment Seek external validity: use survey Seek “thick description”: use case studies. etc. use survey) Within-subjects design (longitudinal. focus groups. indepth interviews. 12 . see Ch.Step 4. observations.
5-9) 13 . How to Collect Data? Sampling (Ch. 4) Instrument Questionnaire (survey or experiment) or coding classification (content analysis) Stimulus (experimental materials) Measuring (Ch.Step 5.
How to Analyze and Interpret Data? (Zhu) Univariate analysis: characteristics of a variable in the sample and the corresponding population (Ch. 12) 14 . 12) Multivariate analysis: relationship among three or more variables in the sample and the corresponding population (Ch.Step 6. 10) Bivariate analysis: relationship between two variables in the sample and the corresponding population (Ch.
How to Present the Results? (Zhu) News releases Business/policy reports Academic journal articles Book/book chapter manuscripts Oral presentations 15 .Step 7.
or method-specific results.Step 8. sample-specific. How to Replicate the Study? Replications are necessary to eliminate designspecific. Basic types of replication: Literal replication (exact duplication) Operational replication (to duplicate only sampling or experimental procedure) Instrumental replication (to duplicate only dependent measures) Constructive replication (deliberately to alter measures and procedures) 16 .
continuous variable Variable vs. reliability 17 . operationalization (Zhu) Constitutive vs. measurement Measurement vs. operational definition Validity vs. variable Variables vs. construct independent vs. Elements of Research Learning Objectives: What are the similarities and differences between the following pairs of terms? Concept vs.Ch 2. scale Conceptualization vs. dependent variable Concept/construct vs. constant (Zhu) Discrete vs.
The difference between a concept and a construct is always arbitrary and relative as a concept may contain sub-concepts whereas a construct may be part of a larger construct 18 .Concepts and Constructs Both are “building blocks” of theory: Concept: a term that expresses an abstract idea formed by generalizing from particulars and summarizing related observations. Construct: a “meta” concept that consists of lowerlevel concepts. usually not directly observable.
DV is what the research wishes to explain. 19 .Independent and Dependent Variables Variables are an “empirical” or “operational” version of concepts/constructs: Independent variable (IV): a force or event that acts as the cause of a process IV can be systematically varied in experiment but observed in survey or content analysis Dependent variables (DV): a force or event that acts as the effect or outcome of a process DV can only be observed in any research setting.
don’t study a constant. In general. variables with a fixed value are not worth studying.e. Constant is a special case of variables that involve a single (i. fixed) value.Variables and Constants (Zhu) Variable has at least two values. 20 . That it..
Discrete and Continuous Variables The values of any variable can be either discrete or continuous: Discrete variable: a finite set of values Continuous variable: an set of values that can be infinitely broken into subparts. 21 .
1) Operational definition: using observable (i..e.Constitutive and Operational Definitions Constitutive definition: using other words or concepts to define or explain the word/concept being defined (see Ch. measurable and quantifiable) variables to define or explain the word/concept being defined 22 .
Operationalization: a thought process to translate an abstract concept into a concrete variable that can be quantitatively measured by a questionnaire (in survey). based on existing theory and past research. coding sheet (in content analysis). The accuracy of the translations is known as “validity” whereas the precision of the translation is known as “reliability”. Both translation processes involve inevitably errors and distortions. and other means of data collection. It involves “translating” concrete events and/or phenomena to abstract symbols and propositions.Conceptualization and Operationalization (Zhu) Conceptualization: a thought process to identify key concepts and formulate their structural relationship. 23 . physiological instruments (in experiment).
Levels of Measurement Measurement is an empirical (i.e. No fine distinction between the two is necessary. 24 . interval and ratio levels are treated in the same way. quantifiable) version of variables: Nominal level (discrete): using arbitrary numbers to classify categories of a variable Ordinal level (discrete): ranking the order of categories of a variable Interval level (continuous): measuring a variable with an equal distance between adjacent points on a scale Ratio level (continuous): with all properties of interval level plus a true zero point Zhu: in practice..
Reliability and Validity Indicators of the quality of a variable (or its measurement): Reliability: the extent to which a measure consistently gives the same results (“precision”) Stability over time Internal consistency among constitutive measures Equivalency between two parallel forms of a measurement Validity: the extent to which a measure captures what it is supposed to measure (“accuracy” or “truth”) Face validity Predictive validity Concurrent validity Constructive validity 25 .
Reliability and Validity Indicators of the quality of a variable (or its measurement): Reliability: the extent to which a measure consistently gives the same results (“precision”) Stability over time Internal consistency among constitutive measures Equivalency between two parallel forms of a measurement Validity: the extent to which a measure captures what it is supposed to measure (“accuracy” or “truth”) Face validity Predictive validity Concurrent validity Constructive validity 26 .
Research Ethics Learning Objectives: What principles.. theoretical grounds and practical consequences)? 27 .Ch 3. guidelines. and codes stipulated that way (i.e. and codes of behavior are available to guide researchers when dilemmas arise? Why are the principles. guidelines.
Why Be Ethical? Unethical practices will harm research subjects damage the reputation of the research community produce misleading results 28 .
Relativism theory: assuming that there is no absolute right or wrong way of behavior.General Ethical Theories The following three theories help researchers to determine what is ethical and what is not: Categorical imperative theory (rule-based): the researcher should act in a way that he or she wants all others to act. the researcher should act based on the established norms and. 29 . better yet. codes of conduct in the local culture. Balancing theory (utilitarianism): the researcher should maximize good and minimize harm from an action.
and decisions of research subjects. values. Beneficence: the researcher should remove existing harms and confer benefits on research subjects. Nonmaleficence: the researcher should not intentionally inflict harm on research subjects. 30 . Justice: the researcher should treat all research subjects equally (to enjoy benefits and avoid harms).Ethical Principles Autonomy (self-determination): the researcher should respect the rights.
Don’t withhold benefits from participants in control groups. Don’t invade the privacy of the participant. Don’t coerce people to participate. Don’t violate the right to self-determination. Don’t expose the participant to physical or mental stress. Don’t fail to treat participants fairly and show them consideration and respect. Always treat every participant with unconditional human regard. Cook’s Code of Behavior Don’t involve people in research without their knowledge or consent. Don’t lead the participant to commit acts that diminish his or her self-respect. 31 .S. Don’t withhold from the participant the true nature of the research or actively lie about it.
plagiarism. deprived authorship 32 .Specific Ethical Problems Voluntary participation and informed consent: remedied by consent forms Concealment (hiding information) and deception (providing false information): remedied by postresearch debriefs Protection of privacy: remedied by measures of confidentiality Federal regulations: human subjects reviews Data analysis and reporting: prevention from tampering with data.
AAPOR’s Code of Professional Ethics & Practices I. 4. 3. Principles of professional practice in conduct A. using only tools and methods well suited to the research problem. not interpreting research results inconsistent with the data. Exercise due care in gathering and processing data. (cont’d on next page) 33 . 2. to assure the accuracy of results B. not implying greater confidence in the results than what the data warrant. not using tools and methods that yield misleading conclusions. Exercise due care in research design and analysis: 1.
respondents: shall not lie. the profession: share as freely as possible ideas and findings to improve to the profession D. clients and sponsors: protect their confidentiality and accept only assignments possible to accomplish within technical limitations C. abuse. the public: disclose methods used and correct possible distortions when releasing results B.AAPOR’s Code of Professional Ethics (cont’d) II. Principles of professional responsibility in dealing with people A. protect their confidentiality 34 . or humiliate them. coerce.
Ch 4. Sampling Learning Objectives: What is probability sampling? Why is it important and necessary to draw probability samples? How to draw probability samples? How to determine sampling error (in relation to confidence level and confidence interval)? How to determine sample size? 35 .
e. differences from the population). “study population”). variables.e.e. Census involves only measurement errors (i. or phenomena under study (i. A study of every member of the study population is known as “census”. Sample involves both measurement errors and “sampling errors” (i. inconsistencies produced by the instrument used). Sample: a sub-set of the study population that is representative of the population. The process of drawing a sample from the study population is known as “sampling”... 36 ..Population and Sample Population: the entire set of subjects. concepts.
37 . With the selection probability unknown.Probability and Nonprobability Samples Probability sample: selected according to random principle whereby each unit’s chance for selection is known. Given the selection probability known. sampling errors of a nonprobability can not be estimated. Nonprobability sample: selected without following random principle so that units are selected with an unknown chance. sampling errors of a probability sample can be estimated.
Zhu: of the above issues.When to Use a Nonprobability Sample? Purpose of the study: when it is not to generalize the results to the population (i. 38 . for inferential purpose).e. or a pilot study to aims to test questionnaire or instrument Cost availability: when there isn’t adequate money to collect a probability sample Time availability: when there isn’t adequate time to collect a probability sample Acceptable errors: when the amount of errors is not a prime concern. it is to investigate the relationships between variables.. instead. the generalizability is often the most important consideration.
Snowball sample: a collection of subjects selected based on “referrals”. Volunteer sample: a collection of subjects selected not by random principle but by self initiation. which eliminates all others who fail to meet the criteria.Nonprobability Samples Available sample (convenience sample): a collection of readily accessible subjects. Quota sample: a collection of subjects selected to meet a predetermined or known percentage (quota). 39 . Purposive sample: a collection of subjects selected for specific characteristics.
] 40 . random digital dialing (RDD) procedure produces an SRS Systematic random sample: every n-th subject is selected from the population. in which two or more of the above are used. In telephone survey. Stratified sample: selected after the population is divided into demographic strata. Cluster sample: selected after the population is divided into geographic clusters [Note: in practice. multistage sampling is often used.Probability Samples Simple random sample (SRS): each subject in the population has an equal chance of being selected.
Kish Random Tables.2) Based on “last-birthday” principle (added): . see Table 4.Random Selection of Individuals from Households Either of the following methods is used to ensure that individuals are selected randomly from the chosen households/organizations: Based on predetermined demographic quota (e. selecting the member of the household/organization whose birthday is the most recent 41 .g..
e..Sampling Error Sampling error (standard error): the degree to which statistics (i. measurements obtained from a sample) differ from parameters (i. but can be estimated 42 ..e. the same measurements that would be obtained from the population) Sampling error is a function of sample size Sampling error is inevitable.
the population parameter would fall within the confidence interval 43 . the larger sampling error The positive and negative sampling errors associated with a given confidence level form a range known as “confidence interval” In practice. Confidence Interval. & Confidence Level Sampling error is a relative concept that takes different values for a given sample depending on the “confidence level” arbitrarily chosen by the researcher The higher confidence level. if 100 samples of the same size were drawn.Sampling Error. 95% is considered to be the minimally acceptable confidence level.
Calculation of Sample Error se = p(1 − p) ×z n where se is the sampling error.57 44 . z = 1. e. and z is the z-value associated with a confidence level. Confidence level = 68%.. n is the sample size. p is the percentage of interest in the population (often assumed to be 50%). z = 1 Confidence level = 95%. z = 2.g.96 Confidence level = 99%.
or explanatory Project complexity: univariate. inferential. pilot study. bivariate. or formal study Project purpose: descriptive. single level or multilevel. or multivariate.Considerations for Sample Size Project type: focus groups. crosssectional or longitudinal Amount of error tolerated: “low incident” events are particularly sensitive for errors Time constraints Financial constraints Previous research in the area 45 .
maximally acceptable sampling error (se) and a predetermined. if confidence level = 99%.Calculation of Sample Size (Zhu) The formulae for sampling error can also be used to determine sample size given a predetermined. z = 1.54.96. z = 2. 46 . minimally acceptable confidence level (which determines the value of z): p(1 − p) 2 n= z 2 se Note that if the chosen confidence level = 95%. etc.
Ch 6. Content Analysis Learning Objectives: What is content analysis? Why do we need to use content analysis? Limitations of content analysis and over-use of content analysis How to conduct content analysis? How to ensure the systematic and objective nature? 47 .
and quantitative manner.What Is Content Analysis? Kerlinger (2000): content analysis is a method of studying and analyzing communication in a systematic. Systematic: content is selected and analyzed according to explicit and consistently applied rules: sampling procedure and coding scheme Objective: content is analyzed based on explicit operational definition and classification rules Quantitative: content is precisely counted 48 . objective.
Why to Use Content Analysis?
To describe communication content: text, graphics, audio, video, hypertext To test hypotheses of message characteristics: “If the source has characteristic A, then messages containing elements x and y will be produced; if the source has characteristic B, then messages with elements w and z will be produced.” To compare media content to the “real world”: portrayal of some group, phenomenon, trait, etc. is assessed against a standard taken from real life. To assess the image of particular groups in society: a specific application of describing communication content To establish a starting point for studies of media effects: content as the input of the effects process
Limitations of Content Analysis
Content analysis alone cannot serve as a basis for making statements about the effects of content on audiences. The findings of a particular content analysis are limited to the coding scheme used (e.g., operational definition of TV violence). It is difficult to content analyze “low incident” phenomenon. Content analysis is often time-consuming and expensive (i.e., labor intensive).
How to Conduct Content Analysis?
Basic steps in content analysis:
Formulate research question or hypothesis 2. Define the study population 3. Select a sample from the population 4. Select and define a unit of analysis 5. Construct classification categories 6. Establish a quantification system 7. Train coders and conduct a pilot study 8. Code the content based on established rules 9. Analyze the coding data 10. Draw conclusions and search for indicators
Formulating a Research Question or Hypothesis Research question or hypothesis is generated based on: Existing theory Prior research Practical issue Zhu: research question or hypothesis should be: Specific Falsifiable Insightful 52 .Step 1.
audio. etc.g. text. etc. Defining the Study Population To specify the boundaries of the body of content: What should be included and why? Topic Format (e.?) Time period 53 . text.Step 2.?) Time period What should be excluded and why? Topic Format (e. audio.g. graphic. graphic... video. video.
Select appropriate media outlets (purposively or randomly) Select appropriate dates (SRS. or stratified to create “composite week”) 12-14 issues per year for print media 2 days per month for broadcast media Select specific section/program (when necessary or appropriate) 54 . systematic.g. 1. Selecting a Sample A census of the entire population is sometimes desirable and feasible in content analysis Sampling in content analysis often involves a multistage procedure. 2. e..Step 3. 3.
episode. Unit of analysis should be clearly defined (i. sentence. paragraph. program. segment. theme. act. by a clear-cut operational definition) Whenever possible. article.. etc. etc.e. lower unit of analysis is preferred because it can be combined into a higher unit of analysis in the future 55 . Selecting a Unit of Analysis Unit of analysis: the smallest element (which cannot be further divided) of a content analysis Examples of unit of analysis in content analysis: Print media: word/symbol. Broadcast media: character.Step 4.
multi-response coding should be used (which creates complications for statistical analysis) Exclusive: every unit of analysis can be placed in a category When necessary. set up a category of “others” or “miscellaneous” (which should not exceed 10% of the sample) Reliable: different coders should agree about the proper category for each unit of analysis (see intercoder reliability) 56 . Constructing Coding Categories Classification categories should be: Mutually exclusive: a unit of analysis can be placed in one and only one category Zhu: when mutually non-exclusivity is either inevitable or desirable.Step 5.
Step 6. Establishing a Quantification System The quantification system can take any of the four levels of measurement: Nominal Ordinal Interval Ratio 57 .
Step 7.e. Training Coders and Doing a Pilot Study Number of coders is used for the same content: 2-6 Coders should not be informed of the research question/hypothesis (i.. “blinded coding”) Training coders: To revise definitions and categories To identify “opinionate” coders (and replace such coders who fail to conform to the rules) Pilot study: to check intercoder reliability 58 .
Step 8. Coding the Content A standardized coding sheet based on the operational definition and classification rules should be used for coding An instruction sheet should be provided as an accompanying instrument for the coding sheet 59 .
Analyzing the Data Univariate analysis: Percentage or mean (and standard deviation) of each dependent variable Bivariate analysis: Difference in percentage or mean of each dependent variable between/among groups of an independent variable Correlation between the dependent and independent variables Multivariate analysis: Difference in percentage or mean of each dependent variable between/among groups of across several independent variables Correlation of a dependent variable with several independent variables 60 .Step 9.
Step 10. independent “benchmark” indicator is needed to help interpret the findings Zhu: Answers to “why” question almost always come from data outside content analysis Example: Zhu (1991) 61 . Interpreting the Results For descriptive studies: additional.
its measurements and procedures must be replicable (i.e.Reliability of Content Analysis Reliability: if a content analysis is to be objective.e. Reliability of content analysis is measured by intercoder reliability (i.. degree of agreement between independent coders) A poor intercoder reliability indicates problems with: coding instrument or instructions coder training unit of analysis 62 .. repeated measurement of the same material results in similar conclusions).
respectively. N1 and N2 are the number of decisions by the first and second coders. O−E R= 1− E where O is the observed agreement (in %) between two coders.Intercoder Reliability Holsti’s formulae: Scott’s pi (which offsets agreement by chance): 2M R= N1 + N 2 where M is the number of coding decisions on which two coders agree. E is the expected agreement (in %) given by: 1 E= p (where P is the number of categories of the variable): 63 .
Validity of Content Analysis The validity of content analysis results is affected by both sampling method and coding method. The validity can be assessed based on: Face validity: if the categories are rigidly and satisfactorily defined and followed Concurrent validity: if the results are consistent with similar data obtained from an independent source Construct validity Predictive validity 64 .
9) How to conduct surveys? Questionnaire design Sampling Data collection Data analysis 65 . Survey Research Learning Objectives: What is survey research? Why do we need to conduct surveys? Advantages and disadvantages of survey Differences between survey and experiment (Ch.Ch 7.
Descriptive surveys: to describe or document what the current condition is Analytical surveys: to explain why the current condition exists 66 . behavior. and other relevant information.What Is Survey Research? A survey involves interviewing a sample of respondents by face-to-face. telephone. mail. or other methods to measure their knowledge. with the aim to project the findings to the study population. attitudes.
reliable. and practical way to gather information representative of the general population 67 .Why to Use Survey Research? Survey research has the following advantages: Able to investigate problems in realistic settings Costs reasonably given the amount of information gathered Able to collect a large amount of information from a variety of people with relative ease Not constrained by geographic boundaries Zhu: the only valid.
9 on experimental research). to establish a causal relationship under study. Inappropriate wording a placement of questions within a questionnaire may bias results.. if possible at all.Disadvantages of Survey Research Real causal factors (i. independent variables) cannot be manipulated (see Ch. 68 .e. It has become increasingly difficult to obtain a high response rate. Survey methods other than face-to-face interviews may not be able to screen out wrong respondents. which makes it difficult.
Make necessary callbacks/revisits 8. Pretest the questionnaire 5. Verify the results (quality control inspection) 9. 69 . Train interviewers 6. Select the sample 3.Basic Procedure of Survey Research Select the method of interviews 2. Carry out fieldwork 7. Tabulate and analyze the data 10. Zhu: calculate response rate 1. Construct the questionnaire 4.
Step 1. the respondents don’t get to see the questionnaire CATI: Computer-Assisted Telephone Interviews Personal interviews: respondents are interviewed face-to-face at either their home/workplace or a field service location CAPI: Computer-Assisted Personal Interviews Group administered surveys: a group of respondents is gathered together and asked to fill individual copies of a questionnaire 70 . Selecting the Method of Interviews Mail surveys: sending self-administrated questionnaires to a sample of respondents with stamped reply envelops enclosed Telephone interviews: trained interviewers ask questions and record the answers over the phone.
Comparisons of Data Collection Methods Mail Phone Personal Group 2nd Most 2nd Most Cost Cheapest Cheapest Expensive Expensive 2nd Time Slowest Quickest 2nd Slowest Quickest Limited Selection of No control Full control Full control Respondents control Interviewer Lowest 2nd Highest Highest 2nd Lowest Bias Assistance Minimal 2nd Minimal 2nd Fullest Fullest Available Response Lowest 2nd Highest Highest 2nd Lowest Rate 71 .
Step 2. Selecting the Sample
Mail surveys: drawn from the appropriate sampling frame (i.e., mailing list with names and addresses of respondents) Telephone interviews: drawn from telephone directory (with necessary modifications, e.g., adding a random digit) or from a random digital dialing program Personal interviews: commonly drawn from a multistage sampling procedure Group administered surveys: drawn a mailing list, a telephone directory (or RDD program), or other sampling procedures
Step 3. Construct the Questionnaire
Basic rules of questionnaire design:
Understand the goals of the project so that only relevant questions are included Questions should be clear and unambiguous Questions must accurately communicate what is required from the respondents Don’t assume respondents understand the questions they are asked Follow Occam’s Razor (the simpler the better)
See later part of this chapter for more details of questionnaire design
Impact of Data Collection Methods on Questionnaire Design
Mail surveys: questions must be easy to read and understand before respondents are unable to seek explanations Telephone interviews: response options for all questions must be fewer and shorter than other methods Personal interviews: the interviewers must tread lightly with sensitive and personal questions because his/her physical presence may make the respondent less willing to answer
Pretest the Questionnaire All questionnaires must be tested at least once before put in formal use Pretest can be done among: a focus group an informal sample of 10-20 people Particular attention should be paid to how easy the respondents in the pretest understand the questions 75 .Step 4.
. writing down exactly the original wording without any personal interpretation) 76 .e. should be trained Training should focus on: what the questions are about how to ask the questions (i. reading out exactly the original wording and instruction without any personal interpretation) how to probe follow-up questions (e. asking “what else” at least once after the respondent gives an answer to openended questions) how to take answers from respondents (i... Train Interviewers All interviewers.e.Step 5.g. experienced or beginner.
000: Mail: 2 months (with 3 rounds of mails) Telephone: 1-2 weeks Personal: 2-4 weeks Excessively long fieldwork may introduce unexpected events into the survey 77 . Carry out the Fieldwork Weekends and evenings of weekdays are generally preferred time for interviews Long public holidays should be avoided Usual duration of the fieldwork for a sample of 1.Step 6.
Make Necessary Callbacks/Revisits Zhu: Any non-contact respondent should be called back or revisited 3-5 times Callbacks/revisits are necessary and effective means to improve response rate Must callback or revisit: Those who were contacted but not available for interview with or without an appointment made Those who have never been contacted Optional callback or revisit: Those who broke off the interview Those who refuse the interview 78 .Step 7.
. Verify the Results Supervisor(s) call back/revisit a subsample (10-20%) of the complete cases by each interviewer to ensure the interviews take place. Zhu: the quality control (QC) verification usually focuses on a few specific details of the survey that are easy to remember. e.g.Step 8. method of the interview (telephone or personal) sex of the interviewer (male or female) general topic of the questions type of the gifts (if any) 79 .
illegal or illogical values. and recode all necessary values Demographic distribution: run frequencies on age. Analyze the Data Data cleaning: run frequencies to identify missing.Step 9. sex (or other key variables) and compare the results with population census data (and weight the sample if necessary) Formal analyses Univariate Bivariate Multivariate 80 .
2003. Of the 6 formulas of AAPOR.Step 10. 81 . Zhu. 130113). RR4 represents the best balance between the most conservative (which tends to underestimate the real SRR) and the most liberal (which tends to overestimate the SRR). & Sun. American Association for Public Opinion Research (AAPOR) has published standard formulas to calculate SRR so that different surveys can be comparable (see Ke. pp. Calculate the Response Rate (Zhu) Survey response rate (SRR) is the most important indicator of the quality of the survey.
etc.g..). O = other eligible but unsuccessful cases. 82 . P = partially completed interviews. UH = households status unknown. which is often estimated by: I + P + R + NC + O e= I + P + R + NC + O + NE where NE is cases known to be non-eligible (e. R = refusals and break-off cases.Calculation of RR4 I+P RR 4 = ( I + P) + ( R + NC + O) + e(UH + UO) where I = completed interviews. NC = noncontacts. nonresidential units. unqualified individuals. UO = other unknown households. and e = estimated proportion of eligible but unknown cases.
How to Design a Questionnaire? Question wording Question types Question formats Introduction Screener/filter questions Instructions for interviewers and respondents Question order Questionnaire length 83 .
..e. suggesting a certain response or hidden premise) Do not ask highly detailed information Avoid potentially embarrassing questions 84 .e. involving two or more questions in one sentence) Avoid biased words or terms Avoid leading questions (i.Guidelines for Question Wording Make questions clear Keep questions short Include only relevant questions Do not ask double-barreled questions (i.
DV=Dependent Variable 85 . IV=Independent Variable.How to Determine Which Questions to Ask in a Survey? (Zhu) Knowledge (DV) Media Exposure (IV) Personal Background (Control) Media Access (IV) Behavior (DV) Attitudes (DV) CV=Control variable.
Question Types Open-ended questions: requiring respondents to generate their own answers Flexible Time consuming Need to use content analysis to process Close-ended questions: requiring respondents to select an answer from a list of predetermined options Ease to quantify Rigid Need an “Other” category for unforeseen answers 86 .
Common Formats of Questions Multiple choices Check list Forced choice (from a pair of statements) Rating scales Likert scale Semantic differential Feeling thermometer Rank ordering Fill in blanks 87 .
Introduction of Questionnaire To inform the respondent about the survey organization the (general) purpose of the survey the duration of the survey the anonymity and confidentiality of the respondent Characteristics of a successful introduction (Backstrom and hursh-Cesar. 1986): short realistically worded nonthreatening serious neutral pleasant but firm 88 .
Screener/Filter Questions To exclude unqualified respondents from the entire or part of the questionnaire Screener for exclusion from the entire questionnaire: placed right after the introduction section Screener for skips over section(s) of the questionnaire: placed before the section(s) to be skipped 89 .
To explain how to answer the questions
Instructions for the interviewer (in telephone or personal interviews):
provided as much as possible usually typed in capital letters and enclosed in brackets or boxes to be distinguished from instructions for the respondent
Instructions for the respondent (particularly important for self-administered questionnaire):
used only when necessary to avoid confusions whenever possible, providing examples for illustration
Generally, the following order is recommended:
Start with simple, easy, and general (i.e., “warmup”) questions Questions serving as dependent variables general go before questions serving as independent variables (to avoid “contamination” or priming effects) Do not ask knowledge questions at the beginning (to avoid embarrassment) Place demographic, personal, and other sensitive questions at the end of the questionnaire
How long is a questionnaire too long?
when there are 10% or more breakoffs (i.e., respondents who drop out before the end of the interview)
Wimmer and Dominick’s recommended maximum length:
Self-administered mail or group survey: 60 min. Face-to-face interview: 60 min. Telephone interview: 20 min.
Zhu: Questionnaires longer than half of the above generally result in high breakoffs among Chinese respondents.
Are explanations offered for abstract concepts or technical terms? 5. Are questions ordered from easy to difficult and from general to specific? 9. 93 . Is there any “double-barrel” question？ 7. Is a mid-ground position provided in the answers of attitudinal question? 3. Are personal questions asked at the end? 10.A Checklist for Questionnaire Design (Zhu) Are the answers in multiple choice questions complete and mutually exclusive? 2. Are similar question placed together? 8. Are the question and answers clearly and unambiguously expressed? 4. Are the question and answers too long? 6. Are screener questions and continued questions appropriately linked? 1.
What to Include in a Survey Report? Section Abstract Introduction Literature Review Methods Results (Statistical Test) News Release No Yes No Yes Yes No Business Report Yes Yes Optional Yes Yes Optional Academic Paper Yes Yes Yes Yes Yes Yes Discussion References No No Yes Optional Yes Yes 94 .
Longitudinal Research Learning Objectives: What is longitudinal research? Differences from cross-sectional research Why do we need longitudinal research? Requirements for causal relationships How do we conduct longitudinal research? Trend study Cohort analysis Panel study 95 .Ch 8.
96 . longitudinal research collects data from the same sample or different samples at different points in time. but a new research design involving existing methods of data collection and analysis. Longitudinal research is not a new method of data collection or statistical analysis.What Is Longitudinal Research? Unlike cross-sectional research that collects data from a representative sample at only one point in time.
Why to Use Longitudinal Research? (Zhu) Compared with cross-sectional research. longitudinal research enables the researcher to: Identify changes over time Trace the time order of a causal relationship Requirements for causality: Time order between independent and dependent variables Association of the two variables Exclusion of all other alternative explanations 97 .
How to Conduct Longitudinal Research? Trend studies: a topic is restudied using different samples drawn from the same population Cohort analysis: tracking specific age cohorts as they change over time Panel studies: the same sample of people is measured at different points in time Retrospective panel: members of a cross-sectional sample reconstruct their past by recalls Follow-back panel: a cross-sectional sample is compared with a corresponding archival data Catch-up panel: a cross-sectional sample in the past is compared with current data available from other sources 98 .
15 .Classification of Longitudinal Research (Zhu) Number of Time Points Unit of Analysis Sample Group Individual 2-30 Trend study Cohort analysis Panel study 99 30+ Time series analysis Source: Ke. Ch. Zhu & Sun (2003).
high attrition. allow secondary analysis Vulnerable to changes in sample or measurement Cohort Analysis Detect the effects of maturation and social changes Panel Studies Able to identify dynamic changes Costly. cohort and period. vulnerable to sample mortality .Comparison of Longitudinal Designs Trend Studies Establish longterm patterns. sensitization to research instrument 100 Advantages Disadvantages Difficult to separate age.
Experimental Research Learning Objectives: What is experiment? Why do we need experiment? Requirements for causality How to design and conduct experiment? How to select the appropriate design? How to manipulate the independent variable? How to randomize the subjects? 101 .Ch 9.
Field experiment takes place outside lab settings to mimic real life in natural settings. 102 . (Zhu) Quasi-experiment does not involve random assignment of subjects to experimental groups. control conditions so that the effects of experimental stimulus (“manipulation”) could be directly tested.What Is Experiment? The classic experiment (“controlled labolatory experiment”) is a research procedure in which subjects are randomly assigned to experimental vs.
e. 103 .Why to Use (Lab) Experimental Research? It helps establish causality (i. experiment is considered “the most rigorous method”. It costs relatively less than other methods. It is easy to replicate. As the first two merits (establishment of causality and control for confounding effects) are the most central in science research. It allows control for confounding effects.. cause and effect).
3. 4. 6. 7. randomization) Conduct a pilot study Administer the experiment Analyze and interpret the results 104 . 8.Procedure of Experimental Research 1. Select the experimental setting Select the experimental design Operationalize the variables Decide how to manipulate the independent variables Select and assign subjects to experimental conditions (i. 5..e. 2.
Select the Experimental Setting Experimental settings: Controlled laboratory settings (to ensure interval validity) Natural settings (to ensure external validity) 105 .Step 1.
.Step 2. without randomization) 106 . interrupted time series design) Pretest-posttest design with nonequivalent control group (i. Select the Experimental Design “True” experimental designs Posttest-only design with control group Pretest-posttest design with control group Solomon design with four groups Factorial design Other experimental designs Repeated measures design without control group (similarly.e.
. display of imitation actions) 107 . Operationalize the Variables Independent variable: experimental stimulus to apply to the subjects Dependent variable(s): responses from the subjects (before and) after exposed to the stimulus Attention (e.g. secondary tasks such as button push) Knowledge (e..g. recalls) Attitudes or perceptions (e..g. questionnaire) Behavior (e..g.Step 3.
verbal instructions.g.Step 4. or other stimuli are presented to the subjects. In general. events or stimuli for presentation to the subjects: Straightforward manipulation: written materials.. Staged manipulation: the researcher constructs events or circumstances (e. Manipulate the Independent Variable Develop a set of specific instructions. by using a “confederate” who pretends to be a subject) to manipulate the independent variable. the manipulation should be as strong as possible to maximize potential differences between the experimental groups. 108 .
Select and Assign the Subjects Selection of experimental subjects: ideally.Step 5. subjects should be randomly selected from the study population to ensure external validity Assignment of experimental conditions: the chosen subjects should be randomly assigned to experimental condition(s) and control condition to eliminate (or minimize) confounding effects that exist among subjects: Randomization: Matching: 109 .
especially to test whether the manipulation of the independent variable is strong enough to have the intended effects.Step 6. 110 . Conduct a Pilot Study A pilot study with a small number of subjects helps reveal problems with stimuli and/or measurement.
.e. Administer the Experiment Formally carry out the main phase of the experiment: Have the subjects to read and sign a “consent form” (required by human subjects review committee) Randomization of subjects Apply the manipulation (i. stimulus) Measure subjects’ responses Debrief the subjects at the end of experiment to inform them the real purpose and potential implications of the study 111 .Step 7.
. nominal scale for the independent variable and interval/ratio scale for the dependent variable). experimental data are mostly analyzed with: t-Test (when there are two groups involved) ANOVA (when there are three or more groups) MANOVA (when there are several parallel or repeatedly measured dependent variables) 112 .Step 8. Analyze and Interpret the Results Given the specific levels of measurement used (e.g.
Introduction to Statistics Learning Objectives: What is descriptive statistics? Distributions vs.Ch 10. summary statistics What is inferential statistics? Sampling distribution How to calculate descriptive statistics? Sample distributions Central tendencies Dispersions Normal curve z-score 113 .
Zhu: statistical indicators used to describe quantitative characteristics of a sample Key descriptive statistics: Distribution of variables: frequencies Central tendency of variables: mean. mode Dispersion of variables: variance.What Is Descriptive Statistics? Descriptive statistics: statistical methods and techniques used to reduce data to allow for easier interpretation. median. standard deviation 114 .
What Is Inferential Statistics (Zhu)? Inferential statistics are indicators used to estimate quantitative characteristics (i.e. parameters) of a population from relevant descriptive statistics of a corresponding sample Key inferential statistics: Sampling distributions Sampling errors (also known as “standard errors”) Confidence levels Confidence intervals 115 ..
Sample Distributions Distribution: a collection of numbers. Zhu: a collection of all possible values of a variable observed/measured from a sample Distribution can be described by a frequency table that contains: Values Counts (frequencies) Percentage Valid Percentage Cumulative Percentage Distribution can also be described by a graphic chat: Bar chat (for nominal or ordinal scale variables) Histogram (for interval or ration scale variables) 116 .
. some of skewed distributions can be transformed to become proximately normal 117 .Shapes of Distribution Skewness of a distribution: Right skewness: the tail of the curve trails off to the right of the distribution Left skewness: the tail of the curve trails off to the left of the distribution Normal distribution: the two halves of the curve are identical (i. symmetrical) Zhu: normal distribution has desirable mathematical properties for many statistical analyses.e.
the sum of all scores divided by N). a “statistic”) to describes the “typical” or “average” feature of a sample distribution Common central tendency statistics: Mean: the arithmetical average of a distribution (i.. which is the most frequently used Median: the midpoint of a distribution with half of the scores above and half below it Mode: the score(s) that occur(s) most frequently 118 .Statistics of Central Tendency Central tendency uses a single number (i.e.e..
. n is the number of cases (i. X is the midpoint of that interval or group 119 . X is any score of the distribution.e.. grouped) scores: where X (read “x bar”) is the mean. sample size) ∑ fX X= n where f is the frequency of each given interval (or group).∑ is the summation symbol.e.How to Calculate Mean? Based on original scores: ∑X X= n Based on aggregated (i.
“spread-out.” or deviation from the central tendency of a distribution Common dispersion statistics: Range (difference between maximum and minimum) Variance Standard deviation 120 .Statistics of Dispersions Dispersion statistics describe the variability.
standard deviation is the squared root of variance.How to Calculate Variance & Standard Deviation? Variance (S2) Standard Deviation (S) S = 2 ( X − X )2 ∑ n −1 S= ( X − X )2 ∑ n −1 Zhu: Notice that variance is just the squared standard deviation. While variance is unitfree. standard deviation takes the same unit as the original scores (X). 121 .
A particular z-score tells how many standard deviations the original score is above or below the mean of the sample. 122 X−X z= s .e. measurement units) because all zscores have a mean of 0 and a standard deviation of 1..Standard Scores (z-Scores) z-scores are transformed values from the original scores based on the mean and standard deviation: z-scores help compare scores obtained from totally different methods (i.
1% 34.e..6% 34.4% 13.6% 1 2 -2 -1 0 z 123 . Distribution of z-Scores) P (z) 50% 95.Standard Normal Curve (i.1% 13.
Sampling distribution is a virtual (i. non-existent) distribution. with a fixed population size (N) Sampling distribution: the collection of all possible values of a statistic (i.e..Sampling Distribution Distributions: Sample distribution: the collection of all values of a sample (actually measured). with a fixed sample size (n) Population distribution: the collection of all values of a corresponding population (possibly but unlikely to be measured).e. with an infinite number of cases in it. 124 . Sampling distribution is the basis to estimate population parameters from sample statistics. X ) that would occur if all possible samples of a fixed size (n) were taken from the population..
Comparison of Three Distributions (Zhu) Sample Distribution (Statistic) Unit Size Mean Std Dev An individual n X Population Distribution (Parameter) An individual N Sampling Distribution A sample ∞ μ se( = μ σ s σ n ) 125 .
4 under “sampling error” where s2 is the variance of the variable in the sample Zhu: Generalized from the above formulas. 126 .How to Calculate Standard Error? For nominal variables: se = p(1 − p) ×z n For interval/ratio variables: 2 s se = ×z n which has been presented in Ch. we can conclude that the standard error of a sampling distribution is simply the variance of a sample divided by the squared root of the sample size plus adjustment for a given confidence level.
Ch 11. Hypothesis Testing Learning Objectives: What is statistical hypothesis (as compared with research question)? Why do we need to set up hypothesis (instead of research question)? How to test hypothesis (the 5-step procedure)? 127 .
statistical hypothesis is formal. exploratory.What Is a Statistical Hypothesis? Unlike research question (which is informal. and preliminary). and predictive. general. Zhu: a good hypothesis need to specify the nature of the relationship (correlated or causal) the direction of the relationship (positive or negative) the form of the relationship (linear or nonlinear) the strength of the relationship (strong or weak) 128 . specific. explanatory. Hypothesis is a tentative generalization about the relationship between two or more variables that predicts an outcome.
research lacks focus and clarity It eliminates trial-and-error research that is time consuming and wasteful It helps rule out intervening and confounding variables It allows for quantification of variables. without it. words that cannot be quantified cannot be included in a hypothesis 129 .Why Do We Need Hypothesis? Major benefits of setting up a hypothesis (instead of a research question): It provides direction for a study.
Any hypothesis that challenges existing knowledge should have a compelling reason.What Constitutes a Useful Hypothesis? It should be compatible with current knowledge in the area. functional) hypothesis. It should be stated concisely It should be testable (i. “if A = B and B = C.... see my previous criteria for a “good” hypothesis 130 .e.e. then A = C”. It should be logically consistent (i. or Aristotle’s syllogism).e. falsifiable) Zhu: the above prescribes a “useful” (i.
Research Hypothesis vs.e. Null hypothesis (denoted as Ho) forms a logical alternative to the research hypothesis (H1) under test Zhu: null hypothesis aims to prevent existing knowledge from easily challenged 131 . sampling error).. Null Hypothesis Null hypothesis (also called “hypothesis of no difference”) assets that the statistical differences or relationships under study are due to chance or random error (i.
2. 4.Procedure of Hypothesis Testing (Zhu) 1. Specify the research hypothesis and the corresponding null hypothesis Select the appropriate statistical test Select the minimally acceptable significance level Collect the required data and perform the chosen testing Make a decision on the acceptance (or rejection) of the research hypothesis based on the testing results 132 . 3. 5.
a significant difference between groups is the same as a significant relationship between the group variable and the outcome variable) 133 Relationship between two or more variables: .Step 1.. there are two types of research hypotheses: Difference between or among groups: H0 : X 1 = X 2 = ⋅ ⋅ ⋅ H1: X 1 ≠ X 2 ≠ ⋅ ⋅ ⋅ . H1: β > 0 Note that a difference and a relationship are mathematically equivalent (i.e. Specifying the Hypotheses (Zhu) Generally. or better yet. H1: X 1 〉 X 2 〉 ⋅ ⋅ ⋅ H0 : β = 0 H1: β ≠ 0 . or better yet.
Select Statistical Test (Zhu) Independent Variable (IV) Dependent Variable (DV) Nominal Ordinal Interval/Ratio Nominal Crosstabs + Chi-square Crosstabs + Chi-square t.Step 2.or F-Test Ordinal Crosstabs + Chi-square Spearman Correlation F-Test Interval/Ratio Crosstabs + Chi-square* Spearman Correlation* Correlation or Regression 134 *After recoding the IV into an ordinal scale .
Determine the Significance Level (Zhu) The significance level is the minimally acceptable probability of error for the hypothesis testing Significance level (denoted as α): predetermined by the researcher.e.05 .01 (an error of 1%).1%) Probability level (denoted as p): the actually obtained in the data analysis. commonly set at α = 0.01. when α = 0. confidence level = 99%..e.Step 3. or α = 0. and when α = 0. confidence level = 95%. when α = 0.001. which could be any value from 0 to 1 Confidence level: the opposite to the significance level (i.9%) 135 .05 (i.001 (an error of 0.. an error of 5% in rejecting H0). α = 0. confidence level = 99.
Chi-square analysis) t-test (for two groups) ANOVA (F-test. 6-9) Statistical analysis based on the choice made in Step 2: Difference between/among groups: Crosstabulation (i. Collect and Analyze the Data (Zhu) Data collection (see Ch.Step 4. for three or more groups) Relationship between variables: Correlation analysis (for non-directional relationships) Regression analysis (for directional or causal relationships) 136 ..e.
Step 5. we reject H0. we reject H1 (the research hypothesis) If p < α. the null hypothesis). consequently.e. fail to reject) H0 (i.e. we accept H1 137 . consequently... we accept (i. Make the Statistical Decision The decision is made by simply comparing the resulting p and the predetermined α: If p ≥ α.
025 (2.475 (47.50 (50%) Region of Rejection (= α/2) Region of retention Region of Rejection (= α/2) .475 (47.5%) .5%) .50 (50%) .05 (Two-tail) .5%) µ .Regions of Rejection for α<.5%) 138 .025 (2.
” is the reverse to Type I error.” Type I error is under the direct control of the researcher. either to accept or reject H0). Zhu: between the two. To reduce the error. Because it happens in the region of rejection (= α). Type I error is called as “alpha error. the researcher can simply set α closer to zero.Errors in Hypothesis Testing When we make a decision (i.. Type II error.e. Type I error is generally more serious and thus should be prevented as the first priority 139 . called as “beta error. Type II error: the acceptance of a false null hypothesis that should be rejected. we run the risk of committing one of the two types of errors: Type I error: the rejection of a true null hypothesis that should be accepted.
Possible Results in Testing an H0 Decision Made Reality Reject H0 Type I error (=α) Correct (=1-β =Power of Analysis) Accept H0 Correct (=1-α =Confidence Level) Type II error (=β) H0 is true H0 is false 140 .
Possible Results in Testing an H1 The difference /relationship in the sample is: Significant The difference/relationship in the population: doesn’t exist Type I error exists Correct Nonsignificant Correct Type II error 141 .
the minimal power of the test (to avoid β error) Power is commonly set to . the necessary sample size (to reject a false H0).80 142 .Power Analysis Power of a test refers to the probability of rejecting a null hypothesis when it is false Zhu: the relevant statement on pp. or 2. given i) the predetermined α level and ii) the possible size of observed difference/relationship: 1. 274-5? is wrong Zhu: power analysis helps the researcher to determine.
Basic Statistical Procedures Learning Objectives: What are the following statistical tests? crosstabulation analysis and Chi-square test t-test and ANOVA correlation and regression analysis? Why should we use any of the above tests (instead of all others)? How can we perform these tests and interpret their results? 143 .Ch 12.
or authority. Statistics is how we advance our knowledge of everything. tenacity. In order to obtain valid and reliable results.Why Do We Need Statistical Tests? Statistical tests are necessary with the scientific method of knowing. Otherwise. 144 . which generate results that cannot be verified. any data must be analyzed using some type of statistical method. the data will be analyzed based on the methods of intuition.
What Statistical Test Should I Use? It depends on the measurement level of your variables Independent Variable (IV) Dependent Variable (DV) Nominal Ordinal Interval/Ratio Nominal CrosstabsChi-square CrosstabsChi-square t.or F-Test Ordinal CrosstabsChi-square Spearman Correlation F-Test Interval/Ratio CrosstabsChi-square* Spearman Correlation* Correlation or Regression 145 *After recoding the IV into an ordinal scale .
Where to Find the Relevant Tool in SPSS? Statistical Test Chi-square test t-test One-way ANOVA Multi-way ANOVA Spearman & Pearson correlation Simple & multiple regression SPSS Procedure Analyze/Descriptive/Crosstabs Analyze/Compare Means/Paired Samples Tests Analyze/Compare Means/One-Way ANOVA General Linear Model/Univariate Correlate/Bivariate Regression/Linear 146 .
or the relationship between a nominal/ordinal IV and a nominal/ordinal DV Chi-square (χ2) test provides the significance test of the null hypothesis underlying the crosstabulation that the observed difference doesn’t exist in the population. either of the following: the difference in a nominal/ordinal DV between/among groups of a nominal/ordinal IV. in a 2-way table format. or the observed relationship doesn’t exist in the population 147 .Crosstabs Analysis & Chi-square Test Crosstabulation analysis displays.
p < .Construction of a 2-Way Crosstable (Zhu) IV DV 1 2 Total N 1 ?% ?% 100% ? 2 ?% ?% 100% ? 3 ?% ?% 100% ? Total (optional) ?% ?% 100% ? 148 χ2 = ?. df = 2.? .
dividing the number of cases in each of the cells by the total number of cases in the corresponding column) Show column totals (100% and the number of cases) at the bottom Optionally. and p level) below the table 149 . degrees of free. show “sample percent” in the last column Show Chi-square test results (χ2 value.Rules for Cross-tables (Zhu) Put the IV in columns. with each row for each group of DV Show “column percent” in each cell (i.e.. with each column for each group of IV Put the DV in rows.
.05). reject the H0 and consequently accept the H1 Remember that the H0 states either of the following: There is no difference in the DV between/among the groups of the IV in the population There is no relationship between the IV and DV in the population 150 . .Interpretation of Chi-square Test Results (Zhu) When the resulting p-level is equal to or greater than the predetermined α-level (e. retain the H0 and consequently reject the H1 When p is smaller than α.g.
therefore. ANOVA can be used practically for comparisons of any number of groups 151 . based on F-test) is used when the IV involves three or more groups t-test is a special case of ANOVA because the t-statistic is the squared root of the corresponding F-statistic.t-Test and ANOVA (Zhu) Both tests examine the difference in an interval/ratio DV between/among groups of a nominal/ordinal IV: t-test is used when the IV involves two groups ANOVA (Analysis of Variance.
? 1 ? ? ? ? 2 ? ? ? ? ? -? -152 . p < . df = ?.Construction of a t-Test Table (Zhu) IV Difference DV Mean Std Dev Std Error (optional) N t = ?.
show the difference in the mean of the DV between the two groups in the last column Show the results of the t-test (including the obtained tvalue. with each column for each group of the IV Put the DV in rows. with one row for mean. standard deviation. respectively Optionally. and number of cases of each group. and p-level) below the table When constrained by the space. t-test table(s) can be replaced by a brief report of the above content in the text. standard error (optional because it can be calculated based on the other three pieces of the information presented). parallel tables can be combined into a multi-panel table 153 .Rules for t-Test Tables (Zhu) Put the IV in columns. df.
there is no difference in the mean of DV between the groups of the IV in the population) and consequently reject the H1 When p is smaller than α. reject the H0 and consequently accept the H1 (i. retain the H0 (i.Interpretation of t-Test Results (Zhu) When the resulting p-level is equal to or greater than the predetermined α-level (e. ...05).e.e.. there is a significant difference in the DV between the groups of the IV in the population) 154 .g.
Construction of an ANOVA Table (Zhu) IV DV Mean Std Dev Std Error (optional) N 1 ? ? ? ? 2 ? ? ? ? 3 ? ? ? ? 155 f = ?. df1 = ?. df2 = 2. p < ? .
** p < .05.An Alternative Table for ANOVA Results: Multi-group Comparisons (Zhu) Group 1 Mean Std N Group 2 Group 3 ? ? ? Difference* Difference** Difference*** 156 Group 2 ? ? ? Group 3 ? ? ? * p < .01.001 . *** p < .
g..e.e. there is a significant difference in the DV between at least one pair of the groups of the IV in the population) Post hoc tests: when the overall test is significant (p < α).05). 157 . a post hoc test is needed to compare each pair of the groups to identify exactly which pair is significantly different.. The logic of t-test between two groups applies here.. retain the H0 (i. reject the H0 and consequently accept the H1 (i.Interpretation of ANOVA Results (Zhu) Overall test: When p ≥ α (e. . there is no difference in the mean of DV among all groups of the IV in the population) and consequently reject the H1 When p < α.
Two-way ANOVA Data Table* IV 1 IV 2 Value=1 Value=2 Total Value=1 Value=2 Value=3 Total x11 x21 x•1 x12 x22 x•2 x13 x23 x•3 x1• x2• x•• 158 This and the following slides on 2-way ANOVA are optional. .
Two-way ANOVA Result Table Source IV 1 IV 2 Interaction Error Total Sum of Squares ? ? ? ? ? df ? ? ? ? ? 159 Mean Square ? ? ? ? F ? ? ? p ? ? ? .
if p < α.e. the effect of IV 1 on DV varies according to the value of IV 2.e.e... IV 2 has a significant effect on the DV) H0 for the interaction between IV 1 and IV 2. if p < α. if p < α. or vise vice. IV 1 has a significant effect on the DV) H0 for IV 2.Interpretation of 2-way ANOVA Results (Zhu) Three H0’s are tested: H0 for IV 1. reject the H0 for IV 2 (i.. reject the H0 (i. reject the H0 for IV 1 (i. the effect of IV 2 on DV varies according to the value of IV 1 160 .
Illustration of Interaction Effect No Interaction DV DV Significant Interaction IV 2 = 2 IV 2 = 2 IV 2 = 1 1 2 IV 1 1 IV 2 = 1 2 IV 1 161 .
measured in both standardized unit and nonstandardized unit (i. for ordinal variables) Pearson’s correlation (r. measured in a standardized unit (from 0 to 1): Spearman’s correlation (ρ. the original unit of the DV): Simple regression (for a DV and an IV) Multiple regression (for a DV and 2+ IVs) 162 .Correlation and Regression Analysis Correlation: the degree of association of 2 or more variables without assuming the causal direction between them..e. for interval/ratio variables) Regression: the degree of association of a DV with 1 or more IVs.
5) X Weak (probably nonsig.1) X Imperfect Negative (r=-0.Possible Correlational Relationships Y Y Y Y Perfectly Positive Correlation (r=1) X Perfectly Zero Correlation (r=0) X Perfectly Negative Correlation (r=-1) X Y Y Imperfect Positive (r=0.5) X 163 .) (r=0.
the ρ or r between X and Y is beyond chance and does hold in the population) If ρ or r > 0. the correlation is medium if the absolute value of ρ or r > .e. there is a negative correlation between X and Y (i.e.Interpretation of Correlation Results (Zhu) Significance of the relationship: If p ≥ α.e. there is a positive correlation between X and Y (i.7. the two vary in an opposition direction) If the absolute value ρ or r < ....3. the correlation is strong 164 Direction of the relationship: Strength of the relationship: . retain H0 (i.3 but < . the two vary in the same direction) If ρ or r < 0.7..e. reject H0 (i. the correlation is weak if the absolute value ρ or r > . the observed ρ or r between X and Y is merely by chance and doesn’t exist in the population) If p > α.
** p < . *** p < .Construction of Correlation Matrix Table X1 X2 X3 X1 1.00 ?*** (n = ?) X3 1.01.001 .00 ?* (n = ?) ?** (n = ?) X2 1.05.00 165 * p < .
Partial Correlation (optional) Partial correlation is a multivariate analysis that examines the net bivariate correlation between X and Y by controlling the impact of “third variable” (e.. Z. W.) on Y. Zhu: the order of partial correlation is determined by the number of “third variables” involved: rxy (bivariate correlation between X and Y) is called “zeroorder correlation coefficient” rxy|z (partial correlation between X and Y with Z controlled) is called “first-order correlation coefficient” rxy|zw (partial correlation between X and Y with Z and W controlled) is called “second-order correlation coefficient” rxy|zw… (partial correlation between X and Y with k variables controlled) is called “k-order correlation coefficient” 166 . etc.g.
a X 167 Y = a + bX b . with one (X) as the IV and another Y as the DV The relationship between X and Y can be expressed by the equation: Y Y = a + bX where a is the intercept of the regression line and b is the slope of the regression line.Simple Regression Analysis A simple regression analysis builds on an assumed causal relationship between two variables.
Construction of Simple Regression Table b Beta t p IV ? ? ? ? Constant ? ? ? ? N=? 168 .
Interpretation of Simple Regression Results (Zhu)
Regression analysis tests the H0 that the effect of X on Y, called “regression coefficient of X” is null (i.e., b = 0) If the resulting p (for b) < α, we reject H0 and conclude that X has a significant effect on Y More specifically, we can predict that a unit increase in X will lead to a change with the amount of b in Y in the population, of course within a confidence interval However, the above (i.e., rejection of H0) still doesn’t prove the causal direction from X to Y; it only suggests the impact of X on Y if the causal assumption is correct.
Multiple Regression Analysis (optional)
Multiple regression analysis involves two or more IVs: Y = a + b1X1 + b2X2 + … + bkXk The test involves:
an H0 for the overall regression equation (i.e., R2 = 0, where R2 is called “coefficient of determination”) multiple H0’s, each corresponding to a particular IV (i.e., b1 = b2 = … = bk = 0)
Table of a Multiple Regression
b IV 1 IV 2 IV 3 Constant ? ? ? ? Beta ? ? ? ? t ? ? ? ? p ? ? ? ?
Adjust R2 = ?, N = ?, (F = ?, df1 = ?, df2 = ?,) p < ? * p < .05, ** p < .01, *** p < .001
R2 N p .Table of Several Multiple Regressions DV1 IV 1 IV 2 IV 3 Constant ? ? ? ? ? ? ? DV2 ? ? ? ? ? ? ? DV3 ? ? ? ? ? ? ? 172 Adj.
we reject the associated H0 to conclude that the corresponding IV has a significant effect on the DV The value of the particular b describes the amount of change in the DV caused by a unit of increase in the IV. when all other IVs are held constant 173 .Interpretation of Multiple Regression Results (optional) If the p for the overall equation < α. we reject the overall H0 to conclude that the IVs jointly have a significant effect on the DV The value of R2 describes the amount of variance in the DV explained by the IVs together If the p for a particular b (ranging form b1 to bk) < α.
optional) Nominal/ordinal scale variables as IVs: recoded by a series of dummy (i.e..Advanced Techniques for Multiple Regression (Zhu. binary) variables Interaction between two IVs: when both are interval/ratio variables when one IV is interval/ratio and another nominal/ordinal variable Nonlinear relationship: Nonlinear transformation of IVs 174 .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.