3 views

Uploaded by Đinh Đức Tâm

powerpoint slide_introduction to meta-analysis

- 2 Interest in Dream Interpretation: A Gender Difference
- Influence of Deliberate Practice on Musical Achievement
- Formula Sheet
- Cawley Keeping Levy 1998
- jurnal
- Notes on Jackson's Research Methods and Statistics 3rd edition Text
- 10.1111@j.1750-8606.2008.00060.x.pdf
- metaanaliza moderatori
- quien esta fumando, tomando y usando drogas
- stres and immune jurnal.pdf
- Correlation Between Adherence Rates Measured by MEMS and SRQ
- Davis - Effects of Second Responder Programs on Repeat Incidents of Family Abuse - CSR
- Panduan INSINAS 2018 Final
- Kvalitet Starih Mexiko (1)
- SUMMARY OF TRAVEL TRENDS_NHTS 2009.pdf
- Stat 115 - 3 Estimation
- EffectSizeBecker.pdf
- Job fair
- conclusionsonspssresults
- Association Between Demographic and Diagnostic Factors and Employment Outcomes for People with Psychiatric Disabilities.pdf

You are on page 1of 12

Introduction to Meta-Analysis

Data Analysis II

Associate Professor

Michigan State University

Coefficients) and Transformations

1 r

ES Zr .5 ln

1 r

Correlation

coefficient Reliability of Measure1 Reliability of Measure2 Sample Size ES Zr

0.150 0.86 0.9 62 0.151

-0.028 999 999 213 -0.028

0.120 0.75 0.9 115 0.121

0.500 999 999 213 0.549

0.350 999 999 156 0.365

0.240 999 999 156 0.245

1

Apr-17

Adjustments

(for Measurement Unreliability)

Measure1 Measure2 Sample Size ES Zr Zr

0.86 0.9 62 0.151 0.172

999 999 213 -0.028 -0.028

0.75 0.9 115 0.121 0.147

999 999 213 0.549 0.549

999 999 156 0.365 0.365

999 999 156 0.245 0.245

(for Correlation Coefficients)

Reliability

adjusted ES Zr n-3 se w w*ES N Sum w*ES Sum w Average ES

0.109 59 0.130 59 6.42 977 235.83 956 0.2467

0.172 59 0.130 59 10.14

-0.028 210 0.069 210 -5.88

0.147 112 0.094 112 16.44

0.549 210 0.069 210 115.35

0.365 153 0.081 153 55.91

0.245 153 0.081 153 37.45

1

se

n3

w n 1

ES

(w ES)

w

Source: Lipsey and Wilson (2001), Practical Meta-Analysis by Sage

2

Apr-17

Reliability

adjusted ES Zr n-3 se w w*ES N Sum w*ES Sum w Average ES se of ES

0.172 59 0.130 59 10.14

-0.028 210 0.069 210 -5.88

0.147 112 0.094 112 16.44

0.549 210 0.069 210 115.35

0.365 153 0.081 153 55.91

0.245 153 0.081 153 37.45

1 1

seES 0.032

w 956

by the sum of the weights.

and Confidence Intervals

95% CI

Average z-test for the 95% CI Lower Upper

ES se of ES Mean ES bound bound

ES 0.2467

Z 7.63

seES 0.03231

95% Confidence Interval

Lower ES 1.96( seES ) 0.247 1.96(.032) 0.183

Upper ES 1.96( seES ) 0.247 1.96(.032) 0.31

3

Apr-17

Q-value Total Q

1.1213 39.8062

0.3309

15.8457 The homogeneity test is

1.1182 based on the Q statistic,

19.2317 which is distributed as a chi-

2.1579 square with k 1 degrees of

0.0006

freedom where k is the

number of effect sizes

(Hedges & Olkin, 1985)

Homogeneity Analysis

that all of the effect sizes are estimating the same

population mean is a reasonable assumption.

If homogeneity is rejected, the distribution of effect

sizes is assumed to be heterogeneous.

Single mean ES not a good descriptor of the distribution

There are real between study differences, that is, studies

estimate different population mean effect sizes.

Two options:

model between study differences

fit a random effects model

4

Apr-17

Heterogeneous Distributions

categorical variables with the analog to the one-way ANOVA

continuous variables and/or multiple variables with weighted

multiple regression

Assume variability is random and fit a random effects

model.

The Analog to the one-way ANOVA

(for Mean Differences)

1 1 -0.33 11.91 -3.93 1.30

2 1 0.32 28.57 9.14 2.93

sums for each

3 1 0.39 58.82 22.94 8.95 subgroup of effect

4 1 0.31 29.41 9.12 2.83

5 1 0.17 13.89 2.36 0.40

sizes.

6 1 0.64 8.55 5.47 3.50

151.15 45.10 19.90

8 2 0.15 10.75 1.61 0.24

9 2 -0.02 83.33 -1.67 0.03

10 2 0.00 14.93 0.00 0.00

118.82 -3.29 1.34

5

Apr-17

The Analog to the ANOVA

(for Mean Differences)

45.10 2

QGROUP _ 1 19.90 6.44

151.15

3.29 2

QGROUP _ 2 1.34 1.25

118.82

The Analog to the ANOVA

(for Mean Differences)

The sum of the individual group Qs = Q within:

and j is the number of groups.

is the Q between:

QB QT QW 14.76 7.69 7.07

6

Apr-17

The Analog to the ANOVA

(for Mean Differences)

within groups Q and a between groups Q.

QW 7.07 dfW 8 QCV _ .05 (8) 15.51 pW .05

QT 14.76 dfT 9 QCV _ .05 (9) 16.92 pT .05

in effect sizes.

(for Mean Differences)

can be calculated for each group:

ESGROUP _1

(w ES ) 45.10 0.30

w 151.15

ESGROUP _ 2

( w ES ) 3.29 0.03

w 118.82

7

Apr-17

(for Correlation Coefficients)

- Same as before: Calculate the mean ES, standard error

and confidence intervals for each group.

- Compare confidence intervals to see if there is overlap

- Calculate Q-statistics for each group

- Remember, these are multiple comparisons and

bivariate assessments

- See Joshi and Roh (2009) Tables 1, 2, and 3

Multiple Regression Analysis

categorical between studies variable.

What if you are interested in a continuous

variable or multiple between study variables?

Multiple Regression Analysis

can use canned programs (e.g., SPSS, SAS)

parameter estimates are correct (R-squared, B weights, etc.)

F-tests, t-tests, and associated probabilities are incorrect

can use Wilson/Lipsey SPSS macros which give correct

parameters and probability values

8

Apr-17

Bivariate vs. OLS vs. WLS vs. HLM

(see Sagie and Koslowsky 1993 Personnel Psych)

Both methods analyze the effects of moderators by

performing multiple regression analysis where:

Observed effects sizes is the dependent variable and

the level of all moderators for each study are

independent variables.

Z = 0 + 1 X1 + 2 X2 + 3 X3 + + I .

where, Z is the z-transformed value of the sample size weighted

correlation between multinationality and performance, s are

parameter estimates, and Xi are categorical variables (dummy-coded)

and continuous variables each X.

Three reasons to use a random effects model

Total Q is significant and you assume that the excess variability

across effect sizes derives from random differences across

studies (sources you cannot identify or measure).

The Q within from an Analog to the ANOVA is significant.

The Q residual from a Weighted Multiple Regression analysis is

significant.

18

Source: Lipsey and Wilson (2001), Practical Meta-Analysis by Sage

9

Apr-17

Fixed effects model assumes that all of the variability

between effect sizes is due to sampling error.

(If differences between studies that lead to differences in effects are

not regarded as random (i.e., if they are regarded as consequences

of purposeful design decisions), then fixed effects methods are

appropriate for analysis).

Random effects model assumes that the variability

between effect sizes is due to sampling error plus

variability in the population of effects.

(the differences in effects are regarded as consequences of a

process that cannot be predicted in advance (i.e., the sources of

influences on the outcome are both numerous and unidentifiable)

The true effect sizes under study are sampled from a

larger population of effect sizes.

Fixed Effect Results

confidence intervals.

Confidence intervals will get bigger.

Effects that were significant under a fixed effect model may no

longer be significant.

Random effects models are therefore more conservative.

10

Apr-17

Dummies for

Moderators

for Level 2

11

Apr-17

Stage

findings in a table (i.e., Descriptive Statistics Table)

Search for moderators using bivariate analysis and multivariate

analysis of effect sizes.

Use random effects models unless you have a strong reason to

believe that fixed effects models are more appropriate for your

analyses.

Despite its limitations, try to analyze the meta-analytical correlation

matrix for theory testing purposes, when appropriate.

Discussion

12

- 2 Interest in Dream Interpretation: A Gender DifferenceUploaded byDaysilirion
- Influence of Deliberate Practice on Musical AchievementUploaded byTal Strauss
- Formula SheetUploaded byYuki Fong
- Cawley Keeping Levy 1998Uploaded byNguyen Hoang Minh Quoc
- jurnalUploaded byNadhira Puspita Ayuningtyas
- Notes on Jackson's Research Methods and Statistics 3rd edition TextUploaded bypereirdp
- 10.1111@j.1750-8606.2008.00060.x.pdfUploaded byJorge
- metaanaliza moderatoriUploaded byCorina Ica
- quien esta fumando, tomando y usando drogasUploaded bydexpira
- stres and immune jurnal.pdfUploaded byChristina Davis
- Correlation Between Adherence Rates Measured by MEMS and SRQUploaded bycgmoraes
- Davis - Effects of Second Responder Programs on Repeat Incidents of Family Abuse - CSRUploaded byAnny Serena
- Panduan INSINAS 2018 FinalUploaded bypandibio87
- Kvalitet Starih Mexiko (1)Uploaded byAmar Sodilo
- SUMMARY OF TRAVEL TRENDS_NHTS 2009.pdfUploaded byslp
- Stat 115 - 3 EstimationUploaded byJude Patrick Sabaybay
- EffectSizeBecker.pdfUploaded byHerlina Mulyastuti
- Job fairUploaded byAyu Wdya
- conclusionsonspssresultsUploaded byapi-301889134
- Association Between Demographic and Diagnostic Factors and Employment Outcomes for People with Psychiatric Disabilities.pdfUploaded bynithinkondapuram
- 1992 Shimoda the Infuence of Dissolved CO2 ConcentrationUploaded bySharif M Mizanur Rahman
- Business Statistic Lab write-upUploaded byBamboo Chen
- Omega-3 fatty acids in high-riskUploaded byseck13
- New Text ciDocumentUploaded bymohammed saad
- ajwed orgnlUploaded byniaz43
- Prosthesis replacement in Mason III radial head fractures. A metaanalysis.pdfUploaded byGino Nanetti
- newresconst_Uploaded byqtipx
- CrtUploaded byEli Wati
- 25371Uploaded byAlle Sam
- Peppermint Oil for the Treatment of Irritable Bowel Syndrome_ a Systematic Review and Meta-AnalysisUploaded byAnonymous vOJH2hLMh6

- Olympic WordformUploaded byĐinh Đức Tâm
- Kirca Et Al 2012b GSJUploaded byĐinh Đức Tâm
- Rubera and Kirca 2012 JMUploaded byĐinh Đức Tâm
- Aguinis et al. 2010 ORM.pdfUploaded byĐinh Đức Tâm
- Phrasal Verb in EnglishUploaded bymdl
- Geyskens Et Al. 2009 JoMUploaded byĐinh Đức Tâm
- Rosenthal 1995 PsychBUploaded byĐinh Đức Tâm
- 4How to Analyze Meta-Analytic Data Stage IUploaded byĐinh Đức Tâm
- 6Best PracticesUploaded byĐinh Đức Tâm
- 1Definition and Role of Meta-AnalysisUploaded byĐinh Đức Tâm
- 3Collecting Data for a Meta-AnalysisUploaded byĐinh Đức Tâm
- 2Specifying the Problem and Retrieving StudiesUploaded byĐinh Đức Tâm
- Jean Sibelius - Koncert za violinu u D-molu (solo partitura)Uploaded byHarel Levi
- Williams, John - Piano and Violin Score - Schindler's List -.pdfUploaded byĐinh Đức Tâm
- GMAT Awa 109 SampleUploaded byĐinh Đức Tâm
- Chap005- The Global EnvironmentUploaded byĐinh Đức Tâm
- Chap004- Chap004-The External Environment [Compatibility Mode]Uploaded byĐinh Đức Tâm
- Chap003- Corporate Social Responsibility and Business EthicsUploaded byĐinh Đức Tâm
- Chap002 Company MissionUploaded byĐinh Đức Tâm
- Chap006 Internal AnalysisUploaded byĐinh Đức Tâm
- Pachelbel Canon in D Violin-solo VersionUploaded byalien_san

- Jorion Frm Exam AnswersUploaded byrahulloni
- Lecture01 Estimation Theory HandoutsUploaded byredcard53
- econ2209_5248tute8Uploaded byRichard Leicheston
- 5logreg1_02Uploaded byAmsalu Walelign
- Allocation Rules for Clinical TrialsUploaded byعلي علي
- Hacking - Telepathy - Origins of Randomization in Experimental DesignUploaded bybfosbrook
- 1692-4229-1-PB.pdfUploaded byHerlinia Yulistia Avionic
- Ap09 Frq Statistics FormbUploaded byMichael Powell
- Regression Using RUploaded bySanjay Mishra
- HBR_Store24B_CaseUploaded bymanojben
- Notes 15 Logistic Regression Confidence IntervalsUploaded byMn pk
- Logistic Regression Mini TabUploaded byAnıl Toraman
- The ANOVA ProcessUploaded bygarrettherr
- Great Presentation Demand Forecasting -SpssUploaded byRead Lead
- Qnt 351 Final Exam Correct Answers 100%Uploaded byMilan5456754
- Using SAS for Econometrics, Fourth Edition - Griffiths, William E.Uploaded byAamir Baig
- Multiple RegressionUploaded bymthmstr
- Asymptotic Statistics - Lecture NotesUploaded byAnonymous gUySMcpSq
- Sample Past Paper CVEN2002Uploaded byShean Hee
- Stats - IntroUploaded byamalia
- Activity 7Uploaded bySameer Satyam Pathak
- Effect Size Statistics in Logistic RegressionUploaded byPrem Nath
- Lab 2 217 SolutionUploaded byieltstutor
- EViews 6 Users Guide IIUploaded byRicardo Consonni
- ANOvaUploaded byنور روسلن
- Forecasting-with-R-notes.pdfUploaded byKarthik
- St 205_lecturer NotesUploaded byPeter Ng'orogita
- OUTPUT 5 2nd SubmissionUploaded byJoshua Arcaira
- Bayesian ModelingUploaded byrishit_93
- RcmdrPlugin.hhUploaded byzaki_b6