Meta-Analysis in Environmental Science

By Swarna Weerasinghe
Keywords: step-by-step approach to meta-analysis, PECO approach for environmental studies, systematic
coding of meta-data, sensitivity and influence analysis, fixed effects and random effects models

Abstract: Meta-analysis is an analytical method of synthesizing, systematically combining,

and integrating results across different studies with similar research focuses. While limiting
the analytical methods applicable to environmental studies, this article presents a step-
by-step approach to meta-analysis. Brief descriptions presented in each section of this
article cover the initial steps of meta-analysis such as formulation of a research question,
development of a literature search strategy, and assessment of quality of the published
literature by reviewing possible biases. Next steps provide essentials of data extraction,
methods of meta-data analysis as well as interpretation of results. An attempt is made
to provide hypothetical examples to illustrate each meta-analytical step. The goal of this
article is to provide an overview of the fundamentals of meta-analysis of environmental
studies.

Introduction
Meta-analysis is a method of pooling results from different studies that minimizes biases and errors of
each individual study [1–4] . The goal of this information mining approach is to acquire summary findings
to answer a research question that has been answered using different study samples from different
populations. Most importantly, meta-analytical findings help to iron out controversies. For example,
there are controversies Impact assessment: Assessing a Local Biological Effect With Before and After
Dataaround the effects of environmental exposure to ozone on upper respiratory health. Some studies
showed a detrimental effect while other studies point to a protective effect. Meta-analysis can serve a
variety of purposes starting from developing, refining, and testing hypotheses through identifying data
gaps to providing directions for future research [4] . Final summary measures of meta-analysis are more
reliable than what is provided by each individual study. Hence, meta-analytical findings have become
an essential component of program planning and policy making environments. The discussion of this

Dalhousie University, Halifax, Nova Scotia, Canada

Based in part on the article “Meta-analysis” by R. L. Tweedie, which appeared in the Encyclopedia of Environmetrics first edition.
c John Wiley & Sons, Ltd and republished
This article was originally published online in 2006 in Encyclopedia of Environmetrics,
in Wiley StatsRef: Statistics Reference Online, 2014.

c 2013 John Wiley & Sons, Ltd. All rights reserved.

Copyright 1
 Meta-Analysis in Environmental Science

article is limited to the applications in the area of physical environments and synthesis of quantitative
data.
By systematically combining research findings from different studies, meta-analysis yields several
advantages. In particular, meta-findings minimize individual study bias and reduce issues arising from
small sample size and type two error [4] . In addition, if the type of meta-analysis allows pooling across
populations with different demographic identities, generalizability of such meta-analytical results to a
demographically broader population is warranted. The systematic review guidelines in meta-analysis
ensure that the literature review is objective oriented [5] . Thus meta-analysis should be undertaken
with great caution. For instance, summarizing information coming from different methods such as
regression coefficients coming from different regression models can lead to findings that are misleading
and impossible to interpret. For example, the effect of ozone on asthma hospital admissions when
unadjusted for temperature is different from that when adjusted for the same since the latter takes into
consideration only the irregular variation in ozone. Biases in meta-analytical findings are due to the
tendency of underreporting insignificant results and inclusion of multiple results coming from the same
study [4] . The following step-by-step approach of meta-analysis covers some of the cautionary measures
that can be undertaken to minimize such biases.

Step-by-Step Approach to Meta-Analysis

Step 1: Formulation of a Research Question

Meta analysis, as a quantitative approach, is object oriented and starts with formulation of a research
question. The research question should include four attributes designated by PECO, the population (P)
of interest, clearly defined exposure (E), crucial confounders or controllers (C), and an outcome measure
(O). This is an alternative to the PICO [6] framework proposed in evidence-based medicine literature.
The last attribute of outcome (O) is not necessary if the impact of environmental exposure is not the
intention of meta-analysis. For example, a meta-analysis to answer a research question of “Is there an
association between ozone exposure and asthma exacerbation among adults below 40 years of age, after
controlling for meteorological factors?” includes studies that have a study population of 0- to 40-year-olds.
Exposure of interest should be ambient ozone concentrations (for example, daily means or three-day
accumulated exposure) and the outcome of interest can vary from asthma-related general practitioner
visits, emergency room visits, and hospital admissions to mortality. If the intention is to understand Trend
Analysis time trends and seasonal patterns of environmental exposure, it is not necessary to include an
outcome measure. However, it is important to consider confounding of environmental exposures with
other variables such as temperature and humidity. Conflicting studies are recommended to be included,
with a brief description of whether or not and to what extent the studies confirm or refute the hypotheses
of study interest [7] .

Step 2: Development of a Literature Search Strategy

The PECO approach described above would allow us to develop search terms, a string of words, which
includes keywords and phrases that combine the population, exposure, and outcome (if applicable) of
interest, all of which are tailored toward a specific meta-analysis. For example, search terms “ozone and
asthma” in the title provided 832 articles, of which only 27 contained ambient ozone exposure. A compre-
hensive list of relevant databases should be identified. After reviewing selected articles, their references
point to other relevant articles from their references [8] . An explicit search strategy should include clear
2 c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright
Meta-Analysis in Environmental Science

inclusion (eligibility) and exclusion criteria of literature. The inclusion criteria emphasis should be on
consistency, quality, and availability of findings [3] . Nevertheless, Analysis of Variance (ANOVA) statistical
significance or power should not be considered as an important feature to be considered for inclusion.
For example, for the above research question, inclusion of only the school-going population of 5- to
19-year-olds will avoid occupational exposures. Careful attention should be paid to weaknesses [7] and
limitations that are mentioned in the articles [7] . Given that electronic literature searches are facilitated
in most educational institutions, literature searches should not be limited to full papers that are only
available electronically. Unpublished studies (after assessing the quality) and published reports should
be included to avoid publication bias [3] .

Step 3: Quality Assessment

Study selection bias can be minimized by having multiple reviewers assess the quality of papers. In
addition, inclusion of an independent adjudicator is recommended to iron out any disagreements among
reviewers. Another measure that can be undertaken to minimize study selection bias in meta-analysis
is to blind the reviewer about the authors, the type of publication, and the journal of publication. When
assessing the quality of studies for meta-analysis, careful examination of the following biases is also
recommended.
In environmental studies, systematic differences that exist in ascertainment of exposed and unexposed
subjects introduce subject selection biases [8] . For instance, in a study of exposure to environmental
tobacco smoking during pregnancy and birth weight, inclusion of smoking mothers in the exposed group
introduces a subject selection bias. Similarly, collecting adverse outcomes from the exposed subjects can
be different from that of the unexposed subjects and this is known as performance bias [8] . Given that
precise instruments capture low birth weights accurately, use of those precise instruments to measure
birth weights of babies born to exposed mothers introduces performance bias. One of the most important
biases in meta-analysis is the publication bias that arises from inclination to publish positive results.
Although inclusion of unpublished reports and non-peer-reviewed articles and reports reduces publication
bias, thorough scrutiny is recommended because of lack of rigor in articles that are not peer reviewed.
Published literature that ignores the bias due to nonresponses and loss to follow-up introduce a drawback
known as attrition bias [8] . There are visual presentations such as funnel plots that help to detect publication
biases. Owing to space limitation, authors tend to underreport information on confounders and it is
recommended to contact the authors to obtain additional information on the factors that are controlled
in the models [3] .
More rigorous meta-analytical studies assess the quality of each study using sensitivity and influence
analysis. Influence analysis would detect the changes in a combined or meta-estimate of a study, by
comparing the changes of the combined estimate when a study is excluded and included.

Sensitivity and Influence Analysis

In environmental studies, length of the exposure is an important factor. For example, some studies
tend to have shorter exposure periods. By carrying out sensitivity analyses [3] of positive and negative
outcomes classified by shorter and longer exposures, the impact of the length of exposure on outcomes
can be detected. Studies with redundant information should be excluded, while keeping the most up-
to-date information for further analyses. The following example shows a hypothetical case of sensitivity
analysis on the significance of length of lead exposure and children’s IQ. Selected studies are cross-
classified according to the duration of exposure as 1–5 years and greater than five years of exposure and
its significance on children’s IQ.
c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright 3
 Meta-Analysis in Environmental Science

Figure 1. Influence analyses plot (hypothetical example).

A simple chi-square test can be used to test the hypothesis that statistical significance of environmental
exposure does not depend on the duration of exposure. If the chi-square test leads to statistical significance,
then the study results are sensitive to the length of exposure. Hence, we are unable to combine studies
with different lengths of exposure.
Similarly, influence analysis (also noted as sensitivity analysis in the literature) will identify certain
studies (which are identified as outliers or exceptional cases) that are influential on the pooled estimate.
Firstly, the studies that provide controversial results that can be considered as outliers (having confidence
intervals (CIs) further away from other studies) in the forest plot (Figure 1) have to be identified. Influence
analysis will identify the sensitivity of these outliers on the pooled estimate and its standard error. A
simple method of sensitivity analysis of influential studies, identified as outliers, is to calculate the pooled
estimate with outliers and then with each outlier removed, one at a time. This method was applied in a
radon exposure and lung cancer study to understand the influence of each study on the pooled estimate
[9]
. This could be an exhaustive process to do with each study and, therefore, it is important to identify
only the studies that are providing exceptional results. Once a pooled estimate and the standard error
are calculated, 95% CI for each round (of influential study removal) is plotted (Figure 1). A significant
departure with a particular study (study 4 for example) removed from the overall estimate would suggest
a greater influence. Other sophisticated methods of influence analysis are described elsewhere [10] .

Step 4: Systematic Coding of Study Information

After selecting a set of published studies (from steps 1–3 above), the next two important steps include data
coding and analysis. An example of systematic coding of studies included in a meta-analysis is shown
4 c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright
Meta-Analysis in Environmental Science

Table 1. Exposure and Study Significance Classification (Oij Indicates Number of Studies)

Number of studies 1–5 years of exposure More than 5 years of exposure

Significant effect O 11 O 12
Nonsignificant effect O 21 O 22

Table 2. Template for Data Coding: a Sample to Examine Effect of Ozone on Asthma

Study (ID) Location Duration (or Design Exposure (E) Confounder Outcome (O)
sample size) Population(P) (C)

(1) Steib et al. Montreal, 1990–2000 Time series All age CO, NO2 , O3 , Temporal Asthma and
2009 [11] Ottawa, groups SO2 , PM10, cycles, CVD ED visits
Edmonton, St and PM2.5 weather, day
John, Halifax, of the week
Toronto,
Canada
(2) Edmonton, 1992–2002 Time series Two age CO, NO2 , O3 , Temperature ED visits for
Szyszkowicz, Canada groups below SO2 , PM10, relative asthma
2008 [12] and above and PM2.5 humidity
10-year olds
(3) Steib et al. Saint John, 1984–1992 Time series All age SO2 , NO2 , Weather ED visits for
1996 [13] New groups SO4 , and TSP asthma
Brunswick, metropolitan
Canada area
a Note: ED, emergency department.

in Table 1. The first task of systematic coding is to tabulate the study characteristics including PECO
information in a meaningful manner (Table 1). Inclusion of an ID number for each study would help
further data coding.
Additional information, such as sample size, type of main outcome (relative risk (RR), odds ratio (OR),
or correlation), and significance of study findings, that provide a comprehensive overview of study results
can be included in a separate data coding sheet.

Step 5: Information Extraction and Data Coding

These tabulated data disclose the original information used for the meta-analysis. Further classification
of significance across demographic groups (for example, male and female) and carrying out a chi-square
test provide directions for subgroup stratification for meta-analysis [1] .
Data extraction is done using a data extraction protocol. Such an objective-oriented data extraction
protocol should be designed to facilitate consistency and repetition of meta-analysis with new data
coming from new publications [3] . Data extraction protocol [3] should include the following:

r criteria for judgement of completeness of the study data;

r compatibility assessment with other selected studies across different exposures, outcome measures,
and route of exposure;
r quality assessment of information to be extracted from the selected studies including sample size,
power, and study designs;
r different outcome types whether human or ecological and the populations of exposure;
r criteria for judgment of quality of information;
r citations of the data sources (for example, study ID as in Table 2).
c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright 5
 Meta-Analysis in Environmental Science

One can develop a meta-database by coding data from each study. By recording data in a spread sheet
format using the study id (assigned in Table 2) across subgroups (age and gender for example) with the
same ID repeated will allow detailed meta-analysis. The data coding sheet should include the following:

r type and scale of the summary data on effects and exposure matrices;
r frequency, duration, and intensity of exposure [3] ;
r sample size, number exposed and unexposed (if applicable), and time period (for time series studies);
r descriptive summary measures of the outcomes and exposures including means and standard errors
(or CIs) for the entire study period as well as for each subgroups;
r study limitations that are identified by the authors;
r different outcomes measures, for example, hospital admissions, emergency room visits, or mortality,
noted in different studies have to be coded separately.

Most of the publications include effects in the form of regression coefficients, RRs, or ORs along with
their standard errors (or CIs) and associated P values. In some studies, because of space limitations, level
of significance is marked with stars and, therefore, P values are missing. This can be misleading since a
P value of 0.045 can be considered significant at a level of significance of 0.05, whereas a P value of 0.051
is considered nonsignificant. There is no meaningful difference between the two P values of 0.045 and
0.051 to reach a meaningful conclusion.

Step 6: Data Analysis

This is the most important step in meta-analysis. Meta-data analysis is presented in two parts: exploratory
data analysis that includes visual display of meta-data and estimation of the pooled estimate and its
standard error. The objective of meta-analysis is to obtain a pooled estimate and to make inference about
this estimate. Two of the visual display methods of meta-data obtained from selected studies are forest
plots and funnel plots.

Visual Display of Meta-Data

In a forest plot, CIs for the outcome (for example, 95% CI for the RR of emergency room visit) extracted
from each study are drawn against the outcome or effect measure (for example, RR). In a forest plot, the
pooled estimate is also plotted and typically a diamond symbol is used to denote it. The pooled estimates
(of all study-combined effect) can be calculated using the DerSimonian and Laird approach, which is
explained below. It is critical to convert estimates and CI for comparability across studies. In general, the
regression coefficient (of an environmental exposure variable) provides excess risk per unit increase. This
has to be converted into a reasonable scale, for example, 20 ppb increase of ozone for daily averages, 30 for
8-h maximum, and 40 for 1-h maximum are recommended [14] . If the studies are organized according to
the increasing order of the effect (for example, RR or OR), then one can observe a trend [8, 14] and identify
influential studies to carry out sensitivity analyses as outlined earlier in this article. Here is a hypothetical
example of a forest plot of the studies ordered according to the increasing order of RR estimate.
The pooled estimate is marked with a diamond in Figure 2. Methods of obtaining the pooled estimates
are discussed later in this article. Some authors have added exposure information as a column of mean
values and standard errors of exposure measures of each study to the plot [8] . This additional information
allows one to visually explore the dose–response relationship of exposure and outcome.
Another meta-data exploration graphical display, a funnel plot, is used to assess publication bias. A
funnel plot is generated by plotting the effect estimate for each study against its precision (reciprocal of
6 c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright
Meta-Analysis in Environmental Science

Figure 2. Forest plot (hypothetical example).

standard error) or standard error to understand biases and validity in meta-data collected from different
studies [3, 8] . Given that the standard error decreases as the sample size increases, it is believed that larger
studies provide more precision than smaller studies. Some authors plot effect size against the sample
size. Hence, small studies tend to scatter at the beginning of a plot where there is less precision. However,
there are no rules as to whether the precision (or sample size or standard error) goes on the x-axis or on
the y-axis. Hence, the shape of the funnel could be upside down or turned sideways. An example of a
funnel plot is shown in Figure 3.
If there is no publication bias, a symmetry is expected around the horizontal line that represents the
pooled estimate line (dotted line in Figure 3). A regression method of detecting asymmetry is suggested
since conventionally symmetry is visually examined [15] . The standard normal deviate (SND) defined as
the effect size divided by the standard error is used in the regression equation SND = " + $x + g, where
x = precision. The test for null hypothesis of $ = 0 is equivalent to the test of symmetry. If the test is
insignificant, then there is publication bias. In addition, the slope $ indicates the size and the direction
of the bias [15] .
c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright 7
 Meta-Analysis in Environmental Science

Figure 3. Sample funnel plot.

Pooled Estimate Calculation

The four different types of effects (denoted by 2 below) that are frequently used in environmental exposure
data analyses are risk difference (RD), RR, OR, and the beta coefficient of a regression of adverse outcome
against environmental exposure(s). In addition, the regression coefficient (2i , shown in the regression
model below) and its standard error are often used in environmental data meta-analysis. Generalized
linear models (GLM) and time series regression models are two of the regression type models used
in environmental data analysis. The coefficient of the model across studies has to be converted into a
reasonable scale and this was discussed earlier in this article. In addition, it is important to pay attention
to exposure lag time. For example, combining studies with the same day exposure with over 2–3 days
exposure would underestimate the multiday effect [14] .
The three risk estimates mostly used in environmental risk assessments are RD, RR, and OR.
Let us define the risk estimate for the ith study as 2i ,Pe denotes the probability of outcome (adverse)
for the exposed community, and Pne denotes the probability of the same outcome for the unexposed
community. Then 2i takes the values of RD, RR, or OR defined in equations (1, 2, 3).
RD = Pe − Pne (1)

RR = Pe /Pne (2)

Pe /(1 − Pe )
OR = (3)
Pne /(1 − Pne )
8 c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright
Meta-Analysis in Environmental Science

These effect estimates are limited to single variable estimates and are of limited use. In general,
multiple regression models with exposure effect adjusted for other confounding variables are used. Two
of such models are generalized linear models and time series regression models given in equations (4)
and (5), respectively.
g (y) = " + 2i X 1 + $1 X 2 + · · · + $k X k + g (4)
·
where g is the random error term, g is the mapping rule (i.e., transformation, log, logit, g = identity for
g(y) = y outcome measure for multiple regression), X1 is the exposure, and X2 ,. . .,Xk are the k different
confounders.
g (yt ) = " + (Yt−1 + 2i X 1 + $1 X 2
(5)
+ . . . $k X k + gt + *gt−1 ,
where ( and * are autoregressive and moving average parameters. In all of the estimates above, environ-
∧
mental exposure effect is denoted by 2i and its estimate is denoted by 2i . Let us denote the standard error
∧ ∧
of 2i by var (2i ) = Si2 . A general method of combining evidence (estimates and standard errors) from
∧
different studies is called inverse variance weighted method [16] . Let the inverse of the variance, 1/var(2i ), be
∧
denoted by wi for the ith study. Then the pooled estimate 2 is calculated as in (6).
P ∧
∧ 2i wi
2= P (6)
wi
∧
where 2i is the estimate of 2i for the ith study, where the sum is calculated over all studies. The variance
∧ P
of 2 is given by 1/ wi . It is important to test whether the variances across studies are homogeneous.
Here, we test the null hypothesis that the pooled variance 12 = 0. The test statistics for the test is given
by equation (7) as weighted sums of squares of all of the studies,
X ∧ ∧
X2 = wi (2i − 2 )2 (7)

which follows a P 2 distribution with degrees of freedom given by the number of studies minus 1.

Methods When the Variance Is Homogeneous

The methods that assume homogeneity are Mantel–Haenszel and Peto methods [16] . In general, a fixed
∧
effect model for the ith study 2i = 2 + gi assumes 2i ∼ N(2,F 2 ), where the errors g i ∼ N(0,F 2 ). However, if
the null hypothesis of homogeneity is rejected, random effect and mixed effect models (described below)
are used for the analysis. If enough data are available across subgroups (for example, age and gender
groups), carrying out homogeneity tests in each subgroup (across studies) would allow one to identify
sources of heterogeneity.
Mantel–Haenszel method combines ORs from case–control or perspective studies [17] . For example, if
Ai ,Bi ,Ci , and Di denote the cell counts as shown in Table 3, then OR for the ith study (ORi ) is denoted by
ORi = Ai Di /Bi Ci . Let m be the number of studies for which we are combining results.

Outcome Positive Negative

Exposed Ai Bi

c 2013 John Wiley & Sons, Ltd. All rights reserved.

Copyright 9
 Meta-Analysis in Environmental Science

Outcome Positive Negative

Not exposed Ci Di

Then the pooled estimate of OR given by Mantel–Haenszel [17] is given by (8).

Pm
Ai Di /Ni
R = Pi=1
m (8)
i=1 Bi C i /Ni

where Ni is the sample size of the ith study. The variance of the pooled OR can be obtained by exponen-
tiating the variance formula in (9) as given by var(ln(R )) [18] .

m m
X Pi Ri X Pi Si + Qi Ri
var (ln(R )) = 2
+
i=1
2R+ i=1
2R+ S+
m
(9)
X Qi Si
+
i=1
2S+2

Pm Pm
where Ri = Ai Di /Ni ,Si = Bi Ci /Ni ,Pi = (Ai + Di )/Ni ,Qi = (Bi + Ci )/Ni and R+ = i=1 Ri and S+ = i=1 Si
for m studies.
In such studies where neither frequencies nor estimates, but the P values, are reported, a method of
combining P values is needed. The formula given in equation (10) is used to combine P values from
different studies [19] .
m
X
P P2 = −2 ln(Pi ) (10)
i=1

This test statistic in equation (10) follows a chi-square distribution with 2m degrees of freedom, where
m is the number of studies. In other words, the combined P value of all studies is given by probability
2
(P2m > P p2 ), where the probability is the chi-square distribution with 2m degrees of freedom that exceeds
the value of the test statistic P P2 as calculated. If the P value is reported as Pi < 0.05 (for example), then
(Pi − 0.05) can be used in place of Pi , but the result is conservative [16] .

Methods When the Variance Is Heterogeneous (Random Effects Models)

These models do not take the homogeneity among studies into consideration. Herein, we make the
assumption that the sample of studies came from a population that follows a distribution with mean
∧
: and standard deviation 1. The estimate 2 that is discussed above is an estimate of :. Moreover, the
chi-square test of homogeneity described above is the same test for the test of null hypothesis of variance
12 = 0. The P 2 test statistic for homogeneity is applicable to different effect estimates such as RDs
and ORs expressed in equations (1, 2, 3). When the variances across studies are unequal (i.e., when the
∧
homogeneity test fails) then the unweighted estimate of 2 is recommended [20] . The alternative formula
to equation (6) is given by replacing the weights wi by

wi = 1/(Si2 + 12w ) (11)

10 c 2013 John Wiley & Sons, Ltd. All rights reserved.
Copyright
Meta-Analysis in Environmental Science

where
 
Pm ∧ ∧
 (
i=1 2i − 2) 
12w = max 0, − S2 (12)
 m−1 

and
m
X Si2
S2 = (13)
i=1
m
∧
Herein Si denotes the standard error of ith study estimate of 2i . In general, the random effects model
can be written as a two-stage model [16] .
∧
A random effects model for the ith study 2i = : i + gi assumes g i ∼ N(0,F 2 ), where : i = 2 + l i and
l i ∼ N(0,J 2 ). In the random effects model, the ith group true effect : i is distributed around the population
∧
mean 2 with the between subgroup variance J 2 , and the observed study effect 2i is distributed around
within sub group mean : i and the variance, F 2 .
In a mixed model presentation, : i represents the ith study group (where subgroup homogeneity
occurred) mean and the associated within study variance is F 2 . Therein 2 represents the overall true effect
and J 2 represents the between study variance. This model can be fitted using mixed model analytical
methods.
These models can be fitted using standard statistical analytical software such as SPSS, SAS, and STATA.
There is specialized software such as REVMAN, which would do exploratory meta-analyses (forest plots,
funnel plots, and homogeneity tests) as well as fitting fixed effect and random effects models.

Presentation of Results
This article presented a six-step process of meta-analysis. Introduction of a meta-analysis article should
be the same as any other article, wherein the background, rational, and the need of the study are
described. Methods section includes the literature search strategy with a list of databases, study inclusion
and exclusion criteria (Step 3), and a listing of selected studies as presented in Table 2. In order to
ensure credibility of study findings, exploratory meta-data presentations of forest plot and funnel plot are
recommended. Inclusion of effect estimate and CI beside the vertical axis of the forest plot would provide
more information on meta-data structures. Results of homogeneous testing need to be included. Finally,
effect estimates and fixed, random effect, or mixed model pooled estimate results should be presented
along with CI estimates. It is paramount to summarize limitations and biases of the meta-analysis
findings.

Related Articles
Resource Selection

References
[1] Hunter, J.E. & Schmidt, F.L. (1990). Methods of Meta-Analysis: Correcting Error and Bias in Research Study Findings, Sage
Publication, Newbury Park.
[2] Greenland, S. (1987). Quantitative methods in the review of epidemiologic literature, Epidemiology Review 9, 1–29.

c 2013 John Wiley & Sons, Ltd. All rights reserved.

Copyright 11
 Meta-Analysis in Environmental Science

[3] Blair, A., Burg, J., Forman, J., Gibb, H., Green, S., Morris, R., Raabe, G., Savitz, D., Teta, J., Wartenberg, D., Wong,
O., & Zimmerman, R. (1995). Guidelines for application of meta analysis in environmental epidemiology, Regulatory
Toxicology and Pharmacology 22, 189–197.
[4] Lu, Y., Chen, L., & Fu, B. (2008). Land-cover effects on red soil rehabilitation in China: a meta-analysis, Progress in Physical
Geography 32(5), 491–502.
[5] Vassanadumrongdee, S., Matsuoka, S., & Shirakawa, H. (2004). Meta-analysis of contingent valuation studies on air
pollution-related morbidity risks, Environmental Economics and Policy Studies 6, 11–47.
[6] Sackett, D.L., Richardson, W.S., Rosenberg, W., & Haynes, R.B. (1997). Evidence-Based Medicine: How to Practice and
Teach EBM, Churchill Livingston, New York.
[7] Lee, P.N. (1993). Correspondence: Environmental tobacco smoke and lung cancer risk in non-smoking women, Journal
of the National Cancer Institute 85(9), 748.
[8] Salmasi, G., Grady, R., Jones, J., & McDonald, S. (2010). Environmental tobacco smoke exposure and perinatal outcomes:
A systematic review and meta-analyses, Acta Obstetricia et Gynecologica 89, 423–441.
[9] Lubin, J.H. & Boice, J.D. (1997). Lung cancer risk from residential radon: meta analyses of eight epidemiological studies,
Journal of the National Cancer Institute 89(1), 49–57.
[10] Chatterjee, S. & Hadi, A.S. (1966). Influential observations, high leverage points, and outliers in regression, Statistical
Science 1(3), 379–416.
[11] Stieb, D.M., Szyszkowicz, M., Rowe, B.H., & Leech, J.A. (2009). Air pollution and emergency department visits for cardiac
and respiratory conditions: a multi-city time-series analysis, Environmental Health 10, 8–25.
[12] Szyszkowicz, M. (2008). Ambient air pollution and daily emergency department visits for asthma in Edmonton, Canada,
International Journal of Occupation Medicine and Environmental Health 21(1), 25–30.
[13] Stieb, D.M., Burnett, R.T., Beveridge, R.C., & Brook, J.R. (1996). Association between ozone and asthma emergency
department visits in Saint John, New Brunswick, Canada, Environmental Health Perspective 104(12), 1354–1360.
[14] Ito, K., Leon, S.F.D., & Lippmann, M. (2005). Association between ozone and daily mortality – analysis and meta-analysis,
Epidemiology 16, 446–457.
[15] Egger, M., Smith, G.D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test,
British Medical Journal 315(7109), 629–634.
[16] Hasselblad, V. (1995). Meta-analysis of environmental health data, The Science of the Total Environment 160–161,
545–558.
[17] Mantel, N. & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease,
Journal of the National Cancer Institute 22, 719–748.
[18] Robin, J., Breslow, N., & Greenland, S. (1986). Estimation of the Mantel–Haensze; variance consistent in both sparse
data and large-data limiting models, Biometrics 42, 311–323.
[19] Fisher, R.A. (1932). Statistical Methods for Research Workers, 4th Edition, Oliver and Boyd, Edinburgh.
[20] DerSimonian, R. & Laird, N. (1986). Meta analysis in clinical trials, Contemporary Clinical Trials 7, 177–188.

12 c 2013 John Wiley & Sons, Ltd. All rights reserved.

Copyright