You are on page 1of 23

Risk Analysis DOI: 10.1111/risa.

12372

Macroeconomics of Natural Disasters: Strengths and


Weaknesses of Meta-Analysis Versus Review of Literature

Peter A. G. van Bergeijk1,∗ and Sara Lazzaroni2

We use the case of the macroeconomic impact of natural disasters to analyze strengths and
weaknesses of meta-analysis in an emerging research field. Macroeconomists have published
on this issue since 2002 (we identified 60 studies to date). The results of the studies are contra-
dictory and therefore the need to synthesize the available research is evident. Meta-analysis
is a useful method in this field. An important aim of our article is to show how one can use
the identified methodological characteristics to better understand the robustness and impor-
tance of new findings. To provide a comparative perspective, we contrast our meta-analysis
and its findings with the major influential research synthesis in the field: the IPCC’s 2012 spe-
cial report Managing the Risks of Extreme Events and Disasters to Advance Climate Change
Adaptation. We show that the IPCC could have been more confident about the negative eco-
nomic impact of disasters and more transparent on inclusion and qualification of studies, if
it had been complemented by a meta-analysis. Our meta-analysis shows that, controlling for
modeling strategies and data set, the impact of disasters is significantly negative. The evidence
is strongest for direct costs studies where we see no difference between our larger sample and
the studies included in the IPCC report. Direct cost studies and indirect cost studies differ
significantly, both in terms of the confidence that can be attached to a negative impact of
natural disasters and in terms of the sources of heterogeneity of the findings reported in the
primary studies.

KEY WORDS: IPCC; macroeconomics; meta-analysis

1. INTRODUCTION higher frequency and intensity of natural disasters


and, consequently, the empirical literature on the
The macroeconomic analysis of natural disasters
macroeconomic impact of natural disasters has
is a fairly recent branch of economic research.(1,2)
grown substantially.(4) Our systematic review helps
For a long time, this topic was mainly investigated
to identify two substrands in the literature. The first
by the technical sciences and other disciplines of
set of studies focuses on disaster direct costs, that is
social sciences.(3) Whenever economics was in-
economic damages and number of people reported
volved, the analysis was typically microeconomic
affected and/or killed when a natural hazard occurs.
and/or case specific. This changed, however and
The second set of studies concentrates on disaster
macroeconomists became involved due to the
indirect (and secondary) costs quantified in terms of
the effects on GDP/income. Note that this distinction
1 ISS Erasmus University, Kortenaerkade 12, Hague 2581 AX, The
is important because the two strands of literature
Netherlands.
2 Department deal with costs that originate at different moments in
of Economics, University of Bologna, Piazza Scar-
avilli 2, Bologna, 40126, Italy. time: upon natural hazards’ occurrence in the case of
∗ Address correspondence to Peter van Bergeijk, ISS Erasmus direct costs and after the disaster has occurred in the
University, Kortenaerkade 12, Hague 2581 AX, The Nether- case of indirect costs. In both sets of empirical stud-
lands; tel: +31704260517; bergeijk@iss.nl. ies, the disaster effect of interest can be summarized

1 0272-4332/15/0100-0001$22.00/1 
C 2015 Society for Risk Analysis
2 van Bergeijk and Lazzaroni

by the estimated t-value. The t-value captures the economic evidence on the effects of natural disasters
sign (positive or negative) of the estimated disaster using a traditional review of the literature. The IPCC
effect and its level of significance (the probability to report is a selective survey of the literature that em-
reject the hypothesis that the estimated effect is zero phasizes what the authors of the survey deem reliable
when it is in fact true). For example, if the t-value of and important. The objectivity of the survey is not
the disaster effect estimated in regression i in study j based on full inclusivity of all available literature, but
is lower than −1.65 (two-tailed t-value threshold for on a rigorous review and peer-review process that is
10% level of significance) it means that, once other a hallmark of the IPCC process.5 In this article, we
factors have been accounted for, the authors found a want to compare the merits of the two methodolo-
negative effect of disasters and can be 90% confident gies and their strengths and weaknesses in research
in rejecting the hypothesis that the estimated disaster synthesis.6 We use our own work (meta-analysis) be-
effect is nil. cause this is the earliest meta-analysis in the field7
Fig. 1 reports the median and average t-value and we provide a critical self-assessment. The IPCC
of 60 macroeconometric studies (our “primary stud- report is a worthy benchmark because substantial in-
ies”) that were identified in a systematic review as tellectual and human resources were available for
part of a meta-analysis.(5) 3 Fig. 1 reports the t-values this report, thus any issue that occurs with the IPCC
for the association between disasters and macroeco- work can be expected to emerge in reviews that are
nomic impact distinguishing studies on disaster direct done with less rigor and resources.
and indirect costs. One of the key messages of Fig. 1 As in the meta-analysis, the IPCC report ac-
is that the findings of the available econometric stud- knowledges the disagreement in the literature. IPCC
ies tend to disagree regarding this fundamental rela- assigns only “medium confidence”8 to the prevalence
tionship given the large divergence for both sign and of a negative economic impact. Indeed, one of the
statistical significance of the impact of natural disas- key issues in this emerging literature is about the sign
ters. For disaster direct costs studies, the average t- of the disaster impact and the confidence that we can
value in our sample is −5.9 with a standard deviation have in this sign (its significance). The usual approach
of 8.2. For disaster indirect costs studies, the average in meta-analysis for a field that is more mature would
t-value in our sample is −1.4 with a standard devia- be to estimate the actual average size effect (the
tion of 5.4. meta-effect), but the studies available until 2013 do
The aims of this article are threefold. First, to not allow estimating the size effect for two reasons.
assess if the two substrands have to be considered First, in the primary studies disaster direct costs are
separately, or as a unique body of literature on dis- measured both in monetary and nonmonetary terms
aster macroeconomic costs. Second, to uncover the
methodological sources for disagreement about the
human-induced climate change, the potential impacts, and the
effects of natural disasters. Third, to draw lessons
adaptation and mitigation options.”(6: vii) It is coordinated by
from the heterogeneity of the findings of the primary Working Groups I and II, respectively dealing with physical sci-
studies. We will do so by contrasting two methodolo- ence basis of climate change and climate change impacts, adap-
gies: a meta-analysis and an authoritative selective tation, and vulnerability.
5 We thank an anonymous referee for this point.
review of literature.
6 Presently our data set comprises 60 primary studies until De-
Our research is motivated by the simultaneous
cember 2013 for a total of 1,688 parameter estimates. Because
availability of our meta-analysis and the important the IPCC report covers only studies until May 2011, we would
International Panel on Climate Change (IPCC) spe- like to point out that different results in the meta-analysis could
cial report Managing the Risks of Extreme Events be driven by more recent studies that logically were not included
and Disasters to Advance Climate Change Adaptation in the report.
7 Our working paper has been available online since April 2013.
(henceforth “the IPCC report”).(6) In 2012, Working
While being in the peer-review process for this journal another
Groups I and II of the IPCC4 jointly produced this meta-analysis on indirect costs studies has become available on-
report aiming, inter alia, at synthesizing the available line as a corrected proof for publication.(7)
8 In the IPCC report the degree of certainty in key findings
3 Upon the advice of the referees we have updated and extended is based on authors’ evaluation of scientific knowledge using
this study that included 22 studies only. the qualitative concept of confidence. Confidence can be low,
4 The IPCC was created in 1988 with the general mandate “to medium, or high, depending on the authors’ evaluation of the
assess in a comprehensive, objective, and transparent manner level (low, medium, high) of agreement and type, amount, and
all the relevant scientific, technical, and socioeconomic informa- consistency (limited, medium, robust) of the evidence included
tion to contribute in understanding the scientific basis of risk of in the literature reviewed.
Meta-Analysis Versus Review of Literature 3

Fig. 1. Empirical studies on the macroeconomic impact of natural disasters. [section symbol] Extreme minimum t-values were cut. *Studies
analyzing both disaster direct and indirect cost. [degree symbol] Studies in IPCC (2012).
4 van Bergeijk and Lazzaroni

and therefore we need a dimensionless parameter the failure to account for the roles of the informal
in order to make useful comparisons. Second, the sector, insurance, foreign aid, and disaster type.(6: 265)
functional forms used in the primary studies dif- This is the second issue for which we want to make
fer and the data necessary to calculate elasticities a comparison: Can meta-analysis shed more light
are not always reported. We will therefore follow on the methodological drivers of the heterogeneity
a somewhat different approach and present the in reported results? Again, we compare our 2013
meta-sign and meta-confidence in order to analyze meta-analysis and findings with the IPCC report in
how sign and confidence issues are tackled in the order to discuss strengths and weaknesses of the two
two methodologies (narrative literature review vs. research synthesis methods. Importantly, disagree-
meta-analysis). Meta-sign and meta-confidence will ment according to the meta-analysis appears to be
then refer, respectively, to the synthesized cumula- related to methodological choices by the authors of
tive sign and level of significance of the estimated the primary studies (including time period, countries
disaster effect across studies and samples in the coverage, and econometric methods).
macroeconomic empirical literature. The remainder of the article is structured as fol-
We use sign and confidence because we are in- lows. Section 2 provides motivation for and expla-
terested in understanding if natural disasters have a nation of our meta-analysis, focusing on how the
negative/insignificant/positive effect at the macroe- methodology is presently being applied and the iden-
conomic level. Note that we are interested in confi- tification of the primary studies. Section 3 presents
dence (significance) rather than preciseness. To be the IPCC report as a benchmark for comparison of
clear, one may be confident about an imprecise es- meta-analysis results. Section 4 shows the results of
timate (large standard error) if it is sufficiently far meta-analysis and IPCC report. Section 5 compares
away from zero.9 Within this framework we are also the strengths and weaknesses of meta-analysis and
able to include primary studies that do not report narrative review of literature.
t-values and/or standard errors but do report coef-
ficients and levels of significance. Fig. 2 illustrates 2. META-ANALYSIS: MOTIVATION
that using the reported significance levels increases AND IDENTIFICATION
our sample by 6.4% (4.6%) for direct (indirect) costs
studies.10 As will become clear, a focus on meta-
2.1. Why Meta-Analysis?
sign and meta-confidence makes it possible to reach
firmer conclusions than the IPCC did.11 This article takes stock of the emerging liter-
The IPCC report relates the disagreement ature on the macroeconomics of natural disasters,
between the primary studies to limitations in the not only because of the relevance of the topic of the
analyses, such as the lack of a counterfactual or costs of natural disasters, but also and especially in
order to discuss the use of meta-analysis as a tool to
9 An estimate’s precision (calculated as the reciprocal of the stan- synthesize knowledge in an emerging and politically
dard error) depends on methodological choices in the primary sensitive field and, relatedly, to compare this to a
studies (empirical strategy and model specification) and is re- traditional review of the literature. This is important
flected in t-values through the regression coefficient because t- because of the apparent disagreement between the
values are calculated as the ratio between the coefficient and the
primary studies. Meta-analysis is a research method
standard error.
10 Excluding these observations reduces the share of significantly that enables to synthesize and summarize previously
negative parameters from 73.5% to 72.5% for direct cost studies obtained empirical findings for a research question
and increases the share of significantly negative parameters from in a quantitative and statistically rigorous fashion,
42% to 43.4% for indirect cost studies. to present the results of several studies in a coherent
11 This is not generally the case. Often, the sign of a relationship is
framework, and ultimately to estimate a meta-effect
unambiguously estimated in a literature. An example is Disdier
and Head,(8) who investigate the size of the distance effect in the across studies.(9,10) In our meta-analysis, the goal
gravity model of international trade. The sign in these models is is not to uncover the “average” or meta-effect that
always negative (larger distance reduces trade), but the interest- could be derived from the contradicting studies
ing question is how this distance effect develops over time and (compare DeCicca and Kenkel(11) ), but instead
therefore the meta size effect needs to be estimated. Our use of
to find out the meta “sign and significance.” In
t-values can only be relevant when a literature is characterized
by disagreement about the sign. Even then we would prefer to our opinion, a modest goal is more appropriate in
work with a meta-size effect, but as argued in the text that is not this emerging field where we have a great many
(yet) a good option in our case. definitions of disaster impact and strength, many
Meta-Analysis Versus Review of Literature 5

Fig. 2. T-values and reported levels of signifi-


cance for studies on disaster direct (1) and in-
direct (2) costs.

functional forms, different (sub) populations of the lows us to identify which methodological char-
data, and different estimation methods. Our second acteristics are associated with finding a specific
goal is to disentangle the impact of methodological result (in our case we have three results:
choices and research strategies on the results re- [a] significantly negative, [b] insignificant, and
ported by the primary studies. We use a somewhat [c] significantly positive).
broader approach to meta-regression analysis and 2. Preliminary state of findings: In the case of
provide descriptive statistics of several aspects of the the macroeconomics of natural disasters, the
literature so as to quantify and qualify results of the meta-analysis is extra useful because, as in
body of primary studies. The “bibliometric” aspects any emerging scientific field, many findings
of the literature are important because these help us are “preliminary” and often contradictory due
to identify differences in country income levels, time to the process of finding out the true effect.
periods, considered control variables, and also in the Fig. 1 illustrates this process: it plots the mean
frequency with which methodologies and statistical and median t-value per study, which are or-
techniques have been applied as well as the need dered by year of publication. A fluctuating pat-
to interact and extend sets of explanatory variables tern emerges because typically the scientific
covered by the literature. debate on new issues is characterized by inde-
From the start it is clear that the disagreement pendent replications (both based on the orig-
between the primary studies in our sample may be inal data set of the primary studies and on
caused by three main sources.12 alternative data sets and samples), updates
(when more data become available because
1. Methodological differences: Behind the dis-
new events occur), and innovations (of mod-
agreement may be heterogeneity in the data
els and tests) that all may result in alternative
sources, the samples (country coverage and re-
findings.14 Typically, many such contradictions
search period), the econometric specifications,
and the estimation procedures.13 In such a
context, a meta-analysis of the reported results 14 Aclear example of this process is the replication study made
can be used to shed light on the impact that by Reed and Mercer(12) of the study by Toya and Skidmore.(13)
the methodology exerts on the results reported Reed and Mercer update and extend the disaster data and per-
form a number of robustness checks to tackle truncation bias
in the primary studies. Meta-analysis thus al-
and skewedness of disaster data, introduce country and time
fixed effects, and cluster standard errors. Contrary to the original
12 Sampling error is expected to play a minor role in macroeco- study by Toya and Skidmore, Reed and Mercer find that eco-
nomics because “the sample may include the entire relevant nomic development variables (income, educational attainment,
population.”(8) size of the government, openness, and financial-sector develop-
13 Economists also refer to “structural heterogeneity” due to the ment) are generally speaking not significant. The only reportedly
selection of subpopulations and “method heterogeneity” due to significant effect is that natural disasters trigger larger economic
measurement and statistical methods.(8) losses for wealthier countries.
6 van Bergeijk and Lazzaroni

occur during the years in which a new scientific to estimate elasticities using only the regression re-
field emerges. Meta-analysis is a tool that can sults based on full samples, but there is a clear cost,
be used to distill an overall finding from the re- namely, the loss of additional information on the ef-
ported empirical results of the primary studies fect of different sample characteristics (size, decades,
as well as measures of statistical confidence. and countries included) on the results. Because one
3. Intrinsic motivation and bias of the re- of the goals of our 2013 meta-analysis is precisely to
searchers: The effect of natural disasters is an understand how sample design influences the results
important topic with substantial societal and in the primary studies, we prefer to avoid this im-
political relevance and therefore researchers portant cost. In the words of Disdier and Head:(8)
may be driven by idealistic or ideological moti- “different estimates often differ in terms of sample
vations to report those results that fit their con- period, method, etc., and therefore within-study vari-
ceptions of the importance of the problem and ation . . . can be used to assess the importance of such
the need to find solutions, as well as the kind of variables.”
instruments that should be used—and, impor-
tantly, may ignore results that contradict their
2.2. Meta Data Set
view of the world.15 A meta-analysis is more
objective and transparent than the traditional In the meta-analysis, we identified 60 macroe-
review of literature because it enables the re- conomic studies up to and including December
searcher to systematically study the sources of 2013 that empirically assess the effects of nat-
variation in the quantitative results reported in ural disasters in an extensive search. We used
the primary studies. Econlit and Google Scholar and broad keyword
listings with the following terminologies: “natural
As mentioned before, we do not analyze the co- disasters,” “impact,” “growth,” “economic develop-
efficients that have been estimated in the primary ment,” “development,” “killed,” “affected,” “insti-
studies but rather the sign and significance of those tutions,” and “econometric.”17 We considered only
coefficients. 16 First, measurement in the primary English-language studies, except for Moreno and
studies is not uniform: some studies focus on human Cardona,(20) written in Spanish and included in
capital damages in terms of people reported affected the IPCC report, so as to enable the inclusion in
and/or killed, whereas other studies concentrate on the meta-analysis of all suitable macroeconomet-
monetary damages in terms of economic damages ric studies considered in the IPCC report.(6) Four
as a share of GDP and/or and GDP/income varia- studies(21–24) could not be included in the meta-
tions triggered by natural disasters. Second, the func- analysis because key statistical characteristics were
tional forms used in the primary studies (log-log, log- not reported.18 Similarly, studies that did not include
linear, linear-log) often differ much between studies. a variable for disaster occurrence (disaster variable)
Elasticities could be calculated with simple formu- in the econometric analysis were automatically ex-
las and using sample means,(17) but this is not always cluded (e.g., Baker and Bloom,(25) Okuyama and
possible because the necessary key statistics are Sahin,(26) Toya and Skidmore(13) ). Finally, we were
sometimes not reported. Indeed, primary studies helped by referees of this journal and participants of
generally report descriptive statistics for the full conferences where we presented the meta-analysis as
sample considered, whereas within the same study they identified missing studies. All in all we were able
authors often present estimations on a variety of
subsamples (e.g., a subsample for OECD countries 17 Literature reviews were not included in our meta-analysis. Be-
or a subsample for earthquakes) for which specific cause we are interested in collecting t-statistics (significance
average characteristics are not provided. To over- levels) of the variables considered, empirical works using vec-
come this limitation it would have been possible tor autoregressive models, input-output, and computable gen-
eral equilibrium (CGE) analyses could not be included since
the former reported the impulse response functions only and
15 Compare Doucouliagos and Paldam(14) on development aid. not the short-and long-term coefficients (Fomby et al.(18) and
Similar issues are here at play because, for example, disaster vul- Cuñado and Ferreira(19) are an exception, duly included in the
nerability is an important factor that has been used to stop LDC meta-analysis) while the results of input-output and CGE analy-
graduation (see Fialho(15) ). ses by design do not provide the standard errors or t-values that
16 The cost of the procedure is of course that we are unable to ar- we need in our meta-analysis.
rive at the precise and exact parameter values that are often nec- 18 We contacted these authors by e-mail asking for additional in-

essary to populate economic models.(16) formation, but received no reply.


Meta-Analysis Versus Review of Literature 7

to collect 1,688 parameter estimates (significance lev- Appendix A). Therefore, our meta-analysis also in-
els) as well as the methodological characteristics of vestigates if these strands can be considered as a
individual regressions reported in the primary stud- unique field of research or if they are character-
ies. ized by significantly different findings (controlling for
The methodology of the primary studies shows other methodological characteristics).
many and obvious similarities, basically because all In our sample, 31 of the primary studies deal
studies in our sample deal with the macroeconomics with direct costs, 24 deal with indirect secondary
of disasters from multicountry and/or multievent per- costs, and 5 studies consider both direct and indirect
spectives. Note that in line with the IPCC report we costs.(27–31)
also include local studies (considering one country
only) as long as they follow a macroeconometric ap-
3. THE IPCC REPORT
proach. Behind these similarities important differ-
ences exist, however, as the empirical literature can The objective of the IPCC report is to provide
be subdivided into two substrands on the basis of the a comprehensive assessment of the literature on cli-
key variable of interest.(3) The first variable is the mate change (the usual domain of IPCC) with a
direct cost, typically quantified by disaster economic focus on disaster recovery, risk management, and
damages, number of people affected, and/or casu- mitigation up to May 2011. The IPCC report de-
alties. The second variable is the indirect cost that fines the “impact of climate extremes and disasters on
identifies the second-order economic effect of natu- economies, societies and ecosystems . . . as the dam-
ral disasters on GDP; this effect is indirect because it age costs and losses of economic assets or stocks, as
is based on the (estimated) impact of destruction of well as consequential indirect effects on economic
the capital stock, loss of labor force, and a reduction flows, such as on GDP or consumption”(6: 264) (italic
of actual working and production days. in the original). So as in our meta-analysis, a catego-
It is useful to distinguish these two approaches. rization into direct and indirect costs is made, explic-
The first approach (Model type 1) tries to under- itly pointing out that the categories are “rarely fully
stand how socioeconomic factors influence the ef- exclusive, and items or activities can have elements
fects of disasters focusing on the direct costs of dis- in all categories.”(6: 264) Disaster costs—the core re-
asters, whereas the second approach (Model type 2) search question analyzed in the meta-analysis—are
focuses on the indirect impact of natural disasters on discussed in Chapters 4 (Section 5), 5 (Section 5.2.1),
GDP (average income in local studies). Model type 1 6 (Section 1), and 8 (Section 2.1). Fig. 3 provides an
studies consider only periods in which disasters actu- overview of the studies in the IPCC report by type of
ally occurred, whereas Model type 2 studies compare analysis.
periods in which disasters took place with periods in Two main research questions are considered in
which countries did not experience natural disasters. the report. The first IPCC research question relates
Both model types use a specific “disaster variable” to the sign of the impact on economic development;
that accounts for the occurrence of the phenomenon. IPCC reaches the conclusion that it is negative, with
The disaster variable can be a dummy, a disaster fre- medium confidence. Table I lists the studies cited in
quency, intensity, and/or (especially in Model type the IPCC report finding a negative or positive im-
2 studies) a variable describing the number of peo- pact on growth in the short and long term. Com-
ple affected or killed or the damages reported in case bining Table I with the correspondent studies in the
the disaster occurred (direct costs). Hence, the vari- meta-analysis allows us to make two observations.
ables accounting for number of affected or killed and First, there is high prevalence of nonmacroecono-
damages are a disaster impact variable in Model type metric studies20 in the IPCC information set. Sec-
1 and a disaster indicator in Model type 2. The two ond, for the period covered by the IPCC (up to and
models use common sets of explanatory variables.
Direct and indirect impact studies are highly 20 Some macroeconometric studies are included in the non-
interrelated(2: 285) in the minds of both the produc- macroeconometric category due to their unsuitability for the
ers and users of the research findings19 and are meta-analysis: Hochrainer(32) uses as dependent variable the
difference between projected and observed GDP in five years
each on their own merits relevant for policy (see
post event instead of GDP level/growth, Raddatz(33) relies on
a VAR analysis, and Benson and Clay,(21) Jaramillo,(22) Simons
19 Cross-referencingbetween Model type 1 and Model type 2 stud- and Sutter(23,24) did not report the number of observations in the
ies and vice versa occurs regularly in this literature. regression results and were thus excluded.
8
Table I. Nature of the Impact of Natural Disasters on Economic Development (GDP/Income—Model Type 2)

Studies in the IPCC Report (Until May 2011) Studies in the Meta-Analysis (Until December 2013)a

Timeb Negative Positive Negative Positive

Short term (<5 years) Benson and Clay (1998,c Albala-Bertrand (1993)d Caselli and Malhotra (2004) Noy and Vu (2010)
2000,d 2003,d 2004d ) Skidmore and Toya (2002) Tavares (2004) Vu and Hammes (2010)
Kellenberg and Mobarak Caselli and Malhotra (2004) Heger et al. (2008) Cuñado and Ferreira (2011)
(2008) Hallegatte and Ghil (2007)d Noy (2009) Ahlerup (2013)
Cuny (1983)d Hsiang (2010)
Otero and Marti (1995)d Moreno and Cardona (2011)
Charveriat (2000)d Strobl (2011)
Crowards (2000)d Bergholt and Lujala (2012)
ECLAC (2003)d Strobl (2012)
Mechler (2004)d Von Peter et al. (2012)
Raddatz (2009)d Zylberberg (2012)
Noy (2009) Deryugina (2013)
Okuyama and Sahin (2009)d Felbermayr and Groschl
Cavallo and Noy (2010)d (2013a)
Fomby et al. (2013)
Ghimire and Ferreira (2013)
McDermott et al. (2013)

Mixed evidence
Murlidharan and Shah (2003)
Jackson (2013)

Noy and Noualsri (2007) Noy and Noualsri (2007) Ahlerup (2013)
Long tem (5 years) Hochrainer (2009)d Fisker (2012) Kim (2010)
Jaramillo (2009)c Deryugina (2013) Cuñado and Ferreira (2011)
Felbermayr and Groschl Skidmore and Toya (2002)
(2013b)
McDermott et al. (2013)

Mixed evidence Mixed evidence


World Bank and UN (2010)d Loayza et al. (2012)
for very severe disasters

Note: [ ] Is a Model type 1 study.


a In the table, we report meta-analysis studies according to the clear prevalence of significant negative/positive results.
b In the meta-analysis we also considered medium term (2–4 years); here we incorporate the medium-term studies in the short-term studies.
c Identifies a study that is in the IPCC information set but not in the meta-analysis due to missing information.
d Stands for reports, books, noneconometric studies, or econometric studies that could not be included in the analysis because using a different model type.

Source: Authors’ elaborations from IPCC (2012: 264–269, 344–345, 443–445), Sections 4.5, 6.1, and 8.2.1.
van Bergeijk and Lazzaroni
Meta-Analysis Versus Review of Literature 9

Fig. 3. Studies (76) on disaster impacts in the IPCC report


by type of analysis.
Source: Authors’ elaborations from IPCC (2012: 264–269,
317, 344–345, 443–445). *This category includes reports,
books, and studies with descriptive analyses.

including May 2011), we note that some readily avail- choices in the source of disaster data and construc-
able macroeconometric studies were not included in tion of the disaster data set (years and countries),
the report. model specification, and estimation technique. Be-
The second research question deals with the fore moving to the meta-regression analysis in the
nexus between development and disaster impact. next section, we first compute composite t-statistics
The IPCC report states with high confidence that dis- for some of the variables that we will use as mod-
aster economic losses are higher in developed coun- erator variables. By broadening the sample and us-
tries and that fatalities and economic losses as a ing the information contained in the samples and
share of GDP are higher in developing countries. subsamples of the primary studies, we can draw
Again we can note in Table II the prevalence of non- conclusions with respect to meta-sign and meta-
macroeconometic studies in the IPCC information significance.
set.21 Following previous meta-analyses(34–36) we cal-
Section 5.5.2 of the IPCC report cites studies that culate the composite t-statistic as:
focus on disaster impact at the local level (Table III).
ti
Nonmacroeconometric analyses prevail, but in this tC = √ ∼ N (0, 1) . (1)
case this is rightly motivated by the nature of the re- N
search question: the impact of natural disasters at the In light of the variability of the number of obser-
local level can be better analyzed with input-output vations in the selected studies, it is important to fol-
and CGE models. low Diebel and Wooster,(37) Djankov and Murrell,(38)
and Mebratie and van Bergeijk(36) and calculate a
weighted composite statistic:
4. META ANALYSIS
n
k=1 wktk
4.1. Descriptive Statistics TWC =  ∼ N (0, 1) , (2)
n
w
k=1 k
2

The variability in the level of significance of the


mean/median t-statistic across the selected studies is where wk is the weight assigned to the nth t-value in
very high (Fig. 1). This could be driven by different the meta data set calculated as the reciprocal of the
number of t-values from the same study, so that each
study is equally represented. This is only one alterna-
21 Tothe extent that the classification of countries in OECD/non-
tive to assess the robustness of the findings. Table IV
OECD categories mirrors the classification in devel-
oped/developing economies, the meta-analysis also tests
presents the composite and weighted t-statistics of all
these relationships using a wider set of studies and controlling robustness checks. Calculations were also conducted
for their methodological characteristics. excluding extremely high t-values (|t|>10) with a
10 van Bergeijk and Lazzaroni

Table II. Nexus Between Development and Disaster Vulnerability

Findings Studies in the IPCC Report (Until May 2011)

Fatality rates and economic losses as share Wildavski (1988)a Hallegatte et al. (2007)a
of GDP higher in developing countries Albala Bertrand (1993)a Parry et al. (2007)a
Burton et al. (1993)a Raddatz (2007)a,b
Otero and Marti (1995)a Toya and Skidmore (2007)a
Tol and Leek (1999)a World Bank (2007)a
Charveriat (2000)a Heger et al. (2008)
Crowards (2000)a Nicholls (2008)a
Murlidharan and Shah (2001) Raschky (2008)
World Bank (2000);a IPCC (2001)a Schumacher and Strobl (2008)c
ECLAC (2003)a Cavallo and Noy (2009)a
Del Ninno et al. (2003)a,b Hallegatte and Dumas (2009)a
Owens et al. (2003)a,b Hochrainer (2009)a
Skoufias (2003)a,b Ibàrraran et al. (2009)a
Benson and Clay (2004)a,b Loayza et al. (2009)
Mechler (2004)a Noy (2009)
Rasmussen (2004) Raddatz (2009)a
UNDP (2004)a Cardona et al. (2010)a,b
Brooks et al. (2005)a IFRC (2010)a,b
Kahn (2005) Lal (2010)a
McKenzie et al. (2006)a Rodriguez-Oreggia et al. (2010)a
O’Brien et al. (2006)a World Bank and UN (2010)a
Thomalla et al. (2006)a

Wealthier countries experience higher total UNDP (2004)a Cummins and Mahul (2009)a
economic and insured losses DFID (2005)a UNISDR (2009a
Birch and Wachter (2006)a Swiss Re (2010)a
O’Brien et al. (2006)a Cavallo and Noy (2010)a
Cutter and Finch (2008)a CRED (2010)a
Kellenberg and Mobarak (2008) Pelham et al. (2011)a

Fatalities decrease with rising income Kahn (2005) Nicholls et al. (2008)a
Bouwer et al. (2007)a World Bank and UN (2010)a
Toya and Skidmore (2007)a

a Stands for reports, books, noneconometric studies, or econometric studies that could not be included in the analysis because using a
different model type.
b Stands for studies that were cited to back a particular policy option.
c The study in the IPCC report is the working paper version whereas in the meta-analysis we reported the published version.

Source: Authors’ elaborations from IPCC (2012: 264–269, 344–345, 443–445), Sections 4.5, 6.1, and 8.2.1.

remaining number of included observations of 538 example, the weighted statistics are in most cases
t-values for Model type 1 and 875 t-values for Model lower than the unweighted ones. Similarly, all com-
type 2. Table IV reports composite t-values distin- posite t-statistics are significant even excluding out-
guished by model type, the use of the EM-DAT liers, but they decrease substantially when we use the
database, and the inclusion of OECD/non-OECD median, suggesting that outliers play some role in de-
countries. Moreover, we compare composite t-values termining the sample disaster outcome. Most impor-
for published and unpublished studies to consider tant, bivariate analysis suggests with high confidence
study quality and/or possible publication biases. Fi- that disasters have a negative and significant effect
nally, we present composite t-values for studies in- (all the composite t-values have negative sign and are
cluded/excluded by the IPCC report. significant at 1%). We can be more confident than
Table IV illustrates the need for more than IPCC both because we aggregate the evidence avail-
one weighting scheme and checking for outliers. For able in the primary studies considered by IPCC and
Meta-Analysis Versus Review of Literature 11

Table III. Studies Estimating the Cost of Managing Disasters at the Local Level

Studies in the IPCC report (Until May 2011) Studies in the Meta-Analysis (Until December 2013)

West and Lenze (1994)a Pielke and Downton (2000)


Rose et al. (1997)a Simmons and Sutter (2008)
Okuyama (2004)a Czajkowski and Kennedy (2010)
Haimes et al. (2005)a Nordhaus (2010)
Rose and Liao (2005)a Noy and Vu (2010)
Smith and McCarty (2006)a Vu and Hammes (2010)
Tsuchiya et al. (2007)a Czaikowski et al. (2011)
Hallegatte (2008)a Mendelsohn et al. (2011)
Strobl (2008)b Strobl (2011)b
Noy and Vu (2010) Deryugina (2013)
Rodriguez-Oreggia et al. (2010)a Anttila-Huges and Hsiang (2013)

a Stands for reports, books, noneconometric studies, or econometric studies that could not be included in the analysis because using a
different model type.
b The study in the IPCC report is the working paper version whereas in the meta-analysis we reported the published version.

Source: Authors’ elaborations from IPCC (2012: 317), Section 5.5.2.1.

because we cover more primary studies that deal with both model types.22 Table V provides descriptions of
the macroeconomic impact of disasters. the control variables included in the meta-analysis,
providing the number of observations (t-values that
could be derived and/or calculated from the primary
4.2. Meta-Regression studies) and their means and standard deviations.
Our meta-analysis studies significance levels (1/0; To compare macroeconometric studies in the meta-
significant or not) rather than continuous t-values to analysis and IPCC samples, we will use the dummy
allow the inclusion of studies that do not (always) re- variable “IPCC” identifying the overlapping studies.
port t-statistics or standard errors, but do report sig-
nificance levels (e.g., *10%, **5%, and ***1%). We 4.3. Results: Meta-Analysis Versus IPCC
assume that the estimated disaster effect yi∗j is a func-
tion of empirical design ED, estimation technique Because the cut-off date of the studies included
ET, and studies outreach SO (publication and IPCC in the IPCC report is May 2011, we initially compare
dummy): meta-analysis results for two samples including both
direct and indirect costs: the “full May 2011 sam-
yi∗j = θ0 + θ1 EDi j + θ2 ET i j + θ3 SOi j + εi j . (3) ple” of 34 macroeconometric studies available at the
cut-off date of the IPCC report and the 17 macroe-
We observe three categories of y: conometric studies actually included in the IPCC re-
port (the “IPCC selection”). The first line shows that
1. The reported t-value in regression i in study the probability of a significantly negative effect (i.e.,
j is negative and significant at least at 10% p[y = 1]) is 65% for the full May 2011 sample and
(t-value lower or equal to −1.65 or an explic- 53% for the IPCC selection. Overall, we find that em-
itly reported significance level in conjunction pirical design of the primary studies is highly relevant
with a negative sign for the coefficient); for the sign and level of significance of the estimated
2. The coefficient is insignificant; and disaster impact. Some of the findings differ. For
3. The coefficient is positive and significant at example, the use of EMDAT data or the limitation
least at 10% (t-value greater or equal to +1.65 to local studies is associated with a lower probability
or an explicitly reported significance level in to find a negative and significant disaster impact in
conjunction with a positive sign for the coef- the IPCC selection, which is not supported when we
ficient).
22 Foreach cost category, the weight of estimate i in study j is cal-
Because a different number of estimates are culated dividing the total number of t-values by the product of
collected from each primary study we follow Bi- the number of estimates from study j and the total number of
jmolt and Pieters(39) and weight observations for studies.
12

Table IV. Composite t-Statistics of the Selected Studies

Using Median Weighted


t-Statistics All Excluding Weighted All Excluding
from Each Study Observations Outliers Observations Outliers

TC N TC N TC N TWC N TWC N

Model type (1) −37.5 36 −159.1 719 −54.3 538 −100.7 719 −50.3 538
Model type (2) −4.8 27 −40.4 882 −33.3 875 −22.9 882 −19.7 875
Database: EM−DAT
Model type (1) −30.0 23 −105.8 513 −35.2 393 −73.2 513 −33.4 393
Model type (2) −3.2 17 −21.5 490 −21.0 489 −16.4 490 −16.2 489
Database: other
Model type (1) −22.6 13 −130.3 206 −46.7 145 −71.9 206 −42.3 145
Model type (2) −3.2 13 −36.6 392 −26.4 386 −16.3 392 −12.4 386
Countries: OECD
Model type (1) −34.9 31 −137.1 571 −48.6 453 −100.5 571 −46.1 453
Model type (2) −3.8 22 −28.0 634 −26.2 630 −16.5 634 −15.4 630
Countries: non-OECD
Model type (1) −31.8 27 −110.5 558 −40.2 433 −80.4 558 −39.5 433
Model type (2) −4.2 25 −39.8 822 −32.4 815 −22.6 822 −18.9 815
Published
Model type (1) −32.0 20 −155.9 290 −58.0 197 −92.6 290 −42.5 197
Model type (2) −3.9 12 −31.8 390 −23.5 387 −25.0 390 −20.5 387
Unpublished
Model type (1) −19.5 16 −77.8 429 −24.2 341 −50.7 429 −29.2 341
Model type (2) −2.9 15 −25.8 492 −23.7 488 −11.7 492 −10.6 488
Included in IPCC (2012)
Model type (1) −24.4 8 −92.1 93 −38.2 59 −60.1 93 −24.8 59
Model type (2) −2.7 9 −13.4 269 −12.8 268 −10.8 269 −10.6 268
Excluded by IPCC (2012)
Model type (1) −29.5 28 −135.0 626 −44.2 479 −81.6 626 −43.7 479
Model type (2) −4.0 18 −39.6 613 −31.5 607 −20.4 613 −16.6 607

Note: Composite t-values can be calculated based only on reported t-values (719 and 882 for direct and indirect costs studies) rather than reported levels of significance (765 and 923
for direct and indirect costs studies; cf. Fig. 2). In the table TC stands for composite unweighted t-values as explained by Equation (1) in the text whereas TW stands for composite
weighted t-statistics as explained by Equation (2) in the text.
Source: Authors’ elaborations.
van Bergeijk and Lazzaroni
Meta-Analysis Versus Review of Literature 13

Table V. Definition of Variables and Descriptive Statistics

Model Type 1 Model Type 2

Variable Description N Mean SD N Mean SD

Empirical design
N observations Number of observations in the original regression 765 2,564 5,085 923 3,021 11,697
EM-DAT 1 if data on disasters were taken from EM-DAT, else 0 765 0.681 0.466 923 0.562 0.496
Climatic disaster 1 if climatic natural disasters were included, else 0 765 0.884 0.321 923 0.856 0.351
Geologic disaster 1 if geologic natural disasters were included, else 0 765 0.320 0.467 923 0.650 0.477
Other disaster 1 if nonnatural disasters were included, else 0 765 0.009 0.095 923 0.190 0.392
N years Period considered in the estimation 765 36.349 16.178 923 32.670 12.462
N countries Number of countries in the sample 757 57 48 891 89 54
Local study 1 if the study focuses on 1 country, else 0 765 0.235 0.424 923 0.051 0.220
OECD 1 if OECD countries are included, else 0 739 0.824 0.381 923 0.687 0.464
Non-OECD 1 if non-OECD countries are included, else 0 739 0.755 0.430 923 0.935 0.247
Medium run 1 if the study considers impact in medium run, else 0 765 0.005 0.072 923 0.049 0.215
Long run 1 if the study considers impact in the long run, else 0 765 0.041 0.197 923 0.291 0.455
ND: Dummy 1 if natural disaster is modeled through a dummy, else 0 765 0.144 0.351 923 0.107 0.310
ND: Count 1 if natural disaster is modeled in terms of frequency, else 0 765 0.272 0.445 923 0.291 0.455
ND: Intensity 1 if natural disaster is modeled in terms of intensity, else 0 765 0.464 0.499 923 0.301 0.459
ND: Killed 1 if natural disaster is modeled in terms of deaths, else 0 765 0.026 0.026 923 0.076 0.265
ND: Affected 1 if natural disaster is modeled in terms of affected, else 0 765 0.094 0.292 923 0.115 0.319
ND: Damages 1 if natural disaster is modeled in terms of damages, else 0 – – – 923 0.173 0.379
Estimation technique
Panel 1 if data set is panel (0 = cross-section) , else 0 765 0.944 0.230 923 0.928 0.258
OLS 1 if estimation is conducted with OLS, else 0 765 0.410 0.492 923 0.572 0.495
Country FE 1 if estimation with country fixed effects, else 0 765 0.337 0.473 923 0.592 0.492
IV 1 if estimation with instrumental variables, else 0 765 0.080 0.271 923 0.089 0.285
NegBin 1 if estimation with negative binomial, else 0 765 0.229 0.420 – – –
ZINB 1 if estimation with zero-inflated neg.bin., else 0 765 0.065 0.247 – – –
GMM 1 if estimation with GMM, else 0 – – – 923 0.178 0.382
Studies outreach
Published 1 if the study is published in a peer-reviewed journal, else 0 765 0.429 0.495 923 0.454 0.498
IPCC 1 if the study is included in IPCC (2012), else 0 765 0.161 0.368 923 0.317 0.466

Source: Authors’ elaborations.

analyze the full May 2011 sample. It is also notewor- direct costs and 43% for indirect costs. The empirical
thy that the publication dummy is significant in the design of the studies is as before highly relevant for
IPCC sample but not in the full May 2011 sample. We the sign and level of significance of estimated disaster
do, however, also find much agreement: the findings impact for both model types. For example, studies
for both samples show the importance of the num- that use as disaster variable the number of disasters
ber of observations (although the effect is very small that occurred rather than a dummy for disaster
in magnitude), “other” disasters, panel studies, long- occurrence are more likely to report a significantly
run focus, and—importantly—the dummy for Model negative result (the probability is 46% pt. higher).
type 2 studies is significant for both samples. The im- The probability to report a significantly negative
plication of the latter finding is clear: a significant result further increases significantly when disaster
difference occurs for direct and indirect cost studies occurrence is measured in terms of intensity, fatali-
so that these groups need to be analyzed in separate ties, and affected persons. The heterogeneity of the
meta-analyses. results of the primary studies also depends on the
Table VII, therefore, provides these two meta- number of years and countries considered in the
analyses for the full sample of 60 studies available at primary studies. Contrary to a priori expectations
the end of 2013. primary studies that analyze non-OECD countries
The first line shows that the probability of a are more likely to report a positive result; the proba-
significantly negative effect (i.e., p[y = 1]) is 73% for bility increases by 18% pt. (direct costs). This result,
14

Table VI. Meta-Regression Analysis for All Studies Until IPCC Cut-Off Date (May 2011)

All Studies Identified—Until May 2011 IPCC Studies

(1) (2) (3) (1) (2) (3)


Significant Significant Significant Significant
Outcome(c) Negative Insignificant Positive Negative Insignificant Positive

P(y = c), sd 0.65 (0.32) 0.28 (0.23) 0.07 (0.12) 0.53 (0.35) 0.38 (0.28) 0.09 (0.21)
N obs 0.00*** (3.94) −0.00*** (−3.57) 0.00*** (−3.73) 0.00*** (4.57) − 0.00*** (−4.08) −0.00*** (−3.33)
EMDAT −0.06 (−1.16) 0.03 (1.13) 0.02 (1.18) −0.19*** (−5.38) 0.13*** (7.47) 0.06*** (2.7)
Nyears −0.00 (−1.05) 0.00 (1.07) 0.00 (1.01) 0.00 (−0.63) 0.00 (0.62) 0.00 (0.65)
Ncountries 0.00 (1.41) −0.00 (−1.47) 0.00 (−1.3) 0.00 (0.71) −0.00 (−0.72) − 0.00 (−0.68)
Local study −0.06 (−0.57) 0.04 (0.57) 0.02 (0.57) −1.08** (−2.17) 0.73** (2.13) 0.35* (1.95)
OECD −0.13*** (−2.6) 0.07*** (2.82) 0.05** (2.2) −0.10 (−0.88) 0.06 (0.92) 0.03 (0.79)
Non-OECD −0.05 (−0.5) 0.03 (0.51) 0.02 (0.48) 0.11 (1.42) −0.07 (−1.36) −0.04 (−1.44)
Climatic 0.03 (0.15) −0.01 (−0.15) −0.01 (−0.15) −0.45* (−1.83) 0.30 (1.64) 0.14** (2.16)
Geologic 0.09 (1) −0.05 (−1.00) −0.04 (−0.97) 0.19 (1) −0.13 (−0.98) −0.06 (−1.01)
Other 0.15*** (2.81) −0.09*** (−2.62) −0.06*** (−2.82) 0.23*** (4.11) −0.15*** (−3.25) −0.07*** (−4.36)
Medium run −0.13* (−1.74) 0.08* (1.78) 0.05 (1.61) −0.09 (−1.16) 0.06 (1.12) 0.03 (1.2)
Long run −0.14*** (−3.13) 0.08*** (3.08) 0.06*** (2.83) −0.11** (−2.17) 0.08** (2.03) 0.04** (2.15)
Type 2 −0.35*** (−4.17) 0.21*** (3.81) 0.14*** (3.82) −1.58*** (−4.39) 1.07*** (3.73) 0.51*** (3.58)
ND: Counta 0.14*** (2.65) −0.08*** (−2.61) −0.06** (−2.46) 0.05 (0.58) −0.04 (−0.59) −0.02 (−0.56)
ND: Intensitya 0.19** (2.47) −0.12** (−2.41) −0.08** (−2.36) −0.07 (−0.32) 0.05 (0.32) 0.02 (0.32)
ND: Killeda 0.02 (0.49) −0.01 (−0.48) −0.01 (−0.5) 0.09 (0.98) −0.06 (−0.98) −0.03 (−0.96)
ND: Affecteda 0.05 (0.76) −0.03 (−0.75) −0.02 (−0.77) 0.08 (0.67) −0.06 (−0.66) −0.03 (−0.67)
ND: Damagesa −0.06 (−1.02) 0.04 (1.03) 0.02 (1) −0.01 (−0.12) 0.01 (0.12) 0.00 (0.12)
Panel 0.24*** (2.78) −0.14*** (−2.84) −0.10** (−2.46) 0.89*** (3.38) −0.60*** (−3.17) −0.29*** (−2.77)
OLS 0.07 (0.99) −0.04 (−1.02) −0.03 (−0.94) 0.50* (1.85) −0.34* (−1.89) −0.16 (−1.61)
IV −0.00 (−0.06) 0.00 (0.06) 0.00 (0.06) 0.41 (1.27) −0.28 (−1.28) −0.13 (−1.2)
GMM −0.08 (−1.29) 0.05 (1.25) 0.03 (1.32) 0.71** (2.39) −0.48** (−2.32) −0.23** (−2.12)
Negative binomial 0.02 (0.13) −0.01 (−0.13) −0.01 (−0.13) −1.21** (−2.27) 0.82** (2.14) 0.39** (2.18)
ZINB −0.10 (−0.89) 0.06 (0.88) 0.04 (0.89)
CountryFE −0.11** (−2.1) 0.06** (2.05) 0.04** (2.04) 0.24 (1.55) −0.16 (−1.49) −0.08 (−1.55)
Published −0.03 (−0.5) 0.02 (0.49) 0.01 (0.51) 0.80** (2.5) −0.55** (−2.46) −0.26** (−2.15)
Pseudo R2 0.354 0.421
N studies 34 17
N obs. 778 376

Notes: Ordered probit, dependent variable: disaster impact effects. Standard errors clustered by studies.
a Reference category for the way natural disasters (ND) are operationalized is the dummy for disaster occurrence.

*, **, *** stand for 10%, 5%, and 1% level of significance.


van Bergeijk and Lazzaroni
Table VII. Meta-Regression Analysis by Model Type

Model Type 1 Model Type 2

(1) (2) (3) (1) (2) (3)


Outcome(c) Significant Negative Insignificant Significant Positive Significant Negative Insignificant Significant Positive

P(y = c), sd 0.73 (0.30) 0.11 (0.09) 0.16 (0.23) 0.43 (0.27) 0.46 (0.29) 0.11 (0.16)
N obs. 0.00 (0.72) 0.00 (0.65) −0.00 (−0.71) 0.00** (2.07) −0.00** (−2.00) −0.00** (−2.12)
EMDAT −0.01 (−0.03) −0.00 (−0.03) 0.01 (0.03) −0.34*** (−6.17) 0.22*** (4.93) 0.13*** (6.79)
N years −0.01** (−3.27) − 0.00* (−1.71) 0.02*** (3.1) −0.01*** (−4.49) 0.01*** (4.53) 0.00*** (3.73)
N countries 0.00 (1.58) 0.00 (1.36) − 0.00 (−1.58) 0.00 (1.16) −0.00 (−1.18) −0.00 (−1.10)
Local study 0.11 (1.50) 0.01* (1.9) −0.12 (−1.54) −0.87*** (−4.09) 0.55*** (3.71) 0.32*** (4.13)
OECD 0.02 (0.66) 0.00 (0.58) −0.02 (−0.66) −0.14*** (−3.02) 0.09*** (2.95) 0.05*** (2.87)
Non-OECD −0.16** (−2.33) −0.01 (−1.61) 0.18** (2.29) −0.07 (−1.22) 0.04 (1.24) 0.03 (1.16)
Meta-Analysis Versus Review of Literature

Climatic 0.28 (1.63) 0.02 (1.32) −0.31 (−1.62) 0.00 (0.02) −0.00 (−0.02) −0.00 (−0.02)
Geologic 0.11 (0.59) 0.01 (0.57) −0.12 (−0.59) 0.05 (0.41) −0.03 (−0.41) −0.02 (−0.42)
Other 0.40 (1.40) 0.03 (1.6) −0.44 (−1.42) 0.15** (2.14) −0.10** (−2.09) −0.06** (−2.13)
Medium run −0.35** (−2.21) −0.03 (−1.44) 0.38** (2.14) −0.24*** (−2.66) 0.15*** (2.84) 0.09** (2.29)
Long run −0.30 (−0.82) −0.03 (−0.66) 0.33 (0.81) −0.21*** (−4.69) 0.14*** (4.44) 0.08*** (4.16)
ND: Counta 0.46*** (2.87) 0.04* (1.89) −0.50*** (−2.82) −0.11 (−1.12) 0.07 (1.12) 0.04 (1.12)
ND: Intensitya 0.37*** (2.72) 0.03** (1.97) −0.40*** (−2.71) −0.26* (−1.94) 0.17** (2.04) 0.10* (1.74)
ND: Killeda 1.18*** (6.25) 0.10*** (3.62) −1.28*** (−6.9) 0.10 (1.27) −0.07 (−1.21) −0.04 (−1.37)
ND: Affecteda 0.40* (1.91) 0.03 (1.57) −0.44* (−1.9) 0.00 (0.03) −0.00 (−0.03) − 0.00 (−0.03)
ND: Damagesa 0.00 (0.01) −0.00 (−0.01) −0.00 (−0.01)
Panel −0.07*** (−8.23) −0.01*** (−2.77) 0.07*** (8.09) 0.39*** (3.51) −0.25*** (−3.56) −0.14*** (−3.07)
OLS 0.09 (1.09) 0.01 (1.34) −0.10 (−1.11) 0.00 (0.01) 0.00 (−0.01) −0.00 (−0.01)
IV 0.25* (1.88) 0.02*** (3.2) −0.27** (−1.96) 0.07 (0.71) −0.04 (−0.69) −0.03 (−0.74)
GMM 0.01 (0.06) 0.00 (−0.06) 0.00 (−0.06)
Negative binomial 0.14 (0.92) 0.01 (1.03) −0.15 (−0.93)
ZINB 0.53*** (2.71) 0.04*** (3.02) −0.57*** (−2.82)
CountryFE 0.06 (1.62) 0.01* (1.92) −0.07* (−1.66) 0.12** (2.01) −0.07** (−2.07) −0.04* (−1.84)
Published 0.43*** (3.95) 0.04* (1.97) −0.47*** (−3.77) 0.45*** (4.94) −0.28*** (−4.59) −0.17*** (−4.44)
IPCC 0.20 (0.65) 0.02 (0.55) −0.22 (−0.64) −0.13** (−1.99) 0.08* (1.95) 0.05 (1.97)
Pseudo R2 0.317 0.176
N studies 36 29
N obs. 739 891

Note: Ordered probit, dependent variable: disaster impact effects. Standard errors clustered by studies.
a Reference category for the way natural disasters (ND) are operationalized is the dummy for disaster occurrence.

*, **, and *** stand for 10%, 5%, and 1% level of significance.
15
16 van Bergeijk and Lazzaroni

however, has to be interpreted with caution, as the confident than the conclusions that we base on the
list of non-OECD countries includes many different meta-analysis of a larger set of papers.23
economies both in terms of regional distribution
and income levels (ranging from LDCs to BRICS
5. CONCLUSIONS AND AREAS FOR
and advanced economies that are not members of
FURTHER RESEARCH
the OECD). Clearly, this is an important issue for
further research. Recent years have witnessed a strong increase
For indirect costs studies, different measure- in the number of available studies on the macroe-
ments of disasters do not influence the probability to conometric impact of natural disasters. These stud-
report a positive or negative impact except for disas- ies, as in any emerging field of science, are
ter intensity, increasing the probability to report pos- heterogeneous both in terms of findings and method-
itive and significant results by 10%. The data source ological characteristics. In this article, we discussed
is highly relevant: using the EMDAT database re- a meta-analysis providing descriptive bibliometric
duces the probability of significantly negative results statistics and an econometric synthesis of the liter-
by 36%. If the indirect costs are analyzed with a ature. The main findings of the meta-analysis are
medium- to long-term perspective, the reported dis- that:
aster outcome has 21–26% pt. lower probability to be r The impact of disasters is significantly negative.
significantly negative and a 8–9% pt. higher probabil-
ity to be significantly positive, reflecting the ability of The evidence is strongest for direct costs studies
economies to recover from disasters as time passes. where we see no difference between our larger
In comparison with IPCC findings, although devel- sample and the studies included in the IPCC re-
port.
oped (OECD) countries may experience high eco- r Direct cost studies and indirect cost studies dif-
nomic losses, the meta-analysis suggests that studies
including OECD countries have a 14% lower prob- fer significantly, both in terms of the confidence
ability to report negative and significant disaster ef- that can be attached to a negative impact of
fects (possibly due to higher preparedness and abil- natural disasters and in terms of the sources of
ity to recover). Introducing country fixed effects in heterogeneity of the findings reported in the pri-
mary studies.
primary studies econometric analyses should capture r Country coverage, time period, and natural dis-
higher/lower wealth and preparedness time-invariant
country characteristics and this is reflected by the aster operationalization significantly influence
finding that studies that include country fixed effects the probability that a study reports a signifi-
have a 12% higher probability to report negative and cantly negative disaster impact in direct costs
studies;
significant results. r For indirect cost studies we find that hetero-
We next turn to two important issues that relate
to bias. The first issue relates to publication bias. geneity is due to the database used, the time
The dummy Published is significant and shows that frame (does the study consider short-/medium-
the likelihood of reporting significantly negative /long-term effects) and the limitation of the
analysis to a single country.
results is 43–45% pt. higher for published studies r Both direct and indirect costs studies suggest
(after controlling for other characteristics of the
primary studies) for both model types. The second publication bias.
issue relates to bias in the selection of studies (this We confronted this methodology with the au-
consists of initial bias as illustrated in Table VI and thoritative selective peer-reviewed literature review
of the noninclusion of studies that were published of the IPCC special report Managing the Risks of
after the IPCC’s cut-off day). Importantly, we find Extreme Events and Disasters to Advance Climate
no evidence of bias for direct cost studies, so that Change Adaptation. Table VIII provides an overview
the IPCC selection represents the field that we know of strengths and weaknesses of the two approaches.
today quite well. For indirect cost studies, we note The traditional narrative literature synthesis
that the studies included in the IPCC report are less clearly has strong points mainly related to its
likely to report significantly negative results (the
probability decreases by 13% pt.). The point here 23 Note again that the IPCC sample covers studies until May 2011,
is that the IPCC selection of macroeconometric so it is possible that this finding partially reflects that we updated
studies could explain why its conclusions are less the sample as requested by the referees of the journal
Meta-Analysis Versus Review of Literature 17

Table VIII. Strengths and Weaknesses of Traditional Literature Review and Meta-Analysis

Traditional Literature Meta-Analysis


Review (IPCC 2012) (Authors 2013)

Qualitative analysis and case studies Yes No


Quantitative analysis Yes Yes, but not CGE, VAR,a and input-output
Identification of methodological differences Yes Yes
Identification of impact of methodological differences No Yes
Possibility to include “incomplete studies” Yes No
Coverage of literature relevant for method Incomplete (selective) Incomplete (search strategy)
Transparent nonsubjective synthesis No Yes

Source: Authors’ elaborations.


a If only impulse response function reported in the study.

flexibility and ability to also include purely qualita- actually a quite general nuisance. Disdier and
tive analysis (including mathematical models), case Head,(8) analyzing the distance effect in 103 gravity
studies, and studies that by their very nature do not models, report that only 1,257 of 1,466 regressions
report significance levels (input-output models and reported standard errors (a 15% loss). Moons and
computable general equilibrium models). Indeed, in Van Bergeijk,(40) analyzing the impact of economic
the case of natural disasters meta-analysis and IPCC diplomacy, report that of the 1,334 coefficients taken
literature review studies, samples do not show high from 30 studies only 643 t-values (48%) were re-
overlap because the bulk of studies in IPCC (sections ported or could be calculated because essential infor-
4.5, 5.5.2.1, 6.1, and 8.2.1) are nonmacroeconometric mation such as standard errors was missing in the pri-
(Fig. 3). mary studies. It should also be noted that this is not
Our meta-analysis does not take non- a problem that is specific to economics—comparable
macroeconometric studies into account (except problems also occur in fields where meta-analysis is
for VAR studies presenting regression coefficients), well established as a tool to synthesize knowledge
although economic tools such as IO models, VARs, (see, e.g., Goodman et al.(41) ). Incomplete or inaccu-
and CGE models certainly generate valuable sci- rate studies (e.g., those that fail to report standard er-
entific knowledge. The method to synthesize the rors, t-values, or the number of observations) can still
knowledge is completely transparent for meta- be considered in the traditional review of literature
analyses and this is a benefit in a highly political area (probably with a lower implicit weight of evidence)
such as the developmental perspective on the impact To conclude, both methodologies have short-
of natural disasters. Importantly, meta-analysis helps comings as far as the studies coverage is concerned.
to uncover the relationship between findings in Traditional literature reviews are more prone to
primary studies and their methodological character- be affected by author decisions to include/exclude
istics. Meta-analysis provides a formal framework to studies depending on their findings. For example, the
test hypotheses. Sometimes, experts and researchers IPCC literature review is by design a selective survey
may see such tests as complex methods that prove of the literature that emphasizes what the authors
what they already know, but the fact that this can of the survey deem reliable and important. As such,
be done objectively and transparently certainly adds the objectivity of the IPCC review is not based on
value. When the expert researchers say “we know full inclusivity of all available literature but mostly
this” or “this result is biased by this methodology,” on the rigorous review and peer-review process. This
the meta-analyst says “yes, and we can test/prove would provide the necessary quality and objectivity,
this.” whereas our meta-analysis has to rely on trans-
One of the major problems with doing meta- parency, full coverage of the literature, and rigorous
analyses in economics is that primary studies do robustness testing.24 Meta-analysis Internet-based
not always provide the necessary parameters. In the search strategies might exclude relevant studies not
case of natural disasters we had to exclude the stud- in the form of academic articles and/or working
ies of Benson and Clay,(21) Jaramillo,(22) and Sim- papers. The incomplete coverage also may reflect the
mons and Sutter(23,24) because of missing number of
observations in the estimations presented. This is 24 We thank an anonymous referee for pointing this out to us.
18 van Bergeijk and Lazzaroni

inability to cope with certain methods and method- March. (Climate Change and African Political
ologies and the data requirements of meta-analyses. Stability Research Brief; no. 9).
It is, however, also a consequence of decisions to 2. Cashin P, Dyczewski P. Government re-
include or exclude studies. sponses to natural disasters in the Caribbean.
Our take on this is that the two methods force In Sahay R, Robinson DO, Cashin P(eds). The
two different perspectives on researchers and that Caribbean: From Vulnerability to Sustained
each perspective helps to uncover other (aspects of) Growth. Washington, DC: IMF, 2006.
literatures. If that is the case then research synthesis 3. Jayraman TK. Macroeconomic reform and re-
would benefit from combining the two approaches. silience building in small states. In Kisanga
So we arrive at two intertwined conclusions. A EJ, Cordina G, Briguglio L. Building the Eco-
meta-analysis without a traditional review of the nomic Resilience of Small States. Malta: Is-
literature is incomplete. A traditional review of the lands and Small States Institute and London:
empirical literature is equally incomplete without a Commonwealth Secretariat, 2006.
meta-analysis: IPCC could have been more confident 4. Kenny C. Why Do People Die in Earth-
about the negative economic impact of disasters if a quakes? The Cost, Benefits and Institutions of
meta-analysis had been used. Disaster Risk Reduction in Developing Coun-
tries. Washington, DC: World Bank, 2009.
(World Bank Policy Research Working Paper;
ACKNOWLEDGMENTS no. 4823).
5. Liborio C. Natural Disasters: From Destruc-
We thank EAERE and the Harvard Center for
tion to Recovery. St. Louis: Federal Reserve
Risk Analysis (HCRA) for financial support and Lisa
Bank of St. Louis, 2011 October. (Liber8 Eco-
A. Robinson and James K. Hammitt and participants
nomic Information Newsletter).
at the HCRA’s October 2013 “Methods for Research
6. OIC. OIC Countries and Natural Disasters:
Synthesis: A Cross-Disciplinary Approach” work-
Assessment of Risk. Proceedings of the 28th
shop for their helpful comments. Research for this ar-
Session of COMCEC-Standing Committee for
ticle was conducted when Sara Lazzaroni was a PhD
Economic and Commercial Cooperation of
student at the Università Cattolica del Sacro Cuore
the Organization of Islamic Cooperation, 2012
of Piacenza (Italy); financial support from the univer-
October 8–11. Istanbul, Turkey: COMCEC.
sity is gratefully acknowledged. We also would like
7. Sawada Y, Oum S (eds). Economic and Wel-
to thank two anonymous referees, Raymond Florax,
fare Impacts of Disasters in East Asia and Pol-
and professors and participants at the 2013 Belpasso
icy Responses. Jakarta: ERIA, 2011. (ERIA
International Summer School and the 2013 MAER-
Research Project Report; no. 2011-8).
Net conference in Greenwich for helpful comments.
8. UNISDR. From Shared Risk to Shared
We thank authors of primary studies who kindly
Value—The Business Case for Disaster Risk
replied providing the additional data needed for the
Reduction. Global Assessment Report on Dis-
compilation of the meta data set and other valu-
aster Risk Reduction. Geneva, Switzerland:
able insights for our analysis: Jesse Anttila-Hughes,
United Nations Office for Disaster Risk Re-
Laura Bakkensen, Oscar Becerra, Eduardo Cav-
duction (UNISDR), 2013.
allo, Juncal Cuñado, Tatyana Deryugina, Solomon
9. World Bank. World Development Report
Hsiang, Matthew Kahn, Carolyn Kousky, Norman
2014: Managing Risk for Development. Wash-
Loayza, Tom McDermott, Ilan Noy, Roger Pielke
ington, DC: World Bank, 2014.
Jr., Tobias Rasmussen, Bob Reed, Mark Skidmore,
José Tavares, Dean Yang, Yuki Yikeda, and Yanos
Zylberberg.
APPENDIX B: ADDITIONAL REFERENCES
FOR THE LITERATURE CITED IN THE
APPENDIX A: REFERENCES FOR IPCC REPORT (TABLES I-III)
POLICY REPORTS
10. Albala-Bertrand JM. Political Economy of
1. Bussell J, Colligan A. Institutional Capacity Large Natural Disasters with Special Refer-
for Natural Disasters: Methodology for Case ence to Developing Countries. Oxford: Calen-
Studies in Africa. Austin, TX: CCASPS, 2013 dron Press, 1993.
Meta-Analysis Versus Review of Literature 19

11. Benson C, Clay EJ. Developing countries and for Public Intervention. Washington, DC:
the economic impacts of natural disasters. Pp. World Bank, 2009.
11–21 in Kreimer A, Margaret A (eds). Man- 23. Cuny FC. Disasters and Development. Ox-
aging Disaster Risk in Emerging Economies. ford, UK: Oxford University Press, 1983.
Washington, DC: World Bank, 2000 (Disaster 24. Cutter SL, Finch C. Temporal and spatial
Risk Management Series; no. 2). changes in social vulnerability to natural haz-
12. Benson C, Clay E. Disasters, vulnerability, ards. Proceedings of the National Academy of
and global economy. Pp. 3–31 in Kreimer A, Science, 2008;105(7):2301–2306.
Arnold M, Carlin A (eds). Building Safer 25. del Ninno C, Dorosh PA, Smith LC. Public
Cities: The Future of Disaster Risk. Washing- policy, markets and household coping strate-
ton, DC: World Bank, 2003. (Disaster Risk gies in Bangladesh: Avoiding a food security
Management Series; no. 3). crisis following the 1998 floods. World Devel-
13. Benson C, Clay E. Understanding the Eco- opment, 2003;31(7):1221–1238.
nomic and Financial Impacts of Natural 26. DFID. Natural Disaster and Disaster Risk
Disasters. Washington, DC: World Bank, Reduction Measures: A Desk Review of
2004. (Disaster Risk Management Series Costs and Benefits. London, UK: UK Depart-
no. 4). ment for International Development, The Sta-
14. Birch EL, Wachter SM (eds). Rebuilding Ur- tionery Office, 2005.
ban Places After Disaster: Lessons from Hur- 27. ECLAC. Handbook for Estimating the
ricane Katrina. Philadelphya, PA: University Socio-Economic and Environmental Effects
of Pennsylvania Press, 2006. of Disasters. Santiago, Chile: LC/MEX/G.5,
15. Bouwer LM, Crompton RP, Faust E, Höppe LC/L 1874 Economic Commission for Latin
P, Pielke RA Jr. Confronting disaster losses. America and the Caribbean, 2003. Avail-
Science, 2007: 318, 753. able at www.undp.org/cpr/disred/documents/
16. Brooks N, Adger WN, Kelly PM. The publications/eclac_handbook.pdf, Accessed
determinants of vulnerability and adaptive June 15, 2012.
capacity at the national level and the implica- 28. Haimes Y, Horowitz B, Lambert J, San-
tions for adaptation. Global Environ Change, tos J, Lian C, Crowther K. Inoperability
2005;15:151–163. input-output model for interdependent
17. Burton K, Kates R, White G. The Environ- infrastructure sectors. I: Theory and method-
ment as Hazard, 2nd ed. New York: Guilford ology. Journal of Infrastructure Systems,
Press, 1993. 2005;11:67–79.
18. Cardona OD, Ordaz MG, Marulanda MC, 29. Hallegatte S. An adaptive regional input-
Carreno ML, Barbat AH. Disaster risk from output model and its application to the assess-
a macroeconomics perspective: A metric ment of the economic cost of Katrina. Risk
for fiscal vulnerability evaluation. Disasters. Analysis, 2008;28(3):779–799.
2010;34(4):1064–1083. 30. Hallegatte S, Dumas P. Can natural disasters
19. Charveriat C. Natural Disasters in Latin have positive consequences? Investigating the
America and the Caribbean: An Overview role of embodied technical change. Ecological
of Risk. Washington, DC: IDB, 2000. (IDB Economics, 2009;68:777–786.
Working Paper; no. 434). 31. Hallegatte S, Ghil M. Endogenous Business
20. CRED. EM-DAT: The OFDA/CRED In- Cycles and the Economic Response to Ex-
ternational Disaster Data Base. Louvain, ogenous Shocks. Milan, Italy: Fondazione Eni
Belgium: Centre for Research on the Epi- Enrico Mattei, 2007. (Working Paper; no.
demiology of Disasters, Université Catholique 2007.20).
de Louvain, 2010. Available at: www.emdat. 32. Hallegatte S, Hourcade JC, Dumas P. Why
be/Database/, Accessed June 10, 2012. economic dynamics matter in assessing cli-
21. Crowards T. Comparative Vulnerability mate change damages: Illustration on extreme
to Natural Disasters in the Caribbean. events. Ecological Economics, 2007;62:330–
Bridgetown, Barbados: Caribbean Develop- 340.
ment Bank, 2000. 33. Ibarraran ME, Ruth M, Ahmad S, London M.
22. Cummins J, Mahul O. Catastrophe Risk Fi- Climate change and natural disasters: Macroe-
nancing in Developing Countries: Principles conomic performance and distributional
20 van Bergeijk and Lazzaroni

impacts. Environment, Development and Sus- of the Intergovernmental Panel on Climate


tainability, 2009;11:549–569. Change. Cambridge, UK: Cambridge Univer-
34. IFRC. World Disasters Report 2010: Focus sity Press, 2007.
on Urban Risk. Geneva, Switzerland: Interna- 44. Pelham L, Clay E, Braunholz T. Natural Dis-
tional Federation of Red Cross and Red Cres- asters: What is the Role for Social Safety Nets?
cent Societies, 2010. Washington, DC: World Bank, 2011.
35. IPCC. Climate Change 2001: Impacts, Adapta- 45. Rodriguez-Oreggia E, de la Fuente, de la
tion and Vulnerability. Contribution of Work- Torre R, Moreno H, Rodriguez C. The Impact
ing Group II. In McCarthy JJ, Canziani OF, of Natural Disasters on Human Development
Leary NA, Dokken DJ, White KS (eds). Third and Poverty at the Municipal Level in Mexico.
Assessment Report of the Intergovernmental Cambridge, MA: Center for International De-
Panel on Climate Change. Cambridge, UK: velopment at Harvard University, 2010. (CID
Cambridge University Press, 2001. Working Paper; no. 43).
36. Lal PN. Vulnerability to natural disasters: An 46. Rose A, Liao S. Modeling regional economic
economic analysis of the impact of the 2009 resilience to disasters: A computable general
floods on the Fijian sugar belt. Pacific Eco- equilibrium analysis of water service disrup-
nomic Bulletin, 2010; 25(2):62–77. tions. Journal of Regional Science, 2005;45:75–
37. McKenzie E, Prasad B, Kaloumaira A. Eco- 112.
nomic Impact of Natural Disasters on De- 47. Rose A, Benavides J, Chang S, Szczesnjak P,
velopment in the Pacific, Vol. I. Suva, Fiji: Lim D. The regional economic impact of an
SOPAC and USP, 2005. earthquake: Direct and indirect effects of elec-
38. Mechler R. Natural Disaster Risk Manage- tricity lifeline disruptions. Journal of Regional
ment and Financing Disaster Losses in Devel- Science, 1997;37:437–458.
oping Countries. Karlsruhe, Germany: Verlag 48. Skoufias E. Economic crises and natural
für Versicherungswirtschaft, 2004. disasters: Coping strategies and policy implica-
39. Nicholls N. Recent trends in the seasonal tions. World Development, 2003;13(7):1087–
and temporal behaviour of the El Niño 1102.
Southern Oscillation. Geophysical Research 49. Smith SK, McCarty C. Florida’s 2004 hurri-
Letters, 2008; 35:L19703. cane season: Demographic response and re-
40. O’Brien G, O’Keefe P, Rose J, Wisner B. Cli- covery. Proceedings of Southern Demographic
mate change and disaster management. Disas- Association Meeting, Durham, NC, 2–4 Nov
ters, 2006;30(1):64–80. 2006.
41. Otero RC, Marti RZ. The impacts of nat- 50. Swiss Re. Natural Catastrophes and Man-
ural disasters on developing economies: Im- Made Disasters in 2009: Catastrophe Claims
plications for the international development Few Victims, Insured Losses Fall. Zurich,
and disaster community. Pp. 11–40 in Munas- Switzerland: Swiss Reinsurance Company,
inghe M, Clarke C (eds). Disaster Prevention 2010. (Sigma, No. 1/2010).
for Sustainable Development: Economic and 51. Thomalla F, Downing T, Spanger-Siegfried E,
Policy Issues. Washington, DC: World Bank, Han G, Rockström J. Reducing hazard vulner-
1995. ability: Towards a common approach between
42. Owens T, Hoddinott T, Kinsey B. Ex-ante ac- disaster risk reduction and climate adaptation.
tions and ex-post public responses to drought Disasters, 2006;30(1): 39–48.
shocks: Evidence and simulations from Zim- 52. Tol RSJ, Leek FPM. Economic analysis of nat-
babwe. World Development, 2003;31(7):1239– ural disasters. Pp. 308–327 in Downing TE, Ol-
1255. sthoorn AA, Tol RSJ (eds). Climate, Change
43. Parry ML, Canziani OF, Palutikof JP, van del and Risk. London, UK: Routledge, 1999.
Linden PJ, Hanson CE. Technical summary. 53. Tsuchiya S, Tatano H, Okada N. Economic
Pp. 23–78 in Parry, ML, Canziani OF, Pa- loss assessment due to railroad and high-
lutikof JP, van der Linden PJ, Hanson CE way disruptions. Economic Systems Research,
(eds). Climate Change 2007: Impacts, Adapta- 2007;19(2):147–162.
tion and Vulnerability. Contribution of Work- 54. UNDP. Reducing Disaster Risk: A Challenge
ing Group II to the Fourth Assessment Report for Development. New York: United Nations
Meta-Analysis Versus Review of Literature 21

Development Programme, Bureau for Crisis 3. Cavallo E, Noy I. The Economics of Natural Disasters: A Sur-
Prevention and Recovery, 2004. vey. Washington: Inter-American Development Bank, 2010.
(IDB Working Paper Series; no.124).
55. UNISDR. Global Assessment Report on Dis- 4. Raschky PA. Institutions and the losses from natural disasters.
aster Risk Reduction—Risk and Poverty in a Nat Hazards Earth Sys Sci, 2008; 8(4):627–634.
Changing Climate: Invest Today for a Safer 5. Lazzaroni S, van Bergeijk PAG. Natural Disasters Impact,
Factors of Resilience and Development: A Meta-Analysis of
Tomorrow. Geneva, Switzerland: United Na- the Macroeconomic Literature. The Hague, Netherlands: In-
tions International Strategy for Disaster Re- ternational Institute of Social Studies of Erasmus University
duction, 2009. Rotterdam, 2013. (ISS Working Paper Series/General Series;
no. 554).
56. West CT, Lenze DG. Modeling the regional 6. IPCC (Intergovernmental Panel on Climate Change). Manag-
impact of natural disasters and recovery: A ing the Risks of Extreme Events and Disasters to Advance
general framework and an application to Hur- Climate Change Adaptation. (The SREX Report). Cambridge
and New York: Cambridge University Press, 2012 Available
ricane Andrew. International Regional Sci- at: http://ipcc-wg2.gov/SREX/report, Accessed May 20, 2013.
ence Review 1994;17:121–150. 7. Klomp J, Valckx K. Natural disasters and economic growth: A
57. Widawsky DA, O’Toole JC. Prioritizing the meta-analysis. Global Environmental Change, 2014; 26:183–
195.
Rice Biotechnology Research Agenda for 8. Disdier AC, Head K. The puzzling persistence of the distance
Eastern India. New York: Rockefeller Foun- effect on bilateral trade. Review of Economic Statistics, 2008;
dation Press, 1990. 90(1):37–48.
9. Florax RJGM, de Groot HLF, de Mooij RA. Meta-Analysis:
58. World Bank. Cities, Seas and Storms: Man- A Tool for Upgrading Inputs of Macroeconomic Policy Mod-
aging Change in Pacific Island Economies. els. Amsterdam: Tinbergen Institute, 2002. (Tinbergen Insti-
Volume IV, Adapting to Climate Change, tute Discussion Paper; no. 2002: TI 2002-041/3).
10. Stanley TD, Doucouliagos HS. Meta-Regression Analysis in
Papua New Guinea and Pacific Islands Coun- Economics and Business. Oxon, UK: Routledge, 2012.
try Unit. Washington, DC: International 11. DeCicca P, Kenkel D. Synthesizing Econometric Evidence:
Bank for Reconstruction and Develop- The Case of Price Elasticity Estimates Methods for Research
Synthesis: A Cross-Disciplinary Approach, 2013 Oct 3–4.
ment/World Bank, 2000. Available at: Cambridge, MA: Harvard School of Public Health, 2013.
<web.worldbank.org/WBSITE/EXTERNAL/ 12. Reed WR, Mercer R. REPLICATION STUDY: Toya and
COUNTRIES/EASTASIAPACIFICEXT/ Skidmore (Economics Letters, 2007). Christchurch, New
Zealand: Department of Economics and Finance, Univer-
PACIFICISLAND-SEXTN/0,contentMDK: sity of Canterbury. 2013. (Working Papers in Economics; no.
20218394pagePK:141137piPK:217854the 13/05).
SitePK:4 41883,00.html>. 13. Toya H, Skidmore M. Economic development and the impacts
of natural disasters. Economics Letters, 2007; 94(1):20–25.
59. World Bank. Disasters, Climate Change, 14. Doucouliagos H, Paldam M. The aid effectiveness literature:
and Economic Development in Sub-Saharan The sad results of 40 years of research. Journal of Economic
Africa: Lessons and Future Directions. Eval- Surveys, 2009; 23(3): 433–461.
15. Fialho D. Slicing Up the Developing World. Differentiation
uation Brief 3. Washington, DC: Independent in the Special Treatment of Developing Countries: Cui Bono?
Evaluation Group, World Bank, 2007. [Dissertation]. Rotterdam, Netherlands: Erasmus University
60. World Bank and UN. Natural Hazards, Un- Rotterdam, 2015.
16. Betranou E, Mallender J. Relationship between the data col-
Natural Disasters: The Economics of Ef- lection approaches and the application of health economics
fective Prevention. Washington, DC: Inter- models in practice: A case study approach. Methods for Re-
national Bank for Reconstruction and De- search Synthesis: A Cross-Disciplinary Approach, 2013 Oct 3-
4. Cambridge, MA: Harvard School of Public Health, 2013.
velopment/World Bank, 2010. Available at: 17. Gujarati DN, Porter DC. Basic Econometrics, 5th ed. New
<gfdrr.org/gfdrr/nhud-hom>, Accessed June York: McGraw Hill International Editions, 2009.
17, 2013. 18. Fomby T, Ikeda Y, Loayza NV. The growth aftermath of nat-
ural disasters. Journal of Applied Econometrics, 2013; 28:412–
434. 
19. Cuñado J, Ferreira S. The macroeconomic impacts of natural
REFERENCES (REFERENCES MARKED disasters: The case of floods. Land Economy, 2014; 90(1):149–
168. 
WITH  INDICATE STUDIES INCLUDED 20. Moreno A, Cardona OD. Effects of natural disasters on
IN THE META-ANALYSIS) growth, employment, inflation and income distribution: an
evaluation of the case of Colombia and Mexico. Back-
1. Okuyama Y. Economic modeling for disaster impact analysis: ground paper prepared for the 2011 Global Assessment
Past, present and future. Economic Systems Research, 2007; Report on Disaster Risk Reduction. Geneva, Switzerland:
19(2):115–124. United Nations International Strategy for Disaster Reduction,
2. Pelling M, Ozerdem A, Barakat S. The macro-economic im- 2011.
pact of disasters. Program in Development Studies, 2002; 21. Benson C, Clay E. The Impact of Drought on Sub-Saharan
2(4):283–305. African Economies: A Preliminary Examination. Washington,
22 van Bergeijk and Lazzaroni

DC: World Bank, 1998 (World Bank Technical Paper Series; 40. Moons S, van Bergeijk PAG. Does economic diplo-
no. 401). macy work? A meta-analysis on the effect of economic
22. Jaramillo CRH. Do Natural Disasters Have Long-Term Ef- diplomacy on international economic flows.(August 12,
fects on Growth? Manuscript. Bogotá, Colombia: Universidad 2011). Available at SSRN: http://ssrn.com/abstract=1908699 or
de los Andes, 2009. (Documentos CEDE; ISSN 1657-5334). http://dx.doi.org/10.2139/ssrn.1908699, Accessed June 20, 2012
23. Simmons KM, Sutter D. WSR-88D radar, tornado warnings, 41. Goodman JE, Petito Boyce C, Sax SN, Beyer LA,
and tornado casualties. Weather Forecasting, 2005; 20:301– Prueitt RL. Rethinking meta-analysis: Applications for
310. air pollution data and beyond. In Methods for Research
24. Simmons KM, Sutter D. False alarms, tornado warnings, and Synthesis: A Cross-Disciplinary Approach, October
tornado casualties. Weather, Climate, and Society, 2009; 1:38– 3–4, 2013. Cambridge, MA: Harvard School of Public
53. Health, 2013. Available at: http://cdn1.sph.harvard.edu/wp-
25. Baker SR, Bloom N. Does Uncertainty Reduce Growth? content/uploads/sites/1273/2013/09/Goodman-et-al.-Sept-
Using Disasters as Natural Experiments’. Cambridge, MA: 2013.pdf, Accessed October 3, 2013.
NBER, 2013. (Working Paper; no. 19475). Available at: 42. Ahlerup P. Are Natural Disasters Good for Economic
http://www.nber.org/papers/w19475, Accessed November 26, Growth? [Letter]. Gothenburg, Sweden: University of
2013. Gothenburg, School of Business, Economics and Law, 2013.
26. Okuyama Y, Sahin S. Impact Estimation of Disasters: A (University of Gothenburg Working Papers in Economics;
Global Aggregate for 1960 to 2007. Washington, DC: World no. 553). 
Bank, 2009. (World Bank Policy Research Working Paper; no. 43. Ahmed M, Iqbal K. Disaster and Decentralization. Northfield,
4963). MN: Department of Economics, Carleton College. 2009. (Car-
27. Anttila-Hughes JK, Hsiang SM. Destruction, Disinvestment, leton College Department of Economics Working Paper Se-
and Death: Economic and Human Losses Following Environ- ries; no. 2009-03). 
mental Disaster. SSRN, 2013. Available at: http://ssrn.com/ 44. Anbarci N, Escaleras M, Register CA. Earthquake fatalities:
abstract=2220501 or http://dx.doi.org/10.2139/ssrn.2220501, The interaction of nature and political economy. Journal of
Accessed May 1, 2013. Public Economics, 2005; 89:1907–1933.
28. Deryugina T. The Role of Transfer Payments in Miti- 45. Bakkensen LA. Natural Disasters and Adaptation: Ev-
gating Shocks: Evidence from the Impact of Hurricanes. idence from Global Tropical Cyclone Damages. New
SSRN, 2013. Available at: http://ssrn.com/abstract=2314663or Haven, CT: Yale University, 2013. Available at: http://cbey.
http://dx.doi.org/10.2139/ssrn.2314663, Accessed May 1, yale.edu/uploads/Environmental%20Economics%20Seminar/
2013. Bakkensen_Cyclones.pdf, Accessed December 15,
29. Felbermayr G, Gröschl J. Naturally Negative: The Growth 2013.
Effects of Natural Disasters. Munich: Center for Economic 46. Bergholt D, Lujala P. Climate-related natural disasters, eco-
Studies and Ifo Institute, 2013. (CESifo Working Paper; nomic growth, and armed civil conflict. Journal of Peace Re-
no. 4439).  search, 2012; 49(1):147–162.
30. Jackson O. Natural disasters, foreign aid, and economic 47. Caselli F, Malthotra P. Natural Disasters and Growth: From
growth. SSRN, 2013. http://ssrn.com/abstract=2329404 or Thought Experiment to Natural Experiment. Washington,
http://dx.doi.org/10.2139/ssrn.2329404, Accessed May 1, DC: International Monetary Fund, 2004.
2013. 48. Cavallo E, Galiani S, Noy I, Pantano J. Catastrophic natural
31. Zylberberg Y. Natural Natural Disasters and Economic Dis- disasters and economic growth. Review of Economic Statis-
ruption. Paris, France: Paris School of Economics, 2012. (PSE tics, 2013; 95(5):1549–1561. 
Working Papers). 49. Cavallo E, Powell A, Becerra O. Estimating the direct eco-
32. Hochrainer S. Assessing Macroeconomic Impacts of Natural nomic damages of the earthquake in Haiti. Economics Jour-
Disasters: Are There Any? [Letter]. Washington, DC: World nals, 2010; 120(August):F298–F312. 
Bank, 2009. (World Bank Policy Research Working Paper; no. 50. Czajkowski J, Kennedy E. Fatal tradeoff? Toward a better un-
4968). derstanding of the costs of not evacuating from a hurricane
33. Raddatz C. The Wrath of God: Macroeconomic Costs of Nat- in landfall counties. POP Environment, 2010; 31(1–3):121–
ural Disasters. Washington, DC: World Bank, 2009. (World 149. 
Bank Policy Research Working Paper; no. 5039). 51. Czajkowski J, Simmons K, Sutter D. An analysis of coastal and
34. Sinani E, Meyer K. When and where does foreign direct inland fatalities in landfalling US hurricanes. Natural Hazards,
investment generate positive spill-overs? A meta-analysis 2011; 59(3):1513–1531. 
[letter]. Journal of International Business Studies, 2009; 52. Escaleras M, Anbarci N, Register CA. Public sector corrup-
40(7):1075–1094. tion and major earthquakes: A potentially deadly interaction.
35. Havránek T, Iršová Z. Meta-analysis of intra-industry FDI Public Choice, 2007; 132(1/2):209–230. 
spill-overs: Updated evidence. Czech Journal of Economics 53. Escaleras M, Register CA. Fiscal decentralization and natural
and Finance, 2010; 60(2):151–174. hazards. Public Choice, 2012; 151:165–183. 
36. Mebratie AD, van Bergeijk PAG. Firm heterogeneity and de- 54. Fankhauser S, McDermott TKJ. Understanding the Adap-
velopment: A meta analysis of FDI productivity spillovers. tation Deficit: Why are Poor Countries More Vulnerable to
Journal of International Trade & Economic Development, Climate Events than Rich Countries? [Letter]. London: Cen-
2013; 22(1):53–74. tre for Climate Change Economics and Policy and Grantham
37. Diebel D, Wooster R. Productivity spillovers from for- Research Institute on Climate Change and the Environment,
eign direct investments in developing countries: A meta- 2013. (Centre for Climate Change Economics and Policy
regression analysis. Review of Development and Economic, Working Paper; no. 150). 
2010; 14(3):640–655. 55. Felbermayr G, Groschl J. Natural disasters and the effect of
38. Djankov S, Murrel P. Enterprise restructuring in transition: trade on income: A new panel IV approach. European Eco-
A quantitative survey. Journal of Economic Literature, 2002; nomic Review, 2013; 58(February):18–30. 
40(3):739–792. 56. Ferreira S, Hamilton K, Vincent JR. Nature, Socioeconomics
39. Bijmolt THA, Pieters RGM. Meta-analysis in marketing when and Adaptation to Natural Disasters. Washington, DC: World
studies contain multiple measurements. Marketing Letters, Bank. 2011. (World Bank Policy Research Working Paper; no.
2001; 12(2):157–169. 5725).
Meta-Analysis Versus Review of Literature 23

57. Fisker PS. Earthquakes and Economic Growth. Copenhagen, Hawaii, 2007. (University of Hawaii Working Paper; no. 07-
Denmark: Institute for Advanced Development Studies, 2012. 28).
(CREEMA Development Research Working Paper Series; 75. Noy I, Vu TB. The economics of natural disasters in a devel-
no. 01/2012).  oping country: The case of Vietnam. Journal of Asian Eco-
58. Gaiha R, Hill K, Thapa G, Kulkarni VS. Have Natural Dis- nomics, 2010; 21(4):345–354.
asters Become Deadlier? [Letter]. Manchester, UK: Univer- 76. Patt AG, Tadross M, Nussbaumer P, Asante K, Metzger M,
sity of Manchester, Brooks World Poverty Institute, 2013; Rafael J, Goujon A, Brundrit G. Estimating least-developed
57:181. countries’ vulnerability to climate-related extreme events over
59. Ghimire R, Ferreira S (2013) Economic Shocks and Civil the next 50 years. Proceedings of the National Academy Sci-
Conflict. Southern Agricultural Economics Association ence, 2010; 107(4):1333–1337.
(SAEA) Annual Meeting, February 3–5, 2013. Available at: 77. Pielke RA Jr., Downton MW. Precipitation and damaging
http://ageconsearch.umn.edu/bitstream/142587/2/ghimire%20 floods: Trends in the United States, 1932–97. J Clim, 2000;
and%20ferreira_013.pdf, Accessed October 14, 2013.  12(20): 3625–3637. 
60. Heger M, Julca A, Paddison O. Analysing the Impact of 78. Raschky P, Schwindt M (2009) Aid, Natural Disasters and
Natural Hazards in Small Economies: The Caribbean Case. the Samaritan’s Dilemma. Washington, DC: World Bank,
Helsinki, Finland: UNU/WIDER, 2008. (UNU/WIDER Re- 2009. (World Bank Policy Research Working Paper; no.
search Paper; no. 2008/25).  4952). 
61. Hsiang SM. Temperatures and cyclones strongly associated 79. Raschky P, Schwindt M. Aid, Catastrophes and the Samari-
with economic production in the Caribbean and Central tan’s Dilemma. Clayton, Australia: Monash University, 2011.
America. Proceedings of the National Academy of Science, (Monash University Working Paper). 
2010; 107(35):15367–15372.  80. Rasmussen TN. Macroeconomic Implications of Natural Dis-
62. Hsiang SM, Narita D. Adaptation to cyclone risk: Evidence asters in the Caribbean. Washington, DC: IMF, 2004. (IMF
from the global cross-section. Climate Change Economics, Working Paper; no. WP/04/224). 
2012; 3(2):1250011–28.  81. Sadowski NC, Sutter D. Fatalities and hurricane damages: Are
63. Kahn ME. The death toll from natural disasters: The role of in- safer hurricanes more damaging? [letter]. South Economic
come, geography, and institutions. Review of Economics and Journal, 2005; 72(2):422–432. 
Statistics, 2005; 87(2):271–284. 82. Schumacher I, Strobl. Economic development and losses due
64. Keefer P, Neumeyer E, Plümper T. Earthquake propensity to natural disasters: The role of hazard exposure. Ecological
and the politics of mortality prevention. World Development, Economics, 2011; 72:97–105. 
2011; 39(9):1530–1541. 83. Simmons KM, Sutter D. Tornado warnings, lead time, and
65. Kellenberg DK, Mobarak AM. Does rising income increase or tornado casualties: An empirical investigation. Weather Fore-
decrease damage risk from natural disasters? [letter]. Journal casting, 2008; 23(2):246–258. 
of Urban Economics, 2008; 63(3):788–802. 84. Skidmore M, Toya H. Do natural disasters promote long-run
66. Kim C. The effects of natural disasters on long-run economic growth? Economic Inquiry, 2002; 40(4):664–687.
growth. Michigan Journal of Business, 2010; 41:15–49. 85. Strobl E. The economic growth impact of hurricanes: Evi-
67. Loayza N, Olaberria E, Rigolini J, Christiaensen L. Natural dence from U.S. coastal counties. Review of Economics and
disasters and growth: going beyond the averages. World De- Statistics, 2002; 93(2):575–589.
velopment, 2012; 40(7):1317–1336. 86. Strobl E. The economic growth impact of natural disasters in
68. McDermott T, Barry F, Tol RSJ. Disasters and development: developing countries: Evidence from hurricane strikes in the
Natural disasters, credit constraints, and economic growth. central American and Caribbean regions. Journal of Develop-
Oxford Economic Papers, 2013(November):1–24.  ment Economics, 2012; 97:130–141.
69. Mendelsohn R, Emanuel K, Chonabayashi S. The Impact of 87. Tavares J. The open society assesses its enemies: Shocks, dis-
Climate Change on Global Tropical Storm Damages. Wash- asters and terrorist attacks. Journal of Monetary Economics,
ington, DC: World Bank, 2011. (World Bank Policy Research 2004; 51(5):1039–1070.
Working Paper; no. 5562).  88. Von Peter G, von Dahlen S, Saxena S. Unmitigated Disas-
70. Murlidharan TL, Shah HC. Catastrophes and macro- ters? New Evidence on the Macroeconomic Cost of Natural
economic risk factors: An empirical study. Proceedings of Catastrophes. Switzerland: BIS, 2012. (BIS Working Papers;
the Conference on Integrated Disaster Risk Management: no. 394).
Reducing Socio-Economic Vulnerability, 2001. Laxenburg, 89. Vu TB, Hammes D. Dustbowls and high water, the economic
Austria: International Institute for Applied Systems Analysis impact of natural disasters in China. Asia Pacific Journal of
(IIASA). Sport and Social Science, 2010; 1(December):122–132.
71. Neumeyer E, Plümper T, Barthel F. The political economy of 90. Yamamura E. Death Caused by Natural Disasters: The Role
natural disaster damage. Global Environmental Change, 2014; of Ethnic Heterogeneity. Brussels, Belgium: Economics and
24(1):8–19.  Econometrics Research Institute, 2011. (EERI Research Pa-
72. Nordhaus W. The economics of hurricanes and implications of per Series; no. 10/2011). 
global warming. Climate Change Economics, 2010; 1(1):1–20. 91. Yamamura E. Institution, Economic Development, and Im-
 pact of Natural Disasters. Munich, Germany: Munich Personal
73. Noy I. The macroeconomic consequences of disasters. Journal RePec Archive, 2011. (MPRA Paper; no. 32069). 
of Development Economics, 2009; 88(2):221–231. 92. Yang D. Coping with Disaster: The Impact of Hurricanes on
74. Noy I, Nualsri A. What Do Exogenous Shocks Tell Us About International Financial Flows, 1970–2002. B E Journal of Eco-
Growth Theories? [Letter]. Honlolulu, Hawaii: University of nomic Analysis & Policy, 2008; 8(1-Advances):Art.13.

You might also like