See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/330511771

Research synthesis and meta-analysis: A step-by-step approach, Harris

Cooper (Ed.), 5th ed.) . Sage Publications, Inc., Thousand Oaks/California
(2016). xviii þ 360 pp.

Article  in  System · January 2019

DOI: 10.1016/j.system.2017.12.012

CITATIONS READS

0 751

1 author:

Fei Li
China University of Geosciences
4 PUBLICATIONS   1 CITATION   

SEE PROFILE

All content following this page was uploaded by Fei Li on 30 July 2020.

The user has requested enhancement of the downloaded file.

248 Book reviews / System 72 (2018) 240e253

methodological differences used to investigate language contact: the latter work dominantly with data generated in natu-
ralistic settings.

References

Block, D. (2003). The social turn in second language acquisition. 3240 Prospect Street, NW, Washington, DC: Georgetown University Press.
May, S. (Ed.). (2013). The multilingual turn: Implications for SLA, TESOL, and bilingual education. Abingdon: Routledge.

Ursula Lanvers
Department of Education, University of York, YO10 5DD United Kingdom
E-mail address: ursula.lanvers@york.ac.uk

https://doi.org/10.1016/j.system.2017.12.011

Research synthesis and meta-analysis: A step-by-step approach, Harris Cooper (Ed.), 5th ed.) . Sage Publications, Inc.,
Thousand Oaks/California (2016). xviii þ 360 pp.

Over recent decades, using meta-analytic methods for research synthesis has become an indispensable force in moving the
field of social science forward complementing primary research. This is not uncommon in applied linguistics where we
specialize (the first author as an experienced teacher-researcher of SLA; the second author as a PhD candidate of L2 in-
struction). However, given the non-mathematical background of most social scientists, it is often hard to find a compre-
hensive yet accessible book that incorporates conceptual and technical orientations in conducting meta-analysis. Harris
Cooper's fifth edition of Research Synthesis and Meta-Analysis is such a timely addition to addressing the issues for both
emerging and experienced social science researchers.
Our initial contact with this book was in a meta-analysis course taken over half a year ago. This textbook has succeeded in
offering introductory and procedural knowledge for novice meta-analysts like us. Following the seven-step model themed
throughout the book, we initiated and completed our respective research syntheses. It is these positive and educational
experiences that prompted us to write this book review which we believe will be of particular value for new meta-analysts.
As a step-by-step guide book, its most appealing and helpful feature is the organization of themes. To start, the most
critical issues and the most liable mistakes are presented in the form of guiding questions before the discussion of each step
(Chapters 2e8). The questions can serve as a checklist of reference about solutions for novice researchers whenever they
encounter procedural problems, or constant reminders of following the methodological guidelines for experienced re-
searchers. With a big picture of the proper procedures of each step in mind, the author ensures a scientifically designed
synthesis process that leads to a cohesive and trustworthy report with maximum validity. In addition, the meticulously
designed hands-on exercises following the discussion of each step provide readers with opportunities to put the techniques
into practice. More importantly, readers can reflect on their mastery of the procedures ensuring that each step has been
learned and operated appropriately. The combination of guiding questions and hands-on exercises clearly strengthens the
conceptual knowledge presented in each step. For example, in Step 1 (Chapter 2), readers are invited to think about the
question of whether conceptual definition is clearly defined before proceeding to the main text. Meanwhile, they are
encouraged to further their understanding by analyzing the conceptual definitions of two meta-analyses of the same domain
in the exercise. Further, a specially designed and extremely helpful section of this book is the presupposed warnings of
procedural variations that might produce differences in conclusion in each step. These warnings alert readers of the upcoming
pitfalls and prepare them to deal with all procedures cautiously in order to obtain a trustworthy conclusion. For example, in
Step 2 (Chapter 3), Cooper cautions that “variation in searched sources might lead to systematic differences in retrieved
research” (p. 61). This warning primes readers with the pros and cons when making a decision as to whether to include
fugitive literature. In our case, we managed to minimize the potential quality problems that would have threatened the
validity of our conclusions by excluding fugitive literature in our own research.
Another impressive feature is the breadth of the book, which equips readers with a full picture of what they need to know
in conducting research synthesis. Specifically, both basic and advanced techniques are covered, ranging from formulating a
problem to writing up the report. In terms of the basic techniques, the discussion on literature search (Step 2 in Chapter 3) in
particular inspired us and updated our previous beliefs. We originally did not recognize personal contact with researchers as
an important potential channel to search the literature. However, Cooper's discussion of researcher-to-researcher channel led
us to realize that we could not only obtain relevant studies through contacting researchers working on similar problems but
also seek suggestions from them. Following all the possible channels, we succeeded in including as many studies as we could
in our syntheses. In addition, the basic techniques of report writing called “MARS” illustrated in Table 8.2 and Figure 8.1
 Book reviews / System 72 (2018) 240e253 249

(which are both newly added to this edition) in Step 7 (Chapter 8) are typical examples of checklists for synthesists to follow.
Their comprehensiveness has indeed saved us much trouble in presenting a convincing and valid research report.
Besides the basic techniques, the book also touches on some new and advanced techniques that are indicative of the trends
of the field. For example, in contrast to the 4th edition (Cooper, 2009), a new study-level measure of effect called “I2” is
incorporated in this book. Introduction about this indicator is useful for meta-analysists because they can use it to detect
heterogeneity of effect sizes from primary studies, based on which they may make better informed decisions as to whether to
use fixed effects model or random effects model. Moreover, four advanced approaches to meta-analysis emerging in recent
years and the research trend of cumulating results across meta-analyses are also introduced. While these topics give readers a
glimpse of the latest developments of research synthesis, they involve more advanced and complex statistical calculations.
These approaches are more valuable for experienced meta-analysts since they require more profound and comprehensive
statistical knowledge.
As emerging meta-analysists in SLA, we found all our expectations were met in this book. In particular, this book stands
out from many other similar texts with several merits. First, the clear and accessible discussion of all the techniques without
too many equations makes this book very easy to understand. In contrast, the handbook by Cooper, Hedges, and Valentine
(2009) was written in a more advanced manner by over forty authors from five distinct disciplines and contains too many
formulas and contents not necessarily applicable in social science. Second, the 7-step model is a complete pattern to follow in
conducting research synthesis. Comparatively, the work by Borenstein, Hedges, Higgins, and Rothstein (2009) mainly dis-
cusses the techniques of computing the effect size with the software called Comprehensive Meta-analysis (CMA), with little
advice on the other crucial steps in carrying out a research synthesis such as formulating a research problem, collecting
information, and presenting the results. Third, the discussion of the theoretical principles related to the pertaining practical
techniques contributes to a rich understanding of why a certain decision is made and what can be done to make it better.
However, some other texts on literature review, like Littell, Jacqueline, and Vijayan (2008) and Petticrew and Roberts (2006),
are limited to introducing little background knowledge and largely confined to procedures instead of providing compre-
hensive theoretical guidance.
While we believe this book to be indispensable for both novice and expert users, we had a few minor concerns. First, some
important technical terms such as data point, moderator, and meta-regression are not adequately explained, which may cause
difficulty in dealing with the procedures among novice researchers since they might not possess the required knowledge of
these terminologies. It would be more helpful to facilitate the understanding of the different effect sizes calculated if addi-
tional conceptual explanations of the technical terms could be provided. Second, the statistical calculation of effect sizes is
largely conceptual rather than procedural. While benefiting much from the conceptualizations, we found ourselves some-
times lost in the actual calculation process. It would be preferable to have examples using software programs such as CMA,
RevMan, STATA, or SAS included with illustrations.
Overall, this comprehensive guide book is especially valuable for novice researchers interested in meta-analysis. Its
informative discussion of the seven steps of conducting research synthesis not only offers readers a complete model to follow,
but also reminds them of the differences in conclusions caused by procedural variations in each step so that they can avoid
typical pitfalls and mistakes during the process. Besides the basic techniques to fulfill our personal research, the book is also
highly appreciated for its description of the theoretical principles behind the procedures. In spite of some minor limitations,
this book is a most exceptional guide for emerging meta-analysists who “possess only an introductory background in basic
research methods and statistics” (p.16) and a valuable reference for the experienced synthesists to improve their skills.

References

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
Cooper, H. M. (2009). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.
Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York: Russell Sage.
Littell, J. H., Jacqueline, C., & Vijayan, P. (2008). Systematic reviews and meta-analysis. New York: Oxford University Press.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Massachusetts: Blackwell Publishing.

Yansong Feng
School of Foreign Languages, Xuzhou Institute of Technology, Xuzhou, Jiangsu, 221008, PR China
E-mail address: fys720424@126.com
Fei Li*
School of Foreign Languages, Huazhong University of Science and Technology, Wuhan, Hubei, 430074, PR China
 Corresponding author.
E-mail address: fl_403@hust.edu.cn

https://doi.org/10.1016/j.system.2017.12.012

View publication stats