You are on page 1of 5

How to Write an Influential Review Author(s): Michael L. Rosenzweig, Jerrold I. Davis, James H.

Brown Source: Bulletin of the Ecological Society of America, Vol. 69, No. 3 (Sep., 1988), pp. 152-155 Published by: Ecological Society of America Stable URL: . Accessed: 10/06/2011 12:05
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at . JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at . . Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact

Ecological Society of America is collaborating with JSTOR to digitize, preserve and extend access to Bulletin of the Ecological Society of America.

and program di rectors to be conservative. How should a review be fashioned so that it is taken at face value and not discounted? What sort of influence should it try to exert? Surely a review should seek to identify and encourage the most promising and innovative research. National Science Foundation. who errs. Most of us respond whenever we can. How do we write a review that will convince panel members and program directors that a is an emi proposal. and easiest to criticize. panel members. worthwhile risk. The credit to the (one) who is actually in the belongs arena. 1800 G Street N. its programs. above all. But wouldn't ithave been a tragedy ifNatural Se are afraid of mistakes. as reviewers. but. we use this influence It is not much help if unwisely. A one-page description of the ac tivities of the Division of Biotic Svstems and NSF 87-40. at least fails while daring greatly. because it is one of the most impor tant ways that we contribute to the develop ment of our science. or our review is not taken seriously. Often we are asked for our anonymous writ ten opinion of a colleague's research proposal or manuscript. who at the best knows in the end the triumph of high achievement. for all its rough edges. HOWTOWRITEAN INFLUENTIAL REVIEW It is not the critic who counts. BSR has the same Washington. its telephone number is (202)357 address. Yet it is precisely the newest ideas that are the least tested. it is nec If essary that we take risks and actively en courage the development of new concepts. and comes short again and again. Evolution and Systematics tion to the staff members of NSF's Division of Biotic Systems and Resources. NSF 83-57. with relevant NSF publications can be obtained by writing either to the relevant program or to the Forms and Publications Unit. The following are especially useful: NSF 86-20. we must strive to identify and express our enthusiasm for new ideas and innovative ap proaches.?Theodore Roosevelt lection had been rejected because Darwin founded it upon incorrect mechanisms of in heritance? we are to advance our science. the great devotions. an exciting adventure nently with a high payoff if it succeeds? The Re search Support Liaison Committee in Ecolo took this ques gy. is there a tendency of reviewers.. who strives valiantly. and methods. Room 232. Our re views have enormous influence on the direc tion and rate of progress inour discipline. and insist that our peers write airtight proposals. With funds scarce relative to need and the average quality of proposals very high. is the work really likely to produce any surprises or major new discoveries? We have to be honest and point out potential problems. Such peer review is the by which we. DC 20550. The way we phrased it reflects a nagging perception that we shared with them. If we. 7332. Guide to Programs. history reveals that new theories often are incomplete and often do contain serious errors. then who will dare to we require that If tackle the difficult questions? a proposal be so well described that we can visualize every aspect of the research. whose face ismarred by dust and sweat and blood. the com primary mechanism munity of scientists. This is the most useful single source of information on support available throughout the entire Foundation.Some Sources of Information Resources telephone and numbers. Contains everything you need to know about preparing a proposal. Fiscal Year 1988. theories. the most controver sial. who knows the great enthusiasms. and who at the worst.W. affect the distribution of limited grant funds and journal pages. avoid risks and look for grounds to criticize a proposal? Is any fault or loose end likely to prevent fund ing?Must we avoid negative comments inour 152 . Grants for Research and Edu cation in Science and Engineering. In fact.

panels are often swayed by the negative comments. This iswhy it is essential to be broad minded. They may be full of glowing adjectives. should be fund scientists were the same questions to a journal editor. subtracting the value of each counterfeit penny without noticing that they are coming we accept that even from a solid gold box. Furthermore. This is a fact. Neg ative reviews are often full of well-reasoned objections. pernicious. and why the investigators are well qualified to do the work. in truth. but they rarely contain the details and logical arguments that would give them substance. but it is. it is a fact that proposals that receive low scores or average less than 2 (very good) have a difficult time. whereas serious errors will usually be detected and corrected. Remember that NSF panels and ad hoc re viewers are drawn from a wide spectrum of our peers. and explain why they have not put you off. We know no reason why it has to be so. the more detail ithas.we feel a proposal reviews if ed? from BSR The answers unanimous. What else can we ex pect? If NSF is to discount negative com ments about promising but risky and even somewhat it must have flawed proposals. This may sound fair. If there are three pages of negatives. good reasons for doing so. Scores should reflect the priority that we place on supporting a particular program of re search. Many reviewers seem to think it is their pri mary responsibility to discover and call atten tion to all the flaws in a proposal or manu script. spend your time on that one. but it is. Just because we recognize the merits of a proposal does not mean that other reviewers will appreciate them. Point out weaknesses. State why the problem is important." what do you think the panelists notice? Their overall negative impression cannot be re versed. what contribution the proposed research will make. more con structive positive reviews. too. We need to remember that it ismuch more damaging to our discipline to suppress an im portant contribution than to fund or publish a questionable piece of research. Give the program director (or editor) a good excuse to follow your advice. Negativity The Correlation Between in Reviews Detail and The more negative the tenor of a review. either by the investigator before publication or by the scientific community soon afterward. then we should be able to resist the tempta Ifwe tion to dwell on the negative aspects. Faced with positive reviews lacking sub stance and with well-documented negative criticisms. New ideas and conflicting data cannot have any influence un less they are developed. Itmakes us into petty bookkeep 153 . So the Re search Support Committee felt they should be shared. not the number of nits that can be picked from the way the proposal iswritten. novel. You cannot counter that fact with an introductory or concluding paragraph. Positive reviews are more often brief statements of approval. Perhaps they have the attitude that it should be deemed worthy until proved oth erwise. then we should place more emphasis on the positive aspects of a good proposal and write longer. or interesting. adopt the attitude that a proposal does not deserve funding unless the research isdaring. The overall impression your review makes de pends upon the proportion of criticisms that are positive. We all know what a long. Ifyou really think the pro posal is that good. It is time to produce some long. Believe it. and to consider the potential importance of a piece of research as well as to search for flaws. When you encounter a good proposal (or MS. Regulate the Proportion of Positive Comments Do not undermine what is intended to be a positive review by devoting more attention to criticism than to supporting comments. The score assigned by reviewers to NSF proposals is also very influential. and could be addressed the answers would be similar.). Accentuate the Positive ers. Describe in detail what makes it good. negative review looks like. Avoid hyperbole and be specific. and the first sentence says "This is a manifestly im portant proposal and should be fully funded. fill the review with your reasons and mention the negatives briefly. If the best science?and especially science that has not yet been done?must be imperfect. As much as panel members and program officers try to read the reviews carefully. positive ones.

to write such reviews. or program director is driven to sympathize with the victim. ers have expressed similar concerns. and to promote independence and that originality. The panel will then be even more con vinced of your positive opinion because they see you are taking so much time to help. however. For example. feel they must entertain their read ers with a rapier wit. A. But we cannot con vince other scientists (or the public. course. The earth pulses with fascinating ecological and evolutionary questions. This may mean that if you and you alone noticed the flaws. Biologists. send the details directly to the author. and point out in your review that you have done so. consider whether itwould have been wise to discourage R. and acid cuts. whose criticisms are often published and who earn their keep from them. aim of inward. Usually the attacks are oblique. the panel will have little difficulty making an evaluation. Dorigan of the Department of Energy has observed that when scientists are under attack. we should learn to write shorter negative reviews. piece). The physicists aim outward at their op ponents. low blows are against the rules. on the other hand. thus calling attention to proposals that are solid but unexciting and unlikely to Ifother review result insignificant advances. "I have my doubts as to the validity of Mr. But even if Iam correct. Serious criticisms and substantial concerns should al but this can be done in ways be expressed. style. dispassionate language without indulging in ad hominem assaults. Many appear intellectually challenging as those facing any other scientific discipline. such ag gressive attacks leave a lasting impression of on your part. to have decided that criticism is a written ver sion of prizefighting except that inboxing. MacArthur as soon as he made the illogical jump from resource-use overlaps to compet itive alphas? Negative Reviews Unfortunately. wagon-train Simultaneously.Inwriting a review. or of the importance of our government officials) we seem to be calling each other in work if competents. Their weapons. all too many negative sci entific reviews seem to have been written by put-down artists. Dr. Leon Wieseltier calls it "aggression as an intellectual instru ment. It is very important. but it The editor. we would all do well to bear inmind the difficulties we would have in trying to write a proposal to support the most exciting of our own research." And just to be sure that you realize Darwin is hardly the only example. panelist. Moreover. pierc ing put-downs. Darwin's ideas on the process of heredity. Robert Reich has written innovation is largely a process "Technological of imagining radical alternatives towhat is cur 154 . The questions are as The above is not intended to suggest that we should endeavor to be less critical. and threatens with environmental concerns. This is not only cruel and cowardly (at least the literary critic signs his minimizes their influence as well. or give golden fleece awards to projects whose titles make them easy targets. they circle round."1 If you feel compelled to note every flaw. Fisher's researches because his models lacked the of genetic drift. at each other. Critics of art and literature. The panel needs help ensuring that critically short funds are not wasted on pedestrian proj ects. Pol iticians ask what direct. Simply admit that you found nothing particularly exciting or nov el. are disparaging reviews and negative comments. your criticism will be ignored. Why should anyone want to in vest in a bunch of incompetents? Inorder to convey a more accurate impres sion of the value of our collective labors. The answers are es sential to deal with the environmental prob lems that beset the world. Darwin's proposal to visit the Galapagos Islands remains extremely valuable. Would you have component suppressed Eldredge and Gould's work on punctuated equilibria because they first in sisted that punctuation had to be connected to speciation? Would you have retired R. caustic comments. and an expensive one at that. H. we all need to make a conscious effort to tolerate ideas and unconventional diverse ap proaches. but reported them intemperately. Janet V. Mr. a prescient reviewer of Darwin might have written. and Ihave written to him indetail about these doubts. unprofessionalism The Payoff to Our Whole Science Basic science inAmerica is often under at tack as a social luxury. immediate benefits to expect from our work.

Sih and Nishikawa 1988) has been di rected toward the question of the equality. Norton Miller. 2 The New Republic. NY 14853 and James H.H. Brown This may accurately characterize many hu man activities. Dave Allan. attention on these pages and elsewhere (Cole and Zuckerman 1987.rently accepted. It is up to us. Jim Lawrey. James T. 3 VIII 1987:32. American Society of Zoologists. the community of to ecologists. The limitations peer review system has obvious and imperfections. A somewhat more subtle commentary on the acceptance and importance of E & E scientists 155 . Society of Systematic Zoology). AZ 85721 Jerrold I. and degradation of the envi ronment. Bailey Hortohum Cornell University Ithaca." Thus it can thrive only if dissent is tolerated. American So ciety of Plant Taxonomists. we must encourage that dissent and emphasize the ad vances it will make possible. but it is the best means that scientists have devised to evaluate each others' work. Dick Root. Doug Gill. This is the first contribution of the Research in Ecology.1987) views I& II. "We have met the enemy and he is us. 16 XII 1987:42. which is in its very essence mutualistic and collaborative. NM 87131 1 The New Republic. A famous cartoon character (Pogo) once said. U. New York. John Brooks. Rosenzweig Department of Ecology and Evolutionary Biology University of Arizona Tucson. directed by Dr.S. New York. National Science and Re Foundation. in publication rates for academic women in comparison to their male col leagues. Ecological Society of America. Elizabeth Wells. American Society of Limnology and Oceanography." Poston. Rick Vari.Davis L. The ability that women have to publish pa pers is both a major component and a reflec success in entering these tion of women's male-dominated fields in large numbers. or lack of it. Acknowledgments The Division of Biotic Systems sources. and systematists. The following members of the Committee contributed to its planning or discussed an earlier draft of it. Callahan suggested the main idea of this editorial. the rapid advancement of our discipline. USA.Pushcart Press. Bill (1986. Evo Support Liaison Committee lution and Systematics (Constituent societies: American Society of Naturalists. Botanical Society of America. Sih and Nishikawa (1988) found evi that statistically signif dence that suggested icant differences did not exist between men and women incurrent (1986) publication rates within the disciplines of ecology and evolution (henceforth E & E). provided freely of their time to discuss the issues with the Committee. Dr. Jane Lub chenco.2 Inour reviews. Loehle 1987. Walter Eanes. But itcannot be permitted to be true of the scientific enterprise. war.or both: Drs. Larry Slo bodan. evolutionists. Larry Pomeroy. Michael L. Muriel Department of Biology University of New Mexico Albuquerque. Steve Green. such as ethnic and religious prejudice. Society for the Study of Evolution. But notice that even critics often make fools of themselves with professional Rotten Re negative reviews: Henderson. TO OF DIFFERENCESINTHE PROPORTION WOMEN TO MEN INVITED GIVESEMINARS: ISTHEOLD BOYSTILLKICKING? Recently. use the peer review system carefully and wise Only then can itserve the goal we all share: ly. Paul Risser. It is in the public domain.