• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • CommentGo Back
 
This twa-part article reviews the current literature on journul peer review. Research on thissubject bus grown during the 1980s and 1990s and bus increased our awareness of both themyths and facts about
peer
review. Part 1 summarizes research,findings on the participants inthe system (the appointment mechanisms of editors and referees, and reviewer tasks andqual$ications) and systemicproblems of‘reliability accuracy, and bias. Part 2 describes currentresearch on how fraud, .favoritism, and self-interest may affect the review system and on suchpolicy issues as interfrence qfparticularistic criteria; connections among editor.s, authors, andrcferees; und double-blind review. Althou~h the literuture indicates thnt peer review has manyproblems, the author concludes that it is dif/icult to imagine how science could advance withoutsuch a key quality control mechanism.
Peer Review for Journals asIt Stands Today-Part 2
JUAN MIGUEL CAMPANARIO
Universidad de Alcalú
In thefirst purt of this article, which appeared in the March 1998 issue ofScience Communication (Volumc 19, Number 3), 1 reviewed t-esearch ind-ings about the participants in the peer review system and various problemsrelated to reliability, accuracy, and bias. The second part summarizes currentresearch findings about fraud, favoritism, and self-intercst in peer review,bcginning with what happens when manuscripts are rejected. In addition, thereview examines what is known (and not known) about othcr important topicsand policy issues, about how particularistic critcria may affect the reviewprocess, and about therole played by professional connections among editors,
Author k Note: The author thanks Jcrry Keller for his help in the writing of the article and theScience Communication referees
for
their conunents. Address correspondence to Juan MiguelCCampanario, Grupo de Investigación en Aprendizaje de las Ciencias, Departamento de Física,Universidad de Alcalá, 28871 Alcalá de Henares, Madrid, Spain; phone: 34-91-88.54926; fax:34-91-8854942; e-mail: fscarnpanario@akala.es.ScienceCommunication,
Val. 19 No. 4, June 1998 277-306
0
1998 Sage
Publications, nc.
277
 
278SCIENCE COMMUNICATION
authors, and referees. 1 close with consideration of the research on double-blind review.
Reconsideration of Rejected Manuscripts
What happens when authors get rejection letters? A common myth is thata rejection decision is always irrevocable, perhaps because most journals donot have formal appeal mechanisms or describe in their guidelines how anauthor might attempt to reverse a negative decision. The Luncet was the firstjournal to appoint an ombudsman, whose task is “to record and, wherenecessary, to investigate episodes of alleged editorial maladministration”(Horton 1996, 6). However, in psychology, for example, only three of thirtyeditors surveyed by Hartley (1988) responded positively to the question: “Doyou have a formal appeals procedure to resolve possible disputes betweeneditors and authors?” Another eighteen editors stated that they rely oninformal procedures to deal with above-mentioned conflicts. Actually, bymeans of these informal procedures, and contrary to al1 odds, some authorsmanage to convince editors to reverse their decisions concerning the rejec-tion. On occasion, editors must deal with resubmissions, even though authorshave been told in strong language that a manuscript is of no Publishing value(Neal 1994). When authors of rejected manuscripts complain to editors, thisaction can terminate with a reiteration of the original verdict, with a newrefereeing process, or even with a request to the authors to revise and resubmitthe manuscript.Simon, Bakanic, and McPhail(l986) followed the fate of rejected manu-scripts as well as authors’ reactions when a complaint was lodged to the editorof the Ameritan Sociological Review. Sixty percent of the time, the editorrerouted the paper to new reviewers; in 22 percent of the cases, the originaldecision was reiterated without a new refereeing process. Thirteen percent ofauthors who complained were fortunate enough to have the decision reversed,and their manuscripts were accepted for publication. Authors with doctoraldegrees from Ivy League universities were overrepresented among those whotook their complaints to the editorial boards. Interestingly, 27 percent ofauthors who complained had served as referees for the journal during theprevious year, while these referees accounted for 16 percent of submissions.Hence, referees were more likely to complain about other referees’ decisions.This finding may indicate that they knew the journal and the system.From a survey of two groups of sixteen and eighty-six medical journals,Weller (1991) concluded that 62 percent of editors in the first group hadsubmitted rejected papers to a new round of reviewing in favor of the authors
 
Campanario /PEER REVIEW 279
who complained about the final verdict. Editors in the second group didthat only 14 percent of the time, and they contacted authors to explain theverdict 69 percent of the time. Alfred Soffer (1987), editor of Ches& foundthat about 50 percent of rejected authors who engaged in a dialogue withreviewers had their manuscripts accepted for publication.Most leading journals assume that authors of rejected manuscripts willattempt to have them published at any cost, and therefore they offer counselabout where to submit a rejected manuscript (e.g., see the latest guidelines toauthors of Science and Nature; also see Eichom and Van den Boss 1985;Hernon, Smith, and Croxen 1993, table 4; Stossel 1985).
Fraud, Fantasy, and Mistakes
Referees often do not discover main errors in fraudulent papers. Accordingto Stewart and Feder (1987), there were twelve errors per paper in eighteenmanuscripts written by John Darsee and published in major biomedicaljournals. Instances of scandals involving plagiarized papers or papers withfabricated data are publicized on an almost weekly basis in the news sectionsof leading journals such as Science and Nature, and the inability of the peerreview system to detect and prevent fraud seems to be proven beyond doubt.Consider, for example, the controversy involving a paper charged of beingfraudulent, which was published in the February 1994 issue of the Germanjournal Angewandte Chemie. According to the submitted article, a staticmagnetic field can induce chemical reactions in favor of chemical speciesover the other complementary chemical entity or “enantiomers.” After clear-ing the usual peer review process, the paper was published, but it wasretracted when evidente against the results began to accumulate. Accordingto one prestigious chemist, the results were too good to be true (Clery andBradley 1994); however, referees had not detected the mistake.In the behavioral sciences, some classic experiments have proved to beclassic blunders because referees did not discover gross mistakes in manu-scripts published by eminent psychologists (García 1981). Armstrong (1996)cites an experiment done by J. B. Watson on conditioning of behavior, whichwas based on data from a “sample” of one baby. Descriptions of the experi-ment kept appearing in textbooks without referente to the failed replications.One controversia1 economics study of social security contained a majorprogramming error; its correction led to results that were the opposite of thoseoriginally published (Pressman 1994, n. 5).According to Gingold (1973), the cumulative number of publicationsdealing with anomalous water (“polywater”) reached 500. Some theoretical
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
You must be to leave a comment.
Submit
Characters: ...