• Embed Doc
  • Readcast
  • Collections
  • CommentGo Back
 
-Have Referees Rejected Some of the Most-Cited Articlesof All Times?
Juan Miguel CampanarioGrupo de lnvestigaci& en Aprendizaje de las Ciencias, Departamento de Física, Universidad de Alcalá,28871
Alcalá
de Henares, Madrid, Spain. E-mail: fscampanario@alcala.esIn this artícle a quantitative study is reported on the resis-tance that scientists may encounter when they do innova-tive work or when they attempt to publish articles that laterbecome highly cited. A set of
205
commentaries by authorsof some of the most-cited papers of all times have beenexamined in order to identify those articles whose authorsencountered difficulty in getting his or her work published.There are 22 commentaries (‘10.7%) in which authors men-tion some difficulty or resistance in doing or Publishing theresearch reported in the article. Three of the articles whichhad problems In belng published are the most cited fromtheir respective jtimals. According the authors’ commen-taries, although sometimes referees’ negative evaluationscan help improve the articles, In other Instances refereesand editors wrongly rejected the highly cited articles.Introductíon
Sometimcs scicntists cncounter strong resistancefrom pccrs lo thcir ncw idcas. Thc scientific communityoften finds it dificult to acccpt ncw idcas or methods anduncxpccted observations ( Barber, 196 ). The greatestand most harmful source of resistance from scientists toscientific discovery comes precisely from those peerswhose mission is to preserve the quality of scientificwork: The editors and referees of scicntific journals. AI-most all scientific journals monitor the quality of sub-mitted papers by means of the advice of editorial boardmembers or externa1 referees: This is the so-called peerreview system.The peer review system has been frequently criticized.According to its critics, peer review system tends to “fa-vour unadventurous nibblings at the margin of truthrather than quantum leaps” (Lock, 1985), it may delaypublication (Garfield, 1986), it cannot always avoid du-plicate or fraudulent publication (Garfield, 1980; Gar-field & Welljams-Dorof. 1990; Peters& Ceci, 1982), it issometimes biased o positive results (Easternbrook, Ber-lin, Gopalan & Matthews, 1991; Epstein, 1990), it is setup in such a way that particularcriteria can interfere withselection ( Arrillaga, 1992; Cole & Bowers, 1973; Sham-blin, 1970; Willis & McNamee, 1990; Yoels, 1971; Yo-
topoulos, 196 I ), and it oRen allows
excellent manu-scripts to be criticized by refereeswith vested nterests orcontrary views (Biggs, 1990; Meadows, 1977; Rodman,1970). There is evidente that some of the referees
pass
their manuscripts on to a colleague for partial or totalreview (Glogoff, 1988; Lock & Smith,
199 I ) and referee
and editorial board members’ misconduct and conflict ofintercsts havc been noticed (Barinaga, 1992; Huth, 1992;Maddox. 1992). Redner made onc of thc most seriouschargcs against the peer review systcm whcn hc afflrmedthat “onc of the roles ofjournals almost appcars o be toshift out and reject really original contributions”(Redncr, 1987). Medical editor David Horrobin as-sertcd that “there is objective evidcnce that some rcfer-ees, and even some highly respected ones in top aca-demic positions, are at best ignorant and carelessand atworst deliberately obstructive” (Horrobin, 1974).There is a growing interest in the peer review systemand a number of investigations have been published onthis topic. Most articles dealing with the peer review sys-tem address he reliability of the system. Results of thesestudies point out that the agreement among referees svery low in the social and behavioral sciences,althoughin the physical sciences t tends to be greater (Ciccetti,199 ). However, the validity of the peer revicw systemhas not been studied in a systematic way, maybe becauseof the mcthodological and conceptual
dilficultics of suchan entcrprisc. The question here is whcthcr the peer re-view systcm achicves one of its goals: To promote origi-nal
idcas. valuahlc approaches, or ncw methods and torejcct thc mediocre oncs.
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE. 47(4) 302-310. 1996CCC 0002.6231/96/040302-09
 
chological research papers and the number of citationsreceived by these articles (Gottfredson, 1978). Medicaleditor, David F. Horrobin (Horrobin, 1990) presentedan astonishing set of anecdotical casesof importan& orig-inal, or innovative papers hat had been rejected by ref-erees. More cases may be found in Barber ( 1961) andMcCutchen ( 1991). lt is necessary o keep in mind thatsome scientists even had their findings tumed down al-most at once, although they would later win the NobelPrize for the very same research. Documented casesareSevero Ochoa’s work on polynucleoride phosphorylase(Ochoa, 1980); Hans Krebs’s account of the citric acidcycle (Horrobin. 1990 : Rosalind Yalow’s initial workon radioimmunoassay (Yalow, 1982); Harmut Michel’sresearch on photosynthetic processes Garfield, 1989~);Pavel Alekseevich Cherenkov’s article on the radiationnamed after him (Hubbell, 1991); Arthur Kornberg’swork on DNA synthesis (Kornberg, 1989); HenryTaube’s work on inorganic complexes (Gray and Col-lman, 1983;Taube 1988); Sydney Altmant’s findings onribozymes (Baum, 1989; Lewin, 1989); Murray Gell-Mann’s work on quarks (Crozon, 1987); K. Aleti MÜII-er’s and J. Georg Bednorz’s work on high-temperaturesuperconductivity (Combes$?t, 1988); Gerd Binning’sand Heinrich Rohrer‘s work on the scanning tunnellingmicroscope (Armstrong & Hubbard, 1991); and Wil-liam A. Fowler’s work on nucleogenesis (Maddox,1983). i!However, the above examples and others reported inmore or Iess anecdotical litcrature. are not systematicand quantitativc studics on pecrsrcsistance to impor-tant scientihc findings. To till this void 1have uscd a ncwapproach to addrcssing this topic. In short. it consists ofthe use of thc author’s commentarics on important andhighly cited papcrs. Thcsc commcntarics may be foundin the Citation ClassicsH caturc
of Currm Con/ws.
According to thc abovc. this articlc is an attempt toassesshe validity of thc peer revicw. Thc spccific objcc-tives of this articlc are:1) To study he ditficulticsencountcrcdby the authorsof a very selectset of articlcs, ranking among thcmost-citedonesof all times, whcn trying to publishthe highlycited articlcs.2) To compute
hc
rclative rcquencyof thesedifficul-lies.3) To analyze,whcn possiblc, he rcasonsprovidcd byreferees nd editors o advisceditors o rcject he pa-pers.Citation data on articlcs and journals are computedby thc lnstitute for Scicntitic lnformation (El) and canbe obtained from thc .~‘C~¡(VICI~.;/u/ion II&V (SC1 ). Thisindex records citations to previous articlcs givcn eachyear in the 3,500 mosl inllucntial journals (Seglcn,1992). Citations have bccn widely uscd by sociologists ofscicnce as a tool in stutivlng thc structure ol‘ scicnce(Rice, Borgman, ,Bedna&i, & Hart, 1991). Citationrates correlate well with the award of prizes and the inde-pendently expressedopinions of scientists regarding sci-entific merit. There is a vast literature on this topic( Abrams, 1991). Sometimes, citation rates may even de-termine the salariesof researchers Diamond, 1986 . Ci-tation data have also been used n academic evaluationsof individual scientists (Garfield, 1983a, 1983b King,1987; Seglen, 1992).Although the typical article is cited an average of 15times (data from the foreword to the Citation ClassicsRfeature of Current
Conrenrs,
1991 ), the actual distribu-tion of article citations are found to be very skewed. Dis-tribution of article citations within a tield conforms rea-sonably well to linearity in a double-log plot: A few pa-pers are highly cited and many articles are rarely cited(Magyar, 1973; Seglen, 1992). Thus, according to Ham-ilton, 55% of 1981 papers were uncited 5 years after pub-lication and in social sciences he uncitedness ates aver-aged 75 and 98% in arts and humanities, respectively(Hamilton, 1990, 1991). 1 must point out that, accord-ing Bott and Hargens, the uncitedness is lesser han theamount reported by Hamilton (Bott and Hargens,1991). Two final data: Only 0.05% of the over 32 millionarticle which were cited at least once between 1945 and1988 were cited more than 500 times (Garfield, 1990a).From
1977
to
199 1, Currenr Cmlenfs
featured aweekly Citation ClassicsR section where commentariesfrom scientists who had written highly cited articles werepublishcd. From 1992, Citation Classicswere publishedbi-weekly. A Citation ClassicsHpapcr is an article whichis cxtraordinary for thc largc numbcr of citations it hasreccivcd. Thc sclection critcria for nominating an articleas a Citation ClassicsKcandidatc are divcrse and can befound elscwhere (Garfield. 1984. 1989a. 199Ob). lf anauthor conscnts to writing this commcntary, the articlebecomes a Citation ClassicsK. In his cssay, the authorexplains the work, dctails what prompted the research,thc contributions of their coauthors and, sometimes, theobstaclesencountered in both research and publication.It has been said that Citation ClassicsRare “the humanside of scicnce” (Garfield, 198 ). By the end of 1993,close o 5,000 commentaries had been published in
Cur-rení C’onlcnls
(Garfield, 1993 . This interesting databasehas becn used n some studies on the sociology ofscience(Astin, 199 1; Cano and Lind, 1991; Chubin, Porter,Rossini, 1984; Garfield, 1989b, 1990~; Schulz-DuBois,1984), and it can be used to study history ofscience. ASGarlicld pointed, Citation ClassicsK ommentaries”pro-vidc rescarchersand studcnts ofscience with behind-the-scencs inlòrmation aboutscicntific endeavours”(Garíicld. 1993. p. 7 ).MethodsCitatlon (‘lassicsK commentarlcs 13~ utliors ofa frac-
tion 01‘ thc 300 rnost-citcd
;~r-l~clcsII thc tiistor!, 0f‘ sC¡-
JOURNAL OF THE AMERICAN SOCIETY FOA INFORMATION SCIENCE-April1996303
 
ence have been examined. The list of these 400 articleswas published in a series of essaysby Eugene Garfield(Garfield, 1990a, 1990b, 199Od, 1991). An importantfi-action of these highly cited articles are CitationClassicsR and, therefore, their authors’ commentarieshave been published in
Current Contents.
Referentes tothese
Current Contents
issues was also published in theaforementioned list. Among the 400 most-cited articlesof al1 imes, 188 articles were Citation ClassicsRwhenGarfield’s essays n the 400 most-cited articles appeared.In addition, 1 have reviewed al1Citation ClassicsRcom-mentaries published in
Current Contents
in every issue
sitlce Garfield’s essays isted the 400 most-cited articlesofall times. This wasdone to identify additional CitationClassicsR commentaries published that correspond tothe list of the 400 aforementioned most-cited articles. 1have found 17 additional commentaries. 1 have read al1these commentaries to identify articles whose authorshad difficulties in doing the research or in getting thempublished.Most Citation ClassicsR commentaries were. locatedin
Current Contents
issues iled in libraries from the fiveuniversities in the Madrid area and in some research n-stitutes in Madrid. Howevei, the lnstitute for Scientificlnformation provided some Citation ClassicsR ommen-taries published in old
Current Confents
issues hat werehard to find in the. Madrid area.To study thc di&icultiesencountered by individual pa-pers 1have used a classification schehe consisting in twobroad categories:
a) Authors cncountcrcd difficultics during thc rcvisionphasc
but
thc manuscript was finally publishcd in
thc
samc journal to which thc manuscript was submittcd.On somc occasions. authors had to makc changcs orfullill refcrccs’ suggcstions or rcqucsts for additionaldata. On olhcroccasions. papcrs wcrc rcjcctcd but thcauthors argued or inquircd and Ihc papcr was linallyacccptcd.b) Manuscript wcrc rcjcctcd and authors had to look forncw journals 10 publish Ihcm.
1 must note that 1 have idcntified two papers whoseauthors encountered prepublicalion dificultics. Thc au-thors ofthe highly cited papers encountcred some minor
difficulties
before the publication process due to factorsother than refcrces. These instances have not bccn takenin account bccause hesc kinds of problems actually have
no relcvancc lo thc focus of this article.
Results
and
Dìscussion
Thcrc are 22 Citation Classics’ commrn~arics( 10.7%) that mcntion somc dificulty or rcsist;lncc in do-ing or publishing thc rcscarch rcported in lbc art~clc. .fhc
citation frcqucncy Iòr
thc articlcs with problcms rangcfrom I .675 lo 0.390. It is worlh noting lhat. actor-dtng to
Gafield, the top 1,400 articles cited at least 1,000 timesrepresent just 0.004% of all cited publications in the1945-
1988 Science Citation Index
database (Garfield,1990a). Table 1 includes the bibliographic data on themost-cited Citation ClassicsRarticles which encountereddifficulties. According to the lists included in the afore-mentioned essaysby Eugene Gafield, three articles in-cluded in Table 1 would eventually become the most-cited of their joumals. Next, the articles in Table 1 areanalyzed in more detail. Data on the process of publica-tion of these papers were obtained from the ISl prologue
to each
Citation ClassicsR commentary and from thecommentary Asclf,
As
noted above, to study the individ-ual articles 1 have used a classification scheme consistingof two categories:a)
Difficulties during the
revision phase: In this cate-
gory 1 have listed those papers in which the
journal refer-ees raised questions and posed problems but the paperswere finally published. On some occasions, authors hadto make changesor fulfill referees’suggestionor requests for additional data.
-The referce of thc 1967 papcr by Bernard Hin on selectiveextraction of polyoma DNA from cell cultures sent him a sin-gle-spaccd three-page report in a very paternalistic tone. The
referee’seport raiscdmany questions
nd
requested additionaldata. As Hirt recognized. thc referee worked
hard and made
somc goods rcmarks. As ir often occurs. the manuscript wentinto a drawcr for two and a half months unGl. with thc hclp ofsomconc, Hirl rcwrolc it.-Thc 1959 articlc hy I lavcl. Edcr. and Bragdon dcscribcs anaccuratc and
cllicicr~t
mcthod for dctcrmining lipoprolcincomposition. Thc difiicultics cncountcrcd hy thc authors whcnthcy suhmirtcd thc manuscript to thc prcsligious Jottrml N/’C/irli~cl/ //11’<‘.s/;~~n/ro,r had
to
do with conccpts. At thai timethc journal did not acccpt tcrminology thai has sincc bccomcstandard jargon ofthc licld.-Thc origin of thc IY Stcrnbcrgcr and collcagucshighlycitcd articlc is an assignmcnt hc poscd to Stcrnbcrgcr’s stu-dcnts. Thc uncxpcctcd rcsutt was a ncw mcthod lo visualizeerythrocytcs and spirochctcs. Ludwig Stcrnbcrgcr and col-leagucs had somc difiiculty in Publishing the work bccause oneofthc referees fclt that thc findings wcre loo insignificant. How-evcr, thc mcthod rcportcd in thc artictc has widc applications inthc study of fixcd normal and pathologic tissuc. Thc principlcundcrlying this mcthod yiclds high scnsitivity bccausc ofinhcr-ently low background.-Two rcvicwcrs could no1 agr-CC as to thc acccptability of thr1967 papcr by Rcvcrly Murphy on mcthods to mcasurc stcroidsby competitivc protcin-binding radioassay. A third rcfcrcc
wus
involvcd and al1 threc had many suggcslions for rcvision. Thcrcvision of this papcr tool\ wcll ovcr a ycar and. according 10Murphy. this dcl;cy WIS h:umIùl bcc;~usc somc ~ollcagucs towhom shc had g~\‘cn thc mcthods publishcd carlicr.
304 JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE-Apvl1996
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
You must be to leave a comment.
Submit
Characters: ...