The Agony and the Ecstasy— The History and Meaning of the Journal Impact Factor
Presented by Eugene Garfield Chairman Emeritus, Thomson ISI 3501 Market Street, Philadelphia PA 19104 Fax: 215-387-1266 - Tel. 215-243-2205 firstname.lastname@example.org www.eugenegarfield.org at International Congress on Peer Review And Biomedical Publication Chicago, September 16, 2005
I had considered as an alternative title for my talk “Citation Sanity and Insanity -- the Obsession and Paranoia of Citations and Impact Factors.” Others might have preferred “Uses and Abuses of Impact Factors.” Origins of the Impact Factor I first mentioned the idea of an impact factor in Science magazine in 1955.1 That paper is considered the primordial reference for the concept of the Science Citation Index. Five years later, we began the experimental Genetics Citation Index project which led to the publication of the 1961 Science Citation Index. In 1955, it did not occur to me that “impact” would one day become so controversial. Like nuclear energy, the impact factor is a mixed blessing. I expected it to be used constructively while recognizing that in the wrong hands it might be abused. Since Current Contents, no less SCI, did not exist, it would have been precocious indeed to contemplate the influence of the nascent impact factor. In the early 1960s, Irving H. Sher and I created the journal impact factor to help select journals for the new Science Citation Index (SCI). To do this we simply re-sorted the author citation index into the journal citation index. From this simple exercise, we learned that initially a core group of large and highly cited journals needed to be covered in the new SCI.
2 SLIDE 1: TOP JOURNALS SORTED BY ARTICLES PUBLISHED IN 2004
TOP JOURNALS SORTED BY NUMBER OF ARTICLES, 2004
Abbreviated Journal Title J BIOL CHEM P NATL ACAD SCI USA BIOCHEM BIOPH RES CO J IMMUNOL BIOCHEMISTRY-US J VIROL J AGR FOOD CHEM CANCER RES J NEUROSCI BLOOD NUCLEIC ACIDS RES CIRCULATION FEBS LETT NEUROSCI LETT J CLIN MICROBIOL TRANSPLANT P CLIN CANCER RES BRAIN RES J UROLOGY ONCOGENE Slide 1
Total Cites 405017 345309 64346 108602 96809 74388 27992 105196 93263 97885 66057 115133 54417 25138 35117 9048 23585 58204 39589 45546
Impact Articles Factor 6.355 10.452 2.904 6.486 4.008 5.398 2.327 7.690 7.907 9.782 7.260 12.563 3.843 2.019 3.439 0.511 5.623 2.389 3.713 6.318 6585 3084 2312 1793 1687 1464 1261 1253 1233 1206 1160 1129 1112 1101 1090 1070 1052 1037 1029 1003
In Slide 1, we see the top 20 life science journals sorted by the number of articles published in 2004. Journal of Biological Chemistry published 6,500 articles last year.
MOST-CITED JOURNALS.570 6. So we created the journal “impact factor. we also recognized that smaller but important review and specialty journals might not be selected if we depended solely on total publication or citation counts. However.218 3.000 times last year – this includes citations to any articles in its entire history.690 9.903 7. slide 2 shows the list of journals most-cited in 2004.486 2.182 10.105 28.713 12.355 32.2 We needed a simple method for comparing journals regardless of size or citation frequency.563 4.389 21.452 31. 2004
Abbreviated Journal Title J BIOL CHEM NATURE P NATL ACAD SCI USA SCIENCE J AM CHEM SOC PHYS REV LETT PHYS REV B NEW ENGL J MED ASTROPHYS J J CHEM PHYS CELL LANCET CIRCULATION APPL PHYS LETT J IMMUNOL J GEOPHYS RES CANCER RES BLOOD BIOCHEMISTRY-US J NEUROSCI Slide 2. The JBC was cited over 400.075 38.839 7.853 6.008 7.782 4. Total Cites 405017 363374 345309 332803 231890 229765 185905 159498 144264 138693 136472 126002 115133 112516 108602 105601 105196 97885 96809 93263 Impact Articles Factor 6.308 6.3 SLIDE 2: MOST-CITED LIFE SCIENCE JOURNALS 2004 In contrast.907 6585 878 3084 845 3167 3575 4964 316 2478 2772 288 415 1129 3731 1793 2085 1253 1206 1687 1233
431 3725 44.713 316 79 35 84 80 878 845 33 168 288 130 351 191 26 19 138 415 30
Slide 3 shows the life science journals ranked by impact factor. to describe both journal and author impact. it is my 1972 paper in Science on “Citation Analysis as a tool in journal evaluation.557 14671 33. The transplant surgeon Tom Starzl has co-authored over 2. especially in Europe.831 49529 24. I bring
. We used a quarterly issue of the 1969 SCI to identify the most significant journals of science.000 articles.4 While my 1955 paper is considered primordial for citation indexing history.586 88864 24. Note the appearance of small review journals. Journal impact factors generally involve relatively large populations of articles and citations. Individual authors.” that has received most attention from journal editors.4 SLIDE 3: LIFE SCIENCE JOURNALS SORTED BY IMPACT FACTOR
TOP JOURNALS SORTED BY IMPACT FACTOR.5 That paper was published before the Journal Citation Reports existed. The term “impact factor” has gradually evolved.3 Over ten years ago.538 38657 31. It is one thing to use impact factors to compare journals and quite another to use them to compare authors.182 332803 31.695 363374 32.170 5957 32.515 159498 38.695 8093 23.918 9446 33.143 7800 22.355 126002 21. This ambiguity often causes problems.223 136472 28.389 14063 27.853 16487 31. on average.837 18169 22. produce much smaller numbers of articles although some are phenomenal. 2004 Abbreviated Journal Title ANNU REV IMMUNOL CA-CANCER J CLIN NEW ENGL J MED NAT REV CANCER PHYSIOL REV NAT REV MOL CELL BIO NAT REV IMMUNOL NATURE SCIENCE ANNU REV BIOCHEM NAT MED CELL NAT IMMUNOL JAMA-J AM MED ASSOC NAT GENET ANNU REV NEUROSCI PHARMACOL REV NAT BIOTECHNOL LANCET Slide 3 Total Impact Articles Cites Factor 14357 52.570 6618 36. I attended a celebration of Carl Djerassi’s 1000th paper.
which is the number of cites in the current year to any items published in the journal in the previous 2 years.” There are always exceptions to these generalities. which would give even greater weight to rapidly changing fields. While our analysis was based on a large sample of literature. Furthermore. Steve Lock aptly named the application of scientometrics to journals evaluation “journalology.6 The journal Scientometrics was started in 1978. Scientometrics and Journalology Citation analysis has blossomed over the past three decades into the field of scientometrics which now has its own International Society of Scientometrics and Informetrics (ISSI). They also cite all sorts of anecdotal citation behavior which do not represent average behavior. The JCR help page provides instruction for computing five-year impact factors. Over 15 years ago. 7. the annual JCR is not based on a sample. the rankings based on 1-. Nevertheless. The impact factor could just as easily be based on the previous year’s articles alone. The JCR today includes every citation that appears in the 5.000 plus journals that it covers. The half-life (number of retrospective years required to find 50% of the cited references) is longer for a physiology journal than that for a physics journal. Similarly. Hansen and Henrikson11 reported “good agreement between the journal impact factor and the overall [cumulative] citation frequency of papers on clinical physiology and nuclear medicine. I myself deplore the quotation of impact factors to three decimal places.or ten-year period. It matters very little whether the impact of JAMA is quoted as 21. Therefore.5 rather than 21.”7 All citation studies should be normalized to take into account variables such as the discipline. but then the measure would be less current.or 15-year impact factors do not differ significantly.455. citation density. the number of substantive articles (source items) published in the same 2 years. and half-life.10 seven years ago. A less current impact factor could take into account longer periods of citations and/or sources. Citation density (R/S) is significantly lower for mathematics journals than for molecular biology journals. The same can be said about alleged citation errors. These are all unified prior to issuing the JCR each
. Impact critics will usually find them. ISI uses three decimal places to reduce the number of journals with the identical impact rank. and the denominator. when journals are studied within disciplinary categories. discussions of sampling errors in relation to JCR are not particularly meaningful.9. A journal’s impact factor is based on two elements: the numerator. For some fields. When journals were studied across fields. most of which are really variants of one kind or another or do not affect impact since only variants in cited journal abbreviations matter in calculating impact. JCR’s two-year based impact factors may or may not give as complete a picture as would a five.5 this up for an important reason. but the rankings within the physiology category did not change significantly. the ranking for physiology journals improved significantly as the number of years increased. I reported on this in The Scientist. The citation density is the average number of references cited per source article.8.
will generally increase the number of “super-cited” papers. Such journals publish large numbers of items that are not substantive research or review articles. Furthermore. It is well known that there is a skewed distribution of citations in most fields. letters. Size vs. especially in the current year. thousands of other methodology and review papers do not. these must be shared by a larger number of cited articles. The average number of citations per article and the immediacy of citations are the significant elements. Incidentally.8 The assignment of article publication codes is based on human judgment. if at all. NEJM. rather. they don’t affect the calculation of their journal’s impact factor. the citation density and the age of the literature cited. A news story might be perceived as a substantive article. But we all know that they may be cited.6
year. And while a few classic methodology papers exceed a high threshold of citation. and tributes are not included in JCR’s calculation of source items (the denominator). Those that are affected change by 5 or 10%. obituaries. JAMA. and a significant letter might not be.the key determinants in impact are not the number of authors or articles in the field but. Nevertheless. but that is also why they don’t have a significant effect on the impact calculations. practice-based versus research-based articles. Nature. whereas some articles in small fields may have unusual impact. however. But only a small group of journals are affected. The Lowry paper was recently discussed in Journal of Biological
. interviews. Correspondence. no effort is made to differentiate clinical versus laboratory studies or.12 The size of a field. This assumption overlooks the fact that while more authors produce more citations. Publishing mediocre review papers will not necessarily boost your journal’s impact. The impact factors reported by JCR tacitly imply that all editorial items in Science. some distortion will result. Most articles in most fields are not well cited. etc. since they are all a decade or more old. can be neatly categorized. The well-known 80/20 rule applies in that 20% of articles may account for 80% of the citations. To reiterate -. for that matter. review papers on average are cited about twice the average. editorials. Citation Density There is a widespread but mistaken belief that the size of the scientific community that a journal serves significantly affects the journal’s impact factor. especially where they have cross-disciplinary impact. And a huge number of author errors or variants are corrected by the ISI system but unseen to the user. Nevertheless. All these potential variables provide grist for the critical mill of citation aficionados. consider this short list of super-cited papers in the life sciences. SLIDE 4: SUPER CITED PAPERS IN THE LIFE SCIENCES For your amusement. news stories. since the JCR numerator includes citations to these more ephemeral items.
55. ROSEBROUGH. MM
120. OH. M. GHS
31. F. N
SINGLE-STEP METHOD OF RNA ISOLATION BY ACID GUANIDINIUM THIOCYANATE PHENOL CHLOROFORM EXTRACTION ELECTROPHORETIC TRANSFER OF PROTEINS FROM POLYACRYLAMIDE GELS TO NITROCELLULOSE SHEETS . J
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA JOURNAL OF BIOLOGICAL CHEMISTRY JOURNAL OF MOLECULAR BIOLOGY
48. RANDALL. RJ LAEMMLI. STANLEY.671
FOLCH.PROCEDURE AND SOME APPLICATIONS A SIMPLE METHOD FOR THE ISOLATION AND PURIFICATION OF TOTAL LIPIDES FROM ANIMAL TISSUES DETECTION OF SPECIFIC SEQUENCES AMONG DNA FRAGMENTS SEPARATED BY GEL-ELECTROPHORESIS
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA ANALYTICAL BIOCHEMISTRY
63. They have been cited in at least 150.7 Chemistry13 but the authors failed to mention Lowry’s own commentary on this most-cited paper in the history of science.
BRADFORD. UK Title PROTEIN MEASUREMENT WITH THE FOLIN PHENOL REAGENT CLEAVAGE OF STRUCTURAL PROTEINS DURING ASSEMBLY OF HEAD OF BACTERIOPHAGE-T4 RAPID AND SENSITIVE METHOD FOR QUANTITATION OF MICROGRAM QUANTITIES OF PROTEIN UTILIZING PRINCIPLE OF PROTEIN-DYE BINDING DNA SEQUENCING WITH CHAIN-TERMINATING INHIBITORS
1945-July. P. AL. SACCHI.
MOST-CITED ARTICLES IN THE ISI WEB OF SCIENCE
Authors LOWRY. T. SSCI and A&HCI do include citations to books as well as individual chapters of books. NICKLEN.987
Source JOURNAL OF BIOLOGICAL CHEMISTRY NATURE
I have not included here super cited books such as Molecular Cloning: a Laboratory Manual by Maniatis and Sambrook which appeared15 in numerous editions beginning with 1982. J.14 Lowry himself noted that it was not his most important paper. This is my way of reminding those who are book authors.646
SOUTHERN. COULSON. AR
CHOMCZYNSKI. FARR. LEES. STAEHELIN. H. GORDON. 2005
Yr 1951 V 193 Pg 265 Hits 293. NJ. S.000 papers.179
73% 4.02% 0.887 1.995 3.603 2.614 4.6% 19.542 4.077 6.000-9.2% 10. Out of about 38 million source items about half were not cited at all.14% 0.00% 0.000-4. Therefore.999 900-999 800-899 700-799 600-699 500-599 400-499 300-399 200-299 100-199 50-99 25-49 15-24 10-14 5-9 2-4 1 Items Cited
% of WOS 0.064 2.343.999 2.000-3.06% 0.999 1.00% 0.025 343.00% 0.999 3.891.00% 0.000-1.1%
Total Items in File Slide 5
.8 SLIDE 5: CITATION FREQUENCY DISTRIBUTIONS
Citation Frequency Distribution 1900-August.000 5.01% 0.7% 16.000-2.00% 0.232 1.163.03% 0.01% 0.5% 24.762 2.557 27.952 3.319
For a more realistic view of citation frequencies.00% 0.931. 2005 (articles cited at least once) Approx # of Items Receive Citations 61 120 116 215 664 3.83% 10.7% 100.37% 1. slide 5 shows that from 1900-2005.789 19. about one half of one percent of cited papers were cited over 200 times.059 74. Keep in mind that “items” includes not only substantive articles but also ephemera mentioned earlier.106.769
Number of Citations >10.1% 11.000 4.269 953.637 12. these data provide a distorted picture for high impact journals where the number of uncited publications is much smaller.938.226.529 2.
Opthof17 showed how journal impact performance can vary from issue to issue. But in fact there rarely are two journals with identical semantic or bibliographic profiles. that should not significantly affect journal rankings. Some might argue that JCR categories are larger than necessary. I demonstrated that journal titles can be deceiving. the appearance of articles on the same subject in the same issue of a journal may have an upward effect. somewhat subjective methods for categorizing journals are by no means perfect. even though their specialists do use citation analysis to support their decisions. it is preferable to conduct item-by-item journal audits so that any differences in impact for different types of editorial items can be taken into account. some editors would like to see impacts calculated solely on the basis of their most-cited papers so that their otherwise low impact factors can be ignored. However. ISI’s heuristic. Citation analysis proved the Journal of Experimental Medicine was a leading immunology journal. references to articles that are no longer within the JCR two-year window will not be counted. For greater precision.18 Other objections to impact factors are related to the system used in JCR to categorize journals. We rely on the two-way citational relationships between journals to reduce the subjective influence of journal titles.
. The time required to referee manuscripts may also affect impact. In a perfect system it ought to be possible to compare journals with an identical profile. Recent work by Alexander Pudovkin and myself19 is an attempt to group journals more objectively. since most journals experience this skewness.20 It still is one of the five top immunology journals based on its impact factor.16 Alternatively.9
The skewness of citations is well known and repeated as a mantra by critics of the impact factor. Others would like to see rankings by geographic area because of SCI’s alleged English language bias. Three decades ago. On the one hand. Europhiles would like to be able to compare their journals by language or geographic groups especially in the social sciences and humanities. If manuscript processing is delayed.
179 3.327 2. Few would quarrel with the assignment of these journals to this category.617 3.590 3.580 2. GENERAL & INTERNAL Journals sorted by Impact factor
Abbreviated Journal Title NEW ENGL J MED JAMA-J AM MED ASSOC LANCET ANN INTERN MED ANNU REV MED ARCH INTERN MED BRIT MED J CAN MED ASSOC J AM J MED MAYO CLIN PROC MEDICINE ANN MED J INTERN MED AM J PREV MED CURR MED RES OPIN J GEN INTERN MED QJM-INT J MED EUR J CLIN INVEST PREV MED J PAIN SYMPTOM MANAG Slide 6
Impact Articles Factor 316 351 415 189 29 282 623 100 285 161 30 79 135 143 212 163 73 110 287 117
MEDICINE.114 3188 11. In Slide 6. you see the list of journals in the JCR category “Medicine.530 2.570 88864 24.” There are no surprises here. General and Internal.508 7.831 126002 21.821 2. but this tells us little about their actual subject content.200 26525 56807 6736 21000 6816 4255 2626 4793 3972 1148 4686 4073 4332 5372 2941 7.038 5.941 4.928 2.10 SLIDE 6: GENERAL INTERNAL MEDICINE CATEGORY SORTED BY IMPACT 2004.727 3.713 36932 13.746 3.187
. Slide 7 provides the general formula for calculating citation relatedness between two journals and the relatedness coefficient expressing the average of the maximum and minimum.11 SLIDE 7: CALCULATING RELATEDNESS COEFFICIENTS JCR recently added a new feature which provides you the ability to more precisely establish journal categories based on citation relatedness.
23 66. which reflects how often JAMA cites and is cited by each of the journals listed.85 70.25 39.20 30. but many journals have moved up in rank such as Journal of Family Planning and Journal of the American Geriatrics Society.89 31.90 30. General & Internal Category Journal JAMA-J AM MED ASSOC ANN INTERN MED NEW ENGL J MED ARCH INTERN MED J GEN INTERN MED √ √ √ √ √ √ √ √ CONTROL CLIN TRIALS ADV RENAL REPLACE TH MED CARE J FAM PRACTICE HEALTH AFFAIR J AM GERIATR SOC CURR CONTR TRIALS C ACAD MED INQUIRY-J HEALTH CAR CAN MED ASSOC J AM J MED AM J PREV MED √ √ √ √ √ √ √ √ √ ARCH PEDIAT ADOL MED CLIMACTERIC J AM MED INFORM ASSN MENOPAUSE PHARMACOEPIDEM DR S AM J MED QUAL ENDOCRIN METAB CLIN MATURITAS BLOOD PRESS MONIT FAM MED Rcoefficient 274.98 46. The top four journals related to JAMA remain the same as in the sort by impact.70 45.06 52.00 46.64 53.55 34.41 66.75 52.09 89.26 123.20 33.97 127.81 64.02 64. The checks on the left indicate the journal was not assigned to the General Medicine category.73 38.12 SLIDE 8: JOURNALS: JAMA . The relatedness coefficient takes into account the sizes of the journals involved (papers published) as well as the number of times each journal cites the other.89 33.28 34.37 40.16
Slide 8 is a list of the 20 journals most related to JAMA by the citation relatedness coefficient.26 69.RELATED JOURNALS SORTED BY CITATION RELATEDNESS COEFFICIENT
Slide 8 JOURNALS MOST RELATED BY CITATION RELATEDNESS TO JAMA √ = Not in Medicine.84 52.
85 64.72 21.79 22.96 37.11 36. we see differences that are quite striking.15 47. the next slide will demonstrate further that it is relevant to list it in the cardiology category. but the next two are cardiology journals.24 123. as are 9 of the next 12 journals shown. and Archives of Internal Medicine.13 SLIDE 9:
Slide 9 JOURNALS MOST RELATED BY CITATION RELATEDNESS TO NEW ENGLAND JOURNAL OF MEDICINE √ = Not in Medicine.99 33.66 40.62 36.46 22. Annals of Internal Medicine.97 24.66 24.64 45.77 37. JAMA. General & Internal Category Journal NEW ENGL J MED JAMA-J AM MED ASSOC ANN INTERN MED ARCH INTERN MED AM J MED √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ CIRCULATION J AM COLL CARDIOL MAYO CLIN PROC CHEST PROG CARDIOVASC DIS CAN MED ASSOC J CRIT CARE MED CURR PROB CARDIOLOGY J CARD FAIL EUR HEART J AM HEART J AM J CARDIOL AM J MED SCI MED LETT DRUGS THER RESUSCITATION BONE MARROW TRANSPL GASTROENTEROL CLIN N CURR OPIN CARDIOL MED CLIN N AM HEART Rcoefficient 345.31 35. as well.54
NEJM RELATED JOURNALS SORTED BY CITATION RELATEDNESS
Performing the identical exercise for the NEJM. The top four journals are there: NEJM.09 124.36 58. While this observation does not affect the categorization of NEJM as a general medicine journal.49 61.25 22.40 32.19 34.
.13 57.90 27.
563 35038 40841 10890 1497 12390 7618 14243 23887 6023 15028 29703 27826 1702 46 4258 1213 4023 1164 1327 9.271 3. The ranking by impact factor probably conforms to the general idea of the most prestigious journals in the field.972 9.716 4.
Slide 10 JCR Category: Cardiac & Cardiovascular Systems Source: 2004 Journal Citation Reports
Rmax JCR Rank Rank 3 12 2 4 16 13 14 7 17 6 15 5 19 11 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 8 18 10 9 16 17 18 19 20 Circulation
Abbreviated Journal Title
Impact Articles Factor 1129 340 591 250 53 269 163 356 652 314 327 824 654 45 73 205 79 220 118 31
115133 12.118 3.009 3.140 3.681 3.000 2.539 3.247 4.575 4.198 3.263 3.879 2.14 SLIDE 10: JCR CARDIAC & CARDIOVASCULAR SYSTEMS BY IMPACT FACTOR Here is the listing of the cardiac journals category in the 2004 JCR.967 2.813 2.676
Circulation Research Journal of the American College of Cardiology European Heart Journal Trends In Cardiovascular Medicine Cardiovascular Research Journal of Molecular and Cellular Cardiology American Heart Journal American J of Physiology-Heart and Circulatory Physiology Heart Journal of Thoracic and Cardiovascular Surgery American Journal of Cardiology Chest Basic Research in Cardiology European J of Cardiovascular Prevention & Rehabilitation Journal of Cardiovascular Electrophysiology Journal of Cardiac Failure Journal of Heart and Lung Transplantation European Journal of Heart Failure Progress in Cardiovascular Diseases
.133 6.796 2.
04 33.56 24.16 165.9 180.67 170.84 129.94 7.09 145.67 26.96 20.45 4.48 170.65 170.16 113.01 220. Using Circulation.21 46.01 220.74
160.84 129.28 159.97 65.56 112.29
160.09 5.54 10.25 115.89 6.03 24.25 115.31 7.85 35.29
160.16 71.16 165.56 139.73 128.56 39.95 105.95 105.9 180.80 51. this analysis also tells us something about the JCR placement of the journal Coronary Artery Disease.09 145.27 144.56 112. we find that NEJM ranked 7th among the most related journals in this field.81 33.7 14.03 124.45 6.69 156.57 14.48 19.94 125.8 123.77 164.35 124.9 21.41 33. using the JCR relatedness ranking method.69 156.83 119.21 14.94 125.16 118. to represent cardiology.28 159.54 11.71 70.45 8.83 33.01 41.36 55. some journals would be assigned to different JCR categories.77 164.06 169.9 31.54 27.68 12.44 6.27 144.14 142.6 159. JCR assigns it to the category “Peripheral Vascular Disease” but it is in fact the 10th journal in this list.73 128.76 11.56 5.56 139. Heretofore one could only guess at the proximity of NEJM to this or other topics.45 25.83 119.94 153.6 159.57 30.67 170.14 142.15
SLIDE 11: JOURNALS MOST RELATED TO CIRCULATION BY CITATION RELATEDNESS
JOURNALS MOST RELATED BY CITATION RELATEDNESS TO CIRCULATION Journal
CIRCULATION J AM COLL CARDIOL J CARDIOVASC ELECTR AM J CARDIOL EUR HEART J AM HEART J NEW ENGL J MED PROG CARDIOVASC DIS J CARD FAIL CORONARY ARTERY DIS CURR PROB CARDIOLOGY BASIC RES CARDIOL HEART PACE J AM SOC ECHOCARDIOG CARDIOLOGY CURR OPIN CARDIOL CARDIOVASC DRUG THER CATHETER CARDIO INTE J CARDIOV MAGN RESON J INTERV CARD ELECTR J NUCL CARDIOL EUR J HEART FAIL CLIN CARDIOL INT J CARDIOL J ELECTROCARDIOL INT J CARDIOVAS IMAG
160.16 85. Journal Performance Indicators
.8 123.16 113.07 40.40 41.76
Rank by Rcoefficient
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
However.09 48.61 173.31 47. However.00 28.68 32.45 50. the highest impact journal in this area.94 153.81 78.09 32.06 169.14 10.61 173.38 57.
349 39.5.57 35.79 54.148 45.84 19. the impact calculations are more precise.05 48.821 37. 1981 TO 2004
JAMA CITATION IMPACT (ALL ITEMS) IN ONE YEAR PERIODS 1981 TO 2004 Source: ISI Journal Performance Indicators file. Only citations to the substantive items are counted in the denominator.
.21 This annual compilation now covers the period 1981 to 2004.389 34.009 34.219 21.09 3.576 26.34 Citations 16.842 16.257 21.19 58.55 70.094 32.311 1.979 35.54 81.16
SLIDE 12: JPI DATA ON JAMA – CITATION IMPACT (ALL ITEMS) IN ONE YEAR PERIODS.968 30.921 7. Unlike JCR.040 18.76 44.688 30.791 18.26 35.83 47.16 58.174 Papers 551 573 554 618 526 504 597 620 627 656 31.48 65.99 60.5 Articles published in JAMA in 1999
Slide 12 Many of the discrepancies with journal impact factors are eliminated altogether in another ISI database called the Journal Performance Indicators (JPI).11 35. the database links each source item to its own unique citations.257 644 370 588 585 555 550 547 564 497 370 371 377 395 383 351
Citations received 1999-2004 = 84.291 20.20 84.908 32.53 40. 2004 Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Year 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Impact 29.436 24.358 22.70 48. the cumulated impact for JAMA articles published in 1999 was 84.372 31. For example.98 42.48 56. And it is possible to obtain cumulative impact measures covering longer time spans.257 citations received (from 1999 to 2004) by the 370 articles published in 1999. This was derived by dividing the 31.19 75.389 38.40 55.71 49. Therefore.
(5. Veter. the journals with high impact factors include the most prestigious.22 (http://incites. Mycol. SLIDE 13: MYCOLOGY JOURNALS EFFECT OF TIME ON IMPACT RANKINGS FOR ONE.5
In 1999.16) Lichenologist (5. (12.17) Lichenologist (1. Citations to these items were not included in the JPI calculation of impact.
EFFECT OF TIME ON IMPACT RANKINGS OF MYCOLOGY JOURNALS Ranks for one.90)
From: http://in-cites.46) Mycological Research (7. and 24 year period
Rank 1 2 3 4 5 6 7 8 9 10 2004 Impact Factor Fungal Genetics/Biol. FIVE.76) Fungal Genetics/Biol.87) Mycoses (0.43) Fungal Diversity (1.63) Mycopathologia (1.74) Medical Mycology (1. journal impact factors are also used by authors to decide where to submit their articles. consider the topic of mycology which was reported recently in inCites for April 25.53) Impact 1981-2004 Yeast (17. This example seems to contradict the generalization I made when discussing physiology journals. five. and 253 editorials. (3. While the journal Medical Mycology ranked 4th in 2004.05) Yeast (1.com/research/2005/april_25_2005-1.20) Mycological Research (3.45) Mycologia (1.95) Fungal Diversity (1. Some
.13) Lichenologist (0.13) Medical Mycology (4.73) Mycopathologia (0. In addition to helping libraries decide which journals to purchase.69) Impact 2000-2004 Fungal Genetics/Biol.53) Mycorrhiza (3.53) Experimental Mycology (14.html) inCites is a free ISI news bulletin.36) J.com/research/2005/april_25_2005-1.94) Mycorrhiza (1.19) Medical Mycology (6. 2005.81) Yeast (5.70) Mycologia (8.72) Mycorrhiza (7.89) Mycological Research (1. Med.257 370 Citations received 1999-2004 = Articles published in JAMA 84.87) Mycoses (1.16) Mycopathologia (6. AND 24 YEAR PERIOD. (9.html Slide 13
To illustrate the chronological changes in rankings for a group of related journals. JAMA published 1905 items of which 680 were letters.17 31. it moved to 3rd place when five years of data were used but 9th when 23 years of data were used. As a general rule.37) Mycologia (3. The perception of prestige is a murky subject.
when ISI prepares a personal citation report it provides data on the expected citation impact not only for a particular journal but also for a particular year. Bensman23 argued that this 2-year total citation count is a better guide to journal significance and cost-effectiveness than is the impact factor.”24 . But some studies would indicate that web sitation is a harbinger of future citation The assumption that the impact of recent articles cannot be evaluated in SCI is not universally correct. The Scientist magazine waits up to 2 years to select “hot papers” for commentary by authors.
. though there is little evidence that this is any better than traditional citation analysis. Journal impact can also be useful in comparing expected and actual citation frequency. Most of these papers will eventually go on to become “citation classics. the impact factor is used to estimate the expected influence of individual papers which is rather dubious considering the known skewness observed for most journals. This brings us full circle to the first slide I showed you on the most-cited journals.18 would equate prestige with high impact. one must distinguish between readership or downloading and actual citation in new research papers. However. full confirmation of high impact is generally obtained 2 years later. While there may be several years delay on some topics. Thus. Thus. when the author’s recent bibliography is examined. The use of journal impact factors instead of actual article citation counts to evaluate individuals is a highly controversial issue. but they are not the same as Citations. However. Web “sitations” may occur a little earlier. Presumably the mere acceptance of the paper for publication by a high impact journal is an implied indicator of prestige. because impact factors can change from year to year. And allegedly recently published articles may not have had enough time to be cited. Thus. papers that achieve high impact are usually cited within months of publication and certainly within a year or so. Granting and other policy agencies often wish to bypass the work involved in obtaining actual citation counts for individual articles and authors. the impact factors of the journals involved are substituted in lieu of the actual citation count. some librarians argue that the numerator in the impact-factor calculation is itself even more relevant. so it is tempting to use the journal impact factor as a surrogate evaluation tool. This pattern of immediacy has enabled ISI to identify “hot papers” in its bimonthly publication Science Watch. Typically. Today so-called “webometrics” are increasingly brought into play.
Klenk HD. Inst Virol. Univ Frankfurt. Berger A. Ctr Dis Control & Prevent. Natl Reference Ctr Trop Infect Dis. Cinatl J. Goldsmith CS. GA USA. WHO.html “A NOVEL CORONAVIRUS ASSOCIATED WITH SEVERE ACUTE RESPIRATORY SYNDROME” “IDENTIFICATION OF A NOVEL CORONAVIRUS IN PATIENTS WITH SEVERE ACUTE RESPIRATORY SYNDROME”
. Germany. Peoples R China. Urbani C. Off Director. France. D-6000 Frankfurt. Leduc JW. Med Clin 3. Atlanta. Rickerts V. Erasmus Univ. Shieh WJ. Natl Ctr Infect Dis. Bernhard Nocht Str 74. Div Viral & Rickettsial Dis. Hughes JM. Special Pathogens Branch. Ctr Dis Control & Prevent. GA USA. Escriou N. Vieth S. Bangkok. Hanoi. Grywna K. Schmitz H. Germany. Lim W. Rabenau H.com/hotpapers/2005/may05-cli. Panning M. Peret T. Infect Dis Pathol Act. From: http://in-cites. Sturmer M. D-20359 Hamburg. GA USA. Taipei. Ctr Dis Control & Prevent. GA 30333 USA.html
Two recent examples of Hot Papers published in JAMA and NEJM include papers on coronavirus at http://in-cites. Taiwan. Univ Marburg. Inst Pasteur. Van Der Werf S. Brodt HR. Rollin PE. Rotterdam. Paddock CD. Hong Kong. Guarner J. Hong Kong. Natl Ctr Infect Dis. Yang JY. Zaki SR. Singapore. Comer JA. Bellini WJ. Burguiere Am. Dept Pathol. Natl Influenza Ctr No France. San Francisco. Kramme S. D-3550 Marburg. Govt Virus Unit. GA USA. Univ Frankfurt. Div Bacterial & Mycot Dis. Ling AE. Manuguerra Jc. Natl Reference Ctr Trop Infect Dis. Rota P. Humphrey CD.19 SLIDE 14: EXAMPLES OF HOT PAPERS
Slide 14. Natl Ctr Infect Dis. Inst Med Virol. CA 94143 USA. Fouchier Ram. Queen Mary Hosp. Thailand. Germany. HOT PAPER: Citations: 475 Title: IDENTIFICATION OF A NOVEL CORONAVIRUS IN PATIENTS WITH SEVERE ACUTE RESPIRATORY SYNDROME Authors: Drosten C. Singapore Gen Hosp. Dept Virol. D-20359 Hamburg. Atlanta. Kolesnikova L. Atlanta. Osterhaus ADME. Natl Ctr Infect Dis. Ctr Dis Control & Prevent. Bernhard Nocht Inst Trop Med. Vietnam. Inst Virol. Atlanta. Gunther S. Dept Virol. Cox N. Ctr Dis Control. Univ Calif San Francisco. Ctr Dis Control & Prevent. Erdman D. Natl Ctr Infect Dis. Dept Hlth. Tong SX. Natl Ctr Infect Dis. Emery S. Eickmann M. Singapore.com/hotpapers/2005/may05-cli. Doerr HW Source: N ENGL J MED 348: (20) 1967-1976 MAY 15 2003 Addresses: Bernhard Nocht Inst Trop Med. Germany. Muller S. Atlanta. Influenza Branch. Paris. HOT PAPER: Citations: 515 Title: A NOVEL CORONAVIRUS ASSOCIATED WITH SEVERE ACUTE RESPIRATORY SYNDROME Authors: Ksiazek TG. Netherlands. Germany. Int Emerging Infect Dis Program. Preiser W. Fields B. GA USA. Dowell SF. Anderson LJ Source: N ENGL J MED 348: (20) 1953-1966 MAY 15 2003 Addresses: Ctr Dis Control & Prevent. Becker S. Atlanta. Derisi J. Resp & Enter Virus Brach. D-6000 Frankfurt.
garfield.pdf 5. pgs 5-6 (February. Experience has shown that in each specialty the best journals are those in which it is most difficult to have an article accepted.edu/histcomp/index-starzl. http://www. “Citation indexes to science: a new dimension in documentation through association of ideas. Volume 5. Garfield.issi-society.pdf
2. pgs.” Science 122(3159):108-11 (1955). a better evaluation system would involve actually reading each article for quality but then this entire congress is dedicated to the difficulties of reconciling peer review judgments. new full-text capabilities in the web make this more practical to perform.garfield. http://garfield. I have had to rush through a lot material to save time but hope that I have given you a balanced view of a complex and controversial topic. “A tribute to Carl Djerassi: Reflections on a remarkable scientific entrepreneur. We call this citation context analysis. http://www.upenn. Volume.1973). "Citation analysis as a tool in journal evaluation .html 4. therefore. 6. http://www. 527-544 (1977). a good technique for scientific evaluation. Hoeffel25 succinctly." Science. 178 (4060):471-479 (1972).library.pdf 6. http://garfield.library. The use of impact factor as a measure of quality is widespread because it fits well with the opinion we have in each field of the best journals in our specialty.upenn. E. "Choosing physiology journals.edu/essays/V1p527y1962-73.library.7.info/
.” Yes. Essays of an Information Scientist. Fortunately.” Current Contents (42): 5-14 (October 18.edu/essays/v5p721y1981-82. their judgment surely would be tempered by observing the comments of those who have cited the work. 721-730 (Philadelphia: ISI Press) (1983). most people do not have or care to take the time to read the articles any more! Even if they did. Brodman E.20
Conclusion Of the many conflicting opinions about impact factors.upenn. When it comes time to evaluating faculty.1. Garfield E.edu/essays/v6p468y1983. and these are the journals that have a high impact factor.library. pgs. Garfield E. Reprinted in:Current Contents No.upenn.1982) Reprinted in Essays of and Information Scientist." Bulletin of the Medical Library Association 32:479 (1960) 3. Most of these journals existed long before the impact factor was devised.
expressed the situation
“Impact Factor is not a perfect tool to measure the quality of articles but there is nothing better and it has the advantage of already being in existence and is.
6. http://garfield. “How well does journal “impact” work in the assessment of papers on clinical physiology and nuclear medicine?” Clinical Physiology 17(4):409-18 (1997). http://garfield.edu/commentaries/tsv12(14)p12y19980706. Henriksen JH.garfield.pdf
10. Philadelphia: ISI Press (1977).. 1977).library. Opthof T.” Current Contents No. http://garfield. “The most highly cited paper in publishing history: Protein determination by Oliver H.” The Scientist 12(14):12-3 (July 6. 3.pdf 11.library. Pudovkin AI and Garfield E. Yu D-R.MolecularCloning.upenn. Online at www. “Long-term vs.library. “Is the ratio between number of citations and publications cited a true constant?” Current Contents No.upenn. 2nd Edition.pdf 8. Hansen HB.” Scientometrics 64(2):235-246 (2005) 17.upenn.edu/essays/v2p419y1974-76.html
. Garfield E.upenn. b. 12.pdf 15. Wang X-Y. Volume 41.. 1976). (1989). « The influence of publication delays on impact factors. November 17. and Maniatis T." Presented at the American Society for Information Science and Technology Annual Meeting. “Long-term vs. 1998). “Journalology”: Are the quotes needed?” CBE Views 12(4):57-9 (1989). short-term journal impact (part II). Garfield E. 16. Published in Proceedings of the 67th Annual Meeting of the American Society for Information Science & Technology. 280(28):e25 (July 15. 6-7 (January 3. acceptance rate.edu/commentaries/tsv12(03)p10y19980202. pgs. Commentary by Lowry. "Algorithmic procedure for finding semantically related journals. Simoni RD. 14 . http://garfield.edu/papers/ranknormalizationasist2004published. 1998). OH on ”Protein measurement with folin phenol reagent.edu/essays/v13p019y1990. Lock SP.upenn. Fritsch EF. Molecular Cloning : A Laboratory Manual. Which medical journals have the greatest Impact? Annals of Internal Medicine. pgs. “Submission.21
7. “Introducing Citation Classics: the human side of scientific reports. "Rank-normalized Impact Factor: A way to compare journal performance across subject categories.edu/papers/pudovkinsemanticallyrelatedjournals2002. 419-421.library. 18.” Cardiovascular Research 41(1):1-4 (1999).pdf 19. Cold Spring Harbor: Cold Spring Harbor Press.library. Garfield E. pgs.library. Garfield E.pdf 13. a. http://www. 2005). 105(2):313-20 (1986). Lowry." Journal of the American Society for Information Science and Technology (JASIST) 53(13):1113-1119 (November 2002). 2004. Providence.edu/essays/v10p007y1987.upenn. Volume 2.” Current Contents/Life Sciences (1):7 (January 3. Sambrook J. RI. pgs.upenn.507-515 (2004) http://garfield. Available: http://garfield. 5-7 (Feb 9.library.edu/classics1977/A1977DM02300001.pdf
9. and Hill RL. 1977). Reprinted in Essays of an Information Scientist.com.upenn. Garfield E. Pudovkin AI. http://garfield. short-term journal impact: Does it matter?” The Scientist 12(3):10-2
(February 2. Garfield E. rapid review system and Impact Factor. Yu G.library. Kresge N.” Journal of Biological Chemistry.
http://scientific. www. Philadelphia: ISI Press (1977). pgs. pgs 5-8 (June 28.22
20.” Library Resources & Technical Services 42(3):147-242 (July.com/research/2005/april_25_2005-1. “Journals ranked by impact: Mycology. 3. ‘Journal Citation Studies.com/products/jpi/ 22. 1998). Hoeffel C. “Journal impact factors” [letter]. 26.html 23. Journal of Experimental Medicine compared with Journal of Immunology.org 25. 1998). Anonymous. “Scientific and technical serials holdings optimization in an inefficient market: a LSU serials redesign project exercise.thomson.” inCites (April 25. Garfield. Bensman SJ. 1972). 2005). 24. or how much of a clinician is the immunologist?’ Current Contents No. Volume 1. E.
. Allergy 53(12):1225 (December. Reprinted in Essays of an Information Scientist. 21. http://in-cites.citationclassics.