Impact Factor and Citation

Metrics: What do they Really

Mean?
Outline

  • Measurement of the Business of Science

• Background on various methods

• Where things can go wrong

• What that means for librarians

Why Measure?

• Security of knowledge

• Predictability

• Comparison analysis

• Subject and domain dependent

• Quantifiability fallacy
 Historical Background on Impact
Factor
Eugene Garfield, "Citation
Indexes for Science: A
New Dimension in
Documentation through
Association of Ideas,"
Science 122, 3159 (July
15, 1955): 108
 What is an Impact Factor
ratioof articles published to articles cited
during a rolling two-year window
A/B
A = citations in 2008 : articles cited in 2006-7
B = citable items published in 2006-7
year 2008 citations to 2006 + 2007 articles
articles published in 2006 + 2007
Your (real) Impact Factor

 
Use of Impact Factors
Collection development
Choosing journals for publication
Journal assessment/marketing by
publishers
Evaluation of scholarly research &
individual performance, for purposes of
tenure and promotion, and funding
Evaluation of departments, institutions,
and nations
Problems with IF
Two-year window
# of journals published in discipline
ISI coverage
Language
Publication type
Not representative
Journal size
 Other Issues

Citation errors
Citation clubs
Citation bartering
Manipulation

Review articles
Case studies
Editorial interference
 h-index
Hirsch, J. E. (2005).
"An index to
quantify an
individual's
scientific research
output". PNAS 102
(46): 16569–16572.
h-index
h-index, developed by Jorge Hirsch

A scientist has index h if h of [their] Np

papers have at least h citations each, and
the other (Np - h) papers have at most h
citations each.
Criticisms of the h-index
Comparisons
Age
Insensitive
Context
Limitations of citation databases
# of authors
From h to g
Given a set of articles ranked in
decreasing order of the number of
citations that they received, the g-index is
the (unique) largest number such that the
top g articles received on average at least
g citations.
What do the numbers really mean?

Author inaccuracies

ISI calculation inaccuracies

What does an H-index of 1 really mean?

Citation Game-Playing – How to
make it Work for YOU!
• Become famous so everyone gives you
authorship on their paper
• HEP in ArXve submission timing
• Ride the bias – multiple authors: choose the
middle
• Folia Phoniatrica et Logopaedica: 1 articles
cited all the papers published in the last 2
years for IF=1.439 compared to 0.655
• Find and ride the trend – Martin Fleischmann
was once the most highly cited chemist in
Britain
 

Nature 451 2008 pg 766-767

Search Web of Science for Wilfrid Laurier
Wilfrid Laurier as Author in 2007?

 
Wilfrid Laurier as Author in 2007?
Ann Arbor as Author (MUCH worse in
Google Scholar)
Ann Arbor as Author in 2007?
William H. Gates = Bill Gates
Researcher ID = A-1930-2009
Researcher ID Publication List

 
Anyone can write a letter

 
NATURE Vol 53  June
2008 pg. 718
 09-Feb-08
Position h-index patents total number first-year last-year Publications/year
of papers publication publication

president 5 0 13 1984 2008 0.54

dean science 15 0 51 1988 2008 2.55
assistant 5 0 14 1999 2007 1.75
assistant 5 0 6 1998 2007 0.67
assistant 15 2 48 1998 2007 5.33
assistant 6 0 13 1998 2007 1.44
assistant 7 0 12 1998 2007 1.33
assistant 9 0 14 2000 2005 2.80
CAS 3 0 4 1999 2002 1.33
CAS 10 33 25 1979 2007 0.89
professor 8 0 36 1967 2007 0.90
associate 7 0 17 1992 2005 1.31
CAS 0 0 2 2007 2008 2.00
CAS 17 0 24 1984 2001 1.41

Average 8 2.50 19.93 1992.21 2006.14 1.73

Chemistry Department Scopus H-Index

Created Feb 9, 2008 with the faculty at that time. Dependent on information included in
SCOPUS.
ISI Accuracy?

 
Sciencewatch.com

 
Institutional Metrics
http://webometrics.info
What does this mean for Librarians?
Must be familiar and knowledgeable about
the current calculations of Science
Worthiness

International students are aware

Administrations and granting agencies will

ask for these statistics but may not be aware
of their limitations

Avoid the citation culture

Take Home Message

Avoid Disregard Syndrome 

Practice Citation Vigilance