11-10-2016

Dissertation/Project/Internship Plan

2016_2017

Theme 3 – Bibliometrics
Sandra T. Silva

11 and 12 October 2016

Faculdade de Economia, Universidade do Porto

Syllabus - Bibiometrics

1. Bibliographic databases

2. Bibliometric indicators

3. Bibliometrics as a tool for literature reviews

Acknowledgment: Aurora Teixeira and Tefko Saracevic

2

1
 11-10-2016

Outline of the presentation

• Motivation and aims of the session

• Key associated concepts
• Bibliometrics as a tool
• Data, sources, variables & methods
• Laws of bibliometrics
• An example

Motivation – importance of the literature review

• The importance of the literature review for the scientific research:

acknowledgment of the cumulative nature of knowledge and the need to
assess the ‘state of the art’
• A rigorous literature review not only surveys what researches have been
done in the past on your research topic but it also appraises, encapsulates,
compares and contrasts, and correlates various scholarly books, research
articles, and other relevant sources that are directly related to your current
research

2
 11-10-2016

Motivation – importance of the literature review

• The research topic: must be stated as a well-defined question accessible to a

specific academic discipline

• A personal interest is not necessarily

significant to the larger research community
• Usable research must further
current knowledge in the field
through both a synthesis of
present knowledge and
an original contribution
extending that knowledge

https://carolinekuhn.wordpress.com/week-9-part-iii-
literature-review/, accessed on 05.10.2016 5

Motivation – importance of the literature review

• Selecting a Topic – Broad scan of the literature

Phase Main sources Purpose
To assist the
Reviews of the researcher in
Broad scan
literature identifying a
research problem

To develop a
proposal
Focused review Scopus; WoS
prospectus and
research proposal

To provide a
Comprehensive scholarly
All sources
critique foundation for
the study
6

3
 11-10-2016

Selecting a Topic – Broad scan of the literature

• Concentrate on Research Reviews - one of the best ways to conduct the

broad scan is to concentrate on research reviews

http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1468-2370 http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1
467-6419

The quality of research

The Thompson Reuters Impact Factor

• Frequency with which the "average article" in a journal has been cited in a
particular year or period
• A ratio between citations and recent citable items published
• The impact factor of a journal is calculated by dividing the number of
current year citations to the source items published in that journal during
the previous two years

https://jcr.incites.thomsonreuters.com/JCRJournalHomeAction.action?SID=A2-e4s3Af6HvgGfmZfnJEOfPDYYxx9F5jvJ0-
18x2dqq0k9SuuFxxca4tn0JGiq6Ax3Dx3D693D4dsx2BNkk83n9Ud2I3Vwx3Dx3D-9vvmzcndpRgQCGPd1c2qPQx3Dx3D-
wx2BJQh9GKVmtdJw3700KssQx3Dx3D&refineString=null&SrcApp=IC2LS&timeSpan=null&Init=Yes&wsid=Q2gEBXxFwquf12pU54G
8

4
 11-10-2016

The Thompson Reuters Impact Factor

The quality of research

The quality of research

The Thompson Reuters Impact

Factor

5
 11-10-2016

The quality of research

The Scimago Journal Ranking (SJR)

http://www.scimagojr.com/journalrank.php

• It expresses the average number of weighted citations received in the

selected year by the documents published in the selected journal in the
three previous years

11

The quality of research (SJR)

6
 11-10-2016

Rank Title SJR SJR Quartile

1 Quarterly Journal of Economics 20.761 Q1
2 Journal of Finance 14.546 Q1
3 Econometrica 12.485 Q1
4 Review of Financial Studies 9.925 Q1
The quality of research (SJR, 2015)

5 Journal of Financial Economics 9.92 Q1

6 Journal of Economic Literature 9.77 Q1
7 Review of Economic Studies 9.25 Q1
8 American Economic Review 8.048 Q1
9 Journal of Political Economy 7.646 Q1
10 Annual Review of Economics 7.012 Q1
11 Journal of Accounting and Economics 6.834 Q1
12 Journal of Marketing 6.612 Q1
13 Journal of Economic Perspectives 6.077 Q1
14 Journal of Marketing Research 5.764 Q1
15 Journal of Accounting Research 5.733 Q1
16 Journal of Labor Economics 5.631 Q1
17 Journal of Consumer Research 4.896 Q1
18 Review of Economics and Statistics 4.629 Q1
19 Accounting Review 4.478 Q1
20 Marketing Science 4.34 Q1
21 Entrepreneurship: Theory and Practice 4.24 Q1
22 Journal of International Business Studies 4.208 Q1
23 Journal of Monetary Economics 4.15 Q1
24 Quantitative Economics 4.134 Q1
25 Journal of the Academy of Marketing Science 3.861 Q1
26 Journal of Econometrics 3.781 Q1
27 Journal of International Economics 3.723 Q1
28 Annual Review of Financial Economics 3.629 Q1
29 RAND Journal of Economics 3.544 Q1
30 Economic Journal 3.39 Q1
Source: 31 Strategic Entrepreneurship Journal 3.377 Q1
http://www.scimagojr 32 Journal of Economic Growth 3.273 Q1
.com/journalrank.php 33 Journal of Financial Markets 3.233 Q1
34 Energy Economics 3.025 Q1

Motivation – importance of the literature review

The literature review:

• Helps you create a rapport with your audience

• Lists all the past scholarship

• Helps you avoid incidental plagiarism

• Sharpens your research focus within the historical context of the research

14

7
 11-10-2016

Motivation – importance of the literature review

• Clarify some key concepts

• Show how bibliometrics can be of great help in conducting a novel and

insightful literature review

• Comes in many different types:

• Qualitative

• Quantitative  bibliometrics

• Mixed

15

Key concepts

Bibliometrics
• “the application of mathematics and statistical methods to books and other
media of communication” [Pritchard, A. (1969), “Statistical bibliography or
bibliometrics?”, Journal of Documentation, 25(4):348-349]
• “… the quantitative treatment of the properties of recorded discourse and
behavior pertaining to it.” [Fairthorne, R.A. (1969), “Empirical hyperbolic
distributions (Bradford-Zipf-Mandelbrot) for bibliometric description and
prediction”, Journal of Documentation, 25(4): 319-343]
• “is a statistical analysis of books, articles, or other publications.” [OECD, in
http://stats.oecd.org/glossary/detail.asp?ID=198]

16

8
 11-10-2016

Key concepts

Scientometrics – the science of measuring and analysing science

• Pioneers

- Derek J. de Solla Price (1922-1983)

- Eugene Garfield (1925-…): creator of the Science Citation Index

• Methods

- Qualitative (peer review)

- Quantitative (bibliometrics)

- computational (mapping, etc.)

17

Key concepts

Scientometrics (cont.)

• Also used to study broader than books, articles …

• Covering science in general, not just publications

Related concepts

• Infometrics – covers all information objects

• Webmetria or Cybermetria – Web connections, manifestations using

bibliometric techniques to study the relationship or properties of different
sites on the web

18

9
 11-10-2016

Key concepts

Meta-analysis (≠ Bibliometrics)
• The use of statistical methods to combine results of individual studies
• It is focused on contrasting and combining results from different studies, in
the hope of identifying patterns among study results, sources of
disagreement among those results, or other interesting relationships that
may come to light in the context of multiple studies
• Often, but not always, important component of a systematic review
procedure
• Uses some procedures of bibliometrics in order to identify the relevant
studies

19

Key concepts

Meta-analysis (≠ Bibliometrics)

“A meta-analysis is a quantitative literature review method in which statistical

procedures are used to combine results from different studies investigating the
same research question. The aim is to identify patterns among results, sources
of disagreement or other interesting relationships that may come to light in the
context of multiple studies (Greenland & O’Rourke, 2008). In comparison with
traditional literature reviews, meta-analysis has the advantage of summarizing
the findings of the studies in a systematic way, thus eliminating subjectivity and
reducing the chances of making wrong interpretations and drawing misleading
review conclusions (Shadish, 1982).” in Neves, Afonso and Silva (2016: 386)

20

10
 11-10-2016

Bibliometrics as a tool

Reasons for quantitative studies of the literature…

• Analysis of structure and dynamics

• Search for regularities - predictions possible

• Understanding of patterns

• “Order out of documentary chaos”

• Verification of models, assumptions

• Rationale for policies & design

21

Bibliometrics as a tool

Reasons for quantitative studies of the literature…

• Science searches for regularities

• Success of statistical methods in social sciences

• Need for justification & basis for decisions

• Something can be counted – irresistible

22

11
 11-10-2016

Bibliometrics as a tool

Possible applications …

• History of science

• Sociology of science

• Science policy; resource allocation

• Library selection, policies

• Information organization

• Information management

23

Bibliometrics as a tool

Reviewing/critical synthesis of a given area of research

• Leite, D., Silva, S.T. and O. Afonso (2014), “Institutions, Economics and the Development
Quest”, Journal of Economic Surveys, 28(3): 491-515 [Institutions and Dev. Ecs.]
• Teixeira, A.A.C. (2014), "Evolution, roots and influence of the literature on National Systems of
Innovation: a bibliometric account", Cambridge Journal of Economics, Vol. 38 (1): 181-214.
[Innovation]
• Teixeira, A.A.C. and Mota, L. (2012), "A bibliometric portrait of the evolution, scientific roots
and influence of the literature on University-Industry links", Scientometrics, Vol. 93 (3): 719-
743. [Technology Transfer]
• Silva, S.T. and Teixeira, A.A.C. (2009), "On the divergence of evolutionary research paths in the
past fifty years: a comprehensive bibliometric account", Journal of Evolutionary Economics,
Vol. 19 (5): 605-642. [Evolutionary Economics]

24

12
 11-10-2016

Bibliometrics as a tool

Analysing the trends of a given area of research [based on a ‘seed’ journal]

• Teixeira, A.A.C. and Silva, M. (2013), "The intellectual scientific basis of science, technology and
innovation research", Innovation: The European Journal of Social Science Research, Vol. 26 (4):
472-490 [Research Policy]
• Du, Y. and Teixeira, A.A.C. (2012), "A bibliometric account of Chinese economics research
through the lens of the China Economic Review", China Economic Review, Vol. 23 (4): 743–762
[China Economic Review]
• Teixeira, A.A.C. (2011), "Mapping the (In)visible College(s) in the Field of Entrepreneurship",
Scientometrics, Vol. 89 (1): 1-36 [7 journals in Entrepreneurship]
• Castro e Silva, M. and Teixeira, A.A.C. (2011), "A bibliometric account of the evolution of EE in
the last two decades. Is Ecological Economics (becoming) a post-normal science?", Ecological
Economics, Vol. 70 (5): 849-862 [Ecological Economics]

25

Bibliometrics as a tool

Citation analysis
• Teixeira, A.A.C. (2014), “The internationalization of U.Porto's scientific production”, ch. 6 (pp.
235-308), in Pedro Teixeira (ed.), Percursos da internacionalização na Universidade do Porto -
Uma visão centenária, U.Porto 100, U. Porto Editorial, ISBN: 978-989-746-038-8.
• Teixeira, A.A.C. and Sequeira, J. (2011), "Determinants of the international influence of a R&D
organisation: a bibliometric approach”, European Journal of Scientific Research, Vol. 53 (3):
400-430.
• Vieira, P.C. and Teixeira, A.A.C. (2010), "Are Finance, Management, and Marketing
Autonomous Fields of Scientific Research? An Analysis Based on Journal Citations",
Scientometrics, Vol. 85 (3): 627-646.

26

13
 11-10-2016

Bibliometrics as a tool

As the starting point for meta-analysis

• Neves, P.C, Afonso, O. and Silva, S.T. (2016), “A Meta-Analytic Reassessment of the Effects of
Inequality on Growth”, World Development, 78: 386-400 [Inequality and growth]

As the starting point for social network analysis

• Silva, S.T., Mota, I. and Grilo, F. (2015), “The use of game theory in regional economics: A
quantitative retrospective”, Papers in Regional Science, 94: 421–441 [Game Theory and
regional economics]

27

Data, sources, variables & methods

Bibliometrics - data

Governed by data available in documents or information resources in general

- that what can be counted:
• Author(s)

• Organization, country, language

• Source

• Journal, publisher, patent

• …

28

14
 11-10-2016

Data, sources, variables & methods

Bibliometrics - data
• Contents
• Text, parts of text, subject, classes

• Representation
• Citations
• To a document, in a document, co-citation

• Links
• Any other quantifiable attribute

29

Data, sources, variables & methods

Bibliometrics - sources

• Science Citation Index

• Scopus

• Google Scholar

• Compilation of variables from journals in a subject

• Publication counts from indexes, or other data bases

• Web structures, links

30

15
 11-10-2016

Data, sources, variables & methods

Bibliometrics – variables  authors

• Number in a subject, field, institution, country

• Productivity  ranks, assessments

• Collaboration – co-authorship, related networks

• dynamics - productive life

• papers/author in a subject

• mapping

31

Example – Silva, Mota and Grilo (2015)

Collaboration – co-authorship, related networks: Social Network Analysis exercise

Source: Silva, S.T., Mota, I. and Grilo, F. (2015), “The use of game theory in regional economics: A quantitative retrospective”, Papers
in Regional Science, 94: 421–441 32

16
 11-10-2016

Data, sources, variables & methods

Bibliometrics – variables  sources

• Concentration most often on journals

• Growth, dynamics, numbers (bibliometrics laws)

• Information explosion - exponential laws

• Time movements, life cycles

• Various distributions by subject, language, country

33

Data, sources, variables & methods

Bibliometrics – variables 
texts (Zipf's law)

• Distribution of words

• Words, phrases in various

parts

• Subject analysis,
classification Number of times ‘innovation’ mentioned in parliamentary debates

• Co-word analysis
Source: Perren, Lew and Sapsed, Jonathan (2010) Innovation as Politics:
The startling rise and reshaping of innovation in UK parliamentary
discourse 1960-2005 In: Technical Change: History, Economics and Policy:
A Conference in Honour of Nick von Tunzelmann , 29-30 May, Brighton,
UK, http://eprints.brighton.ac.uk/7565/1/Innovation_as_Politics.pdf

34

17
 11-10-2016

Data, sources, variables & methods

Bibliometrics – variables  citations

• Studied a lot; many pragmatic results

• Base for citation indexes, web of science, impact factors, co-citation

studies etc.

• Derived:

• Number of references in articles

• Number of citations to articles

• Bibliographic coupling

35

Data, sources, variables & methods

Bibliometrics – variables  citations

Citation matrix (example)

Source: http://www.scils.rutgers.edu/~tefko
36

18
 11-10-2016

Data, sources, variables & methods

Bibliometrics – variables  citations

• Co-citations

• Author connections, subject structure, networks, maps

• Centrality

• Of authors, papers

• Validation with qualitative methods

• Impact  Impact factor (IF) : increasingly being interpreted as a

fundamental indicator of the quality of journals

37

Data, sources, variables & methods

Bibliometrics – Sleeping Beauties

• “Why do some discoveries fade into obscurity while others blaze a new
trail the moment they are published?”

• “Why do some research papers remain dormant for years and then
suddenly explode with great impact upon the scientific community?” –
Sleeping Beauties (study from the Indiana University Bloomington School
of Informatics and Computing's Center for Complex Networks and Systems,
in the Proceedings of the National Academy of Sciences)

Source: http://news.indiana.edu/releases/iu/2015/05/sleeping-beauties.shtml

38

19
 11-10-2016

Data, sources, variables & methods

Bibliometrics – Sleeping Beauties

• "This study provides empirical evidence that a paper can truly be 'ahead of
its time’”
• "A 'premature' topic may fail to attract attention even when it is introduced
by authors who have already established a strong scientific reputation.“
Alessandro Flammini, corresponding
author on the study

Source: http://news.indiana.edu/releases/iu/2015/05/sleeping-beauties.shtml

39

Data, sources, variables & methods

Bibliometrics – Sleeping Beauties

Examples

• Study from the influential statistician Karl Pearson. His paper that was
published in 1901 in the journal Philosophical Magazine did not "awaken"
until 2002

• A seminal paper by Albert Einstein, Boris Podolsky and Nathan Rosen

(quantum theory), published in 1935, didn’t receive widespread citation
until 1994

• The top journals for the publication of sleeping beauties: Proceedings of

the National Academy of Sciences, Nature and Science
Source: http://news.indiana.edu/releases/iu/2015/05/sleeping-beauties.shtml
40

20
 11-10-2016

Data, sources, variables & methods

Bibliometrics – Sleeping Beauties

41
http://www.helsinki.fi/~pjojala/Impact-Factor-Immediacy_Index.jpg

Data, sources, variables & methods

Bibliometrics – Sleeping Beauties

42
http://www.helsinki.fi/~pjojala/Impact-Factor-Immediacy_Index.jpg

21
 11-10-2016

Source: Sleeping Beauties in International Economics and International Business, by Ângela Fonseca, MEGI Master
Dissertation

Data, sources, variables & methods

Bibliometrics – classical type of studies

• Comparative publications over centuries

• Number of journals founded over time

• Number of abstracts published over time

• National scientific size vs. economy size

• Bibliographic coupling and co-citation

• Web structures, links

44

22
 11-10-2016

Bibliometric laws

1. Lotka’s law (1926) – the frequency distribution of scientific productivity

Lotka, A. J. (1926), "The frequency distribution of scientific productivity“, Journal of the Washington Academy of
Sciences, 16 (12): 317–324. [looked in Chemical Abstracts Index]

Alfred Lotka (1880-1949, American mathematician, chemist and statistician)

“It would be of interest to determine, if possible, the part which men of

different caliber contribute to the progress of science considering first

simple volume of production.” (Lotka, 1926)

45

Bibliometric laws

1. Lotka’s law (1926)

xn y = C

where

x: number of publications

y: number of authors with x publications

n: constant (equals 2 for scientific subjects)

C: constant

The total number of authors y in a given subject, each producing x

publications, is inversely proportional to some exponential function n of x

• Lotka's law is also referred to as the inverse square law of scientific

productivity

46

23
 11-10-2016

Bibliometric laws

1. Lotka’s law (1926)

This means: a large proportion of the total literature in a field is authored by a

small proportion of the total number of authors, falling down regularly, where
the majority of authors produce but one paper

Example: for 100 authors, who on average each wrote one article each over a
specific period, we have also 25 authors with 2 articles (100/22=25), 11 with 3
articles (100/32 ≈ 11), 6 with 4 articles (100/42 ≈ 6), etc. (in Tefko Saracevic)

• Not exact prediction but holds true overall in most fields

47

Bibliometric laws

2. Bradford’s law (1934)

Samuel C. Bradford (1878-1948, British mathematician and librarian)

Pattern first described by Samuel C. Bradford em 1934 that estimates the

exponentially decreasing returns in searching for references in journals

• If scientific journals are arranged in order of decreasing productivity of

articles on a given subject, they may be divided into a nucleus of periodicals
more particularly devoted to the subject and several groups or zones
containing the same number of articles as the nucleus, when the numbers of
periodicals in the nucleus and succeeding zones will be as a : n : n 2 : n3 (in
Tefko Saracevic)

• n is Bradford multiplier
48

24
 11-10-2016

Bibliometric laws

2. Bradford’s law (1934)

• Basically states that most articles in a subject are produced by few journals
(called nucleus) and the rest are made up of many separate sources that
increase in numbers in a regular, exponential way
• Like Lotka’s law this is a law that generally follows laws of diminishing
returns
• Very useful for librarians: for each field it is sufficient to identify the "core
publications" and only stock those; researchers usually do not need to go
outside that set

49

Bibliometric laws

2. Bradford’s law (1934) – Bradford’s Law of Scattering

• He grouped journals with articles relevant to a subject (from a bibliography)

into 3 zones in order of decreasing yield from journals with largest no. of
articles to those with smallest; at the end are journals with one article each on
the subject

• Each group had the same number of articles but different no. of journals

• The number of journals in each zone increases exponentially

Example: if there are 5 journals in the first zone that produced 12 relevant
articles; there may be 10 journals in the second zone for next 12 articles & 20
for next 12. Then, the Bradford multiplier (n) is 10/5=2

(in Tefko Saracevic) 50

25
 11-10-2016

Bibliometric laws

2. Bradford’s law (1934) – Bradford’s Law of Scattering

In any field of interest, relevant journals can be split into three groups; each
group contributes the same number of relevant articles to citations in the field
• # of 1st group journals = k
3 sources = 130 articles nucleus
• # 2nd group journals = k*n
9 sources = 130 articles
• # of 3rd group journals = k*(n^2)
27 sources = 130 articles 3 sources
130 articles
9 sources
130
articles
27
sources
130
51
Source: Tefko Saracevic articles

Bibliometric laws

3. Zipf’s law (1935)

George Kingsley Zipf (1902-1950, American linguist and philologist)

r.f=c

where

r: rank (in terms of frequency)

f: frequency (no. of times the given word is used in the text)

c: constant for the given text

For a given text the rank of a word multiplied by the frequency is a constant

Works well for high frequency words, not so well for low – thus a number of
modifications

52

26
 11-10-2016

Bibliometric laws

3. Zipf’s law (1935) of term distribution

In a document of ca. 10,000 words
• the most frequently used word is “the” at 950 times
• the 2nd most frequently used word is “a” at 490 times
• the 3rd most frequently used term is “in” at 340 times
• the 1000th most frequently used term is “fruit” at 1 time

(in Tefko Saracevic)

53

Bibliometric laws

3. Zipf’s law (1935) of term distribution

• Frequency x occurrences = constant

1 x 950 = 950
2 x 490 = 980
3 x 340 = 1020
1000 x 1 = 1000

Constant = 1000 for this document

54

27
 11-10-2016

The laws of bibliometrics – more uses

• Measuring the scattering of articles on a subject in various journals

(Bradford’s law)

• Measuring the productivity of an author based on the number of published

articles (Lotka’s law)

• Ranking of words in a text based on frequency of occurrence of words

(Zipf’s law)

• Identification of the peers, social change and the core journal, etc.

55

The laws of bibliometrics – more uses

• Indexing and Thesaurus

• Research

• Formulating search strategies in case of automated system

• Bibliographic control

• Library Management

56

28
 11-10-2016

Example
Teixeira, A.A.C. (2014), "Evolution, roots and influence of the literature on National Systems of
Innovation: a bibliometric account", Cambridge Journal of Economics, Vol. 38 (1): 181-214.

The author has developed three major bibliometric exercises:

1. Main trends on NSI: based on the analysis of the abstracts from all (356)
articles published on NSI found in the EconLit and Scopus bibliographic
databases up to 31 December 2010 [using the term ‘National Innovation
Systems’ or ‘National Systems of Innovation’ as the search keyword]
2. The scientific roots of NSI literature: citation analysis taking the
references/citations of 297 (out of the 356) articles listed in the abstract
database, which generated a citation database involving 13,556
references/citations
3. The influence of NSI literature: considering 25 ‘seminal’ journal articles
on NSI (i.e., articles that were cited in 30 and more studies), we
constructed a database of 2,109 studies that were ‘influenced’ by the NSI
literature
57

Example

Main trends NSI

58

29
 11-10-2016

Example

Main trends NSI

59

Example

Main trends NSI

60

30
 11-10-2016

Example

Main trends NSI

61

Example

Main trends NSI

62

31
 11-10-2016

Example

Main trends NSI

63

Example

Roots of NSI

64

32
 11-10-2016

Example

Roots of NSI

65

Example

Roots of NSI

66

33
 11-10-2016

Example

Influence of NSI

67

34