Cardiovasc Intervent Radiol
https://doi.org/10.1007/s00270-020-02535-0
LETTER TO THE EDITOR
Disparities in Citation Metrics Amongst Web of Science, Scopus,
and Google Scholar for Interventional Radiology Journals
Vibhor Wadhwa1 • George K. Vilanilam1 • Jeffrey Forris Beecham Chick2
Received: 17 April 2020 / Accepted: 25 May 2020
 Springer Science+Business Media, LLC, part of Springer Nature and the Cardiovascular and Interventional Radiological Society of Europe
(CIRSE) 2020
Editor,
The scientific community relies on citation metrics to
gauge the reputation of a journal. Academic institutions
also utilize citation metrics as a criteria for promotion and
tenure. Although there are many available citation metrics,
the most important, for scientific journals, is the journal
impact factor (JIF) published by Clarivate (Clarivate;
Philadelphia, PA). The calculation of JIF relies on citation
counts of the articles published in the journal and uses the
Web of Science (WoS) database to count the number of
citations. The other well-known citation indexing services
used by the scientific community include Scopus (Elsevier
Inc; Amsterdam, Netherlands) and Google scholar (GS)
(Alphabet Inc; Mountain View, CA). It has been shown
that disparities exist amongst these services in terms of
citations metrics for other medical specialties. For the field
of interventional radiology (IR), however, no analysis has
reported the differences between these citation indexes.
The purpose of this study was to evaluate the citation
counts obtained from WoS, Scopus, and GS for three well-
known IR journals—Journal of Vascular and Interven-
tional Radiology (JVIR), CardioVascular and Interven-
tional Radiology (CVIR), and Journal of Endovascular
Therapy (JET).
Institutional review board approval was not required for
this study. The 50 most cited articles, published in JVIR,
& Vibhor Wadhwa
vibhorwadhwa90@gmail.com
1
Department of Radiology, University of Arkansas for
Medical Sciences, Little Rock, AR, USA
2
Division of Interventional Radiology, Department of
Radiology, University of Washington, Seattle, WA, USA
CVIR, and JET in 2016 and 2017, were obtained from the
WoS database and included in this study. Citation counts
were recorded for each article on April 10th, 2020 from
WoS, Scopus, and GS. Means and standard deviations were
calculated for citations obtained from all three sources for
the journals. Paired sample t test was used to determine
statistical differences in citations between the three sour-
ces. Additionally, the Altmetric score (Holtzbrinck Pub-
lishing Group; Stuttgart, Germany) for each article was
recorded and the Spearman correlation test was used to
determine correlation between citation counts and Alt-
metric scores. Two-tailed p value of \ 0.05 was considered
significant.
A total of 150 articles were included in this study
(JVIR = 50; CVIR = 50; JET = 50). The mean number of
citations obtained from WoS, Scopus, and GS for JVIR was
24.64 ± 10.43, 27.5 ± 12.43, and 36.8 ± 16.15, respec-
tively; for CVIR were 25.62 ± 16.25, 27.52 ± 16.73, and
37.32 ± 22.11, respectively; and for JET were
22.90 ± 8.69, 24.60 ± 9.24 and 29.38 ± 11.69, respec-
tively (Fig. 1). For JVIR, Scopus found 11.61% more
citations than WoS and GS found 49.35% more citations
than WoS, both differences were statistically significant
(p \ 0.001). Similarly, for CVIR, Scopus found 7.42%
more citations than WoS and GS found 45.67% more
citations than WoS, both differences were again statisti-
cally significant (p \ 0.001). For JET, Scopus found
7.42% more citations than WoS and GS found 28.29%
more citations than WoS (p \ 0.001). The mean Altmetric
scores for the journals were: JVIR 13.4 ± 16.27, CVIR
5.08 ± 5.22, and JET 2.20 ± 2.74. There was no statistical
correlation between the Altmetric scores and the citation
counts obtained from WoS (p = 0.323), Scopus
(p = 0.627), or GS (p = 0.066).
123
 V. Wadhwa et al.: Disparities in Citation Metrics Amongst Web of Science, Scopus, and…
Fig. 1 Mean citations for 50
most cited articles published in
JVIR, CVIR, and JET journals
in 2016 and 2017 obtained from
Web of Science, Scopus, and
Google Scholar
The differences in citation counts for the same articles
indicate the differing sourcing strategies of these services.
WoS has 129 journals under the subject subcategory
‘‘Radiology, Nuclear Medicine, and Medical Imaging,’’
and ranks journals using JIF, of which JVIR ranks 44 and
CVIR ranks 79. Under the subject subcategory ‘‘Peripheral
Vascular Disease’’ on WoS, JET ranks 25 and JVIR ranks
27. On the other hand, Scopus lists 272 journals under the
subject subcategory ‘‘Radiology, Nuclear Medicine, and
Imaging,’’ and ranks journals based upon CiteScore metric,
of which CVIR ranks 102 and JVIR ranks 112. There is no
distinct subject category for IR in either of these indices.
The CiteScore metric uses Scopus citations, rather than
WoS citations, and is calculated by including citations and
publications from the three previous years (rather than the
2 years for JIF). JIF and CiteScore trends of JVIR, CVIR,
and JET over last 5 years are shown in Fig. 2. Note that
while the JIF of JVIR has consistently been higher than
CVIR, the CiteScore of both journals is comparable. Also,
note that while the JIF for JET is comparable with JVIR in
2018, the CiteScore for JET is higher than JVIR.
The present study shows that Scopus and GS find sig-
nificantly more citations than WoS, with the difference
between GS and WoS being larger for all three journals.
Findings of this current study are similar to previous
reports in other specialties, such as cardiology [1], and
highlight the disparities in citation indexing within these
databases. The Journal Citation Report (JCR), of Clarivate,
indexes almost 12,000 journals while Scopus indexes close
to 23,000 active journals; therefore Scopus potentially has
a broader library of citation sources. The size of GS
repository; however, is unknown at the present time, with
123
one study estimating 389 million indexed records [2]. GS
includes [ 90% found by WoS and Scopus, with the
additional citations including theses and dissertations,
books or book chapters, conference proceedings, unpub-
lished materials, and other document types [3]. Another
possible reason for GS finding more citations is the
inclusion of non-English literature while WoS and Scopus
predominantly index English literature. Some studies have
shown that GS may return duplicate citations results, pos-
sibly another reason for higher citation counts [4]. It is
worth noting; however, that differences between citation
databases were consistent across all three journals, despite
the different strategies.
Altmetric is a commercial organization which developed
a social media-based ‘‘alternative metrics’’ scoring system
and publishes a composite Altmetric Attention Score for
each article to aggregate and track online mentions. It is
calculated as a weighted total of the mentions of an article
by various online platforms including social media, with
the weights reflecting the relative reach of each online
source. Importantly, however, conventional citations in
biomedical journals are not included within the score,
making it a different metric altogether. It has been previ-
ously shown that there is a weak correlation between
citations and Altmetric score for radiology articles [5]. This
is consistent with findings of the current study, which
showed no significant correlation between Altmetric score
and citation counts. Despite these findings, such novel
scoring systems are likely to garner more attention with the
continued growth of social media.
Limitations of this study are inclusion of only 50 most
cited manuscripts from the journals, which was purposeful,
V. Wadhwa et al.: Disparities in Citation Metrics Amongst Web of Science, Scopus, and…

Fig. 2 Journal impact factor and CiteScore trends for JVIR, CVIR, and JET from 2014 to 2018

123
 V. Wadhwa et al.: Disparities in Citation Metrics Amongst Web of Science, Scopus, and…
since as highly cited articles are more likely to show dif-
ferences between the various sources. Also, IR studies
published in other journals were not included, since this
study was targeted towards IR-centric journals.
In conclusion, this study demonstrates that Scopus and
GS have a higher mean citation counts than WoS for IR
journals and the differences between WoS and GS are
much larger than WoS and Scopus. Given these differ-
ences, individual researchers, academic institutions, and
journals are more likely to rely on multiple sources to
gather complete citation data. Larger studies need to be
performed to determine the quality of these citation
indices.
Funding None.
Compliance with Ethical Standards
Conflict of interest The authors declare that they have no conflict of
interest.
123
References
1. Anker MS, Hadzibegovic S, Lena A, Haverkamp W. The
difference in referencing in Web of Science, Scopus, and Google
Scholar. ESC Heart Fail. 2019;6(6):1291–312. https://doi.org/10.
1002/ehf2.12583.
2. Gusenbauer M. Google Scholar to overshadow them all? Com-
paring the sizes of 12 academic search engines and bibliographic
databases. Scientometrics. 2019;118(1):177–214. https://doi.org/
10.1007/s11192-018-2958-5.
3. Martı́n-Martı́n A, Orduna-Malea E, Thelwall M, López-Cózar ED.
Google Scholar, Web of Science, and Scopus: a systematic
comparison of citations in 252 subject categories. SocArXiv. 2018.
https://doi.org/10.31235/osf.io/42nkm.
4. Adriaanse LS, Rensleigh C. Web of science, scopus and google
scholar: a content comprehensiveness comparison. Electron Libr.
2013;31(6):727–44. https://doi.org/10.1108/el-12-2011-0174.
5. Rosenkrantz AB, Ayoola A, Singh K, Duszak R. Alternative
metrics (‘‘Altmetrics’’) for assessing article impact in popular
general radiology journals. Acad Radiol. 2017;24(7):891–7.
https://doi.org/10.1016/j.acra.2016.11.019.
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.