Scientometrics

DOI 10.1007/s11192-013-1148-8

Assessment of research fields in Scopus and Web

of Science in the view of national research evaluation
in Slovenia

Tomaz Bartol • Gordana Budimir • Doris Dekleva-Smrekar •

Miro Pusnik • Primoz Juznic

Received: 6 May 2013

Ó Akadémiai Kiadó, Budapest, Hungary 2013

Abstract Web of Science (WOS) and SCOPUS have often been compared with regard to user
interface, countries, institutions, author sets, etc., but rarely employing a more systematic
assessment of major research fields and national production. The aim of this study was to
appraise the differences among major research fields in SCOPUS and WOS based on a stan-
dardized classification of fields and assessed for the case of an entire country (Slovenia).
We analyzed all documents and citations received by authors who were actively engaged in
research in Slovenia between 1996 and 2011 (50,000 unique documents by 10,000
researchers). Documents were tracked and linked to SCOPUS and WOS using complex
algorithms in the Slovenian COBISS bibliographic system and SICRIS research system where
the subject areas or research fields of all documents are harmonized by the Frascati/OECD
classification, thus offsetting some major differences between WOS and SCOPUS in database-
specific subject schemes as well as limitations of deriving data directly from databases.
SCOPUS leads over WOS in indexed documents as well as citations in all research fields. This
is especially evident in social sciences, humanities, and engineering & technology. The
least citations per document were received in humanities and most citations in medical and
natural sciences, which exhibit similar counts. Engineering & technology reveals only half
the citations per document compared to the previous two fields. Agriculture is found in the
middle. The established differences between databases and research fields provide the

T. Bartol (&)
Agronomy Department, Biotechnical Faculty, University of Ljubljana, Ljubljana, Slovenia
e-mail: tomaz.bartol@bf.uni-lj.si

G. Budimir
Institute of Information Science, Maribor, Slovenia

D. Dekleva-Smrekar
M. Pusnik
Central Technological Library at the University of Ljubljana, Ljubljana, Slovenia

P. Juznic
Department of Library and Information Science and Book Studies, Faculty of Arts,
University of Ljubljana, Ljubljana, Slovenia

123
 Scientometrics

Slovenian research funding agency with additional criteria for a more balanced evaluation
of research.

Keywords Bibliometrics
Citation analysis
Research performance

Research evaluation
Research fields
Research information systems
Slovenia

Introduction

Similarities and differences between WOS and SCOPUS have frequently been compared with
regard to various criteria. Most studies have focused on comparing the user interface
(retrieval using different fields) and differences in coverage according to various issues.
Less research has been published that attempts a more comprehensive and balanced
evaluation of the role of the two databases in the assessment of a wider national production
of publications, especially with regard to possible differences among different scientific
fields. It is known that the transparency of research is an important element of scientific
activity. This involves the use of different quantitative methods such as the employment of
bibliometric indicators for the purposes of research evaluation. If the two citation databases
are used concurrently for such evaluation, it is important to know the principal charac-
teristics of both databases in tracking publications and citations in different scientific fields.
The general purpose of our research is to compare SCOPUS and WOS by assessing, more
specifically, differences among different fields of research. The case of Slovenian docu-
ments and citations appearing in both databases will be studied. We assume that differ-
ences exist among databases in coverage and citability that have frequently been addressed
in the literature. More particularly, however, we also wish to establish if there exist some
particular differences among different major research fields, not only between databases
but also within each database. It is important to assess such differences more systemati-
cally. Not only do the databases differ in coverage, but different research fields may also
exhibit different publishing patterns and different levels of citability. Such information
may serve as a basis for a more balanced appraisal of research fields for the purposes of
evaluating researchers and research activities in Slovenia. Specifically, if the differences
are not taken into account, some researchers, for example in the fields of social sciences,
humanities, and possibly some other fields, may not compete for very limited national
research funds on a level playing field with other researchers since certain fields may
exhibit much higher citability. The study covers some 50,000 unique documents published
by almost 10,000 authors and indexed by WOS and SCOPUS between 1996 and 2011. The
study employs a systematic authority-control managed by the COBISS system (the techniques
are presented in detail in the next section), which uses an internationally harmonized
classification of scientific fields for both databases to offset many well-known differences
between the two databases, for example, in classification, missing data, errors, or problems
in capturing consistent sets in the data derived directly from databases. The study thus not
only provides systematically collected and analyzed information on scientific production in
Slovenia for the purposes of evaluation, but also offers a more balanced general overview
of the two databases and field-specific characteristics based on a large set of harmonized
data over a longer period of time.
In preparation, we reviewed some selected previous papers that emphasized the issues
tackled in our research. Comparison of the two databases is usually based on selected

123
Scientometrics

country data, institutions, selected journals, publication types, subject categories (subject
areas), etc. As opposed to our research, however, most authors derived data directly from
the databases, which presents substantial limitations in an analysis.
The assessment of database coverage by countries, institutions, or journals seems to be
the most frequent object of research. Benoit and Marsh (2009) assessed universities with a
political studies program in Ireland (North and South) and compared Irish departments
without any kind of national level research ranking and review with the United Kingdom,
where the Research Assessment Exercise (RAE) permits some identification of centers of
research excellence. Scientific publishing of the health-sciences-related departments of
Navarra University was assessed by Torres-Salinas et al. (2009) on a sample of the 50 most
cited researchers, with SCOPUS providing 14.7 % more citations. A study by Haddow and
Genoni (2009), which was based on Australian education journals, also found that the
coverage in SCOPUS outperformed WOS. SCOPUS also performed better in some areas of the
social sciences (Lasda Bergman 2012). In general, however, the advantage of SCOPUS over
WOS lies in the better coverage of journals in languages other than English (Leydesdorff
et al. 2010). For example, 240 Russian journal titles are indexed by SCOPUS, and 160 by WOS
(Zibareva and Soloshenko 2011). On the other hand, such national journals play a more
peripheral role in the international journal communication system (Lopez-Illescas et al.
2009).
Despite the differences, both databases are comparable in terms of rankings (Bar-Ilan
et al. 2007) with the important exception of certain authors who publish their research in
some major series that are only indexed by SCOPUS (Bar-Ilan 2008). In particular, the top
countries have similar ranks in both databases (Archambault et al. 2009). Here, however, it
needs to be pointed out that analyses based on smaller entities (publications, institutions)
produce considerably different results for the two databases than an analysis based on
larger entities such as research domains and previously mentioned countries (Meho and
Sugimoto 2009).
Researchers frequently report possible errors and limitations involving analyses that
derive data directly from databases. Vieira and Gomes (2009) used such data in an
assessment of two universities and reported possible inaccuracies in the retrieval of bib-
liographic units due to input errors, for example, the country in the authors’ addresses. In a
study covering WOS and several other databases, it was observed that sometimes a city may
be present in the affiliation field while the country is missing (Bartol and Hocevar 2005).
Even though the identification of an author may be good, the missing data can also be
attributed to journal publishers who have not always had consistent policies for including
the country affiliation of the authors (Jacsó 2009). Many Russian-language references are
frequently not taken into account in ‘‘citation reports’’ because of problems involving
transliteration (Zibareva and Soloshenko 2011). Furthermore, the policy of assigning
particular document types may change over time, which creates unclear differentiating
criteria among different subject disciplines (Harzing 2013).
Subject categories or subject areas are database-specific, so differences exist in the
representation of research fields in each database. Important differences in categorization
exist even among the top journals within a particular category (Abrizah et al. 2013). Such
journal classification systems may hold at the macro-level but will be less applicable at
more specialized levels (Tijssen 2010). Some ‘‘marginal’’ fields may be covered by several
different subject categories (Chirici 2012). Researchers therefore sometimes employ a
harmonization based on a common classification. The European classification of the
European Foundation for Science (EFS) was used for the field of social sciences and
humanities by Pumain et al. (2010) in order to offset some differences between WOS and

123
 Scientometrics

SCOPUS in subject classification. Archambault et al. (2009) used the categories of the United
States’ National Science Foundation (NSF) to delineate the fields of natural sciences and
engineering. Bornmann and Leydesdorff (2013) used Frascati (OECD) subject categories
to compare the outputs of six highly productive countries. However, in spite of the
common guidelines in the Frascati manual, differences will remain in the way R&D
statistics are compiled nationally (Wendt et al. 2012).
Finally, it needs to be underlined that the coverage, scope, and search functionalities in
both databases are constantly evolving; therefore, the information in the literature can only
refer to the coverage that exists at the time of the analysis (Lopez-Illescas et al. 2009). The
constant expansion of coverage towards more local or national relevance or additional
publication types thus represents a serious challenge for citation based methods and
indicators (Glänzel and Moed 2012). We may at this point add that Slovenia’s current
research information system, SICRIS, which was used as the main tool for our research,
mitigates many of the above challenges and detects changes in both databases very
promptly through constant updating, thus providing a very good tool for a thorough sci-
entometric assessment of national production in relation to both SCOPUS and WOS. Its
functionalities are presented more systematically in the next section.

Materials and methods

The functionalities of both citation databases have been presented on many occasions so in
this paper we will not again address their major features but will place more emphasis on
the evaluation of data as derived from the authority-controlled SICRIS and COBISS systems
and subsequent links to both SCOPUS and WOS.
Funded by the Slovenian Research Agency (SRA), the Slovenian current research
information system (SICRIS) contains data on research organizations and groups, research
projects and programs, and individual researchers. Bibliographic evaluation of researchers’
publishing activity is provided through the COBISS system and its union bibliographic
database. Software development is managed by the Institute of Information Science (IZUM).
Researcher bibliographies became obligatory in 1997, involving inputs by professional
librarians (Demsar and Juznic 2013). One of the more important features of the system is
the functionality of a scientometric assessment of researchers’ bibliographies, whereby all
bibliographic units that are covered by WOS are provided with links to the corresponding
WOS record (Curk et al. 2006). The system has facilitated several analyses of scientific
output, for example, of collaboration among scientists and co-authorship patterns (Perc
2010; Peclin et al. 2012) or collaboration within selected research disciplines such as
biotechnology, mathematics, physics, and sociology (Kronegger et al. 2011). In order to
complement the WOS-related data in COBISS, in 2012 the SRA decided to provide similar links
to the bibliographic output in SCOPUS. The feature now permits tracing published documents
as well as counting citations to documents according to both WOS- and SCOPUS-derived
indicators.
COBISS links bibliographic records to the general SCOPUS database (SciVerse/Elsevier)
and WOS Citation Databases (Science Citation Index Expanded, Social Sciences Citation
Index, Arts & Humanities Citation Index; Thomson Reuters) for all bibliographic units (co-)
authored by scientists who are subject to the systematic evaluation of the published research
in SICRIS. The input data is checked and verified both by the authors and the respective
bibliographic services, usually at an author’s department responsible for the accuracy of
data. During this process, complex record matching algorithms are used to link the

123
Scientometrics

corresponding records in COBISS, SCOPUS, and WOS. The links for records that remain unlinked
during the automated process are established manually. When the links are identified and
established, they are saved in a special database so it is no longer necessary each time to
determine which bibliographic units in WOS and SCOPUS belong to a researcher.
In order to enable this analysis, all relevant records were arranged in an experimental
archive to facilitate a more thorough exploration. The citations were counted on the
principle of all citations. Some limitations existed due to different ways of recording
authors’ names in COBISS (full names) and in WOS or SCOPUS (surnames and initials). Some
differences were also found in the bibliographic data. Such differences represent a chal-
lenge for the automatic identification and coupling of corresponding records. The largest
variances between the two databases occurred in the titles containing chemical formulae or
other symbols and also, to a lesser extent, in the years of publication, ISSN numbers, DOI
identifiers, volumes, issues, and number of pages. The authority control in COBISS com-
pensates for a significant portion of such differences.
Our analysis involved bibliographic units (co-)authored by 9,828 registered researchers
who were active between 1996 and 2011. Researchers are closely monitored on the basis of
a unique author identifier, i.e. the SRA code (Slovenian Research Agency code). The
analysis covered all researchers who are registered in the SICRIS system and evaluated for
the purposes of the SRA. Besides all the Slovenian authors who are active in public research
organizations, the system includes also Slovenian researchers working abroad and some
foreign researchers who are active in Slovenia and need to be evaluated for the purposes of
the SRA. We therefore included all documents that were co-authored by at least one such
registered researcher.
According to the principles of research policy in Slovenia, in order to better track some
possible differences among different research fields, all researchers also need to associate
their particular research area with one of the six major research fields (Field of Research
Classification) and their respective narrower fields, for example, biology, mathematics, or
physics in the natural sciences or civil engineering, computer science, or geodesy in the
engineering sciences and technologies. This classification, which is used by the SRA, has
been roughly harmonized with the Field of Science and Technology Classification in the
Frascati Manual (OECD), with some modifications related to the specifics of research
groups in Slovenia. Table 1 presents fields of research in Slovenia matched with the major

Table 1 Fields of Science and Technology as used for research classification by the SRA (Slovenian
Research Agency) and Frascati/OECD; researchers active in the field between 1996 and 2011
Field of research Major field of Science and Researchers
classification (SRA) Technology (Frascati/OECD)

Biotechnical Sciences (agriculture) Agricultural Sciences 906

Engineering Sciences and Technologies Engineering and Technology 3,000
Humanities Humanities 718
Medical Sciences Social and Health Sciences 1,838
Natural Sciences and Mathematics Natural Sciences 2,281
Social Sciences Social Sciences 1,085
Total 9,828

123
 Scientometrics

fields of science and technology according to Frascati and the number of different
researchers classified in the respective fields in the 1996–2011 period.
If a document was co-authored by several authors all pertaining to the field of natural
sciences, it was counted only once in this field. However, we also needed to take into
account those selected ‘‘multidisciplinary’’ documents that had been co-authored by
authors from different fields. For example, if a document was co-authored by authors
pertaining to the natural sciences and authors from the medical sciences, it was counted
once in each field. Our method involves the principles of whole counting, which is applied
in most bibliometric analyses. Even though some authors make a case for fractionalized
counting (Aksnes et al. 2012), this was not practical for our analysis since the whole
counting method is also employed by the source data in SICRIS. The same method was used
on both databases on the same controlled source of data, so both databases were compared
on the same principles for all fields of science.

Results

Our more detailed analysis was conducted on 9,828 researchers who classified their
research activity into one of the six major fields of science and technology presented in
Table 1. Unclassified researchers were not assessed. The classified researchers (co-
)authored 44,226 documents in SCOPUS and 38,339 documents in WOS, according to the links
in COBISS. These documents were cited 500,094 and 432,311 times, respectively (Fig. 1).
The analysis covered those more ‘‘authoritative’’ document types that are used for
assessment of research activity in SICRIS such as articles, proceedings papers, and chapters.

(a) (b)

Fig. 1 The total number of documents (a) and citations (b) in the 1996–2011 period in SCOPUS and WOS
databases

123
Scientometrics

Some miscellaneous document types such as ‘‘abstracts-only,’’ corrections, and letters

were not included.
Figure 1 serves as a general reference for more detailed analyses in the continuation of
the paper. At this point it needs to be reiterated that SCOPUS was introduced only recently so
it is possible that some bibliographic records in COBISS may not have been linked to SCOPUS
as yet.
Figures 2 and 3 present the yearly growth of published documents in the observed
period by major fields of science according to the subject matter of an author’s research
activities. The four associated life and applied sciences (agricultural sciences, medical (and
health) sciences, natural sciences, and engineering & technology) are presented together
(Fig. 2), and separately from the social sciences and humanities (Fig. 3). Namely, the total
numbers of documents for the social sciences and humanities are much lower on account of
smaller research groups and different publishing and citation patterns in these two fields.
Figure 2 reveals a continual growth in all four major life and applied sciences fields in
both databases. These numbers must be perceived in relation to the registered researchers
in their respective fields as shown in Table 1. The growth is fairly steady although in some
isolated phases there is some irregularity in natural sciences and engineering & technology,
which can be inferred from the 2004 and 2006 data. The total numbers are the highest in
the natural sciences, indicating specific publishing patterns in this scientific field. Agri-
cultural sciences return the lowest numbers, with only 906 active researchers in this field
(Table 1).
Some other outcomes are more interesting. Even though SCOPUS reveals higher numbers
than WOS in all groups, we can observe some differences in database-specific results
between the natural sciences on the one hand and engineering & technology on the other.
Although SCOPUS shows some lead over WOS in the natural sciences, this lead is much more

Fig. 2 Growth of documents in agricultural sciences, medical sciences, natural sciences, and engineering &
technology in the 1996–2011 period in SCOPUS and WOS databases

123
 Scientometrics

Fig. 3 Growth of documents in social sciences and the humanities in the 1996–2011 period in SCOPUS and
WOS databases

substantial in engineering & technology. The differences are not strongly influenced by the
better coverage of Slovenian national journals in SCOPUS. While 34 Slovenian journals are
indexed by SCOPUS compared to 26 by WOS, the research evaluation policy in Slovenia
encourages presentation of research in the international context, thus giving less incentive
to national publishing with fewer articles appearing in Slovenian publications.
Figure 3 illustrates the trends in social sciences and the humanities. Social sciences
display an accelerated growth after 2006 in both databases. Some stagnation can be
observed in the humanities in 2006 (Fig. 3). This development, however, is more than
effectively offset in the next year. While there is a brief converging between SCOPUS and
WOS in the humanities in 2009, publications in WOS then seem to ‘‘lose some steam.’’ On
average, however, SCOPUS exhibits better inclusion of documents in social sciences and
humanities in comparison to the fields in life and applied sciences.
The final analysis brings the assessment of published documents in relation to citations
received in all respective major fields in both databases (Figs. 4, 5). The data are pre-
sented in two-axis charts to better illustrate the differences among the fields. The left-
hand axes show the number of total documents for each respective field; the right-hand
axes present citations. SCOPUS yields more results in all disciplines but the figures vary
substantially among the disciplines. The difference in database coverage in the life and
applied sciences group (Fig. 4) is most noticeable in engineering & technology. Some
2,000 more documents can be found in SCOPUS. The citations (right-hand axis) reveal a
similar pattern. The difference in percentages between the databases is higher in the
social sciences and humanities (Fig. 5). Both fields display more documents and citations
in SCOPUS than in WOS. To offer some additional perspective, we also provide information
on the Slovenian journals included in SCOPUS and not WOS, and vice versa, by database-
specific general classifications. SCOPUS includes 15 Slovenian journals not indexed by

123
Scientometrics

Fig. 4 Documents (left y-axis scale) and citations (right y-axis scale) in agricultural sciences, medical
sciences, natural sciences, and engineering & technology in the 1996–2011 period in SCOPUS and WOS
databases

WOS, and WOS seven journals not indexed by SCOPUS (Table 2). One of the SCOPUS journals
is listed under both Agricultural and Biological Sciences and Environmental Science.
Altogether, 34 Slovenian journals are indexed by SCOPUS and 26 by WOS. Some of these
journals, however, though recorded on indexed-journals lists in databases, do not return
any results (or just a few dozen). Also, many journals are very rarely or never cited, not
even as self-citations.
With regard to citations, further differences among the disciplines are evident in both
databases. The average number of citations per published document is much lower in
engineering & technology (7.4 cit/doc—WOS) than in the natural sciences (14.4 cit/doc—
WOS) or medical sciences (13.6 cit/doc—WOS). In WOS, 17,678 and 11,134 respective
documents have been attributed to natural sciences and engineering & technology. How-
ever, natural sciences received 253,752 citations in this database, and engineering &
technology only 82,590 citations. SCOPUS shows similar patterns but returns more docu-
ments and citations. In the agricultural sciences, SCOPUS returns some 10 % more docu-
ments and citations, with citations per document producing similar results: 10.8 cit/doc in
SCOPUS and 10.6 cit/doc in WOS. However, the medical sciences show 13.6 cit/doc in WOS
and 15.0 cit/doc in SCOPUS. The difference between the two databases is more evident in the
social sciences and humanities (Fig. 5). In SCOPUS, social sciences display 2,884 documents
with 13,423 citations (4.7 cit/doc), and humanities 1,877 documents with 2,505 citations
(1.3 cit/doc). Citations per document in WOS are similar with 4.2 and 1.2 cit/doc, respec-
tively. In WOS, however, the total citation count for humanities is only 1,656, lower by more
than a third compared to SCOPUS (2,505 citations). In general, the humanities yield the
lowest count of citations per document in both databases.

123
 Scientometrics

Fig. 5 Documents (left y-axis scale) and citations (right y-axis scale) in social sciences and humanities in
the 1996–2011 period in SCOPUS and WOS databases

Discussion

Our study employed the two global citation databases SCOPUS and WOS to assess certain
differences between the databases and to ascertain how such differences are reflected in the
major research fields and the possible applicability of such information in the evaluation of
published research in Slovenia. We employed the Slovenian national system for the
evaluation of research (SICRIS), which is linked to the unique bibliographic functionality of
the COBISS system, wherein all registered researchers are systematically assessed for
inclusion of their documents in the two databases along with pertaining citation counts.
Most similar analyses have focused on a limited set of data and experimental materials
are almost invariably derived directly from the databases. Our analysis, however, sys-
tematically reviewed all scientific fields based on a harmonized classification of fields and
employed data of a vast majority of researchers in Slovenia. This analysis thus involved
almost the entire national production over an extended period of time (1996–2011). To
avoid many possible errors in the identification of authors, incorrect country, missing
author affiliation or changing patterns in the classification of publication types in some
disciplines (Vieira and Gomes 2009; Jacsó 2009; Harzing 2013), our data were subjected to
a rigorous authority control. Our assessment of differences in publishing and citing patterns
among different major fields of science was based on an international classification, thus
offsetting some major problems with subject schemes based on journals (Chirici 2012). It
is also known that database differences exist in the coverage of certain document types
which may even be assigned incorrectly (Bar-Ilan 2008; Meho and Sugimoto 2009;
Harzing 2013). However, since the records in COBISS are supplied with permanent docu-
ment-type indicators, our harmonized input data represented an important factor for the
assessment of databases on equivalent basis.

123
Scientometrics

Table 2 Slovenian journals lis-

Journals
ted only in SCOPUS or only in WOS
SCOPUS subject area
Agricultural and Biological Sciences 2
Arts and Humanities 1
Computer Science 2
Environmental Science 1
Medicine 1
Pharmacology, Toxicology and Pharmaceutics 1
Physics and Astronomy 1
Social Sciences 7
WOS (general groups)
Arts & Humanities (AHCI) 2
Sciences (SCIE) 3
Social Sciences (SSCI) 2

The results of our study show that SCOPUS, generally speaking, provides more results,
both in the number of records as well as in the number of related citations. This is in
agreement with the findings by, for example, Torres-Salinas et al. (2009); Haddow and
Genoni (2009). There is no substantial difference between the two databases in the cov-
erage of Slovenian national journals as opposed to the considerable advantage of SCOPUS
over WOS in the coverage of the national journals of some other countries (Torres-Salinas
et al. 2009; Zibareva and Soloshenko 2011; Leydesdorff et al. 2010). Also, many Slovenian
journals generate very low citation counts in both databases. On the other hand, some more
established Slovenian journals publish works by non-Slovenian authors. A ‘‘national bias’’
thus plays a relatively minor role in our study.
The analysis, more specifically, addressed differences between databases on the basis
of detecting some possible special characteristics in different research fields. Some other
authors have also tackled some specific differences, for example, among specific subject
areas particularly in a more national-language oriented publishing landscape (Lopez-
Illescas et al. 2009) or in the coverage of specific publications (Bar-Ilan 2008). Our
analysis, however, systematically employed a harmonized subject classification of
research fields and involved all scientific fields. The model (case) is thus very com-
prehensive. We found some more noticeable differences between the databases in the
fields of social sciences and humanities, with SCOPUS exhibiting noticeably higher counts
than WOS. Perhaps more interesting are the figures in the so-called life-and-applied-
sciences group comprising agriculture, engineering & technology, medical, and natural
sciences. While SCOPUS shows better coverage in all four major fields, this difference is
much more evident in engineering & technology with SCOPUS covering substantially
more documents than WOS. The differences in agriculture, medical, and natural sciences
are much smaller.
The final part of the analysis tackled the possible differences in citations received by
Slovenian authors in relation to published documents, again for both databases. SCOPUS
shows more citations per document than WOS in all fields. We detected the highest cita-
tions-per-document counts in the natural and medical sciences. Among all the fields,
citations per document are the highest in medical sciences in SCOPUS. In the life-and-

123
 Scientometrics

applied-sciences group, the citation counts are the lowest in engineering & technology.
Agriculture is placed approximately in the middle. Citation counts are much lower in the
social sciences, and the lowest in humanities, clearly indicating more particular publishing
and citation patterns in these two fields. Some other document types such as books may
yield hundreds of additional citations in other bibliographic services; however, such
citations are not monitored on the same principles by the two databases. Consequently, this
study in no way sought to judge the overall quality of coverage of publications by Slo-
venian authors in the respective fields of science.
More complex assessment of such patterns is beyond the scope of the present analysis but
may be carried out in our future research on a smaller sample. Our results nevertheless
clearly indicate the existence of substantial differences in the ‘‘citability’’ of research fields
established on a very comprehensive national collection of data over a longer period of time.
The identification of these differences is important because research grants need to be
approved and funds allocated through a fair evaluation system. For example, SCOPUS seems
to provide more extensive coverage and more citations in social sciences and humanities so
it could be preferred for a more balanced evaluation of these fields. Also, since engineering
& technology scientists receive ‘‘weaker’’ citation counts on average than natural scientists,
the former can compete fairly for funds only within their particular peer group. This study
thus provides some important information for policy makers who can use the findings for the
purposes of a more impartial appraisal of research outcomes.

Conclusions

Our study represents a systematic attempt to evaluate the differences in published docu-
ments and citations across all major research fields in the SCOPUS and Web of Science
databases based on a standardized classification of fields. These differences were assessed
for the case of an entire country (Slovenia). This was possible by employing an in-depth
analysis of SCOPUS- and WOS-linked data in the COBISS system. This national information
system systematically tracks such data on the principles of rigorous authority-control and
the harmonized international subject classification of all documents, thus offsetting many
well-known problems of deriving data directly from these databases. The results corroborate
some previous studies that found SCOPUS to return more results than WOS. However, our study
also brings a detailed analysis of more particular differences among different scientific
fields. The differences are most obvious in the fields of social sciences, humanities, and
engineering & technology, with SCOPUS returning noticeably more results than WOS in the
number of total documents and citations. Some further characteristics can also be observed
within different scientific fields. The least citations per document are received in the
humanities. Social sciences perform better. However, more interesting figures emerge in life
and applied sciences: the field of engineering & technology exhibits much lower citation
counts per document than comparable agricultural and especially medical and natural sci-
ences in both databases. The detection and systematic assessment of these differences
provide Slovenia’s funding agency with additional criteria for a more transparent and
balanced evaluation of research. The results are based on comprehensive data, established
on a very large and inclusive set of documents and citations over a longer period of time,
which also offers some more widely applicable information on both databases.

123
Scientometrics

References

Abrizah, A., Zainab, A. N., Kiran, K., & Raj, R. G. (2013). LIS journals scientific impact and subject
categorization: A comparison between Web of Science and Scopus. Scientometrics, 94(2), 721–740.
Aksnes, D. W., Schneider, J. W., & Gunnarsson, M. (2012). Ranking national research systems by citation
indicators. A comparative analysis using whole and fractionalised counting methods. Journal of In-
formetrics, 6(1), 36–43.
Archambault, E., Campbell, D., Gingras, Y., & Lariviere, V. (2009). Comparing bibliometric statistics
obtained from the Web of Science and Scopus. doi:10.1002/asi.21062.
Bar-Ilan, J. (2008). Which h-index?—A comparison of WoS, Scopus and Google Scholar. Scientometrics,
74(2), 257–271.
Bar-Ilan, J., Levene, M., & Lin, A. (2007). Some measures for comparing citation databases. Journal of
Informetrics, 1(1), 26–34.
Bartol, T., & Hocevar, M. (2005). The capital cities of the ten new European Union countries in selected
bibliographic databases. Scientometrics, 65(2), 173–187.
Benoit, K., & Marsh, M. (2009). A relative impact ranking of political studies in Ireland. The Economic and
Social Review, 40(3), 269–298.
Bornmann, L., & Leydesdorff, L. (2013). Macro-indicators of citation impacts of six prolific countries:
Incites data and the statistical significance of trends. PLoS One, 8(2), e56768.
Chirici, G. (2012). Assessing the scientific productivity of Italian forest researchers using the Web of
Science, SCOPUS and SCIMAGO databases. iForest—Biogeosciences and Forestry, 5(3), 101–107.
Curk, L., Budimir, G., Seljak, T., & Gerkes, M. (2006). Linking the SICRIS—COBISS.SI—Web of Science
systems. Organizacija znanja, 11(4), 230–235.
Demsar, F., & Juznic, P. (2013). Transparency of research policy and the role of librarian. Journal of
Librarianship and Information Science. doi:10.1177/0961000613503002.
Glänzel, W., & Moed, H. F. (2012). Opinion paper: Thoughts and facts on bibliometric indicators.
Scientometrics. doi:10.1007/s11192-012-0898-z.
Haddow, G., & Genoni, P. (2009). Australian education journals: Quantitative and qualitative indicators.
Australian Academic and Research Libraries, 40(2), 88–104.
Harzing, A.-W. (2013). Document categories in the ISI web of knowledge: Misunderstanding the social
sciences? Scientometrics, 94(1), 23–34.
Jacsó, P. (2009). Errors of omission and their implications for computing scientometric measures in eval-
uating the publishing productivity and impact of countries. Online Information Review, 33(2),
376–385.
Kronegger, L., Ferligoj, A., & Doreian, P. (2011). On the dynamics of national scientific systems. Quality &
Quantity, 45(5), 989–1015.
Lasda Bergman, E. M. (2012). Finding citations to social work literature: The relative benefits of using Web
of Science, Scopus, or Google Scholar. The Journal of Academic Librarianship, 38(6), 370–379.
Leydesdorff, L., de Moya-Anegón, F., & Guerrero-Bote, V. P. (2010). Journal maps on the basis of Scopus
data: A comparison with the journal citation reports of the ISI. Journal of the American Society for
Information Science and Technology, 61(2), 352–369.
Lopez-Illescas, C., de Moya Anegon, F., & Moed, H. F. (2009). Comparing bibliometric country-by-country
rankings derived from the Web of Science and Scopus: The effect of poorly cited journals in oncology.
Journal of Information Science, 35(2), 244–256.
Meho, L. I., & Sugimoto, C. R. (2009). Assessing the scholarly impact of information studies: A tale of two
citation databases—Scopus and Web of Science. Journal of the American Society for Information
Science and Technology, 60(12), 2499–2508.
Peclin, S., Juznic, P., Blagus, R., Sajko Cizek, M., & Stare, J. (2012). Effects of international collaboration
and status of journal on impact of papers. Scientometrics, 93(3), 937–948.
Perc, M. (2010). Growth and structure of Slovenia’s scientific collaboration network. Journal of Infor-
metrics, 4(4), 475–482.
Pumain, D., Kosmopoulos, C., & Dassa, M. (2010). JournalBase—A comparative international study of
scientific journal databases in the social sciences and the humanities (SSH). Cybergeo: European
Journal of Geography (article 484), doi:10.4000/cybergeo.22862.
Tijssen, R. J. W. (2010). Discarding the ‘‘basic science/applied science’’ dichotomy: A knowledge utili-
zation triangle classification system of research journals. Journal of the American Society for Infor-
mation Science and Technology, 61(9), 1842–1852.
Torres-Salinas, D., Lopez-Cózar, E., & Jiménez-Contreras, E. (2009). Ranking of departments and
researchers within a university using two different databases: Web of Science versus Scopus.
Scientometrics, 80(3), 761–774.

123
 Scientometrics

Vieira, E. S., & Gomes, J. A. N. F. (2009). A comparison of Scopus and Web of Science for a typical
university. Scientometrics, 81(2), 587–600.
Wendt, K., Aksnes, D. W., Sivertsen, G., & Karlsson, S. (2012). Challenges in cross-national comparisons
of R & D expenditure and publication output. In Proceedings of 17th international conference on
science and technology indicators. Presented at the STI 2012, Montreal, Canada, September 5–8 (Vol.
2, pp. 826–834).
Zibareva, I., & Soloshenko, N. (2011). Russian scientific publications 2005–2009 in the science citation
index, Scopus, and chemical abstracts databases. Scientific and Technical Information Processing,
38(3), 212–223.

123