Scientometrics

https://doi.org/10.1007/s11192-018-2990-5

Digital competences, computer skills and information

literacy in secondary education: mapping and visualization
of trends and concepts

Karmen Stopar1 · Tomaž Bartol1

Received: 28 June 2018

© Akadémiai Kiadó, Budapest, Hungary 2018

Abstract
Digital competences, computer skills, information literacy and related abilities represent a
crucial element in ICT education (Information and Communication Technologies). They
are less frequently investigated in the frame of secondary education than in higher educa-
tion. We assess these contexts in secondary education through science mapping and visu-
alizing techniques, examining publishing patterns and trends. Databases Web of Science
(WOS) and Scopus are used. Publishing exhibits logarithmic inverse relationship between
rank and frequency (power laws). Only a few chief publications among hundreds account
for an important share of all published research. WOS-based visualizations (VOSviewer
software) of concepts used in titles, abstracts and keywords show several clusters of
research: computer-, information-, as well as digital-related. Further analysis reveals that
the major terms which define these clusters predominate in different periods. Computer-
related are earlier terms, followed by information-related, and now digital-related. As some
concepts mature terminology embraces more trendy novel concepts. Clusters of co-citing
and co-cited sources shows differences among publications. Proceedings play an impor-
tant role as sources of co-citations but are cited more weakly. Both co-citing and co-cited
sources exhibit well defined clusters with little convergence between Library and Informa-
tion Science on one side and Education and Educational Research, and Computer Science
on the other even though the respective publications employ similar terminological con-
cepts. The lack of exchange between these research domains calls for more co-operation in
order to boost synergy in these critical twenty-first century skills.

Keywords Secondary education · Digital competences · Information literacy ·

Bibliometrics · Science mapping · Publishing patterns

* Tomaž Bartol
tomaz.bartol@bf.uni‑lj.si
Karmen Stopar
karmen.stopar@bf.uni‑lj.si
1
Biotechnical Faculty, University of Ljubljana, Jamnikarjeva 101, 1000 Ljubljana, Slovenia

13
Vol.:(0123456789)
 Scientometrics

Introduction and background

Research field of digital competences (also referred to as competencies), computer skills

and information literacy is strongly connected with education, computers, information sci-
ence and libraries. This is reflected in the mapping of papers in this field to the broader
classification area of Education and Educational Research in the citation database Web of
Science as well as Information Science and Library Science and Computer Science. Pub-
lications in these research areas are principal contributors of published research outcomes.
However, co-operation and co-citation between different research groups, as reflected by
the respective research areas, seems rather weak. Researchers remain confined in their
particular professional and educational expertise without sufficient awareness of related
research conducted in other areas. The scatter of research across the different areas and
publications needs to be elucidated in order to show potentials for possible future synergy.
Many papers have been published on the topic of information and communication tech-
nologies (ICT), computers, data, digital resources etc., in the educational context of com-
petences, literacies and skills. The state of the definitions and concepts was comprehen-
sively reviewed by Bawden (2001). Ala-Mutka (2011) addressed these key competences
for lifelong learning in the frame of DIGCOMP project (Digital Competence Framework
for Citizens). The concepts computer- information- and ICT literacy, in the scope of sec-
ondary education and ICILS (International Computer and Information Literacy Study)
were used by Siddiq et al. (2016). Van Laar et al. (2017) addressed digital skills as so
called twenty-first century skills. The different definitions notwithstanding, these terms are
used interchangeably by many authors and connote an array of related or associated litera-
cies, skills and competences. These have also been identified as current topics of interest in
the program of annual ECIL conferences (European Conference on Information Literacy)
(Kurbanoğlu et al. 2018). The terminology seems somehow fluid and subject to evolution.
Here it should be pointed out that both terms competences and competencies are used inter-
changeably in the official documents related to education and curricula (Halász and Michel
2011). In fact, both terms are designated as synonyms in Merriam Webster Thesaurus as
well as in Oxford English Dictionary.
The review below tackles various interrelated sets of digital-related competences, skills
and literacies, and publishing patterns in the context of mapping, scattering and related
bibliometric research. We employ terminology as used by the authors. Most papers address
specific topics, especially cases of specific countries, in relation to international educa-
tional standards, and with regard to different principal actors: educational institutions more
in general or specific stakeholders, such as libraries. The two citation databases Scopus and
Web of Science have been extensively used in order to investigate progress in this field.
Research by a very active research group in this field (Pinto et al. 2014) employed both
Scopus and WOS in view of scientific production, and assessed the concepts informa-
tion literacy, -skills and -competence. They also investigated publishing patterns in rela-
tion to logarithmic power laws, such as Lotka’s law and Bradford’s law, and identified
core journals. The authors used program VOSviewer in order to visualize the concepts.
Kumar (2014) selected the term digital literacy in a similar study. A sole term information
literacy was employed by Majid et al. (2015) in both Scopus and WOS, using multiple-
index fields. Some authors emphasized different models, such as digital literacy and digital
skills in the context of education where more weight was put on technological issues (Mur-
ray and Perez 2014). Concept maps and ICT literacy skills were investigated by Weinerth
et al. (2014) in Scopus and WOS. Other relevant terms were also employed, for example

13
Scientometrics

computer literacy in the context of health information seeking and PubMed database (Li
et al. 2015). Health literacy as a specific literacy in medical field was also visualized with
VOSviewer. Links with information literacy and health education were identified (Kokol
et al. 2018). The terms information literacy as well as digital literacy were employed for
the retrieval of documents in WOS Core Collection by Kolle (2017), using topics-fields
although the author restricted the analysis only to research articles. WOS- as well as Sco-
pus-based bibliographic study of mobile learning in science education was presented by
Zydney and Warner (2016). They reviewed this type of learning in elementary, secondary
as well as post-secondary education.
Taşkin et al. (2013) investigated only journals but noted also the importance of pro-
ceedings. Co-citation network was generated using the concept of information literacy.
Olmeda-Gómez et al. (2017) also performed visualizations and co-keyword and document
co-citation analysis on WOS data. Links between information literacy and higher educa-
tion were detected. Also related to education, Alzafari (2017) conducted co-word-analysis
and visualizations based on Scopus data. WOS- and Scopus-related co-word analysis and
science mapping identified topics and relationships (Heradio et al. 2016). Emerging fields
in education were evaluated by way of co-citation analysis (Steinhardt et al. 2017). Wang
(2018) identified computer science as the largest node among the over 50 subject catego-
ries applicable in digital humanities. Besides VOSviewer, authors have also used other vis-
ualization programs, for example CiteSpace in order to assess various information literacy
clusters (Bušelić and Zorica 2018).
Scientific research has been known to exhibits nonlinear patterns which show that
essential papers are published in only a limited number of preferred core publications. This
was addressed already decades ago, for example, in the so called Bradford’s law of scatter-
ing and related models which were comprehensively explained by Garfield (1980). Similar
distributions can also relate to the references in the papers (Price 1965). Guenther (2006)
detected power laws (Bradford’s Law) in core journals with regard to nursing informatics.
The importance of Bradford’s law for raising the students’ awareness of core literature was
highlighted by Shenton and Hay-Gibson (2009). This law was also used, for example, by
Milojević et al. (2011) on assessing frequently occurring title words or phrases and their
distribution in journals, using co-occurrence analysis and clustering. The term information
literacy was also detected. Scatter of documents according to these patterns was observed
in relation to information literacy (Pinto et al. 2014), digital literacy (Kumar 2014) as well
as health information seeking (Li et al. 2015). Steinhardt et al. (2017) detected nonlinear
patterns in the literature on quality assurance of teaching and learning.
Research questions:

• What are the patterns of the scatter of publications and research areas in the field of
digital literacy and secondary education?
• Which are the principal publications and main areas of research?
• How strong are connections between information science, library science, computer
science and education?
• Is it possible to identify some specific clusters of related concepts (fields of research)
by way of visualization?
• How are these concepts or fields of research influenced by time-period?
• What are the relationships between the citing and the cited sources?

The above questions are all investigated in the context of secondary education. This
term, as well as other related contexts, such as secondary schools or high schools, was a

13
 Scientometrics

principal target in the retrieval of documents under analysis. The occurrence is assessed in
the context of digital-and-related competences. Given the vast array of different concepts
such as computer, digital, ICT, information in a possible permuted relationship with com-
petencies (competences), skills, literacy (etc.), we use the term digital literacy when wish-
ing to refer to these related competences in an inclusive meaning.

Materials and methods

We included both global citation databases Scopus and Web of Science Core Collection
(WOS). Scopus is a uniform database which permits searches throughout its collection
on equal principles. WOS now also offers access to its records through a uniform plat-
form (Core Collection) which currently incorporates records from Conference Proceedings
Citation Index, as well as Book Citation Index. WOS products were previously accessi-
ble through separate subscription policies. For example, conference proceeding were not
accessible on the same platform as journals [SCI-Expanded, Social Sciences Citation Index
(SSCI), Arts and Humanities Citation Index (A&HCI)]. The integrated version can now be
launched as a default option so end-users now no longer need to select separate products.
Thus, both databases can now be compared on more equivalent principles than previously.
It is possible to select different document types, such as Articles, Reviews, Proceedings.
Our pilot search detected proceedings as an important source of information in this field so
we employed all document types in order to facilitate comparison between databases which
sometimes classify document types differently.
We paid special attention to the formulation of search statement (query or syntax).
Authors frequently limit their queries only to a few choice phrases such as information
literacy or digital literacy. We however felt that such a limitation excludes many docu-
ments of importance which tackle other similar sets of skills. After an extensive prelimi-
nary research of more frequently occurring terms in abstracts we included the concepts
presented in Table 1. The skill-related concepts were searched for as phrase. Altogether, we
tested 15 different skill concepts in relation to five educational concepts.
We wished to keep the syntax as normalized as possible in order to facilitate repeatabil-
ity. Some tradeoffs were nevertheless involved. For example, phrase-search (such as digital

Table 1  Literacy-, skill-, and competence-related concepts in co-occurrence with secondary-education-

related concepts
Skill-related (15 permuted phrases) Education-related (5 phrases)

“computer competen*” AND “high school*”

“data (literac* OR literate*)” “middle education”
“digital skill*” “middle school*”
“information “secondary education”
“ICT “secondary school*”

The complete search statement was designed as follows: [(“computer competen*” OR “computer literac*”
OR “computer literate*” OR “computer skill*” OR “data competen*” OR “data literac*” OR “data liter-
ate*” OR “data skill*” OR “digital competen*” OR “digital literac*” OR “digital literate*” OR “digital
skill*” OR “ICT competen*” OR “ICT literac*” OR “ICT literate*” OR “ICT skill*” OR “information
competen*” OR “information literac*” OR “information literate*” OR “information skill*”) AND (“high
school*” OR “middle school*” OR “middle education” OR “secondary school*” OR “secondary educa-
tion”)]

13
Scientometrics

literacy) has some limitations since relevant combinations of terms are also possible outside
the strict context of a phrase. We thus tested adjective forms, such as digitally literate but
unique occurrences were very rare and usually occurred together with other concepts and
would have been retrieved anyway.
On the other hand, not employing phrases would yield far too much noise so the search pre-
cision would be very low by returning too many unrelated documents. Optimization of such
queries is thus essential. We employed both “information literac*” as well as “information
literate*” in order to integrate terms “information literacy” as well as “information literate(s)”
and exclude terms such as “information literature”. Other concepts would have been possible,
for example, internet skill, Google skills, search skills, media literacy. But these terms are too
specific and restrictive and do not cover all aspects of more general digital competences or
information literacy.
Regarding the secondary-education-related concepts, our usage of these terms also applies
to middle schools or middle education (Table 1). Middle-related terms can denote junior high
schools, although not always. As junior high schools are nevertheless retrieved by the above
syntax thus the term middle also needs to be employed for the purposes of consistency. Even
though a majority of papers is dedicated to the more ‘standard’ secondary education (‘sen-
ior’ high schools) there is an added value in including also possible junior high (and middle)
schools. With regard to the search statement we should note that search queries involve trade-
offs. If a query is too short then many possibly relevant documents are lost. If it is too long
then it invariably involves some search noise.
On careful consideration we decided to run the searches in both databases with the com-
bined Topics field (Article title, Abstract, Keywords) although such results may not be fully
consistent for the purposes of database comparison—on account of different indexing meth-
ods in respective databases. Both databases employ authors’ keywords. In addition, WOS also
includes KeyWords Plus (KW +) which is a proprietary WOS methodology. Scopus harvests
additional keywords from thesauri. Thus, the same keywords may sometimes retrieve different
documents in either database. The queries based on abstracts-only would be more consist-
ent, however, there is no such possibility in WOS. Authors of similar studies employ differ-
ent methods which all entail tradeoffs. Titles retrieve too few documents although the search
precision is better. Title-retrieval is more suitable for concepts which return high numbers of
papers (Bartol and Stopar 2015).
We downloaded all documents based on the search statement (Table 1) without any limita-
tion for the initial year as there was no need for it even though both databases have different
coverage of early periods as Scopus is a newer database. Namely, in both databases, there were
very few relevant documents prior to 2000. Also, both databases are being constantly updated
for older documents. Our searches were run in the beginning of 2018 so they basically contain
the data from the database beginnings to the end of 2017. At this point it should be noted that
some 2017-documents are still arriving in the first months of 2018. Hardly any records could
be found to the effect of data-related literacies. However, we still used these concepts in the
retrieval of total records as data-literacy was a suggested topic in the program of annual ECIL
conferences (European Conference on Information Literacy) (Kurbanoğlu et al. 2018). Pos-
sible developments to this end may be monitored in the future.

Scattering of publications

Based on a pilot query, a strong manifestation of conference proceedings was expected.

First step was thus to ascertain publishing patterns in each database. This is shown in the

13
 Scientometrics

corresponding subsection of the Results section. The pilot study indicated strong conver-
gence of relevant papers in only a limited number of core publications. We thus first tested
the principles of logarithmic power laws in order to detect possible inverse proportion of
frequency and terms. In our case, we investigated if a small number of relevant publication
titles (core publications) really accounts for a major part of all relevant information, and
if, on the other hand, very large group of publication titles publishes very few or even only
one relevant paper each. This analysis involved both Scopus and WOS.
To this end, we aligned publications in both databases on the same scale and assessed if
the curves exhibit power-law patterns as described by Bradford (1934) who graded sources
into three classes, the first class being represented by a only a few very productive sources.
According to these principles, no definite boundaries between classes can however be indi-
cated as the classes depend on a scientific field. The observations by Bradford are now
known as Bradford’s law of scattering where journals in the field are represented by a core
(first zone) of a few journals [1], a second zone with more journals [n], and a third zone,
with the bulk of journals [n2]. The theoretical formula is generally presented as: 1:n:n2
(Black 2004).
We performed the same analysis also on broader subject classification of publications
as in both databases all publications are mapped to one or several broader research fields
or disciplines. Broader classification in WOS is based on Research Areas which can unify
several topically related categories. In Scopus, Subject Areas are employed to this end.

Mapping and clustering

The initial exploratory assessment of the scatter of publications in Scopus and WOS was
followed by complex bibliometric mapping in WOS which is presented in the correspond-
ing second subsection of the result section. We employed VOSviewer. This program ena-
bles creation, visualization and exploration of bibliometric maps of science (Van Eck and
Waltman 2010). It manages mapping as well as clustering (Waltman et al. 2010). Both
citation databases provide similar functionalities for visualizations of text data and biblio-
graphic data, and are used individually, according to accessibility or an author’s choice.
Complex visualizations are usually not repeated in both databases simultaneously. We
decided to use WOS where we visualized different networks and constructed respective
maps based on text data (network visualization, overlay visualization) as well as biblio-
graphic data (bibliographic coupling network of citing sources, co-citation network of cited
sources).

Maps based on text data (term maps)

In the first part of bibliometric mapping we prepared the maps using an automated term-
identification technique (Van Eck and Waltman 2011). We identified terms (concepts)
i.e. noun-phrases in the titles and abstracts of the 585 WOS documents. Noun-phrase is
a sequence of words where the last word in the sequence is a noun and each other word
is either a noun or an adjective. We took into account phrases which occurred at least 12
times. In the set of 12,147 noun phrases there where 318 that met this threshold, and for
which we calculated relevance score. The 191 (60%) most relevant concepts were selected
for the representation on the map. We excluded a few less relevant highly occurring general
terms which had no specific meaning. These were marked by low relevance score in the
program.

13
Scientometrics

1. Network visualization

First we constructed term map (established on the co-occurrence frequencies) in order to

detect clusters of related research (concepts). This is a two dimensional map where the dis-
tance between any two concepts reflects the relatedness. The stronger the relation between two
concepts, the smaller the distance in the map. Each concept in this term map also is marked by
a distinct color indicating a cluster of related concepts. The terms with the same color tend to
co-occur with each other more frequently.

2. Overlay visualization

In the next step, we constructed a map which monitors possible characteristic developments in
time. This was done using overlay visualization which is the same as previous visualization of
terms except that items are colored differently. If items have ‘scores’ (a publication year in our
case) the average year then determines the color of an item.

Maps based on bibliographic data

In the second part of bibliometric mapping we first identified principal publications which
cite other publications (bibliographic coupling analysis). We then identified those publications
which are the most cited (co-citation analysis).

1. Bibliographic coupling network

In the bibliographic coupling network of ‘citing’ sources, two publications are coupled if there
is a third publication that is cited by both publications (Kessler 1963). VOSviewer asks for the
minimum number of publications which are defined by the same source (threshold) in order to
be included in the network. The suggested threshold is optimized by the program in order to
limit overlapping. In our data set, there were 22 such sources which co-cited another publica-
tion at least four times.

2. Co-citation network

In co-citation network of ‘cited’ sources, two publications are co-cited if there is a third pub-
lication that cites both publications (Small 1973). In this part of analysis we chose the default
value (at least 20 citations per source). Of the 9288 sources, 87 meet this threshold.
In both networks, relatedness is indicated by distance and lines. In general, the closer two
publications are located to each other, the stronger their relatedness in terms of bibliographic
coupling links or co-citation links. Lines indicate the strength of co-citation links between
journals. The subsections in the methods section and results section are numbered and labelled
in similar sequences in order to facilitate the tracking of successive procedures.

Results

Altogether, 894 Scopus records and 585 WOS records were retrieved. All document types
and all years were included. The construction of search query is explained in the Methods
section (Table 1). The results present two basic aspects:

13
 Scientometrics

1. Assessment of the scattering of publications according to Bradford-like power-laws

evaluates both Scopus and WOS records (Figs. 1, 2, Table 2).
2. Bibliometric mapping tackles WOS records and identifies clusters of related research,
factors of a time period, and networks of relations between citing as well as cited sources
(Figs. 3, 4, 5, 6, 7).

Scattering of publications

Figure 1 presents those publication titles which published at least two papers referring to
computer-, digital-, ICT-, information- in phrasal relationship with competencies (com-
petences), skills and literacy (Table 1) in relation to secondary education (also including
terms such as high schools etc.). We notice that both database curves are similar and show
power-laws characteristics of inverse proportion of frequency and terms. A few core pub-
lications account for an important share of all documents—left side of Fig. 1. On the right
side there exists a long list of publications which published only two papers in the entire
period. In addition, there remains a very long ‘tail’ of publications, in both databases,
which only published a single document relative to the subject matter in this study. The
curve is longer in Scopus than in WOS because there are more documents in Scopus.
Some database limitations apply. In WOS, the analytics orders the records by Sources
(Source titles). Documents published in proceedings-series are presented twice in analyt-
ics: once under the unique ISSN/ISBN Book Series and then again under the name of each
conference. For example, in WOS, 24 records have been mapped to the Book Series INTED
Proceedings, and have then also been mapped to yearly events, such as INTED2017 (3
records), INTED2016 (5 records), etc. In order to show correct numbers (and avoid dupli-
cates) we mapped all yearly conferences to the same source publication. In Scopus, some
documents occurring on the faceted left-hand side were not comprised in the proprietary
analytics. We thus needed to include such documents “manually”.

Fig. 1  Documents (papers) published per publication in Scopus and WOS and the number of different pub-
lication titles which published at least two papers each

13
Scientometrics

Fig. 2  Documents (papers) mapped to Scopus Subject Areas and WOS Research Areas

Table 2  Publications which published ten or more documents (papers)

WOS Documents Scopus Documents

Computers and education 45 Computers and education 52

INTED (proceedings) 24 CCIS (proceedings) 26
Procedia (proceedings) 24 Turkish Online J. of Edu. Techn. 16
ICERI (proceedings) 22 ACM (proceedings) 12
EDULARN (proceedings) 19 LNCS (proceedings) 11
CCIS (proceedings) 12 J. of Libr. and Inf. Sci. 10
J. of Libr. and Inf. Sci. 10

The two logarithmic curves are very similar. A further analysis, however, shows
that the core publications are not the same. Substantial differences between databases
exist—notably in the coverage of Books Series (Proceedings) which obviously play a
major role in this research field. Table 2 presents publications which published at least
10 documents each. In Fig. 1, these come about at the very left of the X-axis.
Each database thus indexes several different core documents and publications which
are not included in the other database. Sometimes, different documents from the same
publication are indexed. For example the WOS-indexed documents from highly rank-
ing INTED Proceedings are not indexed by Scopus. Scopus exhibits better coverage of
some other conference series, as well as many national and regional journals which are
not (yet) included in WOS. The latter fact is frequently mentioned in literature when
both databases are compared.

13
 Scientometrics

Fig. 3  Clusters of related concepts in WOS (network visualization)

Proceedings were usually published in Book Series. They were classified as ‘conference
papers’ in Scopus (220 of the 894 total documents) and ‘proceedings papers’ in WOS (190
of the total 585 documents). Proceedings are sometimes excluded from similar studies as
they are not mapped to citation indicators such as Impact Factor/JCR (WOS) and SNIP
(Scopus). It is evident, however, that proceedings play substantial role in the field of digital
literacy and education and cannot be ignored.
The fluid designation of ‘proceedings’ document type may be, for example, noted in the
CCIS series (Communications in Computer and Information Science). This book series is
in Scopus supplied with SNIP and SJR, and ranked accordingly: as a journal (serial). In
WOS, on the other hand, this is only a Book Series and is not rated by JCR. The current
version of WOS Core Collection now nevertheless includes also these document types and
thus provides a much better uniform coverage of research fields which were in the past
neglected by the more traditional journal-based Science Citation Indexes.
Next step was to assess broader subject categories (Fig. 2). In WOS, these are repre-
sented by Research areas. Some research areas merge several associated WOS categories
which are narrower in scope. For example, research area of Education (Edu.) and Educa-
tional Research is represented by three different categories: Edu.-Educational Research,
Edu.-Scientific Disciplines and Edu.-Special. Computer science is represented by seven
categories. Information Science and Library Science is represented only by the eponymous
category. Scopus classification is less detailed in terms of analytics and is based on Sub-
ject areas. A very general subject area of Social Sciences is thus assigned to educational

13
Scientometrics

Fig. 4  Prevailing concepts in a given average time-period in WOS (overlay visualization)

Fig. 5  Number of retrieved records using the respective queries in WOS

13
 Scientometrics

Fig. 6  Bibliographic coupling network of citing sources in WOS

contexts as well as information- and library-related contexts. Computer science, on the

other hand, is a separate subject area.
Just as in the preceding ranking of publications (Fig. 1), we can again observe char-
acteristics of inversely-proportional power-laws. In WOS, the area of Education (E) and
Educational Research is the strongest area, followed by Computer science and Information
Science and Library Science. More than half of all documents under study were thus pub-
lished in publications which had been mapped to the educational subject categories. There
is again a long ‘tail’ of publications which had been mapped to rarely occurring areas.

13
Scientometrics

Fig. 7  Co-citation network of cited sources in WOS

This has been observed also in other scientific fields (Stopar et al. 2016). Four areas (for
example, Environmental Sciences Ecology) designated two papers each. Nineteen areas
(for example, Criminology Penology) labelled only one paper each. Figure 2 presents only
every other area (for the purposes of readability). An insert within the figure then pre-
sents all areas with higher occurrence. A publication can sometimes be mapped to two or
more different areas. To this end we conducted some further exploratory analysis where we
observed very weak convergence between Education on one side and Information Science
and Library Science on the other. Only four papers were classified with both categories.
There is some more convergence with Computer science which is however weak.
Classification is much less detailed in Scopus. Most documents under study have been
mapped to Social Sciences, followed by Computer Science. Figure 2 shows only the ten
most productive areas. These are again followed by a long tail of rarely occurring areas.
This comparison of both databases shows that the same publications can be classified quite
differently in either database. This must be taken into account if these classifications are
used in subject evaluation.

Mapping and clustering

The preceding assessment of the scattering of publications shows only a part of the picture
since it does not provide an information on interactions. To this end, we employed complex
clustering techniques in order to visualize different relationships.

Text data

Term map in Fig. 3 (co-word analysis) is a result of network visualization and shows
clusters of concepts which are the most frequent to occur together in papers. We can

13
 Scientometrics

observe three distinct clusters of information literacies (blue), computer literacies

(green) and digital literacies (red). The fourth group of remaining miscellaneous con-
cepts (yellow) is not organized in a very distinct cluster and does not link a more strictly
defined set of concepts.
Information literacy is the prevailing concept in the blue cluster. Other information-
related synonymous concepts such as information skills or information competences occur
more rarely and are not shown. (this is defined by the program in order to diminish overlap-
ping and facilitate legibility of the map). This cluster highlights the importance of libraries
in teaching information literacy in secondary education. The concept information literacy
is in this frame no longer just a keyword but a consistently used descriptor.
Red cluster converges around the concept of digital competence. Visualization is
defined by the program so for some items the label (concept) is again not displayed in order
to reduce overlapping. Importantly, this cluster is connected with several education- and
learning-related concepts.
Green cluster displays computer-literacy and -skills. ICT-skills and -competences can
also be noted as well as communication technology. This cluster displays fairly frequent
occurrence of gender-related concepts such as gender, gender differences, girl, male indi-
cating specific aspects of computer-related research in connection with specialized social
sciences topics.
The previous part of co-word analysis identified clusters which show possible distinct
fields of research. But are there perhaps other components which play part, such as time-
period as posed in our research question?
To this end, we visualized the same items by the so-called overlay visualization where
the program highlights (through a color scale) the items according to the periods in time
(Fig. 4). Colors range from blue (early publication period), green (medium period), to red
(recent publication period). The colors show that prevailing concepts (major terms) in
Fig. 3 are dominant, on average, in different periods of time (average year of occurrence).
These time periods are then presented by and additional analysis (Fig. 5) which corrobo-
rates the results in Fig. 4.
Computer literacy (and related) are shown in blue in Fig. 4 which means that these con-
cepts occur already before 2010, and have been used the longest. This is also reflected in
Fig. 5: by the year 2010, there were already 56 such documents. Figure 5 then shows that
during the last decade such records have not grown very substantially. This can also be
inferred from the blue color in Fig. 4.
ICT-related literacies, which come about for the first time in 2000, exhibit in the last
period stronger presence than computer-related. These two groups of concepts are likely
associated as they occur in the same cluster in the previous Fig. 3. Figures 4 and 5 then
show that ICT-related concepts are in this recent period more frequently used than
computer-related.
Information-related literacies became important in the mid-period which is marked by
green color in Fig. 4. Although this concept has been around as long as computer-related
literacies and skills (information-related return 35 results by 2010), it suddenly surpasses
computer-related in the mid-period. The supporting Fig. 5 indeed shows that there is a
clear growth only in 2011 which then becomes fairly steady and not very accelerated.
Information-related are still very significant concepts at present but have been very recently
overtaken by digital-related concepts.
Digital competence comes about in a fairly recent period, occurring for the first time
only 10 years ago when computer and related concepts have already begun to stagnate.
Digital competence is not only the most recent ‘trendy’ concept—it also shows a very rapid

13
Scientometrics

growth, surpassing all other related concepts (Fig. 5). This is substantiated by red color in
Fig. 4 which designates current popular topics (“hot topics”).
These different concepts (computer-, digital-, information, ICT-related) may perhaps
seem distinct as shown by different clusters in previous Fig. 3. Based on our observations
in Fig. 4 and verification thereof in Fig. 5 we tentatively assume that a time-period plays
an important role in the choice of terminology. Or, in other words, terminology may evolve
and begin to employ different terms for the fields of research which are perhaps not so
different.
These assumptions are based on visualizations as mapped by the program. Such maps
should always be interpreted with limitations in mind. Term maps invariably contain both
specific as well as more general terms. Terms can thus interconnect also hierarchically
which cannot be shown in such visualizations. Limitations notwithstanding, the data in
both Figs. 4 and 5 are nevertheless in good agreement.

Bibliographic data

In the second and concluding part of bibliometric mapping we assessed sources (publi-
cations) which co-cite other sources (bibliographic coupling network) as well as sources
which are the most cited (co-citation network). These two visualizations address our last
research question which aimed to evaluate the relationships between the citing and the
cited sources. These relationships are expressed in similarities as well as differences which
are described in the following paragraphs.
Figure 6 presents bibliographic coupling network. Conference proceedings play impor-
tant role in all clusters and were also identified as core publications (Table 2). They pro-
vide important citation counts to other publications. We can observe distinct clusters. Com-
puter-education-related publications (red cluster) come about in the same cluster as they
co-cite similar publications. Proceedings in the blue cluster also show strong co-citation
patterns. Both clusters are located close to each other indicating related citation patterns. A
small yellow cluster is more dispersed and less substantial as it includes only a few records.
Journal Computers and Education is the most strongly present among all publications.
We can note a clear library-and-information-related cluster (green) which is distant
from the aforementioned clusters although connection exist which is indicated by lines.
We assume that this cluster is connected with information-literacy concepts as found in
previous Fig. 3. Weak connections and distances between the red and blue clusters on one
side and green cluster on the other indicate that there is less convergence and exchange of
information between researchers in these areas.
Figure 7 presents co-citation network which is based on cited sources. In this network,
two publications are co-cited if there is a third publication that cites both publications. We
can again observe clusters with shared co-citation patterns. In contrast to the preceding
analysis of citing publications (Fig. 6), this network contains mostly journals. Proceedings
are co-cited weakly and are not shown on this figure. The blue cluster is strongly domi-
nated by the journal Computers and Education which is again the most important publica-
tion in the network. We can also observe another education-related cluster (red) of co-cited
journals. This cluster is also connected with the cluster dominated by Computers and Edu-
cation. A smaller (violet) cluster of ‘science education’ can also be noticed. We can detect
an interesting yellow cluster with a convergence of journals in the scope of social sciences
and humanities. Medical/health-related issues are also located in this cluster. Finally, we
can again outline a separate cluster of library-and information related journals. This cluster

13
 Scientometrics

is again the most distant indicating strong co-cited connections within the field of library
science but weak co-cited connections with publications in other clusters. This cluster
nevertheless links also to the blue cluster (computers, education). The other clusters are
located much closer to each other thus indicating more relatedness in citation patterns.
We can summarize the following similarities and differences between the clusters of cit-
ing and cited sources:

• the journal Computers and Education is both the most important citing source in this
field as well as cited source; it has a substantial impact on both citing as well as cited
clusters
• frequently citing sources are different than frequently cited sources: for example, pro-
ceedings contribute strongly to citations of journals although they weakly frequently
cited themselves
• the citing sources form three distinct clusters, two among which are located close to
each other denoting similarities in citing patterns
• the cited sources form several clusters; most clusters, although distinct, show links to
computers/education
• in both networks (citing sources as well as cited sources), library and information
sources form fairly divergent (separate) clusters

Based on figures, we can tentatively conclude that in the sense of education, computers,
information science and library science (in the secondary education context) there exist
two different asymmetric realms with weak co-operation.

Discussion and conclusions

As most digital literacy studies had been conducted in the scope of higher education we
decided to investigate how it applies to secondary education. This has been tackled much
more rarely even though many more students are enrolled in the secondary education. In
fact, even many institutions of higher education have not yet fully embraced digital literacy
as a foundational literacy (Murray and Perez 2014). There exist many relevant schemes
which target and assess these literacies in the context of secondary education, for example
ICILS, but higher education cases nevertheless prevail which may be due to the fact that
authors of such studies are frequently also educators who are used to academic publishing
which, in turn, forms an integral part of their profession where it is frequently essential for
career promotion.
The first challenge in our study was the construction of a comprehensive search query.
Many authors limit such research to concepts which are too restrictive to efficiently
embrace the many possible digital, information, computer (etc.) literacies, competences
and skills in educational contexts. Besides, terminology sometimes seems almost fractal.
Janssen et al. (2013) identify as many as twelve areas and definitions related to the term
digital competence alone. Pegrum (2010) talks about ‘network literacy’ as a core digital
literacy. On top of that, there exist different interpretations of concepts in research on one
side and professional practice on the other (Pilerot 2014). This entropy is exacerbated fur-
ther by random use of either competences or competencies (Halász and Michel 2011). Our
research has essentially employed those terms which are most frequently used by authors
what is also shown in the visualizations. We have not included more specific and restricting

13
Scientometrics

contexts, for example, media literacy which often comes about as a subject on its own. We
intend to investigate this terminological challenges in our future research. Challenges were
also represented in the choice of educational terminology (middle vs. high vs. secondary)
which is internationally not completely harmonized. Thus, our study also included middle-
related concepts.
Our study has shown that the publishing patterns in this field exhibit logarithmic inverse
relationship between the frequency and rank: only a few chief publications (among hun-
dreds) account for an important share of all published research, followed by a long tail of
papers which were published in random publications. In our case it is worth noting that the
principal (core) publications in both databases (Scopus and WOS) are not necessarily the
same. We have also found strong representation of proceedings among the core publica-
tions although it is generally understood that journals are essential. Similar logarithmic
patterns can also be perceived in research areas. This is more revealing in WOS which
possesses a more detailed classification than Scopus. Scopus only offers two applicable
broader areas: Computers Science and Social Sciences which seemingly include informa-
tion science, library science as well as education. In WOS, Education and Educational
Research is thus the principal research area for the field under our study, followed by
Computer Science, and Information Science and Library Science. Journals indexed with
other WOS research areas carry less significance. Here we might mention that highly
ranked publications are believed to provide a description of the core cognitive areas in a
field (Milojević et al. 2011). Links between major areas, especially between Education
and Computers on one hand and Information Science on the other seem very weak even
though educational and computer papers employ similar concepts as information science
and library papers.
Visualization algorithms have shown several clearly defined clusters of seemingly
distinct research topics. But are these topics really so specific? We have thus examined
these topics also for possible specifics of timeline. It turns out that the most frequent terms
(information or computer literacy, digital skills etc.) are in front in different time-periods.
The computer-literacy-based cluster is the ‘oldest’ indicating that these activities emerged
the earliest and were very active already before 2010. Information literacy is articulated
later. The most recent cluster, and now the most active one, revolves around digital com-
petences. As yearly trends cannot be revealed by visualizations we run an additional com-
plementary time-series analysis in WOS which corroborates this findings. Digital-related
“buzzword” concepts are now indeed the most frequent and represent “hotspot” compe-
tences, literacy and skills. They surge very rapidly only after 2010 even though the term
has already been around in the 1990s (Bawden 2001). Computer literacy is now trailing
behind the other literacies. This certainly does not mean that computers are no longer rel-
evant. Just the opposite, they are now taken for granted as the subject has matured. Or, in
other words, the subject has been sufficiently described (Tibaná-Herrera et al. 2018). In
conclusion, instead of representing very specific areas of research these clusters seem to
reflect a progress in terminology rather than an important shift in research topics.
The concluding two visualizations address relations between the citing and cited
sources. They address bibliographic data and citation patterns. Visualization of networks
again exhibit clusters of related research. This is represented by sources (for example, jour-
nals and proceedings). The networks are marked by noteworthy differences. Proceedings
are important contributors of citations although less frequently cited themselves. Confer-
ences therefore play an important role in this field and are a source of citation counts for the
journals thus boosting their scientometric rankings. We expect the role of proceedings to
remain important also in the future especially given their inclusion in the integrated version

13
 Scientometrics

of the WOS Core Collection. Cited publications are dominated by journals. What is most
notable is that library and information cluster on one hand and education- and computer
clusters on the other are quite distant from each other in both networks even though links
exist. It seems as if the researchers in this field investigated the subject of digital competen-
cies in secondary education in two disconnected contexts. We tentatively assume that at
least a part of it is due to the dominance of WOS categorization scheme which then defines
co-operation in respective fields. But are these fields really so different? After all, the terms
under our study, all in relation to education are used in educational and computer-related
as well as library and information publications. And yet this is only weakly reflected in
citation patterns. Each research area favors specialized core publications. The exchange of
experiences and findings is thus rare instead of providing a basis for interaction.
By detecting and explaining some specific publishing and citation patterns related to
digital literacy and secondary education our study wishes to bridge the still existing gap
between different research narratives. It invites researchers to look beyond their more nar-
row domains and create synergy. Such studies should address and assess digital-, infor-
mation-, ITC-, computer-, and related competencies already during secondary education.
Educational research, library and information science as well as computer science should
be involved in a collaborative way. These competencies serve as basis for further scientific
and societal progress. An attempt to meet these needs during higher education comes late
as many precious opportunities have by then already been missed.

Funding This study was funded by the Slovenian Research Agency (ARRS) (Grant Nos. J5-8230, P4-0085).

References
Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding (JRC Technical
Notes (JRC 67075)). Luxembourg: European Commission (JRC-IPTS).
Alzafari, K. (2017). Mapping the literature structure of ‘quality in higher education’ using co-word analysis.
Quality in Higher Education, 23(3), 264–282. https​://doi.org/10.1080/13538​322.2017.14186​07.
Bartol, T., & Stopar, K. (2015). Nano language and distribution of article title terms according to power
laws. Scientometrics, 103(2), 435–451. https​://doi.org/10.1007/s1119​2-015-1546-1.
Bawden, D. (2001). Information and digital literacies: A review of concepts. Journal of Documentation,
57(2), 218–259.
Black, P. E. (2004). Bradford’s law. Dictionary of algorithms and data structures [online]. https​://xlinu​
x.nist.gov/dads/HTML/bradf​ordsL​aw.html. Accessed 23 March 2018.
Bradford, S. C. (1934). Sources of information on specific subject. Engineering, 137, 85–86.
Bušelić, V., & Zorica, M. B. (2018). Information literacy quest. In search of graduate employability. Com-
munications in Computer and Information Science (CCIS), 810, 98–108. https​://doi.org/10.1007/978-
3-319-74334​-9_11.
Garfield, E. (1980). Bradford law and related statistical patterns. Current Contents, 19, 5–12.
Guenther, J. T. (2006). Mapping the literature of nursing informatics. Journal of the Medical Library Asso-
ciation: JMLA, 94(2 Suppl.), E92–E98.
Halász, G., & Michel, A. (2011). Key Competences in Europe: Interpretation, policy formulation
and implementation. European Journal of Education, 46(3), 289–306. https​://doi.org/10.111
1/j.1465-3435.2011.01491​.x.
Heradio, R., de la Torre, L., Galan, D., Cabrerizo, F. J., Herrera-Viedma, E., & Dormido, S. (2016). Virtual
and remote labs in education: A bibliometric analysis. Computers & Education, 98, 14–38. https​://doi.
org/10.1016/j.compe​du.2016.03.010.
Janssen, J., Stoyanov, S., Ferrari, A., Punie, Y., Pannekeet, K., & Sloep, P. (2013). Experts’ views on digi-
tal competence: Commonalities and differences. Computers & Education, 68, 473–481. https​://doi.
org/10.1016/j.compe​du.2013.06.008.
Kessler, M. M. (1963). Bibliographic coupling between scientific papers. Journal of the Association for
Information Science and Technology, 14(1), 10–25.

13
Scientometrics

Kokol, P., Saranto, K., & Vošner, H. B. (2018). eHealth and health informatics competences: A systemic
analysis of literature production based on bibliometrics. Kybernetes, 47(5), 1018–1030. https​://doi.
org/10.1108/K-09-2017-0338.
Kolle, S. R. (2017). Global research on information literacy: A bibliometric analysis from 2005 to 2014.
Electronic Library, 35(2), 283–298. https​://doi.org/10.1108/EL-08-2015-0160.
Kumar, K. (2014). A scientometric study of digital literacy in online library information science and
technology abstracts (LISTA). Library Philosophy and Practice, Paper 1044.
Kurbanoğlu, S., Boustany, J., Špiranec, S., Grassian, E., Mizrachi, D., & Roy, L. (2018). In Information
literacy in the workplace (5th European conference, ECIL) (CCIS/communications in computer
and information science, vol. 810). Springer. https​://doi.org/10.1007/978-3-319-74334​-9.
Li, F., Li, M., Guan, P., Ma, S., & Cui, L. (2015). Mapping publication trends and identifying hot spots
of research on Internet health information seeking behavior: A quantitative and co-word bicluster-
ing analysis. Journal of Medical Internet Research. https​://doi.org/10.2196/jmir.3326.
Majid, S., Yun-Ke, C., Aye, H. N., Khine, M. M. W., & Wai, S. Y. (2015). Analyzing publishing trends
in information literacy literature: A bibliometric study. Malaysian Journal of Library and Informa-
tion Science, 20(2), 51–66.
Milojević, S., Sugimoto, C. R., Yan, E., & Ding, Y. (2011). The cognitive structure of library and infor-
mation science: Analysis of article title words. Journal of the Association for Information Science
and Technology, 62(10), 1933–1953. https​://doi.org/10.1002/asi.21602​.
Murray, M., & Perez, J. (2014). Unraveling the digital literacy paradox: How higher education fails at the
fourth literacy. Issues in Informing Science and Information Technology. https​://doi.org/10.28945​
/1982.
Olmeda-Gómez, C., Ovalle-Perandones, M.-A., & Perianes-Rodríguez, A. (2017). Co-word analysis and
thematic landscapes in Spanish information science literature, 1985–2014. Scientometrics, 113(1),
195–217. https​://doi.org/10.1007/s1119​2-017-2486-8.
Pegrum, M. (2010). ‘I link, therefore I am’: Network literacy as a core digital literacy. E-Learning and
Digital Media, 7(4), 346–354. https​://doi.org/10.2304/elea.2010.7.4.346.
Pilerot, O. (2014). Connections between research and practice in the information literacy narrative: A
mapping of the literature and some propositions. Journal of Librarianship and Information Sci-
ence, 48(4), 313–321. https​://doi.org/10.1177/09610​00614​55914​0.
Pinto, M., Escalona, M. I., Pulgarín, A., & Uribe-Tirado, A. (2014). The scientific production of Ibero-
American authors on information literacy (1985–2013). Scientometrics, 102(2), 1555–1576. https​://
doi.org/10.1007/s1119​2-014-1498-x.
Price, D. J. D. S. (1965). Networks of scientific papers. Science, 149(3683), 510–515.
Shenton, A. K., & Hay-Gibson, N. V. (2009). Bradford’s Law and its relevance to researchers. Education
for Information, 27(4), 217–230. https​://doi.org/10.3233/EFI-2009-0882.
Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective
by learning from the past—A systematic review of assessment instruments that aim to measure pri-
mary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84. https​
://doi.org/10.1016/j.edure​v.2016.05.002.
Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two
documents. Journal of the American Society for Information Science, 24(4), 265–269. https​://doi.
org/10.1002/asi.46302​40406​.
Steinhardt, I., Schneijderberg, C., Götze, N., Baumann, J., & Krücken, G. (2017). Mapping the quality
assurance of teaching and learning in higher education: The emergence of a specialty? Higher Edu-
cation, 74(2), 221–237. https​://doi.org/10.1007/s1073​4-016-0045-5.
Stopar, K., Drobne, D., Eler, K., & Bartol, T. (2016). Citation analysis and mapping of nanoscience and
nanotechnology: Identifying the scope and interdisciplinarity of research. Scientometrics, 106(2),
563–581. https​://doi.org/10.1007/s1119​2-015-1797-x.
Taşkin, Z., Doǧan, G., & Şencan, I. (2013). Analyzing the intellectual structure of world information
literacy literature through citations and co-citations. Communications in Computer and Information
Science (CCIS), 397, 54–60. https​://doi.org/10.1007/978-3-319-03919​-0_6.
Tibaná-Herrera, G., Fernández-Bajón, M. T., & de Moya-Anegón, F. (2018). Global analysis of the
E-learning scientific domain: a declining category? Scientometrics, 114(2), 675–685. https​://doi.
org/10.1007/s1119​2-017-2592-7.
Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliomet-
ric mapping. Scientometrics, 84(2), 523–538. https​://doi.org/10.1007/s1119​2-009-0146-3.
Van Eck, N. J., & Waltman, L. (2011). Text mining and visualization using VOSviewer. ISSI Newsletter,
7(3), 50–54.

13
 Scientometrics

Van Laar, E., Van Deursen, A. J. A. M., Van Dijk, J. A. G. M., & De Haan, J. (2017). The relation between
21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior,
72, 577–588. https​://doi.org/10.1016/j.chb.2017.03.010.
Waltman, L., van Eck, N. J., & Noyons, E. C. M. (2010). A unified approach to mapping and cluster-
ing of bibliometric networks. Journal of Informetrics, 4(4), 629–635. https​://doi.org/10.1016/j.
joi.2010.07.002.
Wang, Q. (2018). Distribution features and intellectual structures of digital humanities: A bibliometric anal-
ysis. Journal of Documentation, 74(1), 223–246. https​://doi.org/10.1108/JD-05-2017-0076.
Weinerth, K., Koenig, V., Brunner, M., & Martin, R. (2014). Concept maps: A useful and usable tool for
computer-based knowledge assessment? A literature review with a focus on usability. Computers &
Education, 78, 201–209. https​://doi.org/10.1016/j.compe​du.2014.06.002.
Zydney, J. M., & Warner, Z. (2016). Mobile apps for science learning: Review of research. Computers &
Education, 94, 1–17. https​://doi.org/10.1016/j.compe​du.2015.11.001.

13