Asia-Pacific Edu Res (2020) 29(1):9–19

https://doi.org/10.1007/s40299-019-00442-8

REGULAR ARTICLE

A Content Analysis of Computational Thinking Research:

An International Publication Trends and Research Typology
Kai-Yu Tang1 • Te-Lien Chou2,3 • Chin-Chung Tsai4,5

Received: 16 September 2018 / Accepted: 9 March 2019 / Published online: 21 March 2019
 De La Salle University 2019

Abstract This paper aims to provide a comprehensive
analysis of publication patterns on computational thinking
(CT) over two recent periods (period I: 2006–2012; period
II: 2013–2018). Based on keyword analysis, a total of 3798
(period I) and 7175 (period II) keywords were found.
Derived from the content analysis, a research typology of
two-period keywords was consolidated and framed
according to its attributes, including background settings,
domain-specific factors, and learning outcomes. Main
findings show as follows: (1) Regarding the research
background, students from secondary and higher education
are the main participants; and computer science, mathe-
matics, and engineering are the major subjects. (2) As the
domain-specific factors, game and peer collaboration were
found to be the main pedagogies, while web-based and
face-to-face learning environments were almost equally
referred to in CT research settings. However, compared
with traditional command-based tools, Scratch, Lego, and
Python were identified as the emerging visual-based pro-
gramming languages. (3) Finally, the keywords related to
learning outcomes were classified based on the Bloom’s
framework of three learning domains. First, knowledge and
mental understanding are the main goals in the cognitive
domain; motivation and attitude are the main tasks in the
affective domain; and social and communication skills are
the central outcomes in the training of psychomotor ability.
Further discussions and research directions are provided.
Keywords Computational thinking
Keyword analysis

Content analysis
International publication trends

Research typology
Introduction

Since Wing (2006) proposed the new concept of ‘‘com-

& Kai-Yu Tang putational thinking’’ (CT) and advocated that everyone
ky.nctu@gmail.com
develop the ‘‘ways to think like a computer scientist’’ from
1
Department of International Business, Ming Chuan kindergarten to higher education, the concept has now
University, 250 Zhong Shan N. Rd., Sec. 5, Shilin Dist., become one of the fundamental skills of the twenty-first
Taipei 111, Taiwan century (Kim et al. 2012; Shailaja and Sridaran 2015;
2
Graduate Institute of Digital Learning and Education, Shute et al. 2017). CT represents an attitude and skill for
National Taiwan University of Science and Technology, 43, solving problems, designing complex systems, and under-
Keelung Rd., Sec.4, Da’an Dist., Taipei 10607, Taiwan
standing human thoughts and behaviors that draw on
3
Department of Library and Information Science, School of concepts fundamental to computer science (Lye and Koh
Continuing Education, Fu-Jen Catholic University, 510,
Zhongzheng Rd., Xinzhuang Dist., New Taipei City 24205,
2014). Through programming practices, students learn to
Taiwan generalize abstractions, process information and detect
4 errors systematically, compose and decompose problems
Program of Learning Sciences, National Taiwan Normal
University, Taipei, Taiwan structurally, and think in iterative, recursive, and parallel
5 ways (Grover and Pea 2013). Moreover, students could
Institute for Research Excellent in Learning Sciences,
National Taiwan Normal University, 162, Section 1, Heping develop logical thinking skills and transfer them into
E. Rd., Da’an Dist., Taipei 106, Taiwan

123
10
different contexts to face more challenging problems in
real life (Voogt et al. 2015).
According to the search from the Web of Science
(WoS), research reviews on CT have gradually increased
since 2013. While some reviews have focused on K-12 to
higher education, others have summarized the definitions,
assessments, and research agenda. Based on a narrative
review, Grover et al. (2013) investigated the K-12 educa-
tion on CT in terms of environments and tools. They
concluded that educators needed to adopt graphical or
game-like programming tools to inspire students’ creativity
and problem-solving skills for K-12 learners. Regarding
the teaching strategies for CT education, Lye and Koh
(2014) surveyed 27 studies and found that CT was mainly
taught through programming. They concluded that the
scaffolding process is a key element in CT cultivation.
Using content analysis, two recent reviews provided more
in-depth analyses of CT characteristics, practices, tools,
and models. Buitrago Flórez, et al. (2017) analyzed 92
papers from the ACM digital library and selected journal
publications. The authors highlighted the importance of
peer-based collaborative environments and concept maps
for enhancing meaningful programming experiences. They
also concluded that programming can be used as a tool for
mind development, and pointed out some challenges faced
by early programmers, tools, and pedagogies used in
schools. Similarly, Shute et al. (2017) searched for research
papers from various databases, such as ERIC, JSTOR, and
Google Scholar. They selected 45 studies for review and
created a working definition. They defined CT as ‘‘the
conceptual foundation required to solve problems
Fig. 1 A line chart of CT
publications (2006–2018): the
amounts of journals articles and
authors. *The timespan of data
retrieval started from 2006 and
300
ends on August 10, 2018
250 2006-2012
average papers: 36.1
200 average authors: 79.7
150 124
100
76
47
48
50
22 34
24
10
0
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018*
Arcle growth stascs
123
K.-Y. Tang et al.
effectively and efficiently with solutions that are reusable
in different contexts.’’ Finally, they categorized CT skills
into six facets, including decomposition, abstraction,
algorithm design, debugging, iteration, and generalization.
To provide a complementary viewpoint to previous
review papers, the present study selected an academic-fo-
cused database from the WoS as the source of research
data. A starting year of 2006 was set to line up with the
year of Wing’s (2006) original paper, and the review ended
in the middle of 2018 to provide the cutting-edge research
status of CT. Last, the proposed typology was grounded on
a huge number of keywords to provide a comprehensive
survey of the literature development. This bottom-up
approach aims to provide solid evidence of the research
trends.
The main foci of this current study are twofold. First, the
focus was to profile international publication patterns of
CT research over two recent periods (period I: 2006–2012;
period II: 2013–2018). According to the line chart of CT
publications from 2006 to 2018, a slow growth trend in
terms of articles and authors occurs in the period I
(2006–2012), followed by more obvious increasing trend in
period II (2013–2018). As shown in Fig. 1, the cutoff point
was thus set between 2012 and 2013 to make a balance for
the analyzed years of publications. A balanced number of
years for the subsequent analysis and comparison led to a
lesser distortion of analysis. Moreover, this setting can
benefit researchers to observe the potential changes in CT
publication patterns in these two periods, such as interna-
tional productivity. The second focus was to identify the
most distinctive keywords as a typology for characterizing
2013-2018
average papers: 80.9
average authors: 186.7
299
209
190
164
144
114 129
94 102 101
96
75 73
57
45 47 45 48
Author growth stascs
A Content Analysis of Computational Thinking Research: An International Publication Trends and… 11

Table 1 The strategy of keywords used in the search

Step Keywords Results

1 TS = (‘‘comput* thinking’’) 136

2 TS = (‘‘comput* education’’ or ‘‘computer* science education’’) 394
3 TS = (‘‘comput* literac*’’) 321
4 TS = (abstraction or automation or decomposition or ‘‘thinking recursively’’) 318
5 TS = (‘‘programming’’ or ‘‘computer* program*’’ or ‘‘cod* thinking’’ or ‘‘algorithm* thinking’’ or ‘‘comput* intelligence’’) 838
6 Two original articles with 394 citing articles 396
7 #6 or #5 or #4 or #3 or #2 or #1 1611
8 Conducting a relevancy test for the search #7 715
In line with the education focus, the search of computational thinking was limited to the category of ‘‘education educational research’’ defined by
the Web of Science. Two original works published in computer science journals by Wing (2006, 2008) were included and treated as seeding
articles to include all their citing journal articles. Based on the setting of the search, research articles related to computational thinking from the
computer science domain to education and educational research were covered as much as possible. The timespan of the search period
(2006–2018) was set and conducted on August 10, 2018. Indexes = SSCI, SCI-expanded

CT research. This is a strategy for research review.
Adopted from earlier studies, researchers used to classify
highly used keywords for presenting research themes and
framework (Maida and Cheon 2014; Hsiao et al. 2015;
Park et al. 2016). However, rather than common used
wordings, this current study focus on selecting keywords
that can represent attributes of CT research. As such, two
methods were adopted to achieve the research goals. While
the keyword analysis is an initial attempt to systematically
profile all the keywords used in CT research, the following
content analysis is to identify the most critical keywords
for the comparison of CT development across the two
periods. In sum, after a brief introduction, the following
section illustrates the process of data retrieval and the
details of the used methods. Finally, the main results are
interpreted and discussed.
Data and Method
The process of data inclusion is critical for conducting a
content analysis of literature reviews. The Web of Science
(WoS) database, the major source of high quality indexed
academic journals that contains both the Sciences Citation
Index (SCI) and the Social Sciences Citation Index (SSCI)
databases, was explored in this study. This paper used a
systematic keyword search and a citation-based relevancy
test to secure our research data.
First, the keyword definitions were based on four highly
cited papers, including two original works on computa-
tional thinking authored by Jeannette Wing (2006, 2008),
and two recent research review papers (Grover and Pea
2013; Lye and Koh 2014). The keywords used for the
query included ‘‘computational thinking,’’ ‘‘computer
education,’’ ‘‘computer science education,’’ ‘‘computer
literacy,’’ ‘‘abstraction,’’ ‘‘automation,’’ ‘‘decomposition,’’
‘‘thinking recursively,’’ ‘‘programming,’’ ‘‘computer pro-
gram,’’ ‘‘coding thinking,’’ ‘‘algorithm thinking,’’ and
‘‘computing intelligence.’’ In addition, a strategy for the
use of keyword searching was applied as shown in Table 1.
The first five steps involved searching for papers that
included the keywords listed above. These multi-keyword
queries were to search for literature with the core concept
(i.e., computational thinking), its synonymous concepts/
terms (e.g., coding thinking), its practical competencies
(e.g., abstraction), and all possible settings or approaches
(e.g., computer education) step by step. The search was
limited to the category of ‘‘education educational research’’
as defined by the WoS to be in line with the education
focus. The truncation searching technique (*) was used to
take in keyword variations. For example, ‘‘comput*
thinking’’ returns results with ‘‘computational thinking’’
and ‘‘computing thinking’’; ‘‘comput* education’’ returns
results with ‘‘computer education,’’ ‘‘computing educa-
tion,’’ and ‘‘computational education.’’ As shown in
Table 1, these five steps found 136, 394, 321, 318, and 838
research papers, respectively. Next, in step 6, two original
articles of computational thinking (Wing 2006, 2008) were
included along with all citing articles to capture a com-
prehensive and the most up-to-date developments in the
literature. As such, we set the data-retrieval time to begin
from 2006 and to end on August 10, 2018. Based on the
keywords search, a total of 1611 articles were retrieved (in
step 7).
Second, the citation-based relevancy test was performed
by a citation network analysis to control those documents
that have no citations and are not citing documents in the
dataset. This practice is conducted because the topic search
in the WoS will search the article title, author-defined
keywords, abstract, and Keywords Plus. However, Key-
words Plus is a column with system-defined keywords,
and it may have less relationship with the interests of

123
12
research. Based on the logic of citation use suggested by
past reviews (Liu et al. 2017), those irrelevant documents
were excluded. As a result, a total of 715 articles remained
(step 8), representing a research pool of computational
thinking research published in high quality journals in the
recent decade. Meanwhile, all the bibliometric information
of the 715 articles was collected for the following keyword
and content analyses.
Keyword Analysis
The term ‘‘keyword’’ used in this paper is a general usage
to indicate the key information appearing in the titles,
keywords, and abstracts. Some researchers have suggested
that the wordings highlighted in the articles’ titles, key-
words, and abstracts was considered as the most essential
but critical element to represent an article (Cook et al.
2007). This idea is also identical with the topic search used
in the WoS. Recently, keyword analysis has been applied
to explore over four decades of articles published in a
renowned education journal (Zawacki-Richter and
Latchem 2018).
As such, a keyword analysis was performed by
decomposing the most valuable content of the corpus from
the 715 CT articles. First, all the wordings in the columns
of title, keyword, and abstract were gathered. Some stop
words which are insignificant were excluded, such as
‘‘and,’’ ‘‘at,’’ ‘‘in,’’ ‘‘on,’’ ‘‘above,’’ ‘‘under,’’ and so on.
Other common terms were also removed, for example
‘‘computer,’’ ‘‘education,’’ ‘‘learning,’’ and ‘‘computational
thinking.’’ The process of scraping common terms is to
reduce noise because those wordings are less discrimina-
tive for identifying the main characteristics of CT research.
Second, the terms with similar or identical meanings in the
context of CT research were regarded as being the same.
For instance, ‘‘algorithm’’ and ‘‘algorithmic,’’ ‘‘APOS’’
and ‘‘APOS theory,’’ ‘‘computer literacy’’ and ‘‘computer
literate’’ are all treated as the same wordings. The APOS is
a procedure of action, process, object, and schema used in
computer education (Dubinsky and McDonald 2001).
After the list of keywords was aligned, each keyword
was then matched and counted if it appeared in a certain
article. Note that each article may have multiple keywords;
however, those duplicated wordings used in a single article
were counted only once. Only the keywords appeared in
the different article were accumulated. The use of unique
counting is helpful to compare how many times a sub-
ject/tool/concept or noteworthy practice in CT research,
which is represented as the keyword, has been addressed
among the 715 articles. Accordingly, two sets of keywords
from the two periods of analysis were identified (3798
keywords from period I: 2006–2012, and 7175 keywords
from period II: 2013–2018). A further comparison of
123
K.-Y. Tang et al.
keywords between two periods was conducted for identi-
fying the research trend among CT publications.
Content Analysis
The second method we used was content analysis. The
main idea of a content analysis is to categorize the scien-
tific findings to uncover the underlying dynamics concep-
tually and categorically, which is similar to grounded
theory (Glaser and Strauss 1967). Past research has sug-
gested that the use of content analysis helps researchers
explore some prominent trends and pivotal insights that
have not yet been found (Lin et al. forthcoming). As such,
the content analysis adopted in this study could even pro-
vide scaffolds to build up a conceptual framework for the
research community (Dyrbye et al. 2007).
All the two-period keywords were then content-ana-
lyzed within the context of computer science education.
First, to consolidate the most distinctive keywords, the
process of induction was continued for several runs by two
experienced researchers. A consensus was reached if the
keyword was selected by the two researchers, while those
keywords that failed to gain consent were resolved upon
discussion. Overall, the intercoder reliability for each
analysis was at least 85%, indicating that the induction was
adequately reliable.
Furthermore, each keyword was assessed based on its
nature and was classified according to its various attributes.
For example, elementary, secondary, and teacher are the
types of participants; arts, biology, chemistry, computer
science, and robotics are the different subjects studied in
the research. The keywords with C/C??, R, Lego, Python
were categorized as programming languages. While some
keywords served as the background settings of a study (i.e.,
participants, subjects), others are domain-specific factors
(i.e., pedagogy, environment, programming lan-
guages/tools). In addition, following past research, key-
words related to learning outcomes were divided into three
domains of learning (Bloom et al. 1956), namely cognitive,
affective, and psychomotor outcomes. Finally, a typology
of 67 selected keywords in CT research was obtained as
shown in Table 2.
Results and Discussions
This study analyzed the CT studies from 2006 to 2018. The
results of the most productive countries, influential articles,
and active authors, as well as some distinctive keywords
that representing main attributes of CT research typology
are presented and discussed below.
A Content Analysis of Computational Thinking Research: An International Publication Trends and… 13

Table 2 A typology of selected keywords

Category Keywords Description

1. Participants Elementary, Higher education, Secondary, Teacher Background information that is often used in the CT
2. Subjects Arts, Biology, Chemistry, Computers science, Ecology, Engineering, research
Languages, Mathematics, Music, Physics, Robotics course
1. Pedagogy APOS, Constructivism, Game, Inquiry, Microworld, Pair Domain-specific factors—some unique features or
programming, Peer/Collaboration, Problem-based learning, Self- characteristics in the research context and setting
regulated learning, Simulation, Situated of CT
2. Environment Blended/Hybrid, Face-to-face, Web-based/Online
3. Programming Alice, Augmented reality, C/C??, Code.org, Java, JavaScript, Kodu,
languages/tools Lego, Python, R, Robots, Scratch
1. Cognitive Abstraction, Academic, Critical thinking, Decomposition, Discourse, Main learning outcomes that have been measured
outcomes Efficacy, Formative evaluation, Knowledge, Learning and discussed in the CT research
effectiveness, Mental, Metacognitive, Perception, Problem solving,
Reasoning, Reflection, Student feedback
2. Affective Attitude, Belief, Engagement, Motivation
outcomes
3. Psychomotor Acquisition, Communication, Efficiency, ICT skill, Social,
outcomes Programming skill

The Profile of International Publication Patterns
in CT Research
The Most Productive Countries
Table 3 lists the distribution of CT research with the
number and percentage of their publications, showing
publication trends of the most productive countries in CT.
To measure each country’s productivity, the first authors’
country affiliation was counted because the first author has
the major contribution to the research. If the first author has
multiple affiliations, the main affiliation was manually
checked and coded. As a result, a total of 52 countries were
found to be listed in CT publications. We also found that
the first 10 countries have contributed over 535 articles,
which is over 73% of the total publications from 2006 to
2018. Therefore, the ten countries were labeled as the top
10 productive countries in CT research, namely the USA,
Taiwan, Spain, Turkey, UK, China, Greece, Australia, the
Netherlands, and Canada.
The result indicates that over one third of the publica-
tions were from the USA (n = 255, 35.7%); this proportion
can also be applied as the most productive country to
period I (n = 87, 35.4%) and period II (n = 168, 35.8%).
We also found that the number of publications from the
USA almost doubled from period I to period II. Countries
such as the USA, Spain, the UK, China, and the Nether-
lands were found to have increasing publication trends in
both numbers and percentage of publications. The second
most publications were from Taiwan (n = 55, 7.7%).

Table 3 The most productive countries on CT publications (2006–2012 vs. 2013–2018)

# Country All years 2006–2018 (n = 715) Period I 2006–2012 (n = 246) Period II 2013–2018 (n = 469) Trend from the
period I to period II
Numbers of Percentage of all Numbers of Percentage of Numbers of Percentage of In In
publications years (%) publications period I (%) publications period II (%) numbers percentage

1 USA 255 35.7 87 35.4 168 35.8 : :

2 Taiwan 55 7.7 26 10.6 29 6.2 : ;
3 Spain 49 6.9 12 4.9 37 7.9 : :
4 Turkey 43 6.0 17 6.9 26 5.5 : ;
5 UK 33 4.6 11 4.5 22 4.7 : :
6 China 29 4.1 9 3.7 20 4.3 : :
7 Greece 23 3.2 10 4.1 13 2.8 : ;
8 Australia 19 2.7 8 3.3 11 2.3 : ;
9 Netherlands 15 2.1 5 2.0 10 2.1 : :
10 Canada 14 2.0 5 2.0 9 1.9 : ;

123
14 K.-Y. Tang et al.

Table 4 Top 20 active authors in CT publications (2006–2012 vs. 2013–2018)

# Authors All year Period I 2006–2012 Period II 2013–2018 First-authored articles Country Cites Active year

Number of articles (h-index)

1 Kafai, Yasmin B. 9 (6) 2 (2) 7 (5) 4 USA 91 2008–2017
2 Denning, Peter J. 8 (4) 5 (3) 3 (2) 8 USA 19 2007–2017
3 Bers, Marina U. 7 (4) 1 (1) 6 (3) 1 USA 20 2012–2018
4 Hwang, Gwo-Jen 7 (4) 2 (2) 5 (2) 1 Taiwan 9 2006–2017
5 Cooper, Stephen 5 (5) 2 (2) 3 (3) 2 USA 9 2009–2015
6 Chen, Gwo-Dong 5 (5) 5 (5) – 2 Taiwan 6 2008–2012
7 Hwang, Wu-Yuin 5 (4) 1 (1) 4 (3) 2 Taiwan 11 2012–2017
8 Magana, Alejandra J. 5 (4) – 5 (4) 3 USA 4 2014–2018
9 Dagiene, Valentina 5 (3) – 5 (3) 3 Lithuania 7 2016–2017
10 Tsai, Chia-Wen 5 (3) 3 (3) 2 (1) 3 Taiwan 3 2008–2017
11 Lau, Wilfred W. F. 4 (4) 3 (3) 1 (1) 4 China 9 2009–2014
12 Chang, Chih-Kai 4 (4) 3 (3) 1 (1) 2 Taiwan 7 2008–2014
13 Ellis, Nick C. 4 (4) 4 (4) – 3 USA 5 2009–2009
14 Burke, Quinn 4 (3) – 4 (3) 2 USA 10 2013–2016
15 Berland, Matthew 4 (3) – 4 (3) 3 USA 9 2013–2015
16 Sentance, Sue 4 (3) – 4 (3) 2 UK 0 2014–2018
17 Roman-Gonzalez, M. 4 (2) – 4 (2) 2 Spain 14 2015–2018
18 Hooshyar, Danial 4 (2) – 4 (2) 4 Malaysia 1 2015–2018
19 Weintrop, David 4 (1) – 4 (1) 2 USA 24 2016–2018
20 Wing, Jeannette M. 2 (2) 2 (2) – 2 USA 385 2006–2008

However, even though the number of publications
increased from period I (n = 26, 10.6%) to period II
(n = 29, 6.2%), the percentage in period II decreased.
Similar results were found for Turkey, Greece, Australia,
and Canada (downtrend signs in percentage). This reveals
the fact that the overall number of international publica-
tions on CT has increased, and the global competition in
CT research has become more intense, especially in recent
years.
The Most Active Authors
The analysis of the most active authors helps identify
active researchers with frequent publications that continue
contributing new findings for benefitting the CT research
community. Table 4 lists top 20 active CT researchers
worldwide. Among a total of 1652 researchers, Yasmin
Kafai was the most active author who had published nine
articles with h-index 6 and 91 citations from 2008 to 2017.
The results also show an increasing trend of her continued
publications from period I (n = 2) to period II (n = 5). The
second-most active author was Peter Denning with eight
first-authored articles, h-index 4 and 19 citations, followed
by Marina Bers, with seven articles with h-index 4 and 20
citations. Overall, a total of 10 active authors were from the
USA, and five were from Taiwan (led by Gwo-Jen Hwang).
The remainders were from Lithuania (Valentina Dagiene),
China (Wilfred Lau), the United Kingdom (Sue Sentance),
Spain (M. Roman-Gonzalez), and Malaysia (Danial
Hooshyar). In other words, over half of the active authors
were affiliated with the US and the western universities/
research institutions; but almost the other half of the
authors were from the Asia–Pacific region.
Taking results of CT publication patterns altogether, we
found that the dominant productivity and impact in CT
research were from the western countries, especially the
USA. However, some growth trends from non-western
countries were also noteworthy. For example, three of the
top ten countries are Taiwan, Turkey, and China (Table 3),
and seven out of top 20 active authors were also from non-
western countries (Taiwan, China, and Malaysia) (Table 4).
We also observed that eight researchers from various
regions who were not published in period I became active
authors in period II, showing that CT research has attracted
many international researchers.
The Most Influential Research Articles
To more specifically discover the publication trend, the
most influential research articles with highly cited times
were reviewed. Note that only the articles cited by other
journal articles in SSCI/SCI were counted as citations.

123
A Content Analysis of Computational Thinking Research: An International Publication Trends and… 15

Overall, the top three highly cited articles have received at
least 201 citations in total and 20.1 times on average (per
year). The most highly cited article is Wing’s (2006)
original article titled ‘‘computational thinking,’’ advocating
the importance and practices of CT for every learner (302
total citations, 23.2 cites per year). Followed by Papaster-
giou’s (2009) empirical study in examining students’
learning effectiveness and motivations in learning the
computer memory concept via computer games. He con-
cluded that that the gaming approach was effective in
promoting students’ knowledge of computer memory
concepts and more motivational than the non-gaming
approach (286 total citations, 28.6 cites per year). The third
one is Resnick et al’s (2009) article titled ‘‘Scratch: pro-
gramming for all,’’ introducing a game-based programming
tool that facilitating individuals to access coding activities
(201 total citations, 20.1 cites per year).
Recently, Denner et al. (2012) conducted experiments
among middle school female students by using a visual
tool, called Statecast Creator, to program a computer game.
They found that game construction involving both design
and programming activities can enhance students’ learning
of computer science concepts. Weintrop et al.’s (2016)
position paper articulated the benefits of including com-
putational thinking in mathematics and science classrooms.
They point out that embedding CT in a learning curriculum
can establish a reciprocal relationship between CT and
learning in mathematics and science, and bring math and
science education closer to current professional practice. In
addition, these articles have integrated various game-pro-
gramming tools into formal curricula. This indicates that
game design can facilitate CT cultivation, enable learners
to solve problems in real life, and empower them the ability
to apply educational knowledge into the practical world.
From these highly cited papers, we found that the idea of
computational thinking does help computer developers and
educators to encourage and inspire students to think like
computer scientists.
The Main Characteristics of CT Research
Based on the keyword-derived typology (Table 2), three
main characteristics of CT research were analyzed,
including background settings, domain-specific factors, and
learning outcomes. Each dimension was reported with the
sub-categorizations and counts of each occurring keyword.
Note that duplications of each keyword from individual
studies were carefully filtered; the number of counts rep-
resents the frequencies of such wording being used in the
number of articles.
Background Settings
Figure 2 shows the results of the background settings,
which were further divided into participants and subjects.
In the first category of participants, secondary students
(n = 118) were found as the major research target in CT
research, followed by those from higher education
(n = 109) and elementary schools (n = 72), whereas tea-
cher (n = 27) and kindergarten (n = 10) participants were
the two least frequent. Next, the results reveal that sec-
ondary students were also the most discussed participants
in period II (30 88), followed by those from higher edu-
cation (42 65), even though they were found the most in
period I. Teachers, however, were still found to be the least
(6 21). This result reveals that there is a lack of teacher
training for CT. This indicates that fostering CT is still a
challenge due to only a few teachers being trained with the
knowledge and skills to integrate CT into course curricula.
Similarly, Mouza et al. (2017) urged that CT skills must be
taught in teacher education curricula to foster deeper
knowledge and skills so that they can graduate well

Fig. 2 Counts and trends of Secondary (118) 30 88

1. Participants

background settings on CT Higher education (109) 42 65

research from 2006 to 2018.
Elementary (72) 16 56
(Number) denotes total counts
for each subcategory from 2006 Teacher* (27) 6 21
to 2018. *Including in-service Kindergarten (10) 3 7
teacher, school teacher, and Computer science (78) 32 46
teacher training. **Including Mathematics (65) 17 48
Linguistic and Morphology Engineering (59) 12 47
Arts (36) 12 24
2. Subjects

Robotics (25) 4 21
Languages**(19) 8 11
Biology (11) 2 9
Physics (11) 4 7
Chemistry (7) 16
Period I (2006-2012) Period II (2013-2018)
Music (7) 34
0 20 40 60 80 100 120 140

123
16 K.-Y. Tang et al.

Game (77) 17 60
Peer/ Collaboration (68) 23 45
Simulation (37) 9 28
Inquiry (13) 3 10

1. Pedagogy
Self-regulated learning (11) 83
Problem-based learning (10) 55
Microworld (7) 43
Pair programming (6) 33
APOS* (5) 23
Situated (4) 31
Constructivism (4) 04
Environm

Web-based/Online (160) 68 92
ent

Face-to-face** (94)
2.

34 60
Blended/ hybrid (13) 8 5
Scratch (44) 4 40
Java (14) 9 5
3.programming languages/ Tools

C/ C++ (11) 47
Robots (11) 29
Lego (6) 15
R language (5) 32
Alice (5) 14
Python (4) 04
JavaScript (2) 02
Kodu (2) 02
Code.org (1) 01 Period I (2006-2012) Period II (2013-2018)
Augmented Reality (1) 01
0 20 40 60 80 100 120 140 160 180

Fig. 3 Counts and trends of domain-specific factors on CT research from 2006–2018. *APOS = APOS (Action, Process, Object, Schema) theory
(Dubinsky and McDonald, 2001). **Including Face-to-face and laboratory. ;: downward trend sign

prepared. In addition, it is also noteworthy that the pedagogical methods dominate the CT research in both
youngest students from kindergarten were found to be period I and II with upward trends, especially games (17
selected as participants in CT research from period I 60). Other approaches, such as inquiry (n = 13), problem-
(n = 3) to period II (n = 7). All of the analyzed participants based learning (n = 10), constructivism (n = 4), APOS
showed uptrend signs for the two research periods. (n = 5), and pair programming (n = 6) were found to be
In the second category of subjects, as expected, Com- equal or even more in period II than period I. These used
puter Science (n = 78) was the main subject in CT pedagogies reflect the fact that the development of CT is
research. This was followed by Mathematics (n = 65), needed to create task-based learning environments to
Engineering (n = 59), and Robotics courses (n = 25), enhance motivation and cognitive outcomes (Belland et al.
indicating that CT was highly discussed in STEM educa- 2013). This also implies that instructors may need to
tion. Other subjects were also found across natural science, construct adaptive learning environments to scaffold stu-
such as Biology (n = 36), Physics (n = 36), and Chemistry dents to take more control in CT learning (Drexler 2010).
(n = 7); and the humanity and arts fields, including Arts Next, regarding learning environments, web-based/
(n = 36), Languages (n = 19), and Music (n = 7). All online (n = 160) was the most frequent setting adopted in
subjects also showed an uptrend from period I to period II. CT research, followed by face-to-face environments
(n = 94). Both showed an uptrend in the two periods.
Domain-Specific Factors However, the results of blended/ hybrid (8 5) which
specifies class settings with mixed environments was found
Figure 3 shows the results of the domain-specific factors, to be less or was even gradually being replaced by today’s
including pedagogy, environment, and programming lan- mobile learning era. It is also noticeable that the differ-
guage/ tools. First, game (n = 77) was the most frequently ences between online and face-to-face decreased in period
adopted pedagogy in CT research, followed by peer/ col- II. CT enrichment activities can be taught with or without
laboration (n = 68), and simulation (n = 37). These the help of technologies. For example, recently educators

123
A Content Analysis of Computational Thinking Research: An International Publication Trends and… 17

Knowledge (151) 53 98
Mental (133) 41 92
Academic (39) 19 20
Decomposition (48) 17 32
Abstraction (54) 13 41

1. cognitive domain
Student feedback (37) 12 25
Critical (44) 6 37
Perception (32) 13 19
Efficacy (36) 14 22
Reasoning (28) 5 23
Reflection (24) 5 19
Problem-solving (9) 4 5
Discourse (8) 4 4
Metacognitive (6) 24 Period I (2006-2012) Period II (2013-2018)
Learning effectiveness (3) 30
Formative evaluation (2) 20
Motivation (55) 18 37
3. psychomotor 2.affective
domain

Attitude (43) 13 30
Engagement (30) 3 27
Belief (18) 9 9
Social (87) 22 65
Communication (49) 23 26
domain

Programming skill (28) 8 20

Acquisition (20) 8 12
Efficiency (11) 8 3
ICT skill (3) 30
0 20 40 60 80 100 120 140 160

Fig. 4 Counts and trends of learning outcomes on CT research from 2006–2018

have adopted board games for younger learners or adult measured the learning outcomes as decomposition
beginners to develop CT skills, including basic coding for (n = 48), abstraction (n = 54), student feedback (n = 37),
8–9-year-old learners (Tsarava et al. 2018) and collabora- critical thinking (n = 44), efficacy (n = 36), reasoning
tion, communication, and problem solving for groups of (n = 28), reflection (n = 24), discourse (n = 8), and
college-age novice players (Berland and Lee 2011). These metacognition (n = 6). Nonetheless, academic (n = 39),
changes in the learning environment leave a clue for edu- formative evaluation (n = 3), and learning effectiveness
cators to be aware of the importance of learners’ CT (n = 2) were found but appeared less in period II. These
cultivation. cognitive findings reflect the importance of CT enhance-
Lastly, the use of programming languages/tools was also ment, as it is a heuristic higher-order thinking skill devel-
discussed. The results show that Scratch (n = 44) was opment for learners.
heavily used in CT research with a great jump from period Next, in the affective domain, participants’ motivation
I (n = 4) to period II (n = 40), and that Java (n = 14) (n = 55) was found to be measured most often, while
occurred most in the earlier period. Note that the use of attitude (n = 43), engagement (n = 30), and belief (n = 18)
Java and R language was also found to decrease; however, were considered as learning goals in the more recent CT
robots, Lego, and Alice were found to be increasingly research. Among them, engagement was found to be less
studied in recent years. Those that were found active in addressed in period I, but much more often in period II (3
recent years were tools more user-friendly, or game-based, 27). This result can be viewed as evidence linking with
including Python, JavaScript, Kodu, and Code.org. One Wing’s (2006) argument. ‘‘Computational thinking is
recent application of augmented reality in CT research was reformulating a seemingly difficult problem into one we
also found in this analysis (Teng et al. 2018), showing know how to solve, perhaps by reduction, embedding,
some needs for visual-based features for programming transformation, or simulation’’ (Wing 2006, p. 1). Thus, CT
learning for the younger generation. education helps develop individuals’ higher-order thinking
skills to cope with the fast-paced society and the mixed
Learning Outcomes reality environments. Facilitating adaptive and engaging
environments to nourish intrinsic motivation for CT
As shown in Fig. 4, the results of the learning outcomes developments for learners of all ages is the key.
were divided into the cognitive, affective, and psychomotor Lastly, in the psychomotor domain, social (n = 87) and
domains. In the cognitive domain, participants’ knowledge communication (n = 49) skills were found to be frequently
(n = 151) and mental understanding (n = 133) were most evaluated in CT research, where social skills were more
frequently assessed. While these two words show general often addressed in period II (22 65). In addition, skills such
meanings for cognitive assessments, some researchers as programming (n = 28), acquisition (n = 20), efficiency

123
18
(n = 11), and ICT (n = 3) were also found. However,
efficiency and ICT skills may already be a basic ability for
online-generation students after two decades of broadband
network development.
Conclusions and Limitations
This study conducted keyword and content analyses on CT
research in the recent 13 years. All in all, our result showed
that research publications across the two analyzed periods
have greatly increased and have spread around the world in
recent years. In terms of productivity, the USA dominated
the CT publications in both periods. However, the rising
impacts from Asia and Europe are noteworthy, with several
active researchers from these countries being identified.
We also have provided a typology of CT research that
was based on the results of the keyword analysis and
content analysis. Several implications for the future
research were concluded. First, the CT-infused courses
were most often conducted in STEM education, and some
humanity and arts domains. This finding highlights that the
philosophy of CT can be directly applied to natural and
social sciences. Second, pedagogy using games, peer/col-
laboration approaches, and task-based learning environ-
ments were found. This higher-order thinking cultivation
environment can foster students’ computational practices
with information processing, scaffolding, and reflection
activities. Moreover, Scratch, Lego, Alice, and Python
were found to be emerging programming language/ tools,
highlighting the use of game-programming tools as a
means of helping learners develop the skills to face real-life
problems.
This study has contributed a research typology from
keyword and content analyses to understanding the litera-
ture structure of computational thinking research, and has
revealed some changes in three major characteristics,
namely background settings, domain-specific factors, and
learning outcome domains. Nevertheless, all methodology
has their own research limitations (Nerur et al. 2008). In
this study, the first limitation is the completeness issue of
keyword-searching strategies. Even though we used 13
keywords derived from four influential papers, the cover-
age may not be complete. For example, the works related to
‘‘computational action’’ and ‘‘system thinking’’ used in the
educational simulation area might have been omitted in our
analysis. However, following the same criteria in this paper
by searching the category of education and educational
research from the WoS, less than 20 new articles were
obtained using the above two keywords. This indicates a
very limited impact on our results. For future analysis,
researchers may add more keywords and search a broader
category to enlarge the data. Second, the unit of analysis in
123
K.-Y. Tang et al.
this study is the keywords, especially from article’s
abstract. In general, the abstract provided at most 300
words. Moreover, writing style varied among researchers:
some researchers prefer concise wording; others may tend
to give details of the research backgrounds and results, and
even according to the rules of each journal. In this study,
about 56% of articles have reported their research results in
the abstract; it is difficult to comment which outcomes
were superficial and which were more critical by merely
using keyword analysis. It is the same for the relationships
between the CT use and curricular topics, as well as the
relationship between CT concepts/practices and higher-
order skills. As a complementary research method, meta-
analysis and extensive content analysis are suggested for
further examination of the above relationships. Last, to
present the latest research status of CT, the timespan set-
ting is not a complete year (until August, 2018). Future
studies can gather data from a complete research year to
continue the observations of CT development. Bearing the
limitations in mind, this keyword analysis of CT publica-
tions has provided a fresh attempt to demonstrate a con-
ceptual and typological overview with bibliographic
evidence to quickly follow the research status and key
elements of CT literature.
Acknowledgements The authors are grateful to the comments from
two anonymous reviewers. This work was financially supported by
the ‘‘Institute for Research Excellence in Learning Sciences’’ of
National Taiwan Normal University (NTNU) from The Featured
Areas Research Center Program within the framework of the Higher
Education Sprout Project by the Ministry of Education (MOE) in
Taiwan. Partly financial support from the Ministry of Science and
Technology, Taiwan, under grant number MOST 107-2511-H-130-
003 was also acknowledged.
References
Belland, B. R., Kim, C., & Hannafin, M. J. (2013). A framework for
designing scaffolds that improve motivation and cognition.
Educational Psychologist, 48(4), 243–270.
Berland, M., & Lee, V. R. (2011). Collaborative strategic board
games as a site for distributed computational thinking. Interna-
tional Journal of Game-Based Learning, 1(2), 65–81.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl,
D. R. (1956). Taxonomy of educational objectives: The classi-
fication of educational goals. Handbook I: Cognitive domain.
New York: David McKay Company.
Buitrago Flórez, F., Casallas, R., Hernández, M., Reyes, A., Restrepo,
S., & Danies, G. (2017). Changing a generation’s way of
thinking: Teaching computational thinking through program-
ming. Review of Educational Research, 87(4), 834–860.
Cook, D. A., Beckman, T. J., & Bordage, G. (2007). A systematic
review of titles and abstracts of experimental studies in medical
education: Many informative elements missing. Medical Edu-
cation, 41(11), 1074–1081.
Denner, J., Werner, L., & Ortiz, E. (2012). Computer games created
by middle school girls: Can they be used to measure
A Content Analysis of Computational Thinking Research: An International Publication Trends and…
understanding of computer science concepts. Computers &
Education, 58(1), 240–249.
Drexler, W. (2010). The networked student model for construction of
personal learning environments: Balancing teacher control and
student autonomy. Australasian Journal of Educational Tech-
nology. https://doi.org/10.14742/ajet.1081.
Dubinsky, E., & McDonald, M. A. (2001). APOS: A constructivist
theory of learning in undergraduate mathematics education
research. In The teaching and learning of mathematics at
university level (pp. 275–282). Dordrecht: Springer.
Dyrbye, L. N., Harris, I., & Rohren, C. H. (2007). Early clinical
experiences from students’ perspectives: A qualitative study of
narratives. Academic Medicine, 82(10), 979–988.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded
theory: Strategies for qualitative theory. New Brunswick: Aldine
Transaction.
Grover, S., & Pea, R. (2013). Computational thinking in K–12: A
review of the state of the field. Educational Researcher, 42(1),
38–43.
Hsiao, C. H., Tang, K. Y., & Liu, J. S. (2015). Citation-based analysis
of literature: A case study of technology acceptance research.
Scientometrics, 105(2), 1091–1110.
Kim, H., Choi, H., Han, J., & So, H. J. (2012). Enhancing teachers’
ICT capacity for the 21st century learning environment: Three
cases of teacher education in Korea. Australasian Journal of
Educational Technology, 28(6), 965–982.
Lin, T. J., Lin, T. C., Potvin, P., & Tsai, C. C. (forthcoming).
Research trends in science education from 2013 to 2017: A
systematic content analysis of publications in selected journals.
International Journal of Science Education.
https://doi.org/10.1080/09500693.2018.1550274.
Liu, J. S., Ho, M. H. C., & Lu, L. Y. (2017). Recent themes in social
networking service research. PLoS ONE, 12(1), e0170293.
Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning
of computational thinking through programming: What is next
for K-12? Computers in Human Behavior, 41, 51–61.
Maida, V., & Cheon, P. M. (2014). Prognosis: The ‘‘missing link’’
within the CanMEDS competency framework. BMC Medical
Education, 14, 93. https://doi.org/10.1186/1472-6920-14-93.
Mouza, C., Yang, H., Pan, Y. C., Ozden, S. Y., & Pollock, L. (2017).
Resetting educational technology coursework for pre-service
teachers: A computational thinking approach to the development
of technological pedagogical content knowledge (TPACK).
Australasian Journal of Educational Technology, 33(3), 61–76.
Nerur, S. P., Rasheed, A. A., & Natarajan, V. (2008). The intellectual
structure of the strategic management field: An author co-
19
citation analysis. Strategic Management Journal, 29(3),
319–336.
Papastergiou, M. (2009). Digital game-based learning in high school
computer science education: Impact on educational effectiveness
and student motivation. Computers & Education, 52(1), 1–12.
Park, J., Wu, B., Morrison, A. M., Shen, Y., Cong, L., & Li, M.
(2016). The tourism system research categorization framework.
Asia Pacific Journal of Tourism Research, 21(9), 968–1000.
Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N.,
Eastmond, E., Brennan, K., et al. (2009). Scratch: Programming
for all. Communications of the ACM, 52(11), 60–67.
Shailaja, J., & Sridaran, R. (2015). Computational thinking the
intellectual thinking for the 21st century. International Journal
of Advanced Networking & Applications, 7, 39–46.
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying
computational thinking. Educational Research Review, 22,
142–158.
Teng, C. H., Chen, J. Y., & Chen, Z. H. (2018). Impact of augmented
reality on programming language learning: Efficiency and
perception. Journal of Educational Computing Research,
56(2), 254–271.
Tsarava, K., Moeller, K., & Ninaus, M. (2018). Training Computa-
tional Thinking through board games: The case of Crabs &
Turtles. International Journal of Serious Games, 5(2), 25–44.
Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015).
Computational thinking in compulsory education: Towards an
agenda for research and practice. Education and Information
Technologies, 20(4), 715–728.
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L.,
et al. (2016). Defining computational thinking for mathematics
and science classrooms. Journal of Science Education and
Technology, 25(1), 127–147.
Wing, J. M. (2006). Computational thinking. Communications of the
ACM, 49(3), 33–35.
Wing, J. M. (2008). Computational thinking and thinking about
computing. Philosophical Transactions of the Royal Society of
London A: Mathematical, Physical and Engineering Sciences,
366(1881), 3717–3725.
Zawacki-Richter, O., & Latchem, C. (2018). Exploring four decades
of research in Computers & Education. Computers & Education,
122, 136–152.
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
123