Algorithmic thinking in early childhood

Investigating predictive factors amid environmental education

Kalliopi Kanaki
Postdoctoral Researcher, Department of Preschool Education, University of Crete, kalkanaki@uoc.gr

Michail Kalogiannakis
Associate Professor, Department of Preschool Education, University of Crete, mkalogian@uoc.gr

The rising interest of educators, researchers and policymakers around the world as far as the development of computational
thinking skills in compulsory education is concerned is echoed in the plethora of research studies discussed in the pertinent
literature. However, the successful injection of computation thinking in formal educational settings demands the construction of
developmentally appropriate assessment tools. In this paper, we discuss a novel framework for assessing computational thinking
skills in early childhood settings. The proposed framework was employed in a relevant quantitative research study conducted in
the city of Heraklion, Crete, from February to June 2019, with a sample of 435 first and second graders and within the context of
the Environmental Study course. This paper also provides evidence regarding the examination of age, gender and learning
achievements in the Environmental Study course as predictive factors of one of the core computational thinking competencies,
namely algorithmic thinking. The research findings revealed that age and learning achievements in the Environmental Study
course constitute predictive factors for algorithmic thinking skills in the first and second grade level of primary school. On the
contrary, algorithmic thinking skills are not related to first and second graders’ gender. The results of this study provide a solid
background for designing and implementing developmentally appropriate tools for cultivating and assessing computational
thinking skills in the early years of schooling.

CCS CONCEPTS • Computational thinking • Environmental science • Student assessment

Additional Keywords and Phrases: Algorithmic thinking, Early childhood education, Gender, Age

1 INTRODUCTION
Nowadays, computational thinking (CT) is being widely injected in K-12 and succeeding curricula all around the
globe [1, 2], provided that it is a set of fundamental competencies for every productive member of the modern world
[3]. Since Wing coined the term [3], a notable amount of research has been dedicated to the study of introducing CT
in compulsory education [1, 4]. Despite growing uptake, various issues and challenges are coming out regarding the
effective cultivation of CT competencies in K-12 [2]. An important matter that demands deeper consideration is CT
assessment, especially in early childhood education [5, 6].
Having focused our research interest on cultivating and assessing CT in the first two grade levels of primary
school, we propose a novel assessment framework developmentally appropriate for young students. In this article,
we discuss part of a research study we conducted exploiting the proposed assessment framework. The study was
implemented within a robust ethical framework [7, 8] in the city of Heraklion, in Crete, and lasted from January to
June 2019. Its objective was the investigation of the association of age [9], gender [10], and learning achievements
in the Environmental Study course with one of the most important CT skills, that is to say algorithmic thinking (AT).
Hence, the research questions that drive this article are Q1. “Are AT skills related to students’ age in the early
primary school years? Q2. “Are AT skills related to students’ gender in the early primary school years?”Q3. “Are AT
skills related to students’ educational achievements in the Environmental Study course in the early primary school
years?”
In order to answer the research questions, we set the hypotheses that: H1.“AT skills are not related to students’
age in the early primary school years”H2. “AT skills are not related to students’ gender in the early primary school
years”H3. “AT skills are not related to students’ educational achievements in the Environmental Study course in the
early primary school years”.
The pillar of the proposed assessment framework employed to test the hypotheses set is an innovative digital
platform PhysGramming (an acronym derived from Physical Science Programming) designed and constructed
under the umbrella of constructivism, play-based learning, and multidisciplinarity. PhysGramming is
developmentally appropriate for first and second graders and introduces a hybrid schema of text-based and visual
programming techniques that exposes young students to object-oriented programming syntax and principles [11].
It opens up the chance for constructing digital games, keeping first and second graders’ interest and participation
alive in the learning process [12, 13, 14].

2 THEORETICAL BACKGROUND

2.1 Correlation between Computational thinking skills and educational grade levels
Over the last two decades, several research studies examined the correlation between CT skills and educational
grade levels. For example, a study implemented in Spain with a sample of 1251 fifth and sixth graders confirmed
this correlation [15]. Another study conducted in Ankara in 2015-2016 on a sample of 156 students ranging from
fifth to twelfth-grade level resulted, among others, that students’ CT skills are analogous to their grade level [16]. A
relevant study that took place in the Netherlands on a sample of 200 students six to twelve years old concluded that
age influences the growth of CT skills [17]. Finally, a study conducted in Switzerland on a sample of 109 K-12
students aged three to sixteen inferred that AT competencies increase while coming up to higher grade levels [1].

2.2 Correlation between Computational thinking skills and gender

In the literature, several studies confirm the absence of gender effect on cultivating CT skills. For example, a study
conducted in Virginia on a sample of 758 five to nine-year-old students revealed that gender does not influence the
development of CT competencies [6]. Another study that examined the cultivation of coding and CT skills for
children ages eight to seventeen concluded the lack of performance differentiation between boys and girls, although
they implement coding activities in different ways [18].
On the other hand, there are studies that confirm the gender effect on developing CT skills. For example, a study
that took place in Spain on a sample of 1251 students, from fifth to tenth grade, revealed gender differentiation in
seventh and eighth-grade levels. This differentiation escalates in ninth and tenth-grade levels, although it is not
observable in fifth and sixth [15]. Another study conducted on a sample of 60 sixth graders, examined CT skills in
terms of their cultivation and assessment through coding activities and concluded that there are gender differences

2
in the way boys and girls are involved in coding, with boys being more confident and achieving higher levels of
performance [19]. There are also studies that concluded that gender is a predictive factor of CT skills when
examined amid Computer Science activities [16, 20, 21].

2.3 Correlation between Computational Thinking and Environmental Science

Appropriately organized environmental education could bring fresh air to the stance of people that distinguish
themselves from nature [22]. Enhancing environmental awareness and sustainability in compulsory education is
essential for cultivating individual and collective sensibility and responsiveness to address present-day health and
environmental stimuli [23].
Although CT competencies, such as AT, are strongly related to environmental issues [24, 25], there is a research
gap regarding the synergistic cultivation of CT and environmental awareness.

2.4 Algorithmic thinking

AT is one of the fundamental CT competencies, which pertains to the ability of developing skills or rules that foster
the formulation of solutions to problems by specifying the exact steps needed [26]. Scholars highlight the
significance of exercising AT even from early childhood settings, targeting to effectively cultivate CT skills and
establish the basis for positive viewpoints towards technical professions [27].

3 MATERIALS AND METHODS

Despite the extended research efforts in the field of cultivating and assessing CT, the construction of relevant
assessment tools that are developmentally appropriate for primary school students remains a challenge that
demands consideration [28]. Supporting the viewpoint that the best way to experience programming for the first
time is via the object-oriented paradigm [11, 29, 30], and attempting to serve the novel idea of introducing object-
oriented programming concepts in early childhood education, we propose PhysGramming [11], which is also
employed in order to assess CT skills, such as AT.
Regarding the assessment of the content understanding, we employed worksheets that examined students’
perceptions about animals’ eating habits. The level of students’ content understanding, which was determined by
their performance in filling in the worksheets, took one of the following values: Excellent, Very Good, Good and
Almost Good. These values came up from the statistical analysis of the data of a relevant pilot study we conducted.

3.1 PhysGramming
In order to unfold the functionality of PhysGramming, we employ the thematic unit of animals’ eating habits, amid
which we implemented our research study. Firstly, students are prompted to select animals that share a common
characteristic. In our case, we asked students to paint, shoot or select animals from PhysGramming’s image pool
that belong to the fauna of Crete. The selected entities automatically appear in command lines and students have to
determine their names (Figure 1).

3
 Figure 1: Assigning values to the animals’ attribute “NAME”

Then, students have to determine each animal’s eating habits (Figure 2). After that, PhysGramming automatically
constructs digital games, which are unique just like students’ paintings and photos. In this article, we present the
puzzle games, since they are employed for assessing AT skills.

Figure 2: Assigning values to the animals’ attribute “NUTRITION HABITS”

PhysGramming constructs puzzles of four, six, nine, or twelve pieces (Figure 3). Each puzzle is a grid of randomly
positioned pieces of an image, with the lower right cell of the grid to be temporarily empty. Students have to
reassemble the image by rearranging its pieces. Nonetheless, only the pieces adjacent to the empty cell can be placed
in it by moving them horizontally, vertically, or diagonally [11]. The rest of the pieces will become moveable only
when one of their adjacent cells becomes empty. Therefore, solving a four-piece puzzle is quite easy, in contrast
with the rest of the puzzles.

4
 Figure 3: Four, six, nine, and twelve-piece puzzles

Considering that increasing the puzzles’ pieces complicates the solution process and demands higher AT skills,
we determine the levels of AT skills based on the most difficult puzzle students manage to solve (Table 1). The
proposed scale embraces the viewpoint of relevant research studies, such as, the one of Chongo et al. [31] that
suggests four CT skills levels i.e., Fail, Pass, Credit, Excellence.

Table 1: AT levels

AT levels The most difficult puzzle solved

Basic four-piece puzzle
Medium six-piece puzzle
Satisfactory nine-piece puzzle
Excellent twelve-piece puzzle

Research sample
The sample constituted of 435 first and second graders participated in the research study and it was gender-
balanced - 210 girls (48.28%) and 225 boys (51.72%), and grade-balanced – 218 first graders (50.11%) and 217
second graders (49.89%).
With the aim of formulating a representative sample, we applied the cluster sampling method [8]. Hence, we
involved schools from different areas of Heraklion city, in order to be sure that the participants of the research
study would appertain to various socio-economic strata, entailing a sample of diverse learning achievements and
stances towards the educational process [32, 33].

3.2 Validation
Pursuing the validity and the reliability of the results obtained by the proposed assessment tool, we employed
adequate methods documented in the international literature [8]. Although the subject of this article is not to
discuss the methods used, we indicatively notice that the Pearson correlation coefficient was calculated to be 0.81.
This value indicates very good test/retest reliability i.e., very good stability of measurements provided by recurrent
studies that involve the same individuals at different times [34, 35].

4 RESULTS
In order to examine the hypotheses set, we employed the chi-square test, and calculated p-value and odds ratio.
Seeking to model the relationship between the variables under study, we used the ordinal logistic regression
analysis method. We also applied the machine learning method [36] to predict the AT levels of populations similar
to our study, using 80% of the survey data to create the prediction equations and the remaining 20% to test them.

4.1 Examining age

Based on the contingency tables of observed and expected frequencies, chi-square is 8.7543, degrees of freedom
(df) is 3, and p-value is 0.03274. Since p-value is less than 0.05, we reject the H1 and we accept that, in the case of
first and second-graders, AT skills are related to age.
Based on the calculated odds ratio, the probability of detecting excellent and satisfactory AT levels is higher for
second graders (odds ratio has been calculated to be 0.883 and 0.512, respectively). On the other hand, medium

5
and basic AT levels are more likely to be detected in first-grade level (odds ratio has been calculated to be 1.689
and 1.097, respectively).
The graph provided when applying the machine learning method (Figure 4) indicates that being a second grader
increases the probability of being classified at the higher AT levels (satisfactory and excellent), and decreases the
probability of being classified at the lower levels of AT (basic and medium).

Figure 4: AT levels in relation to grade level

4.2 Examining gender

Based on the contingency tables of observed and expected frequencies, chi-square is 4.9912, df is 3, and the p-value
0.1724. Since p-value is greater than 0.05, we accept H2 i.e., the absence of correlation between students’ gender
and AT skills in early childhood.

4.3 Examining educational achievements in the Environmental Study

Based on the contingency tables of observed and expected frequencies, chi-square is 36.434, df is 9 and p-value is
3.32e-05. Since p is less than 0.05, we reject H3 and accept that there is association between AT and skills content
understanding of the environmental Study course.
Next, we examine the association of each one of the AT levels with each one of the content understanding levels and
we calculate odds ratio. Results indicate that students grouped into one of the two higher AT levels are more likely
to be grouped into one of the two higher levels of content understanding as well. On the other hand, students placed
into one of the lower AT levels are more likely to be placed into one of the lower content understanding levels.
These results are confirmed by the graph provided by applying the machine learning method (Figure 5).

6
 Figure 5: AT levels in relation to content understanding of the Environmental Study course

5 DISCUSSION
Our research interest was oriented towards the first two grade levels of primary school since they constitute the
basis of the educational structure. Indeed, during the first years of schooling, attitudes and dispositions towards the
learning process are formed, drawing children’s academic trajectory [37, 38]. We focused on investigating
assessment tools since they are beneficial for teachers who need to monitor students' progress, promote learning
and maintain cognitive achievement. Furthermore, without valid and reliable assessment tools, the introduction of
any educational element, not only CT, could not be successful.
The results of our research study are in line with the studies already documented in international literature.
Indeed, our findings regarding the absence of correlation between students’ gender and CT skills have already been
ascertained in previous studies [6, 18]. Works that confirm the association between students’ gender and CT skills
concern older students rather than first and second graders [15, 19], with the apparent discrepancy to be due to
the effect of gender roles on dispositions toward technology [39].
Our findings regarding the correlation between CT skills and educational grade levels and, thus, students’ age
have been confirmed in several studies [1, 15, 16, 17]. These findings are anticipated if we face CT skills as the
reflection of competencies relevant to cognitive development, which is confirmed to be correlated with educational
grade levels [16, 40].
Apropos the investigation of the synergistic learning of CT and Environmental Study in the first two grade levels,
the originality of our work concerns the absence of relevant recorded studies. Furthermore, most of the pertinent
surveys target older children [41, 42], leaving the area of preschool and early school education under-explored.
Our future research plans include the implementation of a study across the country, addressing the sample
limitation of the present study conducted in Crete. We also intend to employ several other thematic units of the
Environmental Study course.
In attempting to discuss the aftertaste left to the participants of our study, we should underline the festive
atmosphere established in the classrooms, especially when solving the puzzles, which students recognized
immediately as the most enjoyable part of the assessment process. The animated fireworks and the rewarding
sound that accompanied each successful attempt were proved very joyful, established a creative atmosphere,
engaged students, and prompted them to try harder in order to solve more difficult puzzles.

7
6 CONCLUSIONS
This article documents a multidisciplinary assessment framework, of game-based and constructivist texture,
designed to examine the potential association between AT skills of first and second graders with age [9], gender
[10], and educational achievements in the Environmental Study course. The research study presented in this article
promotes the novel idea of the synergistic cultivation of fundamental CT skills and environmental awareness,
providing evidence of the joint assessment of the two fields. It also contributes to the efficient cultivation of CT skills
in early education, elaborating on gender equity. Finally, yet importantly, it affirms the correlation of CT skills in
the narrow age range of first and second graders. We steered towards the first school years since the subject of CT
evaluation at this sensitive period of students’ development remains insufficiently investigated [5].
Emphasizing the need for bringing developmentally appropriate CT practices into formal learning settings, we
introduced the digital platform PhysGramming, aiming to support the evaluation of CT competencies and, thus, add
to the field of CT development. The proposed tool not only detects the levels of young students’ CT skills but also
facilitates the assessment of the effectiveness of relevant teaching interventions. It also contributes to the design of
targeted teaching interventions for the cultivation of CT.
In conclusion, the presented work is of interest to scholars that support the diffusion of CT competencies in K-
12, employing multidisciplinary learning scenarios and developmentally appropriate instructional practices, with
the ultimate goal of equipping posterity with 21st-century skills and habilitating it for the modern job market
demands [43]. The results of the study contribute to building a solid base for future research studies regarding the
construction of age-appropriate CT assessment tools that will provide an ecological reflection on CT activities, and
support the end of perpetuating gender stereotypes about women's enrollment in science fields.

REFERENCES
[1] Alberto Piatti, Giorgia Adorni, G, Laila El-Hamamsy, Lucio Negrini, Dorit Assaf, Luca Gambardella, and Francesco Mondada. 2022. The CT-cube:
A framework for the design and the assessment of computational thinking activities. Comput. Hum. Behav. Rep. 5, 100166.
https://doi.org/10.1016/j.chbr.2021.100166.
[2] Stefania Bocconi, Augusto Chioccariello,, Panagiotis Kampylis, Valentina Dagienė, Patricia Wastiau, Katja Engelhardt, Jeffrey Earp, Milena Anna
Horvath, Eglė Jasutė, Chiara Malagoli, Vaida Masiulionytė-Dagienė, and Gabrielė Stupurienė. 2022. Reviewing Computational Thinking in
Compulsory Education. In Inamorato Dos Santos, A., Cachia, R., Giannoutsou, N. and Punie, Y. (Eds.), Publications Office of the European Union,
Luxembourg. ISBN 978-92-76-47208-7, https://doi.org/10.2760/126955, JRC128347.
[3] Jeannette M. Wing. 2006. Computational thinking. Commun. ACM, 49(3), 33-35. https://doi.org/10.1145/1118178.1118215.
[4] Yasmin B. Kafai, and Chris Proctor. 2022. A Revaluation of Computational Thinking in K–12 Education: Moving Toward Computational
Literacies. Educ. Res., 51(2), 146-151.
[5] Emmanouil Poulakis, and Panagiotis Politis. 2021. Computational Thinking Assessment: Literature Review. In Thrasyvoulos Tsiatsos, Stavros
Demetriadis, Anastasios Mikropoulos, and Vasileios Dagdilelis (Eds.), Research on E-Learning and ICT in Education, 111-128. Cham: Springer.
https://doi.org/10.1007/978-3-030-64363-8_7.
[6] Emily Relkin, Laura de Ruiter, and Marina Umaschi Bers. 2020. TechCheck: Development and validation of an unplugged assessment of
computational thinking in early childhood education. J. Sci. Educ. Technol., 29(4), 482-498. https://doi.org/10.1007/s10956-020-09831-x.
[7] Vasiliki Petousi, and Eirini Sifaki. 2020. Contextualizing harm in the framework of research misconduct. Findings from discourse analysis of
scientific publications, Int. J. Sustain. Dev., 23, 149–174. https://doi.org/10.1504/IJSD.2020.10037655.
[8] Louis Cohen, Lawrence Manion, and Keith Morrison. 2007. Research Methods in Education, 3th ed. Routledge: London, UK.
[9] Kalliopi Kanaki, and Michail Kalogiannakis. 2022. Assessing Algorithmic Thinking Skills in Relation to Age in Early Childhood STEM
Education. Education Sciences, 12(6), 380. https://doi.org/10.3390/educsci12060380
[10] Kalliopi Kanaki, and Michail Kalogiannakis. 2022. Assessing algorithmic thinking skills in relation to gender in early childhood, Educational
Process, 11(2), 44-59. https://doi.org/10.22521/edupij.2022.112.3
[11] Kalliopi Kanaki, and Michail Kalogiannakis. 2018. Introducing fundamental object-oriented programming concepts in preschool education
within the context of physical science courses. Education and Information Technologies, 23(6), 2673-2698. https://doi.org/10.1007/s10639-
018-9736-0.
[12] Fredrik S. Breien, and Barbara Wasson. 2021. Narrative categorization in digital game-based learning: Engagement, motivation & learning. Br.
J. Educ. Technol. 2021, 52, 91–111. https://doi.org//10.1111/bjet.13004.

8
[13] Stephen Rushton, Anne Juola-Rushton, and Elizabeth Larkin. 2010. Neuroscience, play and early childhood education: Connections,
implications and assessment. Early Child. Educ. J., 37, 351–361. https://doi.org/10.1007/s10643-009-0359-3.
[14] Mariano Sigman, Marcela Peña, Andrea P. Goldin, and Sidarta Ribeiro. 2014. Neuroscience and education: Prime time to build the bridge. Nat.
Neurosci., 17, 497–502. https://doi.org/10.1038/nn.3672.
[15] Marcos Román-González, Juan-Carlos Pérez-González, and Carmen Jiménez-Fernández. 2017. Which cognitive abilities underlie computational
thinking? Criterion validity of the Computational Thinking Test. Comput. Hum. Behav., 72, 678–691.
https://doi.org/10.1016/j.chb.2016.08.047.
[16] Hatice Yildiz Durak, Mustafa Saritepeci. 2018. Analysis of the relation between computational thinking skills and various variables with the
structural equation model. Comput. Educ., 116, 191–202. https://doi.org/10.1016/j.compedu.2017.09.004.
[17] Wouter J. Rijke, Lars Bollen, Tessa H.S. Eysink, and Jos L. J. Tolboom. 2018. Computational Thinking in Primary School: An Examination of
Abstraction and Decomposition in Different Age Groups. Inform. Educ., 17, 77–92. https://doi.org/10.15388/infedu.2018.05
[18] Sofia Papavlasopoulou, Kshitij Sharma, and Michail N. Giannakos. 2020. Coding activities for children: Coupling eye-tracking with qualitative
data to investigate gender differences. Comput. Hum. Behav., 105, 105939. https://doi.org/10.1016/j.chb.2019.03.003.
[19] Jennifer Jenson, and Milena Droumeva. 2016. Exploring media literacy and computational thinking: A game maker curriculum study. Electronic
Journal of e-Learning, 14(2), 111-121. Retrieved from www.ejel.org.
[20] Charoula Angeli, and Michail Giannakos. 2020. Computational thinking education: Issues and challenges. Comput. Hum. Behav., 105, 106185.
https://doi.org/10.1016/j.chb.2019.106185.
[21] Charoula Angeli, and Nicos Valanides. 2020. Developing young children's computational thinking with educational robotics: An interaction
effect between gender and scaffolding strategy. Comput. Hum. Behav., 105, 105954. https://doi.org/10.1016/j.chb.2019.03.018.
[22] Courtney E. Quinn, and Matthew Cohen. 2021. Using COVID-19 to Teach Sustainability Futures Thinking. In COVID-19: Paving the Way for a
More Sustainable World, 411-426. Springer, Cham. https://doi.org/10.1007/978-3-030-69284-1_21.
[23] Thiago Leite Cruz, Pedro Israel Mota Pinto, and Anderson Hélio Juwer Ferreira. 2021. Environmental Education as a Tool to Improve
Sustainability and Promote Global Health: Lessons from the COVID-19 to Avoid Other Pandemics. In COVID-19: Paving the Way for a More
Sustainable World, 331-347. Springer, Cham. https://doi.org/10.1007/978-3-030-69284-1_17.
[24] Arjan S. Gosal, Janine A. McMahon, Katharine M. Bowgen, Catherine H. Hoppe, and Guy Ziv. 2021. Identifying and Mapping Groups of Protected
Area Visitors by Environmental Awareness. Land, 10(6), 560. https://doi.org/10.3390/land10060560.
[25] Hamed Rezapouraghdam, Arash Akhshik, and Haywantee Ramkissoon. 2021. Application of machine learning to predict visitors’ green
behavior in marine protected areas: Evidence from Cyprus. Journal of Sustainable Tourism, 1-25.
https://doi.org/10.1080/09669582.2021.1887878.
[26] Li Zhao, Xiaohong Liu, Chenhui Wang, and Yu-Sheng Su. 2022. Effect of different mind mapping approaches on primary school students’
computational thinking skills during visual programming learning. Comput. Educ., 104445. https://doi.org/10.1016/j.compedu.2022.104445.
[27] Karin Tengler, Oliver Kastner-Hauler, and Barbara Sabitzer. 2021. Enhancing Computational Thinking Skills using Robots and Digital
Storytelling. In Proceedings of the CSEDU, Online Conference, 157–164. https://doi.org/10.5220/0010477001570164.
[28] Javier del Olmo-Muñoz, Ramón Cózar-Gutiérrez, José Antonio González-Calero. 2020. Computational thinking through unplugged activities in
early years of Primary Education. Comput. Educ., 150, 103832. https://doi.org/10.1016/j.compedu.2020.103832.
[29] A. Ferrari, A. Poggi, M. Tomaiuolo. 2016. Object oriented puzzle programming. Mondo. Digit., 15, 64.
[30] Erica Janke, Philipp Brune, and Stefan Wagner. 2015. Does outside-in teaching improve the learning of object-oriented programming? In
Proceedings of the 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Florence, Italy, Volume 2, 408–417.
https://doi.org/10.1109/icse.2015.173.
[31] Samri Chongo, Kamisah Osman, and Nazrul Anuar Nayan. 2020. Level of Computational Thinking Skills among Secondary Science Student:
Variation across Gender and Mathematics Achievement. Sci. Educ. Int., 31, 159–163. https://doi.org/10.33828/sei.v31.i2.4
[32] Janine Bempechat and David J. Shernoff. 2012. Parental influences on achievement motivation and student engagement. In Handbook of
research on student engagement. Sandra L. Christenson, Amy L. Reschly, and Cathy Wylie (Eds). Springer, Boston, MA, USA, 315–342.
https://doi.org/10.1007/978-1-4614-2018-7_15.
[33] Cheng Yong Tan, Meiyan Lyu, and Baiwen Peng. 2020. Academic benefits from parental involvement are stratified by parental socioeconomic
status: A meta-analysis. Parent. Sci. Pract., 20, 241–287. https://doi.org/10.1080/15295192.2019.1694836.
[34] Sharmila Vaz, Torbjörn Falkmer, Anne Elizabeth Passmore, Richard Parsons, and Pantelis Andreou. 2013. The case for using the repeatability
coefficient when calculating test–retest reliability. PLoS ONE, 8, e73990. https://doi.org/10.1371/journal.pone.0073990.
[35] Gail M. Sullivan. 2011. A primer on the validity of assessment instruments. J. Grad. Med. Educ., 3, 119–120. https://doi.org/10.4300/jgme-d-
11-00075.1.
[36] Ibtehal Talal Nafea. 2008. Machine Learning in Educational Technology. In Machine Learning-Advanced Techniques and Emerging
Applications. Hamed Farhadi (Ed.), IntechOpen: London, UK, 175–183. https://doi.org/10.5772/intechopen.72906.
[37] Linda Bakken, Nola Brown, and Barry Downing. 2017. Early childhood education: The long-term benefits. Journal of research in Childhood
Education, 31(2), 255-269. https://doi.org/10.1080/02568543.2016.1273285.
[38] Dana Charles McCoy, Hirokazu Yoshikawa, Kathleen M. Ziol-Guest, Greg J. Duncan, Holly S. Schindler, Katherine Magnuson, Rui Yang, Andrew
Koepp, and Jack P. Shonkoff1. 2017. Impacts of early childhood education on medium-and long-term educational outcomes. Educational
Researcher, 46(8), 474-487. https://doi.org/10.3102/0013189X17737739.

9
[39] Cathryne Stein, and Kim Nickerson (2004). Botball robotics and gender differences in middle school teams. In 2004 Annual Conference, 9.262.1
- 9.262.10. https://doi.org/10.18260/1-2--13534.
[40] Shuchi Grover, and Roy Pea. 2013. Computational thinking in K–12: A review of the state of the field. Educ. Res., 42, 38–43.
https://doi.org/10.3102/0013189x12463051.
[41] Didem Alsancak Sırakaya. 2020. Investigating computational thinking skills based on different variables and determining the predictor
variables. Particip. Educ. Res, 7(2), 102-114. https://doi.org/10.17275/per.20.22.7.2.
[42] Dalia Alyahya, and Amal Alotaibi. 2019. Computational thinking skills and its impact on TIMSS achievement: An Instructional Design
Approach. Issues and Trends in Learning Technologies, 7(1), 3-19. http://dx.doi.org/10.2458/azu_itet_v7i1_alyahya.
[43] Edelberto Franco Silvaa, Bruno Jose Dembogurskib, and Gustavo Silva Semaan. 2021. A Systematic Review of Computational Thinking in Early
Ages. arXiv preprint arXiv:2106.10275.

10