Computers and Education: Artificial Intelligence 5 (2023) 100155

Contents lists available at ScienceDirect

Computers and Education: Artificial Intelligence

journal homepage: www.sciencedirect.com/journal/computers-and-education-artificial-intelligence

Developing middle school students’ understanding of machine learning in

an African school
Ismaila Temitayo Sanusi a, *, Solomon Sunday Oyelere b, Henriikka Vartiainen c,
Jarkko Suhonen a, Markku Tukiainen a
a
School of Computing, University of Eastern Finland, P.O.Box 111, 80101, Joensuu, Finland
b
Department of Computer Science, Electrical and Space Engineering, Luleå University of Technology, SE-931 87, Skellefteå, Sweden
c
School of Applied Educational Science and Teacher Education, PO Box 111, 80101, Joensuu, Finland

A R T I C L E I N F O A B S T R A C T

Keywords: Researchers’ efforts to build a knowledge base of how middle school students learn about machine learning (ML)
Machine learning education is limited, particularly, considering the African context. Hence, we conducted an experimental classroom study
Data (N = 32) within the context of extracurricular activities in a Nigerian middle school to discern how students
Ethics
engaged with ML activities. Furthermore, we explored whether participation in our intervention program elicit
Middle school
changes in students’ ML comprehension, and perceptions. Using multiple qualitative data collection techniques
Nigeria
including interviews, pre-post open-ended surveys and written assessments, we uncover evidence that indicated
evolution of students’ ML understanding, ethical awareness, and societal implication of ML. In addition, our
findings showed that a middle school student can learn and understand ML, even when one had no prior
knowledge or interest in science related careers. The findings have implication for pedagogical design of AI
instruction in middle school context. We discuss the implication of our results for researchers and relevant
stakeholders, highlight the limitations and chart future work paths.

1. Introduction consensus among relevant stakeholders about the significance of ML

The increasing pervasiveness and permeation of artificial intelli­
gence (AI) & machine learning (ML) in every sphere of our lives has
necessitated the educational ecosystem to take cognizance of the
compelling need to design a curriculum including tools and approaches
that makes children use ML efficiently, become more future-ready,
improve their critical thinking skills (Xia et al., 2022) and raise their
desire to learn ML. While AI encompasses different subsets which in­
cludes ML, the vast majority of advancements in AI today are due to
machine learning models (Rodríguez-García et al., 2021). Owing to the
claim that the current development in AI is mainly carried out by ML
algorithms, it is rational to assume that understanding how ML works is
key to AI literacy. Preparing students for ML will help create a genera­
tion of active creators and designers of future ML-powered technologies.
According to Touretzky and Gardner-McCune (2022), ML is more than
technology; it can help students better appreciate the complexity of
humanity. With the benefits of teaching ML highlighted in literature and
lessons for young learners (Long & Magerko, 2020; Touretzky et al.,
2019; Vartiainen et al., 2021), integrating ML into middle school edu­
cation is crucial. The focus on middle schoolers is germane as re­
searchers’ efforts to build a knowledge base of how middle school
students learn about ML is limited (Sanusi et al., 2022). However,
integrating ML concepts in the formal school system requires weaving of
relevant content into a curriculum, hence the need to create learning
activities appropriate for the target population.
Indeed, a few curricula activities have been developed to support
learners to engage with ML in the middle school context. For instance,
Lee et al. (2021) created AI curriculum activities grounded in child
development literature, ethics, and career development. Lee and col­
leagues implemented the curriculum online with middle students and
reported a significant increase in students’ conceptual understanding of
AI. Sabuncuoglu (2020) developed an AI curriculum consisting of three
main modules which include how computers interact, how computers
see and how computers hear. Sabuncuoglu’s workshops with students

* Corresponding author.
E-mail addresses: ismaila.sanusi@uef.fi (I.T. Sanusi), solomon.oyelere@ltu.se (S.S. Oyelere), henriikka.vartiainen@uef.fi (H. Vartiainen), jarkko.suhonen@uef.fi
(J. Suhonen), markku.tukiainen@uef.fi (M. Tukiainen).

https://doi.org/10.1016/j.caeai.2023.100155
Received 24 April 2023; Received in revised form 6 July 2023; Accepted 8 July 2023
Available online 17 July 2023
2666-920X/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
I.T. Sanusi et al.
showed how the curriculum could be applied in an engaging and fruitful
way. Williams et al. (2022) developed three middle school AI literacy
curricula, called Creative AI, Dancing with AI, and How to Train Your
Robot. The authors evaluation of the curricula indicated that students
developed a critical lens to better grasp how AI systems work and how
they impact society. In addition to curriculum development, different
visual tools and hands-on activities have been employed at the middle
school level to teach ML concepts (Mahipal et al., 2023; Sakulkuea­
kulsuk et al., 2018). While existing works have demonstrated an
important start to how students can be supported to learn ML, more
efforts are required to particularly extend ML knowledge propagation to
middle schoolers in African settings, a region with dearth of studies in
relation to this initiative.
Investigating how students in an African setting learn ML is neces­
sary to understand how the uniqueness of the social context can be
leveraged to best introduce ML to them (Sanusi, 2021; Gwagwa,
Kachidza, Siminyu, & Smith, 2021). Gwagwa et al. (2021) argued that if
there is “absence of significant AI research and development in Africa,
the AI applications deployed in Africa tend to originate from outside the
continent and will thus lack contextual relevance.” In order for youths in
the developing context to keep pace with the global technological
advancement, it is imperative that “these sets of demography be
equipped with AI skills required to prepare them for workplaces where
human-AI teaming is the norm” (Oyelere et al., 2022). A few studies
have demonstrated an important start to how ML can be demystified in
classrooms in both formal and informal (DSN, n.d.; Sanusi et al., 2023)
settings. However, with ML concepts been postulated as difficult to
teach in the compulsory level of education owing to its abstract features
(Kim et al., 2023; Ottenbreit-Leftwich et al., 2022), students cannot be
expected to develop a full understanding of the concepts within the short
intervention (45 min) of the identified studies. Hence, the need for more
study with robust design and implementation of curriculum activities to
understand how students learn and how their attitude shifts towards
learning ML over time.
To provide evidence for policy makers, researchers, and relevant
stakeholders on the feasibility of ML inclusion in the formal school
system in Nigeria, we embarked on an intervention study. Adapting an
existing ethics infused ML curriculum, called AI MyData (Anonymous,
2023), we engaged middle school students in learning ML during a
3-week, 6-sessions intervention program. Student engagement has been
identified as a valuable means to foster and increase student learning
(Renninger & Bachrach, 2015; Tarantino et al., 2013). Fredricks et al.
(2004) described engagement as a multidimensional construct that en­
compasses behavior, emotion, and cognition. It was further stressed that
engagement can result to commitment or investment which is key to
enhancing students learning. We believe that our program will help to
introduce students to the joys of science at an early age and pique their
curiosity to be future ML experts or researchers. Drawing on prior
studies, the purpose of this study was to contribute to our understanding
of the evolution of middle schoolers’ comprehension and perceptions as
they engage in learning ML within the context of extracurricular activ­
ities in a public middle school.
This study addressed the following research questions (RQs):
RQ1. How does middle school students engage with the ML activities
in an African school?
RQ2. Does participation in our intervention program elicit changes
in students’ ML comprehension and perceptions in an African
school?
2. Conceptual framework
2.1. The three dimensions of ML literacy
Even though researchers and educators still grapple with AI learning
objectives, standards, and frameworks (Kim et al., 2023), there is an
2
Computers and Education: Artificial Intelligence 5 (2023) 100155
agreement that basic knowledge of AI, affective component and societal
implication of AI are critical for developing AI competencies (Ayanwale
et al., 2022; Kong et al., 2022, 2023; Lee et al., 2021; Xia et al., 2022). In
their framework for AI, Kong and Zhang (2021) propose a
three-dimensional conceptual framework of AI literacy as a) cognitive,
b) affective, and c) socio-cultural dimensions. The cognitive dimension
of AI literacy refers to educating people about AI basics and developing
their skills; the affective dimension involves equipping individual to
respond appropriately with the ubiquitous use of AI in their lives or
workplaces; and the socio-cultural component focuses on the ethical
implication of AI use in the society.
As shown in Fig. 1, the three dimensions were adapted in this study
to include applicable concepts in the scope of extracurricular activities
designed to promote ML ideas in a public middle school. The partici­
pants a) learned about some ML techniques, specifically supervised
learning which include classification algorithms such as decision trees,
k-nearest neighbor (k-NN), convolutional neural networks (CNN), and
expanded the scope to include algorithm bias (cognitive dimension); b)
engaged with ML platforms such as Google’s Teachable Machine (GTM,
Carney et al., 2020) and Doodleit (Mahipal et al., 2023) among other
unplugged activities to learn to confidently interact with ML as well as to
be inspired to formulate new solutions and understand the impact of ML
for the social good (affective dimension); and were c) prompted to think
ethically through discussions and videos of some negative and positive
examples of facial recognition systems as well as public cameras (so­
cio-cultural dimension).
2.2. Learning ML in middle school
While there is a call for more effort and initiatives regarding ML in
middle school (Sanusi et al., 2022), few studies have laid the foundation
for understanding what and how students learn ML within middle school
education. Ali et al. (2021) and DiPaola et al. (2021) introduced the
Creative AI curriculum which was used to teach how generative
adversarial networks work including deepfakes and its societal impli­
cation. Lee et al. ‘s (2021) DAILy curriculum was built on creative AI
with much focus on ethics and career development. Leveraging on
middle school students interests in dance and art, Jordan et al. (2021)
created Dancing with AI which comprise of AI-powered block-based
coding tools and learning modules which allows students to “design,
build, and reflect on interactive, movement-based, multimedia experi­
ences through a user-friendly gestural interface.” Williams et al. (2021)
designed How to Train Your Robot curriculum with learning activities
centered around machine learning, autonomous robotics, as well as
speech and image recognition. Sabuncuoglu (2020) also developed an AI
Fig. 1. The adapted three dimensions of ML literacy (Kong & Zhang, 2021).
I.T. Sanusi et al.
curriculum with specific focus on three areas including how computers
interact, how computers see & how computers hear. It is worthy of note
that all the curriculum highlighted above in a way incorporate ethics in
their learning activities which shows its significance in raising ethically
aware citizens of future creators of AI.
In addition to the curricula initiatives for middle schoolers, we
recognize other efforts being made by researchers to promote ML. One of
such efforts include development of technology platforms and tools to
ensure novices grasp the basics of ML and how ML works (Mahipal et al.,
2023; Rodríguez-García et al., 2021). Another attempt to popularize ML
among youth has been using manipulatives and hands-on activities (Ma
et al., 2023; Sakulkueakulsuk et al., 2018). As shown above, all the
initiatives on AI and middle school education were reports of experi­
ences from Global North with a high proportion from the United States.
This suggests that more research focusing on youth in the Global South
should be pursued to get a clearer picture of how students can be sup­
ported to learn ML across contexts and make democratization of
teaching and learning of AI indeed a global initiative. Since it has been
established in literature (e.g., Sanusi et al., 2022) that additional ML
resources are needed for middle schools, considering the
under-researched population, especially Africa, is important.
2.3. Initiatives on learning ML with young learners in African context
Different countries have begun to research into ML curricula among
other initiatives to promote ML within the Europe and USA as well as
HongKong among other developed regions. However, more curricula
activities need to be developed especially in the Africa context with
paucity of initiative towards ML in the compulsory level of education
(Sanusi, 2021b). While different African countries may have different
needs for ML because African societies are intrinsically multiethnic and
multicultural, the use of the emerging technology is inevitable. For
instance, a recent comprehensive review of ML research trend in Africa
found that ML technology can help address some of Africa’s most
pervasive problems, such as improving education, providing quality
healthcare services among other sustainability development challenges
(Ezugwu et al., 2023). As a result, it is imperative to build human ca­
pabilities to be able to harness the use of this technology. In an attempt
to build these capabilities, a proactive approach is preparing youth at
their early age to begin to learn the fundamentals of how ML operate.
Utilizing this approach helps to build a foundation for future learning as
several literature outside Africa, especially in North America, and
Europe has shown that learning ML at young age help children to
develop useful mental model (Mahipal et al., 2023) and motivate them
to be AI and ML applications developer and researcher (Touretzky et al.,
2019). Based on the potential benefit attributed to teaching ML in
schools, as suggested in numerous studies (Vartiainen et al., 2021; Xia
et al., 2022), we hold the view that learning ML should be integrated
into the curriculum in African schools or incorporated across the cur­
riculum to orientate them towards the development of future technol­
ogies for their countries as well as prepare them for global citizenship.
The discourse about learning ML within the compulsory level of
education in African context is ill-informed (Miao & Holmes, 2022;
Sanusi et al., 2022d). However, building a knowledge base of how
middle school students learn about ML in the developing context will be
valuable for African scholars, curriculum designers and policy makers to
develop and strengthen ML education in schools. Although ML is not
currently introduced as a new subject area or embedded across the
curriculum in African schools (Miao & Holmes, 2022), initiatives are
emerging among non-governmental organizations (NGO) and re­
searchers to equip school-aged students with ML knowledge. One of
such organizations include Data Scientists Network in Nigeria who
design series of programs such as boot camps and competitions to spur
children’s interest and prepare them for opportunities in Artificial In­
telligence (AI) and Data Science (DSN, n.d.). DSN, in bid to empower
children with present and future skills also unveils AI books for primary
3
Computers and Education: Artificial Intelligence 5 (2023) 100155
and secondary school in Nigeria (Nwite, 2019). Relatedly, “AI IN AF­
RICA,” an organization domiciled in South Africa create platforms to
equip the youth to design an inclusive future with AI through work­
shops, events, and collaborative programs with academic, public, and
private institutions (Aiinafrica, n.d.).
Besides NGOs intervention, researchers and authors of the present
study have begun to provide scientific evidence on how teachers and
students can be supported to teach and learn ML respectively. The effort
based on our previous works can be summarized into three categories of
teacher conception, student perception and learner experience in a non-
formal setting. We identified three papers on teacher conception. Of the
three papers, one explored the initial conceptions of some African
teachers about teaching ML in schools while others focus on examining
factors influencing teachers’ intention and readiness to teach AI. Three
papers were found that reported the student perception of AI which
specifically centers around the role of students’ competencies in
learning considering cultural competence and teamwork among stu­
dents. One paper was found to have explored young learners experience
with ML in a non-formal setting. Our knowledge of the past research
suggests that initiatives regarding instruction design, curriculum
framework or student motivation on ML/AI education were not
considered. This suggests the need to pursue these to uncover how to
make ML learning effective and engaging to meet learning needs. This
study followed Merrill’s (2002) instructional design principles. Merrill
(2002) established five instructional design principles that can be
applied when designing any program or practice to achieve effective and
efficient instruction. This include making the content relatable and
solving real-world problem (problem-centered), activating existing
knowledge as foundation for new knowledge (activation), demon­
strating knowledge to learners (demonstration), applying new knowl­
edge by the learners (application), and integrating new knowledge into
the learner’s world (Integration).
3. Methodology
3.1. Study design and procedure
Drawing from previous literature with similar goals (Lee et al.,
2021), we utilized a pretest-posttest design approach to evaluate our
intervention program on ML education for middle school students. Be­
sides the pretest-posttest phases, to triangulate our findings, there were
written assessments and interviews (focus group discussions, FGD)
during the learning phase of the program. The study was carried out in a
regular school system within the context of extracurricular activities.
The whole program includes six 1-h sessions on ML lessons in the first
three weeks of second term in January 2023.
Participation was voluntary and participants were assured of ano­
nymity. Their parents’ consent to participate in the study was sought
after informing them about the purpose of the research, the activities,
and the kind of data to be collected. We subsequently got the partici­
pants assent at the onboarding phase; a day before the start of the pro­
gram having been apprised of the research goal, as well as the topics that
will be covered during the study, the ML tools that will be adopted and
the data gathering techniques. The policy statement regarding enroll­
ment of children in research in Nigeria (National Health Research Ethics
Committee of Nigeria NHREC, 2016) served as guidance in relation to
ethical consideration.
3.2. Context and participants
This study was carried out at a public middle school in southwestern
Nigeria during its extracurricular activities period. The extracurricular
activities are structured such that teachers and each student belong to a
particular club (e.g., Jet club, Drama Club, Public Speaking club, etc.)
based on their interest. These clubs assemble for an hour bi-weekly to
engage in activities set aside for the day/week. For instance, in Jet club
I.T. Sanusi et al.
which centers around discussion and engagement with hands-on activ­
ities related to STEAM, a meeting could focus on “how do we come to
know that a chemical reaction has taken place?” using intuitive exam­
ples. The club also engage its members in science fairs and excursions to
promote their interest in science, engineering, and technology. All these
clubs were ultimately designed to encourage and prepare students for
STEAM careers. Since the clubs are organized along the lines of science
and art, we reach out to all groups to accommodate participants of
varying interest and career aspirations.
Initially, 47 students expressed interest and signed up for the study
across grade levels within the middle grade bands. Given that we
planned to work with a sizable number 0f students with varying grades,
interest and background, this study reported only the demographics and
activities involving 32 students we selected. The selected students were
eight candidates each across four clubs considering their grade levels
and sex. Ten of the participants were in 7th grade, ten were in 8th grade
while twelve were in 9th grade. In terms of gender representation, 20
were boys and 12 were girls. They have their ages ranging between 11
and 14 and about fifty percent of them are interested in a STEM-related
career in future. Consistent with the design of our program which did
not require knowledge of programming, AI, or ML, all the participants
reported that they did not have background on any of the three concepts.
The students take computer studies since it is a compulsory subject in the
Nigerian school system.
3.3. Materials
This study took a cue from Eguchi et al.’s (2021) work who adapted
an AI ethics middle school curriculum implemented with students in the
USA (Payne, 2019) for Japanese classrooms. We deployed AI MyData
(Anonymous, 2023) with slight modification (see 3.3.2 for the changes)
to further calibrate the curriculum and some of its activities to ensure
contextual relevance. The adapted curriculum was initially piloted with
sixth to ninth graders in the United States. Since the present study focus
on similar set of demography (6th to 9th grade students), we assume it
could be suitable for them even though they reside in different region.
The curriculum’s name was coined based on the premise that without
“data,” there is no AI/ML. The activities in the adopted curriculum
include data science and ML concepts weaved with ethical orientation.
We however focused primarily on the ML-related concepts and its
ethical implication in this study. Detailed below are the pre-test, the
learning, and the post-test phases.
3.3.1. Pretest
The pre-test provided us the opportunity to assess the participants’
entry behavior before unveiling the activities in the intervention pro­
gram. In this phase, we also gathered the participants demographic in­
formation which includes gender, age, grade level, and career
aspiration. Four open-ended items adapted from prior study were used
to assess the students’ prior knowledge. The items centers on how the
participants understand ML, how ML is important to them, how they
would like to use ML in their daily life and discerning their concerns
with ethical implications of ML. To determine whether participation in
our intervention program elicit changes in students’ ML comprehension,
we launched the pre-test to serve as a baseline. The pretest was
completed a day before the learning phase.
3.3.2. Learning phase
The learning phase was organized as six sessions with different ML
topics and learning activities per session. Each of the sessions lasted for
an hour period. The adopted AI MyData curriculum with slight modifi­
cation to its activities to further calibrate it and as well ensure contextual
relevance. Learning the concepts in Fig. 2 contribute to ML literacy as
depicted in Fig. 1 (see 2.1). As shown in Fig. 2, we specifically taught the
students six ML-related concepts which includes: Introduction to AI &
classification, Decision trees, k-nearest neighbors (k-NN), Convolutional
4
Computers and Education: Artificial Intelligence 5 (2023) 100155
Fig. 2. The ML lessons covered in the program.
neural network (CNN), Algorithm bias, and Ethics of ML. Our peda­
gogical strategy across the sessions includes a fair mix of activity-based
learning, group work, instruction, and discussion.
In session 1, we began with the definition of AI and discussed the four
main things AI can do (perceive, learn, decide & interact) with illus­
trations. We further prompted the participants to mention what they use
or see on a daily that has AI in them. What is AI and what is not AI
formed part of our discussion using different examples. Further, we
introduced Google’s Teachable Machine (GTM) and demonstrated how
it works with the students – allowing them to critique if it recognizes two
persons differently. Building on the GTM demonstration, we presented a
fish recognition activity where students were to work with the hypoth­
esis “is GTM able to recognize venomous fish and harmless fish?.” In
session 2, we taught the students decision trees with an unplugged ac­
tivity approach – the use of pasta kits (Lee & Martin, 2018). Having
introduced the concept of decision trees through a PowerPoint presen­
tation and how to execute the activity, we gave the students tasks to
build a decision tree to classify five different types of pasta handed out to
them. After the task, we handed out a mystery pasta and had the stu­
dents classify based on the decision tree they earlier built. We had
conversations about the mystery to know if their tree classifies it with a
closely related pasta shape. The discussion led to conversations about
the tree having bias or not. Session 3 focused on k-nearest neighbor
activity. The paper activity originally used in the study of Ma et al.
(2023) applied k-NN to find the species of three mystery penguins. Since
duck is predominant and easily recognized by the students in the study
area, we had them identify three species of duck (Muscovy duck,
Campbell duck and Pekin Duck) in Nigeria. The activity was introduced
after explaining that k-NN is one of the simplest classification methods,
when k-NN is to be applied and the illustration of when k = 1 and K = 3.
Session 4 centered on how features are extracted from images in CNN.
The aim of the lesson was to learn about how CNN performs image
recognition. The session commenced with the introduction to image
recognition. We then describe CNN after which all the students had a
hands-on practice with DoodleIt, a simple sketch recognizer for learning
CNN (Mahipal et al., 2023). We further introduced an unplugged
paper-based activity about how computer sees different types of animals
to reinforce learning of the CNN concept. In session 5, algorithm bias
was the topic of discussion. The session started off with definition of
algorithm bias and gender shades video (Buolamwini, 2018) showing
I.T. Sanusi et al.
bias in face recognition technology. Since the issue of bias in AI/ML
technology is still pervasive and embedded, we consider it as a topic to
consider. We further introduced an activity using the pose model
segment of GTM (with their bodily expressions) to depict bias by varying
samples uploaded in the classes to be trained. In the final session, we
taught and discussed ethics of ML. We explained the idea of ethics and
why ethics of ML is important. We further showed videos of some
non-desirable and positive examples of facial recognition in the society
to prompt the students to think about the ethical and societal implica­
tions of ML. The students worked with either computer-based applica­
tions or pencil and paper/manipulatives or both in most of the sessions.
The students were assigned to a group with a total of 8 groups with each
group comprising four students. Throughout the program the partici­
pants mostly performed tasks and worked in groups. Our program is
designed to understand how students behave, how they feel, and how
they think about ML which is the overall aim of engagement (Fredricks
et al., 2004).
3.3.3. Posttest
The posttest was carried out immediately after the intervention
where the participants answered the paper-based test. Considering that
our intention is to measure students’ conceptual understanding of ML
and the effect of the intervention vis-à-vis their prior knowledge, we
adopted the items used in the pretest phase. However, we added two
questions to allow participants to reflect on what they found most
interesting and/or most confusing in the learning materials to assist us in
further curating those materials.
3.3.4. Data collection and analysis
Literature has shown that to enhance the credibility of what is found,
triangulation of evidence should be adopted (Lincoln & Guba, 1985). As
this paper focuses on discerning participants’ comprehension and per­
ceptions as they engage in ML lessons, multiple data collection tech­
niques were considered to gauge how they process the learning
materials including their experiences. Specifically, we elicit information
through pre-post open-ended surveys, FGD, and written assessments. Per
the focus group interviews which were conducted in the last 10 min of
each session, we used semi-structured questions which bothers on the
topics/activity for each session. For instance, one of the cognitive
prompts for the first activity was “Does varying the size of your input
data impacts accuracy? What else could impact accuracy? Why?.”
Another example of a question on the algorithm bias lesson was “How
could we make our training data better to avoid bias?.” The written
assessments and deliverable tasks were also topic specific as shown in e.
g.,Figs. 3, 4 & 5 to assess the comprehension of the ML concepts.
Following the approach adopted by earlier studies (Ma et al., 2023;
Mahipal et al., 2023), we showcased some of the assessment including
completed tasks and provided a narrative on how it suggests the par­
ticipants understand our learning materials. While our analysis relied on
a qualitative content analysis (Elo & Kyngäs, 2008), we specifically
adopted the approach of Mahipal et al. (2023). We analyzed the
pretest-posttest by reading through the responses of the participants to
Fig. 3. Decision tree created by the participants using pasta.
5
Computers and Education: Artificial Intelligence 5 (2023) 100155
capture and summarize their thoughts. We also refer to their feedback as
quotes to buttress our assertions. Per the interviews, the transcribed data
provided us with information on students’ comprehension of ML con­
cepts. We used quotes and excerpts from the participants to show how
they were thinking about ML.
4. Result
4.1. Pretest results
In the pretest responses, 80% (N = 32) of the students recognized
only smartphones as the common things they know that have ML in
them. Only two of the participants specifically mentioned face ID
application and automated teller machine. About 60% (N = 23) who
responded to the question “is AI important to you?” believes that AI is
important to them because of income and learning of relevant skills as
reflected in their statements: Yes, one can gain relevant skills that can
provide income; It helps to understand changes in our environment.
Regarding how they would like to use ML technology in their daily life,
students were interested in using the technology in planning their time,
for future career, and providing services for organizations. Finally, when
they were asked about the concerns they have with respect to ethics and
social effect of ML, about 12% (N = 4) had concerns which were coined
as “negative things about the internet,” “less use of ML applications in our
society.”
4.2. Evidence of engagement with learning activities
Here, we detailed some of the evidence that portrayed the partici­
pants’ engagement with the learning activities across the sessions. The
findings presented are based on the transcribed data of the interviews
and written assessment/completed tasks during the activities.
4.2.1. Session 1
The students’ engagement with GTM activity provided us with a
proof of how they began to learn about classification algorithms. Having
demoed the functionality of GTM with the students to determine
whether the learning tool recognized two individuals differently, they
embarked on the main activity for the session – fish recognition activity.
The hypothesis for the activity was “Is GTM able to recognize venomous
fish and harmless fish? The students in groups were handed 12 different
printed cards with 4 each being venomous, harmless and test cards. The
test cards (scorpionfish, bluefish, stargazer fish & croaker fish), a mix of
venomous and harmless fish were unlabeled since they are meant to test
out the output of the trained labelled model. Prior to the lesson, the
participants are not aware of the different types of harmless fish and
venomous fish. Getting to use these ideas to teach classification based on
the categories the fishes belong opens up the possibility of teaching ML
within the context of life sciences.
Before the participants explored the GTM by training both categories
of fishes, they had earlier predicted the two fishes that were either
venomous or harmless in a paper activity. With the learning tool, the
I.T. Sanusi et al.
Fig. 4. Worksheet completed by the participants on classification of ducks with k-NN algorithm.
Fig. 5. Completed tasks used to assess the way participants comprehend how computers recognize images.
students’ grouped both examples of the fishes into categories they
wanted the computer to learn and trained their model. They then tested
the model to see whether it correctly classified the new examples of the
unlabeled fishes. The students were excited to see that the computer
recognized venomous and harmless fish differently. However, in most of
the groups, the computer fails to correctly identify if one of the fishes
(croaker) was venomous or harmless with a high confidence rating. This
contention led to focus group discussions where participants reflect on
what transpired in the session.
We were able to gain insight into how the students understood
classification during conversation about the GTM activity. About 90
percent of the students stated that when they compare the GTM output
with their own conclusions from the unplugged activity, the model
agrees with them. As shown in the excerpts of the FGD below, the stu­
dents indicated that they recognized venomous and harmless fish
differently based on their features. The conversations further revealed
that they assume computers also recognize the two kinds of fish differ­
ently based on their physical attributes.
Researcher: Does the model agree with you when you compare
GTM’s output with your own conclusions from the unplugged
activity?
FGD1: Yes, most (3 out of 4) of the fishes were recognized correctly.
It is unsure if croaker is either venomous or harmless and most of the
time identify it as venomous. The confidence rating is below 50
percent.
Researcher: Why does the computer classify croaker as a venomous
fish?
FGD2: It is possible the computer identifies croaker as a venomous
fish due to the similar characteristics it shares with other venomous
fishes. They have rough body types unlike harmless fish.
Further, we got a glimpse of how students think of input data and its
impact on the output of the model. As shown in some passages of dia­
logue between the researcher(s) and participants below. More than half
of the students opined that having more data may impact accuracy when
testing out the model. They conceived this idea of more data, more
6
Computers and Education: Artificial Intelligence 5 (2023) 100155
accuracy when they were prompted to ascertain if GTM can recognize
themselves and their colleagues differently. The students also believe
using a variety of sample data with different features could also impact
accuracy because the model will only provide outputs similar to or based
on what it was trained on.
Researcher: Does varying the size of your input data impacts
accuracy?
FGD3: Yes, more data in different classes may affect the accuracy of
the output. It recognizes our test data with high confidence rating
when we demonstrate if a computer can differentiate the faces of four
of us.
Researcher: What else could impact accuracy? Why?
FGD2: The use of a variety of data samples can affect. I also think
about the features of those data. We thought of these examples
because you must train your model first to see the effect as output.
4.2.2. Session 2
The decision trees built by the participants and their reflection at the
end of the session indicates how they understood the concept. As shown
in Fig. 3, the students were able to classify different types of pasta
handed to them. The session started with introducing the concept of
decision trees through a 5 questions game with animals, where the
students organize different questions into different trees based on their
characteristics. We further introduce the Pasta land activity with a few
example questions about the pasta to get the kids started. The partici­
pants worked as a group to create pasta decision trees with 5 pasta types
handed to them and then share the model. We provided a new pasta for
the students to test the testing pasta on their own model and discuss the
outcomes.
Vignette of completed tasks (Fig. 3) indicated how the decision tree
method was used for classification. In both (a) and (b), the students
created decision trees based on the data features, in this case pasta
features. Both images adopted different attributes of the pastas to build
their flowchart-like tree structure. This different approach of thinking
about the data categorization showed that the students were making
I.T. Sanusi et al.
their own choices based on their understanding that the decision tree
depicts splitting the source set into subsets based on attributes. The
students thought about the activity further indicate how they think
about decision tree concept. A student mentioned that I classified the
pasta along the line of physical appearance and another stated that the
pasta was categorized according to the similarities they share, when asked
how they were asked how the answers were arrived at. It is further
evident that the students understand that there are not many ways to the
classification other than considering the characteristics of those
elements.
4.2.3. Session 3
Fig. 4 showcases the tasks performed by students where they applied
k-NN to identify three mystery species of ducks. Having described k-NN
as a supervised learning classifier which uses proximity to make classi­
fications, we demonstrated through the species of duck that it classifies
the data point on how its neighbor is classified. We also mentioned that k
represents the number of the nearest neighbors used to classify new data
points. In the task completed by the students, we asked them to classify
when k = 1 and k = 3. As shown in Fig. 4, the completed tasks revealed
the students realized the rationale behind the k-NN algorithm.
In (a) above, the student was able to show that the quality of the
predictions depends on the distance measure, likewise in (b) who could
apply the k-NN idea to identify the mystery ducks. The figures a & b
exemplifies the kind of feedback retrieved from the participants about
the distance-based algorithm. From the cognitive interviews feedback,
we deduced that while students identified when k = 1 differently, they
all understood that the k-NN algorithm considers the closest training
data point to the point they want to make a prediction for. We concluded
that students realized the rationale behind k-NN algorithm based on the
follow-up conversations we had with them. For instance, when the
students that completed tasks in Fig. 4 (a) & (b) were asked how they
arrived at the answers, (a) mentioned that “I measured the spaces between
the different types of duck and unknown duck to determine the closest” and
(b) similar stated that “I was looking for the duck species that are within the
shortest distance to the mystery ducks.” The students reported that they
can now recognize the three species of ducks they see around almost
every day by their name. In response to “where else do you think k-NN
can be applied to,” the students mostly mentioned different species of
animals and objects. However, two students gave descriptions that
suggest recommender systems. A student said facebook suggests friends
and people we may know and another mentioned that an online shopping
platform could suggest items close to what was initially searched or bought.”
This response is an indication that students can learn ML concepts and
infer how it is applied in the real-world through use of practical exam­
ples and approaches.
4.2.4. Session 4
With a computational tool, called Doodleit and an accompanying
paper activity, we introduced CNN concepts to the students. How the
tool operates is that whenever a student sketches on the area dedicated
for drawing, it visualizes the filter and features maps including the
output neurons in percentages based on the six classes of images (cat,
sheep, triangle, cake, door & apple) with which it was trained. The
students interfaced with the Doodleit in their respective groups and
were fascinated with the interactive nature of the tool. However, they
have concerns with how the image drawn is being misrepresented and
attributed to a totally different image in the output neuron. For example,
a sketch of a fish, dog or a random image could be shown as a cat. We
showed the students some examples of datasets with which each of the
six images were trained and we mentioned to them that the image drawn
was identified by feature extraction. They understood that data has so
much influence in the recognition accuracy and that the computer rec­
ognizes images based on similarity in their features as shown in the
completed assessment in Fig. 5.
As shown in Fig. 5, it shows the students got the idea of how CNN
7
Computers and Education: Artificial Intelligence 5 (2023) 100155
works. For instance, the students in (a) likened the duck image to either
chicken, goose, or turkey with the explanation that they share compa­
rable characteristics. Also, in (b), the students even choose only goose
with the justification that it is closer to duck in terms of attributes. The
responses provided for the completed assessment suggests that the stu­
dents understood the basic concept of CNN.
4.2.5. Session 5
The algorithm bias activity which focused on using the pose feature
of GTM provided us some pointers to how students construed algorithm
bias. The activity was conducted by varying the pose samples of different
classes and testing out the model. Their tested model was found to be
biased most times towards the samples with more datasets. The con­
versations with students after the activity revealed that they recognize
that ML depends on the size of the data used to teach it. Below is an
excerpt from two of the groups.
Researcher: What happened after testing your algorithm? Why?
Group 2: It recognizes our poses that have more data. I tripled one
category of the samples during the training.
Group 4: There is bias towards the classes that has much data of
poses. We have few poses captured in one class and several poses
recorded in another class.
The response from the students suggests their understanding of al­
gorithm bias in that they recognize that bias can emerge from decisions
relating to how data is collected and selected to train the algorithm. To
make the students aware that many factors contribute to a computer
system yielding unfair outcomes, we initiated a discussion around the
subject prompted through a video on racial discrimination in face
recognition technology.
With a video by Buolamwini (2018) on facial recognition systems that
did not recognize darker, female faces as well as lighter, male faces, we
further reinforce the idea of “training data matters,”. The 5-min long
video was followed by discussion on why the problem identified in the
video should be of concern and ways it could be addressed. When the
students were asked what they make of the video. Students’ answers
were that recognition technology does not work for people of colors as
exemplified by a student response that “the technology does not work for
people with dark faces.” Regarding why the issues raised in the video
should be a topic of interest, most of the students responded that the
technology would work for selected people which is different from the
intended function of the algorithm if not addressed. For instance, a
student mentioned that “facial recognition technology will only work for
few sets of people if the topic is not discussed” and another student stated
that “mistakes of identifying people wrongly may occur”. In response to
how the racial discrimination pointed out in the video can be managed,
the students recognized that use of variety of data considering different
skin types among other qualities is key.
The students’ responses indicated that when the technology doesn’t
work the same way for all people, it can be misleading and causes
humans to make inaccurate decisions. Their feedback on how to address
bias suggests that datasets should be better curated. The students were
able to establish that one of the major causes of bias in ML is insufficient
training data.
4.2.6. Session 6
We were able to raise the consciousness of the participants on ethics
of ML through videos of scenarios that showed the pros and cons of the
use of facial recognition technology. Further, we discussed some ethical
dilemmas as a prompt to understand how they think about the ethical
implication of ML around social media and facial recognition
technology.
Our discussion was guided by three main questions adapted from
Skirpan et al. (2018): 1) Do you believe facebook/instagram/tiktok is
good or bad for society?, and 2) Do you believe face recognition
I.T. Sanusi et al.
technology is good or bad for society?.
In response to question 1, all the students believe that social media
platforms can either be good or bad for society depending how and what
individuals use it for. While most of them speak highly of the chances it
provides for communication and networking, few reflect on revealing
personal information.
With respect to question 2, almost all the participants saw the
technology from a positive perspective with appreciation for how the
technology can aid in policing. They specifically wonder if the
technology is in use by Nigerian law enforcement agencies and how
it should be adopted to track down suspected criminals, solve crimes
and find missing people. We got an indication that the students have
begun to have ethical orientation based on the dialogue that ensued
among some FGDs:
FDG4: M1 – … ….. if facial recognition technology is being used by
police for example, it means our locations will be monitored. F2: I
also think the technology may be inaccurate, as we all see when
Google’s Teachable Machine recognizes harmless fish as venomous
fish.
FDG7: F1 - … ….. so, with technology, will the government or other
people be aware of what I do in private? M1: Yes, it looks like
someone will be watching whatever we do which I don’t think is fine.
The excerpts above suggest that students have started to think ethi­
cally in relation to use of ML applications which suggest they will be
more attentive to such issues in future. They were also able to identify
why ethics is important – ensuring ML initiatives maintain human dig­
nity and do not in any way cause harm to people. All the participants
indicated they would like more lessons on ethics because it will help
them understand better the effect of ML on society.
4.3. Posttest results
In contrast to the findings from the pretest results, students
mentioned several things they know that have ML in them which in­
cludes social media, google, face ID and voice assistants, image recog­
nition apps. Almost all the students (26 out of 32) found ML more
important with reasons related to preparation for future learning and its
use for social good. For example: It will help me to learn machine language
in the future; It will help to solve peoples and societal challenges. In response
to how they would like to use technology in their daily life, they
mentioned crime detection, to improve security and assist the physically
challenged. Most of the students had concerns about the societal effect of
ML after the program. Their concerns can be summarized into algorithm
bias, misuse of data and infringement of personal privacy as shown in
some of their responses: My concern is that if you did not have enough data,
you will not get accurate results; our data can be used without permission.
Almost all the students reported they found GTM activity more inter­
esting and decision tree lesson less interesting which suggest that the
activity could be more calibrated.
5. Discussion
5.1. Reflection of findings and practical implication
Here, we briefly reflect on our findings with respect to the two
research questions guiding this study and highlight the implication for
practice.
5.1.1. How does students engage with the ML activities?
Our study supports the ample evidence that indicated technology
incorporation in classrooms as an instructional tool enhances student
learning and educational outcomes (Devlin et al., 2013). The
grade-appropriate materials which encompasses hands-on, and collab­
orative approaches to learning adopted in this study suggest how
8
Computers and Education: Artificial Intelligence 5 (2023) 100155
students can be engaged in learning ML concepts. The students’ inter­
action with the computer-based tools and concrete materials across the
learning phase showed that employing such approaches is effective for
promoting children’s participation in ML activities. This is consistent
with past research (Lee et al., 2021; Ma et al., 2023; Williams et al.,
2022) which adopted similar approaches with middle school students in
developed contexts. Having subjects from the developing context
demonstrate their understanding of ML concepts through engagement
with ML platforms, tangibles and manipulatives indicate that students
regardless of their geographical location can learn ML using such
approaches.
As it is widely acknowledged that engagement plays a critical role in
learning (Greene, 2015; D’Mello et al., 2017), advancing the scientific
study of how students learn ML requires investigating students’
engagement with ML activities. This study’s findings align with existing
evidence that indicates when students are engaged in their own
learning, it comes with multifarious benefits including increased moti­
vation and achievement (Aguilar et al., 2022; Sinatra et al., 2015). Our
result suggests the students were cognitively, emotionally, and behav­
iorally engaged based on the three components of engagement proposed
by Fredricks et al. (2004). The result that demonstrates cognitive
engagement pertains to the students’ investment in the learning task,
specifically as it relates to understanding of content and learning ma­
terials. As detailed in the result (section 4.2), we could deduce increase
in learning gains based on the completed tasks, conversations, and re­
sponses to cognitive interviews across the program sessions. Further in
the analysis, the feelings, and attitudes of the students about the ML
activities illustrates emotional engagement (Renninger & Bachrach,
2015). Specific examples include the expression of interest by the stu­
dents to take more lessons on ethics of ML, interest in exploring GTM
more as well as other ML tools utilized in the program and general
satisfaction about most of the program activities. According to King
(2020), “a student who arrives to class prepared to learn and engages in
the learning process through on-task behavior and class participation is
exhibiting the broad indicators of behavioral engagement.” As obtain­
able in the study of Gomes et al. (2023), examples of actions that re­
flected behavioral engagement were completion of exercises, tasks, and
written assessments including interaction with peers and instructors.
There is also an indication of social-behavioral engagement (Pekrun &
Linnenbrink-Garcia, 2012) considering how the students collaboratively
participated in the ML activities with their peers. An instance is inter­
action among the participants during the group work activities to
complete a given task. Such as in Fig. 3 where students deliberated on
pasta classifications in a bid to understand the decision tree concepts.
The quality of the social interactions during the session on ethical
implication of ML using facial recognition technology as a point of dis­
cussion also illustrate the occurrence of social-behavioral engagement
(Linnenbrink-Garcia & Pekrun, 2011).
5.1.2. Does students’ participation in our intervention program elicit
changes in students’ ML comprehension and perceptions?
We analyzed the participants’ responses to our pretest-posttest
questions to determine whether the students’ participation in our pro­
gram elicit qualitative changes. The findings show an obvious difference
from how the students conceptualize ML before and after the interven­
tion study. The participants now realize that ML is encountered by them
daily with the use of certain devices or services. Our findings revealed
that even though there is unequal access to new technology in Nigeria
due to social inequalities in income distribution (Sanusi et al., 2023),
students are not unaware of some of these emerging technologies. They
interact with smart devices, but they do not know they contain ML ap­
plications. It is also interesting to note that students in their posttest
views after the program showed they recognize the use of ML for social
impact, whereby ML applications are developed to tackle social, envi­
ronmental and health challenges in the society. Against the imagination
that ML is only focused on by big tech giants and experts, the young
I.T. Sanusi et al.
students were able to see that this technology impacts their everyday
life, and they are a key stakeholder in ensuring we build ethical ML into
the applications, product, and services in future. Based on
pretest-posttest responses, we could infer that our intervention, that is
the students’ engagement with the ML activities led to the changes
observed. This finding is consistent with past and related research
exploring the pre and post effect of an instructional AI program in
middle school context (Lee et al., 2021). The outcome of this study is an
indication that teaching of ML algorithms concept to middle schoolers is
possible in Nigerian schools given that learning resources are available
including the utilization of relevant approaches.
Our findings have implications for practice. First, since results from
this study corroborate the contention that learning cannot have the
desired impact on learners unless the instruction is engaging and
effective (Chiu, 2021), educators should design ML activities that will
provide students opportunity to engage cognitively, emotionally, so­
cially, and behaviorally. Second, the use of both plugged and unplugged
approaches should be encouraged to effectively support students
learning of ML concepts. Unplugged learning has been found to increase
young students’ mastery of several technology-related concepts (Her­
mans & Aivaloglou, 2017; Wohl et al., 2015). Likewise, past research has
showed that use of plugged activities gets the students engaged and can
foster computer literacy learning including programming concepts
(Brackmann et al., 2017; Sáez-López et al., 2016). To address the limi­
tations that may arise with the use of either of both approaches, it is
recommended to combine both computer-based activities and activities
for which no digital equipment is needed (Brackmann et al., 2017;
Sigayret et al., 2022). Third, the use of collaborative learning approach
as exemplified in this study could be adopted by the teachers. While
Gillies (2014) describes collaborative learning a teaching tool to
encourage students to think critically and solve problem, research has
shown that collaborative learning promotes student STEM interest
(Casad & Jawaharlal, 2012; Mosley et al., 2016). Adopting collaborative
approach is more important in Nigerian context since past research
suggested that there is lack of collaborative skills among students in
Nigerian schools (Akor et al., 2019; Sanusi et al., 2022c). Fourth, based
on the premise that introducing materials, tools and activities that are
familiar to the student for instruction leads to effective learning (Russell
et al., 2012), educators should embark on contextualizing students’
learning experience using items or object that are of interest in their
specific context. This is particularly relevant as prior research indicated
that contextualized resources are effective for AI implementation in
schools (Eguchi et al., 2021; Oyelere et al., 2022). In the present study,
we attempt to introduce context-bound activity by adapting a k-NN
penguin activity to duck which is common in the study area. We also use
images of mammals and birds familiar to the participants to illustrate
the idea of recognition in CNN. We believe that the use of relatable
materials could encourage students to learn about ML. Fifth, school
administrators could popularize ML concepts in their various schools by
creating AI or ML club which would be a platform to promote learning
and design of ML applications. While educators are expected to engage
the learners with series of approaches, they also require professional
learning opportunities to execute effective instruction. Therefore, pro­
fessional development programs designed to equip the teachers with
technological pedagogical and content knowledge to provide them with
sufficient knowledge and skills to integrate ML into teaching (Tang et al.,
2022). Finally, this study’s findings offer the policymakers the feasibility
of teaching ML in classrooms, as such resources can be made available to
support integration of ML in schools.
Additionally, this study provides some suggestions for developing
middle schoolers ML literacy in Africa or similar contexts. Based on the
result of this study, we recommend that educators and practitioners
contextualize the activities based on the needs of each African societies
since the continent is home to diverse populations with each country
having its own tribes, languages, and cultural differences. This study
also suggests that the basic idea behind ML can be understood by middle
9
Computers and Education: Artificial Intelligence 5 (2023) 100155
school students in Nigeria, which is an indication that children in other
Africa countries may be able to process the ML learning materials. This
claim should be validated by future research involving children from
other Africa countries. This study further showed that out-of-school time
can be used to promote the concept to students, to this end, policy­
makers, education leaders and practitioners and non-governmental or­
ganizations in the continent can provide support for educators and
students to learn through out-of-school learning venues. Pursing such
initiatives is important as evidence have shown, especially in the west­
ern world that afterschool and summer learning programs are an
essential tool to support and expand K-12 computer science education
(Dunton & Rodriguez, 2018; Ma et al., 2023).
In summary, considering the three dimensions of the ML literacy
adapted to this study, we analyzed the triangulated data from partici­
pants for pointers to show their a) understanding of basic ML concepts
(cognitive dimension), b) use and interaction with ML applications
(affective dimension), and c) recognition of ethical implications of ML
applications (socio-cultural dimensions). We fulfilled all these di­
mensions by demystifying the concepts of ML with bite sized examples,
facilitating interaction with ML platforms, and initiated discussion
around conduct of facial recognition technology use.
5.2. Limitation and future work
We noted four limitations of this project which borders on the study
design. First, the participants were recruited from a government-owned
school in an urban city in Nigeria. Future research needs to examine
other regions in the country with different ML needs including private-
owned schools and rural areas to address equity issues and the di­
chotomy between school location and ownership (Sanusi & Olaleye,
2022). Second, the program detailed in this paper was conducted as an
extracurricular activity within the formal school system which makes it
optional learning. Future research could conduct the study as part of
regular classroom activities or adopt a longitudinal research design
approach. Third, we did not analyze our results based on the partici­
pants’ interest. This may be an interesting future research direction since
our participants were recruited from the clubs organized along the lines
of sciences, languages, and art, with varying career interest and career
aspirations. Fourth, the sole focus on student’s feedbacks without
teachers’ data is a limitation to this study. Supporting data from teachers
would have provided us more evidence to support how they believe their
students regard ML. However, the series of data collection approaches
we used offered us enough proof to conclude students understood basic
ML concepts and its ethical implication. Lastly, this study attempts to
ensure the credibility and transferability of the findings to fulfil the
trustworthiness requirement of a qualitative research (Shenton, 2004).
This was done by triangulating the data and methods with detailed
reporting of the data collection process. However, we did not consider
investigator triangulation or participants checking during the analytical
phase of the research (Twining et al., 2017) which could further enhance
the trustworthiness of our work.
We have four future work paths. First, we plan to build a no-code
machine learning platform for young learners in an African context
with a bias towards nature, culture, accent, and artefacts. The applica­
tion interface will accommodate interaction among teachers, students,
and parents aside other modules on the learning platform. Developing
tools with more functionalities that support collaborative teamwork and
sharing is important as earlier studies (Gresse von Wangenheim et al.,
2021) observed existing tools do not support teachers monitoring their
students’ learning process. Second, we plan to further revise the content
in this study and create an ML curriculum that meets an interdisciplinary
approach. We will particularly customize the topics addressed to moti­
vate the students by engaging them with problems and approaches that
are of interest in their specific local context, e.g., the use of relatable
examples and contextual games as proposed for students in Nigeria by
Oyelere et al. (2022). Developing an ML virtual tool and curriculum for
I.T. Sanusi et al.
the emerging context is supported by past research (Eguchi et al., 2021;
Oyelere et al., 2022) that exemplified the importance of contextualizing
resources and learning experience for the targeted population. Third, as
widely observed in literature that professional learning opportunities for
ML is lacking, especially in Africa (Ayanwale et al., 2022; Ayanwale &
Sanusi, 2023; Sanusi et al., 2022b), we plan to create teacher training
initiatives through workshops and short courses to prepare educators
adequately for the implementation of the contents in the classrooms.
Teachers’ centrality in promoting ML ideas in school has been estab­
lished (Sanusi et al., 2022e), hence the need to have them ready and
believe in its relevance for students through the provision of ML
knowledge as well as relevant pedagogical and technological content.
Lastly, we aim to carry out a semester and or an academic year-long
study to evaluate the students’ learning outcomes, their understanding
and application of ML ideals and attitudes towards ML. We anticipate
that our resources will be scaled to students and teachers across schools
and learning context in different regions of the context for wider access
and acceptability. In addition, how are the policymakers thinking about
ML education for young learners? is a potential research question. The
point of view of the policymakers would suggest how they regard the
subject and ways they could be convinced since adopting a new subject
as a curricular material requires the analysis of the state’s policy and
future needs (Sanusi et al., 2022c).
6. Conclusion
There is a dearth of research in developing countries, specifically
Africa on ML education within the compulsory level of education.
Consequently, exploring ways and implementing instructional units to
gain insight into how best to approach the increasingly popular concept
in schools is necessary. This paper provided a qualitative insight of how
middle school students comprehend ML while engaging with ML activ­
ities using plugged and unplugged approaches. Our analysis revealed
that participation in our intervention program elicit changes in students’
ML comprehension, perceptions, and practices. The multiple data
collection techniques provided evidence to show that a middle school
student can learn and understand ML, even when one had no prior
knowledge or interest in science related careers. It also demonstrated
that ethically aware citizens in relation to the use of ML applications can
be raised as the participants were able to weigh in on ML ethical issues
and dilemmas. This study contributes to knowledge on middle schoolers
learning of ML concepts using relatable materials and approaches that
future study can leverage in engaging similar populations, particularly
in developing regions.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
List of Acronyms
AI Artificial intelligence
ML Machine learning
k-NN k-nearest neighbor
CNN Convolutional neural networks
NGO Non-governmental organization
DSN Data Scientists Network
NHREC National Health Research Ethics Committee of Nigeria
FGD Focus group discussion
STEAM Science, Technology, Engineering, Art, and Mathematics
GTM Google’s Teachable Machine
10
Computers and Education: Artificial Intelligence 5 (2023) 100155
References
Aguilar, S. J., Galperin, H., Baek, C., & Gonzalez, E. (2022). Live instruction predicts
engagement in K–12 remote learning. Educational Researcher, 51(1), 81–84.
Aiinafrica. (n.d.). AI IN AFRICA. Retrieved January 31, 2023, from https://aiinafrica.
org/.
Akor, T. S., Bin Subari, K., Binti Jambari, H., Bin Noordin, M. K., & Onyilo, I. R. (2019).
Engineering and related programs’ teaching methods in Nigeria. International Journal
of Recent Technology and Engineering, 8(2), 1279–1282.
Ali, S., DiPaola, D., Lee, I., Hong, J., & Breazeal, C. (2021). Exploring generative models
with middle school students. In CHI conference on human factors in computing systems
(CHI ‘21).
Anonymous. (2023). AI MyData: Fostering middle school students’ engagement with
machine learning through an ethics-infused AI curriculum. In redacted. Submitted.
Ayanwale, M. A., & Sanusi, I. T. (2023). Perceptions of STEM vs. Non-STEM teachers
toward teaching artificial intelligence. In 2023 IEEE AFRICON conference. IEEE
(Accepted).
Ayanwale, M. A., Sanusi, I. T., Adelana, P., Aruleba, K., & Oyelere, S. S. (2022). Teachers’
readiness and intention to teach artificial intelligence in schools. Computers in
Education: Artificial Intelligence, 1–11. https://doi.org/10.1016/j.caeai.2022.100099
Brackmann, C. P., Román-González, M., Robles, G., Moreno-León, J., Casali, A., &
Barone, D. (2017). Development of computational thinking skills through unplugged
activities in primary school. In Proceedings of the 12th workshop on primary and
secondary computing education (pp. 65–72).
Buolamwini, J. (2018). Gender shades. Retrieved from https://www.youtube.com/watch
?v=TWWsW1w-BVo.
Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., Jongejan, J.,
Pitaru, A., & Chen, A. (2020). Teachable machine: Approachable web-based tool for
exploring machine learning classification. CHI 2020, April 25–30, 2020, Honolulu, HI,
USA.
Casad, B. J., & Jawaharlal, M. (2012). Learning through guided discovery: An engaging
approach to K-12 STEM education. In 2012 ASEE annual conference & exposition (pp.
25–886).
Chiu, T. K. (2021). Student engagement in K-12 online learning amid COVID-19: A
qualitative approach from a self-determination theory perspective. Interactive
Learning Environments, 1–14.
D’Mello, S., Dieterle, E., & Duckworth, A. (2017). Advanced, analytic, automated (AAA)
measurement of engagement during learning. Educational Psychologist, 52(2),
104–123.
Devlin, T. J., Feldhaus, C. R., & Bentrem, K. M. (2013). The evolving classroom: A study
of traditional and technology-based instruction in a STEM classroom. Journal of
Technology Education, 25(1), 34–54.
DiPaola, D., Ali, S., & Breazeal, C. (2021). What are GANs?: Introducing generative
adversarial net- works to middle school students. In Proceedings of the 11th symposium
on education advances in artificial intelligence (EAAI ‘21).
DSN. (n.d.). Data Scientists Network, AI for Kids and Teens. Retrieved January 31, 2023,
from https://www.datasciencenigeria.org/ai-for-kids-and-teens/.
Dunton, S. T., & Rodriguez, S. (2018). Examining the role of informal education in K-12
computing pathways & CS education reform efforts. In Proceedings of the 49th ACM
technical symposium on computer science education, 1064-1064.
Eguchi, A., Okada, H., & Muto, Y. (2021). Contextualizing AI education for K-12 students
to enhance their learning of AI literacy through culturally responsive approaches. KI-
Künstliche Intelligenz, 35(2), 153–161.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of
Advanced Nursing, 62(1), 107–115.
Ezugwu, A. E., Oyelade, O. N., Ikotun, A. M., Agushaka, J. O., & Ho, Y. S. (2023).
Machine learning research trends in Africa: A 30 Years overview with bibliometric
analysis review. Archives of Computational Methods in Engineering, 1–31.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of
the concept, state of the evidence. Review of Educational Research, 74(1), 59–109.
Gillies, R. (2014). Cooperative learning: Developments in research. International Journal
of Educational Psychology, 3(2), 125–140.
Gomes, S., Costa, L., Martinho, C., Dias, J., Xexéo, G., & Moura Santos, A. (2023).
Modeling students’ behavioral engagement through different in-class behavior
styles. International Journal of STEM Education, 10(1), 1–15.
Greene, B. A. (2015). Measuring cognitive engagement with self-report scales:
Reflections from over 20 years of research. Educational Psychologist, 50(1), 14–30.
Gresse von Wangenheim, C., Hauck, J. C., Pacheco, F. S., & Bertonceli Bueno, M. F.
(2021). Visual tools for teaching machine learning in K-12: A ten-year systematic
mapping. Education and Information Technologies, 26(5), 5733–5778.
Gwagwa, A., Kachidza, P., Siminyu, K., & Smith, M. (2021). Responsible artificial
intelligence in sub-saharan Africa: Landscape and general state of play. AI4D Africa.
Hermans, F., & Aivaloglou, E. (2017). To scratch or not to scratch? A controlled
experiment comparing plugged first and unplugged first programming lessons. In
Proceedings of the 12th workshop on primary and secondary computing education (pp.
49–56).
Jordan, B., Devasia, N., Hong, J., Williams, R., & Breazeal, C. (2021). PoseBlocks: A
toolkit for creating (and dancing) with AI. In Proceedings of the 11th symposium on
education advances in artificial intelligence (EAAI ‘21).
Kim, K., Kwon, K., Ottenbreit-Leftwich, A., Bae, H., & Glazewski, K. (2023). Exploring
middle school students’ common naive conceptions of Artificial Intelligence
concepts, and the evolution of these ideas. Education and Information Technologies,
1–28.
King, K. (2020). Interventions to enhance behavioral engagement. Student Engagement:
Effective Academic, Behavioral, Cognitive, and Affective Interventions at School,
133–156.
I.T. Sanusi et al.
Kong, S. C., Cheung, W. M. Y., & Tsang, O. (2022). Evaluating an artificial intelligence
literacy programme for empowering and developing concepts, literacy and ethical
awareness in senior secondary students. Education and Information Technologies,
1–22.
Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2023). Evaluating an artificial intelligence
literacy programme for developing university students’ conceptual understanding,
literacy, empowerment, and ethical awareness. Educational Technology & Society, 26
(1), 16–30.
Kong, S.-C., & Zhang, G. (2021). A conceptual framework for designing artificial
intelligence literacy programmes for educated citizens. In S.-C. Kong, Q. Wang,
R. Huang, Y. Li, & T.-C. Hsu (Eds.), Conference proceedings (English paper) of the 25th
global Chinese conference on computers in education (GCCCE 2021) (pp. 11–15). The
Education University of Hong Kong.
Lee, I., Ali, S., Zhang, H., DiPaola, D., & Breazeal, C. (2021). Developing middle school
students’ AI literacy. In Proceedings of the 52nd ACM technical symposium on computer
science education (pp. 191–197).
Lee, I., & Martin, F. (2018). Investigating bias in machine learning applications. In CSTA
conference.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. sage.
Linnenbrink-Garcia, L., & Pekrun, R. (2011). Students’ emotions and academic
engagement: Introduction to the special issue. Contemporary Educational Psychology,
36, 1–3.
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design
considerations. In Proceedings of the 2020 CHI conference on human factors in
computing systems (1-16).
Mahipal, V., Ghosh, S., Sanusi, I. T., Ma, R., Gonzales, J. E., & Martin, F. G. (2023).
DoodleIt: A novel tool and approach for teaching how CNNs perform image
recognition. In Proceedings of the 25th australasian computing education conference (pp.
31–38).
Ma, R., Sanusi, I. T., Mahipal, V., Gonzales, J. E., & Martin, F. G. (2023). March).
Developing machine learning algorithm literacy with novel plugged and unplugged
approaches. In Proceedings of the 54th ACM technical symposium on computer science
education (Vol. 1, pp. 298–304).
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research &
Development, 50, 43–59.
Miao, F., & Holmes, W. (2022). International forum on AI and education: Ensuring AI as a
common good to transform education, 7-8 december; synthesis report.
Mosley, P., Ardito, G., & Scollins, L. (2016). Robotic cooperative learning promotes
student STEM interest. American Journal of Engineering Education, 7(2), 117–128.
National Health Research Ethics Committee of Nigeria, NHREC. (2016). Policy statement
regarding enrollment of children in research in Nigeria. http://nhrec.net/nhrec/Final%
20NHREC%20Policy%20Statement%20on%20Enrollment%20of%20Children%20in
%20Research.pdf. (Accessed 13 January 2023).
Nwite, S. (2019). Data science Nigeria unveils AI book for primary and secondary schools.
https://www.tekedia.com/data-science-nigeria-unveils-ai-book-for-primary-and-se
condary-schools/. (Accessed 31 January 2023).
Ottenbreit-Leftwich, A., Glazewski, K., Jeon, M., Jantaraweragul, K., Hmelo-Silver, C. E.,
Scribner, A., … Lester, J. (2022). Lessons learned for AI education with elementary
students and teachers. International Journal of Artificial Intelligence in Education, 1–23.
Oyelere, S. S., Sanusi, I. T., Agbo, F. J., Oyelere, A. S., Omidiora, J. O., Adewumi, A. E., &
Ogbebor, C. (2022). Artificial intelligence in african schools: Towards a
contextualized approach. In 2022 IEEE global engineering education conference
(EDUCON) (pp. 1577–1582). IEEE.
Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student
engagement. In S. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of research on
student engagement (pp. 259–282). New York, NY: Springer.
Renninger, K. A., & Bachrach, J. E. (2015). Studying triggers for interestand engagement
using observational methods. Educational Psychologist, 50(1), 58–69.
Rodríguez-García, J. D., Moreno-León, J., Román-González, M., & Robles, G. (2021).
Evaluation of an online intervention to teach artificial intelligence with learningml
to 10-16-year-old students. In Proceedings of the 52nd ACM technical symposium on
computer science education (pp. 177–183).
Russell, I., Coleman, S., & Markov, Z. (2012). A contextualized project-based approach
for improving student engagement and learning in AI courses. In Proceedings of
second computer science education research conference (pp. 9–15).
Sabuncuoglu, A. (2020). Designing one year curriculum to teach artificial intelligence for
middle school. In Proceedings of the 2020 ACM conference on innovation and technology
in computer science education (pp. 96–102).
Sáez-López, J. M., Román-González, M., & Vázquez-Cano, E. (2016). Visual programming
languages integrated across the curriculum in elementary school: A two year case
study using “scratch” in five schools. Computers & Education, 97, 129–141.
Sakulkueakulsuk, B., Witoon, S., Ngarmkajornwiwat, P., Pataranutaporn, P.,
Surareungchai, W., Pataranutaporn, P., & Subsoontorn, P. (2018). Kids making AI:
11
Computers and Education: Artificial Intelligence 5 (2023) 100155
Integrating machine learning, gamification, and social context in STEM education. In
2018 IEEE international conference on teaching, assessment, and learning for engineering
(TALE) (pp. 1005–1010). IEEE.
Sanusi, I. T. (2021). Intercontinental evidence on learners’ differentials in sense-making
of machine learning in schools. In Proceedings of the 21st koli calling international
conference on computing education research (pp. 1–2).
Sanusi, I. T. (2021b). Teaching machine learning in K-12 education. In Proceedings of the
17th ACM conference on international computing education research (pp. 395–397).
Sanusi, I. T., Jormanainen, I., Oyelere, S. S., Mahipal, V., & Martin, F. (2022e). Promoting
machine learning concept to young learners in a national science fair. In Proceedings
of the 22nd koli calling international conference on computing education research (pp.
1–2).
Sanusi, I. T., & Olaleye, S. A. (2022). An insight into cultural competence and ethics in K-
12 artificial intelligence education. In 2022 IEEE global engineering education
conference (EDUCON) (pp. 790–794). IEEE.
Sanusi, I. T., Olaleye, S. A., Agbo, F. J., & Chiu, T. K. (2022d). The role of learners’
competencies in artificial intelligence education. Computers in Education: Artificial
Intelligence, 3, Article 100098.
Sanusi, I. T., Olaleye, S. A., Oyelere, S. S., & Dixon, R. A. (2022c). Investigating learners’
competencies for artificial intelligence education in an African K-12 setting.
Computers and Education Open, 3, Article 100083.
Sanusi, I. T., Oyelere, S. S., & Omidiora, J. O. (2022b). Exploring teachers’
preconceptions of teaching machine learning in high school: A preliminary insight
from Africa. Computers and Education Open. https://doi.org/10.1016/j.
caeo.2021.100072
Sanusi, I. T., Oyelere, S. S., Vartiainen, H., Suhonen, J., & Tukiainen, M. (2022).
A systematic review of teaching and learning machine learning in K-12 education.
Education and Information Technologies, 1–31.
Sanusi, I. T., Sunday, K., Oyelere, S. S., Suhonen, J., Vartiainen, H., & Tukiainen, M.
(2023). Learning machine learning with young children: Exploring informal settings
in an African context. Computer Science Education. https://doi.org/10.1080/
08993408.2023.2175559
Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research
projects. Education for Information, 22(2), 63–75.
Sigayret, K., Tricot, A., & Blanc, N. (2022). Unplugged or plugged-in programming
learning: A comparative experimental study. Computers & Education, 184, Article
104505.
Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and
measuring student engagement in science. Educational Psychologist, 50(1), 1–13.
Skirpan, M., Beard, N., Bhaduri, S., Fiesler, C., & Yeh, T. (2018). Ethics education in
context: A case study of novel ethics activities for the CS classroom. In Proceedings of
the 49th ACM technical symposium on computer science education (pp. 940–945).
Tang, J., Zhou, X., Wan, X., Daley, M., & Bai, Z. (2022). ML4STEM professional
development program: Enriching K-12 STEM teaching with machine learning.
International Journal of Artificial Intelligence in Education, 1–40.
Tarantino, K., McDonough, J., & Hua, M. (2013). Effects of student engagement with
social media on student learning: A review of literature. The Journal of Technology in
Student Affairs, 1(8), 1–8.
Touretzky, D. S., & Gardner-McCune, C. (2022). Artificial intelligence thinking in k-12.
Computational Thinking Education in K-12: Artificial Intelligence Literacy and Physical
Computing, 153–180.
Touretzky, D., Gardner-McCune, C., Cynthia Breazeal, F. M., & Deborah, S. (2019).
A year in K-12 AI education. AI Magazine, 40(4), 88–90, 2019.
Twining, P., Heller, R. S., Nussbaum, M., & Tsai, C. C. (2017). Some guidance on
conducting and reporting qualitative studies. Computers & Education, 106, A1–A9.
Vartiainen, H., Toivonen, T., Jormanainen, I., Kahila, J., Tedre, M., & Valtonen, T.
(2021). Machine learning for middle schoolers: Learning through data-driven design.
International Journal of Child-Computer Interaction, 29, Article 100281.
Williams, R., Ali, S., Devasia, N., DiPaola, D., Hong, J., Kaputsos, S. P., & Breazeal, C.
(2022). AI+ ethics curricula for middle school youth: Lessons learned from three
project-based curricula. International Journal of Artificial Intelligence in Education,
1–59.
Williams, R., Kaputsos, S., & Breazeal, C. (2021). Teacher perspectives on how to train
your Robot: A middle school AI and ethics curriculum. In Proceedings of the 11th
symposium on education advances in artificial intelligence (EAAI ‘21).
Wohl, B., Porter, B., & Clinch, S. (2015). Teaching computer science to 5-7 year-olds: An
initial study with scratch, cubelets and unplugged computing. In Proceedings of the
workshop in primary and secondary computing education (pp. 55–60).
Xia, Q., Chiu, T. K., Lee, M., Sanusi, I. T., Dai, Y., & Chai, C. S. (2022). A self-
determination theory (SDT) design approach for inclusive and diverse artificial
intelligence (AI) education. Computers & Education, Article 104582.