Ninja Code Village for Scratch: Function

Samples/Function Analyser and Automatic

Assessment of Computational Thinking Concepts
Go Ota Yosuke Morimoto Hiroshi Kato
Master's Program, The Open University of Japan The Open University of Japan
The Open University of Japan morimoto@ouj.ac.jp hkato@ouj.ac.jp
gohome@v006.vaio.ne.jp

Abstract
Ninja Code Village is a comprehensive learning-

support environment for the Scratch, visual programming II. FUNCTION SAMPLES AND ANALYSER
language. It provides more than 60 sample functions that are Many national curricula for computer sciencesuch as
commonly used in Scratch projects, and analyses automatically
Computing (UK) and
Digital Technologies (Australia)
which functions are used in a project in order to foster students focus on CT as one of their goals. The key CT ideas of
competencies in abstraction, modelling, and decomposition. It
abstraction, modelling, and decomposition are important
also provides automatic assessment of computational thinking
concepts such as conditional statements, loops, data, and
competencies for transforming real problems and/or
parallelism in order to develop students programming skills. requirements into computer programs. The primary aim of
NCV is to foster these competencies in children. As such, it
Keywords
Programming learning, Automatic assessment.. comes with sample programs and a function analyser to make
students aware that a program can consist of many functions
assembled into a large and complex program.
I. INTRODUCTION
Children all over the world enjoy learning visual We reviewed many Scratch projects that were developed by
programming on platforms such as Scratch, Minecraft, and The children, as well as the Starter Projects of Scratch and other
Foos. The Japanese government has announced that its primary tutorial websites that are available. We then extracted more
schools will start teaching programming in 2020, while also than 60 functions that are commonly used by children and
warning that appropriate materials have not yet been prepared prepared small sample programs as learning material in NCV.
and the capacity of teachers for programming education is still In the case of Scratch, children can see samples and sequences
insufficient. In order to prepare for this coming situation in of blocks if required. In addition, a function analyser is a
Japan, we are developing Ninja Code Village (NCV), a powerful tool for detecting and showing which functions are
comprehensive learning-support environment for used in a program. As shown in Table I, each sample has a
programming. Although there are many different methods to
learn programming, we chose a
making strategy based on
Paperts concept of constructionism [1]. Generally speaking,

making is better at fostering the creativity of children than the

small step strategy of solving programming quizzes step-by-
step. However, some children may remain at an elementary
programming level, such as only colouring, and not learn any
higher skills. If formal education providers such as primary
schools want to guarantee the quality of their programming
education for
making, teachers and/or tutors must check the Diagnostic results by NCV
activities and outputs of students carefully and give them
appropriate feedback. Inevitably, this requires considerable
human and time resources.
In this showcase, we demonstrate how NCV provides a
rich programming learning environment. Figure 1 shows how Click here.
NCV works in terms of typical block scripts for a target- Show detailed information of
sample programs detected as
shooting game. The output refers to level-1 conditionals and functions in the project.
level-1, 2, and 4 loops as defined by computational thinking
(CT) concepts (see Table II), including
Bounce on edge,

Score, and
Dice functions.
Copyright is held by the owner/author(s).
978-1-5090-0252-8/16/$31.00 ©2016 IEEE Fig. 1. Example: Scratch Scripts and Diagnostic Results by NCV

238
 TABLE I. EXAMPLES OF FUNCTION SAMPLES. TABLE II. FRAMEWORK OF CT CONCEPTS OF NCV
Sample Function Patterns of Scratch blocks 1 2 3 4
Title for detect Conditional If Ifelse Logic Nested
Dice Use random value "['randomFrom:to:'" statements operations conditional
statements
Bounce on Bounce on edge "['doUntil' | 'doForever'| Loops Repeat forever Repeat Repeat until Nested loops
edge 'doRepeat' ['bounceOffEdge'" (count)
Score Count up score "['doIf'|' doIfElse', Procedure More than one Definition Blocks with Use of clones
[' ['changeVar:by:',  , 1]" script of blocks passing
Maze Move to right or left by "['whenKeyPressed', 'left parameters
arrow keys arrow'] ['changeXposBy:', -" Common One block uses Scripts Sprites use Recursive use
and "['whenKeyPressed', 'right procedure more than one share same of blocks
arrow']  ['changeXposBy:', " block blocks blocks
Ping Pong After touching the "['doIf'|' doIfElse' Data Use of Sprites Use of lists Use of cloud
colour, bounce ['touchingColor:'  ['heading: variables share variable
 ]" variables
Events Key pressed Mouse Background Time, video
specific function and a pattern of Scratch blocks that we clicked changed and audio
Parallelism Synchronize Message Message Message and
defined. NCV detects the patterns within a Scratch project and by background in a sprite between wait between
shows the corresponding sample titles on a page of diagnostic sprites sprites
results (the lower-right of Fig. 1). These results could motivate User Input by Detect Detect Use mouse
the children to add more functions to their programs. They Interface keyboard key mouse position
could also help them to understand the functions of programs pressed clicked
that they like by analysing other projects. Moreover, even if
some teachers are programming novices, they could still example, the relevant criteria (Table II) are not listed on the
analyse the programs that their students develop and give NCV website because we wish to encourage children to think
appropriate feedback to help improve them, e.g.
If you want to critically about the meaning of each CT concept and not just
add a score to your game program, see this sample. pick up so-called easy marks. After this evaluation phase, we
would show the criteria to students, if they need.
III. AUTOMATIC ASSESSMENT OF CT CONCEPTS A long-term goal is for NCV to become a Learning
Programs consist not only of functions, but also the general Analytics and Knowledge (LAK) system for programming
concepts of algorithms and data, e.g. sequences, conditional education to give both students and teachers appropriate
statements, and loops. From the viewpoint of assessment in feedback automatically by analysing learning records. We hope
programming education, Brennan and Resnick defined a clear that NCV will eventually solve the problem of insufficient
framework for the study and assessment of CT in Scratch; it programming-education capacity among teachers.
has three key CT facets: concepts, practice, and perspective [2].
Other researchers have also defined frameworks and verified V. RELEVANCE TO VL/HCC
their validity, some of which were developed for automatic
Although the
small step strategy gives clear and
assessment tools for Scratch. Moreno-Leon developed the
Dr.
consistent instructions to both teachers and students, NCV
Scratch web tool to assess CT concepts and programming
provides a more attractive and creative programming-learning
behaviour, and showed its effectiveness by practical research
environment based on the
making strategy by means of its
[3].
novel function analyser and precise assessment criteria.
Because an important issue for automatic assessment is
how to design appropriate criteria, we reviewed some national ACKNOWLEDGMENT
curricula and found that the Computing Progression Pathways
of Computing (UK) defined detailed developmental objectives This work was supported by JSPS KAKENHI Grant No.
related to programming [4]. We then extracted these objectives 26282058.
and organized them in terms of BrennanResnick CT concepts
[2]. Finally, we transferred these objectives into the context of REFERENCES
Scratch and defined our own framework (shown in Table II). [1] S. Papert, Constructionism: A new opportunity for elementary science
NCV performs automatic assessment by detecting the Scratch education MIT, Media Laboratory, Epistemology and Learning Group:
blocks that correspond to this framework (the lower-left- of Fig. National Science Foundation, 1986.
1). By using this function, students are motivated to go on and [2] K. Brennan and M. Resnick, New frameworks for studying and
use additional CT concepts. It is also easier for teachers to assessing the development of computational thinking, Annual Meeting
of the American Educational Research Association, 2012.
assess their students level of progress towards their
[3] J. Moreno-León, G. Robles, and M. Román-González, Dr. Scratch:
developmental objectives. automatic analysis of scratch projects to assess and foster computational
thinking, M RED - Revista de Educación a Distancia, 2015.
IV. FUTURE WORK [4] Computing At School (CAS), Computing Progression Pathways,
http://community.computingatschool.org.uk/resources/1692 (accessed
We have recently released the first version of NCV in 2016.7.4).
Japanese and have begun to evaluate it in practice. For

239