You are on page 1of 8

Computers in Human Behavior 77 (2017) 11e18

Contents lists available at ScienceDirect

Computers in Human Behavior


journal homepage: www.elsevier.com/locate/comphumbeh

Full length article

The effects of teaching programming with scratch on pre-service


information technology teachers' motivation and achievement*
Osman Erol a, *, Adile Aşkım Kurt b
a
Mehmet Akif Ersoy University, Turkey
b
Anadolu University, Turkey

a r t i c l e i n f o a b s t r a c t

Article history: The aim of this study is to examine the effect of programming instruction with Scratch on student
Received 6 November 2016 motivation and their programming achievements. The study group consisted of 52 sophomore students
Received in revised form attending the Department of Computer Education and Instructional Technologies of Mehmet Akif Ersoy
6 August 2017
University's Faculty of Education, Turkey. Participants were randomly divided into two groups in order to
Accepted 10 August 2017
Available online 11 August 2017
have 26 students in both the test group and the control group. During the first seven weeks of the study,
it is aimed that the students will understand programming logic and learn basic programming structures.
For this purpose, participants in the test group were instructed using Scratch, whilst in the control group,
Keywords:
Programming
flowcharting and problem-solving activities were conducted as per the curriculum. During the second
Scratch seven weeks of the study, C# programming language instruction was conducted using the same method
Flowcharts for both the test and control groups. Achievement Test and Motivated Strategies for Learning Ques-
Motivation tionnaire were utilized as data collection tools in the study, and a 3  2 (measurement time x groups)
factorial design was employed. Study findings revealed that programming achievement scores for both
the test and control groups increased at the end of the whole process; however, the increase was
significantly different in favor of the test group at the end of the whole process. It was observed that
motivation scores decreased in the control group, while the test group's scores increased.
© 2017 Published by Elsevier Ltd.

1. Introduction (Kinnunen & Malmi, 2008; Kurland, Pea, Clement, & Mawby, 1986).
Problems experienced in algorithm and programming instruction
Especially in basic courses for beginners in programming, the could lower student motivation in programming courses and the
objective is to teach the students both the basic concepts of pro- resulting negative attitudes could negatively affect student
gramming and programming logic in order to prevent potential achievement related to programming (Anastasiadou & Karakos,
problems experienced in advanced programming courses, where 2011; Lahtinen, Ala-Mutka, & Ja €rvinen, 2005). However, there are
coding instruction is delivered. In this process generally daily lan- many tools that can be employed as a means for solving this
guage and flowcharts, which are designated as so-called codes, are problem through the teaching of algorithm and basic programming
the tools utilized to teach programming logic by developing algo- such as Raptor, Flint, Code.org, Scratch, and Alice. General charac-
rithms. The fact that algorithm instruction is such an abstract teristics of these software are to materialize the algorithm process,
process causes beginner programmers to only comprehend pro- provide rapid testing and editing possibilities to the user (Baldwin
gramming logic at a low level and then fail to transfer this & Kuljis, 2000; Cooper et al., 2006; Maloney, Resnick, Rusk,
knowledge to other programming languages in the future Silverman, & Eastmond, 2010). Some of these, such as Raptor,
Flint, and FCPro, are just simple flowchart creation software and
lack design capabilities (Cooper, Dann, & Pausch, 2003). However,
*
This study is the summary of the doctoral dissertation titled “The Effects of on the other hand, tools that are capable of designing games and
Teaching Programming with Scratch on Pre-Service Information Technology stories such as Alice or Scratch, make it possible for students to
Teachers' Motivation and Achievement” written by Osman Erol and supervised by
design substantial and unique content; rendering the process more
Adile Aşkım Kurt.
* Corresponding author. attractive. One of the most frequently used tools lately in visual
E-mail addresses: oerol@mehmetakif.edu.tr (O. Erol), aakurt@anadolu.edu.tr programming of game design is Scratch (Resnick et al., 2009).
(A.A. Kurt).

http://dx.doi.org/10.1016/j.chb.2017.08.017
0747-5632/© 2017 Published by Elsevier Ltd.
12 O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18

Scratch has been used worldwide to teach algorithm and pro- the motivation and programming achievements of pre-service in-
gramming logic to students of different ages, from primary school formation technologies teachers. Based on this objective, the
to college (Maloney et al., 2010), due to its simple interface and drag following research question has been investigated:
and drop structure. Scratch is an effective tool that teaches pro-
gramming logic and is generally preferred by beginner pro-  Is there a significant difference between the group that learned
grammers rather than learning a mere programming language. In with Scratch and the group that learned with flowchart in
addition, Scratch enables the design of games, stories and anima- participant motivation and their academic achievements?
tions thanks to its design-oriented structure, which in turn makes
learning more fun. As these tools employ game design in pro-
gramming teaching, the game design process could also help 2. Methodology
learners to comprehend programming logic. Digital game design, in
programming instruction is based on Papert (1991) constructionist The current study utilized the pretest-posttest experimental
approach. According to Papert (1980), the most effective learning model with a control group. The dependent variables of the study
occurs when individuals are allowed to design and create were achievement in programming and motivation, while the in-
(construct) a product instead of instructing the knowledge directly. dependent variable was instructional method (Scratch/Flowchart).
Learning takes place through experience while constructing these The research design is presented in Table 1.
products (Harel & Papert, 1991; Kafai, 2006; Mishra & Girod, 2006).
This practice increases the learner's motivation and enables better 2.1. Study group
learning; thus it could be possible for beginners in programming to
gain increased motivation and a positive attitude towards pro- Participants of the study consisted of 52 sophomore students
gramming. Furthermore, using Scratch, student motivation for taking Programming 1 course at the Department of Computer Ed-
programming courses could be improved, which would in turn ucation and Instructional Technologies of Mehmet Akif Ersoy Uni-
increase their achievement (Malan & Leitner, 2007). Also, learners versity during the fall semester of the 2014e2015 academic year.
who understood programming logic could transfer the acquired Participants were divided into test and control groups, each con-
knowledge to the learning of other programming languages (Wolz, sisting of 26 students. Convenience sampling method was applied
Leitner, Malan, & Maloney, 2009). Regarding these, it could be and participants were randomly assigned to the two groups. Equal
argued that the effect of tools and game design processes such as probability assignment of the participants to the test and control
Scratch on the instruction of programming logic and programming groups was obtained through random selection. Participant de-
languages should be scrutinized in connection with learner mographic characteristics (gender and type of high school gradu-
motivation. ated) for both groups are presented in Table 2.
A review of the existing literature revealed numerous studies Overall, 48.07% of the participants were female and 51.93% were
conducted using Scratch in programming instruction. One such male. Analysis of the type of high school that participants gradu-
study was conducted by Rizvi, Humphries, Major, Jones, and Lauzun ated from demonstrated that 73.07% of participants (i.e. from both
(2011) with college students. In their study, the effect of game groups) graduated from vocational or technical high schools, while
design activities performed in basic computer lessons on students' 26.93% graduated from general high schools. In the vocational and
programming achievement in advanced programming lessons was technical high schools, there are programming courses in the
examined. At the end of the two-year long study, it was observed computer science department, but not in the general high school.
that Scratch increased students' attitudes towards programming Accordingly, having experience with programming could be an
and that their achievements were better than those in the group advantage for graduates of vocational and technical high schools
that did not receive Scratch training. The authors stressed that participants. In addition, gender was an influential variable on
game design activities with Scratch developed algorithmic- attitude and success in programming, especially girls were more
thinking skills of the students and that these skills were then unsuccessful in programming (Goode, Estrella, & Margolis, 2006;
transferred to a programming language. In another study, Westcott Kelleher, Pausch, & Kiesler, 2007). Therefore, while distributing
(2008) analyzed the effect of programming instruction using the participants into groups, attention is paid to balance the gender
Scratch on student achievements in Cþþ programming. In the test and the type of high school graduated. After the placement, par-
group, Cþþ code instruction was integrated within game design ticipants were informed about which group (Scratch or Flowchart)
activities using Scratch. Study findings demonstrated no significant they were assigned to. However, participants did not know
difference between the test scores that included basic structures whether their group was the experimental or the control group. In
such as variable definitions, but there was a significant difference addition, similar activities were applied by different methods in
favoring the test group that included more complex and difficult each group before the groups were merged for their subsequent
programming structures such as decision making and loops. In a programming language training. The same instructor was
study conducted by Leiva and Salas (2013) with college students, employed throughout the whole process.
the effect of game design activities using Scratch on students'
programming achievements were indicated. At the end of the 2.2. Data collection tools
study, findings showed that game design activities conducted with
the test group increased student course satisfaction and that their The first tool the study utilized the Motivated Strategies for
academic success was more comparable to that of the control Learning Questionnaire (MSLQ), as developed by Pintrich, Smith,
group. However, in several studies it was determined that game Garcia, and McKeachie (1993) and adapted to Turkish by
design activities conducted with visual tools such as Scratch or Büyüko €ztürk, Akgün, Kahveci, and Demirel (2004). The question-
Alice increased the programming achievements of the students, naire includes two subscales; motivation and learning strategies.
their attitude, and their motivation for programming (Bishop-Clark, There are 31 items and six sub-dimensions in the motivation sub-
Courte, & Howard, 2007; Cooper et al., 2003; Howland & Good, scale, while there are 50 items and nine sub-dimensions in the
2015; Malan & Leitner, 2007; Nikou & Economides, 2014; learning strategies subscale. These subscales can be applied sepa-
Çagıltay, 2007). Hence, the main objective of the current study is rately, and accordingly, for this current study only the motivation
to analyze the effect of programming instruction with Scratch on subscale was employed. In the 7-point, Likert-type scale, each item
O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18 13

Table 1
Research design.

Groups Selection Pretest Process Posttest Process Posttest 2

Test R ATPre MSLQPre PIS ATPost MSLQPost CPLI ATPost2 MSLQPost2


Control R ATPre MSLQPre PIF ATPost MSLQPost CPLI ATPost2 MSLQPost2

R Random selection.
PIS Programming Instruction with SCRATCH.
PIF Programming Instruction with Flowcharts.
CPLI C# Programming Language Instruction.
ATPre Achievement Test e Pretest.
MSLQPre: Motivated Strategies for Learning Questionnaire e Pretest.
ATPost Achievement Test e Posttest.
MSLQPost Motivated Strategies for Learning Questionnaire e Posttest.
ATPost2 Achievement Test e Posttest 2.
MSLQPost2 Motivated Strategies for Learning Questionnaire e Posttest 2.

Table 2
Participant demographic characteristics.

Demographic Group (n ¼ 52) Test Group (n ¼ 26) Control Group (n ¼ 26)

Gender Female f 14 11
% 53.84 42.31
Male f 12 15
% 46.16 57.69
Graduated High School Type Vocational/Technical High School f 18 20
% 69.23 76.92
General High School f 8 6
% 30.77 23.08

is scored using values ranging between “1 e Absolutely not true for Mehmet Akif University, with a two-week duration between ap-
me” to “7 e Absolutely true for me”. In the Turkish adaptation, plications. The correlation between the two measurements was
internal consistency coefficients ranged between 0.52 and 0.86. calculated with Pearson Product-Moment Correlation Coefficient as
The other data collection tool was an achievement test devel- 0.723. The multiple-choice test included 11 algorithm problems
oped by the authors to measure the academic achievements of the and nine C# problems. Algorithm problems had been used to test
students on programming. This achievement test included two programming logic. There were question simple programming
sections with 20 multiple-choice questions on basic programming problems that did not involve coding and could be solved step-by-
structures (variable, decision making and control, loop, string) and step on paper. Many similar level algorithm problems had been
four open-ended questions related to programming problems. In used both during Scratch training and flowchart training. Learning
order to design the part of the achievement test that included outcomes related to the questions in the multiple-choice test are
multiple-choice questions, a pilot test containing 25 items was presented in Table 3.
formed with expert opinion. These experts were instructors who To design the second section of the achievement test that con-
teach programming at Mehmet Akif Ersoy University. As a reli- tained the open-ended programming problems, an item pool was
ability study, the pilot test was applied to 114 students of Pro- created which included analysis and synthesis level questions. Four
gramming Languages I course in the Computer Education and problems were selected based on the feedback obtained from ex-
Instructional Technologies Department at Mehmet Akif University perts and were reorganized to fit the requirement. Interrater reli-
Faculty of Education. Participants in the pilot study were educated ability was tested using Kendall W fitness test to establish scoring
at the third and fourth grades in the same department and had reliability of open-ended programming problems (W ¼ 0.901,
already taken programming courses before. These students were p < .01).
not involved in the current study.
Item analysis was conducted after the pilot application and 2.3. Procedure
discrimination (r) and difficulty (p) indices for each item were
calculated. Due to the fact that discrimination index scores of five The study conducted within a Programming Languages I class
items were below 0.20, these items were excluded from the main lasted for a total of 14 weeks, including the collection process of the
test. Six items with scores ranging between 0.20 and 0.30 in the tests. The objective was to teach basic programming structures
item discrimination index were then reorganized through expert during the initial seven weeks of the study. In the control group,
opinion. After the items with low discrimination index values were basic programming structures instruction was conducted as plan-
excluded from the test, the mean discrimination index was calcu- ned in the current curriculum using flowcharts. Thus, the basic
lated as r ¼ 0.44. Difficulty index values for test items demonstrated structures were explained through narration and presentation
that they varied between 0.23 and 0.79, resulting in a mean diffi- methods and programming problems were solved using flow-
culty index calculation of p ¼ 0.51. Thus, it could be argued that the charts. In the test group, this process was utilized by conducting
test was of intermediate difficulty. Furthermore, KR-20 internal with Scratch software. Participants used the characters and
consistency coefficient was calculated to measure the reliability of graphics in Scratch to solve these problems in a game scenario. For
the test as 0.712. Test-retest method was used to determine the example, a problem on collecting three numbers entered from the
consistency of the achievement test. The developed achievement keyboard, control group solved the problem by using flow dia-
test was applied twice to 61 junior students attending the Com- grams, whereas the test group solved by designing it as a Scratch
puter Education and Instructional Technologies Department of game (Fig. 1). Similar problems and examples were used for both
14 O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18

Table 3
Learning outcomes related to problems.

Learning Outcomes Algorithm Problems C# Problems

Variable (Assignment, Basic operations) Q3-Q6-Q14-Q19 Q1-Q11


Loop (For, while etc.) Q2-Q3-Q5-Q7-Q10-Q16 Q8-Q18-Q20
Decision making (If else etc.) Q12-Q6-Q17-Q19 Q9-Q11-Q15
Arrays Q5-Q7 Q4-Q8

Control Group Test Group

Fig. 1. The solution of same problem in both groups.

groups and in both groups they wrote problem-solving algorithms comparison is needed, Bonferroni correction was implemented and
(solution steps) step by step. Type-I error probability was minimized. Furthermore, to determine
In the second seven-week period of the study, C# programming the magnitude of the difference, h2 effect size value was examined.
language instruction was conducted in both the control and test The Eta square correlation coefficient is interpreted as 0.01, 0.06
groups. In both groups, instruction by presentation, demonstration, and 0.14, respectively, as small, medium and large effect sizes
exercise techniques, and similar problems and examples were (Cohen, 1988). In cases where there was no difference, statistical
employed. In other words, the same methodology was utilized in power was reported.
both groups during their C# programming language instruction. To determine whether or not there was a significant difference,
The content and activities related to the course for both groups are between the group that learned with Scratch and the group where
presented in Table 4. instruction was conducted based on the existing curriculum, in
increasing the participant motivation and academic achievements
2.4. Data analysis with learning programming languages, a 3  2 mixed design
MANOVA was employed. In this analysis, two different instruction
In the analysis of data, in cases where more than one methods (learning with Scratch, and problem solving with

Table 4
Course content of test and control groups.

Week Control Group Test Group

Weeks 1 & 2  Flowcharts & Symbols  Introduction to Scratch


 Variables  Variables
 Solving related problems with flowcharts  Solving related problems with Scratch
Week 3  Control & Decision Making  Control & Decision Making
 Solving related problems with flowcharts  Solving related problems with Scratch
Weeks 4 & 5  Loops  Loops
 Solving related problems with flowcharts  Solving related problems with Scratch
Weeks 6 & 7  Arrays  Arrays
 Solving related problems with flowcharts  Solving related problems with Scratch
Weeks 8-14 C#
 Introduction to C#
 Variables
 IfeElse/Switch Case Codes
 For/While/DoeWhile Codes
 Arrays
 Solving related problems in C# Editor
O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18 15

flowcharts) reflect the in-group factor, while measurement times of examine the interaction of time of measurement and group vari-
pretest, posttest and posttest2 applications of the measurement ables, motivation and programming achievement variables were
tools (achievement and motivation) were the recurring measure- individually addressed in a 3  2 mixed design ANOVA. Analysis of
ment factor. At the end of the analysis, simple main effect analysis variance results for the mean motivation score of the participants
was conducted in order to determine whether or not the differ- based on time of measurement and group variables are presented
ences observed between achievement and motivation scores based in Table 6.
on the group and measurement times were statistically significant. As can be observed in Table 6, there was a significant difference
Thus, while both achievement and motivation scores were between the participants' motivation scores based on the group
compared within themselves separately based on the time of variable (F (1,50) ¼ 6.320, p < .025, h2 ¼ 0.112). It could be argued
measurement, dependent samples t-test was conducted for each that the motivation scores of the participants differed based on the
time of measurement and after Bonferroni adjustment, significance group variable; in other words, the type of instruction method
level was divided by the number of tests. To compare the applied. Furthermore, the effect size value (h2 ¼ 0.112) demon-
achievement and motivation scores based on the groups, inde- strated a large effect (Cohen, 1988). However, there was no signif-
pendent samples t-test was conducted separately for each group icant difference found between the motivation scores based on the
and following Bonferroni adjustment significance level was divided time of measurement variable (F (1.750, 87.494) ¼ 160.920,
by the number of tests. p ¼ 0.437, power ¼ 0.174). There was a significant difference based
on the interaction effect of time of measurement and group vari-
3. Results ables between the motivation scores (F (1.750, 87.494) ¼ 1584.052,
p < .01, h2 ¼ 0.136). Thus, it could be stated that the control and test
MANOVA test results were used to determine whether or not group motivation scores differed based on the time of measure-
there was a significant difference between the group that learned ment. Analysis of variance results for programming achievements
with Scratch and the group where instruction was conducted based of the participants based on time of measurement and group var-
on problem solving with flowcharts in increasing the motivation iables are presented in Table 7.
and academic achievements of the participants on learning pro- According to Table 7, there was a significant difference between
gramming languages. These results are presented in Table 5. the participants' achievement scores based on the group variable (F
As can be seen in Table 5, there was a significant difference (1,50) ¼ 11.021, p < .025, h2 ¼ 0.181). Thus, it could be argued that
between the participants' motivation and achievement scores the achievement scores of the participants differed based on the
based on the group variable (Wilks' L ¼ 0.800, F (2,49) ¼ 6.106, group variable; in other words, the type of instruction method
p < .025). Accordingly, it could be stated that the type of instruction applied. Furthermore, the effect size value (h2 ¼ 0.181) demon-
method applied in both the test and control groups had an effect on strated a large effect (Cohen, 1988). Based on the time of mea-
motivation and achievement scores of the participants. Effect size surement, it was observed that there was a significant difference
value was calculated as .200. Thus, it is possible to talk about a large between the achievement scores (F (1.232, 61.624) ¼ 181.521,
effect (Cohen, 1988). It was observed that there was a significant p < .025, h2 ¼ 0.784). Accordingly, it could be argued that the
difference between the participants' motivation and achievement achievement scores of the participants differed based on pretest,
scores based on the time of measurement variable (Wilks' posttest and posttest2 measurements. A large effect could be
L ¼ 0.172, F (4,47) ¼ 56.434, p < .001). Accordingly, it could be determined from the effect size value (h2 ¼ 0.784) (Cohen, 1988).
stated that motivation and achievement scores of the participants There was a significant difference based on the interaction effect of
differed based on pretest, posttest and posttest2 measurements. time of measurement and group variables between the achieve-
Effect size value was calculated as .828. Thus, it is possible to talk ment scores (F (1.232, 61.624) ¼ 8.339, p < .025, h2 ¼ 0.143). Thus, it
about a large effect (Cohen, 1988). Another finding was that moti- could be stated that control and test group achievement scores
vation and achievement scores of the participants showed a sig- differed based on the time of measurement. To analyze the inter-
nificant difference based on the interaction effect of time of action effect in detail, test and control group achievement and
measurement and group variables (time of measurement * group) motivation scores based on the time of measurement are presented
(Wilks' L ¼ 0.712, F (4,47) ¼ 4.755, p < .025). Accordingly, it could in Fig. 2.
be stated that the motivation and achievement scores of the par- Conducted analyses of variance demonstrated that the time of
ticipants received in pretest, posttest and posttest2 differed based measurement and group variable interaction had a significant ef-
on the group variable; in other words, the instruction method used. fect on motivation and achievement. However, as can be observed
The finding that time of measurement and group (time of mea- from Fig. 2, this interaction had different effects on motivation and
surement * group) interaction created a difference in the motiva- achievement scores of the participants. While the motivation score
tion and achievement scores of the participants was important. continuously increased in the test group, it initially decreased in the
Thus, motivation and programming achievement scores were control group, and then became stable. On the other hand,
examined within the context of the interaction of time of mea- achievement score reflected a continuous increase in both the test
surement and group variables, instead of running a single analysis and control groups. Simple main effect analysis was conducted in
for each variable. This was important for a healthier interpretation order to determine whether or not these observed cases were
of the findings (Huck, 2000). Effect size value was calculated as statistically significant.
.288. Thus, it is possible to talk about a large effect (Cohen, 1988). To When achievement scores within the groups were compared

Table 5
Multivariate analysis of variance (MANOVA) results for motivation and programming achievement mean scores based on time of measurement and group variables.

Source of Variance Wilks' L F Hypothesis df Error df p h2


Group .800 6.106 2.000 49.000 .004 .200
Time of Measurement .172 56.434 4.000 47.000 <.001 .828
Time of Measurement* Group .712 4.755 4.000 47.000 .003 .288
16 O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18

Table 6
Analysis of variance (ANOVA) results for the mean motivation score of participants based on time of measurement and group variables.

Source of the variance Sum of Squares df Mean Square F p h2


Between-groups
Group 6346.314 1 6346.314 6.320 .015 .112
Error 50,204.526 50 1004.091
Within groups
Time of measurement 281.590 1.750 160.920 .802 .437 .016
Time of measurement *Group 2771.897 1.750 1584.052 7.897 .001 .136
Error 17,550.513 87.494 200.591
Total 77,154.84 141.994

Table 7
Analysis of variance (ANOVA) results for programming achievements of participants based on time of measurement and group variables.

Source of the variance Sum of Squares df Mean Square F p h2


Between-groups
Group 3689.130 1 3689.130 11.021 .002 .181
Error 16,737.162 50 334.743
Within groups
Time of measurement 46,290.716 1.232 37,558.903 181.521 .001 .784
Time of measurement *Group 2126.452 1.232 1725.340 8.339 .003 .143
Error 12,750.760 61.624 206.912
Total 81,594.22 115.088

difference between the groups continuously increased, favoring the


test group. As a result, it could be stated that the instruction
methods used in both the control and test groups increased
achievement scores. However, since the difference between the
groups favored the test group, the instructional method imple-
mented in the test group was more effective in increasing
achievement. Furthermore, the reason the increase in achievement
score increased within the measurement interval where a common
instructional method was utilized (posttest e posttest2 measure-
ment interval) could be due to the instructional methods used in
the control and test groups. Thus, based on the significant differ-
ence in scores obtained at the posttest2 measurement favoring the
test group, it could be argued that the method used in the test
Fig. 2. Group achievement & motivation scores based on time of measurement.
group (programming instruction with Scratch) affected the
achievement more in the measurement interval where a common
instructional method was used for programming (posttest e
posttest2 measurement interval).
based on time of measurement (Appendix A), in the control group,
When motivation scores within the groups were compared
there was a significant difference between pretest and posttest
based on time of measurement (Appendix C), in the control group,
favoring posttest (t (25) ¼ 7.427, p < .001); posttest and posttest2
there was no significant difference between pretest and posttest (t
favoring posttest2 (t (25) ¼ 7.079, p < .001), and pretest and
(25) ¼ 2.563, p ¼ 0.017); posttest and posttest2 (t (25) ¼ .331,
posttest2 favoring posttest2 (t (25) ¼ 8.634, p < .001). In the test
p ¼ 0.743), and pretest and posttest2 (t (25) ¼ 2.034, p ¼ 0.053). In
group, there was a significant difference between pretest and
the test group, there was also no significant difference between
posttest favoring posttest (t (25) ¼ 12.054, p < .0001), posttest
pretest and posttest (t (25) ¼ 1.380, p ¼ 0.018), posttest and
and posttest2 favoring posttest2 (t (25) ¼ 8.695, p < .0001), and
posttest2 (t (25) ¼ 2.074, p ¼ 0.049), and pretest and posttest2 (t
pretest and posttest2 favoring posttest2 (t (25) ¼ 11.756,
(25) ¼ 2.943, p ¼ 0.007). However, this could be due to type-II
p < .0001). Accordingly, it could be observed that the achievement
error that could occur as a result of Bonferroni adjustment. In
scores significantly increased from pretest towards posttest2
fact, it is possible that a significant difference was not observed due
continuously in both the test and the control groups. Achievement
to the narrowing of the level of significance, while the comparison
scores were compared based on the groups in order to determine in
could have revealed significant differences. Analysis of mean values
which group this increase was significantly higher (between-
demonstrate that there was a decrease of 6.08 points between the
groups comparison). Analysis of the differences between groups
pretest and posttest measurements, and there was an increase of
(Appendix B) showed that the groups had similar achievement
1.04 points between posttest and posttest2 measurements in the
scores in pretest (t (50) ¼ .589, p ¼ 0.558). In the posttest, the test
control group, while there was a decrease of 7.88 points between
group was 9.191 points significantly higher (t (50) ¼ 3.666,
pretest and posttest2 measurements. Thus, it could be stated that
p < .001) than the control group, and in posttest2, the test group
participants' motivation scores decreased at posttest2 compared to
was 19.025 points significantly higher (t (50) ¼ 3.153, p < .008)
the pretest scores in the control group and there was a decrease
than the control group. Thus, it could be deduced that the
from the first measurement to the third. On the other hand, in the
O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18 17

test group, it was observed that participants' motivation scores comprehensible and entertaining. Scratch helps to visualize the
increased from the first measurement (pretest) to the second algorithm process. The visual structure of Scratch could make
measurement (posttest), and from the second measurement programming structure more concrete and the interesting which is
(posttest) to the third measurement (posttest2). Furthermore, it generally considered to be difficult and boring, thereby increasing
was also observed that motivation scores increased 11.81 points learner motivation and enabling participants to learn programming
between the first (pretest) and third (posttest2) measurements. better (Gee, 2005; Moreno, 2012). Learners' managing all the
Thus, it could be stated that participants' motivation scores design processes (determining problem, design, test etc.) helps
increased at the end of the process in the test group. To see in which identifying this process a more constructive approach. On the
group these differences were significant, scores obtained in the contrary, it could be stated that problem-solving activities with
measurements were compared based on groups (between-group flowcharts found in the current curriculum decreased the motiva-
comparison). Analysis of the differences between groups (Appendix tion of learners for programming, and that this effect continued
D) showed that the groups had similar motivation scores in the during their C# programming language instruction. Thus, it is
pretest (t (50) ¼ .205, p ¼ 0.838), in the posttest, the test group possible that low motivation made it difficult for the participants to
was 16.65 points significantly higher (t (50) ¼ 2.651, p < .008) and acquire programming logic, which in turn affected their program-
in posttest2, the test group was 20.81 points significantly higher (t ming achievements to a lesser degree. Learners who are bored with
(50) ¼ 3.434, p < .001) than the control group. Thus, it could be the flowcharting course and have lowered motivation, thus expe-
deduced that the difference between the groups continuously and rience problems in comprehending programming logic. Accord-
significantly increased in favor of the test group. Furthermore, it ingly, it is seen that activities with Scratch increases the motivation
could be argued that the method used in the test group (pro- of students, whilst flow diagrams decrease motivation.
gramming instruction with Scratch) continuously increased moti- Based on study results, it could be argued that learners were
vation of the participants; whilst the method used in the control able to learn programming logic as a result of the experiences ob-
group (programming instruction with flowcharts) decreased tained during Scratch process, and transfer this experience to the
motivation of the participants, and the effect on motivation learning of a new programming language. On the contrary, it was
continued in the test group. However, motivation stagnated in the observed that the limitations of problem-solving activities using
control group in the interval where the same method was used in flowcharts, as found in the current curriculum, such as their level of
both groups (posttest e posttest2 measurement interval). difficulty and vapidity, insufficiency in materializing the process
and learners remaining passive in the process decreased the
4. Discussion & conclusion motivation of learners and this factor negatively affected the pro-
cess of learning programming logic. According to the results of the
Study findings revealed that participants' programming research, the following could be suggested.
achievements showed an increase in both the test and control
groups; however, this increase significantly differentiated in favor Activities could be conducted in the introduction to program-
of the test group. Furthermore, it was observed that the difference ming courses using visual tools such as Scratch or Alice instead
continued in the third measurement, the posttest2 results, also of flowcharts to ensure the achievement of students under-
favoring the test group. The fact that the difference favoring the test standing programming logic.
group continued after the posttest2 measurement, could be inter- Learner-oriented learning environments could be designed in
preted as the participants having transferred the skills they ob- both programming logic and programming language instruction
tained during the process of Scratch to the process of programming to increase the leaner achievements and motivation.
with C#. This finding is supported by the results of similar studies Further studies could be conducted that would utilize different
found in the literature (e.g., Leiva & Salas, 2013; Rizvi et al., 2011; visual tools other than Scratch activities and compare those to
Westcott, 2008; Çag ıltay, 2007). Another finding of the current flowcharts.
study was that motivation of the participants showed differences in Similar empirical studies could be conducted to research the
both posttest and posttest2 measurements, favoring the test group. effect of activities using Scratch on programming achievements
In other words, it was demonstrated that activities conducted with and motivation at different educational levels (high school,
Scratch implemented in the test group was effective in increasing middle school, primary school).
the motivation of the participants. Furthermore, it was observed
that the increase in motivation in the test group continued during
Appendix A. Achievement within-group comparison
the programming with C# instruction process. Another significant
finding was the fact that at the end of the process, motivation of the
participants in the test group increased, and decreased in the
control group. This finding was parallel to related studies found in
the literature (Nikou & Economides, 2014; Osman, Loke, Zakaria &
Downe., 2012). However, in this current study, contrary to the Groups Tests n Sd df t pa
X
findings of other studies, it was found that problem-solving activ-
ities using flowcharting, which is preferred by several educators Control Group Pretest 26 6.62 5.507 25 7.427 .000
Posttest 26 20.32 8.709
especially in programming logic instruction, decreased the moti- Posttest 26 20.32 8.709 25 7.079 .000
vation of the participants (Leiva & Salas, 2013; Rizvi et al., 2011). Posttest2 26 39.62 20.413
As a result of the analysis of all study data, it could be stated Pretest 26 6.62 5.507 25 8.634 .000
that; it might be possible that high motivation during the process Posttest2 26 39.62 20.413
Test Group Pretest 26 7.58 6.236 25 12.054 .000
where programming logic was taught could have helped the par-
Posttest 26 29.51 9.357
ticipants to comprehend the programming logic better and could Posttest 26 29.51 9.357 25 8.695 .000
have also increased the students' motivation and programming Posttest2 26 58.64 23.027
achievements during their programming language instruction Pretest 26 7.58 6.236 25 11.756 .000
process. The reason for this could be Scratch making the algorithm Posttest2 26 58.64 23.027
a
process where programming logic was instructed more Bonferroni adjustment.025/6 ¼ 0.004.
18 O. Erol, A.A. Kurt / Computers in Human Behavior 77 (2017) 11e18

Appendix B. Achievement between-groups comparison Büyüko €ztürk, Ş., Akgün, O. € E., Kahveci, O.,€ & Demirel, F. (2004). Güdülenme ve
€g
o renme stratejileri o €lçeginin Türkçe formunun geçerlik ve güvenirlik çalışması.
Kuram ve Uygulamada Eg itim Bilimleri, 4(2), 207e239.
ıltay, N. E. (2007). Teaching software engineering by means of computer-game
Çag
development: Challenges and opportunities. British Journal of Educational
Technology, 38(3), 405e415.
Tests Groups n X Sd df t Pa Cohen, J. W. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Hillsdale, NJ: Lawrence Erlbaum Associates.
Pretest Control 26 6.62 5.507 50 .589 .558 Cooper, S., Dann, W., & Pausch, R. (2003). Using animated 3D graphics to prepare
Test 26 7.58 6.236 novices for CSI. Computer Science Education, 13, 3e30.
Posttest Control 26 20.32 8.709 50 3.666 .001 Cooper, S., Powers, K., McNally, M., Goldman, K. J., Proulx, V., & Carlisle, M. (2006).
Test 26 29.51 9.357 Tools for teaching introductory programming: What works? ACM SIGCSE
Posttest2 Control 26 39.62 20.413 50 3.153 .003 Bulletin, 38, 560e561.
Test 26 58.64 23.027 Gee, J. P. (2005). Learning by design: Good video games as learning machines.
ELearning, 2, 5e16.
a
Bonferroni adjustment .025/3 ¼ 0.008. Goode, J., Estrella, R., & Margolis, J. (2006). Lost in translation: Gender and high
school computer science. In J. M. Cohoon, & W. Aspray (Eds.), Women in IT:
Reasons on the underrepresentation (pp. 89e114). Cambridge, MA: The MIT
Press.
Harel, I., & Papert, S. (1991). Software design as a learning environment. Interactive
Learning Environments, 1(1), 1e30.
Appendix C. Motivation within-group comparison
Howland, K., & Good, J. (2015). Learning to communicate computationally with flip:
A bi-modal programming language for game creation. Computers & Education,
80, 224e240.
Huck, S. W. (2000). Reading statistics and research (3rd ed.). New York: Addison
Wesley Longman.
Kafai, Y. B. (2006). Playing and making games for learning instructionist and
Groups Tests n X Sd df t pa constructionist perspectives for game studies. Games and Culture, 1(1), 36e40.
Kelleher, C., Pausch, R., & Kiesler, S. (2007). Storytelling Alice motivates middle
Control Group Pretest 26 151.92 22.509 25 2.563 .017 school girls to learn computer programming. In CHI 2007 proceedings of the
Posttest 26 143.00 22.293 SIGCHI conference on human factors in computing systems (pp. 1455e1464).
Posttest 26 143.00 22.293 25 .331 .743 Kinnunen, P., & Malmi, L. (2008). CS minors in a CS1 course. In Proceedings of the
Posttest2 26 144.04 23.787 fourth international workshop on computing education research, ICER (pp. 79e90)
Pretest 26 151.92 22.509 25 2.034 .053 (New York, USA).
Kurland, D. M., Pea, R. D., Clement, C., & Mawby, R. (1986). A study of the devel-
Posttest2 26 144.04 23.787
opment of programming ability and thinking skills in high school students. In
Test Group Pretest 26 153.04 16.153 25 1.380 .018
E. Soloway, & J. C. Spohrer (Eds.), Studying the novice programmer (pp. 83e112).
Posttest 26 159.35 22.165
Hillsdale. NJ: Lawrence Erlbaum.
Posttest 26 159.35 22.165 25 2.074 .049 Lahtinen, E., Ala-Mutka, K., & Ja €rvinen, H. (2005). A study of the difficulties of novice
Posttest2 26 164.85 19.713 programmers. ACM SIGCSE Bulletin, 37(3), 14e18.
Pretest 26 153.04 16.153 25 2.943 .007 Leiva, F. A. J., & Salas, C. A. C. (2013). Practices of advanced programming: Tradition
Posttest2 26 164.85 19.713 versus innovation. Computer Applications in Engineering Education, 21(2),
a 237e244.
Bonferroni correction .025/6 ¼ 0.004.
Malan, D. J., & Leitner, H. H. (2007). Scratch for budding computer scientists. SIGCSE
Bulletin, 39(1), 223e227.
Maloney, J., Resnick, M., Rusk, N., Silverman, B., & Eastmond, E. (2010). The scratch
Appendix D. Motivation between-groups comparison programming language and environment. ACM Transactions on Computing Ed-
ucation (TOCE), 10(4), 16.
Mishra, P., & Girod, M. (2006). Designing learning through learning to design. The
High School Journal, 90(1), 44e51.
Moreno, J. (2012). Digital competition game to improve programming skills.
Educational Technology & Society, 15(3), 288e297.
Tests Groups n X Sd df t Pa Nikou, S. A., & Economides, A. A. (2014). Transition in student motivation during a
scratch and an app inventor course. In Proceedings of global engineering edu-
Pretest Control 26 151.92 22.509 50 .205 .838 cation conference (EDUCON) 2014 (pp. 1042e1045). Istanbul, Turkey: IEEE.
Test 26 153.04 16.153 Osman, M. A., Loke, S. P., Zakaria, M. N., & Downe, A. G. (2012). Secondary students'
Posttest Control 26 143.00 22.293 50 2.651 .007 perfectionism and their response to different programming learning tools. In
Test 26 159.35 22.165 Proceedings of humanities, science and engineering (CHUSER) (pp. 584e588). IEEE
Posttest2 Control 26 144.04 23.787 50 3.434 .001 (Sabah, Malaysia).
Test 26 164.85 19.713 Papert, S. (1980). Mindstorms: Children, computers and powerful ideas. New York:
Basic Books.
a
Bonferroni adjustment .025/3 ¼ 0.008. Papert, S. (1991). Situating constructionism. In S. Papert, & I. Harel (Eds.), Con-
structionism (pp. 1e11). Norwood, NJ: Ablex Publishing Corporation.
Pintrich, P. R., Smith, D., Garcia, T., & McKeachie, W. J. (1993). Reliability and pre-
dictive validity of the motivated strategies for learning questionnaire (MSLQ).
Washington, DC: Office of Educational Research and Improvement.
References Resnick, M., Maloney, J., Monroy-Herna ndez, A., Rusk, N., Eastmond, E., Brennan, K.,
et al. (2009). Scratch: Programming for all. Communications of the ACM, 52(11),
Anastasiadou, S. D., & Karakos, A. S. (2011). The beliefs of electrical and computer 60e67.
engineering students' regarding computer programming. The International Rizvi, M., Humphries, T., Major, D., Jones, M., & Lauzun, H. (2011). A CS0 course using
Journal of Technology, Knowledge and Society, 7(1), 37e51. scratch. Journal of Computing Sciences in Colleges, 26(3), 19e27.
Baldwin, L. P., & Kuljis, J. (2000). Visualization techniques for learning and teaching Westcott, S. (2008). Effectiveness of using digital game playing in a first-level pro-
programming. Journal of Computing and Information Technology, 8(4), 285e291. gramming course (Unpublished Doctoral Dissertation). USA: Pace University.
Bishop-Clark, C., Courte, J., & Howard, E. V. (2007). A quantitative and qualitative Wolz, U., Leitner, H. H., Malan, D. J., & Maloney, J. (2009). Starting with scratch in
investigation of using Alice programming to improve confidence, enjoyment CS1. SIGCSE Bulletin, 41, 2e3.
and achievement among non-majors. Journal of Educational Computing
Research, 37(2), 193e207.