Action Research Proposal:

Educational Effectiveness of

BrainPOP web presentations.

Robert A. Fiero
Rio Norte Junior High School
Valencia, California
 Introduction
• Stage: Rio Norte Junior High School—Seventh
Grade Life Science.
• Importance: Visual representations in
multimedia have been shown to increase
student learning through a multi-modality effect
and lessen cognitive load.
• Research Question: Do the BrainPOP
animated presentations improve the efficiency of
student learning and motivation seventh grade
life science (cells and genetics) instruction?
 Study Question

What is the effectiveness of BrainPOP

on learning outcomes of motivation,
learning of science content?
 Setting/Stage
• BrainPOP, a multimedia web-based animation
instructional tool has been operation for several
years but had no published studies on its
effectiveness on student learning until late 2009
concurrent with my study.
• Self published information (2003) indicated it
being widely used (15% of school districts) and
effective at accelerating learning for different
learner styles.
• Their funded independent study (not peer
reviewed) showed results of significant increase
of skills for language, reading, and science.
 Importance
• Multimedia
• Instruction is increasingly being delivered via
multimedia applications and online environments.
• Cognitive Load Theory
• Information processing theories have been
increasingly important in designing effective
instructional materials and presentations.
• Modality Effect
• Instructional material, especially in science
education, are widely delivered via combining
visual and verbal information in graphical form.
• Learning style inventories have been a valuable
meta-cognitive tool for students and instructional
design element
 Multimedia
• Interactive multimedia has the potential to create
a student–centered learning multi-sensory
environment with formative and performance
assessment capabilities (Reed, 2006).
• Computer assisted instruction such as
multimedia produces higher student
achievement in an accelerated manner (Zheng,
2007) or can if designed and implemented
effectively.
• Multimedia that utilize simple animations (i.e.
graphic background is neutral, no ambient
sound, focus is on content etc.) and only minimal
text enhancements such as captions are
consistent with good design principles (Milheim,
2006).
 Cognitive Theory
• Information process theory assumes individuals
have a limited working memory, and when
overloaded learning stops. Instructional
materials should be designed to reduce this
cognitive load and using more than one
presentation modality aids in this, such as
narrative animation (Sweller, 1998; Kirschner
2002; Mayer, 2001).

• Prior knowledge (constructed cognitive

schemas) also aid in reducing the stress on
working memory and aid in learning efficiency
(Valcke, 2002)
 Modality Effect
• Individual learning differences (strengths or
preferences) correlate strongly with their
reasoning or cognitive abilities.

• Field dependent learners benefit from the

multimodal presentations because of the support
of visual and audio cues (Zheng, 2008).

• Students with assimilating and accommodating

learning styles had positive attitudes toward
network-based instruction (Frederico, 2000).
 Methodology
• Where: Rio Norte JHS
• Who: 7th grade students
• What: Life Science—California Standards
• When (timeline): Both first and second
quarters during respective Cell and
Genetics units of study.
• How (triangulation): Three data sources
per motivation and content learning
elements in the study.
 Methodology
Where:
• Rio Norte Junior High
School in Valencia,
California

• Administrated by
William S. Hart Union
High School District
and Los Angeles
County Office of
Education.
 Methodology-PARTICIPANTS
Who:
Who 7th grade students in four classes. N=101
• Two classes will be experimental at a time with another two as
controls for comparison.
• Each class is mostly randomized for equity by gender when their
schedules are arranged.
• **Average class size for all four classes:
• Period 1: N=29 (15 M & 14 F)
• Period 2: N=24 (14 M & 10 F)
• Period 3: N=22 (11 M & 11 F)
• Period 6: N=26 (14 M & 12 F)
• Classes are on rotating block creating greater equity in learning
experience relative to time of day.
• Resource students are clustered evenly throughout each class.
**Some roster changes were made and those students were not
included in the study. So the numbers shown are the base minimum
for each class.
 Methodology-PROCEDURE
When (timeline):
• First quarter with the cell biology unit two treatment
classes used BrainPOP and two control classes did not.
At the end of this cell unit the treatment and control
classes switched roles for the genetics unit for further
comparison and data collecting.
• The cell unit was significantly longer than the genetics
unit temporally and content wise as well.
• Cell Unit: 9/14/09 to 11/16-09, Chapters 3 & 4.
• Genetics Unit: 11/18/09 to 12/15/09, Chapter 5 and Chapter
6.1-2

• A fifth class I teach did not participate.

 Methodology-CONTENT
What:
What Life Science—California Standards

• Cell Biology Standards

All living organisms are composed of cells, from just one to many
trillions, whose details usually are visible only through a
microscope. *Chapters 3 and 4.
• Genetics Standards
A typical cell of any organism contains genetic instructions that
specify its traits. Those traits may be modified by environmental
influences. *Chapters 5 and Chapter 6.1 and 6.2.
• BrainPOP
A multimedia web-based animation instructional program.
http://www.brainpop.com/
*Science Explorer California Focus on Life Science 2008 Prentice Hall
publishing.
 Methodology- INSTRUMENTS
How the data was collected:
Student Content Learning
 Pre- and Post-Unit Test
 Standards-based benchmark test.
 Overall grade after each unit.

Student Motivation & Engagement

 Attitudinal Survey on BrainPOP experience.
 Modality Strengths Survey-Learning Style
Inventory Test
 Research Questions
• Do the BrainPOP animated presentations
improve the efficiency of student content learning
of cell biology and genetics?

• How does multimedia animated presentations of

BrainPOP affect student motivation and
achievement in life science 7th grade curriculum?

• Is there a positive correlation between learning

style (modality preference) and the motivation
and learning of students who use BrainPOP?
 Results-Content Learning
Two assessments were given for each unit
to measure content learning:
• Prior content knowledge was assessed
using a 50-item pre-test and post-test
score were compared.
• Standards-based Benchmark Exams.
• None of the test scores showed
significant differences between
experimental and control groups.
 Results-Content Learning
• Pre-test & Post-test scores for both units
showed for all classes (aggregated) significant
differences in student performance. P>0.05
• CELL UNIT Genetics Unit:
Cell Unit: Pre-
Pre- & Post-Test
& Post-Test

45.00
45.00
• Mean of Pre-test:
40.57 39.70
22.2/50 40.24
39.96 38.17
38.54 Period 1 Pre-test
40.00 38.13 37.92
40.00
35.00
35.00
• Mean of Post-test: 39.1/50 Period 1 Post-test
Period 3 Pre-test
•30.00
GENETICS
30.00
25.00 23.32
20.79
UNIT
21.52
20.87 21.08
22.96
22.56 Period 3 Post-test
25.00 21.46
20.00
20.00 • Mean of Pre-test: 21.5/50 Period 2 Pre-test
15.00
15.00 Period 2 Post-test
10.00
10.00 • Mean of Post-test: 39.5/50 Period 6 Pre-test
5.00
5.00 Period 6 Post-test
--
Period
Period1 1 Period
Period33 Period 2 Period66
Period

Potential total score of 50

 Results-Content Learning
Cell UnitQuarter
Genetics-Experimental vs Control
Unit-Experimental
1
2-Benchmark
Benchmark
Groups
vsResults
Control Groups
Scores

40.50
41.00
22.5
Scores

40.57
39.96
23.00
Scores
Scores
Scores

40.50
40.00 22.88
40.24
22.04
22.90
22
40.00 21.75 39.70
39.50
22.80
Test

22.65 Period1-Control
Period 1-Exp.
39.50 Period1-Exp.
Period
Benchmark
Test

22.70
21 Benchmark

39.00 21.39 1-Control

21.5 38.54 Period3-Control
3-Exp.
Unit

22.60
39.00 Period
Period3-Exp.
Period 3-Control
38.50 38.13
Unti

22.50
38.50 38.17 Period
Period 2-Control
22.36 22.38 37.92 Period2-Exp.
Period 2-Control
2-Exp.
Post-Genetics

21
38.00
22.40 20.75
38.00 Period
Period 6-Control
Period6-Exp.
Period 6-Control
Post-Cell

22.30 6-Exp.
37.50
37.50
20.5
Quater

22.20
Quarter

37.00
37.00
22.10
36.50
36.50
20
22.00
Per.1
Per.1
Per.1 Per.3
Per.3
Per.1 Per.3
Per.3 Per.
Per.2222
Per.
Per. Per.666
Per.
Per.
Per. 6

Potential total score of 50

25 P>0.05
 Results-Achievement
• To measure student achievement student grades were
compared between the control and experimental classes for
Quarter
each 1 for
2
unit. No the Genetics
Cellsignificant
statistically Unit: Unit:
PeriodsPeriods
1 & when
differences 3
were
2 & 6 the
were
aggregating experimental
thethe experimental
classes. groups.
P>0.05 groups.

Quarter
Quarter 12 Grades
Grades

Period
Period63
A
A
Period
Period21 B
B
C
C
Period
Period36 D
D
F
F
Period
Period12

0%
0% 20%
20% 40%
40% 60%
60% 80%
80% 100%
100%
 Results-Engagement
• To determine student perceptions of
BrainPOP and how engaging it was to
them a survey was given.
• Survey had 24 questions (see appendices
section).
• Approximately half the students had
viewed/used it in other classes.
• Aggregating results indicated a mild
positive appreciation for the BrainPOP.
 Results-Engagement
Coded the responses accordingly:
Strongly disagree = 1, disagree = 2, neutral = 3, agree = 4, strongly agree
= 5.

They gave a modest score of 4 (mode) to the following:

• I enjoyed or appreciated the BrainPOP episodes
overall.
• I think BrainPOP has significant educational value.
• It truly helped me learn about a specific science topic.
• Brain Pop aided you in performing better on the
benchmark or unit tests (on cells and genetics).
• I experienced BrainPOP as entertaining and yet
educational.
 Results-Learning Modality
 Modality Strengths Survey-Learning Style
Inventory Test
 Most students preferred the visual
(34%)and kinesthetic (29%) modalities
which the literature indicates these two
types would appreciate animations or
visuals of any kind or verbal (words).
 Discussion-Implications
• BrainPOP has effective multimedia presentation
features consistent with cognitive load theories.
• BrainPOP bestowed no significant effect on
learning the science content of cellular biology
and genetics and is inconclusive in regards on
content learning.
• BrainPOP was one many assignments and
interactions or opportunities to learn the content.
• BrainPOP did mildly increase their motivation to
learn science.
 Discussion-Limitations
• The post-tests and Benchmark assessments
were given often weeks after BrianPOP with
many other activities intervening.
• The pre-post-test and benchmark exams in my
study were only for retention and not for transfer
of knowledge.
• Not satisfactorily equal distribution of resource
and SC1students.
• Small participant size. (N=101).
Discussion-Further Questions/Research
• I would implement the full array of activities and
resources that BrainPOP provides to determine
if their full suite of educational resources are
truly effective.
• Test the prevailing theory that narration and
video is much more effective than narration and
text (Mayer 2005).
• I would like to see if students use BrainPOP on
their own, in full of control to stat and stop the
animations and thus view at their own pace
would improve its effectiveness and be more
motivating for them.