Raz-Kids

An Evaluation of an Online Reading Program

Public domain image courtesy of pixabay.com

Prepared for Second Grade teachers at

Starlight Elementary School
Denise Holder
EDTECH 505

Raz-Kids Evaluation Report

Summary
Raz-Kids is an online reading program that Starlight Elementary School second grade students
have participated in for the 2013-2014 and current 2014-2015 school years. The Raz-Kids
program gives students the opportunity to practice reading just-right books, take
comprehension tests, and build reading skills through practice. By adopting Raz-Kids, it was the
teachers hope that students would find more interest in reading online thus creating increased
reading achievement.
This report is an evaluation of the Raz-Kids online reading program. This program was adopted
to address the issue of the decline in reading achievement scores. The purpose of this evaluation
is to examine the objectives of the Raz-Kids program to determine whether or not it was
effective in meeting those objectives and, in turn, determine its future use with Starlight
Elementary second grade students. Students were given the STAR Reading assessment at the
beginning of the evaluation period. Teachers had the ability to monitor student progress using
online reports available in the Raz-Kids program. After the program evaluation period ended,
students and teachers were each given a summative evaluation to determine attitudes towards the
Raz-Kids program.
The results of this evaluation showed that student reading achievement increased above typical
growth. One classroom spent much less time in the program, but that lack of time did not hinder
growth in reading achievement. Out of twenty-four skills assessed, students showed accuracy in
20 of them. Student perceptions of the program were positive, as were teacher perceptions.
Teachers believe that Raz-Kids helped increase student engagement and achievement. Various
reports that identify individual student progress are available to teachers, but those reports were
not used due to lack of training. In the discussion section at the end of this report, suggestions for
improving this already successful program are included.

Program Description
Raz-Kids is an online reading program that the three classrooms of Starlight Elementary School
second grade students have participated in for the past two school years. The Raz-Kids program
gives students the opportunity to practice reading just-right books, listen to just-right books
and books of interest, take comprehension tests (eQuizzes), and record themselves reading for
reading practice and teacher evaluation. These tasks are aimed at increasing student reading
achievement.
Students have the opportunity to use this program at school for at least 20 minutes, four times a
week. There are additional opportunities each day where students are able to log into the
program for independent work. Students may also access this program at home on their
computers or via an app on a cell phone. Two Raz-Kids virtual classrooms were purchased for
Starlight Elementary second graders. Two actual classrooms at Starlight Elementary took one
Raz-Kids classroom, and the third Starlight Elementary classroom was split between the two
Raz-Kids classrooms.

By using this program, students gain practice in reading a variety of texts. Teachers are able to
monitor this practice through a variety of classroom reports. Parents can also access their childs
activity and progress if they choose to create a parent account.

Program Objectives
The team of second grade teachers, which included the evaluator, worked together to establish
the objectives for determining the effectiveness of the Raz-Kids program. The second grade
teachers hope the Raz-Kids program will:

increase student reading achievement

show student effectiveness in skills such as vocabulary, problem/solution, etc.
increase student engagement and perceptions about reading
assist teachers in monitoring student progress

Program Components
Raz-Kids is an online reading program. It has interactive, leveled books that span 27 levels.
There are over 400 online books and quizzes. Each corresponding eQuiz measures
comprehension, providing teachers with skill-driven instruction. Online running records allow
students to record reading a passage so that teachers may go back and digitally assess students.
Raz-Kids is available via the internet, but it may also be accessed through an app available for
download on mobile devices, tablets, or iPods.
Students can listen to books for modeled fluency, read books for practice, and record themselves
reading so teachers can monitor progress. Every leveled book has an accompanying eQuiz to test
reading comprehension. Students are able to practice reading any time, anywhere.
Teachers have the ability to create assignments and track student progress with online
assessments and student recordings. A variety of reports are available for teachers to view
student progress.

Evaluation Method
Participants
Participants in this evaluation included all sixty-three second graders from Starlight Elementary,
a kindergarten through fifth grade school in the Galaxy School District. The three second grade
teachers were included in determining program objectives and participated in the administration
and the evaluation of the program.

Procedures
This evaluation used the goal-based model in which the evaluator worked with stakeholders to
answer whether the Raz-Kids program is meeting its intended objectives. Prior to the evaluation

period, teachers gave their class a tutorial on how to access and log in Raz-Kids at school and at
home. Teachers instructed students how to navigate the different aspects of the programs and
discussed expectations. Baseline data was collected from students in the form of the STAR
Reading assessment. This test determined their initial reading level. Parents were notified of their
childs participation in the program through a letter that Raz-Kids provides. This letter gave each
parent the log in information as well instruction on to access Raz-Kids to check on student
progress. Information on how to send messages to their child in the program was also included in
this letter.
During the evaluation period, students had an opportunity to work on the Raz-Kids program for
twenty minutes, four days a week. Students were also encouraged to use the program in their
classrooms during independent work time. Teachers reminded parents, via emails and weekly
newsletters, that Raz-Kids was available to all students at home. A direct link to the Raz-Kids
website was placed on the front page of the second grade website. The Raz-Kids program has
data collection built into it, so teachers had the ability to access and monitor individual student
progress in the program.
At the end of the program evaluation period, students took the STAR Reading assessment again
to determine current reading achievement. Fifty-eight students and three teachers participated in
summative evaluation surveys to determine attitudes toward the Raz-Kids program. It must be
noted that the program was not over when the evaluation period ended. Due to time constraints
in completing the program evaluation, the program evaluation period ended before students
completed the Raz-Kids program.

Data Sources
To determine student reading achievement, administration of the STAR Reading assessment was
completed at the beginning of the program as well as at the conclusion of the program evaluation
period. The STAR Reading assessment uses a combination of reading comprehension passages
and cloze method questions to assess individual student reading achievement. The two STAR
Reading assessment scores were compared to determine reading achievement progress. Changes
in achievement levels were documented.
The Raz-Kids program automatically tracked program use as students participated. Program use
was tracked to determine a correlation, if any, between time spent in the program to the
achievement data in each class. Individual student program use was available and reviewed.
Skill reports from the Raz-Kids program were accessed in the Classroom Reports section of the
program. Each question is linked to a specific skill, and scores are compiled from all individual
student eQuizzes after a student has read and/or listened to a book. In addition, classroom
activity reports were accessed.
A student survey was designed to determine student engagement in the program and student
attitudes towards reading now that students have worked with the program for an extended
period of time. This survey was available electronically. Students accessed this survey in class
and completed it as the teacher read it out loud. This survey is available in Appendix A. Results
can be seen in Appendix B.

A teacher survey was also created to determine ease of use, perception of change in student
achievement due to the use of the program, and feeling regarding continued use of the program.
There was a 100% teacher response rate. This survey was available to teachers electronically, but
discussion of results occurred in person. This survey is available in Appendix C. Results can be
seen in Appendix D.

Results
STAR Reading Assessment
STAR Reading assessments were administered prior to the program start. They were also
administered at the end of the program evaluation period so data could be compared. The
majority of students in all three classrooms increased reading achievement. For the eight-week
evaluation period, expected reading achievement growth is 0.2.
Class A 21 students
Sixteen students had an increase in reading achievement. The mean achievement increase was
0.8125. Achievement decreased for four students. The mean decrease was 0.425. One students
reading level remained the same.
Class B 21 students
Seventeen students had an increase in reading achievement. The mean achievement increase was
0.735. Achievement decreased for two students. The mean decrease was 0.4. Reading levels for
two students remained the same.
Class C 21 students
Thirteen students had an increase in reading achievement. The mean achievement increase was
0.68. Achievement decreased for three students. The mean decrease was 0.5. Reading levels for
three students remained the same. Two students did not participate in the assessment.
20
15
10
5
0

Class A
Class B
Class C

Program Use

Program use was calculated for

individual classrooms. Two classrooms
had similar time spent in the program,
while one classroom spent
approximately half of the time that the
other two classrooms did on the
program.

Minutes spent in Raz-Kids

5000
4000
3000
2000
1000
0

Class A
Class B
Class C

Skills Assessment
Skill reports were reviewed to assess comprehension strengths. These reports can also be used to
define opportunities for additional instruction for your whole class or to look at individual
student reports to individualize instruction. Students were accurate in the majority of the skills
assessed. Four skills fell below 80% accuracy. The percentage below 80% identifies the skills as
areas that need additional practice.
Skills

Accuracy

Identify Theme
Character Point of View
Identify Genre
Connect to Prior Knowledge
Analyze Characters
Problem and Solution
Classify Information
Main Idea and Details
Recall
Story Elements
Phonological Awareness
Comprehension
Vocabulary
Phonics
Sequence
Make Inferences/Draw Conclusions
Cause and Effect
Compare and Contrast
Authors Point of View
Authors Purpose

100%
100%
100%
93%
91%
91%
89%
88%
88%
88%
87%
86%
85%
85%
84%
84%
83%
82%
82%
81%

Reality/Fantasy
Make Inferences
Fact or Opinion
Draw Conclusions

78%
77%
74%
74%

Student Survey

Students were surveyed to determine program use as well as attitudes about Raz-Kids. Fifty eight
out of sixty three students participated in the survey. The majority of students have a favorable
opinion of Raz-Kids. In spite of that favorable opinion of the Raz-Kids online reading program,
the majority of students would rather read an actual book than read online. The majority of
students also believe that Raz-Kids has made them want to read more and has helped them
become better readers.

Teacher Survey
Teachers were surveyed about how the Raz-Kids program was used, the data collection aspects
of the program, and their overall feelings about the program. All teachers believe that Raz-Kids
increases student engagement in reading and reading achievement. Two of the three teachers did
not find the Classroom Reports helpful. All teachers said they do not have any hesitations about
using Raz-Kids with their students. Below is input from teachers regarding issues that must be
addressed if the Raz-Kids program continues.
Question
What types of support
might you need to
implement this
program successfully?

Response
None.
I would need to learn how to implement the program
successfully and have a better understanding of it (I just need
to make time to play around on the website!)
Tutorial showing me the reports and how to, email suggesting
I look at a certain report

Question
What barriers, if any,
occurred when
implementing the
Raz-Kids program?

Response

Understanding how it works

How students use the rocket? Can I turn off the read aloud
part? Are there tutorials for teachers and students?

Teachers were asked why they would continue/discontinue the Raz-Kids program. All teachers
said they wished to continue and shared these reasons why:
Kids are more engaged in this program.
I like the read aloud/modeled reading and comprehension checks
I believe it empowers the students to become better readers--they feel successful with this
program and love seeing their growth when they move up levels.

Discussion
Teachers at Starlight Elementary School began using Raz-Kids in the 2013-2014 school year.
This evaluation was performed to determine whether or not the Raz-Kids online reading program
is effective in increasing student reading achievement and showing effectiveness in a variety of
skills. The evaluation was to also determine student and teacher attitudes regarding the program.
The primary objective of the program evaluation was to determine if the use of Raz-Kids was
effective in increasing student reading achievement. The results of the STAR Reading
Assessment showed 46 out of 63 students, or 73%, had an increase in reading achievement
during the program evaluation period. In contrast, fourteen percent of students had a decrease in
reading achievement.
Student growth in reading achievement, as measured by the STAR Reading assessment, was
evident in the majority of students in all three classes. When comparing times individual
classrooms spent in the Raz-Kids program, Class A utilized the program half as much as the
other two classrooms. However, this lack of time spent in the program did not affect achievement
in that classroom. Class A has the highest average increase and the average decrease was
comparable to Class B and Class C. Continued, consistent use of the program is recommended
for all classes.
The student survey displayed positive results in the attitudes of the students toward the Raz-Kids
program. Eighty two percent of students had a favorable opinion of the Raz-Kids program. In
addition to that favorable opinion, almost 88% of students believe that they have become better
readers since they have been using the Raz-Kids program. In contrast to the positive feelings that
students have about the program, seventy percent of students would rather read a book in their
hands instead of on a computer. Over half of the students have accessed Raz-Kids outside of the
school day, whether on a home computer, tablet, or mobile telephone. Positive student attitudes
about the program and its results, as well as desire to utilize the program outside of school hours,
could translate into increased student achievement due to increased participation and
engagement. It is recommended that program use and promotion of the program outside of
school continues.

Data analysis in the teacher survey determined that teachers also have a favorable opinion of the
Raz-Kids program and believed it helps increase their students reading achievement. The survey
results determined that teacher use of reports was varied. After discussing this finding with
teachers, teachers stated that they did not utilize all of the reports because they were unsure how
to access them. It is suggested that, prior to the school year, teachers receive training on how to
access reports to determine areas of improvement for individual students. A mid-year refresher
based on teacher needs regarding the program is also suggested.
Based on the data, the Raz-Kids program appears effective and worth the investment. The
majority of students tripled the expected reading growth rate for reading achievement. Students
have interest in the program and believe that it contributes to their reading growth. The data the
program provides is valuable to teachers if they know how to access it. After reviewing all data
sources, the Raz-Kids program has met all intended objectives. It is recommended that the
program continue to be used at Starlight Elementary along with the program improvement
suggestions.

Project Cost
Ms. Denise Holder, program evaluator, developed instruments required to review the Raz-Kids
program. In the role of evaluator, Ms. Holder administered the assessments and surveys,
collected and analyzed data, and compiled the final evaluation report. Because Ms. Holder was a
program facilitator, in addition to the program evaluator, there was no cost for travel as she was
present each day of the program evaluation period. Surveys were created using the cost-free
Google Forms and posted on the second grade website. This chart outlines cost for services
performed for this evaluation.
Personnel
Denise Holder,
program evaluator

Total

8 days

$500/day

$4,000

$4,000

Appendix A
Student Survey
These are the questions included on the student survey.
1. What is your name?
2. Who is your teacher?
3. Do you like to read? Yes/No
4. How much time do you spend reading each day?
a.
b.
c.
d.
e.

Less than 15 minutes

Between 15 and 30 minutes
Between 31 and 45 minutes
Between 46 and 60 minutes
More than an hour

5. I would rather...
a. read a book that is in my hands
b. read a book that is on the computer
6. How much do you enjoy using Raz-Kids?
a.
b.
c.
d.
e.

Really dont like it

Dont like it
Its ok
Like it
Love it

7. Have you ever worked on Raz-Kids on your home computer? Yes/No

8. Have you ever worked on Raz-Kids at home using a tablet or a phone? Yes/No
9. Do you feel like you want to read more because you have been using Raz-Kids? Yes/No
10. Do you think you have become a better reader because you have been using Raz-Kids?
Yes/No

Here is the link to the live student survey: http://tinyurl.com/razstudentsurvey

Appendix B
This appendix contains results from student survey. Names have not been included to protect
student and teacher privacy.

A
ppendix C
Teacher Survey
These are the questions included on the student survey.

1. How many days was Raz-Kids accessible to your students?

1, 2, 3, 4, 5
2. Were your students expected to use Raz-Kids in the computer lab each morning? Yes/No
3. What barriers, if any, occured when implementing the Raz-Kids program?
4. Which of these classroom reports did you access?
a. Activity Report
b. Assignment progress
c. Assessments
d. Skill reports
e. None of the above
5. How helpful did you find the Classroom Reports portion of Raz-Kids?
Likert scale = 1 (Not at all helpful) 5 (Very helpful)
6. How easy was it for you to see the progress of your students using the Classroom
Reports?
Likert scale = 1 (Very difficult) 5 (Very easy)
7. What types of support might you need to implement this program successfully?
8. Do you have any hesitations about using Raz-Kids with your students? Yes/No
9. If you have any hesitations, what are they?
10. Do you believe Raz-Kids increases student engagement in reading? Yes/No
11. Do you believe Raz-Kids increases student reading achievement? Yes/No
12. Would you recommend continued use of Raz-Kids? Yes/No
13. Please give a reason as to why you wish to continue or discontinue Raz-Kids use.

Here is the link to the live teacher survey: http://tinyurl.com/razteachersurvey

Appendix D
This appendix contains results from teacher survey.

1. How many days was Raz-Kids accessible to your students?

4 days 1 response
5 days 2 responses
2. Were your students expected to use Raz-Kids in the computer lab each morning?
Yes 2 responses
No 1 response
3. What barriers, if any, occured when implementing the Raz-Kids program?
o Understanding how it works
o How students use the rocket? Can I turn off the read aloud part? Are there
tutorials for teachers and students?
4. Which of these classroom reports did you access?
Activity Report 2 responses
Assignment progress 0 responses
Assessments 0 responses
Skill reports 1 response
None of the above 0 responses
5. How helpful did you find the Classroom Reports portion of Raz-Kids?
Likert scale = 1 (Not at all helpful) 5 (Very helpful)
1 (Not at all helpful) 2 responses
5 (Very helpful) 1 response
6. How easy was it for you to see the progress of your students using the Classroom
Reports?
Likert scale = 1 (Very difficult) 5 (Very easy)
1 (Very difficult) 1 response
5 (Very easy) 2 responses
7. What types of support might you need to implement this program successfully?
o None.
o I would need to learn how to implement the program successfully and have a
better understanding of it (I just need to make time to play around on the
website!)
o Tutorial showing me the reports and how to, email suggesting I look at a certain
report
8. Do you have any hesitations about using Raz-Kids with your students?
Yes- 0 responses
No 3 responses
9. If you have any hesitations, what are they?
N/A

10. Do you believe Raz-Kids increases student engagement in reading?

Yes- 3 responses
No 0 responses
11. Do you believe Raz-Kids increases student reading achievement?
Yes- 3 responses
No 0 responses
12. Would you recommend continued use of Raz-Kids?
Yes- 3 responses
No 0 responses
13. Please give a reason as to why you wish to continue or discontinue Raz-Kids use.
o Kids are more engaged in this program.
o I like the read aloud/modeling reading and comprehension checks.
o I believe it empowers the students to become better readers they feel successful
with this program and love seeing their growth when they move up levels.