Texas School District

Technology Applications
Program:
An Evaluation Report
Submitted to: Dr. Francisco Zepeda
Educational Technology Director
Texas Independent School District

Submitted by: Nicolas Hernandez

Educational Technology Evaluator
Texas Independent School District

Page |2

SUMMARY
The Grade 3-5 Technology Applications Program at Texas Independent School
District aims to prepare students for 21st Century skills use of technology in the
classroom in middle school and beyond. Additionally, the program aims to teach
students the technology literacy skills they will use throughout their academic careers.
Each campus at the Texas Independent School District has a Technology Applications
Program. Each campus employs a teaching assistant to teach the course and the
students rotate into the class once every nine days. There are no prior formal
evaluations of the Technology Applications program.
The purpose of this evaluation report is to provide information on whether or not
the Grade 3-5 Technology Applications program met the Texas Independent School
District goals and overarching objective to prepare students to use 21st-century
technology skills in the classroom. A secondary purpose of this evaluation is to
determine whether or not the program met the 2014-2015 school district goals for the
teaching of technology applications skills.
The results of the evaluation show that the Texas Independent School District
students and teachers have an opportunity to grow into one of their goals. Based on the
data collected, the program is successful at training students in technology skills.
However, the majority of the technology is used in ways outside of the districts goals
and objectives. Most notably, 21st century related classroom use of technology scored
low on both teacher and student surveys.

DESCRIPTION OF THE PROGRAM EVALUATED

The Grade 3-5 Technology Applications program at the Texas Independent School
District is located on all five elementary campuses. The total population of the program
on these campuses consists of approximately 2,500 students. Each campus utilizes a
teaching assistant to teach the entire population of students in the technology course.
The teaching assistants each have six classes with an average class size of 23
students. They see each of their classes once every nine days on a rotation that permits
them to see all of their students regularly. Each computer lab is outfitted with high-speed
Internet access (Ethernet and WIFI), 30 late model Dell desktop computers running
Windows 7 and Microsoft Office, a ceiling mounted projector, speakers, and a
SmartBoard. The technology applications teachers use a curriculum from learning.com
to teach the students the basic technology skills they need to be successful.
The primary goal of the Grade 3-5 Technology Applications program is to prepare
students to use 21st-century skills in the classroom. A secondary goal of the program is
to provide students with the technical know how required of a 21 st-century classroom.
This is done by teaching students the computer literacy skills they will use throughout

Page |3
their academic careers. Students also have access to technology in the classroom with
their content area teachers. The expectation is that students will take what they learn
from the Technology Applications program and begin to use those skills in the
classroom. Content area teachers are expected to design lessons that incorporate 21 stcentury skills for their students.
Program Goals & Objectives
The Texas Independent School District is preparing students for the workplace by
stressing 21st-century skills such as collaboration, problem-solving, creativity, critical
thinking, and online communication in the classroom. The school districts overarching
objective for the Technology Applications program is:

The Technology Applications program will prepare students to use 21st-century

technology skills in the classroom.

The school district has goals for the program to help maintain alignment and focus.
These are:

By the end of the 2014-2015 school year, 80% of students in the Technology
Applications program will acquire the technology skills and master the
technological processes that are required in 21 st-century classrooms.
By the end of the 2014-2015 school year, at least 80% of students will transfer
the skills they learn in the Technology Applications class to the content area
classroom on a monthly basis.

Program Components
The Technology Applications program features a fully functional lab on each campus
that is dedicated to the programs use. The computer lab includes 30 late model Dell
desktop computers that are connected to high-speed Internet access. The computers all
run Windows 7 and have Microsoft Office installed on them for students to use. The
district also provides each student with a Google Apps for Education account. The lab
also includes a ceiling mounted projector, an audio system, and a SmartBoard.
The curriculum for the Technology Applications classes is from learning.com. This
curriculum was installed last year. The learning.com curriculum features interactive
lessons that teach students 21st century skills. Content area teachers have access to
the curriculum, and they are expected to complete the Technology Applications lessons
before the school year begins. They are expected to design lessons that incorporate
21st century skills.

EVALUATION METHOD
Participants

Page |4
This evaluation was conducted of the entire Texas School District Grade 3-5 Technology
Applications program. The student population consists of 2,576 students on the five
elementary campuses. While participation in the evaluation was mandatory for all third
to fifth-grade students, 34 students failed to complete the survey taking the total number
of student participants in the evaluation to 2,542. This accounts for approximately 23
students per Technology Applications class. These students are all Texas Independent
School District students. The five Technology Applications teachers and all 27 content
area teachers (mandatory participation) from all four core subject areas on the five
campuses also participated in the evaluation.
Procedures
To test for the districts overarching goal of the Technology Applications program, the
data focused on the two district goals which stated:

By the end of the 2014-2015 school year, 80% of students in the Technology
Applications program will acquire the technology skills and master the
technological process that are required in 21 st-century classrooms.
At least 80% of students will transfer the skills they are learning in the
Technology Applications class to the content area classroom on a monthly basis.

The evaluation consisted of surveys that aimed to test whether or not these goals were
met. Surveys were administered to student and teacher participants, and Gradebook
data for the Technology Applications courses and end of course exam grades were
used to determine whether or not students mastered the course curriculum. The student
surveys were conducted over a standard nine-day rotation in the Technology
Applications classroom from April 7th to April 17th. Prior to taking the survey, an email
was sent to all the Technology Applications teachers with instructions on how to have
students complete the survey. A link to the survey was included in the email. ( A
samples of these email messages can be found in Appendix C).
On the day that students were to take the survey, students would log into the computer
as normal, navigate to the survey as directed by the instructor and complete the survey.
On average the surveys were completed in about 20 minutes and students continued
with the technology class as normal. During the survey, the teaching assistant was
permitted only to issue clarifying statements about the survey directions. The teaching
assistants were not permitted to provide students with any other assistance outside of
what is provided to them as part of their special education or 504 accommodations.
Teachers also took a survey over the same nine-day period. A link to the survey was
sent to each teacher specifically. Teachers logged into the account and completed the
survey when it was convenient for them to do so. All 27 participants completed the
survey on time. Outside of having to login and take a 20-minute survey, the teachers
participation in the evaluation did not distract from their normal routine.

Page |5
Permission was obtained to analyze Gradebook data with the understanding that
grades would only be discussed in general terms, and test questions could not be made
public under any circumstances.
Data Sources
Student Survey
The student survey consisted of 17 questions. The questions were geared to get data
from three main categories: student skills, digital citizenship, and classroom technology
use. Because the districts goals referenced monthly as a target, the answer choices
for the classroom technology use and digital citizenship were all based on frequency of
use with 5 answer choices ranging from almost daily to never with monthly at the
midpoint. The student skill questions also had five answer choices ranging from easy
to impossible with somewhat difficult as the midpoint. The complete list of questions
and answer choices for the Student Survey can be found in Appendix A: Student
Survey.
Teacher Survey
The teacher survey included 19 questions. The questions were designed to gather data
regarding classroom technology use. The questions asked teachers about their
behavior and expectations with their students regarding the frequency of use and the
types of technology use in the classroom. Because the districts goals referenced
monthly as a target, the answer choices for the teacher survey were based on the
frequency of technology use. There were five answer choices ranging from almost
daily to never with monthly as the midpoint.
Gradebook Data
Gradebook data from all five of the Technology Applications teachers (for all grade 3-5
Technology Applications students) was analyzed to determine the percentage of
students who showed mastery of the Technology Applications skills. To reach a level of
mastery, students had to have a final grade of 75 or above in either the Technology
Applications course or the end of course exam

RESULTS
The student and teacher surveys were administered in April of 2015 over a nine-day
period. The Gradebook data was analyzed in July of 2015. The results of the data
collection are included below.
Primary Data Source: Student Survey
The student survey results are broken down into three different tables. Each table
consists of data from the student survey that is pertinent to its category. Table 1 includes
data on the frequency of classroom technology use by students. The table includes

Page |6
seven questions. The table is shaded slightly darker in the three columns to the right.
These columns represent scores in the target zones according to district goals. Except
for the generic question involving computer use in the classroom, the other six
questions reflect scores where the majority of students responded that their use of 21 stcentury skills occurred less frequently than the districts goal of monthly.
Table-1 Student Survey Results (Freq.
of Classroom technology use)

Neve
r

Computer use in classroom

Feedback from others in class
Online feedback NOT from teacher
Use video chat or chat apps in class
Use web tools to receive online info.
Write online
Work on e-portfolio

3%
36%
70%
76%
49%
59%
68%

Every
Few
Month
s
13%
16%
14%
7%
16%
12%
15%

Monthly

Weekl
y

Almos
t Daily

11%
23%
10%
7%
15%
12%
11%

26%
16%
5%
9%
10%
11%
6%

47%
9%
2%
0%
9%
6%
0%

The data in Table-2 contains four questions. The table contains information about the
frequency with which students receive training in digital citizenship (a key aspect of 21 stcentury skills training). The data show that a majority of students rate in the target zone
for citing online information, but below the target zone for the other three questions in
this set.
Table-2 Student Survey Results (Freq.
of Digital Citizenship Lessons)

Neve
r

Cite online information

Sharing personal information online
How to act respectfully online
Respond to online bullying

11%
59%
31%
34%

Every
Few
Month
s
29%
18%
27%
32%

Monthly

Weekl
y

Almos
t Daily

35%
14%
21%
17%

15%
9%
15%
15%

10%
0%
6%
2%

The information in Table-3 is about student technology skills. The table contains seven
questions. At least 80% of students claimed they could perform the tasks listed in the
table except for making online purchases which had only 67% of students claiming
they could do this. This particular question may have to be examined to rule out a
possible confounding variable relating to student access to making online purchases.
Table-3 Student Survey Results
(Student Technology Skills)
Sending email
Attaching a printer
Creating a spreadsheet

Impossibl
e
1%
13%
6%

Difficul
t
1%
7%
7%

Somewha
t Difficult
6%
22%
25%

Easy
17%
22%
25%

Very
Easy
74%
36%
36%

Page |7
Sending a text message
Making online purchases
Viewing Friends photos online

5%
13%
13%

1%
20%
4%

1%
26%
5%

8%
22%
18%

85%
19%
61%

Primary Data Source: Teacher Survey

The teacher survey results can be found in Table-5. Table-5 consists of 19 questions
and answers about how teachers incorporate technology into their lessons with
students. Much like what the students reported, except for the generic students use
technology in class question, all of the 21st century related classroom technology use is
used less than once a month in the core content classrooms. Nine of the questions had
a score of over 80% in the never column with online collaboration with other schools
receiving a score of 96% in the never column.

Table-5 Teacher Survey Results (Freq.

of use)

Neve
r

Students use technology in class

Students get feedback in class
Ask students to receive online
feedback NOT from the teacher
Ask Students to use chat or video
chat applications
Have students receive info with web
tools
Have students write online
Have students create e-portfolios
Students collaborate with class
Students collaborate with teacher
Online student collaboration with
other schools
Students share documents online
Students use digital tools to collect
and analyze data
Students conduct experiments or
perform measurements
Conduct research with digital tools
Students use tech to identify and
solve real-world problems
Have students create and upload

6%
56%

Every
Few
Month
s
11%
4%

87%

Monthly

Weekl
y

Almos
t Daily

11%
20%

28%
12%

45%
8%

7%

2%

4%

0%

91%

2%

4%

2%

0%

84%

7%

4%

3%

1%

87%
93%
93%
89%

7%
4%
4%
4%

4%
2%
2%
7%

1%
0%
0%
0%

1%
0%
0%
0%

96%

4%

0%

0%

0%

71%

9%

18%

2%

0%

47%

29%

20%

3%

1%

60%

20%

18%

2%

0%

33%

47%

18%

2%

0%

47%

24%

22%

5%

2%

76%

20%

2%

2%

0%

Page |8
art, music, movies, or webcasts
Have students create animations,
demonstrations, models, or sims.
Have students develop or present
multimedia presentations
Have students use digital camera

82%

13%

4%

0%

0%

49%

42%

9%

0%

0%

73%

22%

2%

2%

0%

Primary Data Source: Teacher Gradebooks

The Technology Applications teachers Gradebooks were also gathered and analyzed to
determine how many students successfully mastered the Technology Applications skills.
The district threshold for mastery is 80%. In order to reach a level of mastery, students
had to have a final grade of 75 or greater in the Technology Applications class or pass
the Technology Applications End of Course Exam with a grade of 75 or above. Students
who achieved both were placed in the mastery category as well. Students who did not
meet either of these thresholds were considered to not have mastered the Technology
Applications course skills. (The End of Course Exam and the Gradebooks are not
available to be included as an attachment due to state and licensing regulations.)
Third grade rated below the mastery threshold, but 4 th and 5th grade rated above the
threshold. Cumulatively, the group had a passing rate of 85%. The data is organized by
grade level in the graph below.

Technology Applications Mastery by Grade Level

100%
94%

80%
60%

86%

85%

5th

Cumulative

75%

40%
20%
0%

3rd

4th

Page |9

DISCUSSION
The purpose of this evaluation was to determine whether the 2014-2015 Grade 3-5
Technology Applications program met the Texas Independent School Districts goals and
objectives. The overarching objective of the Texas Independent School District Grade 35 Technology Applications program is to prepare students for the use of 21 st-century
technology skills in the classroom. The district created two goals for the Technology
Applications program to help measure this objective. The first goal calls for 80% of
students to reach a level of mastery in the Technology Applications courses. The second
goal is for 80% of students to transfer the 21st century skills learned in the Technology
Applications course to the core content classroom by utilizing the skills in the core
content classroom at least once a month. The results of the evaluation are intended to
be used so that the Educational Technology Department can make educated decisions
about the Technology Applications programs future. Two surveys were created and
administered, and Gradebook data was evaluated to measure whether the Technology
Applications program accomplished the goals set by the district for the 2014-2015
school year.
Goal 1 (80% of students will master the Technology Applications skills)
The student survey results show that students met the district expectations concerning
21st-century technology applications skills in six of the seven questions. The one
question where students fall below the districts Technology Applications skills
expectation revolves around making online purchases. Due to the sensitive nature of
making online purchases, a confounding variable may be at play here because some
caregivers may not entrust credit card information to 8-11 year old children.
The Gradebook data was analyzed to determine the numbers of students that mastered
the Technology Applications skills course. The Gradebook data show that cumulatively,
the Technology Applications students mastered the Technology Applications skills.
When analyzed by grade level, however, the third-grade students did not meet the 80%
threshold. Only 75% of the students were considered to have mastered the Technology
Applications skills. Fourth and 5th-grade students both mastered the Technology
Applications skills. The Gradebook data is consistent with the student survey data and
accurately reflects skill attainment with the 80% threshold established by the district.
The lower numbers of the third-grade students will need further evaluation to determine
potential causes for the lower performance.

P a g e | 10
Goal 2 (80% of students will utilize the 21st-century skills learned in the
Technology Applications course to the core content classroom at least monthly.)
The student survey results show that 84% of students report the use of technology in
the classroom at least once a month. This number is consistent with the districts
expectations that 80% of students will use technology in the content area classroom on
a monthly basis. However, when the questions became more detailed about specific
types of technology use that are in line with 21 st-century technology skills, the scores
were drastically different. There were six questions that addressed specific 21 st-century
skills technology use. Over 50% of students reported that their use of these skills in the
classroom was less than once a month; and for all but two of these questions, over 70%
of students reported that that they do not use these skills in the classroom on a monthly
basis. An analysis of the data reflects that student technology use in the classroom is
not in line with the school districts goals and objectives.
The teacher survey was primarily focused on 21st-century technology skills use in the
core content classroom. The results of the survey show a similar pattern to what was
reported by students. Teachers reported that 83% of students used technology in the
classroom. But when asked about specific types of 21 st-century skills related
technology use, teachers reported the frequency of use in the classroom as less than
monthly. Only two of the 18 questions reflected the frequency of use of 21 st-century
skills in the classroom at greater than 25%. Additionally, fourteen of the eighteen
questions contained a response of never by more than 50% of the teachers surveyed.
Based on the survey results, the Grade 3-5 Technology Applications program has failed
to meet the districts goal calling for 80% of students to transfer the 21 st century skills
knowledge from the Technology Applications classroom to the content area classrooms.
The data show that teachers report technology use, but the type of use is inconsistent
with the districts goals. Further evaluation of teachers, lesson plans, and campusspecific goals and objectives will need to be done to understand why this is occurring.
Overall
The Technology Applications program met the districts 21 st-century skills technology
skill acquisition goal, but did not achieve the goal calling for the transfer of these skills to
the core content classrooms. The data show that the Technology Applications program
is effective at teaching the skills that the district believes are essential for students.
However, the skills are not transferring to use in the core content classroom. This
suggests that teacher behavior and/or training may be an issue with this transfer. An
evaluation focused on how teachers use technology, teachers comfort with technology,
teacher knowledge of 21st century skills, how teachers plan lessons, and campus goals
and objectives should be conducted to gather the pertinent information about potential
ways to improve the program so that it can meet the districts goals and objectives.

P a g e | 11

APPENDICES

Appendix A: Student Survey

How often do you use computers in the classroom?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often do you get feedback (on your work) from others in the classroom?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often do you receive online feedback from someone other than your
teacher?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often do you use video or chat applications?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

P a g e | 12

How often do you use web tools to receive online information?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often do you write online?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often do you work on an e-portfolio?
a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often are you taught how to cite online information?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I am taught how to share information about myself online.

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I am taught how to act respectfully online

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

P a g e | 13
I am taught how to respond to online bullying.

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

For me, sending an email is

a) Very easy b) easy c) Moderately Difficult d) Difficult e) Impossible

I find attaching a printer to be

a) Very easy b) easy c) Moderately Difficult d) Difficult e) Impossible

I find creating a spreadsheet to be

a) Very easy b) easy c) Moderately Difficult d) Difficult e) Impossible

I find sending a text message

a) Very easy b) easy c) Moderately Difficult d) Difficult e) Impossible

I find making online purchases

a) Very easy b) easy c) Moderately Difficult d) Difficult e) Impossible

P a g e | 14

I find viewing my friends photos or videos online

a) Very easy b) easy c) Moderately Difficult d) Difficult e) Impossible

Appendix B: Teacher Survey

How often do students use technology in the classroom?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

How often do you ask students to receive digital feedback from other students in the
classroom?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

P a g e | 15

How often do you ask students to receive online feedback from someone other than
you?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to use chat or video chat applications?

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask students to use web tools to receive online information

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask students to write online

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to create e-portfolios

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

My students collaborate online with classmates

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

P a g e | 16

My students collaborate online with me

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to collaborate online with students at other schools

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to store documents online

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to use digital tools to collect and analyze data

b) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to conduct experiments or perform measurements using digital tools

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

My students conduct research using digital tools

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

P a g e | 17

My students use digital tools to identify and solve real-world problems

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to create and upload art, music, movies, or webcasts

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to create animations, demonstrations, models, or simulations

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

My students develop or present multimedia presentations

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

I ask my students to use a digital camera (photo or video)

a) Almost daily b) Weekly c) Monthly d) Every few months e) Never

P a g e | 18
Appendix C: Student Survey Email to Teaching Assistants

Technology Applications teachers,

Beginning Tuesday, the 6th of April, your 3 through 5th grade students will take the
Technology Use survey. Selecting the link now will take you to a page that reflects that
the survey is not taking additional responses at this time. However, it will be open on
Tuesday and it will remain open until Friday the 17th. Please confirm that the survey is
active Tuesday morning prior to giving access to students. If there is a problem you
should notify Nicolas.
Please have the students complete the survey by Friday, the 17 th of April. This survey is
mandatory for all of your students.
Thank you
Nicolas Hernandez

P a g e | 19
Appendix D: Survey Email to Teachers

Teachers,
Please Take 15 minutes to help your school improve use of technology within the
learning and teaching environment.
This is an anonymous survey regarding your technology use. It will take
approximately 15 minutes to complete. (Participation is mandatory.)
Your answers are very important! They will help your school choose the right technology
for your classrooms.
There are no right or wrong answers. Everyone will have a different set of answers to
these questions, depending on his/her experience.
Thank you for your time! Select the link below to start the survey.
Nicolas Hernandez