Evaluation Plan

Stephanie Fong, Sakinah Hunafa, Jenna Jenkins, Spencer Landis

FRIT 7237: Evaluation of Educational Needs and Programs

October 24, 2017

Implementation Evaluation:
 Implementation Data Sources
Evaluation Questions

P1: Were the initial Initial Experience: Data regarding the initial experience could be gathered through professional
learning workshop observations. These observations would determine time accuracy with
experience and follow activities as well as accuracy of participants. They would ensure proposed objectives were covered
during the professional learning activities. Gathering attendance information of teachers and staff
up activities would help to determine actual participants. A further survey could be done to determine this with
teachers confirming times and dates of workshop attendance along with open ended comments on
implemented as regarding if changes occurred. The “Summary of Project activities” table in the proposal details
activities and the dates chosen for completion. This can help determine activity implementation as
planned? stated in regards to time and content covered. Follow up Activities: Data regarding follow up
activities could be gathered through observation of the professional learning workshops to
determine that activities are going as scheduled. Data regarding the electronic communication
activities could be gathered through survey to see if they were implemented as well as the
meetings and the Project Debrief meeting.

P2: What is the Initial program activities: This data can be collected through a focus group of the participating
teachers discussing the useful activities for construction and implementation of the module.
quality of the initial Teachers would also discuss whether or not the activities provided enough practice using the
NSSM to incorporate them into their PBL modules correctly and efficiently. Finally, they would
program activities? discuss technology introduction for the modules. A survey with specific closed and open ended
questions regarding teacher response to the quality of the activities including teacher opinion on
future improvements would be useful too. Teachers could rate communication between the staff
and teachers during the continued module creation. The survey could also determine whether
teachers felt the small group activities with other teachers had sufficient participation to be
helpful. Another question posed could be to ask if teachers noticed the UBD model in use during
the workshops and presentations given. The October 2011 survey completed by the Braxton
County math teachers provides more data for a starting point of teacher NSSM knowledge. This
allows for a comparison of pre-program knowledge and post professional learning knowledge.

P3: Who are the The participants are math teachers in Braxton County, Georgia. Braxton County was identified as
an eligible system for the grant program that focuses on the application of New State Standards for
program participants Mathematics in the creation and use of mathematics problem-based learning modules. Teachers
will be recruited from a pool of educators who will be teaching math at the elementary, middle, or
and how were they high school levels in Braxton county during the 2015-2016 school year. The project will focus
initially on educators teaching middle school grades, but any mathematics teacher in Braxton
recruited? County is welcome to participate up to full capacity of 16 participants. Braxton County’s School’s
Director of Professional Learning, Ms. Carla Lawton, will assist The Project Director in recruiting
participants. Email and paper mailing will be used to advertise the project. This data can be
collected through a teacher survey asking participants if they were recruited or made aware of the
opportunity to participate through email, paper mail, or personal contact.
P4: What is the The follow up and support activities provide many opportunities for participants and faculty
members to share ideas, ask questions, discuss results, and provide feedback. Once PBL modules
quality of follow up are finalized and implemented teacher participants will conduct self-evaluations of the
implementation of modules and share the results with project staff members. Great Southern
and support University faculty members will conduct a mid-implementation meeting with teacher participants
to discuss questions and concerns related to the implementation of the PBL modules. There will
activities? also be a final meeting in which teacher-created self-evaluations, implementation, and best
practices will be discussed. Following the workshop, participants and project staff will
communicate through electronic tools such as email and wikis. Wikis allow for discussion of the
project components and sharing of information during the Fall semester when the modules are
implemented. When the project is finished, public areas of the wiki will be used to share the PBL
modules that were created with the public. Interviews and/or surveys can collect data to determine
if participants felt well supported, if questions were answered in a timely manner, and if the
support activities were meaningful.

Summative Evaluation:
Outcome Evaluation Questions Outcome Evaluation Data Collection Activities

O1:To what extent were teachers
able to develop PBL modules that
were connected to local business and
industries, aligned with NSSM, and
incorporated appropriate uses of
technology?(Objectives 1, 2, and 3)
Data to determine the usefulness and NSSM alignment of PBL modules could be
collected in the following ways:
1.The proposal stated that the PBL modules will be available for viewing on Wiki
after completion. The modules can be viewed with a checklist of important aspects
that must be included.
2.Faculty communication with teachers to complete the modules to ensure that they
meet requirements along with rubrics.
3.Teacher interviews by the faculty members to ascertain how well teachers were
able to develop their PBL modules. Specific questions regarding NSSM alignment
and technology use will further assist in gathering data. This data will help decide
if changes should be made. Further questions regarding teacher understanding of
PBL will determine any possible workshop modifications to improve teacher
knowledge.. Finally, a survey can be conducted to ask questions regarding Wiki
use for the PBL modules. These questions will help to determine whether teachers
came upon some issues with the site such as ease of use or the technology. This
could include computer issue such as Internet connectivity issues. This data will
help to see if Wiki should be used in future as well as the technology incorporated.

O2:To what extent were teachers
able to implement and evaluate
those modules? (Objective 4)
Teacher participants will implement their finalized PBL modules with their
students and conduct a self- evaluation of the implementation. Self-evaluation
results and student work will be used to evaluate the success of the PBL module
and identify areas for improvement.

Objective Indicator Data Source

1. Create PBL Modules Modules draw on content and -Post-Project Survey
based on local businesses and processes from local businesses -Observation/Participation in Professional Learning
industries and industries Workshops

2. Create PBL Modules Module content, tasks, and -Observation/Participation in Professional Learning
addressing NSSM assessments are aligned with Workshops
appropriate NSSM -Wiki Checklist, Self-Evaluation

3. Integrate technology into PBL modules contain activities -Post-Project Survey

PBL experiences that effectively use technology -Wiki Checklist, Self-Evaluation
-Observation/Participation in Professional Learning
Workshops

4. Implement and evaluate Record of implementation -Post-Project Self-Evaluation

PBL Modules Record of self-evaluation -Wiki Checklist, Self-Evaluation

Data Collection Schedule/Summary:

Both qualitative and quantitative data collection instruments were used resulting in a mixed-
methods design. Quantitative surveys were administered to teacher participants before the
project started and also at the conclusion of the project in late 2015 or early 2016. Surveys were
used to collect data in the areas of how participants were selected, level of support available to
participants, self-evaluations, and ease of technology use in creating PBL modules.
Qualitatively, mid-implementation meetings were held with teacher participants as well as a final
meeting. These meetings discussed areas including implementation questions/concerns, self-
evaluation, level of implementation support, and best practices. Additionally, faculty observed
and participated in the professional learning workshop where the design of modules occurred in
order to ensure local businesses were consulted and well-established instructional design models
were used.

Data set Date of collection Instruments Data collected by:

already
developed?

Self-Evaluations of Implementation Fall 2015 or Early 2016 Yes Great Southern Faculty

Mid- Implementation Meeting Fall 2015 Yes Great Southern Faculty

Final Meeting Fall 2015 or Early 2016 Yes Great Southern Faculty

Teacher Perceived Needs and October 2011 Yes Great Southern Faculty
Interest Survey
 Pre-Project Survey Mon, June 25 No Great Southern Faculty

Post-Project Survey Early 2016 No Great Southern Faculty

Observations and Participation in Mon, June 25 through No Great Southern Faculty

Professional Learning Workshops Friday, June 29