Assignment Five Final Report FRIT 8435 Fall 2011 Toni Bowers, MaryFrances Pinson, Kristin Sapp,

Walker Weekly Quiz Evaluation Report Table of Contents

I.

Executive Summarypg 3 Purposepg 3 Information collection..pg 3 Findingspg 4 Judgments...pg 4-5 Recommendations.pg 5 Introduction..pg 6

II.

III.

Focus of the Evaluation.pg 7

a. Description of Evaluation Object..pg 7 b. Evaluative Questionspg 8 c. Information Needed to Complete Evaluation.pg 8

IV.

Overview of Evaluation Plan and Procedures.pg 9 Presentation of Results.pg 10

V.

VI. VII.

Conclusions and Recommendationspg 14 Appendices...pg 18

I. EXECUTIVE SUMMARY: Purpose: The purpose of this evaluation is to analyze the Walker Middle Schools Weekly Walker Quiz Program and its impact on Georgias Criterion Reference Competency Test (CRCT) scores. The evaluation should show the significance in the implementation of the Weekly Walker Quiz Program for the entire school year verses only using the program for half of the school year, or not using the program at all. The program evaluation results should explain the advantages of continuing the use of the Weekly Walker Quizzes. The following evaluation questions were posed to aid in the determination of the significance of implementing the Walker Weekly Quiz Program: Which is a better time frame for the Walker Weekly Quiz to be administered, half a school year or for an entire school year? Does the Walker Weekly Quiz Program improve CRCT scores? Should the Walker Weekly Quizzes be counting as an actual grade in the classroom or only used as evidence of improvement?

Information Collection: A variety of data was collected and used to answer the evaluation questions. Interviews with Walker Middle School teachers and administrators were conducted

to examine the significance in the performance of the Walker Weekly Quiz Program in the classrooms. Teachers were able to share the pros and cons though their experience in previous years and the students results in their classrooms. Quantitative data analysis forms of CRCT scores, from the previous five school years, were observed. This data highlighted trends of the test scores through no implementation, half-year implementation, and full-year implementation of the Weekly Walker Quizzes.

Findings: Longitudinal data of CRCT scores from 2006-2011 was received from Walker Middle School records. The Longitudinal data included CRCT scores in all five subject areas and grades fourth through eighth. The data was combined to show a passing average for the entire student body in each subject area. The first half year of implementation of the Walker Weekly Quiz Program resulted in gains in all subject areas except Social Studies. Among these gains were significant gains of 10% in Math, 8.5% in Science, and 4% in Reading. After the next half year of program implementation, all subject areas showed significant gains. However, the CRCT scores after the full year implementation of the Walker Weekly Quiz program only proved to have small gains in Math, Science, and Social Studies with Reading and Language Arts scored both decreasing around

1%. The most significant difference in the percentage passes were shown in the two half years of program implementation.

Judgments: Based upon the evaluation of program data relating to students gains in CRCT percent passing and the Walker Weekly Quiz program implementation, this evaluation study concludes that there are several strengths shown by using the program. All areas reached or exceeded criteria standards in the first two years of program implementation for half of that particular school year. All subject areas gained at least 2% in the overall passing rate in the first year of the program implementation with the exception in the area of Social Studies. Significant gains, which exceeded criteria and standards, in overall CRCT passing rate for grades fourth through eighth. These results exceeded standards by over double the percentage needed to meet the criteria and standards. These gains continued throughout the second half year of the program implementation with all subject area scores passing rate increasing by at least 2% or more with some reaching near an 8% gain. On the other hand, these weaknesses were shown throughout the program Implementation. During full year implementation of the Walker Weekly Quiz program, CRCT passing rates did not prove to advance. During the full year

Walker Weekly Quiz implantation, only one subject area, Social Studies, met the criteria for proving the program affective. Reading and Language Arts showed a decrease in the passing percentage of all students during the full year execution of the program.

Recommendations: Based on the CRCT passing rate results, Walker Weekly Quizzes should be implemented in all schools in the Long County School district. However, Walker Weekly Quiz program should only be implemented for half of the school year from January to CRCT at the end of the school year. All teachers of core should administer Walker Weekly Quiz program once weekly in their classrooms. Students should take quizzes individual and teacher should review as a class. Also, quizzes should be scored to use as a measurement of student success and projection of success on the Criterion Reference Competency Test.

II. INTRODUCTION: In the spring of 2000, Georgia law required all elementary and middle school students in grades one through eight take the Criterion Reference Competency Test (CRCT). This test measures how well students obtain and use skills that are taught based on the Georgia Performance Standards. Since the first year the CRCT was given, administrators and teachers hope to see their students do well on the test and also see the test scores consistently meeting or exceeding the standards of the test each year. To aid in the success of test scores, schools will implement programs that reinforce student learning. In the 2008-2009 school year, Walker Middle School put into practice the Weekly Walker Quiz Program. This program was designed for half of the school year, that teachers give weekly quizzes to assess the standards being taught in the classroom. The schools administrators and teachers were pleased with the outcome of the program; the decision was made to implement the Weekly Walker Quizzes again for half of the 2009-2010 school year. However, for the 2010-2011 school year, school administrators tried something new; they implemented the program for the entire school year. This evaluation will examine Walker Middle School students CRCT scores when the Weekly Walker Quiz Program was not implemented, implemented during half of the school year, and implemented for a full school year. The study will provide information for Walker County School board members, the

superintendent, Walker Middle School administrators, and teachers about the significance of the Weekly Walker Quiz program and if the program should continue being put into action for a full school year, half a school year, or completely dissolved.

III. Focus of the evaluation a). The focus of the evaluation is the Weekly Walker Quiz Program to see if implementing this program improves CRCT test scores. The quizzes are given at the end of each school week over possible CRCT material covered in the subject areas of reading, language arts, math, science, and social studies. The program began back in the year of 2006 in grades fourth through eighth at Walker Middle School. The main goals of the program are to help students be better prepared for the CRCT and to improve CRCT scores for the Walker School System. Stakeholders at Walker Middle School requested this evaluation be completed to see if the Walker Weekly Quizzes are better to be implemented only half of the school year or for the entire school year. They also requested preexisting data to be reviewed and compiled to show CRCT scores over the past five years that included only implementing the program half the year and implementing the program for the entire year. By compiling this data, stakeholders inside the school system as well as stakeholders in the community could compare both sets of data. The data will be compiled into tables and line/time graphs. Each will be labeled with the school year and if the data was retrieved from half of the school year or the entire school year. Staffing for this evaluation will include an inside evaluator, data clerk, the testing faculty, and the lead administrator. The inside evaluator will interview the

principal and select members of the testing faculty. If needed the inside evaluator will enlist the assistances of grade level chairs to conduct the interviews with two members from the same grade testing faculty. To retrieve the data needed the inside evaluator will work closely with the data clerk and the principal. b). To provide the evaluation with guidance, two questions were decided upon. *Does the Weekly Walker Quiz Program improve CRCT scores? *Which is a better time frame for the WWQ to be administered, half a year or for an entire school year? The inside evaluator will meet with all stakeholders at the start of the evaluation to go over these questions thoroughly so that everyone involved will know which way the evaluation is set to proceed. c). Information to Collect: Data results before the Weekly Walker Quiz program was implemented Data results from half a year starting in the year 2006 going through present Data results from entire year starting in the year 2006 going through present Information to Analyze: Student improvement from before implementation, half a year, and the entire year

Information to Report: Student scores from before implementation, half a year, and the entire year Which method would show the most improvement on student CRCT scores: half a year or the entire year

IV. Brief Overview of the Evaluation Plan and Procedure

The evaluation at Walker Middle School will be conducted by an internal evaluator. This evaluator will consult with the lead administrator, faculty, and staff at the middle school about the student improvement score data. Interviews of the lead administrator and grade level chairs will be conducted by the internal evaluator. If other interviews need to be conducted, then the internal evaluator will instruct the grade level chairs to conduct the interviews. The internal evaluator will keep all stakeholders involved in the evaluation process by holding informal meetings and writing a mid-evaluation report. By consulting with the data clerk, the internal evaluator will be able to collect the data needed for before the Weekly Walker Quiz program was implemented, half a year implementation, and an entire year implementation. After all the data has been collected by the evaluator, the evaluator and the lead administrator will consult together to discuss the data results. The data results will be complied together into line graphs and tables to be read and comprehended more easily. From the data results, the internal evaluator and the lead administrator will draw a conclusion on which method showed the most student score improvement and to be implemented within the school system.

V. PRESENTATION OF EVALUATION RESULTS Presentation of Evaluation Findings: Longitudinal data of CRCT scores from 2006-2011 was received from Walker Middle School records. The Longitudinal data included CRCT scores in all five subject areas and grades fourth through eighth. The data was combined to show a passing average for the entire student body in each subject area. This data includes information throughout the different stages of the Walker Weekly Quiz Program. The results include data before and after the implementation of the program including stages of half year implementation and full year implementation.

Overall CRCT Passing Rate Walker Weekly Year Quiz Program Implementatio n Not Implemented Not Implemented Half Year Implemented Half Year Implemented Full Year Implemented 20062007 20072008 20082009 20092010 20102011 Reading CRCT Percent Passing Rate 79.4 82.8 86.86 90.86 89.5 Language Arts CRCT Percent Passing Rate 81.4 84.4 87.14 89.1 88.56 Math CRCT Percent Passing Rate 74.6 63.4 73.7 77.92 78.82 Science CRCT Percent Passing Rate 64.2 64.6 72.92 80.8 82.7 Social Studies CRCT Percent Passing Rate 83.2 71 66.5 72.72 75.72

Overall Points Increase / Decrease in CRCT Scores Walker Weekly Quiz Program Implementatio n Not Implemented Half Year Implemented Half Year Implemented Full Year Implemented Yea r Reading CRCT Percent Passing Differenc e Language Arts CRCT Percent Passing Differenc e Math CRCT Percent Passing Differenc e Science CRCT Percent Passing Differenc e Social Studies CRCT Percent Passing Differenc e

200 7 to 200 8 200 8 to 200 9 200 9 to 201 0 201 0 to 201 1

3.2

-11.2

0.2

-12.2

4.06

2.74

10.3

8.52

-4.5

1.96

4.22

7.88

6.22

-1.96

-0.54

0.9

1.9

This chart includes the overall point increase or decrease in each subject area for all grades at Walker Middle School. There were many point increase and a few decrease. There was a steady increase in Reading between years of 2007- 2010.

Each half year of the Walker Weekly Quiz implementation the Reading scores had an average increase of 4 points. However, during the one full year of implementation, there was a decrease. Language arts found close to the same results as Reading only with a 2 point increase during half year program implementation and a slight decrease during the first full year of implementation. On the other hand, Math had a significant change from decrease of 12 points before the Walker Weekly Quizzes to increase of 10 points during the first half year of implementation and 4 point increase during the next half year implementation of the Walker Weekly Quiz program. Nevertheless, there was less of an increase in CRCT test scores during the full year program implementation. The results for Science and Social Studies followed the same forgot with a greater overall passing percentage during the half year of program implementation than the full year of program implementation.

This graph illustrations the increase in the overall passing rate for each subject area. In the years of 2006-2007 and 2007-2008, the Walker Weekly Quiz Program was not implemented. During these years, there were significant decreases in the passing rate for Math and Social Studies. For the other subjects, there was little to no increase in the passing rate between the year 2007 and 2008. On the other hand, all subject areas passing rates showed significant increases in the year 2009 and 2010. These results include the years that the Walker Weekly

Quiz Program was implemented for the duration of half of the school year. However, in the year 2011, when the Walker Weekly Quiz was implemented for the entire year, there were small passing rate decrease in Reading and Language Arts and small passing rate increases in Math, Science, and Social Studies.

VI. CONCLUSIONS AND RECOMMENDATIONS: a. Criteria and Standards:

Guiding Question

Criteria

Standard

1. Does the Weekly Walker Quiz Program improve CRCT scores?

Teachers give one weekly walker quiz each week. Teacher reviews over correct answers with the students. Teachers average four weekly walker quiz grades to monitor student achievement. One year, teachers give one weekly walker quiz each week from January through April. The next year, teachers give one weekly walker quiz each week of the school year from August through April. Teacher reviews over correct answers weekly with the students. Teachers average four weekly walker quiz grades to monitor student achievement.

Given previous year's data without the use of the program, Student's CRCT scores will improve by at least 2%

2. Which is a better time frame for the WWQ to be administered, half a year or for an entire school year?

Students CRCT scores will improve 2% from year one, where weekly walker quizzes are administered for only a portion of the year, to year two, where walker quizzes are administered throughout the whole year from August-April.

b. Judgment about evaluation object

Based upon the evaluation of program data relating to students gains in CRCT percent passing and the Walker Weekly Quiz program implementation, this evaluation study concludes:

Strengths: All areas reached or exceeded criteria standards in the first two years of program implementation for half of that particular school year. All subject areas gained at least 2% in the overall passing rate in the first year of the program implementation with the exception in the area of Social Studies. Significant gains, which exceeded criteria and standards, in overall CRCT passing rate for grades fourth through eighth where shown in math with an increase of 10.3%, reading with an increase of 4.06%, and Science with an increase of 8.52% These results exceeded standards by over double the percentage needed to meet the criteria and standards These gains continued throughout the second half year of the program implementation with all subject area scores passing rate

increasing by at least 2% or more with some reaching near an 8% gain. Weaknesses: Full year implementation of the Walker Weekly Quiz program did not prove to advance CRCT passing rates. During the full year Walker Weekly Quiz implantation, only one subject area, Social Studies, met the criteria for proving the program affective. Reading and Language Arts showed a decrease in the passing percentage of all students during the full year execution of the program.

RECOMMENDATIONS: Walker Weekly Quiz program should only be implemented for half of the school year from January to CRCT at the end of the school year. Walker Weekly Quizzes should be implemented in all schools in the Long County School district. All teachers of core should administer Walker Weekly Quiz program once weekly in their classrooms.

 Students should take quizzes individual and teacher should review as a class. Quizzes should be scored to use as a measurement of student success and projection of success on the Criterion Reference Competency Test.

VII. APPENDICES: References

Long County Schools (2011). Longitudinal Data of CRCT Scores. Ludowici, Georgia. Long County Schools Systems. http://www.longcountyps.com.

Appendix A Walker Middle School Overall CRCT Passing Rate 2007-2011 Walker Weekly Year Quiz Program Implementatio n Not Implemented Not Implemented Half Year Implemented Half Year Implemented Full Year Implemented 20062007 20072008 20082009 20092010 20102011 Reading CRCT Percent Passing Rate 79.4 82.8 86.86 90.86 89.5 Language Arts CRCT Percent Passing Rate 81.4 84.4 87.14 89.1 88.56 Math CRCT Percent Passing Rate 74.6 63.4 73.7 77.92 78.82 Science CRCT Percent Passing Rate 64.2 64.6 72.92 80.8 82.7 Social Studies CRCT Percent Passing Rate 83.2 71 66.5 72.72 75.72

Appendix B Walker Middle School Overall Points Increase/Decrease in CRCT Scores 2007-2011

Overall Points Increase / Decrease in CRCT Scores Walker Weekly Quiz Program Implementatio n Not Implemented Half Year Implemented Half Year Implemented Full Year Implemented Yea r Reading CRCT Percent Passing Differenc e Language Arts CRCT Percent Passing Differenc e Math CRCT Percent Passing Differenc e Science CRCT Percent Passing Differenc e Social Studies CRCT Percent Passing Differenc e

200 7 to 200 8 200 8 to 200 9 200 9 to 201 0 201 0 to 201 1

3.2

-11.2

0.2

-12.2

4.06

2.74

10.3

8.52

-4.5

1.96

4.22

7.88

6.22

-1.96

-0.54

0.9

1.9

Appendix C Walker Middle School Student CRCT Passing Rate 2007-2011

Chart Created using http://www.chartgo.com/index.jsp

Appendix D Planning the Evaluation Reporting: Evaluation Audience for Report Question the Report Content 1. Does the Weekly Walker Quiz Program improve CRCT scores? School board members, Findings of students overall performance Assistant (scores) on superintende CRCT nt, testing prior to Walker School Quiz administrato Programs rs; implementat including ion principal, compared to vice students principals, overall and performance instructional after the coordinators programs implementat School ion faculty; including Language Arts, Math, Social Studies, and Science educators Report Format Oral briefing, executive summary, written report, and verbal presentation of results Findings included in the written report will be descriptions in the form of quantitative data analysis of CRCT scores Reporti ng Schedul e Ending of each school year Ending of each nine weeks; after nine weeks post-test are given Context for Presenting Report Consultatio n and discussion of introductory report with faculty and school administrato rs; followed by a final report submitted to them Briefing and introductory report to assistant superintend ent Executive summary submitted to board members

2. Which is a better time frame for the WWQ to be administere d; half of the school year or the entire school year?

Assistant Comparison superintende of students nt, overall achievement School on CRCT administrato during half rs, including year of principal, implementin vice g Walker principals, Quiz and program instructional verses coordinators students overall School achievement faculty, on CRCT including during Language whole year Arts, Math, of Social implementin Studies, and g Walker Science Quiz educators program School faculty; including Language Arts, Math, Social Studies, and Science educators School administrato rs; including principal, Progress of students on testing to date; Findings of students performance in all subject areas

Written report including quantitative data from multiple years of Walker Quiz program implementat ion Verbal presentation

Ending of school year

Ending of the school year; Presentation and Beginni discussions ng of the of report to school school year administrato rs, written report given; Beginning of the school year; Presentation of results at staff meeting; along with a one page summary of reports First staff meeting: One page summary given and discussed at staff meeting; written report given to all faculty. Second staff meeting:

3. Should the Walker Weekly Quizzes be counted as an actual grade or only used as evidence of improveme nt?

Written report comparing students grades on Walker Quizzes. executive summary of results verbal presentation

Ending of each Walker Quiz grading period; once every four weeks

vice principals, and instructional coordinator.

Verbal presentation and discuss of full written report Additional Staff Meetings: One page summaries of all students test scores on Walker Quizzes

Appendix E Evaluation: Guiding Questions, Criteria, and Standard

Guiding Question

Criteria

Standard

3. Does the Weekly Walker Quiz Program improve CRCT scores?

Teachers give one weekly walker quiz each week. Teacher reviews over correct answers with the students. Teachers average four weekly walker quiz grades to monitor student achievement. One year, teachers give one weekly walker quiz each week from January through April. The next year, teachers give one weekly walker quiz each week of the school year from August through April. Teacher reviews over correct answers weekly with the students. Teachers average four weekly walker quiz grades to monitor student achievement. Teachers give one weekly walker quiz each week. Teacher reviews over correct answers with the students. Teachers average four weekly walker

Given previous year's data without the use of the program, Student's CRCT scores will improve by at least 2%

4. Which is a better time frame for the WWQ to be administered, half a year or for an entire school year?

Students CRCT scores will improve 2% from year one, where weekly walker quizzes are administered for only a portion of the year, to year two, where walker quizzes are administered throughout the whole year from August-April.

5. Should the Walker

Weekly Quizzes be counted as an actual grade or only used as evidence of improvement?

When, four quiz scores are averaged, over 80% of the class should score a passing score of 70 or higher.