Running head: Topic Selection Communities in Schools 1

Program Evaluation: Communities in Schools

Whitney R. Langley

University of West Georgia

Topic Selection Communities in Schools 2

Abstract

With its inception in a small home in 1972, Atlanta, Georgia’s Communities in Schools Program

is now considered “the nation’s largest dropout prevention network with a proven track record of

positively impacting graduation rates (cisatlanta.org, n.d.).” Communities in Schools of Atlanta

(CIS) offers many programs to empower youth of all ages like the food pantry, foster care

support, community service opportunities, risky behaviors program, and reading programs. The

purpose of this paper is the select a topic for program evaluation by providing information about

CIS, some of school needs assessments, and the type of evaluation that would take place.
Topic Selection Communities in Schools 3

Program evaluation: Communities in Schools

Description of Program

Communities in Schools of Atlanta is a dropout prevention program that has partnered

with 65 metro Atlanta schools. They offer an array of services based on the needs and ages of the

students. CIS offers these services in a tiered approach with tier 1 for school wide services such

as the Books for Kids Program. Tier 2 students are in the target level programs with group

activities designed to support students with “similar barriers (about us, n.d.).” Some of the group

sessions may include anti bullying, mentoring, and tutoring. The individualized support

programs are in tier 3 where home visits and school check ins are necessary to ensure students

are performing at their highest potential.

At my school

I teach in an elementary school in Atlanta where the Communities in Schools program

has several services. Free books are provided several times a year to the entire school. Tier two

students are identified by their homeroom teachers to take part in the mentoring program. CIS

seeks to increase the attendance rates of these students and encourage students to make

behavioral gains. These students are pulled out of class for 30 minutes twice a week to

participate in the program. These students meet with the site coordinator from September

through the end of the school year in May.

Major stakeholders

Posavac (2016) describes stakeholders as “the people who are personally involved with

the program, derive income from it, sponsor it, or are potential recipients of the programs

services (Posavac, 2016, p. 28).” The stakeholders involved with Communities in Schools are the

students, teacher, parents, and community at large. The program has a direct impact on the
Topic Selection Communities in Schools 4

students because they are the primary recipients of the services, but the effects extend to their

teachers and parents. The purpose of this evaluation is for a graduate class program, so these

primary stakeholders are not expecting anything from this program evaluation.

Process Evaluation

The best type of evaluation for the Communities in Schools Program is the process

evaluation. This focuses on the fidelity of interventions such as the usage of proper strategies, the

correct dosage, and any inefficiencies that hinder the delivery of services (cyfar.org, n.d.). This is

an established program so I want to focus on examining the proposed outcomes. The only role I

play in the program is to select participants and provide any differences that I notice in

academics and behavior to the school CIS coordinator. In previous years, this has been a passive

role where the coordinator came to me with a list of students that received services the previous

school year and said that the caseload was full.

For the purposes of the program evaluation, I would be considered an external evaluator.

Though I work in the same school building, I am not employed by Communities in Schools

Atlanta. Not having this connection allows me to have an impartial view of the processes

observed. Posavac (2016), when describing an external evaluator says that they “might not have

the same level of commitment to working for program improvements (p. 17),” though I want to

program to be effective so that students are successful.

This type of program evaluation is one that would be requested by the superintendent of a

school system in order to determine whether or not the Communities in Schools Program is

actually increasing student graduation rates while concurrently reducing dropout rates. Since CIS

has been a long standing partner with Atlanta Public Schools, it would benefit the stakeholders in
Topic Selection Communities in Schools 5

the school system to get an overall update on the effectiveness of Communities in Schools. This

evaluation would address the questions:

1. Which schools in the Atlanta Public Schools System are partnered with Communities

in Schools?

2. What services are being offered at the sites?

3. How do graduation rates for students participating in the CIS program compare to

those of previous years?

4. Teachers- Do you think that the CIS Atlanta Program has positively impacted your

classroom?

Proposed Methodology

The participants will include Atlanta Public Schools high school students who are

enrolled in the Communities in Schools Program. Of the 15 total high schools in the Atlanta

Public Schools System, nine of the schools are partnered with the CIS program (About us, n.d.).

The number of students in each school varies as the CIS team reevaluates student’s needs. Most

of the identified students are referred to the program from school counselors or teachers.

Anonymity and confidentiality is promised to all participants. No incentives will be provided

within this examination.

Design and Methodology

The design of this evaluation will be mixed method in nature. Quantitatively, this

evaluation will be a documentation review. With its primary goal to see how a program operates

without interrupting the program, a documentation review will supply historical information that

will showcase the effectiveness of Communities in Schools of Atlanta.

Topic Selection Communities in Schools 6

The data for this program evaluation will come from two sources. The first is the

graduation data for students who participated in the program. Question three, “How do

graduation rates for students participating in the CIS program compare to those of previous

years?” Will be answered by examining these records. This data will show the last four years of

graduation rates in order to establish a trend.

The second source of data for the evaluation will come from a teacher support study that

is designed by Communities in Schools Atlanta to gather the teachers’ perceptions of

effectiveness. This survey will address question four, “Do you think that the CIS Atlanta

Program has positively impacted your classroom?” The survey provides an opportunity for

teachers to provide information about themselves, the needs of the school before partnering with

CIS, and their thoughts about the program (see Appendix A).

Procedure

Prior to data collection, approval from the Institutional Review Board at the University of

West Georgia. Once approved, I will submit a research study application packet to the Atlanta

Public Schools Office of Research Evaluation (Appendix B). Data collection will begin after all

parties have approved the program evaluation.

Data will commence at the beginning of the fall 2018 academic semester. An informed

consent form will be sent to the teachers who are working at schools that partner with

Communities in Schools of Atlanta. Because student data and graduation rates may contain

sensitive information, they can be collected without additional consent once Atlanta Public

Schools grants access to this information.

This program evaluation is a documentation review so data will be collected and

analyzed throughout the semester. The surveys will be sent to the teachers who taught at the
Topic Selection Communities in Schools 7

school during the prior school year (see Appendix A). Their perceptions are most valuable

because a teacher who is new to the school would have a limited interaction with the CIS

mentors.

Proposed Analysis

Several types of analysis will be used in this program evaluation because both

quantitative and qualitative methods were used to collect data. Quantitative methods will be used

to analyze graduation rates and teacher responses on survey questions. Statistical analysis will be

used so that stakeholders can clearly understand the findings presented in picture and percentage

form (Posovac, 2016, p. 37). There is a space for open ended responses, which are optional to

teachers, which would be analyzed and grouped into themes.

Budget

The evaluation for this program will be free of charge as this is an assignment for a

graduate level class. The complete budget is outlined in Table 1 that follows.

Table 1- Evaluation Budget

Budget Category Amount Description

Printing and Copying $30 Consent forms
2 copies per teacher
100 teacher participants
$0.15/ copy
Auto Mileage $60 $0.30/mile
Travel to partner schools 200miles
Evaluation Fee $1,500 5 hours/week at $30 per hour=$150

$150/week over 10 weeks = $1,500

Total $1,590
Topic Selection Communities in Schools 8

Evaluator’s Qualifications

The evaluator holds a Bachelor of Arts in Early Childhood Education from Spelman

College in Atlanta, Georgia. She also has a Master of Education with a Mathematics focus from

Georgia State University. She is currently pursuing a Specialists Degree in Instructional

Technology at The University of West Georgia. She has experience in designing quantitative and

qualitative research projects. She worked as a graduate assistance while at Georgia State

University where she evaluated Georgia Baptist Children’s Home services.

Dissemination Plan

Once all of the data has been collected and analyzed, a report of findings will be drafted

and submitted to the major stakeholders at each Atlanta Public Schools high school that has

partnered with Communities in Schools of Atlanta. The findings will be presented at a monthly

principals meeting. The document will also be sent via email to each principal for further

reference. This will also grant principals the opportunity to disseminate these findings to relevant

staff members and personnel. A complete timeline of the dissemination plan is listed in Table 2

which follows.

Table 2 Dissemination Timeline

-Conduct needs assessment

Month 1 -Submit IRB approval to university
-Submit Research study application to Atlanta Public Schools

-IRB approval from the University of West Georgia

Month 2 -Approval of research from Atlanta Public Schools
-Collect informed consent from teachers

-Collect graduation data from participating schools

Month 3 -Comparative data for target students and school

-Distribute reflective surveys to teachers

Month 4 -Aggregate data
Month 5 -Report findings to stakeholders
Topic Selection Communities in Schools 9

References

About Us. (n.d.). Retrieved September 1, 2018, from https://www.cisatlanta.org/about-us

Our School Based Sites. (n.d.). Retrieved September 26, 2018, from

https://www.cisatlanta.org/cis-school-sites .

Different Types of Evaluation. (n.d.). Retrieved September 2, 2018, from

https://cyfar.org/different-types-evaluation#Process

Posavac, E. J. (2016). Program evaluation: Methods and case studies. New York: Routledge.
Topic Selection Communities in Schools 10

Appendix A

Post survey for teachers

Teacher Instructional Assistant Other

Are you Currently a:   

How many years of experience do you have? ____________________________

How many years have you taught at this school? _________________________

Do you teach students who receive services provided by CIS? _______________

How familiar are you with the services CIS provides in your school?
Not Familiar Somewhat Familiar Very Familiar
  

Please respond to the following statements using the five point scale provided. I am able to be
a more effective teacher because CIS helped me…
Strongly Strongly
Disagree Unsure Agree
Disagree Agree
Improve student attendance     
Decrease student tardiness     
Improve student behavior     
Decrease high risk behavior
outside the classroom
(alcohol, drugs, violence,     
gang activity)

Decrease student suspensions     

Improve student attitudes
towards learning     

Students become more

engaged in learning
    

Students resolve distracting

social and emotional issues
    

Students to come to class

better prepared to learn     

Students acquire resources

needed for learning     
Topic Selection Communities in Schools 11

Please respond to the following statements using the five point scale provided.
Strongly Strongly
Disagree Unsure Agree
Disagree Agree
I am better able to improve
student achievement because
of the overall services CIS     
provides.

Part of my overall
effectiveness as a teacher can
be attributed to positive     
differences CIS has made in
my school.

My job would be more

difficult without the presence     
of CIS in my school.

I enjoy teaching more

because CIS has helped
improve the overall climate
    
in the school.

Because of CIS I have

noticed that the parents of my     
students are more involved.

Because of CIS I have

noticed that more community
resources are being brought
    
into my school.
To what extent is each of the following a problem in this school:
Moderate
Serious problem Minor problem Not a problem
problem
Student pregnancy    
Students dropping out    
Student apathy    
Lack of parental involvement    
Poverty    
Students come unprepared to
learn    

Poor student health    

Poor student attendance    
Student behavior problems    
Topic Selection Communities in Schools 12

Appendix B

Office of Research
and Evaluation
130 Trinity Ave., SW, 6th Floor | Atlanta, Georgia 30303 | P: 404.802.2672 |
F: 404.802.1601

Dear Prospective Researcher,

Thank you for your interest in conducting research in the Atlanta Public Schools (APS).

It is the goal of APS and the Office of Research & Program Evaluation to participate in research

efforts that will substantially benefit APS, its students, and/or staff.

Each year, APS receives a number of requests to participate in research investigations.

While we are eager to participate in research that will substantially benefit our system, students,

and/or staff, it is not feasible or desirable for APS to participate in every proposed research

project. Thus, researchers are required to submit a Research Study Application for proposed

research projects. The Research Study Application is designed to provide the review committee

with sufficient information in order to reach a decision about the appropriateness of APS

participation in the research project. Your application will be evaluated based on the following

criteria:

 Use of APS staff and student time and resources

 Value of the research study to APS
 Alignment of proposed research with APS strategic goals and objectives
 Research design: research question and purpose, data collection methods, analytical
approach, tools and methods
 Level of data extraction from central records
 Ability to comply with human research informed consent procedures
 Thoroughness of research plan and application
Topic Selection Communities in Schools 13

Please carefully review the Research Application Resource Guide available on the Office

of Research and Evaluation website prior to completing this application. The Resource Guide

provides detailed information about the application submission process including the timeline for

when proposals will be reviewed and when notifications of committee decision will be sent out.

Questions about the application process and/or application materials should be directed to

404.802.2672 or research_screening@atlanta.k12.ga.us. Again, thank you for your interest. We

look forward to receiving your research application.

Office of Research and Evaluation

Topic Selection Communities in Schools 14

Atlanta Public Schools

Research Study Application Cover-page

Name of

Researcher:_________________________________________________________________

University/Institution/Organization:_________________________________________________

____

Title of Study:

________________________________________________________________________

This checklist is to assist you in submitting a complete research study application packet to the

Atlanta Public Schools Office of Research and Evaluation, please include this page with your

application. Please note the committee will not review an incomplete application packet. Please

submit the following materials electronically.

Basic Elements to include:

Completed Research Study Application
Copy of institution's IRB approval letter or letter of support from graduate

program. Must be
Consent provided
and on official
assent forms (basedstationary.
on research population)
Copies of data collection instruments (interview protocols / survey instrument

/ observation rubric)
Recruitment material (flyers, recruitment letters, enrollment forms etc.)
Topic Selection Communities in Schools 15

Researcher Assurances

By my signature as the principal Investigator/researcher, I certify that all information within the
accompanying application is true, complete and accurate to the best of my knowledge. I assure that I
will comply with all Atlanta Public School guidelines and researcher expectations.

Signature of

Researcher:___________________________________________________________

Date:__________________
Topic Selection Communities in Schools 16

Research Study Application

SECTION A: APPLICATION INFORMATION

Title of Research Project:

Researcher’s Full Name(s): Title/Position:

University/Institution/Organization:
Mailing Address:
Email Address: Daytime Phone:

Is this the final version of the Yes

Date Submitted:
proposal? No
Projected Start Date (mm/yyyy): Projected Completion Date (mm/yyyy):
I have reviewed and understand:
Yes
Research Application Resource Guide No

The research is related to a:

Doctoral Study Masters Study Class Project
Professional Project (Non-profit organization) Professional Project (For-profit organization)

Have you included a copy of your Institutional Review Yes No

Board (IRB) approval form with your application?
Has any APS staff member already agreed to assist you
Yes No (if yes, please
upon approval of this study? (Please note that this agreement is not
provide documentation of the agreement)
binding)

Graduate students only: Please fill out the information below

Atlanta Public Schools reserves the right to contact university faculty associated with a proposed
research project. Advisor contact information is required for ALL graduate student projects.
Advisor’s
Title/Position:
Name:
E-mail
Daytime Phone:
Address:
Mailing
Address:
Yes
Have all advisory/regulatory committee members formally approved this research?
No

Atlanta Public School Employees Only: Please fill out the information below

School Principal' Name: Title/Position:

Email Address: Daytime Phone:
Topic Selection Communities in Schools 17

Mailing Address:
Has the principal formally
Yes No (if yes, please provide documentation of the agreement)
approved this research?

SECTION B: STUDY SPECIFICATIONS

Study includes Elementary Middle
High School
School School
participants at: Charter Administrative
School Alternative School Office
Check all that apply.
Other (please explain):
Please list the specific schools or APS locations you anticipate participating in your study.

Academic At-risk
College readiness
achievement students

Curriculum
English language
or Educational policy
learners
instruction or
leadership
Area(s) of Study:
Ethnic or Physical Professional
Check all that apply. cultural health or development
studies safety

Special Student Supplemental

education social or emotional programs
development
Teacher
professional Other(please explain):
practices
Type of Study
Quantitative Mixed-Methods
Qualitative
Relevance of Research to APS‐ Describe why the research is specifically relevant to APS and its
Strategic Plan and how it would substantially benefit APS, its students, and/or
staff. (Response should not exceed 200 words.)
Topic Selection Communities in Schools 18

Does study employ: Check all that apply.

Non-school personnel surveys (e.g., surveys of District level staff, like superintendents) Yes No
All surveys must be attached.
School administrator surveys. All surveys must be attached. Yes No

Teacher surveys. All surveys must be attached. Yes No

Student surveys. All surveys must be attached. Yes No

Parent surveys. All surveys must be attached. Yes No

Non-school personnel interviews Yes No

School administrator interviews Yes No

Teacher interviews Yes No

Student interviews Yes No

Parent interviews Yes No

Classroom observations Yes No

Audio recording of APS staff Yes No

Audio recording of APS students Yes No

Video recording of APS staff Yes No

Video recording of APS students Yes No

SECTION C: DATA RECORDS REQUEST

Does study access, require, or record: Check all that apply.

De-identified student-level information Yes No

De-identified staff-level information Yes No

Aggregated student-level information Yes No

Does study access, require, and/or record: Check all that apply.
Aggregated staff-level information Yes No

School-level information Yes No

Data Elements: What data elements are you requesting? (Skip this section if you are not requesting
any student record data)

Attendance Demographics Discipline Grades

Student Promotion

Graduation Teacher promotion Test/Assessment scores

Other (please explain):

Data Pull
One time only Annually Every semester Graduation
Schedule:
Other (please explain):
Topic Selection Communities in Schools 19
Please provide a detailed description of any existing data that you are requesting. Please avoid

the use of terms such as "academic information" and "test scores." Include the academic years

that you are requesting and at what point during the research study this data will be needed. For

example: "For the 2010-2011 school year, please provide de-identified student level data

(researcher will provide Student ID numbers, gathered from the permission form) including:

ethnicity, gender, school number, days attended / enrolled by semester, and school year GPA.”
Topic Selection Communities in Schools 20

Instructions: Please answer each question below within this document.

Relevant documents (e.g., questionnaires, consent forms, IRB approvals, etc.) should be

sent electronically with this application.

1. Purpose and research questions: Specify purpose of research study and the primary
research questions to be addressed. (Response should not exceed 500 words)

2. Rationale for Research-

Provide a brief description of the theoretical background for the study,
including references, where appropriate. (Response should not exceed 500 words.)

3. Research Design- This section should include a summary of your research or

evaluation design, including statistical analysis procedures to be applied. Please
include your data collection procedures and activities across the different study groups in
sufficient detail for a review to fully understand the implementation plan. Also include a
description of the sample and proposed analyses. (Note: The technical soundness of your
research methodology, measures, and analysis will be considered in the review process)
Please attach all surveys and assessments you propose to use during the research.

4. Timeline: Describe your timeline for the research study, include when you plan to
analyze and report on the data.

5. How much time will each of the various sets of participants be required to commit
to this study?

6. Describe any existing data that you will need from the school system (e.g.,
demographics, test scores) and how this will help you answer your research
questions.

7. Identify any ethical or privacy issues that may be of concern to APS or parents, and
explain how you have addressed them.
Topic Selection Communities in Schools 21

8. What is your plan to share and disseminate results? -Describe with whom and how
you will share your results.

9. Participant consent: Researchers must obtain written permission from the student
participant’s parent/guardians using a Parent/Guardian consent form. Please attach a copy
of the consent forms you will us to obtain permission from all study participants,
including parents if students are involved.

10. Confidentiality: If central data is requested, researchers will only be provided with de-
identified student data. Please explain how you will maintain the confidentiality of your
study participants. Specifically, who will have access to this data? For what purposes will
research data be shared? What will you do with the data after the analyses are complete?
What security measures will you take with the data?

11. Human Subjects: How do you plan to protect human subjects during this research?

12. Compliance: Are you prepared to comply with all the terms in the Resource Guide? Yes
No