Evaluation and Program Planning 65 (2017) 156–162

Contents lists available at ScienceDirect

Evaluation and Program Planning

journal homepage: www.elsevier.com/locate/evalprogplan

Devil is in the details: Using logic models to investigate program process T

⁎
David J. Peyton , Michael Scicchitano
University of Florida, United States

A R T I C L E I N F O A B S T R A C T

Keywords: Theory-based logic models are commonly developed as part of requirements for grant funding. As a tool to
Process evaluation communicate complex social programs, theory based logic models are an effective visual communication.
Logic model However, after initial development, theory based logic models are often abandoned and remain in their initial
Special education form despite changes in the program process. This paper examines the potential benefits of committing time and
Teacher preparation
resources to revising the initial theory driven logic model and developing detailed logic models that describe key
Personnel preparation
activities to accurately reflect the program and assist in effective program management. The authors use a
funded special education teacher preparation program to exemplify the utility of drill down logic models. The
paper concludes with lessons learned from the iterative revision process and suggests how the process can lead to
more flexible and calibrated program management.

Social programs are complex. Regardless of their scope, they require
effective management of interactions between several groups within
multiple environments to achieve their intended goal. A necessity of
effectively managing social programs is clarity in the vision (why) of
the program and the process (how) to reach its goal. The clarity of how
and why should extend beyond program managers, reaching all levels
of stakeholders down to those carrying out the day-to-day activities of
the program. The use of logic models to crystalize the vision and pro-
cess of a program has become common place for more than twenty
years (Kaplan & Garrett, 2005). A notable function of logic models are
their use as a graphic representations of how a program theoretically
should work, given optimal conditions (McLaughlin & Jordan, 1999;
Silverman et al., 2009; W.K. Kellogg Foundation, 2004). Moreover, the
wide acceptance in the research literature and application in the field,
has led to their routine incorporation into funding proposals (both
private and public). This rather recent development should be wel-
comed, as logic model development shows substantial promise in en-
couraging thoughtful program design. However, despite the require-
ment, prior research suggests that the logic model process rarely moves
beyond the initial theory based logic model generated at the outset of
the program and may not serve as functional tool to support program
management (Kaplan & Garrett, 2005; Millar et al., 2001; Newton,
Poon, Nunes, & Stone, 2013).
The importance of using a logic model as a tool to develop the
program theory or theory of action to support program management
has been well documented (Cooksy, Gill, & Kelly, 2001;
McLaughlin & Jordan, 1999; Patton, 1997). However, a common mis-
interpretation is that developing an initial theory driven model or
outcome based logic model (W.K. Kellogg Foundation, 2004) is the end
of the process, rather than a starting point. Just as social programs will
surely change and recalibrate in response to changes in populations and
environments (Millar et al., 2001), logic models should be revisited to
restore clarity, reflect changes, help to maximize program efficiency,
and orient stakeholders to the path ahead. To this end, previous work
by Millar et al. (2001) suggested a paradigm shift in the utility of logic
models, shifting from an evaluation perspective to one focused on
successful program management. We employ their innovation as an
access point to introduce our drill down1 approach. Specifically, Millar
et al. (2001), suggest shifting logic models from theory driven models
anchored to complex outcomes that are likely out of the control of
program managers to models that have greater utility by identifying
critical interactions necessary to achieve more attainable outcomes, “to
do this we have to reverse the usual manner in which logic models are
developed” (p. 75). We propose that this view of logic models to be of
greater service to programs highly dependent on stakeholder interac-
tions, such as teacher preparation programs (Table 1).
We draw on previous work around teacher preparation reform to
inform our logic model discussion. Specifically, prior research on per-
sonnel preparation grants (to be discussed in detail later) included
surveys that tangentially spoke of barriers to implementation,

⁎
Corresponding author at: 352.273.4244 Norman Hall, Rm. 1403, PO Box 117050, University of Florida, Gainesville, FL 32611, United States.
E-mail address: peyto788@ufl.edu (D.J. Peyton).
1
The term “drilling down” is a phrasal verb that describes the process of examining something in more depth. This verb is receiving increased use as our society becomes ever more data
driven and we are conducting more detailed analyses of data. We use the term “drill down” as adjective to describe a type of logic model. The CDC uses similar terminology, “nested
models”, to describe more detailed examinations of logic models (Silverman, Mai, Boulet, & O’Leary, 2009).

http://dx.doi.org/10.1016/j.evalprogplan.2017.08.012
Received 13 January 2017; Received in revised form 20 August 2017
Available online 01 September 2017
0149-7189/ © 2017 Elsevier Ltd. All rights reserved.
D.J. Peyton, M. Scicchitano Evaluation and Program Planning 65 (2017) 156–162

Table 1
Drill Down Model Approach.

Phase Suggested Steps for Constructing a Drill Down Logic Model

Phase I: Review 1. Establish assumptions about the program under evaluation.

Phase II: Knowledge Development
Phase III: Informed Inquiry
Phase IV: Drill Down
2. Develop or review the initial theory-based logic model of the program
3. Interview program manager(s) to identify program’s intended purpose and encourage reflection about the program in its current state
versus the initial intent of the program.
1. Conduct document analysis (e.g. yearly reports, internal assessments) and compare and contrast with initial theory-based logic model
2. Identify key-staff and personnel mentioned through document analysis
3. Revise original model to represent data collected through document analysis and interviews of program managers (e.g. eliminate or create
goals, activities, or sub-activities)
1. Conduct semi-structured interviews with key staff and management to establish understanding of the intent of the program, identify
activities and outputs crucial to achieving goals
2. and how their participation contributes to the overall intent of the program
3. Revise the initial logic model to reflect a more accurate view of the program by seeking feedback on the revision with program managers
and key staff.
1. Select activities in the revised logic model to “drill-down” into
2. Focusing on critical details that support efficient program management, create separate “drill-down” models that reflect each activity in
the revised theory based logic model
3. Review and revise the drill-down logic models regularly to reflect current program activities.

conducted interviews from the vantage point of the principal in-
vestigator, described program outcomes, and various innovations
within the program (Altieri, Colley, Daniel, & Dickenson, 2015;
Anderson, Smith, Olsen, & Algozzine, 2015; Little, Sobel, McCray, &
Wang, 2015). Missing from this body of research is a view of how the
process of implementation unfolded, corrections that were made, and
how stakeholders were included. In essence, there is little in the way of
actionable information to improve process management of these grants.
Improved process management makes it possible to enhance out-
come evaluations. Before a program can successfully achieve its pro-
gram goals it must be successfully implemented. Indeed, an outcome
evaluation should not begin unless it can be clearly demonstrated the
program has been faithfully and successfully implemented. Drill down
models make it possible to determine if the program has been suc-
cessfully implemented and can guide the necessary management
changes to produce successful program implementation. In this paper,
we demonstrate the utility of detailed drill down logic models as an
effective tool for management and monitoring of social program pro-
cesses. An important finding of our research is that the initial logic
model became obsolete once funding was obtained and implementation
began and was not reflected through a revised model, let alone one that
included greater detail. Moreover, detailed information contained in
the drill down model examples we provide demonstrate how identifying
and altering specific aspects of the actual implementation would not be
visible in the less-detailed conceptual logic model. In doing so we
concede that drill down logic models are a greater investment of time
and are likely to force stakeholders to confront potential shortcomings
or unintended impacts of the program. However, we posit that the
benefit of increased program clarity will provide program managers
with data to make calculated and necessary changes to the program,
outweighing the aforementioned challenges.
The paper is structured in the following manner. Section 1 describes
the function of drill down logic models and their potential impact for
managing and monitoring programs. The authors draw distinctions
between less detailed (e.g. theory approach) logic models versus those
that drill deeper (e.g. activities approach) to explain nuances within a
program. Section 2 describes Project Achieve (pseudonym), a program
established through a teacher personnel preparation grant. Section 2
also introduces the initial theory based logic model of Project Achive
developed as part of the initial grant proposal. Finally, Section 2 dis-
cusses the methods used to develop several drill down logic models that
reflect Project Achieve accurately in its current state and with attention
to details of the process. We explain how this more detailed level of
data offers specific potential courses of action for stakholders. For ex-
ample, Bellini, Henry, and Pratt (2011) found great utility in logic
model development to assist with process and outcome evaluation for
their professional development program for students with autism
spectrum disorder. Their use of a logic model approach helped to es-
tablish communication between stakeholders to systematically monitor,
manage, and report on progress with greater specificity, creating an
interative feedback loop to improve the program. Indeed, the logic
model is a tool for all stakeholders to see a clear graphical presentation
of program activities and outcomes. Moreover, the detail provided in a
drill down model increases opportunities for stakeholders to be involved
in discussions about and revisions to the actual implementation of the
program. The details of implementation will likely be of much greater
interest to stakeholders than the broader conceptual logic model, which
may ignore critical aspects that impact day to day operations (e.g. ac-
countability markers). Section 3 makes recommendations to encourage
the routine use of detailed drill down logic model development to inform
efficient and flexible management of social programs.
1. Drill down logic models
The purpose of drill down logic models is to expand into greater
specificity the actions that connect program objectives and activities to
program outputs. This area of investigation is commonly referred to as
the “black box” (Patton, 1997; Rossi, Lipsey, Freeman, 2004). The drill
down logic model is intended to unpack these components and sub-
components to reveal the inner-workings of the program. It is worth
noting that the utility of drill down logic models is correlated with effort
and commitment to data collection and reporting. For example, this
requires initial meetings and follow up with multiple stakeholder
groups, annual document reviews, and member checking as the pro-
gram moves through its initial temporal sequence. As such, a single
version is inadequate. However, the benefits obtained from the drill
down models make it possible to examine the details of actual program
implementation. Stated differently, a theory based logic model or one
that is intended for use by evaluators serves little utility for program
managers in their day to day operations. Thus, a detailed review of
implementation can facilitate operational changes that will produce an
effective program. This ability to adjust the actual aspects of im-
plementation are simply not possible with the broader conceptual
driven models. As staff changes occur, resources are added or removed,
and objectives are abandoned, the drill down models should be sensitive
to the fluidity of program development and reflect the current processes
of the program. For example, if funding is reduced or staff turnover, the
overall objective of offering education classes may remain, however,
the specifics of how often, may change from weekly to bi-weekly. This
small, but significant change, tends not to be reflected in the more
general theory model, however, it should be acknowledged in a drill
down model as it factors into how the program is managed and

157
D.J. Peyton, M. Scicchitano
implemented (Millar et al., 2001). Moreover, this approach is con-
cerned with the functionality of the program and the reallocation of
resources as conditions “on the ground” change, contrasted against
deriving the theory of change or unattainable goals. Thus, development
of a drill down logic model is intended to offer actionable information to
be taken up by all levels of stakeholders, although we focus our analysis
on the program manager.
Drill down models do not require new headings (i.e. inputs, activ-
ities, and outputs), but rather sub-headings that are expansions of more
generic statements. This level of deconstruction can occur at each of
these elemental levels. For example, “staff” may be listed under inputs
in a theory or output model, however, in a drill down model specific
staff are listed by department or another purposeful categorization.
Additionally, the same model may suggest that “staff will review ma-
terials”. Will all staff review all materials? A drill down model may
specify in the following manner, “instructional staff will review curri-
culum material”. This level of detail gives a clear indication of who is
responsible and the sub-activity they are responsible for under the more
general goal of “reviewing materials”. The deeper the model drills, the
higher the quality of information for program managers to make
changes. Instead of enacting changes that are counter to the mission of
the program (Millar et al., 2001; Scheirer, 1994), they are more likely to
enact decisions that are impactful in reaching program outcomes.
Stated differently, with more precise information the manager can op-
erate with a scalpel rather than a blunt instrument. In the next section,
we provide examples of this process as it was applied in investigating
Project Achieve.
2. Project achieve: using drill down logic models
The following section briefly explains special education personnel
preparation grants and describes, Project Achieve (pseudonym), a re-
structured program funded through a preparation grant by the United
States Department of Education. First, we provide background on spe-
cial education personnel preparation grants and their necessity to the
advancement of special education teacher preparation. Next, we de-
scribe elements of Project Achieve, designed to meet the contextual
problem outlined in a recent special education preservice program
improvement grant application (2007–2011). Followed by our data
collection methods, which offers a detailed recounting of how we
moved from the initial theory based model to drill down models. We
then introduce the initial theory based logic model (see Fig. 1) devel-
oped for Project Achieve. Finally, we demonstrate a revised theory based
model and a subsequent drill down logic model derived from the theory
based model as part of an iterative process to accurately reflect changes
in Project Achieve.
2.1. Background on personnel preparation grants
Federal investment by the U.S. Department of Education in per-
sonnel development of qualified special education professionals has
been in existence for over forty years (Kleinhammer-Tramill, Baker,
Tramill, & Fiore, 2003; Kleinhammer, Mickelson, Barton, 2014). The
continual funding is intended to advance the development of profes-
sionals to improve academic and behavioral outcomes for students with
disabilities. The rapidly accumulating body of research on how to ef-
fectively educate students with disabilities, increased access to educa-
tional opportunities for students with disabilities and the persistent
shortage of qualified special education personnel has only accentuated
the importance of federal funding. Recently, five-year special education
preservice program Improvement Grants (PPIG) were extended an-
nually from 2007 to 2011 to cohorts of institutes of higher education
(n = 72) to improve special education preservice programming in re-
sponse to the Highly Qualified Teacher (HQT) mandate, part of No
Child Left Behind legislation.
158
Evaluation and Program Planning 65 (2017) 156–162
“To accomplish this goal, personnel preparation programs must
ensure that their graduates who expect to be providing instruction in
a core academic subject are able to meet State special education
certification or licensure requirements, as well as have the necessary
content knowledge, consistent with the HQT requirement” (Federal
Register, 2007, p. 33599).
In the following section, we sketch out the areas of need Project
Achieve was designed to address and the proposed actions to address
them.
2.2. Project achieve
Project Achieve is a restructured teacher preparation program in-
tended to dually certify graduates in special and general education. The
program is designed as a response to the following areas of need: (a)
critical shortages in qualified special education teachers locally and
nationally, (b) improve the instruction of general education teachers
through exposure and mastery of evidence based practices (EBP) for
students with disabilities, (c) meet the needs of students with cultural
and linguistically diverse backgrounds and (d) improve the effective-
ness and retention of beginning teachers through high quality induction
and mentoring programs for pre-service and in-service teachers.
Project Achieve’s intended impact is to address the shortage of
highly-qualified teachers with dual certification and improve the out-
comes of students with high-incidence disabilities (e.g. learning dis-
abilities, attention deficit hyper-activity disorder). The overall impact is
supported by the following actions: (a) pre-service teachers will have
increased exposure and implementation of highly effective-research
based practices, (b) sustained supervised practice in high quality, di-
verse classrooms, (c) collaboration with general educators to improve
outcomes for students with disabilities served in general education
classrooms, and (d) high quality indication and mentoring support for
pre-and in-service teachers.
2.3. Data collection methods
Drilling down into the various layers of the Project Achieve, we used
an adapted case study format (Yin, 2013). We began by clearly stating
our assumptions (e.g. personnel preparation grants have been histori-
cally useful in teacher preparation reforms). Acknowledging this posi-
tion, we were conscious of potential evidence that support or refutes
this position to avoid bias. We then came to consensus on the unit of our
analysis, Project Achieve initial logic model and the process described
within, and proceeded with data collection through qualitative methods
(e.g. in-depth interviews and document analysis) commonly associated
with program evaluation (Patton, 2002; U.S. Department of Health and
Human Services, 2016). We proceed by detailing each phase of the
analysis to provide details necessary for practitioners to replicate all or
parts of our process.
Phase 1 of the analysis began with interviews of program manage-
ment, comprised of open-ended questions, permitting participants wide
latitude to introduce the program, its intent, and their perception of its
current utility. For example, we began with, “Why did you apply for the
grant?” and “Who was involved in the conception of the Project
Achieve?” This permitted interview participants to branch off into wide
ranging discussions about the success and struggles of the program and
key members that moved the program along. This also permitted us to
develop context for the program and promoted reflection on the part of
program management. For example, as the conversation progressed the
program managers freely admitted that the initial theory of change
represented by the logic model was a “reach” and was probably “more
ambitious” then they could accomplish. It was important during the
course of this conversation to reserve judgment and secure trust
(Patton, 1997). The establishment of trust led to securing access to
critical documentation that supported Phase II.
D.J. Peyton, M. Scicchitano Evaluation and Program Planning 65 (2017) 156–162

Fig. 1. Initial Project Achieve Program Theory Logic Model.

Phase II included document analysis (Rutman, 1980; Wholey, stakeholders could explain the intent of the Project Achieve beyond their
2003). Project Achieve staff provided the authors with official doc- specific participation (e.g. working with an internal evaluator to revise
umentation (i.e. grant proposal, annual reports, course syllabus, in- a course syllabi). Moreover, with only a single exception, stakeholders
ternal evaluation documentation). Beginning with the initial grant were unable to communicate how their participation fit into the larger
proposal, we worked to develop a sense of what Project Achieve in- theory of change. Interviews were analyzed and then coded themati-
tended to accomplish and the supporting literature to substantiate the cally, combined with previous data, interpreted and used to guide de-
theory of change. The proposal was tightly aligned with the initial velopment of the drill down logic models.
theory based logic model, meeting federal requirements. We used the
initial theory driven logic model of Project Achieve to guide our analysis
2.4. Initial project achieve logic model
of subsequent documents (i.e. annual reports and course syllabus). As
we moved through annual reports we engaged in comparative analysis
The authors reviewed Fig. 1 to gather a general understanding of
of activities and goals reported within the annual reports contrasted
how the initial program was conceived. Assisted by the grant proposal
with what was stated in the initial grant proposal. Where discrepancies
narrative and initial conversations with program managers we were
surfaced, we made note and began augmenting the initial theory based
able generate a sense of the program theory at its inception (Patton,
logic model. We categorized information under the logic model inputs,
1997). This initial step was consistent with accepted process evaluation
activities, and outcomes. When data from the document review did not
methods (Cooksy et al., 2001; McLaughlin & Jordan, 1999; Rossi et al.,
fall under the initial categories, we began to create sub-categories or
2004; W.K. Kellogg Foundation, 2004). As stated earlier, the theory
new categories to drill-down into Project Achieve. For example, as can be
based model provides general descriptions of the process, leaving out
seen in our drill down models below, our document analysis allowed us
details of who specifically will be involved and how the activities will
to unpack “restructure course work and field experiences”. Summarily,
successfully translate to stated outputs. The initial model, as presented
we found through our document analysis that revision of syllabi and
to funders or other outside stakeholders, it is a functional beginning.
self-reporting of restructured courses by faculty was what officially took
However, as we demonstrate, in Fig. 2, significant changes are reflected
place. We employed examples such as this to inform question devel-
in the updated program theory model.
opment for our next round of interviews with program managers and
additional stakeholders.
Phase III of our analysis consisted of a second round of interviews 2.5. Updated project achieve logic model
with Project Achieve investigators along with staff who were involved
with the document creation and mentioned in official documents. A review of annual Project Achieve program reports and interviews
Interviews were conducted in a semi-structured format (see Appendix with program staff shaped several revisions of the theory logic model.
A) with sets of questions designed to stimulate further discussion. We This data was contrasted against the initial theory logic model.
anticipated variance across stakeholder participants in their under- Although Fig. 2 represents a theory based model, a degree of detail was
standing of Project Achieve and their level of intent. However, few added (e.g. temporal alignment, specific terms, external influences) to
the model to improve its utility in program management (Millar et al.,

159
D.J. Peyton, M. Scicchitano Evaluation and Program Planning 65 (2017) 156–162

Fig. 2. Project Achieve revised theory logic model.

Fig. 3. Project Achieve Objective 1 Drill Down Logic Model (Blanton, Pugach, & Bóveda, 2014).

2001). An additional benefit of the revision process was to remove
objectives and activities that were no longer part of the program. When
objectives are removed, there is an impact on how resources are man-
aged, activities are undertaken and the causal linkages to outcomes are
likely to be reconfigured. These changes were uncovered through the
document review process and were verified by the program managers.
As reported by Gugiu & Rodriguez-Campos (2007) this may give pro-
gram managers pause, as any changes to the initial program will need
to be reported to funders. The acknowlegment, while difficult for pro-
gram managers, is necessary to recalibrate the program to meet its in-
tended outcomes. Using the revised activities from Fig. 2 we demon-
strate the utility of drill down models focusing on objective 1(Fig. 3).
2.6. Project achieve drill down logic model
As stated by Scheirer (1994),
“To undertake process measurement, the program must be specified
in detail…process evaluation forces clear thinking and planning
during program development—about what the program is, why it is
expected to produce results, for what types of people it may be ef-
fective and in what circumstances” (p. 40).
Drilling down into objective 1, “Review and revise syllabi to reflect
professional standards, evidence based practices, systems of support”,
we found several details that would assist a program manager in
monitoring progress within the program. For example, not all faculty
would be involved in these activities, but specifically special education
course instructors. This could include adjunct faculty or full time fa-
culty, but not those currently in non-instructional positions; reducing
the number of interactions program managers would have regarding
this activity. A second example is the temporal sequence represented in
the model, from a program manager perspective the temporal elements
can serve as another marker of progress (e.g. when are activities
scheduled to be completed?). Contrasted with the initial theory model
there is no temporal precedence, suggesting that all activities are oc-
curring simultaneously. Attention to temporal order could also be
considered more fine grain “if, then” statements, requiring fewer as-
sumptions about “how” the activities were being conducted when re-
viewing a theory logic model (W.K. Kellogg Foundation, 2004). Finally,

160
D.J. Peyton, M. Scicchitano
the drill down logic model demonstrates the iterative process within the
general activity described in objective 1 (Fig. 2). By examining the
theory model a program manager gets little sense of the steps within the
activities and is unable to make slight adjustments (e.g. add more staff
or divide up staff to evaluate syllabi) or quickly determine with cer-
tainty where and when a breakdown in the process occurs. Ultimately,
the drill down logic model encourages more pointed questions that can
lead to more nuanced analysis of the program process. However, the
analysis hinges on the dedication to update the models to reflect the
program in its most current form.
3. Lessons learned
As any new tool or methodology comes on line and gains wide
adoption, its value becomes diminished if not used as designed. Logic
models are powerful tools that can serve several purposes, developing
program theory, providing a detailed description of program activities,
identify tiered program outcomes (e.g. immediate, intermediate, long-
term), unearth causal linkages and stimulate program redesign (Rossi
et al., 2004). However, to be effective, logic models require a com-
mitment of time and resources to be properly described and revised
(Kaplan & Garrett, 2005; W.K. Kellogg Foundation, 2004). Specifically,
to serve program managers as an effective tool to augment the process
of the program (resources, activities, outputs) there should be a thor-
ough accounting of who or what resources are involved and the sub
activities they are involved in. The benefit of revision and high degree
of detail can be seen in drill down logic models for Project Achieve.
As a tool for program managers, drill down logic models are an
effective communication tool. It gives stakeholders at all levels a clear
sense of their responsibilities and the impact of their sub-activity in
achieving the outputs that are believed to have short and long term
impacts on the situation to be addressed (Kroeger, Borders, & Webster,
2013). Moreover, it increases the likelihood that program managers can
make sophisticated decisions about how program resources are utilized
and what activities are to be continued. Indeed, the drill down ap-
proach has broad applicability to the fields of evaluation, program
planning, education, public administration and even private sector
management.
Another substantive lesson learned from the drill down exercise
with Project Achieve was the finding that initial activities were no longer
applicable. This finding raised questions for the authors. At what point
did it become obvious that the objectives were unattainable? Would a
more detailed logic model have assisted with this analysis? How long
were critical resources allocated to activities that were unlikely to yield
benefits towards the outcomes? This series of questions is not aimed at
judging the initial program theory, but rather accentuates how a more
detailed model may have allowed program managers to shift resources
or alter sub-activities that were yielding little benefit.
Program managers are under significant pressure to secure funding
to support impactful programs. However, despite their best effort to
accurately present the program theory under ideal conditions for
funding consideration, changes within a social program are a reality.
Instead of interpreting the logic model process as a condition for the
grant proposal, the initial model should serve as a starting point to
facilitate continual and more detailed logic model development.
Resources in social programs, be it human or monetary, tend to be
scarce. Program managers require accurate information to deploy them
in the most efficient and impactful ways possible, drill down logic
models are one tool at their disposal to support these critical decisions.
Funding
The researchers did not receive any specific grant from funding
agencies in the public, commercial, or not-for-profit sectors.
161
Evaluation and Program Planning 65 (2017) 156–162
Acknowledgments
In addition to the authors cited in the references, the authors wish to
thank our anonymous reviewers, faculty and staff of Program Achieve
for their time and thoughtful insights of the program.
Appendix A
Project Achieve Interview Protocol
1. Can you talk to me about how you became a part of Project
Achieve?
2. To what degree did it align with other elements of the program,
how did you coordinate the program for both Special Education
and General Education.?
3. What activities did you directly participate in?
4. How did you participation fit into the larger goals of Project
Achieve?
5. What type of shared vision existed across departments of General
and Special Education?
6. Were assessments or assignments designed to complement other
elements of the program?
7. Who was involved? (General Education, Special Education,
Schools, etc.)
8. What was the expectation of collaboration for the process?
9. How frequent were interactions?
10. When you interacted with various faculty what was the context of
the interactions?
11. How would you describe the level of collaboration?
12. To what degree did the course prepare students for a special edu-
cation licensure?
References
Altieri, E. M., Colley, K. M., Daniel, L. S., & Dickenson, K. W. (2015). Merging expertise:
Preparing collaborative educators. Rural Special Education Quarterly, 34(1), 17–22.
Anderson, K., Smith, J. D., & Olsen, J. (2015). Systematic alignment of dual teacher
preparation. Rural Special Education Quarterly, 34(1), [Retrieved from] https://www.
questia.com/library/journal/1P3-3674661581/systematic-alignment-of-dual-
teacher-preparation.
Bellini, S., Henry, D., & Pratt, C. (2011). From intuition to data: Using logic models to
measure professional development outcomes for educators working with students on
the autism spectrum. teacher education and special education. The Journal of the
Teacher Education Division of the Council for Exceptional Children, 34(1), 37–51.
https://doi.org/10.1177/0888406410384153.
Blanton, L. P., Pugach, M. C., & Bóveda, M. S. (2014). Teacher education reform initiatives
and special education: Convergence, divergence, and missed opportunities (Document No.
LS-3). University of Florida.
Cooksy, L. J., Gill, P., & Kelly, P. A. (2001). 2001 − Cooksy, Gill, Kelly − The program logic
model as an integrative framework for a multimethod evaluation pdf, vol. 24, 3–9.
Federal Register (2007, November 27). U.S. Government Printing Office: Washington, DC.
Kaplan, S. A., & Garrett, K. E. (2005). The use of logic models by community-based in-
itiatives. Evaluation and Program Planning, 28(2), 167–172. https://doi.org/10.1016/
j.evalprogplan.2004.09.002.
Kroeger, S., Borders, C., & Webster, K. (2013). Research strand: Program evaluation.
McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your programs
performance story. Evaluation and Program Planning, 22(1), 65–72. https://doi.org/
10.1016/s0149-7189(98)00042-1.
Millar, A., Simeone, R. S., & Carnevale, J. T. (2001). Logic models: A systems tool for
performance management. Evaluation and Program Planning, 24(1), 73–81. https://
doi.org/10.1016/S0149-7189(00)00048-3.
Newton, X. A., Poon, R. C., Nunes, N. L., & Stone, E. M. (2013). Research on teacher
education programs: Logic model approach. Evaluation and Program Planning, 36(1),
88–96. https://doi.org/10.1016/j.evalprogplan.2012.08.001.
Patton, M. (1997). Utilization-focused evaluation: The new century text. Thousand Oaks, CA:
Sage431. [Retrieved from] http://scholar.google.com/scholar?hl=en&btnG=
Search&q=intitle:Utilization-Focused+Evaluation:+The+New+Century+Text#0.
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand
Oaks, CA: Sage.
Rutman, L. (1980). Planning useful evaluations: Evaluability assessment, vol. 96Sage library
of social researchhttps://www.ncjrs.gov/App/abstractdb/AbstractDBDetails.aspx?
id=67317.
Scheirer, M. A. (1994). Designing and using process evaluation. In Wholey, (Ed.).
Handbook of practical program evaluation (pp. 40–66).
Silverman, B., Mai, C., Boulet, S., & O’Leary, L. (2009). Logic models for planning and
D.J. Peyton, M. Scicchitano
evaluation: Resource guide for the CDC state birth defects surveillance program cooperative
agreement. CDChttp://www.cdc.gov/ncbddd/birthdefects/models/resource1-
evaluationguide2009.pdf.
W.K. Kellogg Foundation (2004). Logic model development guide. development. 72.
[Retrieved from] http://www.wkkf.org/knowledge-center/resources/2006/02/WK-
Kellogg-Foundation-Logic-Model-Development-Guide.aspx.
Wholey, J. S. (2003). Evaluability assessment. Handbook of practical program evalua-
tion33–62. https://doi.org/10.1177/0163278703026002005.
David Peyton is a doctoral student in Special Education Psychology and Early Childhood
162
Evaluation and Program Planning 65 (2017) 156–162
department at the University of Florida. His current research interests examine the impact
of federal and stated sponsored education programs and policies on special education
teacher preparation programs.
Michael Scicchitano is an associate professor of political science and the director of the
Master’s Program in Public Affairs at the University of Florida. He directs the Florida
Survey Research Center (FSRC) at the UF and has implemented hundreds of surveys re-
search and program evaluation projects. He is also editor of State and Local Government
Review, the official journal for American Society for Public Administration on inter-
governmental management.