Logic models to enhance

program performance

Ellen Taylor-Powell, Ph.D.

Evaluation Specialist
University of Wisconsin- Extension-Cooperative Extension

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Where are you going?
How will you get there?
What will tell you that
you’ve arrived?

A logic model is your

program ROAD MAP

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Logic model is a…
• Picture of your program or intervention
• Graphic representation of the “theory of
action” – what is invested, what is done, and
what results
• Core of planning and evaluation

Provides a common framework for your work

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Definition
LOGIC
− the principles of reasoning
− reasonable
− the relationship of elements to each other and a whole

MODEL
− small object, representing another, often larger object
(represents reality, isn’t reality)
− preliminary pattern serving as a plan
− tentative description of a system or theory that accounts
for all of its known properties

The American Heritage Dictionary, 2nd Ed

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 The accountability era
• What gets measured gets done
• If you don’t measure results, you can’t tell success from
failure
• If you can’t see success, you can’t reward it
• If you can’t reward success, you’re probably rewarding
failure
• If you can’t see success, you can’t learn from it
• If you can’t recognize failure, you can’t correct it.
• If you can demonstrate results, you can win public support.

Re-inventing government, Osborne and Gaebler, 1992

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Logic model is in widespread use

• Private Sector
• Public Sector: GPRA
• Non-Profit Sector
• International Arena
• Evaluators

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Example: Every day logic model –
Family Vacation

Family Members Drive to state park Family members

learn about each
Budget other; family
Set up camp
bonds; family has
Car a good time
Cook, play, talk,
laugh, hike
Camping
Equipment

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Example: Financial management program
Situation: Individuals with limited knowledge and skills in basic financial
management are unable to meet their financial goals and manage
money to meet their needs.

INPUTS OUTPUTS OUTCOMES

Extension We conduct a Participants gain

invests time variety of knowledge, change
and resources educational activities practices and have
targeted to improved financial
individuals who well-being
participate

WHAT WE INVEST WHAT WE DO WHAT RESULTS

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Example: One component of a comprehensive parent
education and support initiative

Situation: During a county needs assessment, majority of parents reported that

they were having difficulty parenting and felt stressed as a result

INPUTS OUTPUTS OUTCOMES

Parents
Develop increase Parents
Staff parent ed knowledge of identify
curriculum child dev appropriate Improved
actions to child-
Targeted take parent
Money Parents better
Deliver parents relations
understanding
series of attend
their own
interactives parenting style Parents use
Partners essions effective Strong
parenting families
Parents gain
practices
Research Facilitate skills in
support effective
groups parenting
practices

Assumptions: External factors:

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Example: Smoke free worksites
Situation: Secondhand smoke is responsible for lung cancer, respiratory symptoms,
cardiovascular disease, and worsens asthma. Public policy change that creates smoke free
environments is the best known way to reduce and prevent smoking.

Inputs Outputs Outcomes

Assess
worksite Increased Demonstrations
Coalition tobacco Worksite awareness of of public
policies and owners, importance of support for SF
Time practices managers SF worksites worksites

Dollars
Develop Increased SF
Unions SF worksites
community knowledge of worksites
Partners policies
support for SF
Including SF worksite drafted
worksites
youth benefits &
options
Workers;
Organize union SF worksite
and members policies
Increased passed
implement
commitment,
strategy for
Public support and
targeted
demand for
worksites Adherence
SF worksites
to smoke-
free policies

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Example: Logic model training workshop
Situation: Funder requires grantees to include a logic model in funding request;
grantees have limited understanding of logic models and are unable to fulfill the
funding requirement

INPUTS OUTPUTS OUTCOMES

Increase
Trainer 3 hour knowledge of
training logic models Use logic models Improved
Budget
in planning and planning –
•Interactive
Equipment Increase ability to evaluation – in programs
activities
Grantees create a your own work achieve positive
Research
•Group work meaningful logic results
base
•Practice model of program
Training Model quality logic Improved
curriculum •Q and A Increase model practice evaluation -
confidence in more credible
using logic and useful data
models

Accountable here

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Connecting outputs to
outcomes is a challenge
“I think you should be
more explicit here in
Step Two.”

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Programs aren’t linear
Feedback loops and multi-dimensions

INPUTS OUTPUTS OUTCOMES

Long-
term
Program Activities Participation Short Medium
investments

What we What we Who we What results

invest do reach

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Chain of outcomes
SHORT MEDIUM LONG-TERM
Seniors increase Practice safe cooling of Lowered incidence of food
knowledge of food food; food preparation borne illness
contamination risks guidelines

Participants increase Establish financial goals, Reduced debt and

knowledge and skills in use spending plan increased savings
financial management

Community increases Residents and employers Child care needs are met
understanding of discuss options and
childcare needs implement a plan

Empty inner city parking Youth and adults learn Money saved, nutrition
lot converted to gardening skills, nutrition, improved, residents enjoy
community garden food preparation and mgt. greater sense of
community

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Focus of outcomes

•Individual •Child is ready to enter school; farmer

- Child, parent, client, implements nutrient management practice
resident

•Families control spending to maintain

•Group family financial stability
- family, team, community
- group
•Agency institutes policy that encourages
physical activity of staff
•Agency, organization
•Family serving agencies share resources
•System to better meet clientele needs

•Community •Communities develop and preserve

decent safe and affordable housing

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Writing good outcomes
SMART objectives: Specific, measurable, attainable, results-oriented, timed

Who/what Change/desired In what By when

effect

Families increase their use of within one

participating in community year of joining
the Family resources and
Resource
services
Center

4 school adopt policies to by Dec 2005

boards improve
student
nutrition and
physical activity

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
What does a logic model look like?
•Graphic display of boxes and
arrows; vertical or horizontal
- Relationships, linkages
•Any shape possible
- Circular, dynamic
- Cultural adaptations;
storyboards
•Level of detail
- Simple
- Complex
•Multiple models
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Multiple logic models
Multiple models may be needed to describe and
explain complex initiatives or systems.

1.Multi-level programs:
A series of linked models that depict varying levels
such as national-state-county levels OR, institution-
division-unit levels

2.Multi-component programs:
A series of models to depict various components
(goals, sites, target populations) within a
comprehensive initiative

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 State level logic model: Reducing and preventing youth tobacco use
Outcomes - Impact
Inputs Activities Reach Short Medium Long
Increased awareness of
Promote community need to eliminate youth Increased compliance
involvement in restricting access to tobacco
tobacco access to youth and enforcement of laws Decreased
Community products, including and policies
Coalition  Establish access to
baseline of Parents, tobacco industry tactics,
Members tobacco for
current practices Caretakers laws, noncompliance
minors
 Inform/educate Law enforcement
 Eliminate self- Retailer Decreased supply to
service minors
Health Department Increased commitment to
 Facilitate active
Funding enforcement of eliminate access/sources
laws
Social
Increased knowledge and norms less
Facilitate youth involvement Community org, Increased # of youth
Partners skills in participating in supportive
in policy change Businesses actively engaged in
Local policy change of youth
 Regional  Recruit youth Policy makers policy change
 Involve tobacco use
State Adults
youth/adults
Youth serving org
 Educate Increased commitment by
Youth youth and adults for youth Increased adoption of
to participate in policy policy changes that
Promote school and change involve youth in the
community based prevention change process
Research programs and policies Delayed
and best  Establish average age
Schools Increased knowledge
practices baseline of at first use;
existing resources Community about tobacco Increased # of effective reduced
 Educate Families dependence; benefits
prevention programs or initiation
 Assist with Youth serving org and options for youth
policies adopted
planning and Youth prevention (e.g, CDC
implementing guidelines, school-family
programs/services initiatives)
Reduced
Increased # of youth morbidity and
participating in prevention mortality
Promote youth cessation Increased commitment to
See Treating programs
services and Tobacco Addiction adopt effective
policies Youth Logic Model programs/policies for youth
prevention

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Component Logic Model Youth: Youth Advocating for Policy Change
Outcomes - Impact
Inputs Activities Reach Short Medium Long
Coalition Establish baseline for Increased # of youth
Increased # youth,
members policy change in Community actively engaged in
community with help community
organizations, members who: advocating for policy
from youth
businesses change in community
policy makers Understand
tobacco use issues Increased
Time Educate youth and adults in their communities number of
on policy change options Know how to tobacco
and how to achieve them advocate for policy policies in
change Increased # of activities community
Funding Adults or increased intensity
of activities that involve
Identify partners, youth to accomplish
including youth serving policy change
Partners organizations and Social
schools, for engaging Youth serving Increased # youth
Local organizations
norms less
youth in policy change wanting to be
Regional supportive
involved in
State advocating for
of youth
Develop strategy for and policy changes Increased adoption tobacco use
promote engagement of of policies that
youth in policy change involve youth in the
Effective practice Schools
policy change
strategies Increased # youth Counter industry Delayed
skilled in being influence
Assist with development of average age
youth advocacy skills able to advocate
YOUTH for policy change Promote clean at first use;
Local media indoor air reduced
outlets initiation
Promote community Decrease
support for youth availability of
involvement in Increased support
for youth tobacco products in
community affairs/
involvement in the community
policy change
policy change

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Getting started

• Determine purpose of logic model

- Who will use it? For what?
• Involve others
• Set boundaries for logic model
• Understand situation
• Explore research, knowledge base, what
others are doing/have done

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Check your logic model

1. Is it meaningful?

2. Does it make sense?

3. Is it doable?

4. Can it be verified?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Limitations

Logic Model…
• Represents reality, is not reality
• Focuses on expected outcomes
• Challenge of causal attribution
 Many factors influence process and outcomes
• Doesn’t address: Are we doing the right
thing?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Where does evaluation fit?

From beginning to end

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 PLANNING: start with the end in mind

What do you want to know? How will you know it?

EVALUATION: check and verify

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
What does evaluation mean to you?

 Evaluation means asking good,

critical questions about programs to
improve programs and help them be
accountable for the wise use of
resources.

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Parents
Develop increase
parent ed knowledge of Parents
Staff curriculum child dev identify
appropriate Improved
Deliver actions to child-
Money Targeted Parents better
series of understand take parent
parents
interactive their own relations
attend
Partners sessions parenting style
Parents use
effective
Parents gain Strong
Research Facilitate skills in
parenting
families
support practices
effective
groups parenting
practices

EVALUATION: What do you (and others) want to know about this program?
What amount Were all Did all parents attend To what extent did To what extent To what extent
knowledge and are relations
of $ and time sessions that we intended? did behaviors improved?
were delivered? How Who did/not not?Did skills increase? change? For Does this result
invested? effectively? they attend all For whom? Why? whom? Why? in stronger
What else families?
sessions? What else
happened?
happened?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
 Prioritize
Lots of questions and so little time

Prioritize evaluation questions

Evaluation purpose
• Need
• Context
• Process
• Outcomes
Stakeholder needs

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Who wants to know what about your program?

WHO might use WHAT do they HOW will they

the evaluation? want to know? use the info?
You – staff

Participants

Funder

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Developing an evaluation
plan based on your logic model
1. Focus:

2. Questions 3. Indicators 4. Timing 5. Data collection

Sources Methods Sample Instruments

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
http://www.uwex.edu/ces/lmcourse

http://www.uwex.edu/ces/pdande

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation