Chapter 8: Designing Effective Programs

Kettner, Designing and Managing Programs: An Effectiveness-Based Approach 5e © 2017, SAGE

Publications.
 The Significance of Program Design

Program design:

• Process of identifying and defining elements that make up

delivery of a service

• Includes determining what type of clients with what types

of needs will receive what services, in what volume and
with what expected result
Program Design and Diversity
Effective interventions need to be specialized
based on demographic factors including:
• Ethnicity
• Gender
• Socioeconomic status
• Sexual orientation
 Program Design and Diversity
Six factors that strongly influenced special-population-
sensitive program design (Calley):
• Professional associations committed to both broad and specific
aspects of multiculturalism
• Increasing scholarship dedicated to cultural competence
• Cultural competence as a core component of academic
preparation in mental health disciplines
• Promulgation of national standards for cultural competence by the
federal government
• Inclusion of cultural competency-specific accreditation standards
for mental health and human service programs
• Requirement for addressing cultural competence in proposals for
funding new program development
 Defining the Elements of a Program
Logic model framework for defining program elements:
• Inputs—Resources and raw materials
• Process—Activities that use inputs to achieve
objectives with raw materials
• Outputs—Measurement of services provided and
completion of services
• Outcomes—Demonstrated benefits to those
receiving services
• Impact—Measurable changes on organizations,
communities or systems resulting from services
Defining the Elements of a Program:
Inputs
• Inputs include five elements representing an
agency’s resources and raw materials:
– Clients or consumers
– Staff
– Material resources
– Facilities
– Equipment
• Each element needs to be clearly defined in terms
that are useful for analytical and reporting purposes
Defining the Elements of a Program:
Inputs
Defining the Elements of a Program:
Throughputs
• Throughputs are the procedures that will be
implemented to carry out the program
• Throughputs usually involve treatment rehabilitation
or support services
• Each of the following elements needs to be clearly
defined in terms that are useful for analytical and
reporting purposes:
– Service definition
– Service tasks
– Method of intervention
Defining the Elements of a Program:
Throughputs
Service Definition
• Usually a simple definition of services to be provided
that focuses on a specific aspect of client problems
and needs
 Defining the Elements of a Program:
Throughputs
Service Tasks
• Define activities involved in the provision of service
• Activities tend to follow a chronological order which
can be represented by a flowchart
• Identification of tasks brings focus to the question of
who does what with clients, for what purpose and
under what conditions
• Task identification addresses accountability and
facilitates ongoing evaluation of effectiveness
Defining the Elements of a Program:
Throughputs
Method of Intervention
• Specifies the way services are planned for delivery
• Identify best practices and specify a proven method
of treatment or service delivery
Defining the Elements of a Program:
Throughputs
Defining the Elements of a Program:
Outputs
• Direct products of program activities
• Purpose of measuring outputs is to determine:
– how much of available service did client received
(units of service / intermediate output)
– whether client completed treatment or received
full complement of services as specified in the
program design (service completion / final output)
Defining the Elements of a Program:
Outputs
Units of Service / Intermediate Output
• Measured in three different ways:
– as episodes or contact units
(one contact between a worker and a client)
– as material units
(a tangible resource provided to a client)
– as time units
(minutes, hours, days, weeks, or months,
depending on the information needs of
program)
Defining the Elements of a Program:
Outputs

Service Completion / Final Output

• Defined at the time program is designed
• Defining output prior to implementation enables
evaluators to distinguish between individuals
who complete the program and individuals who
drops out
Defining the Elements of a Program:
Outputs
Quality
• Frequently addressed through the use of standards--
specifications accepted by recognized authorities that are
regularly and widely used and have a recognized and
permanent status
• Usually need to identify and operationalize standards (or
less often develop them)
• After quality standards are selected and defined, they
must be associated with units of service / intermediate
outputs and tracked
 Defining the Elements of a Program:
Outputs
Quality
• In measuring quality over time, two units need to be
recorded and tracked:
– the number of units of a given service provided to
clients
– of the units provided, the number that met the pre-
established quality standard
• Based on this information, any quality standard
established within a field can be used in conjunction
with units of service provided to determine the extent
to which quality standards are being achieved
Defining the Elements of a Program:
Outputs
Defining the Elements of a Program:
Outcomes
• Measurable change in quality of life of a client
between entry into and exit from a program
• Four categories of outcome measures:
– Numerical counts
– Standardized measures
– Level of functioning scales
– Client satisfaction
Defining the Elements of a Program:
Outcomes
Numeric Counts
• Nominal measures related to client flow
• Require yes or no answers to specific questions
• Program outcomes are determined by
aggregating findings within a specified period to
establishes a success rate derived from the
numeric count
Defining the Elements of a Program:
Outcomes
Standardized Measures
• Objective instruments validated and widely used by
practitioners
• Program outcomes are determined by periodically
aggregating data on clients receiving a service
 Defining the Elements of a Program:
Outcomes
Level-of-functioning scales
• Instruments developed by staff and other local experts
familiar with a particular population and problem that are
specific to a program or service
• Allow clients to be rated on several aspects of
functioning
• Not tested for validity and reliability
• Program outcomes are determined by aggregating
scores on each scale at client entry and exit
Defining the Elements of a Program:
Outcomes
Client Satisfaction
• Proxy measure for other more tangible outcomes (be
cautious with its use!)
• Requires the development of questions with
responses ranging from “very satisfied” to “very
dissatisfied” and options in between
• Program outcomes are determined by aggregating
scores for a defined interval
 Defining the Elements of a Program:
Outcomes
Intermediate Outcomes and Final Outcomes
• Intermediate outcomes--changes in quality of life of the
client measured at the point of completion of the services
provided
• Final outcomes--changes in quality of life of the client
measured at a designated follow-up point after services are
completed
• Intermediate and final outcomes are interrelated
• Intermediate and final outcomes can be documented by
using any of the four measures: numerical counts,
standardized measures, level-of-functioning scales and
client satisfaction instruments
 Defining the Elements of a Program:
Outcomes
Framework for identifying and defining elements of program
design:
• Inputs--client, staff, and physical resource elements
• Throughputs--service delivery elements
• Outputs--service completion elements
– Intermediate outputs--completions of units of service
– Final outputs--completions of full complement of
services prescribed
• Outcomes--measures of life changes for clients
– Intermediate outcomes--changes at the point of
completion of services
– Final outcomes--changes achieved or maintained as
of a specified point in the follow-up process after
completion of services
Defining the Elements of a Program:
Outcomes and Impact
 Specifying the Program Hypothesis

• Program hypothesis--term used to sum up the

assumptions and expectations of a program (if-then
statement)
• Guides the definition and selection of program
elements
 Specifying the Program Hypothesis

•Framework that provides program’s internal consistency

and integrity:
–Inputs—types of clients and staff, additional resources
–Throughputs—types of services, service tasks, and methods of
intervention are most relevant to address the problems and
work with the client population
–Outputs--mix of services needed for a full complement of
services and minimum quantity of these services that is
expected to produce a measurable result
–Outcomes--what can be achieved given the limitations of the
program
Foundation for a program that is logical and can be
implemented and evaluated