2013 Catholic Charities USA Annual Gathering

Leading for Social Impact

September 16, 2013

Jeff Bialik, Executive Director Karen Erickson, Director of Assessment & Learning

1

Introductions
Jeff Bialik Executive Director Catholic Charities CYO San Francisco, CA 415.972.1287 jbialik@cccyo.org Karen Erickson Dir. Assessment and Learning Catholic Charities CYO San Francisco, CA 415.972.1346 kerickson@cccyo.org

Leading for Social Impact – September 16, 2013

2

Leading for Social Impact
 Description:

o This workshop will challenge participants to consider a paradigm shift in traditional program centric thinking in favor of a mission centric approach focused on social impact, outcomes and the metrics that measure success. Focus areas include a survey of emerging trends and tools for connecting the mission to outcomes and impact.

 Learning Objectives:

o Identify major trends impacting or likely to impact nonprofit entities o Understand why nonprofit leaders should focus on social impact, outcomes and the indicators that measure success o Gain an understanding of a theory of change and logic model framework and how it might be applied

Leading for Social Impact – September 16, 2013

3

What is a paradigm shift and why do we need it?

 A paradigm is an accepted pattern of behavior or framework for how we act.
 There are disruptive changes or “forces” at work that are making it necessary for those of us in nonprofit work to do things differently to make a positive impact  …Thus – the need for a paradigm shift

Leading for Social Impact – September 16, 2013

4

Making the case for an era of disruptive changes

 Pervasive, disruptive transformation in the nonprofit sector
o Sources of stasis: The trends that provide continuity in our lives o Sources of change: The trends that disrupt

• Minor – small, evolutionary changes that occur over a long period of time • Major – large scale change within the same macro structure • Sudden and huge - disruptive, paradigm shifters, “black swans”

Leading for Social Impact – September 16, 2013

5

Six Disruptive Forces *
Purposeful Experimentation Information Liberation Integrating Sciences

Uncompromising Demand for Impact
Branding Causes, Not Organizations Attracting Investors, Not Donors
* Disruptive Forces: Driving a Human Services Revolution, The Alliance for Children and Families, Baker Tilly

Leading for Social Impact – September 16, 2013

6

The growth in number of non-profit organizations is outpacing the growth in donor contribution
There are many nonprofits that have not registered with IRS

Total Giving in the USA
In $Bn -.4%*

Number of Non-profits Registered With IRS
In Mn +4.7%*

327 295

304

285

291

* Average annual growth rate
Source: GivingUSA

Leading for Social Impact – September 16, 2013

7

Significant Environmental Trends  Shifting priorities for government
o o o o o Health care/Medicare Pensions/Social Security Debt reduction/revenue aversion Prison/Military Industrial Complex Education (K – 16)

 The global economy
o o o o Europe China India Middle East

Leading for Social Impact – September 16, 2013

8

Continuing the significant environmental trends  Changing demographics
o Aging population o Increased diversity o Impact of immigration

 Shifting cultural norms
o o o o

Increased acceptance of single parent families Changing attitudes re: sexual identity Explosive ubiquity of social media 24x7 connectivity

Leading for Social Impact – September 16, 2013

9

Still even more significant environmental trends      Hybrid structures Social enterprises Social innovation For profit service providers Social impact bonds *

 Human capital bonds

o Social Impact Bonds Video

* For more information, please read McKinsey & Company's report, "From Potential to Action," which assesses the opportunity for social impact bonds in the U.S. and is the result of 12 months of research and analysis. http://mckinseyonsociety.com/social-impact-bonds

Leading for Social Impact – September 16, 2013

10

Five (Not-So-Easy) Fixes in response to these forces and trends
Move toward social impact mission/vision/ (the what and the why)

Diversify revenue streams

Build sustainability (adaptability and resiliency) into the organization

Lead and manage to impacts, outcomes and metrics/key performance indicators (KPIs)

Leading for Social Impact – September 16, 2013

11

A Picture of Response to Disruptive Forces*
Multi-service organization of the past
Service-centric Holds itself accountable for shortterm, service-specific outcomes Clients receive the single service that they were referred to

Multi-service organization of the future
Client-centric Holds itself accountable for longterm, holistic outcomes Clients receive multiple services (if needed), based on a plan that they create with staff

“Discharges” clients after completing Maintains an extended relationship one service with clients to ensure they achieve and maintain long-term outcomes Program staff are siloed Reports data for compliance with funders Roles and processes in place to deliver multiple coordinated services Uses data to continually improve service delivery

* Disruptive Forces: Driving a Human Services Revolution, The Alliance for Children and Families, Baker Tilly
Leading for Social Impact – September 16, 2013

12

Moving toward a culture of continuous improvement
From To

Data collected for external reports Data as a burden Only the manager knows that Culture of blame and excuses

Data for internal improvement

Data as a useful tool Knowledge sharing across staff Culture of shared accountability

Functional or program “silos”

Cross-functional teams

“That doesn‟t have anything to do “How can I help us reach our with my job!” team goals?”

Fear of breaking rules and patterns
Leading for Social Impact – September 16, 2013

Support for making mistakes and trying new strategies
13

*A New Paradigm for Catholic Charities CYO
Move from a program centric bias where: Solving a problem requires a program solution Funding drives program Programs drive mission Program funders dictate mission Programs evolve into perpetual silos… … until funders change the rules Move toward a mission centric bias where: We provide services not programs What we do is driven by the outcomes we seek The evolving needs of the client determine the services we provide over time Client needs and efficient use of agency resources requires external collaboration, internal integration and/or hand-off to a third party Meeting the mission does not require that we perform the service Resources gravitate to meet the required outcomes.
Leading for Social Impact – September 16, 2013

14

Why is measuring outcomes important?

To improve the quality of our services
To focus our limited resources on those activities that actually advance our mission and achieve our desired outcomes To respond to donors/funders seeking to support measurable impacts To inform our advocacy

Leading for Social Impact – September 16, 2013

15

Making the Case for Impacts and Outcomes
 Differentiation  Return on investment  The rise of the “impact measurement meta-mediary”  The voices of philanthropic thought leaders

 The growth in the number of nonprofits
 The ability to demonstrate impacts is important to funders  Nonprofit stakeholders  The case for quality

Leading for Social Impact – September 16, 2013

16

Charity Navigator is introducing its “third dimension” in charity evaluation

1. Financial Health 2. Accountability and Transparency 3. Results Reporting
“This is the most important work being done in the nonprofit sector” Paul Brest, former president of the William and Flora Hewlett Foundation

CN 3.0

Leading for Social Impact – September 16, 2013

17

“Setting clear goals and finding measures that will mark progress toward them can improve the human condition” *
* Bill Gates‟ 2013 Annual Letter  Bill Gates' 2013 Annual Letter Video

Leading for Social Impact – September 16, 2013

18

“If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra
Where are you going? How will you get there? What will show that you’ve arrived?

Leading for Social Impact – September 16, 2013

19

Furthermore, this ability to articulate and measure impact goes beyond attracting funders, but benefits all key stakeholders
Benefit to Stakeholders Funders  Funders need better quality information to decide which effective organizations to fund o Foundations (e.g., David and Lucile Packard Foundation) and sophisticated individual donors typically want to see impact achieved Quotes ““Government might be satisfied with metric, for now, but foundations typically look for demonstration of impact before funding…” - Consultant to Non-profits

 Beneficiaries receive prevention, intervention and stabilization measures that are tightly linked Clients/ towards a sustainable result over the long term Beneficiaries o Not just getting temporary “piece meal” assistance
 Board and management team are aligned and have clarity in the goals they want to hold Board and themselves accountable for and how they can go Management about to achieve those goals  Staff are clear about the strategic priorities of the organization and how their program fits into the overall mission o Create a tunnel window to channel staff’s creativity and effort on initiatives driving towards the mission
Interviews; Bridgespan: Strongly Led, Under0managed; Booz & Company analysis

““Moving towards real impact means giving a hand up and not just a hand out” - Board Member, Philanthropedia

“This becomes a strategic decision making tool, to identify priorities, what to do and what not to do” - President Emeritus, Great Valley Center ““Our people are so focused on what it is that we’re trying to accomplish from a social impact perspective…it’s about getting to the goals” - CEO, Teach for America

Staff

Source:

Leading for Social Impact – September 16, 2013

20

Leap of Reason – a Basic Premise
 Most nonprofits don‟t know whether or not they are delivering on their promises  Managing to outcomes is …
o o o o o Using information to support decision making About culture and people – not numbers Primarily about helping nonprofits improve A key to maximizing collective impact All too rare

 We must focus on why and what to measure – not just how to measure  The nonprofit must be the driver (and the beneficiary) of the outcomesassessment process  Reasonableness and common sense must guide the outcomesassessment process
o o o o o Hear the constituent voice Assess to learn and do Apply rigor with reason Be practical Create a learning culture

Leading for Social Impact – September 16, 2013

21

Now that we know the WHY, let‟s move on to the HOW

 Theories of Change  Logic Models  Data Collection and Reporting

Leading for Social Impact – September 16, 2013

22

 EXERCISE

Leading for Social Impact – September 16, 2013

23

Theory of Change
 A common problem is that activities and strategies often do not lead to the desired outcomes.

 Check your „if-then‟ statements and ensure that they make sense and lead to the outcomes you want to achieve.  A logic model makes the connections EXPLICIT.

“I think you should be more explicit here in Step Two.”
Leading for Social Impact – September 16, 2013

24

Some Definitions

o The empirical basis for believing we can make a difference o Why we expect we can achieve success

Theory of Change –how we effect change

Logic Model – what we do and how
o The related and interconnected elements of a program, organization or activity that rationally links objectives, activities and outcomes/impacts

  

Inputs – what resources are committed Outputs – what we measure

o Money, time, staff, facilities, methodologies, etc.

o Volume of a program‟s activities, e.g. numbers of clients served, units of service, etc.

Outcomes – what we wish to achieve
o o

Meaningful changes that result from the actions that we take Changes in knowledge, skills, attitudes, behaviors, condition, or status

 

o Specific, observable and measurable demonstration of whether the desired change occurred

Indicators – what we use to stay on course Impact – what we aim to effect

o The long term benefit of our activities beyond the individual services recipient

Leading for Social Impact – September 16, 2013

25

In the Theory of Change, identifying intended impact and the belief of why it will happen help crystalize the strategy
Description: Intended Impact “What does success look like?”
 Explains what the organization is trying to achieve and will hold itself accountable for within some manageable period of time  Identifies both the benefits the organization seeks to provide and the beneficiaries of those activities  Draws on the following to define the who, what, where, and how o Organizational values • Target population (e.g., homeless youth, 12-24) • Target outcomes (e.g., self sufficiency) • Geography (e.g., San Francisco) • Approach (e.g., continuum of care) o Data • Needs of beneficiaries • Availability of services by other organizations o Making tough choices and tradeoffs • Reallocation of resources for greatest impact • Clear on what to do and what not to do
Source: Zeroing in on Impact, Stanford Social Innovation Review, Sept 2004; Delivering on the Promise of Nonprofits, Harvard Business Review; Booz & Company analysis

Description: Theory of Change “Why do we expect we can achieve success?”
 Explains the underlying beliefs of how the organization’s intended impact will actually happen Identify the cause-and-effect logic by which organizational and financial resources will be converted into the desired social results  Considers beyond taking account of its own resources, but also partnership and involvement of others where necessary  Ensures that all stakeholders understand why strategic decisions are made as they are  Helps to unearth assumptions about programs and services that can be tested and revised as necessary  Is broad enough to reflect the organization’s mission but focused enough to be actionable

Both discussions are iterative and there might be several options to achieve the same results – there is no “one right answer”

Leading for Social Impact – September 16, 2013

26

Theory of Change

 Theory of change example – University of Arizona  University of Arizona Theory of Change Video

Leading for Social Impact – September 16, 2013

27

To get started, a set of key questions need to be asked and answered . . .
Some Questions to Clarify Intended Impact  Who are our beneficiaries?  What benefits do our programs create?  Where is our geographic focus?  How do we define success?  What do we hold ourselves accountable for?  What won’t we do?  What would make us obsolete? Some Questions to Clarify Theory of Change  What is the cause-and-effect logic that gets us from our resources (people and funds) to impact?  Where are the gaps or leaps of faith in this logic chain?

 What are the most important elements of our programs’ content and structure?
 What assumptions led us to choose these particular program elements?  Are there other ways in which we could achieve the desired outcomes?  What is the minimum duration our beneficiaries need to be engaged to achieve these outcomes?

Source: Zeroing in on Impact, Stanford Social Innovation Review, Sept 2004; Booz & Company analysis

 What else do our beneficiaries need to achieve these outcomes?

Leading for Social Impact – September 16, 2013

28

...which will then help the organization develop an actionable intended impact and a coherent theory of change
Characteristics of an Actionable Intended Impact  Links in a compelling way to your mission and vision for social change  Specifies the outcomes you seek to create for your beneficiaries  Affords sufficient control over outcomes to enable real accountability  Is realistic and achievable, given your capabilities  Is measurable on an accurate, timely basis  Provides an effective platform for making strategic tradeoffs, especially those related to program focus and resource allocation decisions Note: Theories of change have been largely used as a tool for evaluation. It is difficult to pinpoint the earliest use of the term "Theory of
Change," but a hint at its origins can be found in the evaluation community among the work of notable methodologists, such as Huey Chen, Peter Rossi, Michael Quinn Patton, and Carol Weis Source: ActKnowledge, Theory of Change Community; Zeroing in on Impact, Stanford Social Innovation Review, Sept 2004; Booz & Company analysis

Characteristics of a Coherent Theory of Change  Identifies the most important needs of your chosen beneficiaries  Articulates the most important leverage points to meet those needs  Links your solutions to your beneficiaries’ needs through a chain of cause-and-effect relationships  Is empirically plausible if not proven

Leading for Social Impact – September 16, 2013

29

Mission Centric/Outcome Thinking  Figure out exactly what you‟re trying to accomplish
o Can you define success clearly and succinctly?

 Pick the right indicator
o How do you know that you have achieved success?

 Get good quality numbers
o Can you confirm that you are actually on track to achieving success?

 Show that it was you
o How do you know it was you?

Leading for Social Impact – September 16, 2013

30

Catholic Charities CYO Canal Family Support – Kid‟s Club
 Clarifying our Intended Impacts
o o o o o Who, what, where? How do we define success? What are we accountable for? What won‟t we do? What would make us obsolete?

 Clarifying our Theory of Change
o o o o o o o What is the cause and effect logic that gets us to impact? Where are the gaps or leaps of faith in this logic chain? What are the most important elements? What assumptions have we made? Could we do this some other way? What is the minimum duration? What else do our clients need?

Leading for Social Impact – September 16, 2013

31

Catholic Charities CYO Canal Family Support – Kid‟s Club
 Problem Statement:
o Children living in the Canal District of San Rafael, CA, on average perform at significantly below grade level in reading and writing as measured by test scores. This learning gap is caused in part by living in households where at least one member speaks English as a Second Language, one or more parents has no more than a high school education and parents are unaware of and/or unable to access school and community resources to benefit the child and family. As a result of this gap, these children are less likely to make satisfactory academic progress, more likely to drop out of high school, and less likely to achieve higher education, leading to a higher probability of generational poverty.

 Program Outcomes and Impacts:
o Our outcomes are to increase satisfactory academic progress; increase access to resources; increase high school graduation; and increase achievement of higher education leading to our impacts of a reduction of educational achievement gap in Marin and an empowered and self-sufficient community.
Leading for Social Impact – September 16, 2013

32

The Logic Model

 Logic Model Video

Leading for Social Impact – September 16, 2013

33

Logic model in evaluation

What do you want to know?

How will you know it?

EVALUATION: check and verify

Leading for Social Impact – September 16, 2013

34

Going through the logic model checklist will help ensure that the causal relationship between the activities and outcomes is robust
Logic Model Checklist
 Have you included all the major activities needed to implement your project and achieve expected outcomes and impact?  Does your organization have adequate resources to implement the activities and achieve the desired outcomes and impact?  Have you expressed your outcomes and impact in terms of change? Have you identified who/what will experience that change, and over what time period?  Do activities, outputs, outcomes and impact relate to each other logically (the if-then relationship)?  Does your logic model clearly identify the scope of your project‟s influence?  Have you identified the objectives and underlying assumptions about why your project exists?  Have you considered external forces that may influence your project‟s success?
Source: FSG Social Impact; Booz & Company analysis

Leading for Social Impact – September 16, 2013

35

Logic Model Example: Step Up!
Input/ Resources
      Staff Volunteers Money Equipment Facilities Training

Activities/ Processes
Activities to Nurture Career Awareness and Exploration  Self-assessment of personal interests/ goals  Guest speakers from different professions  Job shadowing Activities for Developing Skills for Finding Employment  Application Writing  Resume Writing  Job Searching  Interviewing Activities for Developing Job Skills  Service-learning experiences  Activities to develop soft skills for work such as communication, teamwork, time management… Activities for Academic Preparation  Visits to universities, colleges, training centers  Research on school programs related to interests  Learning about scholarships/ bursaries  Registering for school and courses Activities Providing Extended Support  Safe, respectful and friendly atmosphere  One-to-one mentoring

Short-term Outputs
 Number of youth attending program  Activities youth took part in  Levels of attendance  Goal completion rates  Number of volunteers and their hours  Hours of community service  Etc.

Intermediate-term Outcomes
Increased:  Awareness of personal interests and skills  Awareness of careers  Understanding of the world or work  Skills for finding work  Social skills for working with others  Work skills  Positive attitudes towards school  Knowledge of education and training programs  Self-discipline  Job experience  Goal setting  Increased education experience

Long-term Impact
Increased likelihood of:  High School graduation  Employment  Post-Secondary Studies  Life-Long Learning  Personal Success

Source: Program Outcome Evaluations; United Way of Winnipeg; Booz & Company analysis

Leading for Social Impact – September 16, 2013

36

The Urban Institute and the Center for What Works have developed a set of common outcomes and measures for non-profits
 Objective o Provide guidance on what & how to measure o Standardize impact/outcome measures for benchmarking and performance comparison  Outcomes and measures have been defined for 14 categories as a reference point o Adult Education o Advocacy o Affordable Housing o Assisted Living o Business Assistance o Community Organizations o Emergency Shelter o Employment Training o Health Risk Reeducation o Performing Arts o Prison Re-entry Source: Building a Common Outcome Framework to Measure Nonprofit Performance (2006), The Urban Institute, o Transitional Housing The Center for What Works; Booz & Company analysis o Youth Mentoring o Youth Tutoring
Leading for Social Impact – September 16, 2013

37

Common Outcome: Youth Tutoring
Output Intermediate Outcome Long-Term Impact

Note: Adapted from The Common Outcome Framework; organizations can modify and move the measures along the time continuum (e.g., from intermediate outcome to long-term impact) depending on what they feel is reasonable to achieve within their resources and means Source: The Urban Institute; The Center for What Works; Booz & Company analysis

Leading for Social Impact – September 16, 2013

38

Theory of Change: If we provide targeted reading intervention to elementary-aged English Language Learners then they will reach and maintain grade level reading proficiency.
Canal Family Support: 3-13-2013
Situation/Problem Statement Elementary school students from the Canal District are reading below grade level which causes them to be significantly less prepared to advance in their education. Inputs Facilities - Location at Pickleweed Park Community Center facilities Staff - Program Administrators (2) - After School Group Leaders (6) Supplies/Other Resources - Accelerated Reader Software - Community Partnerships - Alignment with San Rafael schools - Safe and positive culture - Nutritious Snacks Outputs Participants -# of elementaryaged EnglishLanguage Learners (grades 1-5) -# of families Activities - # of hours of reading practice and assessment provided daily - # of meetings with teachers, family advocates and other school personnel / one-on-one school support meetings with families - Physical Activity - Family Support - Experiential Enrichment Short term Outcomes
1. 1st and 2nd grade students reach and maintain grade level reading proficiency 2. 3rd, 4th and 5th grade students reach and maintain grade level reading proficiency

Intermediate Outcomes 3. 1st, 2nd, and 3rd grade students reach and maintain grade level reading proficiency 4. 4th and 5th grade students reach and maintain grade level reading proficiency

Long Term Outcomes 5. 1st, 2nd, 3rd and 4th grade students reach and maintain grade level reading proficiency 6. 5th grade students reach and maintain grade level reading proficiency

Benefits/Impact All Marin County students are reading at or above grade level and are successfully transitioned to middle school. All Marin County students stay reading at or above grade level and graduate from high school. All Marin County students stay reading at or above grade level and graduate from college.

Indicators: 1. 80% of 1st and 2nd graders are reading at or above Benchmark level on the Star Early Literacy test (at least 40 Percentile Rank) 2. 60% of 3rd, 4th and 5th graders are reading at or above the Benchmark level on the Star Reading test (at least 40 PR)

Indicators: 3. 80% of 1st, 2nd and 3rd graders are reading at or above Benchmark level on the Star Early Literacy test (at least 40 Percentile Rank) 4. 60% of 4th and 5th graders are reading at or above the Benchmark level on the Star Reading test (at least 40 PR)

Indicators: 5. 80% of 1st, 2nd, 3rd and 4th graders are reading at or above Benchmark level on the Star Early Literacy test (at least 40 Percentile Rank) 6. 60% of 5th graders are reading at or above the Benchmark level on the Star Reading test (at least 40 PR)

Leading for Social Impact – September 16, 2013

39

Data Collection and Reporting
 Creating a Data Collection Plan that staff agree on and own  Working with each program or service to make sure that they have realistic, relevant and practical tools with which to collect data  Having a database in which to input and store data  Creating a reporting template or dashboard for showing off the results of the program‟s efforts

Leading for Social Impact – September 16, 2013

40

Data Collection Plan
Program Name and Time Period Outputs YTD

Q1

Q2

Q3

Q4

NOTES

Outcome

Indicators

Target

Results

Short Term Outcomes:

Intermediate Outcomes:

Long Term Outcomes:

Leading for Social Impact – September 16, 2013

41

Sample Report
Canal Kids Club Outcomes and Outputs Data for Fiscal Year 2013 July 1, 2012 - June 30, 2013

Outputs
Participants Number of kids (1-5) Number of parent / families Short Term Outcomes (September) 1. Establish baselines on reading tests Average score in first 30 days of school year Reading Activities Number of hours of after school programming Number of parent education events Intermediate Outcomes (Mid-Year) 2. Kids improve reading level Average score mid-year Reading

Long-term Outcomes (April) 3. Kids improve reading levels Average score in April Reading

Leading for Social Impact – September 16, 2013

42

Summary of Key Learning  There are both internal and external factors that motivate us to measure impacts
o Improve quality o Focus resources o Attract/retain funding

 To measure impact we must translate our mission into an end goal, explain why and how and then measure performance  Theory of Change/Logic Model are good tools to use  Delivering real impact takes time
Leading for Social Impact – September 16, 2013

43

Summary of Key Learning (continued)  Transition to outcome-centric requires a paradigm shift
o Requires changes in culture, org structure, systems o Changes must be both top-down and bottom-up  Measuring outputs is different from measuring outcomes  May require narrowing the focus of services and individual staff performance expectations  Important to socialize the concept and the language

Leading for Social Impact – September 16, 2013

44

Challenges & Achievements
 Major Challenges
o Gaining support from board, agency leaders and program staff o Getting comfortable with the tools and language o Battling against the inertia and moving the culture (internal and external) inch by inch o Anticipating change in a changing environment o Finding good indicators for each outcome

 Select Achievements
o Engaging the discussion with agency leadership, program staff and the board o Applying the Theory of Change/Logic Model tools to each unique program in a multi-program agency o Identifying significant resources that are available to support this journey (see links to resources) o Having good data to give to funders and community partners

Leading for Social Impact – September 16, 2013

45

Links to resources – best practices
                  http://www.performwell.org/ http://dekhconsulting.com/ http://www.vppartners.org/leapofreason/overview http://www.fsg.org/ http://www.trueimpact.com/ http://www.whatworks.org/ http://www.urban.org/center/cnp/projects/outcomeindicators.cfm http://www.mckinsey.com/client_service/social_sector http://www.jimcollins.com/ http://www.missionbased.com/ http://www.bridgespan.org/Home.aspx http://www.ssireview.org/ http://www.socialimpactexchange.org/exchange/knowledgecenter/overview http://www.aecf.org/KnowledgeCenter.aspx http://www.uwex.edy/ces/pdande www.innonet.org http://www.performwell.org/ http://outcomestoolbox.com/
46

Leading for Social Impact – September 16, 2013

Leading for Social Impact
Contact info: Jeff Bialik jbialik@cccyo.org 415.972.1287 Karen Erickson kerickson@cccyo.org 415.972.1346
September 16, 2013
47