You are on page 1of 29

Making Your Data Work

Rob Johnstone & Priyadarshini Chaplot LACCD Achieving the Dream Retreat January 20, 2012

Making Your Data Work Rob Johnstone & Priyadarshini Chaplot LACCD Achieving the Dream Retreat January 20,
Making Your Data Work Rob Johnstone & Priyadarshini Chaplot LACCD Achieving the Dream Retreat January 20,

What we will cover

1.Framing the issue

2.What we learned from the BRIC Initiative, the Aspen Prize, and Completion by Design

3.Making the connection

and Completion by Design 3.   Making the connection Building Stronger Systems for SLOs and Program

WHO IS THE RP GROUP?

Putting our work in context

WHO IS THE RP GROUP? Putting our work in context Building Stronger Systems for SLOs and
The RP Group
The RP Group

Strengthens the abilities of California community colleges to gather, analyze and act on information in order to strengthen student success

Provides research, evaluation, professional development, and technical assistance services that support evidence-based decision-making and inquiry

Work is defined and conducted by community college practitioners

is defined and conducted by community college practitioners   Building Stronger Systems for SLOs and Program

Building Stronger Systems for SLOs and Program Review – September 2011

4

A RESOURCE FOR INSTITUTIONAL CHANGE

A Culture of Inquiry

A RESOURCE FOR INSTITUTIONAL CHANGE A Culture of Inquiry An Applied Inquiry Framework for Student Completion

What is a Culture of Inquiry?

Institutional capacity for supporting open, honest and

collaborative dialog focused on

strengthening the institution

and the outcomes of its students.

the institution and the outcomes of its students. An Applied Inquiry Framework for Student Completion –

An Applied Inquiry Framework for Student Completion – September 2011

6

Culture of Inquiry: Why All the Fuss?

Because this ongoing work is challenging but necessary! Work needs to marry insight & evidence Problems are large and recurring No silver bullet / evident answers Multiple solutions likely needed

Progress is not linear Requires cross constituency interaction

linear ●   Requires cross constituency interaction An Applied Inquiry Framework for Student Completion –

An Applied Inquiry Framework for Student Completion – September 2011

7

Culture of Inquiry: Features

Widespread sharing and easy access to user-friendly information on student outcomes Encouraging more people to ask a wider collection of questions and use their evidence and conclusions to enhance decision making Shared, reflective and dynamic discussions

making ●   Shared, reflective and dynamic discussions An Applied Inquiry Framework for Student Completion –

An Applied Inquiry Framework for Student Completion – September 2011

8

Culture of Inquiry: More Features

Multiple opportunities to discuss information within and across constituency groups Continuous feedback so adjustments can be made along the way and processes can be adapted Culture that values curiosity, questions and robust conversations

that values curiosity, questions and robust conversations An Applied Inquiry Framework for Student Completion –

An Applied Inquiry Framework for Student Completion – September 2011

9

THE IMPETUS FOR OUR WORK

Why does prioritizing a culture of inquiry make sense?

WORK Why does prioritizing a culture of inquiry make sense? Building Stronger Systems for SLOs and

You’re wasting time and money when your outcomes don’t focus on

the right data.

20 year trend for institutional outcomes

20 year trend for institutional outcomes 100% 90% 80% 70% 60% 50% 40% 30% 20% 10%

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

Retention Rate

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Retention Rate Success Rate What
Success Rate

Success Rate

What does that tell us about the usefulness of these metrics in setting 1989 1990
What does that tell us
about the usefulness of
these metrics in setting
1989
1990
1991
1992
institutional strategies?
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2008

1999

Building Stronger Systems for SLOs and Program Review – September 2011

12

Much of what is needed to support good decision making is

beyond the traditional accountability measures.

Much of what is needed to support good decision making is beyond the traditional accountability measures.

What is Our College’s Transfer Rate?

What is Our College’s Transfer Rate?
What is Our College’s Transfer Rate?

Data do not

speak

for themselves.

The vital role of conversation

The vital role of conversation ●   ●   ●   In order to make data

In order to make data useful, ample time and space are needed to discuss and analyze the information and connect it back to the original research question.

Answers are not always immediately apparent, so skilled facilitation may be needed to dig out the deeper meaning.

Multiple perspectives and types of information are often needed to make sense of individual data points.

!
!
are often needed to make sense of individual data points. ! Building Stronger Systems for SLOs

Building Stronger Systems for SLOs and Program Review – September 2011

16

WHAT CAN WE DO?

What the RP Group has learned

WHAT CAN WE DO? What the RP Group has learned Building Stronger Systems for SLOs and
BRIC’s on-campus support helps colleges improve internal practices. •   Technical experts who are community

BRIC’s on-campus support helps colleges improve internal practices.

Technical experts who are community college practitioners

Focus on discrete practices such as program review

Opportunities for dialog

such as program review •   Opportunities for dialog Building Stronger Systems for SLOs and Program

Building Stronger Systems for SLOs and Program Review – September 2011

18

Strategy 1:

Reform processes to make them

simpler and provide more useful information.

Strategy 1: Reform processes to make them simpler and provide more useful information. 19

age 2:

While what colleges developed varied, each system was:

age 2: While what colleges developed varied, each system was: •   Meaningful •   Sustainable

Meaningful

Sustainable

Consistent

For example

Building Stronger Systems for SLOs and Program Review – September 2011

20

Comprehensive Assessment Report Examples http://www.rpgroup.org/resources/comprehensive-

Comprehensive Assessment Report Examples

http://www.rpgroup.org/resources/comprehensive-

assessment-reports-best-practice-examples-around-

country

Resource Compendium on Assessment

http://www.rpgroup.org/resources/assessment-strategies-

http://www.rpgroup.org/resources/assessment-strategies- and-models-help-tell-story Building Stronger Systems for

and-models-help-tell-story

Building Stronger Systems for SLOs and Program Review – September 2011

21

Strategy 2: Create

space and time to talk

so good ideas can be put into action

Get conversations going using inquiry guides. • Assessing Student Learning Outcomes • Using an Equity

Get conversations going using inquiry guides.

Assessing Student Learning Outcomes

Using an Equity Lens to Assess Student Learning

Assessing Student Services Outcomes

Maximizing the Program Review Process

Assessing Non-credit Student Learning Outcomes

•   Assessing Non-credit Student Learning Outcomes Building Stronger Systems for SLOs and Program Review –

Building Stronger Systems for SLOs and Program Review – September 2011

23

Strategy 3:

Make better use of existing processes;

don’t reinvent the wheel.

Strategy 4:

Pick your organizational improvement priorities

and stick to them.

Strategy 5:

To build lasting changes, ensure you involve a

broad range of leadership.

WHAT THIS MEANS FOR YOU

How can you improve your accountability and success efforts?

How can you improve your accountability and success efforts? Building Stronger Systems for SLOs and Program

Brainstorm & Discuss

Pick one of the following questions to answer and pair up with someone to share your ideas.

What process could we simplify at our college?

How could we create more opportunities for conversations about our student outcomes?

What is a process we could redesign to make it more useful?

What should be my college’s top improvement priority?

How could we build stronger and deeper support for improvement efforts?

  How could we build stronger and deeper support for improvement efforts? Making Your Data Work

Making Your Data Work – November 2011

28

Find Out More

The RP Group Website www.rpgroup.org

Rob Johnstone, Senior Research Fellow rjohnstone@rpgroup.org

Bob Gabriner, Leadership Chair gabriner@sfsu.edu

Rob Johnstone, Senior Research Fellow rjohnstone@rpgroup.org Bob Gabriner, Leadership Chair gabriner@sfsu.edu 29
Rob Johnstone, Senior Research Fellow rjohnstone@rpgroup.org Bob Gabriner, Leadership Chair gabriner@sfsu.edu 29