Three Studies to Drive Campus Conversations on Student Success

Rob Johnstone, RP Group
An initiative of the Research & Planning Group for California Community Colleges

Today we’ll cover:
—  Working toward a “Culture of Inquiry” – selected

findings from statewide projects —  Grades & Success in Sequenced Courses —  Levels of Preparedness in GE Courses – success of simultaneously enrolled students in Math / English courses —  Cohort Tracking in Basic Skills

An initiative of the Research & Planning Group for California Community Colleges

Ground Rules
—  Ask lots of questions – especially nagging and tough

ones
—  If I’m talking too fast, throw something at me

(preferably not something sharp)
—  Data doesn’t make meaning; we make meaning from

data
—  Spontaneous outbursts of applause are encouraged

An initiative of the Research & Planning Group for California Community Colleges

Data, Evidence, College Decisionmaking, Student Success, and You…

An initiative of the Research & Planning Group for California Community Colleges

How do colleges make good decisions ?
If you want to learn you have to take part in the conversation

“Learning emerges from an ongoing conversation about things that matter with both passion and discipline”
- Parker Palmer The Courage to Teach
Learning is founded on evidence and inquiry
An initiative of the Research & Planning Group for California Community Colleges

Framing Thought #1
— “Data do not speak for themselves.

What is also needed are occasions that bring educators together to examine evidence about student learning, reflect on its meaning and identify approaches that yield better results.”
¡ 

Carnegie Foundation (2009)

An initiative of the Research & Planning Group for California Community Colleges

Framing Thought #2
—  Decision making in complex systems is a

highly social process, not solely an intellectual exercise.
—  The process is composed of many small

acts, carried out by different people at different points in time.

An initiative of the Research & Planning Group for California Community Colleges

Institutional Research (IR) & You
—  IR is about helping the college and its

practitioners make better decisions —  Institutional research is fundamentally an interventionist discipline. —  We are using non-experimental data to examine behavior in complex human systems —  We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions

An initiative of the Research & Planning Group for California Community Colleges

Linear Model of Institutional Research
Primary Responsibility
Faculty/Staff Researcher

Key Features: •  Request-driven •  Highly delineated tasks •  Emphasis is on generating metrics
An initiative of the Research & Planning Group for California Community Colleges

Collaborative Model of Inst. Research
Primary Responsibility
Faculty/Staff Researcher Joint Activity

Key Features:
•  Dialogue-rich •  Jointly-driven processes
An initiative of the Research & Planning Group for California Community Colleges

Finding from RP Research

Colleges may be focusing on the wrong data.

An initiative of the Research & Planning Group for California Community Colleges

Twenty Year Trend for Common Institutional Outcome Measures
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Retention Rate

Success  Rate  
Consider the multitude of changes over this period: - faculty/staff turnover -  program successes/failures -  changing student demographics -  budget contractions/expansion -  evolving state & accreditation mandates -  leadership turnover and yet these performance metrics remained amazingly stable.

What does that tell us about the usefulness of these metrics in setting institutional strategies?

An initiative of the Research & Planning Group for California Community Colleges

An Observation Much of what is needed to support good decision making lies beyond the current grasp of our institutional databases.

An initiative of the Research & Planning Group for California Community Colleges

What is Our College’s Transfer Rate?

An initiative of the Research & Planning Group for California Community Colleges

Insight from the Field: A Look at Three Studies

An initiative of the Research & Planning Group for California Community Colleges

Section 2: Grades & Success in Sequenced Courses

An initiative of the Research & Planning Group for California Community Colleges

Success  Rates  of  English  836/846  Students  by  Entry  Group
100%

90%

82%
80%

70%

67%

63% 58% 49% 51% 42%

60%

59%

50%

40%

33%

37%

41%

30%

20%

18%

10%

0%

A  in  English  826

B  in  English  826

C  in  English  826
Success  Rate

Other  Grade  in   English  826
Non-­‐Success  Rate

English  836/846   Non-­‐Success

English  836/846  -­‐ First  English  C ourse

An initiative of the Research & Planning Group for California Community Colleges

Success  Rates  of  English  100  Students  by  Entry  Group
100% 90%

88% 80%

80%

70%

65% 58% 53%
47% 45%

60%

55%

50%

42%
40%

35%

30%

20%

20% 12%

10%

0%

A  in  English  836/846 B  in  English  836/846 C  in  English  836/846
Success  Rate
An initiative of the Research & Planning Group for California Community Colleges

Other  Grade  in   English  836/846

English  100/105   Non-­‐Success

English  100/105  -­‐ First  English  C ourse

Non-­‐Success  Rate

Success  Rates  of  Math  110  Students  by  Entry  Group
90%

81%
80%

70%

60%

60% 51%

62% 57% 49% 43% 38% 54%
46%

50%

40%

40%

30%

20%

19%

10%

0%

A  in  Math  811

B  in  Math  811

C  in  Math  811
Success  Rate

Other  Grade  in  Math   Math  110/111  Non-­‐ Math  110/111  -­‐ First   811 Success Math  C ourse
Non-­‐Success  Rate

An initiative of the Research & Planning Group for California Community Colleges

Success  Rates  of  Math  120  Students  by  Entry  Group
90%

80%

77% 65%
55%

79%

70%

60%

55%

57%

50%

45% 35% 23%

45%

43%

40%

30%

21%

20%

10%

0%

A  in  Math  110/112

B  in  Math  110/112

C  in  Math  110/112 Other  Grade  in  Math   Math  120/122  Non-­‐ Math  120/122  -­‐ First   110/112 Success Math  C ourse
Success  Rate Non-­‐Success  Rate

An initiative of the Research & Planning Group for California Community Colleges

Success  Rates  of  Chemistry  220  Students  by  Entry  Group
120%

100%

96%

80%

79% 71%

77% 61%

60%

60%

40%

40% 29% 21%

39% 23%

20%

4%
0%

A  in  C hemistry  210 B  in  C hemistry  210

C  in  C hemistry  210
Success-­‐Rate

Other  Grade  in   Chemistry  210

Chemistry  220  Non-­‐ Chemistry  220  -­‐ First   Success Chemistry  C ourse

Non-­‐Success  Rate

An initiative of the Research & Planning Group for California Community Colleges

Success  Rates  of  Biology  260  Students  by  Entry  Group
100% 90%

94%

91% 81% 85%

89%

80%

76% 64%

70% 60%

50%

40% 30%

36% 24% 19% 6% 9% 15% 11%

20%

10%

0%

A  in  Biology  250 B  in  Biology  250 C  in  Biology  250

Other  Grade  in   Biology  260  Non-­‐ Biology  260  -­‐ First   Simultaneous  /   Biology  250 Success Biology  C ourse Other  Group  for   Biology  260
Non-­‐Success  Rate

Success  Rate
An initiative of the Research & Planning Group for California Community Colleges

Success  Rates  of  Economics  102  Students  by  Entry  Group
100%

91%
90%

81%
80%

70%

68%

71% 66%

71%

60%

50%

40%

32%
30%

34% 29% 29%

20%

19% 9%

10%

0%

A  in  Economics  100 B  in  Economics  100 C  in  Economics  100
Success  Rate
An initiative of the Research & Planning Group for California Community Colleges

Other  Grade  in   Economics  100

Economics  102  Non-­‐ Economics  102  -­‐ First   Success Economics  C ourse

Non-­‐Success  Rate

Section 3: Success in GE Courses by Simultaneously Enrolled English / Math Students

An initiative of the Research & Planning Group for California Community Colleges

Psychology 100 Success of Students Simultaneously Enrolled in English
Simultaneously Enrolled Count in % of PSYC Success in PSYC 100 100 Enr. PSYC 100 English Course Two Levels Below Transfer One Level Below Transfer Transfer A Transfer B Not in English Course Total 139 730 1,209 530 5,207 7,815 2% 9% 15% 7% 67% 100% 31% 48% 68% 75% 64% 63%

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
An initiative of the Research & Planning Group for California Community Colleges

Psychology 100 Success of Students Simultaneously Enrolled in Math
Simultaneously Enrolled Count in % of PSYC Success in PSYC 100 100 Enr. PSYC 100 Math Course Three Levels Below Two Levels Below One Level Below Transfer-Level Math Not in Math Course Total 157 970 773 997 4,918 7,815 2% 12% 10% 13% 63% 100% 43% 51% 63% 76% 64% 63%

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
An initiative of the Research & Planning Group for California Community Colleges

Success in 5 Highly Enrolled GE Courses by English Enrollment Level
Simultaneously Enrolled Success in Success in Success in Success in Success in Psyc 100 SPCH 100 ECON 100 HIST 201 HSCI 130 English Course Two Levels Below Transfer One Level Below Transfer Transfer A Transfer B Not in English Course Total Total Enrollments 31% 48% 68% 75% 64% 63% 7,815 54% 57% 73% 82% 74% 73% 5,793 27% 56% 73% 79% 69% 69% 5,643 48% 43% 51% 64% 59% 58% 4,603 60% 74% 82% 88% 73% 75% 4,475

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
An initiative of the Research & Planning Group for California Community Colleges

Success in 5 More Highly Enrolled GE Courses by English Enrollment Level
Simultaneously Enrolled English Course Two Levels Below Transfer One Level Below Transfer Transfer A Transfer B Not in English Course Total Total Enrollments
Success in FITN 334 Success in ACTG 100 Success in MUS 202 Success in BIOL 250 Success in SOCI 100

48% 58% 68% 76% 66% 66% 4,447

37% 50% 66% 73% 69% 68% 3,718

31% 64% 74% 80% 70% 71% 3,624

N/A 41% 60% 65% 70% 68% 3,577

40% 50% 59% 74% 69% 68% 2,813

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
An initiative of the Research & Planning Group for California Community Colleges

Success in 5 Highly Enrolled GE Courses by Math Enrollment Level
Simultaneously Enrolled Success in Success in Success in Success in Success in PSYC 100 SPCH 100 ECON 100 HIST 201 HSCI 130 Math Course Three Levels Below Two Levels Below One Level Below Transfer-Level Math Not in Math Course Total Total Enrollments 43% 51% 63% 76% 64% 63% 7,815 54% 63% 67% 78% 75% 73% 5,793 26% 46% 62% 80% 69% 69% 5,643 38% 48% 51% 63% 59% 58% 4,603 49% 71% 79% 86% 74% 75% 4,475

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
An initiative of the Research & Planning Group for California Community Colleges

Success in 5 More Highly Enrolled GE Courses by Math Enrollment Level
Simultaneously Enrolled Math Course Three Levels Below Two Levels Below One Level Below Transfer-Level Math Not in Math Course Total Total Enrollments
Success in FITN 334 Success in ACTG 100 Success in MUS 202 Success in BIOL 250 Success in SOCI 100

59% 61% 68% 77% 66% 66% 4,447

43% 46% 55% 77% 70% 68% 3,718

50% 58% 73% 82% 70% 71% 3,624

N/A 51% 54% 69% 70% 68% 3,577

N/A 46% 55% 75% 69% 68% 2,813

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
An initiative of the Research & Planning Group for California Community Colleges

Section 4: Cohort Tracking in Basic Skills Sequences

An initiative of the Research & Planning Group for California Community Colleges

Cohort Tracking
—  Cohorts tracked include students whose first math

course at Skyline was between 2002-03 and 2006-07. Students were tracked for three academic years.
—  Students were tracked based on the first math/

English/ESL course that they took at Skyline (not on their placement).

An initiative of the Research & Planning Group for California Community Colleges

Math Cohort Tracking – Starting in Fundamentals
Starting Level Att. Fund Pass Fund Att. Beg. Pass Beg. Att. Inter. Pass Inter. Att. Tran. Pass Tran.

Fundamentals % of starting cohort % from previous step

1,863

1,215

939

624

331

212

134

99

100%

65% 65%

50%

33% 66%

18%

11% 64%

7%

5% 74%

---

77%

53%

63%

*Note: Fundamentals = Math 811, Beginning Algebra = 110 or 111+112, Intermediate Algebra = 120 or 122+123, Transfer = 130, 150, 200-level

An initiative of the Research & Planning Group for California Community Colleges

Math Cohort Tracking – Starting in Beginning Algebra
Starting Level Att. Beg. Pass Beg. Att. Inter. Pass Inter. Att. Tran. Pass Tran.

Beginning Algebra % of starting cohort % from previous step

3,449

2,306

1,363

934

577

440

100%

67% 67%

40%

27% 27% 69%

17%

13% 76%

---

59%

62%

*Note: Beginning Algebra = 110 or 111+112, Intermediate Algebra = 120 or 122+123, Transfer = 130, 150, 200-level

An initiative of the Research & Planning Group for California Community Colleges

Math Cohort Tracking – Starting in Intermediate Algebra
Starting Level Att. Inter. Pass Inter. Att. Tran. Pass Tran.

Intermediate Algebra % of starting cohort % from previous step

2,086 100% ---

1,494

891 43% 60%

647

72% 72%

31% 73%

*Note: Intermediate Algebra = 120 or 122+123, Transfer = 130, 150, 200-level

An initiative of the Research & Planning Group for California Community Colleges

English Tracking – Starting in Two Levels Below Transfer
Starting Level Att. Two Below Pass Two Below Att. One Below Pass One Below Att. Trans. A Pass Trans. A Att. Trans. B Pass Trans. B

Two Levels 2,328 Below Transfer % of starting cohort % from previous step 100%

1,566 67%

1,179

924 40%

742

591

440

362

51%

32%

25% 80%

19%

16% 72%

---

67%

75%

78%

80%

74%

*Note: Two Levels Below = 826, One Level Below = 836 or 846, Transfer A = English 100 or 105, Transfer B = 110 or 165

An initiative of the Research & Planning Group for California Community Colleges

English Tracking – Starting in One Level Below Transfer
Starting Level Att. One Below Pass One Below Att. Trans. A Pass Trans. A Att. Trans. B Pass Trans. B

One Level Below Transfer % of starting cohort % from previous step

3,017

2,246

1,666

1,391

1,038

879

100%

74% 74%

55%

27% 46% 83%

34%

29% 85%

---

74%

75%

*Note: One Level Below = 836 or 846, Transfer A = English 100 or 105, Transfer B = 110 or 165

An initiative of the Research & Planning Group for California Community Colleges

English Tracking – Starting in Transfer A
Starting Level Transfer A % of starting cohort % from previous step
Att. Trans. A Pass Trans. A Att. Trans. B Pass Trans. B

3,320 100% ---

2,615 79%

1,641 49% 63%

1,406 42%

79%

86%

*Note: Transfer A = English 100 or 105, Transfer B = 110 or 165

An initiative of the Research & Planning Group for California Community Colleges

ESL Tracking – Starting in Pre Intermediate
Starting Level Pre Intermediate % of starting cohort % from previous step
Att. Pre Inter. Pass Pre Inter. Att. Inter. Pass Inter. Att. High Inter. Pass Att. High Trans Inter/ . Pass Trans .

259

177

123

91

78

56

44

32

100%

68% 68% 68%

47%

35% 35% 74%

30%

22% 22% 72%

17%

12% 11% 73%

---

69%

86%

79%

*Note: Pre Intermediate = 820 / 821-822 / 852-862-872; Intermediate = 830 / 831-832 / 853-863-873; High Intermediate = 840 / 841-842; Transfer = 400 or English 100/105

An initiative of the Research & Planning Group for California Community Colleges

ESL Tracking –Starting in Intermediate
Starting Level Intermediate % of starting cohort % from previous step
Att. Inter. Pass Inter. Att. High Inter. Pass High Inter/ Att. Trans . Pass Trans .

361

250

177

133

99

79

100%

69% 69% 69%

49%

37% 37% 75%

27%

22% 22% 80%

---

71%

74%

*Note: Intermediate = 830 / 831-832 / 853-863-873; High Intermediate = 840 / 841-842; Transfer = 400 or English 100/105

An initiative of the Research & Planning Group for California Community Colleges

ESL Tracking – Starting in High Intermediate
Starting Level High Intermediate % of starting cohort % from previous step
Att. High Inter. Pass High Inter/ Att. Trans. Pass Trans.

366 100% ---

243 66% 66%

158 43% 65%

123 34% 34% 78%

*Note: High Intermediate = 840 / 841-842; Transfer = 400 or English 100/105

An initiative of the Research & Planning Group for California Community Colleges

Something to consider…
English Progression 67% 40% 74% 25% 46% 79% Math Progression 65% 33% 67% 11% 27% 72% 5% 13% 31% 16% 29% 42% ESL Progression 68% 35% 22% 11% 22% 34%

69% 37% 66%

An initiative of the Research & Planning Group for California Community Colleges

Something to consider…
English Progression 67% 74% 79% 40% 74% 46% 42% 25% 46% 29% 79% 16% 29% 42% ESL Progression 68% 69% 5% 66% 35% 22% 11% 22% 34%

Math Progression 65% 33% 11%

37% 22% 69% 37% 34% 66%

67% 67% 27% 13% 27% 13% 72% 31% 72% 31%

An initiative of the Research & Planning Group for California Community Colleges

Something to consider…
English Progression 67% 74% 79% 40% 46% 42% 25% 29% 16% ESL Progression 68% 69% 5% 66% 35% 37% 34% 22% 22% 11%

Math Progression 65% 67% 72% 33% 27% 31% 11% 13%

An initiative of the Research & Planning Group for California Community Colleges

BRIC TAP Core Team
Rob Johnstone, Project Director johnstoner@smccd.edu Priyadarshini Chaplot, Project Coordinator pchaplot@rpgroup.org Lead Technical Coaches Darla Cooper Janet Fulks Bob Gabriner Cathy Hasson Ken Meehan Bob Pacheco Greg Stoup Ian Walton

An initiative of the Research & Planning Group for California Community Colleges

Find Out More!
RP Group Website www.rpgroup.org BRIC TAP http://www.rpgroup.org/projects/BRIC.html Rob Johnstone, BRIC Project Director johnstoner@smccd.edu Kathy Booth, RP Group Executive Director kbooth@rpgroup.org Priya Chaplot, BRIC Project Coordinator pchaplot@rpgroup.org
An initiative of the Research & Planning Group for California Community Colleges

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.