You are on page 1of 43

Prepared  by  

LEAPS  for  Environmental  Literacy:    
Engaging  Adolescents  in  Workforce  
 
Development  

!

June 2015

Leadership & Evaluation to Advance Program Success

Table  of  Contents  
Cluster  Study  Overview                                                                                                                                                                                                                                              3

 

Par<cipant  Overview                                                                                                                                                                                                                                                  12

 

Pa@erns  in  Outcomes  Across  Organiza<ons                                                                                                                                                                  16  

Connec<ons  between  Program  Components  and  Outcomes                                                                                                      22                                  

Compensa<on                                                                                                                                                                                                                                                                          30                                  

Avenues  for  Future  Evalua<on  and  Learning                                                                                                                                                              34                                  

Takeaways  for  the  Field                                                                                                                                                                                                                                        36                                  

Appendices                                                                                                                                                                                                                                                                                    38                                  
LEAPS  Cluster  Studies:  Workforce  Development  

Cluster  Study  Overview  

The  Path  to  Cluster  Studies  
§  Seventeen  LEAPS  organizaBons  have  engaged  in  ongoing  technical  assistance  to  
develop  their  evaluaBon  capacity  since  October  2012.  During  the  first  phase  of  this  
iniBaBve,  organizaBon  representaBves  aJended  trainings  focused  on  foundaBonal  
evaluaBon  and  learning  concepts:    theories  of  change,  logic  models,  evaluaBon  plans,  
tool  development,  data  analysis  techniques,  and  communicaBng  effecBveness.  They  
also  engaged  in  a  range  of  leadership  capacity  building  acBviBes.  
§  In  the  phase  that  followed  and  with  conBnued  support  from  the  S.D.  Bechtel,  Jr.  
FoundaBon,  Learning  for  AcBon  worked  with  clusters  of  organizaBons  to  develop  
shared  measures  around  three  cross-­‐cuSng  areas  of  interest.  The  cluster  study  work  
arose  out  of  an  opportunity  for  the  cohort  to  begin  to  document  its  collecBve  story  –  
which  was  further  emerging  aUer  Phase  I’s  work  –    and  the  desire  to  advance  
organizaBons’  evaluaBon  capacity  through  the  hands-­‐on,  group  learning  experience  
this  type  of  project  would  provide.  OrganizaBons  worked  together  to  define  and  form  
the  clusters,  collect  data,  and  analyze  the  emerging  stories.  

4  

LEAPS  Cluster  Studies:  Workforce  Development  

Cluster  Study  Topic  
OrganizaBons  in  the  Workforce  cluster  uBlize  workforce  
development  models  centered  on  culBvaBng  youth  
leadership  and  voice  in  local  environmental  and  social  
jusBce  issues  as  plaYorms  for  preparing  youth  for  future  
job  opportuniBes,  and  fostering  greater  connecBon  with  
the  environment.  They  have  adapted  to  their  local  
communiBes,  engaged  many  different  partners,  and  
drawn  from  diverse  funding  sources  in  the  development  
of  their  models.    
Photo  Credit:  CommunityGrows  

We  used  the  following  unifying  If-­‐Then  statement  to  test  assump<ons  about  our  models:    
If  we  create  relevant,  supporBve  early  employment  experiences  for  youth  that  demonstrate  
appreciaBon  and  value  through  compensaBon,  then  youth  will:    
§  Develop  transferrable  professional  &  leadership  skills;    
§  Increase  commitment  to  environmental  stewardship;  and,    
§  Experience  increased  self-­‐efficacy,  including  workplace  self-­‐efficacy.    

5  

LEAPS  Cluster  Studies:  Workforce  Development  

Cluster  Study  Organiza<ons  
Rising  Sun  Energy  
Center’s    (Rising  Sun)  
mission  is  to  empower  
individuals  to  achieve  
environmental  and  
economic  sustainability  
for  themselves  and  their  
communiBes.  

CommunityGrows  
culBvates  healthy  youth  
through  growing  
gardens  in  low-­‐income,  
diverse  communiBes.  

6  

Student  Conserva<on  
Associa<on’s  (SCA)  
mission  is  to  build  the  next  
generaBon  of  conservaBon  
leaders  and  inspire  lifelong  
stewardship  of  the  
environment  and  
communiBes  by  engaging  
young  people  in  hands-­‐on  
service  to  the  land.  

LEAPS  Cluster  Studies:  Workforce  Development  

Cluster  Study  Organiza<ons  
We  would  also  like  to  
recognize  Youth  Radio  for  
its  parBcipaBon  and  
!
thought  partnership  
throughout  the  cluster  work.  Their  program  
model  and  approach  share  similar  components  
and  intended  impact  as  the  other  organizaBons  
within  the  cluster.  While  Youth  Radio  could  not  
parBcipate  in  data  collecBon  with  the  cluster,  
the  organizaBon  made  significant  contribuBons  
to  every  stage  of  this  work,  parBcularly  survey  
instrument  development.  
Youth  Radio’s  mission  is  to  launch  young  people  on  
career  and  educaBon  pathways  by  engaging  them  in  
work-­‐based  learning  opportuniBes,  creaBve  
expression,  professional  development,  and  health  
and  academic  support  services.  

Photo  Credit:  Youth  Radio  

I   will   apply   the   criBcal   thinking   skills   I've  
learned   here   to   almost   every   aspect   of   my  
life,   including   my   educaBon   and   my  
consumpBon   of   media.   I   will   also   apply   my  
new  consciousness  of  professional  skills  to  all  
professional  spaces  in  my  life.  
-­‐Youth  Radio  ParBcipant1  

1While  Youth  Radio  did  not  parBcipate  in  cluster  data  collecBon,  this  parBcipant  quote  comes  from  Youth  Radio’s  exisBng  

7  

evaluaBon  efforts.    

LEAPS  Cluster  Studies:  Workforce  Development  

Our  Approach  
Research  Ques<ons  

§  What  pa@erns,  if  any,  emerge  across  organizaBons  with  regard  to  rela<ve  gains  
in  professional  skills,  connecBon  to  the  environment,  and  self-­‐efficacy?  
§  What,  if  anything,  can  we  learn  about  the  relaBonship  between  different  parts  of  
our  if-­‐then  statement?  
§  How  important  is  compensa<on  to  youth?  
§  Where  we  have  qualitaBve  informaBon,  in  what  ways  do  youth  describe  the  
rela<onship  between  what  they  did  in  program  and  the  changes  they  
experienced?  

Methods  
• Shared  post-­‐program  survey  aUer  the  summer  or  fall  work  
experience.  
• OrganizaBons  uBlized  many  common  measures,  with  occasional  
customizaBon.  

8  

LEAPS  Cluster  Studies:  Workforce  Development  

Strengths  &  Limita<ons  of  Our  Approach  
Strengths  
§ 

Our  survey  was  designed  to  test  our  if-­‐
then  statement  in  a  way  that  is  based  
on  youth’s  experience  of  a  program,  
and  less  conBngent  upon  program  
models  or  specific  program  acBviBes.    

§ 

The  survey  tool  built  upon  other  
measurement  tools  and  knowledge  in  
the  field  for  a  range  of  survey  secBons  
including  the  program  atmosphere,  
connecBon  to  the  environment,  self-­‐
efficacy,  and  support  secBons.  

§ 

 
9  

The  survey  was  administered  
anonymously,  increasing  the  likelihood  
of  candid  responses.  

Limita<ons  
§ 

While  surveys  are  useful  for  answering  
quesBons  like  “how  much?”  and  “to  
what  extent?”,  they  are  less  useful  for  
answering  quesBons  like  “how?”  or  
“why?”.  While  some  qualitaBve  data  
was  collected  in  the  survey,  the  lack  of  
addiBonal  qualitaBve  methods  –  such  
such  as  interviews  or  focus  groups  –  is  
an  inherent  limitaBon  of  the  design.  

§ 

Youth  responses  were  not  triangulated  
with  other  forms  of  program  data.  

§ 

Due  to  the  Bming  of  cluster  work,  both  
survey  design  and  data  collec<on  took  
place  in  the  fall.  For  the  programs  that  
operate  in  the  summer,  this  was  a"er  
the  interven<on,  limiBng  the  types  of  
data  that  could  be  collected  and  likely  
influencing  response  rates.  
LEAPS  Cluster  Studies:  Workforce  Development  

Addi<onal  Limita<ons  
§  Low  response  rates.  We  had  a  low  response  rate  to  the  
survey,1  which  creates  a  fewchallenges:    
o  It  may  introduce  bias.  In  parBcular,  we  are  most  
concerned  about  voluntary  response  bias.    
o  On  an  organizaBon  level,  too,  a  low  response  rate  
creates  an  issue  with  precision,  reducing  our  confidence  
in  whether  the  data  collected  from  our  sample  
represents  the  program  populaBon  more  broadly.    
 
Nevertheless,  we  find  the  data  valuable  for  looking  for  
pa9erns  and  exploring  our  if-­‐then  statement.  
1Response  rates:  CG  =  9/12,  SCA  =  18/63,  Rising  Sun  =  55/~300    

10  

LEAPS  Cluster  Studies:  Workforce  Development  

Use  of  Aggregated  Data  
§  We  only  aggregated  data  when  looking  at  assumpBons  and  
beliefs  that  the  programs  share  in  common,  when  looking  
for  cross-­‐program  pa@erns,  and  when  we  could  explore  
our  if-­‐then  statement  looking  at  the  rela<onship  between  
survey  measures.  
§  In  cases  where  we  aggregated  data  for  analysis,  we  have  
different  levels  of  representa<on  for  each  organiza<on,  
based  on  different  numbers  of  respondents.    
o  Specifically,  there  are  9  respondents  from  CommunityGrows,  18  
respondents  from  SCA,  and  55  respondents  from  Rising  Sun.    

1Response  rates:  CG  =  9/12,  SCA  =  18/63,  Rising  Sun  =  55/~300    

11  

LEAPS  Cluster  Studies:  Workforce  Development  

ParBcipant  Overview  

Survey  Respondent  Demographic  Data:  
Ethnicity  
Overall  Sample  

CommunityGrows  

65  

5  

Asian  

31%  

Black  /  African  
American  

23%  

Asian  
Black  /  African  
American  

LaBno  /  Hispanic  

19%  

White  /  Caucasian  

14%  

MulBracial  

13  

SCA  

16  

44  
Asian  
20%  

LaBno  /  Hispanic  

Pacific  Islander  /  NaBve  
Hawaiian  

Rising  Sun  

40%  

White  /  Caucasian  

36%  

Asian  

25%  

Black  /  African  
American  

21%  

Black  /  African  
American  

31%  

White  /  Caucasian  

21%  

LaBno  /  Hispanic  

31%  

LaBno  /  Hispanic  

MulBracial  

20%  

MulBracial  

Pacific  Islander  /  
NaBve  Hawaiian  

20%  

Pacific  Islander  /  
NaBve  Hawaiian  

11%  
9%  
2%  

White  /  Caucasian  
MulBracial  

13%  

Pacific  Islander  /  
NaBve  Hawaiian  

11%  

3%  

The  majority  of  the  survey  sample  is  youth  of  color.  Asian/
Asian  American  youth  (31%)  make  up  the  largest  
proporBon  of  youth  that  submiJed  the  survey.    
LEAPS  Cluster  Studies:  Workforce  Development  

Survey  Respondent  Demographic  Data:  
Gender  
Overall  Sample  

70  
Female  

CommunityGrows  

Rising  Sun  

4  

SCA  

17  

49  

Male  
Female  

Male  

Female  

Male  

Female  

Male  

25%  
39%  

40%  

47%  
61%  

60%  

53%  

75%  

Females  comprise  the  majority  of  the  sample  (60%).    

14  

LEAPS  Cluster  Studies:  Workforce  Development  

Survey  Respondent  Demographic  Data:  
Free  or  Reduced  lunch  
Overall  Sample  

69  

CommunityGrows  

5  

Rising  Sun  

SCA  

16  

48  
Yes  
33%  

Yes  
44%  

Yes  
63%  

Yes  
80%  
No  
60%  
No  
20%  

No  
49%  

I  do  not  
know  
7%  

15  

No  
25%  
I  do  not  
know  
6%  

I  do  not  
know  
13%  

The  sample  is  almost  evenly  split  by  youth  who  qualify  for  free  or  
reduced  lunch  (44%)  and  those  who  do  not(49%).  Seven  percent  
of  youth  were  unsure  if  they  qualify.    
LEAPS  Cluster  Studies:  Workforce  Development  

PaJerns  in  Outcomes  Across  
OrganizaBons  

Youth  surveyed  report  varying  levels  of  
proficiency  in  professional  skills  
Average    
(5-­‐point  scale)  
(n=53-­‐54)  

Photo  Credit:  Youth  Radio  

Photo  Credit:  Student  ConservaBon  AssociaBon  

§  Across  programs  surveyed,  dressing  
appropriately  and  following  direc<ons  were  
consistently  among  the  top  three  
professional  skills  with  which  youth  rate  
greatest  proficiency  (on  average).    
§  Similarly,  leading  peers,  handling  
disagreements  produc<vely,  and  being  
responsive  to  text  and  email  consistently  
scored  within  the  boJom  four  skills.  

17  

a.  Dressing  appropriately    

4.7  

b.  Following  direc<ons  

4.5  

c.  Showing  up  on  <me    

4.4  

d.  Knowing  my  schedule  

4.3  

e.  Working  well  with  a  supervisor  

4.3  

f.  Working  well  with  others  on  a  team  

4.3  

g.  Rela<ng  well  with  other  people  

4.2  

h.  Communica<ng  effec<vely  with  a  supervisor  

4.2  

i.  Managing  my  <me  

4.0  

j.  Being  responsive  to  texts  and  email  

4.0  

k.  Handling  disagreements  produc<vely    

3.9  

l.  Leading  my  peers  

3.9  

LEAPS  Cluster  Studies:  Workforce  Development  

Youth  Surveyed  Report  Increases  in  
Professional  Skills  As  A  Result  of  Par<cipa<ng  
In  my  Bme  with  working  with  BEETS  I've  
seen   myself   grow   in   communicaBon  
skills.  
-­‐CommunityGrows  ParBcipant  

18  

Average    
(4-­‐point  scale)  
(n=62-­‐72)  
a.  Communica<ng  effec<vely  with  a  supervisor  

3.1  

b.  Working  well  with  others  on  a  team  

3.1  

c.  Working  well  with  a  supervisor  

3.0  

d.  Rela<ng  well  with  other  people  

2.9  

§  Across  programs  surveyed,  no  clear  pa@ern  
emerged  regarding  top  four  and  boJom  four  
areas  of  improvement.  

e.  Managing  my  <me  

2.9  

f.  Following  direc<ons  

2.9  

§  Communica<ng  effec<vely  with  a  
supervisor  and  managing  my  <me  
consistently  appeared  within  the  top  half  of  
improvement  averages.    

g.  Showing  up  on  <me    

2.8  

h.  Leading  my  peers  

2.8  

i.  Knowing  my  schedule  

2.8  

§  Leading  my  peers  and  knowing  my  schedule  
consistently  appeared  in  the  boJom  half.  

j.  Handling  disagreements  produc<vely    

2.8  

k.  Dressing  appropriately  

2.7  

l.  Being  responsive  to  texts  and  email  

2.6  

LEAPS  Cluster  Studies:  Workforce  Development  

Youth  Surveyed  Report  Increased  
Connec<on  to  the  Environment  
Average  
 (4-­‐point  scale)  
(n=78)  
a.  It  makes  me  happy  to  see  people  take  care  of  the  environment  

3.2  

b.  I  feel  a  personal  responsibility  to  help  protect  the  environment  

3.1  

c.  I  am  interested  in  learning  more  about  the  environment  

3.1  

d.  I  am  aware  of  environmental  issues  facing  my  community  

3.0  

e.  I  feel  personally  impacted  by  environmental  condi<ons  in  my  
community  

2.9  

f.  I  speak  up  and  talk  with  friends  and  family  about  issues  that  
affect  the  environment  

2.8  

g.  I  par<cipate  in  ac<vi<es  that  help  the  environment  

2.7  

h.  I  am  interested  in  pursuing  a  career  that  has  to  do  with  the  
environment  

2.6  

The   program   exposed   me   to   issues   that   I   had   not   previously  
known…  I  will  use  what  I've  learned  to  spread  knowledge  and  
parBcipate  in  acBviBes  that  help  create  a  beJer  community.    
-­‐Student  ConservaBon  AssociaBon  ParBcipant  

19  

Photo  Credit:  Student  ConservaBon  AssociaBon  

§  “It  makes  me  happy  to  see  people  
take  care  of  the  environment”  and  
“I  am  aware  of  environmental  
issues  facing  my  community”  
consistently  appeared  within  the  top  
half  of  improvement  averages.  
§  “I  speak  up  and  talk  with  family  and  
friends  about  issues  that  affect  the  
environment”  and  “I  par<cipate  in  
ac<vi<es  that  help  the  
environment”  consistently  appeared  
in  the  boJom  half.  
LEAPS  Cluster  Studies:  Workforce  Development  

Digging  Deeper:  Speaking  Up  with  Friends  
and  Family  
§  Speaking  up  with  family  and  friends  
appears  to  be  an  outcome  that  may  
take  longer  to  develop.  

I   am   more   familiar   with   the   technical  
aspects   of   simple   eco-­‐friendly   choices,  
though   I   s<ll   struggle   in   sugges<ng  
these  changes  in  my  friends  and  family.    
-­‐Rising  Sun  ParBcipant  

§  Within  SCA,  we  found  there  is  a  strong  
staBsBcally  significant  relaBonship  between  
number  of  sessions  in  the  program  and  
improvement  in  speaking  up  and  talking  
with  family  and  friends  (p<.01).  
§  In  aggregate  we  found  that  greater  prior  
frequency  of  par<cipa<on  in  
environmentally-­‐focused  ac<vi<es  was  
correlated  with  greater  improvement  in  
speaking  up  and  talking  with  family  and  
friends  (p<.05).  
Photo  Credit:  CommunityGrows  

20  

LEAPS  Cluster  Studies:  Workforce  Development  

Youth  Report  Improvement  in  Self-­‐
Efficacy  
§  Youth  surveyed  report  improvement  in  self-­‐
efficacy,  including  improvement  in  their  
beliefs  about  future  workforce  success.  
Average  
 (4-­‐point  scale)  
(n=71-­‐74)  

21  

a.  Even  when  I  don’t  know  how  to  do  something  right  away,  
I  can  learn  how  

3.3  

b.  I  believe  I  will  succeed  in  my  future  jobs  

3.2  

c.  I  am  confident  that  I  can  do  well  on  many  different  
professional  tasks  

3.1  

d.  I  can  reach  my  goals  if  I  try  

3.1  

e.  I  can  remain  calm  when  facing  difficul<es  

3.1  

f.  When  I  am  confronted  with  a  problem,  I  can  usually  find  
several  solu<ons  

3.0  

g.  When  I  need  help,  I  know  where  to  go  

2.9  

Photo  Credit:  CommunityGrows  

§  “Even  when  I  don’t  know  how  to  do  
something  right  away,  I  can  learn  how”  
consistently  appeared  within  the  top  
two  improvement  averages.  
§  “When  I  need  help,  I  know  where  to  go”  
and  “I  can  remain  calm  when  facing  
difficul<es  consistently  appeared  in  the  
boJom  half  of  improvement  averages.  

LEAPS  Cluster  Studies:  Workforce  Development  

ConnecBons  between  Program  
Components  and  Outcomes  

Rela<onship  between  Relevancy  and  
Increased  Connec<on  to  the  Environment  
§  The  more  youth  had  the  opportunity  
to  work  on  a  relevant  project  or  job  
skill,  the  greater  growth  they  
reported  in  connec<on  to  the  
environment.  
§  All  eight  connecBon  to  the  
environment  measures  were  
posiBvely  correlated  with  having  
the  opportunity  to  work  on  a  
relevant  project  or  job  skill.  

Photo  Credit:  Rising  Sun  Energy  Center  

23  

Correla<on  with  
Relevancy  
(n  =  78)  
a.  It  makes  me  happy  to  see  people  take  care  of  
the  environment  

**  

b.  I  feel  a  personal  responsibility  to  help  protect  
the  environment  

***  

c.  I  am  interested  in  learning  more  about  the  
environment  

***  

d.  I  am  aware  of  environmental  issues  facing  my  
community  
e.  I  feel  personally  impacted  by  environmental  
condi<ons  in  my  community    
f.  I  speak  up  and  talk  with  friends  and  family  about  
issues  that  affect  the  environment    

*  
***  
**  

g.  I  par<cipate  in  ac<vi<es  that  help  the  
environment  

***  

h.  I  am  interested  in  pursuing  a  career  that  has  to  
do  with  the  environment  

***  

In  the  chart  above:    
•  Darkness  of  shading  corresponds  to  the  esBmated  effect  size  of  the  relaBonship:  n  =  small,  n  =  
moderate,  n  =  large  
•  Asterisks  (*)  represent  the  p-­‐value  (indicaBng  level  of  significance)  of  the  relaBonship:  *  =  p<.05,  
**  =  p<.01,  ***  =  p<.001    

LEAPS  Cluster  Studies:  Workforce  Development  

I've  always  wanted  to  work  with  animals  growing  up  
and  I  always  thought  that  it  had  to  be  direct  contact  
jobs  such  as  a  veterinarian,  but  during  my  job  at  SCA,  I  
learned  that  we  were  restoring  habitat  because  we  
wanted  to  rebuild  the  homes  of  different  animals.  
Restoring  habitat  was  an  indirect  way  to  help  animals  
and  when  I  learned  that,  I  felt  a  lot  more  passionate  
about  what  I  was  doing.  In  all,  the  job  experience  
opened  my  eyes  to  the  different  career  possibili<es  
that  have  the  same  goal  in  mind,  which  is  caring  for  
and  protecBng  animals.    

 

 -­‐SCA  ParBcipant  

Youth  Ra<ngs  of  Adult  Support  and  Modeling  Were  
Posi<vely  Correlated  with  Various  Outcomes  
§  Youth’s  raBngs  of  supporBve  adults  
were  correlated  with  some  environment  
and  self-­‐efficacy  measures.1    
§  Most  notably,  Bmely  enforcement  of  
rules  and  agreements  was  posiBvely  
correlated  with  seven  of  eight  
dimensions  of  improvement  in  
connecBon  to  the  environment.    
§  In  addiBon,  adults  appears  to  be  one  of  
the  few  program  model  characterisBcs  
correlated  with  dimensions  of  self-­‐
efficacy.  
The  thing  that  helped  me  the  most  to  boost  my  
confidence   and   self   esteem   were   my  
encouraging  and  understanding  managers.  
-­‐Rising  Sun  ParBcipant  

Photo  Credit:  CommunityGrows  

Prior   to   the   program   I   thought   I   couldn't   deal  
with  problems  on  a  crew.  But  this  past  summer  
showed   me   that   not   all   crews   were   nice   and  
calm   and   problems   can   and   will   happen.   So  
even   with   the   problems   in   my   crew   I   was   able  
to   deal   with   them   well.   I   also   discussed   these  
issues   with   my   supervisors   and   learned   a   lot  
about  conflict  mediaBon.    
-­‐Student  ConservaBon  AssociaBon  ParBcipant  

1See  Appendix  A  for  more  details  

25  

LEAPS  Cluster  Studies:  Workforce  Development  

Posi<ve  Connec<on  between  Collabora<on  
and  Environmental  Outcomes  
§  Par<cipants  who  reported  
greater  agreement  that  
they  regularly  collaborate  
and  share  opinions  also  
tended  to  report  greater  
improvement  in  connecBon  
with  the  environment.  

In  this  program,  we  
regularly  collaborate,  
share  our  opinions  with  
others,  and  work  in  teams  
(n  =  78)  

a.  It  makes  me  happy  to  see  people  take  care  of  the  
environment  
b.  I  feel  a  personal  responsibility  to  help  protect  the  
environment  
c.  I  am  interested  in  learning  more  about  the  
environment  

**  
*  

d.  I  am  aware  of  environmental  issues  facing  my  
community  
e.  I  feel  personally  impacted  by  environmental  
condi<ons  in  my  community  
f.  I  speak  up  and  talk  with  friends  and  family  about  
issues  that  affect  the  environment  
g.  I  par<cipate  in  ac<vi<es  that  help  the  
environment    
h.  I  am  interested  in  pursuing  a  career  that  has  to  
do  with  the  environment  
Photo  Credit:  Student  ConservaBon  AssociaBon  

26  

**  
*  
**  
*  

In  the  chart  above:    
•  Darkness  of  shading  corresponds  to  the  esBmated  effect  size  of  the  relaBonship:  n  =  small,  n  =  
moderate,  n  =  large  
•  Asterisks  (*)  represent  the  p-­‐value  (indicaBng  level  of  significance)  of  the  relaBonship:  *  =  p<.05,  
**  =  p<.01,  ***  =  p<.001    

LEAPS  Cluster  Studies:  Workforce  Development  

Connec<ons  Between  Different  Types  of  
Outcomes  
Is  improvement  in  one  set  of  outcomes  -­‐  such  as  professional  skills,  
connec@on  to  the  environment,  or  self  efficacy  –  correlated  with  
improvement  in  other  outcome  measures?  
§  We  saw  many  sta<s<cally  
significant  posi<ve  
correla<ons  between  
different  types  of  outcomes,  
telling  us  that  youth  who  
tended  to  report  greater  
improvement  in  one  
dimension,  also  tended  to  
report  greater  improvement  
in  others.1  
1See  Appendix  B  for  more  details  

Photo  Credit:  CommunityGrows  

27  

LEAPS  Cluster  Studies:  Workforce  Development  

Connec<ons  between  Improvement  in  Professional  
Skills  and  Improvement  in  Self  Efficacy  
§  Improvement  in  
professional  skills  emerged  
as  related  to  three  specific  
dimensions  of  
improvement  in  self  
efficacy:    
o  “I  believe  I  will  succeed  in  
my  future  jobs.”  
o  “I  am  confident  that  I  can  
do  well  on  many  different  
professional  tasks.”  
o  “I  can  reach  my  goals  if  I  
try.”    
Being   able   to   communicate  
and   work   beJer   with   clients  
and   customers.   This   gave   me  
higher  self  esteem.  
-­‐Rising  Sun  ParBcipant  

28  

a.  Communica<ng  effec<vely  
with  a  supervisor  
b.  Working  well  with  others  
on  a  team  
c.  Working  well  with  a  
supervisor  
d.  Rela<ng  well  with  other  
people  

I  believe  I  will  
succeed  in  my  
future  jobs  
(n=59-­‐68)  

I  am  confident  that  I  can  
do  well  on  many  different  
professional  tasks  
(n  =  58-­‐67)  

I  can  reach  my  
goals  if  I  try  
(n=59-­‐69)  

**  

**  

**  

*  

*  

**  

***  

**  

*  

***  

e.  Managing  my  <me  

**  

*  

**  

f.  Following  direc<ons  

***  

*  

***  

g.  Showing  up  on  <me    

***  

**  

***  

h.  Leading  my  peers  

*  

**  

**  

i.  Knowing  my  schedule  

**  

*  

***  

j.  Handling  disagreements  
produc<vely    

*  

*  

*  

k.  Dressing  appropriately  

***  

**  

***  

l.  Being  responsive  to  texts  
and  email  

***  

**  

***  

In  the  chart  above:    
•  Darkness  of  shading  corresponds  to  the  esBmated  effect  size  of  the  relaBonship:  n  =  small,  n  =  moderate,  n  =  large  
•  Asterisks  (*)  represent  the  p-­‐value  (indicaBng  level  of  significance)  of  the  relaBonship:  *  =  p<.05,  **  =  p<.01,  ***  =  p<.
001    

LEAPS  Cluster  Studies:  Workforce  Development  

Digging  Deeper:  Youth  Comments  Related  
to  Self  Efficacy  
§  In  various  open-­‐ended  
responses,  youth  commented  
on  the  way  in  which  certain  
elements  of  the  program  
boosted  their  self  esteem  and  
confidence.  

Before  I  began  the  program,  I  was  very  nervous  about  
facing   challenges,   especially   ones   that   seemed  
overwhelming.   Now   I   feel   much   more   confident   in  
myself   and   the   people   around   me   when   taking   on  
heavy   tasks.   Having   the   group   set   a   goal   helped   set   a  
goal-­‐driven   atmosphere   and   that   is   essenBal   towards  
geSng   tasks   done.   I   became   moBvated   to   accomplish  
the  tasks  as  I  did  not  want  to  let  the  team  down.    
-­‐Rising  Sun  ParBcipant  

The   overall   work   experience,  
which  I've  never  had  before  this  is  
what   increased   my   confidence   in  
my   abiliBes.   I   was   able   to   finally  
see   for   this   first   Bme   how   I   would  
actually  do.  
-­‐Student  ConservaBon  AssociaBon  
ParBcipant  

Photo  Credit:  Rising  Sun  Energy  Center  

29  

LEAPS  Cluster  Studies:  Workforce  Development  

CompensaBon  

How  Important  Is  Compensa<on?  
How  important  was  it  to  you  to  have  a  
paid  internship/posi<on?  (First  Year)  

How  important  was  it  to  you  to  have  a  
paid  internship/posi<on?  (Returning)  

Average  =  3.5  
(n  =  45)  

Average  =  2.9  
(n  =  24)  
33%  

64%  

33%  

25%  

2%  

13%  

20%  

Not  At  All   Somewhat   Important  
Very  
Important   Important  
Important  

8%  

Not  At  All   Somewhat   Important  
Very  
Important   Important  
Important  

§  While  84%  of  first  year  respondents  report  that  compensaBon  was  important  or  very  
important  to  them,  only  66%  of  those  who  have  been  in  the  program  for  more  than  a  year  
rate  it  as  important  or  very  important.  

31  

LEAPS  Cluster  Studies:  Workforce  Development  

Compensa<on  A@racts  Par<cipants  Who  
May  Not  Have  Par<cipated  Otherwise  
If  your  posi<on  were  not  paid,  would  you  s<ll  
have  chosen  to  par<cipate?  (First  Year)  

If  your  posi<on  were  not  paid,  would  you  s<ll  
have  chosen  to  par<cipate?  (Returning)  

Average  =  2.6  
(n  =  45)  
33%  

63%  

31%  
20%  

16%  

Yes  

Average  =  1.8  
(n  =  24)  

Maybe  

Probably  
Not  

No  

29%  

Yes  

Maybe  

4%  

4%  

Probably  
Not  

No  

§  While  just  over  half  (51%)  of  first  year  parBcipants  say  that  they  either  would  not  or  probably  
would  not  have  parBcipated  if  the  posiBon  were  not  paid,  that  percent  drops  to  8%  of  those  
who  have  been  in  the  program  for  more  than  a  year.  

32  

LEAPS  Cluster  Studies:  Workforce  Development  

It  gave  me  confidence  and  told  me  that  
yes,  I  can  make  money,  and  that  I'm  more  
than  just  a  high  school  student.  

-­‐Rising  Sun  ParBcipant  
   

Avenues  for  Future  EvaluaBon  and  
Learning  

Avenues  for  Future  Evalua<on  and  
Learning    
As  organizaBons  conBnue  to  explore  their  effecBveness,  they  may  choose  to:    
§  ConBnue  to  collect  program  data,  re-­‐running  important  analyses  to  beJer  
understand  these  relaBonships  on  an  organizaBon-­‐specific  level  (here  we  
looked  at  aggregate  results).  Moreover,  having  a  greater  range  of  the  
organizaBon's  parBcipants  involved  may  change  the  relaBonships  organizaBons  
see.    
§  Look  at  composite  scores  for  some  of  the  core  concepts  –  professional  skills,  
connecBon  to  the  environment,  and  self-­‐efficacy  –  that  we  explored  through  
individual  items.  
§  Consider  more  qualitaBve  evaluaBon  to  explore  the  “how”  and  “why”  of  some  
of  the  links  that  emerge  in  the  quanBtaBve  data.  
§  Explore  potenBal  drivers  of  change  for  workforce  and  self-­‐efficacy  outcomes.  
§  Collect  data  alongside  program  delivery  to  ensure  high  response  rates.    

35  

LEAPS  Cluster  Studies:  Workforce  Development  

Takeaways  for  the  Field  

Takeaways  for  the  Field  
§  For  environmental  organiza<ons  looking  to  
increase  the  connecBon  to  the  environment,  
creaBng  opportuniBes  for  adolescents  to  work  on  
projects  or  skills  they  care  about  may  support  
greater  increases  in  connec<on  to  the  
environment,  including  greater  interest  in  
environmentally-­‐focused  careers.  
§  For  environmental  educaBon  programs  that  serve  
adolescents,  providing  a  workforce  opportunity  
may  a@ract  youth  who  might  otherwise  not  be  
inclined  to  par<cipate,  and  may  further  support  
their  sense  of  self-­‐efficacy.  
37  

LEAPS  Cluster  Studies:  Workforce  Development  

Appendices  

Appendix  A:  Tables  Related  to  Suppor<ve  
Adults  
Seven  of  eight  connec<on  to  the  environment  dimensions  were  posi<vely  correlated  with  
<mely  enforcement  of  rules  and  agreements.    
Adult  Modeling  

Suppor<ve  Adults  

a.  It  makes  me  happy  to  see  people  
take  care  of  the  environment  

Adults  in  this  
program  
want  to  see  
me  succeed.  
(n=72)  

I  would  feel  comfortable  
asking  a  program  staff  
member,  adult  mentor  or  
professional  if  I  needed  a  
reference  for  school  or  a  job  
(n=71)  

There  is  someone  in  this  
program  –  a  staff  person,  
mentor,  professional,  or  
other  adult  –  who  I  can  
go  to  for  help  when  
making  decisions  that  are  
important  to  me  
(n=72)  

**  

**  

**  

Instructors,  staff,  
When  someone  
and/or  other  leaders   breaks  our  rules  or  
in  this  program  “walk   agreements  in  this  
their  talk”  by  
program,  the  issue  
modeling  the  
is  addressed  
behavior  they  expect  
rather  than  
of  us  
ignored  
(n=78)  
(n=78)  

b.  I  feel  a  personal  responsibility  to  
help  protect  the  environment  

***  

c.  I  am  interested  in  learning  more  
about  the  environment  
d.  I  am  aware  of  environmental  
issues  facing  my  community  

*  
**  

*  

*  

***  
*  

e.  I  feel  personally  impacted  by  
environmental  condi<ons  in  my  
community  

**  

f.  I  speak  up  and  talk  with  friends  
and  family  about  issues  that  affect  
the  environment  

**  

g.  I  par<cipate  in  ac<vi<es  that  help  
the  environment  
h.  I  am  interested  in  pursuing  a  
career  that  has  to  do  with  the  
environment  

*  
*  

*  

***  

*  

*  

Appendix  A:  Tables  Related  to  Suppor<ve  
Adults  
To  some  extent,  par<cipants  who  reported  greater  agreement  with  various  
statements  about  suppor<ve  adults  also  tended  to  report  greater  improvement  in  
self-­‐efficacy.    
Suppor<ve  Adults  
Adult  Modeling  

a.  Even  when  I  don’t  know  how  to  do  
something  right  away,  I  can  learn  how  

Adults  in  this  
program  want  to  
see  me  succeed  
(n=69-­‐72)  

I  would  feel  
comfortable  asking  a  
program  staff  member,  
adult  mentor  or  
professional  if  I  needed  
a  reference  for  school  
or  a  job  
(n=69-­‐71)  

*  

*  

Instructors,  staff,  and/or  
When  someone  
other  leaders  in  this  
breaks  our  rules  or  
program  “walk  their  
agreements  in  this  
talk”  by  modeling  the   program,  the  issue  is  
behavior  they  expect  of  
addressed  rather  
us  
than  ignored  
(n=71-­‐74)  
(n=71-­‐74)  

*  

b.  I  believe  I  will  succeed  in  my  future  jobs.  
c.  I  am  confident  that  I  can  do  well  on  many  
different  professional  tasks  

There  is  someone  in  this  
program  –  a  staff  person,  
mentor,  professional,  or  
other  adult  –  who  I  can  go  
to  for  help  when  making  
decisions  that  are  
important  to  me  
(n=69-­‐72)  

*  

**  

*  

d.  I  can  reach  my  goals  if  I  try  

*  

e.  I  can  remain  calm  when  facing  difficul<es  

f.  When  I  am  confronted  with  a  problem,  I  
can  usually  find  several  solu<ons.  

g.  When  I  need  help,  I  know  where  to  go  

*  

*  
**  

**  

*  

Appendix  B:  Connec<ons  Between  Outcomes  
Par<cipants  who  reported  greater  improvement  on  self  efficacy  measures  also  tended  
to  report  greater  improvement  in  connec<on  to  the  environment.    
Even  when  I  don’t  
know  how  to  do  
something  right  
away,  I  can  learn  
how  
(n=72)  

I  believe  I  will  
succeed  in  my  
future  jobs  
(n=74)  

I  am  confident  that  I  
can  do  well  on  many  
different  professional  
tasks  
(n=73)  

I  can  reach  
my  goals  if  I  
try  
(n=74)  

I  can  remain  
calm  when  
facing  
difficul<es  
(n=71)  

When  I  am  confronted  
with  a  problem,  I  can  
usually  find  several  
solu<ons  
(n=73)  

When  I  need  
help,  I  know  
where  to  go  
(n=72)  

**  

**  

***  

*  

**  

***  

***  

b.  I  feel  a  personal  
responsibility  to  help  
protect  the  environment  

*  

*  

*  

*  

c.  I  am  interested  in  
learning  more  about  the  
environment  

**  

***  

*  

**  

d.  I  am  aware  of  
environmental  issues  facing  
my  community  

**  

**  

**  

**  

*  

**  

*  

**  

*  

*  

a.  It  makes  me  happy  to  
see  people  take  care  of  the  
environment  

e.  I  feel  personally  
impacted  by  environmental  
condi<ons  in  my  
community  
f.  I  speak  up  and  talk  with  
friends  and  family  about  
issues  that  affect  the  
environment  

**  

g.  I  par<cipate  in  ac<vi<es  
that  help  the  environment  
h.  I  am  interested  in  
pursuing  a  career  that  has  
to  do  with  the  environment  

***  

**  

*  

**  

**  

*  

*  

**  

**  

**  

*  

***  

**  

*  

***  

*  
*  

*  

*  
*  

**  

Appendix  B:  Connec<ons  Between  Outcomes  
Youth  who  tended  to  report  greater  improvement  in  professional  skills  also  tended  to  report  
greater  improvement  in  connec<on  to  the  environment  (and  vice  versa).    

a.  Communica<ng  
effec<vely  with  a  
supervisor  

It  makes  me  
happy  to  see  
people  take  
care  of  the  
environment  
(n=62-­‐72)  

I  feel  a  personal  
responsibility  to  
help  protect  the  
environment  
(n=62-­‐72)  

I  am  interested  
in  learning  
more  about  the  
environment    
(n=62-­‐72)  

I  am  aware  of  
environmental  
issues  facing  
my  community    
(n=62-­‐72)  

I  feel  personally  
impacted  by  
environmental  
condi<ons  in  my  
community  
(n=62-­‐72)  

I  speak  up  and  
talk  with  friends  
and  family  about  
issues  that  affect  
the  environment  
(n=62-­‐72)  

I  par<cipate  
in  ac<vi<es  
that  help  the  
environment  
(n=62-­‐72)    
 

**  

**  

**  

**  

**  

*  

*  
*  

**  

**  

**  

b.  Working  well  with  
others  on  a  team  

*  

c.  Working  well  with  a  
supervisor  

I  am  interested  
in  pursuing  a  
career  that  has  
to  do  with  the  
environment  
(n=62-­‐72)  

**  

**  

**  

*  

***  

**  

d.  Rela<ng  well  with  
other  people  

*  

***  

***  

**  

***  

***  

e.  Managing  my  <me  

**  

***  

***  

*  

***  

**  

f.  Following  direc<ons  

*  

**  

***  

**  

***  

**  

**  

**  

*  

**  

**  

**  

**  

***  

**  

g.  Showing  up  on  <me    

**  

**  

*  

***  

**  

*  

**  

**  

*  

*  

j.  Handling  
disagreements  
produc<vely    

**  

**  

***  

**  

k.  Dressing  
appropriately  

*  

***  

*  

**  

**  

*  

*  

*  

h.  Leading  my  peers  
i.  Knowing  my  schedule  

l.  Being  responsive  to  
texts  and  email  

**  

*  

**  
*  

*  

For  more  informaBon  about  the  Workforce  Development  Cluster  Study,  
please  contact  Frances  Tompkins  (Frances@TogetherBrave.com)  or  
Jessica  Xiomara  García  (Jessica.Xiomara.Garcia@learningforacBon.com).    

www.learningforacBon.com  
170  Capp  Street,  Suite  C        |        San  Francisco,  CA  94110