National 4-H Science

Leadership Academy

Year Three
National 4-H Science E-Academy Pre-Program Needs Assessment Report
September 30, 2012

Mary E. Arnold, Ph.D. Project Evaluator Oregon State University
With assistance from

Courtney Archibeque, MpH Graduate Research Assistant Oregon State University

Table of Contents

Evaluator’s Statement Acknowledgements Executive Summary Overview Respondent Demographics Personal Interest in 4-H Science

………………………………………………………………….. ii ………………………………………………………………….. iii ………………………………………………………………….. iv ………………………………………………………………….. 1 ………………………………………………………………….. 3 ………………………………………………………………….. 3 ………………………………….……….. 5 ……….……………….………….……… 7

Knowledge and Use of 4-H Science Tools Increasing the Use of 4-H Science Tools Content of the E-Academy E-Academy Format Time Commitment Pre and Post Academy Assignments Format of E-Academy Sessions Technological Capabilities Summary and Recommendations

………………………………………...………….………….... 9 ………………………………………………………..………... 16 ………………………………………………………..………... 18 ………………………………………………………..………... 18 ………………………………………………………..………... 18 ………………………………………………………..………... 20 ……………………………………………………... 21 ..……………………………... 23

Appendix ONE: Needs Assessment Respondent Demographics

Evaluator’s Statement
This document serves as the final evaluation report for the National 4-H Science Academy: Year Three E-Academy Needs Assessment sponsored by National 4-H Council with funding from the Noyce Foundation. The academies were held in five locations across four regions of the country between January and April, 2012. All academy participants were invited to participate in the program evaluation. All data for the evaluation were entered by participants directly into an on-line data collection system. Access to the system was provided by the evaluator to the participants for data entry, but only the evaluator and her research assistants had access to the actual dataset. The integrity and accuracy of the raw data rests with the individual participants. The integrity and accuracy of the analysis and interpretation rests solely with me as the project evaluator. To this end, I certify that the analysis and results presented in this document are complete and accurate insofar as the data entered by the participants were as well. Any questions or concerns about this report should be addressed to me.

Mary E. Arnold, Ph.D. Project Evaluator, Oregon State University September 30, 2012 mary.arnold@oregonstate.edu

ii

Acknowledgements
This needs assessment could not have taken place without the dedicated help and support of many individuals. First and foremost, I would like to thank National 4-H Council and the Noyce Foundation for the opportunity to conduct the Year Three Virtual Academy evaluation, which includes this formal needs assessment. It continues to be exciting to witness the movement of the 4-H Science professional development program from the national to the regional, and now to the local level, and to document the important work that is taking place as a result of the Academy. I would like to express my thanks and gratitude to Edward Bender, Janet Golden, Jo Turner, Beth Bernstihl, and Maila Oliveria at National 4-H Council for their support and help with various aspects of the needs assessment process. Thank you, also, to the members of the Virtual Academy Leadership Team (VAULT) for your thoughtful contributions to developing the needs assessment instrument. Debbie Nistler, President of NA4-HA was very gracious in allowing us to send e-mail invitations through their e-mail contacts to reach as many front line 4-H educators as possible. A special thank you to Debbie and the NAE4-HA staff for making this happen and for sending out the invitations and reminders. Similarly, I would like to extend my thanks and gratitude to all of the 4-H Science Liaisons across the country who forwarded invitations to participate in the survey to their local 4-H contacts. A very special thank you to my co-authors and graduate research assistant Ms. Courtney Archibeque. Your focused assistance with the data analysis and report preparation could not be replaced. Thank you especially for your cheerful willingness to concentrate your work time on the report so we could meet the expected deadline. I would like to thank each and every 4-H educator who contributed data. The sincerity with which you approached the evaluation was evident in the data and information you provided. Your responses will guide the work of the VALT as they plan the E-Academy based on the interests and needs you shared through this process. Finally, thank you to the Noyce Foundation for the generous support of the National 4-H Science Leadership Academy. The funding provided by the foundation made this important program possible. As a result, 4-H programs across the country are more prepared to develop and sustain programs for youth in science, technology, engineering, and math.

iii

Executive Summary
Respondent Demographics We received 504 responses from 48 states, with the big 4-H science states showing the greatest
participation (e.g. Illinois, Kentucky, Minnesota, and Pennsylvania). The demographic data indicate that the intended audience (i.e. frontline county agents and educators) was reached through the needs assessment process. Additional demographics show that • 81% of respondents were agents or educators at the county level • 92% were female • 88% were NOT in a science specific role • 9.7% attended the National Science Academy in 2010 • 32 % attended a regional academy in 2012

Personal interest in 4-H Science • Over 80% indicated a high or very high personal interest in professional development for 4-H Science • Over 82% indicated a commitment to learn and develop skills for 4-H Science programming • Almost 80% indicated positive support in their work environment for implementing 4-H Science programs Use of 4-H Science Professional Development Resources • Most respondents have NOT HEARD of the national tools available to support 4-H Science. • The exception is for specific training guides, such as Junk Drawer Robotics, and The Power of Wind; 76.8% had HEARD about these resources • Even less have USED these resources, with only 56.5% reporting using specific training guides • 70% of respondents reported that knowing more about the resources would increase the likelihood that they would use them Proposed Content of the E-Academy The topics respondents were most interested in learning about are**: • Tools and resources available to support 4-H Science programs (Doing Science) • Teaching others to inquiry-based science experiences (Doing Science)
iv

How to engage volunteers in 4-H Science as coaches or mentors (Volunteer Development) • How to help traditional volunteers facilitate 4-H Science programs (Volunteer Development) • How to effectively train volunteers in 4-H Science AND PYD principles (Volunteer Development) ______________________________________________ ** 75% or more are moderately to very interested in this topic The topics respondents were next most interested in learning about are***: • Facilitating inquiry-based science experiences (Doing Science) • Scientific engineering practices framework for K-12 Science (Doing Science) • Facilitating science programs to limited resource and/or non-traditional audiences (Doing Science) • How to recruit science-rich volunteers (Volunteer Development) • How to include inquiry in all science programs (Curriculum) • An overview of current 4-H Science curriculum resources (Curriculum) • Strategies for engaging teens as teachers in 4-H Science programs (Curriculum) ______________________________________________ *** 60% or more are moderately to very interested in this topic E-Academy Format Respondents were asked to indicate their preference for the format of the E-academy. This is the only place where the needs assessment data differ in important ways when the respondents are separated out by group. For this analysis we separated out those who are county educators from all other respondents, and in doing so, found an important difference.  Interactive e-learning sessions consisting of presentations with interactive components received the highest overall average rating (4.0/5.0)  Facilitated interactive learning: consisting of presentations viewed as a group at a local site with a facilitator trained to conduct breakout sessions received the greatest number of “high” ratings  BUT- if you consider only the responses from county-level educators the Facilitated Interactive Learning is the most preferred (4.8/5.0 mean rating)

v

Time Commitment for an E-Academy Work
Respondents were asked to indicate the amount of time they are willing to commit to participating in the E-Academy: • 69.8% for ONE part day e-academy (up to four hours total) • 61.5% for ONE full-day academy (5+ hours) • 59.7% for an academy series over 3-5 months with part days of up to 4 hours • 14.9% for three or more full consecutive days • 26.8% for Three or more part consecutive days

Delivery of E-Academy Sessions Respondents were asked their preference for the delivery of the E-Academy session. Only two types of delivery received moderate positive ratings:
• • 71.2% webinar presentations (both live and recorded) 64.9% video presentations

Technological Capacity  It is important to note that while 70% indicated they have attended webinars successfully, this also means that almost 30% have not  In addition, only 58% indicated they have adequate technological support locally in the event of problems with participation that are caused by local equipment  And one out of four people (25%) report they do not have an appropriate setting, such as a quiet room for participating in the e-academy

Summary and Recommendations The results of the needs assessment are, overall, promising for the success of the E-Academy. In particular, the respondents seem interested in the proposed content of the E-Academy, and indicate they are willing to commit time to attending the academy. There are a few important things to note that should be taken into consideration by the E-Academy planners, however. 1. Respondents indicated the greatest interest in content related to science program implementation (Doing Science) and Volunteer Development. Particular aspects of Curriculum also were rated moderately high. There are several items that stand out clearly as interest areas for this audience and should be kept in mind when the content of the academy is planned. 2. The proposed content for the Evaluation and Fund Development areas received low to moderate ratings. The planners should plan content in these areas with the audience in
vi

mind and not plan a program that is not a match with the skills and needs of countybased educators. 3. County educators indicated a very strong preference for the format of the E-Academy to be a facilitated session. While creating this type of session is beyond the purview of the E-Academy per se, some attention should be paid to providing information to states and counties about how to plan for and host the E-Academy at the local level, including a recommendation that learning be done in a facilitated session if possible. 4. Care should be taken to be sure the technology used for the delivery of the E-Academy is tested for potential problems that may occur when the educator participates in the academy. The results of the needs assessment revealed just a fair to moderate technological capacity of the educators. Especially noteworthy is that almost ½ of respondents reported having no on-site technical assistance to help them. Advanced training, clear instructions for participation, and technical support will all be key to the success of the E-Academy.

vii

Year Three: National 4-H Science E-Academy Pre-Program Needs Assessment Overview
The aim of year three of the National 4-H Science Academy is to provide a professional development program for county-based 4-H educators. The program design for year three follows a natural progression from the first academy, held in December, 2010 that reached teams of four from LGUs, to last year’s regional academies that reached a broader audience, to the E-Academy that is intended to reach as many front-line educators as possible. This progression marks the overall development, scope, and reach of the National 4-H Science Academy program. A development team entitled the Virtual Academy Leadership Team (VALT) was formed in the early summer of 2012, under the leadership of Edward Bender. Team members are:  Edward Bender, National 4-H Council  Janet Golden, National 4-H Council  Jim Kahler, NIFA  Suzanne Le Menestrel, NIFA  Doug Swanson, NIFA  Misty Blue-Terry, North Carolina A & T University (1890s)  Trudy Dunham, University of Minnesota (North Central)  Debbie Fajans, University of Vermont (Northeast  Heather Kent, University of Florida (South)  Steven Worker, University of California, Davis (West)  Mary Arnold, Oregon State University (Evaluator) A face to face meeting of the VALT was held in July at the National 4-H Conference. At the meeting the team was charged with the development and delivery of the E-Academy, and began the initial planning phase. Part of the meeting was devoted to a review of the evaluations of the Year One and Two academies, which informed some of the initial planning. One of the end products of the July VALT meeting was the development of the instrument for the pre-program needs assessment. Following the meeting the evaluator drafted the initial instrument based on discussions and planning at the meeting, and worked via e-mail with VALT members to refine and finalize the needs assessment instrument. The needs assessment focused on the following distinct areas:   Respondent Demographics Personal interest in 4-H Science
1

 

Current use of 4-H Science Professional Development Resources Proposed Content of the E-Academy o 4-H Science Program Development and Delivery o Volunteer Development o Curriculum and Program Development o Evaluation o Resource Development Structure and Participation in the E-Academy o The Focus of the E-Academy o Time Commitment and E-Academy Work o Types of E-Academy Sessions o Technological Capacity

Respondents to the needs assessment were recruited through LGU Science Liaisons and through direct e-mails from NAE4-HA to its members. E-mail invitations were sent at least two times through each method, with some liaisons following up more than one time with a reminder. Data were collected on-line through a commercial survey company, and no problems were reported with the data collection process. Data were collected between August 1 and August 20, 2012. Responses were downloaded from the on-line collector into an Excel file and finally into SPSS for analysis.

2

Respondent Demographics
In all, we received 504 responses from 48 states, with some of the bigger 4-H science states showing the greatest participation (e.g. Illinois, Kentucky, Minnesota, Pennsylvania). Additional demographics show that • 81% of respondents were agents or educators at the county level • 92% were female • 88% were NOT in a science specific role • 9.7% attended the National Science Academy in 2010 • 32 % attended a regional academy in 2012 The demographic data indicate that the intended audience (i.e. frontline county agents and educators) was reached through the needs assessment process. A detailed analysis of the respondent demographics can be found in Appendix One. It is important to note that while the respondent demographics did match those of the intended audience, the N of 504 is small relative to the estimated population size of over 3,000. The demographic results, however, do indicate that the responses came from a fair representation of educators across the country, with one exception: Because of the relatively small N, and the percentage of the respondents who indicate a moderate to strong interest in science, it may be safe to assume that these results represent those who are most interested in promoting 4-H Science, and not the overall body of 4-H agents/educators in general. Nonetheless, the E-Academy is being developed as a “train-the-trainer” model, thus the development of the E-Academy should be geared toward those on the local level who show an interest in 4-H Science, as they will most likely become the trainers of others at the local level. To this end, the responses provided in the needs assessment appear to be valuable and meaningful.

Personal Interest in 4-H Science
• • • Over 80% indicated a high or very high personal interest in professional development for 4-H Science Over 82% indicated a commitment to learn and develop skills for 4-H Science programming Almost 80% indicated positive support in their work environment for implementing 4-H Science programs

Table 1.0 represents respondent’s personal interest in 4-H Science. Participants rated “desire to learn and develop skills for developing 4-H Science programs” as the area in which there is
3

the most personal interest. The least interest is in “level of positive support in your work environment for implementing 4-H science programs.” Figure 1.0 shows the percentage of respondents who rated their interest in each item “high” or “very high.”

Table 1.0 Personal Interest in 4-H Science
Very High High and Very High

Very Low

Low

High

Missing

Desire to learn and develop skills for developing 4-H Science programs Commitment to learn and develop skills for developing 4-H Science programs Ability to implement what you learn in order to enhance 4-H Science programming Ability to use what you learn in order to enhance 4-H Science programming Level of positive support in your work environment for implementing 4-H Science programs

3 0.6% 0 2 0.4% 0 9 1.8%

48 9.5% 62 12.3% 45 8.9% 30 6.0% 68 13.5%

234 46.4% 229 45.4% 279 55.4% 277 55.0% 266 52.8%

193 38.3% 187 37.1% 149 29.6% 167 33.1% 134 26.6%

427 84.7% 416 82.5% 428 85.0% 394 88.1% 400 79.4%

26 5.2% 26 5.2% 29 5.8% 30 6.0% 27 5.4%

Figure 1.0
100% 80% 60% 40% 20% 0% Desire to learn Commitment to Ability to Ability to use and develop skills learn and develop implement what what you learn in for developing 4skills for you learn in order order to enhance H Science developing 4-H to enhance 4-H 4-H Science programs Science programs Science programming programming Level of positive support in your work environment for implementing 4-H Science programs 55.4% 46.4% 45.4% 55.0% 52.8% Very High High 38.3% 37.1% 29.6% 33.1% 26.6%

4

Knowledge and Use of 4-H Science Tools
• • • • Most respondents have NOT HEARD of the national tools available to support 4-H Science. The exception is for specific training guides, such as Junk Drawer Robotics, and The Power of Wind; 76.8% had HEARD about these resources Even less have USED these resources, with only 56.5% reporting using specific training guides Table 2.0 presents the awareness of participants of various tools used for 4-H Science Table 2.1 shows those who have used them. The most heard of tool was the “Specific Project Training Guides (Junk Drawer Robotics, Power of Wind)” and the least heard of tool was the “4-H Science Smart Competency Training Guide.” Respectively, these two tools were also the most and the least used by respondents.

Table 2.0 Knowledge of 4-H Science Tools
Heard Of:
4-H Science Smart Competency Training Guide Using inquiry-based learning to support 4-H science on-line course 4-H Science checklist 4-H Science 101 Training Guide 4-H Science Competencies Inquiry-based learning online learning modules Fund Development Toolkit 4-H Science Logic Model Promising Practices Specific Project Training Guides (Junk Drawer Robotics, Power of Wind) Recruiting and Developing Volunteers Archived 4-H Science Academy Webinars Curriculum Rubrics Youth Engagement, Attitudes and Knowledge (YEAK) Survey

Yes
71 14.1% 156 31.0% 177 35.1% 147 29.2% 237 47.0% 174 34.5% 165 32.7% 260 51.6% 135 26.8% 387 76.8% 287 56.9% 144 28.6% 143 28.4% 152 30.2%

No
391 77.6% 306 60.7% 283 56.2% 314 62.3% 223 44.2% 286 56.7% 295 58.5% 202 40.1% 324 64.3% 77 15.3% 173 34.3% 316 62.7% 312 61.9% 306 60.7%

Missing
42 8.3% 42 8.3% 44 8.7% 43 8.5% 44 8.7% 44 8.7% 44 8.7% 42 8.3% 45 8.9% 40 7.9% 44 8.7% 44 8.7% 49 9.7% 41 9.1%

5

Table 2.1 Use of 4-H Science Tools
Used:
4-H Science Smart Competency Training Guide Using Inquiry-Based Learning to Support 4-H Science on-line Course 4-H Science Checklist 4-H Science 101 Training Guide 4-H Science Competencies Inquiry-Based Learning Online Learning Modules Fund Development Toolkit 4-H Science Logic Model Promising Practices Specific Project Training Guides (Junk Drawer Robotics, Power of Wind) Recruiting and Developing Volunteers Archived 4-H Science Academy Webinars Curriculum Rubrics Youth Engagement, Attitudes and Knowledge (YEAK) Survey

Yes
26 5.2% 60 11.9% 122 24.2% 89 17.7% 140 27.8% 65 12.9% 37 7.3% 139 27.6% 68 13.5% 285 56.5% 153 30.4% 51 10.1% 56 11.1% 72 14.3%

No
429 85.1% 397 78.8% 335 66.5% 369 73.2% 318 63.1% 393 78.0% 418 82.9% 318 63.1% 387 76.8% 175 34.7% 303 60.1% 405 80.4% 394 78.2% 379 75.2%

Missing
49 9.7% 47 9.3% 47 9.3% 46 9.1% 46 9.1% 46 9.1% 49 9.7% 47 9.3% 49 9.7% 44 8.7% 48 9.5% 48 9.5% 54 10.7% 53 10.5%

6

Increasing Use of 4-H Science Tools
Table 3.0 displays ways participants would increase their use of the 4-H Science Tools. Most participants (70.6%) reported that “knowing more about them” would increase their usage the most. Figure 3.0 presents the results in a graphic format. Table 3.0 Increasing Usage of 4-H Science Tools Frequency % 356 70.6% 212 42.1% 285 56.5% 173 34.3%

What would increase your likelihood of using tools? Knowing more about them Easier access to the tools Training on how to use the tools A need for them based on my clientele

Missing 148 29.4% 292 57.9% 219 43.5% 331 65.7%

Other Responses Include the Following:  Thirty (30) respondents mentioned that some aspect of time, i.e. more time in the day, more time to learn about tools, more time to use the tools, and more time to implement them, is needed in order for them to better use the tools.  Nine (9) respondents stated that they would use the tools more if they knew where to find them and how to access them. Some complaints about accessing the tools include: the URLs seem to change too frequently, searching/browsing is not effective in locating the tools, State Liaison’s not sharing any of the resources with County Liaisons, enrolling in webinars is difficult, finding archived webinars is difficult, finding the tools is difficult when not all in one spot.  Four (4) respondents mentioned needing more resources and funds for developing educational kits, travel to trainings or for materials.  Three (3) respondents mentioned having more staff especially administrative staff to help with day to day tasks so more time can be spent in learning about and using the tools.  Two (2) respondents would like the focus of team meetings to be on the tools and training resources and to have clearer communication between state and county staff who have already learned about using these tools.  One (1) respondent each reported that they would like: to see how others use/implement these tools and resources, venues to use them (partnership opportunities, project days, community events, etc.), and that more support is needed from supervisors to use science programming.

7

Figure 3.0
Increasing Usage of 4-H Science Tools
100% 80% 60% 40% 20% 0% Knowing more about them Easier access to the tools Training on how to use A need for them based on the tools my clientele 70.6% 56.5% 42.1% 34.3%

8

Content of E-Academy
Respondents were asked about to rate their interest in the proposed content of the EAcademy, which is arranged around five content areas: 1) Doing 4-H Science; 2) Volunteer Development; 3) Curriculum and Program Development; 4) Evaluation; and 5) Resource Development. The first question asked if respondents would attend an e-learning session on each topic. The percentages of those who replied “Yes” are presented in graphic form in Figure 4.0

Figure 4.0 Percentage of Respondents indicating they would attend an E-Learning Session on this Topic
100%

80%

85.1% 78.4% 78.4% 72.2% 66.9%

60%

40%

20%

0% Doing 4-H Science Volunteer Development Curriculum and Program Development Evaluation Resource Development

When asked about specific topics of interest in each of the areas, the primary interest is in the areas of Doing Science and Volunteer Development. A second tier of interest is in curriculum. The topics respondents were most interested in learning about are**: • Tools and resources available to support 4-H Science programs (Doing Science) • Teaching others to inquiry-based science experiences (Doing Science) • How to engage volunteers in 4-H Science as coaches or mentors (Volunteer Development) • How to help traditional volunteers facilitate 4-H Science programs (Volunteer Development)
9

How to effectively train volunteers in 4-H Science AND PYD principles (Volunteer Development) ______________________________________________ ** 75% or more are moderately to very interested in this topic The topics respondents were next most interested in learning about are***: • Facilitating inquiry-based science experiences (Doing Science) • Scientific engineering practices framework for K-12 Science (Doing Science) • Facilitating science programs to limited resource and/or non-traditional audiences (Doing Science) • How to recruit science-rich volunteers (Volunteer Development) • How to include inquiry in all science programs (Curriculum) • An overview of current 4-H Science curriculum resources (Curriculum) • Strategies for engaging teens as teachers in 4-H Science programs (Curriculum) ______________________________________________ *** 60% or more are moderately to very interested in this topic

Detailed results for each proposed topic are presented in Tables 4.0 - 4.4. Items that respondents were 1) least interested in; 2) most interested in; and 3) had the highest combined “moderately and very” interested are highlighted in each table. Also included are summaries of the open-ended responses. In most cases the same item was the one that received both the most “very interested” ratings and the most combined “moderately” and “very interested” ratings, there were two exceptions: 1) Learning about strategies for engaging teens as teachers in 4-H Science programming emerged as the item with the most interest in the areas of curriculum and program development; and 2) Building partnerships for 4-H Science rose to the top in the fund development area, which is the only item that over 60% of respondents indicated interest.

10

Table 4.0 Interest in Doing 4-H Science Topics
Moderately and Very Interested

Not interested

Somewhat interested

Moderately interested

Very interested

Missing

Tools and resources available to support 4-H Science programs

5 1.0%

50 9.9%

142 28.2%

263 52.2%

405 80.4%

44 8.7%

Why inquiry is the basis for 4-H 44 141 161 114 275 44 Science 8.7% 28.0% 31.9% 22.6% 54.5% 8.7% Connection between science 32 114 166 148 314 44 inquiry and experiential learning 6.3% 22.6% 32.9% 29.4% 62.3% 8.7% Scientific engineering practices 17 92 171 178 349 46 framework for K-12 science 3.4% 18.3% 33.9% 35.3% 69.2% 9.1% Facilitating inquiry-based science 15 77 183 185 368 44 experiences 3.0% 15.3% 36.3% 36.7% 73.0% 8.7% Facilitating science programs to 361 13 85 155 206 45 limited resource or non-traditional 71.7% 2.6% 16.9% 30.8% 40.9% 8.9% audiences Teaching others to facilitate 21 68 157 212 369 45 inquiry-based science experiences 4.2% 13.5% 31.2% 42.1% 73.3% 9.1% Asking high quality science 19 99 179 162 341 45 questions 3.8% 19.6% 35.5% 32.1% 67.6% 8.9% Developing science argumentation 45 142 149 118 267 50 skills 8.9% 28.2% 29.6% 23.4% 53.0% 9.9% Other:  Six (6) respondents mentioned that they have extensive training or knowledge in science and science education so more in-depth learning and moving past the basics is needed for them.  Three (3) respondents stated that they would like to see some sessions designed with volunteers in mind with specific science activities that can be taught to and used by volunteers.  Three (3) respondents mentioned that having more specific science resources such as curricula and activities that can be used would be helpful.  One (1) respondent mentioned that having a thorough academy on the tools, practical applications and what has been successful would be helpful.  One (1) respondent mentioned that they would be interested in teaching most of these sessions.  Other responses included: developing science tool kits, evaluation of science content, how animal science can fit into the new 4-H Science/STEM push, informal vs. formal science environment, and making the case for 4-H science.

11

Table 4.1 Interest in Volunteer Development Topics
Moderately and Very Interested

Not interested

Somewhat interested

Moderately interested

Very interested

Missing

How to recruit science-rich 10 80 142 223 365 49 volunteers 2.0% 15.9% 28.2% 44.2% 72.4% 9.7% How to engage volunteers in 4-H 11 65 150 229 379 49 Science as coaches or mentors 2.2% 12.9% 29.8% 45.4% 75.2% 9.7% How to help traditional volunteers 8 64 131 252 383 49 facilitate 4-H Science programs 1.6% 12.7% 26.0% 50.0% 76.0% 9.7% Reframing the volunteer role to 23 93 172 167 339 49 match the needs of 4-H Science 4.6% 18.5% 34.1% 33.1% 67.2% 9.7% programs Implementing the new researchbased national 4-H campaign to 20 89 160 182 342 53 effectively identify, recruit and 4.0% 17.7% 31.7% 36.1% 67.8% 10.5% retain 4-H volunteers Strategies for recruiting corporate 24 95 163 172 335 50 volunteers 4.8% 18.8% 32.3% 34.1% 66.4% 9.9% How to effectively train volunteers 7 73 149 226 375 49 in 4-H Science and PYD 1.4% 14.5% 29.6% 44.8% 74.4% 9.7% Working with limited resource and 19 94 142 200 342 49 multi-cultural audiences in 4-H 3.8% 18.7% 28.2% 39.7% 67.9% 9.7% science programs Other:  Most respondents mentioned that they have had adequate training in volunteer development and recruitment but need help with more specific issues such as finding volunteers in rural areas, specifics about recruiting, training and supporting volunteers, real world help at the local level not theory,  One (1) respondent mentioned they need help with retention of volunteers and one other respondent reported that volunteers and parents complain that 4-H is becoming more like school.

12

Table 4.2 Interest in Curriculum and Program Development Topics

Not interested

Somewhat interested

Moderately interested

Very interested

Moderately and Very Interested

Missing

Using the 4-H science checklist to determine science-ready programs Using the national 4-H Curriculum rubrics to determine high quality curriculum Revising current curriculum to include 4-H Science programs How to include inquiry in all science programs Adapting or revising a curriculum to include science inquiry An overview of current 4-H Science curriculum resources How to participate in the NYSD

23 4.6% 33 6.5% 31 6.2% 24 4.8% 30 6.0% 18 3.6% 57 11.3%

120 23.8% 128 25.4% 111 22.0% 87 17.3% 91 18.1% 77 15.3% 115 22.8%

162 32.1% 157 31.2% 141 28.0% 165 32.7% 164 32.5% 146 29.0% 134 26.6%

136 27.0% 124 24.6% 159 31.5% 166 32.9% 152 30.2% 201 39.9% 135 26.8%

298 59.1% 281 55.8% 300 59.5% 331 65.6% 316 62.7% 347 68.9% 269 53.4%

63 12.5% 62 12.3% 62 12.3% 62 12.3% 67 13.3% 62 12.3% 63 12.5%

Strategies for creating youth16 116 164 143 307 65 adult partnerships in 4-H science 3.2% 23.0% 32.5% 28.4% 60.9% 12.9% programs Strategies for engaging teens as 15 77 158 191 349 63 teachers in 4-H Science 3.0% 15.3% 31.3% 37.9% 69.2% 12.5% programming Designing 4-H Science programs 22 116 167 136 303 63 that integrate the mission 4.4% 23.0% 33.1% 27.0% 60.1% 12.5% mandates Other:  Five (5) respondents mentioned that they are less interested in sessions about things they already do such as the NYSD.  Two (2) respondents mentioned they need help in writing curricula and finding lesson plans.

13

Table 4.3 Interest in Evaluation Topics
Moderately and Very Interested

Not interested

Somewhat interested

Moderately interested

Very interested

Missing

Overview of evaluation basics

52 10.3%

113 22.4%

148 29.4%

123 24.4%

271 53.8%

68 13.5%

Planning and conducting an 33 97 146 159 305 69 evaluation of 4-H Science 6.5% 19.2% 29.0% 31.5% 60.5% 13.7% programs Overview of existing evaluation 24 91 147 173 320 69 instruments and tools 4.8% 18.1% 29.2% 34.3% 63.5% 13.7% Accessing the 4-H online 29 102 152 152 304 69 evaluation basics course 5.8% 20.2% 30.2% 30.2% 60.4% 13.7% Overview of recent 4-H research 38 110 152 134 286 70 that documents the impact of 47.5% 21.8% 30.2% 26.6% 56.8% 13.9% H Science programs Overview of current informal/non-formal science 38 119 152 127 279 68 learning research and how it 7.5% 23.6% 30.2% 25.2% 55.4% 13.5% documents the impact of 4-H Science programs Preparing effective 4-H Science 31 96 151 159 310 67 Program success stories 6.2% 19.0% 30.0% 31.5% 61.5% 13.3% Using photos/images to 34 85 153 163 316 69 document program success 6.7% 16.9% 30.4% 32.3% 62.7% 13.7% Other:  Four (4) respondents mentioned that they want new ideas on evaluation which can include: developing/using tools that allow agents to use record books and demonstrations as evaluation methods, animal science and environmental science evaluation, new ideas and programs to evaluate and what else is out there besides YEAK.  One (1) respondent mentioned that they are not interested in the nationally available evaluation tools since their state does not use them.  One (1) respondent mentioned that they need help in conveying the program stories through evaluation.

14

Table 4.4 Interest in Resource Development Topics
Not interested Somewhat interested Moderately interested Very interested Moderately and Very Interested

Missing

Mission to Market; The End of Fund Raising Implementing the 4-H Science Fund Development Toolkit The top 10 funding strategies for 4-H Science Developing a 4-H Science Advocacy card to use with potential donors Building partnerships for 4-H Science Using the NYSD to market your 4-H Science program Volunteer and alumni funding support Examples of current successful partnerships for 4-H Science Developing an “elevator” speech about your 4-H science program to use with potential donors Using 4-H Science impact research to support your case for potential funders Creating a public value statement for your 4-H Science program Strategies for making a successful “ask” of potential donors

42 8.3% 39 7.7% 32 6.3% 59 11.7% 23 4.6% 41 8.1% 56 11.1% 35 6.9% 56 11.1% 42 8.3% 38 7.5% 41 8.1%

125 24.8% 124 24.6% 89 17.7% 119 23.6% 91 18.1% 101 20.0% 130 25.8% 115 22.8% 117 23.3% 126 25.0% 109 21.6% 124 24.6%

114 22.6% 136 27.0% 138 27.4% 126 25.0% 157 31.2% 139 27.6% 129 25.6% 149 29.6% 115 22.8% 126 25.0% 138 27.4% 131 26.0%

140 27.8% 120 23.8% 162 32.1% 117 23.2% 151 30.0% 139 27.6% 104 20.6% 121 24.0% 132 26.2% 125 24.8% 133 26.4% 121 24.0%

254 50.4% 256 50.8% 300 59.5% 243 48.2% 308 61.2% 278 55.2% 233 46.2% 270 53.6% 247 49.0% 251 49.8% 271 53.8% 252 50.0%

83 16.5% 85 16.9% 83 16.5% 83 16.5% 82 16.3% 84 16.7% 85 16.9% 84 16.7% 84 16.7% 85 16.9% 86 17.1% 87 17.3%

Other:  Two (2) respondents reported that training on fund development might be more helpful if it was tailored to different job positions so that each person knows their role in fund development based on their position. This could help communication between co-workers regarding fund development.  Two (2) respondents stated that finding funding is an issue and would like more resources for finding funding that aren’t competitive.  One (1) respondent would like to have these sessions available after the academy for references when they are needed.  One (1) respondent would like more focus to be placed on other science topics such as animal and environmental science.  One (1) respondent would like help in convincing other extension staff that 4-H science is important and that fundraising is needed.  One (1) respondent mentioned that they need more time to receive all of the training topics and implement them.

15

E-Academy Format
Respondents were asked to indicate their preference for the format of the E-academy. This is the only place where the needs assessment data differ in important ways when the respondents are separated out by group. For this analysis we separated out those who are county educators from all other respondents, and in doing so, found an important difference.  Interactive e-learning sessions consisting of presentations with interactive components received the highest overall average rating (4.0/5.0)  Facilitated interactive learning: consisting of presentations viewed as a group at a local site with a facilitator trained to conduct breakout sessions received the greatest number of “high” ratings  BUT- if you consider only the responses from county-level educators the Facilitated Interactive Learning is the most preferred (4.8/5.0 mean rating)  Table 5.0 represents the preference of all respondents for the format of the E-Academy. Table 5.1 shows the preferences for respondents who are county-based educators. County educators indicated a strong preference for an “facilitated interactive e-learning session” as the format of the academy. All participants reported that they would prefer to participate via distance technologies.

16

Table 5.0 Preference in Format of E-Academy – All Respondents
Average
1=low agreement 5=high agreement

Format Types

I would not participate in this format

On-line e-learning session: Presentation-based learning, 26 3.3 non-interactive presentations about each topic 5.2% Interactive e-learning session: Presentations with some 20 4.0 interactive component 4.0% Enhanced interactive e-learning: Interactive presentations with required additional on-line 43 3.4 participation such as assignments, field trips, breakout 8.5% discussions Facilitated interactive learning: Presentations viewed as 29 a group at a local site with a facilitator trained to 3.9 5.8% conduct breakout sessions I would NOT participate if only offered via distance 30 1.7 technologies 6.0% Other:  Nine (9) respondents mentioned that they would prefer to have face to face training rather than the E-Academy but realized that logistics and costs are inhibitors. Also, E-learning has advantages that are not available with a traditional conference format.  Two (2) respondents reported that the E-learning that blends live interaction is the best format.  One (1) respondent stated that they want information to be archived so that the information can be accessed when it is needed.

Table 5.0 Preference in Format of E-Academy – County Educators
On-line e-learning session: Presentation-based learning, non-interactive presentations about each topic Interactive e-learning session: Presentations with some interactive component Enhanced interactive e-learning: Interactive presentations with required additional on-line participation such as assignments, field trips, breakout discussions Facilitated interactive learning: Presentations viewed as a group at a local site with a facilitator trained to conduct breakout sessions I would NOT participate if only offered via distance technologies 3.3 3.9 24 5.9% 18 4.4% 34 8.3% 22 5.4% 27 6.6% 17

County Educators

All Responses

3.4

4.8 1.8

Time Commitment
Respondents were asked to indicate the amount of time they are willing to commit to participating in the E-Academy. The following percentages indicated a “yes” for the indicated time: • 69.8% for ONE part day e-academy (up to four hours total) • 61.5% for ONE full-day academy (5+ hours) • 59.7% for an academy series over 3-5 months with part days of up to 4 hours • 14.9% for three or more full consecutive days • 26.8% for Three or more part consecutive days

Pre and Post Academy Assignments
Respondents were asked the likelihood that they would participate in pre and post academy assignments. Figure 5.0 presents the percentage of respondents who indicated they were “likely” or “very likely” to complete these assignments.

Figure 5.0 Likelihood of Completing Pre and Post Academy Assignments
100% 80% 21.4% 60% 10.7% 40% 20% 0% Complete assignments BEFORE the eacademy Complete assignments during the eacademy Complete any Attend any post Complete online post-academy academy e- learning modules assignments learning pre or post eopportunities academy 48.4% 54.8% 9.7% 48.0% 12.7% 59.1% 12.9% 56.5% Very Likely Likely

Format of E-Academy Sessions
Respondents were asked to rate their interest in participating and presenting at the E-Academy. The top two topics respondents were interested in participating in are: video (64.9%) and webinar (71.2%) presentations. Participants have the least interest in participating in the Pechakucha sessions (39.7%). All respondents reported that they would be the most interested
18

in presenting “lightning sessions.1” (22.8%) Respondents would be least interested in presenting “Pechakucha2 sessions” (69.6%). Figure 6.0 presents the percentage of respondents who indicated they were interested in a particular format. Figure 6.1 shows the percentage of respondents who are interested in presenting a particular type of session at the E-Academy.
Figure 6.0 Percentage of Respondents Indicating Interest in Types of E-Learning Sessions
100 90 80 70 60 50 40 30 20 10 0 On-line poster presentations Pechakucha session Lightning sessions Video presentation Voice-over power point presentation (recorded) Webinar presentations (live and archived) 47.2 43.7 59.9 64.9 59.7 71.2

1

A Lightning session is a short presentation given at a conference or similar forum that last only a few minutes; several will usually be delivered in a single period by different speakers. 2 Pechakucha is a simple presentation format where 20 images are shown, each for 20 seconds. The images advance automatically and presenter talks along to the images.

19

Figure 6.1 Percentage of Respondents Indicating Interest in Presenting Types of E-Learning Sessions
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% On-line poster presentations Pechakucha session Lightning sessions Video presentation Voice-over power point presentation (recorded) Webinar presentations (live and archived) 18.70% 22.80% 13.90% 14.90% 20.00% 18.50%

Technological Capabilities
Finally, respondents were asked rate their technological capability to attend the e-academy successfully. The most inhibitive piece of technology to the respondents attending an E-Academy is having a “functional voice over internet protocol (VOIP) microphone system.” (27.4%) Other inhibitive aspects include having a “webcam” (25.0%) and “having adequate technical support in the event of a technological problem on your end.” (26.6%).  It is important to note that while 70% indicated they have attended webinars successfully, this also means that almost 30% have not  In addition, only 58% indicated they have adequate technological support locally in the event of problems with participation that are caused by local equipment  And one out of four people (25%) report they do not have an appropriate setting, such as a quiet room for participating in the e-academy Figure 7.0 presents the percentage of respondents who indicated they had the technological capability to attend the e-academy. .

20

Figure 7.0 Percentage of Respondents Indicating Sufficient Capability to attend the E-Academy Successfully

100 90 80 70 60 50 40 30 20 10 0

84.3 70.2 58.3 56.2 59.9 75.2

Past webinar Adequate tech Access to a A functional participation support in the computer for voice over without technical event of a tech online viewing internet protocol difficulties on problem on your (VOIP) your end end microphone system

Webcam

Appropriate settings, such as a quiet room with no interruptions

Summary and Recommendations
The results of the needs assessment are, overall, promising for the success of the E-Academy. In particular, the respondents seem interested in the proposed content of the E-Academy, and indicate they are willing to commit time to attending the academy. There are a few important things to note that should be taken into consideration by the E-Academy planners, however. 1. Respondents indicated the greatest interest in content related to science program implementation (Doing Science) and Volunteer Development. Particular aspects of Curriculum also were rated moderately high. There are several items that stand out clearly as interest areas for this audience and should be kept in mind when the content of the academy is planned. 2. The proposed content for the Evaluation and Fund Development areas received low to moderate ratings. The planners should plan content in these areas with the audience in mind and not plan a program that is not a match with the skills and needs of countybased educators. 3. County educators indicated a very strong preference for the format of the E-Academy to be a facilitated session. While creating this type of session is beyond the purview of the E-Academy per se, some attention should be paid to providing information to states and
21

counties about how to plan for and host the E-Academy at the local level, including a recommendation that learning be done in a facilitated session if possible. 4. Care should be taken to be sure the technology used for the delivery of the E-Academy is tested for potential problems that may occur when the educator participates in the academy. The results of the needs assessment revealed just a fair to moderate technological capacity of the educators. Especially noteworthy is that almost ½ of respondents reported having no on-site technical assistance to help them. Advanced training, clear instructions for participation, and technical support will all be key to the success of the E-Academy.

22

Appendix One Needs Assessment Respondent Demographics
Frequency Gender Percentage

State

Male Female Alabama Arizona Arkansas California Colorado Connecticut Delaware Florida Georgia Hawaii Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Maryland Massachusetts Michigan Minnesota Mississippi Missouri Montana Nebraska Nevada New Hampshire New Jersey New Mexico New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Rhode Island South Carolina

78 390 2 2 5 14 6 2 1 16 16 2 4 35 11 5 25 32 9 8 8 4 9 36 5 22 3 8 7 1 8 1 16 25 3 24 11 12 26 1 6

15.5% 92.5% 0.4% 0.4% 1.0% 2.8% 1.2% 0.4% 0.2% 3.2% 3.2% 0.4% 0.8% 6.9% 2.2% 1.0% 5.0% 6.3% 1.8% 1.6% 1.6% 0.8% 1.8% 7.1% 1.0% 4.4% 0.6% 1.6% 1.4% 0.2% 1.6% 0.2% 3.2% 5.0% 0.6% 4.8% 2.2% 2.4% 5.2% 0.2% 1.2% 23

Region

Number of years in 4-H

State Science Liaison

Official Role

Science Specific Role

Period Hired, Science Specific Role Only

Tennessee Texas Utah Vermont Virginia Washington West Virginia Wisconsin Wyoming Missing Northeast North Central South West 1890s Missing First year 2-5 years 6-10 years 11-15 years 16-20 years 21+years Missing Yes No Missing Volunteer Partner State 4-H Science Liaison State 4-H Science Specialist State 4-H Specialist (not science) County 4-H Educator (agent role) County 4-H Educator (non-agent role) State 4-H Program Leader State 4-H Foundation Staff State 4-H Foundation Director Missing Yes No Missing December 2009 or earlier January to June 2010 July to December 2010 January to June 2011 July to December 2011 January to June 2012

7 16 10 7 5 9 4 11 1 3 88 180 139 78 10 9 52 123 112 79 47 88 3 36 444 24 4 3 6 14 35 345 64 14 1 2 16 53 443 8 41 7 2 10 11 11

1.4% 3.2% 2.0% 1.4% 1.0% 1.8% 0.8% 2.2% 0.2% 0.6% 17.5% 35.7% 27.6% 15.5% 2.0% 1.8% 10.3% 24.4% 22.4% 15.8% 9.4% 17.6% 0.6% 7.1% 88.1% 4.8% 0.8% 0.6% 1.2% 2.8% 6.9% 68.5% 12.7% 2.8% 0.2% 0.4% 3.2% 10.5% 87.9% 1.6% 48.8% 8.3% 2.4% 11.9% 13.1% 13.1% 24

Period Hired, Science Specific Role Only

July to December 2012 Pre December 2009 to December 2010 January 2011 to December 2012 Direct Programming to youth Teaching and preparing volunteers and partners Teaching and preparing 4-H Staff Developing curricula Adapting current curricula Creating science lessons and activities Fund development Evaluation Developing programs Developing partnerships Missing Direct Programming to youth Teaching and preparing volunteers and partners Teaching and preparing 4-H Staff Developing curricula Adapting current curricula Creating science lessons and activities Fund development Evaluation Developing programs Developing partnerships Missing Yes No Missing Curriculum Evaluation Professional Development Fund Development Attendee (non-presenter) Presenter Both Yes No Missing Attendee (non-presenter) Presenter Both

2 50 34 204 87 25 6 22 22 7 2 53 29 47 100 235 40 5 7 5 5 6 40 29 32 49 431 24 8 18 14 6 44 0 2 161 315 28 115 1 44

2.4% 59.5% 40.5% 40.5% 17.3% 5.0% 1.2% 4.4% 4.4% 1.4% 0.4% 10.5% 5.8% 9.3% 19.8% 46.6% 7.9% 1.0% 1.4% 1.0% 1.0% 1.2% 7.9% 5.8% 6.3% 9.7% 85.5% 4.8% 17.4% 39.1% 30.4% 13.0% 95.6% 0 4.4% 31.9% 62.5% 5.6% 71.9% 0.6% 27.5%

Where is time spent?

Where SHOULD time be spent?

Attended National Academy

If yes, which track did you attend?

If yes, what was your role? Attended REGIONAL Academy 2012 If yes, what was your role?

25

Sign up to vote on this title
UsefulNot useful