The Impact of Professional Development on New Technology Initiatives Georgia Southern University Debra J.

Scott FRIT 8132 3/21/2012

The Impact of Professional Development on New Technology Initiatives Introduction Many innovative technology initiatives tend to appeal to only a minority of teachers. Educators provide a variety of reasons for their reluctance to adopt new technologies but two commonly expressed reasons are related to the perception that teachers have been provided insufficient professional development and inadequate technical support. This perception can produce resistance to new technology initiatives regardless of whether or not it is true. The primary objective of this review was to consider research studies that focused on the role formative evaluation plays in revealing underlying perceptions and the impact they have on the progress of new technology initiatives. Research articles were chosen that addressed major issues impacting the integration of new technologies into the curriculum. The research considered several significant variables that contribute to the success or failure of professional development: providing ample opportunities for modeling instruction, facilitating community building and collaboration, long term vision of the leadership which is demonstrated by the level of support provided (availability of appropriate software, technology availability, and adequate technical support), as well as the teachers perception of: technology and its potential impact on learning, their technical ability, their ability to effectively integrate technology in instruction, and their perception of the level of administrative support being provided. The research communicates the challenges involved in collecting data that ultimately can help to reveal the status of a technology initiative at any given time.

Review of Literature Evaluating Professional Development The article, Connecting Instructional Technology Professional Development to Teacher and Student Outcomes provides a reproducible method for the reliable evaluation of a professional development (PD) program. The ultimate goal of professional development is to positively impact student performance therefore formative evaluation was used to fine tune professional development as it related to measurable and observable teacher and student outcomes. The teacher outcomes were observed as improvement in the quality of their lesson plans and the student outcomes were observed as the improvement of their performance on standardized tests (Martin, Strother, Beglau, Bates, Reitzes, & Culp, 2010). Measuring the impact of professional development as it relates to instructional effectiveness and student outcomes is challenging but necessary in order to provide teachers more targeted support. The article summarized the results of an evaluation of an instructional technology professional development (PD) program. The evaluators carefully analyzed the goals and objectives of the PD plan, observed the effect that the PD had on the participating teachers lesson plans, and ultimately observed how the more clearly defined lesson plans impacted student achievement. The researchers created an observation instrument designed to quantify how consistently the programs objectives were being achieved across several schools. Forty-four observable behaviors were identified as an indication of the programs 5 primary objectives: modeling instruction, community building, technology utilization, connection to practice, and promoting

inquiry based learning. During 4 hour observation sessions, observers recorded whether or not 1 or more of the identified behaviors or activities occurred within 15 minute time intervals. The observers were required to attend a 1 day training session in order to increase inter-rater reliability. The focus of the study was enhancing Missouris Instructional Networked Teaching Strategies (eMINTS), a program started by educators at the University of Missouri, Columbia. eMints was a 2 year program of ongoing professional development that included peer reviewed teaching sessions and classroom visits by instructional specialists. During the classroom visits, the specialists recorded how much time was spent: modeling instruction or co-teaching, collaborating on lesson planning, and troubleshooting technology issues. Three years of standardized test scores were assessed (the year prior to the PD and the 2 years of the program.) Researchers observed a significant positive correlation between the degree of alignment to the programs primary objectives and the quality of the teachers lesson plans. The results indicated that the more time instructional specialists spent collaborating with teachers on lesson planning the better the lesson plans. Higher quality lesson plans was then positively related to significantly higher student test scores. Technology utilization had a consistently positive impact on student achievement at all grade levels. The authors noted that the impact of the primary variables seemed to vary with grade level. A strong relationship was observed between community building and achievement in 3rd grade, between modeling instruction and achievement in 4th grade, and between more inquiry-based learning activities and achievement in 5th grade. As one might expect, the amount of time spent troubleshooting issues was negatively related to the quality of lesson plans. The

impact of particular objectives being more closely aligned to specific grade levels seems to support Piagets theory of cognitive development with the 3rd graders being at a stage in which they were developing greater language skills, the preoperational stage, resulting in better communication; the 4th graders being at the concrete operation stage in which they tended to benefit from structured straight forward presentations of material; the 5th graders being more cognitively developed, exhibited characteristics of Piagets final stage of development, formal operations, and were therefore inclined to think more logically making them more successful at inquiry based learning activities (Ormrod, 2004). The evaluators emphasized that effective professional development programs take a significant amount of time for development, evaluation, and fine tuning. A good plan on paper does not necessarily guarantee that it will be executed as intended without sufficient support and follow up. Both time and money are limited resources therefore it is always essential that efforts are focused in areas that provide the greatest return on investment. Without quality evaluation of professional development there will be no way to determine what components of the current professional development plan are adding the greatest value. Collaboration between peers and university sponsors was a key element to the impact of the eMints professional development plan. Because of the limited amount of time and money for professional development, more research is needed on facilitating virtual collaboration and nurturing virtual learning communities. For the most part, the teacher has been basically autonomous in the classroom environment, but education reform is demanding innovative practices that can best be accomplished with educators sharing what works and what doesnt work in real time.

Impact of Technology Initiatives The Path to Pedagogical Reform in the Sciences focused on the issue of expanding the impact of technology initiatives. Widespread systemic change will not occur if educators continue to have the belief that innovative teaching strategies using new technologies is just something that the school techies do. The National Science Foundation (NSF) has expressed concern that effective learning strategies studied in pilot programs are not being widely accepted and therefore are not producing a widespread lasting impact. If a teaching strategy has been researched and is considered best practice then that is how it should be presented and teachers should be participating in ongoing discussion about why it is considered best practice (Kezar, 2012). Providing evidence from peer reviewed research that particular education reform practices improve instructional effectiveness can add credibility to new technology initiatives. An intentional clearly defined plan for developing buy in to the implementation of innovative instructional strategies is required to increase the overall impact on instruction. Professional development is often successful at initiating change in a few but in order to sustain the change, underlying thoughts and beliefs must also change. The implications of this research is similar to that of Martin et al. (2010) emphasizing that professional development needs to focus more on facilitating networking and discussion in order to encourage discussion about best practices (Kezar, 2012). Kezar (2012) discussed two models for promoting greater and more widespread buy in: mutual adaptation and social movement. Mutual adaptation can be encouraged by facilitating learning communities, such as those proposed by Martin et al. (2010), that work together to

customize innovation for a particular setting. According to Kezar (2012), the three key components of both the mutual adaptation and social movement change models are discussion (change tends to last longer when the targeted users mentally accept it as valuable), networking (the ability to connect with other like-minded individuals to exchange ideas), and incentives. Professional development generally helps participants to understand the need for change(s) but it does not necessarily trigger the motivation within participants to actually make the change(s). In the mutual adaptation and social movement model, new technology initiatives are implemented better when individuals collaborate in groups, supporting, and helping each other resolve any issues as they arise. More case study research is needed that describes practical methods for facilitating collaboration among educators. It is not enough to say that it should be done. Professional development plans must incorporate practical ways to build and strengthen learning communities. Encouraging virtual learning communities participating in asynchronous discussion would be particularly effective because it eliminates the issue of the participants having time conflicts. Long Term Vision for Professional Development

Wright (2010) described a professional development program called the Master Technology Teacher (MTT). The program provided training to teachers on how to use technology in classroom instruction. It involved collaboration between both in-service and preservice teachers and university faculty members. The University of Alabama formed the MTT program partnership in 2000 as part of a grant initiative. The objective was to provide better inservice teacher training by training a cohort of middle and high school teachers to become master

technology teachers. The training included professional development and field experiences which were designed to increase awareness of new technologies as well as best practices for improving instructional effectiveness. The Universitys pre-service teachers were required to collaborate with the master teachers to effectively integrate new technologies in instruction. As in Martin et al. (2010) and Kezar (2012), collaboration was again a key element of the professional development.

In the first year of the MTT program, 2 teachers from local school districts teaching various subjects were matched with 2 pre-service teachers training to teach in the same subject area. A traditional workshop model was used to train teachers on the use of basic technology for practical uses such as presentation delivery and internet use. University faculty was also a part of the collaboration as they generally provided technical support to the teachers. The researchers noted that this collaborative approach tended to be less intimidating for new technology users. The following year, the workshop was continued but with more emphasis on promoting instructional strategies that encouraged interactive projects such as web quests. As a result of including more projects in their instruction, the master teachers reported that their colleagues began to take notice and comment on the changes in their instructional methods. During this year, the program faced challenges because several of the collaborating faculty members left the university. During the 3rd year, more technologies were introduced such as digital storytelling, interactive white boards, and digital photography. By the 5th year, the workshop model was revised to include more classroom visits to observe and support instruction. Each visit consisted of a pre-observation discussion in which the teachers explained their lesson objectives and a post-observation discussion about what went right and what went wrong. In some cases the postobservation discussion included the students. As the master teachers bonded they would also

come together to discuss specific challenges and to develop strategies in support of their school improvement plans. By the 9th year the program more aggressively promoted the use of technologies such as GPS, wikis & blogs to encourage greater student collaboration, and digital storytelling for sharing content. Integration of technology across the curriculum was also encouraged by promoting collaborative projects involving multiple disciplines. By the 10th year, the school district budgets were drastically decreased. Because of this, one language arts teacher promoted a paperless plan and students throughout the district were encouraged to use personal blogs, wikis, and social networks for writing assignments as opposed to printed paper. Unfortunately, due to issues with wireless connectivity and insufficient access to computers, most collaborative projects had to be done outside of class time. This observation reinforces the impact that technology access has on new technology initiatives.

At the time the article was written, the program spanned 10 years and involved 14 schools. Wright (2010) as well as Martin et al. (2010) clearly emphasized that promoting technology integration takes time, a concerted effort, formative evaluation, and requires a flexible long term implementation plan. The research also emphasized the importance of cultivating learning communities among both teachers and students. Collaboration was clearly a key component within and among the schools and the university.

The Role of Leadership

Banoglus (2011) research on the role of leadership in technology integration indicates that there needs to be a clear understanding that technology planning involves more than just allocation of funds by the administrative leadership. Leadership standards developed by the International Society for Technology in Education (ISTE) identified several important variables

10

impacting technology implementation: Leadership and Vision; learning and teaching; productivity and professional practice; support, management, and operations; assessment and evaluation; and social, legal, and ethical issues. In Banoglus (2011) research, the validity and the reliability of the Principals Technology Leadership Analysis (PTLA) survey instrument was measured and confirmed using confirmatory factor analysis (CFA) & explanatory factor analysis (EFA). As a result of the analysis it was determined that the following 3 dimensions of data were reliable for the sample population of principals: leadership & vision, learning & teaching, assessment & evaluation, as well as a school principals overall technology leadership competency. The results indicated that the principals had the lowest competency in the leadership & vision area, though female principals tended to demonstrate a greater ability to encourage collaboration. The research results indicated that many teachers see their principals as not being very proficient technology users themselves and that therefore they tended to have unrealistic expectations from teachers. To the contrary, results revealed that the principals tended to have a high self-efficacy for their technology ability and were confident in their ability to provide technology leadership. The literature review included in the article indicated that the role of technology coordinators is vital in assisting principals as they act as change leaders to provide technical expertise and instructional support. This technology coordinator can help to bridge the gap between the teachers perspective and the principals perspective. Using Reliable Evaluation Instruments The purpose of the study performed by Inan and Lowther (2010) was to analyze the effect of various teacher characteristics and environmental factors on technology integration in

11

K-12 schools. The research data was gathered from 1,382 Tennessee public school teachers. The teachers were employed at schools participating in the 1st year of a Title II program called Tennessee EdTech Launch One. A path analysis research method was used to create a visual representation to explain the interactions among teacher-level and environmental or school-level variables identified as having some degree of impact on successful technology integration. As in a flow chart, in path analysis, arrows are used to indicate the direction of influence. The observed effects were categorized as having an indirect or direct impact on the success of new technology initiatives. Teacher-level effects included: academic discipline, gender, self-efficacy as it relates to computer use, years of experience, beliefs, and readiness. School-level variables included: availability of appropriate software, computer availability, adequate technical support, and adequate administrative support. The Path Analysis resulted from the statistical analysis of the teacher demographic information and the accessibility of technology in the classroom. The researchers noted that previous peer reviewed research has typically used the path model to primarily analyze only teacher level variables. The data collection instrument used was the Teacher Technology Questionnaire (TTQ). This instrument has been validated and is commonly used in research and evaluation studies. It provided data related to the teachers perception of technology and its impact on learning, the teachers perception of their technical ability as well as their ability to integrate technology in instruction, and their perception of the level of administrative support as it relates to technology integration. The research results indicated that among all the variables, teachers readiness had the strongest direct effect on technology integration followed by teachers beliefs. Teachers readiness is described as the teachers perception of their ability to effectively integrate technology in their instructional strategies. Teachers beliefs are described as their perception of

12

the level of impact they believe technology can make on student learning. These results are supported by Kezars (2012) research which stated that while most professional development consists of how to instruction on the use of technology it does not always effectively motivate a change in practice or positively change foundational beliefs about the potential impact that technology has on instruction. Computer availability had the 3rd largest direct effect on technology integration supporting the research that indicates that sufficient access is a critical component of any new technology integration.

The purpose of Corns (2010) study was to determine if the School Technology Needs Assessment (STNA) is a valid and reliable instrument for collecting data that describes how educators feel about how new technology initiatives are being implemented in their schools. The STNA is a free online survey tool developed by staff & faculty at the University of North Carolina (UNC) Greensboro. In collaboration with the North Carolina Department of Public Instruction (NCDPI), key variables were identified as well as the data sources required to evaluate the effectiveness of technology implementation. The data sources included staff needs assessments, teacher reflection logs, classroom observations, professional development questionnaires, and rubrics for evaluating lesson plans.

The STNA 3.0 instrument consisted of self-report items and 5-point Likert scale items related to 4 general topics that included various subtopics. The major topics included: supportive environment for technology use (administrators who model and communicate a long term vision tor effective technology integration; sufficient access to a variety of technology; a reliable technology infrastructure), professional development (modeling research based instructional

13

strategies), use of technology for teaching and learning (student use of technology for communication and collaboration in project based learning activities), and the impact of technology on teaching and learning (better quality lesson plans and improved student assessment scores). The authors conducted a literature review to answer the question: To what extent does literature support the significance of these 4 major topics measured by the STNA. These are many of the same topics that were identified as significantly impacting technology initiatives in each of the articles discussed in this literature review.

A review of several alternative technology needs surveys were also studied for comparison. STNA was compared to similar school technology needs surveys that were not state-specific, were still being supported, and had been evaluated for validity and reliability (i.e. Level of technology implementation questionnaire (LoTiQ) and Teacher Technology Questionnaire). The STNA and the LoTiQ were similar in that they were considered comparable in nature but only the STNA was available to any school district online, free of charge.

The research used a correlational design to study relationships between the variables. An exploratory factor analysis (EFA) was used to identify related factors in the STNA and a confirmatory factor analysis (CFA) was used to determine whether the factors supported the reliability of the existing. The research sample consisted of 1918 surveys from 49 schools (32 elementary, nine middle, and eight high schools) in North Carolina. Descriptive statistics (i.e. mean, standard deviation) for each response item and demographic data (years of teaching experience, role, grade, ethnicity, gender) was used in order to reveal any telling patterns.

Nineteen items resulted in significant response differences in the data. It was determined that this was due primarily to unique requirements of high school teachers. In high school

14

teachers are generally grouped by discipline but in elementary and middle school they are grouped by grade level. Most of the significant variation in response patterns related to technology needs seemed to indicate similarities among the high school respondents that significantly different from the elementary and middle school response pattern. Nevertheless 67 of 86 items included on the survey instrument provided meaningful data that was similar across grade levels. The author recommended 2 options for revising the instrument to eliminate the disparity: remove the 19 items so that the instrument can be considered reliable across all grade levels or provide 2 versions of the evaluation instrument, one for high school and one for middle and elementary school.

The author also took note that the schools varied significantly in the way they administered the instrument therefore for greater reliability a uniform training needs to be designed to be administered prior to raters using the instrument for observation. Holden and Radas (2011) research referenced the Technology Acceptance Model (TAM) to describe the level of acceptance of new technology initiatives among K-12 teachers. The research focused on the characteristics of the technology as critical elements impacting the ultimate success or failure of technology integration initiatives. The research results indicate the importance of taking the perceived usability of technology into consideration specifically as it relates to the characteristics of the targeted user population. One would expect that if the users do not consider the technology to be user friendly then they would not be motivated to use it. Research indicates that teachers have a more positive attitude toward technologies that they think are useful. In general, technology users tend to be more motivated to use technology that they are convinced increases their productivity and their overall effectiveness.

15

A survey was administered to K-12 teachers in two rural school districts in Virginia. The survey evaluated the relative effect of computer self-efficacy compared to the effect of overall technology self-efficacy. The researchers found technology self-efficacy to have more of an impact on technology acceptance. Research has found the frequency of a teachers technology usage to be a good indicator of the users self-efficacy. The researchers considered 4 usability measures: learnability (learning how to perform tasks using the technology is relatively easy), functionality (satisfaction with the features provided), navigation (user friendliness; intuitive), and memorability. When adopting new technology, more usability surveys need to be administered. This would allow the end user to see that their preferences are significant to the purchasing decisions. Technology is not one size fits all but often an initiative is launched without much consideration of the characteristics of the target users. New technology pilot programs are often attractive to users with a high self-efficacy for technology but the average user does not necessarily possess a high self-efficacy. Prior to adoption of new technology, a greater percentage of users need to be convinced of its value. The implication of this research is that professional development plans must find a way to target the characteristics and address the needs of the majority, not a minority of users. Custom Application Development

The goal of the research of McAndrew, Taylor, & Clow (2010) was to develop a reliable method of communicating relative impact of multiple variables in the evaluation of a custom designed learning system called the MOBILearn system. The MOBILearn project was the result of a partnership that included participants from a variety of academic disciplines and countries.

16

The focus of the project was a first aid refresher training course. The instructional objective was to design a project that would engage students more than the typical refresher course held in a classroom. Using mobile technology, participants were required to collaborate to determine the best way to resolve scenario based health emergency simulations. The goal was to design challenges that would be complex enough that they would demand communication and collaboration from the participants.

Evaluators of the system were required to respond to the needs of a variety of stakeholders: teachers, software designers, instructional designers, and the students. The usability of the system had to be evaluated as well as the instructional design of the training course. A strength of the study was the use of multiple trials allowing subsequent trials to be tweaked using feedback from the previous trials. All participants were given a questionnaire designed to give feedback about the complex interactions. Adjustments were made to the design from one trial to the next based on the feedback. Practical examples of formative evaluation such as this can be useful for helping people to understand the positive impact offered by pre-planning for periodic evaluation throughout the technology implementation.

Depending on how technologically inclined the target users are, it is probably best that they get involved in the evaluation process only after most of the bugs have been identified and addressed. Many educators are particularly averse to using technology because they have experienced it as more of a distraction than a benefit. Involving them in the early stages of software development may result in reinforcing negative experiences with technology and will only create greater resistance to the new technology implementation.

17

Currently, many mobile applications are simply being adapted to educational settings but they have not necessarily been designed to address specific learning objectives. As instructional design becomes increasingly more digital in nature, the demand for custom applications to be designed with specifications particular to educational requirements will also increase.

Conclusions One of the greatest implications of this research is that it reveals how vital it is for organizations to have a long term plan for professional development as it relates to new technology initiatives. The overall plan for the implementation of various technology initiatives must have sufficient resources allocated for professional development and a plan for the execution of ongoing formative evaluation. Of course, long term planning demands long range vision from administrators and others in leadership. In order to make a realistic plan, administrators will need periodic feedback from teachers and support staff clearly communicating their developmental needs. This will entail the completion of surveys and questionnaires designed to accurately assess the target users level of technology use. All of these requirements depend upon feedback from various sources in order for them to be successfully completed which emphasizes the necessity for personnel that facilitates collaboration. Ongoing collaboration is not an option; it is a requirement if our objective is to deliver professional development that truly addresses the needs of the target audience. Technology users require a long term plan for new technology integration that allows the gradual introduction of features in stages based upon the level of technology use (i.e. entry, adoption, adaptation, appropriation, and transformation) exhibited by the target users (Whitehead et al., 2003). Many new technology initiatives begin as pilot programs intended to model a best

18

practice that will eventually gain widespread acceptance. Pilot programs tend to appeal to a select group of end users who are generally referred to as the techies. Unfortunately, the characteristics of the techies are not the average user characteristics. More professional development needs to be designed to target the typical user. The typical user generally does not possess the level of self-efficacy as it relates to the use of technology or the integration of technology in classroom instruction as that of the techie. The usability of new technologies is being overlooked as it relates to the characteristics of the typical user. This information is critical as we go through the new technology adoption process. Too many bells and whistles oftentimes can overwhelm a new technology user and cause them to avoid using it not because it doesnt address their needs but because they cant figure out how it addresses their needs because of the variety of options. Techies, on the other hand, tend to like options. It is important that the target users evaluate the usability of any new technology and software. Technology is not one size fits all. Users have preferences and these preferences should be an essential focus in the technology adoption process. In summary, there must be a plan that utilizes evaluation instruments similar to those presented in this literature review in order to realistically be able to assess the needs of the target users. Professional development is instruction, and instruction should always be targeted to the target audiences characteristic. Educators are not salespeople and our objective should not be to convince people that they have a need but there should be processes in place that allow people to express authentic needs and then develop instructional strategies that address those needs.

19

References

Banoglu, K. (2011).School Principals' Technology Leadership Competency and Technology Coordinatorship. Educational Sciences: Theory & Practice, 11(1), 208-213.

Corn, J. (2010). Investigating the quality of the school technology needs assessment (STNA) 3.0: A validity and reliability study. Educational Technology Research & Development, 58(4), 353-376. doi:10.1007/s11423-009-9140-y

Holden, H., & Rada, R. (2011). Understanding the Influence of Perceived Usability and Technology Self-Efficacy on Teachers' Technology Acceptance. Journal Of Research On Technology In Education, 43(4), 343-367.

Inan, F. A., & Lowther, D. L. (2010). Factors affecting technology integration in K-12 classrooms: a path model. Educational Technology Research & Development, 58(2), 137154. doi:10.1007/s11423-009-9132-y

Kezar, A. (2012), The Path to Pedagogical Reform in the Sciences, Liberal Education, 98(1), 4045.

Martin, W., Strother, S., Beglau, M., Bates, L., Reitzes, T., & Culp, K. (2010). Connecting Instructional Technology Professional Development to Teacher and Student Outcomes. Journal Of Research On Technology In Education, 43(1), 53-74.

McAndrew, P., Taylor, J., & Clow, D. (2010). Facing the challenge in evaluating technology use in mobile environments. Open Learning, 25(3), 233-249. doi:10.1080/02680513.2010.511959

20

Ormrod, J. (2004). Human Learning (pp.166-167). Upper Saddle River, NJ: Pearson Education, Inc.

Whitehead, B., Jensen, D., & Boschee, F. (2003). Planning for Technology: A Guide for School Administrators, Technology Coordinators, and Curriculum Leaders (pp.81). Thousand Oaks, CA: Corwin Press, Inc.

Wright, V. H. (2010). Professional Development and the Master Technology Teacher: The Evolution of One Partnership. Education, 131(1), 139-146.