Comparative Study of Critical K-12 Teaching Skills Between Florids- Based Military Instructors and Experienced Classroom Teachers By Ron Burton, Florida Troops-to-Teachers Program Manager ‘And John W. Jacobs, Ph.D., CEO, Apr Dav: Olsen, President, Applied Simu (ital Study Report - January 2011) ecutive Summary A pilot stady was competed between September and December, 2010, involving a total of 32 rilitary instructors having varied experience providing stand-up taining in various Subject areas. All instructors were worting on military installations in Florida, and a majority of instoetors had received an advanced astactor qualification rating. All instructors completed_an online ‘Teaching, Skills Profiler (TSP) assessment that measured competency in five eiical K-12 teaching’ skill seas: Providing Feedback to Learers, Classroom Management, Tailoring Instruction, Fostering Leiming-o-Lear Skil, and Facilitating Group Learning Processes. The TSP uses video simulatinn technology, has undergone an extensive validation process and has ‘ben in use for over six years by school disrts, College of Education degre programs, and Altemtive Teacher Certfcation programs. The study results indiated the average (mean) TSP total score forthe entire pilot study (instructor) aroup ss similar, and statistically not diferent, relative to the mean TSP total score of experienced K-12 teachers (based on the TSP normative Sample used during validation). For those military instractors receiving an advance instructor ‘qualification rating, the average (meun) TSP overall score was noticeably higher than those inasuotors with basi question rating, Moreover, the TSP teaching skills score profile for this advanced instrtorproup was higher in thee ofthe five TSP teaching skill reas relative to te experienced K-12 teachers, nn me insince (Paling, Group Processes) was above die Skill profile seore of a mentor teacher group. This mentor teacher group included highly skilled teachers that regularly mentored new or struggling teachers. ‘The results support the view that nilitary instructors have core teaching skills and associated teacher competencies that readily {rans tothe K-12 classroom teaching-learning environmentComparative Study cf riteal K-12 Tenehing Skis — Initial Study Report January, 201) 1.0 Introduction ‘This pilot study was conceived by Ron Burton, the Florida Troops-to-Teachers CTT) Program Manager, for the purpose of comparing the K-12 classroom teaching, skills of Florida-based military instructors with those of experienced K-12 teachers. As part ofthe comparative analysis, these results also provide an initial assessment of the ‘extent to which a militery instructor skill set translates into a K-12 classroom teaching Teaming environment. ‘The plot study uiized an online Teaching Skills Profiler (TSP) assessment instrument for the following reasons a) The online TSP is readily accessible und is available 24/7, ) It has been in we for over six years by school districts, Collegeof Education degree programs, and Altemative Certification programs, and has available results| fiom several teacher groups for comparative purposes, to include experienced K- 12 teachers and highly skilled mentor teachers, and ‘©The TSP has undergone an extensive validation process to include conducting a ‘concurrent validation study that links TSP results to K-12 classroom teaching performance. 2.0 Pilot Study Questions and Methodology ‘The overriding study question for this pilot study was: Do military instructors ‘have a skill set that is comparable to experienced K-12 teachers? ‘Subordinate (Secondary) study questions included: {) Isthere a ncticeable difference between the K-12 teaching skills of military instructors having a basie versus an advanced instructor qualification rating? and b) Are there other factors, such as age, education level and achievement level that slats to the PSP foal sore? ‘The expected results related to these study questions were as follows. For the primary study question, it was expected that qualified military instructors will have @ K= 12 teaching skillset that is comparable to experienced K-12 classroom teachers. This, expected result is due t several factors, including 4) Military instrucor training provides information and techniques for designing and delivering ffeative instructional content using proven instructional systems development (ISD) principles; and ) Military instrucors at both the basic and advanced qualification levels are provided ongoing leadership training that includes such aspects as active listening, providing guidance, and using testing and tet results, that can be readily applied in an instrutional/educational context. For the secondsry questions, it was expected that having an advanced instructor qualification would positively influence TSP total scores and associated teaching skills profile, however, other factors related to age, education level and achievement level ‘would not have a noticeable impact. This latter expectation i based on previous teaching skills analyses using the TSP with other teacher groups, to include individuals entering an alternative teacher cert fication program, that indicate TSP total scores ae typically not‘Comparative Study of Cita! K-12 Teaelag Sil tna Study Report January, 2011) impacted by age, education level and achievement level. Achievement level in this ease reflects the broad career-related achievement options obtained when registering on the TSP website. The following provides a description of how the pilot study sample was obtained, the TSP K-12 teaching skills assessment instrument, and the procedure used wen implementing the pilot study. 2.1 Sample Population of Interest ‘The pilot study sample was drawn ftom instructors stationed in Florida having a basic or advanced instructor qualification, and included Serviee-specific instructors from the Air Foree, Army, Navy, Marine Corp, and Coast Guard. The overall goal was to ‘obtain a sample size ofa least 30 participants from at least wo Services with instructor qualification atthe basis and advanced levels. The pilot study sample was a convenience sample obtained by distributing an email request from Ron Burton to select military instructor leadership personnel and their respective instructor staff. There were over 100 instructors that completed the TSP. registration and many of these participants completed at least one TSP teaching skill assessment. However, the pilot sample was composed of only those 32 participants that ‘completed all ive TSP teaching skill assessments. The reason for only including participants that completed all five TSP ski assessments was to ensure tha: ‘) Each study participant had an “overall score” calculated by adding all five teaching skill assessment scores together, and b) It allowed for a five-part “teaching skills profile” comparative analysis with ‘experienced K-12 teacher and mentor teacher compatison groups Its noteworthy that prior to initiating the pilot study, the study authors considered dropping the Classtoom Management skill area assessment, due to the ‘consideration that military instructors were not likly to deal with behavior management issues, even on an iregalar basis. However, it was decided to include this teaching skill assessment fortwo important reasons: 2) The Classroom Management skill area is an important, if not crucial, aspect of the K-12 teaching environment, and 'b) An important outcome of this study is to determine whether the skillset of tary instructors translates into the K-12 classroom teaching envionment, 22 TSP K-12 Teaching Skill Assessment Instrument ‘As noted previously, the study utilized the TSP assessment instrument to assess the K-12 teaching skills of Porida-based military instructors. The TSP is available online at ‘vowwsteachingskillspeoiler.com. The TSP website provides a detailed deseription of how the instrument was developed and validated along with links allowing users and administrators to register and access the system. Briefly, the online TSP assesses the following five eritial teaching skills that ae core elements of the teaching-learning process: ‘+ Providing Effective Feedback to Leamers ‘* Classroom ManagementComparative Stay o Crit! K-12 Teaching Sil ~ Initia Stady Report ‘© Tailoring Instrction to Fit Individual Learner Needs ‘© Fostering Learting-o-Lear Skills and Abilities ‘+ Facilitating Group Leaming Processes ‘The TSP uses video simulation technology to present realistic classroom situations specifically related to one ofthe above five teaching skills. Moreover, each situation reflects a specific sub-sill component. For example, the TSP includes a video situation ‘elated to Classroom Management that involves setting up classroom and personal conduct rules; one of thee sub-components of the Classroom Management teaching skil area, The five TSP teaching skll-area definitions and their sub-component definitions ere validated as part of the overall system validation process, along with the septs, answer choices and the TSP scoring formula ‘There are a fotal of 16 video situations that comprise the TSP, ‘Three video situations ‘are used to assess each eaching skill, except for the Learning-to-Lear skill area that includes four video situations. Each TSP video situation lasts between 4-7 minutes and ‘completing the entire 16 situations takes approximately 90 minutes. Each situation Judes the following components ‘© A narrated introduction that provides essential background information, ‘© Rolling video stowing the teacher-student or teacher-parent interaction, ‘© Fouranswer chuices shown both as text and acted out via rolling video interaction, and ‘© A final sereen showing al four answer choices (in text format) with ‘accompanying :0-second countdown clock allowing participants to choose the ‘most and least effective answer choices from the four answer choices provided. Figure 1. Example TSP Situation Screen CaptureComparative Stay of Cri K-12 Teaching Stitt Study Report (January, 2011) Participants can stop between situations and return to complete a teaching skill assessment or initiate another teaching skill assessment at thei leisure. The TSP remembers where pat cipants are in the assessment process and will resume their skill assessment at the proper place if they stopped without completing a given skill assessment. Participants can view their teaching skill assessment feedback report immediately after completing the last video situation for that teaching skill and ean download or email ther esults (for example to a mentor or faculty advisor) if they so choose. TSP skill aree feedback reports provide information about overall skill and skill-component strengths and weaknesses, as well as suggestions and information on. ‘how to improve each sub-skll component. One ofthe primary uses of the TSP is to provide an objective taining needs analysis at both the individual and group levels to help teachers and administrators more effectively select and employ limited teacher professional development resources 23 Implementation Procedure ‘Once the pilot study population of interest was determined, the study authors set _up appropriate sub-groap registration codes to allow individual participants to register and access the online TSP teaching shill assessments. ‘There were 17 such sub-group codes, one for each ofthe 16 Service-specific instructor qualifiation classifications and an overall “TTT-FLA" code that could be used for demonstrating the TSP if needed, Setting up individual registration codes allowed the study tear to realy track registrations and teaching skill assessment progress and to address, if needed, any ‘TSP- related issues that may arise ‘When the registration codes were set up and tested, Ron Burton sent an email describing the pilot study objectives along witha brief description ofthe TSP and instructions on how to access the TSP website, to include specifying the appropriate access code to use whea registering. The participants were allowed several months {staring in September and ending in December, 2010) to complete all five assessments, ‘The study authors were able to track participant reyistrations and teaching skill assessment progress inan on-going manner throughout the study. ‘The online TSP is completely automated, so individuals progressed through the teaching skill assessments at their own pace and completed the teaching skll assessments inany order. A large majority of the study participants completed the assessment within ‘three-day period and many completed it during a single (90 minute) session. The online TSP registration includes questions related to age, ethnicity, education level, and (general career) achievement level, The answers to these registration questions were used by the study team to provide « more detailed biographical description ofthe study participant, ‘as well as to conduet selected post-hoc analyses pertaining to the secondary study ‘questions outlined previously. 3.0 Pilot Study Results ‘The following provides information related tothe military instructor sample population characterises and the results of the comparative analysis between this pilot study sample group and selected K-12 teacher groups who completed the TSP previously.Comparative Say of ride K-12 Teaching Skil — Ital 3.1 Plot Study Sample Group Characteristics ty Report Ganuaey, 2011) As noted previcusly, the study sample group was comprised of 32 military instractors having either a basic or advanced instructor qualification, Related to Table 2, the mean age for this goup is 35.2 years, with participant ages ranging from 22 to 58 ‘years, The breakdown of this study group is provided in Table I through Table 6 Table 1. Total Pilet Study Group Breakdown by Service and Gender Servic? Gender_| AirForse | Army Ney | tora Mate & 1 » 28 Female| 0 o 4 4 able 2 Total Pilot Stuy Group Breakdown by Service and Age Serie? ‘Age Level | AirForee | Arm Ney _|_rora_| 20s o ° 1 1 a 3 o “ n 40s s o 2 1 | sos o 1 ° 1 ‘Table 3. Total Pilot Study Group Breakdown by Service and Instructor Qualification Level Service! Instructor | Air Force | Army Navy TOTAL Level - - Basie 3 L 10 Advaned| 5 | oo 0 2Table S. Total Pilot Study Group Breakdown by Service and Education Level TOTAL{Comparative Study ef Crea K-12 Teaching Skis Initial Study Report January, 2011) ‘Table 6. Total Pilot Study Group Breakdown by Service and Achievement Level Service! Achievement | AirKoree | Army Navy TOTAL Level ii Non | Professional | __0 o 2 2 Professional | __0 o 2 1 Supervisory | 3 | 0 10 13 ‘Upper | Management | 4 1 3 8 Missing | 1 0 1 2 3.2 Comparative Analysis of K-12 Teaching Skills ‘The TSP was validated using a normative sample of 110 experienced K-12 teachers. ‘This experiereed teacher group included several teachers having over 30 years experience, as well as teachers from elementary, middle and high school. The TSP total scores for this group were normaly distributed with a mean TSP total score of 41.9 and a standard deviation of 591, This normative teacher sample was used as the basis for ‘comparing the military instructor group to (normal) experienced K-12 teachers, [A second teacher group also wre used for comparison pirpose for this pilot stdy This group was from a single school district located in Georgia and included 52 teacher ‘mentors that worked within the district. The teacher mentors were pr-identified by the istrict in 2007 without the use of the TSP teaching skill assessment results. At the time of the assessment, the TSP results were tobe used to enhance the district's mentoring process by allowing the district to match the teaching skill strengths of individual teacher ‘mentors fo the teaching skill weaknesses of new or struggling teacher mentees. The TSP {otal scores for this mestor teacher group, as expected, were noticeably skewed toward the high end. That is, amajority of these teachers obtained TSP total seores inthe 3° and 4" quartile relative to the TSP total scores ofthe normative teacher group deseribed previously. These results were used as another important real-world validation point for the TSP developer, Applied Simulation Corp. because the TSP total scores and individual teaching skis profiles accurately portrayed the pre-idenlified and independently verified ieaching competences of this highly skilled proup of teacher ‘The following graph and accompanying combined 3" and 4 quartile summary chart provides the basis for the comparative analysis of K-12 teaching skills between the niltary instructor pilot study group and (normal) experienced K-12 teachers and (highly skilled) teacher mentor