chapter Four___ evaluation of Classroom instruction Regardless of the perspective about teaching, and even as the debate on art versus science continues, what is undeniable is the complexity of the teaching-learning process, and. consequently, of the evaluation process, ‘Advocates of the school of thought, which considers teaching gan art, atgue that classroom instruction is a complex activity x puts primary importance on the teacher variable, among jy other interacting variables such as student, curriculum, sroction, and learning environment. Clearly, this concern for sonal variables in the teaching-learning process is a bias towards iat Wiles and Bondi (1991) refer to as the “art of teaching” gespative, Which is glossed over in the “science of teaching” Joc Based on this perspective, when preparing for classroom issuction, important concerns such as what, how, and how much reach, and under what conditions, among others, are decided individval teachers in consideration of personal lenses, ise, ets, values, philosophy, and experiences Onthe other hand, those who favor the “science of teaching” tive emphasize the notion that teaching is a series of highly, (esrted skills that can be acquired and developed. Those who | Ee belong to this school of thought (widely believed to be more humerous) point to the many studies in the 1970s and the 1980s that statistically isolated the critical variables or dimensions of instruction. Because of the results of those studies, it has been reported pervasively that the reliance on the scientific process as a mode for determining as well as a basis for improving teaching performance has been steadily increasing 1 Evalatioh 6f Classroom instruction | 107 However, regardless of the teaching-learning process perspective (science vs. art), there is a consensus that teaching is, indeed, a complex and multi-faceted process. The uniqueness of the people involved (teachers and students) as well as the uniqueness of the context where teaching and learning take place makes the teaching-learning process truly an intricate and difficult activity. This complexity makes observation of classroom instruction a challenging exercise as well. Wiles and Bondi (1991) contend that research on evaluating teaching effectiveness has resulted in more questions than answers. The debate about art versus science extends to evaluation. While many profess that teacher evaluation isa science, evidence suggests, according to the authors, that itis more an art, Furthermore, they contend that it remains ambiguous, based on a survey of literature which presents diverse perspectives on a string of issues. ‘There are still other unresolved issues regarding evaluation. Many controversial questions remain unanswered. Should evaluation be used in making administrative decisions, ie, retention, promotion, merit pay, tenure, and termination? How can it be tied to faculty development? What is its proper focus— teaching competencies or student achievement? How should these two areas be defined or measured? What factors beyond school contribute to student achievement? Who should gauge the cevaluator’s ability to assess? Who should be involved in evaluation: supervisors, peers, and students?108 | Supervision of rstuction: The Philippine Perspective) TN TYPES OF EVALUATION Evaluation may be formative or summative depending on its purpose. When its purpose is primarily to determine the effectiveness of the delivery of instruction that will serve as the basis for improvement, then the evaluation is formative. Thus, this type of evaluation is esentially intended to improve teaching performance: To this extent, the different strategies for supervising instruction which are intended to help teachers (e.g,, clinical supervision, coaching, mentoring, etc.) are mainly based on formative evaluations. When the purpose of evaluation is to use the results for making administrative decisions (promotion, hiring, ‘merit pay, grant of tenure, reassignment, and even dismissal), then the evaluation is summative. Marczely (2001) differentiates between formative and summative evaluation by appropriately referring to formative evaluation as supervision which she considers as “benchmarking for professional growth.” She argues that supervision is a far more intensive and time-consuming task than evaluation, Because of its importance and far-reaching effects, the author suggests that there can be no valid excuse for sacrificing supervision (formative) for evaluation (summative). THE NEED TO EVALUATE CLASSROOM INSTRUCTION Despite the unresolved issues and controversies, evaluation of classroom instruction remains a common practice in many schools in the Philippines. Why isit necessary to evaluate teaching performance? Why is the evaluation of teaching performance a required skill for an instructional supervisor? Evaluation is arguably an important and crucial process in instructional supervision and, therefore, part and parcel of the responsibility of an instructional supervisor. Despite the attendant problems, the evaluation of classroom instruction is used in many schools to: Evaluation of Classroom instruction | 109 help the instructional supervisor know how to assist teachers in their work; assess the quality of instruction; monitor teachers’ progress in instruction; and serve as a basis for administrative decision-making (rehiring, retention, promotion, and termination), The difficulty in evaluating classroom-teaching performance lies in the estimated 200-300 personal interactions that happen each hour in a classroom (Wiles and Bondi, 1991). The quality of these interactions is either enhanced or constrained by many external variables that influence classroom events, Learners and teachers bring into the classroom a variety of influences that impact the teaching-learning process. In general, these influences such as events that happen in the home prior to instruction are not revealed by evaluation. In principle, classroom observation provides a supervisor with data and information about how well or how poorly teacher and students perform during classroom instruction. To a large extent, its results reveal many useful information regarding a teacher's preparedness and capabilities, as well as student involvement in the learning process. However, Popham (1988) argues that although it is almost universally regarded as an essential component in evaluating teaching competencies, it also has serious problems. Due to the complexity of the classroom-learning environment, many imponderables constrain the process. Many things occur in the classrooms that ate probably influenced by factors which are not school-related (i., whether or not a student argued with a parent the night before). Furthermore, because of the shortage of expert evaluators assigned to conduct classroom observation, the practice produces unreliable and invalid results.110 | Supervision of hsttton: The Prion Perspective IN DIMENSIONS OF EFFECTIVE TEACHING One crucial factor in evaluating classroom-teaching performance is determining the dimensions of effective teaching from among the various teaching behaviors discussed in literature, Varying from expert to expert, these dimensions of effective teaching range widely, and correlate with performance on the job. Fortunately, many experts, regardless of orientation, maintain that these dimensions can be measured against well- accepted standards. According to Halsall (1998), the dimensions of effective teaching include knowledge and understanding of the subjects | being taught; setting high expectations to challenge the learner; planning effectively; employing strategies that match not only the needs of the learner, but also the curricular objectives; and managing the learners well to avoid disciplinary problems in the classroom. On the other hand, Moore (1998) emphasizes the importance of the demonstration of a variety of skills such as reflective thinking, decision-making, planning, communicating, and systematic evaluation. This view is shared by Eggen and Kauchak (2001) who maintain that teachers must possess not only the basic skills, but also higher order thinking skills such as analyzing, synthesizing, evaluating, and problem-solving in order to develop these skills, in return, among the learners. ACONTINUUM MODEL OF “ART AND SCIENCE” Reyes (2002) developed a Filipino model of expert teaching based on a study of 69 expert teachers in the Philippines. The study surfaced the following dimensions of expert, as well as responsible teaching: LE Btatuation of Classroom instruction | 111 Expert Teaching + Subject matter expertise Classroom management expertise Instructional expertise Diagnostic expertise Communication expertise Rational expertise Responsible Teaching + Learner-centered teaching + Skills in developing students’ responsibility for learning + Skills in values integration The Filipino model developed by Reyes provides a continuum of the “art of teaching” and “science of teaching” perspectives. It includes two sub-domains which she classified as essentials and enhancers. Essentials include skills such as effective communication, content mastery, instructional clarity, effective classroom management, sensitivity to student's learning, and high rapport with learners. On the other hand, enhancers include those attributes that contribute to effective teaching such as classroom humor, affectionate interaction with students, values integration, charisma, use of a variety of instructional materials and strategies, and maintenance of students’ on-task behavior. A graphical representation of the model shows a pyramid with the essentials comprising the base and the different enhancers gradually tapering off upwards.112 | Supervision of instniction: The Figure 14 ‘A Pyramidal Model of Teaching Expertise in Philippine Higher Education Classroom humor ‘Affectionate interaction with students Values integration ‘Making learning pleasant and enjoyable Magnetismicharisma Use of varied instructional equipmentimaterials Leamer-centeredness Developing students! responsibilty or learning Expressive body language Excellent oral communication skils Use of varied instructional strategies ‘Maintenance of students’ on-task behavior Content mastery, instructional dary, non-threatening disposition, enthusiasm, absence of class disruptions, efficient handling of routine activities and time management, provision for two-way communication, sensitivity to students’ leaming problems and difficulties, anticipation of probable learning problems or misconceptions, and high rapport with students, ‘Source: Reyes, F. (2002). Unveiling teaching expertise: A showcase of 69 exp teachers in the Philppines. Manila: De La Sale University Press, CATEGORIES FOR EVALUATION Based on the different categories cited by various experts, teaching behaviors may be content-related, instruction-related, climate-related (which include the teacher’s personality), and classroom management-related. Content-related behaviors relate to mastery of the subject matter; instruction-related_ behaviors pertain to methods and strategies including effective communication; climate-related behaviors are concerned with the physical and socio-emotional environment conducive to learning: evaluation of Classroom instruction | 113 nd classroom management related behaviors are associated with eflective classroom organization. When observing classroom Instruction, these different teaching behaviors become the proper subjects of scrutiny. TEACHING COMPETENCIES Although there are variations in focus and emphasis, there is a convergence of thought that classroom observation provides the opportunity to assess different major areas of competence. Effective teaching involves the interplay of various competencies. Wilson (1995) proposes that teachers necessarily concern themselves with the following teaching-related issues: students, curriculum, process of learning, social context in which learning lakes place, and resources available o facilitate learning, Satisfying these requirements demands competence. But what competencies are required in teaching? Parry (1998) answers this question by listing down a cluster of related knowledge, attitudes, and skills affecting classroom performance. He considers the following items as constituting teaching competence: Communication skills (verbal and non-verbal) Knowledge of a variety of teaching strategies (instruction) Skills in planning and organization (planning) Mastery of subject matter (content) Skills in assessing the outcomes of learning (evaluation of learning) Skills in classroom management (classroom discipline) Attitudes that foster learning (personality)114 | Supervision of instruction: T Communication Skills Simonds (2001) argues that communication plays an impor role in instruction. Itisa process that involves the use of verbala non-verbal language to stimulate and maintain student interest, ‘well as facilitate the teaching-learning process. Communicatio enables the teacher to set the tone, and establish a climat conducive to learning. Teachers make use of communicatiol to effect learning, and to manage student behavior. It is also powerful tool for mediating, negotiating, and resolving confi in the classroom. Callahan and Clark (1988) maintain that the use of questi is one of the most important teaching techniques. Questior during class discussion stimulates thinking, and motivat students to pay attention. It enables the teacher to assess studen progress, check on teacher clarity, maintain classroom control and emphasize key points, among others. The students’ successit answering questions posed by the teacher is more often determi by the teacher’s questioning techniques. Thus, itis necessary for teachers to develop good questioning techniques as an essenti communication tool. ‘One vital aspect of communication is listening. According to Christenbury (2000), teachers who listen and allow their students to speak in class without interrupting make their students that what they say is important and worth the attention of both teacher and other students. Instructional Skills These skills pertain to the ability of the teacher to use repertoire of instructional strategies that lead students to bé involved actively in the learning process (Danielson and McGre 2000). A skillful teacher is able to make students participate lively in the learning process so that by doing so, they become accountable for their own behavior. Instructional skills are oa aot Crssst00m Inctruction | 118 demonstrated by competent teachers through the application of motivation and presentation strategies suited to the topic and class level; use of appropriate instructional materials; selection and pacing of the learning activities that sustain students’ interest; case in explaining difficult concepts using simple terms; and the art of questioning. The use of appropriate strategies determines, to a large extent, the quality of interactions in the classroom which promote and enhance learning, The interactions may be between teacher and student or among students themselves. A competent teacher is able to promote students’ understanding of the lesson through a variety of strategies and techniques, monitor student performance, and provide useful feedback that will assist learners in the learning process. Organization and Planning ‘The organization of content, materials, and methods of classroom instruction requires careful planning. Lesson plans developed by teachers contain student learning objectives, instructional procedures, required materials, and written description of how the students will be evaluated. The organization of the elements shows logic and congruence. The content, as well as the instruction and evaluation strategies all flow from one another, and are based on the objectives. When classroom lessons are well- planned, teachers become confident, and implement the prepared lessons with relative ease. When properly conducted, these lessons provide for a brisk and smooth transition of learning activities. Mastery of the Subject Matter There is an old adage which says, “You cannot give what you do not have.” If a teacher does not know the subject matter thoroughly, he/she cannot be of much help to students, Insufficient knowledge or grasp of the subject matter constrains the ability to facilitate learning which is accomplished by providing varied116 | Supervision of instruction: The P ‘examples and applications to lead students to higher order thoug processes, and to relate content with other related or allied a1 or disciplines. Besides, the possibility of poor learning, and worse, wro! earning becomes a serious cause for concern when a teacher is well-versed about the topic under discussion. A common remi to teachers is to make sure that he/she is not only one chapt ahead of the students, but several books ahead instead. In oth words, a teacher is expected to be an expert on the subject matter being taught to be an effective facilitator of learning. This is the Teason why as a policy, a teacher is supposed to teach primarily the subject he/she has majored in, and secondarily, what he/she has minored in during pre-service training. Evaluation Skills Classroom evaluation serves as a meaningful source of information for helping students learn. It is an integral part of the instruction process. Because classroom assessment directly relates to instructional goals, the teacher can use the evaluation results to determine whether the concepts and skills emphasized during instruction were understood or not. In effect, an assessment conducted at the end of the lesson is an evaluation of the quality of the teacher's performance. Evaluation results can be used as bases for improving instruction through further clarification, additional explanation, or sometimes even reteaching. ‘Through classroom assessments, teachers are able to facilitate Iearning by providing students with timely feedback on their learning progress, and helping them with their learning difficulties, ‘To achieve this, classroom assessment must be well-planned in. advance. Strategies for evaluating classroom learning at the end of the lesson include, among others, such activities as giving of short quizzes, requiring students to write summaries and other. compositions, and conducting a question-and-answer session. (EP Evaluation of Classroom instruction | 117 Classroom Management ‘According to Kounin instructional management (California State University, Chico, 2005), classroom management includes all of the things a teacher does in pursuit of two important objectives: 1. To foster student involvement and cooperation in all classroom activities 2. To establish a productive working environment in the classroom. ‘To achieve these twin objectives, itis important for a teacher tomake sure that before instruction takes place, there is all-around. readiness in terms of the room, activities, materials, and more importantly, the students. Readiness requires planning and organization which, according to Kounin instructional management, are crucial, interrelated concepts that set the stage for effective classroom management. The smooth transition of one fearning activity to another or “lesson movement” emphasizes the strong relationship between effective management and effective instruction. To this extent, it can be argued that classroom management is one dimension of effective teaching that is highly dependent on the other dimensions (planning, instructional skill, communication skill, and teacher’s attitudes). For instance, failure of the teacher to plan and design appropriate instructional tasks may cause high incidence of discipline problems in the classroom. Lack of motivational skills and variety in strategies and techniques may result in the lack of interest in the subject matter, or boredom even, that may eventually lead to problems in managing the class. Attitudes that Promote Learning ‘Teaching competency largely depends on the teachers’ personality traits, beliefs, desires, and motivations. Thus, its118 | Suporison fst Ts Phi PSC TENN RINNE PT vata or ciasstoom rstcton | 119 inclusion in the observation process is necessary as it influences GUIDING PRINCIPLES ABOUT CLASSROOM effective teaching. Some of the suggested attitudes and behaviors OBSERVATION of the teachers that promote learning, and create proper climate in the classroom include: high expectations, enthusiasm, concern for children, openness to student inquiry, fairness, humor, sensitivity to the needs and concerns of students, and decisiveness (‘Suggestions for observing classroom instruction,” n.d.) Sullivan and Glanz.(2000) propose certain guiding principles about classroom observation that every observer should bear in mind, These principles can help improve the effectiveness of classroom observation in determining teaching performance. i 1. Good supervision depends on reflective thought and ‘The incorporation of the teacher's “persona” in the assessment discussion of observed behavior. is a recognition that teaching isa complex act, which is based on. the “art of teaching” perspective (Wiles and Bondi, 1991), High . ‘The use of observation instruments provides teachers expectations, enthusiasm, and openness to student inquiry which with data on their classroom behaviors that enhance are important distinguishing characteristics of successful teachers their understanding of and commitment to instructional are difficult to assess, and may require “artistic judgment” which, improvement. toa large extent, isa subjective process, Observation involves the factual description of what has occurred, and its interpretation. Hote gna aac eae Room Conclusions about behavior should be based on the Seen description of behavior observed. . The choice of observation instrument is a collaborative ‘Cangelosi (1991) argues that classroom observation should not _ responsibility of both supervisor and teacher. only be concerned with teaching competency variables because the latter are not the sole determinant of a lesson’s success. Classroom Personal bias of the evaluator due to his/her “personal observation must also focus on student outcome variables because lenses” as a result of experience, belief, values, and student achievement is the goal of instruction and, therefore, the philosophy can lead to misinterpretation of observed success ofa lesson depends on how well the students realize the behavior. daring gals Observation is a skill thatis developed through training and practice. In evaluating teaching effectiveness, it is necessary to know a the degree to which the students have achieved the learning goals Soccabaes Byte isso | and i Serr te Auer, Int eae . Feedback is an essential element for successful of the instruction on student progress and development. This is _ ‘Ofaervation. based on the assumption that the quality of the product reflects the quality of the process that produced it. Not alll classroom behaviors can be observed. “Multiple observations with different foci of interests are necessary,120 | Supervision of instruction: The Philippa TYPES OF OBSERVATION Cangelosi (1991) identifies five types of classroom observation, ‘These include structured observation, ecological observation, ethnographic observation, observation based on an in-class rating scale, and informal observation. Structured Observation This type of observation requires the use of an instrument that limits the focus of observation on the items specified in the measurement tool. It is intended for summative evaluation of instruction as one of the bases (among many othets) for arriving at an informed administrative decision (i.e, rehiring, promotion, termination) regarding the faculty. It is conducted, at least, once a year, usually towards the end of the school year. To minimize _ the subjectivity of the evaluation, the items are constructed based on low-inference criteria (i.c., very clear, specific, and focused on observable or measurable behaviors) Ecological Observation ‘This classroom observation involves observing and recording classroom conditions, all learning events, and all types of interaction between teacher-student, as well as student-student, whether verbal or non-verbal, that take place during the entire observation period. The observer comes to class without any predetermined expectations as to what is important, or what types of learning events or interaction will be recorded. The objective of the observation is to obtain a comprehensive and exhaustive record of the teaching-learning that takes place during the period. To insure this, the use of video or audio recording, with the knowledge of the teacher, is sometimes resorted to. This type of observation is most useful in conducting formative evaluation for beginning and marginal teachers, Its result is used for devising an appropriate developmental supervisory (UREN Evaluation of classroom instruction | 122 plan for the target teachers. This type of evaluation is useful in conducting classroom research. Ethnographic Observation ‘Similar to ecological observation, this type of observation does not have predetermined expectations as to what should be the focus of the exercise, The difference is in the recording of the learning, events, Unlike ecological observation which does not involve selective recording of classroom events, ethnographic observation entails selective recording of information based on what the observer considers at the time of the monitoring as significant and worth noting, The observation is recorded through the use of qualitative descriptions rather than scores in rating scales. The narrative descriptions may serve as formative evaluation that can be used as a guide in devising a development supervisory program for individual teachers, Ethnographic observations are best performed by experienced supervisors who have become ‘connoisseurs” of the teaching learning process. Intuitively, only those that have developed the skill can differentiate, for example, between disruptive noise which is a disciplinary concern, and constructive noise that is due to excitement and enthusiasm of the students about the lesson. Observation Based on an In-class Rating Scale ‘This type of observation is similar to structured observation ‘which uses an instrument (rating scale) focused on predetermined aspects of the teaching-learning process. The difference is in the type of rating scale used. While the criteria included in the structured observation are low-inference in order to minimize the subjectivity of the observation, in-class rating scales consist of high inference items which are highly subjective. ‘The high inference of the items fends themselves to different interpretations. For instance, items in the in-class rating scale may include abstract concepts such as teaching effectiveness or teaching122 | Siporison of sti Ts Pipi PBS efficiency which are not described in terms of specific indicators of behavior. What are the indicators of teaching effectiveness and. teaching efficiency? The lack of specifity of behavioral indicators that describe the abstract concepts being observed makes the evaluation vulnerable to biases that result in diverse interpretations. Informal Observation Since it is the most common and the easiest to perform, this type of observation is conducted most frequently, Some educators call it_ “kangaroo observation” which evokes the image of an animal hopping from one point to another which is exactly what the observer does. He/she drops by the classroom unannounced, lingers for a few minutes, observes teacher-student interaction, and then hops to another class. Informal visits are usually referred to as “walk-through” or “management by wandering around” (MBWA). The study conducted by Blasé and Blasé (1998) revealed that teachers view this practice positively. Teachers consider the high visibility of principals as a demonstration of both support and caring that increase their motivation, self-esteem, sense of security, and morale, LENSES OF OBSERVATION It is axiomatic that instructional supervisors should be knowledgeable in determining teaching effectiveness, This is a non- negotiable skill that all supervisors must possess and continue to develop. They should be able to assess the behaviors and practices of teachers during classroom observation, and provide valuable constructive feedback to help teachers improve their teaching skills. What are the areas to be assessed? By what lenses should observation be conducted? Borich (1999) focuses on what he calls lenses of classroom observation as evaluation areas of classroom teaching performance, ‘The tenses include learning climate (physical and emotional (ENE rasta stun | 123 learning environment); classroom management; lesson clarity; instructional variety; teachers’ task orientation; students’ engagement in the learning process; student performance outcomes; and promotion of higher thought processes. Learning Climate ‘This lens includes the physical and socio-emotional aspects conducive to learning, i, facilities and materials, student observance of class rules and procedure, teacher-student and student-student interactions, and teacher's expectations of students. Classroom Management This lens involves a more comprehensive view than learning climate, The foci are on student behavior, cleanliness and orderliness of the environment, smooth flow of the lesson, and observance of classroom rules and routines. Lesson Clarity Its concerns are on the clarity of explanations or directions; logical sequence of the lesson based on the expressed objectives; suitability of the strategies used for motivation, presentation, lesson proper, and evaluation; effective use of audio-visual aids; ‘quality of the questions and answers that promote higher order thought processes; and student understanding of the practical applications of concepts and theories learned. Variety ‘The assessment focuses on the variety of teaching strategies used; provisions for different modes of learning (visual, auditory, tactile, kinesthetic); use of different attention-getting devices; variations in the use of non-verbals (eye contact, voice, gestures); and different uses of rewards (compliment, smile) and reinforcers (examples). Task Orientation ‘The emphases of this evaluation are the extent of the implementation of a well-designed lesson (congruence between124 | Supervision of insiuction: The Phipie Pek Ela ‘Evaluation of Classroom instruction | 125 the written and taught curriculum); the teacher's maximization properly, may discourage interpersonal and collegial relationships, his/her opportunity to focus on instructional tasks that are ‘nd instead promote a superior-subordinate orientation that causes planned; and the effectiveness of the teacher in maintaining ‘mistrust and even antagonism. a sty of the lesson by anticipating and preventing stude An instrument to gather empirical observations is needed i {0 assist the classroom observation process. A well-designed Student Engagement instrument, when properly used by a trained observer, produces ‘The foci of the attention are the quality of learning activiti measurement results that have high degree of validity and or tasks that promote active learning which may include exerci \cliability. Supervisors must be careful in the use of evaluation problem sets, seat works, and group activities; the monitoring instruments. Cangelosi (1991) cautions against what he calls such activities; and the provision of feedback to guide students, “pseudo measuring instruments.” The items in such instruments are high-inference criteria that will yield highly subjective interpretations. For example, a pseudo instrument intended to measure teaching competency might include the items as shown in Table 11. Student Success The assessment focuses on the students’ completion of thei work at moderate or high level of success; the timeliness of feedback and corrections; and the maintenance of classroom pacing and momentum. Table 11 An Example of a Pseudo Instrument Higher Thought Processes a Rating Teisnecessary to assess whether the teacher provides activities, = The teacher deplayed mastery ofthe RT and asks questions that require critical thinking, reasoning, and ‘alaeetinaliec problem-solving. Individual and group activities should afford ‘The teacher used effective and appropriate 412 whee ae the necessary opportunities for higher order thought processes communication and discovery. 4 The teacher conducted the class very well. ee ee aT THE NEED FOR MEASURING INSTRUMENTS Based on the way the items are constructed, the assessment of the teacher's mastery of the subject matter, the use of effective and appropriate communication, and the conduct of the class in general will al depend on the subjective judgment of the observer. What will be the basis for making an objective assessment of how well the teacher knows the subject matter? How will one determine whether the teacher uses effective and appropriate communication? What is the basis for determining whether the teacher conducts the lesson well? If these items are supposed to be rated on a scale of 1 to 5, what criteria will be used fora rating Because of the complex nature of classroom life, instruments for systematic recording of classroom interactions are needed to assist teachers in understanding more fully and becoming more aware of classroom behavior (Good and Brophy, 1997). However, Sullivan and Glanz (2000) argue that the use of observation instruments is problematic when directly connected to evaluation, This is because “life in the classrooms” is based on context and dependent on situation that contributes further to its complexity. Given this condition, the use of evaluation instruments, if not doneof 1 or5? Asa consequence, the variability of the different sco by several observers is expected to be wide because of the lacko indicators for each of the behaviors being rated. The items can be improved by using specific indicators of behavior that are low inference (i.e., measurable and observable) For example, using the following specific indicators of beha\ the evaluator can assess the mastery of the subject matter, the conduct of the class. 4 Table 12 Examples of Low Inference Indicators of Subject Masts item Rating 41._Taught without reading notes, Aaa vee Provided examples to ilustrate dificult terms or concepts. 2a Gave accurate answers to students’ questions, Related the topic to reabife situations, 1 Related the subject matter to other fields. | 1 Likewise, the effective communication m: ; ay be determined, through the observation of the following indicators. 1 Table 13 Low Inference Indicators of Communication Skills Item Used correct grammar in speaking, Maintained eye contact with students. Considered and used students’ ideas and suggestions, ‘Asked probing questions, ‘Spoke in a voice that is clear and loud enough to be heard by everyone. [eS TINNY atten ot classroom natucton | 127 ‘The sample items in both Tables 12 and 13 are specific, and indicate the behaviors being assessed. ‘They are measurable and observable, and as such, do not provide a wide latitude of scores for the observer in assessing the behavior. To this extent, the subjectivity of the observation, which is the major feature of pseudo instruments, is lessened to a considerable degree. When observation instruments are poorly constructed, the results of the observation do not help teachers improve inadequacies in specific teaching behaviors. Vague and fuzzy observations are usually challenged by those observed. When they are not adequately defended (which is usually the case), they can cause conflict and demoralization among the faculty DEVELOPING A MEASURING INSTRUMENT According to Cangelosi (1991), the development of a usable classroom observation measurement thats likely to produce valid and reliable results depends on how well the developers address important issues. Some of these concerns plus other criteria suggested by Shinkfield and Stufflebeam (1995) are the bases for the following proposed guidelines for developing a measuring instrument. 1. The development of a measuring instrument is done collegially. The purpose of evaluation is clarified by defining the evaluation variables with specific sub-variables. “Measurable and observable indicators are identified for each sub-variable. ‘The items are developed and then reviewed for content validity by experts. The instrument is revised based on the comments and suggestions of experts, as well as on statistical analysis (factor analysis).128 | Supervision of instruction: The Philippine F 6. The revised draft is pilot-tested in one or two classrooms. to obtain feedback on clarity of directions and procedures potential for validity and usability, ‘The items are modified based on the feedback obtait during the field-testing. ‘The instrument is field-tested with several classes. Feedback from the field tests is used to finalize the instrument. A formal field test is conducted to assess reliability of the instrument, Based on the above procedure, Balajadia (2004) developed, in consultation with the faculty, a classroom observation instrument consisting of seven evaluation areas: planning and organization; classroom management; command of the subject matter; instructional skills; communications; class rapport; and evaluation of students’ learning consisting of 54 item indicators across these areas. However, through factor analysis, the areas were later changed to teacher's task orientation; lesson clarity; student engagement in the learning process; learning climate; communication skills; instructional skills; and command of the subject matter. Table 14 contains examples of the changes made as a result of the factor analysis in the Balajadia study. ASSESSING MEASUREMENT VALIDITY AND RELIABILITY Assessing the validity involves the determination of the degree to which an instrument measures what it actually intends to measure (Wiseman, 1999). It is an essential, a sine qua non or an indispensable element in determining the quality of the measurement. There are four categories of evidence that indicate the validity of the instrument: 1) concurrent-related evidence, 2) construct-related evidence, 3) content-related evidence, and 4) predictive-related evidence. (uN eWatdation ot Ctassroom Intruction | 129 Table 14 ‘ ‘An Example of the Results of Factor Analysis Category Item: Before Factor | After Factor ‘Analysis Analysis Sew tonatieaming mails | planing and Teach ae organization | grntaton Implemented the planned lesson | Planning and | Instructional fully during class discussion ‘organization skills Lesson Related subject matter with ‘Command of the previous lesson. ‘subject matter | clarity Provided motivation to stimulate | Instructional | Student students’ interest in the lesson, skills ‘engagement Provided at least three-second waiting time before posing questions to class. Instructional | Leaming skills climate ‘Source: Balejadla,R (2004), Development and validation of facuty evaluation system and classroom evaluation insttuments for the archdioesan schools in Pampanga, Unpublished doctoral dissertation, De La Salle University, Mania Concurrent-related Validity Evidence ‘The evidence shows the degree to which performance on one instrument relates to performance in a standardized instrument. For example, if the results of the evaluation using the prepared classroom evaluation instrument have a high correlation with the results of a commercial evaluation measure given at approximately the same time, then, it can be concluded that the prepared evaluation instrument has high concurrent validity. Construct-related Validity Evidence ‘The evidence shows the degree to which an instrument measures a trait (or construct) that is abstract and, therefore, not directly observable. Examples of abstract concepts are anxiety, creativity, caring for students, and openness. They are determined by analyzing or making an informed judgment on whether or not130 | Supervision of instruction: The Philippine Perspective Tas the observable items or specific indicators of the abstract concept describe the trait being measured. Content-related Validity Evidence ‘The evidence demonstrates the appropriateness and comprehensiveness of the content. It provides information on the adequacy of the items to measure the content being assessed, It assumes that the content and the format of the instrument are consistent with the definition of a particular variable being _ measured. Predictive-related Validity Evidence The evidence provides information as to the degree to which estimated performance becomes a reality. It is determined by correlating the results of performance on the instrument with another measure given at some future time. The instrument has high predictive validity ifthe results are consistent with the results of a future measure, ‘Wiseman (1999) emphasizes a view shared by Cangelosi (1991) that a major component of validity of any measuring instrument is reliability, Reliability refers to consistency. An instrument is reliable if it can be depended upon to yield non-contradictory results. There are three categories of evidence that inform about reliability: 1) stability-related evidence, 2) equivalence-related evidence, and 3) internal consistency-related evidence. Stability-related Evidence This evidence refers to the degree to which scores of a group of individuals on the instrument administered on one occasion are consistent with the scores of the same group using the same instrument given at a later date, When the measure is stable, the results on both occasions have high correlation. Equivalence-related Evidence ‘This evidence refers to the extent to which two forms of a ‘measuring instrument yield similar, if not identical, results. The two | EENTENNNNEaaton rasta nstuctn | 192 forms are administered to the same group, and the correlation of the results are computed and analyzed. When the two tests are equivalent, then when administered, the results will show high correlation. Internal Consistency-related Evidence This evidence provides information on the agreement of the different items in one instrument. This is determined by splitting the items in one instrument into two parts. The scores on both halves are then computed and analyzed to determine the reliability of the instrument, The test has high internal consistency if the result from one-half of the test shows high correlation with that from the other half. This procedure is referred to as the split-half method. Aside from validity and reliability of the instruments, Fraenkel and Wallen (1994) maintain that its also important to identify the usability of an instrument. To determine usability, the following questions need to be addressed: How long will it take to administer the instrument? Are the directions clear and easy to understand? Is it appropriate for the intended groups? Is it easy to score and interpret the results? How much does it cost? Do equivalent forms exist? Have there been reports of problems from other users? EVALUATION OF BEGINNING TEACHERS Cangelosi (1991) maintains that the most challenging and difficult evaluations are those involving beginning and marginal teachers. Undoubtedly, as mentioned in Chapter 3, for majority of teachers, the first three years of their professional careers are the most significant, challenging, and threatening. It is during these early years that neophyte teachers try to adapt to their new career and working environment without the benefit of a wealth of professional experiences on which to base their decisions. They have to struggle to prove that they can survive in the classroom, as well as to prove their worth to students, parents, colleagues, and administrators, They do not enjoy security of tenure which isa cause of instability.[EValualion of Classroom Instruction | 183 132 | Supervision of Instruction: The »Perspec [ae cena Given all these conditions, beginning teachers are gener Jo stay on in the profession, and in the process, prevents them from occupied with feelings of doubt and fear of inadequacy (Glickm secking more satisfying careers for themselves. 1985). These feelings are compounded by a heavy workload additional assignments that neophyte teachers are not prepared handle because of their lack of experience. These problems concerns will impact the evaluation of classroom performand On the other hand, the presence of potentially competent jeachers who are not identified and given the necessary direction, guidance, and support may result in either perpetuation of icf in the classroom, or erroneous termination. New teachers who are having adjustment problems, and are . wpa cst) Pe ee Cee aay iven support and encouragement by experienced teachers a hai eae are more ely omen Giana teachers) while the second may ruin reeaealy pe ac zt a be less effective in the classroom. Consequently, they leave th eventually’ eae A te me of a valuat profession in favor of non-teaching jobs (Duke, Cangelosi, and the teaching prot c Knight, 1988 in Cangelosi, 1991). EVALUATION OF EXPERT TEACHERS This is true even for “born teachers” who are easy identify for they usually stand out. Sometimes, non-instructional Expertteachers are valuable resource for: sene ae expectations such as those involving personal relations wit possible, school administrators like i ae Pt a colleagues and administrators might result in their dismiss cali techn pie i amenne eye cer In addition, perceptions about the lack of support for excellent secondarily, to serve as role models for spegoerene ao teaching and high professional standards may be sources of This continues to be a major practice of sc ae ae disenchantment and frustration that may eventually force e in the US., according to Ryan and Soca ne ee rt ew the born teachers to change career. is an aggressive recruitment of experienced teachers fro ‘ Philippines for placement in American schools. Many experience: EVALUATION OF MARGINAL TEACHERS teachers find the opportunity to teach abroad attractive, largely because of the high salaries offered to teachers in developed countries. Asa consequence, the lack of expert teachers, especially in science and mathematics, has now become a bigger problem for many local school administrators. The more problematic area is distinguishing between the Potentially competent and the misplaced individuals. There is a need to differentiate between potential for success and potential for failure (Cangelosi, 1991). The misplaced individual, if i 4 991) lists some of the reasons why allowed to go on without any drastic intervention measures McGrath ¢ re nie: : which are usually costly both emotionally and physically, will lexpert seachers leave: te prot perpetuate instructional incompetence that would be difficult to 1. Lack of opportunities for advancement reverse as time goes by. This has dismal consequences, because : i ed like professionals instructional incompetence is a disservice to the community, and 2. So aalune to bs treated Ae Bs taints the teaching profession. Besides, allowing teachers who are 3. Failure to reward excellence 4. instructionally incompet fe teachin, 1 tructionally incompetent to continue teaching encourages them Thee oF tester decetant making5. Low salaries Inorder to reward and motivate teaching excellence, Ca (1991) suggests summative evaluations based on cost-effe ‘measurements to serve as bases for designing merit-pay prog (based on levels of productivity), and career ladder progeal (schemes to enhance teachers’ opportunities for promotions). success of these strategies depends on how well the eval instruments discriminate meritorious, excellent instruction a teaching performance that is merely competent. Because summative evaluations for advancement of expe teachers affect individual professional satisfaction, as well income, they can also create controversies and conflict in organization. Antagonism and perceptions of unfair treatmen may arise when expert teachers fail to qualify for advance because of evaluation results, or when ‘they think that others been unfairly promoted ahead of them. To avoid this, Ca (1991, p. 176) suggests that these crucial questions be resolved: “1. How well does performance, relative to the summati evaluation variables, correlate with qualifications For example, will an expert teacher whose instructio performance rates higher, according to the summati fora teaching team than one whose teaching receives lower ratings but whose interpersonal skills rate higher? ‘Does the evaluation discriminate only on relevant variables (e., ones subsumed by teaching performance), and not on irrelevant variables (¢g., ethnicity or sex)? Ate criteria, evaluation variables, and the process for making evaluations communicated to all affected parties? (TET atone sree | 198 Well-planned career ladder programs should provide opportunities for expert teachers to have instructional sesponsibilities that extend beyond classroom concerns such as Jeading 1) a curriculum development team, 2) an instructional demonstration team, or 3) a research team. Summative evaluations for expert teachers are designed to \dentify exemplary meritorious instruction. This presupposes that the expert teachers are properly identified and screened. They can be recognized through (1) consistent high summative evaluation ratings on classroom performance over a period of three years; (2) endorsement from peers (colleagues and professional groups); (3) fulfillment of higher level credentials or certification for the advanced position (j.e., completion of graduate programs); and (4) scholarly work (research and publications). This implies the need for a well-established and well-defined evaluation system where all the foregoing concerns are appropriately appraised. EVALUATION SYSTEM While their main arena is the classroom, teachers are also expected to fulfill additional roles and functions outside the classroom that complement, and even enhance their primary task of teaching. This suggests that in making administrative decisions regarding a tenured faculty, especially concerning advancement or promotion, administrators do not simply rely on evaluation that focuses only on classroom teaching. Other criteria that comprehensively describe a teacher’s overall performance are also considered in the decision-making process. The choice of criteria is usually guided by the mission-vision of the school, as well as accepted concepts and principles found in literature. It is not surprising that because of the divergence in the mission-vision of educational institutions, teachers are expected to fulfill roles and functions peculiar to the nature of their respective136 | Supervision of instruction: The: schools. For instance, one can detect differences in the crite set by a sectarian school (Catholic, Protestant, and Islamic) an @ non-sectarian school. For sectarian schools, the criterion @ adherence to gospel values or to the values of a particular patro saintof a Catholic schoo! may figure prominently in the evaluatio non-sectarian schools. Asa rule, the evaluation system implemented in a school clearly defined in faculty and administrative manuals, Evaluat systems include specific elements such as the rationale, arcas 0 evaluation, types of evaluation, and evaluation procedure, amony others. The rationale explains the nature, objectives, and benefits to be derived from the evaluation system. The areas of evaluation identify the different dimensions to be assessed in addition t classroom teaching such as efforts exerted towards professiona growth and development, demonstration of ethical conduc community involvement, and other indicators of what may be considered as “going the extra mile.” Each of the criteria is usually defined by indicators which are directly measurable. For instance, professional growth and development may be demonstrated by attendance in of organizing and conducting seminar-workshops, whether in campus or off-campus; undertaking research or publication of articles in reputable journals; active membership in professional organizations; enrolment in graduate studies; and getting awards or recognition from non-government organizations (NGOs) or government organizations (GOs), Ethical conduct may be gauged by the practice of professionalism, in dealing with students, colleagues, parents, and school administrators, as well as in adherence to the established rules and regulations of the school. Community involvement can be assessed through active participation in the school extension program, as well as active membership in NGOs with specific advocacies for [Bvallaion/of Classréom Inctrucion | 187 Eee improving the life of marginalized communities, and religious organizations whose programs and projects are based on the principle of preferential option for the poor (POP). The phrase ‘going the extra mile” may be measured in terms of a teacher’s yoluntary and wholehearted participation in school activities. Because of the difference in the objectives of formative and summative evaluations, Cangelosi (1991) contends that instructional supervisors who try to help teachers improve their craft through clinical supervision, coaching, mentoring, etc. should not be involved in summative evaluations. According to him, when instructional supervisors are freed from the burden of conducting summative evaluations, they can concentrate on making more effective and efficient in-service and staff development programs. Instructional supervision aims to assist teachers become better practitioners rather than to assess them for employment purposes. Instructional supervision, if done properly and adequately, will also prevent the occurrence of legal problems, Of course, this ideal setup can be done easily in big schools with adequate budgets. However, in schools which are not well-endowed financially, either because of size or tuition structure, the separation of tasks which requires additional manpower will be held back by budgetary constraints, ‘The evaluation procedure specifies the data-gathering process (including the instruments to be used); the feedback mechanism; the needed documents to be submitted to support claims about accomplishments and achievements; the schedule and frequency of the evaluation; and the identification of the evaluators (i.c., principal, assistant principal, subject area coordinator). It also explains the way the different criteria will be assessed, and the weight apportioned for each criterion, Table 15 isan example of an overall faculty evaluation form indicating the areas of evaluation and the corresponding weight per area.188 | Supervision of instriction: The’ Table 15 Overall Faculty Evaluation Form Overall Rating Obtained Per Aroa Area of Evaluation Weight a Rating Teaching Performance Professional Growth and 20 Development 60 (I Evaation of Classroom instruction | 139 inasmuch as the teacher can behave more naturally and more comfortably during the observation. The assignment of weights to the different raters in the archdiocesan schools in Pampanga, as reported by Balajadia, seems to follow this line of thinking because the highest weight is given to the subject area coordinator and the lowest to the principal. Table 16 An Example of Assigning Weights ‘Community Service Professionalism 20. OVERALL RATING: 10 100 Evaluator Weight‘) Principal 10 Vice-principal 20 ‘Sourvo: Balajadia,R, (2004). Development and validation of facuity evaluation sy and instruments for archaiocesan elementary schools in Pampanga. Unputlis ‘dissertation, De La Salle University, Ma In the example, the weight for each of the different evaluator is indicated. The usual weight system used by many schools i based on the principle that the closer the evaluator to the tea. the higher the weight should be, Thus, if the principal, the vi principal, the grade level head, and the subject area coordinat are all involved in the evaluation, itis the rating of the subject coordinator that has the highest weight, followed by a grad decreasing weight for the grade level head, the vice-principal, and the principal. Since middle-level managers are closer to the teacher, they are in a better position to assess the teacher, presumably because they know the teacher better than high-level ‘management. Moreover, because of the better rapport that is supposed to cexist between the middle manager evaluator and the teacher, the evaluation process is less threatening and may yield better results Grade level head 30 Subject area coordinator 40 Finally, when the development of an evaluation system is done collectively by the academic community based on the above guidelines, the process becomes clear, its subjectivity diminished, and the evaluation becomes less troublesome, and possibly, more acceptable to the faculty. Summary. 2 Evaluation requires personal judgment which makes it a difficult process whose results are oftentimes challenged. In order to diminish the subjectivity of evaluation, itis necessary to develop summative evaluation instruments based on accepted criteria that are specific and measurable, These instruments must pass the test for validity and reliability. ‘The clarity and specificity of the measures plus the skill of the evaluator in conducting evaluation will help insure the credibility of the process.140 | Supervision of Instruction! The ‘The evaluation procedure should be explained properly t all those who are concerned. Moreover, it is important that the evaluation system implemented in a school is clearly defined in faculty and administrative manuals. The procedure should specify the data-gathering process (including the instruments to be used); the feedback mechanism; the needed documents to be submitted to support claims about accomplishments and achievements; the schedule and frequency of the evaluation; and the identification of the evaluators (i.e., principal, assistant principal, and subject area coordinator) Additional information about the evaluation systems contained in faculty and administrative manuals may include the elements suggested by literature such as the rationale, the areas of evaluation, and the types of evaluation (formative or summative). The rationale explains the nature, objectives, and benefits to be derived from the evaluation system. The areas of evaluation identify the different dimensions to be assessed in addition to classroom teaching such as efforts exerted towards professional growth and development, demonstration of ethical conduct, community involvement, and other indicators of excellent teaching. ‘When the evaluation procedure is clear, well-documented with specific guidelines, and fully-understood by all those concerned, conflicts that usually surround the process are lessened if not totally avoided. Key Concepts Evaluation Teaching performance Student performance Measuring instrument Observation skills (UN Evaluation of Classroom instruction | 141 Neophyte teachers Instrument validity and reliability Science of teaching Artof teaching Teaching skills REVIEW QUESTIONS 1, In what ways does the “art versus teaching” orientation impact evaluation? ‘What makes evaluation of teaching performance a difficult and complex process? ‘What are the different dimensions of effective teaching? ‘What skills must a supervisor possess and develop in order to make a good classroom observation? ‘What are the guidelines for developing measuring instruments for classtoom observation? How can the validity and reliability of measuri instruments be ascertained? ‘What are the important considerations in the evaluation of beginning and marginal teachers? What are the important considerations in the evaluation of expert teachers? ‘What is the importance of a’ well-designed evaluation system?