CONSTRUCTING PERFORMANCE- @ Basep AssessMENT Dr. Carlo Magno ADVANCE ORGANIZER o Performance based assessment o Authentic assessment © Portfolio assessment o Assessment “for” learningOBJECTIVES © 1. Distinguish performance-based assessment with the traditional paper and pencil tests. © 2. Construct tasks that are performance based. o Design a rubric to assess a performance based task TERMS o Authentic assessment o Direct assessment o Alternative assessment o Performance testing o Performance assessment o Changes are taking place in assessmentMETHOD o Assessment should measure what is really important in the curriculum. © Assessment should look more like instructional activities than like tests. o Educational assessment should approximate the learning tasks of interest, so that, when students practice for the assessment, some useful learning takes place. WHAT IS PERFORMANCE ASSESSMENT? o Testing that requires a student to create an answer or a product that demonstrates his/her knowledge or skills (Rudner & Boston, 1991).FEATURES OF PERFORMANCE ASSESSMENT o Intended to assess what it is that students know and can do with the emphasis on doing. o Have a high degree of realism about them. © Involve: (a) activities for which there is no correct answer, (b) assessing groups rather than individuals, (c) testing that would continue over an extended period of time, (d) self-evaluation of performances. o Likely use open-ended tasks aimed at assessing higher level cognitive skills. PUSH ON PERFORMANCE ASSESSMENT © Bring testing methods more in line with instruction. o Assessment should approximate closely what it is students should know and be able to do.EMPHASIS OF PERFORMANCE ASSESSMENT © Should assess higher level cognitive skills rather than narrow and lower level discreet skills. o Direct measures of skills of interest. PROBLEMS OF IMPLEMENTATION © Experience in constructing and using performance tests is lacking among many educational measurement specialist. o Performance tests take more time to construct, administer, and score than objective tests. © Standardization, reliability, and validity will be difficult to applyGUIDELINES TO ESTABLISH VALIDITY OF PERFORMANCE TESTS: © Consequences: Do teachers teach differently? What do students learn? © Fairness: There is no guarantee © Transfer and generalizability: Extent of small tasks generalized to larger tasks. © Cognitive complexity: no guarantee that high level cognitive skills are tapped. © Content quality: limited sampling of content is possible GUIDELINES TO ESTABLISH VALIDITY OF PERFORMANCE TESTS: © Content coverage: number of tasks chosen is small. o Meaningfulness: evidence that the assessment is meaningful for students © Cost and efficiency. time consuming and costly to construct, administer, and score thn objective forms of assessment.TERMS Performance-and-product —_ | The emphasis is on the students: SRC uh Send Pe en Ruan ie eae Alternative assessment Method that differs from conventional Paper-and-pencil tests, most Particularly objective tests. Authentic assessment Direct examination of student's ability to use knowledge to perform a task that is like what is encountered in real life or in the real world. CHARACTERISTICS OF PERFORMANCE-BASED ASSESSMENT © Students perform, create, construct, produce, or do something. o Deep understanding and/or reasoning skills are needed and assessed. © Involves sustained work, often days and weeks. © Calls on students to explain, justify, and defend. © Performance is directly observable. © Involves engaging in ideas of importance and substance. o Relies on trained assessor's judgments for scoring © Multiple criteria and standards are prespecified and public © There is no single correct answer. © If authentic, the performance is grounded in real world contexts and constraints.LEARNING TARGETS © Skills » Communication and presentation skills » Ex: Speaking 1 Speaking clearly, expressively, and audibly Using voice expressively Speaking articulately and pronouncing words correctly Using appropriate vocal volume Presenting ideas with appropriate introduction, development, and conclusion Presenting ideas in an effective order Providing a clear focus on the central idea Providing signal words, internal summaries, and transitions e ». Developing ideas using appropriate support materials Being clear and using reasoning processes Clarifying, illustrating, exemplifying, and documenting ideas «Using nonverbal cues » Using eye contact Using appropriate facial expressions, gestures, and body movement Selecting language to a special purpose Using language and conventions appropriate for the audienceo Psychomotor skills Fine motor: cutting papers with scissors, drawing a line tracing, penmanship, coloring drawing, connecting dots Gross motor: Walking, jumping, balancing, throwing, skipping, kicking Complex: Perform a swing golf, operate a computer, drive a car, operate a microscope Visual: Copying, finding letters, finding embedded figures, identifying shapes, discrimination Verbal and auditory: identify and discriminate sounds, imitate sounds, pronounce carefully, blend vowels o Products + Write promotional materials + Report on a foreign country » Playing a new songVARIATION OF AUTHENTICITY ae ee es Indicate which parts of Design a garden Greate a garden agarden design are accurate Write a paper on Write a proposal to Write a proposal to zoning change fictitious, present to city council Zoning laws to change zoning laws Explain what would Show how to perform Play a basketball you teach to students basketball skills in game. learning basketball practice CONSTRUCTING PERFORMANCE BASED TASKS \. Identify the performance task in which students will be engaged 2 Develop descriptions of the task and the context in which the performance is to be conducted. 3 Write the specific question, prompt, or problem that the student will receive. Structure: Individual or group? Content: Specific or integrated? Complexity: Restricted or extended?COMPLEXITY OF TASK © Restricted-type task + Narrowly defined and require brief responses » Task is structured and specific * Ex: © Construct a bar graph from data provided © Demonstrate a shorter conversation in French about what is onamenu © Read an article from the newspaper and answer questions © Flip a coin ten times. Predict what the next ten flips of the coin will be, and explain why. © Listen to the evening news on television and explain if you believe the stories are biased. © Construct a circle, square, and triangle from provided materials that have the same circumference. ial Pr o Extended-type task » Complex, elaborate, and time-consuming. » Often include collaborative work with small group of students. » Requires the use of a variety of information » Examples: © Design a playhouse and estimate cost of materials and labor © Plan a trip to another country: Include the budget and itinerary, and justify why you want to visit certain places © Conduct a historical reenactment (e. g. impeachment trial of ERAP) « Diagnose and repair a car problem © Design an advertising campaign for a new or existing product oOIDENTIFYING PERFORMANCE TASK DESCRIPTION © Prepare a task description © Listing of specifications to ensure that essential if criteria are met co Includes the ff.: » Content and skill targets to be assessed » Description of student activities © Group or individual © Help allowed Resources needed ‘Teacher role Administrative process Scoring procedures PERFORMANCE-BASED TASK QUESTION —7 PROMPT © Task prompts and questions will be based on the task descriptions. o Clearly identifies the outcomes, outlines what the students are encourage dot do, explains criteria for judgment.EXAMPLE OF A TASK PROMPT: ature of Fanal What Students Ane Requised te Do Inte the Product eB CHARACTERISTICS OF TASKS Should integrate the most essential aspects of the content being assessed with the most essential skills. 2 Should be authentic = Realistic » Require judgment and innovation » Ask the student to do the subject * Replicates or stimulates * Assess the students ability to efficiently and effectively use a repertoire of knowledge and skill to negotiate a complex task » Allows opportunities to rehearse, practice, consult resources, and get feedback and refine performances @ and products.Structure the task to assess multiple learning targets 4 Structure the task so that you can help students succeed. Think through what students will do to be sure that the task is feasible © The task should allow for multiple solutions The task should be clear ® The task should be challenging and stimulating to students » Include explicitly stated scoring criteria as part of the task Include constraints in completing the task PERFORMANCE CRITERIA © What you look for in student responses to evaluate their progress toward meeting the learning target. o Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency. © Start with identifying the most important dimensions of the performance © What distinguishes an adequate to an inadequate demonstration of the target?QUESTIONS TO ASK: co What are the attributes of good writing, or good scientific thinking, or good collaborative group process, of effective oral presentation? More generally, by what qualities or features will | know whether students have produced an excellent response to my assessment task? co What do | expect to see if this task is done excellently, acceptably, or poorly? o Do | have samples or models of student work, from my class or other sources, that exemplify some of the criteria | might use in judging this task? o What criteria for this or similar task exist in my state curriculum framework, my state assessment program, my district curriculum guides, my school assessment program? o What dimensions might | adapt from work done by natural curriculum councils, by other teachers?EXAMPLE OF CRITERIA © Learning target: * Students will be able to write a persuasive paper to encourage the reader to accept a specific course of action or point of view. o Criteria: » Appropriateness of language for the audience * Plausibility and relevance of supporting arguments. « Level of detail presented » Evidence of creative, innovative thinking * Clarity of expression = Organization of ideas e RATING SCALES o Indicate the degree to which a particular dimension is present. o Three kinds: Numerical, qualitative, combined qualitative/quantitativeo Numerical Scale + Numbers of a continuum to indicate different level of proficiency in terms of frequencyor quality Example: Complete Understanding 5 43 2 1 No understanding No organization 54321 Clear organization Emergent reader 54321 Fluent reader © Qualitative scale » Uses verbal descriptions to indicate student performance. * Provides a way to check the whether each dimension was evidenced © Type A: Indicate different gradations of the dimension Type B: Checklisto Example of Type A: + Minimal, partial, complete » Never, seldom, occasionally, frequently, always » Consistent, sporadically, rarely ~ None, some, complete » Novice, intermediate, advance, superior » Inadequate, needs improvement, good excellent + Excellent, proficient, needs improvement « Absent, developing, adequate, fully developed » Limited, partial, thorough + Emerging, developing, achieving + Not there yet, shows growth, proficient » Excellent, good, fair, poor @ o Examnle of Tyne A: Checklist FIGURE i Jspan Presentation Criteria Content Yes No | . Two or more axpecisof Japanese society imentionest + —— 2 Thesimlarty between the United States and Jopan presented —— — - } Bplained th ciference between the United States and Japan eae orca Spat ) a mnie© Holistic scale = The category of the scale contains several criteria, yielding a single score that gives an overall impression or rating Example level 4: Sophisticated understanding of text indicated with constructed meaning level 3: Solid understanding of text indicated with some constructed meaning level 2: Partial understanding of text indicated with tenuous constructed meaning level 1: superficial understanding of text with little or no constructed meaning @ EXAMPLE HOLISTIC SCALE FICURES.IO Example of Holistic Kating Scale and Rubi _ Kenting Rt _ - Rating Sele Exuluative Crit _ 4 Reader displaysa sophisticated understanding of the text with sub tel apes eee ce ate tad the author's style, questions the text, provides alternative interpeetations; Views the text from multiple perspectives), " 3 Reader doplayna solid understand solid understanding ofthe text with clear evidence fconsiruct. Sagsneaning: Connections arenas Beiwen the extand the reader's ‘experiences, Interpretations are made and generally supported by appropriate testeforences Reale may reveal acnicalstnce toad tle 2 ‘eer plas prt seconds 5 2 paral urdeotndg ute tex wih some evden contr Ingomar, Acton may tnd nea at srr dlp renee ae crn edb appropriate at ns Tenders no stance toward the text. node of ec 1 Reader displays asuperfcial understanding ofthe text with liaited evidence of {Grstecting meaning. Nocnnections are made betwen the text and the eder's ideas /expertences. Reader provides no interpretations or evident of critical sagceo Analytic Scale » One in which each criterion receives a separate score. Example Crear ror a ec a) cs ) Creative ideas Logical organization Relevance of detail Variety in words and sentences Vivid images | Rusrics o When scoring criteria are combined with a rating scale, a complete scoring guideline is produced or rubric. © Ascoring guide that uses criteria to differentiate between levels of student proficiency.Oral Presentation Rubric Organeaation Language Coneeniions ‘ Coherent organization Excellent volume; fluent Highly effective we of throughout logical delivery with varied language enbances the quent smooth taseitions; intonstionjeecive body message few any Uictive induction and. language andeye contact. grammatical mistakes tonctuson 2 Good organization gener Adyuate volume and Generally effective use Slyybureithocme freakin intenstion; generally Of language supports the the logeal low olideas, fluent generally fective message: mincr gram. iMentiable introduction body language and eye matical errors do not ‘and conclision contut interfere with message. 2 Flawed organization: eas Vohumne is toalow or too _Useoflanguage nota ret developed weak Jour: delivery ienot ways aligned with the fansttions ineffective uent hody language message; grammatical candusion. andeyecontactdomot errors may interfere inhance message. ‘with message 1 Lackofowganization; _—Messagecanrotbeunder- Major grammatical errors flowofideasdifficut to stood duct low volime, make themessage very follow:noevidence of strained delivery; neffece difficult or impossible transitions: nointzoduction five body language; ack follow. ‘oreonctusion, of eye contact o Rubrics should answer the following questions: ~ By what criteria should performance be judged? « Where should we look and what should we look for to judge performance success? « What does the range in the performance quality look like? = How do we determine validity, reliability, and fairly what scores should be given and what that score means? » How should the different levels of quality be described and distinguished from one another? PraGUIDELINES IN CREATING A RUBRIC Be sure the criteria focus on important aspects of the performance Match the type of rating with the purpose of the assessment The descriptions of the criteria should be directly observable 4 The criteria should be written so that students, parents, and others understand them. The characteristics and traits used in the scale should be clearly and specifically defined. Take appropriate steps to minimize scoring frame @ PORTFOLIO ASSESSMENT: EXPLORATION o Have you ever done a portfolio? © Tell me about this experience. Did you enjoy it? o What elements did you include in your portfolio? o Are the materials placed in the portfolio required? dcWHAT ARE PORTFOLIOS? o Purposeful, systematic process of collecting and evaluating student products to document progress toward the attainment of learning targets or show evidence that a learning target has been achieved. o Includes student participation in the selection and student self-reflection. e “Acollection of artifacts accompanied by a reflective narrative that not only helps the learner to understand and extend learning, but invites the reader of the portfolio to gain insight about learning and the learner (Porter & Cleland, 1995) CHARACTERISTICS OF PORTFOLIO ASSESSMENT o Clearly defined purpose and learning targets o Systematic and organized collection of student products o Preestablished guidelines for what will be included o Student selection of some works that will be included © Student self-reflection and self-evaluation o Progress documented with specific products and/or evaluations © Portfolio conferences between students and teachersA PORTFOLIO IS: o Purposeful o Systematic and well-organized o Prestablished guidelines are set-up © Students are engaged in the selection of some materials o Clear and well-specified scoring criteria PURPOSE OF PORTFOLIO o Showcase portfolio: Selection of best works. Student chooses work, profile are accomplishments and individual profile emerges. o Documentation portfolio: Like a scrapbook of information and examples. Inlcudes observations, tests, checklists, and rating scales. o Evaluation portfolio: More standardized. Assess student learning with self-reflection. Examples are selected by teachers and predetermined.ADVANTAGES OF PORTFOLIO © Students are actively involved in self-evaluation and self-reflection © Involves collaborative assessment © Ongoing process where students demonstrate performance, evaluate , revise , and produce quality work. o Focus on self-improvement rather than comparison with others © Students become more engaged in learning because both instruction and assessment shift from teacher controlled to mix of internal and external control. © Products help teachers diagnose learning difficulties © clarify reasons for evaluation e@ o Flexible DISADVATNTAGES © Scoring difficulties may lead to low reliability © Teacher training needed © Time-consuming to develop criteria, score and meet students © Students may not make good selections of which of which material to include o Sampling of student products may lead to weak generalization o Parents find the portfolio difficult to underdstandSTEPS IN PLANNING AND IMPLEMENTING © “== PORTFOLIO ASSESSMENT Determine the the purpose Identify physical structure Determine sources of content «Determine sources of content s Determine student reflective guidelines and scoring criteria « Review with students Portfolio content supplied by teacher and/or student Student self-evaluation of contents Teacher evaluation of content and student self- evaluation Student-teacher conference ® Portfolios returned to students for school PURPOSE © Based on specific learning targets © Ideal for assessing product, skill, and reasoning targets Uses: © Showcase portfolio-to illustrate what students are capable of doing © Evaluation of portfolio-standardization of what to include o For parents-what will make sense to parents “Provide specific attention to purpose and corresponding implications when implementing a portfolio.”PHYSICAL STRUCTURE o What will it look like? oe How large will the portfolios be? o Where are they stored so that students can easily access them? © Will it be in folders or scrap books? o How will the works be arranged in the portfolio? o What materials are needed to separate the works in the portfolio? SOURCES OF CONTENT o Work samples © Student and teacher evaluations Guidelines: o Select categories that will allow you to meet the pupose of the portfolio. co Show improvement in the portfolio o Provide feedback on the students on the procedures they are putting together o Provide indicator systemSELF-REFLECTIVE GUIDELINES AND SCORING o Establish guidelines for student self-reflection and the scoring criteria © Scoring guidelines are explained to the students before they begin instruction IMPLEMENTING PORTFOLIO ASSESSMENT o Review with students: Explain to students what is involved in doing a portfolio. o Begin with learning targets o Show examples © Give opportunities to ask questions © Provide just enough structure so that they can get started without telling them exactly what to do. © Selection of content will depend on the age and previos experience of students © Students and teachers decide together what to include with nonrestrictive guidelinesSOME ORGANIZATION © Include table of contents o Brief description of activities o Date produced © Date submitted o Date evaluated STUDENT SELF-EVALUATIONS Reflective and self-evaluation activities need to be taught. © Some guide questions for students: » Can you tell me what you did? » What did you like best abut this sample of your writing? + What will you do next? o Self-reflective questions: + What cid you learn from writing this piece? What would you have done differently if you had more time? What are your greatest strengths and weaknesses in this sample? What would you do differently if you did this over? @PEER EVALUATIONS e Analysis and constructive, supportive criticism of strategies, styles, and other concrete aspects of the product. © Can include comments or a review by parents Teacher evaluations: © Checklist of content © Portfolio structure evaluation: selection of samples, thoroughness, appearance, self-reflection, and organization. © Evaluation of individual entries: use rubrics o Evaluation of entire content: use rubrics @ a ee — 2 spree pnt mgr oem ce 2 Spr pentagon ys et a SS pee pt ge ay he Poitiers pe a apFIGURE 9, Example of Student Portfolio Checklist Portfolio Checklist (For Language Arts Class Only) Name —) By the end ofthe yeas your portfolio mua contain te vigil expies ofthe following tems: Student Assessment Letters) — eaing Log and ook Reviews Reading Atatude Survey WeitingSamples ———— — Sune, From Parflamonrt Cling vl by AlanA.De Pea. Copyright O92by Alan. De Fina. ‘he! by Scholae Prfcnal Woks New Yrk p79 Raped by Permlso 0 Scat.ie tensiniee Unni mg terete CERISE MACURE 99 Example of Parent Review and Bvahuation Form Parent Folder Review and Reflection Smadent Render tee = Please reed everything ia your child's iting folder inclading rafts and comanentary, Each prose © ext up in backto fren oer, rom rough draft final copy Further each ple & accomparved by both ytodent aac! eacher comments on the Piece ‘and weng process Finaly the flere sleotnchade wet questionnaires wher ‘Suenos weltesbout their srs and woukawsses te wets. "Wcbclieve that the best arseseavnt of student writing begins with the students themacives ut madt be brosdened to include the widest posible audience. We ‘Cncourege you to become parte the audience: When yoru have rend the folder please ulkto your children aboot thir writing. Inodilidon please tke a Sew minules to reepond t9 these questions “= Which piewe of writing inthe folder tells yourmact about your child's writing? + Whaat doesit tll your” 1 What de you seca the strengths in your child's weitng? What de you see that noode tobe addresce in your child's growth and development aeawrier ‘+ What suggestions do you have that might aid the class’s growth a > Other comments suggestions? “Thank you so much for investing this time in your child's writing, Dd —_—_——— ‘Sarr Weta pnt Comers rovhing noel (97) Used with permin fhe Pitergh PD Schou Ftcbaogh PA (Moke Und Srng 195). KENTUCKY MATHEMATIN PorTOUO Sosnowiec! HOLISTIC SCORING GUIDE io aes oe See = FIGURE 9.1 Mathematis Holistic Scoring Gulde ‘Sure: Kentucky Deparment of Education (Mf fCuricalum, Asam and Acct) 1955, Used with permiionSTUDENT-TEACHER CONFERENCES o Conference is conducted with sudents before returning the portfolio o Scheduled throughout the school year; some have it monthly o Clarify purposes and procedure with students, answer questions and establish trust o Give guidelines to prepare for each conference © Allow the students to do most of the talking o Have students compare your reflections with theirs o Weaknesses and areas for improvement need to be communicated —show them what is possible for progress (nae STUDENT-TEACHER CONFERENCES o At the end of the conference there is an action plan for the future © Limit the conference to no more than 10 minutes o Students are encouraged to take notes © Focus on one or two major areas of each conference-helps to have a thoughtful discussion© Wrapping up on assessing student learning Effect of Previous Practices: rank students on achievement by graduation New Expectation: Assure competence in Math, Reading, Writing, etc.MISTAKEN BELIEFS ABOUT HOW TO USE ASSESSMENT TO SUPPORT SCHOOL IMPROVEMENT: 1. High-stakes tests are good for all students because they motivate learning 2. If [threaten to fail you, it will cause you to try harder 3. If a little intimidation doesn’t work, use a lot of intimidationPROFOUND MISTAKE Teachers and leaders don’t need to understand sound assessment practices — the testing people will take care of us. R BELIEF do need to understand sound ossegagiet practices. ASSESSMENT LEGACY 1. Assessment has been far more a matter of compliance than of teaching and learning 2. Disregard of the information needs students and teachers who make the most frequent and highest impact decisions 3. Assessment that drive as many students to give up in hopelessness as they spur to more learning 4. And we fail to provide practitioners with the assessment understandings needed to help