You are on page 1of 10

Feedback:HowLearningOccurs

byGrantP.Wiggins

InowknowthatIhavetwothingsincommonwithMichaelJordan.First,like Michael,Iwascutasasophomoretryingoutforthevarsityteam(soccer,inmycase). Second,asyoumightguessfrommyvoice,isthatImgettingovertheflu.WhenI considertheeffortittooktogetonanairplanetobewithyoutoday,Imstaggeredat JordonsperformanceintheNBAfinals,playingwiththefluandleadinghisteamto victory. Hisperformanceisactuallyagoodleadintoourtopic:feedbackawordweusea lot,awordfrontandcentertowhywereallatthisconference.Butfeedbackisaword thatIvelearnedovertheyearsreallyneedssomeanalysis,somecarefulthought. Unthinkingly: peopleoftenmisusetheword.SoIwanttotalkverybasicallyaboutwhatfeedbackis andisnt. TogetbacktoMichaelJordanandathletics:Oneobviousthinginwatchinggood athletes,andeveninlisteningtothemininterviews,isthattheyoftenmakeclearjust howvitalongoingfeedbackistotheirmastery:IwasparticularlystruckbyTiger WoodssrecentremarkswhenhewontheMasters.Whenaskedhowheturnedaround hisearlypoorperformance,hedescribedhow,onthebacknine,whenhewasnot playingwell,hesaidtohimselfthathehadtoadjusthisperformance.Buttoknowyou needtoadjust,youneedongoingfeedback.Tigerknewheneededtoadjustonthe basisofthefeedbackthathewasreceivingnotfromanyperson,psychometrician,or indirectproxytestbutfromtherealthing,theunintendedeffectsofhisputtsandhis drives. SoIwantustothinktodayaboutapointthatisutterlycommonsensicalinthewider performanceworld,butstillhardtograsp,oddlyenough,inschools(whereonewould thinkthatexemplarylearningtakesplace):Youdontgetgoodatanythingwithout feedbacknotfeedbackinthesensethatanexperttranslatesthingsforyou,but feedbackinthesenseofwatchingtheball,whereitgoesandwhereitdoesntgo,and realizingwhattheresultmeansforyournextactions.Toshowthatfeedbackentersinto alllearningandselfadjustment,considersomesimpleexamples:Weusefeedbackas wedrivebylookingatroadsignstomakingsurewereheadingintherightdirection; orweusefeedbackfromavideotapethattellsuswhatwedidanddidnotdoonour swingorinyourclassroom. Lestthereferencestodrivingorsportsstrikeyouasatadunintellectualinaforum suchasthis,letmerefertoPlatoandtotheSocrateswhomwefindinPlatosdialogues. Theveryideaofadialogueimmediatelyalertsusthatfeedbackandreactionwillbe

central.Indeed,whatmakesthedialoguesnotonlysointerestingandcharmingbutalso sopowerfulisthatSocratesinvariablytakeshiscuefromthewordsandideasofhisco participants.Hedoesnthaveacannedspeechtooffer.Indeed,ifyoucomparetwo dialoguessay,forinstance,theMenoandtheTheaetetustheybeginvery similarlybutprogressdifferentlyasaresultofthereactionsofthetwonamesakes.Each ispuzzledbySocratessrequestforasingleconception(virtueandknowledge, respectively)aftereachhaslistedexamples.Iwanttoknowwhattheexampleshavein common,saysSocrates.Tryagain.Andthenthedialoguesdivergeintoneand depthbecauseMenofailstograspthenatureoftherequestandTheaetetusimmediately seestheerrorinhisfirstresponse.Onemightsaythattherestofeachdialoguetakes shapearoundthefeedbackthatSocratesreceivestoallofhisquestionsandtoallofhis answers. Soletmesayitagain,commonsensically:Ifyouwanttoaccomplishapurpose,you needfeedbackSo,todayletussuspendfancytalkofvalidityandreliabilityIsimply wanttotalkaboutthatpartofassessmentthatconcernsjudgingwhethertheperformer isaccomplishinggoalsandabouttheroleoffeedbackinbothlearningandassessing. Butcommonsenseendsattraditional]testing.Ourinitialaxiomthatallaccomplish mentsrequirefeedback,whenappliedtostudentperformanceduringandafterassess ment,beginstolookquiteradical.Ironicallyandsadlyenough,thoughtherestofthe worldseemstounderstandtheimportanceoffeedbackinhowitconductsitsaffairs,we intheacademystillclingtotheviewthatlearningisastraightforward,linearaffair:We teach,studentslearn,andtestsrevealwhatandwhethertheylearned.Further,itseems asifteachingitselfdoesnotneedfeedback.Manysyllabiareimpervioustofeedback (namely,diversestudentresponses,interests,ortrouble),somuchsothatsome professorsreusesyllabiyearafteryear:IfitsthefirstweekinNovember,itmustbe supplyanddemandorParadiseLost.Whenthereisformalfeedbackfromstudents,itis requestedandgivenattheendofthecoursewhenitcandolittlegood(especiallyin termsofgivingthestudentsasensethattheprofessorislistening). Considerthewiderworldbycontrast.Thisistheeraofconsumerism,andweare betterforit.Onmyflighthere,whenitwastimetolandandIhadtoputmycomputer away;Ipickeduptheairlinesmagazinefromthepocketinfrontofme.Thelittleinsert thatIhavehereinmyhandsays:AtContinental,werelistening.Wecareaboutwhat youthink,sogiveusacallanyhour,anyday.AtContinental,wewontjustlendyou anear.We]lpayforthecall.Wewanttoknowwhatyouthinkaboutflyingwithus. Whatarethingsyoulike?Isthereanythingyoudlikeustochange?Whateveryour thoughts,wewanttohearthem.Sogiveusacall,faxus,orsimplyfillouttheresponse cardandsenditin. Imoneofthosesillypeoplewhoactuallyreplies.Ihavecalledanairlineorhotelor carrental,sometimesjusttotellthemthatthingsareokay(Theylikethatbecause theyrenotusedtothosekindsofphonecalls.)But,invariably,Imtreatedwithrespect, evenwhencompanies.Thearticlesaid,inclosing,Ihaveabittercomplaint.Imtreated withreasonablesympathy,ifnotempathyAllwellandgood,allcommonandfamiliar tothoseofyouwhotravelandflyinairplanesorstayinhotels.Andyet,andyet... whenwasthelasttimeyourcollegeoruniversityreallymadeavigorouspushtofind outnotonlywhatstudentclientsthoughtandbelievedandfeltbutwhatinstitutional

clientsbelieveandfeelaboutyourformerstudentsnotasasideeffortbyyour alumniofficebutasamajorinitiativebydeansandprovosts? IamremindedofanunusualsucheventthathappenedintheLouisville,Kentucky schoolsystemwhentheGheensAcademywasresponsibleforprofessional developmentandreformadecadeago.Influencedbythetotalqualitymanagement (TQM)movementandadesiretoreallyprovetothecitizensofLouisvillethatthe schoolsweremoreresponsive,schoolofficialsdidsomethingextraordinaryJefferson CountySchoolsisaboutthethirtiethlargestschooldistrictinAmerica,withmorethan ahundredthousandstudents.BetweenSeptember1standOctober1stinthatyear,the schoolsystemcalledeverysinglefamilyinthedistrict,everysingleone.Andthey beganthephonecallbyasking,Howarewedoing?Well,thiswasunheardofthink aboutwhenandwhyparentsgetcallsfromschoolabouttheirkids.ItgeneratedanOut pouringnotonlyofgoodinformationbutofgoodwillthatwashardtobeat. Feedbackisneitheraluxurynorancillarytoperformance.Arecentarticleinthe NewYorkertalkedaboutthehistoryofthedevelopmentofHDTVhighdefinition television.Itwasdevelopedbyacuriousprocess,nottheusualsortofmythicwayin whichpeoplewithgoodideasandathinktankhavelightbulbsflashing,getagreat idea,andrunwithitasentrepreneurs.No,onthecontrary;saidtheauthorofthearticle. Theultimateproductwasdevelopedbyfeedback,basedonmanyconversations betweenthegovernmentandthetelevisioncompanies.Thearticlesaid,inclosingthat thisisnowthewayoftheworld.Recentstudieshaveshownthatmostmodern, computerrelatedinnovations,70to80percentoftherefinementsandthemajorfea tures,areproposedbycustomersthroughfeedbackIndeed,thedownsideofthis,and allofyouwholiveanddiewithsoftwareknowit,isthatitsbecomecommonpractice toreleasebetasoftwareforsaletogetfeedbackfrompeopleaboutwherethebugsarein ordertomakethesoftwarebetter. Somemightevensaythattheresultpollingandfocusgroups,thecoinoftherealm ofmodernpoliticians,istherealizationinthelatetwentiethcenturythatwedontreally understandallofwhatweneedtodountilwehearwhatmanydifferentpeoplethink. Now;somepeoplehaveblamedClintonandGingrichandotherpoliticiansforhaving nospine,nowill,novision,noleadership.Isay,bycontrast,Hallelujah!:Thisall soundslikedemocracytome.Findoutwhatpeoplethink.(Whichisnottocondone panderinginplaceofwisejudgment;letserronthesideofresponsivenessfora change.) Whyisit,however,thatwedontgatherfeedbackregularlyinschoolsandcolleges anduseittoimproveservice,toimproveteaching?Thatsapuzzletome.Letsthink aboutitalittlefurther.Idliketomakefoursimplepointsaboutthispuzzle: (1)Youcantlearnwithoutfeedback Thenextthreefollowfromthisfirstpoint,whichIvealreadymade,butthey apparentlyarenotselfevidenttomanyeducators.

(2) Itsnotteachingthatcauseslearning.Itstheattemptsbythelearnertoperform thatcauselearning,dependentuponthequalityofthefeedbackand opportunitiestouseit (3) Asingletestofanythingis,therefore,anincompleteassessment.Weneedto knowwhetherthestudentcanusethefeedbackfromtheresults. (4) Werewastingourtimeinventingincreasinglyarcanepsychometricsolutionsto theproblemofaccountability.Accountabilityisafunctionoffeedbackthats usefultothelearner,nottoahandfulofpeoplewhodesignthemeasures.The morearcanethemeasure,thelesslikelyitisthatitwillcauseanyuseful progress,despiteitsvalidityandreliabilityOrtosayittheotherwayaround,the moreselfevidentthefeedbacktotheperformer,themorelikelythegains. Lets think about these points a little bit further by clarifying what I mean by feedback. If I did a poll about your definition off feedback, you would probably say somethinglike,Feedbackinvolvestellingsomeonewhatyoudidanddidnotlikeor whatyoudidordidnotjudgetoberightinwhattheydidsomepraiseandsome blame.Ifyouaskpeopleabouttheirbadfeedbackexperiences,theyusuallysaythings like,Oh,Ireallygothammeredbytheperson.Theimplicationisthat,inthispro fession,westillthinkthatfeedbackiswhatyougetfrompeoplewhodoordonotlike somethingyoudid.That,ofcourse,isamistakenview.Feedbackisnotaboutpraiseor blame,approvalordisapproval.Thatswhatevaluationisplacingvalue.Feedbackis valueneutral.Itdescribeswhatyoudidanddidnotdo. WhenIwastravelingthroughBostontheotherday,IreadintheBostonGlobeabout mybelovedbutdepressingRedSox.Thearticlecontainedanexplanationfromthe pitchingcoachaboutwhytheRedSoxschiefreliefpitcher,Mr.Slocum,hadbeen recentlybanishedtothebullpen.Itseemsthepitchingcoachsawinlookingat videotape,thatHeathcliffdidnotfindhislocationspot22outof29times.Andwhen thathappens,youknowthathesnotstridingproperly.Andwhenthathappens,you lookathisdeliveryinsuchawaythatheplantedhisfootfour,five,sixinchestotheleft ofwherehenormallyplantsit,throwingtheballconsistentlyoutside.Noticethatthere isnotonenegativeorpositivevaluejudgmentinthataccountbythecoach,merelya descriptionofwhatthevideotaperevealed.Thatswhatfeedbackis.Nopraise.No blame.Itjustdescribeswhatyoudidanddidnotdointermsofyourgoal. Thebestscoringrubricsforstudentperformancedothesamething.Infact,when weworkwithpeopleonthedesignofrubrics,wealwayssay,Therubricswillbe powerfulandusefultotheextentthatyouridthemofvalueandcomparativelanguage, suchasexcellent,good,fair,poor,betterthan,worsethan,clearerthan,andlessdearthan. Substituteforallthatphraseologydiscretedescriptorsofwhatisactuallytrueofa certainlevel.So,indeed,wedounderstandtheimportanceofdescriptionintermsof rubricdesign. Ihaveaniceexamplefrommysonwhenhewasfour.Writinghisname,hesaid, Look!WhenIsawwhathewasdoing,IputonmyPiagethat.(Dontbealarmed;

theyreontomenow.Youdonthavetocallthechildabusepeople.)Isaid,Gee, Justin,thatsreallyinteresting.Whatdoesitsay?Hereplied,Itsmyname.Isaid, Showme.Hedid:Justin,Justin.ButhisNreallywasambiguous:Itcouldhave beenanH.SoIasked,Whatsthatlastletter?Hesaid,N.AfterwritingablockN andablockH,Iaskedhimtoidentifyeach,andhedid.ThenIasked,pointingtothe letterthathedwritten,Whatsthat?Nowheresthemostimportantpartofthis wholespeechforgeteverythingelse,butrememberthiswithalongpauseandthe congenitalWigginsfurrowedbrow;mysonspreciouswordswereNotwhatI wanted! Noticethathedidntsay;ImsorryyoudidntlikeitorNotwhatyouwanted. Hesaid,NotwhatIwanted!Andthatsthewayrealfeedbackworks.Ihaveanintent, Icauseaneffect,IdiscovertomydismayitwasnottheeffectIintended,andIwork veryhardtohonoritdifferently.Iownthisproblem.Whenitsrealfeedback,Iownit. Imnotangryatthepersonwhogaveit.Peoplearehungryforrealfeedbackthathelps. Whensomeonetakesthetimetocarefullylookatanddescribewhattheyhavedone, fromtheirvantagepoint,thatsagoodfeedbacksystem. Letsbumpitupfromchildrentocollege.Someofyouknowabouttwosignificant findingsintheworkofDickLightandtheHarvardAssessmentSeminar.Thechief findingfromtheHarvardAssessmentSeminaraboutthemosteffectivecoursesat Harvard,asjudgedbystudentsandalumnus,wastheimportanceofquickanddetailed feedbackStudentsoverwhelminglyreportedthatthesinglemostimportantingredient formakingacourseeffectiveisgettingrapidresponse.Studentssuggesteditshouldbe possibleinmanycoursestogetimmediatefeedbackAsecondmajorfindingisthatan overwhelmingmajorityofstudentswereconvincedthattheirbestlearningtakesplace whentheyhaveachancetosubmitanearlyversion,getdetailedfeedbackand criticism,andthenhandinafinalrevisedversion.Manystudentsobservedthattheir mostmemorablelearningcamefromcourseswheresuchopportunitieswereroutine policy. WhenIwasintheeducationdepartmentatBrown,workingwithTedSizer,Itaught onecoursepersemester.InoneofthecoursesthatItaught,IdidsomethingthatIhad alwaysdoneasahighschoolteacher,whichwastomakethefirstpaperassignment andthelastpaperassignmentofthecoursethesameassignment.IhadBrownjuniors, seniors,andMATstellmethatthisassignmentwasoneofthemostsignificanteventsin theiryearsatBrown,intermsofhelpingthemunderstandwhattheyhadandhadnot knowninitially,howmuchtheyhadandhadnotmadeprogressontheideasofthe course. Whatmyanecdotes,theHarvardexample,andmyexperiencesuggestis,infact, radicalinthecornmonsensical.Itstheideathatassessmentisnotanepisodeinwhicha testistaken,itsover,andresultsaregiven.Anassessmentmustincludethestudents abilitytousethefeedback,becausethatswhateventualautonomousperformance requires. Weveheardalotduringthepasttenorfifteenyearsinboththehighereducation andK12settingsabouttheimportanceofstudentselfassessment.Despitethe importanceoftheidea,itisamisleadingphrase.Selfassessmentisnotthegoal.Self

adjustmentisthegoal.ThatswhatmakesTigerWoodsandMichaelJordangreat. ThatswhatmakesSocratessoimpressive.Thatswhatourbeststudentsandteachers do.Theyselfadjust,withminimaleffortandoptimaleffect. Allwellandgood.Butsupposestudentshaveneverbeentaughttheimportanceof selfadjustment.Indeed,howaretheyevergoingtobetaughtitinascopeand sequencecoveragecurriculumwithaoneshottest?Regrettablywestillliveinan assessmentframeworkinheritedfromtheMiddleAges,onepredicatedonadefunct theoryoflearning.Thattheoryoflearningsays: Takeitallin,contemplateit,playwithitalittlebit,giveitback,andwellthen certifythatyouunderstand.Andifyoudontunderstand,well,youcantenterthe guild,themedievaltraditionoftheuniversity Themodemview,however,says:No,thatsnothowitworks.Itsmorelike software.Itslikebasketball.Itslikelearningtoprintyourname.Youdontreally understanditunlessyoucanadjust.UnlessyoucancopewithfeedbackUnlessyoucan innovatewithwhatyoulearn. Twoexamplesfromprofessionalacademicpracticesareillustrative.Asimple examplehappensatthiskindofforum.Howmanytimeshasithappenedtoyou?You hearsomebodygiveareallyinterestingtalk,andthenwhenthequestionandanswer periodcomes,thepersonembarrasseshimorherself.Youconcludethatthespeaker doesnotreallyunderstandthesubject.Orconsiderthepinnacleperformanceinformal education.Whydowehavethedissertationandalsoitsdefense?Wehavethedefense becausethedissertationisinsufficient.Fourhundredpagesandeighthundred footnotesarenecessarybutnotsufficientevidenceofunderstanding.Assuggested above,understandingisoftenonlyrevealedthroughdialogueinthebroadestsense: dialoguewithpeopleordialoguewithexperienceandphenomena.Assessmentmust reflectthisfact. Someveryinnovativepracticesinplacesabroaddothisnow.Oneofmyfavoritesis inGreatBritain,inscienceK12.Anumberofperformancetasksdesignedforthe nationalassessmentaskthestudenttodesignsimplescientificexperiments.For instance,oneoftheminvolveswatchingandplayingwithawinduptoy.Thequestion thenaskedis,Whatstherelationshipofwindupstolineardistance?appropriate becausethetoygoesalloverandisnotverypredictable.Thestudenthastodesigna littleexperimenttoanswerthequestion.Afteranhourorso,thestudentwritesuphis orherresults.Thentheassessorasks,Great,nowthatyouknowwhatyouknow,how wouldyouredesigntheexperiment,inlightoftheresults,inlightofthequestion,and inlightofthescientificmethod?Evaluatorsfoundaninterestingthing.Theyfound thatalmosthalfofthestudentswhohadperfectlyreasonableanswersnonetheless couldnotjustifytheirmethodsorproposedeficienciesintheirmethod. Weareonlyslowlylearningtograpplewiththephenomenonofstudent misconception,thatwonderfulbutdisturbingresearchthatbeganinphysicstwenty yearsagoatJohnsHopkins(andisnownotedeverywhere,inpartthankstoHoward GardnerswonderfulbookTheUnschooledMind,inwhichhesummarizesallthis research).Intheabsenceofinteractiveassessment,intheabsenceofassessmentthat

doesntrequirestudentstousefeedbackandrespondtoit,studentmisunderstandings canbehiddenbehindtheircorrectanswers.IntheBritishpractice,theBritish experimentersactuallygaveasmanypointstotheanswertothequestionastothe originalexperimentaldesign.andheresthesadfootnote.TheNationalAssessmentof EducationalProgress(NAEP)borrowedtheBritishtaskandsixothersandusedthemin itshandsonscienceassessmentsevenoreightyearsago.Butittookoutthelatterpart ofthetest.Itdidntscorestudentsontheirresponsetoquestions.Itonlyaskedthe studentstodesigntheexperiment. Letmegiveyouadifferentexample.TheProvinceofQuebechasbeguntopractice whatImpreachingasasystem.Thesixthtenthgradewritingassessmentforstudents ismultiday;andthestudentsbringpreviouslywrittenandevaluatedworktotheexam. Theyhavetorevisepreviouslywrittenandgradedpapers.Theyseekfeedbackduring theexam.Thatscalledcheatinginmostplaces.Andthatshowsyouhowfarweare fromunderstandingthiscommonsensicalmessage.Whatisconsideredintelligent, adaptive,andresponsivebehaviorbyperformersineveryothervenueisconsidered cheatinginschoolsandcolleges. SomeonewhohadthisdowntoasciencewasaneleventhgradeEnglishteacherin ruralOhio.Hesaidtohisstudents: Iwantyoutodopeerediting,andIwantyoutodotwothings,andtwothingsonly. Imgoingtoteachyouhowtodoitbecausetoomuchpeereditingistheblindleading theblind,pooledignorance.Hereswhatyouregoingtodo:(1)Attachtoyourdrafta statementofpurpose.WhatwasItryingtodointhispaper?Thepeereditorswill respondintermsofthatpurpose.Not,justAh,Ilikedthis,Ididntlikethat,aridthe other kindofrandomthingsthatsometimesarehelpfulandsometimes notin peer reviewBut,Okay,youweretryingtodothis,buthereshowitseemedtome.(2)Mark theplaceonthepaperwhereyoulostinterestandexplainwhy. IvetoldthisEnglishteacherstoryalotoftimesduringthepastfewyears,andthe amazingpartisthenumberofteachersandprofessorswhoaredisturbedbythisidea. Theyactuallyfinditdifficulttoimaginethattheymightconfrontstudentswiththefact thattheirwritingisuninterestingorboringinplaces.Andyet,whatsthemostcommon thingyouhearaboutstudentpapersanywhereintheworld?Thattheyreboring. ThisteacherfinishesoffthelessoninjusttherightwayUltimately,feedbackisfor empowering the performer. He doesnt set up the peer reviewers as God. On the contrary,whenstudentssubmitthefinaldraft,theymusttellhimwhichfeedbackthey tookandwhywhichfeedbacktheydidnttakeandwhyandthenattachafinalself assessment to their final paper Thats a teacher who understands todays message. Thatsateacherwhogetsextraordinaryresultsfromnotparticularlygiftedstudents. Thatsateacherwhounderstandsnotonlythatweshouldnotshyawayfromfeedback butalsothatitisessentialtohowwegetgoodatthings. Dick Light, in the Harvard Assessment Seminar report, went on to describe the parallelismforprofessorsofwhathehadaskedstudentsdents:Facultymembersat Harvard were asked what single change most improved their teaching. Two ideas swampedallothers.Oneisenhancingstudentawarenessofthebigpicture,thebig

point of it all. The second is the importance of helpful and regular feedback from students,SOaprofessorcanmakemidcoursecorrections. Probablyeverybodyinthisroomworksataninstitutionwheresomeformoffinal courseevaluationisnormaloreven,perhaps,policy.Butfinalcourseevaluationmakes thesamemistakeasfinaltesting.Itsattheend,whenitstoolatetoolateinavery importantsense.Youmightsaywell,itsnottoolate.Theprofessorisgoingtoteachthe courseagain.No,itstoolateforthestudents.Thestudentsdonthavethesatisfaction andthemoralrespectofseeingtheirfeedbackmakeadifference.Orworse,theysee thatitdoesnt. Ihadacolleague,whenItaught,whoaskedthesametwoquestionseveryFriday HehandedoutindexcardsonwhichstudentsansweredthequestionsWhatworked foryouthisweek?andWhatdidnt?Noticethelanguage.Not.Whatdidyoulike? Rather: What worked? He always was surprised. And hes a good teacher. What workedforJoedidnotworkforJill.IgetfeedbacktoalmosteverythingthatIdonowin writing,eitheronscansheetsorInwrittencomments,andImalwaysstunnedatthe enormousdisparityamongsomeofthecomments.SomepeoplethinkImthegreatest thingsinceslicedbread.OtherpeoplethinkIshouldberunoutoftownonarail.Some peoplethinkImsensitivetotheproblems,theissues,andtheaudience.Otherpeople thinkImarrogantandaloofWhatthehelldoImakeofit?Butthatsthechallenge! Thatsteaching.Thatsperformance.Thatscustomerrelations.Thatsassessment. Interestingly,ahighschoolstudentwhousedtoworkformewentofftoHarvard this year as a freshman. I asked him to collect for me examples of Harvard exams (HarvardisunusualinthatitplacesallfreshmanexamsonreserveintheFreshman LibraryInfact,examshavechangedenormouslyovertheyears.Examsfrom1901are allTrivialPursuit.Ifyouthinkwehaventmadegains,golookatsomeofthoseearly exams.)OurfriendfoundtwoexamsthathesaidheknewIdbeinterestedin,andI was.Theprofessorsgavethestudents20or15percentcreditforafullappraisalofthe coursesstrengthsandweaknessesintermsofthecriteriathattheprofessorslaidout, andforstudentrecommendationsonhowtoimprovethethingsthattheythoughtwere deficient.Yes,itsdoneattheend.Butgivingitexamcreditsendsamessage. AsIsuggestedafewminutesagointheBritishstory;andasmanyoftheother examples from the wider world indicate, the next great leap in assessment is to understand thatasolitary test, inwhichthere Isnointeraction between theperson taking the test and the assessor, will turn Out to be as foolish, dimwitted, and premodernassomeofthepracticesinvolvingrodsandcaneswereahundredyears ago.Ifwewanttoknowifstudentsunderstandsomething,wehavetoseeiftheycan deal with feedback and with counterarguments to their arguments and their own ideas,justaswedointhedissertationsdefense. Butasthestoriesalsoillustrate,itsnotnecessarilyafunctionorhumanoneonone. Feedbackbuiltintoassessmentisaboutcompellingthestudenttohaveherorhisideas Intersectwithreality;toseeifthebalsabridgewillholdtheweightpredictedbythe physicsthestudentproposesusingtoseeifthestudentdentcanconvincetheclient that this solution to the problem of environmental pollution is, in fact, feasible scientificallyeconomically,andpoliticallyIndeed,oneofthemostexcitingthingstome

aboutthesocalledproblembasedlearningintheprofessionalschoolsInmedical school, business, engineering, and the like thats now finding its way into the collegiateandprecollegiateworldisthatproblembasedlearning,byitsverynature, buildsfeedbackandtheneedtouseitintotheworkEvenifweareonlyabletosimulate it,wecannonethelessalertstudentsthattheyhavetoshowthattheycandealwith feedbackTheyhavetoshowthattheycandealwiththeunexpectedtobesaidtotruly understandandbeskilled. Indeed,ifwetakethislessontoheart,wewillcometoaverydisturbingtruththat followsfromthecommonsensicalpremisethatwebeganwith.Noneofuswhohas beenateacherisanywherenearasgoodaswecanbe,ifwearenotroutinelygetting feedback from students. Its as if Michael Jordan had to wait until some psychometrictangavehimthescorefromthegameaweeklaterItsasiftheplayercant seetheballgoin.Itsasifthepersonspeakingtohisaudiencecanttellifhesboring them and putting them to sleep. if all we get back is a psychometrically derived, indirectscore,fallwegetbackiswhatweputintoourstudents,wecar,besurethat theyrenotgettingallofwhattheycouldgetandgive.Wecanbesurethattheresso muchtounderstandaboutwhattheydontunderstand. In this regard, one of the most striking things about the Harvard Assessment Seminarreportsisthatprofessorsrelatethatthesinglemostusefuladditiontotheir teaching repertoire, arid the way to hone the two points earlier identified as most helpfulinimprovingteaching,isthesocalledOneMinuteEssay(whereoldfriendof AAHEPatCrossiscitedastheoriginator).Attheendofeachlecture,professorsask Whatwerethemostimportantpointstoday?andwithwhatquestionsdoyouleave? Whatashockitwastosomeprofessorstodiscoverthatwhatwassoclear;elegant, thorough,concise,andselfevidentmadeabsolutelynosensetoallthoseeagerbeaver Harvardundergrads.Thatisthepointoffeedback SoIleaveyouthenwithaplea.Insteadofresistingregularfeedback,embraceit.In termsofyourownassessing,stopconfusingfeedbackwithpraiseandblameandgive farmoregenuinefeedbackorcontrivesituationsthatforcethestudenttoseekand respondtosituationalfeedback.Praiseisnecessary.(Blameissometimesalso necessary.)Butpraiseonlykeepsyouinthegame.Itdoesntgetyoubetter.Feedback getsyoubetter.Andthestudentisentitledtomoreofitthanthestudentnormallygets. Inclosing,asacautionaryreminder,InotethestoryItoldsixorsevenyearsagoin thisforum,forthoseofyouwhowerehere.Mikecomesuptohistenthgradeteacherat theendoftheyearandsays,Ireallylikeyourteaching,butyoukeptwritingthisword allovermypapers,andIdontknowwhatitis.Whatwastheword?sheasks. Vagooisthereply.AsaformerEnglishteacherwhowrotethewordvaguemany times,onmanypapers,Ididntfindtheincidentasfunnyastheaudience.Itwas,in fact,painful.Andthat,ofcourse,isthepoint.Whatseemslikeselfevidentfeedback fromourpointofviewisntnecessarilysotothelearner.Andthereinliesthepathosof teaching. Nowsomewouldsaythatwhenstudentsbecomeundergraduatesorgraduate students,itstheirproblem.Letthemfigureoutwhatswrongwiththeirwork,Such responsesshowmetatwerenotthereyet;werenotyetunderstandinghowlearning

aridassessmentwork.TheyshowmethatContinentalAirlinesstillhasmuchtoteach usonthissubject.Weneedthefeedbackof(even)thenovicetoachieveexpertise. SoIleaveyou,ten,withthisidea.Feedbackisnotpraiseorblame.Itswhatyoudid anddidnotdo,whetheryourealizeditorintendedit.Assessmentshouldmakeitschief businesstheconfrontingofperformerswiththeeffectoftheirwork,including performerscalledteachers.Andthenperformersmustdosomethingabouttheeffect, eithertoexplainit,tojustit,ortocorrectit. Iawaityourfeedback.Thankyou.