Assessment and Evaluation Training Module

SECTION1:WhyDoWeDoAssessment? Wedoassessment: Tomeasuregoalaccomplishment Toidentifyprogramstrengths Tocelebratesuccesses Toidentifyimpactsonprogramparticipants Toidentifyimpactsoncommunity Toidentifyunmetneeds Toinformplanninganddecisionmaking Toidentifyopportunitiesforimprovement Tosharelessonslearned Toreporttofunders Totellourstoriestostakeholders WhatisAssessment? Assessmentis: Assessmentistheprocessofdocumentingandcollectingdatausuallyinmeasurabletermsabout information,knowledge,skills,attitudesandbeliefs. Assessmentscanbeclassifiedinmanydifferentways.Inprogramdevelopmentandmanagement, assessmenttypesthatyoumayseeare:(1)formativeandsummative;(2)objectiveandsubjective;(3) referencing(criterionreferenced,normreferenced,andipsative);and(4)informalandformal. Datapointsanitemofspecific,factualinformationbeingcollectedfrommeasurementorresearch. QuantitativeData:Numericaldatacollectionderivedfromcountinglikedatapoints.Forexample countingthenumberofprogramparticipantssuchasnumberofmentors,numberofmentees,number ofmenteesinmiddleschool,numberofnonwhite,menteesinmiddleschool,numberofmentees receivingfreeorreducedschoollunch.Numberoftrainingsortraininghoursprovidedtoeachmentor. QualitativeData:Nonnumericdatacollectionderivedfromdescriptiveinformation,suchasnarrative, text,audioorvideosources.Thesedataoftenarecollectedinopenendedquestions,feedbacksurveys, observations,orsummaryreports.Theyaremoredifficulttocompare,reproduce,andgeneralize becauseoftheunique,nonduplicativenatureoftheinformation. WhoCaresaboutOurAssessment? WashingtonCampusCompactworkstoassessbothquantitativeandqualitativedatapointsto communicatethesignificanceofourprogramsto: CampuspresidentsandpartnerswhoWashingtonCampusCompactrepresents 2

Yourcommunitypartnersnonprofit,governmental,andprivateentitiesthatinvesttheirtime inyourcampusprogramorpartnership. NationalCampusCompactandotherstateCampusCompacts NationalandStateMentoringPartnerships TheCorporationforNationalService TheWashingtonCommissionforCommunityandNationalService Currentandfutureprivateandcorporatedonors Practitionersinthefieldofcollegecampusstudentengagement. WashingtonCampusCompactstaff You!

Attimeswecollectinformationoncertaindatapointsto: Satisfyreportingrequirementsforgrantorsthoseentitiesthatdonatemoneyorother resourcestosupportyourcollegecampusintheirworkthroughWashingtonCampusCompact. Contributedatatolargerentitiesinthefieldsofcollegecampusstudentengagement,civic engagement,volunteerism,education,atriskyouthinterventionandprevention. Haveinformationtoanalyzeinbeingabletounderstandthedynamics,issues,andpatternsthat existinourcollectivework. Determineifwecanappropriatelyapplyforfundsorresourcesfromotherfundingsourcesnow orinthefuture,ifweworktoincreaseourimpactincertainareas. SECTION2:IfThatsAssessmentWhatisEvaluation? Evaluationis: Analyzingtheinformationcollectedforpatterns,commonalities,abnormalities,orothercharacteristics thatcanhelpusmakesenseoutoftheassessmentdata. Throughevaluationwecanclassify,generalize,drawconclusion,identifyqualityandeffectivebest practices,andunderstandbarriersandchallengesthatexistinprogramdelivery. Assessmentistheworkittakestocollecttheinformation Evaluationistheworkittakestounderstandthevalueoftheinformationcollected. PleaseBeOurPartnersin Assessment&Evaluation! ForWashingtonCampusCompacttoeffectivelyassessandevaluatetheworkofourCampusCompact programsandnetwork,itiscriticalthatwecan: Workcollaborativelywithyoutogatherdatathathasvaluetoallthestakeholdersinvolved. Relyonyoutocollectandreportaccuratedata. Dependonyoutosubmitinformationinatimelymanner. 3

Haveopen,authenticdialogabouthowtoimproveourassessmentpractices. Partnerwithyoutohelpinterpretthedata Expectthatyouwillhelpcommunicatesignificantfindingsaboutyourindividualprogramand ourcollectiveprogramimpacts. SECTION3:WhyDoWeDoAssessment? APreflectionActivity: ThinkaboutatimeduringyourexperienceasaRetentionProjectMemberwhenyoueither(a) useddatatomakeadecisionOR(b)wishedyouhaddatatomakeadecision. Describethesituation: Whatdatadidyouuse?ORWhatkindsofdatawouldhavebeenhelpful? Whatinsightsdidthedataprovide?ORWhatinsightscouldhavethedataprovided? Whatwastheoutcome? Howcouldhavetheoutcomebeendifferentifyouhadmore/betterdata?

AssessmentandEvaluationModule CurriculumInformation LearningObjectives Thetrainingparticipantswill: Understandtheimportanceofintegratingassessmentandevaluationintotheirmentoring modelandotherservicedeliverymodels. Beabletoaccesstherequiredandoptionalassessmentandevaluationtoolsprovidedby WashingtonCampusCompact. Knowhowtocollectrequiredassessmentdataandhowtosubmitcompleted,required assessmentstoWashingtonCampusCompact. Reflectontheintegrationofassessmentandevaluationwithintheircurrentprogrammodel. 4

CurriculumOutline Participantsareguidedtoworkthroughthecurriculumandmaterialsinthisorder: 1. CompletethePreflectionActivity:WhyDoWeDoAssessment? 2. ReadHandout:WhyDoWeDoAssessment? 3. ReviewtheRequiredandOptionalAssessmentToolsprovidedbyWashingtonCampusCompact. 4. ReadHandout:WhatarePerformanceMeasures 5. CompletethePraxisActivity:PerformanceMeasuresPraxisActivityMakingtheMeasures MeaningfulwithinYOURModel. 6. Completeassessment/evaluation SECTION4:WhatisaPerformanceMeasure? APerformanceMeasureisameasurablegoalsetattheoutsetofaneffortorprogramthatsetsa realistic,achievableexpectationforwhatsuccesswilllooklikeattheendoftheendeavor. Ameasurablegoalmeansthatanumericalorquantifiablevaluecanbeassignedasabenchmarkfor success. Examples: Apersonwishestolose10pounds. Alongdistancerunnerwishestoincreaseenduranceby10%eachyear. Amotherwishestodecreasethenumberoftimesherchildrenargueonaweeklybasis. Aprogramwishestorecruit100mentors Tomeasureagoal,itisimportanttohaveabaselinemeasurement.Abaselinemeanstoknowhowyou measureupinthebeginningsothatyouhaveawaytocomparechangefromthebeginningtoend. Examples: Apersoncurrentlyweighs200pounds. Alongdistancerunnercurrentlycanrunupto30milesdaily. Amotherschildrencurrentlyargue21timesonaweeklybasis. Aprogramcurrentlyhas55mentors. TheRetentionProjects2010PerformanceMeasures: PerformanceMeasure#1:Youth/MenteeDevelopment 1,500atrisk/disadvantagedmiddleandhighschoolstudentswillreceiveacademicsupportand encouragementtoattendpostsecondaryeducation,and/orparticipateinmeaningfulservice activities. 5

80%ofteachers/administratorswhorespondtoasurveywillreportthatatrisk/disadvantaged middleandhighschoolstudentmenteesdisplayedimprovedacademicachievementandstayed inschooltheentireacademicyear. 80%ofatrisk/disadvantagedmiddleandhighschoolstudentmenteeswhorespondtoasurvey willreportimprovedattitudestowardacademicengagementandadvancement.Specific indicatorsinclude(a)increasedinterestindoingwellinclasses,(b)increasedinterestingoingto college,(c)increasedresourcestohelpfindcollegeofinterest,(d)increasedpreparationto applyforcollege,and(e)improvedoutlookaboutfutureeducation.

PerformanceMeasure#2:CollegeStudentVolunteerCivicDevelopment 1,200collegestudentvolunteers/servicelearnerswillberecruited,trained,andsupported toparticipateinmeaningfulserviceactivities. 80%ofcollegestudentvolunteers/servicelearnerswhorespondtoasurveywillreport improvedcivicknowledge,attitudes,skills,andabilitiesattheendoftheirserviceexperiences. Specificindicatorsinclude(a)increasedawarenessofcriticalcommunityneeds,issues, strengths,andresources;(b)improvedabilitytothinkcriticallyaboutcommunityneedsand issues;(c)increasedskillsandabilitiesnecessarytomeetcommunityneedsandissues;(d) increasedawarenessofpersonalbiasesandprejudices;and(e)increasedclarificationofown beliefsaboutvariouscommunityissues. 75%ofcollegestudentvolunteers/servicelearnerswhorespondtoasurveywillreportan increasedcapacitytoinitiatepositivechangeforthecommongood.Specificindicatorsinclude (a)improvedunderstandingofhowtouseownknowledge,skills,andabilitiestoimprovethe community;(b)improvedcapacitytocreatepositivecommunitychange;(c)improvedcapacity tobecomeacommunityleader;and(d)increasedlikelihoodtoinitiatechangeforthecommon good.

PerformanceMeasure#3:CollegeStudentVolunteerWorkforceSkillDevelopment 1,200collegestudentvolunteers/servicelearnerswillberecruited,trained,andsupported toparticipateinmeaningfulserviceactivities. 80%ofcollegestudentvolunteers/servicelearnerswhorespondtoasurveywillreport improvedworkforceknowledge,attitudes,skills,andabilitiesattheendoftheirservice experiences.Specificindicatorsinclude(a)improvedabilitytowriteand/orspeakeffectively; influenceothersthroughwriting,speaking,and/orartisticexpression;andcommunicateideasin arealworldcontext;(b)increasedabilitytoestablishand/ormaintainmutuallyrewarding relationshipswithcolleagues;workcooperatively/collaborativelywithothers;listentoand considerothers'pointsofview;andaskforfeedbackfromothers;(c)increasedinvolvement withpeopledifferentfromoneself;(d)increasedabilitytocontributetotheachievementof groupgoals;andimprovedunderstandingofgroupdynamics;(e)improvedleadershipskills, selfreliantbehaviors,andabilitytomanagetimeeffectively;and(f)increasedknowledgeof variouscareersandjobs;clarityofowncareer/professionalgoals;andunderstandingof preferredworkenvironment. 75%ofcollegestudentvolunteers/servicelearnerswhorespondtoasurveywillreportan increasedinterestinpursuingcareersforthecommongood. 6

PerformanceMeasure#4:MemberCivicDevelopment 60nationalservicememberswillberecruited,trained,andsupportedtoparticipateinintensive servicetoimproveacademicachievementofdisadvantaged/atriskyouth. 80%ofnationalservicememberswhorespondtoasurveywillreportimprovedcivicknowledge, attitudes,skills,andabilitiesattheendoftheirserviceexperiences.Specificindicatorsinclude (a)increasedawarenessofcriticalcommunityneeds,issues,strengths,andresources;(b) improvedabilitytothinkcriticallyaboutcommunityneedsandissues;(c)increasedskillsand abilitiesnecessarytomeetcommunityneedsandissues;(d)increasedawarenessofpersonal biasesandprejudices;and(e)increasedclarificationofownbeliefsaboutvariouscommunity issues. 75%ofnationalservicememberswhorespondtoasurveywillreportanincreasedcapacityto initiatepositivechangeforthecommongood.Specificindicatorsinclude(a)improved understandingofhowtouseownknowledge,skills,andabilitiestoimprovethecommunity;(b) improvedcapacitytocreatepositivecommunitychange;(c)improvedcapacitytobecomea communityleader;and(d)increasedlikelihoodtoinitiatechangeforthecommongood. SECTION5:PerformanceMeasuresPraxisActivity MakingtheMeasuresMeaningfulwithinYOURModel. Howwouldyourqualitativelyorquantitativelystateyourprogramscurrentbaselineforeachofthe RetentionProjectscurrentyearperformancemeasures? Areeachoftheseperformancemeasuresrelevanttoyourprogrammodel? Yes IfYes:Listeachbynumberandbrieflynotetheirconnectiontoyourprogram. IfNo:Listeachbynumberandbrieflynotewhatthereasonisthattheyareirrelevant. IfIdontKnow:Scheduleameetingtotalktoyoursupervisoraboutthisissue. 7 No Idontknow

WhatwouldCampusCompactneedtoknowaboutyourprogramtomakethePerformanceMeasures morerelevanttoyourprogram? Calculateorarticulatemeasureablegoalsthatwouldapplytoyourprogramtohelpachievethese RetentionProjectperformancemeasures. ForExample: Includeatleasta10minutesegmentabouthowmentoringcanpositivelyimpactthementors understandingofdiversepopulationsat100%ofthetrainingsweprovide. Revisethecommunitypartnermemorandumofunderstandingthatgoestoall9schoolpartners toincludetherequirementtocompletetheyearendsurvey. #1:Youth/MenteeDevelopment Howmanyatrisk/disadvantagedmiddleandhighschoolstudentswillreceiveacademicsupport andencouragementtoattendpostsecondaryeducation,and/orparticipateinmeaningful serviceactivities?

Howinmeasurabletermswillyourprogramcontributetotheabilityofteachers/administrators whorespondtoasurveywillbeabletoreportthatatrisk/disadvantagedmiddleandhigh schoolstudentmenteesdisplayedimprovedacademicachievementandstayedinschoolthe entireacademicyear?

Howinmeasurabletermswillyourprogramcontributetoensuringthatatrisk/disadvantaged middleandhighschoolstudentmenteeswhorespondtoasurveywillreportimprovedattitudes towardacademicengagementandadvancement?

#2:CollegeStudentVolunteerCivicDevelopment Howmanycollegestudentvolunteers/servicelearnerswillberecruited,trained,andsupported toparticipateinmeaningfulserviceactivitiesatyoursite?

Howinmeasurabletermswillyourprogramcontributetoensurethatcollegestudent volunteers/servicelearnerswhorespondtoasurveywillreportimprovedcivicknowledge, attitudes,skills,andabilitiesattheendoftheirserviceexperiences?

 Howinmeasurabletermswillyourprogramcontributetoensurecollegestudent volunteers/servicelearnerswhorespondtoasurveywillreportanincreasedcapacitytoinitiate positivechangeforthecommongood?

#3:CollegeStudentVolunteerWorkforceSkillDevelopment Howinmeasurabletermswillyourprogramcontributetoensurecollegestudent volunteers/servicelearnerswhorespondtoasurveywillreportimprovedworkforce knowledge,attitudes,skills,andabilitiesattheendoftheirserviceexperiences?

Howinmeasurabletermswillyourprogramcontributetoensurecollegestudent volunteers/servicelearnerswhorespondtoasurveywillreportanincreasedinterestin pursuingcareersforthecommongood?

#4:MemberCivicDevelopment Howinmeasurabletermswillyourprogramensurethatthenationalservicememberrecruited, trained,andsupportedtoparticipateinintensiveservicewillimproveacademicachievementof disadvantaged/atriskyouth?

Howinmeasurabletermswillyourprogramensurethatthenationalservicemember(s)who respondtoasurveywillreportimprovedcivicknowledge,attitudes,skills,andabilitiesatthe endoftheirserviceexperiences?

Howinmeasurabletermswillyourprogramensurethatthenationalservicemember(s)who respondtoasurveywillreportanincreasedcapacitytoinitiatepositivechangeforthecommon good?