DesignDocumentfor

CreatingEffectiveDigitalAssessmentProducts
ByJenniferFett
PurposeoftheCourse ThepurposeofthetrainingistoultimatelyimprovecustomersatisfactionbyincreasingtheabilityofHumanitiesStudio
LearningDesignerstodevelophighqualitydigitalassessmentsandusefulmetadata(e.g.,rubrics,rationales,textcomplexity
information,etc.)usingthecurrentdigitaltoolset(MasonCMS>Learnosity>MyMHP).
AudienceDescription

ThetargetaudienceforthetrainingistheLearningDesigners(includingAssociateLDsandSeniorLDs)oftheMason
HousePublishingHumanitiesStudiowhoareeithercurrentlyassignedtoormaybeassignedtoanyEnglishlanguagearts
programforthecomingyear.TheseLDswillbeidentifiedbytheLeadLearningDesignersandDirectorsofthestudio,and
willbeconfirmedbytheVPofthestudio.
GeneralLearnerCharacteristics
Ages2865
65%female;35%male
Education:allholdatleastBA;manyholdgraduatedegree(s)
Workexperience:3monthsto20+years
Allareconscientiouspeopleconcernedwithhelpingstudentsandteachersfacilitatethelearningprocess
EntryCharacteristics
Prerequisiteskillandknowledge:allarefamiliarwithtraditionalassessments,mostareawareofchangeinmarkettrends
inassessments,fewarefamiliarwithcurrentmarketrequirementsandstandards,fewarefamiliarwithcurrentMHP
assessmentplatformsandbestpractices
Attitudinalandmotivationalcharacteristics:LDswithpreviousassessmentexperiencemaynotfeelasthoughtheyhave
muchtolearn,manymayberesentfulofhavingtotaketimeawayfrombusyschedulesfortrainingwithlittleperceived
immediatevalue(forthosenotassignedtoassessmentprojects),somelearnersmaybemotivatedtotakethetraining
becausetheyfeelitwillhelpthemunderstandtheirownK12kidstestpreparationneeds
Priorexperience:somehaveusedlegacytoolstocreateassessmentcontent,veryfewhaveusedthecurrenttoolset

321753006.doc

6/20/2016

MajorCourse
Objectives(Terminal)

[PhaseI]
I.
II.

Explainhowcreationofassessmentpracticeandpreparationproductsfornextgenerationassessmentisalignedwith
companyvaluesandmission(understandcognitive)
DescribethepotentialvalueoftheCommonCoreStateStandardsinitiativetotheU.S.educationsystem
(understandcognitive)

III.

Elucidatethekeydifferencesbetweentraditionalassessmentsandnextgenerationassessments(understand
cognitive)

IV.

ApplyknowledgeofrigorandcontentrequirementsofnextgenerationassessmentsasdefinedbytheMHPitem
writingcriteriabysuccessfullycompletingtwoitemwritingmodules(applycognitive)

[PhaseII]
V.
VI.

Usingthepreparedtestdevelopmentblueprint,createabudgetestimateforFinancebasedoninformationfromTPL,
AVL,andVRM(ifapplicable)(createcognitive)

VII.

Formulatehighlevelplanforsourcingofpassages/media,includingrepurposingoflegacycontentwherepossible
(evaluatecognitive)

VIII.

Writethoroughandeffectivetestdevelopmentspecificationdocument(s)forvendor/internalcontentdevelopment
partner(createcognitive)

IX.

321753006.doc

Usingrelevantstateorconsortiatestdevelopmentspecifications,createatestdevelopmentblueprint(create
cognitive)

CollaboratewithcrossfunctionalteamstofacilitateworkflowstepsfromMasonCMS>Learnosity>MyMHP(apply
cognitiveandaffective)

6/20/2016

CourseEnabling
Objectives

[PhaseI]
I.

II.

III.

Explainhowcreationofassessmentpracticeandpreparationproductsfornextgenerationassessmentisalignedwith
companyvaluesandmission(understandcognitive)
a. Giveanexampleofhowperformingbelowexpectationsonnextgenerationassessmentscannegatively
impactstudentsand/orteachers.(understandcognitive)
b. ExtendtheimportanceofhelpingstudentsandteacherssucceedinnextgenerationassessmentstoMason
Houseslargermissionstatement.(understandcognitive)
DescribethepotentialvalueoftheCommonCoreStateStandardsinitiativetotheU.S.educationsystem
(understandcognitive)
a. DelineatethereasonsbehindtheCommonCoreStateStandardsinitiative(understandcognitive)
b. DescribethekeyfeaturesoftheCommonCoreStateStandardsforEnglishLanguageArts(understand
cognitive)
c. ExplainhowtheCommonCoreStateStandardshavecreatedaneedforanextgenerationassessment
system(understandcognitive)
Elucidatethekeydifferencesbetweentraditionalassessmentsandnextgenerationassessments(understand
cognitive)
a. Articulatetheuniquefeaturesofthenextgenerationassessmentsprovidedbythetwoconsortia(SBACand
PARCC)(understandcognitive)
b. Summarizethemaintextcomplexitydemandsfornextgenerationassessments(understandcognitive)
c. Explainhoweachofthethreemainitemtypesuniquetonextgenerationassessmentshelpstoachievethe
levelofrigorprescribedbytheCommonCoreStateStandardsforEnglishLanguageArts(understand
cognitive)

IV.

ApplyknowledgeofrigorandcontentrequirementsofnextgenerationassessmentsasdefinedbytheMHPitem
writingcriteriabysuccessfullycompletingtwoitemwritingmodules(applycognitive)
a. SpecifytheMHPcriteriaforitemwriting(understandcognitive)
b. Selectexampleevidencebasedselectedresponse(EBSR)itemsthatmeetthecriteria(evaluatecognitive)
c. Revisethreeexampletechnologyenhanceditems(TEI)sothattheymeetthecriteria(applycognitive)
d. Createasampleperformancetaskscenarioforeachofthethreetypes(narrativewriting,research
simulation,literaryanalysis)(createcognitive)
e. Enumeratethedifferentpiecesofmetadatarequiredforeachitemtype(understandcognitive)
f. Correlateexampleitemstothefollowingpiecesofmetadata:standardanddepthofknowledge(DOK)
g. CreateonesampleitemforeachofthefollowingtypesinLearnosity:EBSR,classification,association,
orderlist,andtextselect(createcognitive)
[Phase2]
321753006.doc

6/20/2016

321753006.doc

V.

Usingrelevantstateorconsortiatestdevelopmentspecifications,createatestdevelopmentblueprint(create
cognitive)
a. Givenanexampleprojectoutline,locateonlinethespecificationdocumentsneededtocreatetheblueprint
(understandcognitive)
b. DesignablueprintinExcelthatprovidesallofthenecessaryinformationneededtoestimatethebudgetfor
theexampleproject(createcognitive)
c. Foreachpieceofinformationneededtoestimatethebudget,identifytheresourceorgrouptoconsultto
obtaintheactualestimate(understandcognitive)

VI.

Usingthepreparedtestdevelopmentblueprint,createabudgetestimateforFinancebasedoninformationfromTPL,
AVL,andVRM(ifapplicable)(createcognitive)
a. Formulatehighlevelplanforsourcingofpassages/media,includingrepurposingoflegacycontentwhere
possible(createcognitive)
b. Alignidentifiedcontentdevelopmenttaskstohistoricalrateschargedbypreferredcontentdevelopment
vendor(createcognitive)

VII.

Writethoroughandeffectivetestdevelopmentspecificationdocument(s)forvendor/internalcontentdevelopment
partner(createcognitive)
a. Summarizeallofthecurrentcontentconsiderationsasspecifiedbytherelevantstatedepartmentof
educationand/ortestdevelopmentconsortium(evaluatecognitive)
b. RelateanyadditionalcontentconsiderationsspecifiedbyinternalMasonHousePublishingteams.(evaluate
cognitive)
c. Specifypreferredprocessesforobtainingand/oraligningmetadatatostimulianditems(evaluate
cognitive)

IX.

CollaboratewithcrossfunctionalteamstofacilitateworkflowstepsfromMasonCMS>Learnosity>MyMHP(apply
cognitiveandaffective)
a. CollaboratewithProjectManagement,DevelopmentSolutions,andTPStoagreeuponworkflowstepsand
createrealisticmilestoneschedule(applycognitiveandaffective)
b. CollaboratewithDevelopmentSolutionsgrouptooutlineandconstructtheassessmentprojectstructurein
MasonCMS(applycognitive)
c. CreateassignmentspergradelevelinMasonCMSforinternalorexternalcontentdevelopmentvendors
(applycognitive)
d. Explaintovendorpartnerstheprocessofmanuscript>MasonCMSreviewstepssothattheyunderstand
eachsteprequiredofthem(understandcognitive)
e. ManagecontentqualityinMasonCMSbyreviewingandprovidingclearandtimelyfeedbackper
passage/itemsettovendor(evaluatecognitiveandaffective)
4

6/20/2016

f.

CollaboratewithLearnosityProductOwnertodetermineandrequestanyneededenhancementsforthe
project(analyzecognitiveandaffective)
g. CollaboratewithDevelopmentSolutionstocreateactivitytemplatesinLearnositythatreflectagreedupon
featuresanduserexperience(applycognitive)
h. PerformcontentqualitychecksinLearnositysothatthefinalproductis95%errorfree(analyzecognitive)
i. LiasewithMyMHPteamtoplanforintegrationofassessmentactivitiesfromLearnosityintoMyMHP
assessmentdashboard(applycognitiveandaffective)
j. WorktoensurequalityofenduserexperienceofMyMHPassessmentcontentbycreatingbugsinJIRAfor
anyexistingissuesthatfallwithinthedomainofLearningDesignerure(evaluatecognitive)
k. WorkwithcrossfunctionalteamtoensurequalityofenduserexperienceofMyMHPassessmentcontentby
respondingtoallassignedbugsinJIRA(analyzecognitive)

321753006.doc

6/20/2016

RLOEnabling
Objective

Articulatetheuniquefeaturesofthenextgenerationassessmentsprovidedbythetwoconsortia(SBACandPARCC)

LearningAssessment Theassessmentforthecoursewillbeadministeredthroughquizzes,knowledgechecksinthetwoelearningmodules,anda
finalsummativetest,allofwhichwillbescoredusingthefollowingratingsystem:
forCourse
3needsimprovement
2meetsexpectations
1exceedsexpectations
Althoughmuchofthescoringintheelearningmoduleswillbeautomated,boththeelearningmodulesandthesummative
testwillrequiremanualscoringbytheLLAsandcourseadministrator.
Pleasenote:onlythePhaseIobjectiveswillbeassessedduringthetraining.ThePhaseIIobjectiveswillbeassessedduring
theyearlyperformancereviewprocessbyLLAsandmanagers.
LearningAssessment Therewillbe2knowledgechecksintheRLO,consistingofselectone,selectmultiple,anddraganddropitems
forRLO
InstructionalDelivery Thoughtheprimaryinstructionalstrategyemployedinthetrainingwillbeonsite/webbaseddirectinstructionintheformof
presentationsandelearningmodules,therewillalsobeafairamountofindirectinstructionintheformoffrequentQ&A
methodforCourse
sessionsandsmallgroupdiscussions.
(overall)
InstructionalStrategy Theprimaryinstructionalstrategyisdirectinstructionwithtwoknowledgechecks
forRLO
Media
508Accommodations

321753006.doc

TheRLOwillconsistoftext,audionarration,graphicorganizers(charts,tables),andgraphics

Atranscriptwillbeprovidedforallaudionarration

508alttextwillbeprovidedforallimages

Userswillbeabletocontrolvolume,pacing(byskippingaheadorrevisitingscreens),andsizeoftext

6/20/2016

CourseStructure
Description

SeeAppendixA

SeatTimeofCourse

17.520hoursover5days

SeatTimeofRLO

2030mins.

321753006.doc

6/20/2016

RLOOutline

I.
II.
III.
IV.

V.
VI.

VII.
VIII.
IX.
RLOFlowchart

Title
Objectives/Instructions
Whatarethetwoassessmentconsortia?
a. SBAC
b. PARCC
WhatfeaturesarecommontobothSBACandPARCCassessments?
a. Itemtypes
1.
EvidencebasedSelectedResponse(EBSR)
2.
Technologyenhanced(TEI)
3.
Performancetasks
b. Detailedcriteriaforstimuli
1. Authenticinformationaltexts
2. Thematicallylinkedstimuli
3. Multimediastimuli
KnowledgeCheck1
Whatfeaturesareuniquetooneconsortiumonly?
a. SBAC
1. Itemsareadaptive
2. Includesshortconstructedresponseitemsthroughout
3. Separategrammar,usage,andmechanicssection
4. Separatespeakingandlisteningsection
5. Performancetasksareseparateassessments
6. Performancetasksincludeargumentative,explanatory,andnarrative
b. PARCC
1. Fixedformonly
2. Onlyconstructedresponseitemisperformancetaskfullwrite
3. Noitemstomeasureeditingorgrammar/mechanicsskills
4. Performancetasksareintegratedintosummativeassessments
5. Performancetasksincluderesearchsimulation,literaryanalysis,andnarrative
KnowledgeCheck2
Summary/Review
SummativeAssessment

http://www.gliffy.com/go/publish/10655155

Screens/PagesinRLO 1720

321753006.doc

6/20/2016

KnowledgeChecksor
OtherAssessmentsor
PracticesforRLO

3Dichotomous(T/F,Y/N,etc.)
1MultipleChoice

3MultipleSelect

2DragandDrop

____Customdescribe;ifappropriate,supplyflowchartinanAppendixandreferenceithere.
____Otherdescribe

Rollovers/clickevents

3Rollovers

19ClickEvents

RLONavigation

Themainnavigationpanelwillincludethefollowingbuttons/icons:

321753006.doc

Mainmenu
Help
Pause
Play
Volume(up/downslider)
Transcript
Previous
Next
Progressbar
Scrollbar

6/20/2016

ScreenLayoutsfor
RLO

321753006.doc

10

6/20/2016

DevelopmentTools
forRLO

Storyboardingtool:BalsamiqMarkups3,PowerPoint2011
Graphicstool:AdobeIllustrator15.1.0
Primaryauthoringtool:AdobeCaptivate9

Ownership

ThetrainingisbeingdevelopedforMasonHousePublishing,aleadingproviderofK12educationalproductsandservices.
JenniferFettwilldeveloptheinitialcourse;theCDGALearningDesignerswillmaintainthecoursewithJenniferasa
consultant.

DevelopmentTimeof Entirecourse:68weeks
entirecourseand
RLO:4050hours
RLO
Supportrequirements SupportfromVPandLLAintheformof30minutepresentationalignedwithlearningobjectives
forRLOandcourse
Onsitefacilitiessupportforprojectorsetup(mainconferenceroom)andITsupport(traininglab)
ITsupportneededforRLOanditemwritingmoduleintegrationandadministrationincompanytrainingsite(Content
DevelopmentGroupAcademy)

321753006.doc

11

6/20/2016

Project
Signoff[optional]

Pleasesignbelowindicatingagreementwiththeproposedcourseplanandapprovingstartupofthestoryboardand
developmentphases.

InstructionalDesigner
Date

ProjectManager/Sponsor
Date

321753006.doc

12

6/20/2016

AppendixA:CourseStructureOutline
Note:Aprecoursesurvey/questionnairewillbedistributedpriortothefirstafternoonof
thetraining,askinglearnerstoratespecificstatementsabouttheirjobsandvalues(to
increasemotivation),toratestatementsabouttheirlevelofapprehensioninlearningnew
technologicaltools(togaugethelevelofscaffoldingneededinthemodulesthatdealwith
thenewtoolset),andtoanswerquestionsaboutpriorexperiencescreatingassessment
contentandproducts(tobetterassesslearnersspecificentrycharacteristics).

Module1(Day1):
[Objectivesdisplayedonprojectorscreenatbeginningofmodule]
1.Videoshowingimpactoflowperformancesonnextgenassessmentsshowshow
studentsandteachersneedhelppreparingforthesehighstakestests
2.Presentationofquestionnaireresultsshowinggroupattitudesandmotivation
3.BriefQ&A
[Break]
4.PresentationbyVPandLLAsabouttheirvisionforhowthecontentdevelopment
groupwillusethetrainingandtoolsettodesignandcreatethebestproducts
5.BriefQ&A
6.Presentationofsurveyresultsshowingcollectivegroupexperienceincreating
assessmentsandcommonpainpoints
7.Smallgroupdiscussionwithworksheet
[Break]
Module1Assessment
8.(Homework)Presentationw/additionalreferencereadingsandvideo(tobe
viewed/readpriortotheseconddaystraining)

Module2(Day2):

[Objectivesdisplayedonprojectorscreenatbeginningofmodule]
1.Q&Aabouthomeworkpresentationandresources
2.SmallgroupactivityinwhichparticipantsbrainstormwaystheCCSShave
impacted/mayimpactcontentdevelopmentatMHP;responsesareenteredintoaGoogle
doc
3.Break(duringwhichfacilitatororganizesGoogledocwithsmallgroupresponsesto
brainstorming)
4.Discussionofbrainstormingresultsandomissionsinwhichfacilitatorgivesrecent
examplesofwaystheCCSShasimpactedcontentdevelopmentoverthepast
developmentcycle
5.Q&A
[Break]
6.RLOonenablingobjectivea)Articulatetheuniquefeaturesofthenextgeneration
assessmentsprovidedbythetwoconsortia.(intraininglab)
7.Presentationontextcomplexitydemandsandimpactonpassagesourcing

321753006.doc

13

6/20/2016

8.Q&A
9.Presentationofthreemainitemtypesindetailfollowedbysmallgroupdiscussion
[Break]
10.Module2assessment

Module3(Day3)

[Objectivesdisplayedonprojectorscreenatbeginningofmodule]
1.RLOforenablingobjectivea)SpecifytheMHPcriteriaforitemwriting.(intraining
lab)
2.Threepartcomputerbasedassessmentcoveringthefollowingobjectives:
b.Selectexampleevidencebasedselectedresponse(EBSR)itemsthatmeetthe
criteria.
c.Revisethreeexampletechnologyenhanceditems(TEI)sothattheymeetthe
criteria.
d.Createasampleperformancetaskscenarioforeachofthethreetypes(narrative
writing,researchsimulation,literaryanalysis)

Module4(Day4)

[Objectivesdisplayedonprojectorscreenatbeginningofmodule]
1.RLOforenablingobjectivee)Enumeratethedifferentpiecesofmetadatarequiredfor
eachitemtype(intraininglab)
2.Twopartcomputerbasedassessmentcoveringthefollowingobjectives:
f.Correlateexampleitemstothefollowingpiecesofmetadata:standardand
depthofknowledge(DOK)
g.CreateonesampleitemforeachofthefollowingtypesinLearnosity:EBSR,
classification,association,orderlist,andtextselect

Module5(Day5)
CourseAssessment

321753006.doc

14

6/20/2016