LikertScales

arethemeaningoflife:

DaneBertram

Note:Aglossaryisincludedneartheendofthishandoutdefiningmanyofthetermsusedthroughoutthisreport.

CPSC681TopicReport

DaneBertram

LikertScale\lickurt\,n.

Definition:

Apsychometricresponsescaleprimarilyusedinquestionnairestoobtainparticipants preferencesordegreeofagreementwithastatementorsetofstatements.Likertscalesarea noncomparativescalingtechniqueandareunidimensional(onlymeasureasingletrait)in nature.Respondentsareaskedtoindicatetheirlevelofagreementwithagivenstatementby wayofanordinalscale.

Variations: Mostcommonlyseenasa5pointscalerangingfromStronglyDisagreeononeendto
StronglyAgreeontheotherwithNeitherAgreenorDisagreeinthemiddle;however,some practitionersadvocatetheuseof7and9pointscaleswhichaddadditionalgranularity. Sometimesa4point(orotherevennumbered)scaleisusedtoproduceanipsative(forced choice)measurewherenoindifferentoptionisavailable.Eachlevelonthescaleisassigneda numericvalueorcoding,usuallystartingat1andincrementedbyoneforeachlevel.For example:

Figure1.SamplescaleusedinLikertscalequestions

Origin:

NamedafterDr.RensisLikert,asociologistattheUniversityof Michigan,whodevelopedthetechnique.Hisoriginalreport entitledATechniquefortheMeasurementofAttitudeswas publishedintheArchivesofPsychologyin1932.Hisgoalwasto developameansofmeasuringpsychologicalattitudesina scientificway.Specifically,hesoughtamethodthatwould produceattitudemeasuresthatcouldreasonablybeinterpreted asmeasurementsonapropermetricscale,inthesamesense thatweconsidergramsordegreesCelsiustruemeasurement scales(Uebersax,2006).
Fromhttp://www.performancezoom.com/performanceszoom_fichiers/likert.gif

Example:

SupposewearecomparingtheopinionsofMastersandPhDstudentsinCPSC.

Pleaseindicatehowmuchyouagreeordisagreewitheachofthefollowingstatements:
Strongly disagree 1 1 1 Somewhat disagree 2 2 2 Neitheragree nordisagree 3 3 3 Somewhat agree 4 4 4 Strongly agree 5 5 5

1.TheUofCThisisnowwebsiteiseasytouse. 2.TheMyUofCwebsiteiseasytouse. 3.ThePeoplesoftStudentCenterwebsiteiseasytouse.

LikertScales

1|P a g e

CPSC681TopicReport

DaneBertram

Analysis:

Eachspecificquestion(oritem)canhaveitsresponseanalyzedseparately,orhaveitsummed withotherrelateditemstocreateascoreforagroupofstatements.ThisisalsowhyLikert scalesaresometimescalledsummativescales.Forourexamplewewillevaluatetheresultsasa wholeusingdescriptivestatistics,andalsothespecificresultsforquestion1(seeMannWhitney Utestsectionbelow). Individualresponsesarenormallytreatedasordinaldatabecausealthoughtheresponselevels dohaverelativeposition,wecannotpresumethatparticipantsperceivethedifferencebetween adjacentlevelstobeequal(arequirementforintervaldata).Inpractice,manyresearchersdo treatLikertscaleresponsedataasifitwereintervaldata;however,fromastatisticalstandpoint thiscanbedangerous.Forexample,thereisnowaytoensurethatparticipantsviewthe differencebetweenagreeandstronglyagreethesameastheymightviewthedifference betweenagreeandneutral.

Theaverageoffairandgoodisnotfairandahalf;whichistrueevenwhen oneassignsintegerstorepresentfairandgood! SusanJamiesonparaphrasingKuzonJretal.(Jamieson,2004) TherawdataforourexampleisoutlinedinTable1below.Theparticipantresponses havebeengroupedaccordingtoMastersandPhDstudentsinordertohelprelatethis datatothestatisticswewillcalculateinthefollowingsections.

ParticipantID 1 2 3 4 MSc 5 6 7 8 9 10 11 12 PhD 13 14 15 16 1 2 4 1 3 2 3 1 2 3 3 2 5 4 3 4 3 2 2 4 3 2 3 4 4 5 4 1 3 2 3 4 3 2 2 3 Category Q1.President 4 3 4 2 Q2.GSA 4 4 3 3 Q3.CSGS 3 3 2 4

Table1.RawData

LikertScales

2|P a g e

CPSC681TopicReport

DaneBertram

Tables2,3,4,and5providetwovariationsofthedescriptivestatisticsthatcanbe calculatedfortheabovedata.Tables2and3showthemedian,mode,range,andinter quartilerangefortherawdatawhereTable2treatsalltheresponsestogetherasa wholeandTable3breaksdownthesamestatisticsintoourtwoparticipantcategories (MastersandPhDstudents)inordertoaidinthecomparisonofthesegroups.

Q1.UofC Q2.MyUofC Q3.Peoplesoft

Median 3 3 3

Mode 4 3 3

Range 4 4 2

InterquartileRange 2 1.25 1

Table2.DescriptiveStatistics1A

Q1.UofC Q2.MyUofC Q3.Peoplesoft

Median MSc 4 3 3 PhD 2 3 2.5 MSc 4 3 3

Mode PhD 2 4 3 MSc 3 2 2

Range PhD 3 4 1

InterquartileRange MSc 1 1 0.5 PhD 1.5 2.25 1

Table3.DescriptiveStatistics1B

Tables4and5goontoaggregatethenumberofresponsesforeachLikertlevelineach questionwhereTable4againtreatsalltheresponsesasawholewhileTable5 distinguishesbetweenMastersandPhDstudentresponses.

Strongly disagree # % 2 13% 2 13% 0 0%

Somewhat disagree 4 25% 2 13% 6 38%

Neitheragree nordisagree 3 19% 6 38% 8 50%

Somewhat agree 6 38% 5 31% 2 13%

Stronglyagree 1 6% 1 6% 0 0%

Q1.UofC

Q2.MyUofC # % Q3.Peoplesoft # %

Table4.DescriptiveStatistics2A

LikertScales 3|P a g e

CPSC681TopicReport
Strongly disagree MSc # % 0 0% 0 0% 0 0% PhD 2 25% 2 25% 0 0% Somewhat disagree MSc 1 13% 1 13% 2 25% PhD 3 38% 1 13% 4 50% Neitheragree nordisagree MSc 2 25% 4 50% 4 50% PhD 1 13% 2 25% 4 50% Somewhat agree MSc 4 50% 3 38% 2 25% PhD 2 25% 2 25% 0 0%

DaneBertram
Stronglyagree MSc 1 13% 0 0% 0 0% PhD 0 0% 1 13% 0 0%

Q1.UofC

Q2.MyUofC # % Q3.Peoplesoft # %

Table5.DescriptiveStatistics2B

Methods:

DependingonhowtheLikertscalequestionsaretreated,anumberofdifferentanalysis methodscanbeapplied: 1. Analysismethodsusedforindividualquestions(ordinaldata): barchartsanddotplots nothistograms(dataisnotcontinuous) centraltendencysummarisedbymedianandmode notmean variabilitysummarisedbyrangeandinterquartilerange notstandarddeviation analyzedusingnonparametrictests(differencesbetweenthemediansof comparablegroups) MannWhitneyUtest(seebelow) Wilcoxonsignedranktest KruskalWallistest 2. WhenmultipleLikertquestionresponsesaresummedtogether(intervaldata): allquestionsmustusethesameLikertscale mustbeadefendableapproximationtoanintervalscale(i.e.codingindicates magnitudeofdifferencebetweenitems,butthereisnoabsolutezeropoint) allitemsmeasureasinglelatentvariable(i.e.avariablethatisnotdirectly observed,butratherinferredfromothervariablesthatareobservedand directlymeasured) analyzedusingparametrictests analysisofvariance(ANOVA)

3. Analysismethodsusedwhenreducedtonominallevelsofagreevs.disagree:
LikertScales Chisquaretest CochranQtest McNemartest 4|P a g e

CPSC681TopicReport

DaneBertram

MannWhitneyUtest:
TogiveanexampleofhowyoumightevaluateasingleLikertscalequestionwewillusethe MannWhitneyUtest(alsocalledtheMannWhitneyWilcoxon,Wilcoxonranksumtest,or WilcoxconMannWhitneytest)tocomparetheopinionsofMastersvs.PhDstudentswith respecttotheusabilityofthemainUofCwebsite(question1fromtheexample).Thisisanon parametrictest,andisthereforewellsuitedtoourLikertscaledataaswecannotpresumethat theunderlyingpopulationfitsanormaldistribution(oranyotherparameterizeddistributionfor thatmatter).Thistestrequiresthatourtwosamplesbestatisticallyindependent(i.e.results fromonesampledonotaffectresultsintheothersample),andthattheobservationsbe ordinal.Wecanusethismethodtotestthenullhypothesisthatthereisanequalprobability thatanobservationfromonesamplewillexceedanobservationfromtheothersample essentiallystatingthatthetwosamplescomefromthesamepopulation.

RunningtheMannWhitneyUtest:
1.CalculatetheUstatistic. TocalculatetheUstatisticwecombinetheobservationvaluesfrombothsamplesandwrite themdowninrankorder.Beloweachobservationvaluewemarkwhichsampleitcamefrom (alternatingbetweenthetwosampleswhenthesameobservationvalueisrepeatedandcanbe seeninbothsamples).Thishasbeendonewiththeobservationvaluesforquestion1asfollows (P=PhDsample,M=MScsample): Rankordered: 1, Originsample: P, 1, P, 2, M, 2, P, 2, P, 2, P, 3, M, 3, P, 3, M, 4, M, 4, P, 4, M, 4, P, 4, M, 4, M, 5 M

Next,movingfromlefttoright,wetakeeachobservationfromsample1(Mastersstudents responses)andcountthenumberofobservationsfromsample2(PhDstudentsresponses) occurringafterit(totheright)inthelist.Whentherearematchingresponses(thesame observationvalue)fromeachsamplewecountahalf. Forexample,withthefirstMastersstudentresponsewehavethefollowing: Rankordered: 1, Originsample: P, 1, P, 2, M,

Tie

2, P,

2, P,

2, P,

3, M,

3, P,

3, M,

4, M,

4, P,

4, M,

4, P,

4, M,

4, M,

5 M

Sincethereisatie,wecount0.5andthen1foreachofthePhDresponses(afterthetie) appearingtotherightofthisMastersresponseinthelistgivingusacountof5.5. ForthesecondMastersstudentresponsewehavethefollowing: 1, P, 2, M, 2, P, 2, P, 2, P, 3, M,

Tie

Rankordered: 1, Originsample: P,

3, P,

3, M,

4, M,

4, P,

4, M,

4, P,

4, M,

4, M,

5 M

LikertScales

5|P a g e

CPSC681TopicReport

DaneBertram

Againwehaveatie,sowecount0.5andthen1foreachadditionalPhDresponseappearingto therightgivingusacountof2.5. ForthethirdMastersstudentresponsewedonthaveatie,sowesimplycount1foreachPhD responseappearingtotherightgivingusacountof2.Thisprocesscontinuesuntilweve calculatedacountforeachoftheMastersstudentresponses.Thesecountsareaddedtogether togiveustheUstatisticwhenstartingthecountingprocesswithsample1: =5.5+2.5+2+1.5+0.5+0+0+0 =12 Similarly,weperformthesamecalculationforeachobservationfromsample2. InthisexamplethereisnotieforthefirstPhDstudentresponseandalloftheMastersstudent responsescomeafteritinthelist,givingacountof8.Justasbefore,wecontinuethisprocess foreachPhDstudentresponseyieldingthefollowingUstatisticwhenstartingwithsample2: =8+8+7.5+7+7+6.5+4.5+3.5 =52 Note: Aconvenientchecktoensureyournumbersarecorrectistoensurethat: U1+U2=(#ofobservationsinSample1)x(#ofobservationsinSample) Thischeckworksbecauseinthemostextremecase,allthevaluesfromonesamplewouldcome beforethevaluesfromtheothersample.Thus,movinglefttoright,eachofthecountswouldbe either8or0intheexampleabove.

UMSc

UPhD

2. AftercalculatingtheUstatistics,consultthetableofcriticalvaluesfortheMannWhitneyUdistribution (Table6)usingthelowerofthetwocalculatedUstatistics(UMSc=12inthiscase).Note:Table6isonlya portionofthefulltableadaptedfrom(Bissonnette,2004). n1=#ofobservationsinsample1(8inthiscase) n2=#ofobservationsinsample2(8inthiscase) =levelofsignificance IfyourUstatisticisbelowthevalueindicatedinthe table,youcanrejectthenullhypothesisandstate withagivenconfidencelevelthatthe results/samplesaresignificantlydifferent.So,in thiscasewecanseethatourUstatistic(UMSc=12) isbelowthevalueindicatedinthetable(13)ata significancelevelof.05whenworkingwithtwo samplesof8observationseach.Thus,wecanreject thenullhypothesisatthe.05levelandstatethat theMScandPhDsamplesaresignificantlydifferent intheiropinionofthemainUofCwebsite. Table6.MannWhitneyUDistributionCriticalValues LikertScales 6|P a g e

CPSC681TopicReport

DaneBertram

LikertScaleStrengths:
simpletoconstruct likelytoproduceahighlyreliablescale easytoreadandcompleteforparticipants

LikertScaleWeaknesses:
centraltendencybias participantsmayavoidextremeresponsecategories acquiescensebias participantsmayagreewithstatementsaspresentedinordertopleasethe experimenter socialdesirabilitybias portraythemselvesinamoresociallyfavourablelightratherthanbeinghonest lackofreproducibility validitymaybedifficulttodemonstrate areyoumeasuringwhatyousetouttomeasure?

Glossary:
interquartilerange thedifferencebetweenthe3rdquartile(Q3)andthe1stquartile(Q1);themiddle50%ofthedata 1.Usethemediantosplitdataintwo(don'tincludethemedianneitherhalf) i 2.Lowerquartilevalueisthemedianofthelowerhalf;upperquartilevalueisthemedianof upperhalf intervalscale numbersassignedtoresponsesindicatemagnitudeofdifferencebetweenitems,butthereisno absolutezeropoint(i.e.differencesbetweenpairsofmeasurementscanbemeaningfully compared) latentvariable avariablethatisnotdirectlyobserved,butratherinferredfromothervariablesthatareobserved anddirectlymeasured median themiddlenumberinasortedlistofdata mode themostfrequentnumberinasetofdata noncomparativescaling eachitemisscaledindependentlyfromtheothers(ex.HowdoyoufeelaboutX?) contrastscomparativescalingwhereitemsarecomparedwitheachother(ex.DoyoupreferXor Y?)

LikertScales

7|P a g e

CPSC681TopicReport

DaneBertram

nonparametric underlyingpopulationdoesnothaveapredefineddistribution(e.g.anormaldistribution) onetailedtest onlyinterestedinadifferenceinasingledirection(i.e.hypothesispredictsthedirectionof differenceaheadoftime) ordinalscale classificationintoorderedcategories,butthereisnoinformationaboutthemagnitudeof differencesbetweencategories psychometric measurementofpsychologicalvariablessuchasattitudes,abilities,personalitytraits,etc. quartile anyofthreevaluesthatsegmentsorteddataintofourequalparts Firstquartile(Q1)cutsoffthelowest25%ofthedata Secondquartile(Q2)isthesameasthemedian Thirdquartile(Q3)cutsoffthehighest25%ofthedata range differencebetweenthelargestandsmallestvalueinasetofdata statisticalindependence theoccurrenceofoneeventmakesitneithermorenorlessprobablethattheothereventoccurs twotailedtest interestedinthedifferenceaswellasthedirectionofthedifference unidimensional measuresonlyasingleunderlyingtrait

Resources:
LikertscaleWikipedia,thefreeencyclopedia(http://en.wikipedia.org/wiki/Likert_scale) goodoverviewofthemethod;servesasagoodjumpoffpageforfindingoutmoreaboutspecific analysismethods,relatedscales,andbackgroundinformation LikertScaleDr.DelSiegleshomepage,NeagSchoolofEducation,UniversityofConnecticut (http://www.gifted.uconn.edu/siegle/research/instrument%20Reliability%20and%20Validity/Likert. html) collectionofcommonlyusedLikertscales(categories/levels)forvarioustypesofattitude measurement(agreement,frequency,importance,quality,likelihood,etc.) Furtherinformationonthevariousanalysismethodsmentionedabovecanbefoundasfollows: MannWhitneyUtesthttp://en.wikipedia.org/wiki/MannWhitney_test Wilcoxonsignedranktesthttp://en.wikipedia.org/wiki/Wilcoxon_signedrank_test KruskalWallistesthttp://en.wikipedia.org/wiki/KruskalWallis_test LikertScales 8|P a g e

CPSC681TopicReport AnalysisofVariance(ANOVA)http://en.wikipedia.org/wiki/Anova Chisquaretesthttp://en.wikipedia.org/wiki/ChiSquare CochranQtesthttp://en.wikipedia.org/wiki/Cochran%27s_theorem McNemartesthttp://en.wikipedia.org/wiki/McNemarTest

DaneBertram

References:
Bissonnette,VictorL."StatisticalTables."VictorBissonnettesHomePage.23Mar.2004.Dept.of Psychology,BerryCollege.23Oct.2007 <http://fsweb.berry.edu/academic/education/vbissonnette/tables/tables.html> containsvariousstatisticallookuptables,specificallytheoneabbreviatedasTable6inthisreport otherareasofthissiteoutlineanddemonstratevariousstatisticalmethodsthroughtheuseof interactiveapplets Jamieson,Susan."LikertScales:Howto(Ab)UseThem."MedicalEducation38(2004):12171218. shortarticleoutlinessomecommonpitfallsseeninpracticewhenusingLikertscales specificallyitelaboratesontheinherentproblemsintreatingLikertscaleresultdataasinterval datawhenitshouldgenerallybetreatedasordinaldata alsoservesasaconcisesummaryofotherworkintheareaexpressingsimilarconcerns KuzonWM.Jr,UrbanchekMB.,andMcCabeS.Thesevendeadlysinsofstatisticalanalysis.AnnPlastic Surg37(1996):26572 (includedforcompleteness)theoriginalpaperthatJamiesonparaphrasesinherpaper(referenced above)whichIthenquoteinthisreport Likert,Rensis."ATechniquefortheMeasurementofAttitudes."ArchivesofPsychology140(1932):155. Dr.Likertsoriginalpublicationaboutthescalesthatwouldlatercometobearhisname unfortunatelyIwasunabletoobtainacopyofthispaperindigitalformat(orotherwise)dueto theageofitspublication Mogey,Nora."SoYouWanttoUseaLikertScale?"LearningTechnologyDisseminationInitiative.25Mar. 1999.HeriotWattUniversity.20Oct.2007 <http://www.icbl.hw.ac.uk/ltdi/cookbook/info_likert_scale/index.html>. pagereferencedfromtheWikipediaarticle(seetheresourcessectionofthisreport) offersaconcise,highleveloverviewofLikertScalesaswellasthedescriptiveandinferential techniquesthatcanbeappliedtothem disappointinglylightonspecificswhenitcomestoexamplesandanalysis PageBucci,Hilary."TheValueofLikertScalesinMeasuringAttitudesofOnlineLearners."HKADesigns.Feb. 2003.20Oct.2007<http://www.hkadesigns.co.uk/websites/msc/reme/likert.htm>. areportonthevirtuesofLikertscalesinthecontextofmeasuringtheattitudesofonlinelearners outlinesabriefoverviewandcomparisonofvariousrelatedscales discussesadvantages,disadvantagesandsomeofthereliabilityandvalidityconcerns

LikertScales

9|P a g e

CPSC681TopicReport

DaneBertram

Shneiderman,Ben.DesigningtheUserInterface:StategiesforEffectiveHumanComputerInteraction.3rded. AddisonWesleyLongman,Inc.,1998.136143. providesexcerptsfromaquestionnaireforuserinterfacesatisfactionwhichshowsanumberof LikertscaletypequestionsandhowtheycanbeappliedtotheHumanComputerInteractionfield

Uebersax,JohnS."LikertScales:DispellingtheConfusion."StatisticalMethodsforRaterAgreement.31Aug. 2006.20Oct.2007<http://ourworld.compuserve.com/homepages/jsuebersax/likert.htm>. doesagreatjobofdefininganddifferentiatingthevarioustermsusedinconnectionwithLikert scales givesconcreteexamplesofeachvariationandpointedargumentsfortherisksinvolvedinthe commonmisguidedassumptionsofordinalvs.intervaldata

LikertScales

10|P a g e