LikertScales

arethemeaningoflife:

DaneBertram

Note:Aglossaryisincludedneartheendofthishandoutdefiningmanyofthetermsusedthroughoutthisreport.
CPSC681TopicReport DaneBertram

LikertScale\lickurt\,n.

Definition: Apsychometricresponsescaleprimarilyusedinquestionnairestoobtainparticipants
preferencesordegreeofagreementwithastatementorsetofstatements.Likertscalesarea
noncomparativescalingtechniqueandareunidimensional(onlymeasureasingletrait)in
nature.Respondentsareaskedtoindicatetheirlevelofagreementwithagivenstatementby
wayofanordinalscale.

Variations: Mostcommonlyseenasa5pointscalerangingfromStronglyDisagreeononeendto
StronglyAgreeontheotherwithNeitherAgreenorDisagreeinthemiddle;however,some
practitionersadvocatetheuseof7and9pointscaleswhichaddadditionalgranularity.
Sometimesa4point(orotherevennumbered)scaleisusedtoproduceanipsative(forced
choice)measurewherenoindifferentoptionisavailable.Eachlevelonthescaleisassigneda
numericvalueorcoding,usuallystartingat1andincrementedbyoneforeachlevel.For
example:

Figure1.SamplescaleusedinLikertscalequestions

Origin: NamedafterDr.RensisLikert,asociologistattheUniversityof
Michigan,whodevelopedthetechnique.Hisoriginalreport
entitledATechniquefortheMeasurementofAttitudeswas
publishedintheArchivesofPsychologyin1932.Hisgoalwasto
developameansofmeasuringpsychologicalattitudesina
scientificway.Specifically,hesoughtamethodthatwould
produceattitudemeasuresthatcouldreasonablybeinterpreted
asmeasurementsonapropermetricscale,inthesamesense
thatweconsidergramsordegreesCelsiustruemeasurement
scales(Uebersax,2006).
Fromhttp://www.performancezoom.com/performanceszoom_fichiers/likert.gif

Example: SupposewearecomparingtheopinionsofMastersandPhDstudentsinCPSC.

Pleaseindicatehowmuchyouagreeordisagreewitheachofthefollowingstatements:

Strongly Somewhat Neitheragree Somewhat Strongly

disagree disagree nordisagree agree agree

1.TheUofCThisisnowwebsiteiseasytouse. 1 2 3 4 5

2.TheMyUofCwebsiteiseasytouse. 1 2 3 4 5

3.ThePeoplesoftStudentCenterwebsiteiseasytouse. 1 2 3 4 5

LikertScales 1|P a g e

CPSC681TopicReport DaneBertram
Analysis: Eachspecificquestion(oritem)canhaveitsresponseanalyzedseparately,orhaveitsummed
withotherrelateditemstocreateascoreforagroupofstatements.ThisisalsowhyLikert
scalesaresometimescalledsummativescales.Forourexamplewewillevaluatetheresultsasa
wholeusingdescriptivestatistics,andalsothespecificresultsforquestion1(seeMannWhitney
Utestsectionbelow).

Individualresponsesarenormallytreatedasordinaldatabecausealthoughtheresponselevels
dohaverelativeposition,wecannotpresumethatparticipantsperceivethedifferencebetween
adjacentlevelstobeequal(arequirementforintervaldata).Inpractice,manyresearchersdo
treatLikertscaleresponsedataasifitwereintervaldata;however,fromastatisticalstandpoint
thiscanbedangerous.Forexample,thereisnowaytoensurethatparticipantsviewthe
differencebetweenagreeandstronglyagreethesameastheymightviewthedifference
betweenagreeandneutral.

Theaverageoffairandgoodisnotfairandahalf;whichistrueevenwhen
oneassignsintegerstorepresentfairandgood!
SusanJamiesonparaphrasingKuzonJretal.(Jamieson,2004)

TherawdataforourexampleisoutlinedinTable1below.Theparticipantresponses
havebeengroupedaccordingtoMastersandPhDstudentsinordertohelprelatethis
datatothestatisticswewillcalculateinthefollowingsections.

ParticipantID Category Q1.President Q2.GSA Q3.CSGS

1 4 4 3

2 3 4 3

3 4 3 2

4 2 3 4
MSc
5 5 3 3

6 4 2 2

7 3 3 3

8 4 4 4

9 3 4 3

10 2 5 2

11 2 4 2

12 4 1 3
PhD
13 1 3 2

14 2 2 3

15 4 3 3

16 1 1 2
Table1.RawData

LikertScales 2|P a g e

CPSC681TopicReport DaneBertram

Tables2,3,4,and5providetwovariationsofthedescriptivestatisticsthatcanbe
calculatedfortheabovedata.Tables2and3showthemedian,mode,range,andinter
quartilerangefortherawdatawhereTable2treatsalltheresponsestogetherasa
wholeandTable3breaksdownthesamestatisticsintoourtwoparticipantcategories
(MastersandPhDstudents)inordertoaidinthecomparisonofthesegroups.

Median Mode Range InterquartileRange

Q1.UofC 3 4 4 2

Q2.MyUofC 3 3 4 1.25

Q3.Peoplesoft 3 3 2 1

Table2.DescriptiveStatistics1A

Median Mode Range InterquartileRange

MSc PhD MSc PhD MSc PhD MSc PhD

Q1.UofC 4 2 4 2 3 3 1 1.5

Q2.MyUofC 3 3 3 4 2 4 1 2.25

Q3.Peoplesoft 3 2.5 3 3 2 1 0.5 1

Table3.DescriptiveStatistics1B

Tables4and5goontoaggregatethenumberofresponsesforeachLikertlevelineach
questionwhereTable4againtreatsalltheresponsesasawholewhileTable5
distinguishesbetweenMastersandPhDstudentresponses.

Strongly Somewhat Neitheragree Somewhat

Stronglyagree
disagree disagree nordisagree agree

Q1.UofC # 2 4 3 6 1

% 13% 25% 19% 38% 6%

Q2.MyUofC # 2 2 6 5 1

% 13% 13% 38% 31% 6%

Q3.Peoplesoft # 0 6 8 2 0

% 0% 38% 50% 13% 0%

Table4.DescriptiveStatistics2A

LikertScales 3|P a g e

CPSC681TopicReport DaneBertram

Strongly Somewhat Neitheragree Somewhat

Stronglyagree
disagree disagree nordisagree agree

MSc PhD MSc PhD MSc PhD MSc PhD MSc PhD

Q1.UofC # 0 2 1 3 2 1 4 2 1 0

% 0% 25% 13% 38% 25% 13% 50% 25% 13% 0%

Q2.MyUofC # 0 2 1 1 4 2 3 2 0 1

% 0% 25% 13% 13% 50% 25% 38% 25% 0% 13%

Q3.Peoplesoft # 0 0 2 4 4 4 2 0 0 0

% 0% 0% 25% 50% 50% 50% 25% 0% 0% 0%

Table5.DescriptiveStatistics2B

Methods: DependingonhowtheLikertscalequestionsaretreated,anumberofdifferentanalysis
methodscanbeapplied:

1. Analysismethodsusedforindividualquestions(ordinaldata):
barchartsanddotplots
nothistograms(dataisnotcontinuous)
centraltendencysummarisedbymedianandmode
notmean
variabilitysummarisedbyrangeandinterquartilerange
notstandarddeviation
analyzedusingnonparametrictests(differencesbetweenthemediansof
comparablegroups)
MannWhitneyUtest(seebelow)
Wilcoxonsignedranktest
KruskalWallistest

2. WhenmultipleLikertquestionresponsesaresummedtogether(intervaldata):
allquestionsmustusethesameLikertscale
mustbeadefendableapproximationtoanintervalscale(i.e.codingindicates
magnitudeofdifferencebetweenitems,butthereisnoabsolutezeropoint)
allitemsmeasureasinglelatentvariable(i.e.avariablethatisnotdirectly
observed,butratherinferredfromothervariablesthatareobservedand
directlymeasured)
analyzedusingparametrictests
analysisofvariance(ANOVA)

3. Analysismethodsusedwhenreducedtonominallevelsofagreevs.disagree:
Chisquaretest
CochranQtest
McNemartest
LikertScales 4|P a g e

CPSC681TopicReport DaneBertram
MannWhitneyUtest:

TogiveanexampleofhowyoumightevaluateasingleLikertscalequestionwewillusethe
MannWhitneyUtest(alsocalledtheMannWhitneyWilcoxon,Wilcoxonranksumtest,or
WilcoxconMannWhitneytest)tocomparetheopinionsofMastersvs.PhDstudentswith
respecttotheusabilityofthemainUofCwebsite(question1fromtheexample).Thisisanon
parametrictest,andisthereforewellsuitedtoourLikertscaledataaswecannotpresumethat
theunderlyingpopulationfitsanormaldistribution(oranyotherparameterizeddistributionfor
thatmatter).Thistestrequiresthatourtwosamplesbestatisticallyindependent(i.e.results
fromonesampledonotaffectresultsintheothersample),andthattheobservationsbe
ordinal.Wecanusethismethodtotestthenullhypothesisthatthereisanequalprobability
thatanobservationfromonesamplewillexceedanobservationfromtheothersample
essentiallystatingthatthetwosamplescomefromthesamepopulation.

RunningtheMannWhitneyUtest:
1.CalculatetheUstatistic.
TocalculatetheUstatisticwecombinetheobservationvaluesfrombothsamplesandwrite
themdowninrankorder.Beloweachobservationvaluewemarkwhichsampleitcamefrom
(alternatingbetweenthetwosampleswhenthesameobservationvalueisrepeatedandcanbe
seeninbothsamples).Thishasbeendonewiththeobservationvaluesforquestion1asfollows
(P=PhDsample,M=MScsample):
Rankordered: 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 4, 4, 4, 4, 5
Originsample: P, P, M, P, P, P, M, P, M, M, P, M, P, M, M, M

Next,movingfromlefttoright,wetakeeachobservationfromsample1(Mastersstudents
responses)andcountthenumberofobservationsfromsample2(PhDstudentsresponses)
occurringafterit(totheright)inthelist.Whentherearematchingresponses(thesame
observationvalue)fromeachsamplewecountahalf.

Forexample,withthefirstMastersstudentresponsewehavethefollowing:

Tie
Rankordered: 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 4, 4, 4, 4, 5
Originsample: P, P, M, P, P, P, M, P, M, M, P, M, P, M, M, M

Sincethereisatie,wecount0.5andthen1foreachofthePhDresponses(afterthetie)
appearingtotherightofthisMastersresponseinthelistgivingusacountof5.5.

ForthesecondMastersstudentresponsewehavethefollowing:

Tie
Rankordered: 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 4, 4, 4, 4, 5
Originsample: P, P, M, P, P, P, M, P, M, M, P, M, P, M, M, M

LikertScales 5|P a g e

CPSC681TopicReport DaneBertram
Againwehaveatie,sowecount0.5andthen1foreachadditionalPhDresponseappearingto
therightgivingusacountof2.5.

ForthethirdMastersstudentresponsewedonthaveatie,sowesimplycount1foreachPhD
responseappearingtotherightgivingusacountof2.Thisprocesscontinuesuntilweve
calculatedacountforeachoftheMastersstudentresponses.Thesecountsareaddedtogether
togiveustheUstatisticwhenstartingthecountingprocesswithsample1:

UMSc =5.5+2.5+2+1.5+0.5+0+0+0
=12

Similarly,weperformthesamecalculationforeachobservationfromsample2.

InthisexamplethereisnotieforthefirstPhDstudentresponseandalloftheMastersstudent
responsescomeafteritinthelist,givingacountof8.Justasbefore,wecontinuethisprocess
foreachPhDstudentresponseyieldingthefollowingUstatisticwhenstartingwithsample2:

UPhD =8+8+7.5+7+7+6.5+4.5+3.5
=52

Note: Aconvenientchecktoensureyournumbersarecorrectistoensurethat:
U1+U2=(#ofobservationsinSample1)x(#ofobservationsinSample)

Thischeckworksbecauseinthemostextremecase,allthevaluesfromonesamplewouldcome
beforethevaluesfromtheothersample.Thus,movinglefttoright,eachofthecountswouldbe
either8or0intheexampleabove.

2. AftercalculatingtheUstatistics,consultthetableofcriticalvaluesfortheMannWhitneyUdistribution
(Table6)usingthelowerofthetwocalculatedUstatistics(UMSc=12inthiscase).Note:Table6isonlya
portionofthefulltableadaptedfrom(Bissonnette,2004).

n1=#ofobservationsinsample1(8inthiscase)
n2=#ofobservationsinsample2(8inthiscase)
=levelofsignificance

IfyourUstatisticisbelowthevalueindicatedinthe
table,youcanrejectthenullhypothesisandstate
withagivenconfidencelevelthatthe
results/samplesaresignificantlydifferent.So,in
thiscasewecanseethatourUstatistic(UMSc=12)
isbelowthevalueindicatedinthetable(13)ata
significancelevelof.05whenworkingwithtwo
samplesof8observationseach.Thus,wecanreject
thenullhypothesisatthe.05levelandstatethat
theMScandPhDsamplesaresignificantlydifferent
intheiropinionofthemainUofCwebsite.
Table6.MannWhitneyUDistributionCriticalValues
LikertScales 6|P a g e

CPSC681TopicReport DaneBertram
LikertScaleStrengths:
simpletoconstruct
likelytoproduceahighlyreliablescale
easytoreadandcompleteforparticipants

LikertScaleWeaknesses:
centraltendencybias
participantsmayavoidextremeresponsecategories
acquiescensebias
participantsmayagreewithstatementsaspresentedinordertopleasethe
experimenter
socialdesirabilitybias
portraythemselvesinamoresociallyfavourablelightratherthanbeinghonest
lackofreproducibility
validitymaybedifficulttodemonstrate
areyoumeasuringwhatyousetouttomeasure?

Glossary:
interquartilerange
thedifferencebetweenthe3rdquartile(Q3)andthe1stquartile(Q1);themiddle50%ofthedata
1.Usethemediantosplitdataintwo(don'tincludethemedianineitherhalf)
2.Lowerquartilevalueisthemedianofthelowerhalf;upperquartilevalueisthemedianof
upperhalf

intervalscale
numbersassignedtoresponsesindicatemagnitudeofdifferencebetweenitems,butthereisno
absolutezeropoint(i.e.differencesbetweenpairsofmeasurementscanbemeaningfully
compared)

latentvariable
avariablethatisnotdirectlyobserved,butratherinferredfromothervariablesthatareobserved
anddirectlymeasured

median
themiddlenumberinasortedlistofdata

mode
themostfrequentnumberinasetofdata

noncomparativescaling
eachitemisscaledindependentlyfromtheothers(ex.HowdoyoufeelaboutX?)
contrastscomparativescalingwhereitemsarecomparedwitheachother(ex.DoyoupreferXor
Y?)

LikertScales 7|P a g e

CPSC681TopicReport DaneBertram
nonparametric
underlyingpopulationdoesnothaveapredefineddistribution(e.g.anormaldistribution)

onetailedtest
onlyinterestedinadifferenceinasingledirection(i.e.hypothesispredictsthedirectionof
differenceaheadoftime)

ordinalscale
classificationintoorderedcategories,butthereisnoinformationaboutthemagnitudeof
differencesbetweencategories

psychometric
measurementofpsychologicalvariablessuchasattitudes,abilities,personalitytraits,etc.

quartile
anyofthreevaluesthatsegmentsorteddataintofourequalparts
Firstquartile(Q1)cutsoffthelowest25%ofthedata
Secondquartile(Q2)isthesameasthemedian
Thirdquartile(Q3)cutsoffthehighest25%ofthedata

range
differencebetweenthelargestandsmallestvalueinasetofdata

statisticalindependence
theoccurrenceofoneeventmakesitneithermorenorlessprobablethattheothereventoccurs

twotailedtest
interestedinthedifferenceaswellasthedirectionofthedifference

unidimensional
measuresonlyasingleunderlyingtrait

Resources:
LikertscaleWikipedia,thefreeencyclopedia(http://en.wikipedia.org/wiki/Likert_scale)
goodoverviewofthemethod;servesasagoodjumpoffpageforfindingoutmoreaboutspecific
analysismethods,relatedscales,andbackgroundinformation

LikertScaleDr.DelSiegleshomepage,NeagSchoolofEducation,UniversityofConnecticut
(http://www.gifted.uconn.edu/siegle/research/instrument%20Reliability%20and%20Validity/Likert.
html)
collectionofcommonlyusedLikertscales(categories/levels)forvarioustypesofattitude
measurement(agreement,frequency,importance,quality,likelihood,etc.)

Furtherinformationonthevariousanalysismethodsmentionedabovecanbefoundasfollows:
MannWhitneyUtesthttp://en.wikipedia.org/wiki/MannWhitney_test
Wilcoxonsignedranktesthttp://en.wikipedia.org/wiki/Wilcoxon_signedrank_test
KruskalWallistesthttp://en.wikipedia.org/wiki/KruskalWallis_test
LikertScales 8|P a g e

CPSC681TopicReport DaneBertram
AnalysisofVariance(ANOVA)http://en.wikipedia.org/wiki/Anova
Chisquaretesthttp://en.wikipedia.org/wiki/ChiSquare
CochranQtesthttp://en.wikipedia.org/wiki/Cochran%27s_theorem
McNemartesthttp://en.wikipedia.org/wiki/McNemarTest

References:
Bissonnette,VictorL."StatisticalTables."VictorBissonnettesHomePage.23Mar.2004.Dept.of
Psychology,BerryCollege.23Oct.2007
<http://fsweb.berry.edu/academic/education/vbissonnette/tables/tables.html>

containsvariousstatisticallookuptables,specificallytheoneabbreviatedasTable6inthisreport
otherareasofthissiteoutlineanddemonstratevariousstatisticalmethodsthroughtheuseof
interactiveapplets

Jamieson,Susan."LikertScales:Howto(Ab)UseThem."MedicalEducation38(2004):12171218.

shortarticleoutlinessomecommonpitfallsseeninpracticewhenusingLikertscales
specificallyitelaboratesontheinherentproblemsintreatingLikertscaleresultdataasinterval
datawhenitshouldgenerallybetreatedasordinaldata
alsoservesasaconcisesummaryofotherworkintheareaexpressingsimilarconcerns

KuzonWM.Jr,UrbanchekMB.,andMcCabeS.Thesevendeadlysinsofstatisticalanalysis.AnnPlastic
Surg37(1996):26572

(includedforcompleteness)theoriginalpaperthatJamiesonparaphrasesinherpaper(referenced
above)whichIthenquoteinthisreport

Likert,Rensis."ATechniquefortheMeasurementofAttitudes."ArchivesofPsychology140(1932):155.

Dr.Likertsoriginalpublicationaboutthescalesthatwouldlatercometobearhisname
unfortunatelyIwasunabletoobtainacopyofthispaperindigitalformat(orotherwise)dueto
theageofitspublication

Mogey,Nora."SoYouWanttoUseaLikertScale?"LearningTechnologyDisseminationInitiative.25Mar.
1999.HeriotWattUniversity.20Oct.2007
<http://www.icbl.hw.ac.uk/ltdi/cookbook/info_likert_scale/index.html>.

pagereferencedfromtheWikipediaarticle(seetheresourcessectionofthisreport)
offersaconcise,highleveloverviewofLikertScalesaswellasthedescriptiveandinferential
techniquesthatcanbeappliedtothem
disappointinglylightonspecificswhenitcomestoexamplesandanalysis

PageBucci,Hilary."TheValueofLikertScalesinMeasuringAttitudesofOnlineLearners."HKADesigns.Feb.
2003.20Oct.2007<http://www.hkadesigns.co.uk/websites/msc/reme/likert.htm>.

areportonthevirtuesofLikertscalesinthecontextofmeasuringtheattitudesofonlinelearners
outlinesabriefoverviewandcomparisonofvariousrelatedscales
discussesadvantages,disadvantagesandsomeofthereliabilityandvalidityconcerns

LikertScales 9|P a g e

CPSC681TopicReport DaneBertram
Shneiderman,Ben.DesigningtheUserInterface:StategiesforEffectiveHumanComputerInteraction.3rded.
AddisonWesleyLongman,Inc.,1998.136143.

providesexcerptsfromaquestionnaireforuserinterfacesatisfactionwhichshowsanumberof
LikertscaletypequestionsandhowtheycanbeappliedtotheHumanComputerInteractionfield

Uebersax,JohnS."LikertScales:DispellingtheConfusion."StatisticalMethodsforRaterAgreement.31Aug.
2006.20Oct.2007<http://ourworld.compuserve.com/homepages/jsuebersax/likert.htm>.

doesagreatjobofdefininganddifferentiatingthevarioustermsusedinconnectionwithLikert
scales
givesconcreteexamplesofeachvariationandpointedargumentsfortherisksinvolvedinthe
commonmisguidedassumptionsofordinalvs.intervaldata

LikertScales 10|P a g e