LikertScales

arethemeaningoflife:

DaneBertram

Note:Aglossaryisincludedneartheendofthishandoutdefiningmanyofthetermsusedthroughoutthisreport.

CPSC681TopicReport

DaneBertram

LikertScale\lickurt\,n.

Definition:

Apsychometricresponsescaleprimarilyusedinquestionnairestoobtainparticipants
preferencesordegreeofagreementwithastatementorsetofstatements.Likertscalesarea
noncomparativescalingtechniqueandareunidimensional(onlymeasureasingletrait)in
nature.Respondentsareaskedtoindicatetheirlevelofagreementwithagivenstatementby
wayofanordinalscale.

Variations: Mostcommonlyseenasa5pointscalerangingfromStronglyDisagreeononeendto
StronglyAgreeontheotherwithNeitherAgreenorDisagreeinthemiddle;however,some
practitionersadvocatetheuseof7and9pointscaleswhichaddadditionalgranularity.
Sometimesa4point(orotherevennumbered)scaleisusedtoproduceanipsative(forced
choice)measurewherenoindifferentoptionisavailable.Eachlevelonthescaleisassigneda
numericvalueorcoding,usuallystartingat1andincrementedbyoneforeachlevel.For
example:

Figure1.SamplescaleusedinLikertscalequestions

Origin:

NamedafterDr.RensisLikert,asociologistattheUniversityof
Michigan,whodevelopedthetechnique.Hisoriginalreport
entitledATechniquefortheMeasurementofAttitudeswas
publishedintheArchivesofPsychologyin1932.Hisgoalwasto
developameansofmeasuringpsychologicalattitudesina
scientificway.Specifically,hesoughtamethodthatwould
produceattitudemeasuresthatcouldreasonablybeinterpreted
asmeasurementsonapropermetricscale,inthesamesense
thatweconsidergramsordegreesCelsiustruemeasurement
scales(Uebersax,2006).
Fromhttp://www.performancezoom.com/performanceszoom_fichiers/likert.gif

Example:

SupposewearecomparingtheopinionsofMastersandPhDstudentsinCPSC.

Pleaseindicatehowmuchyouagreeordisagreewitheachofthefollowingstatements:

Strongly
disagree

Somewhat
disagree

Neitheragree
nordisagree

Somewhat
agree

Strongly
agree

1.TheUofCThisisnowwebsiteiseasytouse.

2.TheMyUofCwebsiteiseasytouse.

3.ThePeoplesoftStudentCenterwebsiteiseasytouse.

LikertScales

1|P a g e

CPSC681TopicReport

Analysis:

DaneBertram

Eachspecificquestion(oritem)canhaveitsresponseanalyzedseparately,orhaveitsummed
withotherrelateditemstocreateascoreforagroupofstatements.ThisisalsowhyLikert
scalesaresometimescalledsummativescales.Forourexamplewewillevaluatetheresultsasa
wholeusingdescriptivestatistics,andalsothespecificresultsforquestion1(seeMannWhitney
Utestsectionbelow).
Individualresponsesarenormallytreatedasordinaldatabecausealthoughtheresponselevels
dohaverelativeposition,wecannotpresumethatparticipantsperceivethedifferencebetween
adjacentlevelstobeequal(arequirementforintervaldata).Inpractice,manyresearchersdo
treatLikertscaleresponsedataasifitwereintervaldata;however,fromastatisticalstandpoint
thiscanbedangerous.Forexample,thereisnowaytoensurethatparticipantsviewthe
differencebetweenagreeandstronglyagreethesameastheymightviewthedifference
betweenagreeandneutral.

Theaverageoffairandgoodisnotfairandahalf;whichistrueevenwhen
oneassignsintegerstorepresentfairandgood!

SusanJamiesonparaphrasingKuzonJretal.(Jamieson,2004)
TherawdataforourexampleisoutlinedinTable1below.Theparticipantresponses
havebeengroupedaccordingtoMastersandPhDstudentsinordertohelprelatethis
datatothestatisticswewillcalculateinthefollowingsections.
ParticipantID

Category

Q1.President

Q2.GSA

Q3.CSGS

10

11

13

14

15

16

4
MSc

12
PhD

Table1.RawData

LikertScales

2|P a g e

CPSC681TopicReport

DaneBertram

Tables2,3,4,and5providetwovariationsofthedescriptivestatisticsthatcanbe
calculatedfortheabovedata.Tables2and3showthemedian,mode,range,andinter
quartilerangefortherawdatawhereTable2treatsalltheresponsestogetherasa
wholeandTable3breaksdownthesamestatisticsintoourtwoparticipantcategories
(MastersandPhDstudents)inordertoaidinthecomparisonofthesegroups.

Median

Mode

Range

InterquartileRange

Q1.UofC

Q2.MyUofC

1.25

Q3.Peoplesoft

Range

InterquartileRange

Table2.DescriptiveStatistics1A

Median

Mode

MSc

PhD

MSc

PhD

MSc

PhD

MSc

PhD

Q1.UofC

1.5

Q2.MyUofC

2.25

Q3.Peoplesoft

2.5

0.5

Table3.DescriptiveStatistics1B

Tables4and5goontoaggregatethenumberofresponsesforeachLikertlevelineach
questionwhereTable4againtreatsalltheresponsesasawholewhileTable5
distinguishesbetweenMastersandPhDstudentresponses.

Q1.UofC

Q2.MyUofC #

%
Q3.Peoplesoft #

Strongly
disagree

Somewhat
disagree

Neitheragree
nordisagree

Somewhat
agree

Stronglyagree

13%

25%

19%

38%

6%

13%

13%

38%

31%

6%

0%

38%

50%

13%

0%

Table4.DescriptiveStatistics2A

LikertScales

3|P a g e

CPSC681TopicReport

DaneBertram

Strongly
disagree

Q1.UofC

Q2.MyUofC #

%
Q3.Peoplesoft #

Somewhat
disagree

Neitheragree
nordisagree

Somewhat
agree

Stronglyagree

MSc

PhD

MSc

PhD

MSc

PhD

MSc

PhD

MSc

PhD

0%

25%

13%

38%

25%

13%

50%

25%

13%

0%

0%

25%

13%

13%

50%

25%

38%

25%

0%

13%

0%

0%

25%

50%

50%

50%

25%

0%

0%

0%

Table5.DescriptiveStatistics2B

Methods:

DependingonhowtheLikertscalequestionsaretreated,anumberofdifferentanalysis
methodscanbeapplied:
1. Analysismethodsusedforindividualquestions(ordinaldata):
barchartsanddotplots

nothistograms(dataisnotcontinuous)
centraltendencysummarisedbymedianandmode

notmean
variabilitysummarisedbyrangeandinterquartilerange

notstandarddeviation
analyzedusingnonparametrictests(differencesbetweenthemediansof
comparablegroups)

MannWhitneyUtest(seebelow)

Wilcoxonsignedranktest

KruskalWallistest
2. WhenmultipleLikertquestionresponsesaresummedtogether(intervaldata):
allquestionsmustusethesameLikertscale
mustbeadefendableapproximationtoanintervalscale(i.e.codingindicates
magnitudeofdifferencebetweenitems,butthereisnoabsolutezeropoint)
allitemsmeasureasinglelatentvariable(i.e.avariablethatisnotdirectly
observed,butratherinferredfromothervariablesthatareobservedand
directlymeasured)
analyzedusingparametrictests

analysisofvariance(ANOVA)

3. Analysismethodsusedwhenreducedtonominallevelsofagreevs.disagree:

LikertScales

Chisquaretest
CochranQtest
McNemartest

4|P a g e

CPSC681TopicReport

DaneBertram

MannWhitneyUtest:

TogiveanexampleofhowyoumightevaluateasingleLikertscalequestionwewillusethe
MannWhitneyUtest(alsocalledtheMannWhitneyWilcoxon,Wilcoxonranksumtest,or
WilcoxconMannWhitneytest)tocomparetheopinionsofMastersvs.PhDstudentswith
respecttotheusabilityofthemainUofCwebsite(question1fromtheexample).Thisisanon
parametrictest,andisthereforewellsuitedtoourLikertscaledataaswecannotpresumethat
theunderlyingpopulationfitsanormaldistribution(oranyotherparameterizeddistributionfor
thatmatter).Thistestrequiresthatourtwosamplesbestatisticallyindependent(i.e.results
fromonesampledonotaffectresultsintheothersample),andthattheobservationsbe
ordinal.Wecanusethismethodtotestthenullhypothesisthatthereisanequalprobability
thatanobservationfromonesamplewillexceedanobservationfromtheothersample
essentiallystatingthatthetwosamplescomefromthesamepopulation.

RunningtheMannWhitneyUtest:
1.CalculatetheUstatistic.
TocalculatetheUstatisticwecombinetheobservationvaluesfrombothsamplesandwrite
themdowninrankorder.Beloweachobservationvaluewemarkwhichsampleitcamefrom
(alternatingbetweenthetwosampleswhenthesameobservationvalueisrepeatedandcanbe
seeninbothsamples).Thishasbeendonewiththeobservationvaluesforquestion1asfollows
(P=PhDsample,M=MScsample):
Rankordered: 1,
Originsample: P,

1,
P,

2,
M,

2,
P,

2,
P,

2,
P,

3,
M,

3,
P,

3,
M,

4,
M,

4,
P,

4,
M,

4,
P,

4,
M,

4,
M,

5
M

Next,movingfromlefttoright,wetakeeachobservationfromsample1(Mastersstudents
responses)andcountthenumberofobservationsfromsample2(PhDstudentsresponses)
occurringafterit(totheright)inthelist.Whentherearematchingresponses(thesame
observationvalue)fromeachsamplewecountahalf.
Forexample,withthefirstMastersstudentresponsewehavethefollowing:
Rankordered: 1,
Originsample: P,

1,
P,

Tie

2,
M,

2,
P,

2,
P,

2,
P,

3,
M,

3,
P,

3,
M,

4,
M,

4,
P,

4,
M,

4,
P,

4,
M,

Sincethereisatie,wecount0.5andthen1foreachofthePhDresponses(afterthetie)
appearingtotherightofthisMastersresponseinthelistgivingusacountof5.5.

ForthesecondMastersstudentresponsewehavethefollowing:

Rankordered: 1,
Originsample: P,

1,
P,

2,
M,

2,
P,

2,
P,

2,
P,

Tie

3,
M,

3,
P,

3,
M,

4,
M,

4,
P,

4,
M,

4,
P,

4,
M,

4,
M,

5
M

4,
M,

5
M

LikertScales

5|P a g e

CPSC681TopicReport

DaneBertram

Againwehaveatie,sowecount0.5andthen1foreachadditionalPhDresponseappearingto
therightgivingusacountof2.5.

ForthethirdMastersstudentresponsewedonthaveatie,sowesimplycount1foreachPhD
responseappearingtotherightgivingusacountof2.Thisprocesscontinuesuntilweve
calculatedacountforeachoftheMastersstudentresponses.Thesecountsareaddedtogether
togiveustheUstatisticwhenstartingthecountingprocesswithsample1:

UMSc

=5.5+2.5+2+1.5+0.5+0+0+0
=12

Similarly,weperformthesamecalculationforeachobservationfromsample2.

InthisexamplethereisnotieforthefirstPhDstudentresponseandalloftheMastersstudent
responsescomeafteritinthelist,givingacountof8.Justasbefore,wecontinuethisprocess
foreachPhDstudentresponseyieldingthefollowingUstatisticwhenstartingwithsample2:

UPhD

=8+8+7.5+7+7+6.5+4.5+3.5
=52
Note: Aconvenientchecktoensureyournumbersarecorrectistoensurethat:
U1+U2=(#ofobservationsinSample1)x(#ofobservationsinSample)
Thischeckworksbecauseinthemostextremecase,allthevaluesfromonesamplewouldcome
beforethevaluesfromtheothersample.Thus,movinglefttoright,eachofthecountswouldbe
either8or0intheexampleabove.

2. AftercalculatingtheUstatistics,consultthetableofcriticalvaluesfortheMannWhitneyUdistribution
(Table6)usingthelowerofthetwocalculatedUstatistics(UMSc=12inthiscase).Note:Table6isonlya
portionofthefulltableadaptedfrom(Bissonnette,2004).
n1=#ofobservationsinsample1(8inthiscase)
n2=#ofobservationsinsample2(8inthiscase)
=levelofsignificance
IfyourUstatisticisbelowthevalueindicatedinthe
table,youcanrejectthenullhypothesisandstate
withagivenconfidencelevelthatthe
results/samplesaresignificantlydifferent.So,in
thiscasewecanseethatourUstatistic(UMSc=12)
isbelowthevalueindicatedinthetable(13)ata
significancelevelof.05whenworkingwithtwo
samplesof8observationseach.Thus,wecanreject
thenullhypothesisatthe.05levelandstatethat
theMScandPhDsamplesaresignificantlydifferent
intheiropinionofthemainUofCwebsite.

Table6.MannWhitneyUDistributionCriticalValues
LikertScales

6|P a g e

CPSC681TopicReport

DaneBertram

LikertScaleStrengths:

simpletoconstruct
likelytoproduceahighlyreliablescale
easytoreadandcompleteforparticipants

LikertScaleWeaknesses:

centraltendencybias

participantsmayavoidextremeresponsecategories
acquiescensebias

participantsmayagreewithstatementsaspresentedinordertopleasethe

experimenter
socialdesirabilitybias

portraythemselvesinamoresociallyfavourablelightratherthanbeinghonest
lackofreproducibility
validitymaybedifficulttodemonstrate

areyoumeasuringwhatyousetouttomeasure?

Glossary:
interquartilerange
thedifferencebetweenthe3rdquartile(Q3)andthe1stquartile(Q1);themiddle50%ofthedata
1.Usethemediantosplitdataintwo(don'tincludethemedianineitherhalf)
2.Lowerquartilevalueisthemedianofthelowerhalf;upperquartilevalueisthemedianof
upperhalf
intervalscale
numbersassignedtoresponsesindicatemagnitudeofdifferencebetweenitems,butthereisno
absolutezeropoint(i.e.differencesbetweenpairsofmeasurementscanbemeaningfully
compared)
latentvariable
avariablethatisnotdirectlyobserved,butratherinferredfromothervariablesthatareobserved
anddirectlymeasured
median
themiddlenumberinasortedlistofdata
mode
themostfrequentnumberinasetofdata
noncomparativescaling
eachitemisscaledindependentlyfromtheothers(ex.HowdoyoufeelaboutX?)
contrastscomparativescalingwhereitemsarecomparedwitheachother(ex.DoyoupreferXor
Y?)

LikertScales

7|P a g e

CPSC681TopicReport

DaneBertram

nonparametric
underlyingpopulationdoesnothaveapredefineddistribution(e.g.anormaldistribution)
onetailedtest
onlyinterestedinadifferenceinasingledirection(i.e.hypothesispredictsthedirectionof
differenceaheadoftime)
ordinalscale
classificationintoorderedcategories,butthereisnoinformationaboutthemagnitudeof
differencesbetweencategories
psychometric
measurementofpsychologicalvariablessuchasattitudes,abilities,personalitytraits,etc.
quartile
anyofthreevaluesthatsegmentsorteddataintofourequalparts
Firstquartile(Q1)cutsoffthelowest25%ofthedata
Secondquartile(Q2)isthesameasthemedian
Thirdquartile(Q3)cutsoffthehighest25%ofthedata
range
differencebetweenthelargestandsmallestvalueinasetofdata
statisticalindependence
theoccurrenceofoneeventmakesitneithermorenorlessprobablethattheothereventoccurs
twotailedtest
interestedinthedifferenceaswellasthedirectionofthedifference
unidimensional
measuresonlyasingleunderlyingtrait

Resources:
LikertscaleWikipedia,thefreeencyclopedia(http://en.wikipedia.org/wiki/Likert_scale)
goodoverviewofthemethod;servesasagoodjumpoffpageforfindingoutmoreaboutspecific
analysismethods,relatedscales,andbackgroundinformation
LikertScaleDr.DelSiegleshomepage,NeagSchoolofEducation,UniversityofConnecticut
(http://www.gifted.uconn.edu/siegle/research/instrument%20Reliability%20and%20Validity/Likert.
html)
collectionofcommonlyusedLikertscales(categories/levels)forvarioustypesofattitude
measurement(agreement,frequency,importance,quality,likelihood,etc.)
Furtherinformationonthevariousanalysismethodsmentionedabovecanbefoundasfollows:
MannWhitneyUtesthttp://en.wikipedia.org/wiki/MannWhitney_test
Wilcoxonsignedranktesthttp://en.wikipedia.org/wiki/Wilcoxon_signedrank_test
KruskalWallistesthttp://en.wikipedia.org/wiki/KruskalWallis_test
LikertScales

8|P a g e

CPSC681TopicReport

DaneBertram

AnalysisofVariance(ANOVA)http://en.wikipedia.org/wiki/Anova
Chisquaretesthttp://en.wikipedia.org/wiki/ChiSquare
CochranQtesthttp://en.wikipedia.org/wiki/Cochran%27s_theorem
McNemartesthttp://en.wikipedia.org/wiki/McNemarTest

References:
Bissonnette,VictorL."StatisticalTables."VictorBissonnettesHomePage.23Mar.2004.Dept.of
Psychology,BerryCollege.23Oct.2007
<http://fsweb.berry.edu/academic/education/vbissonnette/tables/tables.html>

containsvariousstatisticallookuptables,specificallytheoneabbreviatedasTable6inthisreport
otherareasofthissiteoutlineanddemonstratevariousstatisticalmethodsthroughtheuseof
interactiveapplets
Jamieson,Susan."LikertScales:Howto(Ab)UseThem."MedicalEducation38(2004):12171218.

shortarticleoutlinessomecommonpitfallsseeninpracticewhenusingLikertscales
specificallyitelaboratesontheinherentproblemsintreatingLikertscaleresultdataasinterval
datawhenitshouldgenerallybetreatedasordinaldata
alsoservesasaconcisesummaryofotherworkintheareaexpressingsimilarconcerns
KuzonWM.Jr,UrbanchekMB.,andMcCabeS.Thesevendeadlysinsofstatisticalanalysis.AnnPlastic
Surg37(1996):26572

(includedforcompleteness)theoriginalpaperthatJamiesonparaphrasesinherpaper(referenced
above)whichIthenquoteinthisreport
Likert,Rensis."ATechniquefortheMeasurementofAttitudes."ArchivesofPsychology140(1932):155.

Dr.Likertsoriginalpublicationaboutthescalesthatwouldlatercometobearhisname
unfortunatelyIwasunabletoobtainacopyofthispaperindigitalformat(orotherwise)dueto
theageofitspublication
Mogey,Nora."SoYouWanttoUseaLikertScale?"LearningTechnologyDisseminationInitiative.25Mar.
1999.HeriotWattUniversity.20Oct.2007
<http://www.icbl.hw.ac.uk/ltdi/cookbook/info_likert_scale/index.html>.

pagereferencedfromtheWikipediaarticle(seetheresourcessectionofthisreport)
offersaconcise,highleveloverviewofLikertScalesaswellasthedescriptiveandinferential
techniquesthatcanbeappliedtothem
disappointinglylightonspecificswhenitcomestoexamplesandanalysis
PageBucci,Hilary."TheValueofLikertScalesinMeasuringAttitudesofOnlineLearners."HKADesigns.Feb.
2003.20Oct.2007<http://www.hkadesigns.co.uk/websites/msc/reme/likert.htm>.

areportonthevirtuesofLikertscalesinthecontextofmeasuringtheattitudesofonlinelearners
outlinesabriefoverviewandcomparisonofvariousrelatedscales
discussesadvantages,disadvantagesandsomeofthereliabilityandvalidityconcerns

LikertScales

9|P a g e

CPSC681TopicReport

DaneBertram

Shneiderman,Ben.DesigningtheUserInterface:StategiesforEffectiveHumanComputerInteraction.3rded.
AddisonWesleyLongman,Inc.,1998.136143.

providesexcerptsfromaquestionnaireforuserinterfacesatisfactionwhichshowsanumberof
LikertscaletypequestionsandhowtheycanbeappliedtotheHumanComputerInteractionfield

Uebersax,JohnS."LikertScales:DispellingtheConfusion."StatisticalMethodsforRaterAgreement.31Aug.
2006.20Oct.2007<http://ourworld.compuserve.com/homepages/jsuebersax/likert.htm>.

doesagreatjobofdefininganddifferentiatingthevarioustermsusedinconnectionwithLikert
scales
givesconcreteexamplesofeachvariationandpointedargumentsfortherisksinvolvedinthe
commonmisguidedassumptionsofordinalvs.intervaldata

LikertScales

10|P a g e