Rankingmethods

WarrenBuffetofferedabilliondollarsfora perfectNCAAmensbasketballbracket. Noonewon. Likely,hecouldoffertheprizefor400years beforesomeonewon. Thedifficultyisithadtobeperfect,butlotsof matchesare5050. However,agoodrankingmethodwillhelpget manypredictionscorrect(evenifitcanthelp call5050matches).

(c)2014T.RussellHanes

Rankingmethods

(c)2014T.RussellHanes

Rankingmethods

(c)2014T.RussellHanes

Rankingmethods

(c)2014T.RussellHanes

Rankingmethods

Thisisanetworkgraphofasimple,threeteam tournament. TheCondorcetcriterionisthatthewinner shouldbethewinnerofallheadtohead comparisons. Here,AdominatedB(notaboutscore,justthe termweuseinsteadofwinsover). BdominatedC.AndAdominatedC. Therefore,AistheCondorcetwinner. Thisseemsverysimple,butrealliferanking methodsviolatetheCondorcetcriterionallthe time.

(c)2014T.RussellHanes

Rankingmethods

Firstpastthepostelectionscanviolatethe Condorcetcriterion. Here,wewouldnormallysaycandidateAwins theelectionbecauseitreceivedthemostfirst placevotes. However,theCondorcetcriterionsaysweshould lookatheadtoheadcomparisonsonly. IfAhadnotrun,his43voteswouldsplit betweenBandC,30to13.WithoutAinthe race,candidateBwouldget67votes,andC wouldget33. Letslookatallthepairwisecomparisons.

(c)2014T.RussellHanes

Rankingmethods

Forexample,withoutcandidateB,her37voteswouldgo toAandC,10to27.Awouldstillwin,butjustbarely. ThemostinterestingpointisthatwithoutC,all20ofhis voteswouldgotoB,andcandidateBwouldwin. Therefore,BisactuallytheCondorcetwinner,notA.C actedasaspoiler. Lotsofvotingmethodshavebeeninventedtoensure theCondorcetwinneractuallywins. Thesimplestistheinstantrunoff.Peoplemarkafirst andsecondchoiceontheirballots.Thelowestplace candidatesvotesthengetsenttotheirsecondchoice. Inthisexample,Cwouldbeeliminated,andhisvotes wouldgotoB,andBwouldwin.

(c)2014T.RussellHanes

Rankingmethods

Anadjacencymatrixshowsdominances.For example,row1,column2showsthatB dominatedA. Thisistheadjacencymatrixforourelection example. ThismatrixshowsthereisaclearCondorcet winnerbecauseitcanbewritteninupperright triangularform(nodominancesbelowthe upperlefttobottomright diagonal).

(c)2014T.RussellHanes

Rankingmethods

Adjacencymatricescanbeusedtoencodeall sortsofnetworkgraphs.Hereisanetworkgraph oflifts(uparrows)andruns(coloredlines)at Mt.HoodMeadowsskiresortinOregon.

(c)2014T.RussellHanes

Rankingmethods

Herearethelifts.Row1,column2showsaskier cantakealiftfrompointAtopointB.

(c)2014T.RussellHanes

10

Rankingmethods

Andherearetheruns. Adjacencymatricesallowpathstobemapped fromlifttoruntorunbacktoalift. AdjacencymatriceslikethesearebehindGoogle Mapsdirections.

(c)2014T.RussellHanes

11

Rankingmethods

HowdoestheCondorcetcriterionapplytofigure skating? Onedifficultyisthatjudgescantbeallowedtojudge competitorsfromtheirowncountry. Here,fournationsarecompeting.CompetitorsA,B,C andDarefromfourdifferentnations. JudgesA,B,C,andDarefromthesamenations. Therefore,onlyjudgesB,C,andDcanjudgecompetitor A. Theotherdifficultyisthatscoresaresubjective. JudgeAseemstobeloosewithpoints,whilejudgeDis stingy.SincejudgeAdoesnotjudgecompetitorA,that seemstoputcompetitorAatadisadvantage. Pointstotals,justlikefirstplacevotes,isaflawed method.

(c)2014T.RussellHanes

12

Rankingmethods

Thesolutionistolookatheadtohead comparisons.JudgesCandDbothjudged competitorsAandB,andbothjudgessaidAwas thesuperiorskater.SoAdominatesB,andwe canignorethejudgesactualpoints. Fillinginallheadtoheadcomparisons,wesee thatAistheCondorcetwinner.

(c)2014T.RussellHanes

13

Rankingmethods

Ifwelookattotalpoints,yes.JudgeBhas cheatedtomakecompetitorBthewinnerof totalpoints. Butno,onejudgecannoteffectivelycheatifwe lookatheadtoheadcomparisons.AlljudgeB hasdoneisgivencompetitorsAandChisone tiedvote.

(c)2014T.RussellHanes

14

Rankingmethods

HereistherevisednetworkgraphwithjudgeBs attempttocheat.CompetitorAstilldominates competitorC,becausejudgeDvotedforA. AisstilltheCondorcetwinner.

(c)2014T.RussellHanes

15

Rankingmethods

(c)2014T.RussellHanes

16

Rankingmethods

Atie,ormorespecificallyaCondorcetparadox, occurswhenthereisnoCondorcetwinner. Innetworkgraphterms,werefertoitasacycle.

(c)2014T.RussellHanes

17

Rankingmethods

Considerthiscycleintermsoftheadjacency matrix.

(c)2014T.RussellHanes

18

Rankingmethods

Thereisnowaytoreranktheteamstoavoid onedominancebeingoutsideupperright triangularform. Wehavetooverrideoneresulttorankthe teams i.e.,breakthetie.

(c)2014T.RussellHanes

19

Rankingmethods

Tiebreakingbytotalpointsis,justlikeelections andfigureskating,asillymethod. Bwouldwinundertotalpoints,butBwas clobberedbyA. Instead,lookatthepointdifferentials.

(c)2014T.RussellHanes

20

Rankingmethods

Thepointdifferentialsshowthattheresultwe shouldoverrideisCsdominanceofA.This resultisthemostlikelytohavebeenanupsetor fluke.

(c)2014T.RussellHanes

21

Rankingmethods

(c)2014T.RussellHanes

22

Rankingmethods

Hereitis.Onecanusepointdifferentialstoresolve thecycle.

(c)2014T.RussellHanes

23

Rankingmethods

(c)2014T.RussellHanes

24

Rankingmethods

WhathappensifyoubreakthetieBCDby flippingCD?YoucreateaNEWcycle. Wehavetocomeupwithmoresophisticated methods.Ratherthanbreakingcyclesoneby one,wehavetofindwaystoranktheteamsthat MINIMIZEthetotalnumberofcycles.

Note:Wewillusethisrunningexample throughslide48.

(c)2014T.RussellHanes

25

Rankingmethods

Isitbettertohaveoneextremeupsettooverride,like theresultontheleft?Inthatranking,theworstteam beatthebestteam. Isitbettertohavemultiple,smallerupsetsto override,liketheresultontheright?Inthatranking, thesecondbestteambeatthefirstbest,thethird bestbeatthesecondbest,andtheworstbeatthe thirdbest. Theresnoaprioriwaytodecidewhichoneisbetter. ItdependsonhowbigthegapisbetweenDandB.If Bismuch,muchweaker,theoutcomeontheleftis quiteunlikely.IfDandBareevenlymatched,thenthe outcomeontheleftismorelikelythantheoutcome ontheright.

(c)2014T.RussellHanes

26

Rankingmethods

Theprobabilityofupsetsfollowsalogisticcurve.Ifthe differenceinteamsstrengthsiscloseto0,thenitisa 5050match.Ifthedifferenceinstrengthsislarge,the probabilityofanupsetdropsdramatically. Insomesports,thecurveismuchsharperthaninothers. Infootball,forexample,therearealotofupsets.Even weakteamssometimesknockoffmuchstrongerteams inpartbecausethediscrepancyinstrengthfrombest teamtoworstteamisnotthatgreat. Rankingmethodsusevariouswaystoextractfrom winsandpointsthestrengthsofeachteam.Ifamethod findsgreatdisparitiesinstrengths,extremeupsetsare unlikely.Ifamethodfindsminimaldisparitiesin strengths,extremeupsetsmightoccurmorefrequently.

(c)2014T.RussellHanes

27

Rankingmethods

Thismethodissimple,easytoimplementinExcelor codeintoothersoftware,andtransparent.Ithasa highlikelihoodtobeacceptedasanofficialtie breakerforsomekindsoftournaments. Itissomewhatsensitivetoupsets. Thisisamethodofmyowninvention.Themethod factorsinopponentstrength.

(c)2014T.RussellHanes

28

Rankingmethods

Thetournamentresultsareintheupperleftadjacencymatrix. Foreachwin,ateamcanearnadditionalpointsforitsdefeated opponentswins.Lookattheupperrightmatrix.Forexample,A beatBandC.TeamsBandCbothwononematch,soAgetsone additionalpointforeachwin:2weightedwins. TeamDbeatAandC.BeatingteamA(21)isworth2additional points;beatingCisworthonly1.SoDgets3weightedwins. Thesameforlosses:ateamcanloseadditionalpointsforits defeatingopponentslosses.Lookatthelowerleftmatrix.For example,teamBlosttoAandC.LosingtoA(21)isworth1 additionalloss;losingtoC(12)isworth2additionallosses. Therefore,teamBhas3weightedlosses.(Readincolumn3) Addtheoriginalwinsandlossestotheweightedwinsandlosses toseethefinalrankings.Translatedintoathreeround tournament,Agoes21,Dgoes1.91.1,etc.

(c)2014T.RussellHanes

29

Rankingmethods

MarkovchainsaresimpletoimplementinExcel orothersoftware. However,itisnotverytransparentandunlikely tobeusedasanofficialtiebreaker. Itisverysensitivetoupsets.Evenupsetsatthe bottomcanchangerankingsatthetop.

(c)2014T.RussellHanes

30

Rankingmethods

TheideabehindMarkovchainsissimple:eachwin isavoteateamcastsforitself,whileeachlossis avotefortheteamthatbeatsit.Thesevotes areexpressedasprobabilities. Forexample,afairweatherfanwhostartedout withteamAwouldhavea2/3chanceofsticking withAanda1/3chanceofdefectingtoteamD. Notecarefully:Thedirectionofthearrowsis reversedfromaregularnetworkgraph. Afterwanderingaroundalltournament,whereis thefairweatherfanlikelytoendup?

(c)2014T.RussellHanes

31

Rankingmethods

Thetopmatrix[m]loadsintheprobabilitiesfor ourfairweatherfan. Thebottommatrixis[m]^n,wherenisavery largenumber. Thebottommatrixindicatesthatthefairweather fanhasapproximatelya30%chanceofendingup withteamA,a40%chanceofendingupwithD, etc. Theseprobabilitiesareusedtogeneratethe ranking.Theprobabilitiesalsoindicatesome measureofrelativestrength.

(c)2014T.RussellHanes

32

Rankingmethods

Ofcourse,wecoulduseMarkovchainswithpoint differentialsaswell.Herearepossibledifferentials forourtournament.

(c)2014T.RussellHanes

33

Rankingmethods

(c)2014T.RussellHanes

34

Rankingmethods

HereisthegraphfortheMarkovchainbasedon differentials.

(c)2014T.RussellHanes

35

Rankingmethods

Thetopmatrix[m]istheprobabilitiesforour fairweatherfan,basedonpointdifferentials. Thebottommatrixis[m]^n,wherenisalarge number. Thebottommatrixshowsthefairweatherfan hasa55%chanceofstayingwithA,a27% chanceofstayingwithD,etc. Inthisranking,Acomesoutontopduetothe largemarginsitranup.Aisratedfarhigherthan DthanDisratedthanB.

(c)2014T.RussellHanes

36

Rankingmethods

Thesemethodsrequiresomebasiclinearalgebra, but theycanbeimplementedinExcel. ThesemethodsareusedbytheBCSandother organizationstopickpostseasonteams;however, giventhelackoftransparency,thesemethodsare unlikelytobeadoptedastiebreakersforsimpler tournaments.

(c)2014T.RussellHanes

37

Rankingmethods

TheMasseymethodusespointdifferentials.The righthandmatrixispoint differentialsforthefirst threeteams,andfinally,thelastteamsdifferential issetto0. tistotalgamesplayed. gistheindividualmatchesanytwoteamsplayed. Thefinalrowofthelefthandmatrixissettoall1s. Thepurposeistomakealltheratingssumto0. Thersarethefinalratingsfortheteams.

(c)2014T.RussellHanes

38

Rankingmethods

TheteamsareinorderA,D,B,C. Nowweneedtoinvertthelefthandmatrixand lefthandmultiplyonbothsidestofindourratings.

(c)2014T.RussellHanes

39

Rankingmethods

(c)2014T.RussellHanes

40

Rankingmethods

TheColleymethoddoesnotusepointdifferentials, onlywinsandlosses. The meaningoft,g,andrarethesameasinthe Masseymethod.

(c)2014T.RussellHanes

41

Rankingmethods

OurrunningexampleintheorderA,D,B,C. Now weneedtoinvertthelefthandmatrixand lefthandmultiplyonbothsidestofindourratings.

(c)2014T.RussellHanes

42

Rankingmethods

TheColleymethoddoesnotbreakthetiesbetween AandDandbetweenBandC.Inmorecomplex tournaments,theColleymethodwouldbreakthese ties.

(c)2014T.RussellHanes

43

Rankingmethods

Thismethodmakesalotofsensewhenteams clearlyhavebivariatestrength.Footballisaperfect example:theoffensiveteamanddefensiveteamare literallytwodifferentsetsofplayers.Thismethod workswellforbaseball:battingandpitching/fielding aretwoverydifferentskillsets. Thegoalofthismethodisbreakascoreintothe contributionmadebytheoffense(aboveorbelow averageeffort)andtheopposingdefense(aboveor belowaverageeffort). Thisisunlikelytobeusedasanofficialtiebreaker, butitiseasytousetoadjustmatchesmid tournament.

(c)2014T.RussellHanes

44

Rankingmethods

Herearesixteams thathaveplayedtwoopponents each. Thescoresforeachoffensearerecordedbyrow.For example,Ascored10pointsagainstC,whereasin thesamegame,Cscored5pointsagainstA. ThereforeAwon. Thewinnerofeachmatchishighlightedinorange. Theaverageoffense(pointsscored)andaverage defense(pointsallowed)arerecordedforeachteam.

(c)2014T.RussellHanes

45

Rankingmethods

Eachteamspointsscoredineachgameisdividedby itsaveragepoints scored. Thus,Asscoreof10pointsagainstCisdividedbyAs averageof8.5toproducethe1.176.Asscoreagainst CisaboveAsaveragescore.AsscoreagainstEis belowAsaveragescore. Theadjusteddefensiveratingsarecalculatedfor eachteam. Forexample,incolumn1,AsdefensekeptCandE bothtobelowtheiraveragescores.Thus,Asdefense isstrong. CsdefenseisweakbecausebothAandEranup aboveaveragescores.

(c)2014T.RussellHanes

46

Rankingmethods

Wedividetheoriginalscoresbythoseadjusted defensiveratings. AsscoreagainstCdrops from10to8.876,because Csdefenseisweak. Weusetheseadjustedscorestocalculateadjusted offensivescores. Asoveralloffensivescorecomesdownbecauseboth ofitsopponentswereweak. Wecanreiteratetheprocessacoupletimesuntilthe defensiveratingsandoffensivescoresstabilize.

(c)2014T.RussellHanes

47

Rankingmethods

We stageallthehypotheticalgames. Ineachgame,wefindthescorebymultiplying adjustedoffensescoreagainstadjusteddefensive rating.Forexample,Asadjustedoffensivescore againstBsdefensiveratingindicatesAmightscore 7.82points. However,BsoffensetimesAsdefensiverating indicatesBmightscore9.9points,thuswinning. Actualwinsareinorange;hypotheticalwinsarein yellow.

(c)2014T.RussellHanes

48

Rankingmethods

(c)2014T.RussellHanes

49