Professional Documents
Culture Documents
Amortized Analysis Explained by Rebecca Fiebrink Princeton University
Amortized Analysis Explained by Rebecca Fiebrink Princeton University
Contents Introduction Overview Keyideas History Comparisontootheranalysistechniques Thebasics Thethreeapproachestoamortizedanalysis Theaccountingmethod Overview Example Thepotentialmethod Overview Example Comparingtheaccountingandpotentialmethods Indepthexamples Movetofront Splaytrees Moreexamples Redblacktrees Fibonacciheaps Disjointsets Maximumflow Dynamicarrays/hashtables Scapegoattrees Conclusions References
Introduction Overview Amortizedanalysisisatechniqueforanalyzinganalgorithm'srunningtime.Itisoftenappropriate whenoneisinterestedinunderstandingasymptoticbehavioroversequencesofoperations.For example,onemightbeinterestedinreasoningabouttherunningtimeforanarbitraryoperationtoinsert anitemintoabinarysearchtreestructure.Incasessuchasthis,itmightbestraightforwardtocomeup withanupperboundby,say,findingtheworstpossibletimerequiredforanyoperation,then multiplyingthisbythenumberofoperationsinthesequence.However,manyrealdatastructures,such assplaytrees,havethepropertythatitisimpossibleforeveryoperationinasequencetotaketheworst casetime,sothisapproachcanresultinahorriblypessimisticbound!Withabitofcleverreasoning aboutpropertiesoftheproblemanddatastructuresinvolved,amortizedanalysisallowsatighterbound thatbetterreflectsperformance. Keyideas Amortizedanalysisisanupperbound:it'stheaverageperformanceofeachoperationinthe worstcase. Amortizedanalysisisconcernedwiththeoverallcostofasequenceofoperations.Itdoesnot sayanythingaboutthecostofaspecificoperationinthatsequence.Forexample,itisinvalidto reason,TheamortizedcostofinsertionintoasplaytreewithnitemsisO(logn),sowhenI insert'45'intothistree,thecostwillbeO(logn).Infact,inserting'45'mightrequireO(n) operations!Itisonlyappropriatetosay,WhenIinsertmitemsintoatree,theaveragetimefor eachoperationwillbeO(logn). Amortizedanalysisisconcernedwiththeoverallcostofarbitrarysequences.Anamortized boundwillholdregardlessofthespecificsequence;forexample,iftheamortizedcostof insertionisO(logn),itissoregardlessofwhetheryouinsertthesequence'10,''160,''2'orthe sequence'2','160','10,''399',etc. Thetwopointsaboveimplythatbothamortizedandworstcaseboundsshouldbeunderstood whenchoosinganalgorithmtouseinpractice.Becauseanamortizedboundsaysnothingabout thecostofindividualoperations,itmaybepossiblethatoneoperationinthesequencerequires ahugecost.Practicalsystemsinwhichitisimportantthatalloperationshavelowand/or comparablecostsmayrequireanalgorithmwithaworseamortizedcostbutabetterworstcase peroperationbound. Amortizedanalysiscanbeunderstoodtotakeadvantageofthefactthatsomeexpensive operationsmaypayforfutureoperationsbysomehowlimitingthenumberorcostof expensiveoperationsthatcanhappeninthenearfuture. Ifgoodamortizedcostisagoal,analgorithmmaybedesignedtoexplicitlyperformthisclean upduringexpensiveoperations! Thekeytoperformingamortizedanalysisispickingagoodcreditorpotentialfunctionthat captureshowoperationswithdifferentactualcostsaffectadatastructureandallowsthedesired boundstobeshown. History Infinance,amortizationreferstopayingoffadebt,suchasaloanormortgage,bysmallerpayments madeovertime.Thepaperintroducingamortizedanalysisforalgorithmsasageneraltechniquewas publishedbyTarjanin1985,thoughherecognizespriorapplicationofthebanker'smethodinworkby BrownandTarjan(1980)andHuddlestonandMelhorn(1981,1982)andcreditsSleatorwiththe
conceptofthepotentialmethod.Aggregateanalysisisanoldermethod(Ahoetal.1974)thatcannow beunderstoodasaformofamortizedanalysis. Earlyapplicationsofamortizedanalysisincludedthemovetofrontlistupdatingheuristic (SleatorandTarjan1985a),selfadjustingandbalancedbinarytrees(Tarjan1985,SleatorandTarjan 1985b),andpathcompressionheuristicsforthedisjointsetunionproblem(Tarjan1983,1985). Amortizationplaysanimportantroleintheanalysisofmanyotherstandardalgorithmsanddata structures,includingmaximumflow,Fibonacciheaps,anddynamicarrays.Alltheseapplicationsof amortizedanalysisarediscussedbelow.Amortizedanalysishasrecentlybeenusedindiverse applicationsincludingsecurity(Maoetal.2006),databases(Agarwaletal.2006),anddistributed computing(FomitchevandRuppert2004),anditcontinuestobeimportanttotheoreticalworkon algorithmsanddatastructures(e.g.,Mendelsonetal.2006;Georgiadisetal.2006). Comparisontootheranalysistechniques Asmentionedabove,worstcaseanalysiscangiveoverlypessimisticboundsforsequencesof operations,becausesuchanalysisignoresinteractionsamongdifferentoperationsonthesamedata structure(Tarjan1985).Amortizedanalysismayleadtoamorerealisticworstcaseboundbytaking theseinteractionsintoaccount.Notethattheboundofferedbyamortizedanalysisis,infact,aworst caseboundontheaveragetimeperoperation;asingleoperationinasequencemayhavecostworse thanthisbound,buttheaveragecostoveralloperationsinanyvalidsequencewillalwaysperform withinthebound. Amortizedanalysisissimilartoaveragecaseanalysis,inthatitisconcernedwiththecost averagedoverasequenceofoperations.However,averagecaseanalysisreliesonprobabilistic assumptionsaboutthedatastructuresandoperationsinordertocomputeanexpectedrunningtimeof analgorithm.Itsapplicabilityisthereforedependentoncertainassumptionsaboutprobability distributionsonalgorithminputs,whichmeanstheanalysisisinvalidiftheseassumptionsdonothold (orthatprobabilisticanalysiscannotbeusedatall,ifinputdistributionscannotbedescribed!)(Cormen etal.2001,923).Amortizedanalysisneedsnosuchassumptions.Also,itoffersanupperboundonthe worstcaserunningtimeofasequenceofoperations,andthisboundwillalwayshold.Anaveragecase bound,ontheotherhand,doesnotprecludethepossibilitythatonewillgetunluckyandencounteran inputthatrequiresmuchmorethantheexpectedcomputationtime,eveniftheassumptionsonthe distributionofinputsarevalid.Thesedifferencesbetweenprobabilisticandamortizedanalysis thereforehaveimportantconsequencesfortheinterpretationandrelevanceoftheresultingbounds. Amortizedanalysisiscloselyrelatedtocompetitiveanalysis,whichinvolvescomparingthe worstcaseperformanceofanonlinealgorithmtotheperformanceofanoptimalofflinealgorithmon thesamedata.Amortizationisusefulbecausecompetitiveanalysis'sperformanceboundsmusthold regardlessoftheparticularinput,whichbydefinitionisseenbytheonlinealgorithminsequencerather thanatthebeginningofprocessing.SleatorandTarjan(1985a)offeranexampleofusingamortized analysistoperformcompetitiveanalysis. Thebasics Thethreeapproachestoamortizedanalysis Thereexistthreemainapproachestoamortizedanalysis:aggregateanalysis,theaccountingmethod, andthepotentialmethod(Cormenetal.2001,p.405).Aggregateanalysisisasimplemethodthat involvescomputingaboundonasequenceofoperations,thendividingbythenumberofoperationsto obtaintheamortizedcost.Alloperationsareassignedthesameamortizedcost,eveniftheyareof differenttypes(e.g.,addingoneitemtoastackorpoppingmultipleitemsfromastack)(p.40610).
Theothertwomethodsaresomewhatmoreinterestingandpowerful,andbyreasoningaboutparticular localorglobalpropertiesofthedatastructures,theyallowforassigningdifferentamortizedcoststo differenttypesofoperationsinthesamesequence. Theaccountingmethod Overview Theaccountingmethod,orthebanker'sview,assignschargestooperationsasifthecomputerwere coinoperated.Onecanthinkofeachoperationalwaysbeingaccompaniedbyinsertingoneormore coinsintothecomputertopayfortheoperation,accordingtoapredeterminedchargeforthatoperation type(e.g.,astackpushmighthaveonecharge,andapopofseveralitemsfromthestackmighthave anothercharge).Thechargedoesnotnecessarilycorrespondtotheactualtimerequiredforthe particularoperation;itispossiblethattheoperationwillcompleteinlesstimethanthetimecharged,in whichcasesomepositiveaccumulationofcreditisleftafterthetimerequiredfortheoperationtofinish issubtractedfromtheoperationpayment.Or,itispossiblethattheoperationwillneedmoretimethan thetimecharged,inwhichcasetheoperationcanbepaidforbypreviouslyaccumulatedcredit. Intheaccountingmethod,theamountchargedforeachoperationtypeistheamortizedcost forthattype.Aslongasthechargesaresetsothatitisimpossibletogointodebt(i.e.,onecanshow thattherewillneverbeanoperationwhoseactualcostisgreaterthanthesumofitschargeplusthe previouslyaccumulatedcredit),theamortizedcostwillbeanupperboundontheactualcostforany sequenceofoperations.Therefore,thetricktosuccessfulamortizedanalysiswiththeaccounting methodistopickappropriatechargesandshowthatthesechargesaresufficienttoallowpaymentfor anysequenceofoperations. Moreformally,denotetheactualcostoftheithoperationby c i andtheamortizedcost(charge) i .If c ic i ,theithoperationleavessomepositiveamountofcredit, oftheithoperationby c credit i=c i ci thatcanbeusedupbyfutureoperations.Andaslongas
n i =1 n
ci c i
i= 1
,(1)
thetotalavailablecreditwillalwaysbenonnegative,andthesumofamortizedcostswillbeanupper boundontheactualcost(Cormenetal.2001,p.410). Itiscommontokeeptrackofstoredcreditbyassigningittoaparticularlocationinthedata structure,forexamplesomepartofthestructurethatwillbealteredlaterbysomeexpensiveoperation thatwas(partially)madepossiblebytheoperationthatprovidedthecredit.Thislocationcouldbea nodeinatree,apositioninastack,orsomethingelse.Differenttypesofoperationsthatprovidecredit mightstoretheircreditindifferentstructures,butasingleoperationtypewillhaveasingle,well definedlocationtostoreitscredit.Reasoningaboutwhencreditisproduced,whereitisstored,and howitisusedupiskeytoshowingthatEquation1holdsforalloperationsequences.Finally,remember thatamortizationisananalyticaldeviceonly!Thecreditsstoredinthedatastructureareimaginary toolsweusetoreasonaboutthebehaviorofanalgorithmordatastructure,notsomethingthatis explicitoraccessibleinadatastructureabstractionoritsimplementationcode. Example Thisfirstexamplesuppliesatrivialillustrationoftheaccountingmethod,adaptedfromTarjan(1985) andCormenetal.(2001,p.410).Supposeyouhaveasimplestackofitems,whereyoucanpushanitem ontothestackorpopanitemfromthestackinconstanttime.Thestackisconstrainedtonaturally
contain0ormoreitemsatalltimes.Supposethereisonlyoneoperationtypedefinedforthisstack, calledOP,whichinvolvesapplyingzeroormorepopsfollowedbyonepush:
OP(n){ //requiresatleastnitemsareonthestack popnitemsfromstack push1itemontostack
Figure1:OP(3)appliedtoastack Alsosupposethatmsuchoperationshavebeenperformed.TheworstcaserunningtimeforanOP operationinthesequenceisO(m),whichoccursifoperations1throughm1haveeachbeenOP(0) (eachpoppingnoitemsandpushingone),andthemthoperationisOP(m1)(poppingallitemsfrom thestack,thenpushingone).Asimpleworstcaseanalysiswouldconsidermoperationswithaworst caseperoperationcostofO(m)tohaveanupperboundofO(m2). ItshouldbeintuitivelyobviousthatnoteverysingleOPoperationcanhaveacostofm,since previousoperationshavetopushitemsontothestackbeforetheycanbepopped!Amortizedanalysis capturesthisintuition.ConsiderassigningeachOPachargeof2.ThefirstOPmustbeOP(0),popping nothingfromtheemptystackandpushingoneitem.Theactualcostofthisoperationis1,andthe amortizedcostis2,sothecreditremainingis21=1.Thiscreditof1isstoredwiththenewitemin thestack.Ifthisitemispoppedfromthestackinthefuture,thepopwillbepaidforbythisstored1 credit.Inthisway,anysequenceofOPsmaintainstheinvariantthatthereisonecreditstoredper stackeditem.Thealgorithmwillneverrunoutofcredit,becauseeachpopisalreadyprepaid,andeach pushonlyusesup1ofthetwocreditspaidtheOP.TheamortizedcostofeachOPis2,sotheamortized costofanysequenceofmOPsis2m,whichisO(m).ThisisquiteanimprovementovertheO(m2) analysis!
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
Ifadatastructurepossesseslittleornostoredcredit,thisisanindicationthatthebadness resultingfromasubsequentparticularoperationhavingactualcostthatexceedsitsamortizedcostis limited,asthestoredcreditcannotfallbelow0.Ontheotherhand,ifadatastructurepossessesmuch storedcredit,itmightbepossiblethatasubsequentoperationmayhaveactualcostthatfaroutweighs itsamortizedcost,asitmayeatuptheprepaymentsofthepreviousoperations. Thepotentialmethod Overview Thepotentialmethod,orphysicist'sview,definesafunctionthatmapsadatastructureontoareal valued,nonnegativepotential(Tarjan1985;Cormenetal.2001,p.412).Thepotentialstoredinthe datastructuremaybereleasedtopayforfutureoperations.Inthisway,thepotentialissimilarto consideringthetotalcreditstoredthroughoutadatastructureusingtheaccountingmethod.However, thepotentialmethodhasafewotherconceptualdifferencesfromtheaccountingmethod,asdiscussed below. Letusrefertothedatastructureattimei(oralternatively,justafteroperationi),asDi.The potentialfunction( Di)mapsDiontoarealvalue.Again,denotetheactualcostofoperationias c i . Inthepotentialmethod,theamortizedcostofoperationiisequaltotheactualcostplusthe increaseinpotentialduetotheoperation: c i=c i Di Di 1 (2) Thatis,inperforminganalysis,theamortizedcostandpotentialfunctionmustbedefinedsothatthis equivalencealwaysholds.Thisequivalenceimpliesthatanoperationwhoseactualcostislessthanits amortizedcostresultsinanoverallincreaseinpotential,andanoperationwhoseactualcostisgreater thanitsamortizedcostrequiresadecreaseinpotential.Adecreaseinpotentialthereforepaysfor particularlyexpensiveoperations,justasadecreaseincreditsdidfortheaccountingmethod. Justasintheaccountingmethod,wemustenforceanadditionalconditiontoensurethatthe totalamortizedcostofasequenceisalwaysanupperboundonthetotalactualcost.Byequation2,we canderivearelationshipbetweenthetotalamortizedandactualcosts:
n i=1 n
ci= c i Di Di 1
i=1
(3)
ci= c i Dn D 0
i= 1
(4)
Toensurethatthetotalamortizedcostisanupperboundonthetotalactualcost,forasequenceofany length,wemerelymustensurethat Di D0 foralli.Typically,( D0)isdefinedtobe0,and thepotentialisdefinedsothatitisalwaysnonnegative(Cormenetal.2001,p.413). Example Consideragainthesimplestackexampleintroducedabove.Anaturalpotentialfunctionis( Di)=the numberofitemsinthestackafteroperationi.ConsiderperformingOP(k)asoperationi,which performskpopsandonepush.OP(k)hasanactualcostofc=k+1.Ifthestackoriginallycontained nitems,thepotentialbeforetheoperationis( Di1)=n,andthepotentialaftertheoperationis( Di)= (nk+1).TheamortizedcostofOP(k)istherefore:
c i=c i Di D i 1 = k 1 n k 1 n =2 .Furthermore,itisclearthatthepotential definedthiswayisnevernegative.Therefore,wecanconcludethattheamortizedcostofOPis2 (O(1)),andthetotalamortizedcostforanysequenceofOPoperationsisanupperboundontheactual costofthesequence. Inpractice,potentialfunctionsareoften,unsurprisingly,abitmorecomplicated.Forexample, onemightwishtodefinethepotentialbasedonmorecomplicatedpropertyofthedatastructure.Also, itmaybenecessarytoenumerateallthepossibleoutcomesofanoperationtoshowthatequation2 holdsforeachofthem.Suchslightlymorecomplicatedapplicationsofthepotentialmethodappearin theindepthexamplesbelow.
Comparingtheaccountingandpotentialmethods Theaccountingmethodchargeseachoperationacertainamountaccordingtoitstype(e.g.,insertor delete),puttingthefocusoneachoperation'sprepaymenttooffsetfutureexpensiveoperations.The potentialmethodinsteadputsthefocusontheeffectofaparticularoperationataparticularpointin time,basedontheeffectsoftheoperationonthedatastructureandthecorrespondingchangein possibilityforfutureoperationstoincurexpense.Theaccountingmethodstorescreditinparticular areasofadatastructure,suchasnodesofatreeoritemsinastack,whereasthepotentialmethod generallyconsiderspropertiesoftheentiredatastructure(suchashowmanynodesareinatree,orhow manyitemsareinthestack)tocomputethepotential. Theaccountingmethodandpotentialmethodareequivalentintermsoftheirapplicabilityto particularproblemsandtheboundstheyprovide(Tarjan1985).However,oneortheothermaybeeasier toapplytoagivenproblem.Tarjansuggeststhatthepotentialmethodmaybemorenaturalwhen fractionalamountsoftimemustbeconsidered,whiletheaccountingmethodislessabstract. Indepthexamples Inthissection,wetakeastepbysteplookatapplyingamortizedanalysistosomerealproblems.The readerisalsoencouragedtoreadChapter17ofCormenetal.(2001),whichprovidesadditionalsimple examples. Movetofront Thefirstexampleappliesthepotentialmethodtothemovetofront(MTF)heuristicforlistaccess andupdating,adaptedfromTarjan1985andSleatorandTarjan1985a(thereaderisreferredtothe originalsourcesformoreinformationonthebackgroundofthisproblem,andaverydetailedanalysis anddiscussionofthealgorithm).AspectsofthisexamplearealsodrawnfromtheCOS423Spring 2000handout(Tarjan2000). TheproblemMTFaddressesisthis:Consideralinearlistofitems(suchasasinglylinkedlist). Toaccesstheitemintheithpositionrequirestimei.Also,anytwocontiguousitemscanbeswappedin constanttime(notincludingthetimetoaccesstheminthelist).Thegoalistoallowaccesstoa sequenceofnitemsinaminimalamountoftime(oneitemmaybeaccessedmanytimeswithina sequence),startingfromsomesetinitiallistconfiguration.Ifthesequenceofaccessesisknownin advance,onecandesignanoptimalalgorithmforswappingitemstorearrangethelistaccordingtohow oftenitemsareaccessed,andwhen.However,ifthesequenceisnotknowninadvance,aheuristic methodforswappingitemsmaybedesirable.Thisproblemisoneformulationofthestandard dictionaryproblem(SleatorandTarjan1985a,p.203). TheMTFisaheuristicthattakesadvantageofthefactthat,forrealproblemssuchaspaging,if itemiisaccessedattimet,itislikelytobeaccessedagainsoonaftertimet(i.e.,thereislocalityof
a b c d e
a b c d e
a c b d e
c a b d e
Figure3:Movetofront:Toaccess'c'inthe originallist(left),walkdownfrom'a',thenmove 'c'tofrontbyswappingwith'b'then'a'(right) AmortizedanalysiscanbeusedtoshowthatMTFalwaysperformswithinafactorof4ofany algorithmevenanoptimalalgorithmthatknowstheaccesssequenceinadvance,andevenwithout makingassumptionsaboutlocalityofreference.Here,toanalyzewithrespecttoanarbitrary(perhaps optimal)algorithmA,definethepotentialofMTFattimetasthetwotimesnumberofpairsofitems whoseorderintheMTF'slistdiffersfromtheirorderinA'slistattimet.Forexample,ifMTF'slistis ordered(a,b,c,e,d)andA'slistisordered(a,b,c,d,e),thenthepotentialforMTFwillbeequalto2, becauseonepairofitems(dande)differintheirorderingbetweenA'slistandMTF'slist.Thepotential att=0is0,asbothalgorithmsbeginwiththesamelistbydefinition.Also,itisimpossibleforthe potentialtobenegative.Therefore,thetotalamortizedcostisanupperboundonthetotalactualcostof anyaccesssequence. Now,considertheanalysisofaccessingasingleitem,x.LetxbeatpositionkinMTF'slistand atpositioniinA'slist.ThecosttoMTFofaccessingxandmovingittothefrontis2(k1).Thecost forAtoaccessxisi.Notethatmovingxtothefrontofthelistreversestheorderingofallpairs includingxandanitemoriginallyinlocation1tok1(i.e.,k1pairsintotal).Therelativepositions ofallotherpairsareunchangedbythemove.InA'slist,therearei1itemsaheadofx;allofthesewill bebehindxinMTF'slistoncexismovedtothefront.Therefore,thereareatmostmin{k1,i1} pairinversions(i.e.,disagreementsinpairorderbetweenMTFandA)thatareaddedbythemoveto frontofx.Allotherorderingreversals(atleastk1min{k1,i1})mustresultinpairinversion removals(i.e.,thepairorderingsnowagreebetweenMTFandA). Therefore,thepotentialchangeincurredinthissingleaccessandmovetofrontisbounded aboveby2(min{k1,i1}(k1min{k1,i1}))=4min{k1,i1}2(k1).Byequation 2,theamortizedcostofthissingleaccessisboundedaboveas: c = c 2 k 1 4min { k 1, i 1 } 2 k 1 4min {k 1, i 1 } 4 i So,theamortizedcostofasingleaccessandmovetofrontbyMTFisboundedabovebyfourtimesthe costoftheaccessbyA. OneadditionalconsiderationisthatAmightindependentlyperformswapsinresponsetoanew accessrequest,andthisisnottakenintoaccountabove.SayAdoesswaptwoitems.Thisincursno additionalactualcostonthepartofMTF,butitwillincreaseordecreasethenewpotentialby2,andit
willincreasethecostofaccessforAby1.TheboundonMTF'samortizedcoststillholds,sincethe amortizedcostisincreasedbyatmost2,buttheboundisincreasedby4.Thisistruenomatterhow manyswapoperationsAperforms. Splaytrees Thenextexampleappliesamortizedanalysistosplaytrees.Thispresentationoftheexampleassumes nopriorknowledgeaboutsplaytrees,astheyarequitestraightforwardstructureswhoseessential behaviorisexplainedbelow.Thereadershouldhaveabasicunderstandingofbinarysearchtrees, however,andshouldunderstandtheconceptofatreerotation.(Ifnothingelse,rememberthatarotation isamodificationtosomesubtreeofthesearchtree,whichpreservesthebinarysearchtreeorder propertybuthastheeffectofmovingsomenodestowardstherootandsomenodesawayfromtheroot by1.Asinglerotationcanbeperformedinconstanttimeusingastandardbinarytreeimplementation.) SplaytreesareaselfadjustingbinarysearchtreeintroducedbySleatorandTarjan(1985b).The valuesofthenodesareorderedsymmetrically,meaningthattheleftsubtreeofnodexcontainsnodes whosekeyvaluesarestrictlylessthanx,andtherightsubtreecontainsnodeswhosevaluesaregreater thanx.Splaytreesarenotbalancedtreeslikeredblacktrees;theirstructuremaybearbitrarilybad, (e.g.,Figure7),meaningthatanaccessonatreewithnnodesmayrequirenoperations.However, muchlikethemovetofrontheuristicdescribedabove,splaytreesmoveanelementtotherootofthe treethroughaseriesofsplaysteps(rotations).Thismovehastheeffectoflocalcleanupalongthe pathfromtheaccessednodetotheroot,whichtendstoreducethedepthofthetree,anditiswhat makessplaytreesselfadjusting.Splaytreeshavesomenicepropertiesincomparisontobalanced treessuchasredblacktrees,forexamplesimplerimplementationandanalysis.Usingamortized analysis,itispossibletoshowthatsplaytreesarejustasefficientasbalancedtreesforthestandard operationssuchasaccess,insert,anddelete(SleatorandTarjan1985b).Thefollowingexampleshows thisinastepbystepanalysis,whichiscompiledfromSleatorandTarjan(1985b)andTarjan(1985),as wellascoursematerialsavailableonlinebyWayne(2001),TarjanandMulzer(2007),andKargerand Zhang(2005)Thereaderisencouragedtoconsultthesesourcesforadditionaldiscussionand clarification. First,letusintroducetheSPLAY(x,S)operationonsomeelementxandasplaytree,S.This operationhastheeffectofrotatingxuptotherootofSifxisinS,orelserotatingthenextbiggestor nextsmallestelementtoxuptotheroot(dependingontheimplementation).SPLAYworksby repeatedlyperformingrotations.Ateachiterationofthewhileloop,tworotations(cases2ad)are performedtoreplacex'sgrandparentwithx,oronerotation(cases1ab)isperformedtoputxatthe root,ifx'sparentistheroot.Figures4through6illustratetherotationsforcases2a,2c,and1a;cases 2b,2d,and1baresymmetric.
SPLAY(x,S){ whilexisnotrootofS{ ifx'sparentisroot{ ifxisaleftchild,doarightrotation(case1a) elsexisaleftchild,sodoaleftrotation(case2a) //xisnowroot }else{ ifxisaleftchildanditsparent,y,isaleftchild,do2rightrotations(case2a) elseifxisarightchildandyisarightchild,do2leftrotations(case2b) elsexisarightchildandyisaleftchild,sodo1leftthen1rightrotation(case2c) elseifxisaleftchildandyisarightchild,do1rightthen1leftrotation(case2d) } } }
z y x C D A
x y B C B z A D y x C A A y
z D x A y B
x z C D
B A Figure4:SPLAYcase2a
C B B A Figure5:SPLAYcase2c x y C B
B A A B Figure6:SPLAYcase1a
SPLAYmovesxuponelevelinthetreeeachtimearotationisdone.Thiscouldresultinatotal of0rotations,inthecasewherexisalreadytheroot,oritcouldben1rotations,whenthetree containsnnodesinasinglechainandxisthelowestinthechain(e.g.,item1inFigure7).Therefore,a worstcaseboundontheSPLAYoperationisO(n)fornnodes. Figure7showsthesequenceoftreeconfigurationsthatoccuroverthewhileloopin SPLAY(1,S).Indeed,9rotationsareperformedonthetreewith10nodes.However,noticethatthe treeisshallowerafterthisaccessandsplay;thereisnowaytodoanotheraccessthatwillresultin SPLAYtaking9operations.Infact,considertheoperationSPLAY(2,S)onthisnewtree(Figure8). Thissplayonlyrequired5rotations,anditleftthetreeevenmorebalanced.Amortizedanalysisis usefulforreasoningaboutthefactthatperformingaveryexpensiveoperation,suchasSPLAY(1,S) ontheoriginaltreeinFigure7,performssomesortofcleanupthatlimitstheexpenseoffuture operations. 10 9 6 7 10 9 6 4 3 7 5 10 9 1 2 4 3 9 10 10 1 8 6 9 4 7 2 5 3 1 6 5 8 10 9
1 Figure7:SPLAY(1,S)ona"bad"tree(fromWayne2001)
8 6
4 2
1 2
4 3
1 6 5 8
10 9
1 3
2 4 6 5
3 Figure8:SPLAY(1,S)ontreefrom Figure7 TheSPLAYoperationiscalledonceforeachInsert,Delete,Join,andFindoperation.ToFind elementxintreeS,forexample,wefirstcallSPLAY(x,S),tomakextherootofSifxisinthetree. ThecostoftheSPLAYoperationisthereforecrucialtoanalyzingallotheroperations. Now,wedefineapotentialfunctioninordertoperformamortizedanalysis.LetSi(x)denotethe subtreeofSrootedatnodexattimei(specifically,aftertheithiterationoftheSPLAYwhileloop,for example),and|Si(x)|denotethenumberofnodesinthatsubtreeatthattime.Definetherankfunction tobe r i x = log S i x .Letri(x)indicatethelocalpotentialofnodexinthetreeSattimei. Finally,letouroverallpotentialfunctionattimeibe i = r i x . ItcanbeshownthattheamortizedcostofanysplaysteponnodexisO(logn).Toprovidethis bound,wecanenumerateallthepossiblechangestothetreethatasingleiterationoftheSPLAYwhile loopmightcause.Then,foreachcase,wecomputethepotentialfortheentiretreebeforeandafterthe loopiteration,andtheactualcostrequiredtoperformtheiteration.Thisgivesusaboundonthe amortizedcostofasingleloopiteration,whichwecanthensumoveralliterationsofthelooptoobtain aboundontheamortizedcostoftheentireSPLAYoperation. Choosecase2afromSPLAYtostartwith.Figure4showstheconfigurationbeforeandafterthe tworightrotations.Thetotalactualcostofthe2rotationsis2.Thechangeinpotentialintroducedby thesetworotationsisthesumofthechangesinpotentialateachnode: i i1= r i x r i1 x = r i x r i1 x Notethatcase2aonlychangesthepotentialofnodesx,y,andz:thesizesofallothernodes'subtrees remainunchanged.Furthermore,observethat r i x = r i1 z , r i 1 y r i 1 x ,and r i y r i x .Byequation2,theamortizedcostofthissinglecaseis:
c i = c i i i1 = 2 r i x r i 1 x r i y r i 1 y r i z r i 1 z = 2 r i x r i1 z r i y r i z r i1 x r i1 y 2 0 r i x r i z r i 1 x r i 1 x = 2 r i x r i z 2 r i1 x . (5)
x tree x tree x tree x tree
10 9
S i 1 x S i z 2
so, r i z 2 r i x r i 1 x 2 . Substitutingthisbackintoequation5,weseethat
c i 2 r i x 2 r i x r i1 x 2 2 r i1 x = 3 r i x 3 r i 1 x = 3 r i x r i1 x
(6).
loopofSPLAY.ThetotalamortizedcostofanentireSPLAYoperationis c = c i ifthewhileloop
i=1
executesmtimes.Usingequation6,andtakingadvantageofthetelescopingpropertyandthefactthat eithercase1aor1bwillonlybeexecutedonce,weget:
m m
c =
ci
i=1
3 r i x r i 1 x 1= 3 r m x r 0 x 1
i =1
Notethatrm(x)=nbecause,attimem,xistheroot,soallnnodesareinitssubtree.Noteagainthat r0(x)maybeassmallas1,ifxisataleaf.Therefore,ouramortizedboundforoneSPLAYoperation, c ,isO(logn). Finally,thisboundonSPLAYcanbeusedtoeasilyshowamortizedboundonalloperationsof splaytrees.Trivially,theFind(x)operationrequiresoneSPLAY,soitscostisalsoO(logn).Other operationsareslightlymoreinvolved;thereaderisdirectedtoKargerandZhang(2005),Sleatorand Tarjan(1985b),orTarjan(1983,p.536)formoredetails. Moreexamples Theexamplesaboverequirelittlebackgroundknowledgeofthedatastructuresorproblemarea.This sectionprovidesbriefoutlinesoftheuseofamortizedanalysisinseveralotheralgorithmsanddata structuresthataresomewhatmoreinvolved.Thissectionisprovidedsothatreadersfamiliarwiththese topicsmayappreciatetheroleamortizationhasplayedintheiranalysisand/ordesign,andsothat readerscomfortablewiththeexamplespresentedthusfarmayconsultsourcesforthefollowingtopics inordertolearnmore. Redblacktrees Redblacktreesareatypeofbalancedbinarysearchtreethatenforcesbalancebycoloringeachnodein thetreeeitherredorblack,thenreadjustingthetreeafteroperationssuchasinsertanddelete, accordingtothecolorsofthenodes(Cormenetal.2001,Chapter13).Insimple(unbalanced)binary
searchtrees,basicoperationssuchasinsertanddeletecanruninO(n)timeonatreeofsizen.Forred blacktrees,theseoperationsruninO(logn)time.However,thereisincreasedoverheadinvolvedinthe structuralmodificationsperformedtomaintaintreebalance.Intheworstcase,anyoperationcancause O(logn)modifications.However,amortizedanalysiscanbeusedtoshowthatanysequenceofminsert anddeleteoperationscausesO(m)structuralmodificationsintheworstcase(Cormenetal.p.428).It cansimilarlybeshownthatthetotaltimeformconsecutiveinsertionsinatreeofnnodesisO(n+m) (Tarjan1985). Tarjan(1985)usestheaccountingmethodtoshowthissecondbound.Theanalysisshowsthat theinvariantismaintainedthateveryblacknodeinthetreecontains0,1,or2credits,dependingon whetherithasoneredchild,noredchildren,ortworedchildren,respectively.Initially,O(n)creditsare addedtothetree.Theanalysistakesthefollowingform:allpossiblelocaleffectsofarebalancing operationfollowinganinsertoperationareenumerated.Theinvariantisshowntoholdforallofthese cases,wheretheterminatingcasesrequirepaymentofO(1)credits,andnonterminatingcases(which requireoneormoreadditionalrebalancingoperations)useupavailablecreditinthetreetoperform theiroperations.Becauseeachoperationincludesonlyoneterminatingcasebydefinition,eachinsert operationmustpayonlyaconstantamountofcredit.Therefore,theoverallamortizedcostofasingle insertoperationisO(1),andsotheamortizedcostofanysequenceofminsertoperationsisO(n(to inserttheinitialncredits)+m). Fibonacciheaps Fibonacciheapsareaheapdatastructurerepresentedbyaforestofheaporderedtrees(Fredmanand Tarjan1987).TheyallowdeletionfromaheapwithnitemsinO(logn)amortizedtimeandsupport otherheapoperations(insert,findmin,meld,decreasekey)inO(1)amortizedtime.Fibonacciheaps havebeenusedasunderlyingdatastructurestoachievebettertheoreticalboundsonminimumspanning treesandshortestpathproblems. Cormenetal.(2001,Chapter20)employthepotentialmethodtoanalyzetherunningtimeof Fibonacciheapoperations.Thepotentialfunctionusedis H =t H 2 m H wheretisthe numberoftreesintherootlistofheapHandm(H)isthenumberofmarkednodesinH.Withthis potentialfunction,itisfairlyeasytoshowO(1)amortizedtimeforeachnondeleteoperation individually,andO(lgn)timeforextractminanddelete(seethebookfordetails). Disjointsets Disjointsetdatastructuresmaintainacollectionofdisjointdynamicsets(Cormenetal.2001,Chapter 21)andsupportoperationstomakeaset,joint(union)twosets,andfindthesetthatcontainsa particularelement.Thestraightforwardunionalgorithmforadisjointsetforestimplementationtakes O(n)timefornelements,usingbothamortizedandworstcaseanalysis.However,usingtheunionby rankandpathcompressionheuristics,onecanshowthatanysequenceofmmake,union,andfind operationsrunsinO(m( n))time,whereistheslowlygrowingAckermannfunction.Tarjan performsthisanalysisusingtheaccountingmethod(1983,p.2430),andCormenetal.(2001,p.5138) useapotentialfunction.Bothanalysesrequireunderstandingofthedetailsofpathcompressionthatlie outsidethescopeofthispaper,butthecuriousreaderisdirectedtotheTarjanandCormentextbooks formoredetailedtreatment. Maximumflow Thegoalofmaximumflowproblemsistofindthemaximumflowthroughagraph,whereonenodeis asource,onenodeisasink,andeachedgeinthegraphhasaknowncapacity(Cormenetal.2001,
Chapter26).Thiswellstudiedproblemhasseveralsolutionapproaches.Onesuchapproach,called pushrelabel,includesmanyoftheasymptoticallyfastestsolutionstothemaximumflowproblem (Cormenetal.,p.609).Ananalysisofpushrelabelhingesonfindingaboundonthenumberof nonsaturatingpushesperformedbythealgorithm.Thisisdoneusingapotentialfunctiondependenton theheightsoftheverticesinthegraph(Cormenetal.,p.6789). Dynamicarrays/hashtables Adynamictableisexpandedorcontractedasdataisaddedtoorremovedfromit.Itistherefore necessarytodesignandreasonaboutalgorithmsformanagingtablesize.Usingonesimplealgorithm describedinCormenetal.(2001,p.41624),asequenceofmoperationscanbeshowntotakeO(m) amortizedtime. Scapegoattrees Scapegoattreesareanothertypeofbalancedbinarysearchtree(GalperinandRivest1993).Unlikered blacktrees,theydonotstoreadditionalinformationateachnode.Amortizationusingapotential functionbasedonthesumoflocalnodepotentialsyieldsO(logn)insertionanddeletion. Conclusions Amortizedanalysisisausefultoolthatcomplementsothertechniquessuchasworstcaseand averagecaseanalysis.Ithasbeenappliedtoavarietyofproblems,anditisalsocrucialtoappreciating structuressuchassplaytreesthathavebeendesignedtohavegoodamortizedbounds. Tounderstandtheapplicationofamortizedanalysistocommonproblems,itisessentialtoknow thebasicsofboththeaccountingmethodandthepotentialmethod.Theresourcespresentedhere supplymanyexamplesofbothmethodsappliedtorealproblems. Toperformanamortizedanalysis,oneshouldchooseeithertheaccountingmethodorthe potentialmethod.Theapproachesyieldequivalentresults,butonemightbemoreintuitively appropriatetotheproblemunderconsideration.Thereisnomagicformulaforarrivingatapotential functionoraccountingcreditschemethatwillalwayswork;themethoduseddependsonthedesired boundsandthedesiredcomplexityoftheanalysis.Somestrategiesthatsometimeswork,however, includeenumeratingthewaysinwhichanalgorithmmightoperateonadatastructure,thenperforming ananalysisforeachcase;computingthepotentialofadatastructureasasumoflocalpotentialsso thatonecanreasonabouttheeffectsoflocalchangeswhileignoringirrelevantandunchanging componentsofthestructure;designingthepotentialmethodaroundthedesiredformoftheresult(e.g., relatingthepotentialtothelogofthesubtreesizeforsplaytrees,asthedesiredoutcomeisa logarithmicbound);andreasoningabouteachtypeofoperationinasequenceindividuallybefore comingupwithaboundonanarbitrarysequenceofoperations. Anunderstandingofamortizedanalysisisessentialtosuccessinanalgorithmscourse,to understandingtheimplicationoftheoreticalboundsonrealworldperformance,andtothoroughly appreciatingthedesignandpurposeofcertaindatastructures.Thereaderisthereforeagainurgedto consultanyofthesourcesmentionedheretoimprovehisorherunderstandingofamortizedanalysis andtoexplorethesealgorithmsingreaterdepth.
References Aho,A.V.,J.E.Hopcroft,andJ.D.Ullman.1974.Thedesignandanalysisofcomputeralgorithms. AddisonWesley. Agarwal,P.K,J.Xie,J.Yang,andH.Yu.2006.Scalablecontinuousqueryprocessingbytracking hotspots.Proceedingsofthe32ndInternationalConferenceonVeryLargeDatabases32:31 42. Brown,M.R.,andR.E.Tarjan.1980.Designandanalysisofadatastructureforrepresentingsorted lists.SIAMJournalonComputing9(3):594614. Cormen,T.H.,C.E.Leiserson,R.L.Rivest,andC.Stein.2001.Introductiontoalgorithms,2nded. MITPress. Fomitchev,M.,andE.Ruppert.2004.Lockfreelinkedlistsandskiplists.Proceedingsofthe23rd AnnualACMSymposiumonPrinciplesofDistributedComputing,509. Fredman,M.L.,andR.E.Tarjan.1987.Fibonacciheapsandtheirusesinimprovednetwork optimizationalgorithms.JournaloftheACM34(3):596615. Galperin,I.,andR.L.Rivest.1993.Scapegoattrees.ProceedingsoftheFourthAnnualACMSIAM SymposiumonDiscreteAlgorithms,16574. Georgiadis,L.,R.E.Tarjan,andR.F.Werneck.2006.Designofdatastructuresformergeabletrees. Proceedingsofthe17thAnnualACMSIAMSymposiumonDiscreteAlgorithms,394403. Huddleston,S.,andK.Melhorn.1981.RobustbalancinginBtrees.5thGIConferenceon TheoreticalComputerScience:LectureNotesinComputerScience104.NewYork:Springer Verlag.23444. Huddleston,S.,andK.Melhorn.1982.Anewdatastructureforrepresentingsortedlists.Acta Informatica,17:15784. Karger,D.,andX.Zhang.2005.Splaytrees.ScribenotesforSeptember12,6.854Advance Algorithms,MIT.Available:http://courses.csail.mit.edu/6.854/05/scribe/dzhangsplaytrees.ps. Mao,Y.,Y.Sun,M.Wu,andK.J.R.Liu.2006.JET:Dynamicjoinexittreeamortizationand schedulingforcontributorykeymanagement.IEEE/ACMTransactionsonNetworking14(5): 112840. Mendelson,R.,R.E.Tarjan,M.Thorup,andU.Zwick.2006.Meldingpriorityqueues.ACM TransactionsonAlgorithms2(4):53556. Sleator,D.D.,andR.E.Tarjan.1985a.Amortizedefficiencyoflistupdateandpagingrules. CommunicationsoftheACM28(2):2028.
Sleator,D.D.,andR.E.Tarjan.1985b.Selfadjustingbinarysearchtrees.JournaloftheACM32(3): 65286. Tarjan,R.E.1983.Datastructuresandnetworkalgorithms.CBMSNSFRegionalConferenceSeries inAppliedMathematics. Tarjan,R.E.1985.Amortizedcomputationalcomplexity.SIAMJournalonAlgebraicandDiscrete Methods6(2):30618. Tarjan,R.E.2000.Amortizedanalysisofmovetofront(MF)listrearrangement.Coursehandoutfor COS423,TheoryofAlgorithms,PrincetonUniversity.Available: http://www.cs.princeton.edu/courses/archive/spr00/cs423/handout2.pdf. Tarjan,R.E.,andW.Mulzer.2007.ScribenotesforFebruary17,COS423,TheoryofAlgorithms, PrincetonUniversity.Available: http://www.cs.princeton.edu/course s/archive/spring07/cos423/lectures/lecture4.pdf. Wayne,K.2001.Amortizedanalysis.CourseslidesforCOS423,TheoryofAlgorithms,Princeton University.Available:http://www.cs.princeton.edu/~wayne/cs423/lectures/amortized4up.pdf.