Week12- Lecture Notes Topics : dynamic Frog ranuning - memoizetion avd subproblems - Fivonaeei | examples - Shortest paths . questing and DAG views Computational Complexity Dynamic Frog ramming (vp) * Big idea, herd, yet simple. * Powerful algorithmic design technique . Lorge class of Seemingly exponential problems have @ polynomial solution (“only”) vie dP. Faxticularty for optimization problems (minlmen) - Enomple: Shortest paths. A. dymawnie programming i¢ o conlrolled brute- fore methool. Tt uses recursion end re-use, D0 & “controlled. brute- force” DP a “recursion and re-use”Fibonacci Alumbers Filconacei numbers ave of the Porm hehe t, Fae Fiat feo Goal: Compute Fy Noive Algori ton follows vecursive definition. fib in): iP ngo return fe) 2 else veturn 4: fibln) +4ibln-2) => The Tlr-) + Tha) + Ol) » A =e" % 2Tln-2) +0) y gre Enponential- SAS! nN / Ga) /\ J &) @O& aMemoized bd? Algorithm memo = {3 fib \W): z iF nm is in memo: return memoln] 3. else: if neQ: fel 4. else fe fioln-) +fibln-2) fe memo LnJs f ‘. teturn + * $i Lk) only yecurses Pest time called VK - onlynveememvized cells : Keh2,e0n ememoired cells free (OL) Hme) * OW time per coll Ggquoving veeursion) Polynomial - Good) © DP x “recursion + memoizetion” * memoize (vemember) and re-use solutions to Subproblems +net help solve problem — in Fivonaccé , subprovlems are fy Fay---1Fe =—ptime + a subproblems » Lime per Subproblem) +Fitonecei + # subproblems =n. Hime pey subproblems> Ol) a Hime = On) Uignoring vecursions )Bottom-up dP Algoritiom ' Hbe= Tf 2 for K im Lutyens 3 if K aes jel Bln) 4. else: Pe filo Leer] + fib le-27 | Ol. 5 site) + F 6. Teturn fib (wd senactly the seme, computation ab memoized bP (recursion “unrottied*) a general 7 ‘Pepelogicel sort of sub problem dependency DAG. . proctically jester: no secur sion : anely sis more olovious ‘con seve space: last 2fies > Ol)Shortest Paths + Recursive formulation Stun) swim {votuv) + 5L6ud | tow) €€ J + Memoized DP algorithm : takes infinite Hme if cycles. (necessary to handle negative cycles) + Works for directed aeyelic gla) aphs iv Ore) | ~(effectively v°5/ 5ts.00 topological gost + Gellman 1 Sls.) ford rolled into single recursion) $v) $lss) i Suloprovelem dependency should be (sv) an Oeyelic « -more subproblems temove eyelie dependence SylSv) = Shortest Sv path using & k edges ~Tecurrence: Sue Ld) + min { Sey So(d¥) = 0 for SHV | base —— Sy (53) + 0 for amy " / —_— - Goal » BU) = Sieg lv) iP vo negative cyplee -Mewoi ze ~time : # Subproblems - [¥ime per sulpzoblems) wiwt olv) € olv’) - actually 8 Lindeqree lw) jor $y (5¥) =D>tme OVE indegree(a)) = OLVE) [s.u) avotud |tuv) € J BELMAN FORK |Guessing How to design tecurrence wart shortest sev peth es whet is the last edge in path? den't Know s guess tt is (uw) path is shortest su poth + edge (uv) eee eee by optimal substructure scot 18s isu) + voturv) Ce Onother Subproblem * to Bind best ques, ty all (vl choices) ond use best. o*key: small (pelynomial’) # possible quesses per Sub problem — typically this dominates time] subproblem, * DP zx recursion + memoi zation +quessing DAG view ~\ike replicating POs . waph to vepreseut Hme qrar pr ~ converting shortest He 2 paths in graph to shortest pots in BAG 7 " DPe shortest paths in some DAGSuen ary * DP & careful lorute fore = questing 4 vecursion + memoi cation 2 dividing into teasonolle y# suloproblems whose Solution relate - acyoliely - usually via questing ports of solution ctimes de suloproblems x [time per suproblem) and treating recursive calls os oli) (usually wainly guessing) - eosentially On amortization - Count each suloproblem only once ; ofter First time, costs OL) vie memoizetion “DP x Shortest paths in some DAG.5 aby steps to Dynomie Geograrmming define subproblems count 4 subproblems queee (pert of solytion) count + choices telate suloproblem selutions compute time per subproblem Yeturse + memoize problems Hme + lime per sulaproble) o X® suoproblems build DP table betom-up cheek subproblems acyclic |+epologice! order. Solve origival problem: => extva time = 0 Subproblem oR by wunking oubproblem solutions. Enomples Fitoona cei Shortest paths Sulopro blems Fe &, (2) for VEN, 0 KE IVI jor \exen = min gv poth using £ Kedges #subproblems n u* guess nothing edge into v Lf any) # choices L indeq reel) +1 Teeurrence Fee Feat Fea 5qb5,8) = ymin V5 re Us,u) + lu? He per Sulproblem Bw Mumnie Ff i. ols indegree(™) logical eee tor Kel 9 fox Ke 00, Wie) for veEV fotel He atm) oe . * +O) unless efficient sbout tt indeqree prdbolem Fn Sys to9) dor veV entra ime ou) ou)C omnp utational Compleaity definitions: Ps {problems solvable in (n‘) me 3, Cpokyromiet EXP: { problems soluable in oo") time } (eaponentia) ak . { prololems solvable in Pinite tine 3 “recursive” computational! = eS ai tFiculty __ uncomputabole / r i erg R undeddeble Examples: negative - weight cycles detection EP nan Chess EEXP wut ¢P Who wins from given coord canFiguration | Tetris € EXP but don't Know whether € P Sumvive given pieced Prom given tooard. Halting (eoblem Given a eomputer program does it ever hatt Lstop) ? * uncomputable ( ¢R) + no algoritim solver it leorvectly in Binite me on all inputs) * decision problem: answer is YES or NOMost Decision Goblems ave Uncomputable + program x binary string x nonnegative integer EN) s decision problem = @ function Prom binary strings (= non- neg. intergers) to [YES(n, No lor} oz inbnite sequewe of bits = real number ER WI not nearly enovg P7eg yams fos all problems * ea0n program solves only one problem * => almost all problems cannot be solved NP NPs { Decision problems solveble in polynowiel Hme via % \ucky algorithm} “The lucky algorithm can make. . \ueky gueses, alooys “wight” without trying all opttons * nondeterministic model + a\qori thn makes gueses ond then says YES or NO * guesses queranteed to lead to YES outcome iP possible Example: Tetns ENP * nondetermini she algoritowm t quess each move, did I survive 1 + prooP oP Yes: Vist what moves to make (rules of Tetris are cosy )Nf NPs {decision problems with solubons hat con be checked” iw polynomial time} => when answer is YES, it cow be proved, and polyromial -time algorithm con check proof. —_— computation Eat diffi culty ne Ve eG ae uncom putable / undedidatole R P#AP ——— Ik ie a big conjecture (worth $1,000,000) + zconnot engineer luck . £ generating (preots of) solutions con be herder then checking themHadness and completeness Claim: Tf P#ENP, then Tetris ENP- P Proof? * Tetris is AP- hard = “es hard os” every problem ent Injoct . Tetris is AP complete = NP i [wP-hord) NP- Word = : Ex?- hard were PES. ——— chess AP : es Exe uncomputolde/ iassi sees undecidalole * Chess is EAP- complete = EXPN EXP-herd. ExP- hard is as hard as every protlem in GxP. TP NP # EXP, then Chess ¢ Ext\ne. Whether NP EXP ib also an open problem but less” Jomous /™ impo:Reductions Convert the problem into @ problem that i> already known Wow to solve | instead of solving Prom serateh) «© most Common algoritnun design technique + unweighted -shortest path -> wei ghted (set weights 9) + min product path » shortest path Lake 1oqs) * longest path + shortes path (megotive eee * shortest order tour » shortest path (« er of the graph + Cheapest leaky-tonk path > Shortest path 4 fa (gre ny seduction) All oP the aloove ave One-call reductions: A problem — Bproviem +» Bsolution — A solution Multicall seductions: ssobe A using Sree calls 2B, “in tas sense, every algorithen reduces problem > modelo? computation ”N¢- Complete Problems NP- Complete problems ave ell interreducible usin Polynomial Hime teducHons (some difPculty) We can use reductions to prove NP-hardness Tetris. Ertam ples of NP- Cow plete Peovolems « Knap sack ¢ 3- partition : given w integers, divide them into triples of equal sum? ° Travelling Selesmen Problem: ~ shortest path that visits all vertices fo geen raph is nea weight gal Becision version) \ongest common subsequence of k strings Mine sweeper, Soduku and most puzzles SAT : given o Boolean jormula Lend,or, not), is it ever true? * Shortest paths amidst obstacles in SB ° 3 coloring a given graph « Find largest clique in@ given q7eph-