Par

t2: Al
gor
it
hm Desi
gnTechni
ques

Af
teri
ntr
oduci
ngt
hef
ramewor
kandmet
hodsf
oral
gor
it
hm anal
ysi
sint
hepr
ecedi
ngsect
ions,
wear
e
r
eadyt
oembar
konadi
scussi
onofal
gor
it
hm desi
gnst
rat
egi
es.

I
ntr
oduct
ion

Oneoft
het
hingst
hatst
udent
sfoundmostdi
ff
icul
twast
hatonedoesnotknowwher
etost
art
.

Ther
efor
ethe1stquest
ionher
eis:Hav
eyoudesi
gnedal
gor
it
hmsbef
ore.I
fyeswhatappr
oacheshav
eyou
f
oundhel
pful
?

And2ndi
s:Gi
venanal
gor
it
hmi
cpr
obl
em,
wher
edoy
ouev
enst
art
?

Exper
ienceshowt
hatmostal
gor
it
hmsf
oll
owoneofsev
eralwel
l
-knownt
echni
quessuchas:br
ute-f
orce,
di
vi
de-
and-
conquer
,cont
ract
ion (
reduci
ng one pr
obl
em t
o anot
her
),dy
nami
c pr
ogr
ammi
ng,gr
eedy
appr
oach,et
c.I
nthi
slect
ure,wewi
l
ldi
scusst
heset
echni
ques,whi
char
eempl
oyedi
nbot
hsequent
ial
andpar
all
elal
gor
it
hms.

Not
e.Tobecomegoodatdesi
gni
ngal
gor
it
hms,r
equi
regai
ningasmuchexper
ienceaspossi
blebybot
h
f
ormul
ati
ngpr
obl
emsour
sel
vesandsol
vi
ngt
hem byappl
yi
ngt
het
echni
quest
hatwewi
l
ldi
scov
erher
e.

Ov
erv
iewl
et’
shi
ghl
i
ghtsomeoft
hosedesi
gnt
echni
quesanddi
scusseachi
nmor
edet
ail
slat
er.

Br
uteFor
ce(
BF)
:TheBFappr
oachi
nvol
vest
ryi
ngal
lpossi
blesol
uti
onsandpi
cki
ngt
hebestoroneoft
he
sat
isf
ying sol
uti
ons (
usual
l
ythe f
ir
sti
tencount
ers)
.The br
utef
orce appr
oach t
her
efor
etr
ied al
l
per
mut
ati
ons.
I
nsomepr
obl
ems,
itsuf
fi
cesonl
ytopi
ckasol
uti
on;
insuchcases,
iti
snotnecessar
ytoconsi
der
al
lpossi
blesol
uti
ons,
aBFal
gor
it
hm canr
etur
nthef
ir
stencount
ered.
I
nmanyot
herpr
obl
emst
her
ear
epol
ynomi
almanypossi
bil
i
ties.
Q.Gi
vent
hatt
heyar
eof
teni
nef
fi
cient
,whydowecar
eaboutBFal
gor
it
hms?
Onepl
acet
heBFappr
oachcanbev
eryusef
uli
swhenwr
it
ingat
estr
out
inet
ocheckt
he
cor
rect
nessofmor
eef
fi
ciental
gor
it
hms.
Ev
eni
finef
fi
cientf
orl
argen t
heBFappr
oachcoul
dwor
kwel
lfort
est
ingsmal
linput
s.The
appr
oachi
susual
l
ythesi
mpl
estsol
uti
ont
oapr
obl
em,
butnotal
way
s.
Q.Whent
ryi
ngt
oappl
ytheBFt
echni
quet
osol
veapr
obl
em,
wher
ewoul
dyoust
artf
ir
st?

Si
nceBFr
equi
resenumer
ati
ngal
lpossi
bler
esul
tsandchecki
ngt
hem,i
tisof
tenhel
pfult
ost
artby
i
dent
if
yingt
he:
o st
ruct
ureoft
heexpect
edr
esul
ts(
out
put
)(al
lst
ri
ngs,
all
nat
ural
number ween0andn)and
sbet
o awayt
oassi
gnv
aluest
othem sot
haty
oucanpi
ckt
hebest
.

Reduci
ngt
oanot
herpr
obl
em
Somet
imest
heeasi
estappr
oacht
osol
vi
ngapr
obl
em i
sjustt
oreducei
ttoanot
herpr
obl
em f
orwhi
ch
knownal
gor
it
hmsexi
st.
Wheny
ougetmor
eexper
iencedwi
thal
gor
it
hmsy
ouwi
l
lst
artr
ecogni
zi
ngsi
mil
ari
ti
esbet
ween
pr
obl
emst
hatont
hesur
faceseem v
erydi
ff
erent
.
Butoneexer
ciset
haty
oumi
ghtf
indhel
pfulwoul
dbet
ocont
inual
l
yfor
mul
atenew al
gor
it
hmi
c
pr
obl
ems,
ast
hiscanhel
pinunder
standi
nghowseemi
ngl
ydi
ff
erentpr
obl
emsmayber
elat
ed.

Ov
erv
iewoft
heI
nduct
ivet
echni
ques:

Thei
deabehi
ndi
nduct
ivet
echni
quesi
stosol
veoneormor
esmal
l
eri
nst
ancesoft
hesamepr
obl
em,
usual
l
yref
err o as subpr
edt obl
ems,t
o sol
vet
heor
igi
nalpr
obl
em.Thet
echni
quemostof
tenuses
r
ecur
siont
osol
vet
hesubpr
obl
ems.Commont
echni
quest
hatf
all
int
hiscat
egor
yincl
ude:

Di
vi
deandconquer(
DAC)

DACdi
vi
desapr
obl zeni
em ofsi ntok>1i
ndependentsubpr
obl zesn1,n2...nk,sol
emsonsi veeach
pr
obl
em r
ecur
siv
ely
,andcombi
net
hesol
uti
onst
ogett
hesol
uti
ont
otheor
igi
nal
probl
em.
I
tisi
mpor
tantt
hant
hesubpr
obl
emsar
eindependent
;thi
smakest
heappr
oachamenabl
eto
par
all
eli
sm becausei
ndependentpr
obl
emscanbesol
vedi
npar
all
el

Fi
gur
eA:
Str
uct
ureofadi
vi
de-
and-
conqueral
gor
it
hm i
l
lust
rat k=1)
ed( .

TheGr
eedyAl
gor
it
hm: Forapr
obl zen,usesomegr
em ofsi eedyappr
oacht
opul
loutoneel
ement
l
eav
ingapr
obl zen-
em ofsi 1.Sol
vet
hesmal
l
erpr
obl
em.

TheCont
ract
ionAl
gor
it
hm CA:Forapr
obl zengener
em ofsi ateasi
gni
fi
cant
lysmal
l
er(
cont
ract
ed)
i
nst
anceori
nst
ances(
e.g.
, zen=2)
ofsi ,sol
vet
hesmal
l
eri
nst
ancessequent
ial
l
y,andt
henuset
her
esul
ts
t
osol
vet
heor
igi
nalpr
obl
em.Cont
ract
iononl
ydi
ff
ersf
rom DACi
nthati
tal
l
owst
her
etobeonl
yone
subpr
obl
em ormanydependentsubpr
obl
ems.

Fi
gur
eB:
Str
uct
ureofacont
ract
ional
gor
it
hm i
l
lust
rat
ed.
Dy
nami
cPr
ogr
ammi
ng DP:Li
keDAC,DP di
vi
dest
hepr
obl
em i
nto smal
l
ersubpr
obl
emsand t
hen
combi
nest
hesol
uti
onst
othesubpr
obl
emst
ofi
ndasol
uti
ont
otheor
igi
nalpr
obl
em.Thedi
ff
erence,
t
hough,i
sthatt
hesol
uti
onst
osubpr
obl
emsar
ereusedmul
ti
plet
imes.I
tist
her
efor
eimpor
tantt
ost
ore
t
hesol
uti
onsf
orr
euseei
therusi
ngmemor
izat
ionorbybui
l
dingupat
abl
eofsol
uti
ons.

Whati
sagoodsol
uti
ont
oapr
obl
em?

Wheny
ouencount
eranal
gor
it
hmi
cpr
obl
em,
youcanl
ooki
ntoy
ourbagoft
echni
quesand,
wit
hpr
act
ice;
y
ouwi
l
lfi
ndagoodsol
uti
onst
othepr
obl
em.

Whenwesayagoodsol
uti
onwemean:

1.Cor
rect
:Cl
ear
lycor
rect
nessi
smosti
mpor
tant
.

2.Lowcost
:Outoft
hecor
rectsol
uti
ons,
wewoul
dpr
efert
heonewi
tht
hel
owestcost
.

Q.Howdoy
oushowt
haty
oursol
uti
oni
sgood?

I
deal
l
yyoushoul
dpr
ovet
hecor
rect
nessandef
fi
ciencyofy
oural
gor
it
hm.Oneshoul
dmakeahabi
tof
doi
ngt
hisi
nreall
i
fe,
becausei
tisof
tent
ooeasyt
oconv
inceonesel
fthatapooral
gor
it
hm i
scor
rect
andef
fi
ci
ent
.

Q.Howcany
oupr
ovet
hecor
rect
nessofanal
gor
it
hm desi
gnedusi
ngani
nduct
ivet
echni
que?Cany
ou
t
hinkofapr
ooft
echni
quet
hatmi
ghtbeusef
ul?

Al
gor
it
hmsdesi
gnedwi
thi
nduct
ivet
echni
quescanbepr
ovedcor
rectusi
ng(
str
ong)i
nduct
ion.

Q.How cany
ouanal
yzeanal
gor
it
hm desi
gnedbyusi
ngani
nduct
ivet
echni
que?Cany
out
hinkofa
t
echni
quet
hatmi
ghtbeusef
ul?

Wecanof
tenexpr
esst
hecompl
exi
tyofi
nduct
iveal
gor
it
hmswi
thr
ecur
sionsandsol
vet
hem t
o
obt
ainacl
osed-
for
m sol
uti
on.
TheBr
utef
orceTechni
que
BFisast r
aight
forwardappr oachtosolvi
ngapr obl
em, usuall
yitisdir
ect
lybasedontheproblem
st
atementanddef ini
tionsoftheconceptsinvol
ved.
The“for
ce”impl i
edbyt hestr
ategy’
sdefini
ti
onist hatofacomput erandnott
hatofone’s
i
ntel
lect.“
Justdoit !
”wouldbeanot herwaytodescr i
bethepr escri
pti
onoftheBFapproach.And
of
ten,theBFstrat
egyi sindeedtheonet hati
seasiesttoapply.

Incomput ersci
ence,BFsearchorexhausti
vesearch,
alsoknownasgenerateandt est
,isav er
y
generalproblem-sol
vi
ng techni
quet hatconsi
stsofsy st
emati
cal
lyenumer ati
ng allpossibl
e
candidat
es forthe solut
ion and checki
ng whet
hereach candi
date sat
isfi
es the probl
em's
stat
ement .

Fori
nst
ance:
o ABFalgori
thm tofi
ndthedivi
sorsofanat
uralnumbernwoul denumerat
eal
lint
eger
s
f
rom 1ton,andcheckwhet
hereachoft
hem di
vi
desnwithoutr
emainder
.

o ABFappr oachfortheeightqueenspuzzlewoul
dexami
neallpossi
blearr
angement
sof8
piecesont he64-squar
echessboar d,and,f
oreachar
rangement,checkwhethereach
(queen)pi
ececanat t
ackanyother.

Whil
eaBFsear chi
ssimpletoi
mplement,andwi llal
waysf
indasol
uti
onifitexists,i
tscosti
s
pr
oporti
onalt
othenumberofcandi
datesolut
ions–whi chi
nmanypracti
calproblemst endst
o
gr
owv eryqui
ckl
yasthesi
zeoft
heproblem i
ncreases.

Ther
efor
e,BFsear
chshoul
dbeusedwhen:
o theproblem sizeisli
mited,or
o therear eproblem-specif
icheuri
sti
cst hatcanbeusedt or educethesetofcandi date
solut
ionst oamanageabl esize.
o Thesimpl ici
tyofimplementati
onismor eimportantthanspeed.Thi
sisthecase,incri
ti
cal
appli
cationswher eanyer r
orsinthealgori
thm wouldhavev eryser
iousconsequences;or
whenusi ngacomput ertoproveamat hemati
caltheorem.
o Itist o serveasabasel inemet hod;whenbenchmar king ot
heralgori
thmsormet a-
heuri
stics.

TheBFmethodforf
indi
ngani
tem i
nat
abl
e—t
hatcheckal
lent
ri
esoft
hel
att
er,
sequent
ial
l
yis
edl
cal
l i
nearsear
ch.

Basicalgor i
thm f eature
I
nor dertoappl yBFsear chtoaspeci ficcl
assofpr oblems, onemustimpl ementfourprocedures:
fi
rst
,next ,vali
d,andout put.Thesepr ocedur
est akeasapar ameterthedat aPf ort
hepar ti
cular
i
nstanceoft hepr oblem t obesol ved,anddothef oll
owi ng:
fi
rst(P): gener at
eaf i
rstcandidatesol
utionforP.
next( P,c):gener atethenextcandi dat
ef orPaf terthecurr
entonec.
valid(P,c): checkwhet hercandidatecisasol utionforP.
output( P,c) :uset hesoluti
oncofPasappr opriatetotheappl
icati
on.
o Thenextpr ocedur emustal sot ellwhent herearenomor ecandidatesfortheinstanceP,
 aft
erthecur
rentonec.Awayt odot hati
st oretur nul
na" lcandi
dat
e",someconv
ent
ional
datav
alueΛthati
sdist
inctf
rom anyr
ealcandidat
e.

o Thef
ir
stpr
ocedur
eshoul
dret
urnΛi
fther
ear
enocandi
dat
esatal
lfort
hei
nst
anceP.

TheBFmet
hodcanbeexpr
essedbyt
hef
oll
owi
ngpseudocode:

c← f i
rst
(P) //init
iali
zecwi t
hthe1stcandi
date
whil
ec≠Λdo
i
fvali
d(P,
c)thenout put(
P,c)
c← next(P,
c)
endwhi l
e
Forexample,whenlookingf ort
hedivi
sor
sofani nt
egern,
thei
nst
ancedat
aPi
sthenumbern.

Cal
lfi
rst(
n)shoul
dreturnthei
nteger1ifn≥1,orΛother
wise;
Cal
lnext(
n,c)shoul
dreturnc+1i fc<n,andΛother
wise;and
Cal
lvali
d(n,
c)shoul
dr et
urntr
ueifandonlyi
fcisadivi
sorofn.
Theal
gori
thm abovewil
lcal
loutputf
orev er
ycandi
datet
hatisasolut
iont
othegi
veni
nst
anceP.

Theal
gor
it
hm caneasil
ybemodifi
edtostop:
o aft
erfi
ndingthef
ir
stsol
uti
on,or
o aspecifi
ednumberofsol
uti
ons;or
o aft
ertesti
ngaspeci
fi
ednumberofcandi
dat
es,
or
o aft
erspendingagi
venamountofCPUtime

ThePr
obl
em ofCombi
nat
ori
alexpl
osi
on
Themaindisadvant
ageoft
heBFmet hodi
sthat
,formanyr
eal
-wor
ldpr
obl
ems,t
henumberof
nat
ural
candidat
esispr
ohi
bit
ivel
ylar
ge.

Therei sshar pgr owthint

henumberofcandi dates,ast hesizeofthedat aincreases,occursi
nall
sortsofpr oblems.
Fori nstance,i fweareseeki ngapar ticul
arr earr
angementof10l ett
ers,thenwehav e10!=
3,628,800candi datestoconsider ,whichat y
picalPCcangener ateandt estinl essthanone
second.
Howev er,addi ngonemor el ett
er— whi chi sonl ya10% i ncreasei nthedat asize— wi l
l
mul t
iplyt henumberofcandi datesby11—a1000%i ncr ease.
For20l etters,thenumberofcandi datesis20! ,whichi sabout2.4×1018or2. 4qui nt
il
li
on;and
thesear chwi ll t
akeabout10y ear s.
This phenomenon i s commonl y call
ed t he combi nat
ori
alexpl osion,or t he cur se of
dimensi onal i
ty.

Speedi
ngupbr
ute-
for
cesear
ches

OnewaytospeedupaBFal gor
it
hm ist
oreducethesear
chspace,thatis,t
hesetofcandi
dat
e
sol
uti
ons,
byusi
ngheur
ist
ics(t
ri
alander
ror
)speci
fi
ctothepr
oblem class.

Example1:i
nchessgame,t
hechal
lengeoftheeightqueens’i
showt opl
acethequeensona
standar
dchessboar
dsot
hatnoqueenattackseachother.Si
nceeachqueencanbeplacedi
n
anyoft
he64squar
es,
inpr
inci
plet
her
ear
e64Λ8=281,
474,
976,
710,
656possi
bil
i
tiest
oconsi
der
.
Howev er
,becausethequeensareallal
i
ke, andthatnotwoqueenscanbeplacedonthesame
squar
e,thecandidat
esar eal
lpossibl
eway sofchoosi ngasetof8squaresfr
om thesetall64
squar
es;whichmeans64choose8=64! /(56!*8!
)=4,426,
165,
368candidat
esoluti
ons— about
1/60,
000ofthepreviousest
imate.

Fur
ther
,noar r
angementwi
tht woqueensont hesamerow ort
hesamecol umncanbea
sol
uti
on.Ther
efor
e,wecanfur
therrest
ri
ctt
hesetofcandi
dat
est
othosear
rangement
s.

From t
hisexample,ali
tt
lebi
tofanal
ysi
swilloft
enleadt
odr amati
creduct
ionsi
nthenumberof
candi
datesolut
ions,
andmayt ur
nanint
ract
ableprobl
em i
ntoatri
vi
alone.

Exampl e2:foraproblem off

indi
ngall
integer
sbetween1and1, 000,000evenl
ydi
visi
bleby417a
naiveBFsolutionwouldgenerateal
li
ntegersi
ntherange, andtesteachofthem f
ordivi
sibi
l
ity.
However,thepr obl
em canbesol vedmor eeffi
cientl
ybyst ar
ti
ngwi t
h417andr epeatedl
y
adding417unt i
lthenumberexceeds1, 000,
000— whi cht akesonly2398( =1,
000,000÷417)
steps,andnot est
s.

Reor
der
ingt
hesear
chspace

Sincetherunni
ngtimeofaBFsear choftendependont heorderi
nwhi
chthecandi
dat
esar
e
tested.
Asageneral
rul
e,oneshoul
dtestt
hemostpr omi
singcandi
dat
esf
ir
st.
I
nsummar y,t
houghBr ut e-f
or cei srarelyacl everoref ficiental gori
thm, i
tshouldnotbe
over
lookedasani mpor tantal gor it
hm desi gnst rategyf ort hef oll
owingreasons:
o unl i
kesomeot herst rategi es,BFi sappl i
cablet oav erywi dev ari
etyofpr oblems.I nf act,i
t
seemst obet heonl ygener alapproachf orwhi chi tismor ediff
icul
ttopoi ntoutpr oblemsi t
cannottackle.
o For some i mportant pr oblems—e. g.,sor t
ing,sear ching,mat r
ix mul ti
pli
cation,st r
ing
mat chi
ng—theBFappr oachy ieldsreasonabl ealgor ithmsofatl eastsomepr acti
calv alue
withnolimit
ationoni nstancesi ze.
o Theexpenseofdesi gningamor eef fi
ciental gor i
thm maybeunj usti
fiablei fonlyaf ew
i
nstancesofapr obl em needt obesol vedandaBFal gor i
thm cansol v
et hosei nstanceswi th
acceptabl
espeed.
o ABFal gori
thm canst il
lbeusef ul f
orsol vi
ngsmal l-si
zei nstancesofapr oblem.
o ABFal gori
thm canser veani mpor t
antt heoreticaloreducat i
onalpurposeasay ardsti
ckwi th
whichtojudgemor eef ficientalternati
v esf orsolvingapr oblem.

Appl
icat
ionofBr
uteFor
ce:Sel
ect
ionSor
tandBubbl
eSor
t
Let’
sconsi dertheappl
icati
onoft heBFappr oachtotheproblem ofsor
ti
ng
Givenal istofn or derabl
eitems( e.
g.,number s,char
actersfrom someal
phabet
,char
act
er
str
ings)
,rear r
angethem innon-decreasi
ngor der
.

Select i
onSor t
Selection-
sort,sor
tbyscanningtheentir
egivenl
isttofi
ndi t
ssmal lestelementandexchangei t
witht hefi
rstelement,put
ti
ngthesmal l
estel
ementinit
sf i
nalposit
ioni nthesortedl i
st.Thenthe
l
istscanagai n,star
ti
ngwi ththesecondelement,tofindt hesmal lestamongt hel astn−1
element sandexchangei twiththesecondelement,putt
ingt hesecondsmal lestel ementinits
fi
nal posit
ion.
Generall
y,onthei t
hpasst hroughtheli
st,number edf r
om 0t o n −2,t heal gor
it
hm
sear
chesf
ort
hesmal
l
esti
tem amongt astn−i
hel element
sandswapsi
twi
thAi
:

Af
tern−1passes,
thel
i
sti
ssor
ted.

Herei
spseudocodeoft
hisal
gor
it
hm,
forsi
mpl
i
cit
y,assumest
hatt
hel
i
sti
simpl
ement
edasan
ar
ray:
ALGORI THM Sel ect i
onSor
t(A[
0..
n−1] )
//Sortsagi v
enar raybyselect
ionsort
//Input:
Anar rayA[ 0.n−1]ofor
. derabl
e
element s
//Output:ArrayA[0..n−1]sort
edin
nondecr easi
ngor der
fori ←0t on−2do
Figure:Exampleofsortingwi t
hselectionsort.
mi
n←i Each l ine corr
esponds t o one i terat
ion of the
algori
thm,i.
e.,apasst hrought heli
st’stai
ltotheri
ght
orj
f ←i on−1do
+1t
oft hev er
ti
calbar;anel ementi nbol dindi
catesthe
fA[
i j]
<A[
min]mi
n←j smal l
estelementf ound.El ementst ot heleftofthe
swapA[i
]andA[
min] verti
calbarar eint heirf i
nalpositionsandar enot
consideredinthi
sandsubsequenti terati
ons

Asanexample,seetheacti
onoftheal
gori
thm ont st89,45,68,90,29,34,17asi
hel
i ll
ust
rat
ed
i
ntheFi
gureabove.
Theanaly
sisofselecti
onsortisstr
aight
for
ward.Theinputsizeisgivenbyt henumberof
el
ementsn;t
hebasicoperat
ioni
sthekeycomparsonA[
i j]
<A[mi
n].

Thenumberoft
imesi
tisexecut
eddependsonl
yont
hear
raysi
zeandi
sgi
venbyt
hef
oll
owi
ng
sum:

Sincewehavealreadyencounter
edthelastsum i
nanalyzi
ngtheal
gor
it
hm earl
i
er,
youshouldbe
abletocomputeitnowony ourown.
Whetheryoucomput ethi
ssum bydistr
ibuti
ngthesummat i
onsymbolorbyimmediat
elyget
ti
ng
thesum ofdecr
easingint
egers,t
heanswermustbet hesame:

Thus,sel
ecti
onsortisaΘ( n2)algor
ithm onallinputs.Note,however,t
hatthenumberofkey
swapsisonlyΘ(n)
,or,
morepr eci
sely
, n−1(oneforeachr epet
it
ionoftheil
oop).
Thisproper
tydi
sti
nguishesselect
ionsortposit
ivel
yfrom manyothersor
ti
ngal
gori
thms.
BubbleSort
AnotherBFappli
cati
ontot hesort
ingproblem ist
ocompar eadj
acentelement softhel
i
stand
exchangethem i
ftheyar eoutofor der
.Bydoi ngitrepeat
edl
y,weendup“ bubbl
i
ngup”the
l
argestel
ementtothelastposi
ti
onont heli
st.
Thenextpassbubblesupthesecondl ar
gestelement
,andsoon, unt
ilaf
tern−1passest hel
i
st
i
ssor ed.Passi
t (
0≤i ≤n−2)ofbubbl esortcanberepresent
edbythefoll
owingdiagr
am:

Her
eispseudocodeoft
hisal
gor
it
hm.

ALGORI THM Bubbl eSor

t(A[
0..
n−1])
//Sortsagi v
enarraybybubblesort
//Input:
Anar rayA[0.
.n−1]oforder
abl
e
element s
//Output:ArrayA[0.
.n−1]sort
edin
nondecr easi
ngorder
fori ←0t on−2do
orj
f on−2−i
←0t do
fA[
i j+1]
<A[
j]swapA[
j]andA[
j+1]

Fi
gur
e:showsthefi
rstt
wopassesofbubbl
esor
tonthelst89,
i 45,68,
90,29,
34,17.
Anew l i
neisshownaf t
eraswapoft woelementsi
sdone.Theel ementstotheri
ghtoft
he
verti
calbarareint
heirfi
nalposi
ti
onsandarenotconsi
der
edi nsubsequenti
ter
ati
onsoft
he
algori
thm

Thenumberofkeycomparisonsf
orthebubbl
e-sor
tv er
siongi
venabov
eisthesamef oral
l
arr
ay zen;
sofsi iti
sobt
ainedbyasum t
hati
salmostident
ical
tot
hesum f
orsel
ect
ionsor
t:

Thenumberofkeyswaps,
however,dependsonthei
nput
.Int
hewor
stcaseofdecr
easi
ngar
ray
s,
i
tist
hesameast henumberofkeycompar i
sons:

Asi sof tent hecasewi thanappl icat

ionoft heBFst rategy, wecanmaket hefol
l
owing
observation:
Ifapasst hr
oughthelistmakesnoexchanges,t hel i
sthasbeensortedandwecanstop
thealgorithm.
Thought her untimeiti
sst i
l
linΘ( n2)intheworstandaveragecases,youcanobser
vet
hateven
mongel ement ar
ysorti
ngmet hods,bubblesortisaninferi
orchoice,andifitwer
enotfori
ts
catchyname, youwouldprobablyhav eneverhear
dofit
.
Thedi
vi
de-
and-
conquer(
DAC)appr
oach
I
nthissecti
on,weexamineanalt
ernat
ivedesi
gnapproach,knownas“ di
vi
deand-conquer
,”we
wi
llusedi
videand-
conquert
odesi
gnasor t
ingal
gor
it
hm whosewor st
-caserunni
ngti
mei smuch
l
essthanthatofi
nser
ti
onsort.

TheDACispr obabl
ythebest-
knowngeneralalgor
it
hm desi
gntechni
que.Thoughi
tsf
amemay
havesomethi
ngt odowi thitscatchyname,i tiswelldeser
ved;qui
teaf ew v
eryef
fi
cient
al
gori
thmsarespeci
fici
mplementat
ionsofthi
sgeneral
strat
egy.

Manyusef ulal
gorit
hmsar er ecur
sive in st
ructure:to solv
eagi ven problem,theycall
themselvesrecur
sivel
yoneormor etimest odealwithclosel
yrelat
edsubpr oblems.
Thesealgori
thmsusuallyfolow a DAC appr
l oach:theybr eakt hepr obl
em into sever
al
subproblemsthataresimil
artotheor i
ginalprobl
em butsmal lerinsize,solvethesubprobl
ems
recursi
vely
,andthencombinethesesolutionstocreateasolutiontotheor i
ginal
problem.

DACal
gor
it
hmswor
kaccor
dingt
othef
oll
owi
nggener
alpl
an:

o Divi
de-theprobl
em intoanumberofsubpr obl
emst hataresmalleri
nstancesofthesame
probl
em.
o Conquer-thesubproblemsbysol vi
ngt hem r
ecursi
vely.I
fthesubprobl
em sizesaresmal
l
enough,
howev er
,justsolv
et hesubproblemsinastrai
ghtforwar
dmanner .
o Combinethesoluti
onst othesubproblemsintothesoluti
onfortheori
ginalpr
oblem.

Whent
hesubpr
obl
emsar
elar
geenought
osol
ver
ecur
siv
ely
,wecal
lt her
hatt ecur
sivecase.
Oncet
hesubpr
obl
emsbecomesmal
lenought
hati
tdosenotr
equi
rer
ecur
sion,wesayt
hatt
he
r
ecur
si bot
on“ tomsout
”andt
hatwehav
egot
tendownt
othebasecase.
Somet
imes,
inaddi
ti
onwemaywehav
etosol
vesubpr
obl
emst
hatar
enotqui
tet
hesameas
t
heor
igi
nal
probl
em.Weconsi
dersol
vi
ngsuchsubpr
obl
emsaspar
toft
hecombi
nest
ep.

TheDACt echniqueasshownabov edepict

sthecaseofdi
vi
dingaprobl
em i
ntotwosmall
er
subproblems;thisi
sbyf arthemostwidel
yoccur
ri
ngcasef
orDACalgori
thmsdesi
gnedt
obe
executedonasi ngl
e-pr
ocessorcomput
er.
 Asanexampl e,letusconsi
dertheprobl
em ofcomputingt hesum ofnnumber sa0,...,an−1.
Ifn>1,wecandi v
idetheproblem i
ntotwoinstancesoft hesamepr oblem:thencomput et he
sum oft
hefirst(n/2)numbersandcomput ethesum oft her emai
ning(n/2)numbers.(ifn=1,
wesimplyr
eturna0ast heanswer.
)
Eachoft
het wosumsi scomputedbyapply
ingthefol
lowi ng:

Thef
oll
owingquest
ionsmayarise:
I
sthi
saneffi
cientwaytocomputethesum ofnnumber
s?
Andt
hequesti
on:
Whycoulditbemoreeff
ici
entthant
heBFsummat i
on?

NoteveryDACalgori
thm i
snecessari
l
ymoreef
fi
cientt
hanevenaBFsolut
ion.Butoft
enthetime
spentonexecuti
ngtheDACplanturnsoutt
obesigni
fi
cant
lysmall
ert
hansolvi
ngaproblem bya
di
ffer
entmethod.
Infact
,theDACapproachyiel
dssomeofthemostimport
antandef
fici
ental
gori
thmsin
computersci
ence.

Thoughweconsi deronl
ysequenti
alal
gori
thmsher e,i
tisworthyofnotethatt
heDAC
techniqueisideal
l
ysui t
edforparal
l
elcomput
ations,i
nwhicheachsubpr
obl
em canbesol
ved
simultaneousl
ybyitsownprocessor
.

I
nmostt y
picalcasesofDAC, aprobl
em’si
nst
anceofsizeni
sdivi
dedint
otwoinst
ancesofsi
ze
n/2.
Moregenerall
y,ani
nstanceofsizencanbedi
videdi obi
nt nst zen/
ancesofsi b,
wit yaof
honl
them needingtobesolved.(Her
e,aandbareconstant
s;a≥1andb>1. )

Assumingt zeni
hatsi sapowerofb,
tosi
mpl
i
fyouranal
ysi
s;wegett
hef
oll
owi
ngr
ecur
rencef
or
therunni
ngtmeT(
i n)
:
T(
n)=aT(
n/b)+f(
n), Eqn1
wheref( n)isafunct
ionthataccount
sforthet i
mespentondi
vi
dingani
nst zeni
anceofsi nto
i
nstancesofsizen/bandcombi ni
ngthei
rsolut
ions.
Forthesummationexampleabove,eta=b=2andf(
l n)=1.

Theabove(
Eqn1)iscal
ledthegeneralDACrecur
rence.
Theor
derofgr
owt hofT(n)dependsonthevaluesoft
heconst
ant
s:aandbandt
heor
der
ofgr
owthofthef
unctonf(
i n).

Bef
orecont
inui
ngwi
thcl
assi
cexampl
esofDAC,
let
’sbr
ief
lydi
scussr
ecur
rence.

WhatisRecur
rences?
Recur
rencesgohandi nhandwi
tht
heDACpar
adi
gm,becauset
heygi
veusanat
uralwayt
o
char
act
eri
zet
her
unni
ngt
imesofDACal
gor
it
hms.
Ar
ecur
rencei
sanequat
ionori
nequal
i
tyt
hatdescr
ibesaf
unct
ioni
nter
msofi
tsv
alueon
smal
l
eri
nput
s.
Ear
li
erwedescr
ibedt
hewor
st-
caser
unni
ngt
imeT(
n)oft
heMer
ge-
Sor
tpr
ocedur
ebyt
he
r
ecur
rence:
 whosesol
uti
onwecl
aimedt
obeT(
n)=Θ(
nlogn)
.

Recur
rencescant
akemanyf
orms.
Forex
ampl
e,ar
ecur
siv
eal
gor
it
hm mi
ghtdi
vi
desubpr
obl
emsi
ntounequalsi
zes,suchasa2/
3-
t
o-1/
3spl
i
t.
I
ftheDACst
epst
akel
i
neart
ime,
suchanal
gor
it
hm woul
dgi
ver
iset
other
ecur
rence:

So,
subpr
obl
emsmustnotnecessar
il
ybeaconst
antf
ract
ionoft
heor
igi
nal
probl
em si
ze.
Forexampl
e,i
far
ecur
siv
ever
sionofl
i
near
-sear
chcr
eat
esj
ustonesubpr
obl
em cont
aini
ng
oneel
ementl
esst
hant
heor
igi
nal
probl
em.
Eachr
ecur
siv
ecal
lwoul
dtakeconst
antt
imepl
ust
het
imef
ort
her
ecur
siv
ecal
l
sitmakes,
y
iel
dingt
her
ecur
rence:

Met
hodsofSol
vi
ngRecur
rences
Ther
ear
ethr
eemet
hodsf
orsol
vi
ngr
ecur
rences—t
hati
s,f
orobt
aini
ngasy
mpt
oti
c“Θ”or“
O”,
or
“
Ω”boundsont
hesol
uti
on:
o I
nthesubst
it
uti
onmet
hod,
weguessaboundandt
henusemat
hemat
ical
induct
iont
o
pr
oveourguesscor
rect
.
o Ther
ecur
sion-
tr
eemet
hodconv
ert
sther
ecur
rencei
ntoat
reewhosenodesr
epr
esentt
he
cost
sincur
redatv
ari
ousl
evel
soft
her
ecur
sion.Weuset
echni
quesf
orboundi
ng
summat
ionst
osol
vet
her
ecur
rence.
o Themast
ermet
hodpr
ovi
desboundsf
orr
ecur
rencesoft
hef
orm

,
wher
ea>=1,
b>1,
andf
(n)i
sagi
venf
unct
ion.
Occasi
onal
l
y,we see r
ecur
rences t
hatar
einequal
i
ties,such as T(
n)<=2T(
n)+Θ(
n).
Becausesuchar
ecur
rencest
atesonl
yanupperboundonT(
n),weexpr
essi
tssol
uti
onusi
ngO-
not
ati
on.
Si
mil
arl
y,i
fthei
nequal
i
tywer
erev
ersedt
oT(
n)>=2T(
n)+Θ(
n),becauset
her
ecur
rence
gi
vesonl
yal
owerboundonT(
n),
weuseΩ-
not
ati
oni
nit
ssol
uti
on.

TheFol
l
owi
ngar
esomest
andar
dal
gor
it
hmst
hatar
eDACal
gor
it
hms.

1)Bi
nar
ySear
ch-I
neachst
ep,t
heal
gor
it
hm compar
est
hei
nputel
ementxwi
tht
hev
alueof
 themi ddleel
ementinarr
ay.I
ftheval
uesmatch,
itret
urnst
heindexofmiddl
e.Ot
herwi
se,
ifx
i
sl esst hanthemiddleel
ement,t
henthealgor
it
hm recur
sforlef
tsideofmiddl
eelement
,
elserecursforr
ightsi
deofmiddl
eelement
.
2)Quick-sor
t-Theal
gorit
hm picksapi votel
ement,r
earr
angesthearr
ayelementsi
nsuchaway
thatallel
ementssmal l
erthant hepickedpivotel
ementmov etol ef
tsi
deofpivot,andal
l
greaterel
ementsmov etorightside.Fi
nall
y,t
healgor
it
hm recur
siv
elysor
tsthesubarr
ayson
l
eftandr i
ghtofpi
votelement.
3)Merge–Sort-Thi
sal
gor
ithm di
videst
hear
rayi
ntwohal
ves;r
ecur
siv
elysor
tst
hem and
f
inal
lymer
gesthet
wosor
tedhalv
es.
4)ClosestPairofPoi
nts-Thepr obl
em i
st of
indthemi ni
mum/maximum cl
osestpai
rofpoi
nts
i
nasetofpoi nt
sinx-yplane.I
tsol
vestheproblem i
nO(n^
2)ti
mebycalcul
ati
ngdist
ancesof
everypairofpoint
sandcompar i
ngthedistancestofi
ndtheminimum.TheDACal gor
it
hm
solvestheprobl
em inO(nLogn)ti
me.
5)Str
assen’
sAlgori
thm -Aneffi
ciental
gor
it
hm usetomul
ti
plytwomat
rices.Asi
mpl
emethod
tomulti
plyt
womat r
icesneed3nest edloopsandi
sO(n^3)
.ButSt
rassen’
smult
ipl
i
estwo
matri
cesinO(n^
2.8974)ti
me.

Let
’snowdi
scusssomeappl
i
cat
ionofDACal
gor
it
hm

TheMer
ge-
sor
t
Mer
ge-
sor
tsor
tsagi
venar
rayA[
0..n−1]bydi
vi
dingi
tint
otwohal
vesA[
0..
|n/
2−1]andA[
n/ .
2|
.n−1]
,sor
teachoft
hem r
ecur
siv
ely
,andt
henmer
get
het
wosmal
l
ersor
tedar
ray
sint
oasi
ngl
e
sor
tedone.
Thusmer
gesor
tal
gor
it
hm cl
osel
yfol
l
owst
heDACpar
adi
gm.
Ther
ecur
sion“
bot
tomsout
”whent
hesequencet
obesor
tedhasl
engt
h1;
noi
swor
ktobedone,
si
nceev
erysequenceofl
engt
h1i
sal
readyi
nsor
tedor
der
.
Thekeyoper
ati
onoft
hemer
gesor
tal
gor
it
hm i
sthemer
gingoft
wosor
tedsequencesi
n
t
he“
combi
ne”st
ep.
Themer
gingpr
ocedur
ecal
l
sthef
oll
owi
ngauxi
l
iar
ypr
ocedur
e:

Anyoft
hemer
gingpr
ocedur
esbel
owmaybeused;
thet
woal
gor
it
hmsar
edoi
ngt
hesamet
hing:
mer
gingt
wosor
tedar
ray
sint
oar
rayA.

Al
gor
it
hm (
procedur
e)A Al
gor
it
hm (B

Recallt
hecardplayi
ngmot if(i
nthetext
book:T.Conmen),wher
et wopil
esofcar
dsfaceupona
tabl
e.Eachpi
leissorted,withthesmall
estcar
dsontop,andthealgor
it
hm Bwasusedt omerge
them.
MERGE(A,p,q,r),whereAi sanarr
ayandp,q,andrar eindi
cesint
othearr
aysuchthat
p<=q<r
.Thepr
ocedur
eassumest
hatt
hesubar
ray
sA[
p..q]andA[
q+1.
.r]ar
einsor
tedor
der
.It
mer
gest
hem t
ofor
m asi
ngl
esor
tedsubar
ray
.
TheMERGEpr
ocedur
etakest
imeΘ(
n),
wher
en=r–p+1i
sthet
otalnumberofel
ement
s
bei
ngmer
ged.

Themer
gingoft
wosor
tedar
ray
scanbedoneasf
oll
ows.Twopoi
nter
s(ar
rayi
ndi
ces)ar
e
i
nit
ial
i
zedt
opoi
ntt
othef
ir
stel
ement
soft
hear
ray
sbei
ngmer
ged.
Theel
ement
spoi
ntedt
oar
ecompar
ed,andt
hesmal
l
eroft
hem i
saddedt
oanewar
ray
bei
ngconst
ruct
ed;af
tert
hat
,thei
ndexoft
hesmal
l
erel
ementi
sincr
ement
edt
opoi
ntt
oit
s
i
mmedi
atesuccessori
nthear
rayi
twascopi
edf
rom.Thi
soper
ati
oni
srepeat
edunt
iloneoft
he
t
wogi
venar
ray
sisexhaust
ed,andt
hent
her
emai
ningel
ement
soft
heot
herar
rayar
ecopi
edt
o
t
heendoft
henewar
ray
.

Gi
vent
hel
i
stA:
8,3,
2,9,
7,1,
5,4 Gi
vent
hel
i
stA=(
5,2,
4,7,
1,3,
2,6)
.
 Not
icethelengthsofthesor
tedsequencesbei
ng
mergedincreaseastheal
gori
thm pr
ogressesf
rom
bot
tom totop

Fi
gur
eA: Fi
gur
eB.

Fi
gur
esA&B:
Exampl
eofmer
gsor
toper
ati
onsont
wol
i
sts

Ther
ear
etwomai
nideasl
eadi
ngt
ovar
iat
ionsofmer
ge-
sor
t:t
hebot
tom-
upandmul
ti
way
o Theal gorithm canbei mplement edbott
om-upbymergingpai r
softhearray’
selements,t
hen
mer gingt hesor t
edpai rs,andsoon.( Ifnisnotapowerof2,onl ysli
ghtbookkeeping
compl icati
onsar i
se.)Thisav oidstheti
meandspaceov erheadofusingast acktohandle
recursivecalls.
o Al i
stt obesor tedcanbedi videdint
omor ethantwoparts,sorteachrecursi
vel
y,andthen
mer get hem t ogether.Thisscheme,whi chispar
ti
cul
arlyusefulforsor
ti
ngf il
esresi
dingon
secondar ymemor ydevices,i
scal edmul
l ti
waymergesort.

Therecur
rencef
ort
her
unni
ngt
imeofaDACal
gor
it
hm i
sobt
ainedf
rom t
het
hreest
epsoft
he
basi
cparadi
gm.

Asbef ore,weletT(n)bet herunningtimeonapr obl em ofsizen.

Ifthepr obl
em sizeissmal lenough,sayn<=cf orsomeconst antc,thestr
aight f
orward
soluti
ont akesconstantti
me, whichwewr i
teasΘ( 1).
Supposewedi videthepr oblem int
o‘a’subproblems,eachofsi ze1/ boftheori
ginal,andfor
exampl e,bothaandbar e2, (notethatther
emanyDACal gori
thmsi nwhicha≠b).
Ifittakest i
meT( n/b)tosol veonesubpr obl
em ofsi zen/ b,thenitwillt
aketmea*
i T(n/b)to
solveaoft hem.
IfittakesD( n)timet odi vi
det heproblem i
ntosubpr oblemsandC( n)ti
met ocombi nethe
soluti
onsoft hesubproblemsi ntothesoluti
onfortheor iginalpr
oblem;

Sowegett
her
ecur
rence:
Wereasonasfol
l
owstosetupther
ecur
renceforT(n)
, f
orthewor
st-
caser
unni
ngt
imeofmer
ge
sor
tonnnumberswhena=b=2.
o Themerge-
sor
tonj
ustoneel
ementn=1takesconstantt
ime.

o Whenn>1,
webr
eakdownt
her
unni
ngt
imeasf
oll
ows.

eqn2.
The“
mast
ert
heor
em,
”canbeusedt
oshow,

Butwedonotneedt
hemast
ert
heor
em t
ounder
standwhyt
hesol
uti
onr
ecur
renceeqn2i
sT(
n)=
(
nlogn)
.
Sowecanr
ewr
it
eeqn2as:

egn3

wher
etheconst
antcr
epr
esent
sthet
imer
equi
redt
osol
vepr
obl
emsofsi
ze1aswel
last
het
ime
perar
rayel
ementoft
hedi
vi
deandcombi
nest
eps.

I
fweaddt
hecost
sacr
osseachl
evel
oft
het
ree;
o t
het
opl
evel
hast
otal
costcn,
o t
henextl
evel
downhast
otal
costc(
n/2)
+c(
n/2)
=cn,
o t
hel
evel
aft
ert
hathast
otal
costc(
n/4)
+c(
n/4)+c(
n/4)+c(
n/4)
=cn,
andsoon.

I
ngener
al,
thel
evel
ibel
owt ophas2inodes,
het eachcont
ri
but
ingacostofc(
n/2i)
,sot
hat
:
othei
thlev
elbel
owt hetophastotalcost2ic(n/2i)
=cn.
oThebot
tom l
eveli
nf i
gureB,hasnnodes, eachcont r
ibut
ingacostofc,f
oratot
alcostofcn.
oThetot
alnumberofl ev
elsoftherecursi
ont reeislgn+1,wher eni sthenumberofleaves,
 cor
respondi
ngt
othei
nputsi
ze.

oTherefore,tocomput ethetotalcostrepresent
edbyt herecur
renceeqn3,wesimpl
yaddup
thecost sofallthelevels.
oTher ecursi
ontreehasl gn+1l evel
s,eachcostingcn,foratot
alcostofcn(l
gn+1)=cn(
lgn
+cn) .
oI
gnor i
ngt helow-orderterm andtheconstantcgivesthedesi
redresul
tofΘ(nlgn)
.

Anot
herExampl
e:Themaxi
mum-
subar
raypr
obl
em
Supposey
ouar
eof
fer
edt
heoppor
tuni
tyt
oinv
esti
nthest
ockofaVol
ati
l
eCor
por
ati
on.Youar
e
al
l
owedt
obuyoneuni
tofst
ockonl
yonceandt
hensel
litatal
aterdat
e,af
tert
hecl
oseof
t
radi
ngf
ort
heday
.Tocompensat
efort
hisr
est
ri
cti
on,
youar
eal
l
owedt
olear
nwhatt
hepr
iceof
t
hest
ockwi
l
lbei
nthef
utur
e.Yourgoali
stomaxi
mizey
ourpr
ofi
t.Tabl
ebel
owshowst
hepr
ice
oft
hest
ockov
era17-
dayper
iod.
Youmaybuyt
hest
ockatanyonet
ime,
star
ti
ngaf
terday0,
whent
hepr
icei
s$100pershar
e.Of
cour
se,y
ouwoul
dwantt
o“buyl
ow,sel
lhi
gh”
—buyatt
hel
owestpossi
blepr
iceandl
atersel
lat
t
hehi
ghestpossi
blepr
ice—t
omaxi
miz
eyourpr
ofi
t.Unf
ort
unat
ely
,youmi
ghtnotbeabl
etobuy
att
hel
owestpr
iceandt
hensel
latt
hehi
ghestpr
icewi
thi
nagi
venper
iod.I
nthet
abl
e,t
hel
owest
pr
iceoccur
saf
terday7,
whi
choccur
saf
tert
hehi
ghestpr
ice,
aft
erday1.

I
nst
eadofl
ooki
ngatt
hedai
l
ypr
ices,
letusi
nst
eadconsi
dert
hedai
l
ychangei
npr
ice,
wher
ethe
changeondayi
ist
hedi
ff
erencebet
weent
hepr
icesaf
terdayi
-1andaf
terdayi
.Thet
abl
ein
t
het
abl
eshowst
hesedai
l
ychangesi
nthebot
tom r
ow.I
fwet
reatt
hisr
owasanar
rayA,
shown
i
nFi
gur
ebel
ow,
wenowwantt
ofi
ndt
henonempt
y,cont
iguoussubar
rayofAwhosev
alueshav
e
t
hel
argestsum.Wecal
lthi
scont
iguoussubar
r hemaxi
ayt mum subar
ray
.
Forexampl
e,i
nthear
rayofFi
gur
ebel
ow,
themaxi
mum subar
rayofA[
1…16]i
sA[
8..
11]
,wi
tht
he
sum 43.Thus,
youwoul
dwantt
obuyt
hest
ockj
ustbef
oreday8(
thati
s,af
terday7)andsel
li
t
af
terday11,
ear
ningapr
ofi
tof$43pershar
e.

Fi
gur
e:Thechangei
nst
ockpr
icesasamaxi
mum-
subar
raypr
obl
em.

Here,
thesubar
rayA[
8..
11]
,wi
thsum 43,
hast
hegr
eat
estsum ofanycont
iguoussubar
rayof
ar
rayA.
ADACSol
uti
on
Let
’st
hinkabouthowwemi
ghtsol
vet
hemaxi
mum-
subar
raypr
obl
em usi
ngt
heDACt
echni
que.
Supposewewantt
ofi
ndamaxi
mum subar
rayoft
hesubar
r l
ayA[ow.
.hi
gh]
.DACsuggest
sthat
wedi
vi
det
hesubar
rayi
ntot
wosubar
ray
sofasequalsi
zeaspossi
ble.Thati
s,wef
indt
he
mi
dpoi
ntsaymi
, d,oft
hesubar
ray
,andconsi
dert
hesubar
ray l
sA[ow.
.mi
d]andA[
mid+1…hi
gh]
.
Anycont
iguoussubar
rayA[
i…j l
]ofA[ow…hi
gh]mustl
i
einexact
lyoneoft
hef
oll
owi
ngpl
aces:

Ther
efor
e,amaxi
mum subar
r l
ayofA[ow ..hi
gh]mustl
i
einexact
lyoneoft
hesepl
aces.Thea
maxi
mum subar
r l
ayofA[ow..hi
gh]musthav
ethegr
eat
estsum ov
eral
lsubar
ray
s;ei
ther
:
o ent
ir
elyi l
nA[ow..mi
d],
or
o ent
ir
elyi mi
nA[ d+1..hi
gh]
,or
o cr
ossi
ngt
hemi
dpoi
nt.
Wecanf
indmaxi
mum subar
ray l
sofA[ow ..mi
d]andA[
mid+1..hi
gh]r
ecur
siv
ely
,because
t
hesesubpr
obl
emsar
esmal
l
eri
nst
ancesoft
hepr
obl
em off
indi
ngamaxi
mum subar
ray
.
Thus,al
lthati
slef
ttodoi
sfi
ndamaxi
mum subar
rayt
hatcr
ossest
hemi
dpoi
nt,andt
akea
subar
raywi
tht
hel
argestsum oft
het
hree.

Fi
gure(a):Possi
blel
ocati
onofsubar
r l
ayofA[ow.
.hi
gh]
:ent
ir
elyi
n[l
ow.
.mi
d],
ent
ir
elyi mi
nA[ d+1…hi
gh]
,
orcr
ossingthemidpointmid.

Fi
gur
e(b):
Anysubar
r l
ayofA[ow.
.high]cr
ossi
ngthemidpoi
ntcompr
isest
wosubar
ray i.
sA[ .mi
d]and
mi
A[ dow+1..j
], el
wher ow<=I<=midandmi d<j<=hi
gh.

Wecanf
indamaxi
mum subar
raycr
ossi
ngt
hemi
dpoi
nti
nti
mel
i
neari
nthesi
zeoft
hesubar
ray
l
A[ow::hi
gh]
.
Thi
spr
obl s notasmal
em i l
eri
nst
anceofouror
igi
nalpr
obl
em,becausei
thast
headded
r
est
ri
cti
ont
hatt
hesubar
rayi
tchoosesmustcr
osst
hemi
dpoi
nt.Asi
nfi
gur
e(b)
,anysubar
ray
cr
ossi
ngt
hemi
dpoi
nti
sit
sel
fmadeoft
wosubar
ray i..mi
sA[ d]andA[
mid+1..j
], el
wher ow<=
mi
dandmi
d<j<=hi
gh.
Ther
efor
e,wej
ustneedt
ofi
ndmaxi
mum subar
ray
soft
hef
or i::mi
m A[ d]andA[
mid+1..j
]and
t
hencombi
net
hem.
Thepr
ocedur
eFI
ND-
MAX-
CROSSI
NG-
SUBARRAY bel
ow t
akesasi
nputt
hear
rayA and t
he
i cesl
ndi ow,mi
d,andhi
gh,andi
tret
urnsat
upl
econt
aini
ngt
hei
ndi
cesdemar
cat
ingamaxi
mum
subar
rayt
hatcr
ossest
hemi
dpoi
nt,
alongwi
tht
hesum oft
hev
aluesi
namaxi
mum subar
ray
.

Wet
her
efor
ehav
eal
i
near
-t
imeFI
ND-
MAX-
CROSSI
NG-
SUBARRAYpr
ocedur
e.
Wecanwr
it
epseudocodef
oraDACal
gor
it
hm t
osol
vet
hemaxi
mum subar
raypr
obl
em as:
Anal
yzi
ngt
hedi
vi
de-
and-
conqueral
gor
it
hm
Wesetup a r
ecur
rencet
hatdescr
ibest
her
unni
ng t
imeoft
her
ecur
siv
eFI
NDMAXI
MUM-
SUBARRAY pr
ocedur
e.Aswedi
dwhenweanal
yzedmer
gesor
t,wemaket
hesi
mpl
i
fyi
ng
assumpt
iont
hatt
heor
igi
nalpr
obl
em si
zei
sapowerof2,sot
hatal
lsubpr
obl
em si
zesar
e
i
nteger
s.
Wedenot
ebyT(
n)t
her
unni
ngt
imeofFI
ND-
MAXI
MUM-
SUBARRAYonasubar
rayofnel
ement
s.
l
i
ne1t
akesconst
antt
ime.Thebasecase,
whenn=1,
iseasy
:li
ne2t
akesconst
antt
ime,
andso
T(
1)=Θ(
1). Eqn4.

The r
ecur
siv
e case occur
s when n > 1.Li
nes 1 and 3 t
ake const
antt
ime.Each oft
he
subpr
obl
emssol
vedi
nli
nes4and5i
sonasubar
rayofn/
2el
ement
s(ourassumpt
iont
hatt
he
or
igi
nal
probl
em si
zei
sapowerof2ensur
est
hatn/
2isani
nteger
),andsowespendT(
n/2)t
ime
sol
vi
ngeachoft
hem.
Becausewehav
etosol
vet
wosubpr
obl
ems—f
ort
hel
eftsubar
rayandf
ort
her
ightsubar
ray
,the
cont
ri
but
iont
other
unni
ngt
imef
rom l
i
nes4and5i
s2T(
n/2)
.
Aswehav
eal
readyseen,t
hecal
ltoFI
ND-
MAX-
CROSSI
NG-
SUBARRAYi
nli
ne6t
akesΘ(
n)t
ime.
Li
nes7–11t
akeonl
yΘ(
1)t
ime.
Fort
her
ecur
siv
ecase,
ther
efor
e,wehav
e:

Eqn5
Combi
ningequat
ions(
4)and(
5)gi
vesusar
ecur
rencef
ort
her
unni
ngt
imeT(
n)ofFI
ND-
MAXI
MUM-
SUBARRAY:

Eqn6

Thi
srecur
rencei
sthesameasr
ecur
rencef
ormer
gesor
t.
Asweshal
lseef
rom t
hemast
ermet
hodl
ater
,thi
srecur
rencehast
hesol
uti
on:
T(
n)=Θ(
nlgn)
.
Youmi
ghtal
sor
evi
si
ttheear
li
err
ecur
siont
reet
ounder
standwhyt
hesol
uti
onshoul
dbeT(
n)=
Θ(
nlgn)
.
Thus,wecancheckt
hatt
heDACmet
hody
iel
dsanal
gor
it
hm t
hati
sasy
mpt
oti
cal
l
yfast
ert
han
heBFmet
t hod.
Wi
thmer
gesor
tandnowt
hemaxi
mum-
subar
raypr
obl
em,
webegi
ntogetani
deaofhow
power
fult
heDACmet
hodcanbe.Somet
imesi
twi
l
lyi
eldt
heasy
mpt
oti
cal
l
yfast
estal
gor
it
hm f
or
apr
obl
em,
andot
hert
imeswecandoev
enbet
ter
.

Al
gor
it
hm f
ormat
ri
xmul
ti
pli
cat
ion
Theassumpt
ioni
syouknowhowt
omul
ti
plymat
ri
ces.(
Other
wise,r
efr
eshy
our
sel
f)I
fA=(
aij
)
andB=(
bij
)ar
esquar
en*nmat
ri
ces,
theni
nthepr
oductC=A*B,
wedef
inet
heent
ryci
j
,fori
;j
=1,
2,....n,
by:

Eqn7.
en2 mat
Wemustcomput ri
xent
ri
es,andeachi
sthesum ofnv
alues.Thef
oll
owi
ngpr
ocedur
e
t
akesn*nmat
ri
cesAandBandmul
ti
pli
est
hem,
ret
urni
ngt
hei
rpr
oductC.
Eachmat
ri
xhasanat
tri er
but ows,
giv
ingt
henumberofr
owsi
nthemat
ri
x.

TheSQUARE-
MATRI
X-MULTI
PLYpr
ocedur
ewor
ksasf
oll
ows:
Thef
orl
oopofl
i
nes3–7comput
est
heent
ri
esofeachr
owi
,andwi
thi
nagi
venr
owi
,thef
or
l
oopofl
i
nes4–7comput
eseachoft
heent
ri
esci
j
,foreachcol
umnj
.Li
ne5i
nit
ial
i
zesci
jto0as
west
artcomput
ingt
hesum gi
veni
nequat
ion(
7),
andeachi
ter
ati
onoft
hef
orl
oopofl
i
nes6–7
addsi
nonemor
eter
m ofequat
ion(
7).
Becauseeachoft
het
ri
ply
-nest
edf
orl
oopsr
unsexact
lyni
ter
ati
ons,
andeachexecut
ionofl
i
ne7
t
akesconst
antt
ime,
theSQUARE-
MATRI
X-MULTI
PLYpr
ocedur
etakes‚
Θ(n3)t
ime.

Asi
mpl
eDACal
gor
it
hm
LetususeaDACal
gor
it
hm t
ocomput
ethemat
ri
xpr
oductC=A*B,weassumet
hatni
san
exactpowerof2i
neachoft
hen*nmat
ri
ces.Wemaket
hisassumpt
ionbecausei
neachdi
vi
de
st
ep,wewi
l
ldi
vi
den*nmat
ri
cesi
ntof
ourn/
2*n/
2mat
ri
ces,sot
hisexactpowerof2
guar
ant
eest
hatt
hedi
mensi
onn/
2isani
nteger
.
Supposet
hatwepar
ti
ti
oneachofA,
B,andCi
ntof
ourn/
2*n/
2mat
ri
ces

Eqn8

So,
wer
ewr
it
etheequat
ionC=A*Bas:
 Eqn9

Equat
ion(
9)cor
respondst
othef
ourequat
ions:

Eqns:
10,
11,
12and13

Eachofthesef
ourequat
ionsspeci
fi
est
womul
ti
pli
cat
ionsofn/
2*n/
2mat
ri
cesandt
headdi
ti
on
ofthei
rn/2*n/
2product
s.

Wecanuset
heseequat
ionst
ocr
eat
east
rai
ght
for
war
d,r
ecur
siv
e,DACal
gor
it
hm:

5 elsepar
ti
ti
onA,
B,andCasinequat
ions(
8)
6C11=SMMR( A11,B11)+SMMR(A12,B21)
7C11=SMMR( A11,B12)+SMMR(A12,B22)
8C11=SMMR( A21,B11)+SMMR(A22,B21)
9C11=SMMR( A21,B12)+SMMR(A22,B22)
10 r etur
nC

Not
e:SMMR=Squar
e-Mat
ri
x-Mul
ti
ply
-Recur
siv
e

Let
’sder
ivear
ecur
rencet
ochar
act
eri
zet
her
unni
ngt
imeofSMMR.
LetT(
n)bet
het
imet
omul
ti
plyt
won*nmat
ri
cesusi
ngt
hispr
ocedur
e.I
nthebasecase,
whenn=1,
weper
for
mjustt
heonescal
armul
ti
pli
cat
ioni
nli
ne4,
andso
T(
1)=Θ(
1) Eqn13

Ther
ecur
siv
ecaseoccur
swhenn>1.
Asdi
scussed,
par
ti
ti
oni
ngt
hemat
ri
cesi
nli
ne5t
akesΘ(
1)t
ime,
usi
ngi
ndexcal
cul
ati
ons.
I
nli
nes:6–9,wer
ecur
siv
elycal
l“SMMR”
,at
otalofei
ghtt
imes.Becauseeachr
ecur
siv
ecal
l
mul
ti
pli
est
won/
2*n/
2mat
ri
ces,t
her
ebycont
ri
but
ingT(
n/2)t
otheov
eral
lrunni
ngt
ime,t
he
t
imet
akenbyal
lei
ghtr
ecur
siv
ecal
l
sis8T(
n/2)
.
Weal
somustaccountf
ort
hef
ourmat
ri
xaddi
ti
onsi
nli
nes6–9.Eachoft
hesemat
ri
ces
cont
ainsn2/
4ent
ri
es,
andsoeachoft
hef
ourmat
ri
xaddi
ti
onst
akes,
Θ(n2)t
ime.
Si
ncet
henumberofmat
ri
xaddi
ti
onsi
saconst
ant
,thet
otalt
imespentaddi
ngmat
ri
cesi
nli
nes
6–9i n2)
s,Θ( .(Agai
n,weusei
ndexcal
cul
ati
onst
opl
acet
her
esul
tsoft
hemat
ri
xaddi
ti
onsi
nto
t
hecor
rectposi
ti
onsofmat
ri
xC,
wit
hanov
erheadofΘ(
1)t
imeperent
ry.
)
Thet
otalt
imef
ort
her
ecur
siv
ecase,t
her
efor
e,i
sthesum oft
hepar
ti
ti
oni
ngt
ime,t
he
t
imef
oral
lther
ecur
siv
ecal
l
s,andt
het
imet
oaddt
hemat
ri
cesr
esul
ti
ngf
rom t
her
ecur
siv
ecal
l
s:

Eqn14
Not
icet
hati
fwei
mpl
ement
edpar
ti
ti
oni
ngbycopy
ingmat
ri
ces,
whi
chwoul
dcost
‚ n2)t
Θ( ime,
the
r
ecur
rencewoul
dnotchange,andhencet
heov
eral
lrunni
ngt
imewoul
dincr
easebyonl
ya
const
antf
act
or.
Combi
ningequat
ions(
13)and(
14)gi
vesust
her
ecur
rencef
ort
her
unni
ngt
imeofSQUARE-
MATRI
X-MULTI
PLY-
RECURSI
VE:

Eqn15

Readpage78oft
hemai
ntext
bookt
oseehowequat
ion15wasobt
ained.
Howt
hecomponent
sofequat
ion15cameabout
.Par
ti
ti
oni
ngeachnxnmat
ri
xby

backatt her ecur

siont r
ee,withtherecursi
vecaseT( n)=@T( n/2)+Θ(n)
.The
factorof2det er
mi nedhow manychi ldr
eneachtreenodehad,whi chinturn
determinedhowmanyt ermscont r
ibutedtothesum ateachleveloft
hetr
ee.If
wewer etoignoret hefactorof8inequat i
on(14)
,therecur
siontreewoul
djust
belinear,andeachl evelwouldcont
ributeonl
yonetermtothesum.

Bearinmind,therefor
e,thatal
thoughasymptot
icnot
ati
onsubsumesconst
ant
mult
ipl
i
cat
ivefactors,r
ecur
siv
enotati
onsuchasT(n/
2)doesnot
.
Themast
ermet
hodf
orsol
vi
ngr
ecur
rences
Mostt ypicalcaseofdi vide-
and-
conquerapr oblem’si
nstanceofsize n isdiv
idedintotwo
i
nstancesofsi zen/ 2.Moregeneral
ly
,ani nst
anceofsizencanbedi vi
dedi obi
nt nst
ancesof
sizen/b,wi haoft
t hem needi
ngtobesol ved.( e,aandbar
Her econstants;a≥1andb>1. )
Assumingt hatsizeni sapowerofbtosimpl i
fyouranal
ysi
s,wegetthefol
lowingrecur
rencefor
therunningtimeT( n):
T(n)=aT(n/b)+f( n)
, eqn16

Themast
ermet
hoddependsont
hef
oll
owi
ngt
heor
em.
The f(n)isaf unct
ionthataccountsforthetimespentondi vi
dingani nst zeni
anceofsi nt
o
i
nstancesofsizen/bandcombi ni
ngthei
rsolut
ions.(Forthesum exampl eabove,a=b=2andf
(n)=1.)Recurr
ence(**)iscal
ledthegeneraldi
vide-and-
conquerr ecur
rence.
Obviously
,theorderofgrowthofitssol
utonT(
i n)dependsont hev al
uesoftheconst
antsaand
bandt heorderofgr owthoft hefunctonf(
i n).Theef fi
ciencyanalysi
sofmanydi v
ide-
and-
conqueralgori
thmsisgreatl
ysimpli
fi
edbyt hefoll
owingtheorem.

St
atedot
her
wisewehav
e,

Anal
ogousr
esul
tshol
df heOand_not
ort ati
ons,
too.

Forexampl
e,t
her
ecur
rencef
ort
henumberofaddi
tonsA(
i n)madebyt
hedi
vi
de-
and-
conquer
sum-
comput
ati
onal
gor
it
hm (
seeabov
e)oni
nput zen=2ki
sofsi s
 A(
n)=2A(
n/2)+1.
Thus,
fort
hisexampl
e,a=2,
b=2,
andd=0;
hence,
sincea>bd,

Usi
ngt
hemast
ermet
hod
Tousethemastermet
hod,wesimpl
ydet
ermi
newhi
chcase(
ifany
)oft
hemast
ert
heor
em
appl
i
esandwr
it
edowntheanswer
.

Let
’sconsi
dert
hef
oll
owi
ngexampl
es:
i
ntot
hegapbet
weencase2andcase3.

Recal
lourear
li
erequat
ion14:

SUMMARY

Di
vi
de-
and-
conqueri
sagener
alal
gor
it
hm desi
gnt
echni
quet
hatsol
vesapr
obl
em bydi
vi
dingi
t
i
ntosev
eralsmal
l
ersubpr
obl
emsoft
hesamet
ype(
ideal
l
y,ofaboutequalsi
ze)
,sol
vi
ngeachof
t
hem r
ecur
siv
ely
,andt
hencombi
ningt
hei
rsol
uti
onst
ogetasol
uti
ont
otheor
igi
nal
probl
em.
Manyef
fi
ciental
gor
it
hmsar
ebasedont
hist
echni
que,al
thoughi
tcanbebot
hinappl
i
cabl
eand
i
nfer
iort
osi
mpl
eral
gor
it
hmi
csol
uti
ons.
Runni
ngt
imeT(
n)ofmanydi
vi
de-
and-
conqueral
gor
it
hmssat
isf
iest
her
ecur
rence:
T(
n)=aT(
n/b)+f(
n).
TheMast
erTheor
em est
abl
i
shest
heor
derofgr
owt
hofi
tssol
uti
ons.

Mer
gesor
tisadi
vi
de-
and-
conquersor
ti
ngal
gor
it
hm.I
twor
ksbydi
vi
dingani
nputar
rayi
ntot
wo
hal
ves,
sor
ti
ngt
hem r
ecur
siv
ely
, henmer
andt gingt
het
wosor
tedhal
vest
ogett
heor
igi
nalar
ray
sor
ted.Theal
gor
it
hm’
sti
meef
fi
ciencyi
sin_
(nl
og n)i
nal
lcases,wi
tht
henumberofkey
compar
isonsbei
ngv
erycl
oset
othet
heor
eti
calmi
nimum.I
tspr
inci
paldr
awbacki
sasi
gni
fi
cant
ext
rast
orager
equi
rement
.

Qui
cksor
tisadi
vi
de-
and-
conquersor
ti
ngal
gor
it
hm t
hatwor
ksbypar
ti
ti
oni
ngi
tsi
nputel
ement
s
accor
dingt
othei
rval
uer
elat
ivet
osomepr
esel
ect
edel
ement
.Qui
cksor
tisnot
edf
ori
tssuper
ior
ef
fi
ciencyamong n l
og n al
gor
it
hmsf
orsor
ti
ng r
andoml
yor
der
ed ar
ray
sbutal
so f
ort
he
quadr
ati
cwor
st-
caseef
fi
ciency
.
Dy
nami
cPr
ogr
ammi
ng(
DP)

Dy namicpr ogrammi ngisanal gorit

hm designtechniquewi thar atherinteresti
nghistor
y .Itwas
i
nv entedbyapr ominentU. S.mat hematici
an,RichardBel l
man, i
nt he1950sasagener almet hod
foropt i
mizingmul ti
stagedeci sionprocesses.Thus, t
hewor d“ progr ammi ng”i nthenameoft hi
s
techniquest andsfor“planning”anddoesnotr efertocomput erprogr ammi ng.
Aft
erpr ovingitswor thasani mpor t
anttoolofappl iedmat hemat i
cs,dynami cprogrammi ng
hasev ent
ual l
ycomet o beconsi der
ed,atl easti n comput ersci enceci rcl
es,asagener al
algori
thm desi gntechniquet hatdoesnothav et obel imitedt ospeci altypesofopt i
mi zati
on
problems.I tisfr
om thispointofv iewthatwewi llconsiderthistechni queher e.

DP,l i
ke t he DAC met hod,sol ves problems by combi ni
ng the sol uti
ons to subpr obl
ems.
(“
Programmi ng”inthiscont extrefer
st oat abularmet hod,nott owrit
ingcomput ercode.
)Aswesaw
abovet heDAC al gorit
hmspar t
iti
ont heproblem i nt
odi sjoi
nt( i
ndependent)subpr obl
ems,sol vethe
subproblemsr ecursi
vely,andthencombi netheirsoluti
onst osol vetheori
ginalproblem.
In cont rast,DP appl i
es when t he subpr obl
ems ov er
lap—that is,when subpr oblems share
subsubproblems.I nthiscontext,aDACal gor
ithm doesmor ewor kthannecessary,repeat
edlysolvi
ngthe
commonsubsubpr oblems.
ADPal gori
thm solveseachsubsubpr obl
em j ustonceandt hensavesi t
sansweri nat abl
e,ther
eby
avoi
dingt hewor kofrecomput ingtheanswerev erytimei tsolveseachsubpr oblem.
DPi
sther
efor
eat
echni
quef
orsol
vi
ngpr
obl
emswi
thov
erl
appi
ngsubpr
obl
ems.

Typi
cal
ly,thesesubproblemsar i
sef rom ar ecurr
encer el
atingagi v
enpr obl
em’ssolut
ionto
sol
uti
onsofi t
ssmallersubproblems.
Weusual lyapplyDP t o optimizat
ionproblems.Suchpr obl
emscanhav emanypossi bl
e
sol
uti
ons.Eachsoluti
onhasav alue,andwewi sht ofi
ndasol uti
onwiththeopti
mal (
mini
mum or
maximum)v alue.Wecallsuchasol ut
ionanopt imalsol
ut i
ont othepr
oblem,asopposedtothe
opti
malsoluti
on,si
ncetheremaybesev er
alsoluti
onsthatachievetheopti
mal v
alue.

Whendev elopingady nami c-pr

ogr ammi ngal gori
thm, wef oll
owasequenceoffoursteps:
1.Characterizethest ructur eofanopt imalsolution.
2.Recursivelydef i
net hev alueofanopt i
mal solution
3.Comput et hev alueofanopt i
mal solut
ion,typicallyinabot
tom-upfashi
on
4.Constructanopt imal solutionfrom comput edi nformati
on.
Steps1–3f ormt hebasi sofaDPsol ut
iont oapr oblem.I fweneedonlyt heval
ueofanopt i
mal
soluti
on,andnott hesol utioni t
sel f
,thenwecanomi tst ep4.Whenwedoper form step4,we
somet i
mesmai ntainaddi t
ionali nformationdur i
ngst ep3sot hatwecaneasi l
yconst r
uctan
optimalsol
ution.

Theil
lustrat
ionst hatfoll
owuset heDPmet hodtosolv
esomeopt i
mizat
ionproblems.
Thi
st echni quecanbei ll
ust
ratedbyrevi
siti
ngtheFibonacci
.TheFibonaccinumbersar
ethe
el
ement soft hesequence:
0, 1,1,2,3,5,8,13,21,34,
..,whichcanbedefinedbythesi
mpl erecurr
ence
F(n)=F( n−1)+F( n−2)f orn>1 Eqn.18
andtwoi nit
ialconditi
ons
F(0)=0, F(1)=1. Eqn.19

Ifwet ryt ouser ecurr

ence( eqn.18)di
rectl
yt ocomput ethent hFibonaccinumberF(n),wewould
hav etor ecomput ethesamev aluesofthi
sf unct
ionmanyt i
mes.
Not et hatthepr oblem ofcomput ngF(
i n)isexpr essedi ntermsofi t
ssmal l
erandov erl
appi
ng
subpr oblemsofcomput i
ng:F(n−1)andF( n−2) .
Sowecansi mplyfil
lel ement
sofaone- di
mensi onalarraywi t
ht hen+1consecut i
ve
valuesofF( n)byst art
ing,i
nv iewofiniti
alconditi
ons( eqn.19),wit
h0and1andusi ngequati
on
(eqn.18)ast heruleforproducingallt
heot herelements.Obv iousl
y,thelastel
ementoft hi
sarr
ay
willcont nF(
ai n)
.
TheFi
bonacci
algor
it
hm i
sgi
venasf
oll
ows.
ALGORITHM Fib(n)
//
Comput esthenthFibonaccinumberi
ter
ati
vel
ybyusi
ngi
tsdef
ini
ti
on
//
Input
:Anonnegat i
veintegern
//
OutputThent
: hFibonaccinumber
F[
0]←0;F[1]←1
ori
f ondo
←2t
F[i
]←F[
i−1]
+F[
i−2]
ret
urnF[
n]

Sowhetheroneusestheclassi
calbot t
om- upv ersionofDPorit
stop-downvar i
ati
on,t
hecrucial
stepindesi
gni
ngsuchanal gor
ithm remainst hesame:der i
vi
ngar ecurr
encerelat
ingasolut
ion
totheprobl
em t
osoluti
onsofit
ssmal lersubpr oblems.
Thei
mmediateavail
abi
li
tyofequat ion( eqn.18)forcomputi
ngthent hFibonaccinumberis
oneofthefewexcepti
onstothi
sr ul
e.

Sinceamaj or
it
yofDPappli
cati
onsdealwi
thopti
mizat
ionpr
oblems,wealsoneedtomentiona
generalpri
nci
plet
hatunderl
i
nessuchappli
cati
ons.Ri
chardBell
mancall
edi tthepr
inci
pleof
optimali
ty.
I
tsaysthatanopt
imalsolut
iontoanyi
nst
anceofanopt i
mizat
ionpr
oblem i
scomposedof
optimalsol
uti
onst
oit
ssub-i
nstances.

Let
’sdi
scussDPmor
evi
aty
pical
exampl
es.

Example1:Coin-r
owpr obl
em
Letssupposewehav earowofncoinswhosevaluesaresomeposit
iveint sc1,
eger c2,...,cn,
notnecessari
lydi
stinct
.Thegoalistopi
ckupt hemaxi mum amountofmoneysubjecttothe
constr
aintthatnot wocoinsadjacenti
nt heinit
ialr
ow canbepi ckedup.LetF(n)bet he
maximum amountt hatcanbepi
ckedupfr
om therowofncoins.

Toder iv
ear ecurrenceforF(n),
wepar t
iti
onalltheall
owedcoi nselecti
onsi nt
otwogroups:
t hose
thatincl
udet helastcoinandthosewithoutit.
Thelargestamountwecangetf r
om thef i
rstgroupisequalt ocn+F( n−2) ;t
hev al
ueoft he
nthcoinpl usthemaxi mum amountwecanpi ckupf rom thefi
rstn−2coi ns.
Themaxi mum amountwecangetf rom thesecondgr oupisequaltoF( n−1)bythedefinit
ion
ofF( n)
.Thus, wehav ethefol
lowingr
ecurrencesubjecttot hei
niti
alcondi
tions:
F(
n)=max{
cn+F(
n−2)
,F(
n−1)
}forn>1,
F(
0)=0, F(1)=c1. (Eqn20)
WecancomputeF(n)byf il
li
ngtheone- r
owt ablelefttoright.
ALGORI THM Coi nRow( C[
1..n]
)
//
Appliesf ormula(20)bottom upt of i
ndthemaxi mum amountofmoney
//
thatcanbepi ckedupf r
om acoi nr owwi thoutpicki
ngtwoadjacentcoi
ns
//
Input:ArrayC[1..n]ofpositi
veint
eger sindicati
ngthecoinval
ues
//
Output :Themaxi mum amountofmoneyt hatcanbepickedup
F[
0]←0; F[1]←C[ 1]
ori
f ondo
←2t
F[i
]←max(C[
i+F[
] i−2]
,F[
i−1]
)
ret
urnF[
n]
Theappl
i
cat
ionoft
healgor
it
hm tothecoi
nrowofdenomi
nat
ions5,
1,2,
10,
6,2i
sshownbel
ow.
I
tyiel
dst
hemaximum amountof17.

FI
GURE:Sol
vi
ngt
hecoi
n-r
owpr
obl
em byDPf
ort
hecoi
nrow5,
1,2,
10,
6,2.

Itisworthpointi
ngoutt hat
,infact
,weal sosolvedthepr
oblem forthefi
rsticoi
nsintherow
giv
enforevery1≤i ≤6.Forexample,ori
f =3,t
hemaximum amounti sF(3)=7.
Tofindthecoinswiththemaximum totalval
uefound,weneedtobacktracethecomput
ationst
o
seewhichofthetwopossi bi
li
ti —cn+F(
es: n−2)orF(n−1)—producedthemaxima

Inthelastappl i
cationofthef ormula,i
twast hesum c6+F( 4),
whichmeanst hatthecoi nc6=2
i
sapar tofanopt imalsoluti
on.Mov i
ngt ocomput ngF(
i 4),themaximum waspr oducedbyt he
sum c4+F( 2),whichmeanst hatthecoi nc4=10i sapar tofanoptimal soluti
onaswel l
.
Finall
y,themaxi mum incomput ingF( 2)waspr oducedbyF( 1),i
mpl yingthatthecoi nc2is
notthepar tofanopt imalsolutionandt hecoi nc1=5i s.Thus, t
heoptimal soluti
oni s{c1,c4,c6}
.
Toav oidr epeati
ngthesamecomput at ionsduringthebackt r
acing,thei nformationabout
whichoft het wot ermsin( eqn20)wasl argercanber ecordedinanext raar r
aywhent hev al
ues
ofFarecomput ed.
Usingt heCoi nRow tof ndF(
i n),thel argestamountofmoneyt hatcanbepi ckedup,as
wellasthecoi nscomposi nganopt i
mal set,
cl earl
ytakesΘ( n)t
imeandΘ( n)space.

Example2:Change- makingpr obl

em
Considerthegener alinstanceoft hef ol
lowi ngwel l
-knownpr oblem:Gi vechangef oramountn
usi
ng t he mi ni
mum numberofcoi ns ofdenomi natons d1<d2 <...
i <dm.Fort he coi
n
denominationsusedi ntheUni tedStates,asf orthoseusedi nmosti fnotallot hercountr
ies.
Here,weconsi deraDPal gorit
hm f ort hegener alcase,assumingav ailabil
it
yofunlimit
ed
quanti
ti
esofcoi nsf
oreachoft hem denomi nationsd1<d2<... <dm wher ed1=1.
LetF(n)bet hemi ni
mum numberofcoi nswhosev al
uesaddupt on;itisconv enienttodefneF(
i 0)
=0.Theamountncanonl ybeobt ainedbyaddi ngonecoi nofdenomi nat
iondjt ot heamountn−
djforj=1, 2,...,m sucht hatn≥dj .
Ther
efore,
wecanconsideral
lsuchdenominati
onsandsel
ectt
heonemini
mi ngF(
zi n−dj)+1.
Si
nce1i saconst
ant,wecan,ofcourse,f
indthesmalestF(
l n−dj)fi
rstandthenadd1toi
t.
Hence,wehav
ethefoll
owingr
ecurr
enceforF(n)
:

Eqn21
WecancomputeF(
n)byfi
ll
i
ngaone-
rowtablel
eftt
ori
ghti
nthemannersimilart
othewayitwas
doneabov
efort
hecoi
n-r
owprobl
em,butcomputi
ngatabl
eentryher
erequi
resfi
ndi
ngthe
mini om number
mum ofupt s.

ALGORITHM ChangeMaking(D[1.
.m],n)
//Appli
esDOtofi
ndt hemini
mum numberofcoi nsofdenominat
ions:d1<d2<...<dm
where
//d1=1t hataddupt oagivenamountn
//I
nput:Posi
ti
vei egernandar
nt ayD[
r 1.
.m]ofincreasi
ngposi
ti
veinteger
sindi
cat
ingt
he
coin
//denominati
onswher eD[
1]=1
//Output
:Theminimum numberofcoinsthataddupt on
F[0]
←0
ori
f ondo
←1t
t
emp← j←1
;
whi
lej
≤m andi
≥D[
j]do
t
emp← mi
n(F[
i−D[
j],
]temp)
j←j+1
F[
i←t
] emp+1
r
et nF[
ur n]

Theapplicat
ionoft heal
gori
t oamountn=6anddenomi
hm t nat
ions1,
3, 4i
sshownbel
ow.The
answerity i
eldsistwocoins.Theti
meandspaceef
fi
cienci
esofthealgor
it eO(
hm ar nm)and
Θ(n)
,respecti
v el
y.
FI
GURE:Appl
i
cat
ionofAl
gor
ihm Mi
t nCoi
nChanget
oamountn=6andcoi
ndenomi
nat
ions1,
3,and4.

Tof i
ndthecoinsofanoptimalsol
uti
on,weneedtobacktracethecomputat
ionst
oseewhi
chof
thedenominationspr
oducedtheminimain(eqn21)
.
Fortheinstanceconsider
ed,thel
astappli
cati
onoftheformula(forn=6) ,t
hemini
mum was
producedbyd2=3.Thesecondmi ni
mum ( orn=6−3)wasal
f soproducedforacoi
nofthat
denominati
on.Thus,themini
mum-coinsetforn=6istwo3’s.

TheGr
eedyTechni
que
Letusr ev isitt hechange- maki ngpr oblem f aced,bymi l
lionsofcashi er sal lov ert hewor ld:gi ve
changef oraspeci f camountnwi
i tht heleastnumberofcoi nsoft hedenomi nat onsd1>d2>..
i
.>dm usedi nt hatl ocal e.( Her e,weassumet hatt hedenomi nat i
onsar eor der edi ndecr easi ng
order .)Forexampl e,t hewi delyusedcoi ndenomi nationsi nt heUni tedSt atesar e d1=25
(quar ter), d2=10( di me) ,d3=5( nickel ,
)andd4=1( penny )
.
Howwoul dy ougi v echangewi thcoi nsoft hesedenomi nationsof ,say ,48cent s?
Ifyoucameupwi tht heanswer1quar ter,2di mes,and3penni es,y ouf oll
owedconsci ousl yor
notal ogi calst rategyofmaki ngasequenceofbestchoi cesamongt hecur rentlyav ailable
alt
er nat ives.I ndeed,i nt he f irstst ep,y ou coul d hav e gi v en one coi n ofanyoft he f our
denomi nat ions.
“Gr eedy ”thinki ngl eadst ogi vingonequar terbecausei tr educest her emai ni ngamountt he
most ,namel y ,to23cent s.I nthesecondst ep, youhadt hesamecoi nsaty ourdi sposal , buty ou
coul dnotgi veaquar ter,becausei twoul dhav ev iol
at edt hepr oblem’ sconst raint s.Soy ourbest
select i
oni nt hisst epwasonedi me, reducingt her emai ningamountt o13cent s.Gi vi
ngonemor e
dimel efty ouwi th3cent st obegi venwi t
ht hr eepenni es.
I
st hissol utiont othei nstanceoft hechange- maki ngpr obl em opt i
mal ?
Yes,i ti s.I nf act,onecanpr ov et hatt hegr eedyal gor it
hm y ieldsanopt imalsol uti
onf orev ery
posi t
iv ei nt egeramountwi ththesecoi ndenomi nations.Att hesamet i
me,i ti seasyt ogi vean
exampl eofcoi ndenomi nat ionst hatdonoty ieldanopt imalsol utionf orsomeamount se. g.,d1=
25, d2=10, d3=1andn=30.
Theappr oachappl iedi nt heopeni ngpar agrapht ot hechange- maki ngpr oblem i scal led
greedy .Comput ersci entist sconsi deri tagener aldesi gnt echni quedespi tet hef actt hati tis
appl i
cabl et oopt imi zat i
onpr obl emsonl y.
Thegr eedyappr oachsuggest sconst ruct i
ngasol ut i
ont hroughasequenceofst eps, each
expandi ngapar ti
allyconst ruct edsol uti
onobt ainedsof ar,unt i
lacompl etesol utiont ot he
probl em i sr eached.
Oneachst ep—andt hisi st hecent ral pointoft hi stechni que—t hechoi cemademustbe:
o f easi bl e, i.
e.,ithast osat i
sfyt hepr oblem’ sconst raints
o l ocal lyopt imal , i.
e., i
thast obet hebestl ocalchoi ceamongal lfeasi blechoi cesav ailableon
thatst ep
o i rrev ocabl e,i.e.,oncemade, itcannotbechangedonsubsequentst epsoft heal gorithm
Theser equi rement sexpl aint het echnique’ sname:oneachst ep, itsuggest sa“ greedy ”gr ab
ofthebestal ternat iveav ailablei nthehopet hatasequenceofl ocallyopt imalchoi ceswi llyielda
(globall
y)opti
malsol
ut i
ont otheentir
epr obl
em.
Wer efr
ainfr
om aphi losophi
caldiscussi
onofwhet hergreedisgoodorbad.(
Ify
ouhavenot
seent hemov iefr
om whicht hetopic’
sepi gr
aphistaken,it
sherodidnotendupwell.
)Fr
om our
algori
thmicperspect
ive,thequesti
oni swhethersuchagr eedystr
ategywor
ksornot.

Asweshal lsee,therearepr oblemsf orwhi chasequenceofl ocall

yopt i
malchoi cesdoesy ield
anopt i
malsol ut
ionf oreveryinstanceoft hepr oblem inquest i
on.Howev er,t herear eot her
sf or
whicht hisisnott hecase;f orsuchpr oblems,agr eedyalgor i
thm canst i
llbeofv alueifwear e
i
nterestedi norhav etobesat isfi
edwi thanappr oximat esolut i
on.
Asar ul
e,greedyal gori
thmsar ebothi ntui
tivel
yappeal ingandsi mple.Giv enanopt imizati
on
problem,i tisusual l
yeasyt of igureouthow t opr oceedi nagr eedymanner ,possi blyafter
consideringaf ewsmal li
nstancesoft hepr oblem.
Whati susual l
ymor ediff
icultistopr ov ethatagr eedyal gori
thm yiel
dsanopt i
malsol uti
on
(wheni tdoes) .Oneoft hecommonway st odot hisisusi ngmat hemat icali nduction,toshow
thatapar ti
all
yconst ruct
edsol utionobtainedbyt hegr eedyal gori
thm oneachi terati
oncanbe
extendedt oanopt i
mal soluti
ont othepr oblem.
Thesecondwayt opr oveopt imali
tyofagr eedyal gori
t hm istoshow t hatoneachst epi t
doesatl eastaswel l
asanyot heralgorit
hm coul dinadv ancingt owardthepr obl em’sgoal .

AnotherExampl e
l
et’
sconsi dert hepr oblem off i
ndingt hemi ni
mum numberofmov esneededf orachesskni ght
togof rom onecor nerofa100×100boar dt othedi agonallyopposi tecorner.(Thekni ght’
s
mov esar eL-shapedj umps:t wosquar eshor i
zontall
yorv erti
callyfollowedbyonesquar ei nthe
perpendiculardirecti
on.)
Agr eedysol uti
oni sclearher e:jumpascl oset othegoalaspossi bl
eoneachmov e.Thus,
i
fitsstartandf inishsquar esare(1,1)and( 100,100) ,respectively,asequenceof66mov essuch
as(1,1)−( 3, 2)−( 4,4)−...−( 97, 97)−( 99,98)−( 100, 100)sol vest heprobl
em.( Thenumberk
oftwo-mov eadv ancescanbeobt ainedf r
om theequat on1+3k=100.
i
Whyi sthisami nimum- mov esolution?
Becausei fwemeasur et hedi stancet othegoalbyt heManhat t
andistance,whichi sthe
sum oft hedi fferencebet ween t her ow number sand t hedi fferencebetween t hecol umn
number soft wosquar esinquest i
on,t hegreedyal gorithm decr easesi tby3oneachmov e;t he
bestthekni ghtcando.
Thet hirdwayi ssimpl ytoshow t hatthef i
nalr esultobt ainedbyagr eedyal gori
thm is
opti
mal basedont healgorit
hm’sout putratherthant hewayi toperates.

Anot herexampl e:consi derthepr oblem ofplaci

ngt hemaxi mum numberofchi psonan8×8
boar dsot hatnot wochi psar epl acedont hesameoradj acent-v er
ti
call
y ,hor
izontal
ly,or
diagonallysquar es.
Tof ol
lowt hepr escripti
onoft hegreedyst r
ategy ,
weshoul dplaceeachnewchi psoast o
l
eav easmanyav ai
lablesquar esaspossi blefornextchi ps.Forexample,start
ingwitht heupper
l
ef tcorneroft heboar d, wewi l
lbeabl etoplace16chi psasshowni nFi
gure( a)
.
Whyi sthissolutionopt i
mal ?
Toseewhy ,par ti
ti
ont heboar dintosixteen4×4squar esasshowni nFi gure( b)below.
Obv iousl
y ,i
tisimpossi bletopl acemor ethanonechi pineachoft hesesquar es,whichimpl i
es
thatt hetotalnumberofnonadj acentchipsont heboar dcannotexceed16.
FI
GURE(a)Pl
acementof16chi
psonnon-
adj
acentsquar
es.(
b)Par
ti
ti
onoft
heboar
dpr
ovi
ng
i
mpossi
bil
i
tyofpl
aci
ngmor et
han16chi
ps.

Asaf i
nalcomment ,weshould menti
ont hatarathersophisti
cat
ed theor
yhasbeen
developedbehi ndthegreedyt
echni
que,whichisbasedont heabst
ractcombinator
ialst
ruct
ure
call
ed“ matr
oid.”
Aninterestedreadercancheckbookssuchas[ Cor
09]aswel lasavar i
etyofInter
net
resourcesont hesubject
.

Huf
fmanTr
eesandCodes
Supposewehavetoencodeat extt
hatcompr
isessymbol
sfrom somen- sy
mbolal
phabetby
assi
gningt
oeachoft
hetext
’ssy
mbolssomesequenceofbi
tscal
l
edt hecodewor
d.

Wecanuseaf ixed-l
engthencodi ngthatassignstoeachsy mbolabi tst ri
ngofthesamel ength
m( m ≥log2n).Thi si
sexact lywhatthestandardASCI Icodedoes.
Onewayofget ti
ngacodi ngschemet hatyieldsashorterbi tstri
ngont heaver
agei s
basedont heol dideaofassi gni
ngshortercodewor dst omoref requentsy mbol sandlonger
codewordst ol essfrequentsy mbols.Thisideawasused,i npar t
icular,inthetelegr
aphcode
i
nventedinthemi 19thcent
d- urybySamuel Mor se.
Inthatcode,frequentlett ssuchase(
er .
)anda( .
−)ar eassi
gnedshor tsequencesofdotsand
dasheswhileinf r
equentlett
erssuchasq( −−.−)andz( −−..
)havelongerones.

Anotheroptioni stheuseofVar iable-

lengt
hencodi ng,whichassi gnscodewor dsofdi ffer
ent
l
engthstodi ff
er entsymbol s.Thisintr
oducesapr obl em thatf
ixed-
lengthencodingdoesnothav e.
Namel y,how canwet el
lhow manybi t
sofanencodedt extrepresentthef ir
st( or,more
general
ly,hei
t th)sy mbol
?
Toav oidt hiscompl icati
on,wecanl i
mi tour selvestotheso- cal
ledpr ef
ix-
free( orsimply
pref
ix)codes.I napr efi
xcode, nocodewor disapr efixofacodewor dofanot hersymbol .
Hence, wi t
hsuchanencodi ng,wecansi mpl yscanabi tstri
ngunt i
lwegett hef irstgroup
ofbitsthatisacodewor df orsomesy mbol ,replacet hesebi t
sbyt hissymbol ,andr epeatt hi
s
operati
onunt i
lt hebitst
ring’sendi sreached.

Ifwewantt ocreateabi nar

ypr ef
ixcodef orsomeal phabet,iti snormaltoassociat
ethe
alphabet’ssy
mbol swithleavesofabi nar
yt r
eeinwhi challtheleftedgesarelabel
ledby0andall
ther i
ghtedgesarelabell
edby1.
Thecodewordofasy mbolcant henbeobt ainedbyr ecordingthelabel
sont hesimpl
e
pathf r
om theroottothesy mbol’
sleaf.
Si
ncether
ei snosimpl epathtoal eafthatcontinuestoanot herl
eaf,nocodewordcanbe
apr efi
xofanothercodewor d;hence,anysucht r
eey i
eldsapr efi
xcode.

Amongt hemanytr
eesthatcanbeconst r
uct
edint
hismannerf
oragi
venal
phabetwit
hknown
fr
equenciesoft
hesymboloccur
rences,howcanweconst
ructatr
eet
hatwoul
dassignshor
ter
bi
tstri
ngstohigh-
fr
equencysymbolsandlongeronestol
ow-fr
equencysymbol
s?
Itcanbedonebyt hef
oll
owinggreedyalgor
ithm,i
nvent
edbyDav i
dHuff
manwhi
l
ehewas
agraduatest
udentatMIT.

Huffman’salgor
it
hm
Step1I ni
ti
alzenone-
i nodet
reesandlabelthem wit
ht hesymbol
softhealphabetgi
ven.Record
thefrequencyofeachsymboli
nitst
ree’sroottoindi
cat et
hetr swei
ee’ ght.(Moregeneral
l
y,the
weightofat r
eewil
lbeequalt
othesum ofthefrequenciesi
nthetr
ee’
sleav
es.)

St
ep2Repeatthefoll
owingoper
ati
onunt il
asingletr
eeisobtai
ned.
o Findtwotreeswit
hthesmallestweight(t
iescanbebrokenarbi
trar
il
y.
)
o Maket hem thel
eftandri
ghtsubtreeofanewt r
eeandrecor
dt hesum oft
hei
rwei
ght
sin
therootofthenewtreeasit
swei ght.

At r
eeconstr
uctedbyt
heabovealgor
it
hm i edaHuf
scal
l fmant
ree.I
tdef
ines—i
nthemanner
descri
bedabove—aHuf
fmancode.

EXAMPLE:Consi
derthefiv
e-sy
mbolalphabet{A,B,C,D,_
}wi
tht
hef
oll
owi
ngoccur
rence
fr
equenci
esi
natextmadeupofthesesy
mbols:

TheHuf
fmant
reeconst
ruct
ionf
ort
hisi
nputi
sshowni
nFi
gur
ebel
ow.
FI
GURE:Exampl
eofconst
ruct
ingaHuf
fmancodi
ngt
ree.

Ther
esul
ti
ngcodewor
dsar
easf
oll
ows:

Hence,DADisencodedas011101, and10011011011101isdecodedasBAD_ AD.

Withtheoccur
rencefrequenci
esgivenandthecodewor dlengt
hsobtai
ned,t
heaver
agenumber
ofbit
spersymbol i
nthiscodeis(
no.ofbits*%occurrence):
( 0.
2* 35)+(
3* 0.
1)+(
2*0.
2)+(
2*0.2)+(3*0.
15)=2.25.

Hadweusedaf i
xed-lengthencodingforthesameal phabet
,wewoul dhav etouseatl east3bits
pereachsymbol .
Thus,fort
histoyexampl e,Huff
man’scodeachievest hecompressionrat
io—ast andardmeasur e
ofacompr essi
onal gorit
hm’ seff
ecti
venessof(3−2.25)/3*100%=25%.
Inotherwords,Huf fman’ sencodingofthetextwil
luse25%l essmemor ythanitsf i
xed-l
ength
encoding.
Ext
ensiveexper i
ment swi t
hHuf f
mancodeshav eshownt hatthecompr essionr at
iofor
thi
sschemet ypi
call
yf allsbetween20%and80%,dependi ngont hecharacter
isti
csoft hetext
bei
ngcompr essed.

Huff
man’sencodi
ngisoneoft
hemostimportantf
ile-
compressi
onmet hods.
I
nadditi
ontoi t
ssimpli
cit
yandver
sati
li
ty,i
ty i
eldsanopt i
mal,i.
e.,minimal-
length,encodi
ng
(
provi
dedthefr
equenci
esofsy
mboloccur
rencesar ei
ndependentandknowni nadv ance)
.

ThesimplestversionofHuf fmancompr essioncal l

sfor:
o Apr el
iminar
yscanni ngofagi ventextt
ocountt hefrequenciesofsymbol occurrencesi
nit.
o Thent hesefrequenciesar eusedtoconst ructaHuf fmancodi ngtreeand
o Encodet hetextasdescr ibedabov e.
Thisschememakesi tnecessar y,however,toincludet hecodingtableintot heencodedtextto
makei tsdecodi ng possible.Thi sdrawbackcan beov er
comebyusi ng dy namicHuf f
man
encoding,inwhicht hecodingt reeisupdat edeacht i
meanewsy mboli sreadf r
om thesource
text
.
Furt
her,moder nal t
ernatiessuchas Lempel
v -Zivalgor i
thms( e.
g.,assigncodewor dsnott o
i
ndivi
dualsy mbolsbutt ost ri
ngsofsy mbols,allowingt hem toachievebetterandmor er
obust
compressionsinmanyappl i
cat i
ons.
 TheLi
mit
ati
onsofAl
gor
it
hm Power
I
ntel
lectdi
sti
nguishesbetweent hepossibleandtheimpossibl
e;
reasondist
inguishesbetweent hesensibleandthesensel
ess.
Event hepossibl
ecanbesenseless.
—MaxBor
n(1882–1970)
,MyLi
feandMyVi
ews,
1968

Inthepr ecedingsect i
ons,weencount eredsomeal gori
thmsf orsolv
ingav ari
etyofdi f
fer
ent
problems.Af airassessmentofal gor
it
hmsaspr oblem-solvingtoolsisinescapabl e:theyar
every
power ful
inst
rument s,especiall
ywhentheyar eexecutedbymoder ncomput ers.
Butt hepowerofal gori
thmsi snotunli
mi t
ed,anditsli
mi t
sar ethesubjectoft hissect i
on.
Asweshal lsee,somepr oblemscannotbesol vedbyanyal gori
thm.Ot herpr oblemscanbe
solvedalgori
thmi cal
lybutnoti npol
ynomi alti
me.Andev enwhenapr obl
em canbesol vedin
polynomialti
mebysomeal gor i
thms,
therear eusuall
ylowerboundsont hei
ref fi
ciency.

Asubsectiondealswit
ht hequest ionofint
ract
abili
ty:whichprobl
emscanandcannotbesol v
ed
i
n polynomialtime.This wel l
-devel
oped area oft heor
eti
calcomput erscience is cal
l
ed
computati
onalcomplexi
tyt heory.
Wepr esentthebasicel ement softhistheoryanddi scussinf
ormallysuchfundamental
not
ionsasP, NP,andNP- compl eteprobl
ems, i
ncludi
ngt hemostimpor t
antunresol
vedquesti
on
oft
heoreti
calcomputerscienceaboutt herel
ati
onshipbet weenPandNPpr obl
ems.

P,
NP,
andNP-
Compl
etePr
obl
ems
Inthestudyofthecomputati
onalcompl
exi
tyofpr
obl
ems,t
hefi
rstconcer
nofbothcomput
er
sci
enti
stsandcomputingpr
ofessi
onal
siswhet
heragi
venpr
obl
em canbesolv
edinpol
ynomi
al
ti
mebysomeal gori
thm.

DEFINITI
ON1
Wesayt hatanalgori
thm solvesapr obl
em inpoly nomialti
mei fi
tsworst-caseti
meef fi
ciency
bel
ongstoO( p(n)
)wherep( n)isapolynomialofthepr oblem’soni zen.(
nputsi Notet
hatsi nce
wear eusingbig-
ohnot at
ionher e,pr
oblemssol vablein,say,l
ogari
thmict i
mear esolvablein
poly
nomial t
imeaswell.
)
Problemsthatcanbesol vedinpolynomialtimear ecaledt
l ract
able,andproblemst hat
cannotbesolvedi
npolynomi alt
imearecaledi
l ntractable.

Ther ear esev eral r

easonsf ordr awi ngt hei ntractabili
t ylinei nt hisway .
o Fi rst,thet ableofappr oxi mat edv aluesofsomef unct ioni npr ev i
oussect ionimpl yt hatwe
cannotsol v ear bit
raryi nst ancesofi ntractablepr obl emsi nar easonabl eamountoft ime
unlesssuchi nstancesar ev erysmal l.
o Second,al thought heremi ghtbeahugedi fferencebet weent her unningt imesi nO( p(n))for
polynomi alsofdr asticallydi f
ferentdegr ees,t her e ar ev er yf ew usef ulpol ynomi al
-t
ime
algorithmswi t
ht hedegr eeofapol ynomi alhighert hant hr ee.I naddi t
ion,pol ynomi alsthat
boundr unni ngtimesofal gor it
hmsdonotusual lyhav eext remel yl argecoef fici
ents.
o Thi r
d,poly nomi alfunct i
onspossessmanyconv eni entpr oper ties;i npar ti
cular,botht hesum
andcomposi ti
onoft wopol ynomi alsar eal way spol ynomi alst oo.
o Four th,thechoi ceoft hiscl asshasl edt oadev elopmentofanext ensiv etheorycal led
comput ationalcompl exi ty,whi chseekst ocl assi fypr obl emsaccor dingt ot hei
ri nherent
diffi
culty.Andaccor dingt ot hist heor y,apr oblem’ si ntract abi l
ityr emai nst hesamef orall
principalmodel s ofcomput ations and al lr easonabl ei nput -encodi ng schemes f ort he
probl em underconsi derat i
on.
Wej ustt ouchonsomebasi cnot ionsandi deasofcompl exityt heor yi nthissect i
on.Ify ouar e
i
nt erestedi namor eformalt reatmentoft hi stheor y,y ouwi llhav enot roublef i
ndingaweal t
hof
textbooksdev ot edtot hesubj ect

PandNPPr
obl
ems
Mostpr oblemswedi scussedsof arcanbesol vedinpolynomialti
mebysomeal gorit
hm.They
i
ncl udecomput ingthepr oductandt hegreatestcommondi vi
soroftwoi nt
egers,sort
ingal i
st,
sear chi
ngf orakeyi nal i
storf orapat t
erninat extst
ri
ng,checkingconnect
ivi
tyofagr aph,and
findingami ni
mum spanni ngt r
eeet c.
Informal l
y,wecant hi
nkaboutpr oblemst hatcanbesol v
edi npolynomialti
meast hesetthat
comput ersciencet heoreti
cianscal lP.A mor ef or
maldef i
nit
ionincl
udesinP onl ydecision
pr oblems,whi chareproblemswi thy es/
noanswer s.

DEFINITI ON2
ClassPi sacl assofdeci sionpr oblemst hatcanbesol v edi npoly nomial t
imeby( det ermi nisti
c)
al
gorithms.Thi sclassofpr oblemsi scal ledpolynomi al.
Theyar epr oblemst hatcanbesol v edi
nt i
meO( nk)f orsomeconst antk, wher eni sthe
si
zeoft hei nputt ot hepr obl em.
Therest ri
ct onofPt
i odeci sionpr oblemscanbej usti
fiedbyt hef oll
owingr easons.
o Fir
st ,iti ssensi blet oexcl udepr oblemsnotsol v ablei npol ynomialt imebecauseoft heir
exponent iallylargeout put .Suchpr oblemsdoar i
senat ur
ally— e.g.,gener atingsubset sofa
givensetoral ltheper mut ationsofndi st
inctitems— buti tisappar entf r
om t heout setthat
theycannotbesol vedi npol ynomi altime.
o Second,manyi mpor tantpr oblemst hatar enotdeci sionpr oblemsi nt heirnat ur alf ormul ati
on
canber educedt oaser i
esofdeci sionpr oblemst hatar eeasiert ostudy .Forexampl e,instead
ofaski ngf orthemi nimum numberofcol orsneededt ocolort hev er
ticesofagr aphsot hatno
twoadj acentv erti
ceshav esamecol or,wecanaskwhet hert hereexistssuchacol oringoft he
graph’ sv erticeswi thnomor ethanm col or
sf orm =1, 2,...(Thel at
teriscal ledt hemcol ori
ng
probl em. )
Thef i
rstv alueofm i nt hisser i
esf orwhi cht hedeci sionpr oblem ofm- col ori
nghasa
solut
ionsol vest heopt i
mi zat i
onv ersionoft hegr aph-color i
ngpr oblem aswel l
.

Onemaywonderwhet herev erydeci si

onpr obl
em canbesol v
edi npolynomialtime.Theanswer
tot hi
squestionturnsoutt obeno.I nfact
,somedeci sionpr obl
emscannotbesol v edatal lby
anyal gori
thm.Suchpr oblemsar ecal edundeci
l dable,asopposedt odecidablepr oblemst hat
canbesol vedbyanal gor i
thm.
Af amousexampl eofanundeci dabl
epr obl
em wasgi venbyAl anTur i
ngin1936.Thepr obl em in
questioniscall
edthehal ti
ngpr oblem:
Givenacomput erpr ogram andani nputtoit
,det er
minewhet herthepr ogram wi llhalton
thatinputorconti
nuewor kingindef i
nitel
yonit.Bel
owi sashor tproofofthi
sr emar kablef act.
Bywayofcont radiction,assumet hatAi sanal gori
thm thatsolvesthehal t
ingpr oblem.
Thatis,foranyprogram Pandi nputI ,

Wecanconsi
derpr
ogram Pasani
nputt
oit
sel
fanduset
heout
putofal
gor
ihm Af
t r(
orpai P,P)
t
oconst
ructapr
ogram Qasfol
l
ows:

Thenonsubst
it
utngQf
i orP,
weobt
ain

Thi
sisacontr
adict
ionbecausenei
theroft
het
woout
comesf
orpr am Q i
ogr spossi
ble,whi
ch
compl
etest
heproof
.

Ar
ether
edeci
dabl
ebuti
ntr
act
abl
epr
obl
ems?
Yes,therear e,butt henumberofknown exampl esissmal l
,especiall
yoft hosethatar i
se
natur
allyr
atherthanbeingconstr
uctedfort
hesakeofat heoreti
calargument.
Therearemanyi mport
antproblems,however,f
orwhichnopol ynomial
-t
imealgori
thm has
beenfound, norhastheimpossi
bil
it
yofsuchanal gori
thm beenproved.
Hereisjustasmal lsampleofsomeoft hebest
-knownproblemst hatfal
li
ntothi
scategory:

Hami lt
oni anci rcuitpr obl em:Det ermi newhet heragivengr aphhasaHami l
tonianci r
cui t—thati s;
apat ht hatst artsandendsatt hesamev ertexandpassest hroughal ltheotherv erticesexact ly
once.
Travel ingsal esmanpr obl em:Findt heshor testt ourt oughnci
hr t
ieswi thknownposi tiveinteger
distancesbet weent hem ( i
.e.
,f i
ndt heshor testHami ltonianci r
cui ti nacompl etegr aphwi th
positivei nt egerwei ght s).
Knapsackpr oblem:Fi ndt hemostv aluablesubsetofni temsofgi venposi ti
veintegerwei ghts
andv aluest hatf itintoaknapsackofagi venposi ti
veintegercapaci t
y.
Par t
itionpr oblem:Gi v ennposi ti
v ei ntegers,det erminewhet heritispossi bletopar tit
iont hem
i
nt otwodi sjointsubset swi t
ht hesamesum.
Bin-packi ngpr oblem:Gi venni temswhosesi zesareposi tiv
er ati
onalnumber snotl ar gerthan1,
putt hem i ntot hesmal lestnumberofbi nsofsi ze1.
Graph- color ingpr oblem:Foragi v engr aph,f inditschr omat i
cnumber ,whichist hesmal l
est
numberofcol or st hatneedt obeassi gnedt othegr aph’ sv er
ticessot hatnot woadj acent
verti
cesar eassi gnedt hesamecol or .
Integerl inearpr ogr ammi ngpr oblem:Fi ndt hemaxi mum ( ormi ni
mum)v al
ueofal i
nearf unction
ofsev er ali nteger -valuedv ari
ablessubj ectt oaf ini
tesetofconst raintsint hef orm ofl inear
equal it
iesandi nequal iti
es.

Someoft hesepr obl emsar edeci sionpr oblems.Thoset hatar enothav edeci sion-
v ersi on
count erparts( e.g.,t hem- color i
ng pr oblem f ort he graph- col oring pr oblem) .Whatal lt hese
problemshav ei ncommoni sanexponent ial(orwor se)gr owt hofchoi ces, asaf unct i
onofi nput
size,from whi chasol ut i
onneedst obef ound.
Not e,howev er,thatsomepr oblemst hatalsof al
lundert hi sumbr ellacanbesol v
edi npolynomi al
ti
me.Forex ampl e,t heEul erianci r
cui tprobl em—t hepr oblem oft heexi stenceofacy clet hat
traversesal ltheedgesofagi v engraphexact l
yonce—canbesol vedi nO( n2)t imebychecki ng, i
n
addi t
iont ot hegr aph’ sconnect iv
ity
,whet heral lt
hegr aph’sv erticeshav eev endegr ees.
Thisexampl ei spar ti
cul arl
yst ri
king:i tisqui tecount erintuitivet oexpectt hatt hepr obl em
aboutcy clest raversi ngal ltheedgesexact lyonce( Euleri
anci r cuits)canbesomucheasi ert han
theseemi ngl ysi mi larpr obl em aboutcy clesv i
sit
ingal lt hev er ti
cesexact lyonce( Hami ltoni an
circuit
s(tourorcy cle)).
Anot hercommonf eat ur eofav astmaj ori
tyofdeci sionpr oblemsi st hef actt hatalthoughsol ving
suchpr oblemscanbecomput ati
onal lydifficult
,checki ngwhet herapr oposedsol uti
onact ual ly
solvest hepr obl em i scomput at i
onallyeasy ,i.
e.,i
tcanbedonei npol ynomi altime.( Wecant hink
ofsuchapr oposedsol utionasbei ngr andoml ygener atedbysomebodyl eav inguswi t
hthet ask
ofv erif
yingi t sv ali
di ty.)
Forexampl e, itiseasyt ocheckwhet herapr oposedl istofv er ti
cesi saHami l
tonianci rcui t
foragi vengr aphwi thnv er ti
ces.Al lweneedt ochecki st hatt hel i
stcont ainsn+1v er
ticesof
thegr aphi nquest ion, thatt hef istnv
r erti
cesar edistinctwher east hel astonei st hesameast he
fi
rst,andt hatev eryconsecut i
v epairoft hel i
st’sver ti
cesisconnect edbyanedge.

Thi
sgeneralobser
vati
onaboutdeci
sionpr
obl
emshasl
edcomput
ersci
ent
ist
stot
henot
ionofa
nondet
ermini
sti
calgor
it
hm.

DEFI
NITION3
Anondeter
ministi
calgor
it
hm isat wo-st
ageproceduret
hattakesasit
sinputani anceIofa
nst
deci
si
onprobl
em anddoest hefoll
owing.
Nondet
ermini
stic(“
guessi
ng”)stage:Anarbi
tr
arystr
ingSisgenerat
edthatcanbethoughtofas
acandidatesol
utiont othegiv
eni nstanceI(butmaybecompl etegi
bber
ish(r
ubbi
sh)aswel l
).
Deter
mi ni
sti
c(“verifi
cati
on”)stage:Adet er
mi ni
stical
gori
thm t hIandSasi
akesbot t
si nputand
out
putsy esifSr epresent
sasol uti
ont oinstanceI.(fSi
I snotasoluti
ontoinstanceI,the
al
gori
thm eit
herreturnsnoori sallowednott ohaltatal
l.
)

Wesayt hatanondet erministicalgor i

thm solv
esadeci si
onpr obl
em i
fandonl yi
fforeveryyes
i
nstanceoftheproblem itretur nsyesonsomeexecut ion.
(I
notherwords,wer equi
reanondet erminist
icalgor
it
hm tobecapableof“guessi
ng”asolut
ionat
l
eastonceandt obeabl et ov eri
fyitsvali
dit
y.And,ofcourse,wedonotwanti ttoeveroutputa
yesansweronani nstancef orwhi cht heanswershouldbeno. )
Fi
nal
l
y ,
anondet ermi ni
st icalgori
thm i
ssai obenondet
dt er
mini
sti
cpolynomiali
fthetime
eff
ici
encyofit
sverif
icati
onst agei spolynomial.

DEFI
NITION4
assNPi
Cl stheclassofdecisi
onpr oblemst hatcanbesol vedbynondet er
mi nisti
cpolynomial
al
gor
ithms.Thisclassofprobl
emsi scalednondet
l er
mini
sticpolynomial.
TheclassNPconsi st
soft hosepr oblemst hatare“
verif
iabl
e”inpolynomi alti
me.
Whatdowemeanbyapr oblem beingv eri
fi
able?
Ifwewer esomehow gi v ena“ cert
ifi
cate”ofasol uti
on,t henwecoul dv eri
fythatthe
cert
if
icat
eiscorrecti
ntimepol ynomi alinthesizeoftheinputtothepr oblem.

Mostdeci sionpr oblemsar ei nNP.Thi scl assi ncludesallthepr oblemsinP: P⊆NP.

Thi
si st r
uebecause,i fapr oblem i si nP,wecanuset hedet er
mi nisti
cpol ynomialt i
me
algori
thm t hatsol vesi tint hev erification-stageofanondet erminist
ical gori
thm t hatsimpl y
i
gnor esstringSgener atedini t
snondet ermi nisti
c( “guessing”)stage.
ButNP al socont ai
nst heHami lt
onianci rcuitproblem suchas:t hepar tit
ionpr oblem,
decisionv ersionsoft het ravellingsal esman,t heknapsack,gr aphcol oring,andmanyot her
diff
icultcombi natori
al opti
mi zationpr oblems.
Thehal ti
ngpr oblem, ont heot herhand, i
samongt herareexampl esofdeci si
onpr oblems
thatareknownnott obei nNP.
Thi
sl eadst othemosti mpor tantopenquest i
onoft heoreticalcomput erscience:
IsPapr opersubsetofNP, orar et heset wocl asses, infact,t
hesame?
Wecanputt hissymbol i
cal
lyas:

P?=NP.OR
NotethatP=NPwoul dimplythateachofmanyhundr edsofdiff
icul
tcombi nat
ori
aldeci
sion
probl
emscanbesol vedbyapol ynomi al-
ti
meal gori
thm,al
thoughcomput erscient
ist
shav e
fai
ledtofi
ndsuchalgor
ithmsdespit
et heirpersi
stentef
fort
sovermanyy ear
s.
Mor eover
,manywel l
-knowndeci sionproblemsar eknownt obe“ NP-complet
e”,which
seemst ocastmoredoubtsonthepossi bil
it
ythatP=NP.

NP-CompletePr obl
ems
I
nformall
y,anNP- compl et
epr obl
em i
sapr obl nNPt
em i hati
sasdi
ff
icul
tasanyotherprobl
em i
n
t
hisclassbecause, bydefi
niti
on,anyotherprobl
em inNPcanber
educedtoi
tinpolynomial
time
(
showni nFigur
e11a) .
Herearemor eformal defi
nit
ionsoftheseconcepts.

DEFI NITION5
Adeci sionproblem D1i ssaidtobepol y
nomi all
yreduci
blet oadecisi
onproblem D2,i
fther
e
existsaf unct
iontt hattransf
ormsinstancesofD1t oi ancesofD2sucht
nst hat:
1.tmapsal lyesinstancesofD1t oy esi ancesofD2andal
nst l noi ancesofD1t
nst ono
i
nst ancesofD2
2.ti scomput ablebyapol ynomialt
imeal gor
ithm.Thisdefinit
ionimmedi
atelyimpli
esthati
fa
problem D1i spolynomi all
yreduci
bletosomepr emD2t
obl hatcanbesolvedinpolynomialt
ime,
t
henpr em D1canal
obl sobesol
vedi
npol
ynomi
alt
ime(
why
?).

DEFINITION6
Adecisionpr em Di
obl ssai
dtobeNP-completei
f:
1.i
tbelongstoclassNP
2.ever
ypr obl nNPi
em i spol
ynomi
all
yreduci
bl oD
et

Thef actt hatcl oselyrelateddeci sionpr obl emsar epol ynomi allyreduciblet oeachot heri snot
verysur pr ising.
Forexampl e,l etuspr ov ethatt heHami ltonianci rcuitpr oblem ispol ynomi al
lyr educi bl eto
thedeci si onv ersi onoft hetravell
ingsal esmanpr oblem.
Thel attercanbest atedast heex i
st encepr oblem ofaHami lt
onianci rcuitnotl ongert han
agi venposi ti
vei ntegerm inagi vencompl etegr aphwi thposi tiveintegerwei ght s.
Wecanmapagr aphGofagi v
eni nstanceoft heHami l
tonianci r
cuitprobl em t oacompl ete
weight edgr aphGr epresentingani nst anceoft het ravell
ingsal esmanpr oblem byassi gni ng1as
thewei ghtt oeachedgei nGandaddi nganedgeofwei ght2bet weenanypai rofnonadj acent
verti
cesi nG.
Ast heupperboundm ont heHami ltoni anci rcuitlengt h,wet akem =n,wher eni st he
numberofv erticesi nG( andG’ ).Obv iously ,thi stransfor mat i
oncanbedonei npol y
nomi al ti
me.
LetGbeay esi nst anceoft heHami lt
oni anci rcuitpr oblem.ThenGhasaHami ltonianci r cuit,and
i
tsi magei nG’wi l
lhav elengt hn,maki ngt hei mageay esi nstanceoft hedeci siont r
av eli
ng
salesmanpr obl em.
Conv er sely,i fwehav eaHami l
toni anci rcui toft hel engt hnotl argert hanni nG’ ,theni t
s
l
engt hmustbeexact yn(
l why ?)andhencet heci rcuitmustbemadeupofedgespr esenti nG,
maki ngt hei nver sei mageoft hey esi nst anceoft hedeci siont r
av el
li
ngsal esmanpr obl em bea
yesi nst anceoft heHami l
tonianci r
cui tprobl em.Thi scompl etest heproof.

FIGURE11a:NotonofanNP-
i compl
etepr
obl
em.Pol
ynomi
al-
ti
mer
educt
ionsofNPpr
obl oanNP-
emst compl
etepr
obl
em ar
e
shownbyarrows.

Thenot onofNP-
i compl et enessr equires,howev er ,
pol y
nomi alreduci bil
i
tyofallproblemsi nNP,
bothknownandunknown, tothepr oblem inquest i
on.
Giventhebewi lder ingv ar
ietyofdeci sionpr oblems, itisnot hingshortofamazi ngthat
specifi
cexampl esofNP- compl etepr oblemshav ebeenact ual l
yf ound.
Nev er
thel
ess, thismat hemat ical f
eatwasaccompl i
shedi ndependent l
ybySt ephenCook
i
nt heUnitedStatesandLeoni dLev ininthef ormerSov ietUni on.I nhis1971paper ,Cook[ Coo71]
showedt hattheso- calledCNF- satisfiabil
it
ypr oblem isNPcompl ete.
InBooleanl ogic, af ormulai sinconj unctivenor malf or m( orcasualnormalf orm)ifi tisa
conjuncti
onofcl auses,wher eacl ausei sadi sjuncti
onofl iterals;otherwiseput,iti
sanANDof
ORs.Asanor mal form, itisusef ulinaut omat edt heorem pr ov i
ng.
TheCNF- sat i
sfiabi l
ityproblem deal swi thBool eanexpr essi ons.Eachbooleanexpr ession
canber epresentedinconj uncti
venor mal f
orm, suchast hef ollowi ngexpressi
oni nvolvi
ngt hree
booleanv ari
abl esx1, x2, andx3andt heirnegationsdenot ed. x1,x2,
. and. x3,r
espect i
vely:
(
x1∨. x2∨. x3) &(x1∨x2)
. &( x1∨.
. x2∨. x3).
TheCNF- sati
sfiabili
typr oblem askswhet herornotonecanassi gnvaluest r
ueandf alset o
vari
ablesofagi venBool eanexpr essi oni nitsCNFf ormt omaket heent i
reexpressiontr ue.(Iti
s
easytoseet hatt hiscanbedonef ortheabov efor mula:
Ifx1=t rue,x2=t rue, andx3=f alse, t
heent ireexpressionistrue.)
SincetheCook- Lev i
ndi scov eryoft hefirstknownNP- compl eteproblems, computersci entists
havefoundmanyhundr eds, ifnotthousands, ofot herexampl es.
Inparticular
,thewel l
-knownpr oblems( ort hei
rdeci si
onv ersions)ment ioned
above—Hami ltonianci rcuit
, t
ravell
ingsal esman, par ti
tion,bi
npacki ng,andgr aphcol orng—ar
i eal
l
NP-compl et
e.
Iti
sknown, howev er,thatifP≠NPt heremustex istNPpr oblemst hatneitherareinPnorar eNP-
compl ete.

Forawhi l
e,t hel eadi ngcandi dat etobesuchanexampl ewast hepr oblem ofdet ermining
whet heragi veni ntegeri spri
meorcomposi te.
Buti n an i mpor t
antt heor eti
calbr eakt hrough,Pr of
essorMani ndr a Agr awaland hi s
student s Neer ajKay aland Ni t
in Saxena oft he Indian Insti
tute ofTechnol ogyin Kanpur
announcedi n2002adi scoveryofadet erminist
icpol ynomi al
-t
imeal gori
thm f orpr i
malit
ytesting
[Agr04].
Theiralgor i
thm doesnotsol ve,howev er,t
her elat
edpr oblem offactor i
ngl argecomposi te
i
nteger s,whichl i
esatt hehear toft hewi delyusedencr y pti
onmet hodcal ledt heRSAal gor
ithm
[Riv
78] .
Showi ngthatadeci si
onpr oblem i sNP- compl etecanbedonei ntwost eps.
o Fi r
st,oneneedst oshow t hatt hepr oblem i nquest ionisinNP;i .
e.,ar andoml ygenerated
str
ingcanbecheckedi npol ynomi al t
imet odet erminewhet herornoti trepr esentsasolution
tothepr oblem.Ty picall
y,thisst episeasy .
o Thesecondst epi st oshowt hatev erypr oblem inNPi sreducibl
et othepr oblem inquestion
i
npol ynomi alti
me.Becauseoft het ransiti
vit
yofpol y nomialreducti
on, thisstepcanbedone
byshowi ngt hataknown NP- compl etepr oblem canbet r
ansformedt ot hepr obl
em i n
questioninpol ynomi alt
ime( seeFi gure11b) .

Althoughsuchat r
ansfor
mati
onmayneedt obequitei
ngenious,i
tisi
ncomparabl
ysimplerthan
provingtheexistenceofat r
ansformationforever
yproblem inNP.Forexample,ifwealready
know t hattheHami lt
onianci
rcuitprobl sNP-
em i complet
e,itspoly
nomialr
educibi
li
tytot he
decisiontravel
l
ingsalesmanproblem impli
esthatthelat
terisalsoNP-compl
ete(aft
eraneasy
checkt hatthedeci
siontr
avel
li
ngsalesmanpr obl
em isi
nclassNP) .

Fi
gur
e11b

Thedef
ini
tonofNP-
i compl
etenessi
mmedi
atel
yimpl
i
est
hati
fther
eexi
stsadet
ermi
nist
ic
pol
ynomial-
ti
meal gorit
hm forjustoneNP- compl
etepr obl
em,thenev eryprobl nNPcanbe
em i
sol
vedinpolynomialtimebyadet er
mi ni
sticalgor
it
hm, andhenceP=NP.
Inotherwor ds,fi
ndi
ngapol ynomial -t
imealgor
ithm foroneNP- completeprobl
em would
meant hatthereisnoqual itat
ivedi
ffer
encebet weent hecomplexit
yofchecki ngapr oposed
sol
uti
onandf indi
ngitinpol
y nomialt
imef orthevastmaj or
it
yofdecisi
onpr oblemsofallki
nds.

Suchi mpl i
cati
onsmakemostcomput ersci entist
sbel i
evethatP ≠NP,al thoughnobodyhas
beensuccessf ul sofarinf i
ndingamat hemat ical proofoft hi
sintri
guingconjecture.
Surprisi
ngly,inintervi
ewswi t
htheaut hor sofabookaboutt heliv
esanddi scoveriesof15
promi nentcomput ersci enti
sts[Sha98] ,Cook seemed t o be uncertain aboutt he ev entual
resol
ut ionoft hisdil
emmawher easLevincont endedt hatweshoul dexpecttheP=NPout come.
What evert heev entualanswert ot heP ? =NP quest ionprovest obe,knowi ngt hata
problem i sNP- completehasi mportantpr acti
cal impl i
cati
onsf ortoday.
I
tmeanst hatfacedwi thaprobl
em knownt obeNP- complete,weshoul dpr obablynotai m
atgai ni ng fame and f ort
une bydesi gning a pol y
nomi al
-t
ime al gor
ithm f orsol v
ing al lit
s
i
nstances.Rat her,weshoul dconcent rateonsev eralapproachest hatseekt oal leviatethe
i
ntractabi l
it
yofsuchpr oblems.

Usef
ulFor
mul
asf
ort
heAnal
ysi
sofAl
gor
it
hms

Propert
iesofLogarit
hms
Alll
ogari
thm basesareassumedtobegreat
ert
han1inthefor
mulasbel
ow;
lgxdenot
esthe
l
ogarit
hm base2, nxdenot
l esthel
ogar
ihm basee=2.
t 71828...;
x,yar
ear
bit
rar
yposi
ti
ve
numbers.