NP-complete problems

1 Ndingshortestpathsandminimumspanningtree singraphs, matchingsinbipartitegraphs, maximumincreasingsub-sequences, maximum owsinnetworks, cient, becauseineach casetheirtimerequirementgrowsasa polynomialfunction(such asn,n2, orn3) ofthesizeof betterappreciatesuch ef cientalgorithms, considerthealternative:Inalltheseprob-le mswearesearchingfora solution(path,tree, matching, etc.)fromamonganexponentialpopulationofp ossibilities. Indeed,nboyscanbematchedwithngirlsinn!di fferentways, agraphwithnverticeshasnn 2spanningtrees, anda typicalgraphhasanexponentialnum-berof pathsfromstot. Alltheseproblemscouldinprinciplebesolved inexponentialtimebycheckingthroughallcan didatesolutions, onebyone. Butanalgorithmwhoserunningtimeis2n, orworse, is allbutuselessinpractice(seethenextbox).T hequestforef cientalgorithmsis about ndingcleverwaystobypassthisprocessofexha ustivesearch,usingcluesfromtheinputinord ertodramaticallynarrowdownthesearch thisquest,algorithmictech-niquesthatdefe atthespecterofexponentiality:greedyalgor ithms, dynamicprogramming,linearprogramming(whi ledivide-and-conquertypicallyyieldsfaste ralgorithmsforproblemswecanalreadysolvei npolynomialtime).

2 Nowthetimehascometomeetthequest's mostembarrassingandpersistentfailures. We shallseesomeother search problems , inwhichagainweareseekinga solutionwithparticularpropertiesamongane xponentialchaosofal-ternatives. Butforthesenewproblemsnoshortcutseemspos sible. Thefastestalgorithmsweknowforthemarealle xponential ,thegameofchesswasinventedbytheBrahminSi ssatoamuseandteach hisking. Askedbythegratefulmonarch whathewantedinreturn,thewisemanrequested thatthekingplaceonegrainofriceinthe rstsquareofthechessboard,twointhesecond, fourinthethird,andsoon,doublingtheamount ofriceuptothe64thsquare. Thekingagreedonthespot,andasa resulthewasthe rstpersontolearnthevaluable -albeithumbling lessonofexponentialgrowth. Sissa's requestamountedto264 1 = 18;446;744;073;709;551;615grainsofrice, enoughricetopaveallofIndiaseveraltimesov er!Allovernature, fromcoloniesofbacteriatocellsina fetus, weseesystemsthatgrowexponentially fora while.

3 In1798,theBritishphilosopherT. RobertMalthuspublishedanessay inwhich hepredictedthattheexponentialgrowth(heca lledit geometricgrowth )ofthehumanpopulationwouldsoondepletelin earlygrowingresources, anargumentthatin uencedCharlesDarwindeeply. , , ,moderatedtoa doublingevery18monthsandextendedtocomput erspeed,is knownasMoore's Andthesearethetworootcausesoftheexplosio nofinformationtechnologyinthepastdecades :Moore's lawandef 's lawprovidesa disincentivefordevelopingpolynomialal-go rithms. Afterall,if analgorithmisexponential,whynotwaitit outuntilMoore's lawmakesit feasible?Butinrealitytheexactoppositehap pens:Moore's law is a hugeincen-tivefordevelopingef cientalgorithms, becausesuch algorithmsareneededinordertotakeadvantag eof If, forexample, anO(2n)algorithmforBooleansatis ability(SAT) weregivenanhourtorun,it wouldhavesolvedinstanceswith25variablesb ack in1975,31vari-ablesonthefastercomputersa vailablein1985,38variablesin1995,andabou t45variableswithtoday's machines.

4 Quitea bitofprogress exceptthateach extravariablerequiresayearanda half's wait,whiletheappetiteofapplications(many ofwhich are, ironically, re-latedtocomputerdesign)growsmuch faster. Incontrast,thesizeoftheinstancessolvedby anO(n)orO(nlogn)algorithmwouldbemultipli edbya factorof about100each (n2)algorithm,theinstancesizesolvableina xedtimewouldbemul-tipliedbyabout10each decade. EvenanO(n6)algorithm,polynomialyetunappe tizing,wouldmorethandoublethesizeofthein stancessolvedeach decade. Whenit comestothegrowthofthesizeofproblemswecan attack withanalgorithm,wehavea reversal:expo-nentialalgorithmsmakepolyn omiallyslowprogress, whilepolynomialalgorithmsadvanceexponent iallyfast!ForMoore's law tobere ectedintheworldweneedef ,exponentialexpansioncannotbesustainedin -de nitelyinour ; 's law willstopdoublingthespeedof ourcomputerswithina decadeortwo. Andthenprogresswilldependonalgorithmicin genuity orotherwiseperhapsonnovelideassuch asquantumcomputation, Dasgupta, , Vazirani249 Satis abilitySATISFIABILITY, orSAT( ),is a problemofgreatpracticalimportance, withapplicationsrangingfromchiptestingan dcomputerdesigntoimageanaly-sisandsoftwa reengineering.

5 It is alsoa 's whataninstanceofSATlookslike:(x_y_z) (x_y) (y_z) (z_x) (x_y_z):ThisisaBooleanformulainconjuncti venormalform(CNF). Itisa collectionofclauses(theparentheses),each consistingofthedisjunction(logicalor, denoted_) ofseveralliterals,wherea literaliseithera Booleanvariable(such asx) orthenegationofone(such asx).Asatisfyingtruthassignmentisanassig nmentoffalseortruetoeach variablesothateveryclausecontainsa literalwhosevalueistrue. TheSATproblemis thefollowing:givenaBooleanformulainconju nctivenormalform,either nda , settingallvariablestotrue, forexample, satis truthassignmentthatsatis esallclauses?Witha littlethought,it isnothardtoarguethatinthisparticularcase nosuch truthassignmentexists. (Hint: Thethreemiddleclausesconstrainallthreeva riablestohavethesamevalue.) Buthowdowedecidethisingeneral?Ofcourse, wecanalwayssearch throughalltruthassignments, onebyone, butforformulaswithnvariables, thenumberofpossibleassignmentsis exponential, typicalsearch problem.

6 We aregivenaninstanceI(thatis, someinputdataspecifyingtheproblemathand, inthiscasea Booleanformulainconjunctivenormalform),a ndweareaskedto ndasolutionS(anobjectthatmeetsa particularspeci cation,inthiscaseanassignmentthatsatis eseach clause).If nosuch solutionexists, wemustsay cally, a search problemmusthavethepropertythatanypropose dsolutionStoaninstanceIcanbequickly checkedforcorrectness. Whatdoesthisentail?Foronething,Smustatle astbeconcise(quick toread),withlengthpolynomiallyboundedbyt hatofI. Thisis clearlytrueinthecaseofSAT, forwhichSis anassignmenttothevariables. To formalizethenotionof quick checking, wewillsay thatthereis a polynomial-timealgorithmthattakesasinput IandSanddecideswhetherornotSis a solutionofI. ForSAT, thisis easyasit justinvolvescheckingwhethertheassignment speci edbySindeedsatis willbeusefultoshiftourvantagepointandtot hinkof thisef cientalgorithmforcheckingproposedsolutio nsasde ningthesearch :Asearch problemis speci edbyanalgorithmCthattakestwoinputs, aninstanceIanda proposedsolutionS, andrunsintimepolynomialinjIj.

7 We saySisasolutiontoIif andonlyifC(I;S) = problem,researchersoverthepast50yearshav etriedhardto ndef cientwaystosolveit,butwithoutsuccess. ,interestingly, therearetwonaturalvariantsofSATforwhich wedohavegoodalgo-rithms. If allclausescontainatmostonepositivelitera l,thentheBooleanformulais , showninbold, , anda satisfyingtruthassignment,if oneexists, , if allclauseshaveonlytwoliterals, thengraphthe-orycomesintoplay, andSATcanbesolvedinlineartimeby ndingthestronglyconnectedcomponentsof a particulargraphconstructedfromtheinstanc e( ).Infact,inChapter9,we'llseea differentpolynomialalgorithmforthissames pecialcase, which ,if wearejusta littlemorepermissiveandallowclausestocon tainthreeliterals, thentheresultingproblem,knownas3 SAT(anexampleof which wesaw earlier),onceagainbecomeshardtosolve!The travelingsalesmanproblemInthetravelingsa lesmanproblem(TSP) wearegivennvertices1;:::;nandalln(n 1)=2distancesbetweenthem,aswellasabudget b.

8 We areaskedto ndatour, a cycle thatpassesthrougheveryvertexexactlyonce, of totalcostborless ortoreportthatnosuch , weseeka permutation (1);:::; (n)oftheverticessuch thatwhentheyaretouredinthisorder, thetotaldistancecoveredis atmostb:d (1); (2)+d (2); (3)+ +d (n); (1) (onlysomeof thedistancesareshown;assumetherestarever ylarge).Noticehowwehavede nedtheTSPasasearch problem: givenaninstance, nda tourwithinthebudget(orreportthatnoneexis ts).Butwhyareweexpressingthetravelingsal esmanprobleminthisway, wheninrealityit isanoptimizationproblem, inwhich theshortestpossibletouris sought?Whydressit upassomethingelse?Fora Theframeworkofsearch problemsis helpfulinthisregard,becauseit encompassesoptimizationproblemsliketheTS Pinadditiontotruesearch search problemdoesnotchangeitsdif cultyatall,becausethetwoversionsreduceto oneanother. AnyalgorithmthatsolvestheoptimizationS. Dasgupta, , Vazirani251 TSPalsoreadilysolvesthesearch problem: ndtheoptimumtourandif it is withinbudget,returnit;if not,thereis , analgorithmforthesearch seewhy, rstsupposethatwesomehowknewthecostof theoptimumtour;thenwecould ndthistourbycallingthealgorithmforthesea rch problem, , buthowdowe ndtheoptimumcost?

9 Easy:Bybinarysearch!( )Incidentally, thereisa subtletyhere:Whydowehavetointroducea budget?Isn'tanyoptimizationproblemalsoa search probleminthesensethatwearesearchingfora solutionthathasthepropertyofbeingoptimal ?Thecatch isthatthesolutiontoa search problemshouldbeeasytorecognize, orasweputit earlier, polynomial-timecheckable. Givena po-tentialsolutiontotheTSP, it is easytocheck theproperties isa tour (justcheck thateachvertexis visitedexactlyonce)and hastotallength b. Buthowcouldonecheck theproperty isoptimal ?AswithSAT, therearenoknownpolynomial-timealgorithms fortheTSP, despitemucheffortbyresearchersovernearly a century. Ofcourse, thereis anexponentialalgorithmforsolvingit,bytry ingall(n 1)!tours, a faster, yetstillexponential, (MST) problem,forwhich wedohaveef cientalgorithms,providesa starkcontrasthere. To phraseit asa search problem,weareagaingivena distancematrixanda boundb, andareaskedto nda treeTwithtotalweightP(i;j)2 Tdij b.

10 TheTSPcanbethoughtofasa toughcousinoftheMSTproblem,inwhich thetreeis notallowedtobranch andis thereforea much (pronounced Oiler ),thefamousSwissmathemati-cian,waswalkin gthebridgesoftheEastPrussiantownofK onigsberg. Aftera while, henoticedinfrustrationthat,nomatterwhere hestartedhiswalk,nomatterhowcleverlyheco ntinued,it wasimpossibletocrosseach bridgeexactlyonce. Andfromthissillyambition,the eldof edatoncetherootsofthepark's de ciency. First,youturnthemapoftheparkintoa graphwhoseverticesarethefourlandmasses(t woislands, twobanks)andwhoseedgesarethesevenbridges :1 ActuallytheTSPdemandsa cycle, butonecande neanalternativeversionthatseeksa path,andit is nothardtoseethatthisis bankNorthern bankSmallislandBigislandThisgraphhasmult ipleedgesbetweentwovertices afeaturewehavenotbeenallowingsofarinthis book,butonethatis meaningfulforthisparticularproblem,since each bridgemustbeaccountedforseparately.