Xen and the Art of Virtualization - University of Cambridge

1 XenandtheArt ofVirtualizationPaulBarham , Boris Dragovic, KeirFraser, StevenHand,TimHarris,Alex Ho, RolfNeugebauery, IanPratt,Andrew War eldUniversityof CambridgeComputerLaboratory15 JJThomsonAvenue, Cambridge , UK,CB30 FDf beendesignedwhichusevirtualizationtosubd ividetheampleresourcesofa moderncomputer. Somerequirespecializedhardware, offerresourceisolationorperformanceguara ntees;mostpro-videonlybest-effortprovisi oning, ,anx86virtualmachinemonitorwhichallowsmu ltiplecommodityoperatingsystemsto shareconventionalhardwareina safeandresourcemanagedfashion,butwithout sac-ri cingeitherperformanceorfunctionality. Thisisachievedbyprovidinganidealizedvirt ualmachineabstractiontowhichoper-atingsy stemssuchasLinux,BSDandWindowsXP, targetedathostingupto100virtualmachinein -stancessimultaneouslyona modernserver.

2 Thevirtualizationap-proachtakenbyXenis extremelyef cient:weallow operatingsys-temssuchasLinuxandWindowsXP tobehostedsimultaneouslyfora negligibleperformanceoverhead atmosta considerablyoutperformcompetingcommercia landfreelyavailablesolutionsina [OperatingSystems]: ProcessManagement; [Opera-tingSystems]: StorageManagement; [OperatingSystems]:PerformanceGeneralTer msDesign,Measurement,PerformanceKeywords VirtualMachineMonitors,Hypervisors,Parav irtualization MicrosoftResearchCambridge,UKyIntelResea rchCambridge,UKPermissiontomake digitalorhardcopiesofallorpartofthiswork forpersonalorclassroomuseis grantedwithoutfeeprovidedthatcopiesareno tmadeordistributedforpro torcommercialadvantageandthatcopiesbeart hisnoticeandthefullcitationonthe copy otherwise,torepublish,topostonserversort oredistributetolists,requirespriorspeci cpermissionand/ora '03,October19 22,2003,BoltonLanding,New York, 58113 757 5/03 $ cientlypowerfultousevirtualizationtopres enttheillusionofmany smallervirtualmachines(VMs)

3 ,eachrunninga resurgenceofinterestinVMtechnology. InthispaperwepresentXen,a highperformanceresource-managedvirtualma chinemon-itor(VMM)whichenablesapplicatio nssuchasserverconsolida-tion[42,8],co-lo catedhostingfacilities[14],distributedwe bser-vices[43],securecomputingplatforms[ 12,16]andapplicationmobility[26,37].Succ essfulpartitioningofa , virtualmachinesmustbeisolatedfromoneanot her:it is notacceptablefortheexecutionofonetoadver selyaffecttheperfor-manceofanother. Thisis , it is necessarytosupporta , ,albeitwithsomesourcemodi ;eachinstanceexportsanapplicationbinaryi nter-faceidenticaltoa executewhatever they project[15,35]wearedeployingXenonstandar dserverhardwareat virtualmachinesandexpecteachVMtopayin somefashionfortheresourcesit discussourideasandapproachinthisdirectio nelsewhere[21]; numberofwaystobuilda systemtohostmultipleapplicationsandserve rsona oneormorehostsrunninga standardoperatingsys-temsuchasLinuxorWin dows,andthentoallowuserstoinstall lesandstartprocesses time-consumingtaskduetocomplex con , suchsystemsdonotadequatelysupportper-for manceisolation.

4 Theschedulingpriority, memorydemand,net-worktraf adequateprovisioninganda closedusergroup(suchasinthecaseofcom-put ationalgrids,ortheexperimentalPlanetLabp latform[33]),butnotwhenresourcesareovers ubscribed, greaterorlesserdegreewithresourcecontain ers[3],Linux/RK[32],QLinux[40]andSILK[4] .Onedif cultywithsuchapproachesis ensuringthatallresourceusageis accountedtothecorrectprocess consider, forexample,thecomplex effectivelytheproblemof QoScrosstalk [41] a low levelcanmitigatethisproblem,asdemonstrat edbytheExokernel[23]andNemesis[27] usethissamebasicapproachto buildXen,whichmultiplexesphysicalresourc esatthegranularityofanentireoperatingsys temandis multiplexingthisalsoallowsa rangeofguestoperatingsystemstogracefully coexistratherthanmandatingaspeci a pricetopayforthis exibility runninga fullOSis moreheavyweightthanrunninga process,bothintermsofinitialization( ), ,webelieve thispriceis worthpaying.

5 It allowsindividualuserstorununmodi edbinaries,orcollectionsofbinaries,ina resourcecontrolledfashion(forinstanceanA pacheserveralongwitha PostgreSQLbackend).Furthermoreit providesanextremelyhighlevel of gurationinteractionsbe-tweenvariousservi cesandapplicationsareavoided(forexample, eachWindowsinstancemaintainsitsownregist ry).Theremainderofthispaperis structuredasfollows:in describeskey usesindustrystandardbenchmarksto eval-uatetheperformanceofXenoLinuxrunnin gabove Xenincompar-isonwithstand-aloneLinux,VMw areWorkstationandUser-modeLinux(UML).Sec tion5 reviewsrelatedwork,and :APPROACH&OVERVIEWIna traditionalVMMthevirtualhardwareexposedi s function-allyidenticaltotheunderlyingmac hine[38].

6 Althoughfullvirtu-alizationhastheobvious bene tofallowingunmodi edoperatingsystemstobehosted,it alsohasa ,orx86, ,butexecutingthesewithin-suf cientprivilegefailssilentlyratherthancau singa convenienttrap[36].Ef cientlyvirtualizingthex86 MMUisalsodif , 's ESXS erver[10] appliedtotheentireguestOSkernel(withasso ciatedtrans-lation,execution,andcachingc osts) versionsofsystemstructuressuchaspagetabl esandmaintainsconsistency withthevirtualtablesbytrappingeveryup-da teattempt thisapproachhasa highcostforupdate-intensiveoperationssuc hascreatinga new , , therearesituationsinwhichit is desirableforthehostedoperatingsystemstos eerealaswellasvirtualresources:providing bothrealandvirtualtimeallowsa guestOStobettersupporttime-sensitive tasks,andtocor-rectlyhandleTCPtimeoutsan dRTTestimates,whileexposingrealmachinead dressesallowsa guestOStoimprove performancebyusingsuperpages[30]orpageco loring[24].

7 We avoidthedrawbacksoffullvirtualizationbyp resentinga vir-tualmachineabstractionthatis similarbutnotidenticaltotheun-derlyingha rdware anapproachwhichhasbeendubbedparavir-tual ization[43].Thispromisesimprovedperforma nce,althoughit doesrequiremodi isimportanttonote,however, thatwedonotrequirechangestotheapplicatio nbinaryinterface(ABI),andhencenomodi distillthediscussionsofarintoa edapplicationbinariesisessential, ,asthisallowscomplex servercon gurationstobevirtualizedwithina machinearchitectures, [44].Denaliis de-signedtosupportthousandsofvirtualmach inesrunningnetworkservices,thevastmajori tyofwhicharesmall-scaleandunpopu-lar. Incontrast,Xenis ,it is instructive to contrastDenali' , DenalidoesnottargetexistingABIs, ,Denalidoesnotfullysupportx86segmentatio nalthoughit isexported(andwidelyused1) intheABIsofNetBSD,Linux, , theDenaliimplementationdoesnotaddressthe prob-lemofsupportingapplicationmultiplex ing,normultipleaddressspaces,withina , applicationsarelinkedexplicitlyagainstan instanceoftheIlwacoguestOSina mannerratherreminiscentofa libOSin theExokernel[23].

8 Henceeachvir-tualmachineessentiallyhosts a single-usersingle-applicationun-protecte d operatingsystem .InXen,bycontrast,a singlevirtualmachinehostsa realoperatingsystemwhichmayitselfsecurel ymultiplex thousandsofunmodi eduser-level prototypevirtualMMUhasbeendevelopedwhich mayhelpDe-naliinthisarea[44],weareunawar eofany , perhapsrelatedtothelackofmemory-manageme ntsupportatthevirtualizationlayer. Pagingwithinthe1 Forexample, ,butupdatesarebatchedandvalidatedbythehy pervisor. A a lowerprivilegelevel , `fast'handlerforsystemcalls, timerinterfaceandis awareofboth`real'and`virtual' ,Disk, contrarytoourgoalofperformanceisolation: maliciousvirtualmachinescanencouragethra shingbehaviour, (anideapreviouslyexploitedbyself-paging[ 20]).

9 Finally, Denalivirtualizesthe`namespaces'ofallmac hinere-sources,takingtheview thatnoVMcanaccesstheresourcealloca-tions ofanotherVMif it cannotnamethem(forexample,VMshavenoknowl edgeofhardwareaddresses,onlythevirtualad dressescreatedforthembyDenali).Incontras t,webelieve thatsecureac-cesscontrolwithinthehypervi soris suf cienttoensureprotection;furthermore,asdi scussedpreviously, a guestOSmustbemodi edto conformto thispaperwereserve thetermguestoperatingsystemtorefertooneo ftheOSesthatXencanhostandweusethetermdom ainto referto a runningvirtualmachinewithinwhicha guestOSexecutes;thedistinctionis analogoustothatbe-tweenaprogramandaproce ssina callXenitselfthehypervisorsinceit operatesat a higherprivilegelevelthanthesupervisorcod eoftheguestoperatingsystemsthatit presentsanoverview oftheparavirtualizedx86interface,factore dintothreebroadaspectsofthesystem.

10 Memorymanage-ment,theCPU, ,anddiscusshoweachis ,suchasmemorymanagement,arespeci ctothex86,many aspects(suchasourvirtualCPUandI/Odevices )canbereadilyappliedto ,x86representsaworst casein theareaswhereit differssigni cantlyfromRISC-styleprocessors forexample,ef cientlyvirtualizinghardwarepagetablesis moredif cultthanvirtualizinga cultpartofparavirtualizinganarchitecture ,bothintermsofthemechanismsrequiredinthe hypervisorandmodi easierif thearchitectureprovidesa software-managedTLBasthesecanbeef cientlyvirtualizedina simplemanner[13].AtaggedTLBisanotherusef ulfeaturesupportedbymostserver-classRISC architectures,includingAlpha, ertagwitheachTLBentryallowsthehypervisor andeachguestOStoef cientlycoexistinseparateaddressspacesbec ausethereis noneedto , x86doesnothave a software-managedTLB; thebestpossibleperformance, , becausetheTLBisnottagged,addressspaceswi tchestypicallyrequirea completeTLB ,wemadetwo decisions:(i)guestOSesareresponsiblefora llocatingandmanagingthehardwarepagetable s,withminimalinvolvementfromXentoensures afetyandisolation.