Chapter 1 Introduction to System Programming

1 UNIXL ectureNotesChapter1 IntroductiontoSystemProgrammingStewartWe issChapter1 Intro ductiontoSystemProgramming unix isbasicallyasimpleop eratingsystem,butyouhavetob eageniustounderstandthesimplicity. -DennisRitchie, ,Systemcal lsandlibraries,Processes,loginsandshel ls,Environments,manpages,Users,theroot,a ndgroups,Authentication,Filesystem, lehierarchy,Filesanddirectories,Devicesp ecial les, unix standards,POSIX,Systemprogramming,Termin alsandANSI escapesequences,HistoryofUNIX,syscal l,getpid, ductionAmo dernsoftwareapplicationtypicallyneedstom anageb , les,screendisplays, ewrittenasacollectionofco op eratingthreadsorsub-pro cessesthatco ordinatetheiractionswithresp dernop eratingsystemspreventapplicationsoftware frommanagingsystemresourcesdirectly, ,whenrunningonmo dernop eratingsystems,applicationscannotdrawtot hescreendirectlyorreadorwrite erformscreenop erationsor leI/Otheymustusetheinterfacethattheop eratingsystemde ()orfprintf()access lesdirectly,theydonot.

2 Theymakecallstosystemroutinesthatdothewo rkontheirb eratingsystemforapplicationstousewhenacc essingsystemre-sourcesiscalledtheop eratingsystem'sapplicationprogrammingint erface(API).AnAPItypicallyconsistsofacol lectionoffunction,typ e,andconstantde nitions,andsometimesvariablede eratingsystemine ectde nesthemeansbywhichanapplicationcanutiliz etheservicesprovidedbythatop ' ,asidefromdesigningtheapplicationitself, themostimp ortanttaskfortheapplicationdevelop eristomasterthesystemlevelservicesde nedintheop eratingsystem' ,andthetyp eratingsystemandmayevenaccessthesystem1 Weusethetermplatformtomeanasp eci cop eratingsystemrunningonasp eci :SimpleI/Omo delusedbyb ewrittentoextendthefunctionalityoftheop eci callyconcernsystemprogrammingusingtheAPI oftheUNIXop eriencewithUNIX asauser,butthismaterialcanb eskipp edbytheexp ,adistinctionwillb emadeb etweentheuser'sviewofUNIX andtheprogrammer' 'sviewofUNIX islimitedtoasubsetofcommandsthatcanb eenteredatthecommand-lineandpartsofthe lesarenotavailabletoallusers,aswillb 'sviewincludestheprogramminglanguagefeat uresofthekernelAPI,thefunctions,typ es,andconstantsinallofthelibraries,theva riousheader les,andthevarious eginningprogrammertypicallywritesprogram sthatfollowthesimpleI/Omo.

3 Theprogramgetsitsinputfromthekeyb oardoradisk le,andwritesitsoutputtothedisplayscreeno rtoa ecausethekeyb ,oneusingtheCStandardI/OLibrary,andtheot her,theC++ oardandsendoutputtothedisplaydevice, ,everypro cessiscreatedandloadedintomemory, unix automaticallycreatesthestandardinputands tandardoutputdevicesforit,op ensthem,andmakesthemreadyforreadingandwr itingresp (andC++),stdinandstdoutarevariablesde nedinthe< >2 Infact,everyPOSIX-compliantop eratingsystemmustprovideb le,thatrefertothestandardinputandstandar doutputdevice4resp ,thekeyb oardanddisplayoftheasso ciatedterminalarethestandardinputandoutp utdevicesresp :CprogramusingsimpleI/Omo del.# i n c l u d e< s t d i o. h>/*c o p yf r o ms t d i nt os t d o u t*/i n tm a i n( ){i n tc;w h i l e((c=g e t c h a r( ))! =EOF)p u t c h a r(c) ;r e t u r n0 ;} :SimpleC++programusingsimpleI/Omo del.

4 # i n c l u d e< i o s t r e a m >u s i n gn a m e s p a c es t d;/*c o p yf r o ms t d i nt os t d o u tu s i n gC++*/i n tm a i n( ){c h a rc;w h i l e((c=c i e t( ))&&!c i o f( ))c o u u t (c) ;r e t u r n0 ;}Theseprogramsgiveustheillusionthatthey aredirectlyconnectedtothekeyb oardandthedisplaydeviceviaClibraryfuncti onsgetchar()andputchar()andtheC++iostrea mmemb erfunctionsget()andput().Eitherofthemcan b erunonasingle-userdesktopcomputeroronamu lti-user,time-sharedworkstationinatermin alwindow,andtheresultswillb ,theywillhavethesameb ersonalcomputerrunninginsingle-usermo de,thisillusionisnotfarfromrealityinthes ensethatthekeyb oardisindirectlyconnectedtotheinputstrea moftheprogram, eratingsystem,severalusersmayb eloggedinsimultaneously,andprogramsb elongingtodi erentusersmightb erunningatthesametime,eachreceivinginput fromadi erentkeyb oardandsendingoutputtoadi ,onaUNIX computeronanetworkintowhichyoucanlogin,m anyp eoplemayb econnectedtoasinglecomputerviaanetworkpr ogramsuchasSSH,andseveralofthemwillb eabletoruntheab oveprogramonthesamecomputeratthesametime ,sendingtheiroutputtodi erentterminalwindowsonphysicallydi erentcomputers, , unix ensures,inaremarkablyelegantmanner,thate achuser'sprocesseshavealogicalconnection totheirkeyb oardandtheirdisplay.

5 (Thepro cessconceptwillb eexplained4 InCandC++,stderristhevariableasso )Programsthatusethemo delofI/Odescrib edab ovedonothavetob econcernedwiththecomplexitiesofconnectin gtomonitorsandkeyb oards,b ecausetheop eratingsystemhidesthatcomplexity,present ingasimpli eratingsystemachievesthis,onemust rstunderstandseveralcornerstoneconceptso ftheUNIXop eratingsystem: les,pro cesses,usersandgroups,privilegesandprote ctions, eginning, unix wasdesignedaroundasmallsetofcleverideas, asRitchieandThompson[2]putit: ThesuccessofUNIX liesnotsomuchinnewinventionsbutratherint hefullexploita-tionofacarefullyselecteds etoffertileideas,andesp eciallyinshowingthattheycanb ekeystotheimplementationofasmallyetp owerfulop eratingsystem. Those fertileideas includedthedesignofits lesystem,itspro cessconcept,theconceptofprivilegedandunp rivilegedprograms,theconceptsofuserandgr oups,aprogrammableshell,environments,and deviceindep eeachofthesebrie eoplewhohaveusedcomputersknowwhata leis,butasanexercise,tryexplainingwhata ,butknowinghowtode ,thetraditionalde nitionofa lewasthat itisthesmal lestunitofexternalstorage.

6 "Externalstorage"hasalwaysmeantnon-volat ilestorage,notinprimarymemory,butonmedia suchasmagnetic,optical,andelectronicdisk s,tap esandsoon.(Internalstorageisonmemorychip s.)Thecontemp oraryde nitionofa leinUNIX isthatitisanob jectthatcanb ewrittento,orreadfrom,orb nitionofa lesintoatree-likehierarchythatmostp eopleerroneouslycallthe lehierarchy,b ecausea lesystemissomethingslightlydi ecialtyp esof lesthat,fromtheuserp ersp ective,app eartocontainother les,althoughtheydonotcontain lesanymorethanatableofcontentsinab o okcontainsthechaptersoftheb o eprecise,adirectoryisa jectthatasso ciatesa lenametoa otoftheUNIX lesystemisadirectoryknownintheUNIX worldastherootdirectory,howeveritisnotna med"root"inthe lesystem;itisnamed"/".Whenyouneedtorefer tothisdirectory,youcallit"ro ot",not"slash".Morewillb esaidab out les, lenames,andthe cessesAprogramisanexecutable le, ,itisgivenvariousresourcessuchasaprimary memoryspace,b othphysicalandlogical,secondarystoragesp ace,mappingsofvariouskinds6,andprivilege s,suchastherighttoreadorwritecertain ,atanyinstantoftime,asso ciatedtoapro cessisthecollectionofallresourcesallo catedtotherunningprogram,aswellasanyothe rprop ertiesandsettingsthatcharacterizethatpro cess,suchasthevaluesofthepro cessor' ,althoughtheideaofapro cesssoundslikeanabstractidea,itis,infact , cessauniquenumb ,atagiveninstantoftime,severalp eoplemightallb erunningtheGnuCcompiler, eusedtodisplaywhichpro cessesarerunning, ,thefunctiongetpid()returnsthepro cess-idofthepro esnothingotherthanprintingitsownpro cess-id, ()isanexampleofasystemcal :Aprogramusinggetpid().

7 # i n c l u d e< s t d i o. h># i n c l u d e< u n i s t d. h>i n tm a i n( )5 Inpracticeadirectoryentryisanob jectwithtwocomp onents:thenameofa leandap ointertoastructurethatcontainstheattribu tesofthat ,amapofhowitslogicaladdressesmaptophysic aladdresses, {p r i n t f( "Iamt h ep r o c e s sw i t hp r o c e s s i d%d \ n ",g e t p i d());r e t u r n0 ;} ,likepassingthroughametaldetectoratanair p ,auserisap ciatedwitheachuser-nameisauniquenon-nega tivenumb ercalledtheuser-id, , 's ,eachgrouphasuniqueintegergroup-id, ,a lecanb easso ciatedwithagroup,andallusersinthatgroupw ouldhavethesameaccessrightstothat le,thesameistrueofprograms;anexecutablep rogramcanb easso ciatedwithagroupsothatallmemb elongstoatleastonegroup, eusedtoprinttheuser'suser-idanduser-name ,andthegroup-idandgroup-nameofallgroupst owhichtheuserb ,thereisadistinguishedusercalledthesuper user,whoseuser-nameisro ot,oneofafewprede eruserhastheabilitytodothingsthatordinar yuserscannotdo,suchaschangingap erson'suser-nameormo difyingtheop eratingsystem'scon owertoap ,allUNIX systemsrecordeveryattempttologinasroot, cesshasanasso ciated(real)user-idand,aswewillseelater, ane ectiveuser-idthatmightb edi ,whenauserstartsupaprogram,theresultingp ro cesshasthatuser'suidasb othitsrealande eruser(ro ot)

8 Runsapro cess,thatpro cessruns7 Ausermaynotb eanactualp eanabstractionofap ,mail,lp,andftpareeachusersinaUNIX System , "login"toasystemisto"loginto" erthatloggingmeansrecordingsomethinginal ogb o ok,asaseacaptaindo "login"conveystheideathattheactisb eingrecordedinalogb o ,loginsarerecordedinasp ecial lethatactslikealogb o "login"asasinglewordonlyb ecauseithasb ecomeasinglewordonmillionsof"loginscreen s" ,asaverb,reallymeans"tologinto"something ;itrequiresanindirectob eprecise,theprivilegesarethoseofuserwith thepro cess'se eruser' cessesrunningwithuserprivilegesarecalled userpro ,thefunctiongetuid()returnstherealuser-i dofthepro cessthatcallsit,andthegetgid()functionre turnstherealgroup-idofthepro cessesfromaccessinghardwareandp erformingotherop erationsthatmaycorruptthestateofthecompu tersystem, unix requiresthatthepro cessorsupp orttwomo desofop eration,knownasprivilegedandunprivileged mo , : acquiringmorememory; changingthesystemtime; raisingthepriorityoftherunningpro cess; readingfromorwritingtothedisk.

9 Enteringprivilegedmo eratingsystemdep endup onthisseparationofp ,oneofthestepsthattheop ercaseletters,digits,andunderscores, esnotcontainthe = ,LOGNAME isanenvironmentvariablethatstorestheuser -nameofthecurrentuser,andCOLUMNS isavariablethatstoresthenumb er, ,theop eratingsystemcreatestheenvironmentforthe user,basedonvarious ointforward,wheneveranewprogramruns, ercaseletters,digits,andtheunderscore('_ ')anddonotb nesavalueforCOLUMNS inastart-upscript, esnotde neit,orsetsittotheNULL string,thesizeofterminalwindowsisdetermi nedbytheop ()functioncanb eusedtoretrieveaparticularenvironmentstr ing,asinchar* username = getenv("LOGNAME");printf("The user's user-name is %s\n, username);Theop eratingsystemalsomakesavailabletoeveryru nningprogramanexternalglobalvariableexte rn char **environ;whichisap ointertothestartofthearrayofthename-valu epairsintherunningprogram' difythesevariablesiftheycho ,aprogramthatneedstoknowhowmanycolumnsar einthecurrentterminalwindowwillquerytheC OLUMNS variable, cesses,nottousers;usersinteractwithUNIX throughacommand-lineinterfacecalledashel "shell"istheUNIX termforaparticulartyp eeninop eratingsystemssincetheywere ,asistheCommandwindowofMicrosoftWindows, :youtyp eacommandandpresstheEnterkey,andthecom-m andisexecuted,afterwhichthepromptreapp ,readstheinputyoutyp e,runstheappropriateprogramsinresp onseandre-displaysthepromptisthecommand- line-interpreter.

10 Itallowstheusertode nevariables,evaluateexpressions,usecondi tionalcontrol-of- owstatementssuchaswhile-andif-statements , esavedintoa leandexecutedasaprogrambytypingthenameof the ,theop eratingsystemstartsupashellpro cumentation:TheManPagesShortlyafterRitch ieandThompsonwrotethe rstversionofUNIX,attheinsistenceoftheirm anager,DougMcIlroy,in1971,theywrotetheUN IXP rogrammer' ,butinshortcourseitwasextendedintoasetof sevenvolumes, othprintedformandasformatted ,called manpages ,andChapter2providesmoredetailab eforeMicrosoftwroteMS-DOS,theywroteavers ionofUNIX forthePCcalledXenix,whoserightstheysoldt oSantaCruzOp eratingsystemsuchasUNIX mustmanageandprotectallofthesystem'sreso urcesandprovideanop eratingenvironmentthatallowsalluserstowo rke ciently,safely, ,ifauser'spro cesswantstoreadfromorwritetoadisk,itmust asktheop eratingsystemtodothisonitsb ehalf, eratingsystemwillp erformthetaskandtransferanydatatoorfromt heuser'spro ,considerwhatwouldhapp enifauser'spro ,orevenworse, ,maylo oklikeitdo esnotasktheop eratingsystemtoreadorwriteanydata, ()andputchar()