Architecture of the Windows Kernel - Computer Science, FSU

1 Architecture of theWindows KernelBerlinApril 2008 Dave Probert, Kernel ArchitectWindows Core Operating Systems DivisionMicrosoft CorporationMS/HP 2008 Microsoft Corporation 2008 Over-simplified OS historyMulticsUNIX v6/v7 BSD/SVR4 Linux/MacOSRSX-11 VMSNTCP/MMS/DOSWin9xOf all the interesting operating systemsonly unix and NT matter (and maybe Symbian)MachTenexAccentSystem38OS/360VM/ 370 SymbianMCPNT vs unix Design EnvironmentsEnvironment which influencedfundamental design decisionsWindows (NT)UNIX32-bit program address spaceMbytes of physical memoryVirtual memoryMbytes of disk, removable disksMultiprocessor (4-way)Micro-controller based I/O devicesClient/Server distributed computingLarge, diverse user populations16-bit program address spaceKbytes of physical memorySwapping system with memory mappingKbytes of disk, fixed disksUniprocessorState-machine based I/O devicesStandalone interactive systemsSmall number of friendly usersEffect on OS DesignNT vs UNIXA lthough both Windows and Linux have adapted to changes in theenvironment, the original design environments ( in 1989 and 1973) heavilyinfluenced the design choices:Unit of concurrency:Process creation:I/O:Namespace root:Security.

2 Threads vs processesCreateProcess() vs fork()Async vs syncVirtual vs FilesystemACLs vs uid/gidAddr space, uniprocAddr space, swappingSwapping, I/O devicesRemovable storageUser populationsToday s Environment64-bit addressesGbytes of physical memoryVirtual memory, virtual processorsMultiprocessors (64-128x)High-speed internet/intranet, Web ServicesSingle user, but vulnerable to hackers worldwideTV/PC ConvergenceCellphone/Walkman/PDA/PC Convergence Compare & Contrast drives innovation Studying foo is fine But if you also study bar , students will compare &contrast Result is innovation: Students mix & match concepts to create newideas Realizing there is not a single right solution,students invent even more approaches Learning to think critically is an important skill forstudentsTeaching unix AND Windows Microsoft Corporation 20087NT the accidental secretHistorically little information on NT available Microsoft focus was end-users and Win9x Source code for universities was too encumberedMuch better internals information today Windows Internals, 4th Ed.

3 , Russinovich & Solomon Windows Academic Program (universities only): CRK: Curriculum Resource Kit (NT Kernel in PowerPoint) WRK: Windows Research Kernel (NT Kernel in source) Design Workbook: soft copies of the original specs/notes Chapters in leading OS textbooks (Tanenbaum,Silberschatz, Stallings) Microsoft Corporation 20088NT Kernel philosophy Reliability, Security, Portability, Compatibility areall paramount Performance important Multi-threaded, asynchronous General facilities that can be re-used Support Kernel -mode extensibility (for better or worse) Provide unified mechanisms that can be shared Kernel /executive split provides a clean layering model Choose designs with architectural headroom Microsoft Corporation 20089 Important NT Kernel features Highly multi-threaded in a process-like environment Completely asynchronous I/O model Thread-based scheduling Unified management of Kernel data structures, kernelreferences, user references (handles), namespace,synchronization objects, resource charging, cross-process sharing Centralized ACL-based security reference monitor Configuration store decoupled from file system Microsoft Corporation 200810 Important NT Kernel features (cont)

4 Extensible filter-based I/O model with driver layering,standard device models, notifications, tracing, journaling,namespace, services/subsystems Virtual address space managed separately from memoryobjects Advanced VM features for databases (app managementof virtual addresses, physical memory, I/O, dirty bits, andlarge pages) Plug-and-play, power-management System library mapped in every process provides trustedentrypoints Microsoft Corporation 200811 Windows ArchitectureUser-modeKernel-modeNTOS Kernel layerSystem library (ntdll) / run-time libraryKernel32win32 DLLsApplicationsSystem ServicesSubsystemserversLogon/GINAC ritical servicesNTOS executive layerDriversHALF irmware, Hardware Microsoft Corporation 200812 Windows user-mode Subsystems OS Personality processes Dynamic Link Libraries Why NT mistaken for a microkernel System services (smss, lsass, services) System Library ( ) Explorer/GUI (winlogon, explorer) Random executables (robocopy, cmd) Microsoft Corporation 200813 Windows Kernel -mode NTOS (aka the Kernel ) Kernel layer (abstracts the CPU) Executive layer (OS Kernel functions) Drivers ( Kernel -mode extension model) Interface to devices Implement file system, storage, networking New Kernel services HAL (Hardware Abstraction Layer)

5 Hides Chipset/BIOS details Allows NTOS and drivers to run unchanged Microsoft Corporation 200814NT API stubs (wrap sysenter) -- system library ( )usermodekernelmodeKernel-mode Architecture of WindowsNTOS executive layerTrap/Exception/Interrupt DispatchCPU mgmt: scheduling, synchr, ISRs/DPCs/APCsDriversDevices, Filters,Volumes,Networking,GraphicsHardw are Abstraction Layer (HAL): BIOS/chipset detailsfirmware/hardwareCPU, MMU, APIC, BIOS/ACPI, memory, devicesNTOS kernellayerCaching MgrSecurityProcs/ThreadsVirtual MemoryIPCglueI/OObject MgrRegistry Microsoft Corporation 200815 Kernel /Executive layers Kernel layer aka ke (~ 5% of NTOS source) Abstracts the CPU Threads, Asynchronous Procedure Calls (APCs) Interrupt Service Routines (ISRs) Deferred Procedure Calls (DPCs aka Software Interrupts) Providers low-level synchronization Executive layer OS Services running in a multithreaded environment Full virtual memory, heap, handles Note: VMS had four layers: Kernel / Executive / Supervisor / User Microsoft Corporation 200816NT (Native) API examplesNtCreateProcess (&ProcHandle, Access, SectionHandle,DebugPort, ExceptionPort.)

6 NtCreateThread (&ThreadHandle, ProcHandle, Access,ThreadContext, bCreateSuspended, ..)NtAllocateVirtualMemory (ProcHandle, Addr, Size, Type,Protection, ..)NtMapViewOfSection (SectHandle, ProcHandle, Addr,Size, Protection, ..)NtReadVirtualMemory (ProcHandle, Addr, Size, ..)NtDuplicateObject (srcProcHandle, srcObjHandle,dstProcHandle, dstHandle, Access, Attributes, Options) Kernel AbstractionsKernels implement abstractions Processes, threads, semaphores, files, ..Abstractions implemented as data and code Need a way of referencing instancesUNIX uses a variety of mechanisms File descriptors, Process IDs, SystemV IPC numbersNT uses handles extensively Provides a unified way of referencing instances ofkernel abstractions Objects can also be named (independently of the filesystem)17NT Object Manager Generalizes access to Kernel abstractions Provides unified management of:!

7 Kernel data structures! Kernel references! user references (handles)! namespace! synchronization objects! resource charging! cross-process sharing! central ACL-based security reference monitor! configuration (registry)18\ObjectTypesObject Manager: Directory, SymbolicLink, TypeProcesses/Threads: DebugObject, Job, Process, Profile,Section, Session, Thread, TokenSynchronization:Event, EventPair, KeyedEvent, Mutant, Semaphore,ALPC Port, IoCompletion, Timer, TpWorkerFactoryIO: Adapter, Controller, Device, Driver, File, Filter*PortKernel Transactions: TmEn, TmRm, TmTm, TmTxWin32 GUI: Callback, Desktop, WindowStationSystem: EtwRegistration, WmiGuid Microsoft Corporation 200819 Microsoft Corporation 200820\Global??\C:\Device\HarddiskVolume 1<directory>L Global?

8 ? <directory>L C: L \ <symbolic link>\Device\HarddiskVolume1<directory>L Device <directory>L HarddiskVolume1 L \ <device>by I/OmanagerimplementedNaming example Microsoft Corporation 200821\Global??\C:\foo\ <device object>by I/Omanagerimplemented, foo\ deviceobject->ParseRoutine == IopParseDeviceObject Manager Parsing exampleNote: namespace rooted in object manager, not FS Microsoft Corporation 200822I/O Support: IopParseDeviceuserkernelTrap mechanismDev StackNtCreateFile()ObjMgr LookupcontextIopParseDevice()DevObj,cont extSecurityRefMonAccesscheckFile objectFile SysFile System Fills in File objectAccesscheckReturns handle to File object Microsoft Corporation 200823 Why not root namespace in filesys?A few Hard to add new object types Device configuration requires filesys modification Root partition needed for each remote client End up trying to make a tiny root for each client Have to check filesystem very earlyWindows uses object manager + registry hives Fabricates top-level namespace in Kernel Uses config information from registry hive Only needs to modify hive after system stable Microsoft Corporation 200824 Object referencingObjectManagerNTOSK ernelData ObjectNamelookupAccess checksSecurity Ref MonitorReturns ref d ptrRef d ptr used until derefAppNameHandle Microsoft Corporation 200825 Handle Table NT handles allow user code to referencekernel data structures (similar, but moregeneral than unix file descriptors)

9 NT APIs use explicit handles to refer toobjects (simplifying cross-process operations) Handles can be used for synchronization,including WaitMultiple Implementation is highly scalable Microsoft Corporation 200826 Process Handle TablespHandleTableEPROCESSpHandleTableEP ROCESSS ystemProcessHandle TableHandle TableKernel Handlesobjectobjectobjectobjectobject Microsoft Corporation 200827 One level: (to 512 handles)TableCodeA: Handle Table Entries [512 ]Handle TableObjectObjectObject Microsoft Corporation 200828 Two levels: (to 512K handles)TableCodeA: Handle Table Entries [512 ]Handle TableObjectObjectObjectB: Handle Table Pointers [1024 ]C: Handle Table Entries [512 ] Microsoft Corporation 200829 Three levels: (to 16M handles)TableCodeA: Handle Table Entries [512 ]Handle TableObjectObjectObjectB: Handle Table Pointers [1024 ]C: Handle Table Entries [512 ]D: Handle Table Pointers [32 ]E: Handle Table Pointers [1024 ]F: Handle Table Entries [512 ] Microsoft Corporation 200830 Process/Thread structureObjectManagerAny HandleTableProcessObjectProcess Handle TableVirtualAddressDescriptorsThreadThre adThreadThreadThreadThreadFilesEventsDev icesDriversuser-mode executionread(handle)MemoryManagerStruct ures Microsoft Corporation 200831 OBJECT_HEADERP ointerCountHandleCountpObjectTypeoNameIn fopQuotaBlockChargedpSecurityDescriptorC reateInfo + NameInfo + HandleInfo + QuotaInfoOBJECT BODY [optional DISPATCHER_HEADER] oHandleInfooQuotaInfoFlagsEvent Type.

10 Notification or SynchronizationWaiter ListSignaled Microsoft Corporation 200832 WaitListHeadWaitListEntryWaitBlockListKP RCBT hreadThreadWaitListEntryWaitBlockListWai tListEntryNextWaitBlockWaitBlockWaitList EntryNextWaitBlockWaitBlockWaitListEntry NextWaitBlockWaitBlockWaitListEntryNextW aitBlockWaitBlockWaitListEntryNextWaitBl ockWaitBlockWaitListEntryNextWaitBlockWa itBlockWaitListHeadObject->HeaderSignale dWaitListHeadObject->HeaderSignaledWaitL istHeadObject->HeaderSignaledWaitListHea dObject->HeaderSignaledStructure used byWaitMultiple Microsoft Corporation 200833 Summary: Object Manager Foundation of NT namespace Unifies access to Kernel data structures Outside the filesystem (initialized form registry) Unified access control via Security Ref Monitor Unified Kernel -mode referencing (ref pointers) Unified user-mode referencing (via handles) Unified synchronization mechanism (events) Microsoft Corporation 200834 Processes An environment for program execution Binds namespaces virtual address mappings ports (debug, exceptions) threads user authentic