Example: tourism industry

PCI Express I/O Virtualization Explained

PCI Express I/O Virtualization Explained Richard Solomon LSI Corporation 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. Agenda PCI Express I/O Virtualization (IOV) Overview Single Root (SR-IOV). Multi-Root (MR-IOV). Secret Three Letter Acronym (TLA) Decoder Ring Programming IOV Devices Configuration Space Mapping Memory Space Mapping Programming Registers 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 2. PCI Express I/O Virtualization 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 3. IOV Overview System System System Image Image Makes one device Image . #2 #3. #1. look like multiple devices SAS Controller Generally motivated SAS Controller SAS Controller by cost SAS Controller Seek performance within the cost Disk Disk Disk Disk envelope Disk Disk Disk Disk 2010 Storage Developer Conference.

E.g. VMWare running Win32 & Linux on a single CPU E.g. Blade server running multi -OS each on a single blade Each “System Image” (SI) needs to “see” it’s own PCI hierarchy Even if NO end devices are actually shared Only its “portion” of shared end devices 6

Tags:

  Express, Explained, Vmware, Virtualization, Express i o virtualization explained

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of PCI Express I/O Virtualization Explained

1 PCI Express I/O Virtualization Explained Richard Solomon LSI Corporation 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. Agenda PCI Express I/O Virtualization (IOV) Overview Single Root (SR-IOV). Multi-Root (MR-IOV). Secret Three Letter Acronym (TLA) Decoder Ring Programming IOV Devices Configuration Space Mapping Memory Space Mapping Programming Registers 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 2. PCI Express I/O Virtualization 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 3. IOV Overview System System System Image Image Makes one device Image . #2 #3. #1. look like multiple devices SAS Controller Generally motivated SAS Controller SAS Controller by cost SAS Controller Seek performance within the cost Disk Disk Disk Disk envelope Disk Disk Disk Disk 2010 Storage Developer Conference.

2 2010 LSI Corporation. All Rights Reserved. 4. IOV Overview From an adapter point of view: One physical device looks like multiple devices Virtual devices appear completely independent May occupy different PCI memory ranges May have different settings for various PCI. Configuration registers Need to keep cross- device traffic isolated Each virtual device *may* be a multi-function device 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 5. IOV Overview From a system point of view: System Image is a real or virtual system of CPU(s), Memory, O/S, I/O, etc Multiples may run on one or more sets of hardware vmware running Win32 & Linux on a single CPU. Blade server running multi-OS each on a single blade Each System Image (SI) needs to see it's own PCI hierarchy Even if NO end devices are actually shared Only its portion of shared end devices 2010 Storage Developer Conference.

3 2010 LSI Corporation. All Rights Reserved. 6. IOV Overview Attachment of existing PCIe Base components Root Complexes, Switches, Endpoints, and Bridges A solution to use a combination of existing base and IOV-aware components: Multi-Root Single Root capabilities are a superset of the PCIe Base Single Root specification Multi-Root capabilities are superset of the Single Root Base &. ATS. capabilities IOV-capable components are backwards-compatible with existing software Concentric Circles model Although some or all of the new IOV capabilities may not be supported in these circumstances 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 7. Single Root IOV (SR-IOV). Fits into existing PCI CPU #1.

4 CPU #2. CPU #3. CPU #n hierarchies today Single and Multi-CPU. Chipset boxes with traditional PCI Express single point of attachment Root Complex . Port(s). to PCI. PCI Express PCI Express Same address space . PCI Express Endpoint Endpoint Switch Device Device partitioned and allocated above the Root PCI Express Endpoint Device PCI Express Switch PCI Express Switch PCI Express Endpoint Device Complex PCI Express PCI Express PCI Express PCI Express PCI Express Endpoint Endpoint Endpoint Endpoint Endpoint Device Device Device Device Device 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 8. Single Root IOV (SR-IOV). Existing or absolutely minimally changed Root Complex ( chipset) and Switch silicon New Endpoint silicon Presumes existence of a Virtualization Intermediary (VI) aka a Hypervisor Direct result of don't change the chipset!

5 Philosophy Opens market to lots of existing or simply- derived systems might need new BIOS or chipset revision Shifts substantial burden to software 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 9. Multi-Root IOV (MR-IOV). Most obvious example is a blade server with a PCIe backplane . New PCIe hierarchy construct - (mini) fabric Logicallypartitions the hierarchy into multiple Virtual Hierarchies (VHs) all sharing the same physical hierarchy Targets small systems (16-32 Root Ports likely max). Workgroup saying Our yardstick is a yardstick the typical implementation is a system occupying not more than about 3 feet cubed Architected to allow larger, but not optimized that way 2010 Storage Developer Conference.

6 2010 LSI Corporation. All Rights Reserved. 10. MR-IOV Hierarchy CPU #1 CPU #1. CPU #2 CPU #1 CPU #2. CPU #3 CPU #2 CPU #3. CPU #n CPU #3 CPU #n CPU #n Chipset Chipset Chipset PCI Express PCI Express PCI Express Root Complex Root Complex . Root Complex . Port(s) Port(s). Port(s). PCI Express PCI Express Endpoint PCI Express PCI Express PCI Express PCI Express Endpoint Device Endpoint Endpoint Endpoint Endpoint Device Device Device Device Device Enhanced . PCI Express Switch PCI Express PCI Express PCI Express PCI Express Endpoint Endpoint Switch Switch Device Device PCI Express PCI Express PCI Express PCI Express PCI Express Endpoint Endpoint Endpoint Endpoint Endpoint Device Device Device Device Device 2010 Storage Developer Conference.

7 2010 LSI Corporation. All Rights Reserved. 11. Multi-Root IOV (MR-IOV). Existing or absolutely minimally changed Root Complex ( chipset) silicon New Switch silicon Allows for use of existing or minimally changed switches in a reduced capacity in certain places New Endpoint silicon Management model 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 12. Secret Three Letter Acronym (TLA) Decoder Ring 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 13. Decoder Ring Function The PCIe-addressed portion of a device devoted to one distinct chunk of that device's operation one SAS port, or the ethernet side of a combination ethernet/SAS device Virtual Function (VF).

8 A lightweight implementation of one view of a single device Function which is being virtualized to appear as multiple 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 14. Decoder Ring (cont'd). Physical Function (PF). Contains the SR-IOV control structure Used to manage a set of associated Virtual Functions Base Function (BF) [MR-IOV *ONLY*]. Contains the MR-IOV control structure Used to manage Virtual Hierarchies and Physical Functions Is *NOT* usable for real work of the device 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 15. Programming IOV Devices 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 16. Configuration Space Mapping 2010 Storage Developer Conference.

9 2010 LSI Corporation. All Rights Reserved. 17. Configuration Space Mapping Determine desired number of Virtual Functions from InitialVFs field Program NumVFs field to match Multi-Root adds a further layer where configuration software first allocates VFs to Virtual Hierarchies thus InitialVFs may be less than TotalVFs 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 18. Configuration Space Mapping Bus Number / Device Number / Example Simple Single-Function Configuration Space on Bus Number nn Function Number (BDF) field VF0,4. Configuration now known as RoutingID (RID) Space RID=nn04. Virtual RIDs of VFs found from SR-IOV. VF0,3. Functions Configuration Space configuration RID=nn03.

10 VF0,2. Configuration Space RID=nn02. VF0,1. Configuration Space RID=nn01. PF0. Configuration Physical Space Function RID=nn00. 2010 Storage Developer Conference. 2010 LSI Corporation. All Rights Reserved. 19. Configuration Space Mapping More complex offset/stride Example Complex Multi-Function combinations are possible, Configuration Space on Bus Number nn particularly with multi-function VF1,4. Configuration Space devices RID=nn0F. VF1,3. Virtual Functions Configuration PF0 and PF1 Space RID=nn0D. both set to: VF0,2. Configuration VF1,2. Configuration Offset=8. Space Space RID=nn0A RID=nn0B. VF0,1 VF1,1. Stride=2 Configuration Space Configuration Space RID=nn08 RID=nn09. PF0 PF1. Configuration Configuration Physical Space Space Functions RID=nn00 RID=nn01.


Related search queries