Intel® VMD-Enabled NVMe Driver for VMware* ESXi with …

1 intel VMD-Enabled nvme Driver for vmware * ESXi with RAID1 User Guide Author: Jay Lynn Cassio intel VMD/VROC Application Engineer intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 2 Legal Notices and Disclaimers intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information. intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

2 Benchmark results were obtained prior to implementation of recent software patches and firmware updates intended to address exploits referred to as "Spectre" and "Meltdown". Implementation of these updates may make these results inapplicable to your device or system. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. Results have been estimated based on internal intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. All documented performance test results are obtained in compliance with JESD218 Standards; refer to individual sub-sections within this document for specific methodologies.

3 See for detailed definitions of JESD218 Standards. intel does not control or audit the design or implementation of third party benchmark data or Web sites referenced in this document. intel encourages all of its customers to visit the referenced Web sites or others where similar performance benchmark data are reported and confirm whether the referenced benchmark data are accurate and reflect performance of systems available for purchase. The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. For copies of this document, documents that are referenced within, or other intel literature please contact you intel representative. Contact your local intel sales office or your distributor to obtain the latest specifications and before placing your product order.

4 *Other names and brands may be claimed as the property of others. Copyright 2022 intel Corporation. All rights reserved. intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 3 Revision History Revision Number Description Revision Date intel VMD vmware * Release with RAID1 Details Feb 2022 intel VMD-Enabled nvme Driver For vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* Februrary2022 User Guide 4 Contents Revision History .. 3 1 Overview .. 6 References .. 7 Terms and Acronyms .. 8 2 Product Requirements .. 9 Platform Requirements .. 9 If testing VMD with Bootable nvme , the intel VMD-UEFI drivers must be loaded into the BIOS .. 9 Enabling intel 9 3 Product 11 Suggested Installation and Management Tools .. 11 Installing the out-of-box intel VMD-Enabled Driver on nvme RAID Volume.

5 11 Enable VMD and Configure RAID in the 11 vmware Installation with out-of-box intel VMD-Enabled nvme Driver on SATA or nvme Not behind a VMD controller .. 12 How To Set Up Network DHCP .. 14 Set up system for Remote Console and 16 4 Manage intel VMD-Enabled nvme Driver for RAID .. 20 5 Upgrading the vmware * ESXi* inbox intel VMD Driver to Out of Box intel VMD RAID and LED Management Tools .. 22 Update Process .. 22 Installing and using the intel -vmdr-user tool for RAID and LED Management .. 23 6 Workaround for DataStores Missing After Updating to intel VMD nvme Driver From Inbox vmware nvme Driver .. 25 7 Helpful WMWare* ESXi* Commands for intel VMD-Enabled nvme Driver Management .. 28 Installing software .. 28 Debugging Set debug level for kernel log data when issues occur .. 28 Formatting nvme Devices .. 28 Change LBA.

6 29 Disk Listing .. 29 Use ISDCT Tool for vmware .. 29 Disk Management .. 29 Performing IO using IOZONE .. 29 Perform IO .. 30 Power management .. 30 Saving Core Dump 30 Others .. 30 8 Instructions for Removing / Injecting Out of Box intel VMD-Enabled nvme Driver into ESXi bootable image .. 31 Tools Needed .. 31 Software Needed .. 31 intel VMD-Enabled nvme Driver for vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 5 Tables Table 1: Standard Information referenced in this document .. 7 Table 2: Glossary of Terms and Acronyms .. 8 intel VMD-Enabled nvme Driver for vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 6 1 Overview This document describes the intel Volume Management Device ( intel VMD) enabled nvme Driver for vmware * ESXi*.

7 It includes the installation method for adding or upgrading the out-of-box Driver to existing vmware * ESXi* version Operating Systems on intel Xeon Scalable Platforms that have enabled intel VMD in the BIOS. At the time of this writing, an inbox Driver does not exist, and therefore the intel VMD-Enabled nvme Driver requires manual installation or upgrade into the existing vmware * ESXi* Operating System solution. The intel VMD-Enabled nvme Driver is designed to work with intel VMD hardware logic implemented on the intel Xeon Scalable family. intel VMD is much like an HBA controller for nvme SSDs, and adds robust management for nvme SSDs in the Data Center, such as Surprise Hot Plug, LED management, and error handling. The intel VMD-Enabled nvme Driver for vmware * ESXi now also supports RAID 1 Volumes for Boot and Data. intel VMD-Enabled nvme Driver for vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 7 References Table 1: Standard Information referenced in this document Date Title Location October 2019 LED Management for PCI Express SSDs intel CDI/IBL doc# 567916 Sept 2021 intel VMD-Enabled nvme Driver Consult Latest BKC announcements February 2022 intel VMD LED Management Tools Provided on intel Resource and Documentation Center intel VMD-Enabled nvme Driver for vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 8 Terms and Acronyms Table 2.

8 Glossary of Terms and Acronyms Term Definition nvme Non-Volatile Memory Express ISO International Standards Organization SSD Solid State Drive intel VMD-Enabled nvme Driver for vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 9 2 Product Requirements Platform Requirements intel Generation 1, 2, 3, 4 intel Xeon Scalable Processor intel nvme SSDs in format attached to a Hot Swappable Backplane (HSBP) for Surprise Hot-plug management. LED Management must be implemented with intel VMD method using the White Paper located on CDI/IBL: LED Management for PCI Express SSDs CCL doc# 567916 If testing VMD with Bootable nvme , the intel VMD-UEFI drivers must be loaded into the BIOS The following intel VMD-Enabled nvme UEFI drivers are required for nvme devices attached to intel VMD enabled PCIe lanes: These UEFI drivers should be used Enabling intel VMD intel VMD must be enabled in the BIOS.

9 Please check with your BIOS vendor for instructions on enabling intel VMD in the BIOS set up menu. Depending on the validation platform of choice intel VMD must be enabled on the Surprise hot plug capable lanes of the platform. For LED Management, the nvme devices should be attached to the Hot swappable Backplane through an internal Retimer Add In Card (AIC), switch, or connectors directly to the motherboard. If using a retimer card for PCIe nvme SSDs, a PEM Cable must be attached from the Add-in-Card to the motherboard PEM CPU connection. For Surprise hot plug to work correctly, some platforms allow a jumper to be set to disable ACPI hot plug. PCIe hotplug is required to support intel VMD. The latest intel VROC UEFI drivers must be installed in the BIOS: After these UEFI drivers are loaded when the BIOS loads, there will be a HII menu in the BIOS setup screen: intel Virtual RAID on CPU (BIOS menu paths may differ depending on BIOS vendor) intel Virtual RAID on CPU is the HII menu for creating intel Proprietary metadata for RAID support in Windows, Linux, and ESXi Operating Systems.

10 intel VMD-Enabled nvme Driver for vmware * ESXi* intel VMD-Enabled nvme Driver for vmware * ESXi* February 2022 User Guide 10 Supported RAID Options. The intel Virtual RAID on CPU menu should be used to create the desired RAID volume(s) and RAID level(s). Currently supported RAID configurations are: RAID1 boot volume (ESXi redundant boot). Only 1 boot volume is supported. Dual RAID1 boot volumes are not supported as of this writing. RAID1 data volume is supported only with VMD Driver and newer. The RAID1 data volume can exist simultaneously with the RAID1 boot volume with the following requirements: Boot and data volume placement requirements depending upon if vSAN is being used. If vSAN is NOT being used: Boot and data volumes can share a VMD domain If vSAN IS being used: Boot volume must be on separate controller (VMD Domain) from data volumes.