Enabling VMware Enhanced VMotion …

1 Enabling VMware Enhanced VMotion compatibility on HP proliant servers Technical white paper Table of contents Executive 2. VMware VMotion 2. VMotion CPU compatibility .. 2. Enhanced VMotion compatibility .. 3. VMware Enhanced VMotion compatibility requirements .. 7. Intel-based HP proliant servers allowed in an EVC cluster .. 8. AMD Opteron-based HP proliant servers allowed in an EVC cluster .. 9. Summary .. 10. Appendix A Enabling processor virtualization options .. 11. Appendix B migration logs .. 17. For more information .. 18. Executive summary VMware VMotion technology allows running virtual machines to move from one physical machine to another with no impact to the virtual machines.

2 VMotion offers improved system utilization with load balancing, increased serviceability and manageability, as well as Enhanced flexibility. Administrators can reduce unplanned downtime and can eliminate planned downtime to perform hardware maintenance, such as disruptive firmware updates. Successful VMotion migration requires CPU. compatibility between source and destination ESX hosts. This document explains the use of HP. proliant servers with VMware Enhanced VMotion compatibility (EVC) to ensure all hosts in a cluster are VMotion compatible. Target audience: The intended audience for this document is VMware administrators who intend to deploy HP proliant servers in EVC clusters, and purchasing managers who wish to add new HP.

3 proliant servers to an EVC cluster. It is assumed that you have working knowledge of VMware Infrastructure 3 (VI3) and/or VMware vSphere 4. VMware VMotion overview The VMotion process starts by VMware vCenter performing several checks to verify that the virtual machine to be migrated is in a stable state on the source host and that the destination host is compatible. Next, vCenter begins an iterative pre-copying of the memory state of the source guest to the destination host. The memory pre-copy completes when memory changed is below a given threshold or no forward progress is made. The amount of time needed to perform the memory pre- copy depends on the workload, amount of memory and type of network used for VMotion .

4 This step can happen in seconds, or it can take minutes. Next the virtual machine is quiesced, and the remaining state is sent to the destination host. At this point, control is transferred from the source host to the destination host. This step typically takes under one second. The last step is to send the remaining modified memory, and start the virtual machine on the destination host. The amount of time needed for this step depends on workload and memory size. In the event that the VMotion operation fails, the virtual machine continues to run on the source host. A VMotion operation can fail for several reasons, such as network latency or unresponsive storage.

5 In many cases, vCenter will have an error message detailing the cause of the failed migration. The ESX. hosts also log migration information. See Appendix B migration logs for information on logs to check to help diagnose a failed VMotion migration. VMotion CPU compatibility Successful VMotion migration requires that the processors of the destination host be able to execute using equivalent instructions to those the processors of the source host were using when the virtual machine was migrated off of the source host. Processor clock speeds, cache sizes, and the number of processor cores can vary, but processors must come from the same vendor (Intel or AMD ) and present an identical CPU feature set.

6 VMotion compatibility rules prevent unsafe migration that can make a virtual machine unstable. By default, vCenter only allows live migration with VMotion between source and destination processors with a compatible feature set. If processors do not have a compatible feature set, a CPU. mask must be used to make the CPU feature set on the destination host appear identical to the CPU. feature set on the source host. For information on VMotion CPU compatibility requirements for Intel processors, see VMware KB Article 1991, For information on VMotion CPU compatibility requirements for AMD processors, see VMware KB Article 1992, 2. Note VMware does not recommend or support the use of CPU compatibility masks in production environments.

7 For more information on CPU. compatibility masks, see Enhanced VMotion compatibility Enhanced VMotion compatibility (EVC) removes the need to set CPU masks manually. EVC is a cluster setting that automatically configures all hosts in the cluster to be VMotion compatible with each other. All guests in the cluster can migrate live to any host in the cluster because guests always see an identical CPU feature set from all hosts in the EVC cluster. Figures 1 and 2 show the EVC options available in VI3 vCenter cluster settings. VI3 has one EVC mode for Intel hosts and one EVC mode for AMD hosts. Note Before Enabling EVC, or adding a new host to an EVC cluster, it is recommended that the host be updated to the latest HP proliant system ROM version.

8 Figure 1. VI3 EVC cluster setting for AMD hosts 3. Figure 2. VI3 EVC cluster setting for Intel hosts Figures 3 shows the VMware EVC modes available in vSphere 4 for AMD hosts. vSphere 4 has four EVC modes for AMD hosts (AMD Opteron Generation 1, AMD Opteron Generation 2, AMD. Opteron Generation 3 (no 3 DNow!), and AMD Opteron Generation 3). Figure 3. vSphere EVC cluster setting for AMD hosts 4. Figures 4 shows the EVC modes available in vSphere 4 for Intel hosts. vSphere 4 has four EVC modes for Intel hosts (Intel Xeon Core 2, Intel Xeon 45nm Core 2, Intel Xeon Core i7, and Intel Xeon 32nm Core i7). Figure 4. vSphere EVC cluster setting for Intel hosts EVC uses Intel VT FlexMigration and AMD-V Extended Migration, concepts jointly developed by VMware and the CPU manufacturers, to dynamically turn off selected CPUID feature bits.

9 Intel VT. FlexMigration is available in Intel processors with the Intel Core 2 microarchitecture and newer. AMD-V Extended Migration is available in Second-Generation AMD Opteron processors and newer. In VI3, with vCenter U2 and hosts using ESX U2 or later, there is one EVC baseline for each CPU vendor. For Intel Xeon processor-based EVC clusters, the baseline is CPU features supported by Intel Core 2 (Merom) processors. For AMD processor-based EVC clusters, the baseline is CPU features supported in AMD Opteron First and Second Generation (Revision E/F) processors. vSphere 4 includes support for multiple baselines, such as Penryn and Nehalem baselines for Intel- based EVC clusters, and a Greyhound baseline for AMD-based EVC clusters.

10 This allows more control over which CPU features are exposed to the guest. There is a tradeoff between compatibility and capability. The most capable baseline will expose the largest subset of CPU features supported by CPUs in the cluster, but older hardware may not be added to the cluster. The most compatible baseline will expose a minimal set of CPU features to the guest so older hardware may be added to the cluster. Table 1 lists HP proliant server processors supported in EVC clusters. 5. Table 1. Processors in HP proliant Servers Supported in EVC Clusters Baseline Processors Supported Intel Xeon Core 2 (Merom). 51xx, 53xx, 72xx, 73xx series Intel Xeon 45nm Core 2 (Penryn).