Host Power Management in VMware vSphere 7

1 VMware , I nc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www. VMware . com Copyright 2021 VMware , Inc. All rights reserved. This product is protected by and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at VMware is a registered trademark or trademark of VMware , I nc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Host Power Management in VMware vSphere Performance Study for Optimal Power Consumption September 28, 2021 Host Power Management in VMware vSphere | Page 2 Table of Contents Executive Summary.

2 3 Introduction .. 3 Background .. 4 C-States (I dle States/ Power States) .. 4 P-State (Operational States/Performance State): Demand-Based Switching (DBS) .. 4 Power Management Settings .. 5 BI OS Settings .. 6 Host Power Management in vSphere 7. 0 .. 6 Defining Custom Power Policy in vSphere .. 8 Summary: vSphere Power Policies .. 8 Experimental Setup .. 9 Hardware .. 9 vSphere Power Management and esxtop .. 10 Results .. 12 I mpact of P-States and C-States on Host Power Consumption .. 12 Observation 1: Only P-State Management is Enabled .. 13 Observation 2: Both P-State and C-State Management are Enabled.

3 13 LoadGen .. 14 SPEC Power .. 15 VMmark .. 17 View Planner (VDI) .. 20 Impact of vSphere Power Policies on Turbo 23 Best Practices .. 25 References .. 26 About the Authors .. 27 Acknowledgments .. 27 Appendix A .. 28 Host Power Management in VMware vSphere | Page 3 Executive Summary Modern datacenters, which are vital to the computing infrastructure, are expensive to build and operate. A significant part of the operational cost of a modern data center can be attributed to Power consumption and cooling. In an ideal world, we would want to minimize Power consumption without any impact on application performance.

4 H owever, practical cases require a compromise. Finding the right balance entails considerable financial gains. VMware vSphere (ESXi) provides several pre-configured policies for Power Management , targeting applications to optimize for latency, throughput, and Power consumption. Prior technical papers have described the available Power policies in vSphere , their functionality, and present performance and Power consumption data [1] [2]. This technical paper provides an updated guide to vSphere 7. 0 users on different Power policies performance and Power tradeoffs. In addition, the article uses experimental results to gain insights regarding the Power Management capabilities of VMware vSphere on the latest processors using modern workloads.

5 Note that there are no algorithmic changes in the implementation of Power Management in vSphere I nstead, the article evaluates the vSphere Power Management on the latest processors with several recent workloads. While observing the performance and Power consumption on a range of workloads, the paper recommends the use of Power policies as follows: (a) The Balanced policy is the default recommended policy in vSphere , because it maximizes performance-per-watt overall and provides an optimal point between performance and Power consumption for a wide variety of application characteristics.

6 (b) The High Performance profile is preferred in the case of latency-sensitive applications at the cost of higher Power consumption, and (c) Low Power profile for low utilization servers, resulting in higher Power savings at the cost of performance. Introduction The Power used by servers and data centers accounts for ~ 1. 5% of the total Power consumption in the world [3]. CPUs in modern data centers consume ~ 30% of the overall system Power . As CPU hardware complexity has grown significantly in recent years, computer architects have introduced several Power -focused enhancements. These designs have enabled operating systems to capitalize on architectural tools to improve performance-per-watt on various applications.

7 The introduction of demand-based switching (DBS) from Intel has been the critical factor behind Power savings in modern processors. Host Power Management in VMware vSphere | Page 4 The Advanced Configuration and Power Interface (ACPI) specification is an open standard initially developed by several hardware vendors and software developers [4]. It establishes standard interfaces that enable operating system directed motherboard device configuration and Power Management . ACPI can appl y to both individual hardware components and the entire system. In addition, it helps monitor the system s status and employs Power Management algorithms by changing the CPU operating frequency and putting unused components to sleep [5].

8 Background The ACPI standard defines C-States (commonly known as idle states or Power states) and P-States (widel y kn own as operational states or performance states). Hardware vendors such as Intel and AMD have introduced hardware support for P-States and C-States, which are accessible through the on-chip Power control unit (PCU) [6] [ 7]. C-States (Idle States/ Power States) C-States are Power states that aid in saving Power by turning off sub-sections of the CPU when not in use. CPUs are designed to support various Power levels. C0-state is the operational state where all the components are active, and the processor can actively execute instructions.

9 C1 is a shallow state where the clock is gated (switched off). However, all the modules remain active, and the processor can go back to the active C0 state instantaneously. Furthermore, C2-Cn are sleep states where specific sections of the CPU are turned off. The higher the C-State, the deeper into sleep mode the CPU goes. Thus, higher C-States result in significant Power savings. However, it takes more time for the CPU to return to the operational state from deeper sleep states. Therefore, changing the C-state in the BIOS setting doesn t have an impact on throughput. H owever, the latency to get back from a deeper sleep state would be higher.

10 P-State (Operational States/Performance State): Demand-Based Switching (DBS) P-States correspond to different performance levels that are applied while the processor is actively executing instructions. P-States are relevant only when the processor is in the active C0-state. P-State is both a frequency and voltage operating point defined as performance states in the ACPI specification. Both frequency and voltage are scaled as the P-State increases. This process is referred to as dynamic voltage and frequency scaling (DVFS). Hardware vendors such as Intel incorporate several hardware-level P-States that divide the energy and frequency demands into several tiers.