Resource Central: Understanding and Predicting Workloads ...

1 Resource Central: Understanding and PredictingWorkloads for Improved Resource Management inLarge Cloud Platforms Eli BondeMicrosoft MuzioITA, BianchiniMicrosoft research to date has lacked data on the characteris-tics of the production virtual machine (VM) Workloads oflarge cloud providers. A thorough Understanding of thesecharacteristics can inform the providers Resource manage-ment systems, VM scheduler, power manager, serverhealth manager. In this paper, we first introduce an exten-sive characterization of Microsoft Azure s VM workload,including distributions of the VMs lifetime, deployment size,and Resource consumption.

2 We then show that certain VMbehaviors are fairly consistent over multiple lifetimes, is an accurate predictor of future behavior. Based onthis observation, we next introduce Resource Central (RC),a system that collects VM telemetry, learns these behaviorsoffline, and provides predictions online to various resourcemanagers via a general client-side library. As an example ofRC s online use, we modify Azure s VM scheduler to leveragepredictions in oversubscribing servers (with oversubscrib-able VM types), while retaining high VM performance. Usingreal VM traces, we then show that the prediction-informedschedules increase utilization and prevent physical resourceexhaustion.

3 We conclude that providers can exploit theirworkloads characteristics and machine learning to improveresource management substantially. Cortez and Bonde contributed equally. Muzio was an intern at to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee.

4 Requestpermissions from 17, October 28, 2017, Shanghai, China 2017 Association for Computing ISBN 978-1-4503-5085-3/17/10.. $ CONCEPTS Computer systems organization Cloud comput-ing; Computing methodologies Machine learningapproaches;KEYWORDSC loud Workloads , machine learning, predictive managementACM Reference Format:Eli Cortez, Anand Bonde, Alexandre Muzio, Mark Russinovich,Marcus Fontoura, and Ricardo Bianchini. 2017. Resource Central: Understanding and Predicting Workloads for Improved ResourceManagement in Large Cloud Platforms. InProceedings of SOSP , New York, NY, USA, 15 pages.

5 Computing has been expanding at a fastpace, especially as enterprises continue to move their opera-tions to large cloud providers such as Microsoft Azure, Ama-zon Web Services (AWS), and Google Cloud Platform (GCP).Due to heated marketplace competition, providers have beenunder pressure to produce attractive features and services,while controlling their datacenter costs. These factors com-bine to expose providers to a wide variety of Workloads (fromboth external customers and their own internal services) thatmust share a common datacenter infrastructure. Providinggood performance, availability, and reliability under theseconditions can be expensive without sophisticated (but prac-tical and scalable) Resource , research on cloud Resource managementto date has lacked a thorough Understanding of the key char-acteristics of the Workloads of large commercial example, no prior study has explored the lifetime (timebetween creation and termination) or Resource consumptiondistributions of these providers production virtual machinesSOSP 17, October 28, 2017, Shanghai, ChinaE.

6 Cortez, A. Bonde, A. Muzio, M. Russinovich, M. Fontoura, R. Bianchini(VMs). Instead, the prior work has mostly used real but non-VM Workloads , synthetic VM Workloads , and/or focused onmanaging resources via general but often impractical tech-niques for a large cloud provider. For example, many papersexplore (sometimes offline) workload profiling and aggres-sive online Resource reallocation, via dynamic monitoring,scheduling, and/or live VM migration [2,7,20,21,24,27]. Inpractice, offline profiling is infeasible because the Workloads inputs are often unavailable until VMs run in profiling is challenging, as it is hard to determinewhen an arbitrary VM has shown representative performance ( , tail latency) monitoringis usually not possible, as it requires help from , live migration retains contended resources for a rela-tively long time ( , it cannot free up memory pages beforesuccessfully migrating them) and can cause widespread net-work traffic bursts.

7 Practical uses of these techniques requireextreme argue that Resource management can become more ef-fective and practical for large providers with a deeper under-standing of their VM Workloads key characteristics. More-over, if these characteristics can be accurately predicted, im-provements could be even greater. For example, accuratelypredicting Resource utilization at VM deployment time wouldallow Resource -contention-aware VM co-location (mitigat-ing the need for VM migration). Similarly, run-time lifetimepredictions would allow the health management system toestimate when all the VMs running on a misbehaving serverwill likely terminate (facilitating server maintenance with-out VM migration or downtime).

8 We are unaware of priorwork showing that the characteristics of large providers pro-duction VMs can be accurately predicted for better , there is a need for software that can produce suchpredictions and enable the providers Resource managementsystems ( , the VM scheduler, the server health manager)to leverage them. Some prediction-serving systems [5,6,10]have been proposed recently, but have not been studied inthe context of prediction-based Resource this paper, we first introduce a characteri-zation of Azure s (first- and third-party) VM workload, in-cluding distributions of the VMs size, lifetime, resourceconsumption, utilization pattern, and deployment size.

9 Re-searchers can use these distributions to produce realisticcloud Workloads for their own work. Alternatively, theycan use the sanitized production traces we have placed The traces con-tain a subset of our data, but exhibit roughly the same overalltrends as the full characterization shows that many types of VM be-havior are fairly consistent over multiple lifetimes, whenobserved from the perspective of each cloud customer. Thisobservation suggests that prior history may be an accuratepredictor of the future behavior of the customers VMs, somachine learning algorithms could be used online to produceVM behavior on this observation, we introduce Resource Central(RC), a system that collects VM telemetry, periodically learnsthese behaviors into prediction models offline, and providesbehavior predictions online to various Resource managementsystems.

10 Unlike other systems, RC serves predictions from aclient-side library, which caches prediction results, models,and feature data. The library API is simple yet general, so itcan be used with many types of Resource managers, learningalgorithms, prediction models, and feature data. RC s modelsand feature data are currently in production, being usedmanually by engineers and data scientists for analysis andsystem design. The changes to the systems that will leverageRC are still being an example of RC s online use, we describe our modi-fications to Azure s VM scheduler, which selects a physicalserver for each new VM.