Resource Central: Understanding and Predicting Workloads ...

1 Resource Central: Understanding and PredictingWorkloads for Improved Resource Management inLarge Cloud Platforms Eli BondeMicrosoft MuzioITA, BianchiniMicrosoft research to date has lacked data on the characteris-tics of the production virtual machine (VM) Workloads oflarge cloud providers. A thorough Understanding of thesecharacteristics can inform the providers Resource manage-ment systems, VM scheduler, power manager, serverhealth manager. In this paper, we first introduce an exten-sive characterization of Microsoft Azure s VM workload,including distributions of the VMs lifetime, deployment size,and Resource consumption. We then show that certain VMbehaviors are fairly consistent over multiple lifetimes, is an accurate predictor of future behavior. Based onthis observation, we next introduce Resource Central (RC),a system that collects VM telemetry, learns these behaviorsoffline, and provides predictions online to various resourcemanagers via a general client-side library.

2 As an example ofRC s online use, we modify Azure s VM scheduler to leveragepredictions in oversubscribing servers (with oversubscrib-able VM types), while retaining high VM performance. Usingreal VM traces, we then show that the prediction-informedschedules increase utilization and prevent physical resourceexhaustion. We conclude that providers can exploit theirworkloads characteristics and machine learning to improveresource management substantially. Cortez and Bonde contributed equally. Muzio was an intern at to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee.

3 Requestpermissions from 17, October 28, 2017, Shanghai, China 2017 Association for Computing ISBN 978-1-4503-5085-3/17/10.. $ CONCEPTS Computer systems organization Cloud comput-ing; Computing methodologies Machine learningapproaches;KEYWORDSC loud Workloads , machine learning, predictive managementACM Reference Format:Eli Cortez, Anand Bonde, Alexandre Muzio, Mark Russinovich,Marcus Fontoura, and Ricardo Bianchini. 2017. Resource Central: Understanding and Predicting Workloads for Improved ResourceManagement in Large Cloud Platforms. InProceedings of SOSP , New York, NY, USA, 15 pages. computing has been expanding at a fastpace, especially as enterprises continue to move their opera-tions to large cloud providers such as Microsoft Azure, Ama-zon Web Services (AWS), and Google Cloud Platform (GCP).Due to heated marketplace competition, providers have beenunder pressure to produce attractive features and services,while controlling their datacenter costs. These factors com-bine to expose providers to a wide variety of Workloads (fromboth external customers and their own internal services) thatmust share a common datacenter infrastructure.

4 Providinggood performance, availability, and reliability under theseconditions can be expensive without sophisticated (but prac-tical and scalable) Resource , research on cloud Resource managementto date has lacked a thorough Understanding of the key char-acteristics of the Workloads of large commercial example, no prior study has explored the lifetime (timebetween creation and termination) or Resource consumptiondistributions of these providers production virtual machinesSOSP 17, October 28, 2017, Shanghai, ChinaE. Cortez, A. Bonde, A. Muzio, M. Russinovich, M. Fontoura, R. Bianchini(VMs). Instead, the prior work has mostly used real but non-VM Workloads , synthetic VM Workloads , and/or focused onmanaging resources via general but often impractical tech-niques for a large cloud provider. For example, many papersexplore (sometimes offline) workload profiling and aggres-sive online Resource reallocation, via dynamic monitoring,scheduling, and/or live VM migration [2,7,20,21,24,27]. Inpractice, offline profiling is infeasible because the Workloads inputs are often unavailable until VMs run in profiling is challenging, as it is hard to determinewhen an arbitrary VM has shown representative performance ( , tail latency) monitoringis usually not possible, as it requires help from , live migration retains contended resources for a rela-tively long time ( , it cannot free up memory pages beforesuccessfully migrating them) and can cause widespread net-work traffic bursts.

5 Practical uses of these techniques requireextreme argue that Resource management can become more ef-fective and practical for large providers with a deeper under-standing of their VM Workloads key characteristics. More-over, if these characteristics can be accurately predicted, im-provements could be even greater. For example, accuratelypredicting Resource utilization at VM deployment time wouldallow Resource -contention-aware VM co-location (mitigat-ing the need for VM migration). Similarly, run-time lifetimepredictions would allow the health management system toestimate when all the VMs running on a misbehaving serverwill likely terminate (facilitating server maintenance with-out VM migration or downtime). We are unaware of priorwork showing that the characteristics of large providers pro-duction VMs can be accurately predicted for better , there is a need for software that can produce suchpredictions and enable the providers Resource managementsystems ( , the VM scheduler, the server health manager)to leverage them.

6 Some prediction-serving systems [5,6,10]have been proposed recently, but have not been studied inthe context of prediction-based Resource this paper, we first introduce a characteri-zation of Azure s (first- and third-party) VM workload, in-cluding distributions of the VMs size, lifetime, resourceconsumption, utilization pattern, and deployment size. Re-searchers can use these distributions to produce realisticcloud Workloads for their own work. Alternatively, theycan use the sanitized production traces we have placed The traces con-tain a subset of our data, but exhibit roughly the same overalltrends as the full characterization shows that many types of VM be-havior are fairly consistent over multiple lifetimes, whenobserved from the perspective of each cloud customer. Thisobservation suggests that prior history may be an accuratepredictor of the future behavior of the customers VMs, somachine learning algorithms could be used online to produceVM behavior on this observation, we introduce Resource Central(RC), a system that collects VM telemetry, periodically learnsthese behaviors into prediction models offline, and providesbehavior predictions online to various Resource managementsystems.

7 Unlike other systems, RC serves predictions from aclient-side library, which caches prediction results, models,and feature data. The library API is simple yet general, so itcan be used with many types of Resource managers, learningalgorithms, prediction models, and feature data. RC s modelsand feature data are currently in production, being usedmanually by engineers and data scientists for analysis andsystem design. The changes to the systems that will leverageRC are still being an example of RC s online use, we describe our modi-fications to Azure s VM scheduler, which selects a physicalserver for each new VM. Specifically, we modify the sched-uler to collect high-percentile utilization predictions to usein oversubscribing physical servers with oversubscribable VM types ( , first-party VMs that run non-customer-facingworkloads), while retaining good VM evaluation starts by quantifying the accuracy of RC spredictions of six metrics, using our VM telemetry. The re-sults show overall prediction accuracies between 79% and90%, depending on the metric.

8 For example, RC predicts theaverage CPU utilization of a new VM with 81% then quantify the performance of RC s components, andconfirm that models and feature data are compact and fastenough to be executed on the client-side. To evaluate thebenefit of predictions, we explore our modified VM schedulerusing real VM traces and simulation. Our results show thatprediction-informed VM schedules enable safe oversubscrip-tion, while keeping VM deployment failures and contributions are: We present a detailed characterization of several VM work-load behaviors from Microsoft Azure; We show that these behaviors can be accurately predicted; We describe Resource Central, a large-scale system forproducing, storing, and using such predictions; We describe modifications to Azure s VM scheduler thatleverage predictions to improve server selection; We present extensive results evaluating Resource Centraland our modified VM scheduler; and We make a large VM dataset available to the conclude that cloud providers can exploit the charac-teristics of their Workloads and machine learning techniquesto improve Resource management Central: Understanding and Predicting Cloud WorkloadsSOSP 17, October 28, 2017, Shanghai, China2 RELATED WORKC loud VM workload are unawareof other characterizations of the VM workload of real publiccloud providers, except for [15] which focused solely onresource demand volatility and pricing.

9 Other works studiedcloud platform performance [16] and how users implementtheir front-end services in the cloud [11].The closest prior works [8,19,22] didnotaddress publiccloud Workloads . Rather, they characterized a month-longtrace of 12k bare-metal servers running first-party container-based ( , non-VM) Workloads at Google [25]. In contrast, wecharacterize Azure sentire VM workloadover three months,including third-party VMs. In fact, third-party VMs oftenexhibit different characteristics than first-party ones, as weshow in Section importantly, VM Workloads fundamentally differfrom bare-metal container Workloads . Mainly for securityreasons, public cloud providers must encapsulate their cus-tomers Workloads using VMs. Unfortunately, VMs imposehigher creation/termination overheads than containers run-ning on bare metal, so they are likely to live longer, pro-duce lower Resource utilization, and be deployed in smallernumbers. For example, a user who wants to run multipleMapReduce jobs may create a set of VMs sized for the jobs maximum Resource needs; she would only destroy/shutdownthe VMs when all jobs have completed.

10 In a container-basedsystem, each map and reduce task of each job would likelyreceive a different container, which can be more accuratelysized for its task. As another example, services that assign aVM to each user ( , gaming) may pool and reuse VMs, sothat the overhead of VM creation becomes invisible to learning and prediction-serving have recently proposed several frameworks, such asTensorFlow [1], Caffe [13], and MLLib [18], for producingmachine learning models. Our work is orthogonal to them,as RC relies on TLC, a Microsoft-internal state-of-the-artframework that implements many learning , RC is comparable to recent prediction-servingsystems [5,6,10]. In contrast with these systems, RC cachesprediction results, models, and feature data on approach enables the system to operate even when thedata store or the connectivity to it is unavailable. In addition,it provides higher performance by removing the interconnectand store from the critical performance cloud literature on predictingworkload behaviors is extensive.