Azure Accelerated Networking: SmartNICs in the Public Cloud

1 Azure Accelerated Networking: SmartNICs in the Public CloudDaniel FirestoneAndrew PutnamSambhrama MundkurDerek ChiouAlireza DabaghMike AndrewarthaHari AngepatVivek BhanuAdrian CaulfieldEric ChungHarish Kumar ChandrappaSomesh ChaturmohtaMatt HumphreyJack LavierNorman LamFengfen LiuKalin OvtcharovJitu PadhyeGautham PopuriShachar RaindelTejas SapreMark ShawGabriel SilvaMadhan SivakumarNisheeth SrivastavaAnshuman VermaQasim ZuhairDeepak BansalDoug BurgerKushagra VaidDavid A. MaltzAlbert GreenbergMicrosoftAbstractModern Cloud architectures rely on each server running itsown networking stack to implement policies such as tun-neling for virtual networks, security, and load , these networking stacks are becoming increas-ingly complex as features are added and as network speedsincrease.

2 Running these stacks on CPU cores takes awayprocessing power from VMs, increasing the cost of run-ning Cloud services, and adding latency and variability tonetwork present Azure Accelerated Networking (AccelNet),our solution for offloading host networking to hardware,using custom Azure SmartNICs based on FPGAs. Wedefine the goals of AccelNet, including programmabilitycomparable to software, and performance and efficiencycomparable to hardware. We show that FPGAs are the bestcurrent platform for offloading our networking stack asASICs do not provide sufficient programmability, and em-bedded CPU cores do not provide scalable performance,especially on single network SmartNICs implementing AccelNet have beendeployed on all new Azure servers since late 2015 in afleet of>1M hosts.

3 The AccelNet service has been avail-able for Azure customers since 2016, providing consis-tent<15 s VM-VM TCP latencies and 32 Gbps through-put, which we believe represents the fastest network avail-able to customers in the Public Cloud . We present thedesign of AccelNet, including our hardware/software co-design model, performance results on key workloads, andexperiences and lessons learned from developing and de-ploying AccelNet on FPGA-based Azure IntroductionThe Public Cloud is the backbone behind a massive andrapidly growing percentage of online software services [1,2, 3].

4 In the Microsoft Azure Cloud alone, these servicesconsume millions of processor cores, exabytes of stor-age, and petabytes of network bandwidth. Network per-formance, both bandwidth and latency, is critical to mostcloud workloads, especially interactive a large Public Cloud provider, Azure has built itscloud network on host-based software-defined network-ing (SDN) technologies, using them to implement almostall virtual networking features, such as private virtual net-works with customer supplied address spaces, scalable L4load balancers, security groups and access control lists(ACLs)

5 , virtual routing tables, bandwidth metering, QoS,and more. These features are the responsibility of the hostplatform, which typically means software running in cost of providing these services continues to in-crease. In the span of only a few years, we increased net-working speeds by 40x and more, from 1 GbE to 40 GbE+,and added countless new features. And while we built in-creasingly well-tuned and efficient host SDN packet pro-cessing capabilities, running this stack in software on thehost requires additional CPU cycles.

6 Burning CPUs forthese services takes away from the processing power avail-able to customer VMs, and increases the overall cost ofproviding Cloud Root I/O Virtualization (SR-IOV) [4, 5] has beenproposed to reduce CPU utilization by allowing direct ac-cess to NIC hardware from the VM. However, this di-rect access would bypass the host SDN stack, makingthe NIC responsible for implementing all SDN these policies change rapidly (weeks to months), werequired a solution that could provide software-like pro-grammability while providing hardware-like this paper we present Azure Accelerated Network-ing (AccelNet), our host SDN stack implemented on theFPGA-based Azure SmartNIC.

7 AccelNet provides near-native network performance in a virtualized environment,offloading packet processing from the host CPU to theAzure SmartNIC. Building upon the software-based VFPhost SDN platform [6], and the hardware and software in-frastructure of the Catapult program [7, 8], AccelNet pro-vides the performance of dedicated hardware, with theprogrammability of software running in the goal is to present both our design and our experiencesrunning AccelNet in production at scale, and lessons Traditional Host Network ProcessingIn the traditional device sharing model of a virtualizedenvironment such as the Public Cloud , all network I/O toand from a physical device is exclusively performed in thehost software partition of the hypervisor.

8 Every packetFigure 1: An SR-IOV NIC with a PF and and received by a VM is processed by the VirtualSwitch (vSwitch) in the host networking stack. Receiv-ing packets typically involves the hypervisor copying eachpacket into a VM-visible buffer, simulating a soft inter-rupt to the VM, and then allowing the VM s OS stack tocontinue network processing. Sending packets is similar,but in the opposite order. Compared to a non-virtualizedenvironment, this additional host processing: reduces per-formance, requires additional changes in privilege level,lowers throughput, increases latency and latency variabil-ity, and increases host CPU Host SDNIn addition to selling VMs, Cloud vendors sellingInfrastructure-as-a-Service (IaaS) have to provide rich net-work semantics, such as private virtual networks with cus-tomer supplied address spaces, scalable L4 load balancers,security groups and ACLs, virtual routing tables, band-width metering, QoS, and more.

9 These semantics are suf-ficiently complex and change too frequently that it isn tfeasible to implement them at scale in traditional switchhardware. Instead, these are implemented on each host inthe vSwitch. This scales well with the number of servers,and allows the physical network to be simple, scalable andvery Virtual Filtering Platform (VFP) is our Cloud -scaleprogrammable vSwitch, providing scalable SDN policyfor Azure . It is designed to handle the programmabil-ity needs of Azure s many SDN applications, providinga platform for multiple SDN controllers to plumb com-plex, stateful policy via match-action tables.

10 Details aboutVFP and how it implements virtual networks in softwarein Azure can be found in [6]. SR-IOVMany performance bottlenecks caused by doing packetprocessing in the hypervisor can be overcome by usinghardware that supports SR-IOV. SR-IOV-compliant hard-ware provides a standards-based foundation for efficientlyand securely sharing PCI Express (PCIe) device hardwareamong multiple VMs. The host connects to a privilegedphysical function (PF), while each virtual machine con-nects to its own virtual function (VF).