Optics Technology Trends in Data Centers - OFC

1 Optics Technology Trends in DataCentersMarc A. TaubenblattIBMOFC Market Watch 2014 Outline/Summary data movement/aggregate BW increasing andincreasingly important Historically, per channel data rate increases underliegains in optical interconnect cost and power efficiency It s getting harder to keep doing that Proliferation of alternate Technology choices More Customized ( Technology choice, EOEoptimization, packaging) Market fragmentation, fight for volumes ( cost) Make the most of limited PHY resources higher up thestack ( SDN) or new networks (OCS?) data Movement Becoming Critical Compute density increasing (multi-core, VM s), BW need increases EW traffic increasing* (Distributed workloads, virtualization, big data , randomworkloads, graph analytics) 44% CAGR in DC traffic, 76% within DC(Cisco 2012*) 80%+ of Google traffic now internal facing(B.)

2 Koley, Google OI Conf. 2012) Every 1kb of external traffic entering the datacenter generates 930kb ofinternal traffic(N. Farrington, Facebook OI Conf. 2013) Flatter topologies for lower latency ( higher radix, longer links) Resiliency (redundancy, path diversity, hot ) Caveat: Not all datacenters have the same needs ( Mega- center vs Enterprise)..TORA ccessAggregationMORD ecreasing volumesFlatter networkInterconnect Volumes vs Network Hierarchy* : after , Nov 2013, IEEE 400G study groupWill networks dominate DC cost and power concerns? Growing concern on power and cost of network, but it s not the biggest piecetoday: Optics becoming most of network cost( *) Growing faster than compute and memory But DC efficiency is gated by the network Latency, bottlenecks, predictable latency So has been easy to exploit increased Optics /network cost for performanceefficiencies at the DC level Leverage of benefits have to level off when?

3 After , Microsoft, ACMS igcomm 2009after Barrosom, DataCenter as Computer, 2013after * , Google, IEEE Summer Topicals 2013)Amortized CostChallenges for Physical Layer Optical Interconnects: Cost and Power Historically, gains in $/Gbps and mW/Gbps driven by increased data rates: Aggregate data rates easier to manage with fewer channels 400GE advantageousat 8x50 Gbp PCIe Gen5 likely to be 32Gb/s, Gen6 could be 64Gb/s. 64G Fibre Channel expected to be OIF has defined 3 serial 56Gb/s interfaces for future optical interconnects But NRZ increasingly difficult at these data rates and beyond.

4 E packaging losses, driver and receiver performance, Link budget CMOS logic not getting any faster, increased power for mux/demuxData courtesy of Ken Jackson, prices at volume production110100100010000198019902000201 02020 FCEthernetAOCSNAP12110100100010000101000 100000 FCEthernetAOCSNAP12 Power(Ethernet)Power(AOC)10M100M1G10G100 G (10x10) Raw data Rate (Gbps)$/Gbps$/GbpsYearWhat are the Technology choices to stay on trend? MultiMode (MM) VCSELs Getting faster but distance limited SingleMode (SM) Si Photonics Expensive but getting cheaper with Si Ph, no distance issues,link budget and cost (packaging!)

5 More challenging SM VCSELs and MM Si Photonics Best of both worlds or worst of both worlds? WDM, PAM4, DMT .. Trades fiber cost for Trx cost & power ( NRZ 56G a few dBbetter link budget than PAM4) Fwd Error Correction (FEC) likely needed, adds some latency( ~100ns for 4-6dB gain*) and don t forget Copper 25G possible for a few meters, and it scheaper than Optics !* et al, Huawei IEEE 400GE study group, Nov 13 Competing Technology SpaceOFC 2014 : D. Kuchta et al 64Gb/s Transmission over 57m MMF using an NRZ Modulated850nm VCSEL (also >250m at 40G and > 100m at 60G)ChannelSpeed(Gbps)Distance (m)1101001101001000SM OpticsMM OpticsCopperBeware that incumbant technologies are not standing still ( VCSELs, Copper) data Rate & Power: A Packaging Challenge High data rates cause high electrical to optical link power tighter packagingintegration of opticsFirst-level packageLogic:mproc, memory, switch, edge of drawer module.

6 Easy to use, replace and cable but electrical link willbe costly/expensive at higher channel data rates ( need SerDes in module, )First-level packageLogic:mproc, memory, switch, Optics : Harder to cable and replace, but mitigates electrical link power and cost,opportunities for EOE optimizationopticalmodulejumperOn Package Optics : Difficult to design, cable and replace, but best opportunity for lowerpower and cost more amenable to HPC and highly custom needsFirst-level packageLogic:mproc, memory, switch, Staying on the cost and power trend is creating moretechnology choices and more possibilities for customization Proliferation of alternate Technology choices Volumes are key to lower costs, could too many choices could inhibitvolume?

7 Cause longer time for market to adopt? More opportunities for Technology niches with reasonable volumes? More Customized ( Technology Choice, EOE optimization,packaging) Different DC s will have different needs, cost tolerance and focus areas Advanced packaging favors more sophisticated users, harder to do thiswith one size fits all designs Does market fragmentation dominate OR open HW standards?