Fujitsu HPC and AI Processors

1 Connect. Challenge. Rights Reserved, Copyright Fujitsu LIMITED 2015 ISC 2017 June 20th2017 Fujitsu HPC and AI ProcessorsTakumi MaruyamaSenior DirectorAI Platform Business UnitAdvanced System Research & Development UnitAgenda K computer Fujitsu s latest Processors HPC UNIX Future Fujitsu Processors under development Post K AI processor : DLU Summary1 Copyright 2017 Fujitsu LIMITED1K Computer2 Copyright 2017 Fujitsu LIMITEDK ComputerWR# PFlops(Top500, 2011/11)38,621 GTEPS (Graph500, 2016/11) TFLOPS (HPCG, 2016/11)3 Copyright 2017 Fujitsu LIMITED3 High Performance Processor8coreLiquid Cooling4 ProcessorsTorus Technologies in the K computer864 racks82,944 Compute nodes5,184 IO nodesHigh Density Rack24boards4 Copyright 2017 Fujitsu LIMITEDThe Latest Fujitsu Processors5 Copyright 2017 Fujitsu LIMITEDF ujitsu processor DevelopmentPerpetual Evolution > 60 years.

2 Always Targeting 2003 SPARC64 SPARC64 IISPARC64 VSPARC64 GPGS8900GS21600GS8600GS8800 BSPARC64 VIIGS211600 SPARC64V+SPARC64 VIGS8800GS21900 MainframePerformanceReliabilityStore AheadBranch HistoryPrefetchSingle-chip CPUNon-Blocking $O-O-O ExecutionSuper-ScalarL2$ on DieHPC-ACE System on ChipHardware BarrierMulti-core Multi-thread2004 20072008 2011 SPARC64GP2012 2015 2016 SPARC64 IXfxSPARC64 VIIIfxVirtual Machine ArchitectureSoftware on Chip High-speed InterconnectSPARC64X+130nm250nm /220nm180nm:Technology generation90nm350nm28nmTr=1 BCMOS Cu40nm65nmHPCUNIX$ ECCR egister/ALU ParityInstruction Retry$ Dynamic DegradationError Checkers/HistoryMainframe/UNIX/HPC + AIincremental developmentGS21260045nm40nmNextGSSPARC64 XIfxSPARC64X20nmDLUSPARC64 XIIPost-KARMAI6 NextSPARCC opyright 2017 Fujitsu LIMITEDSPARC64 XIfxChip (HPC) Architecture Features 32 computing cores + 2 assistant cores HPC-ACE2 (256bit SIMD) Fujitsu s ISA enhancements Sector Cache.

3 Cache with SW controllability 24 MB L2 cache 20nm CMOS 3,750M transistors Performance (peak) HMC 240GB/s x 2 (in/out) Tofu2 125GB/s x 2 (in/out)corecorecorecorecorecorecorecore corecorecorecorecorecorecorecoreAssistan tcoreAssistantcorecorecorecorecorecoreco recorecorecorecorecorecorecorecorecoreco reTofu2 interfaceTofu2 controllerHMC interfaceHMC interfaceL2 cache L2 cache PCI interfaceMAC MAC MAC MAC PCI controller7 Copyright 2017 Fujitsu LIMITED7 Many (32+2)cores, Medium CPU GHzSPARC64 XII Chip (UNIX) Architecture Features 12 cores x 8 threads SWoC( Software on Chip ) Fujitsu s ISA enhancements 32MB L3 cache Embedded MAC and IOC 20nm CMOS x 5,450M transistors (up to with High Speed Mode enabled) Performance (peak)

4 417 GIPS / 835 GFlops 153GB/s memory throughputDDR4 interfaceDDR4 interfaceCoreCoreL2 CacheCoreCoreCoreCoreCoreCoreCoreCoreCor eCoreL3 CacheMACMACSERDESPCIeGen3 SERDESI nterconnectL3 CacheL3 CacheL3 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheInterconnect & Coherence ControlCopyright 2017 Fujitsu LIMITED8 Multiple big cores, High CPUGHzSPARC64 TMXIfx(HPC) PipelineFLBL1 I$64KB 4waysBranchTargetAddressDecode& IssueRSERSARSFRSBRGUBGPR188 RegistersEXAEXBEAGAEXCEAGBEXDFPR128x4 PortL1 D $64KB4 WayMACF etchIssueDispatchReg-ReadExecuteCache and MemoryCSEC ommitPCControlRegistersL2$Write BufferPatternHistoryTableIOC CPU-CPU I/F34 2017 Fujitsu LIMITED9L1 InstructionCache64 KBRSER eservation Stationfor ExecutionRSAR eservation Stationfor Address generationRSFR eservation Stationfor Floating-pointRSBRR eservation Stationfor BranchGUBEXAEXBEAGAEAGBFUBFPR Update BufferFLAFLBF

5 EtchPortStorePortL1 DataCache32 KBFetchDecodeIssueReg-ReadExecuteCache and MemoryCommitStackEntryCommitFLCFLDS toreBuffer12 coresPipeline-0 Pipeline-1 MACL3 CacheIOCCPU-CPU i/fL2 CachedTLBSPARC64 TMXII(UNIX) PipelineBranchPredictionGPRx4 FPRx4 ProgramCounter x4 ControlRegisters x4 DecodeInstructionBufferShared Micro-architectureCopyright 2017 Fujitsu LIMITED10 Future Fujitsu Processors Under Development-Post K11 Copyright 2017 Fujitsu LIMITED Project Overview RIKEN and Fujitsu are currently developing the post-K computer, which is aims to be the most advanced general-purposesupercomputer in the world Goals of Japan s Post-K Development Project Application performance Low power consumption User convenience Ability to produce ground-breaking resultsCopyright 2017 Fujitsu LIMITEDJ apan s Post-K Computer Development Project12 Functions& ArchitecturePost-KK computerProcessorBaseISA + SIMD ExtensionsARMv8-A+SVESPARCv9+HPC-ACESIMD width [bit]512128FP16 (half precision) support -FMA.

6 Floating-pointmultiplyand add primitives Enhanced Inter-core barrier Sector cache Enhanced Hardware prefetch assist Enhanced InterconnectTofu Enhanced Post-K processor and Interconnect Features Fujitsu processor , adopting ARM ISA and enhanced Tofu interconnect Inherits and enhances the K computer s innovative featuresCopyright 2017 Fujitsu LIMITED13 Post-K processor Supports FP16 Provides optimized precision for a wide range of applications Superior performance Reduces required bandwidth and power consumption Target applications.

7 Existing numerical applications Brand-new applications, including Deep LearningHigh PerformanceforMore ApplicationsDouble PrecisionSingle PrecisionHalf PrecisionCopyright 2017 Fujitsu LIMITED14 Future Fujitsu processor Development-AI processor (DLUTM)15 All Rights Reserved, Copyright 2017 Fujitsu LIMITEDP rocessor Designed for Deep LearningFeatures of DLU Architecture designed for Deep Learning Low power consumption design Optimized precision Goal: 10x Performance / Watt compared to competitors Scalable design with Tofu interconnect technology Ability to handle large-scale neural networks The photograph is an image, and it is different from the thing.

8 DLU(Deep Learning Unit)FY2018 TMUtilizing technologies derived from the K computerCopyright 2017 Fujitsu LIMITED16 DLU Design TargetCopyright 2017 Fujitsu LIMITEDHighPerformanceLowPowerConflictin g Demands Less Transistors less control logic fewerexecution units/$ Lower Frequency More transistors state of the art O-O-O many execution units/$ Higher Frequency High Deep Learning performance / watt:10x performance / watt However, high performance and low power is not easy to achieve at the same time 17 Need for a New Architecture A new architecture is required for the DLU to achieve the target.

9 The architecture is domain specific Deep LearningGeneralPurposeComputingBrainComp utingSupercomputerAcceleratorQuantumComp uterDeepLearningInferenceSpecializationR equired ProcessingCopyright 2017 Fujitsu LIMITED18 What s the New Architecture for the DLU?High PrecisionGeneral UseConventional ArchitectureThe New Architecture2. Optimal Precision1. Domain SpecificSequential+ Parallel3. Massively ParallelMany cores w/ on-chip networkMultiple strong coresDouble/Single precision FPDeep Learning IntegerComplicated O-O-O coresDomain specific cores Domain specific, Optimal precision, and Massively 2017 Fujitsu LIMITED19 HBM2 DPU: Deep learning Processing Unit, DPE: Deep learning Processing ElementHost I/FDPU-0 DPU-1 DPUDPUDPUDPU-nDPEDPEDPEDPEDPEDPEDPEDPEDP EDPEDPEDPEDPEDPEDPEL arge scale DLU interconnect through off-chip networkDPEDPEDPEDLUTM(Deep LearningUnit)DLUA rchitectureInter-chipI/F1.

10 Domain specificDomain specific Cores-Newly designed ISA-Simplified -architecture-Fullysoftware visible and controllable-Heterogeneous cores -DPE and Large RF 3. Massively ParallelMany DPUs with an On-chipNetwork2. Optimal PrecisionDeep Learning Integer Copyright 2017 Fujitsu LIMITED20 DPU: Execution Execute DL operations based on master core s controlHow to utilize many DPUs(convolution example) one CH-out / DPU multiple batch / DPU Heterogeneous CoresDPUDPUDPUDPU DPUDPUDPUDPUM asterMemoryMemoryControllerInstructions/ Data Core:Memory Access andDPU control Push & Pull instructions and data for DLUs.