Fujitsu HPC and AI Processors

1 Connect. Challenge. Rights Reserved, Copyright Fujitsu LIMITED 2015 ISC 2017 June 20th2017 Fujitsu HPC and AI ProcessorsTakumi MaruyamaSenior DirectorAI Platform Business UnitAdvanced System Research & Development UnitAgenda K computer Fujitsu s latest Processors HPC UNIX Future Fujitsu Processors under development Post K AI processor : DLU Summary1 Copyright 2017 Fujitsu LIMITED1K Computer2 Copyright 2017 Fujitsu LIMITEDK ComputerWR# PFlops(Top500, 2011/11)38,621 GTEPS (Graph500, 2016/11) TFLOPS (HPCG, 2016/11)3 Copyright 2017 Fujitsu LIMITED3 High Performance Processor8coreLiquid Cooling4 ProcessorsTorus Technologies in the K computer864 racks82,944 Compute nodes5,184 IO nodesHigh Density Rack24boards4 Copyright 2017 Fujitsu LIMITEDThe Latest Fujitsu Processors5 Copyright 2017 Fujitsu LIMITEDF ujitsu processor DevelopmentPerpetual Evolution > 60 years.

2 Always Targeting 2003 SPARC64 SPARC64 IISPARC64 VSPARC64 GPGS8900GS21600GS8600GS8800 BSPARC64 VIIGS211600 SPARC64V+SPARC64 VIGS8800GS21900 MainframePerformanceReliabilityStore AheadBranch HistoryPrefetchSingle-chip CPUNon-Blocking $O-O-O ExecutionSuper-ScalarL2$ on DieHPC-ACE System on ChipHardware BarrierMulti-core Multi-thread2004 20072008 2011 SPARC64GP2012 2015 2016 SPARC64 IXfxSPARC64 VIIIfxVirtual Machine ArchitectureSoftware on Chip High-speed InterconnectSPARC64X+130nm250nm /220nm180nm:Technology generation90nm350nm28nmTr=1 BCMOS Cu40nm65nmHPCUNIX$ ECCR egister/ALU ParityInstruction Retry$ Dynamic DegradationError Checkers/HistoryMainframe/UNIX/HPC + AIincremental developmentGS21260045nm40nmNextGSSPARC64 XIfxSPARC64X20nmDLUSPARC64 XIIPost-KARMAI6 NextSPARCC opyright 2017 Fujitsu LIMITEDSPARC64 XIfxChip (HPC) Architecture Features 32 computing cores + 2 assistant cores HPC-ACE2 (256bit SIMD) Fujitsu s ISA enhancements Sector Cache.

3 Cache with SW controllability 24 MB L2 cache 20nm CMOS 3,750M transistors Performance (peak) HMC 240GB/s x 2 (in/out) Tofu2 125GB/s x 2 (in/out)corecorecorecorecorecorecorecore corecorecorecorecorecorecorecoreAssistan tcoreAssistantcorecorecorecorecorecoreco recorecorecorecorecorecorecorecorecoreco reTofu2 interfaceTofu2 controllerHMC interfaceHMC interfaceL2 cache L2 cache PCI interfaceMAC MAC MAC MAC PCI controller7 Copyright 2017 Fujitsu LIMITED7 Many (32+2)cores, Medium CPU GHzSPARC64 XII Chip (UNIX) Architecture Features 12 cores x 8 threads SWoC( Software on Chip ) Fujitsu s ISA enhancements 32MB L3 cache Embedded MAC and IOC 20nm CMOS x 5,450M transistors (up to with High Speed Mode enabled) Performance (peak) 417 GIPS / 835 GFlops 153GB/s memory throughputDDR4 interfaceDDR4 interfaceCoreCoreL2 CacheCoreCoreCoreCoreCoreCoreCoreCoreCor eCoreL3 CacheMACMACSERDESPCIeGen3 SERDESI nterconnectL3 CacheL3 CacheL3 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheL2 CacheInterconnect & Coherence ControlCopyright 2017 Fujitsu LIMITED8 Multiple big cores, High CPUGHzSPARC64 TMXIfx(HPC)

4 PipelineFLBL1 I$64KB 4waysBranchTargetAddressDecode& IssueRSERSARSFRSBRGUBGPR188 RegistersEXAEXBEAGAEXCEAGBEXDFPR128x4 PortL1 D $64KB4 WayMACF etchIssueDispatchReg-ReadExecuteCache and MemoryCSEC ommitPCControlRegistersL2$Write BufferPatternHistoryTableIOC CPU-CPU I/F34 2017 Fujitsu LIMITED9L1 InstructionCache64 KBRSER eservation Stationfor ExecutionRSAR eservation Stationfor Address generationRSFR eservation Stationfor Floating-pointRSBRR eservation Stationfor BranchGUBEXAEXBEAGAEAGBFUBFPR Update BufferFLAFLBF etchPortStorePortL1 DataCache32 KBFetchDecodeIssueReg-ReadExecuteCache and MemoryCommitStackEntryCommitFLCFLDS toreBuffer12 coresPipeline-0 Pipeline-1 MACL3 CacheIOCCPU-CPU i/fL2 CachedTLBSPARC64 TMXII(UNIX) PipelineBranchPredictionGPRx4 FPRx4 ProgramCounter x4 ControlRegisters x4 DecodeInstructionBufferShared Micro-architectureCopyright 2017 Fujitsu LIMITED10 Future Fujitsu Processors Under Development-Post K11 Copyright 2017 Fujitsu LIMITED Project Overview RIKEN and Fujitsu are currently developing the post-K computer, which is aims to be the most advanced general-purposesupercomputer in the world Goals of Japan s Post-K Development Project Application performance Low power consumption User convenience Ability to produce ground-breaking resultsCopyright 2017 Fujitsu LIMITEDJ apan s Post-K Computer Development Project12 Functions& ArchitecturePost-KK computerProcessorBaseISA + SIMD ExtensionsARMv8-A+SVESPARCv9+HPC-ACESIMD width [bit]

5 512128FP16 (half precision) support -FMA: Floating-pointmultiplyand add primitives Enhanced Inter-core barrier Sector cache Enhanced Hardware prefetch assist Enhanced InterconnectTofu Enhanced Post-K processor and Interconnect Features Fujitsu processor , adopting ARM ISA and enhanced Tofu interconnect Inherits and enhances the K computer s innovative featuresCopyright 2017 Fujitsu LIMITED13 Post-K processor Supports FP16 Provides optimized precision for a wide range of applications Superior performance Reduces required bandwidth and power consumption Target applications: Existing numerical applications Brand-new applications, including Deep LearningHigh PerformanceforMore ApplicationsDouble PrecisionSingle PrecisionHalf PrecisionCopyright 2017 Fujitsu LIMITED14 Future Fujitsu processor Development-AI processor (DLUTM)15 All Rights Reserved, Copyright 2017 Fujitsu LIMITEDP rocessor Designed for Deep LearningFeatures of DLU Architecture designed for Deep Learning Low power consumption design Optimized precision Goal: 10x Performance / Watt compared to competitors Scalable design with Tofu interconnect technology Ability to handle large-scale neural networks The photograph is an image, and it is different from the thing.

6 DLU(Deep Learning Unit)FY2018 TMUtilizing technologies derived from the K computerCopyright 2017 Fujitsu LIMITED16 DLU Design TargetCopyright 2017 Fujitsu LIMITEDHighPerformanceLowPowerConflictin g Demands Less Transistors less control logic fewerexecution units/$ Lower Frequency More transistors state of the art O-O-O many execution units/$ Higher Frequency High Deep Learning performance / watt:10x performance / watt However, high performance and low power is not easy to achieve at the same time 17 Need for a New Architecture A new architecture is required for the DLU to achieve the target. The architecture is domain specific Deep LearningGeneralPurposeComputingBrainComp utingSupercomputerAcceleratorQuantumComp uterDeepLearningInferenceSpecializationR equired ProcessingCopyright 2017 Fujitsu LIMITED18 What s the New Architecture for the DLU?

7 High PrecisionGeneral UseConventional ArchitectureThe New Architecture2. Optimal Precision1. Domain SpecificSequential+ Parallel3. Massively ParallelMany cores w/ on-chip networkMultiple strong coresDouble/Single precision FPDeep Learning IntegerComplicated O-O-O coresDomain specific cores Domain specific, Optimal precision, and Massively 2017 Fujitsu LIMITED19 HBM2 DPU: Deep learning Processing Unit, DPE: Deep learning Processing ElementHost I/FDPU-0 DPU-1 DPUDPUDPUDPU-nDPEDPEDPEDPEDPEDPEDPEDPEDP EDPEDPEDPEDPEDPEDPEL arge scale DLU interconnect through off-chip networkDPEDPEDPEDLUTM(Deep LearningUnit)DLUA rchitectureInter-chipI/F1. Domain specificDomain specific Cores-Newly designed ISA-Simplified -architecture-Fullysoftware visible and controllable-Heterogeneous cores -DPE and Large RF 3.

8 Massively ParallelMany DPUs with an On-chipNetwork2. Optimal PrecisionDeep Learning Integer Copyright 2017 Fujitsu LIMITED20 DPU: Execution Execute DL operations based on master core s controlHow to utilize many DPUs(convolution example) one CH-out / DPU multiple batch / DPU Heterogeneous CoresDPUDPUDPUDPU DPUDPUDPUDPUM asterMemoryMemoryControllerInstructions/ Data Core:Memory Access andDPU control Push & Pull instructions and data for DLUs. Start/stop execution of DLUs The combination of few large core (Master) and many small execution cores (DPU) results in more performance with less power consumption, compared to a conventional homogeneous structureCopyright 2017 Fujitsu LIMITED21 DPE & Large RF (Register File)DPUCNTLDPU: 128 SIMD* / 16 DPE DPU consists of 16 DPEs connected with on-chip network DPE incudes large RF and wide SIMD execution units to realize an efficient Deep Learning engine.

9 RF is fully SW controllable unlike cache to extract full HW potentialDPE: 8 SIMD* with large RF(~100x of typical CPU core)ExecUNITExecUNITRFExecUNITRFExecUNI TRFExecUNITRFExecUNITRFExecUNITRFExecUNI TRF* For FP32 Copyright 2017 Fujitsu LIMITED22 Register FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileRegister FileNameRF/$ structureUNIXSPARC64 XIIRF+ $HPCSPARC64 XIfxRF + sector$AIDLUL argeRFMore SW controllabilityDeep Learning Integer Fujitsu s Deep Learning Integer realizes necessary accuracy for Deep Learning with only a 16 or 8 bit data size ( less power consumption compared with FP32)

10 Copyright 2017 Fujitsu LIMITEDData SizeEffective PrecisionFP3216-bitINT8-bitINTR equired Precision for Deep Learning>INT8,16 AccumulatorINT16 INT8 INT8 INT16 INT8 INT8FP16FP16FP16FP32int16int16int16int16 Int>16+ Int>16+int8int8int8int8int8int8int8int8 +Int>8+Int>8+Int>8+Int>8 Deep Learning IntegerData Size and PrecisionDLU Data TypeSmall Large16/8bit area with minimum accuracy lossINT16 INT8 INT8 INT16 INT8 INT8HW gatheredstatistics23 Deep Learning Integer AccuracyCopyright 2017 Fujitsu LIMITED(*) ImageNet(subset): image size=96x96, #categories=25FP32 Deep Learning Integer (16bit/8bit) Deep Learning Integer has shown similar accuracy with FP32for Deep LearningINT8 INT8 Deep Learning Integer (8bit)FP32 Deep Learning Integer (16bit)24 DLU Roadmap Multiple generations of DLUs over time, as we currently do for HPC/UNIX/Mainframe Processors Host CPU required Inter-DLU direct connection1stGeneration Embedded host CPU2ndGenerationOther special Processors Neuromorphic Combinatorial optimizationFutureFY2018* Subject to change without noticeCopyright 2017 Fujitsu LIMITED25 Summary26 Copyright 2017 Fujitsu LIMITEDF ujitsu processor Design Style MainframeUNIXHPCAI(DLU)Instruction Set Architecture(HW-SW I/F)Micro-architecture(CPU internal structure)