NVIDIA A2 TENSOR CORE GPU

1 Versatile Entry-Level InferenceThe NVIDIA A2 TENSOR Core GPU provides entry-level inference with low power, a small footprint, and high performance for NVIDIA AI at the edge. Featuring a low-profile PCIe Gen4 card and a low 40-60 watt (W) configurable thermal design power (TDP) capability, the A2 brings adaptable inference acceleration to any server. A2's versatility, compact size, and low power exceed the demands for edge deployments at scale, instantly upgrading existing entry-level CPU servers to handle inference. Servers accelerated with A2 GPUs deliver higher inference performance versus CPUs and more efficient intelligent video analytics (IVA) deployments than previous GPU generations all at an entry-level price point. NVIDIA -Certified Systems featuring A2 GPUs and NVIDIA AI, including the NVIDIA Triton Inference Server, deliver breakthrough inference performance across edge, data center, and cloud.

2 They ensure that AI-enabled applications deploy with fewer servers and less power, resulting in easier deployments, faster insights, and significantly lower costs. Up to 20X More Inference PerformanceAI inference is deployed to make consumer lives more convenient through real-time experiences, and enables them to gain insights on trillions of end-point sensors and cameras. Compared to CPU-only servers, the servers built with NVIDIA A2 TENSOR Core GPU offer up to 20X more inference performance, instantly upgrading any server to handle modern AI. DATASHEETNVIDIA A2 TENSOR CORE GPUE ntry-level GPU that brings NVIDIA AI to any A2 TENSOR CORE GPU | DATASHEET | 1 SYSTEM SPECIFICATIONS Peak TFTF32 TENSOR Core9 TF | 18 TF BFLOAT16 TENSOR Core18 TF | 36 TF Peak FP16 TENSOR Core18 TF | 36 TF Peak INT8 TENSOR Core36 TOPS | 72 TOPS Peak INT4 TENSOR Core72 TOPS | 144 TOPS RT Cores10 Media engines1 video encoder2 video decoders (includes AV1 decode)GPU memory16GB GDDR6 GPU memory bandwidth200GB/sInterconnectPCIe Gen4 x8 Form factor1-slot, Low-Profile PCIeMax thermal design power (TDP)40-60W (Configurable)vGPU software support NVIDIA virtual PC (vPC), NVIDIA virtual Applications (vApps), NVIDIA RTX virtual workstation (vWS), NVIDIA AI Enterprise, NVIDIA virtual Compute Server (vCS)

3 With sparsity Supported in future vGPU releaseSystem on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | omputer V s on Ef c entDet-D0 ( O O, 512x512) | TensorRT 8 2, Prec s on INT8, BS 8 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 8 ( PU)6X10X8X1X8X2X4 XComputer Vision (EfficientDet-DO)System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | NLP BERT-Large (Sequence length 384, SQuAD v1 1) | TensorRT 8 2, Prec s on INT8, BS 1 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 1 ( PU)8 XNatural Language Processing (BERT-Large)System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | Text-to-Speech Tacotron2 + Waveglow end-to-end p pel ne ( nput length 128) | PyTorch 1 9, Prec s on FP16, BS 1 ( PU) | PyTorch 1 9, Prec s on FP32, BS 1 ( PU)15X20X25X20X1X5X10 XText-to-Speech (Tacotron2 + Waveglow)MobileNet Performance (Video Streams 1080p30)1 0X1 2X1 0X1 3 XNVIDIA T4 ShuffleNet v2 NVIDIA A2 SystemConfiguration: [Supermicro SYS-1029GQ-TRT, 2S Xeon Gold 6240 , 512GB DDR4, 1x NVIDIA A2 OR 1x NVIDIA T4] | Measured performance with Deepstream Networks.

4 ShuffleNet-v2 (224x224), MobileNet-v2 (224x224) | Pipeline represents end-to-end performance with video capture and decode, pre-processing, batching, inference, and Improves Performance by Up to Versus T4 IVA Performance (Normalized) NVIDIA A240657075 TDP Operat ng Range (Watts)A2 Reduces Power Consumption by Up to 40% Versus T4 Lower Power and Configurable TDP55604550 NVIDIA T46X7X1X2X4 XInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUSystem on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | omputer V s on Ef c entDet-D0 ( O O, 512x512) | TensorRT 8 2, Prec s on INT8, BS 8 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 8 ( PU)6X10X8X1X8X2X4 XComputer Vision (EfficientDet-DO)

5 System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | NLP BERT-Large (Sequence length 384, SQuAD v1 1) | TensorRT 8 2, Prec s on INT8, BS 1 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 1 ( PU)8 XNatural Language Processing (BERT-Large)System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | Text-to-Speech Tacotron2 + Waveglow end-to-end p pel ne ( nput length 128) | PyTorch 1 9, Prec s on FP16, BS 1 ( PU) | PyTorch 1 9, Prec s on FP32, BS 1 ( PU)15X20X25X20X1X5X10 XText-to-Speech (Tacotron2 + Waveglow)MobileNet Performance (Video Streams 1080p30)1 0X1 2X1 0X1 3 XNVIDIA T4 ShuffleNet v2 NVIDIA A2 SystemConfiguration: [Supermicro SYS-1029GQ-TRT, 2S Xeon Gold 6240 , 512GB DDR4, 1x NVIDIA A2 OR 1x NVIDIA T4] | Measured performance with Deepstream Networks.

6 ShuffleNet-v2 (224x224), MobileNet-v2 (224x224) | Pipeline represents end-to-end performance with video capture and decode, pre-processing, batching, inference, and Improves Performance by Up to Versus T4 IVA Performance (Normalized) NVIDIA A240657075 TDP Operat ng Range (Watts)A2 Reduces Power Consumption by Up to 40% Versus T4 Lower Power and Configurable TDP55604550 NVIDIA T46X7X1X2X4 XInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUSystem on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | omputer V s on Ef c entDet-D0 ( O O, 512x512) | TensorRT 8 2, Prec s on INT8, BS 8 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 8 ( PU)6X10X8X1X8X2X4 XComputer Vision (EfficientDet-DO)

7 System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | NLP BERT-Large (Sequence length 384, SQuAD v1 1) | TensorRT 8 2, Prec s on INT8, BS 1 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 1 ( PU)8 XNatural Language Processing (BERT-Large)System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | Text-to-Speech Tacotron2 + Waveglow end-to-end p pel ne ( nput length 128) | PyTorch 1 9, Prec s on FP16, BS 1 ( PU) | PyTorch 1 9, Prec s on FP32, BS 1 ( PU)15X20X25X20X1X5X10 XText-to-Speech (Tacotron2 + Waveglow)MobileNet Performance (Video Streams 1080p30)1 0X1 2X1 0X1 3 XNVIDIA T4 ShuffleNet v2 NVIDIA A2 SystemConfiguration: [Supermicro SYS-1029GQ-TRT, 2S Xeon Gold 6240 , 512GB DDR4, 1x NVIDIA A2 OR 1x NVIDIA T4] | Measured performance with Deepstream Networks.

8 ShuffleNet-v2 (224x224), MobileNet-v2 (224x224) | Pipeline represents end-to-end performance with video capture and decode, pre-processing, batching, inference, and Improves Performance by Up to Versus T4 IVA Performance (Normalized) NVIDIA A240657075 TDP Operat ng Range (Watts)A2 Reduces Power Consumption by Up to 40% Versus T4 Lower Power and Configurable TDP55604550 NVIDIA T46X7X1X2X4 XInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUInference Speedup ompar sons of one NVIDIA A2 TENSOR ore PU versus a dual-socket Xeon old 6330N PU0 XNVIDIA A2 PUNVIDIA A2 TENSOR CORE GPU | DATASHEET | 2 Higher IVA Performance for Intelligent EdgeServers equipped with A2 offer up to more performance in intelligent edge use cases, including smart cities, manufacturing, and retail.

9 NVIDIA A2 GPUs running IVA workloads result in more efficient deployments with up to better price-performance and ten percent better energy efficiency than previous GPU A2 Brings Breakthrough NVIDIA Ampere Architecture InnovationsTHIRD-GENERATION TENSOR CORESThe third-generation TENSOR Cores in A2 support integer math, down to INT4, and floating point math, up to FP32, to deliver high AI training and inference performance. The NVIDIA Ampere architecture also supports TF32 and NVIDIA s automatic mixed precision (AMP) OF TRUST SECURITYP roviding security in edge deployments and end-points is critical for enterprise business operations. A2 optionally supports secure boot through trusted code authentication and hardened rollback protections to protect against malicious malware RT CORESA2 includes dedicated RT Cores for ray tracing that enable groundbreaking technologies at breakthrough speed.

10 With up to 2X the throughput over the previous generation and the ability to concurrently run ray tracing with either shading or denoising TRANSCODING PERFORMANCEE xponential growth in video applications demand real-time scalable performance, requiring the latest in hardware encode and decode capabilities. A2 GPUs use dedicated hardware to fully accelerate video decoding and encoding for the most popular codecs, including , , VP9, and AV1 on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | omputer V s on Ef c entDet-D0 ( O O, 512x512) | TensorRT 8 2, Prec s on INT8, BS 8 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 8 ( PU)6X10X8X1X8X2X4 XComputer Vision (EfficientDet-DO)System on gurat on PU HPE DL380 en10 Plus, 2S Xeon old 6330N 2 2 Hz, 512 B DDR4 | NLP BERT-Large (Sequence length 384, SQuAD v1 1) | TensorRT 8 2, Prec s on INT8, BS 1 ( PU) | OpenVINO 2021 4, Prec s on INT8, BS 1 ( PU)8 XNatural Language Processing (BERT-Large)