Vitis AI User Guide - Xilinx

1 Vitis AI User GuideUG1414 ( ) February 3, 2021 Revision HistoryThe following table shows the revision history for this Summary02/03/2021 Version documentUpdated links12/17/2020 Version documentMinor changesDeep-Learning Processor UnitAdded new topics: Alveo U200/U250: DPUCADF8H, AlveoU50/U50LV/U280: DPUCAHX8L, and Versal AI Core Version (vai_q_tensorflow2)Added new sectionPyTorch Version (vai_q_pytorch)Added new topics: Module Partial Quantization, vai_q_pytorch Fast Finetuning, and vai_q_pytorch 5: Compiling the ModelAdded new section: Compiling with an XIR-based 10: Integrating the DPU into Custom PlatformsAdded new A: Vitis AI Programming InterfaceAdded new section: VART Version documentMinor changes07/07/2020 Version document Added Vitis AI Profiler topic.

2 Added Vitis AI unified API NamingAdded new topicChapter 2: Getting StartedUpdated the chapter03/23/2020 Version new topicEntire documentAdded contents for Alveo U50 support, U50 DPUV3enablement, including compiler usage and modeldeployment HistoryUG1414 ( ) February 3, 2021 AI User Guide 2 Send FeedbackTable of ContentsRevision 1: Vitis AI Content by Design AI Tools AI System Flow 2: Getting Started ..25 Installation and 3: Understanding the Vitis AI Model Zoo 41 Chapter 4: Quantizing the AI Quantizer Version (vai_q_tensorflow).

3 46 tensorflow Version (vai_q_tensorflow2)..58 PyTorch Version (vai_q_pytorch)..65 Caffe Version (vai_q_caffe)..76 Chapter 5: Compiling the 82 Vitis AI with an XIR-based with 6: Deploying and Running the and Running Models on Alveo U200 ( ) February 3, 2021 AI User Guide 3 Send FeedbackProgramming with Debug with TVM and Microsoft ONNX 7: Profiling the AI 8: Optimizing the 127 Chapter 9: Accelerating Subgraph with ML Functional API Call in 128 Partitioning Support in 10: Integrating the DPU into Custom A: Vitis AI Programming 133 VART 133 Appendix B: Legacy N2 Cube Examples.

4 149 DNNDK Programming for 153 DNNDK 159 Profiling Using the DNNDK Programming 170 Appendix C: Additional Resources and Legal 229 Xilinx Navigator and Design Read: Important Legal 230UG1414 ( ) February 3, 2021 AI User Guide 4 Send FeedbackChapter 1 Vitis AI OverviewThe Vitis AI development environment accelerates AI inference on Xilinx hardware platforms,including both Edge devices and Alveo accelerator cards. It consists of optimized IP cores, tools,libraries, models, and example designs. It is designed with high efficiency and ease of use in mindto unleash the full potential of AI acceleration on Xilinx FPGAs and on adaptive computeacceleration platforms (ACAPs).

5 It makes it easier for users without FPGA knowledge to developdeep-learning inference applications, by abstracting the intricacies of the underlying FPGA 1: Vitis AI StackModel ZooCustom ModelsAI Compiler | AI Quantizer | AI OptimizerAI Profiler | AI LibraryXilinx Runtime library (XRT)Deep Learning Processing Unit (DPU) Vitis AI ModelsFrameworksVitis AI Development KitOverlayUser ApplicationX24893-120920 Chapter 1: Vitis AI OverviewUG1414 ( ) February 3, 2021 AI User Guide 5 Send FeedbackNavigating Content by Design ProcessXilinx documentation is organized around a set of standard design processes to help you findrelevant content for your current development task.

6 This document covers the following designprocesses: Machine Learning and Data Science: Importing a machine learning model from a Caffe,Pytorch, tensorflow , or other popular framework onto Vitis AI, and then optimizing andevaluating its effectiveness. Topics in this document that apply to this design process include: Chapter 2: Getting Started Chapter 4: Quantizing the Model Chapter 5: Compiling the Model System and Solution Planning: Identifying the components, performance, I/O, and datatransfer requirements at a system level.

7 Includes application mapping for the solution to PS,PL, and AI Engine. Topics in this document that apply to this design process include: Chapter 3: Understanding the Vitis AI Model Zoo Networks Embedded Software Development: Creating the software platform from the hardwareplatform and developing the application code using the embedded CPU. Also covers XRT andGraph APIs. Topics in this document that apply to this design process include: Chapter 10: Integrating the DPU into Custom Platforms Host Software Development: Developing the application code, accelerator development,including library, XRT, and Graph API use.

8 Topics in this document that apply to this designprocess include: Chapter 6: Deploying and Running the Model Chapter 9: Accelerating Subgraph with ML Frameworks Hardware, IP, and Platform Development: Creating the PL IP blocks for the hardwareplatform, creating PL kernels, subsystem functional simulation, and evaluating the Vivado timing, resource use, and power closure. Also involves developing the hardware platform forsystem integration. Topics in this document that apply to this design process include: Chapter 10: Integrating the DPU into Custom Platforms System Integration and Validation: Integrating and validating the system functionalperformance, including timing, resource use, and power closure.

9 Topics in this document thatapply to this design process include: Chapter 7: Profiling the ModelChapter 1: Vitis AI OverviewUG1414 ( ) February 3, 2021 AI User Guide 6 Send FeedbackFeaturesVitis AI includes the following features: Supports mainstream frameworks and the latest models capable of diverse deep learningtasks. Provides a comprehensive set of pre-optimized models that are ready to deploy on Xilinxdevices. Provides a powerful quantizer that supports model quantization, calibration, and fine advanced users, Xilinx also offers an optional AI optimizer that can prune a model by up to90%.

10 The AI profiler provides layer by layer analysis to help with bottlenecks. The AI library offers unified high-level C++ and Python APIs for maximum portability fromEdge to Cloud. Customizes efficient and scalable IP cores to meet your needs for many different applicationsfrom a throughput, latency, and power AI Tools OverviewDeep-Learning Processor UnitThe Deep-Learning Processor Unit (DPU) is a programmable engine optimized for deep neuralnetworks. It is a group of parameterizable IP cores pre-implemented on the hardware with noplace and route required.