Number of parameters (M)

1 YOLOX: Exceeding YOLO Series in 2021. Zheng Ge Songtao Liu Feng Wang Zeming Li Jian Sun Megvii Technology {gezheng, liusongtao, wangfeng02, lizeming, 51 41. 50. YOLOX-S. [ ] 6 Aug 2021. 39. 49. 37. 48 EfficientDet-Lite3. 35. 47. COCO AP (%). COCO AP (%). 46 33 EfficientDet-Lite2. YOLOX-Tiny 45 31. YOLOX-L. 44 29 EfficientDet-Lite1. 43 YOLOv5-L. 27. 42 YOLOX-DarkNet53 YOLOX-Nano 25 EfficientDet-Lite0. 41 YOLOv5-Darknet53. 40 23. EfficientDet NanoDet PPYOLO-Tiny YOLOv4-Tiny 39 21. 5 8 11 14 17 20 23 26 29 32 35 38 41 44 V100 batch 1 Latency (ms) Number of parameters (M). Figure 1: Speed-accuracy trade-off of accurate models (top) and Size-accuracy curve of lite models on mobile devices (bottom) for YOLOX and other state-of-the-art object detectors. researchers in practical scenes, and we also provide de- Abstract ploy versions with ONNX, TensorRT, NCNN, and Openvino supported.}

2 Source code is at In this report, we present some experienced improve- Megvii-BaseDetection/YOLOX. ments to YOLO series, forming a new high-performance detector YOLOX. We switch the YOLO detector to an anchor-free manner and conduct other advanced detection 1. Introduction techniques, , a decoupled head and the leading label With the development of object detection, YOLO se- assignment strategy SimOTA to achieve state-of-the-art re- ries [23, 24, 25, 1, 7] always pursuit the optimal speed and sults across a large scale range of models: For YOLO- accuracy trade-off for real-time applications. They extract Nano with only parameters and FLOPs, we the most advanced detection technologies available at the get AP on COCO, surpassing NanoDet by AP;. time ( , anchors [26] for YOLOv2 [24], Residual Net [9]. for YOLOv3, one of the most widely used detectors in in- for YOLOv3 [25]) and optimize the implementation for best dustry, we boost it to AP on COCO, outperform- practice.

3 Currently, YOLOv5 [7] holds the best trade-off ing the current best practice by AP; for YOLOX-L. performance with AP on COCO at with roughly the same amount of parameters as YOLOv4- Nevertheless, over the past two years, the major ad- CSP, YOLOv5-L, we achieve AP on COCO at a vances in object detection academia have focused on speed of FPS on Tesla V100, exceeding YOLOv5-L. anchor-free detectors [29, 40, 14], advanced label assign- by AP. Further, we won the 1st Place on Streaming ment strategies [37, 36, 12, 41, 22, 4], and end-to-end Perception Challenge (Workshop on Autonomous Driving (NMS-free) detectors [2, 32, 39]. These have not been inte- at CVPR 2021) using a single YOLOX-L model. We hope grated into YOLO families yet, as YOLOv4 and YOLOv5. this report can provide useful experience for developers and 1 we choose the YOLOv5-L model at 640 640 resolution and test the * Equal contribution.

4 Model with FP16-precision and batch=1 on a V100 to align the settings of Corresponding author. YOLOv4 [1] and YOLOv4-CSP [30] for a fair comparison 1. are still anchor-based detectors with hand-crafted assigning Models Coupled Head Decoupled Head rules for training. Vanilla YOLO That's what brings us here, delivering those recent ad- End-to-end YOLO ( ) ( ). vancements to YOLO series with experienced optimiza- tion. Considering YOLOv4 and YOLOv5 may be a little Table 1: The effect of decoupled head for end-to-end YOLO. over-optimized for the anchor-based pipeline, we choose in terms of AP (%) on COCO. YOLOv3 [25] as our start point (we set YOLOv3-SPP as the default YOLOv3). Indeed, YOLOv3 is still one of the most widely used detectors in the industry due to the limited latency in this report are all measured with FP16-precision computation resources and the insufficient software support and batch=1 on a single Tesla V100.

5 In various practical applications. As shown in Fig. 1, with the experienced updates of the above techniques, we boost the YOLOv3 to YOLOv3 baseline Our baseline adopts the architec- AP (YOLOX-DarkNet53) on COCO with 640 640 res- ture of DarkNet53 backbone and an SPP layer, referred olution, surpassing the current best practice of YOLOv3 to YOLOv3-SPP in some papers [1, 7]. We slightly ( AP, ultralytics version2 ) by a large margin. More- change some training strategies compared to the orig- over, when switching to the advanced YOLOv5 architec- inal implementation [25], adding EMA weights updat- ture that adopts an advanced CSPNet [31] backbone and an ing, cosine lr schedule, IoU loss and IoU-aware branch. additional PAN [19] head, YOLOX-L achieves AP We use BCE Loss for training cls and obj branch, on COCO with 640 640 resolution, outperforming the and IoU Loss for training reg branch.

6 These gen- counterpart YOLOv5-L by AP. We also test our de- eral training tricks are orthogonal to the key improve- sign strategies on models of small size. YOLOX-Tiny and ment of YOLOX, we thus put them on the baseline. YOLOX-Nano (only parameters and FLOPs) Moreover, we only conduct RandomHorizontalFlip, outperform the corresponding counterparts YOLOv4-Tiny ColorJitter and multi-scale for data augmentation and and NanoDet3 by 10% AP and AP, respectively. discard the RandomResizedCrop strategy , because we We have released our code at https://github. found the RandomResizedCrop is kind of overlapped com/Megvii-BaseDetection/YOLOX, with ONNX, with the planned mosaic augmentation. With those en- TensorRT, NCNN and Openvino supported. One more thing hancements, our baseline achieves AP on COCO val, worth mentioning, we won the 1st Place on Streaming Per- as shown in Tab.

7 2. ception Challenge (Workshop on Autonomous Driving at CVPR 2021) using a single YOLOX-L model. Decoupled head In object detection, the conflict between classification and regression tasks is a well-known prob- 2. YOLOX lem [27, 34]. Thus the decoupled head for classification and localization is widely used in the most of one-stage and YOLOX-DarkNet53. two-stage detectors [16, 29, 35, 34]. However, as YOLO. We choose YOLOv3 [25] with Darknet53 as our base- series' backbones and feature pyramids ( , FPN [13], line. In the following part, we will walk through the whole PAN [20].) continuously evolving, their detection heads re- system designs in YOLOX step by step. main coupled as shown in Fig. 2. Our two analytical experiments indicate that the coupled detection head may harm the performance. 1). Replacing Implementation details Our training settings are mostly YOLO's head with a decoupled one greatly improves the consistent from the baseline to our final model.

8 We train converging speed as shown in Fig. 3. 2). The decoupled the models for a total of 300 epochs with 5 epochs warm- head is essential to the end-to-end version of YOLO (will up on COCO train2017 [17]. We use stochastic gradi- be described next). One can tell from Tab. 1, the end-to- ent descent (SGD) for training. We use a learning rate of end property decreases by AP with the coupled head, lr BatchSize/64 (linear scaling [8]), with a initial lr =. while the decreasing reduces to AP for a decoupled and the cosine lr schedule. The weight decay is head. We thus replace the YOLO detect head with a lite de- and the SGD momentum is The batch size is 128 by coupled head as in Fig. 2. Concretely, it contains a 1 1. default to typical 8-GPU devices. Other batch sizes in- conv layer to reduce the channel dimension, followed by clude single GPU training also work well.

9 The input size two parallel branches with two 3 3 conv layers respec- is evenly drawn from 448 to 832 with 32 strides. FPS and tively. We report the inference time with batch=1 on V100. 2 in Tab. 2 and the lite decoupled head brings additional 3 ms ( ms ms). 2. Feature YOLOv3~v5. #&'( *+ - Cls. 1 1 conv Coupled Head +. 3 3 conv ! # #&'( *+ 4 Reg. +. #&'( *+ 1 Obj. 1024. ! # 512. 256 YOLOX. Decoupled Head Cls. ! # C. 2. !5 ! # 256. FPN !4. feature !3 Reg. ! # 4. ! # 256. 2. IoU. ! # 1. ! # 256. Figure 2: Illustration of the difference between YOLOv3 head and the proposed decoupled head. For each level of FPN. Step feature, we firstheadadoptStepa 1 Baseline Decoupled 1 conv layer to reduce the feature channel to 256 and then add two parallel branches with two 10 10 3 320conv layers each for20classification and regression tasks respectively.)))

10 IoU branch is added on the regression branch. 30 30 40 40 50 50 60. 60 70 70 thus train all the following models from scratch. 80. 80 90 90 100 100 110 110 COCO AP (%). 120 120 Anchor-free Both YOLOv4 [1] and YOLOv5 [7] fol- 130 130 140 140 low the original anchor-based pipeline of YOLOv3 [25]. 150 150 However, the anchor mechanism has many known prob- 160. 160 170 170 lems. First, to achieve optimal detection performance, one 180 180 Decoupled head 190. 190 YOLO needs to conduct clustering analysis to determine a set of 200 200 2100 210 optimal anchors before training. Those clustered anchors 220 0 50 100 220 150. 200. 250 230 230 Epochs are domain-specific and less generalized. Second, anchor 240 240 mechanism increases the complexity of detection heads, as 250 250 260 260 Figure 3: Training curves for detectors with YOLOv3 head well as the Number of predictions for each image.