f g@huawei.com jyguo@pku.edu.cn c.xu@sydney.edu.au arXiv ...

1 GhostNet: More Features from Cheap OperationsKai Han1 Yunhe Wang1Qi Tian1 Jianyuan Guo2 Chunjing Xu1 Chang Xu31 Noah s Ark Lab, Huawei of Computer Science, Faculty of Engineering, University of convolutional neural networks (CNNs) on em-bedded devices is difficult due to the limited memory andcomputation resources. The redundancy in feature mapsis an important characteristic of those successful CNNs,but has rarely been investigated in neural architecture de-sign. This paper proposes a novel Ghost module to gener-ate more feature maps from cheap operations. Based ona set of intrinsic feature maps, we apply a series of lineartransformations with cheap cost to generate many ghostfeature maps that could fully reveal information underlyingintrinsic features. The proposed Ghost module can be takenas a plug-and-play component to upgrade existing convo-lutional neural networks. Ghost bottlenecks are designedto stack Ghost modules, and then the lightweight Ghost-Net can be easily established.

2 Experiments conducted onbenchmarks demonstrate that the proposed Ghost module isan impressive alternative of convolution layers in baselinemodels, and our GhostNet can achieve higher recognitionperformance ( accuracy) than MobileNetV3with similar computational cost on the ImageNet ILSVRC-2012 classification dataset. Code is available IntroductionDeep convolutional neural networks have shown excellentperformance on various computer vision tasks, such as imagerecognition [30,13], object detection [43,33], and semanticsegmentation [4]. Traditional CNNs usually need a largenumber of parameters and floating point operations (FLOPs)to achieve a satisfactory accuracy, ResNet-50 [16] parameters and FLOPs to pro-cess an image of size224 224. Thus, the recent trendof deep neural network design is to explore portable andefficient network architectures with acceptable performancefor mobile devices ( smart phones and self-driving cars). Corresponding authorHUAWEI TECHNOLOGIES CO.

3 , Confidential1 Figure 1. Visualization of some feature maps generated by the firstresidual group in ResNet-50, where three similar feature map pairexamples are annotated with boxes of the same color. One featuremap in the pair can be approximately obtained by transforming theother one through cheap operations (denoted by spanners).Over the years, a series of methods have been proposed toinvestigate compact deep neural networks such as networkpruning [14,39], low-bit quantization [42,26], knowledgedistillation [19,57],etc. Hanet al. [14] proposed to prunethe unimportant weights in neural networks. [31] utilized`1-norm regularization to prune filters for efficient CNNs.[42] quantized the weights and the activations to 1-bit datafor achieving large compression and speed-up ratios. [19]introduced knowledge distillation for transferring knowl-edge from a larger model to a smaller model. However,performance of these methods are often upper bounded bypre-trained deep neural networks that have been taken astheir them, efficient neural architecture design has avery high potential for establishing highly efficient deep net-works with fewer parameters and calculations, and recentlyhas achieved considerable success.

4 This kind of methodscan also provide new search unit for automatic search meth-ods [62,55,5]. For instance, MobileNet [21,44,20] utilizedthe depthwise and pointwise convolutions to construct aunit for approximating the original convolutional layer withlarger filters and achieved comparable performance. Shuf-fleNet [61,40] further explored a channel shuffle operation1 [ ] 13 Mar 2020to enhance the performance of lightweight and even redundant information in the featuremaps of well-trained deep neural networks often guaranteesa comprehensive understanding of the input data. For exam-ple, Figure 1 presents some feature maps of an input imagegenerated by ResNet-50, and there exist many similar pairsof feature maps, like aghostof each another. Redundancyin feature maps could be an important characteristic for asuccessful deep neural network. Instead of avoiding the re-dundant feature maps, we tend to embrace them, but in acost-efficient this paper, we introduce a novel Ghost module to gen-erate more features by using fewer parameters.

5 Specifically,an ordinary convolutional layer in deep neural networks willbe split into two parts. The first part involves ordinary con-volutions but their total number will be rigorously the intrinsic feature maps from the first part, a seriesof simple linear operations are then applied for generatingmore feature maps. Without changing the size of outputfeature map, the overall required number of parameters andcomputational complexities in this Ghost module have beendecreased, compared with those in vanilla convolutionalneural networks. Based on Ghost module, we establish anefficient neural architecture, namely, GhostNet. We firstreplace original convolutional layers in benchmark neuralarchitectures to demonstrate the effectiveness of Ghost mod-ules, and then verify the superiority of our GhostNets onseveral benchmark visual datasets. Experimental resultsshow that, the proposed Ghost module is able to decreasecomputational costs of generic convolutional layer whilepreserving similar recognition performance, and GhostNetscan surpass state-of-the-art efficient deep models such asMobileNetV3 [20], on various tasks with fast inference onmobile rest of the paper is organized as follows: section 2briefly concludes the related work in the area, followed bythe proposed Ghost module and GhostNet in section 3, theexperiments and analysis in section 4, and finally, conclusionin section Related WorkHere we revisit the existing methods for lightening neuralnetworks in two parts: model compression and compactmodel Model CompressionFor a given neural network, model compression aimsto reduce the computation, energy and storage cost [14,48,11,54].

6 Pruning connections [15,14,50] cuts out theunimportant connections between neurons. Channel prun-ing [51,18,31,39,59,23,35] further targets on removinguseless channels for easier acceleration in practice. Modelquantization [42,24,26] represents weights or activationsin neural networks with discrete values for compressionand calculation acceleration. Specifically, binarization meth-ods [24,42,38,45] with only 1-bit values can extremelyaccelerate the model by efficient binary operations. Tensordecomposition [27,9] reduces the parameters or computa-tion by exploiting the redundancy and low-rank property inweights. Knowledge distillation [19,12,3] utilizes largermodels to teach smaller ones, which improves the perfor-mance of smaller models. The performances of these meth-ods usually depend on the given pre-trained models. Theimprovement on basic operations and architectures will makethem go Compact Model DesignWith the need for deploying neural networks on embed-ded devices, a series of compact models are proposed inrecent years [7,21,44,20,61,40,53,56].

7 Xception [7] uti-lizes depthwise convolution operation for more efficient useof model parameters. MobileNets [21] are a series of lightweight deep neural networks based on depthwise separableconvolutions. MobileNetV2 [44] proposes inverted resid-ual block and MobileNetV3 [20] further utilizes AutoMLtechnology [62,55,10] achieving better performance withfewer FLOPs. ShuffleNet [61] introduces channel shuffle op-eration to improve the information flow exchange betweenchannel groups. ShuffleNetV2 [40] further considers theactual speed on target hardware for compact model these models obtain great performance with veryfew FLOPs, the correlation and redundancy between featuremaps has never been well ApproachIn this section, we will first introduce the Ghost moduleto utilize a few small filters to generate more feature mapsfrom the original convolutional layer, and then develop anew GhostNet with an extremely efficient architecture andhigh Ghost Module for More FeaturesDeep convolutional neural networks [30,46,16] oftenconsist of a large number of convolutions that results inmassive computational costs.

8 Although recent works suchas MobileNet [21,44] and ShuffleNet [40] have introduceddepthwise convolution or shuffle operation to build efficientCNNs using smaller convolution filters (floating-numberoperations), the remaining1 1convolution layers wouldstill occupy considerable memory and the widely existing redundancy in intermediatefeature maps calculated by mainstream CNNs as shown inFigure 1, we propose to reduce the required resources, filters used for generating them. In practice,given the input dataX Rc h w, wherecis the numberof input channels andhandware the height and width of2the input data, respectively, the operation of an arbitraryconvolutional layer for producingnfeature maps can beformulated asY=X f+b,(1)where is the convolution operation,bis the bias term,Y Rh w nis the output feature map withnchannels,andf Rc k k nis the convolution filters in this layer. Inaddition,h andw are the height and width of the outputdata, andk kis the kernel size of convolution filtersf,respectively.

9 During this convolution procedure, the requirednumber of FLOPs can be calculated asn h w c k k,which is often as large as hundreds of thousands since thenumber of filtersnand the channel numbercare generallyvery large ( 256 or 512).HUAWEI TECHNOLOGIES CO., Confidential1 ConvInputOutput(a) The convolutional TECHNOLOGIES CO., 1 2 InputOutput(b) The Ghost 2. An illustration of the convolutional layer and the proposedGhost module for outputting the same number of feature maps. represents the cheap to Eq. 1, the number of parameters (infandb)to be optimized is explicitly determined by the dimensionsof input and output feature maps. As observed in Figure 1,the output feature maps of convolutional layers often containmuch redundancy, and some of them could be similar witheach other. We point out that it is unnecessary to generatethese redundant feature maps one by one with large numberof FLOPs and parameters. Suppose that the output featuremaps are ghosts of a handful of intrinsic feature mapswith some cheap transformations.

10 These intrinsic featuremaps are often of smaller size and produced by ordinaryconvolution filters. Specifically,mintrinsic feature mapsY Rh w mare generated using a primary convolution:Y =X f ,(2)wheref Rc k k mis the utilized filters,m nand thebias term is omitted for simplicity. The hyper-parameterssuch as filter size, stride, padding, are the same as those inthe ordinary convolution (Eq. 1) to keep the spatial size ( andw ) of the output feature maps consistent. To furtherobtain the desirednfeature maps, we propose to apply aseries of cheap linear operations on each intrinsic feature inY to generatesghost features according to the followingfunction:yij= i,j(y i), i= 1,..,m, j= 1,..,s,(3)wherey iis thei-th intrinsic feature map inY , i,jin theabove function is thej-th (except the last one) linear oper-ation for generating thej-th ghost feature mapyij, thatis to say,y ican have one or more ghost feature maps{yij}sj=1. The last i,sis the identity mapping for pre-serving the intrinsic feature maps as shown in Figure 2(b).