Example: marketing

Abstract arXiv:1807.11626v2 [cs.CV] 18 Apr 2019

MnasNet: platform -Aware Neural Architecture Search for MobileMingxing Tan1Bo Chen2 Ruoming Pang1 Vijay Vasudevan1 Mark Sandler2 Andrew Howard2 Quoc V. Le11 Google Brain,2 Google Inc.{tanmingxing, bochen, rpang, vrv, sandler, howarda, convolutional neural networks (CNN) formobile devices is challenging because mobile models needto be small and fast, yet still accurate. Although significantefforts have been dedicated to design and improve mobileCNNs on all dimensions, it is very difficult to manuallybalance these trade-offs when there are so many architec-tural possibilities to consider.}

MnasNet: Platform-Aware Neural Architecture Search for Mobile Mingxing Tan 1Bo Chen2 Ruoming Pang Vijay Vasudevan1 Mark Sandler2 Andrew Howard2 Quoc V. Le1 1Google Brain, 2Google Inc. ftanmingxing, bochen, rpang, vrv, sandler, howarda, qvlg@google.com

Tags:

  Platform

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Abstract arXiv:1807.11626v2 [cs.CV] 18 Apr 2019

1 MnasNet: platform -Aware Neural Architecture Search for MobileMingxing Tan1Bo Chen2 Ruoming Pang1 Vijay Vasudevan1 Mark Sandler2 Andrew Howard2 Quoc V. Le11 Google Brain,2 Google Inc.{tanmingxing, bochen, rpang, vrv, sandler, howarda, convolutional neural networks (CNN) formobile devices is challenging because mobile models needto be small and fast, yet still accurate. Although significantefforts have been dedicated to design and improve mobileCNNs on all dimensions, it is very difficult to manuallybalance these trade-offs when there are so many architec-tural possibilities to consider.}

2 In this paper, we proposean automated mobile neural architecture search (MNAS)approach, which explicitly incorporate model latency intothe main objective so that the search can identify a modelthat achieves a good trade-off between accuracy andlatency. Unlike previous work, where latency is consideredvia another, often inaccurate proxy ( , FLOPS), ourapproach directly measures real-world inference latencyby executing the model on mobile furtherstrike the right balance between flexibility and searchspace size, we propose a novel factorized hierarchicalsearch space that encourages layer diversity throughoutthe network.

3 Experimental results show that our approachconsistently outperforms state-of-the-art mobile CNNmodels across multiple vision the ImageNetclassification task, our MnasNet achieves top-1accuracy with 78ms latency on a Pixel phone, which faster than MobileNetV2 [29] with higheraccuracy faster than NASNet [36] with accuracy. Our MnasNet also achieves better mAPquality than MobileNets for COCO object detection. Codeis IntroductionConvolutional neural networks (CNN) have made signif-icant progress in image classification, object detection, andmany other applications.

4 As modern CNN models becomeincreasingly deeper and larger [31, 13, 36, 26], they also be-come slower, and require more computation. Such increasesin computational demands make it difficult to deploy state-of-the-art CNN models on resource-constrained platformsSample models from search spaceTrainerMobile phonesMulti-objective rewardlatencyrewardControlleraccuracyFig ure 1:An Overview of platform -Aware Neural Archi-tecture Search for Latency (ms)7071727374757677 Imagenet Top 1 Accuracy (%)MobileNetV1 MobileNetV2 MobileNetV2( )NASNet-AAmoebaNet-AMnasNetFigure 2:Accuracy vs. Latency Comparison Our Mnas-Net models significantly outperforms other mobile models[29, 36, 26] on ImageNet.

5 Details can be found in Table as mobile or embedded restricted computational resources available onmobile devices, much recent research has focused on de-signing and improving mobile CNN models by reducingthe depth of the network and utilizing less expensive oper-ations, such as depthwise convolution [11] and group con-volution [33]. However, designing a resource-constrainedmobile model is challenging: one has to carefully balanceaccuracy and resource-efficiency, resulting in a significantlylarge design [ ] 29 May 2019In this paper, we propose an automated neural architec-ture search approach for designing mobile CNN 1 shows an overview of our approach, where themain differences from previous approaches are the latencyaware multi-objective reward and the novel search approach is based on two main ideas.

6 First, we formu-late the design problem as a multi-objective optimizationproblem that considers both accuracy and inference latencyof CNN models. Unlike in previous work [36, 26, 21] thatuse FLOPS to approximate inference latency, we directlymeasure the real-world latency by executing the model onreal mobile devices. Our idea is inspired by the observa-tion that FLOPS is often an inaccurate proxy: for exam-ple, MobileNet [11] and NASNet [36] have similar FLOPS(575M vs. 564M), but their latencies are significantly dif-ferent (113ms vs. 183ms, details in Table 1). Secondly, weobserve that previous automated approaches mainly searchfor a few types of cells and then repeatedly stack the samecells through the network.

7 This simplifies the search pro-cess, but also precludes layer diversity that is important forcomputational efficiency. To address this issue, we proposea novelfactorized hierarchical search space, which allowslayers to be architecturally different yet still strikes the rightbalance between flexibility and search space apply our proposed approach to ImageNet classifica-tion [28] and COCO object detection [18]. Figure 2 sum-marizes a comparison between our MnasNet models andother state-of-the-art mobile models. Compared to the Mo-bileNetV2 [29], our model improves the ImageNet accuracyby with similar latency on the Google Pixel the other hand, if we constrain the target accuracy, thenour MnasNet models fasterthan fasterthans NASNet [36] with better to the widely used ResNet-50 [9], our MnasNetmodel achieves slightly higher ( )

8 Accuracy fewerparameters and10 fewermultiply-add plugging our model as a feature extractor into the SSDobject detection framework, our model improves both theinference latency and the mAP quality on COCO datasetover MobileNetsV1 and MobileNetV2, and achieves com-parable mAP quality ( vs ) as SSD300 [22] with42 lessmultiply-add summarize, our main contributions are as follows:1. We introduce amulti-objectiveneural architecturesearch approach that optimizes both accuracy and real-world latency on mobile We propose a novelfactorized hierarchical searchspaceto enable layer diversity yet still strike the rightbalance between flexibility and search space We demonstrate new state-of-the-art accuracy on bothImageNet classification and COCO object detectionunder typical mobile latency Related WorkImproving the resource efficiency of CNN models hasbeen an active research topic during the last several commonly-used approaches include 1)

9 Quantizing theweights and/or activations of a baseline CNN model intolower-bit representations [8, 16], or 2) pruning less impor-tant filters according to FLOPs [6, 10], or to platform -awaremetrics such as latency introduced in [32]. However, thesemethods are tied to a baseline model and do not focus onlearning novel compositions of CNN common approach is to directly hand-craft moreefficient mobile architectures: SqueezeNet [15] reduces thenumber of parameters and computation by using lower-cost 1x1 convolutions and reducing filter sizes; MobileNet[11] extensively employs depthwise separable convolutionto minimize computation density; ShuffleNets [33, 24] uti-lize low-cost group convolution and channel shuffle; Con-densenet [14] learns to connect group convolutions acrosslayers.

10 Recently, MobileNetV2 [29] achieved state-of-the-art results among mobile-size models by using resource-efficient inverted residuals and linear bottlenecks. Unfortu-nately, given the potentially huge design space, these hand-crafted models usually take significant human , there has been growing interest in automatingthe model design process using neural architecture approaches are mainly based on reinforcement learn-ing [35, 36, 1, 19, 25], evolutionary search [26], differen-tiable search [21], or other learning algorithms [19, 17, 23].Although these methods can generate mobile-size modelsby repeatedly stacking a few searched cells, they do not in-corporate mobile platform constraints into the search pro-cess or search space.


Related search queries