Caffe: Convolutional Architecture for Fast Feature …

1 Caffe: Convolutional Architecture for fast Feature Embedding . Yangqing Jia , Evan Shelhamer , Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor Darrell UC Berkeley EECS, Berkeley, CA 94702. ABSTRACT 1. INTRODUCTION. [ ] 20 Jun 2014. Caffe provides multimedia scientists and practitioners with A key problem in multimedia data analysis is discovery of a clean and modifiable framework for state-of-the-art deep effective representations for sensory inputs images, sound- learning algorithms and a collection of reference models. waves, haptics, etc.

2 While performance of conventional, The framework is a BSD-licensed C++ library with Python handcrafted features has plateaued in recent years, new de- and MATLAB bindings for training and deploying general- velopments in deep compositional architectures have kept purpose Convolutional neural networks and other deep mod- performance levels rising [8]. Deep models have outper- els efficiently on commodity architectures. Caffe fits indus- formed hand-engineered Feature representations in many do- try and internet-scale media needs by CUDA GPU computa- mains, and made learning possible in domains where engi- tion, processing over 40 million images a day on a single K40 neered features were lacking entirely.

3 Or Titan GPU ( ms per image). By separating model We are particularly motivated by large-scale visual recog- representation from actual implementation, Caffe allows ex- nition, where a specific type of deep Architecture has achieved perimentation and seamless switching among platforms for a commanding lead on the state-of-the-art. These Con- ease of development and deployment from prototyping ma- volutional Neural Networks, or CNNs, are discriminatively chines to cloud environments. trained via back-propagation through layers of Convolutional Caffe is maintained and developed by the Berkeley Vi- filters and other operations such as rectification and pooling.

4 Sion and Learning Center (BVLC) with the help of an ac- Following the early success of digit classification in the 90's, tive community of contributors on GitHub. It powers on- these models have recently surpassed all known methods for going research projects, large-scale industrial applications, large-scale visual recognition, and have been adopted by in- and startup prototypes in vision, speech, and multimedia. dustry heavyweights such as Google, Facebook, and Baidu for image understanding and search. Categories and Subject Descriptors While deep neural networks have attracted enthusiastic interest within computer vision and beyond, replication of [Pattern Recognition]: [Applications Computer vi- published results can involve months of work by a researcher sion]; [Software Engineering]: [Design Tools and or engineer.]

5 Sometimes researchers deem it worthwhile to Techniques Software libraries]; [Pattern Recognition]: release trained models along with the paper advertising their [Models Neural Nets] performance. But trained models alone are not sufficient for rapid research progress and emerging commercial applica- General Terms tions, and few toolboxes offer truly off-the-shelf deployment Algorithms, Design, Experimentation of state-of-the-art models and those that do are often not computationally efficient and thus unsuitable for commercial deployment. Keywords To address such problems, we present Caffe, a fully open- Open Source, Computer Vision, Neural Networks, Parallel source framework that affords clear access to deep architec- Computation, Machine Learning tures.

6 The code is written in clean, efficient C++, with Corresponding Authors. CUDA used for GPU computation, and nearly complete, The work was done while Yangqing Jia was a graduate student at Berkeley. He is well-supported bindings to Python/Numpy and MATLAB. currently a research scientist at Google, 1600 Amphitheater Caffe adheres to software engineering best practices, pro- Pkwy, Mountain View, CA 94043. viding unit tests for correctness and experimental rigor and speed for deployment. It is also well-suited for research use, due to the careful modularity of the code, and the clean sep- aration of network definition (usually the novel part of deep learning research) from actual implementation.

7 In Caffe, multimedia scientists and practitioners have an orderly and extensible toolkit for state-of-the-art deep learn- ing algorithms, with reference models provided out of the box. fast CUDA code and GPU computation fit industry needs by achieving processing speeds of more than 40 mil- . Core Open Pretrained Framework License language Binding(s) CPU GPU source Training models Development Python, Caffe BSD C++ distributed MATLAB. cuda-convnet [7] unspecified C++ Python discontinued Decaf [2] BSD Python discontinued OverFeat [9] unspecified Lua C++,Python centralized Theano/Pylearn2 [4] BSD Python distributed Torch7 [1] BSD Lua distributed Table 1: Comparison of popular deep learning frameworks.

8 Core language is the main library language, while bindings have an officially supported library interface for Feature extraction, training, etc. CPU indicates availability of host-only computation, no GPU usage ( , for cluster deployment); GPU indicates the GPU. computation capability essential for training modern CNNs. lion images per day on a single K40 or Titan GPU. The may be used to construct networks and classify inputs. The same models can be run in CPU or GPU mode on a vari- Python bindings also expose the solver module for easy pro- ety of hardware: Caffe separates the representation from the totyping of new training procedures.

9 Actual implementation, and seamless switching between het- Pre-trained reference models. Caffe provides (for aca- erogeneous platforms furthers development and deployment demic and non-commercial use not BSD license) reference Caffe can even be run in the cloud. models for visual tasks, including the landmark AlexNet . While Caffe was first designed for vision, it has been adopted ImageNet model [8] with variations and the R-CNN detec- and improved by users in speech recognition, robotics, neu- tion model [3]. More are scheduled for release. We are roscience, and astronomy.

10 We hope to see this trend con- strong proponents of reproducible research: we hope that tinue so that further sciences and industries can take advan- a common software substrate will foster quick progress in tage of deep learning. the search over network architectures and applications. Caffe is maintained and developed by the BVLC with the active efforts of several graduate students, and welcomes Comparison to related software open-source contributions at We summarize the landscape of Convolutional neural net- We thank all of our contributors for their work! work software used in recent publications in Table 1.