MatConvNet: Convolutional Neural Networks for …

1 MatConvNetConvolutional Neural Networks for MATLABA ndrea VedaldiKarel LencAnkush GuptaiiiAbstractMatConvNetis an implementation of Convolutional Neural Networks (CNNs)for MATLAB. The toolbox is designed with an emphasis on simplicity and exposes the building blocks of CNNs as easy-to-use MATLAB functions, providingroutines for computing linear convolutions with filter banks, feature pooling, and manymore. In this manner,MatConvNetallows fast prototyping of new CNN architec-tures; at the same time, it supports efficient computation on CPU and GPU allowingto train complex models on large datasets such as ImageNet ILSVRC.

2 This documentprovides an overview of CNNs and how they are implemented inMatConvNetandgives the technical details of each computational block in the Introduction to Getting started .. a glance .. Documentation and examples .. Speed .. Acknowledgments ..72 Neural network Overview .. network structures .. acyclic graphs .. Computing derivatives with backpropagation .. of tensor functions .. of function compositions .. Networks .. in DAGs .. backpropagation Networks ..183 Wrappers and pre-trained Wrappers .. Pre-trained models .. Learning models.

3 Running large scale experiments .. Reading images ..234 Computational Convolution .. Convolution transpose (deconvolution) .. Spatial pooling .. Activation functions .. Spatial bilinear resampling .. Region of interest pooling .. Normalization .. response normalization (LRN) .. normalization .. normalization .. Categorical losses .. losses .. losses .. Comparisons ..385 Preliminaries .. Simple filters .. in Caffe .. Convolution transpose .. Transposing receptive fields .. Composing receptive fields.

4 Overlaying receptive fields ..446 Implementation Convolution .. Convolution transpose .. Spatial pooling .. Activation functions .. Spatial bilinear resampling .. Normalization .. response normalization (LRN) .. normalization .. normalization .. Categorical losses .. losses .. losses .. Comparisons .. Other implementation details .. sampler .. s algorithm ..54 Bibliography55 Chapter 1 Introduction to MatConvNetMatConvNetis a MATLAB toolbox implementingConvolutional Neural Networks (CNN)for computer vision applications. Since the breakthrough work of [8], CNNs have had amajor impact in computer vision, and image understanding in particular, essentially replacingtraditional image representations such as the ones implemented in our own VLFeat [13] opensource most CNNs are obtained by composing simple linear and non-linear filtering op-erations such as convolution and rectification, their implementation is far from trivial.

5 Thereason is that CNNs need to be learned from vast amounts of data, often millions of images,requiring very efficient implementations. As most CNN libraries,MatConvNetachievesthis by using a variety of optimizations and, chiefly, by supporting computations on other machine learning, deep learning, and CNN open source libraries cite some of the most popular ones: CudaConvNet,1 Torch,2 Theano,3and Caffe4. Manyof these libraries are well supported, with dozens of active contributors and large user , why creating yet another library?The key motivation for developingMatConvNetwas to provide an environment par-ticularly friendly and efficient for researchers to use in their this by its deep integration in the MATLAB environment, which is one of the mostpopular development environments in computer vision research as well as in many other particular,MatConvNetexposes as simple MATLAB commands CNN building blockssuch as convolution, normalisation and pooling (chapter 4); these can then be combined andextended with ease to create CNN architectures.

6 While many of such blocks use optimisedCPU and GPU implementations written in C++ and CUDA (section section ), MATLAB native support for GPU computation means that it is often possible to write new blocksin MATLAB directly while maintaining computational efficiency. Compared to writing newCNN components using lower level languages, this is an important simplification that cansignificantly accelerate testing new ideas. Using MATLAB also provides a bridge towards1 :start3 from a user perspectiveMatConvNetcurrently relies on MATLAB, the library is being devel-oped with a clean separation between MATLAB code and the C++ and CUDA core; therefore, in the futurethe library may be extended to allow processing Convolutional Networks independently of 1.

7 INTRODUCTION TO matconvnet other areas; for instance,MatConvNetwas recently used by the University of Arizona inplanetary science, as summarised in this NVIDIA learn large CNN models such AlexNet [8] and the very deep networksof [11] from millions of images. Pre-trained versions of several of these powerful models canbe downloaded from theMatConvNethome page7. While powerful,MatConvNetre-mains simple to use and install. The implementation is fully self-contained, requiring onlyMATLAB and a compatible C++ compiler (using the GPU code requires the freely-availableCUDA DevKit and a suitable NVIDIA GPU). As demonstrated in fig.

8 And section ,it is possible to download, compile, and installMatConvNetusing three MATLAB com-mands. Several fully-functional examples demonstrating how small and large Networks canbe learned are included. Importantly, severalstandard pre-trained networkcan be immedi-ately downloaded and used in applications. A manual with a complete technical descriptionof the toolbox is maintained along with the features makeMatConvNetuseful in an educational context open-source released under a BSD-like license. It can be well as from Getting startedMatConvNetis simple to install and use. fig. provides a complete example that clas-sifies an image using a latest-generation deep Convolutional Neural network .

9 The exampleincludes downloading matconvnet , compiling the package, downloading a pre-trained CNNmodel, and evaluating the latter on one ofMATLAB s stock key command in this example isvl_simplenn, a wrapper that takes as input theCNNnetand the pre-processed imageim_and produces as output a structureresof particular wrapper can be used to model Networks that have a simple structure, namelyachainof operations. Examining the code ofvl_simplenn(edit vl_simplenninMatCon-vNet) we note that the wrapper transforms the data sequentially, applying a number ofMATLAB functions as specified by the network configuration.

10 These function, discussed indetail in chapter 4, are called building blocks and constitute the backbone most blocks implement simple operations, what makes them non trivial is theirefficiency (section ) as well as support for backpropagation (section ) to allow learningCNNs. Next, we demonstrate how to use one of such building blocks directly. For the sake ofthe example, consider convolving an image with a bank of linear filters. Start by reading animage inMATLAB, say usingim=single(imread(' ')), obtaining aH W Darrayim, whereD= 3 is the number of colour channels in the image. Then create a bankofK= 16 random filters of size 3 3 usingf= randn(3,3,3,16,'single').