François Chollet with J. J. Allaire SAMPLE CHAPTER

1 MANNINGFran ois Chollet with J. J. AllaireDeep Learning with Rby Fran ois Chollet with AllairChapter 3 Copyright 2018 Manning Publicationsvbrief contentsPART 1 FUNDAMENTALS OF DEEP 11 What is deep learning? 32 Before we begin: the mathematical building blocks of neural networks 243 Getting started with neural networks 504 Fundamentals of machine learning 84 PART 2 DEEP LEARNING IN 1095 Deep learning for computer vision 1116 Deep learning for text and sequences 1647 Advanced deep-learning best practices 2188 Generative deep learning 2509 Conclusions 29350 Getting started withneural networksThis CHAPTER is designed to get you started with using neural networks to solve realproblems.

2 You ll consolidate the knowledge you gained from our first practicalexample in CHAPTER 2, and you ll apply what you ve learned to three new problemscovering the three most common use cases of neural networks: binary classifica-tion, multiclass classification, and scalar regression. In this CHAPTER , we ll take a closer look at the core components of neural net-works that we introduced in CHAPTER 2: layers, networks, objective functions, andoptimizers. We ll give you a quick introduction to Keras, the deep-learning library that we ll usethroughout the book. You ll set up a deep-learning workstation with TensorFlow,This CHAPTER covers Core components of neural networks An introduction to Keras Setting up a deep-learning workstation Using neural networks to solve basic classification and regression problems51 Anatomy of a neural networkKeras, and GPU support.

3 We ll dive into three introductory examples of how to useneural networks to address real problems: Classifying movie reviews as positive or negative (binary classification) Classifying news wires by topic (multiclass classification) Estimating the price of a house, given real-estate data (regression)By the end of this CHAPTER , you ll be able to use neural networks to solve simplemachine problems such as classification and regression over vector data. You ll thenbe ready to start building a more principled, theory-driven understanding of machinelearning in CHAPTER of a neural networkAs you saw in the previous chapters, training a neural network revolves around the fol-lowing objects: Layers, which are combined into a network (or model) The input data and corresponding targets The loss function, which defines the feedback signal used for learning The optimizer, which determines how learning proceedsYou can visualize their interaction as illustrated in figure.

4 The network, composedof layers that are chained together, maps the input data to predictions. The loss func-tion then compares these predictions to the targets, producing a loss value: a measureof how well the network s predictions match what was expected. The optimizer usesthis loss value to update the network s weights. Let s take a closer look at layers, networks, loss functions, and (data transformation)Input XWeightsLayer(data transformation)PredictionsY'Weightupdate True targetsYWeightsLoss functionOptimizerLoss scoreFigure Relationship between the network, layers, loss function, and optimizer52 CHAPTER 3 Getting started with neural : the building blocks of deep learningThe fundamental data structure in neural networks is the layer, to which you wereintroduced in CHAPTER 2.

5 A layer is a data-processing module that takes as input one ormore tensors and that outputs one or more tensors. Some layers are stateless, butmore frequently layers have a state: the layer s weights, one or several tensors learnedwith stochastic gradient descent, which together contain the network s knowledge. Different layers are appropriate for different tensor formats and different types ofdata processing. For instance, simple vector data, stored in 2D tensors of shape(samples, features), is often processed by densely connected layers, also called fullyconnected or dense layers (the layer_dense function in Keras).

6 Sequence data, stored in3D tensors of shape (samples, timesteps, features), is typically processed by recur-rent layers such as layer_lstm. Image data, stored in 4D tensors, is usually processedby 2D convolution layers (layer_conv_2d). You can think of layers as the LEGO bricks of deep learning, a metaphor that ismade explicit by frameworks like Keras. Building deep-learning models in Keras isdone by clipping together compatible layers to form useful data-transformation pipe-lines. The notion of layer compatibility here refers specifically to the fact that every layerwill only accept input tensors of a certain shape and will return output tensors of a cer-tain shape.

7 Consider the following example:layer <- layer_dense(units = 32, input_shape = c(784))We re creating a layer that will only accept as input 2D tensors where the first dimen-sion is 784 (the first dimension, the batch dimension, is unspecified, and thus anyvalue would be accepted). This layer will return a tensor where the first dimension hasbeen transformed to be 32. Thus this layer can only be connected to a downstream layer that expects 32-dimensional vectors as its input. When using Keras, you don t have to worry aboutcompatibility, because the layers you add to your models are dynamically built tomatch the shape of the incoming layer.

8 For instance, suppose you write the following:model <- keras_model_sequential() %>%layer_dense(units = 32, input_shape = c(784)) %>%layer_dense(units = 32)The second layer didn t receive an input shape argument instead, it automaticallyinferred its input shape as being the output shape of the layer that came before. : networks of layersA deep-learning model is a directed, acyclic graph of layers. The most commoninstance is a linear stack of layers, mapping a single input to a single output. But as you move forward, you ll be exposed to a much broader variety of networktopologies.

9 Some common ones include the following: Two-branch networks Multihead networks Inception blocks53 Anatomy of a neural networkThe topology of a network defines a hypothesis space. You may remember that in chap-ter 1, we defined machine learning as searching for useful representations of someinput data, within a predefined space of possibilities, using guidance from a feedbacksignal. By choosing a network topology, you constrain your space of possibilities(hypothesis space) to a specific series of tensor operations, mapping input data to out-put data. What you ll then be searching for is a good set of values for the weight ten-sors involved in these tensor operations.

10 Picking the right network architecture is more an art than a science; and althoughthere are some best practices and principles you can rely on, only practice can helpyou become a proper neural-network architect. The next few chapters will both teachyou explicit principles for building neural networks and help you develop intuition asto what works or doesn t work for specific problems. functions and optimizers: keys to configuring the learning processOnce the network architecture is defined, you still have to choose two more things: Loss function (objective function) The quantity that will be minimized duringtraining.