Example: air traffic controller

Lecture 10: Recurrent Neural Networks

Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20171 Lecture 10: Recurrent Neural NetworksFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20172 AdministrativeA1 grades will go out soonA2 is due today (11:59pm)Midterm is in-class on Tuesday!We will send out details on where to go soonFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20173 Extra Credit: Train GameMore details on Piazza by early next weekFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20174 Last Time: CNN ArchitecturesAlexNetFigure copyright Kaiming He, 2016. Reproduced with permission. Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20175 Last Time: CNN ArchitecturesFigure copyright Kaiming He, 2016.

image -> sequence of words. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 13 May 4, 2017 Recurrent Neural Networks: Process Sequences e.g. Sentiment Classification sequence of words -> sentiment. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 10 - 14 May 4, 2017

Fullscreen Download

Tags:

  Image, Recurrent

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Lecture 10: Recurrent Neural Networks

1 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20171 Lecture 10: Recurrent Neural NetworksFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20172 AdministrativeA1 grades will go out soonA2 is due today (11:59pm)Midterm is in-class on Tuesday!We will send out details on where to go soonFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20173 Extra Credit: Train GameMore details on Piazza by early next weekFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20174 Last Time: CNN ArchitecturesAlexNetFigure copyright Kaiming He, 2016. Reproduced with permission. Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20175 Last Time: CNN ArchitecturesFigure copyright Kaiming He, 2016.

2 Reproduced with permission. 3x3 conv, 128 Pool3x3 conv, 643x3 conv, 64 Input3x3 conv, 128 Pool3x3 conv, 2563x3 conv, 256 Pool3x3 conv, 5123x3 conv, 512 Pool3x3 conv, 5123x3 conv, 512 PoolFC 4096FC 1000 SoftmaxFC 40963x3 conv, 5123x3 conv, 512 PoolInputPoolPoolPoolPoolSoftmax3x3 conv, 5123x3 conv, 5123x3 conv, 2563x3 conv, 2563x3 conv, 1283x3 conv, 1283x3 conv, 643x3 conv, 643x3 conv, 5123x3 conv, 5123x3 conv, 5123x3 conv, 5123x3 conv, 5123x3 conv, 512FC 4096FC 1000FC 4096 VGG16 VGG19 GoogLeNetFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20176 Last Time: CNN ArchitecturesFigure copyright Kaiming He, 2016. Reproduced with permission. InputSoftmax3x3 conv, 647x7 conv, 64 / 2FC 1000 Pool3x3 conv, 643x3 conv, 643x3 conv, 643x3 conv, 643x3 conv, 643x3 conv, 1283x3 conv, 128 / 23x3 conv, 1283x3 conv, 1283x3 conv, 1283x3 conv, conv, 643x3 conv, 643x3 conv, 643x3 conv, 643x3 conv, 643x3 conv, 64 PoolreluResidual blockconvconvXidentityF(x) + xF(x)reluXFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20177 Figures copyright Larsson et al.

3 , 2017. Reproduced with Block 1 ConvInputConvDense Block 2 ConvPoolConvDense Block 3 SoftmaxFCPoolConvConv1x1 conv, 641x1 conv, 64 InputConcatConcatConcatDense BlockDenseNetFractalNetFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20178 Last Time: CNN ArchitecturesFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 20179 Last Time: CNN ArchitecturesAlexNet and VGG have tons of parameters in the fully connected layersAlexNet: ~62M parametersFC6: 256x6x6 -> 4096: 38M paramsFC7: 4096 -> 4096: 17M paramsFC8: 4096 -> 1000: 4M params~59M params in FC layers!Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201710 Today: Recurrent Neural NetworksFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201711 Vanilla Neural Networks Vanilla Neural NetworkFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201712 Recurrent Neural Networks : Process image Captioningimage -> sequence of wordsFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201713 Recurrent Neural Networks : Process Sentiment Classificationsequence of words -> sentimentFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201714 Recurrent Neural Networks : Process Machine Translationseq of words -> seq of wordsFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201715 Recurrent Neural Networks .

4 Process Video classification on frame levelFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201716 Sequential Processing of Non-Sequence DataBa, Mnih, and Kavukcuoglu, Multiple Object Recognition with Visual Attention , ICLR et al, DRAW: A Recurrent Neural Network For image Generation , ICML 2015 Figure copyright Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra, 2015. Reproduced with images by taking a series of glimpses Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201717 Sequential Processing of Non-Sequence DataGregor et al, DRAW: A Recurrent Neural Network For image Generation , ICML 2015 Figure copyright Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra, 2015. Reproduced with images one piece at a time!

5 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201718 Recurrent Neural NetworkxRNNFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201719 Recurrent Neural NetworkxRNNyusually want to predict a vector at some time stepsFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201720 Recurrent Neural NetworkxRNNyWe can process a sequence of vectors x by applying a recurrence formula at every time step:new stateold stateinput vector at some time stepsome functionwith parameters WFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201721 Recurrent Neural NetworkxRNNyWe can process a sequence of vectors x by applying a recurrence formula at every time step:Notice: the same function and the same set of parameters are used at every time Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201722(Vanilla) Recurrent Neural NetworkxRNNyThe state consists of a single hidden vector h:Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201723h0fWh1x1 RNN.

6 Computational GraphFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201724h0fWh1fWh2x2x1 RNN: Computational GraphFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, x2x1 RNN: Computational GraphhTFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, x2x1 WRNN: Computational GraphRe-use the same weight matrix at every time-stephTFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, x2x1 WRNN: Computational Graph: Many to ManyhTy3y2y1 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, x2x1 WRNN: Computational Graph: Many to ManyhTy3y2y1L1L2L3 LTFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, x2x1 WRNN: Computational Graph: Many to ManyhTy3y2y1L1L2L3 LTLFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, x2x1 WRNN: Computational Graph: Many to OnehTFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, xWRNN: Computational Graph.

7 One to ManyhTy3y3y3 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201732 Sequence to Sequence: Many-to-one + x2x1W1hTMany to one: Encode input sequence in a single vectorFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201733 Sequence to Sequence: Many-to-one + Many to one: Encode input sequence in a single vectorOne to many: Produce output sequence from single input vectorfWh1fWh2fWW2 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201734 Example: Character-levelLanguage ModelVocabulary:[h,e,l,o]Example trainingsequence: hello Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201735 Example: Character-levelLanguage ModelVocabulary:[h,e,l,o]Example trainingsequence: hello Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201736 Example: Character-levelLanguage ModelVocabulary:[h,e,l,o]Example trainingsequence: hello Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201737 Example: Character-levelLanguage ModelSamplingVocabulary.

8 [h,e,l,o]At test-time sample characters one at a time, feed back to e l l o SampleFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, e l l o SampleExample: Character-levelLanguage ModelSamplingVocabulary:[h,e,l,o]At test-time sample characters one at a time, feed back to modelFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, e l l o SampleExample: Character-levelLanguage ModelSamplingVocabulary:[h,e,l,o]At test-time sample characters one at a time, feed back to modelFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, e l l o SampleExample: Character-levelLanguage ModelSamplingVocabulary.

9 [h,e,l,o]At test-time sample characters one at a time, feed back to modelFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201741 Backpropagation through timeLossForward through entire sequence to compute loss, then backward through entire sequence to compute gradientFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201742 Truncated Backpropagation through timeLossRun forward and backward through chunks of the sequence instead of whole sequenceFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201743 Truncated Backpropagation through timeLossCarry hidden states forward in time forever, but only backpropagate for some smaller number of stepsFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201744 Truncated Backpropagation through timeLossFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, gist: 112 lines of Python( )Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201746xRNNyFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201747train moretrain moretrain moreat first.

10 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201748 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201749 The Stacks Project: open source algebraic geometry textbookLatex stacks project is licensed under the GNU Free Documentation LicenseFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201750 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201751 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201752 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 201753 Generated C codeFei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4, 2017 Fei-Fei Li & Justin Johnson & Serena YeungLecture 10 -May 4.

Show more

Documents from same domain

Convolutional Neural Networks for Visual Recognition

Convolutional Neural Networks for Visual Recognition

cs231n.stanford.edu

Progressive GAN, Karras 2018. Models from Single RGB Images”, ECCV 2018 Beyond recognition: Segmentation, 2D/3D Generation. Fei-Fei Li, Ranjay Krishna, Danfei Xu Lecture 1 - 15 March 30, 2021 Scene Graphs Krishna et al., Visual Genome: Connecting Vision and Language using Crowdsourced Image Annotations, IJCV 2017

  Network, Visual, Recognition, Neural, Convolutional, Karar, Convolutional neural networks for visual recognition

Lecture 9: CNN Architectures

Lecture 9: CNN Architectures

cs231n.stanford.edu

Lecture 9 - 22 May 2, 2017 ImageNet Large Scale Visual Recognition Challenge (ILSVRC) winners First CNN-based winner. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 9 - 23 May 2, 2017 ImageNet Large Scale Visual Recognition Challenge (ILSVRC) winners ZFNet: Improved hyperparameters over AlexNet. Fei-Fei Li & Justin Johnson & Serena Yeung ...

  2017, Challenges, Scale, Visual, Recognition, Ilsvrc, Scale visual recognition challenge

Attention and Transformers Lecture 11

Attention and Transformers Lecture 11

cs231n.stanford.edu

graph with shared weights h 0 f W h 1 f W h 2 f W h 3 x 3 y T ... Extract spatial features from a pretrained CNN Image Captioning using spatial features 11 CNN Features: H x W x D h 0 [START] Xu et al, “Show, Attend and Tell: Neural Image Caption Generation with Visual Attention”, ICML 2015 z 0,0 z 0,1 z 0,2 z 1,0 z 1,1 z 1,2 z 2,0 z 2,1 z ...

  Transformers, Attention, Graph, Spatial, Attention and transformers

Lecture 11: Detection and Segmentation

Lecture 11: Detection and Segmentation

cs231n.stanford.edu

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 - 1 May 10, 2017 Lecture 11: Detection and Segmentation

  Detection, Segmentation

NaveenAppiah SagarVare - Stanford University

NaveenAppiah SagarVare - Stanford University

cs231n.stanford.edu

NaveenAppiah Mechanical Engineering [email protected] SagarVare Stanford ICME [email protected] ... the popular mobile game - Flappy Bird. It involves navi-gating a bird through a bunch of obstacles. Though, this ... the game emulator and learns to make good decisions over time. It is this simple learning framework and their

  Make, Games, Flappy, Naveenappiah

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 2 ...

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 2 ...

cs231n.stanford.edu

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 2 - April 6, 2017 Administrative: Piazza For questions about midterm, poster session, projects,

  Li amp justin johnson amp serena yeung lecture

CNNs for Face Detection and Recognition

CNNs for Face Detection and Recognition

cs231n.stanford.edu

development of object classification, localization and detec-tion techniques. 2.1. Sliding Window In the early development of face detection, researchers tended to treat it as a repetitive task of object classifica-tion, by imposing sliding windows and performing object classification with the neural networks on the window re-gion.

  Technique, Faces, Recognition, Object, Detection, For face detection and recognition

Vector, Matrix, and Tensor Derivatives

Vector, Matrix, and Tensor Derivatives

cs231n.stanford.edu

Erik Learned-Miller The purpose of this document is to help you learn to take derivatives of vectors, matrices, and higher order tensors (arrays with three dimensions or more), and to help you take ... At this point, we have reduced the original matrix equation (Equation 1) …

  Have, Learned

Lecture 14: Reinforcement Learning

Lecture 14: Reinforcement Learning

cs231n.stanford.edu

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 14 - May 23, 2017 Markov Decision Process 19 - Mathematical formulation of the RL problem - Markov property: Current state completely characterises the state of the

  Learning, Reinforcement, Reinforcement learning

Lecture 13: Generative Models

Lecture 13: Generative Models

cs231n.stanford.edu

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - May 18, 2017 Generative Models 17 Training data ~ p data (x) Generated samples ~ p model (x) Want to learn p

  Generative

Related documents

Introduction to Medical Image Processing

Introduction to Medical Image Processing

www.csie.ntu.edu.tw

Δ Medical Image Segmentation Segmentation, separation of structures of interest from the background and from each other, is an essential analysis function for which numerous algorithms have been developed in the field of image processing. The principal goal of the segmentation process is to partition an image into regions

  Introduction, Medical, Image, Processing, Segmentation, Introduction to medical image processing, Image segmentation segmentation

1 DeepLab: Semantic Image Segmentation with Deep ...

1 DeepLab: Semantic Image Segmentation with Deep ...

arxiv.org

tic segmentation typically employs a cascade of bottom-up image segmentation, followed by DCNN-based region classification. For instance the bounding box proposals and masked regions delivered by [47], [48] are used in [7] and [49] as inputs to a DCNN to incorporate shape information into the classification process. Similarly, the authors of [50]

  With, Image, Deep, Segmentation, Semantics, Image segmentation, Deeplab, Semantic image segmentation with deep

UNETR:Transformersfor3DMedicalImageSegmentation

UNETR:Transformersfor3DMedicalImageSegmentation

arxiv.org

Image segmentation plays an integral role in quantitative medical image analysis as it is often the first step for analysis of anatomical structures [33]. Since the advent of deep learn-ing,FCNNsandinparticular“U-shaped“encoder-decoderar-Transformer Encoder

  Image, Segmentation, Image segmentation

Indoor Segmentation and Support Inference from RGBD …

Indoor Segmentation and Support Inference from RGBD …

cs.nyu.edu

metric structure from a depth image, such as graph cut segmentation of planar surfaces and ways to use the structure to improve segmentation. Finally, we o er a new large dataset with registered RGBD images, detailed object labels, and annotated physical relations. 2 Dataset for Indoor Scene Understanding

  Form, Image, Indoor, Support, Inference, Segmentation, Indoor segmentation and support inference from

PANet: Few-Shot Image Semantic Segmentation With …

PANet: Few-Shot Image Semantic Segmentation With …

openaccess.thecvf.com

PANet can provide satisfactory segmentation results, out-performing the state-of-the-arts. Furthermore, it imposes a prototype alignment regularization by forming a new sup-port set with the query image and its predicted mask and performing segmentation on the original support set. We find this indeed encourages the prototypes generated from

  Image, Segmentation

Semantic Segmentation - Department of Computer Science ...

Semantic Segmentation - Department of Computer Science ...

www.cs.toronto.edu

What is semantic segmentation 1. What is segmentation in the first place? 1. Input: images 2. Output: regions, structures 3. Most of the time, we need to "process the image"

  Image, Segmentation

Invariant Information Clustering for Unsupervised Image ...

Invariant Information Clustering for Unsupervised Image ...

openaccess.thecvf.com

each image and its random transformation, or each patch and a neighbour. We show that maximising MI automat-ically avoids degenerate solutions and can be written as a convolution in the case of segmentation, allowing for effi-cient implementation with any deep learning library. We perform experiments on a large number of

  Image, Segmentation

Segmentation Targeting Positioning 3 - EurekaFacts

Segmentation Targeting Positioning 3 - EurekaFacts

www.eurekafacts.com

preferences such as quality, price, style, image, etc. Behavioral Measures The third category of segmentation variables is behavioral measures. It includes product usage and actual behavior such as buying patterns, usage data, channel, ownership, quantities, brand loyalty, attitudes, etc. Wilkie (1990) explains that variables in the first

  Image, Segmentation

Related search queries

Introduction to Medical Image Processing, Image Segmentation Segmentation, Image, Segmentation, DeepLab: Semantic Image Segmentation with Deep, Image segmentation, Indoor Segmentation and Support Inference from