Transcription of Attention and Transformers Lecture 11
{{id}} {{{paragraph}}}
Example: barber
Attention and Transformers Lecture 11
Fei-Fei Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20211 Lecture 11: Attention and TransformersFei-Fei Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20212 Administrative: Midterm-Midterm was this Tuesday-We will be grading this week and you should have grades by next Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20213 Administrative: Assignment 3-A3 is due Friday May 25th, 11:59pm Lots of applications of ConvNets Also contains an extra credit notebook, which is worth an additional 5% of the A3 grade. Extra credit will not be used when curving the class Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20214 Last Time: Recurrent Neural NetworksFei-Fei Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20215 Last Time: Variable length computation graph with shared x2x1 WhTy3y2y1L1L2L3 LTLFei-Fei Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 2021 Let's jump to Lecture 10 - slide 436 Fei-Fei Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20217 Today's Agenda:- Attention with RNNs-In Computer Vision-In NLP-General Attention Layer-Self- Attention -Positional encoding-Masked Attention -Multi-head Attention -TransformersFei-Fei Li, Ranjay Krishna, Danfei XuLecture 11 -May 06, 20218 Today's Agenda:- Attention with RNNs-In Computer Vision-I
att (.) is an MLP. Fei-Fei Li, Ranjay Krishna, Danfei Xu Lecture 11 - May 06, 2021 Extract spatial features from a pretrained CNN Image Captioning with RNNs & Attention 18 CNN Features: H x W x D h 0 Xu et al, “Show, Attend and Tell: Neural Image Caption Generation with Visual Attention”, ICML 2015 z 0,0 z 0,1 z 0,2 z 1,0 z 1,1 z 1,2 z 2,0 ...
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: