Lecture 13: Generative Models

1 Lecture 13: Generative Models Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 1 May 18, 2017. Administrative Midterm grades released on Gradescope this week A3 due next Friday, 5/26. HyperQuest deadline extended to Sunday 5/21, 11:59pm Poster session is June 6. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 2 May 18, 2017. Overview Unsupervised Learning Generative Models PixelRNN and PixelCNN. Variational Autoencoders (VAE). Generative Adversarial Networks (GAN). Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 3 May 18, 2017.

2 Supervised vs Unsupervised Learning Supervised Learning Data: (x, y). x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, regression, object detection, semantic segmentation, image captioning, etc. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 4 May 18, 2017. Supervised vs Unsupervised Learning Supervised Learning Data: (x, y). x is data, y is label Cat Goal: Learn a function to map x -> y Examples: Classification, regression, object detection, Classification semantic segmentation, image captioning, etc.

3 This image is CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 5 May 18, 2017. Supervised vs Unsupervised Learning Supervised Learning Data: (x, y). x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, DOG, DOG, CAT. regression, object detection, semantic segmentation, image Object Detection captioning, etc. This image is CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 6 May 18, 2017. Supervised vs Unsupervised Learning Supervised Learning Data: (x, y).

4 X is data, y is label Goal: Learn a function to map x -> y GRASS, CAT, Examples: Classification, TREE, SKY. regression, object detection, semantic segmentation, image Semantic Segmentation captioning, etc. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 7 May 18, 2017. Supervised vs Unsupervised Learning Supervised Learning Data: (x, y). x is data, y is label Goal: Learn a function to map x -> y A cat sitting on a suitcase on the floor Examples: Classification, regression, object detection, Image captioning semantic segmentation, image captioning, etc.

5 Caption generated using neuraltalk2. Image is CC0 Public domain. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 8 May 18, 2017. Supervised vs Unsupervised Learning Unsupervised Learning Data: x Just data, no labels! Goal: Learn some underlying hidden structure of the data Examples: Clustering, dimensionality reduction, feature learning, density estimation, etc. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 9 May 18, 2017. Supervised vs Unsupervised Learning Unsupervised Learning Data: x Just data, no labels! Goal: Learn some underlying hidden structure of the data Examples: Clustering, K-means clustering dimensionality reduction, feature learning, density estimation, etc.

6 This image is CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 10 May 18, 2017. Supervised vs Unsupervised Learning Unsupervised Learning Data: x Just data, no labels! Goal: Learn some underlying hidden structure of the data 3-d 2-d Examples: Clustering, Principal Component Analysis dimensionality reduction, feature (Dimensionality reduction). learning, density estimation, etc. This image from Matthias Scholz is CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 11 May 18, 2017. Supervised vs Unsupervised Learning Unsupervised Learning Data: x Just data, no labels!

7 Goal: Learn some underlying hidden structure of the data Examples: Clustering, dimensionality reduction, feature Autoencoders learning, density estimation, etc. (Feature learning). Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 12 May 18, 2017. Supervised vs Unsupervised Learning Unsupervised Learning Data: x Figure copyright Ian Goodfellow, 2016. Reproduced with permission. Just data, no labels! 1-d density estimation Goal: Learn some underlying hidden structure of the data Examples: Clustering, dimensionality reduction, feature 2-d density estimation learning, density estimation, etc.

8 2-d density images left and right are CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 13 May 18, 2017. Supervised vs Unsupervised Learning Supervised Learning Unsupervised Learning Data: (x, y) Data: x x is data, y is label Just data, no labels! Goal: Learn a function to map x -> y Goal: Learn some underlying hidden structure of the data Examples: Classification, regression, object detection, Examples: Clustering, semantic segmentation, image dimensionality reduction, feature captioning, etc. learning, density estimation, etc.

9 Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 14 May 18, 2017. Supervised vs Unsupervised Learning Supervised Learning Unsupervised Learning Training data is cheap Data: (x, y) Data: x Holy grail: Solve x is data, y is label Just data, no labels! unsupervised learning => understand structure of visual world Goal: Learn a function to map x -> y Goal: Learn some underlying hidden structure of the data Examples: Classification, regression, object detection, Examples: Clustering, semantic segmentation, image dimensionality reduction, feature captioning, etc.

10 Learning, density estimation, etc. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 15 May 18, 2017. Generative Models Given training data, generate new samples from same distribution Training data ~ pdata(x) Generated samples ~ pmodel(x). Want to learn pmodel(x) similar to pdata(x). Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 13 - 16 May 18, 2017. Generative Models Given training data, generate new samples from same distribution Training data ~ pdata(x) Generated samples ~ pmodel(x). Want to learn pmodel(x) similar to pdata(x).