Transcription of Self-Prediction and Contrastive Learning
{{id}} {{{paragraph}}}
Example: quiz answers
Self-Prediction and Contrastive Learning
Self-Supervised LearningSelf-Prediction and Contrastive LearningLilian Weng, Jong Wook KimNeurIPS 2021 Tutorial Introduction: motivation, basic concepts, examples. Early work: look into connection with old methods. Methods Self-Prediction Contrastive Learning Pretext tasks: a wide range of literature review. Techniques: improve training efficiency. Future directionsOutline2 IntroductionWhat is self-supervised Learning and why we need it?3 Self-Supervised Learning (SSL) is a special type of representation Learning that enables Learning good data representation from unlabelled is motivated by the idea of constructing supervised Learning tasks out of unsupervised is Self-Supervised Learning ?4 Self-Supervised Learning (SSL) is a special type of representation Learning that enables Learning good data representation from unlabelled is motivated by the idea of constructing supervised Learning tasks out of unsupervised datasets.
Denoising autoencoder (Vincent et al. 2008) Add noise = Randomly mask some pixels Only reconstruction loss Context autoencoder (Pathak et al. 2016) Mask a random region in the image Reconstruction loss + adversarial loss Vision Pretext Tasks: Masked Prediction 50
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: