Transcription of Abstract
{{id}} {{{paragraph}}}
Example: quiz answers
Abstract
Representation Learning withContrastive Predictive CodingAaron van den supervised learning has enabled great progress in many applications, unsu-pervised learning has not seen such widespread adoption, and remains an importantand challenging endeavor for artificial intelligence. In this work, we propose auniversal unsupervised learning approach to extract useful representations fromhigh-dimensional data, which we call Contrastive Predictive Coding. The key in-sight of our model is to learn such representations by predicting the future inlatentspace by using powerful autoregressive models. We use a probabilistic contrastiveloss which induces the latent space to capture information that is maximally usefulto predict future samples. It also makes the model tractable by using negativesampling. While most prior work has focused on evaluating representations fora particular modality, we demonstrate that our approach is able to learn usefulrepresentations achieving strong performance on four distinct domains: speech,images, text and reinforcement learning in 3D IntroductionLearning high-level representations from labeled data with layered differentiable models in an end-to-end fashion is one of the biggest successes in artificial intelligence so far.
being the prediction of the model. Let us write the optimal probability for this loss as p(d = ijX;ct) with [d = i] being the indicator that sample xi is the ’positive’ sample. The probability that sample xi was drawn from the conditional distribution p(xt+kjct) rather than the proposal distribution p(xt+k) can be derived as follows: p(d ...
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: