Transcription of Abstract - arxiv.org
{{id}} {{{paragraph}}}
Example: stock market
Abstract - arxiv.org
Representation Learning withContrastive predictive CodingAaron van den supervised learning has enabled great progress in many applications, unsu-pervised learning has not seen such widespread adoption, and remains an importantand challenging endeavor for artificial intelligence. In this work, we propose auniversal unsupervised learning approach to extract useful representations fromhigh-dimensional data, which we call Contrastive predictive Coding. The key in-sight of our model is to learn such representations by predicting the future inlatentspace by using powerful autoregressive models. We use a probabilistic contrastiveloss which induces the latent space to capture information that is maximally usefulto predict future samples. It also makes the model tractable by using negativesampling. While most prior work has focused on evaluating representations fora particular modality, we demonstrate that our approach is able to learn usefulrepresentations achieving strong performance on four distinct domains: speech,images, text and reinforcement learning in 3D IntroductionLearning high-level representations from labeled data with layered differentiable models in an end-to-end fashion is one of the biggest successes in artificial intelligence so far.
2.2 Contrastive Predictive Coding Figure 1 shows the architecture of Contrastive Predictive Coding models. First, a non-linear encoder g enc maps the input sequence of observations xt to a sequence of latent representations zt = g enc(xt), potentially with a lower temporal resolution. Next, an autoregressive model g ar summarizes all z t in
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: