Transcription of Neural Discrete Representation Learning
{{id}} {{{paragraph}}}
Example: quiz answers
Neural Discrete Representation Learning
Neural Discrete Representation LearningAaron van den useful representations without supervision remains a key challenge inmachine Learning . In this paper, we propose a simple yet powerful generativemodel that learns such Discrete representations. Our model, the Vector Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways: theencoder network outputs Discrete , rather than continuous, codes; and the prioris learnt rather than static. In order to learn a Discrete latent Representation , weincorporate ideas from vector quantisation (VQ). Using the VQ method allows themodel to circumvent issues of posterior collapse - where the latents are ignoredwhen they are paired with a powerful autoregressive decoder - typically observedin the VAE framework.
Lastly, once a good discrete latent structure of a modality is discovered by the VQ-VAE, we train a powerful prior over these discrete random variables, yielding interesting samples and useful applications. For instance, when trained on speech we discover the latent structure of language
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
Related search queries
Discrete, Discrete, Binomial & Geometric Random Variables, Discrete Random, Random, Probability Theory, Probability, Joint, RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS, 1 Discrete-time Markov chains, Columbia University, Discrete-event simulation, Continuous Random, Discrete uniform distribution from, Discrete uniform distribution, Random variable