Neural Discrete Representation Learning
Neural Discrete Representation LearningAaron van den useful representations without supervision remains a key challenge inmachine Learning . In this paper, we propose a simple yet powerful generativemodel that learns such Discrete representations. Our model, the Vector Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways: theencoder network outputs Discrete , rather than continuous, codes; and the prioris learnt rather than static. In order to learn a Discrete latent Representation , weincorporate ideas from vector quantisation (VQ). Using the VQ method allows themodel to circumvent issues of posterior collapse - where the latents are ignoredwhen they are paired with a powerful autoregressive decoder - typically observedin the VAE framework.
Lastly, once a good discrete latent structure of a modality is discovered by the VQ-VAE, we train a powerful prior over these discrete random variables, yielding interesting samples and useful applications. For instance, when trained on speech we discover the latent structure of language
Download Neural Discrete Representation Learning
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
Related search queries
Probability Theory, Random, Discrete Random, Probability, Joint, Random variable, 1 Discrete-time Markov chains, Columbia University, Discrete, RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS, Discrete-event simulation, Discrete, Binomial & Geometric Random Variables, Discrete uniform distribution from, Discrete uniform distribution, Continuous Random