Extracting and Composing Robust Features with Denoising ...

Extracting and Composing Robust Features with DenoisingAutoencodersPascal e de Montr eal, Dept. IRO, CP 6128, Succ. Centre-Ville, Montral, Qubec, H3C 3J7, CanadaAbstractPrevious work has shown that the difficul-ties in learning deep generative or discrim-inative models can be overcome by an ini-tial unsupervised learning step that maps in-puts to useful intermediate introduce and motivate a new trainingprinciple for unsupervised learning of a rep-resentation based on the idea of making thelearned representations Robust to partial cor-ruption of the input pattern. This approachcan be used to train autoencoders, and thesedenoising autoencoders can be stacked to ini-tialize deep architectures. The algorithm canbe motivated from a manifold learning andinformation theoretic perspective or from agenerative model perspective. Comparativeexperiments clearly show the surprising ad-vantage of corrupting the input of autoen-coders on a pattern classification IntroductionRecent theoretical studies indicate that deep architec-tures (Bengio & Le Cun, 2007; Bengio, 2007) may beneeded toefficientlymodel complex distributions andachieve better generalization performance on challeng-ing recognition tasks.

this purpose: Restricted Boltzmann Machines (RBMs) trained with contrastive divergence on one hand, and various types of autoencoders on the other. The present research begins with the question of what. Extracting and Composing Robust Features with Denoising Autoencoders

Loading..

Tags:

  Feature, With, Robust, Restricted, Extracting, Boltzmann, Composing, Restricted boltzmann, Autoencoder, Extracting and composing robust features with denoising, Denoising, Extracting and composing robust features with denoising autoencoders

Information

Related search queries