Extracting and Composing Robust Features with Denoising ...

Extracting and Composing Robust Features with DenoisingAutoencodersPascal e de Montr eal, Dept. IRO, CP 6128, Succ. Centre-Ville, Montral, Qubec, H3C 3J7, CanadaAbstractPrevious work has shown that the difficul-ties in learning deep generative or discrim-inative models can be overcome by an ini-tial unsupervised learning step that maps in-puts to useful intermediate introduce and motivate a new trainingprinciple for unsupervised learning of a rep-resentation based on the idea of making thelearned representations Robust to partial cor-ruption of the input pattern. This approachcan be used to train autoencoders, and thesedenoising autoencoders can be stacked to ini-tialize deep architectures. The algorithm canbe motivated from a manifold learning andinformation theoretic perspective or from agenerative model perspective. Comparativeexperiments clearly show the surprising ad-vantage of corrupting the input of autoen-coders on a pattern classification IntroductionRecent theoretical studies indicate that deep architec-tures (Bengio & Le Cun, 2007; Bengio, 2007) may beneeded toefficientlymodel complex distributions andachieve better generalization performance on challeng-ing recognition tasks.

coder we just described. We will now train it to recon-struct a clean “repaired” input from a corrupted, par-tially destroyed one. This is done by first corrupting the initial input x to get a partially destroyed version ˜x by means of a stochastic mapping x˜ ∼ q D(x˜|x). In our experiments, we considered the following corrupt-

Loading..

Tags:

  Feature, With, Decor, Robust, Clean, Extracting, Composing, Extracting and composing robust features with denoising, Denoising

Information

Related search queries