Transcription of Extracting and Composing Robust Features with Denoising ...
{{id}} {{{paragraph}}}
Example: dental hygienist
Extracting and Composing Robust Features with Denoising ...
Extracting and Composing Robust Features withDenoising AutoencodersPascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine ManzagolDept. IRO, Universit e de Montr 6128, Montreal, Qc, H3C 3J7, lisaTechnical Report 1316, February 2008 AbstractPrevious work has shown that the difficulties in learning deep genera-tive or discriminative models can be overcome by an initial unsupervisedlearning step that maps inputs to useful intermediate representations. Weintroduce and motivate a new training principle for unsupervised learningof a representation based on the idea of making the learned representa-tions Robust to partial corruption of the input pattern. This approach canbe used to train autoencoders, and these Denoising autoencoders can bestacked to initialize deep architectures. The algorithm can be motivatedfrom a manifold learning and information theoretic perspective or from agenerative model perspective.
IH(x,z) is a negative log-likelihood for the example x, given the Bernoulli parameters z. Equation 1 with L = L IH can be written θ?,θ0? = argmin θ,θ0 EE q0(X) [L IH (X,g θ0(f θ(X)))] (3) where q0(X) denotes the empirical distribution associated to our n training inputs. This optimization will typically be carried out by stochastic ...
Loading..
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: