PDF4PRO ⚡AMP

Modern search engine that looking for books and documents around the web

Example: stock market

Deep Sparse Recti er Neural Networks

315 Deep Sparse Rectifier Neural NetworksXavier GlorotAntoine BordesYoshua BengioDIRO, Universit e de Montr ealMontr eal, QC, UMR CNRS 6599 UTC, Compi`egne, FranceandDIRO, Universit e de Montr ealMontr eal, QC, Universit e de Montr ealMontr eal, QC, logistic sigmoid neurons are more bi-ologically plausible than hyperbolic tangentneurons, the latter work better for train-ing multi-layer Neural Networks . This pa-per shows that rectifying neurons are aneven better model of biological neurons andyield equal or better performance than hy-perbolic tangent Networks in spite of thehard non-linearity and non-differentiabilityat zero, creating Sparse representations withtrue zeros, which seem remarkably suitablefor naturally Sparse data.

tion regime. This is biologically implausible and hurts gradient-based optimization (LeCun et al., 1998; Bengio and Glorot, 2010). Important divergences between biological and machine learning models concern non-linear activation functions. A common biological model of neuron, the leaky integrate-and- re (or LIF) (Dayan and Abott, 2001), gives ...

Loading..

Tags:

  Biologically

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Spam in document Broken preview Other abuse

Transcription of Deep Sparse Recti er Neural Networks

Related search queries