Transcription of Rectified Linear Units Improve Restricted Boltzmann Machines
{{id}} {{{paragraph}}}
Example: air traffic controller
Rectified Linear Units Improve Restricted Boltzmann Machines
Rectified Linear Units Improve Restricted Boltzmann MachinesVinod E. of Computer Science, University of Toronto, Toronto, ON M5S 2G4, CanadaAbstractRestricted Boltzmann Machines were devel-oped using binary stochastic hidden can be generalized by replacing eachbinary unit by an infinite number of copiesthat all have the same weights but have pro-gressively more negative biases. The learningand inference rules for these Stepped Sig-moid Units are unchanged. They can be ap-proximated efficiently by noisy, Rectified lin-ear Units . Compared with binary Units , theseunits learn features that are better for objectrecognition on the NORB dataset and faceverification on the Labeled Faces in the Wilddataset.
each feature detector to be 1 with probability p(hj = 1) = 1 1+exp(−bj − P i∈vis viwij) (2) where bj is the bias of j and vi is the binary state of pixel i. Once binary states have been chosen for the hidden units we produce a “reconstruction” of the training image by setting the state of each pixel to be 1 with probability p(vi = 1 ...
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: