Dropout: A Simple Way to Prevent Neural Networks from …

Journal of Machine Learning Research 15 (2014) 1929-1958 Submitted 11/13; Published 6/14 Dropout: A Simple Way to Prevent Neural Networks fromOverfittingNitish of Computer ScienceUniversity of Toronto10 Kings College Road, Rm 3302 Toronto, Ontario, M5S 3G4, :Yoshua BengioAbstractDeep Neural nets with a large number of parameters are very powerful machine learningsystems. However, overfitting is a serious problem in such Networks . Large Networks are alsoslow to use, making it difficult to deal with overfitting by combining the predictions of manydifferent large Neural nets at test time. Dropout is a technique for addressing this key idea is to randomly drop units (along with their connections) from the neuralnetwork during training.

The choice of which units to drop is random. In the simplest case, each unit is retained with a xed probability pindependent of other units, where pcan be chosen using a validation set or can simply be set at 0:5, which seems to be close to optimal for a wide range of networks and tasks.

Tags:

  Form, Network, Simple, Choice, Prevent, Neural, Simple way to prevent neural networks from

Information

Related search queries