Transcription of Understanding deep learning requires rethinking …
{{id}} {{{paragraph}}}
Example: dental hygienist
Understanding deep learning requires rethinking …
Understanding deep learning requires RE-THINKING GENERALIZATIONC hiyuan Zhang Massachusetts Institute of BengioGoogle HardtGoogle Recht University of California, VinyalsGoogle their massive size, successful deep artificial neural networks can exhibit aremarkably small difference between training and test performance. Conventionalwisdom attributes small generalization error either to properties of the model fam-ily, or to the regularization techniques used during extensive systematic experiments, we show how these traditional ap-proaches fail to explain why large neural networks generalize well in , our experiments establish that state-of-the-art convolutional networksfor image classification trained with stochastic gradient methods easily fit a ran-dom labeling of the training data.
pletely unstructured random noise. We corroborate these experimental ndings with a theoretical construction showing that simple depth two neural networks al-ready have perfect nite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice.
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: