Batch Normalization: Accelerating Deep Network …
[ ] 2 Mar 2015Batch normalization : Accelerating deep Network training byReducing Internal Covariate ShiftSergey IoffeGoogle SzegedyGoogle deep Neural Networks is complicated by the factthat the distribution of each layer s inputs changes duringtraining, as the parameters of the previous layers slows down the training by requiring lower learningrates and careful parameter initialization, and makes it no-toriously hard to train models with saturating nonlineari-ties. We refer to this phenomenon asinternal covariateshift, and address the problem by normalizing layer in-puts. Our method draws its strength from making normal-ization a part of the model architecture and performing thenormalizationfor each training mini- Batch . Batch Nor-malization allows us to use much higher learning rates andbe less careful about initialization.
arXiv:1502.03167v3 [cs.LG] 2 Mar 2015 Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe
Download Batch Normalization: Accelerating Deep Network …
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: