Transcription of arXiv:1505.00387v2 [cs.LG] 3 Nov 2015
{{id}} {{{paragraph}}}
Example: tourism industry
arXiv:1505.00387v2 [cs.LG] 3 Nov 2015
highway NetworksRupesh Kumar urgen Swiss AI Lab IDSIAI stituto Dalle Molle di Studi sull Intelligenza ArtificialeUniversit`a della Svizzera italiana (USI)Scuola universitaria professionale della Svizzera italiana (SUPSI)Galleria 2, 6928 Manno-Lugano, SwitzerlandAbstractThere is plenty of theoretical and empirical evi-dence that depth of neural networks is a crucialingredient for their success. However, networktraining becomes more difficult with increasingdepth and training of very deep networks remainsan open problem. In this extended abstract, weintroduce a new architecture designed to easegradient-based training of very deep refer to networks with this architecture ashighway networks, since they allow unimpededinformation flow across several layers oninfor-mation highways.
zero-padding to ensure that the block state and transform gate feature maps are the same size as the input. 2.2. Training Deep Highway Networks For plain deep networks, training with SGD stalls at the beginning unless a specific weight initialization scheme is used such that the variance of the signals during forward
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: