arXiv:1505.00387v2 [cs.LG] 3 Nov 2015

arxiv.org

zero-padding to ensure that the block state and transform gate feature maps are the same size as the input. 2.2. Training Deep Highway Networks For plain deep networks, training with SGD stalls at the beginning unless a specific weight initialization scheme is used such that the variance of the signals during forward

  Network, Highway, Padding, Highway networks