Search results with tag "Neural net works"
Generating Sequences With Recurrent Neural Networks
arxiv.orgRecurrent Neural Networks Alex Graves Department of Computer Science University of Toronto graves@cs.toronto.edu Abstract This paper shows how Long Short-term Memory recurrent neural net-works can be used to generate complex sequences with long-range struc-ture, simply by predicting one data point at a time. The approach is
Frequency Principle: Fourier Analysis Sheds Light on Deep ...
ins.sjtu.edu.cnWe study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. We demonstrate a very universal Frequency Principle (F-Principle) — DNNs often fit target functions from low to high frequencies — on high-dimensional benchmark datasets such as MNIST/CIFAR10 and deep neural net-works such as VGG16.
High-Frequency Component Helps Explain the Generalization ...
openaccess.thecvf.comtion behaviors of neural networks, such as the capacity in memorizing label shuffled data [65] and the vulnerability towards adversarial examples [54, 21] To explain the generalization behaviors of neural net-works, many theoretical breakthroughs have been made progressively, including studying the properties of stochas-
ImageNet Classification with Deep Convolutional Neural ...
papers.nips.ccThe learning rates for each net-work were chosen independently to make train-ing as fast as possible. No regularization of any kind was employed. The magnitude of the ... Linear Units (ReLUs). Deep convolutional neural net-works with ReLUs train several times faster than their equivalents with tanh units. This is demonstrated in Figure 1, which ...
Sequence to Sequence Learning with Neural Networks
arxiv.orgsearch decoder. This is by far the best result achieved by direct translation with large neural net-works. For comparison,the BLEU score of an SMT baseline on this dataset is 33.30 [29]. The 34.81 BLEU score was achieved by an LSTM with a vocabulary of …