Unsupervised Data Augmentation for Consistency ... - NeurIPS
Unsupervised Data Augmentation for Consistency Training Qizhe Xie 1, 2, Zihang Dai , Eduard Hovy , Minh-Thang Luong , Quoc V. Le1 1 Google Research, Brain Team, 2 Carnegie Mellon University {qizhex, dzihang, hovy}@cs.cmu.edu, {thangluong, qvl}@google.com Abstract Semi-supervised learning lately has shown much promise in improving deep learn-
Download Unsupervised Data Augmentation for Consistency ... - NeurIPS
Information
Domain:
Source:
Link to this page:
Documents from same domain
Generative Adversarial Imitation Learning
proceedings.neurips.ccnetworks [8], a technique from the deep learning community that has led to recent successes in modeling distributions of natural images: our algorithm harnesses generative adversarial training to fit distributions of states and actions defining expert behavior. We test our algorithm in Section 6, where
Prototypical Networks for Few-shot Learning
proceedings.neurips.cc˚: RD!RMwith learnable parameters ˚. Each prototype is the mean vector of the embedded support points belonging to its class: c k= 1 jS kj X (x i;y i)2S k f ˚(x i) (1) Given a distance function d: R M R 
Inductive Representation Learning on Large Graphs
proceedings.neurips.ccnode classification, clustering, and link prediction [11, 28, 35]. ... (e.g., citation data with text attributes, biological data with functional/molecular markers), our approach can also make use of structural features that are present in all graphs (e.g., node degrees). ... through theoretical analysis, that GraphSAGE is capable of learning ...
Bootstrap Your Own Latent A New Approach to Self ...
proceedings.neurips.ccmining strategies [14, 15] to retrieve the nega-tive pairs. In addition, their performance criti-cally depends on the choice of image augmenta- ... to prevent collapsing while preserving high performance. To prevent collapse, a straightforward solution …
Spatial Transformer Networks - NeurIPS
proceedings.neurips.ccConvolutional Neural Networks define an exceptionally powerful class of models, ... localisation, semantic segmentation, and action recognition tasks, amongst others. ... can take any form, such as a fully-connected network or a convolutional network, but should include a final regression layer to produce the transformation ...
Semi-supervised Learning with Deep Generative Models
proceedings.neurips.ccapproximately invariant to local perturbations along the manifold. The idea of manifold learning ... We show for the first time how variational inference can be brought to bear upon the prob- ... probabilities are formed by a non-linear transformation, with parameters , of a set of latent vari-ables z. This non-linear transformation is ...
Unsupervised Learning of Visual Features by Contrasting ...
proceedings.neurips.ccpseudo-labels to learn visual representations. This method scales to large uncurated dataset and can be used for pre-training of supervised networks [7]. However, their formulation is not principled and recently, Asano et al. [2] show how to cast the pseudo-label assignment problem as an instance of the optimal transport problem.
PyTorch: An Imperative Style, High-Performance Deep ...
proceedings.neurips.ccFacebook AI Research benoitsteiner@fb.com Lu Fang Facebook lufang@fb.com Junjie Bai Facebook jbai@fb.com Soumith Chintala Facebook AI Research soumith@gmail.com Abstract Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals
Visualizing the Loss Landscape of Neural Nets
proceedings.neurips.cctask that is hard in theory, but sometimes easy in practice. Despite the NP-hardness of training general neural loss functions [3], simple gradient methods often find global minimizers (parameter configurations with zero or near-zero training loss), even when data and labels are randomized before training [43].
InfoGAN: Interpretable Representation Learning by ...
proceedings.neurips.ccof the digit (0-9), and chose to have two additional continuous variables that represent the digit’s angle and thickness of the digit’s stroke. It would be useful if we could recover these concepts without any supervision, by simply specifying that an MNIST digit is generated by an 1-of-10 variable and two continuous variables.
Related documents
BERT: Pre-training of Deep Bidirectional Transformers for ...
aclanthology.orgpre-training for language representations. Un-likeRadford et al.(2018), which uses unidirec-tional language models for pre-training, BERT uses masked language models to enable pre-trained deep bidirectional representations. This is also in contrast toPeters et al.(2018a), which uses a shallow concatenation of independently
![arXiv:1408.5882v2 [cs.CL] 3 Sep 2014](/cache/preview/2/c/b/c/4/4/d/0/thumb-2cbc44d022447f6e8ab127ee8a6e3bd2.jpg)
arXiv:1408.5882v2 [cs.CL] 3 Sep 2014
arxiv.orgple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the ar-chitecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which
Understanding deep learning requires rethinking ... - arXiv
arxiv.orgremarkably small difference between training and test performance. Conventional ... In this work, we problematize the traditional view of generalization by showing that it is incapable ... standard architectures trained on the CIFAR10 and ImageNet classication benchmarks. While
![arXiv:2005.14165v4 [cs.CL] 22 Jul 2020](/cache/preview/4/5/0/8/0/d/2/2/thumb-45080d222facc6d76a53aa8ccbd3eddb.jpg)
arXiv:2005.14165v4 [cs.CL] 22 Jul 2020
arxiv.orgRecent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples.
REALM: Retrieval-Augmented Language Model Pre ... - Kenton L
kentonl.comthree popular Open-QA benchmarks, and find that we outperform all previous methods by a significant margin (4-16% absolute accuracy), while also providing qualitative benefits such as interpretability and modularity. 1. Introduction Recent advances in language model pre-training have shown that models such as BERT (Devlin et al.,2018),
Domain-Adversarial Training of Neural Networks
www.jmlr.orgresults on traditional deep learning image data sets|such as MNIST (LeCun et al., 1998) and SVHN (Netzer et al., 2011)|as well as on Office benchmarks (Saenko et al., 2010), where domain-adversarial learning allows obtaining a deep architecture that considerably improves over previous state-of-the-art accuracy.
Performance Monitoring Indicators - MEASURE Evaluation
www.measureevaluation.orgIndicator benchmarks and international comparators 19 ... Further, the task force pointed out that the Bank’s traditional method of appraisal and evaluation of development impact—the calculation of economic rate of return or ... training. indicators: C. performance. Performance.
Chapter 3 O&M Management - Energy
www1.eere.energy.govTraining, and Administration, form the basis for a solid O&M organization, the key lies in the ... Traditional thinking in the O&M ield focused on a single metric, reliability, for program ... tracking and trending metrics against industry benchmarks (NASA 2000). Table 3.1.1. Industry O&M metrics and benchmarks .