Supervised Contrastive Learning - NIPS
contrastive learning which uses only a single positive). These positives are drawn from samples of the same class as the anchor, rather than being data augmentations of the anchor, as done in self-supervised learning. While this is a simple extension to the self-supervised setup, it is non-
Download Supervised Contrastive Learning - NIPS
Information
Domain:
Source:
Link to this page:
Documents from same domain
On Discriminative vs. Generative Classifiers: A …
papers.nips.ccOn Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes Andrew Y. Ng Computer Science Division University of California, Berkeley
SAGA: A Fast Incremental Gradient Method With Support for ...
papers.nips.ccSAGA is preferred over SVRG both theoretically and in practice. For neural networks, where no theory is available for either method, the storage of gradients is generally more expensive than the
Thinking Fast and Slow with Deep Learning and Tree Search
papers.nips.ccSystem 1 is a fast, unconscious and automatic mode of thought, also known as intuition or heuristic process. System 2, an evolutionarily recent process unique to humans, is a slow, conscious, explicit
A Growing Neural Gas Network Learns Topologies
papers.nips.ccA Growing Neural Gas Network Learns Topologies 627 a) Delaunay triangulation b) induced Delaunay triangulation Figure 1: Two ways of defining closeness among a set of points.
Attention is All you Need - Neural Information Processing ...
papers.nips.ccAttention Is All You Need Ashish Vaswani Google Brain avaswani@google.com Noam Shazeer Google Brain noam@google.com Niki Parmar Google Research nikip@google.com
ImageNet Classification with Deep Convolutional Neural ...
papers.nips.ccChallenge, an annual competition called the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) has been held. ILSVRC uses a subset of ImageNet with roughly 1000 images in each of 1000 categories. In all, there are roughly 1.2 million training images, 50,000 validation images, and 150,000 testing images. ILSVRC-2010 is the only version ...
Generative Adversarial Nets - NIPS
papers.nips.ccGenerative adversarial networks has been sometimes confused with the related concept of “adversar-ial examples” [28]. Adversarial examples are examples found by using gradient-based optimization directly on the input to a classification network, in order to find examples that are similar to the data yet misclassified.
Time-series Generative Adversarial Networks
papers.nips.ccA good generative model for time-series data should preserve temporal dynamics, in the sense that new sequences respect the original relationships between variables across time. Existing methods that bring generative adversarial networks (GANs) into the sequential setting do not adequately attend to the temporal correlations unique to time ...
Hidden Technical Debt in Machine Learning Systems
papers.nips.ccaccount for in system design. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns. 1 Introduction As the machine learning (ML) community continues to accumulate years of experience with live
Character-level Convolutional Networks for Text Classification
papers.nips.ccApplying convolutional networks to text classification or natural language processing at large was explored in literature. It has been shown that ConvNets can be directly applied to distributed [6] [16] or discrete [13] embedding of words, without any knowledge on the syntactic or semantic structures of a language.
Related documents
Bootstrap Your Own Latent A New Approach to Self ...
arxiv.orgContrastive approaches avoid a costly generation step in pixel space by bringing representation of different views of the same image closer (‘positive pairs’), and spreading representations of views from different images (‘negative pairs’) apart [39, 40]. Contrastive methods often require
Mother-Tongue Interference in the Acquisition of English ...
files.eric.ed.govContrastive analysis is concerned with the study of a pair of languages with the aim of discovering their structural similarities and differences. Contrastive Analysis is a method that was widely used in the 1960s and early 1970s to explain why some features of a target language were more difficult to learn than others. (Mozlan, 2015)
Understanding Contrastive Representation Learning through ...
proceedings.mlr.pressThe term contrastive loss has also been generally used to refer to various objectives based on positive and negative samples, e.g., in Siamese networks (Chopra et al., 2005; Hadsell et al., 2006). In this work, we focus on the spe-cific form in Equation (1) that is widely used in modern unsupervised contrastive representation learning literature.
Dense Contrastive Learning for Self-Supervised Visual Pre ...
openaccess.thecvf.comcontrastive loss, which extends the conventional InfoNCE loss [29] to a dense paradigm. With the above approaches, we perform contrastive learning densely using a fully con-volutional network (FCN) [26], similar to target dense pre-diction tasks. Our main contributions are thus summarized as follows.
A Simple Framework for Contrastive Learning of Visual ...
proceedings.mlr.press• A contrastive loss function defined for a contrastive pre-diction task. Given a set fx~ kgincluding a positive pair of examples x~ iand x~ j, the contrastive prediction task aims to identify x~ jin fx~ kg k6=ifor a given x~ i. We randomly sample a minibatch of Nexamples and define the contrastive prediction task on pairs of augmented exam-
DetCo: Unsupervised Contrastive Learning for Object Detection
openaccess.thecvf.comcontrastive learning [5,19,5,3,18] currently achieved state-of-the-art performance, arousing extensive attention from researchers. Unlike generative methods, contrastive learning avoids the computation-consuming generation step by pulling representations of different views of the same im-age (i.e., positive pairs) close, and pushing representations
Dimensionality Reduction by Learning an Invariant Mapping
yann.lecun.comA contrastive loss function is employed to learn the param-eters W of a parameterizedfunction GW, in such a way that neighborsare pulled togetherand non-neighborsare pushed apart. Priorknowledgecan beused to identifythe neighbors for each training data point. The method uses an energy based model that uses the
Exploring Simple Siamese Representation Learning
arxiv.orgContrastive learning. The core idea of contrastive learn-ing [16] is to attract the positive sample pairs and repulse the negative sample pairs. This methodology has been recently popularized for un-/self-supervised representation learning [36,30,20,37,21,2,35,17,29,8,9]. Simple and effective instantiations of contrastive learning have been ...
Self-Prediction and Contrastive Learning
neurips.ccContrastive Learning: Inter-Sample Classification Given both similar (“positive”) and dissimilar (“negative”) candidates, to identify which ones are similar to the anchor data point is a classification task. There are creative ways to construct a set of data point candidates: 1. The original input and its distorted version