Supervised Contrastive Learning - NIPS

← Back to document page

Supervised Contrastive LearningPrannay Khosla Google ResearchPiotr Teterwak Boston UniversityChen Wang Snap Sarna Google ResearchYonglong Tian MITPhillip Isola MITAaron MaschinotGoogle ResearchCe LiuGoogle ResearchDilip KrishnanGoogle ResearchAbstractContrastive Learning applied to self- Supervised representation Learning has seena resurgence in recent years, leading to state of the art performance in the unsu-pervised training of deep image models. Modern batch Contrastive approachessubsume or significantly outperform traditional Contrastive losses such as triplet,max-margin and the N-pairs loss. In this work, we extend the self-supervisedbatch Contrastive approach to thefully-supervisedsetting, allowing us to effec-tively leverage label information. Clusters of points belonging to the same classare pulled together in embedding space, while simultaneously pushing apart clus-ters of samples from different classes. We analyze two possible versions of thesupervised Contrastive (SupCon) loss, identifying the best-performing formula-tion of the loss.

contrastive learning which uses only a single positive). These positives are drawn from samples of the same class as the anchor, rather than being data augmentations of the anchor, as done in self-supervised learning. While this is a simple extension to the self-supervised setup, it is non-

  Contrastive