Dense Contrastive Learning for Self-Supervised Visual Pre ...
Dense Contrastive Learning for Self-Supervised Visual Pre-TrainingXinlong Wang1,Rufeng Zhang2,Chunhua Shen1*,Tao Kong3,Lei Li31The University of Adelaide, Australia2Tongji University, China3ByteDance AI LabAbstractTo date, most existing Self-Supervised Learning methodsare designed and optimized for image classification. Thesepre-trained models can be sub-optimal for Dense predictiontasks due to the discrepancy between image-level predic-tion and pixel-level prediction. To fill this gap, we aim todesign an effective, Dense Self-Supervised Learning methodthat directly works at the level of pixels (or local features)by taking into account the correspondence between localfeatures. We present Dense Contrastive Learning (DenseCL),which implements Self-Supervised Learning by optimizing apairwise Contrastive (dis)similarity loss at the pixel levelbetween two views of input to the baseline method MoCo-v2, our methodintroduces negligible computation overhead (only<1%slower), but demonstrates consistently superior perfor-mance when transferring to downstream Dense predictiontasks including object detection, semantic segmentation andinstance segmentation; and outperforms the state-of-the-artmethods by a large margin.
• We propose a new contrastive learning paradigm, i.e., dense contrastive learning, which performs dense pair-wise contrastive learning at the level of pixels (or local features). • With the proposed dense contrastive learning, we de-sign a simple and effective self-supervised learning method tailored for dense prediction tasks, termed
Download Dense Contrastive Learning for Self-Supervised Visual Pre ...
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: