Dense Contrastive Learning for Self-Supervised Visual Pre ...
Dense Contrastive Learning for Self-Supervised Visual Pre-TrainingXinlong Wang1,Rufeng Zhang2,Chunhua Shen1*,Tao Kong3,Lei Li31The University of Adelaide, Australia2Tongji University, China3ByteDance AI LabAbstractTo date, most existing Self-Supervised Learning methodsare designed and optimized for image classification. Thesepre-trained models can be sub-optimal for Dense predictiontasks due to the discrepancy between image-level predic-tion and pixel-level prediction. To fill this gap, we aim todesign an effective, Dense Self-Supervised Learning methodthat directly works at the level of pixels (or local features)by taking into account the correspondence between localfeatures. We present Dense Contrastive Learning (DenseCL),which implements Self-Supervised Learning by optimizing apairwise Contrastive (dis)similarity loss at the pixel levelbetween two views of input to the baseline method MoCo-v2, our methodintroduces negligible computation overhead (only<1%slower), but demonstrates consistently superior perfor-mance when transferring to downstream Dense predictiontasks including object detection, semantic segmentation andinstance segmentation; and outperforms the state-of-the-artmethods by a large margin.
labeling, making it hard to collect data at a massive scale to pre-train a universal feature representation. Recently, unsupervised visual pre-training has attracted much research attention, which aims to learn a proper vi-sual representation from a large set of unlabeled images. A few methods [17, 2, 3, 14] show the effectiveness in down-
Download Dense Contrastive Learning for Self-Supervised Visual Pre ...
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: