Dense Contrastive Learning for Self-Supervised Visual Pre ...
Dense Contrastive Learning for Self-Supervised Visual Pre-TrainingXinlong Wang1,Rufeng Zhang2,Chunhua Shen1*,Tao Kong3,Lei Li31The University of Adelaide, Australia2Tongji University, China3ByteDance AI LabAbstractTo date, most existing Self-Supervised Learning methodsare designed and optimized for image classification. Thesepre-trained models can be sub-optimal for Dense predictiontasks due to the discrepancy between image-level predic-tion and pixel-level prediction. To fill this gap, we aim todesign an effective, Dense Self-Supervised Learning methodthat directly works at the level of pixels (or local features)by taking into account the correspondence between localfeatures. We present Dense Contrastive Learning (DenseCL),which implements Self-Supervised Learning by optimizing apairwise Contrastive (dis)similarity loss at the pixel levelbetween two views of input to the baseline method MoCo-v2, our methodintroduces negligible computation overhead (only<1%slower), but demonstrates consistently superior perfor-m
contrastive loss, which extends the conventional InfoNCE loss [29] to a dense paradigm. With the above approaches, we perform contrastive learning densely using a fully con-volutional network (FCN) [26], similar to target dense pre-diction tasks. Our main contributions are thus summarized as follows.
Download Dense Contrastive Learning for Self-Supervised Visual Pre ...
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: