Transcription of A Simple Framework for Contrastive Learning of Visual ...
{{id}} {{{paragraph}}}
Example: bachelor of science
A Simple Framework for Contrastive Learning of Visual ...
A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen1 Simon Kornblith1 Mohammad Norouzi1 Geoffrey Hinton1 AbstractThis paper presentsSimCLR: a Simple frameworkfor Contrastive Learning of Visual simplify recently proposed Contrastive self-supervised Learning algorithms without requiringspecialized architectures or a memory bank. Inorder to understand what enables the contrastiveprediction tasks to learn useful representations,we systematically study the major components ofour Framework . We show that (1) composition ofdata augmentations plays a critical role in definingeffective predictive tasks, (2) introducing a learn-able nonlinear transformation between the repre-sentation and the Contrastive loss substantially im-proves the quality of the learned representations,and (3) Contrastive Learning benefits from largerbatch sizes and more training steps compared tosupervised Learning .
pretraining (learning encoder network f without labels) is done using the ImageNet ILSVRC-2012 dataset (Rus-sakovsky et al.,2015). Some additional pretraining experi-ments on CIFAR-10 (Krizhevsky & Hinton,2009) can be found in AppendixB.9. We also test the pretrained results on a wide range of datasets for transfer learning. To evalu-
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: