Transcription of A Simple Framework for Contrastive Learning of Visual ...
{{id}} {{{paragraph}}}
Example: barber
A Simple Framework for Contrastive Learning of Visual ...
A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen1 Simon Kornblith1 Mohammad Norouzi1 Geoffrey Hinton1 AbstractThis paper presentsSimCLR: a Simple frameworkfor Contrastive Learning of Visual simplify recently proposed Contrastive self-supervised Learning algorithms without requiringspecialized architectures or a memory bank. Inorder to understand what enables the contrastiveprediction tasks to learn useful representations,we systematically study the major components ofour Framework . We show that (1) composition ofdata augmentations plays a critical role in definingeffective predictive tasks, (2) introducing a learn-able nonlinear transformation between the repre-sentation and the Contrastive loss substantially im-proves the quality of the learned representations,and (3) Contrastive Learning b
diction accuracy without improving representations. We ad-dress this issue by aggregating BN mean and variance over all devices during the training. Other approaches include shuffling data examples across devices (He et al.,2019), or replacing BN with layer norm (Hénaff et al.,2019). 2With 128 TPU v3 cores, it takes ˘1.5 hours to train our
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: