Transcription of A Simple Framework for Contrastive Learning of Visual ...
{{id}} {{{paragraph}}}
Example: tourism industry
A Simple Framework for Contrastive Learning of Visual ...
A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen1 Simon Kornblith1 Mohammad Norouzi1 Geoffrey Hinton1 AbstractThis paper presentsSimCLR: a Simple frameworkfor Contrastive Learning of Visual simplify recently proposed Contrastive self-supervised Learning algorithms without requiringspecialized architectures or a memory bank. Inorder to understand what enables the contrastiveprediction tasks to learn useful representations,we systematically study the major components ofour Framework . We show that (1) composition ofdata augmentations plays a critical role in definingeffective predictive tasks, (2) introducing a learn-able nonlinear transformation between the repre-sentation and the Contrastive loss substantially im-proves the quality of the learned representations,and (3) Contrastive Learning benefits from largerbatch sizes and more training steps compared tosupervised Learning . By combining these findings,we are able to considerably outperform previousmethods for self-supervised and semi-supervisedlearning on ImageNet.
We do not sample negative examples explicitly. Instead, given a positive pair, similar to (Chen et al.,2017), we treat the other 2(N 1) augmented examples within a minibatch as negative examples. Let sim(u;v) = u>v=kukkvkde-note the dot product between ‘ 2 normalized u and v (i.e. cosine similarity). Then the loss function for a positive pair
Loading..
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: