Search results with tag "Language understanding"
Improving Language Understanding by Generative Pre …
s3-us-west-2.amazonaws.comImproving Language Understanding by Generative Pre-Training Alec Radford OpenAI alec@openai.com Karthik Narasimhan OpenAI karthikn@openai.com Tim Salimans OpenAI tim@openai.com Ilya Sutskever OpenAI ilyasu@openai.com Abstract Natural language understanding comprises a wide range of diverse tasks such
BERT: Pre-training of Deep Bidirectional Transformers for ...
aclanthology.orgLanguage Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova Google AI Language fjacobdevlin,mingweichang,kentonl,kristoutg@google.com Abstract We introduce a new language representa-tion model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language repre-
![arXiv:2112.15283v1 [cs.CV] 31 Dec 2021](/cache/preview/c/8/9/2/1/9/c/e/thumb-c89219ce53d02972e27bd3f6ad3863e9.jpg)
arXiv:2112.15283v1 [cs.CV] 31 Dec 2021
arxiv.orgWhile most of VLP methods focus on pre-training for the vision-language understanding tasks, some works have noticed the importance of pre-training for improving the performance of cross-modal generation tasks, mainly image captioning task. Zhou et al. [7] propose the first unified model to improve the performance of both understanding and
Improving Language Understanding by Generative Pre …
cdn.openai.comdiscriminatively trained models to perform adequately. We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input