PDF4PRO ⚡AMP

Modern search engine that looking for books and documents around the web

Example: stock market

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguisticsand the 11th International Joint Conference on Natural Language Processing, pages 4582 4597 August 1 6, 2021. 2021 Association for Computational Linguistics4582 prefix -Tuning: Optimizing Continuous Prompts for GenerationXiang Lisa LiStanford LiangStanford is the de facto way of leveraginglarge pretrained language models for down-stream tasks. However, fine-tuning modifiesall the language model parameters and there-fore necessitates storing a full copy for eachtask. In this paper, we propose prefix -tuning, alightweight alternative to fine-tuning for natu-ral language Generation tasks, which keeps lan-guage model parameters frozen and instead op-timizes a sequence ofcontinuous task-specificvectors, which we call theprefix. prefix -tuningdraws inspiration from prompting for languagemodels, allowing subsequent tokens to attendto this prefix as if it were virtual tokens.

ing. Consider the task of generating a textual de-scription of a data table, as shown in Figure1, where the task input is a linearized table (e.g., “name: Starbucks jtype: coffee shop”) and the out-put is a textual description (e.g., “Starbucks serves coffee.”). Prefix-tuning prepends a …

Loading..

Tags:

  Prefix

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Spam in document Broken preview Other abuse

Transcription of Prefix-Tuning: Optimizing Continuous Prompts for Generation

Related search queries