Transcription of Prefix-Tuning: Optimizing Continuous Prompts for Generation
{{id}} {{{paragraph}}}
Example: confidence
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Proceedings of the 59th Annual Meeting of the Association for Computational Linguisticsand the 11th International Joint Conference on Natural Language Processing, pages 4582 4597 August 1 6, 2021. 2021 Association for Computational Linguistics4582 Prefix- tuning : Optimizing Continuous Prompts for GenerationXiang Lisa LiStanford LiangStanford is the de facto way of leveraginglarge pretrained language models for down-stream tasks. However, fine- tuning modifiesall the language model parameters and there-fore necessitates storing a full copy for eachtask. In this paper, we propose prefix- tuning , alightweight alternative to fine- tuning for natu-ral language Generation tasks, which keeps lan-guage model parameters frozen and instead op-timizes a sequence ofcontinuous task-specificvectors, which we call theprefix. Prefix-tuningdraws inspiration from prompting for languagemodels, allowing subsequent tokens to attendto this prefix as if it were virtual tokens.
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Stanford University xlisali@stanford.edu Percy Liang Stanford University pliang@cs.stanford.edu Abstract Fine-tuning is the de facto way of leveraging large pretrained language models for down-stream tasks. However, fine-tuning modifies
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: