arXiv:2109.05687v1 [cs.CL] 13 Sep 2021

1 Raise a Child in Large language Model:Towards Effective and Generalizable Fine-tuningRunxin Xu1 , Fuli Luo2 , Zhiyuan Zhang1, Chuanqi Tan2,Baobao Chang1 , Songfang Huang2 , Fei Huang21 Key Laboratory of Computational Linguistics, Peking University, MOE, China2 Alibaba pretrained language models extendfrom millions to billions of parameters. Thusthe need to fine-tune an extremely large pre-trained model with a limited training corpusarises in various downstream tasks. In this pa-per, we propose a straightforward yet effec-tive fine-tuning technique, CHILD-TUNING,which updates a subset of parameters (calledchild network)

2 Of large pretrained models viastrategically masking out the gradients of thenon-child network during on various downstreamtasks in GLUE benchmark show that CHILD-TUNING consistently outperforms the vanillafine-tuning score amongfour different pretrained models, and surpassesthe prior fine-tuning techniques Furthermore, empirical results ondomain transfer and task transfer show thatCHILD-TUNINGcan obtain better generaliza-tion performance by large IntroductionPretrained language Models (PLMs) have had aremarkable effect on the natural language process-ing (NLP) landscape recently (Devlin et al., 2019;Liu et al.)

3 , 2019; Clark et al., 2020). Pretraining andfine-tuning have become a new paradigm of NLP,dominating a large variety of its great success, how to adapt suchlarge-scale pretrained language models with mil-lions to billions of parameters to various scenarios,especially when the training data is limited, is stillchallenging. Due to the extremely large capac-ity and limited labeled data, conventional trans-fer learning tends toaggressivefine-tuning (Jianget al., 2020), resulting in: 1) degenerated resultson the test data due to overfitting (Devlin et al.,2019; Phang et al., 2018; Lee et al., 2020), and 2) Equal Contribution. Joint work between Alibaba andPeking University.

4 Corresponding authors.+=ForwardVanilla BackwardCHILD-TUNINGB ackward ! ! "+=Task-Freeor Task-DrivenGradients MaskPretrainedWeightsWeights at1-th Iteration ! ! "Figure 1: The illustration of :It forwards on the whole network while backwardingon a subset of network ( , child network).Right:To achieve this, a task-free or task-driven mask is per-formed on the gradients of the non-child network, re-setting them to zero (grey diagonal grids).poor generalization ability in transferring to out-of-domain data or other related tasks (Mahabadi et al.,2021; Aghajanyan et al., 2021).Preventing the fine-tuned models to deviate toomuch from the pretrained weights ( , with lessknowledge forgetting), is proved to be effective tomitigate the above challenges (Gouk et al.)

5 , 2020).For instance, RecAdam (Chen et al., 2020) intro-ducesL2distance penalty between the fine-tunedweights and their pretrained weights. In addition,Mixout (Lee et al., 2020) randomly replaces part ofthe model parameters with their pretrained weightsduring fine-tuning. The core idea behind them isto utilize the pretrained weights to regularize thefine-tuned this paper, we propose to mitigate the aggres-sive fine-tuning problem from a new on the observation that it is unnecessary toupdate all the parameters within the large-scalemodel during fine-tuning, we propose an effec-tive fine-tuning technique,CHILD-TUNING, whichstraightforwardly updates a subset of parameters(calledchildnetwork) via strategicallymaskingout the gradients of non-child network in the back-ward process, as illustrated in Figure 1.

6 Note [ ] 13 Sep 2021it is different from model pruning, since it still for-wards on the whole network, thus making the fulluse of knowledge hidden in the pretrained detail, we propose two variants,CHILD-TUNINGFandCHILD-TUNINGD, which respec-tively detect the child network in outthe child network in the absence of task data via aBernoulli distribution. It introduces noise to the fullgradients, playing a role of regularization, hencepreventing overfitting to small datasets and lead-ing to better generalization. Furthermore,CHILD-TUNINGD utilizes the downstream task data to de-tect the most task-related parameters as the childnetwork and freezes the parameters in non-childnetwork to their pretrained weights.

7 It decreases thehypothesis space of the model via a task-specificmask applied to the full gradients, helping to ef-fectively adapt the large-scale pretrained model tovarious tasks and meanwhile greatly maintain itsoriginal generalization extensive experiments on the GLUE benchmark show thatCHILD-TUNINGcan bemore excellent at fine-tuning different PLMs,with up score improvementon CoLA/RTE/MRPC/STS-B tasks compared tovanilla fine-tuning (Section. ). Moreover, itachieves better generalization ability in transferringto out-of-domain data and other related tasks (Sec-tion. ). Experimental results also demonstratethatCHILD-TUNING yields consistently greater im-provements than state-of-the-art fine-tuning meth-ods.

8 More importantly, sinceCHILD-TUNINGis orthogonal to these prior methods, integratingCHILD-TUNING with them can even lead to furtherimprovements (Section. ).In summary, our contributions are three-fold: We proposeCHILD-TUNING, a straightfor-ward yet effective fine-tuning technique thatonly updates the parameters in the child net-work. We explore to detect the child networkin both task-free and task-driven ways. CHILD-TUNINGcan effectively adapt thelarge-scale pretrained model to various down-stream scenarios, from in-domain to out-of-domain, and cross-task transfer learning. SinceCHILD-TUNINGis orthogonal to priorfine-tuning methods, integratingCHILD-TUNING with them can further boost the fine-tuning MethodologyTo better adapt large-scale pretrained languagemodel to various downstream tasks, we propose asimple yet effective fine-tuning technique,CHILD-TUNING.

9 We firstly introduce a gradient mask inthe backward process to achieve the aim of updat-ing a subset of parameters ( , child network),while still utilizing the knowledge of the wholelarge model in the forward process (Section ).Then, we explore two ways to detect the childnetwork ( , generate different gradient masks):CHILD-TUNINGF that are in a task-free way (Sec-tion ), andCHILD-TUNINGD that are in a task-driven way ( Section ). Overview of CHILD-TUNINGWe start the introduction ofCHILD-TUNING bygiving a general formulation of the back propaga-tion during the vanilla fine-tuning. We denote theparameters of the model at thet-th iteration aswt(w0refers to the pretrained weights).

10 The vanillafine-tuning computes the gradient of the lossL(wt)and then applies gradient descent to all parameters,which can be formulated as:wt+1=wt L(wt) wt(1)where L(wt) wtare the gradients corresponding tothe model parameterswt, is the learning backwardly computes thegradients of all trainable parameters like standardfine-tuning. However, the key difference is thatCHILD-TUNING determines a child networkCtatthet-th iteration, and only updates this part of pa-rameters. To achieve this, we firstly define a0-1mask that is the same-sized aswas follows:M(i)t={1,w(i)t Ct0,w(i)t/ Ct(2)whereM(i)tandw(i)tdenote thei-th element ofthe maskMtand parameterswtat thet-th trainingiteration, , we formally defineCHILD-TUNING tech-nique by simply replacing Eq.}