Learning to Reweight Examples for Robust Deep Learning

1 Learning to Reweight Examples for Robust Deep LearningMengye Ren1 2 Wenyuan Zeng1 2 Bin Yang1 2 Raquel Urtasun1 2 AbstractDeep neural networks have been shown to bevery powerful modeling tools for many supervisedlearning tasks involving complex input , they can also easily overfit to trainingset biases and label noises. In addition to variousregularizers, example reweighting algorithms arepopular solutions to these problems, but theyrequire careful tuning of additional hyperparam-eters, such as example mining schedules andregularization hyperparameters. In contrast topast reweighting methods, which typically consistof functions of the cost value of each example,in this work we propose a novel meta-learningalgorithm that learns to assign weights to trainingexamples based on their gradient directions.

2 Todetermine the example weights, our methodperforms a meta gradient descent step on thecurrent mini-batch example weights (which areinitialized from zero) to minimize the loss ona clean unbiased validation set. Our proposedmethod can be easily implemented on any typeof deep network, does not require any additionalhyperparameter tuning, and achieves impressiveperformance on class imbalance and corruptedlabel problems where only a small amount ofclean validation data is IntroductionDeep neural networks (DNNs) have been widely used formachine Learning applications due to their powerful capacityfor modeling complex input patterns. Despite their success,it has been shown that DNNs are prone to training setbiases, the training set is drawn from a joint distributionp(x,y)that is different from the distributionp(xv,yv)of theevaluation set.

3 This distribution mismatch could have many1 Uber Advanced Technologies Group, Toronto ON, CANADA2 Department of Computer Science, University of Toronto,Toronto ON, CANADA. Correspondence to: Mengye of the35thInternational Conference on MachineLearning, Stockholm, Sweden, PMLR 80, 2018. Copyright 2018by the author(s).different forms. Class imbalance in the training set is a verycommon example. In applications such as object detectionin the context of autonomous driving, the vast majorityof the training data is composed of standard vehicles butmodels also need to recognize rarely seen classes such asemergency vehicles or animals with very high will sometime lead to biased training models that donot perform well in popular type of training set bias is label train a reasonable supervised deep model, we ideallyneed a large dataset with high-quality labels, which requiremany passes of expensive human quality assurance (QA).

4 Although coarse labels are cheap and of high availability,the presence of noise will hurt the model performance, et al. (2017) has shown that a standard CNN canfit any ratio of label flipping noise in the training set andeventually leads to poor generalization set biases and misspecification can sometimes beaddressed with dataset resampling (Chawla et al., 2002), the correct proportion of labels to train a networkon, or more generally by assigning a weight to each exampleand minimizing a weighted training loss. The exampleweights are typically calculated based on the training loss,as in many classical algorithms such as AdaBoost (Freund& Schapire, 1997), hard negative mining (Malisiewicz et al.,2011), self-paced Learning (Kumar et al., 2010), and othermore recent work (Chang et al.)

5 , 2017; Jiang et al., 2017).However, there exist two contradicting ideas in trainingloss based approaches. In noisy label problems, we preferexamples with smaller training losses as they are morelikely to be clean images; yet in class imbalance problems,algorithms such as hard negative mining (Malisiewicz et al.,2011) prioritize Examples with higher training loss sincethey are more likely to be the minority class. In cases whenthe training set is both imbalanced and noisy, these existingmethods would have the wrong model assumptions. In fact,without a proper definition of an unbiased test set, solvingthe training set bias problem is inherently ill-defined. As themodel cannot distinguish the right from the wrong, strongerregularization can usually work surprisingly well in certainsynthetic noise settings.

6 Here we argue that in order to learngeneral forms of training set biases, it is necessary to havea small unbiased validation to guide training. It is actuallyLearning to Reweight Examples for Robust Deep Learningnot uncommon to construct a dataset with two parts - onerelatively small but very accurately labeled, and anothermassive but coarsely labeled. Coarse labels can come frominexpensive crowdsourcing services or weakly superviseddata (Cordts et al., 2016; Russakovsky et al., 2015; Chen &Gupta, 2015).Different from existing training loss based approaches, wefollow a meta- Learning paradigm and model the most basicassumption instead:the best example weighting shouldminimize the loss of a set of unbiased clean validationexamples that are consistent with the evaluation , validation is performed at the end of training,which can be prohibitively expensive if we treat the exampleweights as some hyperparameters to optimize; to circumventthis, we perform validation ateverytraining iteration todynamically determine the example weights of the currentbatch.

7 Towards this goal, we propose an online reweightingmethod that leverages an additional small validation setand adaptively assigns importance weights to Examples inevery iteration. We experiment with both class imbalanceand corrupted label problems and find that our approachsignificantly increases the robustness to training set Related WorkThe idea of weighting each training example has been wellstudied in the literature. Importance sampling (Kahn &Marshall, 1953), a classical method in statistics, assignsweights to samples in order to match one distribution toanother. Boosting algorithms such as AdaBoost (Freund &Schapire, 1997), select harder Examples to train subsequentclassifiers. Similarly, hard example mining (Malisiewiczet al., 2011), downsamples the majority class and exploitsthe most difficult Examples .

8 Focal loss (Lin et al., 2017) addsa soft weighting scheme that emphasizes harder Examples are not always preferred in the presenceof outliers and noise processes. Robust loss estimatorstypically downweigh Examples with high loss. In self-paced Learning (Kumar et al., 2010), example weightsare obtained through optimizing the weighted trainingloss encouraging Learning easier Examples first. In eachstep, the Learning algorithm jointly solves a mixed integerprogram that iterates optimizing over model parameters andbinary example weights. Various regularization terms onthe example weights have since been proposed to preventoverfitting and trivial solutions of assigning weights to be allzeros (Kumar et al., 2010; Ma et al., 2017; Jiang et al.)

9 , 2015).Wang et al. (2017) proposed a Bayesian method that infersthe example weights as latent variables. More recently,Jiang et al. (2017) proposed to use a meta- Learning LSTMto output the weights of the Examples based on the trainingloss. Reweighting Examples is also related to curriculumlearning (Bengio et al., 2009), where the model reweightsamong many available tasks. Similar to self-paced Learning ,typically it is beneficial to start with easier crucial advantage of reweighting Examples is Robust -ness against training set bias. There has also been amultitude of prior studies on class imbalance problems,including using dataset resampling (Chawla et al., 2002;Dong et al., 2017), cost-sensitive weighting (Ting, 2000;Khan et al.

10 , 2015), and structured margin based objectives(Huang et al., 2016). Meanwhile, the noisy label problemhas been thoroughly studied by the Learning theory commu-nity (Natarajan et al., 2013; Angluin & Laird, 1988) andpractical methods have also been proposed (Reed et al.,2014; Sukhbaatar & Fergus, 2014; Xiao et al., 2015; Azadiet al., 2016; Goldberger & Ben-Reuven, 2017; Li et al.,2017; Jiang et al., 2017; Vahdat, 2017; Hendrycks et al.,2018). In addition to corrupted data, Koh & Liang (2017);Mu noz-Gonz alez et al. (2017) demonstrate the possibilityof a dataset adversarial attack ( dataset poisoning).Our method improves the training objective through aweighted loss rather than an average loss and is an in-stantiation of meta- Learning (Thrun & Pratt, 1998; Lakeet al.