Transcription of Learning to Reweight Examples for Robust Deep Learning
{{id}} {{{paragraph}}}
Example: bankruptcy
Learning to Reweight Examples for Robust Deep Learning
Learning to Reweight Examples for Robust Deep LearningMengye Ren1 2 Wenyuan Zeng1 2 Bin Yang1 2 Raquel Urtasun1 2 AbstractDeep neural networks have been shown to bevery powerful modeling tools for many supervisedlearning tasks involving complex input , they can also easily overfit to trainingset biases and label noises. In addition to variousregularizers, example reweighting algorithms arepopular solutions to these problems, but theyrequire careful tuning of additional hyperparam-eters, such as example mining schedules andregularization hyperparameters. In contrast topast reweighting methods, which typically consistof functions of the cost value of each example,in this work we propose a novel meta-learningalgorithm that learns to assign weights to trainingexamples based on their gradient directions.
objective towards an online approximation that can fit into any regular supervised training. We give a practical implementation suitable for any deep network type and provide theoretical guarantees under mild conditions that our algorithm has a convergence rate of O(1= 2). Note that this is the same as that of stochastic gradient descent (SGD ...
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: