Bagging and Boosting - mit.edu

Bagging and Class 10, 13 March 2006 Sasha RakhlinPlan Bagging and sub-sampling methods Bias-Variance and stability for Bagging Boosting and correlations of machines Gradient descent view of boostingBagging (Bootstrap AGGregatING)Given a training setD={(x1, y1), ..(xn, yn)}, sampleTsets ofnelements fromD(with replacement)D1, D2, .. DT Tquasi replica training sets; train a machine on eachDi,i= 1, .., Tand obtain asequence ofToutputsf1(x), .. fT(x). Bagging (cont.)The final aggregate classifier can be for regression f(x) =T i=1fi(x),the average offifori= 1, .., T; for classification f(x) = sign(T i=1fi(x))or the majority vote f(x) = sign(T i=1sign(fi(x)))Variation I: Sub-sampling methods- Standard Bagging : each of theTsubsamples has sizenand created with Sub- Bagging : createTsubsamples of size only ( <n).

Plan † Bagging and sub-sampling methods † Bias-Variance and stability for bagging † Boosting and correlations of machines † Gradient descent view of boosting

Loading..

Tags:

  Bagging

Information

Related search queries