Transcription of 勾配ブースティング Gradient Boosting
1 0 Gradient Boosting Gradient Boosting Decision Tree (GBDT) ( ) 1 : (Loss Function) [ ]2 L(y(i), f(x(i)))( )( )()()( )( )()()21,2iiiiL yfyf= xx y(i): i x(i): i f: f(x(i)) : i ( )( )()()( )( )(),iiiiL yfyf= xx : (Loss Function) [ ]3 L(y(i), f(x(i))) y(i): i x(i): i f: f(x(i)) : i ( )( )()(),,1,lnKiij kj kkL yfpp== xAdaboost w(i)Adaboost w(i) K: pj,k: i j k : (Loss Function) [ ]4 L(y(i), f(x(i)))( )( )()()( )(),logiiicL yfp= xx( )()( )()()( )()()1expexpikikKijjfpf== xxxK K y (1 or 1) K ( 1 1)c: y(i) k fk : [ ]5 : L(y(i), f(x(i))) f(x(i)) ( )( )()()( )( )()()21,2iiiiL yfyf= xx( )( )()()( )()( )( )(),iiiiiL yfyff = xxx ) : [ ]6( )( )()()( )()( )(),iiikikL yfqpf = xxx : L(y(i), f(x(i))) f(x(i)) ( )( )()()( )(),logiiimL yfp= xx )k q: k y(i) q = 1, q= 0 f (0) m= 1, 2.
2 , M i y Rm,j(j ) m,j f (m) ( )f (m)(x(i)) = f (m-1)(x(i)) + (x(i) Rm,j m,j)7( )()( )()()( ),1,,im jimim jRL yf + xx scikit-learn Gradient Boosting : : XGBoost (eXtreme Gradient Boosting ) LightGBM (Light Gradient Boosting Model) XGBoost, LightGBM 8 XGBoost 9 E +wT: w: ( y ) , : m m-1 LightGBM Gradient -based One-Side Sampling (GOSS) m a 100 % b 100 % (1-a)/b Exclusive Feature Bundling (EFB) 0 bundle 0 10 A 0 20 B B 10 A B bundle bundle 10 Python XGBoost LightGBM (Anaconda ) XGBoost: LightGBM: , method_flag , optuna optuna optuna: optuna.
3 X ( ), y ( ) (19, 20 ) 12 A. Natekin, A. Knoll, Gradient Boosting machines, a tutorial,Front. Neurobot., 7, 1-21, 2013 T. Hastie, R. Tibshirani, J. Friedman, The Elements of StatisticalLearning: Data mining, Inference, and Prediction, Second Edition,Springer, 2009 ~hastie/ElemStatLearn// T. Chen, C. Guestrin, XGBoost: A Scalable Tree Boosting System, , 2016 G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T. Y. Liu, LightGBM: A Highly Efficient Gradient Boosting Decision Tree,NIPS Proceedings, 2017
