Gradient Boosted Feature Selection - Alice Zheng

Gradient Boosted Feature SelectionZhixiang (Eddie) Xu Washington University in Brookings Louis, HuangTsinghua University30 Shuangqing , Q. WeinbergerWashington University in Brookings Louis, X. Zheng GraphLab936 N. 34th St. Ste 208 Seattle, Feature Selection algorithm should ideally satisfy four con-ditions: reliably extract relevant features; be able to iden-tify non-linear Feature interactions; scale linearly with thenumber of features and dimensions; allow the incorpora-tion of known sparsity structure. In this work we propose anovel Feature Selection algorithm, Gradient Boosted FeatureSelection (GBFS), which satisfies all four of these require-ments. The algorithm is flexible, scalable, and surprisinglystraight-forward to implement as it is based on a modifi-cation of Gradient Boosted Trees. We evaluate GBFS onseveral real world data sets and show that it matches or out-performs other state of the art Feature Selection it scales to larger data set sizes and naturally allows fordomain-specific side and Subject [Information Storage and Retrieval]: Miscellaneous; [Pattern Recognition]: Design Methodology Fea-ture evaluation and selectionGeneral TermsLearningKeywordsFeature Selection ; Large-scale; Gradient boosting Work done

to existing nonlinear FS algorithms, its time and memory complexity scales as O(dn), where ddenotes the number of features dimensionality and nthe number of data points1, and is very fast in practice.

Loading..

Tags:

  Feature, Selection, Boosted, Derating, Gradient boosted feature selection

Information

Related search queries