Leakage in Data Mining: Formulation, Detection, and …

Leakage in data mining : formulation , detection , and AvoidanceShachar Kaufman School of Electrical Engineering Tel-Aviv University 69978 Tel-Aviv, Israel Rosset School of Mathematical Sciences Tel-Aviv University 69978 Tel-Aviv, Israel Perlich Media6 Degrees 37 East 18th Street, 9th floor New York, NY 10003 ABSTRACT Deemed one of the top ten data mining mistakes , Leakage is essentially the introduction of information about the data mining target, which should not be legitimately available to mine from. In addition to our own industry experience with real-life projects, controversies around several major public data mining competi-tions held recently such as the INFORMS 2010 data mining Challenge and the IJCNN 2011 Social Network Challenge are evidence that this issue is as relevant today as it has ever been. While acknowledging the importance and prevalence of Leakage in both synthetic competitions and real-life data mining projects, existing literature has largely left this idea unexplored.

to each page-view record at the end of the session. A solution is to replace this attribute with "page number in session" which de-scribes the session length up to the current page, where prediction is required. Subsequent work by Kohavi . et al. [3] presents the common busi-ness analysis problem of characterizing big spenders among cus-tomers.

Tags:

  Data, Mining, Detection, Formulation, In data mining

Information

Related search queries