The HPSPLIT Procedure - SAS

1 SAS/STAT User s GuideThe HPSPLIT ProcedureThis document is an individual chapter fromSAS/STAT User s correct bibliographic citation for this manual is as follows: SAS Institute Inc. User s Guide. Cary, NC:SAS Institute User s GuideCopyright 2015, SAS Institute Inc., Cary, NC, USAAll Rights Reserved. Produced in the United States of a hard-copy book: No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or byany means, electronic, mechanical, photocopying, or otherwise, without the prior written permission of the publisher, SAS a web download or e-book.

2 Your use of this publication shall be governed by the terms established by the vendor at the timeyou acquire this scanning, uploading, and distribution of this book via the Internet or any other means without the permission of the publisher isillegal and punishable by law. Please purchase only authorized electronic editions and do not participate in or encourage electronicpiracy of copyrighted materials. Your support of others rights is Government License Rights; Restricted Rights:The Software and its documentation is commercial computer softwaredeveloped at private expense and is provided with RESTRICTED RIGHTS to the United States Government.

3 Use, duplication, ordisclosure of the Software by the United States Government is subject to the license terms of this Agreement pursuant to, asapplicable, FAR , DFAR (a), DFAR (a), and DFAR , and, to the extent required under law, the minimum restricted rights as set out in FAR (DEC 2007). If FAR is applicable, this provisionserves as notice under clause (c) thereof and no other notice is required to be affixed to the Software or documentation. TheGovernment s rights in Software and documentation shall be only those set forth in this Institute Inc., SAS Campus Drive, Cary, NC 27513-2414 July 2015 SAS and all other SAS Institute Inc.

4 Product or service names are registered trademarks or trademarks of SAS Institute Inc. in theUSA and other countries. indicates USA brand and product names are trademarks of their respective 61 The HPSPLIT ProcedureContentsOverview: HPSPLIT Procedure ..4576 PROC HPSPLIT Features ..4577 Getting Started: HPSPLIT Procedure ..4578 Syntax: HPSPLIT Procedure ..4585 PROC HPSPLIT Statement ..4585 CLASS Statement ..4594 CODE Statement ..4595 GROW Statement ..4595ID Statement ..4597 MODEL Statement ..4597 OUTPUT Statement ..4598 PARTITION Statement ..4598 PERFORMANCE Statement.

5 4599 PRUNE Statement ..4600 RULES Statement ..4601 Details: HPSPLIT Procedure ..4602 Building a Decision Tree ..4602 Splitting Criteria ..4605 Splitting Strategy ..4607 Pruning ..4608 Memory Considerations ..4613 Primary and Surrogate Splitting Rules ..4613 Handling Missing Values ..4614 Unknown Values of Categorical Predictors ..4614 Scoring ..4615 Measures of Model Fit ..4616 Variable Importance ..4619 ODS Table Names ..4620 ODS Graphics ..4621 SAS Enterprise Miner Syntax and Notes ..4621 Examples: HPSPLIT Procedure ..4624 Example : Building a Classification Tree for a Binary Outcome.

6 4624 Example : Cost-Complexity Pruning with Cross Validation ..4636 Example : Creating a Regression Tree ..4641 Example : Creating a Binary Classification Tree with Validation Data ..4644 Example : Assessing Variable Importance ..46534576 FChapter 61: The HPSPLIT ProcedureExample : Applying Breiman s 1-SE Rule with Misclassification Rate ..4654 References ..4658 Overview: HPSPLIT ProcedureThe HPSPLIT Procedure is a high-performance Procedure that builds tree-based statistical models forclassification and regression. The Procedure produces classification trees, which model a categorical response,and regression trees, which model a continuous response.

7 Both types of trees are referred to as decision treesbecause the model is expressed as a series of if-then predictor variables for tree models can be categorical or continuous. The model is based on a partitionof the predictor space into nonoverlapping segments, which correspond to the terminal nodes or leaves ofthe tree. The partitioning is done recursively, starting with the root node, which contains all the data, andending with the terminal nodes. At each step of the recursion, the parent node is split into child nodes throughselection of a predictor variable and a split value that minimize the variability in the response across the models are built from training data for which the response values are known, and these models aresubsequently used to score (classify or predict) response values for new data.

8 For classification trees, themost frequent response level of the training observations in a leaf is used to classify observations in that regression trees, the average response of the training observations in a leaf is used to predict the responsefor observations in that leaf. The splitting rules that define the leaves provide the information that is neededto score new process of building a decision tree begins with growing a large, full tree. Various measures, such asthe Gini index, entropy, and residual sum of squares, are used to assess candidate splits for each node.

9 Thefull tree can overfit the training data, resulting in a model that does not adequately generalize to new prevent overfitting, the full tree is pruned back to a smaller subtree that balances the goals of fittingtraining data and predicting new data. Two commonly applied approaches for finding the best subtree arecost-complexity pruning (Breiman et al. 1984) and pruning (Quinlan 1993). For more information, seethe section Building a Decision Tree on page software provides many different methods of regression and classification. Compared with othermethods, an advantage of tree models is that they are easy to interpret and visualize, especially when the treeis small.

10 Tree-based methods scale well to large data, and they offer various methods of handling missingvalues, including surrogate , tree models have limitations. Regression tree models fit response surfaces that are constant overrectangular regions of the predictor space, and so they often lack the flexibility needed to capture smoothrelationships between the predictor variables and the response. Another limitation of tree models is that smallchanges in the data can lead to very different splits, and this undermines the interpretability of the model(Hastie, Tibshirani, and Friedman 2009; Kuhn and Johnson 2013).