Generalized Additive Models

1 Generalized Additive ModelsOverviewMany nonparametric methods do not perform well when there are a large number of independent variables inthe model. The sparseness of data in this setting inflates the variance of the estimates. The problem of rapidlyincreasing variance for increasing dimensionality is sometimes referred to as the curse of dimensionality. Interpretability is another problem with nonparametric regression based on kernel and smoothing splineestimates (Hastie and Tibshirani 1990).To overcome these difficulties, Stone (1985) proposed Additive Models . These Models estimate an additiveapproximation to the multivariate regression function. The benefits of an Additive approximation are at leasttwofold. First, because each of the individual Additive terms is estimated using a univariate smoother, thecurse of dimensionality is avoided, at the cost of not being able to approximate universally.

2 Second, estimatesof the individual terms explain how the dependent variable changes with the corresponding and Tibshirani (1990) proposed Generalized Additive Models . These Models assume that the meanof the dependent variable depends on an Additive predictor through a nonlinear link function. Generalizedadditive Models permit the response probability distribution to be a member of the exponential family ofdistributions. Many widely used statistical Models belong to this general class, including Additive modelsfor Gaussian data, nonparametric logistic Models for binary data, and nonparametric log- linear Models forPoisson GAM MethodSuppose thatYis a response random variable andX1;:::;Xpis a set of predictor variables. A regressionprocedure can be viewed as a method of estimating how the value ofYdepends on the values ofX1;:::; standard linear regression model assumes that the expected value ofYhas a linear ;:::;Xp/D 0C 1X1C C pXpGiven a sample of values forYandX, estimates of 0; 1;:::; pare often obtained by the least Additive model generalizes the linear model by modeling the expected value of Y ;:::; ,iD1;:::;p, are smooth functions.

3 These functions are estimated in a nonparametric Additive Models extend traditional linear Models in another way, namely by allowing for a ;:::;Xp/and the expected value ofY. This amounts to allowing for an alternative distributionfor the underlying random variation besides just the normal distribution. Although Gaussian Models can beused in many statistical applications, there are types of problems for which they are not appropriate. Forexample, the normal distribution might not be adequate for modeling discrete responses such as counts orbounded responses such as Additive Models consist of a random component, an Additive component, and a link function thatrelates these two components to each other. The responseY, the random component, is assumed to have adensity in the exponential ; /Dexp y b.

4 / . ; / where is called the natural parameter and is the scale parameter. The normal, binomial, and Poissondistributions are all in this family. The quantity /;:::;sp. /are smooth functions defines the Additive component. Finally, the relationship betweenthe mean of the response variable and is defined by a link functiong. /D . The most commonly usedlink function is the canonical link, for which D .A combination of backfitting and local scoring algorithms is used in the actual fitting of the Additive Models and Generalized linear Models can be applied in similar situations, but they servedifferent analytic purposes. Generalized linear Models emphasize estimation and inference for the parametersof the model; Generalized Additive Models focus on exploring data software provides two procedures that fit Generalized Additive Models : the GAM procedure andthe GAMPL procedure.

5 The features, comparative differences, and an example that uses both procedures arepresented in the discussion that GAM ProcedureThe GAM procedure implements the Generalized Additive model proposed by Hastie and Tibshirani (1990).PROC GAM does the following: fits nonparametric or semiparametric Additive Models supports the use of multidimensional data supports multiple SCORE statementsThe GAMPL ProcedureF3 enables you to specify the model degrees of freedom or smoothing parameterPROC GAM can fit Gaussian, binomial, Poisson, gamma, and inverse Gaussian distributions. Althoughtheoretically more than one link can exist for each distribution, PROC GAM always uses the canonical is because the difference between link alternatives is less pronounced for nonparametric Models , as aresult of the flexibility of nonparametric model GAMPL ProcedureThe GAMPL procedure is a high-performance procedure that fits Generalized Additive Models that are basedon low-rank regression splines (Wood 2006).

6 Each spline term is constructed by the thin-plate regressionspline technique (Wood 2003). A roughness penalty is applied to each spline term by a smoothing parameterthat controls the balance between goodness of fit and the roughness of the spline curve. PROC GAMPL fits Models for standard distributions in the exponential family, such as binomial, gamma, Gaussian, inverseGaussian, Poisson, and negative binomial GAMPL runs in either single-machine mode or distributed :Distributed mode requiresSAS High-Performance GAMPL offers the following basic features: estimates the regression parameters of a Generalized Additive model that has fixed smoothing parametersby using penalized likelihood estimation estimates the smoothing parameters of a Generalized Additive model by using either the performanceiteration method or the outer iteration method estimates the regression parameters of a Generalized linear model by using maximum likelihoodtechniques tests the total contribution of each spline term based on the Wald statistic provides model-building syntax in the CLASS statement and effect-based parametric effects in theMODEL statement, which are used in other SAS/STAT analytic procedures (in particular, the GLM,LOGISTIC, GLIMMIX, and MIXED procedures)

7 Provides response-variable options enables you to construct a spline term by using multiple variables provides control options for constructing a spline term, such as fixed degrees of freedom, initialsmoothing parameter, fixed smoothing parameter, smoothing parameter search range, user-suppliedknot values, and so on provides multiple link functions for any distribution provides a WEIGHT statement for weighted analysis provides a FREQ statement for grouped analysis4F provides an OUTPUT statement to produce a data set that has predicted values and other observation-wise statistics produces graphs via ODS GraphicsThe GAMPL procedure is a high-performance analytical procedure. It does everything in the preceding list,plus it also does the following: enables you to run in distributed mode on a cluster of machines that distribute the data and thecomputations enables you to run in single-machine mode on the server where SAS is installed exploits all the available cores and concurrent threads, regardless of execution modePROC GAMPL Contrasted with PROC GAMBoth the GAMPL procedure and the GAM procedure in SAS/STAT software fit Generalized Additive , the GAMPL procedure uses different approaches for constructing spline basis expansions, fittinggeneralized Additive Models , and testing smoothing components.

8 The GAMPL procedure focuses onautomatic smoothing parameter selection by using global model-evaluation criteria to find optimal GAM procedure focuses on constructing Models by fitting partial residuals against each smoothing general, you should not expect similar results from the two procedures. The following sections summarizethe Spline Basis ExpansionsThe GAMPL procedure uses thin-plate regression splines to construct basis expansions for each spline term,and each term allows multiple variables. The GAM procedure uses univariate or bivariate smoothing splinesto construct basis expansions, and each term allows only one or two variables. The thin-plate regressionsplines that PROC GAMPL uses are low-rank approximations of multivariate smoothing splines.

9 PROCGAM also allows loess Generalized Additive ModelsThe GAMPL procedure fits a Generalized Additive model that has fixed smoothing parameters by usinga global design matrix and a roughness penalty matrix. The GAM procedure uses partial residuals to fitagainst single smoothing terms. For Models that have unknown smoothing parameters, PROC GAMPL estimates smoothing parameters simultaneously by optimizing global criteria such as Generalized crossvalidation (GCV) and the unbiased risk estimator (UBRE). PROC GAM estimates each smoothing parameterby optimizing the local criterion GCV one spline term at a Families and Link FunctionsF5 Distribution Families and Link FunctionsThe GAMPL procedure supports all the distribution families and all the link functions that the GAM proceduresupports.

10 In addition, PROC GAMPL fits Models in the negative binomial family. PROC GAMPL supportsany link function for each distribution, whereas PROC GAM supports only the canonical link for Smoothing ComponentsThe GAMPL procedure tests the total contribution for a spline term, including both linear and nonlineartrends. The GAM procedure tests the existence of nonlinearity for a spline term beyond the linear InferenceA global Bayesian posterior covariance matrix is available for Models that are fitted by the GAMPL confidence limits for prediction of each observation are available, in addition to componentwise confi-dence limits. For Generalized Additive Models that are fitted by the GAM procedure, only the componentwiseconfidence limits are available, and they are based on the partial residuals for each smoothing term.