GLM Residuals and Diagnostics - MyWeb

1 Building blocksDiagnosticsSummaryGLM Residuals and DiagnosticsPatrick BrehenyMarch 26 Patrick BrehenyBST 760: Advanced Regression1/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixIntroductionAfter a model has been fit, it is wise to check the model tosee how well it fits the dataIn linear regression, these Diagnostics were build aroundresiduals and the residual sum of squaresIn logistic regression (and all generalized linear models), thereare a few different kinds of Residuals (and thus, differentequivalents to the residual sum of squares)Patrick BrehenyBST 760.

2 Advanced Regression2/24 Building blocksDiagnosticsSummaryResidualsThe hat matrix The 2testBefore moving on, it is worth noting that both SAS andRreport by default a 2test associated with the entire modelThis is a likelihood ratio test of the model compared to theintercept-only (null) model, similar to the overallFtest inlinear regressionThis test is sometimes used to justify the modelHowever, this is a mistakePatrick BrehenyBST 760: Advanced Regression3/24 Building blocksDiagnosticsSummaryResidualsThe hat matrix The 2test (cont d)Just like all model-based inference, the likelihood ratio test isjustified under the assumption that the model holdsThus, theFtest takes the model as given and cannotpossibly be a test of the validity of the modelThe only thing one can conclude from a significant overall 2test is that, if the model is true, some of its coefficients arenonzero (is this helpful?)

3 Addressing the validity and stability of a model is much morecomplicated and nuanced than a simple test, and it is herethat we now turn our attentionPatrick BrehenyBST 760: Advanced Regression4/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixPearson residualsThe first kind is called thePearson residual , and is based onthe idea of subtracting off the mean and dividing by thestandard deviationFor a logistic regression model,ri=yi i i(1 i)Note that if we replace iwith i, thenrihas mean 0 andvariance 1 Patrick BrehenyBST 760: Advanced Regression5/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixDeviance residualsThe other approach is based on the contribution of each pointto the likelihoodFor logistic regression,`= i{yilog i+ (1 yi) log(1 i)}By analogy with linear regression, the terms should correspondto 12r2i.

4 This suggests the following residual , called thedeviance residual :di=si 2{yilog i+ (1 yi) log(1 i)},wheresi= 1ifyi= 1andsi= 1ifyi= 0 Patrick BrehenyBST 760: Advanced Regression6/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixDeviance and Pearson s statisticEach of these types of Residuals can be squared and addedtogether to create anRSS-like statisticCombining the deviance Residuals produces thedeviance:D= d2iwhich is, in other words, 2`Combining the Pearson Residuals produces thePearsonstatistic:X2= r2iPatrick BrehenyBST 760: Advanced Regression7/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixGoodness of fit testsIn principle, both statistics could be compared to the 2n pdistribution as a rough goodness of fit testHowever, this test does not actually work very wellSeveral modifications have been proposed, including an earlytest proposed by Hosmer and Lemeshow that remains popularand is available in SASO ther, better tests have been proposed as well (an extensivecomparison was made by Hosmeret al.)

5 (1997))Patrick BrehenyBST 760: Advanced Regression8/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixThe hat matrix for GLMsAs you may recall, in linear regression it was important todivide by 1 Hiito account for the leverage that a pointhad over its own fitSimilar steps can be taken for logistic regression; here, theprojection matrix isH=W1/2X(XTWX) 1 XTW1/2,whereW1/2is the diagonal matrix withW1/2ii= wiPatrick BrehenyBST 760: Advanced Regression9/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixProperties of the hat matrixIn logistic regression, 6=Hy no matrix can satisfy thisrequirement, as logistic regression does not produce linearestimatesHowever, it has many of the other properties that weassociate with the linear regression projection matrix.

6 Hr=0 His symmetricHis idempotentHW1/2X=W1/2 XandXTW1/2H=XTW1/2whereris the vector of Pearson residualsPatrick BrehenyBST 760: Advanced Regression10/24 Building blocksDiagnosticsSummaryResidualsThe hat matrixStandardized residualsThe diagonal elements ofHare again referred to as theleverages, and used to standardize the Residuals :rsi=ri 1 Hiidsi=di 1 HiiGenerally speaking, the standardized deviance Residuals tendto be preferable because they are more symmetric than thestandardized Pearson Residuals , but both are commonly usedPatrick BrehenyBST 760.

7 Advanced Regression11/24 Building blocksDiagnosticsSummaryDefinitionsPlots Leave-one-out diagnosticsYou may recall that in linear regression there were a numberof diagnostic measures based on the idea of leavingobservationiout, refitting the model, and seeing how variousthings changed ( Residuals , coefficient estimates, fitted values)You may also recall that for linear regression, it was notactually necessary to refit the modelntimes; explicitshortcuts based onHwere availableThe same idea can be extended to generalized linear models,although we cannot take advantage of the explicit-solutionshortcuts without making approximationsPatrick BrehenyBST 760.

8 Advanced Regression12/24 Building blocksDiagnosticsSummaryDefinitionsPlots One-step approximationsThe resulting approximate statistics are said to beone-stepapproximationsto the true valuesThe issue is that we can quickly calculate the one-stepapproximations based on the current weights{wi}withoutrefitting anything, but to calculate the exact value, we wouldneed to go throughnIRLS algorithmsThe approximations are usually pretty good, although if onepoint has a very large influence, then the approximation maybe quite different from the true valuePatrick BrehenyBST 760: Advanced Regression13/24 Building blocksDiagnosticsSummaryDefinitionsPlots One-step approximationsOne-step approximations allow us to quickly calculate the followingdiagnostic statistics for GLMs:Studentized deleted Residuals (for assessing the change in individual coefficients)Cook s distance (for assessing overall influence over the modelfit)Patrick BrehenyBST 760.

9 Advanced Regression14/24 Building blocksDiagnosticsSummaryDefinitionsPlots Variance inflation factorsIt is worth mentioning variance inflation factors (VIF) brieflyhereVIF is a function ofXalone, and therefore how VIF iscalculated and what it means is essentially equivalent to thelinear regression case ( essentially equivalent because we dohave weights for GLMs)InR, we can use theviffunction from thecarpackage:> vif(fit)Age Sex SAS, this is a bit painful, as we have to usePROC REG,which doesn t support theCLASS statement or interactions intheMODEL statement, and you have to calculate andincorporate the weights manually (see code for the messydetails)Patrick BrehenyBST 760: Advanced Regression15/24 Building blocksDiagnosticsSummaryDefinitionsPlots MulticollinearityIf you believe multicollinearity to be a problem, it is often agood idea to look at the correlation matrix forX.

10 Cor( (fit)[,-1]) this model, we are certainly introducing a lot of variabilityby including an interaction; on the other hand, the interactiondid seem to be importantp= BrehenyBST 760: Advanced Regression16/24 Building blocksDiagnosticsSummaryDefinitionsPlots LeverageTo get a sense of the information these statistics convey, let s lookat various plots of the Donner party data, starting with BrehenyBST 760: Advanced Regression17/24 Building blocksDiagnosticsSummaryDefinitionsPlots Cook s DistanceAgeCook's BrehenyBST 760: Advanced Regression18/24 Building blocksDiagnosticsSummaryDefinitionsPlots Delta-beta (for effect of age)Age BrehenyBST 760: Advanced Regression19/24 Building blocksDiagnosticsSummaryDefinitionsPlots Residuals / proportional 2 101 d(Studentized deleted) d2(Studentized deleted)llDiedSurvivedPatrick BrehenyBST 760: Advanced Regression20/24 Building blocksDiagnosticsSummarySummaryResiduals are certainly less informative for logistic regressionthan th