Simple Logistic Regression.notes - mc.vanderbilt.edu

1 1 III. INTRODUCTION TO Logistic REGRESSION1. Simple Logistic Regressiona) Example: APACHE II Score and Mortality in SepsisThe following figure shows 30 day mortality in a sample of septic patients as a function of their baseline APACHE II Score. Patients are coded as 1 or 0 depending on whether they are dead or alive in 30 days, II Score at BaselineDiedSurvived30 Day Mortality in Patients with SepsisWe wish to predict death from baseline APACHE II score in these (x) be the probability that a patient with score xwill that linear regression would not work well here since it could produce probabilities less than zero or greater than II Score at BaselineDiedSurvived30 Day Mortality in Patients with Sepsis2b)

2 Sigmoidal family of Logistic regression curvesLogistic regressionfits probability functions of the following form:pab ab() exp()/( exp())xxx=+ ++1 This equation describes a family of sigmoidal curves, three examples of which are given below.()ab+ x0x - p()/()x +=010 0 For negative values of x, exp asand (x)c) Parameter values and the shape of the regression curvepab ab() exp()/(exp())xxx=+ ++1 For now assume that > very large values of x, and henceexp()ab+ xp()()x + = (x)pab ab() exp()/(exp())xxx=+ ++1xx=-+ =ab a b/,0p().

3 X=+=11 1 05bgWhen and henceThe slope of (x) when (x)=.5 is controls how fast (x) rises from 0 to given , controls were the 50% survival point is located. Data with a lengthy transition from survival to death should have a low value of . 010510152025303540xDiedSurvivedWe wish to choose the best curve to fit the that has a sharp survival cut off point between patients who live or die should have a large value of .4pab ab() exp()/(exp())xxx=+ ++11-=p()xd) The probability of death under the Logistic modelThis probability is{ }Hence probability of survival =++-+++11exp() exp()exp()abababxxxlog( ( ) (( ))ppabxx x1-=+The log odds of death equals{ }, and the odds of death ispp ab( ) (( )) exp()xxx1-= +=+ +11(exp())abxe) The logit functionFor any number between 0 and 1 the logit function is defined bylogit() log( /())ppp=-1 Let di=xibe the APACHE II score of theithpatient10.

4 Patient dies patient livesththiiRST() () Pr[ 1]ii iEdxd=p ==Then the expected value of diis Thus we can rewrite thelogistic regression equation{ } as{ }logit( ( ))( )iiiEdxx=p =a+b52. Contrast Between Logistic and Linear RegressionIn linear regression , the expected value ofyigiven xiisforEyxii()=+abin=12, ,..,ab+xiis the linear is the random component of the model, which has a normal a normal distribution with standard deviation .In Logistic regression , the expected value of given xiis E(di) = idlogit(E(di)) = + xi for i= 1, 2.

5 , n[]iixp=pidis dichotomous with probability of event []iixp=pit is the random component of the modellogit is the link function that relates the expected value of the random component to the linear Maximum Likelihood EstimationIn linear regression we used the method of least squares to estimate regression coefficients. In generalized linear models we use another approach called maximum likelihood estimation. The maximum likelihood estimate of a parameter is that value that maximizes the probability of the observed estimate and by those values and that maximize the probability of the observed data under the Logistic regression model.

6 A b6 Bas e line APACHE II ScoreNu mb e r of PatientsNu mb e r of De ath sBas eline APACHE II S c o reNu mb e r of Patie ntsNu mb e r of De ath s010 20136210 21179341 2214124 11 0 23 13 7593 241186 14 3 25 12 87124 26628225 27759333 283110 19 6 29 7 411 31 5 30 5 412 17 5 31 3 313 32 13 32 3 314 25 7 33 1 115 18 7 34 1 116 24 8 35 1 117 27 8 36 1 118 19 13 37 1 119 15 7 41 1 0 This data is analyzed as tabulate fateDeath by 30 |days | Freq.

7 Percent +-----------------------------------aliv e | 279 | 175 +-----------------------------------Tota l | 454 histogram apache [fweight=freq ], discrete(start=0, width=1) Score at Baseline. scatter proportion of Deaths with Indicated Score010203040 APACHE Score at Baseline8 Logit estimates Number of obs = 454LR chi2(1) = > chi2 = likelihood = Pseudo R2 = | Coef.

8 Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------- ---------------------------------------- -------apache | .1156272 .0159997 .0842684 .146986_cons| .2765283 a b= .1156272= ab() exp()/( exp())xxx=+ ++1exp( + .1156272 )1 exp( + .1156272 )xx=+()exp( + .1156272 20) exp( + .1156272 20) p==+ logit( ( ))( )iiiEdxx=p =a+ of Death by 30 days010203040 APACHE Score at BaselineProportion of Deaths with Indicated ScorePr(fate)94.

9 Odds Ratios and the Logistic regression Modela) Odds ratio associated with a unit increase in xThe log odds that patients with APACHE II scores of xand x+ 1 will die arelogit{ }(())pabxx=+(())()pababbxxx+=+ +=++11andlogit{ } { } from { } gives(())(())ppxx+-1logit = logitlog())()log()()ppppxxxx+-+FHGIKJ--F HGIKJ111 1=and henceexp( ) is the odds ratio for death associated with a unit increase in x.( ())( ( ))ppxx+-1logit = logitpppp()/(())()/(())xxxx+-+-FHGIKJ111 1= logA property of Logistic regression is that this ratio remains constant for all values of x.

10 105. 95% Confidence Intervals for Odds Ratio EstimatesIn our sepsis example the parameter estimate for apache( ) was .1156272 with a standard error of .0159997. Therefore, the odds ratio for death associated with a unit rise in APACHE II score is exp(.1156272) = a 95% confidence interval of (exp( - ), exp( + )) = ( , ).6. Quality of Model fitIf our model is correct then() logit observed proportionix=a+bIt can be helpful to plot the observed log odds against ixa+b11-2-10123 Log odds of Death in 30 days010203040 APACHE Score at Baselineobs_logoddsLinear predictionThen it can be shown that the standard error of is= sex a+b 222 2xxaabbs+ s + s7.