Logistic Regression: Univariate and Multivariate

1 Logistic regression : Univariate andMultivariate1 Events and Logistic RegressionILogisitic regression is used for modelling of an event: Mrs. Smith had a myocardialinfarction between 1/1/2000 and 31/12 occurrence of an event is a binary (dichotomous)variable. There are two possibilities: the event occurs or itdoes not this reason, event occurrence variables can always becoded with 0, 1 personibecame pregnant in personidid not become pregnant in the Probability of an EventIThere are many equivalent ways of measuring theprobability of an will use three.

2 1probability of the event2odds in favour of the event3log-odds in favour of the eventIThese are equivalent in the sense that if you know thevalue of one measure for an event you can compute thevalue of the other two measures for the same a distance in kilometres, statute miles ornautical miles3 The Probability of an EventIThis is a number between 0 and 1. We write =P(Y=1)to mean is the probability thatY= =1 means we know the event is certain to =0 means we know the event is certainnotto between 0 and 1 represent intermediate states ofcertainty, ordered we are certain one ofY=1 andY=0 is true andbecause they cannot be true simultaneously:P(Y=0) =1 P(Y=1) =1.

3 4 Odds in Favour of an EventIThe odds in favour of an event is defined as theprobability the event occurs divided by the probability theevent does not odds in favour ofY=1 is defined as:ODDS(Y=1) =P(Y=1)P(Y6=1)=P(Y=1)P(Y=0)= 1 .INote:ODDS(Y=0) =1 ODDS(Y=1)=1 .soODDS(Y=1) ODDS(Y=0) = the Odds in Favour of an EventIAn odds is a number between 0 and .IAn odds of 0 means we are certain the event increased odds corresponds to increased belief in theoccurrence of the odds of 1 corresponds to a probability of 1 odds of corresponds to certainty the event in Favour of an EventIThe log odds in favour of an event is defined as the log ofthe odds in favour of the event:log ODDS(Y=1) =logP(Y=1)P(Y=0)=log 1.

4 INotelog ODDS(Y=1) = log ODDS(Y=0) =log1 7 Interpreting the Log-odds in Favour of an EventIA log-odds is a number between and .IA log odds of means we are certain the event increased log-odds corresponds to increased belief inthe occurrence of the log-odds of 0 corresponds to a probability of 1 log-odds of corresponds to certainty the event between Probability, Odds and Log-oddsIYou can use the following table to compute one measureof probability from another:PODDSlog ODDSP(Y=1) = 1 log 1 ODDS(Y=1) =oo1+ologolog ODDS(Y=1) =xex1+exexIChoose the row corresponding to the quantity you startwith and the column corresponding to the quantity youwant to 1 is often written logit( ).

5 Iexp(x)1+exp(x)is often written (x)(sometimes expit(x)).9 Motivation for ( Multivariate ) Logistic RegressionIWe want to modelP(Y=1)in terms of a set of predictorvariablesX1,X2,..Xp(for Univariate regressionp=1).IIn linear regression we use the regression equationE(Y) = 0+ 1X1+ 2X2+..+ pXp(1)IHowever, for a binaryY(0 or 1),E(Y) =P(Y=1).IWe cannot now use equation (??), because the left handside is a number between 0 and 1 while the right handside is potentially a number between and .ISolution: replace the LHS with logitEY:logitE(Y) = 0+ 1X1+ 2X2+.

6 + pXp10 Logistic regression Equation Written on Three ScalesIWe defined the regression equation on the logit orlog ODDS scale:log ODDS(Y=1) = 0+ 1X1+ 2X2+..+ pXpIOn the ODDS scale the same equation may be written:ODDS(Y=1) =exp( 0+ 1X1+ 2X2+..+ pXp)IOn the probability scale the equation may be written:P(Y=1) =exp( 0+ 1X1+ 2X2+..+ pXp)1+exp( 0+ 1X1+ 2X2+..+ pXp)11 Interpreting the InterceptIIn order to obtain a simple interpretation of the interceptwe need to find a situation in which the other parameters( 1, .., p) happens whenX1, ,Xpare all equal to we can interpret 0in 3 equivalent ways:1 0is the log-odds in favour ofY=1 whenX1= 0is such that exp( 0)is the odds in favour ofY=1 whenX1= 0is such thatexp( 0)1+exp( 0)is the probability thatY=1 whenX1= can choose any one of these three interpretationswhen you make a Picture: Intercept 2 (Y=1) = ( 0+ 1X1)exp( 0)1+exp( 0)IP(Y=1)vs.

7 X1whenp=1 ( Univariate regression ).13 Univariate Picture: Sign of 1 (Y=1)IWhen 1>0,P(Y=1)increases withX1(blue curve).IWhen 1<0,P(Y=1)decreases withX1(red curve).14 Univariate Picture: Magnitude of 1 (Y=1)I 1=2 (blue curve), 1=4 (red curve).IWhen| 1|is greater, changes inX1more stronglyinfluence the probability that the event 1: Univariate Logistic RegressionITo obtain a simple interpretation of 1we need to find away to remove 0from the regression the log-odds scale we have the regression equation:log ODDS(Y=1) = 0+ 1X1 IThis suggests we could consider looking at the differencein the log odds at different values ofX1, sayt+ ODDS(Y=1|X1=t+z) log ODDS(Y=1|X1=t)which is equal to 0+ 1(t+z) ( 0+ 1t) =z 1.

8 Univariate Logistic RegressionIBy puttingz=1 we arrive at the following interpretationof 1: 1is the additive change in the log-odds in favour of Y=1when X1increases can write an equivalent second interpretation on theodds scale:exp( 1)is the multiplicative change in the odds in favour ofY=1when X1increases 1as a Log-odds RatioIThe first interpretation of 1expresses the equation:logODDS(Y=1|X1=t+z)ODDS(Y=1|X1= t)=z 1whilst the second interpretation expresses the equation:ODDS(Y=1|X1=t+z)ODDS(Y=1|X1=t)= exp(z 1).

9 IThe quantityODDS(Y=1|X1=t+z)ODDS(Y=1|X1=t)is the odds-ratio in favourofY=1 forX1=t+z vs. X1= Coefficients in Multivariate LogisticRegressionIThe interpretation of regression coefficients inmultivariate Logistic regression is similar to theinterpretation in Univariate dealt with general the coefficient k(corresponding to the variableXk) can be interpreted as follows: kis the additive change in the log-odds in favour of Y=1when Xkincreases by1unit, while the other predictor variablesremain in the Univariate case, an equivalent interpretation canbe made on the odds a Logistic regression in RIWe fit a Logistic regression in R using theglmfunction.

10 > output <- glm(sta ~ sex, data= , family=binomial)IThis fits the regression equationlogitP(sta=1) = 0+ 1 data are stored in the fit a Logistic an aside, we can useglmas an alternative tolmto fit alinear model, by specifyingfamily= Regression: glmOutput in RCall:glm(formula = sta ~ sex, family = binomial, data = )Deviance Residuals:Min 1Q Median 3Q :Estimate Std. Error z value Pr(>|z|)(Intercept) **sex1 codes: 0 ** ** * . 1 ISummary of the distribution of the deviance residuals measure how well the observations fitthe model.