Multivariate Logistic Regression - McGill University

1 1 Multivariate Logistic RegressionAs in univariate Logistic Regression , let (x) represent the probability of an event thatdepends onpcovariates or independent variables. Then, using an formulationfor modeling the probability, we have: (x) =e 0+ 1X1+ 2X2+..+ pXp1 +e 0+ 1X1+ 2X2+..+ pXpSo, the form is identical to univariate Logistic Regression , but now with more than onecovariate. [Note: by univariate Logistic Regression , I mean Logistic Regression withone independent variable ; really there are two variables involved, the independentvariable and the dichotomous outcome, so it could also be termed bivariate.]To obtain the corresponding logit function from this, we calculate (lettingXrepresentthe whole set of covariatesX1,X2,..,Xp):logit[ (X)] = ln[ (X)1 (X)]= ln e 0+ 1X1+ 2X2+.

2 + pXp1+e 0+ 1X1+ 2X2+..+ pXp1 e 0+ 1X1+ 2X2+..+ pXp1+e 0+ 1X1+ 2X2+..+ pXp = ln e 0+ 1X1+ 2X2+..+ pXp1+e 0+ 1X1+ 2X2+..+ pXp11+e 0+ 1X1+ 2X2+..+ pXp = ln[e 0+ 1X1+ 2X2+..+ pXp]= 0+ 1X1+ 2X2+..+ pXpSo, again, we see that the logit of the probability of an event givenXis a simplelinear summarize, the two basic equations of Multivariate Logistic Regression are: (X) =e 0+ 1X1+ 2X2+..+ pXp1 +e 0+ 1X1+ 2X2+..+ pXpwhich gives the probabilities of outcome events given the covariate valuesX1,X2,..,Xp,andlogit[ (X)] = 0+ 1X1+ 2X2+..+ pXpwhich shows that Logistic Regression is really just a standard linear Regression model,once we transform the dichotomous outcome by the logit transform. This transformchanges the range of (X) from 0 to 1 to to + , as usual for linear analogously to univariate Logistic Regression , the above equations are for meanprobabilities, and each data point will have an error term.

3 Once again, we assumethat this error has mean zero, and that it follows a binomial distribution with mean (X), and variance (X)(1 (X)). Of course, nowXis a vector, whereas before itwas a scalar of the coefficients in multiple logis-tic regressionInterpretation of the intercept, 0:Notice that regardless of the number of co-variate values, if they are all set to zero, then we have (x) =e 01 +e 0exactly the same as in the univariate case. So, the interpretation of 0remainsthe same as in the simpler case: 0sets the baseline event rate, through theabove function, when all covariate values are set equal to example, if 0= 0 then (x) =e 01 +e 0=e01 +e0=11 + 1= if 0= 1 then (x) =e 01 +e 0=e11 +e1= if 0= 1 then (x) =e 01 +e 0=e 11 +e 1= so before, positive values of 0give values greater than , while negativevalues of 0give probabilities less than , when all covariates are set to of the slopes, 1, 2.

4 , p:Recall the effect on the proba-bility of an event asXchanges by one unit in the univariate case. There, wesaw that the coefficient 1is such thate 1is the odds ratio for a unit change inX, and in general, for a change ofzunits, theOR=ez 1=(e 1) much changes for the Multivariate case, except: When there is more than one independent variable , if all variables arecompletely uncorrelated with each other, then the interpretations of allcoefficients are simple, and follow the above pattern:We haveOR=ez ifor any variableXi,i= 1,2,..,p, where the ORrepresents the odds ratio for a change of sizezfor that variable . When the variables are not uncorrelated, the interpretation is more diffi-cult. It is common to say thatOR=ez irepresents the odds ratio fora change of sizezfor that variableadjusted for the effects of the this is essentially correct, we must keep in mind thatconfounding and collinearity can change and obscure these estimated rela-tionships.

5 The way confounding operates is identical to what we saw forlinear the coefficients given a data setAs in the univariate case, the distribution associated with Logistic Regression is thebinomial. For a single subject with covariate valuesxi={x1i,x2i,..,xpi}, the like-lihood function is: (xi)yi(1 (xi))1 yiFornsubjects, the likelihood function is:n i=1 (xi)yi(1 (xi))1 yi4To derive estimates of the unknown parameters, as in the univariate case, we needto maximize this likelihood function. We follow the usual steps, including taking thelogarithm of the likelihood function, taking (p+ 1) partial derivatives with respectto each parameter and setting these (p+ 1) equations equal to zero, to form a setof (p+ 1) equations in (p+ 1) unknowns. Solving this system of equations gives themaximum likelihood again omit the details here (as in the univariate case, no easy closed form formulaeexists), and will rely on statistical software to find the maximum likelihood estimatesfor typically rely on SE formulae for confidence intervals, and likelihood ratiotesting for hypothesis tests.

6 Again, we will omit the details, and rely on next look at several Logistic Regression ExamplesWe will look at three examples: Logistic Regression with dummy or indicator variables Logistic Regression with many variables Logistic Regression with interaction termsIn all cases, we will follow a similar procedure to that followed for multiple linearregression:1. Look at various descriptive statistics to get a feel for the data. For logisticregression, this usually includes looking at descriptive statistics, for examplewithin outcome = yes = 1 versus outcome = no = 0 The above by outcome group descriptive statistics are often sufficient fordiscrete covariates, but you may want to prepare some graphics for continuousvariables.

7 Recall that we did this for the age variable when looking at the For all continuous variables being considered, calculate a correlation matrix ofeach variable against each other variable . This allows one to begin to investigatepossible confounding and Similarly, for each categorical/continous independent variable pair, look at thevalues for the continuous variable in each category of the other Finally, create tables for all categorical/categorical independent variable Perform a separate univariate Logistic Regression for each independent begins to investigate confounding (we will see in more detail next class),as well as providing an initial unadjusted view of the importance of eachvariable, by Think about any interaction terms that you may want to try in the Perform some sort of model selection technique, or, often much better, thinkabout avoiding any strict model selection by finding a set of models that seemto have something to contribute to overall Based on all work done, draw some inferences and conclusions.

8 Carefully inter-pret each estimated parameter, perform model criticism , possibly repeatingsome of the above steps (for example, run further models), as Other inferences, such as predictions for future observations, and so with linear Regression , the above should not be considered as rules , but ratheras a rough guide as to how to proceed through a Logistic Regression Regression with dummy or indicator variablesChapter 1 (section ) of the Hosmer and Lemeshow book described a data setcalled ICU. Deleting the ID variable , there are 20 variables in this data set, which wedescribe in the table below:6 DescriptionCodingvariable nameVital Status0 = LivedSTA(Main outcome)1 = DiedAgeYearsAGESex0 = MaleSEX1 = FemaleRace1 = WhiteRACE2 = Black3 = OtherService at ICU Admission0 = MedicalSER1 = SurgicalCancer Part of Present0 = NoCANP roblem1 = YesHistory of Chronic RenalO = NoCRNF ailure1 = YesInfection Probable at ICU0 = NoINFA dmission1 = YesCPR Prior to ICU Admission0 = NoCPR1 = YesSystolic Blood Pressure atmm HgSYSICU AdmissionHeart Rate at ICU AdmissionBeats/minHRAP revious Admission to an ICU 0 = NoPREwithin 6 Months1 = YesType of Admission0 = ElectiveTYP1 = EmergencyLong Bone, Multiple, Neck,0 = NoFRAS ingle Area.

9 Or Hip Fracture1 = YesPO2 from Initial Blood Gases0>60PO21 60PH from Initial Blood Gases0 < from initial Blood0 45 PCOG ases1>45 Bicarbonate from Initial0 18 BICB lood Gases1<18 Creatinine from Initial Blood0 > of Consciousness at ICUO = No ComaLOCA dmissionor Stupor1 = Deepstupor2 = Coma7 The main outcome is vital status, alive or dead, coded as 0/1 respectively, under thevariable name sta. For this illustrative example, we will investigate the effect of thedichotomous variables sex, ser, and loc. Later, we will look at more of the variables.# read the data into R> <- (file="g:\\ ", header = T)# Take a quick look at the data> summary( )sta age sex race serMin. Min. Min. Min. Min.

10 Qu. 1st Qu. 1st Qu. 1st Qu. 1st Qu. Median Median Median Median Mean Mean Mean Mean Qu. 3rd Qu. 3rd Qu. 3rd Qu. 3rd Qu. Max. Max. Max. Max. crn inf cpr sysMin. Min. Min. Min. Min. : Qu. 1st Qu. 1st Qu. 1st Qu. 1st Qu. Median Median Median Median Mean Mean Mean Mean Qu. 3rd Qu. 3rd Qu. 3rd Qu. 3rd Qu. Max. Max. Max. Max. pre typ fra po2 Min. : Min. Min. Min. Min. Qu.: 1st Qu. 1st Qu. 1st Qu. 1st Qu.