Multinomial Response Models - data.princeton.edu

1 Chapter 6 Multinomial ResponseModelsWe now turn our attention to regression Models for the analysis of categoricaldependent variables with more than two Response categories. Several ofthe Models that we will study may be considered generalizations of logisticregression analysis to polychotomous data. We first consider Models thatmay be used with purely qualitative ornominaldata, and then move on tomodels forordinaldata, where the Response categories are The Nature of Multinomial DataLet me start by introducing a simple dataset that will be used to illustratethe Multinomial distribution and Multinomial Response The Contraceptive Use DataTable was reconstructed from weighted percents found in Table ofthe final report of the Demographic and Health Survey conducted in ElSalvador in 1985 (FESAL-1985).

2 The table shows 3165 currently marriedwomen classified by age, grouped in five-year intervals, and current use ofcontraception, classified as sterilization, other methods, and no fairly standard approach to the analysis of data of this type couldtreat the two variables as responses and proceed to investigate the questionof independence. For these data the hypothesis of independence is soundlyrejected, with a likelihood ratio 2of on 12 Rodr guez. Revised September, 20072 CHAPTER 6. Multinomial Response MODELST able : Current Use of Contraception By AgeCurrently Married Women. El Salvador, 1985 AgeContraceptive MethodAllSter. OtherNone15 1936123229620 248013740061725 2921613130164830 342687620354735 391975018843540 441502416433845 499110183284 All10054891671 3165In this chapter we will view contraceptive use as the Response and age asa predictor.

3 Instead of looking at the joint distribution of the two variables,we will look at the conditional distribution of the Response , contraceptive use,given the predictor, age. As it turns out, the two approaches are The Multinomial DistributionLet us review briefly the Multinomial distribution that we first encounteredin Chapter 5. Consider a random variableYithat may take one of severaldiscrete values, which we index 1,2,..,J. In the example the Response iscontraceptive use and it takes the values sterilization , other method and no method , which we index 1, 2 and 3. Let ij= Pr{Yi=j}( )denote the probability that thei-th Response falls in thej-th category. Inthe example i1is the probability that thei-th respondent is sterilized .Assuming that the Response categories are mutually exclusive and ex-haustive, we have Jj=1 ij= 1 for eachi, the probabilities add up toone for each individual, and we have onlyJ 1 parameters.

4 In the exam-ple, once we know the probability of sterilized and of other method weautomatically know by subtraction the probability of no method .Forgrouped datait will be convenient to introduce auxiliary randomvariables representing counts of responses in the various categories. Letnidenote the number of cases in thei-th group and letYijdenote the THE Multinomial LOGIT MODEL3of responses from thei-th group that fall in thej-th category, with our exampleirepresents age groups,niis the number of women inthei-th age group, andyi1,yi2,andyi3are the numbers of women steril-ized, using another method, and using no method, respectively, in thei-thage group. Note that jyij=ni, the counts in the various responsecategories add up to the number of cases in each age datani= 1 andYijbecomes an indicator (or dummy)variable that takes the value 1 if thei-th Response falls in thej-th categoryand 0 otherwise, and jyij= 1, since one and only one of the indicatorsyijcan be on for each case.

5 In our example we could work with the 3165records in the individual data file and letyi1be one if thei-th woman issterilized and 0 probability distribution of the countsYijgiven the totalniis givenby themultinomialdistributionPr{Yi1=yi1,.., YiJ=yiJ}=(niyi1,..,yiJ) yiJiJ( )The special case whereJ= 2 and we have only two Response categories isthe binomial distribution of Chapter 3. To verify this fact equateyi1=yi,yi2=ni yi, i1= i, and i2= 1 The Multinomial Logit ModelWe now consider Models for the probabilities ij. In particular, we wouldlike to consider Models where these probabilities depend on a vectorxiofcovariates associated with thei-th individual or group. In terms of ourexample, we would like to model how the probabilities of being sterilized,using another method or using no method at all depend on the woman s Multinomial LogitsPerhaps the simplest approach to Multinomial data is to nominate one ofthe Response categories as a baseline or reference cell, calculate log-odds forall other categories relative to the baseline, and then let the log-odds be alinear function of the we pick thelastcategory as a baseline and calculate the oddsthat a member of groupifalls in categoryjas opposed to the baseline as i1/ iJ.

6 In our example we could look at the odds of being sterilized rather4 CHAPTER 6. Multinomial Response Models than using no method, and the odds of using another method rather thanno method. For women aged 45 49 these odds are 91:183 (or roughly 1 to2) and 10:183 (or 1 to 18).sssssssagelogit2030405060-5-4-3-2-10 1oooooooSterilizationOtherFigure : Log-Odds of Sterilization vs. No Method andOther Method vs. No Method, by AgeFigure shows the empirical log-odds of sterilization and other method(using no method as the reference category) plotted against the mid-pointsof the age groups. (Ignore for now the solid lines.) Note how the log-oddsof sterilization increase rapidly with age to reach a maximum at 30 34 andthen decline slightly. The log-odds of using other methods rise gently up toage 25 29 and then decline Modeling the LogitsIn the Multinomial logit model we assume that the log-odds of each responsefollow a linear model ij= log ij iJ= j+x i j,( )where jis a constant and jis a vector of regression coefficients, forj=1,2.

7 ,J 1. Note that we have written the constant explicitly, so we THE Multinomial LOGIT MODEL5assume henceforth that the model matrixXdoes not include a column model is analogous to a logistic regression model, except that theprobability distribution of the Response is Multinomial instead of binomialand we haveJ 1 equations instead of one. TheJ 1 Multinomial logitequations contrast each of categories 1,2,..J 1 with categoryJ, whereasthe single logistic regression equation is a contrast between successes andfailures. IfJ= 2 the Multinomial logit model reduces to the usual logisticregression that we need onlyJ 1 equations to describe a variable withJresponse categories and that it really makes no difference which category wepick as the reference cell, because we can always convert from one formulationto another.

8 In our example withJ= 3 categories we contrast categories 1versus 3 and 2 versus 3. The missing contrast between categories 1 and2 can easily be obtained in terms of the other two, since log( i1/ i2) =log( i1/ i3) log( i2/ i3).Looking at Figure , it would appear that the logits are a quadraticfunction of age. We will therefore entertain the model ij= j+ jai+ ja2i,( )whereaiis the midpoint of thei-th age group andj= 1,2 for sterilizationand other method, Modeling the ProbabilitiesThe Multinomial logit model may also be written in terms of the originalprobabilities ijrather than the log-odds. Starting from Equation andadopting the convention that iJ= 0, we can write ij=exp{ ij} Jk=1exp{ ik}.( )forj= 1,..,J. To verify this result exponentiate Equation to obtain ij= iJexp{ ij}, and note that the convention iJ= 0 makes this formulavalid for allj.

9 Next sum overjand use the fact that j ij= 1 to obtain iJ= 1/ jexp{ ij}. Finally, use this result on the formula for that Equation will automatically yield probabilities that addup to one for 6. Multinomial Response Maximum Likelihood EstimationEstimation of the parameters of this model by maximum likelihood proceedsby maximization of the Multinomial likelihood ( ) with the probabilities ijviewed as functions of the jand jparameters in Equation This usu-ally requires numerical procedures, and Fisher scoring or Newton-Raphsonoften work rather well. Most statistical packages include a Multinomial terms of our example, fitting the quadratic Multinomial logit modelof Equation leads to a deviance of on 8 The associated P-valueis , so we have significant lack of quadratic age effect has an associated likelihood-ratio 2of four ( = and 12 8 = 4), and is highly that we have accounted for 96% of the association between age andmethod choice ( = ) using only four : Parameter Estimates for Multinomial Logit ModelFitted to Contraceptive Use DataParameterContrastSter.

10 Vs. None Other vs. shows the parameter estimates for the two Multinomial logitequations. I used these values to calculate fitted logits for each age from , and plotted these together with the empirical logits in Figure figure suggests that the lack of fit, though significant, is not a seriousproblem, except possibly for the 15 19 age group, where we overestimate theprobability of these circumstances, I would probably stick with the quadraticmodel because it does a reasonable job using very few parameters. However,I urge you to go the extra mile and try a cubic term. The model should passthe goodness of fit test. Are the fitted values reasonable? The Equivalent Log-Linear Model* Multinomial logit Models may also be fit by maximum likelihood workingwith an equivalent log-linear model and the Poisson likelihood.