HIERARCHICAL LINEAR MODELS FOR ELECTORAL …

1 HIERARCHICAL LINEAR MODELS FORELECTORAL research :A Worked Example in StataMarco R. SteenbergenDepartment of Political ScienceUniversity of Zurich2012 AbstractThese notes discuss a worked example of a HIERARCHICAL LINEAR model withextensive discussion about modeling decisions and interpretation. Note thatvariable names and stata commands appear in we are interested in evaluations of Barack Obama in 2008. We distinguishbetween two levels of analysis: the individual (level-1) and thestatein whichhe/she resides (level-2). Evaluations of Barack Obama are measured on a feelingthermometer (obamafeel), which we treat as a continuous outcome. Our primarylevel-1 predictor is the race of the respondent (1 =black; 0 = white). In addition,we consider partisanship (pid) and the individual s ideological distance to Obama(iddist) as predictors.

2 Partisanship runs from 0 (= strong Democrat) to 6 (=strong Republican), while ideological distance runs from 0 (the respondent andObama are placed at the same ideological position) to 10 (the respondent andObama are at opposite ends of the ideological spectrum). The analysis is based onthe 2008 American National Election data are downloadable Effects ANOVAAs a starting point for the analysis, it is useful to run a random effects allows us to determine what portion of the variance in candidate evaluationsis due to cross-state differences as compated to individual differences. The modelmay be written asobamafeelij= 0j+ ij 0j= 00+ 0jfor individualiin statej. Here, 0jcan be interpreted as the mean evaluationof Obama in a state, whereas 00is the grand mean ( , the mean across allindividuals and states).

3 The level-1 error term ( ij) shows how an individual sevaluation deviates from the mean evaluation in the state in which he/she level-2 error term ( 0j) shows how the mean evaluation in a particular statedeviates from the grand mean. In addition to estimating 00, we are interested inestimatingV ar( ij) = 2andV ar( 0j) = estimate the model using stata sxtmixedcommand, using the followingsyntax:xtmixed obamafeel if flagmis==0, || state:, varwhereflagmis=0selects those cases that are not missing on the level-1 covariatesor level-2 covariate (to be discussed later). Thevaroption causes stata to reportvariances rather than standard deviations, which is the default inxtmixed. Theoutput is shown in Figure output is divided into four sections.

4 At the top one finds descriptive infor-mation about the analysis. This is followed by a listing of the fixed effects third section displays the variance component estimates. Finally, at the bottomof the output, a likelihood ratio test is on the descriptive part, we see that there are 1673 level-1 units inthe analysis (number of obs), and 33 level-2 units (number of groups). Thecluster size varies from 11 to 258, with an average cluster size of just shy of 51observations. The log-likelihood is the fixed effects part, we see an estimate associated withcons. This estimateof is the estimate of the grand mean 00. Thus, averaging acros respondents2 The parts about the Wald test are left empty because there are no level-1 covariates in themodel whose effects would be 1: StataxtmixedOutput for a Random Effects ANOVAand states, the expected feeling thermometer rating of Obama was just shy of standard error is , resulting in az-test statistic of , and ap-value Clearly, the grand mean is significantly different from zero, which should notcome as too great of a surprise considering that Obama won the election.

5 Moretelling is the 95% confidence interval, which runs roughly from to Onfeeling thermometers, a value of 50 is sometimes interpreted as a neutral candidateevaluation. Since this value is not included in the confidence interval, one has clearevidence of a positive attitude toward Obama on the the third section of the output, we find the estimates of the variance com-ponents. The estimate of 00is given asvar(cons)and is , with a 95%confidence interval running from to The estimate of 2is givenasvar(Residual)and is , with a 96% confidence interval from final portion of the output provides the likelihood (LR) ratio test statisticfor the null hypothesis that 00= 0, , that there is no cross-state variation inevaluations of Obama.

6 The LR test compares to fit of the random effects ANOVAto that of a an ordinary regression model with a constant only. The latter modelallows only for individual variation. The test statistic has one degree of freedom we are testing just one variance component and comes out to The reported3p-value is .000 and should actually be halved to obtain a less conservative this case, halving does not affect the conclusion: the null hypothesis has to berejected there is evidence of cross-state variation in the evaluations of CorrelationBased on the estimates in the third part of the output,we can compute the intra-class correlation (ICC). using the definition of the ICC,its estimator is = 00 00+ 2 Substituting for 00and for 2, the ICC is estimated as =.

7 Means that around 9 percent of the variance in Obama evaluations is due todifferences across states, with the remaining 91 percent attributable to Intercept ModelWe now add level-1 covariates to the model , for now assuming fixed effects. Theintercept, however, is allowed to vary across states, in order to accommodate cross-state differences in the baseline evaluation of Obama. The mixed model that is tobe estimated is thusobamafeelij= 00+ 10pidij+ 20iddistij+ 30blackij+ 0j+ ijThis can be estimated in stata usingxtmixed obamafeel pid iddist black if flagmis==0, || state:,varThe estimates are shown in Figure three covariates have statistically significant effects. The Wald test shows,not surprisingly, that they are also jointly significant (W= ,p=.)

8 000). Asone moves up one unit on the partisanship scale, ratings of Obama are expected todecrease by around 7 points. Compared to whites, the expected ratings of blacksare some 12 points higher. An anomalous result is that for ideological distance, asthe coefficient suggests that an increased distance to Obama on the left-right scale3 Remember that we are testing a variance component, for which the alternative hypothesisis of necessity one-sided, namely 00>0(negative variances, which would be allowed under atwo-sided test, do not make sense).4 Figure 2: StataxtmixedOutput for a Random Intercept Modelactually improves his ratings. This result may possibly be a consequence of modelmisspecification, , ideological distance picking up some other is evidence of variation in the intercepts.

9 The variance of the 0js isestimated as , which is sizable. Moreover, comparing the fit of the randomintercept model to that of a regression model yieldsLR= a halvedp-value of .000. Thus, we can clearly reject the null hypothesis that the interceptis the same across all of the states, as the regression model VarianceHow much of the variance do the three level-1 covariatesexplain? To answer this question, we compare the variance in the outcome in therandom intercept model and that in the random effects ANOVA. Specifically, weevaluateR21= 1 V arNew(Y)V arOld(Y)whereV arNew(Y)is the variance in Obama feelings in the random intercept modelandV arOld(Y)is the variance in the random effects ANOVA. From the estimatesin Figure 1,V arOld= + = From Figure 2, we getV arNew= + = Consequently,R21=.

10 48so that the three covariates50102030 Random InterceptsCOCTWAWINYCAAZTXNJMASCORILOKNM KSGARIMNDEFLMSVAPANCNDINTNMIOHNVALLAF igure 3: Empirical Bayes Estimates of the Random Intercepts Across Statesaccount for about 48 percent of the variation in the Bayes EstimatesIt is useful to get a sense of how the intercepts varyacross the states. To that effect, we compute 0j= 00+ 0jwhere 0jis the empirical Bayes residual. We can then display the estimates in adot plot. The following syntax accomplishes this:predict ebres*, reffects level(state)gen beta0=obamafeel[cons]+ebres1graph dot (mean) beta0, over(V081201a, gap(V081201a) sort(beta0))cw linetype(line) ytitle(Random Intercepts)whereV081201ais a variable containing the abbreviation of the state name. Theresulting estimates are shown in Figure 3.