Multilevel Analysis - Princeton University

1 PU/DSS/OTRM ultilevel Analysis (ver. )Oscar Torres-ReynaData Multilevel model whenever your data is grouped (or nested) in more than one category (for example, states, countries, etc). Multilevel models allow: Study effects that vary by entity (or groups) Estimate group level averagesSome advantages: Regular regression ignores the average variation between entities. Individual regression may face sample problems and lack of generalizationMotivationPU/DSS/OTR3-40-2 002040y0204060schoolScorey_meanuse : egeny_mean=mean(y)twowayscatter y school, msize(tiny) || connected y_meanschool, connect(L) clwidth(thick) clcolor(black) mcolor(black) msymbol(none) || , ytitle(y)Variation between entitiesPU/DSS/OTR4statsbyinter=_b[_cons ] slope=_b[x1], by(school) saving(ols, replace): regress y x1sort schoolmerge schoolusing olsdrop _mergegen yhat_ols= inter + slope*x1sort school x1separate y, by(school)separate yhat_ols, by(school)twowayconnected yhat_ols1-yhat_ols65 x1 || lfity x1, clwidth(thick) clcolor(black) legend(off) ytitle(y)-20-100102030y-40-2002040 Reading testIndividual regressions (no-pooling approach)PU/DSS/OTRLR test vs.

2 Linear regression: chibar2(01) = Prob >= chibar2 = sd(Residual) .1030214 sd(_cons) .3999163 : Identity Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] _cons .536272 .9193634 y Coef. Std. Err. z P>|z| [95% Conf. Interval] Log likelihood = Prob > chi2 =.

3 Wald chi2(0) = . max = 198 avg = Obs per group: min = 2 Group variable: school Number of groups = 65 Mixed-effects ML regression Number of obs = 4059. xtmixed y || school: , mle nolog5 Standard deviation at the school level (level 2)Standard deviation at the individual level (level 2)Mean of state level )()(_)(_)_()_()_(_222222222=+=+=+=residu alsdconssdconssdesigmausigmausigmancorre latioIntraclass An intraclass correlation tells you about the correlation of the observations (cases) within a cluster ( ) If the interclass correlation (IC) approaches 0 then the grouping by counties (or entities) are of no use (you may as well run asimple regression).

4 If the IC approaches 1 then there is no variance to explain at the individual level, everybody is the model (null)iijiy +=][Ho: Random-effects = 0PU/DSS/OTR6 Standard deviation at the school level (level 2)Standard deviation at the individual level (level 2)Mean of state level )()(_)(_)_()_()_(_222222222=+=+=+=residu alsdconssdconssdesigmausigmausigmancorre latioIntraclass An intraclass correlation tells you about the correlation of the observations (cases) within a cluster ( ) If the interclass correlation (IC) approaches 0 then the grouping by counties (or entities) are of no use (you may as well run asimple regression). If the IC approaches 1 then there is no variance to explain at the individual level, everybody is the model (one level-1 predictor)LR test vs. linear regression: chibar2(01) = Prob >= chibar2 = sd(Residual).

5 0841759 sd(_cons) .3052516 : Identity Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] _cons .0238706 .4002258 .8082987 x1 .5633697 .0124654 .5389381 .5878014 y Coef. Std. Err. z P>|z| [95% Conf. Interval] Log likelihood = Prob > chi2 = Wald chi2(1) = max = 198 avg = Obs per group: min = 2 Group variable: school Number of groups = 65 Mixed-effects ML regression Number of obs = 4059.

6 Xtmixed y x1 || school: , mle nologiiijixy ++=][Ho: Random-effects = 0PU/DSS/OTR7 Standard deviation at the school level (level 2)Standard deviation at the individual level (level 2)Mean of state level )()1()(_)1()(_)_()_()_(_22222222222=+++= +++=+=residualsdxsdconssdxsdconssdesigma usigmausigmancorrelatioIntraclassVarying -intercept, varying-coefficient modelNote: LR test is conservative and provided only for test vs. linear regression: chi2(3) = Prob > chi2 = sd(Residual) .0839482 corr(x1,_cons) .4975474 .1487416 .1572843 .7322131 sd(_cons) .3044138 sd(x1) .1205631 .0189827 .0885508 .1641483school: Unstructured Random-effects Parameters Estimate Std.

7 Err. [95% Conf. Interval] _cons .3978336 .6646554 x1 .5567291 .0199367 .5176539 .5958043 y Coef. Std. Err. z P>|z| [95% Conf. Interval] Log likelihood = Prob > chi2 = Wald chi2(1) = max = 198 avg = Obs per group: min = 2 Group variable: school Number of groups = 65 Mixed-effects ML regression Number of obs = 4059.

8 Xtmixed y x1 || school: x1, mle nolog covariance(unstructure)iiijijixy ++=][][Ho: Random-effects = 0PU/DSS/OTR8 Standard deviation at the school level (level 2)Standard deviation at the individual level (level 2)Mean of state level interceptsVarying-slope modelLR test vs. linear regression: chibar2(01) = Prob >= chibar2 = sd(Residual) .089372 sd( ) .0003388 .1806391 0 ._all: Identity Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] _cons.

9 1263914 .2357746 x1 .5950551 .0127269 .5701108 .6199995 y Coef. Std. Err. z P>|z| [95% Conf. Interval] Log likelihood = Prob > chi2 = Wald chi2(1) = max = 4059 avg = Obs per group: min = 4059 Group variable: _all Number of groups = 1 Mixed-effects ML regression Number of obs = 4059. xtmixed y x1 || _all: , mle nologiiijixy ++=][PU/DSS/OTR9 PostestimationPU/DSS/OTR10 Comparing models using likelihood-ration testUse the likelihood-ratio test (lrtest) to compare models fitted by maximum likelihood.

10 This test compares the log likelihood (shown in the output) of two models and tests whether they are significantly different. /*Fitting random intercepts and storing results*/quietly xtmixedy x1 || school:, mlenologestimates store ri/*Fitting random coefficients and storing results*/quietly xtmixedy x1 || school: x1, mlenologcovariance(unstructure)estimates store rc/*Running the likelihood-ratio test to compare*/lrtestri rcNote: LR test is conservative(Assumption: ri nested in rc) Prob > chi2 = test LR chi2(2) = lrtest ri rcThe null hypothesis is that there is no significant difference between the two models. If Prob>chi2< , then you may reject the null and conclude that there is a statistically significant difference between the models. In the example above we reject the null and conclude that the random coefficients model provides a better fit (it has the lowest log likelihood)PU/DSS/OTR11 Standard deviation at the school level (level 2)Standard deviation at the individual level (level 2)Mean of state level )var()1var()var(_)1var()var(_)_()_()_(_2 =+++=+++=+=residualxconsxconsesigmausigm ausigmancorrelatioIntraclassVarying-inte rcept, varying-coefficient model: postestimationNote: LR test is conservative and provided only for test vs.