Example: bachelor of science

Linear Mixed-Effects Regression - Statistics

Linear Mixed-Effects RegressionNathaniel E. HelwigAssistant Professor of Psychology and StatisticsUniversity of Minnesota (Twin Cities)Updated 04-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 1 CopyrightCopyright 2017 by Nathaniel E. HelwigNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 2 Outline of Notes1) Correlated Data:Overview of problemMotivating ExampleModeling correlated data2) One-Way RM-ANOVA:Model Form & AssumptionsEstimation & InferenceExample: Grocery Prices3) Linear Mixed-Effects Model:Random Intercept ModelRandom Intercepts & SlopesGeneral FrameworkCovariance StructuresEstimation & InferenceExample: TIMSS DataNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 3 Correlated DataCorrelated DataNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 4 Correlated DataOverview of ProblemWhat are Correlated Data?

Nesting typically introduces correlation into data at level-1 Students are level-1 and schools are level-2 Dependence/correlation between students from same school We need to account for this dependence when we model the data. Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects Regression Updated 04-Jan-2017 : Slide 8

Tags:

  Correlations, Regression

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Linear Mixed-Effects Regression - Statistics

1 Linear Mixed-Effects RegressionNathaniel E. HelwigAssistant Professor of Psychology and StatisticsUniversity of Minnesota (Twin Cities)Updated 04-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 1 CopyrightCopyright 2017 by Nathaniel E. HelwigNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 2 Outline of Notes1) Correlated Data:Overview of problemMotivating ExampleModeling correlated data2) One-Way RM-ANOVA:Model Form & AssumptionsEstimation & InferenceExample: Grocery Prices3) Linear Mixed-Effects Model:Random Intercept ModelRandom Intercepts & SlopesGeneral FrameworkCovariance StructuresEstimation & InferenceExample: TIMSS DataNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 3 Correlated DataCorrelated DataNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 4 Correlated DataOverview of ProblemWhat are Correlated Data?

2 So far we have assumed that observations are :(yi,xi)are independent for allnANOVA:yiare independent within and between groupsIn a Repeated Measures (RM) design, observations are observed fromthe same subject at multiple : multipleyifrom same subjectANOVA:same subject in multiple treatment cellsRM data are one type of correlated data, but other types E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 5 Correlated DataOverview of ProblemWhy are Correlated Data an Issue?Thus far, all of our inferential procedures have required : b N(b, 2(X X) 1)requires the assumption(y|X) N(Xb, 2In)where b= (X X) 1X yANOVA: L N(L, 2 aj=1c2j/nj)requires the assumptionyijiid N( j, 2)where L= aj=1cj jCorrelated data are (by definition) the independence assumptionNeed to account for correlation for valid inferenceNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 6 Correlated DataMotivating ExampleTIMSS Data from 1997 Trends in International Mathematics and Science Study (TIMSS)1 Ongoing study assessing STEM education around the worldWe will analyze data from 3rd and 4th grade studentsWe havenT=7,097 students nested withinn=146 schools> timss = (paste(datapath," ",sep=""),header=TRUE,+ colClasses=c(rep("factor",4),rep("numeri c",3)))> head(timss)idschool idstudent grade gender science math hoursTV1 10 100101 3 girl 32 10 100103 3 girl 23 10 100107 3 girl 44 10 100108 3 girl 35 10 100109 3 boy 36 10 100110 3 boy 21 E.

3 Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 7 Correlated DataMotivating ExampleIssues with Modeling TIMSS DataData are collected from students nested within typically introduces correlation into data at level-1 Students are level-1 and schools are level-2 Dependence/correlation between students from same schoolWe need to account for this dependence when we model the E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 8 Correlated DataModeling Correlated DataFixed versus Random EffectsThus far, we have assumed that parameters are unknown :bis some unknown (constant) coefficient vectorANOVA: jare some unknown (constant) meansThese are referred to as fixed effectsUnlike fixed effects, random effects are NOT unknown constantsRandom effects are random variables in the populationTypically assume that random effects are zero-mean GaussianTypically want to estimate the variance parameter(s)Models with fixed and random effects are called Mixed-Effects E.

4 Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 9 Correlated DataModeling Correlated DataModeling Correlated Data with Random EffectsTo model correlated data, we include random effects in the effects relate to assumed correlation structure for dataIncluding different combinations of random effects can account fordifferent correlation structures present in the dataGoal is to estimate fixed effects parameters ( , b) and randomeffects variance parameters are of interest, because they relate to modelcovariance structureCould also estimate the random effect realizations (BLUPs)Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 10 One-Way Repeated Measures ANOVAOne-Way RepeatedMeasures ANOVAN athaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 11 One-Way Repeated Measures ANOVAM odel Form and AssumptionsModel FormThe One-Way Repeated Measures ANOVA model has the formyij= i+ j+eijfori {1.}

5 ,n}andj {1,..,a}whereyij Ris the response fori-th subject inj-th factor level j Ris the fixed effect for thej-th factor level iiid N(0, 2 )is the random effect for thei-th subjecteijiid N(0, 2e)is a Gaussian error termnis number of subjects andais number of factor levelsNote: each subject is observedatimes (once in each factor level).Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 12 One-Way Repeated Measures ANOVAM odel Form and AssumptionsModel AssumptionsThe fundamental assumptions of the one-way RM ANOVA model are:1xijandyiare observed random variables (known constants)2 iiid N(0, 2 )is an unobserved random variable3eijiid N(0, 2e)is an unobserved random variable4 iandeijare independent of one another5 1,.., aare unknown constants6yij N( j, 2Y)where 2Y= 2 + 2eis the total variance ofYUsing effect coding, j= + jwith aj=1 j=0 Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 13 One-Way Repeated Measures ANOVAM odel Form and AssumptionsAssumed Covariance Structure (same subject)For two observations from the same subjectyijandyikwe haveCov(yij,yik) =E[(yij j)(yik k)]=E[( i+eij)( i+eik)]=E[ 2i+ i(eij+eik) +eijeik]=E[ 2i] = 2 given thatE( ieij) =E( ieik) =E(eijeik) =0 by model E.

6 Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 14 One-Way Repeated Measures ANOVAM odel Form and AssumptionsAssumed Covariance Structure (different subjects)For two observations from different subjectsyhjandyikwe haveCov(yhj,yik) =E[(yhj j)(yik k)]=E[( h+ehj)( i+eik)]=E[ h i+ heik+ iehj+ehjeik]=0given thatE( h i) =E( heik) =E( iehj) =E(ehjeik) =0 due to themodel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 15 One-Way Repeated Measures ANOVAM odel Form and AssumptionsAssumed Covariance Structure (general form)The covariance between any two observations isCov(yhj,yik) ={ 2 = 2 Yifh=iandj6=k0ifh6=iwhere = 2 / 2 Yis the correlation between any two repeatedmeasurements from the same subject. is referred to as the intra-class correlation coefficient (ICC).Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 16 One-Way Repeated Measures ANOVAM odel Form and AssumptionsCompound SymmetryAssumptions imply covariance pattern known as compound symmetryAll repeated measurements have same varianceAll pairs of repeated measurements have same covarianceWitha=4 repeated measurements the covariance matrix isCov(yi) = 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y 2Y = 2Y 1 1 1 1 whereyi= (yi1,yi2,yi3,yi4)is thei-th subject s vector of E.}

7 Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 17 One-Way Repeated Measures ANOVAM odel Form and AssumptionsNote on Compound Symmetry and SphericityAssumption of compound symmetry is more strict than we valid inference, we need the homogeneity of treatment-differencevariances (HOTDV) assumption to hold, which states thatVar(yij yik) = for anyj6=k, where is some is the sphericity assumption for covariance matrixIf compound symmetry is met, sphericity assumption will also be (yij yik) =Var(yij) +Var(yik) 2 Cov(yij,yik)=2 2Y 2 2 =2 2eNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 18 One-Way Repeated Measures ANOVAE stimation and InferenceOrdinary Least Squares EstimationParameter estimates are analogue of balanced two-way ANOVA: =1na aj=1 ni=1yij= y i=(1a aj=1yij) = yi y j=(1n ni=1yij) = y j y which implies that the fitted values have the form yij= + i+ j= yi + y j y so that the residuals have the form eij=yij yi y j+ y Nathaniel E.

8 Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 19 One-Way Repeated Measures ANOVAE stimation and InferenceSums-of-Squares and Degrees-of-FreedomThe relevant sums-of-squares are given bySST= aj=1 ni=1(yij y )2 SSS=a ni=1 2iSSA=n aj=1 2jSSE= aj=1 ni=1 e2ijwhere SSS = sum-of-squares for subjects; corresponding dfs aredfSST=na 1dfSSS=n 1dfSSA=a 1dfSSE= (n 1)(a 1)Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 20 One-Way Repeated Measures ANOVAE stimation and InferenceExtended ANOVA Table andFTestsWe typically organize the SS information into an ANOVA table:SourceSSdfMSFp-valueSSSa ni=1 2in 1 MSS F sp sSSAn aj=1 2ja 1 MSA F ap aSSE aj=1 ni=1(yij yjk)2(n 1)(a 1)MSESST aj=1 ni=1(yij y )2na 1 MSS=SSSn 1,MSA=SSAa 1,MSE=SSE(n 1)(a 1)F s=MSSMSE Fn 1,(n 1)(a 1)andp s=P(Fn 1,(n 1)(a 1)>F s),F a=MSAMSE Fa 1,(n 1)(a 1)andp a=P(Fa 1,(n 1)(a 1)>F a),F sstatistic andp s-value are testingH0: 2 =0 versusH1: 2 >0 Testing random effect of subject, but not a valid testF astatistic andp a-value are testingH0: j=0 jversusH1: ( j {1.)}

9 ,a})( j6=0)Testing main effect of treatment factorNathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 21 One-Way Repeated Measures ANOVAE stimation and InferenceExpectations of Mean-SquaresThe MSE is an unbiased estimator of 2e, ,E(MSE) = MSS has expectationE(MSS) = 2e+a 2 IfMSS>MSE, can use 2 = (MSS MSE)/aThe MSA has expectationE(MSA) = 2e+n aj=1 2ja 1 Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 22 One-Way Repeated Measures ANOVAE stimation and InferenceQuantifying Violations of SphericityValid inference requires sphericity assumption to be sphericity assumption is violated, ourFtest is too liberalGeorge Box (1954) proposed a measure of sphericity =( aj=1 j)2(a 1) aj=1 2jwhere jare the eigenvalues ofa apopulation covariance 1 1 such that =1 denotes perfect sphericityIf sphericity is violated, thenF a F (a 1), (a 1)(n 1)Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 23 One-Way Repeated Measures ANOVAE stimation and InferenceGeisser-Greenhouse AdjustmentLetY={yij}n adenote the data matrixZ=CnYwhereCn=In 1n1n1 ndenotesn ncentering matrix =1n 1Z Zis sample covariance matrix c=Ca Cais doubled-centered covariance matrixThe Geisser-Greenhouse estimate is defined =( aj=1 j)2(a 1) aj=1 2jwhere jare eigenvalues of that is the empirical version of using cto estimate.

10 Nathaniel E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 24 One-Way Repeated Measures ANOVAE stimation and InferenceHuynh-Feldt AdjustmentGG adjustment is too conservative when is close to and Feldt provide a corrected estimate of =n(a 1) 2(a 1)[n 1 (a 1) ]where is the GG estimate of .. note that .HF adjustment is too liberal when is close to E. Helwig (U of Minnesota) Linear Mixed-Effects RegressionUpdated 04-Jan-2017 : Slide 25 One-Way Repeated Measures ANOVAE stimation and InferenceAn R Function for One-Way RM ANOVAaov1rm <- function(X){X = (X)n = nrow(X)a = ncol(X)mu = mean(X)rhos = rowMeans(X) - mualphas = colMeans(X) - mussa = n*sum(alphas^2)msa = ssa / (a - 1)mss = a*sum(rhos^2) / (n - 1)ehat = X - ( mu + matrix(rhos,n,a) + matrix(alphas,n,a,byrow=TRUE) )sse = sum(ehat^2)mse = sse / ( (a-1)*(n-1) )Fstat = msa / msepval = 1 - pf(Fstat,a-1,(a-1)*(n-1))Cmat = cov(X)Jmat = diag(a) - matrix(1/a,a,a)Dmat = Jmat%*%Cmat%*%Jmatgg = ( sum(diag(Dmat))^2 ) / ( (a-1)*sum(Dmat^2) )hf = (n*(a-1)*gg - 2) / ( (a-1)*(n - 1 - (a-1)*gg) )pgg = 1 - pf(Fstat,gg*(a-1),gg*(a-1)*(n-1))phf = 1 - pf(Fstat,hf*(a-1),hf*(a-1)*(n-1))list(mu = mu, alphas = alphas, rhos = rhos,Fstat = c(F=Fstat,df1=(a-1),df2=(a-1)*(n-1)),pva ls = c(pGG=pgg,pHF=phf,p=pval),epsilon = c(GG=gg,HF=hf),vcomps = c( , ((mss-mse)/a)) )}Nathaniel E.


Related search queries