Deciding between competing models: Chi-square difference …

1 Introduction to Structural Equation Modeling with LISREL Version February Christina Werner and Prof. Dr. Karin Schermelleh-Engel goethe university , FrankfurtDeciding between competing Models: Chi-square difference Tests1 Research SituationUsing structural equation modeling to investigate a research question, the simplest strategywould involve constructing just a single model corresponding to the hypotheses, test it againstempirical data, and use a model fit test and other fit criteria to judge the underlying , frequently, it is not particularly satisfactory to analyze a single model, but moreappropriate to analyze severalcompeting modelsandcompare the results, e.

2 G. in the followingsituations: The favorite model fits the data well, but there is a competing model based on different hy-potheses which may explain the observed relationships as well. If possible, this competingmodel should be rejected (it should fit the data worse compared to the favorite model). There are several competing models which are all theoretically plausible. Differences inmodel fit would be the only criteria to decide which model to prefer. The original, presumed model did not fit the data well. It has been modified and it is tobe shown that the modifications actually result in better model particular, the following model comparisons may be made: A model with an additional path compared to an otherwise identical model without thispath: Is there an effect between two latent variables or not?

3 Is there a direct effect of a -variable on an -variable, or an indirect effect only? A model assuming a relationship between two latent variables compared to a model wherethese latent variables are presumed to be unrelated: Are the factors 1and 2independentof each other or not? A model with an additional loading of a manifest variable on a latent variable, comparedto a model without such an additional loading: Is the manifest variablex1an exclusiveindicator of construct 1, or does it also measure aspects of a different latent variable 2at the same time?A decision between competing models may be clear-cut if there are completely obvious differencesin model fit criteria, or if a parameter in question turns out to be both insignificant (|t|< )and of marginal size (close to zero in the completely standardized solution).

4 1 Introduction to Structural Equation Modeling with LISREL Version February Christina Werner and Prof. Dr. Karin Schermelleh-Engel goethe university , FrankfurtHowever, frequently differences in model fit are more subtle, and an objective criterion for adecision between competing models may be desired. For this purpose, different models can becompared with regard to their model fit by computing a 2difference test. This test allowsto decide whether a given model fits significantly better or worse than a competing AssumptionA 2difference test is meaningful only if the models in question arenested models, i. e. oneof the models could be obtained simply byfixing/eliminating parametersin the other comparing models, this is frequently the case, e.

5 G. in the situations described above whereone model just contains anadditionalpath in the structural model anadditionalloading in a measurement model anadditionalcorrelation/covariance between latent variableswhich the other model does not contain (where the parameter in question is fixed to zero).This test is not directly applicable to non-nested models containingstructurally differentparameters, e. g. a model in whichx1serves as an indicator of latent variable 1, compared toan alternative model wherex1is an indicator of 2instead. In this case, there are the followingalternatives: Testing each of the non-nested models against a common parent model in which all modelsin question are nested.

6 Example: To compare a model assuming thatx1is an indicator ofthe latent variable 1against a model that assumesx1to be an indicator of 2instead, acommon parent model would containbothloadings (x1on 1as well asx1on 2). Eachof the models of interest could then be compared to the common parent model using 2difference tests. Ideally, only one of the models would significantly differ from the parentmodel in terms of model fit, allowing to indirectly compare the non-nested models anddecide between them. Descriptive model comparison using criteria suitable for non-nested models, for exampleAIC (Akaike Information Criterion). This does not allow significance testing, ProcedureTo compute a 2difference test, thedifference of the 2valuesof the two models in questionis taken as well as thedifference of the degrees of freedom.

7 Frequently, models underinvestigation differ from each other by just one more free parameter or one more fixed parameter,respectively, so in these cases the difference of the degrees of freedom is 1. 2diff= 2s 2landdfdiff=dfs dfl2 Introduction to Structural Equation Modeling with LISREL Version February Christina Werner and Prof. Dr. Karin Schermelleh-Engel goethe university , FrankfurtHere, s denotes the smaller model with fewer parameters and therefore more degrees of free-dom, whereas l denotes the larger model with more parameters and therefore fewer degrees 2diff-value is distributed withdfdiffdegrees of freedom and can be checked manually forsignificance using a the 2diff-value is significant, the larger model with more freely estimated parameters fitsthe data better than the smaller model in which the parameters in question are fixed.

8 So it pays off to estimate the additional parameters and to prefer the larger model. In case the 2diff-value is insignificant, both models fit equally well statistically, so the parameters in questioncan be eliminated from the model (fixed to zero) and the smaller model can be accepted justas difference tests applied to nested models have essentially the same strengths andweaknesses as 2-tests applied to any single model: They are directly affected by sample size,and for large samples even trivial differences may become , K. A. (1989).Structural equations with latent variables. New York: John Wiley & , R. B. (1998).Principles and practice of structural equation modeling.

9 New York: , K., Moosbrugger, H., & M uller, H. (2003). Evaluating the fit of structuralequation models: Tests of significance and descriptive goodness-of-fit of Psychological Research Online,8, 23 74. Available