Classical Linear Regression Model: Assumptions and ...

1 Classical Linear Regression Model: Assumptions and Diagnostic TestsYan ZengVersion , last updated on 10/05/2016 AbstractSummary of statistical tests for the Classical Linear Regression Model (CLRM), based on Brooks [1],Greene [5] [6], Pedace [8], and Zeileis [10].Contents1 The Classical Linear Regression Model (CLRM)32 Hypothesis Testing: The t-test and The F-test43 Violation of Assumptions : Detection of multicollinearity.. Consequence of ignoring near multicollinearity.. Dealing with multicollinearity..64 Violation of Assumptions : Detection of heteroscedasticity.. The Goldfeld-Quandt test.

2 The White s general test.. The Breusch-Pagan test.. The Park test.. Consequences of using OLS in the presence of heteroscedasticity.. Dealing with heteroscedasticity.. The generalised least squares method.. Transformation.. The White-corrected standard errors..95 Violation of Assumptions : Detection of autocorrelation.. Graphical test.. The run test (the Geary test).. The Durbin-Watson test.. The Breusch-Godfrey test.. Consequences of ignoring autocorrelation if it is present.. Dealing with autocorrelation.. The Cochrane-Orcutt procedure.. The Newey-West standard errors.

3 Dynamic models.. First difference.. Miscellaneous issues..136 Violation of Assumptions : Non-Stochastic Regressors147 Violation of Assumptions : Non-Normality of the Disturbances148 Issues of Model Omission of an important variable.. Inclusion of an irrelevant variable.. Functional form: Ramsey s RESET.. Parameter stability / structural stability tests.. The Chow test.. Predictive failure tests.. The Quandt likelihood ratio (QLR) test.. Recursive least squares (RLS): CUSUM and CUSUMQ..169 The Generalized Linear Regression Model (GLRM) Properties of OLS in the GLRM.. Robust estimation of asymptotic covariance matrices for OLS.

4 HC estimator.. HAC estimator.. for robust covariance estimation of OLS..1921 The Classical Linear Regression Model (CLRM)Let the column vectorxkbe theTobservations on variablexk,k= 1; ; K, and assemble these datain anT Kdata matrixX. In most contexts, the first column ofXis assumed to be a column of1s:x1= 1so that 1is the constant term in the model. Letybe theTobservationsy1, ,yT, and let"be thecolumn vector containing theTdisturbances. TheClassical Linear Regression Model(CLRM) can bewritten asy=x1 1+ +xK K+";xi= 1or in matrix formyT 1=XT K K 1+"T 1: Assumptions of the CLRM(Brooks [1, page 44], Greene [6, page 16-24]):(1)Linearity:The model specifies a Linear relationship betweenyandx1, , +"(2)Full rank:There is no exact Linear relationship among any of the ndependent variables in the assumption will be necessary for estimation of the parameters of the model (see formula (1)).

5 Xis aT Kmatrix with rankK.(3)Exogeneity of the independent variables:E["ijxj1; xj2; ; xjK] = 0. This states that theexpected value of the disturbance at observationiin the sample is not a function of the independentvariables observed at any observation, including this one. This means that the independent variables willnot carry useful information for prediction of" ["jX] =0:(4)Homoscedasticity and nonautocorrelation:Each disturbance,"ihas the same finite variance, 2, and is uncorrelated with every other disturbance," ["" jX] = 2I:(5)Data generation:The data in(xj1; xj2; ; xjK)may be any mixture of constants and be fixed or random.

6 (6)Normal distribution:The disturbances are normally distributed."jX N(0; 2I):In order to obtain estimates of the parameters 1, 2, , K, theresidual sum of squares(RSS)RSS=^" ^"=T t=1^"2t=T t=1(yt K i=1xit i)23is minimised so that the coefficient estimates will be given by theordinary least squares (OLS) estimator^ =2664^ 1^ 2 ^ k3775= (X X) 1X y:(1)In order to calculate the standard errors of the coefficient estimates, the variance of the errors, 2, isestimated by the estimators2=RSST K= Tt=1^"2tT K(2)where we recallKis the number of regressors including a constant. In this case,Kobservations are lost asKparameters are estimated, leavingT Kdegrees of the parameter variance-covariance matrix is given byVar(^ ) =s2(X X) 1:(3)And the coefficient standard errors are simply given by taking the square roots of each of the terms on theleading diagonal.

7 In summary, we have (Brooks [1, page 91-92])8> <>:^ = (X X) 1X y= + (X X) 1X "s2= Tt=1^"2tT KVar(^ ) =s2(X X) 1:(4)The OLS estimator is the best Linear unbiased estimator (BLUE), consistent and asymptotically normallydistributed (CAN), and if the disturbances are normally distributed, asymptotically efficient among all Hypothesis Testing: The t-test and The F-testThet-statistic for hypothesis testing is given by^ i hypothesized valueSE(^ i) t(T K)whereSE(^ i) = Var(^ )ii, and is used to test single hypotheses. TheF-test is used to test more than onecoefficient theF-test framework, two regressions are required.

8 Theunrestricted regressionis the one inwhich the coefficients are freely determined by the data, and therestricted regressionis the one in whichthe coefficients are restricted, the restrictions are imposed on some s. Thus theF-test approach tohypothesis testing is also termedrestricted least statistic for testing multiple hypotheses about the coefficient estimate is given byRRSS U RSSU RSS T Km F(m; T K)(5)whereU RSSis the residual sum of squares from unrestricted Regression ,RRSSis the residual sum of squaresfrom restricted Regression ,mis the number of restrictions1,Tis the number of observations, andKis thenumber of regressors in the unrestricted see why the test centres around a comparison of the residual sums of squares from the restricted andunrestricted regressions, recall that OLS estimation involved choosing the model that minimised the residual1 Informally, the number of restrictions can be seen as the number of equality signs under the null hypothesis.

9 4sum of squares, with no constraints imposed. Now if, after imposing constraints on the model, a residualsum of squares results that is not much higher than the unconstrained model s residual sum of squares, itwould be concluded that the restrictions were supported by the data. On the other hand, if the residualsum of squares increased considerably after the restrictions were imposed, it would be concluded that therestrictions were not supported by the data and therefore that the hypothesis should be can be further stated thatRRSS U under a particular set of very extreme circumstanceswill the residual sums of squares for the restricted and unrestricted models be exactly equal.

10 This would bethe case when the restriction was already present in the data, so that it is not really a restriction at , we note any hypothesis that could be tested with at-test could also have been tested using anF-test, sincet2(T K) F(1; T K):3 Violation of Assumptions : MulticollinearityIf the explanatory variables were orthogonal to one another, adding or removing a variable from aregression equation would not cause the values of the coefficients on the other variables to make it impossible to invert the(X X)matrix since it would not be of full , the presence of high multicollinearity doesn t violate any CLRM Assumptions .