t-tests and F-tests in regression - Jos Elkink

1 t-tests andf - tests in regressionJohan A. ElkinkUniversity College Dublin5 April 2012 Johan A. Elkink (UCD)tandF-tests5 April 20121 / 25 Outline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 20122 / 25 Simple linear regressionOutline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 20123 / 25 Simple linear regressionModelOutline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 20124 / 25 Simple linear regressionModelLinear equationsy=Intercept+Slope xJohan A. Elkink (UCD)tandF-tests5 April 20125 / 25 Simple linear regressionModelLinear equationsIntercepty=Intercept+Slope xJohan A. Elkink (UCD)tandF-tests5 April 20126 / 25 Simple linear regressionModelLinear equations xy=Intercept+Slope xJohan A. Elkink (UCD)tandF-tests5 April 20127 / 25 Simple linear regressionModelLinear equationsSlope x xy=Intercept+Slope xJohan A.

2 Elkink (UCD)tandF-tests5 April 20128 / 25 Simple linear regressionModelSimple regression model02460246810xyJohan A. Elkink (UCD)tandF-tests5 April 20129 / 25 Simple linear regressionModelSimple regression model02460246810xyy=Intercept+Slope xJohan A. Elkink (UCD)tandF-tests5 April 201210 / 25 Simple linear regressionModelSimple regression model02460246810xyy= 0+ 1xJohan A. Elkink (UCD)tandF-tests5 April 201210 / 25 Simple linear regressionModelSimple regression model02460246810xyyi= 0+ 1xi+ iJohan A. Elkink (UCD)tandF-tests5 April 201210 / 25 Simple linear regressionModelNotationyiValue on the dependent variable for caseixiValue on the independent variable for casei xMean value on the independent variable for casei iThe error for casei: i=yi yi kTrue coefficient for variablek kEstimated coefficient for variablek yiPredicted value on the dependent variable for caseiJohan A. Elkink (UCD)tandF-tests5 April 201211 / 25 Simple linear regressionModelOrdinary Least Squares Quickly put, the regression line is chosen to minimize the RSS; it hasslope 1, intercept 0, and goes through the point ( x, y).

3 Furthermore,the estimate for 2is 2=RSS/(n 2) (Verzani 2005: 280).Johan A. Elkink (UCD)tandF-tests5 April 201212 / 25 Simple linear regressionVariance andR2 Outline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 201213 / 25 Simple linear regressionVariance andR2 Breakdown of varianceTotal Sum of Squares (TSS): Ni=1(yi y)2 Explained Sum of Squares (ESS): Ni=1( yi y)2 Residual Sum of Squares (RSS): Ni=1(yi yi)2= Ni=1 2iTSS=ESS+RSSJ ohan A. Elkink (UCD)tandF-tests5 April 201214 / 25 Simple linear regressionVariance andR2 Breakdown of varianceTotal Sum of Squares (TSS): Ni=1(yi y)2 Explained Sum of Squares (ESS): Ni=1( yi y)2 Residual Sum of Squares (RSS): Ni=1(yi yi)2= Ni=1 2iTSS=ESS+RSSS ometimes the second is called regression sum of squares (RSS) and the third errors sum of squares (ESS), which might in fact be more accurate, since really represents errors, not residuals, in this specification.

4 Beware the confusion!Johan A. Elkink (UCD)tandF-tests5 April 201214 / 25 Simple linear regressionVariance andR2R2 How much of the variance did we explain?Johan A. Elkink (UCD)tandF-tests5 April 201215 / 25 Simple linear regressionVariance andR2R2 How much of the variance did we explain?R2= 1 RSSTSS= 1 Ni=1(yi yi)2 Ni=1(yi y)2= Ni=1( yi y)2 Ni=1(yi y)2 Can be interpreted as theproportion of total variance explained by A. Elkink (UCD)tandF-tests5 April 201215 / 25 Simple linear regressionVariance andR2R2 How much of the variance did we explain?R2= 1 RSSTSS= 1 Ni=1(yi yi)2 Ni=1(yi y)2= Ni=1( yi y)2 Ni=1(yi y)2 Can be interpreted as theproportion of total variance explained by this interpretation, the model must include an intercept. Generally,one should not attach too much value to having a highR2- it is usuallymore important to understand whetherxaffectsyand by how much,rather than to understand how much ofyhas not yet been A. Elkink (UCD)tandF-tests5 April 201215 / 25 Simple linear regressionVariance andR2R2 How much of the variance did we explain?

5 R2= 1 RSSTSS= 1 Ni=1(yi yi)2 Ni=1(yi y)2= Ni=1( yi y)2 Ni=1(yi y)2 Can be interpreted as theproportion of total variance explained by this interpretation, the model must include an intercept. Generally,one should not attach too much value to having a highR2- it is usuallymore important to understand whetherxaffectsyand by how much,rather than to understand how much ofyhas not yet been simple linear regression ( one independent variable),R2is the sameas the correlation coefficient, Pearson sr, A. Elkink (UCD)tandF-tests5 April 201215 / 25 InferenceOutline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 201216 / 25 Inferencet-testOutline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 201217 / 25 Inferencet-testInference from regressionIn linear regression , thesampling distributionof the coefficient estimatesform a normal distribution, which is approximated by at distributiondueto approximating A.

6 Elkink (UCD)tandF-tests5 April 201218 / 25 Inferencet-testInference from regressionIn linear regression , thesampling distributionof the coefficient estimatesform a normal distribution, which is approximated by at distributiondueto approximating we can calculate aconfidence intervalfor each estimated A. Elkink (UCD)tandF-tests5 April 201218 / 25 Inferencet-testInference from regressionIn linear regression , thesampling distributionof the coefficient estimatesform a normal distribution, which is approximated by at distributiondueto approximating we can calculate aconfidence intervalfor each estimated perform a hypothesis test along the lines of:H0: 1= 0H1: 16= 0 Johan A. Elkink (UCD)tandF-tests5 April 201218 / 25 Inferencet-testInference from regressionTo calculate the confidence interval, we need to calculate thestandarderrorof the A. Elkink (UCD)tandF-tests5 April 201219 / 25 Inferencet-testInference from regressionTo calculate the confidence interval, we need to calculate thestandarderrorof the of thumb to get the 95% confidence interval: 2SE< < + 2 SEJohan A.

7 Elkink (UCD)tandF-tests5 April 201219 / 25 Inferencet-testInference from regressionTo calculate the confidence interval, we need to calculate thestandarderrorof the of thumb to get the 95% confidence interval: 2SE< < + 2 SEThus if is positive, we are 95% certain it is different from zero when 2SE> A. Elkink (UCD)tandF-tests5 April 201219 / 25 Inferencet-testInference from regressionTo calculate the confidence interval, we need to calculate thestandarderrorof the of thumb to get the 95% confidence interval: 2SE< < + 2 SEThus if is positive, we are 95% certain it is different from zero when 2SE>0. (Or when thet valueis greater than 2 or less than 2.)Johan A. Elkink (UCD)tandF-tests5 April 201219 / 25 InferenceF-testOutline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 201220 / 25 InferenceF-testBreakdown of varianceTotal Sum of Squares (TSS): Ni=1(yi y)2 Explained Sum of Squares (ESS): Ni=1( yi y)2 Residual Sum of Squares (RSS): Ni=1(yi yi)2= Ni=1 2iTSS=ESS+RSSJ ohan A.

8 Elkink (UCD)tandF-tests5 April 201221 / 25 InferenceF-testF-testIn simple linear regression , we can do an F-test:H0: 1= 0H1: 16= 0 Johan A. Elkink (UCD)tandF-tests5 April 201222 / 25 InferenceF-testF-testIn simple linear regression , we can do an F-test:H0: 1= 0H1: 16= 0F=ESS/1 RSS/(n 2)=ESS 2 F1,n 2with 1 andn 2 degrees of A. Elkink (UCD)tandF-tests5 April 201222 / 25 InferenceF-testF-testIn simple linear regression , we can do an F-test:H0: 1= 0H1: 16= 0F=ESS/1 RSS/(n 2)=ESS 2 F1,n 2with 1 andn 2 degrees of multiple regression , this would generalize to:F=ESS/(k 1)RSS/(n k) Fk 1,n kJohan A. Elkink (UCD)tandF-tests5 April 201222 / 25 ExercisesOutline1 Simple linear regressionModelVariance andR22 Inferencet-testF-test3 ExercisesJohan A. Elkink (UCD)tandF-tests5 April 201223 / for both industry (IND) and services (AGR):1 Regress percentage in sector on income per the regression the model thet- A. Elkink (UCD)tandF-tests5 April 201224 / investigate:whether left-right self-placement (lr) and explains trust in politics(trustpol)whether left-right self-placement influences attitude towards EUmembership (eumember)Johan A.

9 Elkink (UCD)tandF-tests5 April 201225 / 25