Transcription of Analysis of Variance for Regression/Multiple Regression
1 AnalysisofVarianceforRegression/ multiple RegressionLectureNotesXVIIS tatistic112,Fall2002 Announcements Thesecondmidtermis next Thursday. Extra OfficeHoursNext Week:Monday, 1:30-2:30;Wednesday, 9-10,1:30-2:30orby , 1-2and4:30-5:30. Haipengwillholdofficehourstomorrow from10:30-12:30. A tutorfromtheuniversitytutorservicewillho lda reviewsessiononTuesday (Nov. 12) HuntsmanHall, notaffiliatedwiththecourse. I willposta setof exercisesforthisweek s lecturesby Analysisof varianceforregression. Multipleregression. Basicmodel. Estimatingandinterpretingtheparameters. Theimpactof lurkingvariables.
2 (analysisofvarianceforregressionpart), Theanalysisof Variance (ANOVA)providesa convenientmethodof comparingthefit of two ormoremodelsto thesamesetof areinterestedin simplelinearregressionmodelin whichtheslopeis zero, simplelinearregressionmodelin whichtheslopeis notzero, .Forbothmodelsit is assumedthat , independent. Analysisof variancesummarizesinformationaboutthesou rcesofvariationin thedata. Totalvariationin theresponse is expressedby thedeviations "!# .Two reasonswhy doesnotequal# : Responses correspondto differentvaluesoftheexplanatoryvariable . Thefittedvalues estimatesthemeanresponseforeachspecific.
3 Thedifference "!# reflectsvariationinmeanresponsesdueto differencesin . Individualobservationswillvary aboutmeanbecauseofvariationwithinsubpopu lationof responsesto a fixed . Thisvariationis representedby theresiduals "! .SumsofSquares Basicideabehindanalysisof Variance :If , canestimatetheamountof variationduetotheresponses correspondingto differentvaluesoftheexplanatory variable andbaseourtestonthisestimate. !# !# " ! Algebraicfact: !# !# ! (1) We write(1)as SSstandsforsumof squaresandT,MandE standfortotal,modelanderrorrespectively. TotalvariationSSTis thesumofvariationdueto thestraight-linemodelfortheregressionfun ction(SSM)andvariationdueto deviationsfromthismodel(SSE).
4 If weretrue, thenSSMshouldbesmall. squaresreflects.. ! ! . Meansquare(MS) sumofsquaresdegreesof freedom Interpretationof : fractionof variationin thevaluesof thatis explainedby theleastsquaresregressionof on . !# !# ! ! !# TheANOVA F test ( is notlinearly relatedto ) teststatisticis willtendto besmallwhen is trueandlargewhen is true. Under , thestatistic hasan distributionwith degreeof freedomin thenumeratorand ! degreesof freedominthedenominator(Table E). Forsimplelinearregression,the testis equivalentto the -testof versus .Multipleregression Consideragaintheproblemof decidinghow many yearsyoushouldstay in have availablethejointdistributionsof earnings( ), education( ) andIQs( ) fora samplefroma populationof peoplelike yourself.
5 Youcouldjustusetheregressionfunction to makeyourpredictionbutgiventheextra informationaboutIQin thesample, it is natural to try to useit. Thenatural way tousetheextra informationis to usethemultipleregressionfunction to make yourprediction(yousubstitutein yourIQto make theprediction). Populationregressionfunction: . Dataformultipleregression:Person1 " " Person2 ..Person MultipleLinearRegressionModel Onepossible modelforthepopulationregressionfunctioni sthemultiplelinearregressionmodel,ananal ogueof thesimplelinearregressionmodel: " " Interpretationof : Thechangein themeanof if isincreasedby oneunitandallotherexplanatory variables, " areheldfixed.
6 Themultiplelinearregressionmodelis very flexible. Examplesof multiplelinearregressionmodels: " ProbabilityModelforMultipleLinearRegress ion Thestatisticalmodelformultiplelinearregr essionis for . Themeanresponse is a linearfunctionof theexplanatoryvariables Theresiduals areindependentandnormallydistributedwith mean0 andstandarddeviation . In otherwords, they areasimplerandomsamplefroma " . Let denotetheestimatorsof . Forthe thobservationsthepredictedresponseis The thresidual,thedifferencebetweentheobserv edandpredictedresponse, is observedresponse!predictedresponse !
7 ! ! ! ! To estimate , we usethemethodof the s thatmakesthesumof thesquaresassmallaspossible, , choose tominimize ! ! ! ! Theparameter " measuresthevariabilityof estimate by whichis anaverageof thesquaredresiduals ! ! ! We estimate by . We call Youwantto predictwhatyourearningswillbeif youobtainacertainnumberof yearsof earnings, " yearsof educationand samplethatonlycontainsearningsandyearsof educationdata,people s IQsarenotrecoreded. IQis probably a lurkingvariable, , a variable thathasanimportanteffectontherelationshi pamongthevariablesin astudybutis notincludedamongthevariablesstudied.
8 Supposethatthepopulationregressionfuncti onfortheexpectedvalueof earningsgiveneducationandIQis amultiplelinearregressionfunction: " " andalsosupposethat " " Whatis ? If we useleastsquaresto estimatethesimplelinearregressionfunctio n , theslopeoftheleastsquareslinewillbeanunb iasedestimateof whichdoesnotgenerallyequal . Thus, by regressingearningsononlyyearsofeducation ,youwillnotobtaintherightslopeforestimat ingtheimpactof additionalyearsof educationgivenyourfixedIQ. Two circumstancesin which : , , IQdoesnothelpto predictearningsonceeducationis included. , , yearsof educationdoesnothelpto omittingabilityonestimatesof thereturnstoeducation: logearnings.
9 Theinterpretationof theleastsquarescoefficientis thatit approximatelymeasuresthepercentincreasei n earningsforoneextra yearof s, trainedapplicants(1962)NLSYM(1969) CPS(1964) (1973)
