Econometrics Lecture Notes (OMEGA)

1 GraduateEconometricsLectureNotesMichaelC reel ,Copyright(C)Jan28,2002byMichaelCreelCon tents1 License, .. ;YSpace..16 ,UniversitatAut .. (Gauss-Markov theorem).. ofMLE.. r-Raolowerbound.. (Raotests,Lagrangemultipliertests).. ,Waldandscoretests.. estimation.. (1).. (1).. briefasideondummyvariables.. translogmodel.. objects:theprobitmodel.. (q)processes.. (p)processes.. (q)process.. :Consistency ofLeastSquares.. :Linearizationofa nonlinearmodel..26718 Generalizedmethodofmoments(GMM) .. specificationtest.. :Nonlinearrationalexpectations..30620 Nonlinearleastsquares(NLS) .. :ThePoissonmodelforcountdata.. :Limiteddependentvariablesandsampleselec tion.. :LaborSupply..319721 .. :estimation.. :hypothesistesting.. norm.. ofconcepts.. width:Cross-validation.. : :Marginalizationoflatentvariables.. (SML).. :multinomialprobit.

2 (MSM).. (EMM).. : estimationofauctionmodels.. :estimationofstochasticdifferentialequat ions.. :estimationofa , , know is thatyouarefreeto modifyanddistributethesenotesinany wayyoulike, , (Open-sourceMaterialsforEconometrics,GPL A rchive) They werepreparedusingLYX( ).LYXisanopensource whatyouseeiswhatyoumean wordprocessor. It canexportyourworkinTEX,HTML, willrunonUnix,Windows, , ,aswellasPDF, HTML, ,forstudy, preparinga course, findthata hardcopy is ofmostuseforlecturingorstudy, whilethehtmlversionis usefulforquickreferenceoransweringstuden ts wouldgreatlyappreciatethatyouinformmeofa ny d alsowelcomecontributionsinany area, modelfromeconomictheory:xi=xi(pi;mi;zi) xiisG 1 vectorofquantitiesdemanded piisG 1 vectorofprices miis income ziis a vectorofindividualcharacteristicsrelated topreferencesSupposea sampleofoneobservationofnindividuals demandsattimeperiodt(thisis acrosssection).

3 Themodelis notestimableasit stands. Theformofthedemandfunctionis differentforalli: Somecomponentsofziaresubjecttofluctuatio nsthatarenotobservabletooutsidemodeler(p eopledon t eatthesameluncheveryday).Breakziintotheo bservablecomponentswiandanunobservableco mponent i:Anestimable( ,econometric)modelisxi= 0+p0i p+mi m+w0i w+ iWe have imposeda numberofrestrictionsonthetheoreticalmode l: Thefunctionsxi( )whichmaydifferforallihave Ofallparametricfamiliesoffunctions,wehav e , ,theserestrictionshave notheoreticalbasis. Thevalidityofany ,specificationtestingwillbeneeded, at leastapproximatelycorrectshouldweuseit a modelis : themodelis a linearfunctionoftheparametervector 0:yt=x0t 0+ t;orinmatrixform,y=X 0+ ;whereyisn 1;X= x1x2 xn 0;wherextisK 1;and 0and 0 in 0meansthisis s mightfirstappear, sinceonecanemploynonlineartransformation softhevariables: 0(zt) = 1(wt) 2(wt) p(wt) 0+ t(The i()areknownfunctions).

4 Definingyt= 0(zt);xt1= 1(wt); modelintheformofequation(??). Forexample,theCobb-Douglasmodelz=Aw 22w 33exp( )canbetransformedlogarithmicallytoobtain lnz=lnA+ 2lnw2+ 3lnw3+ errors:E( )=0 Var( ) =E( 0) = ,linearlyindependentregressors(a)Xhasran kK(b)Xis nonstochastic(c)limn! 1nX0X=QX;a finitepositive (Optional): is is togaininformationabouttheunknownparamete rs 0and 20: =arg mins( ) =n t=1 yt x0t 2s( ) =(y X )0(y X )=y0y 2y0X + 0X0X =ky X k2 Thislastexpressionmakesit clearhow theOLSestimatorchooses : it minimizestheEuclideandistancebetweenyand X : To minimizethecriterions( );take :D s( ) = 2X0y+2X0X =014so = (X0X) 1X0y: To verifythatthisis a minimum, :D2 s( ) =2X0 XSince (X) =K;thismatrixis positive definite,sinceit s a (identitymatrixofordern), so is infacta minimizer. Thefittedvaluesareinthevector y=X : Theresidualsareinthevector =y X Notethaty=X + =X + 20isc 20=1n K 0 ;YSpaceDoa plotwiththetrueline, observationsandtheestimatedline.

5 Wewanttoplotinobservationspace,we ll needtouseonlytwo s observations,wecan t haveK>1:Drawa picturewithtwoobservationsandoneregresso r. We candecomposeyintotwo components:theorthogonalprojectionontoth eK dimensionalspacespannedbyX,X ;andthecomponentthatistheor-thogonalproj ectionontothen Ksubpacethatis orthogonaltothespanofX; : Since is chosento make asshortaspossible, willbeorthogonalto thespacespannedbyX:SinceXis inthisspace,X0 =0 We have thatX is theprojectionofyonthespanofX;orX =X X0X 1X0yTherefore,thematrixthatprojectsyonto thespanofXisPX=X(X0X) 1X0sinceX =PXy: is theprojectionofy offthespacespannedbyX(thatis ontothespacethatis16orthogonaltothespano fX):We have that =y X =y X(X0X) 1X0y= In X(X0X) 1X0 y:Sothematrixthatprojectsyoff thespanofXisMX=In X(X0X) 1X0=In PX:We have =MXy: Thereforey=PXy+MXy=X + : NotethatbothPXandMXaresymmetricandidempo tent.

6 A symmetricmatrixAis onesuchthatA=A0: AnidempotentmatrixAis onesuchthatA=AA: Theonlynonsingularidempotentmatrixis 0is simply i= (X0X) 1X0 i y=c0iyThisis howwedefinea linearestimator- it s a s a linearcombinationoftheobservationsonthed ependentvari-able,wheretheweightsaredete minedbytheobservationsontheregressors,so meobservationsmayhave (PX)tt=e0tPXet=kPXetk2 ketk2=1htis thetthelementonthemaindiagonalofPX(etis anvectorofzeroswitha 1 inthetthposition).So0 ht 1;andT rPX=K)h=K=n:A bettermethodis withoutusingthetthobser-vation(designate thisestimatoras (t)):Onecanshow (seeDavidsonandMacKinnon, )that (t)= 11 ht (X0X) 1X0t t18sothechangeinthetthobservationsfitted valueisXt Xt (t)= ht1 ht tWhileandobservationmaybeinfluentialif it doesn t affectitsownfittedvalue,itcertainlyisinf luentialif it fastmeansofidentifyinginfluentialobserva tionsistoplot ht1 ht tasa ,oneneedstodeterminewhythey commoncauseis a dataentryerror, thedatais thatpurerandomnesscausedustosamplea + Take theinnerproduct:y0y= 0X0X +2 0X0 + 0 ButthemiddletermoftheRHSis zerosinceX0 =0, soy0y= 0X0X + 0 19 TheuncenteredR2uis definedasR2u=1 0 y0y= 0X0X y0y=kPXyk2kyk2=cos2( );where is theanglebetweenyandthespanofX(showwithth eoneregressor, twoobservationexample).

7 TheuncenteredR2changesif weadda constanttoy;sincethischanges . An-other, morecommondefinitionmeasuresthecontribut ionofthevariables,otherthantheconstantte rm,toexplainingthevariationiny: Let = (1;1; :::;1)0;an-vector. SoM =In ( 0 ) 1 0=In 0=nM definedasR2c=1 0 y0M y=1 ESST SSSupposingthatXcontainsa columnofones( ,thereis a constantterm),X0 =0) t t=020soM = :Inthiscasey0M y= 0X0M X + 0 SoR2c=RSST SS Supposingthata columnofonesis inthespacespannedbyX(PX = );thenonecanshow that0 R2c 1 wehave =(X0X) 1X0y=(X0X) 1X0(X + )= 0+ (X0X) 1X0 E( ) = 0:For 2wehave21c 20=1n K 0 =1n K 0M E(c 20) =1n KE(T r 0M )=1n KE(T rM 0)=1n KT rE(M 0)=1n K 20T rM=1n K 20 n T rX(X0X) 1X0 =1n K 20 n T r(X0X) 1X0X = = 0+ (X0X) 1X0 Thisis a linearfunctionof ;whichis N 0;(X0X) 1 20 (Gauss-Markov theorem)TheOLSestimatoris alinearestimator, whichmeansthatit is a linearfunctionofthedependentvariable,y: = (X0X) 1X0 y=Cy22It is alsounbiased, ll =Wy:Iftheestimatoris unbiasedE(Wy)=E(W X 0+W )=W X 0= 0)W X=IKThevarianceof isV( ) =WW0 20:DefineD=W (X0X) 1X0soW=D+ (X0X) 1X0 SinceW X=IK;DX=0;soV( ) = D+ (X0X) 1X0 D+ (X0X) 1X0 0 20= DD0+ (X0X) 1 20 SoV( ) V( ):Thisis a proofoftheGauss-Markov (Gauss-Markov)Undertheclassicalassumptio ns,thevarianceofanylinearunbiasedestimat orminusthevarianceof theOLSestimatoris a positivesemidef-initematrix.

8 It is worthnotingthatwehave notusedthenormalityassumptioninany waytoprove theGauss-Markov theorem,soit is validif theerrorsarenotnormallydistributed, is usefulto reviewtheMLEestimator, sinceundertheassumptionofnormalerrorsthe two sampleofsizenofa randomvectory. SupposethejointdensityofY= y1: : :yn is characterizedbya parametervector 0:fY(Y; 0):Thiswilloftenbereferredtousingthesimp lifiednotationf( 0):Thelikelihoodfunctionis justthisdensityevaluatedat othervalues L(Y; ) =fY(Y; ); 2 ;where is aparameterspace. If thenobservationsareindependent,thelikeli hoodfunctioncanbewrittenasL(Y; ) =n t=1f(yt; )wheretheftarepossiblyofdifferentform. Evenif thisis notpossible,wecanalwaysfactorthelikeliho odintocontributionsofobservations,byusin gthefactthata jointdensitycanbefactoredintotheproducto fa marginalandconditional(doingthisiterativ ely)L(Y; ) =f(y1; )f(y2jy1; )f(y3jy1;y2; ) f(ynjy1;y2; : : :yt n; )25To simplifynotation,definext=fy1;y2; :::;yt 1g;t 2=S;t=1whereSis thesamplespaceofY:(Withthis,conditioning onx1hasnoeffectandgivesa marginalprobability).

9 Now thelikelihoodfunctioncanbewrittenasL(Y; ) =n t=1f(ytjxt; )Thecriterionfunctioncanbedefinedastheav eragelog-likelihoodfunction:sn( ) =1nlnL(Y; ) =1nn t=1lnf(ytjxt; )Themaximumlikelihoodestimatoris definedas =arg maxsn( );wherethesetmaximizedoveris definedbelow. Sinceln( )is a monotonicincreasingfunction,lnLandLmaxim izeat thesamevalueof :Dividingbynhasnoeffecton ,andthereforeit is show consistency oftheMLE,weneedtomake 2 ;a openboundedsubsetof K:Maximixationis26over ;whichis is aninteriorpointoftheparameterspace .Uniformconvergencesn( )u:a:s!limn! E 0sn( ) s ( ; 0);8 2 :We have suppressedYhereforsimplicity. ( )is continuousin ; 2 :Thisimpliesthats ( ; 0)is continuousin :Identifications ( ; 0)hasa willusetheseassumptionstoshow that a:s:! 0:First, certainlyexists,sincea continuousfunctionhasa maximumona ,forany 6= 0E ln L( )L( 0) ln E L( )L( 0) byJensen s inequality( ln( )is a concave function).

10 Now, theexpectationontheRHSisE L( )L( 0) = L( )L( 0)L( 0)dy=1;sinceL( 0)isthedensityfunctionoftheobservations: Therefore,sinceln(1) =0;E ln L( )L( 0) 0;27orE(sn( )) E(sn( 0)) 0:Takinglimits,thisiss ( ; 0) s ( 0; 0) 0exceptona setofzeroprobability(bytheuniformconverg enceassumption).Bytheidentificationassum ptionthereis a uniquemaximizer, sotheinequalityisstrictif 6= 0:s ( ; 0) s ( 0; 0)<0;8 6= 0;However, since is a maximizer, independentofn;wemusthaves ( ; 0) s ( 0; 0) 0:Theselasttwo inequalitiesimplythatlimn! = 0;a:s:Thiscompletestheproofofstrongconsi stency weakconsistency (convergenceinprobabilityto 0) functionDifferentiabilityAssumethatsn( )is twicecontinuouslydifferentiableinN( 0), atleastwhennis maximizethelog-likelihoodfunction,take derivatives:gn(Y; ) =D sn( )=1nn t=1D lnf(ytjxx; ) 1nn t=1gt( ):Thisisthescore vector(withdimK 1):NotethatthescorefunctionhasYasanargum ent,whichimpliesthatit is a , butoneshouldnotforgetthatit is setsthederivativestozero:gn( ) =1nn t=1gt( ) 0:We willshow thatE [gt( )] =0;8t:Thisis theexpectationtakenwithrespecttothedensi tyf( );notnecessarilyf( 0):E [gt( )]= [D lnf(ytjx; )]f(ytjx; )dyt= 1f(ytjxt; )[D f(ytjxt; )]f(ytjxt; )dyt= D f(ytjxt; )dyt:Givensomeregularityconditionsonboun dednessofD f;wecanswitchtheorderofintegrationanddif ferentiation, [gt( )]=D f t(ytjxt.)