Maximum Likelihood Estimation and Nonlinear …

1 Maximum Likelihood Estimation andNonlinear Least Squares in StataChristopher F BaumFaculty Micro Resource CenterBoston CollegeJuly 2007 Christopher F Baum (Boston College FMRC)ML / NL in StataJuly 20071 / 53 Maximum Likelihood Estimation in StataA key resourceMaximum Likelihood estimationA key resource is the bookMaximum Likelihood Estimation in Stata,Gould, Pitblado and Sribney, Stata Press: 3d ed., 2006. A good deal ofthis presentation is adapted from that excellent treatment of thesubject, which I recommend that you buy if you are going to work withMLE in perform Maximum Likelihood Estimation (MLE) in Stata, you mustwrite a short Stata program defining the Likelihood function for yourproblem.

2 In most cases, that program can be quite general and may beapplied to a number of different model specifications without the needfor modifying the F Baum (Boston College FMRC)ML / NL in StataJuly 20072 / 53 Maximum Likelihood Estimation in StataA key resourceMaximum Likelihood estimationA key resource is the bookMaximum Likelihood Estimation in Stata,Gould, Pitblado and Sribney, Stata Press: 3d ed., 2006. A good deal ofthis presentation is adapted from that excellent treatment of thesubject, which I recommend that you buy if you are going to work withMLE in perform Maximum Likelihood Estimation (MLE) in Stata, you mustwrite a short Stata program defining the Likelihood function for yourproblem.

3 In most cases, that program can be quite general and may beapplied to a number of different model specifications without the needfor modifying the F Baum (Boston College FMRC)ML / NL in StataJuly 20072 / 53 Maximum Likelihood Estimation in StataExample: binomial probitLet s consider the simplest use of MLE: a model that estimates abinomial probit equation, as implemented in official Stata by theprobitcommand. We code our probit ML program as:program myprobit_lfversion lnf xbquietly replace lnf = ln(normal( xb )) ///if $ML_y1 == 1quietly replace lnf = ln(normal( - xb )) ///if $ML_y1 == 0endChristopher F Baum (Boston College FMRC)ML / NL in StataJuly 20073 / 53 Maximum Likelihood Estimation in StataExample: binomial probitThis program is suitable for ML Estimation in thelinear formorlfcontext.

4 The local macrolnfcontains the contribution to log-likelihoodof each observation in the defined sample. As is generally the casewith Stata sgenerateandreplace, it is not necessary to loop overthe observations. In the linear form context, the program need not sumup the F Baum (Boston College FMRC)ML / NL in StataJuly 20074 / 53 Maximum Likelihood Estimation in StataExample: binomial probitSeveral programming constructs show up in this example. Theargsstatement defines the program sarguments:lnf, the variable that willcontain the value of log- Likelihood for each observation, andxb, thelinear form: a single variable that is the product of the X matrix andthe current vectorb.

5 The arguments are local macros within program replaces the values oflnfwith the appropriatelog- Likelihood values, conditional on the value of$ML_y1: the firstdependent variable or y -variable. Thus, the program may be appliedto any 0 1 variable as a function of any set of X variables F Baum (Boston College FMRC)ML / NL in StataJuly 20075 / 53 Maximum Likelihood Estimation in StataExample: binomial probitSeveral programming constructs show up in this example. Theargsstatement defines the program sarguments:lnf, the variable that willcontain the value of log- Likelihood for each observation, andxb, thelinear form: a single variable that is the product of the X matrix andthe current vectorb.

6 The arguments are local macros within program replaces the values oflnfwith the appropriatelog- Likelihood values, conditional on the value of$ML_y1: the firstdependent variable or y -variable. Thus, the program may be appliedto any 0 1 variable as a function of any set of X variables F Baum (Boston College FMRC)ML / NL in StataJuly 20075 / 53 Maximum Likelihood Estimation in StataExample: binomial probitGiven the program stored in the theADOPATH how do we execute it?webuse auto, cleargen gpm = 1/mpgml model lf myprobit_lf ///(foreign = price gpm displacement)ml maximizeTheml modelstatement defines the context to be the linear form(lf), the Likelihood evaluator to bemyprobit_lf, and then specifiesthe model.

7 The binary variableforeignis to be explained by thefactorsprice, gpm, displacement, by default including aconstant term in the relationship. Theml modelcommand onlydefines the model: it does not estimate it. That is performed with theml F Baum (Boston College FMRC)ML / NL in StataJuly 20076 / 53 Maximum Likelihood Estimation in StataExample: binomial probitGiven the program stored in the theADOPATH how do we execute it?webuse auto, cleargen gpm = 1/mpgml model lf myprobit_lf ///(foreign = price gpm displacement)ml maximizeTheml modelstatement defines the context to be the linear form(lf), the Likelihood evaluator to bemyprobit_lf, and then specifiesthe model.

8 The binary variableforeignis to be explained by thefactorsprice, gpm, displacement, by default including aconstant term in the relationship. Theml modelcommand onlydefines the model: it does not estimate it. That is performed with theml F Baum (Boston College FMRC)ML / NL in StataJuly 20076 / 53 Maximum Likelihood Estimation in StataExample: binomial probitYou can verify that this routine duplicates the results of applyingprobitto the same model. Note that our ML program producesestimation results in the same format as an official Stata command. Italso stores its results asereturns, so that postestimationcommands such astestandlincom are course, we need not roll our own binomial probit.

9 To understandhow we might apply Stata s ML commands to a Likelihood function ofour own, we must establish some notation, and explain what the linearform context F Baum (Boston College FMRC)ML / NL in StataJuly 20077 / 53 Maximum Likelihood Estimation in StataExample: binomial probitYou can verify that this routine duplicates the results of applyingprobitto the same model. Note that our ML program producesestimation results in the same format as an official Stata command. Italso stores its results asereturns, so that postestimationcommands such astestandlincom are course, we need not roll our own binomial probit. To understandhow we might apply Stata s ML commands to a Likelihood function ofour own, we must establish some notation, and explain what the linearform context F Baum (Boston College FMRC)ML / NL in StataJuly 20077 / 53 Maximum Likelihood Estimation in StataBasic notationThe log- Likelihood function can be written as a function of variables andparameters:`=lnL{( 1j, 2j.)}

10 , Ej;y1j,y2j,..,yDj),j=1,N} ij=xij i= i0+xij1 i1+ +xijk ikor in terms of the whole sample:`=lnL( 1, 2,.., E;y1,y2,..,yD) i=Xi iwhere we haveDdependent variables,Eequations (indexed byi) andthe data matrixXifor theithequation, containingNobservationsindexed F Baum (Boston College FMRC)ML / NL in StataJuly 20078 / 53 Maximum Likelihood Estimation in StataBasic notationIn the special case where the log- Likelihood contribution can becalculated separately for each observation and the sum of thosecontributions is the overall log- Likelihood , the model is said to meet thelinear form restrictions:ln`j=ln`( 1j, 2j,.., Ej;y1j,y2j,..,yDj)`=N j=1ln`jwhich greatly simplify the task of specifying the model.