maxLik: A package for maximum likelihood estimation R

1 Comput Stat (2011) 26:443 458 DOI PAPERmaxLik: A package for maximum likelihood estimationinRArne Henningsen Ott ToometReceived: 15 October 2009 / Accepted: 7 September 2010 / Published online: 22 September 2010 Springer-Verlag 2010 AbstractThis paper describes the packagemaxLikfor the statistical environmentR. The package is essentially a unified wrapper interface to various optimizationroutines, offering easy access to likelihood -specific features like standard errors orinformation matrix equality (BHHH method). More advanced features of the optimi-zation algorithms, such as forcing the value of a particular parameter to be fixed, arealso likelihood OptimizationJEL ClassificationC871 IntroductionThe maximum likelihood (ML) method is one of the most important techniquesin statistics and econometrics.

2 Most statistical and econometric software packagesinclude ready-made routines for maximum likelihood estimations of many standardmodels such as logit, probit, sample-selection, count-data, or survival models. How-ever, if practitioners and researchers want to estimate non-standard models or developA. Henningsen (B)Institute of Food and Resource Economics, University of Copenhagen,Rolighedsvej 25, 1958 Frederiksberg C, Denmarke-mail: ToometDepartment of Economics, Aarhus School of Business, University of Aarhus,Hermodsvej 22, 8230 byh j, Denmarke-mail: ToometDepartment of Economics, University of Tartu,Narva 4, 51009 Tartu, Estonia123444A. Henningsen, O. Toometnew estimators, they have to implement the routines for the maximum likelihoodestimations themselves.

3 Several popular statistical packages include frameworks forsimplifying the estimation , allowing the user to easily choose between a number ofoptimization algorithms, different ways of calculating variance-covariance matrices,and easy reporting of the results. The examples include themlcommand instataandthemaxliklibrary free software environment for statistical computing and graphicsR(RDevel-opment Core Team 2009) has included built-in optimization algorithms since its earlydays. The first general-purpose ML framework, functionmlein the built-in packagestats4, was added in 2003, and an extension,mle2in packagebbmle(Bolker 2009),in 2007. However, both of these packages are based on a general-purpose optimizeroptimwhich does not include an option to use the Newton-Raphson algorithm.

4 Inparticular, its variant, the Berndt-Hall-Hall-Hausman algorithm (Berndt et al. 1974),is very popular for ML problems. TheRpackagemaxLik(Toomet and Henningsen2010) is intended to fill this package can be used both by end-users, develop-ing their own statistical methods, and by package developers, implementing ML esti-mators for specific models. For instance, the packagescensReg(Henningsen 2010),mhurdle(Carlevaro et al. 2010),mlogitBMA(Sevcikova and Raftery 2010),pglm(Croissant 2010),sampleSelection(Toomet and Henningsen 2008), andtruncreg(Croissant 2009)usethemaxLikpackage for their maximum likelihood (currently version ) is available from CRAN( ), R-Forge ( ), and its homepage ( ). This paper focuseson the maximum likelihood related usage of the package ; the other features (includingfinite-difference derivatives and optimization) are only briefly paper proceeds as follows: in the next section we explain the implementationof the package .

5 Section3describes the usage of the package , including the basic andmore advanced features, and ImplementationTheRpackagemaxLikis designed to provide a single, unified interface for dif-ferent optimization routines, and to treat the results in a way suitable for max-imum likelihood (ML) estimation . The package implements a flexible multi-pur-pose Newton-Raphson type optimization routine in internal function is not intended to be called by users but functionmax-NRprovides a convenient user-interface and callsmaxNRComputefor the ac-tual optimization. This Newton-Raphson type algorithm is also used as the basisof functionmaxBHHH, which implements a Berndt-Hall-Hall-Hausman type algo-rithm (Berndt et al. 1974) that is popular for ML problems. In addition, the Broy-den-Fletcher-Goldfarb-Shanno algorithm (Broyden 1970;Fletcher 1970;Goldfarb1 The other ML-related packages offer a few capabilities that are missing inmaxLik( likelihoodprofiling).)

6 This may be suboptimal from the user s perspective who might prefer a single ultimate MLpackage. However, a number of partially overlapping projects seems to characterize a large part of the freesoftware : maximum likelihood estimation4451970;Shanno 1970), the Nelder-Mead routine (Nelder and Mead 1965), and asimulated annealing method (B lisle 1992) are available in a unified way in func-tionsmaxBFGS,maxNM, andmaxSANN, respectively. These three functions arepredominantly wrapper functions around the internal functionmaxOptim, whichcalls functionoptim(from the built-in packagestats) for the actual optimiza-tion. ThemaxLikpackage provides a further implementation of the BFGS opti-mizer,maxBFGSR, which unlike theoptim-based methods is written is designed in two layers.

7 The first (innermost) is the opti-mization (maximization) layer: all the maximization routines are designed to have aunified and intuitive interface which allows the user to switch easily between the main arguments have identical names and similar order; only method-specificparameters may vary. These functions can be used for different types of optimizationtasks, both related and not related to the likelihood . They return an S3 object of classmaximincluding both estimated parameters and various diagnostics second layer is the likelihood maximization layer. The most important toolof this layer is the functionmaxLik. Its main purpose is to treat the inputs andmaximization results in a ML-specific way (for instance, computing the variance-covariance matrix based on the estimated Hessian).

8 ThemaxBHHH function belongsto this layer as well, being essentially a call formaxNRusing the information ma-trix equality as the way to approximate the Hessian matrix. A new classmaxLikisadded to the returned maximization object for automatic selection of the maximization layer supports linear equality and inequality constraints. Theequality constraints are estimated using the sequential unconstrained maximizationtechnique (SUMT), which is also implemented in themaxLikpackage. This isachieved by adding a (initially tiny) penalty term, related to violation of the con-straints, to the objective function. Thereafter the problem is repeatedly solved whilethe penalty is increased for every new repetition. The inequality constraints are dele-gated toconstrOptimin the is aware of theconstraints and is able to select a suitable optimization method; however, no attemptis made to correct the resulting variance-covariance matrix (just a warning is printed).

9 As the constrained optimization should still be considered as experimental, we referthe reader to the documentation of the package for is implemented using S3 classes. Corresponding methodscan handle the likelihood -specific properties of the estimate including the fact that theinverse of the negative Hessian is the approximate variance-covariance matrix of theestimated parameters. The most important methods for objects of class"maxLik"are:summaryfor returning (and printing) summary results,coeffor extracting theestimated parameters,vcovfor calculating the variance covariance matrix of the esti-mated parameters,stdErfor calculation standard errors of the estimates,logLikfor extracting the log- likelihood value, andAICfor calculating the Akaike supplies more debugging information compared tooptim-based are grateful to Yves Croissant for providing the core of this Henningsen, O.

10 Toomet3 Using the maxLik Basic usageLike otherRpackages, themaxLikpackage must be installed and loaded before itcan be used. The following command loads themaxLikpackage:> library("maxLik")The most important user interface of themaxLikpackage is a function with the(same) namemaxLik. As explained above,maxLikis mostly a wrapper for differ-ent optimization routines with a few additional features, useful for ML function has two mandatory arguments,logLikandstart. The first argument(logLik) must be a function that calculates the log- likelihood value as a function ofthe parameter (usually parameter vector). The second argument (start)mustbeavector of starting demonstrate the usage of themaxLikpackage by a simple example: we esti-mate the parameters of a normal distribution based on a random sample.