Chapter 6 Likelihood Inference - Department of Statistical ...

1 Chapter 6 Likelihood InferenceCHAPTER OUTLINES ection 1 The Likelihood FunctionSection 2 Maximum Likelihood EstimationSection 3 Inferences Based on the MLES ection 4 Distribution-Free MethodsSection 5 Large Sample Behavior of the MLE (Advanced)In this Chapter , we discuss some of the most basic approaches to Inference . In essence,we want our inferences to depend only on the model{P : *}and the methods are very minimal in the sense that they require few assumptions. Whilesuccessful for certain problems, it seems that the additional structure of Chapter 7 orChapter 8 is necessary in more involved Likelihood function is one of the most basic concepts in Statistical theories of Inference have been constructed based on it.

2 We discuss likeli-hood methods in Sections , , , and In Section , we introduce somedistribution-free methods of Inference . These are not really examples of likelihoodmethods, but they follow the same basic idea of having the inferences depend on asfew assumptions as Likelihood FunctionLikelihood inferences are based only on the datasand the model{P : *} theset of possible probability measures for the system under theseingredients we obtain the basic entity of Likelihood Inference , namely, the motivate the definition of the Likelihood function, suppose we have a statisticalmodel in which eachP is discrete, given by probability functionf.

3 Having observeds,consider the functionL( |s)defined on the parameter space*and taking valuesinR1,givenbyL( |s)=f (s).We r e f e r t oL( |s)as thelikelihood functionde-termined by the model and the data. The valueL( |s)is called thelikelihoodof .297298 Section : The Likelihood FunctionNote that for the Likelihood function, we arefixing the data and varying the value ofthe see thatf (s)is just the probability of obtaining the dataswhen the true valueof the parameter is .This imposes a belief ordering on*, namely, we believe in 1asthetruevalueof over 2wheneverf 1(s)>f 2(s).

4 This is because the inequalitysays that the data are more likely under 1than are indifferent between 1and 2wheneverf 1(s)=f 2(s). Likelihood Inference about is important to remember the correct interpretation ofL( |s).The valueL( |s)is the probability ofsgiven that is the true value it isnotthe probability of giventhat we have , it is possible that the value ofL( |s)is very small forevery value of .So it is not the actual value of the Likelihood that is telling us howmuch support to give to a particular , but rather its value relative to the likelihoods ofother possible parameter {1,2.}

5 }and that the Statistical model is{P : {1,2}},whereP1isthe uniform distribution on the integers41,..,1035andP2is the uniform distributionon41,.., suppose that we observes=10. ThenL(1|10)=1/103andL(2|10)=1/106. Both values are quite small, but note that the Likelihood sup-ports =1 a thousand times more than it supports = , we are only interested inlikelihood ratios L( 1|s)/L( 2|s)for 1, 2 *when it comes to determining inferences for based on the implies that any function that is a positive multiple ofL( |s), ,L ( |s)=cL( |s)for somefixedc>0,can serve equally well as a likelihoodfunction.

6 We call two likelihoods equivalent if they are proportional in this way. Ingeneral, we refer to any positive multiple ofL( |s)as a Likelihood that a coin is tossedn=10 times and thats=4 heads are observed. Withno knowledge whatsoever concerning the probability of getting a head on a singletoss, the appropriate Statistical model for the data is the Binomial(10, )model with *=[0,1].The Likelihood function is given byL( |4)=21043 4(1 )6,( )which is plotted in Figure Likelihood peaks at = and takes the value there. We will ex-amine uses of the Likelihood to estimate the unknown and assess the accuracy of theestimate.

7 Roughly speaking, however, this is based on where the Likelihood takes itsmaximum and how much spread there is in the Likelihood about its 6: Likelihood : Likelihood function from the Binomial(10, )model whens=4is is a range of approaches to obtaining inferences via the Likelihood one extreme is the Likelihood Principle: If two model and data combinations yield equivalentlikelihood functions, then inferences about the unknown parameter mustbe the principle dictates that anything we want to say about the unknown value of must be based only onL( |s).

8 For many statisticians, this is viewed as a very severeproscription. Consider the following a coin is tossed in independent tosses until four heads are obtained and thenumber of tails observed until the fourth head iss= distributed Negative-Binomial(4, ), and the Likelihood specified by the observed data isL( |6)=2963 4(1 ) that this Likelihood function is a positive multiple of ( ).So the Likelihood principle asserts that these two model and data combinations mustyield the same inferences about the unknown . In effect, the Likelihood principle sayswe must ignore the fact that the data were obtained in entirely different ways.

9 If, how-ever, we take into account additional model features beyond the Likelihood function,then it turns out that we can derive different inferences for the two situations. In partic-ular, assessing a hypothesized value = 0can be carried out in different ways whenthe sampling method is taken into account. Many statisticians believe this additionalinformation should be used when deriving : The Likelihood FunctionAs an example of an Inference derived from a Likelihood function, consider a set ofthe formC(s)={ :L( |s) c},for somec setC(s)is referred to as alikelihood region.

10 It contains all those values for which their Likelihood is at Likelihood region, for somec,seemslike a sensible set to quote as possibly containing the true value of . For, if / C(s),thenL( |s)<L( |s)for every C(s)and so is not as well-supported by theobserved data as any value inC(s). The size ofC(s)can then be taken as a measure ofhow uncertain we are about the true value of .We are left with the problem, however, of choosing a suitable value forcand, asExample seems to indicate, the Likelihood itself does not suggest a natural way todo this. In Section , we will discuss a method for choosingcthat is based uponadditional model properties beyond the Likelihood far in this section, we have assumed that our Statistical models are comprisedof discrete distributions.