SOLUTION FOR HOMEWORK 3, STAT 4352

1 SOLUTION FOR HOMEWORK 3, stat 4352 Welcome to your third HOMEWORK . We finish the point estimation; your Exam 1 is nextweek and it will be close to thatXn:= (X1, .. , Xn) denotes the vector to find mistakes (and get extra points) in my solutions . Typically they are sillyarithmetic mistakes (not methodological ones). They allowme to check that you did yourHW on your own. Please do not e-mail me about your findings just mention them on thefirst page of your SOLUTION and count extra let us look at your Problem LetX1.

2 , Xnbe iid according toExpon( ), sofX (x) = (1/ )e x/ I(x >0), := (0, ).Please note that it is important to write this density with indicator function showing itssupport. In some cases the support may depend on a parameter of interest, and then thisfact is always very important. We shall see such an example inthis the exponential distribution we know thatE (X) = (you may check this by a directcalculation), so we get a simple method of moments estimator M M E= is the answer. But I would like to continue a bit.

3 The method of moments estimator(or a generalized one) allows you to work with any moment (or any function). Let us considerthe second moment and equate sample second moment to the theoretical one. Recall thatV ar (X) = 2, and thusE (X2) =V ar (X) + (E (X))2= 2 sample second moment isn 1 ni=1X2i, and we get another method of moments estimator M M E= [n 1n i=1X2i/2]1 that these MM estimators are different, and this is OK. Then a statistician shouldchoose a better one. Which one do you think is better?

4 You may use the notion of efficiencyto resolve the issue (compare their MSEs (mean squared errors)E( )2and choose anestimator with the smaller MSE). By the way, which estimatoris based on the sufficientstatistic?2. Problem HereX1, .. , XnareP oisson( ). Recall thatE (X) = andV ar (X) = .The MME is easy to get via the first moment, and we have M M E= is the answer. But again, as an extra example, I can suggest a MME based on thesecond moment. Indeed,E (X2) =V ar (X) + (E X)2= + 2and this yields that M M E+ 2M M E=n 1n i= you need to solve this equation to get the MME.

5 Obviouslyit is a more complicatedestimator, but it is yet another Problem LetX1, .. , Xnbe iid according to the pdfg (x) = 1e (x )/ I(x > ).Please note that this is a location-exponential family becauseX= +Z,whereZis a classical exponential RV withfZ (z) = 1e z/ I(z >0). I can go either furtherby saying that we are dealing with a location-scale family becauseX= + Z0,wherefZ0(z) =e zI(z >0).So now we know the meaning of parameters and : the former is the location (shift)and the latter is the scale (multiplier).

6 Note that this understanding simplifies all calculations because you can easily figure out(otherwise do calculations) thatE , (X) = + , V ar , (X) = two familiar results yieldE , (X2) = 2+ ( + )2, and we get the following system oftwo equations to find the pair of MMEs: + = X,2 2+ 2 + 2=n 1n i= solve this system, we square the both sides of the first equality and then subtract theobtained equality from the second equality. We get a new system + = X, 2=n 1n i=1X2i , together with a simple algebra, yields the answer M M E= X [n 1n i=1X2i X2]1/2, M M E= [n 1n i=1X2i X2]1 : We need to check thatn 1 ni=1X2i X2 0 for the estimator to be welldefined.

7 This may be done via famous H older inequality(m j=1aj)2 mm j= Problem HereX1, .. , XnareP oisson( ), = (0, ). Recall thatE (X) = andV ar (X) = . Then, by definition of the MLE: M LE:=argmax n l=1f (Xl) =:argmax LXn( )=argmax n l=1ln(f (Xl)) =: arg max lnLXn( ).For the Poisson pdff (x) =e x/x! we getlnLXn( ) = n +n l=1 Xlln( ) n l=1ln(Xl!).Now we need to find M LEat which the above loglikelihood attains its maximum over all . You can do this in a usual way: take derivative with respectto ( that is, calculate lnLXn( )/ , then equate it to zero, solve with respect to , and then check that thesolution indeed maximizes the loglikelihood).

8 Here equating of the derivative to zero yields n+ nl=1Xl/ = 0, and we get M LE= that for the Poisson setting the MME and MLE coincide; ingeneral they may Problem HereX1, .. , Xnare iidN( , 2) with the mean being known andthe parameter of interest being the variance 2. Note that 2 = (0, ). Then we areinterested in the MLE. Write: 2M LE=argmax 2 lnLXn( 2).HerelnLXx( 2) =n l=1ln([2 2] 1/2e (Xl )2/(2 2)) = (n/2) ln(2 2) (1/2 2)n l=1(Xl ) expression takes on its maximum at 2M LE=n 1n l=1(Xl ) that this is also the Problem LetX1.

9 , Xnbe iid according to the pdfg (x) = 1e (x )/ I(x > ).ThenLXn( , ) = ne nl=1(Xl )/ I(X(1)> ).Recall thatX(1)= min(X1, .. , Xn) is the minimal observation [the first ordered observation].This is the case that I wrote you about earlier: it is absolutely crucial to take into accountthe indicator function (the support) because here the parameter defines the its definition,( M LE, M LE) :=argmax ( , ), (0, )ln(LXn( , )).Note thatL( , ) := ln(LXn( , )) = nln( ) 1n l=1(Xl ) + lnI(X(1) ).

10 Now the crucial step: you should graph the loglikelihoodLas a function in and visualizethat it takes on maximum when =X(1). So we get M LE=X(1). Then by taking aderivative we get that M LE=n 1 nl=1(Xl X(1)).Answer: ( M LE M LE) = (X(1), n 1 nl=1(Xl X(1)).Please note that M LEis a biasedestimator; this is a rather typical Problem Consider iid uniform observationsX1, .. , Xnwith the parametric pdff (x) =I( 1/2< x < + 1/2).As soon as the parameter is in the indicator function you should be very cautious: typicallya graphic will help you to find a MLE estimator, and not a differentiation.)