SOLUTION FOR HOMEWORK 2, STAT 4352

1 SOLUTION FOR HOMEWORK 2, stat 4352 Welcome to your second HOMEWORK . Here we begin our discussion of Point Estimation:Unbiasedness, Efficiency, usual, try to find mistakes (and get extra points) in my solutions . Typically they aresilly arithmetic mistakes (not methodological ones). Theyallow me to check that you didyour HW on your own. Please do not e-mail me about your findings just mention themon the first page of your SOLUTION and count extra let us look at your Problem We have a parametric family of probability spaces (S,F, P , ).It is assumed that an underlying parameter is fixed but unknown to the statistician.

2 It isgiven that two estimators, based on available observationsfrom a space corresponding to anunderlying , satisfyE ( 1) = andE ( 2) = for all considered . As a result, thesetwo estimators are called a statistician suggests to consider a new estimator (a function of observations) 3=k1 1+k2 that this new estimator is a linear combination of the former two. The statisticianwants this new estimator to be unbiased as well. To this end, we needE ( 3) = forall . Write, using the property that the expectation of a sum is equal to the sum ofexpectations and that the expectation of a factor times RV isthe factor times the expectationof the RV,E ( 3) =k1E ( 1) +k2E ( 2) = (k1+k2).

3 This implies that the linear combination of the two unbiasedestimators is again anunbiased estimator iffk1+k2= 1. This is the Problem Here we have a sample of sizenfrom a population with the knownmean and the finite variance 2. Then we can write:E{n 1n i=1(Xi )2}=n 1n i=1E(Xi )2=n 1n i=1 2= that the last line holds for any underlying and , and thus the estimator is Problem Here we again have a sample of sizenfrom a population with theknown mean and the finite variance 2. Then we can write:E X2=E( X + )2=E( X )2+ we know (or check this by a direct calculation)V ar( X) =E( X )2= 2 we getE( X2) = 2+ 2/n,1which yields that X2(the squared sample mean, and recall that Xis always an unbiasedestimator of ) is an asymptotically unbiased estimator of Problem LetE ( ) = , ; this means that the estimator is unbiased.

4 Itis given thatV ar( )6= 0. Then we can writeE ( 2) =E ( + )2=E ( )2+ 2=V ar( ) + : Please recall that what is written above is the familiar formula stating thatthe variance of a RV is equal to the expectation of the squaredRV minus squared mean ofthe RV, and here due to unbiasedness the mean of the RV (the estimator) is equal to conclude that 2is not an unbiased estimator of Problem LetX1, .. , Xnbe iid Poisson( ). Recall thatE (Xi) = andV ar (Xi) = . Also, as usual,E ( X) = for any >0. This yields that Xis an unbiasedestimator of the parameter .To evaluate its efficiency (the minimum variance over all unbiased estimators), we shalluse Cramer-Rao inequality.

5 To use it, we need to calculate variance of the estimator andcompare it with the reciprocal of Fisher information for thesample: if the two coincide thenthe estimator is , on one hand we haveV ar ( X) =V ar (X1)/n= the other hand, we can calculate Fisher information. Recall that Poisson pmf isf (x) =e x/x!,x= 0,1, .., andIX( ) =E ( ln(f (X)/ )2=E [ ( +Xln( ) ln(X!))/ ]2=E [ 1 +X/ ]2=E [1 2X/ +X2/ 2] == 1 2 / + ( + 2)/ 2= 1/ .Here the sample is a sample of iid Poisson RVs, soI(X1,..,Xn)( ) =nIX( ). Thus weconclude that here we have equality in the Cramer-Rao inequality:V ar ( X) =1nIX( ).)

6 This establishes efficiency of the sample mean estimate amongall unbiased estimators of .6. Problem Because 1and 2are independent, and using additional informa-tion that these estimators are unbiased estimators of the parameter , andV ar ( 1) =3V ar ( 2), we can write for 3:=a1 1+a2 2:E ( 3) = (a1+a2) 2andV ar( 3) =a21V ar ( 1) +a22V ar ( 2)= (3a21+a22)V ar( 2).Now we are using those results in turn. First, for 3to be an unbiased estimator we musthavea1+a2= 1. For its variance this implies that3a21+a22= 3(1 2a2+a22) +a22= 3 6a2+ minimize the variance, we need to minimize ina2the above written expression.

7 Thatparabola takes on its minimal value (take derivative, set itto zero, solve the equation, andcheck that this is the point of minimum) ona 2= 3/4. Thena 1= 1 : Choose (a1, a2) = (1/4,3/4).7. Problem Let X1 N( , 21/n) and X2 N( , 22/n). It is also given that thesetwo sample means are independent. [Note that the latter information simplifies calculationof the variance of any linear combination of the two statistics.]Then we study a linear combination :=w X1+ (1 w) X2withw (0,1).(a) Let us establish unbiasedness of . Write,E ( ) =wE ( X1) + (1 w)E ( X2) = (w+ (1 w)) = .What was wished to show.

8 (b) Let us explore the variance of the linear combination. Write,V ar( ) =w2V ar( X1) + (1 w)2V ar( X2)=w2 21/n+ (1 w)2 22/n=n 1[w2( 21+ 22) 2w 22+ 22].The minimum of this variance inwis attained atw = 22/( 21+ 22). [To see the lattertake derivative, set it to zero and solve the equation.] Please note that the optimalw hassense, because the larger 22/ 21the larger our trust into Problem LetY Unif( , + 1) andY(1)is the smallest observation among asample (Y1, .. , Yn) of sizen. Then for any 0< <1 we can writeP(Y(1)> + ) =P(Y1> + , .. , Yn> + )=n i=1P(Yi> + ) =n i=1(1 ) = (1 ) established thatP(|Y(1) |> ) = (1 )n 0 asn.

9 ThusY(1)is a consistent estimator of .39. Problem LetX1, .. , Xnbe iid fromExpon( ). Then, as we known from thetheory of exponential distributionE ( X) =E (X1) = andV ar ( X) =V ar (X1)/n= 2 these results, together with Chebyshev inequality, yieldsP(| X |> ) V ar ( X)/ 2= 2/[ 2n] 0asn . This proves that Xis a consistent estimator of .10. Problem LetX1, .. , Xnbe iid fromExpon( ) distribution. The correspondingjoint pdf isf(X1,..,Xn) (x1, .. , xn) =n i=1(1/ )e xi/ = ne X/[ /n].This factorization in writing the joint pdf, according to the Factorization Theorem, yieldssufficiency of the statistic Problem LetX1andX2be independent andX1 Bin( , n1) andX1 Bin( , n1).

10 Thenf(X1,X2) (x1, x2) =fX1 (x1)fX2 (x2) [this is due to independence]=n1!x1!(n1 x1)! x1(1 )n1 x1n2!x2!(n2 x2)! x2(1 )n x2n1!n2!x1!(n1 x1)!x2!(n x2)! x1+x2(1 )n1+n2 (x1+x2)=n1!n2!x1!(n1 x1)!x2!(n x2)![(1 )n1+n2( /(1 ))x1+x2].This factorization, together with the Factorization Theorem, implies thatX1+X2is asufficient statistic. Plainly a one-to-one transformation of this statistic into a statistic (X1+X2)/(n1+n2) is also a sufficient statistic. Note that the latter is also anunbiased Problem LetX1, .. , Xnbe iid fromGeom( ) distribution. Recall thatXiis the number of failures until first success (sometimes we use the total number of trialsuntil the first success the difference is just 1 trial betweenthe two approaches).