HW-Sol-5-V1 - MIT

1 Statistics for Engineers and Scientists Apr 11. MIT, Spring 2006 Handout #13. Solution 5. Problem 1: Let a > 0 be a known constant, and let > 0 be a parameter. Suppose X1 , X2 , .. , Xn is a sample from a population with one of the following densities. (a) The beta, ( , 1), density: fX (x | ) = x 1 , for 0 < x < 1. a (b) The Weilbull density: fX (x | ) = axa 1 e x , for x > 0. a . (c) The Pareto density: fX (x | ) = x( +1). , for x > a. In each case, nd a real-valued su cient statistic for . Solution Let X (X1 , X2 , .. , Xn ) be a collection of random variables Xi 's, and let x (x1 , x2.)

2 , xn ) be a collection of observed data. (a) For any x, the joint pdf is . n (x x x ) 1 , if i, 0 < xi < 1. 1 2 n fX (x | ) =. 0, otherwise;. = n (x1 x2 xn ) 1 I(0,1) (x1 )I(0,1) (x2 ) I(0,1) (xn ) .. g(T (x) | ) h(x). Factorization theorem implies that T (x) x1 x2 xn is a su cient statistic for . (b) For any x, the joint pdf is . n an (x x x )a 1 e . n xa 1 2 n i=1 i , if i, xi > 0;. fX (x | ) =. 0, otherwise;. n xa = n e . i=1 i . g(T (x) | ). an (x1 x2 xn )a 1. I(0, ) (x1 )I(0, ) (x2 ) I(0, ) (xn ) .. h(x). 1. Factorization theorem implies that n . T (x) xai i=1. is a su cient statistic for.

3 (c) For any x, the joint pdf is . n an . , if i, xi > a;. (x1 x2 xn ) +1. fX (x | ) =. 0, otherwise;. n an . = I(a, ) (x1 )I(a, ) (x2 ) I(a, ) (xn ) . (x1 x2 xn ) +1 .. h(x). g(T (x) | ). Factorization theorem implies that T (x) x1 x2 xn is a su cient statistic for . Problem 2: a) Let X1 , X2 , .. , Xn be independent random variables, each uniformly dis- tributed on the interval [ , ], for some > 0. Find a su cient statistic for . b) Let X1 , X2 , .. , Xn be a random sample of size n from a normal N ( , ). distribution , for some > 0. Find a su cient statistic for . Solution a) For any x (x1 , x2.

4 , xn ), the joint pdf is given by n 1 , if i, xi ;. 2 . fX (x | ) =. 0, otherwise;. n 1 , if min(x1 , .. , xn ) and max(x1 , .. , xn ) ;. 2 . =. 0, otherwise;. 1 n = I[ , ) (min(x1 , .. , xn ))I( , ] (max(x1 , .. , xn )) . 1 . 2 h(x). g(T(x) | ). 2. Factorization theorem implies that T(x) min(x1 , .. , xn ), max(x1 , .. , xn ). is jointly su cient for . b) For any x (x1 , x2 , .. , xn ), the joint pdf is given by 1 n 1 n 2. fX (x) = e 2 i=1 (xi ).. 2 . 1 n 1 n 2 n 2. = e 2 ( i=1 xi 2 i=1 xi +n ).. 2 . 1 n 1 n 2 n n . = e 2 i=1 xi + i=1 xi 2.. 2 . 1 n n 1 n 1 ni=1 x2i n . = e i=1 xi e 2 2.

5 2 .. h(x) g(T (x) | ). Factorization theorem implies that n . T (x) x2i i=1. is a su cient statistic for . Problem 3: Let X be the number of trials up to (and including) the rst success in a sequence of Bernoulli trials with probability of success , for 0 <. < 1. Then, X has a geometric distribution with the parameter : P {X = k} = (1 )k 1 , k = 1, 2, 3, .. Show that the family of geometric distributions is a one-parameter exponential family with T (x) = x. [Hint: x = e ln x , for x > 0.]. Solution Recall that the pmf of a one-parameter ( ) exponential family is of the form p(x | ) = h(x) e ( )T (x) B( ) , where x X.

6 Rewriting the pmf of a Geometric random variable yields P {X = x} = e(x 1) ln(1 )+ln . = ex ln(1 ) (ln(1 ) ln ) , 3. where x {1, 2, 3, .. }. Thus, the geometric distribution is a one-parameter exponential family with h(x) 1 ( ) ln(1 ). T (x) x B( ) ln(1 ) ln . X {1, 2, 3, .. }. Problem 4: Let X1 , X2 , .. , Xn be a random sample of size n from the trun- cated Bernoulli probability mass function (pmf), . p, if x = 1;. P {X = x | p} =. (1 p), if x = 0. (a) Show that the joint pmf of X1 , X2 , .. , Xn is a member of the exponential family of distribution . (b) Find a minimal su cient statistic for p.

7 Solution (a) Let x (X1 , X2 , .. Xn ) denote the collection of Bernoulli random variables. The joint pmf is given by P {X = x | p} = px1 (1 p)1 x1 px2 (1 p)1 x2 pxn (1 p)1 xn n xi n . n xi =p (1 p).. i=1 i=1. n n (ln p) xi e[ln(1 p)][n xi ]. =e . i=1 i=1. n [ln p ln(1 p)] xi +n ln(1 p). =e i=1 , for x {0, 1}n . Therefore, the joint pmf is a member of the exponential family, with the mappings: =p h(x) = 1. n . (p) = ln p ln(1 p) T (x) = xi i=1. B(p) = n ln(1 p) X = {0, 1}n . (b) Let x, y {0, 1}n be given. Consider the likelihood ratio, P {X = x | p} n . n = e[ln p ln(1 p)][ i=1 xi i=1 yi ].

8 P {X = y | p}. 4. De ne a function k(x, y) h(x)/h(y) = 1, which is bounded and non-zero for any x X and y X . n n Note that x and y such that i=1 xi = i=1 yi are equivalent because function k(x, y) satis es the requirement of likelihood ratio partition. n Therefore, T (x) i=1 xi is a su cient statistic. Problem 5: Let X1 , X2 , .. , Xm and Y1 , Y2 , .. , Yn be two independent sam- ples from N ( , 2 ) and N ( , 2 ) populations, respectively. Here, < < , 2 > 0, and 2 > 0. Find a minimal su cient statistic for ( , 2 , 2 ). Solution Let X (X1 , X2 , .. , Xm ) and Y (Y1 , Y2 , .. , Yn ) denote the col- lections of random samples.

9 The joint pdf (of Xj 's and Yi 's), evaluated at x (x1 , x2 , .. , xm ) and y (y1 , y2 , .. , yn ), is given by m m 2. n n 2. 1 j=1 (xj ) 1 i=1 (yi ). fX,Y (x, y | ) = e 2 2 e 2 2. 2 2. 2 x + . 2. 2 12 m x2j 2 12 n yi2 + 2 m n yi B( , 2 , 2 ). =e j=1 i=1 j=1 j 2 i=1 , 2 2. m n . where B( , 2 , 2 ) m 2 n 2. 2 ln 2 + 2 ln 2 + 2 2 + 2 2 . Notice that the joint pdf belongs to the exponential family, so that the minimal statistic for is given by m n m n . T(X, Y) Xj2 , Yi2 , Xj , Yi . j=1 i=1 j=1 i=1. Note: One should not be surprised that the joint pdf belongs to the exponen- tial family of distribution .

10 Recall that gaussian distribution is a member of the exponential family of distribution and that random variables, Xi 's and Yj 's, are mutually independent. Thus, their joint pdf belongs to the exponential family as well. Note: To derive the minimal su cient statistic, one may alternatively consider likelihood ratio partition. The set D0 is de ned to be .. D0 (x, y) Rm+n for all , for all 2 > 0, for all 2 > 0.. fX,Y x, y | , 2 , 2 = 0. = (empty set). 5. Let (x, y) . / D0 and (v, w) . / D0 be given. Their likelihood ratio is given by m m n n fX,Y (x, y | ) 1 2 2 1 2 2. = exp 2 xj vj 2 yi wi fX,Y (v, w | ) 2 j=1 j=1.