Chapter 7 Pearson’s chi-square test

1 Chapter 7 pearson s chi-square Null hypothesis asymptoticsLetX1,X2, be independent from a multinomial(1,p) distribution, wherepis ak-vectorwith nonnegative entries that sum to one. That is,P(Xij= 1) = 1 P(Xij= 0) =pjfor all 1 j k( )and eachXiconsists of exactlyk 1 zeros and a single one, where the one is in the componentof the success category at triali. Note that the multinomial distribution is a generalizationof the binomial distribution to the case in which there arekcategories of outcome insteadof only purpose of this section is to derive the asymptotic distribution of the pearson chi-squarestatistic 2=k j=1(nj npj)2npj,( )wherenjis the random variablenXj, the number of successes in thejth category for trials1.

2 , n. In a real application, the true value ofpis not known, but instead we assumethatp=p0for some null valuep0. We will show that 2converges in distribution to thechi- square distribution onk 1 degrees of freedom, which yields to the familiar chi-squaretest of goodness of fit for a multinomial ( ) implies that VarXij=pj(1 pj). Furthermore, Cov (Xij, Xi`) = EXijXi` 109pjp`= pjp`forj6=`. Therefore, the random vectorXihas covariance matrix = p1(1 p1) p1p2 p1pk p1p2p2(1 p2) p1pk p2pk pk(1 pk) .( )Since EXi=p, the central limit theorem implies n(Xn p)d Nk(0, ).

3 ( )Note that the sum of thejth column of ispj pj(p1+ +pk) = 0, which is to say thatthe sum of the rows of is the zero vector, so is not now present two distinct derivations of this asymptotic distribution of the 2statisticin equation ( ), because each derivation is instructive. One derivation avoids dealing withthe singular matrix , whereas the other does the first approach, define for eachiYi= (Xi1, .. , Xi,k 1). That is, letYibe thek 1-vector consisting of the firstk 1 components ofXi. Then the covariance matrix ofYiisthe upper-left (k 1) (k 1) submatrix of , which we denote by.

4 Similarly, letp denote the vector (p1, .. , pk 1).One may verify that is invertible and that( ) 1= 1p1+1pk1pk 1pk1pk1p2+1pk 1pk 1+1pk .( )Furthermore, the 2statistic of equation ( ) by be rewritten as 2=n(Y p )T( ) 1(Y p ).( )The facts in Equations ( ) and ( ) are checked in Problem If we now defineZn= n( ) 1/2(Y p ),then the central limit theorem impliesZnd Nk 1(0, I). By definition, the 2k 1distributionis the distribution of the sum of the squares ofk 1 independent standard normal randomvariables. Therefore, 2= (Zn)TZnd 2k 1,( )110which is the result that leads to the familiar chi-square a second approach to deriving the limiting distribution ( ), we use some properties ofprojection symmetric matrixPis called a projection matrix if it is idempotent;that is, ifP2= following lemmas, to be proven in Problem , give some basic facts about a projection matrix.

5 Then every eigenvalue ofPequals 0or 1. Suppose thatrdenotes the number of eigenvalues ofPequal to 1. Then ifZ Nk(0, P),ZTZ trace of a square matrixM, Tr (M), is equal to the sum of itsdiagonal entries. For matricesAandBwhose sizes allow them to be multipliedin either order, Tr (AB) = Tr (BA).Recall (Lemma ) that if a square matrixMis symmetric, then there exists an orthogonalmatrixQsuch thatQM QTis a diagonal matrix whose entries consist of the eigenvalues ofM. By Lemma , Tr (QM QT) = Tr (QTQM) = Tr (M), which proves yet another lemma:Lemma symmetric, then Tr (M) equals the sum of the eigenvalues = diag (p), and let be defined as in Equation ( ).

6 Equation ( ) implies n 1/2(X p)d Nk(0, 1/2 1/2).Since may be written in the form ppT, 1/2 1/2=I 1/2ppT 1/2=I p pT( )has tracek 1; furthermore,(I p pT)(I p pT) =I 2 p pT+ p pT p pT=I p pTbecause pT p= 1, so the covariance matrix ( ) is a projection n 1/2(X p). Then we may check (in problem ) that 2= (An)TAn.( )Therefore, since the covariance matrix ( ) is a projection with tracek 1, Lemma andLemma prove that 2d 2k 1as for Section (1),X(2), ..are independent and identicallydistributed from somek-dimensional distribution with mean and finite nonsin-gular covariance matrix.

7 LetSndenote the sample covariance matrixSn=1n 1n j=1(X(j) X)(X(j) X) testH0: = 0againstH1: 6= 0, define the statisticT2= (V(n))TS 1n(V(n)),whereV(n)= n(X 0). This is called Hotelling sT2statistic.[Notes: This is a generalization of the square of a unidimensional t statistic. If thesample is multivariate normal, then [(n k)/(nk k)]T2is distributed asFk,n pearson chi square statistic may be shown to be a special case of Hotelling sT2. ](a)You may assume thatS 1nP 1(this follows from the weak law of largenumbers sinceP(Snis nonsingular) 1). Prove that under the null hypothesis,T2d 2k.

8 (b)Let{ (n)}be alternatives such that n( (n) 0) . You may assumethat under{ (n)}, n(X (n))d Nk(0, ).Find (with proof) the limit of the power against the alternatives{ (n)}of thetest that rejectsH0whenT2 c , whereP( 2k> c ) = .(c)An approximate 1 confidence set for based on the result in part (a)may be formed by plotting the elliptical set{ :n(X )TS 1n(X ) =c }.For a random sample of size 100 fromN2(0, ), where =(1 3/53/5 1), pro-duce a scatterplot of the sample and plot 90% and 99% confidence sets on :In part (c), to produce a random vector with theN2(0, ) distribution,take aN2(0, I) random vector and left-multiply by a matrixAsuch thatAAT=112.

9 It is not hard to find such anA(it may be taken to be lower triangular). Oneway to graph the ellipse is to find a matrixBsuch thatBTS 1nB=I. Then notethat{ :n(X )TS 1n(X ) =c }={X B : T =c /n},so it remains only to find points , closely spaced, such that T equals a con-stant. To find a matrixBsuch as the one specified, note that the matrix ofeigenvectors ofSn, properly normalized, gives an orthogonal matrix that Equations ( ) and ( ).Exercise Lemma and Lemma , then verify Equation ( ).Exercise s chi-square for a 2-way table: Product multinomial categorical variables with 2 andklevels, respectively, and wecollect random samples of sizemandnfrom levels 1 and 2 ofA, then classifyeach individual according to its level of the variableB, the results of this studymay be summarized in a 2 ktable.

10 The standard test of the independence ofvariablesAandBis the pearson chi-square test, which may be written as all cells in table(Oj Ej)2Ej,whereOjis the observed count in celljandEjis the estimate of the expectedcount under the null hypothesis. Equivalently, we may set up the problem asfollows: IfXandYare independent Multinomial(m,p) and Multinomial (n,p)random vectors, respectively, then the pearson chi-square statistic isW2=k j=1{(Xj mZj/N)2mZj/N+(Yj nZj/N)2nZj/N},whereZ=X+YandN=n+m. (Note: I used W2to denote the chi-squarestatistic to avoid using yet another variable that looks like anX.)