Chapter 7 Pearson’s chi-square test

1 Chapter 7 Pearson s chi-square Null hypothesis asymptoticsLetX1,X2, be independent from a multinomial(1,p) distribution, wherepis ak-vectorwith nonnegative entries that sum to one. That is,P(Xij= 1) = 1 P(Xij= 0) =pjfor all 1 j k( )and eachXiconsists of exactlyk 1 zeros and a single one, where the one is in the componentof the success category at triali. Note that the multinomial distribution is a generalizationof the binomial distribution to the case in which there arekcategories of outcome insteadof only purpose of this section is to derive the asymptotic distribution of the Pearson chi-squarestatistic 2=k j=1(nj npj)2npj,( )wherenjis the random variablenXj, the number of successes in thejth category for trials1, .. , n. In a real application, the true value ofpis not known, but instead we assumethatp=p0for some null valuep0. We will show that 2converges in distribution to thechi-square distribution onk 1 degrees of freedom, which yields to the familiar chi-squaretest of goodness of fit for a multinomial ( ) implies that VarXij=pj(1 pj).

2 Furthermore, Cov (Xij, Xi`) = EXijXi` 109pjp`= pjp`forj6=`. Therefore, the random vectorXihas covariance matrix = p1(1 p1) p1p2 p1pk p1p2p2(1 p2) p1pk p2pk pk(1 pk) .( )Since EXi=p, the central limit theorem implies n(Xn p)d Nk(0, ).( )Note that the sum of thejth column of ispj pj(p1+ +pk) = 0, which is to say thatthe sum of the rows of is the zero vector, so is not now present two distinct derivations of this asymptotic distribution of the 2statisticin equation ( ), because each derivation is instructive. One derivation avoids dealing withthe singular matrix , whereas the other does the first approach, define for eachiYi= (Xi1, .. , Xi,k 1). That is, letYibe thek 1-vector consisting of the firstk 1 components ofXi. Then the covariance matrix ofYiisthe upper-left (k 1) (k 1) submatrix of , which we denote by . Similarly, letp denote the vector (p1.)

3 , pk 1).One may verify that is invertible and that( ) 1= 1p1+1pk1pk 1pk1pk1p2+1pk 1pk 1+1pk .( )Furthermore, the 2statistic of equation ( ) by be rewritten as 2=n(Y p )T( ) 1(Y p ).( )The facts in Equations ( ) and ( ) are checked in Problem If we now defineZn= n( ) 1/2(Y p ),then the central limit theorem impliesZnd Nk 1(0, I). By definition, the 2k 1distributionis the distribution of the sum of the squares ofk 1 independent standard normal randomvariables. Therefore, 2= (Zn)TZnd 2k 1,( )110which is the result that leads to the familiar chi-square a second approach to deriving the limiting distribution ( ), we use some properties ofprojection symmetric matrixPis called a projection matrix if it is idempotent;that is, ifP2= following lemmas, to be proven in Problem , give some basic facts about a projection matrix. Then every eigenvalue ofPequals 0or 1.

4 Suppose thatrdenotes the number of eigenvalues ofPequal to 1. Then ifZ Nk(0, P),ZTZ trace of a square matrixM, Tr (M), is equal to the sum of itsdiagonal entries. For matricesAandBwhose sizes allow them to be multipliedin either order, Tr (AB) = Tr (BA).Recall (Lemma ) that if a square matrixMis symmetric, then there exists an orthogonalmatrixQsuch thatQM QTis a diagonal matrix whose entries consist of the eigenvalues ofM. By Lemma , Tr (QM QT) = Tr (QTQM) = Tr (M), which proves yet another lemma:Lemma symmetric, then Tr (M) equals the sum of the eigenvalues = diag (p), and let be defined as in Equation ( ). Equation ( ) implies n 1/2(X p)d Nk(0, 1/2 1/2).Since may be written in the form ppT, 1/2 1/2=I 1/2ppT 1/2=I p pT( )has tracek 1; furthermore,(I p pT)(I p pT) =I 2 p pT+ p pT p pT=I p pTbecause pT p= 1, so the covariance matrix ( ) is a projection n 1/2(X p).

5 Then we may check (in problem ) that 2= (An)TAn.( )Therefore, since the covariance matrix ( ) is a projection with tracek 1, Lemma andLemma prove that 2d 2k 1as for Section (1),X(2), ..are independent and identicallydistributed from somek-dimensional distribution with mean and finite nonsin-gular covariance matrix . LetSndenote the sample covariance matrixSn=1n 1n j=1(X(j) X)(X(j) X) testH0: = 0againstH1: 6= 0, define the statisticT2= (V(n))TS 1n(V(n)),whereV(n)= n(X 0). This is called Hotelling sT2statistic.[Notes: This is a generalization of the square of a unidimensional t statistic. If thesample is multivariate normal , then [(n k)/(nk k)]T2is distributed asFk,n Pearson chi square statistic may be shown to be a special case of Hotelling sT2. ](a)You may assume thatS 1nP 1(this follows from the weak law of largenumbers sinceP(Snis nonsingular) 1).

6 Prove that under the null hypothesis,T2d 2k.(b)Let{ (n)}be alternatives such that n( (n) 0) . You may assumethat under{ (n)}, n(X (n))d Nk(0, ).Find (with proof) the limit of the power against the alternatives{ (n)}of thetest that rejectsH0whenT2 c , whereP( 2k> c ) = .(c)An approximate 1 confidence set for based on the result in part (a)may be formed by plotting the elliptical set{ :n(X )TS 1n(X ) =c }.For a random sample of size 100 fromN2(0, ), where =(1 3/53/5 1), pro-duce a scatterplot of the sample and plot 90% and 99% confidence sets on :In part (c), to produce a random vector with theN2(0, ) distribution,take aN2(0, I) random vector and left-multiply by a matrixAsuch thatAAT=112 . It is not hard to find such anA(it may be taken to be lower triangular). Oneway to graph the ellipse is to find a matrixBsuch thatBTS 1nB=I. Then notethat{ :n(X )TS 1n(X ) =c }={X B : T =c /n},so it remains only to find points , closely spaced, such that T equals a con-stant.

7 To find a matrixBsuch as the one specified, note that the matrix ofeigenvectors ofSn, properly normalized, gives an orthogonal matrix that Equations ( ) and ( ).Exercise Lemma and Lemma , then verify Equation ( ).Exercise s chi-square for a 2-way table: Product multinomial categorical variables with 2 andklevels, respectively, and wecollect random samples of sizemandnfrom levels 1 and 2 ofA, then classifyeach individual according to its level of the variableB, the results of this studymay be summarized in a 2 ktable. The standard test of the independence ofvariablesAandBis the Pearson chi-square test, which may be written as all cells in table(Oj Ej)2Ej,whereOjis the observed count in celljandEjis the estimate of the expectedcount under the null hypothesis. Equivalently, we may set up the problem asfollows: IfXandYare independent Multinomial(m,p) and Multinomial (n,p)random vectors, respectively, then the Pearson chi-square statistic isW2=k j=1{(Xj mZj/N)2mZj/N+(Yj nZj/N)2nZj/N},whereZ=X+YandN=n+m.

8 (Note: I used W2to denote the chi-squarestatistic to avoid using yet another variable that looks like anX.)Using the result of Equation ( ) on p. 329, prove that ifN in such away thatn/N (0,1), thenW2d 2k s chi-square for a 2-way table: Multinomial model. Now con-sider the case in which (X,Y) is a single multinomial (N,q) random still denote the (1, i) entry in a 2 ktable, andYiwill still denote the(2, i) entry.(a)In this case,qis a 2k-vector. Let =q1/(q1+qk+1) and definepto be thek-vector such that (q1, .. , qk) = p. Prove that under the usual null hypothesisthat variableAis independent of variableB( , the row variable and the columnvariable are independent),q= ( p,(1 )p) andp1+ +pk= 1.(b)As in Problem , letZ=X+Y. Assume the null hypothesis is true andsuppose that for some reason is known. The Pearson chi-square statistic maybe written asW2=k j=1{(Xj Zj)2 Zj+(Yj (1 )Zj)2(1 )Zj}.

9 ( )Find the joint asymptotic distribution of N (1 )(X1N Y1N(1 ), .. ,XkN Ykn(1 ))and use this result to prove that W2d Problem (b), it was assumed that was known. However, in mostproblems this assumption is unrealistic. Therefore, we replace all occurrences of in Equation ( ) by = ki=1Xi/N. This results in a different asymptoticdistribution for the W2statistic. Suppose we are given the following multinomialprobabilities for a 2 2 table with independent row and column variables:P(X1= 1) =.1P(X2= 1) =. (Y1= 1) =.3P(Y2= 1) =. that =.25 in the above table. LetN= 50 and simulate 1000 multinomialrandom vectors with the above probabilities. For each, calculate the value ofW2using both the known value =.25 and the value estimated from thedata. Plot the empirical distribution function of each of these two sets of 1000values. Compare with the theoretical distribution functions for the 21and :To generate a multinomial random variable with expectation vectormatching the table above, because of the independence inherent in the table youcan generate two independent Bernoulli random variables with respective successprobabilities equal to the margins: That is, letP(A= 2) = 1 P(A= 1) =.

10 6114andP(B= 2) = 1 P(B= 1) =.75, then classify the multinomial observationinto the correct cell based on the random values following example comes from genetics. There is a particular char-acteristic of human blood (the so-called MN blood group) that has three types:M, MN, and N. Under idealized circumstances, which we assume to be true forthe purposes of this problem, these three types occur in the population withprobabilitiesp1= 2M,p2= 2 M N, andp3= 2N, respectively, where Mis thefrequency of the M allele in the population and N= 1 Mis the frequency ofthe N the value of Mwere known, then the asymptotic distribution of the Pearson 2statistic would be given in the development earlier in this section. However,of course we usually don t know M. Instead, we estimate it using the maximumlikelihood estimator (2n1+n2)/2n.(a)DefineBn= n 1/2(X p), where pis the MLE forp.