Hypothesis Testing

1 Hypothesis TestingECE 3530 Spring 2010 Antonio PaivaWhat is Hypothesis Testing ?A statistical Hypothesis is an assertion or conjectureconcerning one or more prove that a Hypothesis is true, or false, withabsolutecertainty, we would needabsolute knowledge. That is, wewould have to examine theentire , Hypothesis Testing concerns on how to use a randomsample to judge if it is evidence that supports or not is Hypothesis Testing ? (cont.) Hypothesis Testing is formulated in terms oftwohypotheses: H0: the null Hypothesis ; H1: the alternate is Hypothesis Testing ? (cont.)The Hypothesis we want to test is ifH1is likely , there are two possible outcomes: RejectH0and acceptH1because of sufficient evidence inthe sample in favor orH1; Do not rejectH0because of insufficient evidence is Hypothesis Testing ?

2 (cont.)Very important!!Note that failure to rejectH0does not mean the nullhypothesis is true. There is no formal outcome that says acceptH0. It only means that we do not have sufficientevidence to is Hypothesis Testing ? (cont.)ExampleIn a jury trial the hypotheses are: H0: defendant is innocent; H1: defendant is (innocent) is rejected ifH1(guilty) is supported byevidence beyond reasonable doubt. Failure to rejectH0(prove guilty) does not imply innocence, only that theevidence is insufficient to reject studyA company manufacturing RAM chips claims the defectiverate of the population is 5%.

3 Letpdenote thetruedefectiveprobability. We want to test if: H0:p= H1:p > are going to use a sample of 100 chips from theproduction to study (cont.)LetXdenote the number of defective in the sample of 10(chosen arbitrarily in this case).Xis called thetest ,p > notrejectH0criticalvaluecritical regionCase study (cont.)Why did we choose a critical value of 10 for this example?Because this is a Bernoulli process, the expected number ofdefectives in a sample isnp. So, ifp= should expect100 = 5defectives in a sample of 100 chips. Therefore,10 defectives would be strong evidence thatp > problem of how to find a critical value for a desired levelof significance of the Hypothesis test will be studied of errorsBecause we are making a decision based on a finite sample,there is a possibility that we will make possible outcomes are:H0is trueH1is trueDo notrejectH0 CorrectdecisionType IIerrorRejectH0 Type IerrorCorrectdecisionTypes of errors (cont.)

4 DefinitionThe acceptance ofH1whenH0is true is called a Type I probability of committing a type I error is called thelevelof significanceand is denoted by .ExampleConvicting the defendant when he is innocent!The lower significance level , the less likely we are to commita type I error. Generally, we would like small values of ;typically, or of errors (cont.)Case study continued = Pr(Type I error) = Pr(rejectH0whenH0is true)= Pr(X 10whenp= )=100 x=10b(x;n= 100,p= ),binomial distribution=100 x=10(100n) x= , the level of significance is = of errors (cont.)DefinitionFailure to rejectH0whenH1is true is called a Type II probability of committing a type II error is denoted by.

5 Note: It is impossible to compute unless we have a specificalternate of errors (cont.)Case study continuedWe cannot compute forH1:p > the truepisunknown. However, we can compute it for testingH0:p= the alternative Hypothesis thatH1:p= , for instance. = Pr(Type II error) = Pr(rejectH1whenH1is true)= Pr(X <10whenp= )=9 x=0b(x;n= 100,p= ) = of errors (cont.)Case study continuedWhat is the probability of a type II error ifp= = Pr(Type II error)= Pr(X <10whenp= )=9 x=0b(x;n= 100,p= ) = of the critical valueMoving the critical value provides a trade-off between and . A reduction in is always possible by increasing the size ofthe critical region, but this increases.

6 Likewise, reducing ispossible by decreasing the critical of the critical value (cont.)Case study continuedLets see what happens when we change the critical value from10 to 8. That is, we rejectH0ifX notrejectH0criticalvalueold critical regionnew critical regionEffect of the critical value (cont.)Case study continuedThe new significance level is = Pr(X 8whenp= )=100 x=8b(x;n= 100,p= ) = expected, this is a large value than before (it was ).Effect of the critical value (cont.)Case study continuedTesting against the alternate hypothesisH1:p= , = Pr(X <8whenp= )=7 x=0b(x;n= 100,p= ) = ,which is lower than against the alternate hypothesisH1:p= , =7 x=0b(x;n= 100,p= ) = ,again, lower than of the sample sizeBoth and can be reducedsimultaneouslyby increasing thesample study continuedConsider that now the sample size isn= 150and the criticalvalue is 12.

7 Then, rejectH0ifX 12, whereXis now thenumber of defectives in the sample of 150 of the sample size (cont.)Case study continuedThe significance level is = Pr(X 12whenp= )=150 x=12b(x;n= 150,p= ) = that this value is lower than forn= 100andcritical value of of the sample size (cont.)Case study continuedTesting against the alternate hypothesisH1:p= , = Pr(X <12whenp= )=11 x=0b(x;n= 150,p= ) = ,which isalsolower than before (it was ).Approximating the binomial distributionusing the normal distributionFactorials of very large numbers are problematic to computeaccurately, even with Matlab.

8 Thankfully, the binomialdistribution can be approximated by the normal distribution(see Section of the book for details).Approximating the binomial distributionusing the normal distribution (cont.)TheoremIfXis a binomial random variable withntrials and probabilityof success of each trialp, then the limiting form of thedistribution ofZ=X np np(1 p)n is the standard normal approximation is good whennis large andpis notextremely close to 0 or the binomial distributionusing the normal distribution (cont.)Case study continuedLets recompute with the normal approximation. = Pr(Type I error) = Pr(X 12whenp= )=150 x=12b(x;n= 150,p= ) Pr(Z 12 150 150 )= Pr(Z )= 1 Pr(Z ) = 1 = too bad.

9 (It was )Approximating the binomial distributionusing the normal distribution (cont.)Case study continuedWhat if we increase the sample size ton= 500and thecritical value to 40?The normal approximation should be better sincenis larger. Pr(Z 40 500 500 )= Pr(Z )= 1 Pr(Z ) = 1 = unlikely to commit type I the binomial distributionusing the normal distribution (cont.)Case study continuedTesting against the alternate hypothesisH1:p= , =39 x=0b(x;n= 500,p= ) Pr(Z 39 500 500 )= Pr(Z ) = interpretation with normal approximation2533 RejectH0H0is true: = (Type I error rate)H1is true:p= to rejectH0 = (Type II error rate)Visual interpretation with normal approximation2533 RejectH0H0is true: = (Type I error rate)H1is true:p= to rejectH0 = (Type II error rate)Visual interpretation with normal approximation2533 RejectH0H0is true: = (Type I error rate)H1is true.

10 P= to rejectH0 = (Type II error rate)Power of a testDefinitionThepowerof a test is the probability of rejectingH0giventhat a specific alternate Hypothesis is true. That is,Power= 1 .SummaryProperties of Hypothesis testing1. and are related; decreasing one generally increasesthe can be set to a desired value by adjusting the criticalvalue. Typically, is set at or Increasingndecreases both and .4. decreases as the distance between the true value andhypothesized value (H1) vs. two-tailed testsIn our examples so far we have considered: H0: = 0 H1: > is a one-tailed test with the critical region in the right-tailof the test statisticX.