Hypothesis Testing

1 Hypothesis TestingECE 3530 Spring 2010 Antonio PaivaWhat is Hypothesis Testing ?A statistical Hypothesis is an assertion or conjectureconcerning one or more prove that a Hypothesis is true, or false, withabsolutecertainty, we would needabsolute knowledge. That is, wewould have to examine theentire , Hypothesis Testing concerns on how to use a randomsample to judge if it is evidence that supports or not is Hypothesis Testing ? (cont.) Hypothesis Testing is formulated in terms oftwohypotheses: H0: the null Hypothesis ; H1: the alternate is Hypothesis Testing ? (cont.)The Hypothesis we want to test is ifH1is likely , there are two possible outcomes: RejectH0and acceptH1because of sufficient evidence inthe sample in favor orH1; Do not rejectH0because of insufficient evidence is Hypothesis Testing ? (cont.)Very important!!Note that failure to rejectH0does not mean the nullhypothesis is true.

2 There is no formal outcome that says acceptH0. It only means that we do not have sufficientevidence to is Hypothesis Testing ? (cont.)ExampleIn a jury trial the hypotheses are: H0: defendant is innocent; H1: defendant is (innocent) is rejected ifH1(guilty) is supported byevidence beyond reasonable doubt. Failure to rejectH0(prove guilty) does not imply innocence, only that theevidence is insufficient to reject studyA company manufacturing RAM chips claims the defectiverate of the population is 5%. Letpdenote thetruedefectiveprobability. We want to test if: H0:p= H1:p > are going to use a sample of 100 chips from theproduction to study (cont.)LetXdenote the number of defective in the sample of 10(chosen arbitrarily in this case).Xis called thetest ,p > notrejectH0criticalvaluecritical regionCase study (cont.)Why did we choose a critical value of 10 for this example?Because this is a Bernoulli process, the expected number ofdefectives in a sample isnp.

3 So, ifp= should expect100 = 5defectives in a sample of 100 chips. Therefore,10 defectives would be strong evidence thatp > problem of how to find a critical value for a desired levelof significance of the Hypothesis test will be studied of errorsBecause we are making a decision based on a finite sample,there is a possibility that we will make possible outcomes are:H0is trueH1is trueDo notrejectH0 CorrectdecisionType IIerrorRejectH0 Type IerrorCorrectdecisionTypes of errors (cont.)DefinitionThe acceptance ofH1whenH0is true is called a Type I probability of committing a type I error is called thelevelof significanceand is denoted by .ExampleConvicting the defendant when he is innocent!The lower significance level , the less likely we are to commita type I error. Generally, we would like small values of ;typically, or of errors (cont.)Case study continued = Pr(Type I error) = Pr(rejectH0whenH0is true)= Pr(X 10whenp= )=100 x=10b(x;n= 100,p= ),binomial distribution=100 x=10(100n) x= , the level of significance is = of errors (cont.)

4 DefinitionFailure to rejectH0whenH1is true is called a Type II probability of committing a type II error is denoted by .Note: It is impossible to compute unless we have a specificalternate of errors (cont.)Case study continuedWe cannot compute forH1:p > the truepisunknown. However, we can compute it for testingH0:p= the alternative Hypothesis thatH1:p= , for instance. = Pr(Type II error) = Pr(rejectH1whenH1is true)= Pr(X <10whenp= )=9 x=0b(x;n= 100,p= ) = of errors (cont.)Case study continuedWhat is the probability of a type II error ifp= = Pr(Type II error)= Pr(X <10whenp= )=9 x=0b(x;n= 100,p= ) = of the critical valueMoving the critical value provides a trade-off between and . A reduction in is always possible by increasing the size ofthe critical region, but this increases . Likewise, reducing ispossible by decreasing the critical of the critical value (cont.)Case study continuedLets see what happens when we change the critical value from10 to 8.

5 That is, we rejectH0ifX notrejectH0criticalvalueold critical regionnew critical regionEffect of the critical value (cont.)Case study continuedThe new significance level is = Pr(X 8whenp= )=100 x=8b(x;n= 100,p= ) = expected, this is a large value than before (it was ).Effect of the critical value (cont.)Case study continuedTesting against the alternate hypothesisH1:p= , = Pr(X <8whenp= )=7 x=0b(x;n= 100,p= ) = ,which is lower than against the alternate hypothesisH1:p= , =7 x=0b(x;n= 100,p= ) = ,again, lower than of the sample sizeBoth and can be reducedsimultaneouslyby increasing thesample study continuedConsider that now the sample size isn= 150and the criticalvalue is 12. Then, rejectH0ifX 12, whereXis now thenumber of defectives in the sample of 150 of the sample size (cont.)Case study continuedThe significance level is = Pr(X 12whenp= )=150 x=12b(x;n= 150,p= ) = that this value is lower than forn= 100andcritical value of of the sample size (cont.)

6 Case study continuedTesting against the alternate hypothesisH1:p= , = Pr(X <12whenp= )=11 x=0b(x;n= 150,p= ) = ,which isalsolower than before (it was ).Approximating the binomial distributionusing the normal distributionFactorials of very large numbers are problematic to computeaccurately, even with Matlab. Thankfully, the binomialdistribution can be approximated by the normal distribution(see Section of the book for details).Approximating the binomial distributionusing the normal distribution (cont.)TheoremIfXis a binomial random variable withntrials and probabilityof success of each trialp, then the limiting form of thedistribution ofZ=X np np(1 p)n is the standard normal approximation is good whennis large andpis notextremely close to 0 or the binomial distributionusing the normal distribution (cont.)Case study continuedLets recompute with the normal approximation. = Pr(Type I error) = Pr(X 12whenp= )=150 x=12b(x;n= 150,p= ) Pr(Z 12 150 150 )= Pr(Z )= 1 Pr(Z ) = 1 = too bad.

7 (It was )Approximating the binomial distributionusing the normal distribution (cont.)Case study continuedWhat if we increase the sample size ton= 500and thecritical value to 40?The normal approximation should be better sincenis larger. Pr(Z 40 500 500 )= Pr(Z )= 1 Pr(Z ) = 1 = unlikely to commit type I the binomial distributionusing the normal distribution (cont.)Case study continuedTesting against the alternate hypothesisH1:p= , =39 x=0b(x;n= 500,p= ) Pr(Z 39 500 500 )= Pr(Z ) = interpretation with normal approximation2533 RejectH0H0is true: = (Type I error rate)H1is true:p= to rejectH0 = (Type II error rate)Visual interpretation with normal approximation2533 RejectH0H0is true: = (Type I error rate)H1is true:p= to rejectH0 = (Type II error rate)Visual interpretation with normal approximation2533 RejectH0H0is true: = (Type I error rate)H1is true:p= to rejectH0 = (Type II error rate)Power of a testDefinitionThepowerof a test is the probability of rejectingH0giventhat a specific alternate Hypothesis is true.

8 That is,Power= 1 .SummaryProperties of Hypothesis testing1. and are related; decreasing one generally increasesthe can be set to a desired value by adjusting the criticalvalue. Typically, is set at or Increasingndecreases both and .4. decreases as the distance between the true value andhypothesized value (H1) vs. two-tailed testsIn our examples so far we have considered: H0: = 0 H1: > is a one-tailed test with the critical region in the right-tailof the test statisticX. 0rejectH0 One-tailed vs. two-tailed tests (cont.)Another one-tailed test could have the form, H0: = 0 H1: < 0,in which the critical region is in the left-tail. 0rejectH0 One-tailed vs. two-tailed tests (cont.)In a two-tailed test check for differences: H0: = 0 H1: 6= 0, 0rejectH0rejectH0 Two-tailed test: exampleConsider a production line of resistors that are supposed to be100 Ohms. Assume = 8. So, the hypotheses are: H0: = 100 H1: 6= 100,Let Xbe the sample mean for a sample of sizen= not rejectH0 RejectH0In this case the test statistic is the sample mean because thisis a continuous random test: example (cont.)

9 98102 = 100area /2area /2We know the sampling distribution of Xis a normaldistribution with mean and standard deviation / n= to the central limit test: example (cont.)Therefore we can compute the probability of a type I error as = Pr( X <98when = 100) + Pr( X >102when = 100)= Pr(Z <98 1008/ 100) + Pr(Z >102 1008/ 100)= Pr(Z < ) + Pr(Z > )= 2 Pr(Z < ) = 2 = intervalTestingH0: = 0againstH1: 6= 0at a significance level is equivalent to computing a100 (1 )%confidenceintervalfor andH0if 0is outside this the previous example the confidence interval at asignificance level = 100 (1 )is[98,102].Tests concerning sample mean(variance known)As in the previous example, we are often interested in Testing H0: = 0 H1: 6= 0,based on the sample mean Xfrom samplesX1,X2,..,Xn,withknownpopulation variance : = 0, the probability of a type I error iscomputed using the sampling distribution of X, which, due tothe central limit theorem, is normal distributed with mean and standard deviation / concerning sample mean (cont.)

10 (variance known)From confidence intervals we know thatPr( z /2< X 0 / n< z /2)= 1 ab X 0area /2area /2area1 rejectH0rejectH0 Tests concerning sample mean (cont.)(variance known)Therefore, to design a test at the level of significance wechoose the critical valuesaandbasa= 0 z /2 nb= 0+z /2 n,then we collect the sample, compute the sample mean XandrejectH0if X < aor X > concerning sample mean (cont.)(variance known)Steps in Hypothesis testing1. State the null and alternate hypothesis2. Choose a significance level 3. Choose the test statistic and establish the critical region4. Collect the sample and compute the test the test statistic is in the critical region, , do not concerning sample mean (cont.)(variance known)ExampleA batch of 100 resistors have an average of 102 a population standard deviation of 8 Ohms, testwhether the population mean is 100 Ohms at a significancelevel of = 1:H0: = 100H1: 6= 100,Note: Unless stated otherwise, we use a two-tailed 2: = concerning sample mean (cont.)