Equivalence Tests - MMB Home Page

1 Equivalence TestsMathews Malnar and Bailey, Third Street, Fairport Harbor, OH 44077 Phone: 440-350-0911 Website: : 2015 2015 Mathews Malnar and Bailey, We ll start with a review of significance Tests and then transition toequivalence Tests . We ll use the two-samplettest for means for this discussion. Understand that these methods also apply to: other parameters ( standard deviations, proportions,counts, etc.) problems involving one, two, and many Malnar & Bailey, Inc., Equivalence Tests2 Review:Two-SampletTest The hypotheses for the two-sample location problem areH0: 1 2HA: 1 2or in terms of 1 2H0: 0HA: 0 This test is asignificance testbecause its purpose is to demonstrateHA-that there is a significant difference between the two population means. So acceptable decisions are limited to either rejectingH0in favor ofHAor reserving judgement, we never ExampleExample: An experiment was performed to test for a differencebetween two population means.

2 The two samples yielded the followingresults:n1 8,x 1 ,s1 10,x 2 ,s2 the hypothesis test usingH0: 1 2versusHA: 1 2and construct the 95% confidence interval for the bias between thepopulation Malnar & Bailey, Inc., Equivalence Tests4 Two-SampletTest ExampleSolution: Using MINITABStat Basic Statistics 2-Sample tthetest statistic ist the confidence interval isP There are three ways to decide whetherto rejectH0or not. t so rejectH0 p so rejectH0 the confidence interval does not contain zero so rejectH076543210-1-2x 5 Design of Hypotheses To test the hypothesesH0: Something ordinary happensversusHA: Something extraordinary happens,"the extraordinary claim requires extraordinary evidence."-Carl Sagan The extraordinary claim is alwaysHA. HAcomes just its complement.

3 There is no special interest in or value toH0. Sagan says: The extraordinary claimHAis what we want to demonstrate. The data must be very strong to rejectH0and acceptHA. There is no opportunity to accept the ordinary claimH0. Put the burden of proof on the data. That is, if the data are weak orinsufficient then we can t Malnar & Bailey, Inc., Equivalence Tests6 Design of HypothesesExample: Formulate hypotheses to test the claim that there are (outerspace) aliens here among us. What evidence is required?Solution: The extraordinary claim is that there are aliens here amongus soH0: there are no aliens here among usHA: there are aliens here among usTo rejectH0in favor ofHA: Extraordinary evidence - we must meet a walking talking alien, beallowed to perform some physiology/medical Tests on him/her/it, seetheir space ship, visit their planet to confirm that there are others likehim/her/it, etc.

4 Weak evidence - the evidence that the aliens are here is that we can t seethem because they re too smart to get of HypothesesExercise: Formulate the hypotheses to test the claim "there are nomice in my house." What data would be required to support the claim?Solution:Mathews Malnar & Bailey, Inc., Equivalence Tests8 Run the ExperimentCollect sample data and calculate a test statistic that approximates thehypothesized parameter value. For our two-samplettest examplewe re using x to estimate .9 Hypothesis Test InterpretationThe hypothesis test can be interpreted: by comparing thettest statistic tot /2 by comparing thepvalue to by checking if the confidence interval for contains zero or rejectH0: 0when |t| t /2or p or when the confidence interval for does not contain Malnar & Bailey, Inc.

5 , Equivalence Tests10 Equivalence Test The goal of asignificance testis to show that there is a statisticallysignificant difference between the population means. The goal of anequivalence testis to show that the population means areequivalent to each other. The hypotheses for the two-sample Equivalence test are:H0: 1 2HA: 1 2or in terms of 1 2H0: 0HA: 0 Notice that these hypotheses are just the inverted hypotheses from thesignificance test. How do we perform the Equivalence test?11 Why Not Just AcceptH0?If we retain the original significance test hypothesesH0: 0versusHA: 0what s wrong with acceptingH0? Using the confidence interval interpretation we would acceptH0if theconfidence interval contains zero. The confidence interval has the formP x x 1 where the confidence interval half-width is t /2s 1n1 1n2 If we run a bad experiment so that is very large or if the sample sizesare too small then the will be very large and the confidence intervalwill very wide and almost certainly contain zero.

6 Absence of evidence is not evidence of absence. This approach - acceptingH0: 0 - does not put a sufficient burdenof proof on the data. The solution to the problem is anequivalence Malnar & Bailey, Inc., Equivalence Tests12 Equivalence Test Procedure When the purpose of a test is to demonstrate the equality of twotreatment means we perform anequivalence testusing hypotheses:H0: 0HA: 0 Note that what we want to demonstrate - the extraordinary claim - isHA. It is impossible to show thatHAisexactlytrue because there could be apractically small and insignificant bias between 1and 2so we usedifferent forms for these hypotheses - forms that formally consider thepossibility of such a bias:H0:| | HA:| | where is called thelimit of practical equivalance(LOPE).13 Equivalence Test Procedure LOPE is the largest value of for which you would consider the twopopulation means to be practically equivalent to each other.

7 LOPE must be sufficiently small so that the claim that 0 is, for allpractical purposes, still justified. For this reason the value of the LOPE must be chosen by or besatisfactory to the process owner. When we perform the Equivalence test, if the data supportHA:| | then we say that 1and 2arepractically equivalentto each Malnar & Bailey, Inc., Equivalence Tests14 Equivalence Test Procedure The Equivalence test is performed using two one-sided testsof means(TOST). The absolute values in the original Equivalence test hypothesesare broken up into two separate Tests :H01: versusHA1: H02: versusHA2: ttx12 0-t H : O2 >= 0t H :O1 <= 0 Accept H01 Accept H02 Reject H01 Reject H0215 Equivalence Test Procedure The hypotheses test approach is a bit complicated but the confidenceinterval approach requires a very simple calculation and has a verysimple interpretation.

8 To perform the Equivalence test by the confidence interval methodconstruct the 1 2 100% confidence interval for . If the confidenceinterval falls completely inside of the interval then rejectH0in favorofHA. To be conservative some experts advise using the 1 100%confidence interval instead. Note that the confidence interval can be interpreted in two ways in terms of the significance test by checking the location oftheconfidence interval with respect to 0 in terms of the Equivalence test by checking the location of theconfidence interval with respect to the Malnar & Bailey, Inc., Equivalence Tests16 TOST MethodExample: Suppose that the goal of our example problem was todemonstrate that 1and 2were practically equivalent and that beforethe data were collected the process owner chose LOPE to be the confidence interval in terms of the significance test andthe Equivalence : The plot below shows the confidence inteval for.

9 (Significance test) The confidence interval does not include 0 so we mustrejectH0: 0 in favor ofHA: 0, there is a statisticallysignificant difference between 1and 2. ( Equivalence test) The confidence interval does not fall completelywithin the interval 3 so we cannot rejectH0:| | , we donot have evidence for MethodExercise: Use the TOST confidence interval method for equivalencetests to interprete cases A to D in the plot below. Also interprete theintervals in terms of significance 0A)B)C)D)Mathews Malnar & Bailey, Inc., Equivalence Tests18 Sample Size Calculation The sample size required for the two-samplettest to rejectH0: 1 2with power 1 when the difference between the means is isgiven byn 2 t /2 t 2 The sample size required for the two-sample Equivalence test with limitof practical Equivalence and true bias between the means isn 2 t t 219 Sample Size CalculationExample: Determine the sample size required for the two-sampleequivalence test if the LOPE is 2and 90% powerand 1 2 : With 2the hypotheses to be tested areH01: 2versusHA1: 2H02: 2versusHA2: obtain 90% power we have 1 the sample sizeisn 2 t t 2 2 22 22 Further iterations of thetvalues given Malnar & Bailey, Inc.

10 , Equivalence Tests20 Solution using MINITAB(V17)Stat Power and Sample Size Equivalence Tests 2-Sample:21 Superiority Test The goal of a superiority test is to demonstrate that one treatment mean isgreater than the other. Superiority Tests are one-sided so asense ofdirection, either larger is better or smaller is better, must be specified. If a larger response is better, then to demonstrate that 1is superior to 2the hypotheses areH0: 1 2or 0HA: 1 2or 0 These hypotheses are identical to those of the one-sided significance test. If a larger response is better, then rejectH0if the confidence interval for is completely to the right of and does not contain Malnar & Bailey, Inc., Equivalence Tests22 Noninferiority Test The goal of a noninferiority test is to demonstrate that one population smean is not inferior to another population s mean or, if it isinferior, it isnot inferior by very much.