Repeatability and Reproducibility - Engineered Software

1 Repeatability and ReproducibilityCopyright 1999 by Engineered Software , is the variability of the measurements obtained by one person whilemeasuring the same item repeatedly. This is also known as the inherent precision of themeasurement equipment. Consider the probability density functions shown in Figure density functions were constructed from measurements of the thickness of a piece ofmetal with Gage A and Gage B. The density functions demonstrate that Gage B is morerepeatable than Gage AGage BFigure 1. Probability density functions for the thickness of 2 is the variability of the measurement system caused bydifferences in operator behavior. Mathematically, it is the variability of the averagevalues obtained by several operators while measuring the same item. Figure 2 displaysthe probability density functions of the measurements for three operators.

2 The variabilityof the individual operators are the same, but because each operator has a different bias,the total variability of the measurement system is higher when three operators are usedthan when one operator is used. Figure 3 also displays the probability density functionsof the measurements for three operators using the same scale as Figure 2. Notice thatthere is more difference in the means of the measurements shown in Figure 3 than thoseshown in Figure 2. The Reproducibility of the system shown in Figure 3 is higher thanthe Reproducibility of the system shown in Figure 2. Reproducibility 3. Reproducibility most commonly used method for computing Repeatability and reproducibilityis the Range and Average method. The ANOVA method is more accurate, but becauseof the complex mathematics involved it has been shunned historically.

3 With desktopcomputers there is no excuse for not using the more accurate ANOVA & Average MethodThe Range & Average Method computes the total measurement systemvariability, and allows the total measurement system variability to be separated intorepeatability, Reproducibility , and part ANOVA method, discussed in the next section, is preferred to the averagerange method. The ANOVA method quantifies the interaction between Repeatability andreproducibility, and is considered to be more accurate than the average and quantify Repeatability and Reproducibility using average and range method,multiple parts, appraisers, and trials are required. The recommended method is to use 10parts, 3 appraisers and 2 trials, for a total of 60 measurements. The measurement systemrepeatability isRepeatability= R is the average of the ranges for all appraisers and parts, andd2 is found in Appendix A with Z = the number of parts times the number ofappraisers, and W = the number of measurement system Reproducibility isReproducibility = 22nr2where Xrange is the average of the difference in the average measurements between theappraiser with the highest average measurements, and the appraiser with thelowest average measurements, for all appraisers and parts,d2 is found in Appendix A with Z = 1 and W = the number of appraisers,n is the number of parts.

4 Andr is the number of measurement system Repeatability and Reproducibility isRR&=+RepeatabilityReproducibility223 The part variability isVRdPP= Rp is the difference between the largest average part measurement and the smallestaverage part measurement, where the average is taken for all appraisers and alltrials, andd2 is found in Appendix A with Z = 1 and W = the number of total variability, measurement system variability and part variation combined isVRRVTP=+&225 Example 1 The thickness, in millimeters, of 10 parts have been measured by 3 operators, using thesame measurement equipment. Each operator measured each part twice, and the data isgiven in Table 1. Range & Average method example 1 Trial 2 Trial 1 Trial 2 Trial 1 Trial is computed using the average of the ranges for all appraiser and allparts.

5 This data is given in Table 2. Example problem range Trial 1 Trial 2 RTrial 1 Trial 2 R Trial 1 Trial 2 average of the 30 ranges, R, is From Appendix A, with Z = 30 (10 partsmultiplied by 3 appraisers) and W = 2 (2 trials), d2 is The Repeatability is() Repeatability ==515 520112823 average reading for appraiser A is , the average reading for appraiser B is ,and the average reading for appraiser C is To compute Reproducibility , the averageof the range between the appraiser with the smallest average reading (appraiser B in thisexample) and the appraiser with the largest average reading (appraiser C in this example)is needed. Table 3 shows this 3.

6 Reproducibility example BOperator average of the ranges, Xrange, is From Appendix A, using Z = 1 and W = 3for 3 appraisers, is The Reproducibility is()() Reproducibility = =515 7 01519123 710 218 Repeatability and Reproducibility isRR&..=+=23 718 229 922 The part variability is computed using the difference between the largest and smallestpart measurement, where the average is taken for all parts and appraisers. This data isshown in Table 4. Example part variability 1 Trial 2 Trial 1 Trial 2 Trial 1 Trial part with the largest average belongs to part 10, The lowest average belongsto part 5, This difference , , is Vp. From Appendix A, using Z = 1 and W =10 for 10 parts, d2 = The part variability is()VP==515 44 2531871 total measurement system variability isVT=+=29 971 777.

7 Analysis of Variance MethodThe analysis of variance method (ANOVA) is the most accurate method forquantifying Repeatability and Reproducibility . In addition, the ANOVA method allowsthe variability of the interaction between the appraisers and the parts to be ANOVA method for measurement assurance is the same statistical techniqueused to analyze the effects of different factors in designed experiments. The ANOVA design used is a two-way, fixed effects model with replications. The ANOVA table isshown in Table 5. Two-Way ANOVA ofVariationSum ofSquaresDegreesofFreedomMeanSquareF StatisticAppraiserSSAa-1 MSASSAa= 1 FMSAMSE=PartsSSBb-1 MSBSSBb= 1 FMSBMSE=Interaction(Appraiser,Parts)SSAB (a-1)(b-1)MSABSSABab= ()()11 FMSABMSE=Gage(Error)SSEab(n-1)MSESSEab n= ()1 TotalTSSN-1 SSAYbnYNiia= = ()..2216 SSBYanYNjjb= = ().

8 2217 SSABYnYNSSA SSBijjbia= == ().22118 TSSYYN ijkknjbia= === 221119 SSETSSSSA SSB SSAB= 10a = number of appraisers,b = number parts,n = the number of trials, andN = total number of readings (abn)When conducting a study, the recommended procedure is to use 10 parts, 3appraisers and 2 trials, for a total of 60 measurements. The measurement systemrepeatability isRepeatability= measurement system Reproducibility isReproducibility= interaction between the appraisers and the parts isI= measurement system Repeatability and Repeatability isRRI&=+ +RepeatabilityReproducibility22214 The measurement system part variation isVMSBMSABanP= total measurement system variation isVRRVTP=+&2216 Example 2 The thickness, in millimeters, of 10 parts have been measured by 3 operators, using thesame measurement equipment.

9 Each operator measured each part twice, and the data isgiven in Table 6. ANOVA method example Trial 1 Trial 2 Trial 1 Trial 2 Trial 1 Trial compute the characteristics of this measurement system, the two-wayANOVA table must be completed. The sum of the 20 readings (10 parts multiplied by 2trials) for appraiser A is The sum of the 20 readings for appraiser B is sum of the 20 readings for appraiser C is , and the sum of all 60 readings The sum-of-squares for the appraisers is()()()SSA=++ =1710 210 21657 710 21798 010 25165 960502 sum of the 6 readings for each part (3 appraisers multiplied by 2 trials) is given inTable 7 along with the square of this sum, and the square of this sum divided by 7. Part sum-of-squares SquaredSum , , , , , , , , , , , , , , , , , , , , , sum-of-squares for the parts isSSB= =456 320 95165 96011 545 52.

10 ,.The sum of the 2 trials for each combination of appraiser and part is given in Table 8along with the square of this sum, and the square of this sum divided by 8. Interaction sum-of-square SquaredSum , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , sum-of-squares for the interaction between the appraisers and the parts, SSAB, isSSAB= =456 859 05165 960502 5 11 545 535 62,..,..Squaring all 60 individual reading and summing the values gives 457, The totalsum-of-squares isTSS= =457 405 85165 96012 630 42,..,.The sum-of-squares for the gage or error isSSE= =12 630 4 502 5 11 545 5 35 6546 8,.. ,..There are 2 degrees-of-freedom for the appraisers, the number of appraisersminus one; 9 degrees-of-freedom for the parts, the number of parts minus one, 18degrees-of-freedom for the interaction between the appraisers and the parts, the numberof appraisers minus one multiplied by the number of parts minus one; 59 total degrees-of-freedom; the total number of readings minus one, and 30 degrees-of-freedom for thegage, total degrees-of-freedom minus the degrees-of-freedom for the appraisers minus thedegrees-of-freedom for the parts minus the degrees-of-freedom for the interaction.