TUTORIAL No 2 How to determine repeatability and ...

1 3rdEuropean-American Workshop on NDE ReliabilityTUTORIAL No 2 How to determine repeatability and reproducibility (R&R)Dr Damir of Zagreb, Department of Quality, Faculty of Mechanical Engineering & Naval Architecture, CroatiaTutorial , terminology / experiment(how to perform,layout, DoE) analysis (statistical analysis, examples) remarks& discussionLiterature21. IntroductionConcept of R&R as a measure of precision in measurement laboratories, was introduced & developed during 1978-1983 [2]. It was widely accepted but also criticised, and finally resulted with ISO 5725 in any other field,first essential prerequisite is to exchange information, data and knowledge and to understand each other properly so, convention about definitions is "Each term must have the same meaning for all of its users; it must therefore at the same time express a well defined concept and not to be in conflict with everyday language" [1].

2 First, we ll discuss two terms; accuracyand precision; so we ll come to R& IntroductionIt is known from common everyday use that when we repeatsome measurement or want to reproducemeasurement result in some other occasion,we will almost never get exactly the same result. We could observe same phenomena if we want to measure some quantity with different systems!..51. Introductionjust now !.. now, we can carry out simple measurement:6 What time is ..1. IntroductionChristine s watch10:0211:0212:0213:0214:0215:02 Which one you more prefer ?

3 True time10:0011:0012:0013:0014:0015:00My watch10:0011:0112:0213:0314:0215 your watch an accurateand/or preciseone ?72. Definitions / Accuracy, trueness, precision R&R conditions and limits Elements of test system Systematic and random errors Accepted reference value8 Accuracydesignates closeness of agreement between a test result(measurement result) and the accepted reference value (true value) [1,3].NotesThe term precision should not be used for accuracy [1].9 Truenessdesignates closeness of agreement between the average valueobtained from a large series of test result and an accepted reference value [3].

4 NotesThe measure of trueness is usually expressed in terms of bias[3].Trueness has been referred to as accuracy of the mean , what is not recommended [3].10 Precisiondesignates variability of results , or closeness of the agreement between test resultsof repetitive measurements [3].In total, the results of particular set of measurements will be (randomly) distributed around expected measure of dispersionof results gives us information about (trueness and/or precision) .. variability of resultsA)B)C)12 Both, trueness and precision are quantitative can estimatedispersion (precision) by calculation of standard , we can estimateso-called "true value" by calculation of a variability of results13variabilityexample A)14variabilityexample B)15variabilityexample C)16 More important is that we can compare and analyse differences between particular sets of test results .

5 This directs us to the subject of repeatability and reproducibility.. variability of results17 Differentvariabilitiesbetween differentsets of test resultsregarding todifferent sourcesof variabilities. CADFEB18 Regarding factors (elements)of the systemwhich contribute to the variability of the test results ,two conditions of precision, termed repeatability and reproducibilityconditions, have been found necessary and useful for describing the variability of a measurement method [3].. definitions of R&R.

6 19 Conditions where independent test results are obtained with the same methodon identical test itemsin the same laboratoryby the same operatorusing the same equipmentwithin short intervals of time [3].Repeatabilityis precision under repeatability conditions 20 Conditions where test results are obtained with the same methodon identical test itemsin different laboratorieswith different operatorsusing different equipment [3].Reproducibilityis precision under reproducibility conditions 21 Thus repeatability and reproducibility are the two extremes of minimumandreproducibilitythe maximum variabilityin extremes of variability22 The value less than or equal to which the absolute difference betweentwo test resultsobtained under repeatability conditionsmay be expected to be with a probability of 95 % [3].

7 repeatability limit,r232rsfr =The value less than or equal to which the absolute difference betweentwo test resultsobtained under reproducibility conditionsmay be expected to be with a probability of 95 % [3].Reproducibility limit, R242 RsfR =RepeatabilityIllustration ofthe meaninig ofrepeatabilitystandard deviationand ofthe meaninig ofreproducibilitystandard deviationand elements of test system [4] Operator Equipment Calibration Time27 Elements of test system regarding NDT/mine_det Personnel(operator/technician human factor) Equipment(instruments, tools, utensils, probes/sensors,manipulators, spare parts & materials, reference blocks.)

8 Procedure(measurement/testmethod, calibration) Test object / measured entity(any characteristic: microstructure, geometry, surface conditions, orientation, ) Environment (temperature, humidity, viewing conditions ..)28 Trueness and systematic errorspecified toleranceexpected value29 bias (B) for infinite groups ofmeasurementsBis randomlydistributedarroundexpected valueIs a meanthat would result from an infinite number of measurements of the same measurand carried out under repeatability conditionsminusa true valueof the measurand [1].

9 Systematic error30specified toleranceexpected valuePrecision and random error31Is a resultof a measurement minusthe meanthat would result from an infinite number of measurements of the same measurand carried out under repeatability conditions[1].Because only a finite number of measurements can be made, it is possible to determine only an estimate of random error [1].Random error32 Accepted reference valueA value that serves as an agreed-upon referencefor comparison [3].Could be derived as theoretical or established value, an assigned or certified value, a consensus.

10 When not available, the expectationof the quantity, the mean of a specified population of measurements [3].333. R&R experiment Experiment layout How to organise and carry out the experiment DoE34 Experiment layout The objective of the experiment is to determine the precisionobtainable by the general population of operators performing standard measurement (test) method. Each combination of a laboratory and a level is called cell. Each cell contains nrepetitive test results . 35 Experiment layout36p 1level 2level 3125,337,543,9A225,237,744,4325,337,744, 0125,437,444,4B225,437,544,5325,337,744, 9125,137,744,3C225,337,344,9325,037,644, 0125,137,444,6D225,537,244,6325,537,544, 0125,337,544,3E225,237,444,5325,337,744, 7 Experiment layoutqlevelsplaboratoriesnreplicate test results CADFEB37 Performance of the experiment(1) Perform the measurements according to the specified standard method !