Homogeneity of EQA samples requirements …

1 Frank Baumeister TGZ AQS-BW at Institute for Sanitary Engineering, Water Quality and Solid Waste Management University of Stuttgart AQS Baden-W rttemberg Bandt le 2 70569 Stuttgart GERMANY Tel.: +49 711 685 65442 / Fax: +49 711 685 55442 E-Mail: 1 Homogeneity of EQA samples requirements according to ISO/IEC 17043 Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 2 ISO/IEC 17043 Homogeneity (I) Criteria for suitable sample Homogeneity must be available Inhomogeneities must be considered when participants performance is evaluated Assessment of Homogeneity must be conducted with appropriate statistical designs Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest ISO/IEC 17043 Homogeneity (II) PT provider shall use a statistically random selection of a representative number of PT items from the whole batch of test material Homogeneity assessment should be performed after the PT items have been packaged in the final form before distribution to participants Baumeister.

2 Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 4 ISO/IEC 17043 Annex Demonstration of proficiency test item Homogeneity Demonstration of sufficient Homogeneity with valid statistical methods is required Application of procedures described in ISO 13528 and IUPAC International Hamonized Protocol Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 5 ISO 13528 Homogeneity - method and packaging of the samples selection of at least 10 samples per batch (g 10) of two test portions (subsample) from each sample are measured in random order under repeatability conditions Measurements are done from one laboratory Not all measurands, but those which are sensitive to heterogeneity Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 6 ISO 13528 Homogeneity - criterion Comparison of the standard deviation ss (between the samples ) with the standard deviation for proficiency assessment (SDPA) The samples may be considered to be adequately homogeneous if: 3,0 ssBaumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 7 Scheme of Homogeneity testing xt: x1 x2 x3 xt,k: x1,1 x1,2 x2,1 x2,2 x3,1 x3,2 w1 w2 w3 1x2x3 Xwt xStandard deviation of sample averages: Within-sample standard deviation: Between-sample standard deviation: )1/()(2.

3 ,.,. gxxstx )2/(2gwstw)2/(22wxssss Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 8 ISO 13528 Homogeneity criterion not fullfilled Sample preparation procedure must be examined if improvements are possible Heterogeneity is to be included in the standard deviation of SDPA Between-sample SD increases SDPA of z-score calculation (usage of z -score) 221 ss Note: not if SDPA is derived from the data of the participants Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 9 IUPAC Homogeneity additional remarks Method must allow a sufficient precise and satisfactory estimation of ss Check the data for noticeable problems Visual: plot the results against the sample number Trends or discontinuity Excessive rounding Outlying results within the samples Cochran-test: Determination of extreme differences within the samples Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 10 IUPAC Homogeneity Cochran-Test (I) 1791801811821831841851861871881891900123 45678910111213141516 SampleResultResult 1 Result 2D4 the sum of square SDD of the m differences: 2.

4 Calculate the ratio of the largest squared difference to this sum of squared differences: DDSDCmax2 Dmax miiDDDS12 Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 11 IUPAC Homogeneity Cochran-Test (II) 3. Compare the ratio with the appropriate critical value from tables m95%99%70,7270,83880,680,79490,6380,7541 00,6020,718110,570,684120,5410,653130,51 50,624140,4920,599150,4710,575160,4520,5 53170,4340,532180,4180,514190,4030,49620 0,3890,48 Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 12 IUPAC Homogeneity Cochran-Test (III) 1791801811821831841851861871881891900123 45678910111213141516 SampleResultResult 1 Result 2 Delete before ANOVA at 99% confidence level Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 13 IUPAC Homogeneity Cochran-Test (IV) 1781801821841861881901920123456789101112 13141516 SampleResultResult 1 Result 2 Delete complete data set Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 14 IUPAC Homogeneity Cochran-Test (V) 1791801811821831841851861871881891900123 45678910111213141516 SampleResultResult 1 Result 2No elimination in the case of outlying mean value Baumeister.

5 Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 15 IUPAC Homogeneity additional remarks If everything is ok: Estimation of the variances resulting from the analysis and the samples ANOVA Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 16 IUPAC test for significant Homogeneity ANOVA (I) Sum of the squared differences: equivalent to within-sample standard deviation mDsmiian2/122 Variance Vs of the sums Si: equivalent to the standard deviation of the sample average )1/()(21 mSSvmIiS Sampling variance: equivalent to between-sample standard deviation 2/)2/(2anssamsVs Allowable sampling variance: 22)3,0(pall Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 17 IUPAC test for significant Homogeneity ANOVA (II) Critical value c for the significance test (F-test): 2221anallsFFc m*2019181716151413121110987F11,591,61,62 1,641,671,691,721,751,791,831,881,942,01 2,1F20,570,590,620,640,680,710,750,80,86 0,931,011,111,251,436m* is the number of samples that have been measured in duplicateHomogeneity test has been passed, if: cssam 2 Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest 18 Free Sample tvalue#1, xt,1value#2, xt,2sample average ( ), xt.

6 Between-test-portion ranges ( ), wt110,510,410,450,129,69,59,550,1310,49, 910,150,549,59,99,70,45109,79,850,369,61 0,19,850,579,810,410,10,689,810,2100,491 0,810,710,750,11010,21010,10,2119,89,59, 650,31210,21010,10,213141516171819201210 ,020833330,3400924560,2474873730,2916125 49 Expected standard deviation for proficiency assessment1,14 Homogeneityok Dr. Michael Koch, AQS Baden-W rttemberg, Universit t Stuttgartwithin- samples STD ( ), swbetween- samples STD ( ), sSHomogeneity check (ISO 13528 Annex B)Example from ISO 13528general average ( )STD of sample averages ( ), sxnumber of samples g .,.xHomogeneity check9,49,69,81010,210,410,610,811051015 2025 SampleAnalytical result1st result2nd resultSample tvalue#1, xt,1value#2, xt,2D=a-bS=a+bD =(a-b) 110,510,40,120,90,010,73673629,69,50,119 ,10,010,886736310,49,90,520,30,250,06673 649,59,9-0,419,40,160,4117365109,70,319, 70,090,11673669,610,1-0,519,70,250,11673 679,810,4-0,620,20,360,02506989,810,2-0, 4200,160,001736910,810,70,121,50,012,126 7361010,2100,220,20,040,025069119,89,50, 319,30,090,5500691210,2100,220,20,040,02 50691314151617181920number of samples12 Cochran test procedure for duplicate resultsDmax20,36 SDD1,47C0,24489796critical value (99%)0,653critical value (95%)0,541no outlying pairTest for significant inhomogeneitysan20,0612520,0416667VS0,46 265152ssam20,08503788 Expected standard deviation for proficiency assessment1,14 Allowable sampling variance0,116964critical F11,79critical value0,26204056critical F20,86 HomogeneityOK Dr.

7 Michael Koch, AQS Baden-W rttemberg, Universit t Stuttgartno outlying pairHomogeneity check (Intern. Harmonized Protocol Appendix 1)Example from ISO 13528 Homogeneity check02468101205101520 SampleAnalytical result1st result2nd resultS2)(SSi Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest Example (quantitative): Serum (C-reactive protein) Quality Control Centre Switzerland, testingInternal codeMaterialSerumDeviceFuji dri-chem 3500i656 ParameterCRP (C-reactive protein)700 Parameter typeQuantitativeMethodFuji719 Prepared samples2500 CRTolerance11% ,000400,0002360,000410,0003380,000410,00 04430,000390,0005400,000420,0006420,0003 90,0007430,000380,000 Duplicates? STD of sample averages sX < within-sample STD sw Standard deviation of analytical method is around 7% Is this sufficiently precise ? Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest Example (quantitative): Serum (Creatine kinase) Quality Control Centre Switzerland, testingInternal codeMaterialSerumDeviceFuji dri-chem 3500i656 ParameterCK-MB (Creatine kinase)32 Parameter typeQuantitativeMethodFujiPrepared samples50 Tolerance25% ,0042,00245,0041,00343,0039,00 Duplicates?

8 Number of samples is too low Recommendation: Decide from the visual plot if it is fit for the purpose or take more samples Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest Example (semi-quantitative): Urine (Protein) Quality Control Centre Switzerland, testingInternal codeMaterialUrineDeviceUrisys 1800577 ParameterProtein135 Parameter typeSemi-quantitativeMethodStrips409 Prepared +++(2-5g/L)2+++(2-5g/L)3+++(2-5g/L) Number of samples is definitly too low Evaluation of Homogeneity only by comparison of the results possible No statistical tool available Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest Example (qualitative): Urine (Opiates) No statistical method available Comparison of the expected property value Quantification by chemical analyses possible ? Conventional test for Homogeneity possible ? Quality Control Centre Switzerland, testingMaterialUrineDeviceTox/seeParamet erOpiatesParameter typeCategoricalPrepared : Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest Example: copper in soya flour Sample Value 1 Value 2 1 12,1 10,4 2 9,6 9,5 3 10,4 9,9 4 9,5 9,9 5 10 9,7 6 9,6 10,1 7 9,8 10,4 8 9,8 10,2 9 10,8 10,7 10 10,2 10 11 9,8 9,5 12 10,2 10 = 1,14 outlier increases the critical value elimination is required, otherwise the result is too much influenced by the analytical precision After elimination of the outlier, the samples are considered to be homogeneous ISO 13528 is more sensitive in this case Baumeister: Homogeneity of EQA samples , EQALM Symposium 2013, Bucharest Thank you for your attention !

9