RESEARCH REPORT 454 - Health and Safety Executive

1 HSE. Health & Safety Executive Probability of Detection (PoD) curves Derivation, applications and limitations Prepared by Jacobi Consulting Limited for the Health and Safety Executive 2006. RESEARCH REPORT 454. HSE. Health & Safety Executive Probability of Detection (PoD) curves Derivation, applications and limitations George A Georgiou Jacobi Consulting Limited 57 Ockendon Road London N1 3NL. There is a large amount of Probability of Detection' (PoD) data available (eg National NDT Centre (UK), NORTEST (Norway), NIL (Netherlands) and in particular NTIAC (USA)). However, it is believed that PoD. curves produced from PoD data are not very well understood by many who use and apply them.

2 For example, in producing PoD curves, a certain material and thickness may have been used and yet one can find the same PoD quoted for a range of thicknesses. In other cases, PoD curves may have been developed for pipes, but they have been applied to plates or other geometries. Similarly, PoD curves for one type of weld (eg single sided) have been used for other welds (eg double sided). PoD data are also highly dependent on the Non-Destructive Testing (NDT) methods used to produce them and these data can be significantly different, even when applied to the same flaws and flaw specimens. It is often assumed that the smallest flaw detected is a good measure of PoD, but there is usually a large gap between the smallest flaw detected and the largest flaw missed.

3 Similarly, it is often assumed that human reliability is a very important factor in NDT procedures, and yet it is usually found not to be as important as other operational and physical parameters. It is important to question the validity of how PoD curves are applied as well as their limitations. This REPORT aims to answer such questions and in particular their relevance to fitness for service issues involving PoD. The overall goal of this project is to provide clear, concise, understandable and practical information on PoD curves, which will be particularly useful for Health and Safety Inspectors when discussing Safety cases involving PoD curves. This REPORT and the work it describes were funded by the Health and Safety Executive (HSE).

4 Its contents, including any opinions and/or conclusions expressed, are those of the author alone and do not necessarily reflect HSE policy. HSE BOOKS. Crown copyright 2006. First published 2006. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the prior written permission of the copyright owner. Applications for reproduction should be made in writing to: Licensing Division, Her Majesty's Stationery Office, St Clements House, 2-16 Colegate, Norwich NR3 1BQ or by e-mail to ii TABLE OF CONTENTS. TABLE CAPTIONS AND FIGURE CAPTIONS v Executive SUMMARY vi Background vi Objectives vi Work Carried Out vi Conclusions vii Recommendations vii 1.

5 INTRODUCTION 1. 2. OBJECTIVES 2. 3. DERIVATION OF POD CURVES 2. A HISTORICAL BACKGROUND AND DEVELOPMENT OF NDT RELIABILITY METHODS 2. EXPERIMENTAL REQUIREMENTS TO PRODUCE POD CURVES 3. THE AVAILABLE PROBABILITY METHODS TO PRODUCE POD CURVES 4. PoD Curves for Hit/Miss Data 4. PoD Curves for Signal Response Data 5. Sample Sizes 6. CONFIDENCE LIMITS (OR CONFIDENCE INTERVALS) 7. PUBLISHED WORK ON THE MODELLING OF POD 8. An Overview 8. The PoD-generator (The Netherlands) 9. Iowa State University (USA) 9. National NDT Centre (UK) 9. 4. THE PRACTICAL APPLICATION OF POD CURVES 10. HOW POD CURVES ARE USED IN INDUSTRY 10. PUBLISHED WORK ON POD CURVES IN DIFFERENT INDUSTRIES 10.

6 Aerospace (NASA) 10. Aircraft Structures, Inclusions in Titanium Castings 10. NORDTEST Trials 11. Nuclear Components (The PISC Trials) 11. Offshore tubular Joints 11. Dutch Welding Institute (NIL) 11. Railways (National NDT Centre (UK)) 12. LPG Storage Vessels 13. 5. THE LIMITATIONS OF APPLYING POD CURVES 13. COMMENTS ON HIT/MISS DATA AND SIGNAL RESPONSE DATA 13. IMPORTANT OPERATING AND PHYSICAL PARAMETERS 14. NDT Method 15. Fluorescent Penetrant NDT 15. Material Properties 15. Specimen Weld Geometry 15. Flaw Characteristics 16. iii Human Reliability 16. 6. DISCUSSION 16. INTRODUCTION 16. AIMS AND OBJECTIVES 17. HISTORICAL DEVELOPMENT 17. FLAW SAMPLE SIZES FOR HIT/MISS' DATA AND SIGNAL RESPONSE' DATA 18.

7 Model for Hit/Miss Data 18. Model for Signal Response Data 19. To Compute PoD parameters 20. To Achieve the Desired PoD/Confidence Limit Combination 20. POD MODELLING 20. PRACTICAL APPLICATIONS OF POD 21. Aircraft Structures, Inclusions in Titanium Castings 21. NORDTEST Trials 21. Nuclear Components (The PISC Trials) 21. Offshore Tubular Joints 21. Dutch Welding Institute (NIL) 22. Railways 22. LPG Storage Vessels 22. DEPENDENCE OF POD ON OPERATIONAL AND PHYSICAL PARAMETERS 22. Important Operational and Physical Parameters 22. NDT Method 23. Fluorescent Penetrant NDT 23. Material Properties 23. Specimen Weld Geometry 23. Flaw Characteristics 23. Human Reliability 24.

8 7. INDEPENDENT VERIFICATION 24. 8. CONCLUSIONS 24. 9. RECOMMENDATIONS 25. 10. ACKNOWLEDGEMENTS 25. 11. REFERENCES 25. 12. VERIFICATION STATEMENT. TABLES. 1. FIGURES. 1 - 13. APPENDIX A. GLOSSARY OF TERMS, STATISTICAL TERMINOLOGY AND OTHER. RELEVANT INFORMATION. APPENDIX B. AN AUDIT TOOL FOR THE PRODUCTION AND APPLICATION OF. POD CURVES. APPENDIX C. THE VALIDITY OF THE JCL INDEX OF DETECTION' MODEL. iv TABLE CAPTIONS AND FIGURE CAPTIONS. TABLE CAPTIONS. Table 1 Maximum Probability Tables FIGURE CAPTIONS. Figure 1 Example of detection percentages for a handheld Eddy-Current inspection and a log- odds' distribution fit to the data. Figure 2 Ultrasonic NDT hit/miss data illustrating the relatively large gap between the smallest flaw detected and the largest flaw missed.

9 Figure 3 The linear relationship between the log-odds and log flaw size. Figure 4 Schematic of the PoD for flaws of fixed dimension for hit/miss' data. Figure 5 Schematic of the PoD for flaws of fixed dimension for signal response' data. Figure 6 A comparison between the log-odds and cumulative log-normal distribution functions for the same parameters =0 and = Figure 7 An example of when the log-odds model was not applicable to the data collected Figure 8 PoD (a) log-odds model results for different NDT methods applied to the same flaw specimen. Figure 9 PoD (a) log-odds model results for fluorescent penetrant: no developer and developer applied to the same flaw specimen Figure 10 PoD (a) log-odds model results for manual eddy currents: different materials but nominally the same flaws Figure 11 PoD (a) log-odds model results for X-ray radiography: different weld conditions but nominally the same flaws Figure 12 PoD (a) log-odds model results for fluorescent penetrant: different flaws but nominally the same specimens Figure 13 PoD (a) log-odds model results for Ultrasound (Immersion): different operators but inspecting the same flaw specimen v Executive SUMMARY.

10 Background There is a large amount of Probability of Detection' (PoD) data available ( National NDT. Centre (UK), NORTEST (Norway), NIL (Netherlands) and in particular NTIAC (USA)). However, it is believed that PoD curves produced from PoD data are not very well understood by many who use and apply them. For example, in producing PoD curves, a certain material and thickness may have been used and yet one can find the same PoD quoted for a range of thicknesses. In other cases, PoD curves may have been developed for pipes, but they have been applied to plates or other geometries. Similarly, PoD curves for one type of weld ( single sided) have been used for other welds ( double sided).