Verification and Validation in Computational Fluid Dynamics1

1 SAND2002 - 0529. Unlimited Release Printed March 2002. Verification and Validation in Computational Fluid Dynamics1 . William L. Oberkampf Validation and Uncertainty Estimation Department Timothy G. Trucano Optimization and Uncertainty Estimation Department Sandia National Laboratories P. O. Box 5800. Albuquerque, New Mexico 87185. Abstract Verification and Validation (V&V) are the primary means to assess accuracy and reliability in Computational simulations. This paper presents an extensive review of the literature in V&V in Computational Fluid dynamics (CFD), discusses methods and procedures for assessing V&V, and develops a number of extensions to existing ideas.

2 The review of the development of V&V. terminology and methodology points out the contributions from members of the operations research, statistics, and CFD communities. Fundamental issues in V&V are addressed, such as code Verification versus solution Verification , model Validation versus solution Validation , the distinction between error and uncertainty, conceptual sources of error and uncertainty, and the relationship between Validation and prediction. The fundamental strategy of Verification is the identification and quantification of errors in the Computational model and its solution. In Verification activities, the accuracy of a Computational solution is primarily measured relative to two types of highly accurate solutions: analytical solutions and highly accurate numerical solutions.

3 Methods for determining the accuracy of numerical solutions are presented and the importance of software testing during Verification activities is emphasized. The fundamental strategy of 1 Accepted for publication in the review journal Progress in aerospace Sciences. 3. Validation is to assess how accurately the Computational results compare with the experimental data, with quantified error and uncertainty estimates for both. This strategy employs a hierarchical methodology that segregates and simplifies the physical and coupling phenomena involved in the complex engineering system of interest. A hypersonic cruise missile is used as an example of how this hierarchical structure is formulated.

4 The discussion of Validation assessment also encompasses a number of other important topics. A set of guidelines is proposed for designing and conducting Validation experiments, supported by an explanation of how Validation experiments are different from traditional experiments and testing. A description is given of a relatively new procedure for estimating experimental uncertainty that has proven more effective at estimating random and correlated bias errors in wind-tunnel experiments than traditional methods. Consistent with the authors' contention that nondeterministic simulations are needed in many Validation comparisons, a three-step statistical approach is offered for incorporating experimental uncertainties into the Computational analysis.

5 The discussion of Validation assessment ends with the topic of Validation metrics, where two sample problems are used to demonstrate how such metrics should be constructed. In the spirit of advancing the state of the art in V&V, the paper concludes with recommendations of topics for future research and with suggestions for needed changes in the implementation of V&V in production and commercial software. 4. Acknowledgements The authors sincerely thank Frederick Blottner, Gary Froehlich, and Martin Pilch of Sandia National Laboratories, Patrick Roache consultant, and Michael Hemsch of NASA/Langley Research Center for reviewing the manuscript and providing many helpful suggestions for improvement of the manuscript.

6 We also thank Rhonda Reinert of Technically Write, Inc. for providing extensive editorial assistance during the writing of the manuscript. 5. Contents 1. 8. 8. Outline of the 10. 2. Terminology and Development of terminology for Verification and 11. Contributions from Fluid Methodology for Methodology for 19. 3. Verification 24. Fundamentals of Definitions and general 24. Developing the case for code Error and the Verification of Role of Computational error estimation in Verification Convergence of A priori error 34. A posteriori error 36. Need for Verification Algorithm and software quality Algorithm 48. Software quality 4. Validation Fundamentals of Validation and 56.

7 Validation error and Construction of a Validation experiment 61. Hierarchy 61. Hierarchy Guidelines for Validation 67. Statistical estimation of experimental 74. Uncertainty quantification in Hypothesis Validation 82. Recommended 82. Validation metric 83. Zero experimental measurement Random error in experimental 87. 5. Recommendations for Future Work and Critical Implementation 6. Figures 1 Phases of Modeling and Simulation and the Role of V& 2 Verification 3 Validation 20. 4 Validation 5 Demonstration of Extrapolated Error Estimation for Mixed First and Second Order 41. 6 Integrated View of Verification Assessment for 45. 7 Relationship of Validation to 8 Validation Hierarchy for a Hypersonic Cruise 64.

8 9 Validation Pyramid for a Hypersonic Cruise 66. 10 Domain of Boundary Value 84. 11 Proposed Validation Metric as a Function of Relative 86. 12 Validation Metric as a Function of Relative Error and Data 89. Table 1 Major Software Verification 7. 1. introduction Background During the last three or four decades, computer simulations of physical processes have been used in scientific research and in the analysis and design of engineered systems. The systems of interest have been existing or proposed systems that operate at design conditions, off-design conditions, failure-mode conditions, or accident scenarios. The systems of interest have also been natural systems.

9 For example, computer simulations are used for environmental predictions, as in the analysis of surface-water quality and the risk assessment of underground nuclear-waste repositories. These kinds of predictions are beneficial in the development of public policy, in the preparation of safety procedures, and in the determination of legal liability. Thus, because of the impact that modeling and simulation predictions can have, the credibility of the Computational results is of great concern to engineering designers and managers, public officials, and those who are affected by the decisions that are based on these predictions. For engineered systems, terminology such as virtual prototyping and virtual testing is now being used in engineering development to describe numerical simulation for the design, evaluation, and testing of new hardware and even entire systems.

10 This new trend of modeling- and-simulation-based design is primarily driven by increased competition in many markets, , aircraft, automobiles, propulsion systems, and consumer products, where the need to decrease the time and cost of bringing products to market is intense. This new trend is also driven by the high cost and time that are required for testing laboratory or field components, as well as complete systems. Furthermore, the safety aspects of the product or system represent an important, sometimes dominant element of testing or validating numerical simulations. The potential legal and liability costs of hardware failures can be staggering to a company, the environment, or the public.