Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-77c89778f8-m8s7h Total loading time: 0 Render date: 2024-07-18T01:13:09.018Z Has data issue: false hasContentIssue false

2 - Fundamental concepts and terminology

from Part I - Fundamental concepts

Published online by Cambridge University Press:  05 March 2013

William L. Oberkampf  and
Christopher J. Roy
Christopher J. Roy
Affiliation:
Virginia Polytechnic Institute and State University
Get access

Summary

This chapter discusses the fundamental concepts and terminology associated with verification and validation (V&V) of models and simulations. We begin with a brief history of the philosophical foundations so that the reader can better understand why there are a wide variety of views toward V&V principles and procedures. Various perspectives of V&V have also generated different formal definitions of the terms verification and validation in important communities. Although the terminology is moving toward convergence within some communities, there are still significant differences. The reader needs to be aware of these differences in terminology to help minimize confusion and unnecessary disagreements, as well as to anticipate possible difficulties in contractual obligations in business and government. We also discuss a number of important and closely related terms in modeling and simulation (M&S). Examples are predictive capability, calibration, certification, uncertainty, and error. We end the chapter with a discussion of a conceptual framework for integrating verification, validation, and predictive capability. Although there are different frameworks for integrating these concepts, the framework discussed here has proven very helpful in understanding how the various activities in scientific computing are related.

Type
Chapter
Information
Verification and Validation in Scientific Computing , pp. 21 - 82
Publisher: Cambridge University Press
Print publication year: 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aeschliman, D. P. and Oberkampf, W. L. (1998). Experimental methodology for computational fluid dynamics code validation. AIAA Journal. 36(5), 733–741.CrossRefGoogle Scholar
AIAA (1998). Guide for the verification and validation of computational fluid dynamics simulations. American Institute of Aeronautics and Astronautics, AIAA-G-077–1998, Reston, VA.Google Scholar
Almond, R. G. (1995). Graphical Belief Modeling. 1st edn., London, Chapman & Hall.CrossRefGoogle Scholar
Anderson, M. G. and Bates, P. D., eds. (2001). Model Validation: Perspectives in Hydrological Science. New York, NY, John Wiley & Sons Ltd.Google Scholar
Ang, A. H.-S. and Tang, W. H. (2007). Probability Concepts in Engineering: Emphasis on Applications to Civil and Environmental Engineering. 2nd edn., New York, Wiley.Google Scholar
ANS (1987). Guidelines for the verification and validation of scientific and engineering computer programs for the nuclear industry. American Nuclear Society, ANSI/ANS-10.4–1987, La Grange Park, IL.Google Scholar
Apostolakis, G. (1990). The concept of probability in safety assessments of technological systems. Science. 250(4986), 1359–1364.CrossRefGoogle ScholarPubMed
Arthur, J. D. and Nance, R. E. (1996). Independent verification and validation: a missing link in simulation methodology? 1996 Winter Simulation Conference, Coronado, CA, 229–236.Google Scholar
ASME (2006). Guide for verification and validation in computational solid mechanics. American Society of Mechanical Engineers, ASME Standard V&V 10–2006, New York, NY.Google Scholar
Ayyub, B. M. (2001). Elicitation of Expert Opinions for Uncertainty and Risks, Boca Raton, FL, CRC Press.CrossRefGoogle Scholar
Ayyub, B. M. and Klir, G. J. (2006). Uncertainty Modeling and Analysis in Engineering and the Sciences, Boca Raton, FL, Chapman & Hall.CrossRefGoogle Scholar
Bae, H.-R., Grandhi, R. V., and Canfield, R. A. (2006). Sensitivity analysis of structural response uncertainty propagation using evidence theory. Structural and Multidisciplinary Optimization. 31(4), 270–279.CrossRefGoogle Scholar
Beck, M. B. (1987). Water quality modeling: a review of the analysis of uncertainty. Water Resources Research. 23(8), 1393–1442.CrossRefGoogle Scholar
Beven, K. (2002). Towards a coherent philosophy of modelling the environment. Proceedings of the Royal Society of London, Series A. 458(2026), 2465–2484.CrossRefGoogle Scholar
Blottner, F. G. (1990). Accurate Navier--Stokes results for the hypersonic flow over a spherical nosetip. Journal of Spacecraft and Rockets. 27(2), 113–122.CrossRefGoogle Scholar
Bogen, K. T. and Spear, R. C. (1987). Integrating uncertainty and interindividual variability in environmental risk assessment. Risk Analysis. 7(4), 427–436.CrossRefGoogle ScholarPubMed
Bossel, H. (1994). Modeling and Simulation. 1st edn., Wellesley, MA, A. K. Peters.CrossRefGoogle Scholar
Boughton, B., Romero, V. J., Tieszen, S. R., and Sobolik, K. B. (2003). Integrated modeling and simulation validation plan for W80–3 abnormal thermal environment qualification – Version 1.0 (OUO). Sandia National Laboratories, SAND2003–4152 (OUO), Albuquerque, NM.Google Scholar
Boyack, B. E., Catton, I., Duffey, R. B., Griffith, P., Katsma, K. R., Lellouche, G. S., Levy, S., Rohatgi, U. S., Wilson, G. E., Wulff, W., and Zuber, N. (1990). Quantifying reactor safety margins, Part 1: An overview of the code scaling, applicability, and uncertainty evaluation methodology. Nuclear Engineering and Design. 119, 1–15.CrossRefGoogle Scholar
Bradley, R. G. (1988). CFD validation philosophy. Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, AGARD-CP-437, Lisbon, Portugal, North Atlantic Treaty Organization.Google Scholar
Cacuci, D. G. (2003). Sensitivity and Uncertainty Analysis: Theory, Boca Raton, FL, Chapman & Hall/CRC.CrossRefGoogle Scholar
Carnap, R. (1963). Testability and meaning. Philosophy of Science. 3(4), 419–471.CrossRefGoogle Scholar
Casti, J. L. (1990). Searching for Certainty: What Scientists Can Know About the Future, New York, William Morrow.Google Scholar
Chiles, J.-P. and Delfiner, P. (1999). Geostatistics: Modeling Spatial Uncertainty, New York, John Wiley.CrossRefGoogle Scholar
Choi, S.-K., Grandhi, R. V., and Canfield, R. A. (2007). Reliability-based Structural Design, London, Springer-Verlag.Google Scholar
Churchman, C. W. (1968). The Systems Approach, New York, Dell.Google Scholar
Coleman, H. W. and Stern, F. (1997). Uncertainties and CFD code validation. Journal of Fluids Engineering. 119, 795–803.CrossRefGoogle Scholar
Cosner, R. R. (1995). CFD validation requirements for technology transition. 26th AIAA Fluid Dynamics Conference, AIAA Paper 95–2227, San Diego, CA, American Institute of Aeronautics and Astronautics.Google Scholar
Cox, E. (1999). The Fuzzy Systems Handbook: a Practitioner's Guide to Building, Using, and Maintaining Fuzzy Systems. 2nd edn., San Diego, CA, AP Professional.Google Scholar
Cullen, A. C. and Frey, H. C. (1999). Probabilistic Techniques in Exposure Assessment: a Handbook for Dealing with Variability and Uncertainty in Models and Inputs, New York, Plenum Press.Google Scholar
Davis, P. A., Olague, N. E., and Goodrich, M. T. (1991). Approaches for the validation of models used for performance assessment of high-level nuclear waste repositories. Sandia National Laboratories, NUREG/CR-5537; SAND90–0575, Albuquerque, NM.Google Scholar
Davis, P. K. (1992). Generalizing concepts and methods of verification, validation, and accreditation (VV&A) for military simulations. RAND, R-4249-ACQ, Santa Monica, CA.Google Scholar
de Cooman, G., Ruan, D., and Kerre, E. E., eds. (1995). Foundations and Applications of Possibility Theory. Singapore, World Scientific Publishing Co.CrossRefGoogle Scholar
DoD (1994). DoD Directive No. 5000.59: Modeling and Simulation (M&S) Management. from .
DoD (1996). DoD Instruction 5000.61: Modeling and Simulation (M&S) Verification, Validation, and Accreditation (VV&A), Defense Modeling and Simulation Office, Office of the Director of Defense Research and Engineering.
DoD (1997). DoD Modeling and Simulation Glossary. from .
DoD (2008). Department of Defense Standard Practice: Documentation of Verification, Validation, and Accreditation (VV&A) for Models and Simulations. US Washington, DC, Department of Defense.
Dowding, K. (2001). Quantitative validation of mathematical models. ASME International Mechanical Engineering Congress Exposition, New York, American Society of Mechanical Engineers.Google Scholar
Drosg, M. (2007). Dealing with Uncertainties: a Guide to Error Analysis, Berlin, Springer.Google Scholar
Dubois, D. and Prade, H. (1988). Possibility Theory: an Approach to Computerized Processing of Uncertainty, New York, Plenum Press.CrossRefGoogle Scholar
Dubois, D. and Prade, H., eds. (2000). Fundamentals of Fuzzy Sets. Boston, MA, Kluwer Academic Publishers.CrossRefGoogle Scholar
Easterling, R. G. (2001a). Measuring the predictive capability of computational models: principles and methods, issues and illustrations. Sandia National Laboratories, SAND2001–0243, Albuquerque, NM.Google Scholar
Easterling, R. G. (2001b). “Quantifying the Uncertainty of Computational Predictions.” Sandia National Laboratories, SAND2001–0919C, Albuquerque, NM.Google Scholar
Elishakoff, I. (2004). Safety Factors and Reliability: Friends or Foes?, Norwell, MA, Kluwer Academic Publishers.CrossRefGoogle Scholar
Ferson, S. and Ginzburg, L. R. (1996). Different methods are needed to propagate ignorance and variability. Reliability Engineering and System Safety. 54, 133–144.CrossRefGoogle Scholar
Ferson, S. and Hajagos, J. G. (2004). Arithmetic with uncertain numbers: rigorous and (often) best possible answers. Reliability Engineering and System Safety. 85(1–3), 135–152.CrossRefGoogle Scholar
Ferson, S., Nelsen, R. B., Hajagos, J., Berleant, D. J., Zhang, J., Tucker, W. T., Ginzburg, L. R., and Oberkampf, W. L. (2004). Dependence in probabilistic modeling, Dempster -- Shafer theory, and probability bounds analysis. Sandia National Laboratories, SAND2004–3072, Albuquerque, NM.Google Scholar
Fetz, T., Oberguggenberger, M., and Pittschmann, S. (2000). Applications of possibility and evidence theory in civil engineering. International Journal of Uncertainty. 8(3), 295–309.CrossRefGoogle Scholar
Gass, S. I. (1993). Model accreditation: a rationale and process for determining a numerical rating. European Journal of Operational Research. 66, 250–258.CrossRefGoogle Scholar
Grabe, M. (2005). Measurement Uncertainties in Science and Technology, Berlin, Springer.Google Scholar
Guan, J. and Bell, D. A. (1991). Evidence Theory and Its Applications, Amsterdam, North Holland.Google Scholar
Haimes, Y. Y. (2009). Risk Modeling, Assessment, and Management. 3rd edn., New York, John Wiley.Google Scholar
Haldar, A. and Mahadevan, S. (2000). Probability, Reliability, and Statistical Methods in Engineering Design, New York, John Wiley.Google Scholar
Helton, J. C. (1993). Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliability Engineering and System Safety. 42(2–3), 327–367.CrossRefGoogle Scholar
Helton, J. C. (1994). Treatment of uncertainty in performance assessments for complex systems. Risk Analysis. 14(4), 483–511.CrossRefGoogle Scholar
Helton, J. C. (1997). Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty. Journal of Statistical Computation and Simulation. 57, 3–76.CrossRefGoogle Scholar
Helton, J. C. (1999). Uncertainty and sensitivity analysis in performance assessment for the waste isolation pilot plant. Computer Physics Communications. 117(1–2), 156–180.CrossRefGoogle Scholar
Helton, J. C., Oberkampf, W. L., and Johnson, J. D. (2005). Competing failure risk analysis using evidence theory. Risk Analysis. 25(4), 973–995.CrossRefGoogle ScholarPubMed
Hills, R. G. and Trucano, T. G. (1999). Statistical validation of engineering and scientific models: background. Sandia National Laboratories, SAND99–1256, Albuquerque, NM.Google Scholar
Hills, R. G. and Trucano, T. G. (2001). Statistical validation of engineering and scientific models with application to CTH. Sandia National Laboratories, SAND2001–0312, Albuquerque, NM.Google Scholar
Hills, R. G. and Trucano, T. G. (2002). Statistical validation of engineering and scientific models: a maximum likelihood based metric. Sandia National Laboratories, SAND2001–1783, Albuquerque, NM.Google Scholar
Hodges, J. S. and Dewar, J. A. (1992). Is it you or your model talking? A framework for model validation. RAND, R-4114-AF/A/OSD, Santa Monica, CA.Google Scholar
Hoffman, F. O. and Hammonds, J. S. (1994). Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. Risk Analysis. 14(5), 707–712.CrossRefGoogle ScholarPubMed
IEEE (1984). IEEE Standard Dictionary of Electrical and Electronics Terms. ANSI/IEEE Std 100–1984, New York.
IEEE (1991). IEEE Standard Glossary of Software Engineering Terminology. IEEE Std 610.12–1990, New York.
ISO (1991). ISO 9000–3: Quality Management and Quality Assurance Standards – Part 3: Guidelines for the Application of ISO 9001 to the Development, Supply and Maintenance of Software. Geneva, Switzerland, International Organization for Standardization.Google Scholar
Kafka, P. (1994). Important issues using PSA technology for design of new systems and plants. Reliability Engineering and System Safety. 45(1–2), 205–213.CrossRefGoogle Scholar
Kaplan, S. and Garrick, B. J. (1981). On the quantitative definition of risk. Risk Analysis. 1(1), 11–27.CrossRefGoogle Scholar
Kearfott, R. B. and Kreinovich, V., eds. (1996). Applications of Interval Computations. Boston, MA, Kluwer Academic Publishers.CrossRefGoogle Scholar
Kleindorfer, G. B., O’Neill, L., and Ganeshan, R. (1998). Validation in simulation: various positions in the philosophy of science. Management Science. 44(8), 1087–1099.CrossRefGoogle Scholar
Klir, G. J. (1969). An Approach to General Systems Theory, New York, NY, Van Nostrand Reinhold.Google Scholar
Klir, G. J. and Wierman, M. J. (1998). Uncertainty-Based Information: Elements of Generalized Information Theory, Heidelberg, Physica-Verlag.Google Scholar
Klir, G. J. and Yuan, B. (1995). Fuzzy Sets and Fuzzy Logic, Saddle River, NJ, Prentice Hall.Google Scholar
Klir, G. J., Clair, U. St., and Yuan, B. (1997). Fuzzy Set Theory: Foundations and Applications, Upper Saddle River, NJ, Prentice Hall PTR.Google Scholar
Kloeden, P. E. and Platen, E. (2000). Numerical Solution of Stochastic Differential Equations, New York, Springer.Google Scholar
Kohlas, J. and Monney, P.-A. (1995). A Mathematical Theory of Hints – an Approach to the Dempster -- Shafer Theory of Evidence, Berlin, Springer.CrossRefGoogle Scholar
Konikow, L. F. and Bredehoeft, J. D. (1992). Ground-water models cannot be validated. Advances in Water Resources. 15, 75–83.CrossRefGoogle Scholar
Kozine, I. (1999). Imprecise probabilities relating to prior reliability assessments. 1st International Symposium on Imprecise Probabilities and Their Applications, Ghent, Belgium.Google Scholar
Krause, P. and Clark, D. (1993). Representing Uncertain Knowledge: an Artificial Intelligence Approach, Dordrecht, The Netherlands, Kluwer Academic Publishers.CrossRefGoogle Scholar
Kuhn, T. S. (1962). The Structure of Scientific Revolutions. 3rd edn., Chicago and London, University of Chicago Press.Google Scholar
Kumamoto, H. and Henley, E. J. (1996). Probabilistic Risk Assessment and Management for Engineers and Scientists. 2nd edn., New York, IEEE Press.Google Scholar
Law, A. M. (2006). Simulation Modeling and Analysis. 4th edn., New York, McGraw-Hill.Google Scholar
LeGore, T. (1990). Predictive software validation methodology for use with experiments having limited replicability. In Benchmark Test Cases for Computational Fluid Dynamics. Celik, I. and Freitas, C. J. (eds.). New York, American Society of Mechanical Engineers. FED-Vol. 93: 21–27.Google Scholar
Lewis, R. O. (1992). Independent Verification and Validation. 1st edn., New York, John Wiley.Google Scholar
Lin, S. J., Barson, S. L., and Sindir, M. M. (1992). Development of evaluation criteria and a procedure for assessing predictive capability and code performance. Advanced Earth-to-Orbit Propulsion Technology Conference, Huntsville, AL, Marshall Space Flight Center.Google Scholar
Lipton, P. (2005). Testing hypotheses: prediction and prejudice. Science. 307, 219–221.CrossRefGoogle ScholarPubMed
Marvin, J. G. (1988). Accuracy requirements and benchmark experiments for CFD validation. Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, AGARD-CP-437, Lisbon, Portugal, AGARD.
Marvin, J. G. (1995). Perspective on computational fluid dynamics validation. AIAA Journal. 33(10), 1778–1787.CrossRefGoogle Scholar
Mehta, U. B. (1990). The aerospace plane design challenge: credible computational fluid dynamics results. Moffett Field, NASA, TM 102887.CrossRef
Melchers, R. E. (1999). Structural Reliability Analysis and Prediction. 2nd Edn., New York, John Wiley.Google Scholar
Modarres, M. (1993). What Every Engineer Should Know about Reliability and Risk Analysis, New York, Marcel Dekker.Google Scholar
Modarres, M., Kaminskiy, M., and Krivtsov, V. (1999). Reliability Engineering and Risk Analysis; a Practical Guide, Boca Raton, FL, CRC Press.Google Scholar
Moore, R. E. (1979). Methods and Applications of Interval Analysis, Philadelphia, PA, SIAM.CrossRefGoogle Scholar
Morgan, M. G. and Henrion, M. (1990). Uncertainty: a Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. 1st edn., Cambridge, UK, Cambridge University Press.CrossRefGoogle Scholar
Naylor, T. H. and Finger, J. M. (1967). Verification of computer simulation models. Management Science. 14(2), 92–101.CrossRefGoogle Scholar
NRC (1990). Severe Accident Risks: An Assessment for Five U.S. Nuclear Power Plants. U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research, Division of Systems Research, NUREG-1150, Washington, DC.Google Scholar
Oberkampf, W. L. and Aeschliman, D. P. (1992). Joint computational/experimental aerodynamics research on a hypersonic vehicle: Part 1, experimental results. AIAA Journal. 30(8), 2000–2009.CrossRefGoogle Scholar
Oberkampf, W. L. and Barone, M. F. (2004). Measures of agreement between computation and experiment: validation metrics. 34th AIAA Fluid Dynamics Conference, AIAA Paper 2004–2626, Portland, OR, American Institute of Aeronautics and Astronautics.Google Scholar
Oberkampf, W. L. and Barone, M. F. (2006). Measures of agreement between computation and experiment: validation metrics. Journal of Computational Physics. 217(1), 5–36.CrossRefGoogle Scholar
Oberkampf, W. L. and Blottner, F. G. (1998). Issues in computational fluid dynamics code verification and validation. AIAA Journal. 36(5), 687–695.CrossRefGoogle Scholar
Oberkampf, W. L. and Ferson, S. (2007). Model validation under both aleatory and epistemic uncertainty. NATO/RTO Symposium on Computational Uncertainty in Military Vehicle Design, AVT-147/RSY-022, Athens, Greece, NATO.
Oberkampf, W. L. and Helton, J. C. (2005). Evidence theory for engineering applications. In Engineering Design Reliability Handbook. Nikolaidis, E., Ghiocel, D. M., and Singhal, S., eds. New York, NY, CRC Press: 29.Google Scholar
Oberkampf, W. L. and Trucano, T. G. (2000). Validation methodology in computational fluid dynamics. Fluids 2000 Conference, AIAA Paper 2000–2549, Denver, CO, American Institute of Aeronautics and Astronautics.Google Scholar
Oberkampf, W. L. and Trucano, T. G. (2002). Verification and validation in computational fluid dynamics. Progress in Aerospace Sciences. 38(3), 209–272.CrossRefGoogle Scholar
Oberkampf, W. L. and Trucano, T. G. (2007). Verification and Validation Benchmarks. Albuquerque, NM, Sandia National Laboratories, SAND2007–0853.CrossRefGoogle Scholar
Oberkampf, W. L. and Trucano, T. G. (2008). Verification and validation benchmarks. Nuclear Engineering and Design. 238(3), 716–743.CrossRefGoogle Scholar
Oberkampf, W. L., Blottner, F. G., and Aeschliman, D. P. (1995). Methodology for computational fluid dynamics code verification/validation. 26th AIAA Fluid Dynamics Conference, AIAA Paper 95–2226, San Diego, CA, American Institute of Aeronautics and Astronautics.Google Scholar
Oberkampf, W. L., Trucano, T. G., and Hirsch, C. (2004). Verification, validation, and predictive capability in computational engineering and physics. Applied Mechanics Reviews. 57(5), 345–384.CrossRefGoogle Scholar
Oksendal, B. (2003). Stochastic Differential Equations: an Introduction with Applications. 6th edn., Berlin, Springer.CrossRefGoogle Scholar
Oren, T. I. and Zeigler, B. P. (1979). Concepts for advanced simulation methodologies. Simulation, 69–82.CrossRefGoogle Scholar
Oreskes, N., Shrader-Frechette, K., and Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the Earth Sciences. Science. 263, 641–646.CrossRefGoogle ScholarPubMed
Paez, T. and Urbina, A. (2001). Validation of structural dynamics models via hypothesis testing. Society of Experimental Mechanics Annual Conference, Portland, OR, Society of Experimental Mechanics.Google Scholar
Parry, G. W. (1988). On the meaning of probability in probabilistic safety assessment. Reliability Engineering and System Safety. 23, 309–314.CrossRefGoogle Scholar
Parry, G. W. and Winter, P. W. (1981). Characterization and evaluation of uncertainty in probabilistic risk analysis. Nuclear Safety. 22(1), 28–41.Google Scholar
Paté-Cornell, M. E. (1996). Uncertainties in risk analysis: six levels of treatment. Reliability Engineering and System Safety. 54, 95–111.CrossRefGoogle Scholar
Pilch, M., Trucano, T. G., Moya, J. L., Froehlich, G. K., Hodges, A. L., and Peercy, D. E. (2001). Guidelines for Sandia ASCI Verification and Validation Plans – Content and Format: Version 2. Albuquerque, NM, Sandia National Laboratories, SAND2000–3101.Google Scholar
Platt, D. S. (2007). Why Software Sucks…and what you can do about it, Upper Saddle River, NJ, Addison-Wesley.Google Scholar
Popper, K. R. (1959). The Logic of Scientific Discovery, New York, Basic Books.Google Scholar
Popper, K. R. (1969). Conjectures and Refutations: the Growth of Scientific Knowledge, London, Routledge and Kegan.Google Scholar
Rabinovich, S. G. (2005). Measurement Errors and Uncertainties: Theory and Practice. 3rd edn., New York, Springer-Verlag.Google Scholar
Raczynski, S. (2006). Modeling and Simulation: the Computer Science of Illusion, New York, Wiley.CrossRefGoogle Scholar
Refsgaard, J. C. and Henriksen, H. J. (2004). Modelling guidelines -- terminology and guiding principles. Advances in Water Resources. 27, 71–82.CrossRefGoogle Scholar
Roache, P. J. (1990). Need for control of numerical accuracy. Journal of Spacecraft and Rockets. 27(2), 98–102.CrossRefGoogle Scholar
Roache, P. J. (1995). Verification of codes and calculations. 26th AIAA Fluid Dynamics Conference, AIAA Paper 95–2224, San Diego, CA, American Institute of Aeronautics and Astronautics.Google Scholar
Roache, P. J. (1998). Verification and Validation in Computational Science and Engineering, Albuquerque, NM, Hermosa Publishers.Google Scholar
Roza, Z. C. (2004). Simulation Fidelity, Theory and Practice: a Unified Approach to Defining, Specifying and Measuring the Realism of Simulations, Delft, The Netherlands, Delft University Press.Google Scholar
Rubinstein, R. Y. and Kroese, D. P. (2008). Simulation and the Monte Carlo Method. 2nd edn., Hoboken, NJ, John Wiley.Google Scholar
Rykiel, E. J. (1996). Testing ecological models: the meaning of validation. Ecological Modelling. 90(3), 229–244.CrossRefGoogle Scholar
Saltelli, A., Tarantola, S., Campolongo, F., and Ratto, M. (2004). Sensitivity Analysis in Practice: a Guide to Assessing Scientific Models, Chichester, England, John Wiley & Sons, Ltd.Google Scholar
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., and Tarantola, S. (2008). Global Sensitivity Analysis: the Primer, Hoboken, NJ, Wiley.Google Scholar
Sargent, R. G. (1979). Validation of simulation models. 1979 Winter Simulation Conference, San Diego, CA, 497–503.Google Scholar
Schlesinger, S. (1979). Terminology for model credibility. Simulation. 32(3), 103–104.Google Scholar
Serrano, S. E. (2001). Engineering Uncertainty and Risk Analysis: a Balanced Approach to Probability, Statistics, Stochastic Modeling, and Stochastic Differential Equations, Lexington, KY, HydroScience Inc.Google Scholar
Shannon, R. E. (1975). Systems Simulation: the Art and Science, Englewood Cliffs, NJ, Prentice-Hall.Google Scholar
Sheng, G., Elzas, M. S., Oren, T. I., and Cronhjort, B. T. (1993). Model validation: a systemic and systematic approach. Reliability Engineering and System Safety. 42, 247–259.CrossRefGoogle Scholar
Sindir, M. M., Barson, S. L., Chan, D. C., and Lin, W. H. (1996). On the development and demonstration of a code validation process for industrial applications. 27th AIAA Fluid Dynamics Conference, AIAA Paper 96–2032, New Orleans, LA, American Institute of Aeronautics and Astronautics.CrossRef
Smithson, M. (1989). Ignorance and Uncertainty: Emerging Paradigms, New York, Springer-Verlag.CrossRefGoogle Scholar
Stern, F., Wilson, R. V., Coleman, H. W., and Paterson, E. G. (2001). Comprehensive approach to verification and validation of CFD simulations -- Part 1: Methodology and procedures. Journal of Fluids Engineering. 123(4), 793–802.CrossRefGoogle Scholar
Stockman, C. T., Garner, J. W., Helton, J. C., Johnson, J. D., Shinta, A., and Smith, L. N. (2000). Radionuclide transport in the vicinity of the repository and associated complementary cumulative distribution functions in the 1996 performance assessment for the Waste Isolation Pilot Plant. Reliability Engineering and System Safety. 69(1–3), 369–396.CrossRefGoogle Scholar
Suter, G. W. (2007). Ecological Risk Assessment. 2nd edn., Boca Raton, FL, CRC Press.Google Scholar
Taylor, H. M. and Karlin, S. (1998). An Introduction to Stochastic Modeling. 3rd edn., Boston, Academic Press.Google Scholar
Tieszen, S. R., Chu, T. Y., Dobranich, D., Romero, V. J., Trucano, T. G., Nakos, J. T., Moffatt, W. C., Hendrickson, T. F., Sobolik, K. B., Kempka, S. N., and Pilch, M. (2002). Integrated Modeling and Simulation Validation Plan for W76–1 Abnormal Thermal Environment Qualification – Version 1.0 (OUO). Sandia National Laboratories, SAND2002–1740 (OUO), Albuquerque.
Trucano, T. G., Easterling, R. G., Dowding, K. J., Paez, T. L., Urbina, A., Romero, V. J., Rutherford, R. M., and Hills, R. G. (2001). Description of the Sandia Validation Metrics Project. Albuquerque, NM, Sandia National Laboratories, SAND2001–1339.Google Scholar
Trucano, T. G., Pilch, M., and Oberkampf, W. L. (2002). General Concepts for Experimental Validation of ASCI Code Applications. Albuquerque, NM, Sandia National Laboratories, SAND2002–0341.CrossRefGoogle Scholar
Tsang, C.-F. (1989). A broad view of model validation. Proceedings of the Symposium on Safety Assessment of Radioactive Waste Repositories, Paris, France, Paris, France, OECD, 707–716.
Urbina, A. and Paez, T. L. (2001). Statistical validation of structural dynamics models. Annual Technical Meeting & Exposition of the Institute of Environmental Sciences and Technology, Phoenix, AZ.Google Scholar
Vose, D. (2008). Risk Analysis: a Quantitative Guide. 3rd edn., New York, Wiley.Google Scholar
Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, London, Chapman & Hall.CrossRefGoogle Scholar
Wilson, G. E. and Boyack, B. E. (1998). The role of the PIRT in experiments, code development and code applications associated with reactor safety assessment. Nuclear Engineering and Design. 186, 23–37.CrossRefGoogle Scholar
Wilson, G. E., Boyack, B. E., Catton, I., Duffey, R. B., Griffith, P., Katsma, K. R., Lellouche, G. S., Levy, S., Rohatgi, U. S., Wulff, W., and Zuber, N. (1990). Quantifying reactor safety margins, Part 2: Characterization of important contributors to uncertainty. Nuclear Engineering and Design. 119, 17–31.CrossRefGoogle Scholar
Wulff, W., Boyack, B. E., Catton, I., Duffey, R. B., Griffith, P., Katsma, K. R., Lellouche, G. S., Levy, S., Rohatgi, U. S., Wilson, G. E., and Zuber, N. (1990). Quantifying reactor safety margins, Part 3: Assessment and ranging of parameters. Nuclear Engineering and Design. 119, 33–65.CrossRefGoogle Scholar
Zeigler, B. P. (1976). Theory of Modelling and Simulation. 1st edn., New York, John Wiley.Google Scholar
Zeigler, B. P., Praehofer, H., and Kim, T. G. (2000). Theory of Modeling and Simulation: Integrating Discrete Event and Continuous Complex Dynamic Systems. 2nd edn., San Diego, CA, Academic Press.Google Scholar
Zuber, N., Wilson, G. E., Ishii, M., Wulff, W., Boyack, B. E., Dukler, A. E., Griffith, P., Healzer, J. M., Henry, R. E., Lehner, J. R., Levy, S., and Moody, F. J. (1998). An integrated structure and scaling methodology for severe accident technical issue resolution: development of methodology. Nuclear Engineering and Design. 186(1–2), 1–21.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×