Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification & validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Independent Verification and Validation Of SAPHIRE 8 Software Quality Assurance Plan Project Number: N6423 U.S. Nuclear Regulatory Commission

This report provides an evaluation of the Software Quality Assurance Plan. The Software Quality Assurance Plan is intended to ensure all actions necessary for the software life cycle; verification and validation activities; documentation and deliverables; project management; configuration management, nonconformance reporting and corrective action; and quality assessment and improvement have been planned and a systematic pattern of all actions necessary to provide adequate confidence that a software product conforms to established technical requirements; and to meet the contractual commitments prepared by the sponsor; the Nuclear Regulatory Commission.

99 GENERAL AND MISCELLANEOUS↗

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors

The document compiles in a single volume the verification and validation works done for the PLTEMP/ANL code during the years of its development and improvement. The verification of sixteen capabilities of the PLTEMP/ANL Version 4.3 code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis are described and documented. Each chapter of the document deals with the verification or validation of a specific model. The model verification is usually done by comparing the code with hand calculation, Microsoft spreadsheet calculation, or Mathematica calculation. The model validation is done by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL verification works that are available in the open literature are simply referenced and not included in the document. PLTEMP/ANL has been used in conversion safety analysis reports of several US and foreign research reactors that have been licensed and converted. A list of such reactors is given.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Survey of Product-line Verification and Validation Techniques

This report presents the results from the first task of the SARP Center Initiative, 'Product Line Verification of Safety-Critical Software.' Task 1 is a literature survey of available techniques for product line verification and validation. Section 1 of the report provides an introduction to product lines and motivates the survey of verification techniques. It describes what is reused in product-line engineering and explains the goal of verifiable conformance of the developed system to its product-line specifications. Section 2 of the report describes six lifecycle steps in product-line verification and validation. This description is based on, and refers to, the best practices extracted from the readings. It ends with a list of verification challenges for NASA product lines (2.7) and verification enablers for NASA product lines (2.8) derived from the survey. Section 3 provides resource lists of related conferences, workshops, industrial and defense industry experiences and case studies of product lines, and academic/industrial consortiums. Section 4 is a bibliography of papers and tutorials with annotated entries for relevant papers not previously discussed in sections 2 or 3.

testing↗

CFD Verification and Validation of Wire-Wrapped Pin Assemblies

TerraPower is participating in a cooperative project among industry, the national lab system, and academia to perform Verification and Validation (V and V) of Computational Fluid Dynamics (CFD) methods for predicting the flow and heat transfer within liquid-metal-cooled nuclear fuel assemblies with wire-wrapped fuel pins. This project, consisting of both experimental and numerical components, uses surrogate fluids and electrically heated fuel pins to substitute for the liquid metal coolant and the nuclear heat source. The experiments include both unheated assemblies to measure the flow velocities with Laser Doppler Velocimetry (LDV) and Particle Image Velocimetry (PIV), as well as heated assemblies to measure the temperatures of the fluid and the simulated fuel pins with thermocouples. Both sets of experiments also measure the pressure at multiple locations in the system. The numerical component of this work involves high-fidelity Large-Eddy Simulation (LES) modeling and industry-standard Reynolds-Averaged Navier-Stokes (RANS) modeling of the experiments. Furthermore, the experiments and simulations will be conducted on two duct geometries: the first is a non-deformed hexagonal duct that has flat internal duct faces that represents a beginning of life geometry. The other is a deformed duct where the internal hexagonal flow area enlarges then decreases along the axial height to represent the changes that occur to the duct geometry from pressure and irradiation effects in the core. This paper describes the methods and results of the industrial, RANS-based CFD code verification, solution verification, and solution validation. The V and V simulations are being performed for the following experiment test geometries: - Isothermal non-deformed bundle; - Heated non-deformed bundle; - Isothermal deformed bundle; - Heated deformed bundle. These flows have been computed at first with the pre-test design specifications, without the aid of any experimental results. Later, post-test calculations will be performed incorporating the actual test conditions and input from some of the experimental results. These RANS simulations of helically wire-wrapped fuel assemblies employ meshes of bare pins without the wire-wrap in the bundle. The effect of wire-wrap on the flow is accounted for by introducing a momentum source into the governing fluid equations. The momentum source is only applied to cells corresponding to the location of the wire-wrap, and its components in each cell are based on the local flow field. This methodology simplifies the geometry and mesh generation and reduces the cell count, while still capturing the effects of the wire-wrap. The momentum source approach is used for all of the assembly simulations performed in this study. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

MPACT Verification and Validation Manual (rev 6)

The MPACT code provides the capability to calculate the steady-state and transient reactor core neutron distribution for operating nuclear power plant (PWR) conditions across multiple fuel cycles. As required in the CASL SQA plan, it is the responsibility of UM and ORNL as joint owners of MPACT to ensure that verification and validation activities are performed and documented in a V&V manual with supporting publications and CASL technical reports which can be distributed for reference and distribution within VERA. This document provides the fifth revision of the MPACT verification and validation (V&V) manual and describes the current state of MPACT V&V and updates the framework for future MPACT V&V activities. During FY19 MPACT V&V activities in the areas of code / solution verification and code validation have continued to mature. Several specific tasks have also been completed to include the Ganapol and IPEN Benchmarks which have provided significant improvements in MPACT verification and validation, respectively. Based on the results of V&V efforts this past year, the confidence level has increased in the ability of MPACT to model an operational Pressurized Water Reactor. Specific areas in both verification and validation were identified which require increased emphasis during FY20.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Bison Verification and Validation Activities for TRISO

Numerical modeling and simulation (M&S) tools play a key role in the research, development, and overall safety assessments of next-generation nuclear energy systems. One such tool, Bison, is a nuclear fuel performance code that is applicable to many fuel forms (e.g., light-water reactor fuel, oxide and metallic fuel for fast reactors, tri-structural isotropic (TRISO) fuel, and plate fuel), and it uses the finite element method to model the thermo- mechanical response of nuclear fuels. One fuel form widely utilized in Generation-IV high-temperature gas-cooled and fluoride- salt-cooled nuclear reactor concepts is TRISO fuel. Recently, Bison’s capabilities were significantly expanded to enable it to model the performance of TRISO particles and compacts. It is important that Bison’s computational results be reliable and predictive, since this code is used to inform high-consequence decisions. The various processes developed to address this issue generally entail two fundamental steps: verification and validation (V&V). Verification ensures that the code functions correctly and is reliable. Code/solution verification, code benchmark, and software quality assurance exercises are examples of verification activities. On the other hand, validation is the process of assessing a code’s capability to accurately model physical problems. Comparisons between code results and experiments quantify the validation level. Application of V&V procedures is crucial to the development of computational tools that are free of coding mistakes and can accurately represent reality. The current study presents an overview of Bison V&V activities relevant to the TRISO fuel concept, which include code/solution verification exercises, CRP-6 Benchmark—a Coordinated Research Program through the International Atomic Energy Agency (IAEA)—exercises, and validation exercises with the Advanced Gas Reactor (AGR)- 1/2/3/4 experiment series.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Verification and Validation of Open-Source Computational Fluid Dynamics Capabilities in MOOSE

The Multiphysics Object Oriented Simulation Environment (MOOSE) advances predictive modeling in nuclear research by simplifying the application design process and making modeling more efficient and accessible to researchers and the broader community. Because of MOOSE’s critical role in the scientific computation field, verification and validation are important to evaluate the reliability and credibility of MOOSE’s code, simulations, and models. To assess MOOSE’s Navier Stokes module and turbulence models, the following verification and validation University of Manchester ERCOFTAC cases were simulated in MOOSE, and the results were compared: Case 032 Turbulent Flow in a 2D Channel Case 079 Natural Convection in an Enclosed Cavity

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

Methodologies for Verification and Validation of Space Launch System (SLS) Structural Dynamic Models: Appendices

Verification and validation (V&V) is a highly challenging undertaking for SLS structural dynamics models due to the magnitude and complexity of SLS subassemblies and subassemblies. Responses to challenges associated with V&V of Space Launch System (SLS) structural dynamics models are presented in Volume I of this paper. Four methodologies addressing specific requirements for V&V are discussed. (1) Residual Mode Augmentation (RMA). (2) Modified Guyan Reduction (MGR) and Harmonic Reduction (HR, introduced in 1976). (3) Mode Consolidation (MC). Finally, (4) Experimental Mode Verification (EMV). This document contains the appendices to Volume I.

Coppolino, Robert N.↗

Computational solution verification and validation applied to a thermal model of a ruggedized instrumentation package

This study details a methodology for quantification of errors and uncertainties of a finite element heat transfer model applied to a Ruggedized Instrumentation Package (RIP). The proposed verification and validation (V&V) process includes solution verification to examine errors associated with the code's solution techniques, and model validation to assess the model's predictive capability for quantities of interest. The model was subjected to mesh resolution and numerical parameters sensitivity studies to determine reasonable parameter values and to understand how they change the overall model response and performance criteria. To facilitate quantification of the uncertainty associated with the mesh, automatic meshing and mesh refining/coarsening algorithms were created and implemented on the complex geometry of the RIP. Automated software to vary model inputs was also developed to determine the solution’s sensitivity to numerical and physical parameters. The model was compared with an experiment to demonstrate its accuracy and determine the importance of both modelled and unmodelled physics in quantifying the results' uncertainty. An emphasis is placed on automating the V&V process to enable uncertainty quantification within tight development schedules.

97 MATHEMATICS AND COMPUTING↗

Verification and validation of the PLTEMP/ANL code for thermal hydraulic analysis of experimental and test reactors

The document compiles in a single volume several verification and validation works done for the PLTEMP/ANL code during the years of its development and improvement. Some works that are available in the open literature are simply referenced at the outset, and are not included in the document. PLTEMP has been used in conversion safety analysis reports of several US and foreign research reactors that have been licensed and converted. A list of such reactors is given. Each chapter of the document deals with the verification or validation of a specific model. The model verification is usually done by comparing the code with hand calculation, Microsoft spreadsheet calculation, or Mathematica calculation. The model validation is done by comparing the code with experimental data or a more validated code like the RELAP5 code.

Kalimullah, M.↗

Autonomy software verification and validation might not be as hard as it seems

The verification and validation of autonomy software is widely believed to be a challenging unsolved problem. To a certain extent this is true, but in this paper I argue that the problem is not nearly as severe as seems to be widely perceived. many of the perceived hard problems in autonomy software V&V also exist for traditional software, and can be solved using many of the same methods and techniques used for traditional spacecraft software. In particular, the problem of intractably large state spaces exists for any non-trivial software system.

Gat, Erann↗

NASA Orbital Debris Engineering Model (ORDEM) 3.1: Model Verification and Validation

The NASA Orbital Debris Engineering Model (ORDEM) 3.1 Model Verification and Validation (V&V) document accompanies the delivery of the latest ORDEM 3.1 model (Vavrin & Manis, 2019) and provides a detailed description of the V&V activities used to verify that the model was built correctly and validate the model against independent, real world sources of data obtained from sampling the orbital debris environment. This ORDEM 3.1 Model V&V document, along with the related ORDEM 3.1 Model Process document – which covers details of the mathematical, statistical, and physical basis of the model – are intended to inform credibility assessments, risk analyses, uncertainty characterizations, and other applications derived from use of the model by the ORDEM 3.1 user community.

Timothy Kennedy↗

Verification and validation of developed short-term forecasting models

Recent advancements in machine learning (ML) and artificial intelligence (AI) technologies provide an opportunity for leveraging data-driven algorithms to predict future nuclear power plant (NPP) operating conditions by using recorded plant process data. Successfully implementing these models can lead to cost-reducing, conditioned-based predictive maintenance through optimized maintenance schedules and a reduction of unnecessary maintenance activities. This report discusses the verification and validation of short-term forecasting processes (i.e., data cleaning, feature selection, model optimization, and forecasting) developed in previous reports. The verification and validation (V&V) process demonstrates the expected precision and accuracy when the ML model encounters new datasets from different systems. Shapley additive explanations were used as the primary means of feature selection across these different data set. Individual models were trained for each data set, then validated through a cross-validation procedure. In this report, two different ML models were tasked to predict variables from three different plant process data sets with varying prediction horizons. The results indicate that support vector regression (SVR) outperformed long short-term memory (LSTM) neural networks in regard to each data set and each prediction horizon in this study, but further tuning and optimization could improve long short-term memory results. However, each forecasting model showed reduced performance as the prediction horizon was extended from 1 hour to 1 day ahead. Research is ongoing to evaluate the optimal input variable space, which is based on a given set of process parameters, to further improve forecasting accuracy.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The NPARC Alliance Verification and Validation Archive

The NPARC Alliance (National Project for Applications oriented Research in CFD) maintains a publicly-available, web-based verification and validation archive as part of the development and support of the WIND CFD code. The verification and validation methods used for the cases attempt to follow the policies and guidelines of the ASME and AIAA. The emphasis is on air-breathing propulsion flow fields with Mach numbers ranging from low-subsonic to hypersonic.

Slater, John W.↗

Turbulence Modeling Verification and Validation

Computational fluid dynamics (CFD) software that solves the Reynolds-averaged Navier-Stokes (RANS) equations has been in routine use for more than a quarter of a century. It is currently employed not only for basic research in fluid dynamics, but also for the analysis and design processes in many industries worldwide, including aerospace, automotive, power generation, chemical manufacturing, polymer processing, and petroleum exploration. A key feature of RANS CFD is the turbulence model. Because the RANS equations are unclosed, a model is necessary to describe the effects of the turbulence on the mean flow, through the Reynolds stress terms. The turbulence model is one of the largest sources of uncertainty in RANS CFD, and most models are known to be flawed in one way or another. Alternative methods such as direct numerical simulations (DNS) and large eddy simulations (LES) rely less on modeling and hence include more physics than RANS. In DNS all turbulent scales are resolved, and in LES the large scales are resolved and the effects of the smallest turbulence scales are modeled. However, both DNS and LES are too expensive for most routine industrial usage on today's computers. Hybrid RANS-LES, which blends RANS near walls with LES away from walls, helps to moderate the cost while still retaining some of the scale-resolving capability of LES, but for some applications it can still be too expensive. Even considering its associated uncertainties, RANS turbulence modeling has proved to be very useful for a wide variety of applications. For example, in the aerospace field, many RANS models are considered to be reliable for computing attached flows. However, existing turbulence models are known to be inaccurate for many flows involving separation. Research has been ongoing for decades in an attempt to improve turbulence models for separated and other nonequilibrium flows. When developing or improving turbulence models, both verification and validation are important steps in the process. Verification insures that the CFD code is solving the equations as intended (no errors in the implementation). This is typically done either through the method of manufactured solutions (MMS) or through careful step-by-step comparisons with other verified codes. After the verification step is concluded, validation is performed to document the ability of the turbulence model to represent different types of flow physics. Validation can involve a large number of test case comparisons with experiments, theory, or DNS. Organized workshops have proved to be valuable resources for the turbulence modeling community in its pursuit of turbulence modeling verification and validation. Workshop contributors using different CFD codes run the same cases, often according to strict guidelines, and compare results. Through these comparisons, it is often possible to (1) identify codes that have likely implementation errors, and (2) gain insight into the capabilities and shortcomings of different turbulence models to predict the flow physics associated with particular types of flows. These are valuable lessons because they help bring consistency to CFD codes by encouraging the correction of faulty programming and facilitating the adoption of better models. They also sometimes point to specific areas needed for improvement in the models. In this paper, several recent workshops are summarized primarily from the point of view of turbulence modeling verification and validation. Furthermore, the NASA Langley Turbulence Modeling Resource website is described. The purpose of this site is to provide a central location where RANS turbulence models are documented, and test cases, grids, and data are provided. The goal of this paper is to provide an abbreviated survey of turbulence modeling verification and validation efforts, summarize some of the outcomes, and give some ideas for future endeavors in this area.

Rumsey, Christopher L.↗

Cognitive Bias in the Verification and Validation of Space Flight Systems

Cognitive bias is generally recognized as playing a significant role in virtually all domains of human decision making. Insight into this role is informally built into many of the system engineering practices employed in the aerospace industry. The review process, for example, typically has features that help to counteract the effect of bias. This paper presents a discussion of how commonly recognized biases may affect the verification and validation process. Verifying and validating a system is arguably more challenging than development, both technically and cognitively. Whereas there may be a relatively limited number of options available for the design of a particular aspect of a system, there is a virtually unlimited number of potential verification scenarios that may be explored. The probability of any particular scenario occurring in operations is typically very difficult to estimate, which increases reliance on judgment that may be affected by bias. Implementing a verification activity often presents technical challenges that, if they can be overcome at all, often result in a departure from actual flight conditions (e.g., 1-g testing, simulation, time compression, artificial fault injection) that may raise additional questions about the meaningfulness of the results, and create opportunities for the introduction of additional biases. In addition to mitigating the biases it can introduce directly, the verification and validation process must also overcome the cumulative effect of biases introduced during all previous stages of development. A variety of cognitive biases will be described, with research results for illustration. A handful of case studies will be presented that show how cognitive bias may have affected the verification and validation process on recent JPL flight projects, identify areas of strength and weakness, and identify potential changes or additions to commonly used techniques that could provide a more robust verification and validation of future systems.

Larson, Steve↗

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors (Code Version 4.3)

The document compiles in a single volume the verification and validation works done for the PLTEMP/ANL code during the years of its development and improvement. The verification of sixteen capabilities of the PLTEMP/ANL Version 4.3 code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis are described and documented. Each chapter of the document deals with the verification or validation of a specific part of the software that calculates a specific phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. The software verification work is usually done by comparing the code with hand calculation, Microsoft spreadsheet calculation, or Mathematica calculation. The software validation is done by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL verification works that are available in the open literature are simply referenced and not included in the document. PLTEMP/ANL has been used in conversion safety analysis reports of several US and foreign research reactors that have been licensed and converted. A list of such reactors is given.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Argonne Reactor Code Software Verification and Validation Plan for the Versatile Test Reactor

This Argonne Reactor Code (ARC) software has taken a pivotal role in the design work of the Versatile Test Reactor. The ARC software itself consists of several independent or semiindependent pieces of software that are used in concert to carry out specific analysis and design objectives. In this report, we discuss the validation that is needed for the ARC software overall and discuss available, usable experimental data and how it relates to the VTR project. We also discuss the verification that is required for the ARC software with respect to its usage in the VTR project and give some concrete examples as to what it must accomplish. A list of goals for the verification and validation work are presented as the general procedure that will be implemented for every individual piece of software in ARC. The ARC software has been continuously developed for over 50 years and many of the features used for the VTR project have thus been rigorously tested for accuracy during that timeframe. The ARC software is presently under source code control and undergoes nightly regression testing of not only the individual pieces, but the interconnected aspects that are common to most analysis projects. Its greatest weakness, from a software verification and validation perspective, is both a lack of documentation and outdated documentation. As an example, much of the previous validation work will require contemporary effort to re-evaluate given the updated base data libraries and modifications to the software that have occurred since. This requires new documentation to be produced to support the usage of the software in the VTR project. The bulk of the ARC software is focused on steady state neutronics and TH behavior of a reactor system. In this regard, we have identified the ZPPR, FFTF, and EBR-II programs as the best candidates for sources of validation data for the VTR project. The ZPPR-15 series in the ZPPR program was chosen as it is the best representation of the VTR fuel form and reactor core layout of all of the ZPPR measurements. Additional support for it comes from the fact that most of the data has already been processed into usable ARC models for VTR validation. The FFTF and EBR-II reactors were both functional reactors that provide reactor scale measurements. FFTF is clearly a closer match to VTR and its startup measurements would be a very valuable validation basis. EBR-II has invaluable burnup related measurements that will play a key role in the ARC software validation work. Each piece of ARC software will require a report detailing the software verification. The validation work will be logically split into the results of modeling each identified experimental resource (ZPPR, EBR-II, FFTF). The intended timeline for the VTR project is to have the ARC software verified and validated by the end of fiscal year 2020. This is incumbent on sufficient funding allocated to V&V activities and no change in programmatic priorities.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗