Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification & validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

HYBRID Modeling Validation and Verification Status Matrix

The HYBRID modeling repository is a premier resource for integrated energy systems modeling. HYBRID models have been developed since 2015 to describe the physical operation of tightly coupled thermal systems including power generators, thermal transport systems, thermal storage, thermal-to-electric conversion systems, and other thermal applications. Due to the increased size of the repository, a concise summary matrix of available models is desired. This matrix will consolidate not only the list of available models but also indicate original information sources, publications that have model examples, and level of validation and verification that exists for the models. Validation and verification (V&V) levels begin from simplified algebraic relationship and advance to dynamic data validation. Moving forward, as new models are contributed to HYBRID, their V&V level will be included, updating this matrix.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Challenges and opportunities in verification and validation of low temperature plasma simulations and experiments

This paper describes the verification and validation (V&V) framework developed for the stochastic Particle-in-Cell, Direct Simulation Monte Carlo code Aleph. An ideal framework for V&V from the viewpoint of the authors is described where a physics problem is defined, and relevant physics models and parameters to the defined problem are assessed and captured in a Phenomena Identification and Ranking Table (PIRT). Furthermore, numerous V&V examples guided by the PIRT for a simple gas discharge are shown to demonstrate the V&V process applied to a real-world simulation tool with the overall goal to demonstrably increase the confidence in the results for the simulation tool and its predictive capability. Although many examples are provided here to demonstrate elements of the framework, the primary goal of this work is to introduce this framework and not to provide a fully complete implementation, which would be a much longer document. Comparisons and contrasts are made to more usual approaches to V&V, and techniques new to the low-temperature plasma community are introduced. Specific challenges relating to the sufficiency of available data (e.g., cross sections), the limits of ad hoc validation approaches, the additional difficulty of utilizing a stochastic simulation tool, and the extreme cost of formal validation are discussed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA software independent verification and validation (IV and V): An upfront approach to mission success

Software independent verification and validation (IV and V) implementation methods, specific approaches and accomplishments in the application of IV and V to NASA programs are discussed. Because software is mission critical, software assurance is a critical aspect of NASA's safety and mission assurance program. An overview of NASA's software IV and V facility is given.

Gregory, Frederick D.↗

Land Ice Verification and Validation Kit

To address a pressing need to better understand the behavior and complex interaction of ice sheets within the global Earth system, significant development of continental-scale, dynamical ice-sheet models is underway. The associated verification and validation process of these models is being coordinated through a new, robust, python-based extensible software package, the Land Ice Verification and Validation toolkit (LIVV). This release provides robust and automated verification and a performance evaluation on LCF platforms. The performance V&V involves a comprehensive comparison of model performance relative to expected behavior on a given computing platform. LIVV operates on a set of benchmark and test data, and provides comparisons for a suite of community prioritized tests, including configuration and parameter variations, bit-4-bit evaluation, and plots of tests where differences occur.

Source record↗

Land Ice Verification and Validation Kit

To address a pressing need to better understand the behavior and complex interaction of ice sheets within the global Earth system, significant development of continental-scale, dynamical ice-sheet models is underway. The associated verification and validation process of these models is being coordinated through a new, robust, python-based extensible software package, the Land Ice Verification and Validation toolkit (LIVV). This release provides robust and automated verification and a performance evaluation on LCF platforms. The performance V&V involves a comprehensive comparison of model performance relative to expected behavior on a given computing platform. LIVV operates on a set of benchmark and test data, and provides comparisons for a suite of community prioritized tests, including configuration and parameter variations, bit-4-bit evaluation, and plots of tests where differences occur.

Source record↗

Verification and Validation of a Coordinate Transformation Method in Axisymmetric Transient Magnetics.

We present a verification and validation analysis of a coordinate-transformation-based numerical solution method for the two-dimensional axisymmetric magnetic diffusion equation, implemented in the finite-element simulation code ALEGRA. The transformation, suggested by Melissen and Simkin, yields an equation set perfectly suited for linear finite elements and for problems with large jumps in material conductivity near the axis. The verification analysis examines transient magnetic diffusion in a rod or wire in a very low conductivity background by first deriving an approximate analytic solution using perturbation theory. This approach for generating a reference solution is shown to be not fully satisfactory. A specialized approach for manufacturing an exact solution is then used to demonstrate second-order convergence under spatial refinement and tem- poral refinement. For this new implementation, a significant improvement relative to previously available formulations is observed. Benefits in accuracy for computed current density and Joule heating are also demonstrated. The validation analysis examines the circuit-driven explosion of a copper wire using resistive magnetohydrodynamics modeling, in comparison to experimental tests. The new implementation matches the accuracy of the existing formulation, with both formulations capturing the experimental burst time and action to within approximately 2%.

97 MATHEMATICS AND COMPUTING↗

The HTR-Proteus Benchmark: Analysis and Use as a Verification and Validation Case

This presentation outlines the evaluation and application of the HTR-Proteus benchmark as a verification and validation (V&V) case for advanced reactor modeling tools. The work supports the U.S. Department of Energy’s HALEU Availability Program (HAP) and the joint DOE/NRC DNCSH project, which aims to reduce criticality safety uncertainties in commercial-scale HALEU fuel cycle and transportation systems. The HTR-Proteus experiments, conducted at the Paul Scherrer Institute, provide high-fidelity data for TRISO-fueled, graphite-moderated pebble bed reactors with high neutron leakage—conditions relevant to HALEU transport scenarios. This study focuses on Cores 4.2 and 4.3 of the HTR-Proteus benchmark, analyzing key sources of uncertainty including pebble packing, TRISO particle positioning, and core height. Using Project Chrono for realistic pebble geometries and Serpent, SHIFT, and MCNP for neutronics simulations, the study quantifies the impact of these uncertainties on the effective multiplication factor (keff). Results show that a sample size of 110 pebble configurations is sufficient to converge keff, with ±30 pcm uncertainty due to packing randomness. TRISO positioning and core height variations also significantly influence keff, highlighting the importance of detailed modeling in V&V efforts. The benchmark serves as a valuable test case for validating the Griffin reactor physics code and improving confidence in HALEU system simulations.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Knowledge-based system verification and validation

The objective of this task is to develop and evaluate a methodology for verification and validation (V&V) of knowledge-based systems (KBS) for space station applications with high reliability requirements. The approach consists of three interrelated tasks. The first task is to evaluate the effectiveness of various validation methods for space station applications. The second task is to recommend requirements for KBS V&V for Space Station Freedom (SSF). The third task is to recommend modifications to the SSF to support the development of KBS using effectiveness software engineering and validation techniques. To accomplish the first task, three complementary techniques will be evaluated: (1) Sensitivity Analysis (Worchester Polytechnic Institute); (2) Formal Verification of Safety Properties (SRI International); and (3) Consistency and Completeness Checking (Lockheed AI Center). During FY89 and FY90, each contractor will independently demonstrate the user of his technique on the fault detection, isolation, and reconfiguration (FDIR) KBS or the manned maneuvering unit (MMU), a rule-based system implemented in LISP. During FY91, the application of each of the techniques to other knowledge representations and KBS architectures will be addressed. After evaluation of the results of the first task and examination of Space Station Freedom V&V requirements for conventional software, a comprehensive KBS V&V methodology will be developed and documented. Development of highly reliable KBS's cannot be accomplished without effective software engineering methods. Using the results of current in-house research to develop and assess software engineering methods for KBS's as well as assessment of techniques being developed elsewhere, an effective software engineering methodology for space station KBS's will be developed, and modification of the SSF to support these tools and methods will be addressed.

Johnson, Sally C.↗

Verification and validation of rulebased systems for Hubble Space Telescope ground support

As rulebase systems become more widely used in operational environments, the focus is on the problems and concerns of maintaining expert systems. In the conventional software model, the verification and validation of a system have two separate and distinct meanings. To validate a system means to demonstrate that the system does what is advertised. The verification process refers to investigating the actual code to identify inconsistencies and redundancies within the logic path. In current literature regarding maintaining rulebased systems, little distinction is made between these two terms. In fact, often the two terms are used interchangeably. Verification and validation of rulebased systems are discussed as separate but equally important aspects of the maintenance phase. Also described are some of the tools and methods that were developed at the Space Telescope Science Institute to aid in the maintenance of the rulebased system.

Vick, Shon↗

Verification and Validation of Digitally Upgraded Control Rooms

As nuclear power plants undertake main control room modernization, a challenge is the lack of a clearly defined human factors process to follow. Verification and validation (V&V) as applied in the nuclear power community has tended to involve efforts such as integrated system validation, which comes at the tail end of the design stage. To fill in guidance gaps and create a step-by-step process for control room modernization, we have developed the Guideline for Operational Nuclear Usability and Knowledge Elicitation (GONUKE). This approach builds on best practices in the software industry, which prescribe an iterative user-centered approach featuring multiple cycles of design and evaluation. Nuclear regulatory guidance for control room design emphasizes summative evaluation—which occurs after the design is complete. In the GONUKE approach, evaluation is also performed at the formative stage of design—early in the design cycle using mockups and prototypes for evaluation. The evaluation may involve expert review (e.g., software heuristic evaluation at the formative stage and design verification against human factors standards like NUREG-0700 at the summative stage). The evaluation may also involve user testing (e.g., usability testing at the formative stage and integrated system validation at the summative stage). An additional, often overlooked component of evaluation is knowledge elicitation, which captures operator insights into the system. In this report we outline these evaluation types across design phases that support the overall modernization process. The objective is to provide industry-suitable guidance for steps to be taken in support of the design and evaluation of a new human-machine interface (HMI) in the control room. We suggest the value of early-stage V&V and highlight how this early-stage V&V can help improve the design process for control room modernization. We argue that there is a need to overcome two shortcomings of V&V in current practice—the propensity for late-stage V&V and the use of increasingly complex psychological assessment measures for V&V.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Validation and verification of deep-space missions

The growing technical complexity of and increased cost pressure on deep-space science missions poses challenges for the system engineering discipline of mission validation and verification. In the wake of several recent mission failures, the aerospace community is still searching for a stable middle ground between the highly-reliable yet cost-prohibitive approaches of the past and cheaper but excessively risky methods of project implementation.

system engineering validation verification↗

Verification and Validation of Carbon-Fiber Laminate Low Velocity Impact Simulations.

Presented is a model verification and validation effort using low - velocity impact (LVI) of carbon fiber reinforced polymer laminate experiments. A flat cylindrical indenter impacts the laminate with enough energy to produce delamination, matrix cracks and fiber breaks. Included in the experimental efforts are ultrasonic scans of the damage for qualitative validation of the models. However, the primary quantitative metrics of validation are the force time history measured through the instrumented indenter and initial and final velocities. The simulations, whi ch are run on Sandia's Sierra finite element codes , consist of all physics and material parameters of importance as determined by a sensitivity analysis conducted on the LVI simulation. A novel orthotropic damage and failure constitutive model that is cap able of predicting progressive composite damage and failure is described in detail and material properties are measured, estimated from micromechanics or optimized through calibration. A thorough verification and calibration to the accompanying experiment s are presented. Specia l emphasis is given to the four - point bend experiment. For all simulations of interest, the mesh and material behavior is verified through extensive convergence studies. An ensemble of simulations incorporating model parameter unc ertainties is used to predict a response distribution which is then compared to experimental output. The result is a quantifiable confidence in material characterization and model physics when simulating this phenomenon in structures of interest.

English, Shawn Allen↗

MPACT Verification and Validation Manual (V.4.3)

As the VERA SQA plan requires, it is the responsibility of the University of Michigan (UM) and Oak Ridge National Laboratory (ORNL), as co-owners of MPACT, to ensure that verification and validation activities are performed and documented in a V&V manual with supporting publications and CASL technical reports which can be provided for reference and distribution within VERA. This document provides the current revision of the MPACT verification and validation (V&V) manual and describes the current state of MPACT V&V and updates the plans for future MPACT V&V activities. The following sections provide an overview of the V&V process used in MPACT, as well as a summary of the status of each component of V&V in the code.

97 MATHEMATICS AND COMPUTING↗

Verification and Validation in NASA-CUIP LOX/methane Injector Research Efforts

The NASA-CUIP program is supporting experimental and modeling/computational research efforts investigating wall heat flux characteristics of LOX/methane single-element injectors. A Verification and Validation (V&V) task has begun with the objective of quantifying the degree of accuracy of the models for wall heat flux distributions at specific sets of conditions. The V&V approach used is that being drafted as a standard by the ASME Performance Test Codes Committee, PTC 61: Verification and Validation in Computational Fluid Dynamics and Heat Transfer. The approach is based on well-established concepts from experimental uncertainty analysis. Initially the V&V and uncertainty estimation efforts will use data obtained previously in the same facility using oxygen/hydrogen in single-element injector testing. In this paper the V&V process, experimental facility, and computer code are described, and approaches to estimating the associated uncertainties are discussed.

Coleman, Hugh W.↗

Methodologies for Verification and Validation of Space Launch System (SLS) Structural Dynamic Models

Responses to challenges associated with verification and validation (V&V) of Space Launch System (SLS) structural dynamics models are presented in this paper. Four methodologies addressing specific requirements for V&V are discussed. (1) Residual Mode Augmentation (RMA), which has gained acceptance by various principals in the NASA community, defines efficient and accurate FEM modal sensitivity models that are useful in test-analysis correlation and reconciliation and parametric uncertainty studies. (2) Modified Guyan Reduction (MGR) and Harmonic Reduction (HR, introduced in 1976), developed to remedy difficulties encountered with the widely used Classical Guyan Reduction (CGR) method, are presented. MGR and HR are particularly relevant for estimation of "body dominant" target modes of shell-type SLS assemblies that have numerous "body", "breathing" and local component constituents. Realities associated with configuration features and "imperfections" cause "body" and "breathing" mode characteristics to mix resulting in a lack of clarity in the understanding and correlation of FEM- and test-derived modal data. (3) Mode Consolidation (MC) is a newly introduced procedure designed to effectively "de-feature" FEM and experimental modes of detailed structural shell assemblies for unambiguous estimation of "body" dominant target modes. Finally, (4) Experimental Mode Verification (EMV) is a procedure that addresses ambiguities associated with experimental modal analysis of complex structural systems. Specifically, EMV directly separates well-defined modal data from spurious and poorly excited modal data employing newly introduced graphical and coherence metrics.

Coppolino, Robert N.↗

MPACT Verification and Validation Manual Version 4.4

As the VERA SQA plan requires, it is the responsibility of the University of Michigan (UM) and Oak Ridge National Laboratory (ORNL), as co-owners of MPACT, to ensure that verification and validation activities are performed and documented in a V&V manual with supporting publications and CASL technical reports which can be provided for reference and distribution within VERA. This document provides the current revision of the MPACT verification and validation (V&V) manual and describes the current state of MPACT V&V and updates the plans for future MPACT V&V activities. The following sections provide an overview of the V&V process used in MPACT, as well as a summary of the status of each component of V&V in the code.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Software Verification and Validation Plan Activities, 2011, Project Number: N6423, SAPHIRE Version 8

The SV&V Plan experienced changes over the past year to bring it into the operational software life cycle of SAPHIRE 8 and to maintain its sections on design features. Peer review of the SVVP with the former IV&V members identified the need for the operational use of metrics as a tool for quality maintenance and improvement. New tests were added to the SVVP to verify the operation of the new design features incorporated into SAPHIRE 8. Other additions to the SVVP were the addition of software metrics and the PDR and CDR processes. Audit support was provided for the NRC Technical Manager and Project Manager for the NRC OIG Audit performed throughout 2011. The SVVP is considered to be an up to date reference and useful roadmap of verification and validation activities going forward.

42 ENGINEERING↗

Why do verification and validation?

In this discussion paper, we explore different ways to assess the value of verification and validation (V&V) of engineering models. We first present a literature review on the value of V&V and then use value chains and decision trees to show how value can be assessed from a decision maker's perspective. In this context, the value is what the decision maker is willing to pay for V&V analysis with the understanding that the V&V results are uncertain. As a result, the 2014 Sandia V&V Challenge Workshop is used to illustrate these ideas.

42 ENGINEERING↗