Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification & validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Package Testing Program Process for Validation and Verification of Software

The purpose of this document is to define the process for validation and verification (V&V) of software used to test the thermal data acquisition equipment in the Oak Ridge National Laboratory (ORNL) Package Testing Program (PTP) as part of the testing configuration(s). This process adheres to all applicable ORNL Standards Based Management System requirements and correlates with the PTP Quality Management System and the Software Quality Assurance Plan.

97 MATHEMATICS AND COMPUTING↗

Applying Independent Verification and Validation to Automatic Test Equipment

This paper describes a general overview of applying Independent Verification and Validation (IV&V) to Automatic Test Equipment (ATE). The overview is not inclusive of all IV&V activities that can occur or of all development and maintenance items that can be validated and verified, during the IV&V process. A sampling of possible IV&V activities that can occur within each phase of the ATE life cycle are described.

Calhoun, Cynthia C.↗

Independent Verification and Validation (IV and V) - Adding Mission Assurance to NASA Flight Software

The NASA Independent Verification and Validation (IV&V) Facility objective is to identify potential defects in flight software using independent analysis techniques. This paper describes the tailored IV&V techniques that have been developed in support of critical interactions on the Mars Science Laboratory (MSL) project, scheduled to launch in November, 2011. The IV&V techniques for interface analysis use independently developed sequence diagrams of critical scenarios. The results from these analyses have had a positive impact on the requirements flow down, consistency amongst MSL requirements and identification of missing requirements. The results of these analyses and the positive impact to the MSL project are provided.

performance evaluation↗

Verification and Validation of EnergyPlus Phase Change Material Model for Opaque Wall Assemblies

Phase change materials (PCMs) represent a technology that may reduce peak loads and HVAC energy consumption in buildings. A few building energy simulation programs have the capability to simulate PCMs, but their accuracy has not been completely tested. This study shows the procedure used to verify and validate the PCM model in EnergyPlus using a similar approach as dictated by ASHRAE Standard 140, which consists of analytical verification, comparative testing, and empirical validation. This process was valuable, as two bugs were identified and fixed in the PCM model, and version 7.1 of EnergyPlus will have a validated PCM model. Preliminary results using whole-building energy analysis show that careful analysis should be done when designing PCMs in homes, as their thermal performance depends on several variables such as PCM properties and location in the building envelope.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

EMPHASIS™/Nevada Unstructured FEM Implementaiton Version 2.1.3. Description, verification, and validation

EMPHASIS™/NEVADA is the SIERRA/NEVADA toolkit implementation of portions of the EMPHASIS TM code suite. The purpose of the toolkit implementation is to facilitate coupling to other physics drivers such as radiation transport as well as to better manage code design, implementation, complexity, and important verification and validation processes. This document describes the theory and implementation of the unstructured finite- element method solver, associated algorithms, and selected verification and validation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Consistency, Verification, and Validation of Turbulence Models for Reynolds-Averaged Navier-Stokes Applications

In current practice, it is often difficult to draw firm conclusions about turbulence model accuracy when performing multi-code CFD studies ostensibly using the same model because of inconsistencies in model formulation or implementation in different codes. This paper describes an effort to improve the consistency, verification, and validation of turbulence models within the aerospace community through a website database of verification and validation cases. Some of the variants of two widely-used turbulence models are described, and two independent computer codes (one structured and one unstructured) are used in conjunction with two specific versions of these models to demonstrate consistency with grid refinement for several representative problems. Naming conventions, implementation consistency, and thorough grid resolution studies are key factors necessary for success.

Rumsey, Christopher L.↗

Verification and validation of the maximum entropy method for reconstructing neutron flux, with MCNP5, Attila-7.1.0 and the GODIVA experiment

Verification and validation of reconstructed neutron flux based on the maximum entropy method is presented in this paper. The verification is carried out by comparing the neutron flux spectrum from the maximum entropy method with Monte Carlo N Particle 5 version 1.40 (MCNP5) and Attila-7.1.0-beta (Attila). A spherical 100% 235U critical assembly is modeled as the test case to compare the three methods. The verification error range for the maximum entropy method is 15–21% where MCNP5 is taken to be the comparison standard. Attila relative error for the critical assembly is 20–35%. Validation is accomplished by comparing a neutron flux spectrum that is back calculated from foil activation measurements performed in the GODIVA experiment (GODIVA). The error range of the reconstructed flux compared to GODIVA is 0–10%. The error range of the neutron flux spectrum from MCNP5 compared to GODIVA is 0–20% and the Attila error range compared to the GODIVA is 0–35%. The maximum entropy method is shown to be a fast reliable method, compared to either Monte Carlo methods (MCNP5) or 30 multienergy group methods (Attila) and with respect to the GODIVA experiment.

99 GENERAL AND MISCELLANEOUS↗

Verification and Validation of Two Hydrodynamic Methods for Simulations of High Energy Density Physics Problems

A 3D verification and validation suite of test problems is presented and used to evaluate hydrodynamic methods within a radiation hydrodynamics code, xRAGE. These test problems exercise different levels of complexity, building towards ICF problems which in addition to hydrodynamics also include three temperature plasma physics, thermal conduction, and radiation diffusion. Among the problems in the test suite are the Kidder ball problem, the Verney shell problem, and a 5-material compression problem, which exercise different purely hydrodynamic methods implemented within xRAGE. There is excellent agreement between 2D and 3D XRAGE simulation results and between the xRAGE results and the benchmark solutions. Two 3D ICF test problems are also presented, based on an OMEGA direct drive capsule experiment and on a NIF indirect drive capsule experiment. It is demonstrated that the newer unsplit hydrodynamic method in xRAGE produces more vorticity relative to the older default method. For the indirect drive capsule, the 3D simulations are in reasonable agreement with the experimental values of ion temperature and neutron production.

47 OTHER INSTRUMENTATION↗

Expanded verification and validation studies of hypersonic aerodynamics with multiple physics-fidelity models

Hypersonic aerothermodynamics is an important domain of modern multiphysics simulation. The Multi-Fidelity Toolkit is a simulation tool being developed at Sandia National Laboratories to predict aerodynamic properties for compressible flows from a range of physics fidelities and computational speeds. These models include the Reynolds-averaged Navier–Stokes (RANS) equations, the Euler equations with momentum-energy integral technique (MEIT), and modified Newtonian aerodynamics with flat-plate boundary layer (MNA+FPBL) equations, and they can be invoked independently or coupled with hierarchical Kriging to interpolate between high-fidelity simulations using lower-fidelity data. However, as with any new simulation capability, verification and validation are necessary to gather credibility evidence. This work describes formal code- and solution-verification activities, as well as model validation with uncertainty considerations. Code verification activities on the MNA+FPBL model build on previous work by focusing on the viscous portion of the model. Viscous quantities of interest are compared against those from an analytical solution for flat-plate, inclined-plate, and cone geometries. The code verification methodology for the MEIT model is also presented. Test setup and results of code verification tests on the laminar and turbulent models within MEIT are shown. Solution-verification activities include grid-refinement studies on simulations that model the HIFiRE-1 wind tunnel experiments. These experiments are used for validation of all model fidelities. A thorough validation comparison with prediction error and uncertainty is also presented. Three additional HIFiRE-1 experimental runs are simulated in this study, and the solution verification and validation work examines the effects of the associated parameter changes on model performance. Finally, a study is presented that compares the computational costs and fidelities from each of the different models.

42 ENGINEERING↗

Verification and Validation Tests of the Gamma Library of the ARC Software Package (Rev.1)

As part of the verification and validation (V&V) efforts of the Argonne Reactor Code (ARC) software system to support the Versatile Test Reactor (VTR) project of the U.S. Department of Energy (DOE), work has been performed to verify and validate the gamma (or photon) library of the multigroup cross section generation code MC 2 -3 of the ARC system, which is the essential part of the coupled neutron and gamma heating calculations to determine the heat generation rate or power distribution in the core. This V&V study was carried out in three steps: 1) review of the procedures and utility programs for generating the MC 2 -3 gamma library, 2) verification tests of cross sections and KERMA factors by comparing the total heat generation rate of coupled neutron and gamma calculation with that obtained with reaction Q values under the assumption of local gamma energy deposition and by comparing the prompt heat generation rate with that calculated with the MCNP6.2 code, and 3) validation tests of cross sections and KERMA factors by analyzing the ZPPR-15D gamma dose data measured with thermoluminescent dosimeters (TLD).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗

MPACT Verification and Validation Manual (rev 1)

The MPACT code provides the capability to calculate the steady-state and transient reactor core neutron distribution for operating nuclear power plant (PWR) conditions across multiple fuel cycles. As required in the CASL SQA plan, it is the responsibility of UM and ORNL as joint owners of MPACT to ensure that verification and validation activities are performed and documented in a V&V manual with supporting publications and CASL technical reports which can be distributed for reference and distribution within VERA. This document provides the first revision of the MPACT verification and validation (V&V) manual and describes the current state of MPACT V&V and updates the framework for future MPACT V&V activities. During FY16, MPACT V&V activities in the areas of code verification have continued to mature and work has progressed on code error evaluation using the Method of Manufactured Solutions (MMS). Several specific tasks have also been completed to improve MPACT code verification to include tasks to improve the unit and regression test coverage. A joint UM/ORNL task was completed to develop standardize coding standards and workflow for the continued collaborative development of MPACT. During the past year, MPACT validation work has continued in both the areas of measured data from critical experiments as well as measured data from operating nuclear power plants. Preliminary work was also performed in FY16 on the use of post-irradiation examinations for evaluation and validation of the isotopic depletion capability in MPACT. Based on the results of V&V efforts this past year, the confidence level has increased in the ability of MPACT to model an operational Pressurized Water Reactor. Areas were identified which require increased emphasis during FY17 to include updating all the V&V problems with the new 51-group library, the completion of cycle 2 of the BEAVRS benchmark, initial validation for BWR applications, the addition of Uncertainty Quantification results into the benchmark results, and the completion of specific validation tasks for the CASL Challenge problems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Systematic approach to verification and validation: High explosive burn models

Most material models used in numerical simulations are based on heuristics and empirically calibrated to experimental data. For a specific model, key questions are determining its domain of applicability and assessing its relative merits compared to other models. Answering these questions should be a part of model verification and validation (V and V). Here, we focus on V and V of high explosive models. Typically, model developers implemented their model in their own hydro code and use different sets of experiments to calibrate model parameters. Rarely can one find in the literature simulation results for different models of the same experiment. Consequently, it is difficult to assess objectively the relative merits of different models. This situation results in part from the fact that experimental data is scattered through the literature (articles in journals and conference proceedings) and that the printed literature does not allow the reader to obtain data from a figure in electronic form needed to make detailed comparisons among experiments and simulations. In addition, it is very time consuming to set up and run simulations to compare different models over sufficiently many experiments to cover the range of phenomena of interest. The first difficulty could be overcome if the research community were to support an online web based database. The second difficulty can be greatly reduced by automating procedures to set up and run simulations of similar types of experiments. Moreover, automated testing would be greatly facilitated if the data files obtained from a database were in a standard format that contained key experimental parameters as meta-data in a header to the data file. To illustrate our approach to V and V, we have developed a high explosive database (HED) at LANL. It now contains a large number of shock initiation experiments. Utilizing the header information in a data file from HED, we have written scripts to generate an input file for a hydro code, run a simulation, and generate a comparison plot showing simulated and experimental velocity gauge data. These scripts are then applied to several series of experiments and to several HE burn models. The same systematic approach is applicable to other types of material models; for example, equations of state models and material strength models.

36 MATERIALS SCIENCE↗

A practical experience with independent verification and validation

One approach to reducing software cost and increasing reliability is the use of an independent verification and validation (IV & V) methodology. The Software Engineering Laboratory (SEL) applied the IV & V methodology to two medium-size flight dynamics software development projects. Then, to measure the effectiveness of the IV & V approach, the SEL compared these two projects with two similar past projects, using measures like productivity, reliability, and maintain ablilty. Results indicated that the use of the IV & V methodology did not help the overall process nor improve the product in these cases.

Page, Gerald↗

Expert system verification and validation survey, delivery 4

The purpose is to determine the state-of-the-practice in Verification and Validation (V and V) of Expert Systems (ESs) on current NASA and Industry applications. This is the first task of a series which has the ultimate purpose of ensuring that adequate ES V and V tools and techniques are available for Space Station Knowledge Based Systems development. The strategy for determining the state-of-the-practice is to check how well each of the known ES V and V issues are being addressed and to what extent they have impacted the development of ESs.

Source record↗

Verification and Validation of the General Mission Analysis Tool (GMAT)

This paper describes the processes and results of Verification and Validation (V&V) efforts for the General Mission Analysis Tool (GMAT). We describe the test program and environments, the tools used for independent test data, and comparison results. The V&V effort produced approximately 13,000 test scripts that are run as part of the nightly buildtest process. In addition, we created approximately 3000 automated GUI tests that are run every two weeks. Presenting all test results are beyond the scope of a single paper. Here we present high-level test results in most areas, and detailed test results for key areas. The final product of the V&V effort presented in this paper was GMAT version R2013a, the first Gold release of the software with completely updated documentation and greatly improved quality. Release R2013a was the staging release for flight qualification performed at Goddard Space Flight Center (GSFC) ultimately resulting in GMAT version R2013b.

GMAT↗