Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verifications”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors

The document compiles in a single volume several verification and validation works done for the PLTEMP/ANL code during the years of its development and improvement. Some works that are available in the open literature are simply referenced at the outset, and are not included in the document. PLTEMP has been used in conversion safety analysis reports of several US and foreign research reactors that have been licensed and converted. A list of such reactors is given. Each chapter of the document deals with the verification or validation of a specific model. The model verification is usually done by comparing the code with hand calculation, Microsoft spreadsheet calculation, or Mathematica calculation. The model validation is done by comparing the code with experimental data or a more validated code like the RELAP5 code.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2017

Physics Verification Overview

The purpose of the verification project is to establish, through rigorous convergence analysis, that each ASC computational physics code correctly implements a set of physics models and algorithms (code verification); Evaluate and analyze the uncertainties of code outputs associated with the choice of temporal and spatial discretization (solution or calculation verification); and Develop and maintain the capability to expand and update these analyses on demand. This presentation describes project milestones.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

An Investigation into Solution Verification for CFD-DEM

This report presents the study of the convergence behavior of the computational fluid dynamicsdiscrete element method (CFD-DEM) method, specifically National Energy Technology Laboratory’s (NETL) open source MFiX code (MFiX-DEM) with a diffusion based particle-tocontinuum filtering scheme. In particular, this study focused on determining if the numerical method had a solution in the high-resolution limit where the grid size is smaller than the particle size. To address this uncertainty, fixed particle beds of two primary configurations were studied: i) fictitious beds where the particles are seeded with a random particle generator, and ii) instantaneous snapshots from a transient simulation of an experimentally relevant problem. Both problems considered a uniform inlet boundary and a pressure outflow. The CFD grid was refined from a few particle diameters down to 1/6 th of a particle diameter. The pressure drop between two vertical elevations, averaged across the bed cross-section was considered as the system response quantity of interest. A least-squares regression method was used to extrapolate the grid-dependent results to an approximate “grid-free” solution in the limit of infinite resolution. The results show that the diffusion based scheme does yield a converging solution. However, the convergence is more complicated than encountered in simpler, single-phase flow problems showing strong oscillations and, at times, oscillations superimposed on top of globally non-monotonic behavior. The challenging convergence behavior highlights the importance of using at least four grid resolutions in solution verification problems so that (over-determined) regression-based extrapolation methods may be applied to approximate the grid-free solution. The grid-free solution is very important in solution verification and VVUQ exercise in general as the difference between it and the reference solution largely determines the numerical uncertainty. By testing different randomized particle configurations of the same general problem (for the fictitious case) or different instances of freezing a transient simulation, the numerical uncertainties appeared to be on the same order of magnitude as ensemble or time averaging uncertainties. By testing different drag laws, almost all cases studied show that model form uncertainty in this one, very important closure relation was larger than the numerical uncertainty, at least with a reasonable CFD grid, roughly five particle diameters. In this study, the diffusion width (filtering length scale) was mostly set at a constant of six particle diameters. A few exploratory tests were performed to show that similar convergence behavior was observed for diffusion widths greater than approximately two particle diameters. However, this subject was not investigated in great detail because determining an appropriate filter size is really a validation question which must be determined by comparison to experimental or highly accurate numerical data. Future studies are being considered targeting solution verification of transient simulations as well as validation of the filter size with direct numerical simulation data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

GALE 3.1 Verification Report

This report is documentation of the verification activities that have been performed on GALE-3.1 to ensure that the codes meet the requirements and specifications set forth by NRC and that the code meets its intended purpose. Additional benchmark tests for GALE-3.1 are documented in Appendix A of NUREG-0016, Revision 2 for GALE-BWR 3.1 and Appendix A of NUREG-0017, Revision 2 for GALE-PWR 3.1. These activities are done in accordance with the GALE Software Quality Assurance Plan (PNNL-24249) and the GALE Configuration Management & Maintenance Plan (PNNL-24250). These two plans meet the requirements set for by both PNNL software quality assurance requirements and the NRC’s Software Quality Assurance Program and Guidelines (NUREG/BR-0167). The verification activities are documented to conform to Level 2 requirements of NUREG/BR-0167. Verifications were performed for both GALE-3.1 BWR and GALE-3.1 PWR Codes. The GALE 3.1 BWR and PWR codes are computerized mathematical models for calculating the releases of radioactive material in gaseous and liquid effluents. These computer codes are tools which can be used to determine conformance with the requirements of Appendix I to Title 10, Part 50 of the Code of Federal Regulations (10 CFR 50).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

GALE 3.2 Verification Report

This report is documentation of the verification activities that have been performed on GALE-3.2 to ensure that the codes meet the requirements and specifications set forth by NRC and that the code meets its intended purpose. These activities are done in accordance with the GALE Software Quality Assurance Plan (PNNL-24249) and the GALE Configuration Management & Maintenance Plan (PNNL-24250). These two plans meet the requirements set for by both PNNL software quality assurance requirements and the NRC’s Software Quality Assurance Program and Guidelines (NUREG/BR-0167). The verification activities are documented to conform to Level 2 requirements of NUREG/BR-0167. Verifications were performed for both GALE-3.2 BWR and GALE-3.2 PWR Codes. The GALE 3.2 BWR and PWR codes are computerized mathematical models for calculating the releases of radioactive material in gaseous and liquid effluents. These computer codes are tools which can be used to determine conformance with the requirements of Appendix I to Title 10, Part 50 of the Code of Federal Regulations (10 CFR 50).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Program Verification for Extreme-Scale Applications (Final Scientific/Technical Report)

The project seeks to develop tools and techniques to help software developers verify the correctness and accuracy of their code. The target domain is scientific software of the kind widely used and developed in the Department of Energy research community, with a particular focus on "extreme scale" programs - those that are expected to involve possibly millions of parallel threads of execution. The report covers the University of Delaware contribution to the collaborative project. The project had a number of successful outcomes, especially regarding the development and extension of the CIVL software verification framework. CIVL is a verification tool for C or Fortran programs that use MPI, OpenMP, CUDA, and/or Pthreads for parallelization. CIVL went through vast improvements and extensions, and was successfully applied to a number of challenging codes. It found a subtle bug in the Devito PDE framework. It was able to verify the functional correctness of a conjugate gradient solver using a novel probabilistic technique with vanishingly small chance of error. CIVL was used very successfully in verification competitions, and the CIVL solutions to the competition challenges were presented and published. CIVL was also used successfully by other researchers on a computational chemistry kernel.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Supplement to M&V Guidelines: Measurement and Verification for Performance-Based Contracts Version 4.0

Measurement and verification (M&V) are key to determining and confirming the continued operation of, and savings associated with, performance-based contracts. The Federal Energy Management Program (FEMP) updates these guidelines to reflect the evolution of M&V to incorporate industry best practices and ensure that savings from these performance-based projects are properly planned, communicated, and realized. FEMP released M&V Guidelines: Measurement and Verification for Federal Energy Projects Version 3.0 in 2008. This document was developed with input from industry-government working groups, previous M&V guideline authors, and national laboratory subject matter experts. The document provided the methods and guidelines, with a focus on federal energy savings performance contracts (ESPCs). When FEMP released M&V Guidelines: Measurement and Verification for Performance-Based Contracts Version 4.0 in 2015, FEMP expanded the intended audience to include anyone (energy managers, procurement officers, and contractors) involved in implementing performance-based contracts. The streamlined M&V Guidelines Version 4.0 provided the procedures and guidelines needed to quantify savings resulting from installing energy-efficient equipment, water conservation, improved operation and maintenance (O&M), renewable energy, and cogeneration projects with a performance-based contract; however, it removed some of the associated details on topics such as developing regression models and sampling plans. The details and information provided in M&V Guidelines v. 3.0 are still pertinent to M&V for performance-based contracts. The objective is to keep FEMP M&V version 4.0 condensed, providing the reference document for specifying M&V methods and procedures and a resource for developing project-specific M&V plans; this supplement provides additional details. The content of this supplement has been updated to include pertinent revisions between M&V version 3.0 and M&V version 4.0 by providing details and updating the content to ensure relevancy.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS2003

Trends in software verification

steady increase in computational power of general purpose computers, our ability to analyze routine software artifacts is also steadily increasing. As a result, we are witnessing a shift in emphasis from the verification of abstract hand-built models of code, towards the direct verification of implementation level code. This change in emphasis poses a new set of challenges in software verification. We explore some of them in this paper.

software verification↗
NASA NTRS2006

Validation (not just verification) of Deep Space Missions

ion & Validation (V&V) is a widely recognized and critical systems engineering function. However, the often used definition 'Verification proves the design is right; validation proves it is the right design' is rather vague. And while Verification is a reasonably well standardized systems engineering process, Validation is a far more abstract concept and the rigor and scope applied to it varies widely between organizations and individuals. This is reflected in the findings in recent Mishap Reports for several NASA missions, in which shortfalls in Validation (not just Verification) were cited as root- or contributing-factors in catastrophic mission loss. Furthermore, although there is strong agreement in the community that Test is the preferred method for V&V, many people equate 'V&V' with 'Test', such that Analysis and Modeling aren't given comparable attention. Another strong motivator is a realization that the rapid growth in complexity of deep-space missions (particularly Planetary Landers and Space Observatories given their inherent unknowns) is placing greater demands on systems engineers to 'get it right' with Validation.

verification↗
NASA NTRS2014

Toward Automatic Verification of Goal-Oriented Flow Simulations

We demonstrate the power of adaptive mesh refinement with adjoint-based error estimates in verification of simulations governed by the steady Euler equations. The flow equations are discretized using a finite volume scheme on a Cartesian mesh with cut cells at the wall boundaries. The discretization error in selected simulation outputs is estimated using the method of adjoint-weighted residuals. Practical aspects of the implementation are emphasized, particularly in the formulation of the refinement criterion and the mesh adaptation strategy. Following a thorough code verification example, we demonstrate simulation verification of two- and three-dimensional problems. These involve an airfoil performance database, a pressure signature of a body in supersonic flow and a launch abort with strong jet interactions. The results show reliable estimates and automatic control of discretization error in all simulations at an affordable computational cost. Moreover, the approach remains effective even when theoretical assumptions, e.g., steady-state and solution smoothness, are relaxed.

Simulations↗
NASA NTRS2017

Verification Testing: Meet User Needs Figure of Merit

Verification is the process through which Modeling and Simulation(M&S) software goes to ensure that it has been rigorously tested and debugged for its intended use. Validation confirms that said software accurately models and represents the real world system. Credibility gives an assessment of the development and testing effort that the software has gone through as well as how accurate and reliable test results are. Together, these three components form Verification, Validation, and Credibility(VV&C), the process by which all NASA modeling software is to be tested to ensure that it is ready for implementation. NASA created this process following the CAIB (Columbia Accident Investigation Board) report seeking to understand the reasons the Columbia space shuttle failed during reentry. The reports conclusion was that the accident was fully avoidable, however, among other issues, the necessary data to make an informed decision was not there and the result was complete loss of the shuttle and crew. In an effort to mitigate this problem, NASA put out their Standard for Models and Simulations, currently in version NASA-STD-7009A, in which they detailed their recommendations, requirements and rationale for the different components of VV&C. They did this with the intention that it would allow for people receiving MS software to clearly understand and have data from the past development effort. This in turn would allow the people who had not worked with the MS software before to move forward with greater confidence and efficiency in their work. This particular project looks to perform Verification on several MATLAB (Registered Trademark)(The MathWorks, Inc.) scripts that will be later implemented in a website interface. It seeks to take note and define the limits of operation, the units and significance, and the expected datatype and format of the inputs and outputs of each of the scripts. This is intended to prevent the code from attempting to make incorrect or impossible calculations. Additionally, this project will look at the coding generally and note inconsistencies, redundancies, and other aspects that may become problematic or slow down the codes run time. Certain scripts lacking in documentation also will be commented and cataloged.

validation↗
NASA NTRS2017

MESA: Message-Based System Analysis Using Runtime Verification

In this paper, we present a novel approach and framework for run-time verication of large, safety critical messaging systems. This work was motivated by verifying the System Wide Information Management (SWIM) project of the Federal Aviation Administration (FAA). SWIM provides live air traffic, site and weather data streams for the whole National Airspace System (NAS), which can easily amount to several hundred messages per second. Such safety critical systems cannot be instrumented, therefore, verification and monitoring has to happen using a nonintrusive approach, by connecting to a variety of network interfaces. Due to a large number of potential properties to check, the verification framework needs to support efficient formulation of properties with a suitable Domain Specific Language (DSL). Our approach is to utilize a distributed system that is geared towards connectivity and scalability and interface it at the message queue level to a powerful verification engine. We implemented our approach in the tool called MESA: Message-Based System Analysis, which leverages the open source projects RACE (Runtime for Airspace Concept Evaluation) and TraceContract. RACE is a platform for instantiating and running highly concurrent and distributed systems and enables connectivity to SWIM and scalability. TraceContract is a runtime verication tool that allows for checking traces against properties specified in a powerful DSL. We applied our approach to verify a SWIM service against several requirements.We found errors such as duplicate and out-of-order messages.

Message-based System↗
NASA NTRS2017

Thermal Protection for Mars Sample Return Earth Entry Vehicle: A Grand Challenge for Design Methodology and Reliability Verification

Mars Sample Return is our Grand Challenge for the coming decade. TPS (Thermal Protection System) nominal performance is not the key challenge. The main difficulty for designers is the need to verify unprecedented reliability for the entry system: current guidelines for prevention of backward contamination require that the probability of spores larger than 1 micron diameter escaping into the Earth environment be lower than 1 million for the entire system, and the allocation to TPS would be more stringent than that. For reference, the reliability allocation for Orion TPS is closer to 11000, and the demonstrated reliability for previous human Earth return systems was closer to 1100. Improving reliability by more than 3 orders of magnitude is a grand challenge indeed. The TPS community must embrace the possibility of new architectures that are focused on reliability above thermal performance and mass efficiency. MSR (Mars Sample Return) EEV (Earth Entry Vehicle) will be hit with MMOD (Micrometeoroid and Orbital Debris) prior to reentry. A chute-less aero-shell design which allows for self-righting shape was baselined in prior MSR studies, with the assumption that a passive system will maximize EEV robustness. Hence the aero-shell along with the TPS has to take ground impact and not break apart. System verification will require testing to establish ablative performance and thermal failure but also testing of damage from MMOD, and structural performance at ground impact. Mission requirements will demand analysis, testing and verification that are focused on establishing reliability of the design. In this proposed talk, we will focus on the grand challenge of MSR EEV TPS and the need for innovative approaches to address challenges in modeling, testing, manufacturing and verification.

Design and Verification for Reliability↗
NASA NTRSDate not supplied

Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions

Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).

Validation↗
DOE OSTI2013

Verification Challenges at Low Numbers

Many papers have dealt with the political difficulties and ramifications of deep nuclear arms reductions, and the issues of “Going to Zero”. Political issues include extended deterrence, conventional weapons, ballistic missile defense, and regional and geo-political security issues. At each step on the road to low numbers, the verification required to ensure compliance of all parties will increase significantly. Looking post New START, the next step will likely include warhead limits in the neighborhood of 1000 . Further reductions will include stepping stones at1000 warheads, 100’s of warheads, and then 10’s of warheads before final elimination could be considered of the last few remaining warheads and weapons. This paper will focus on these three threshold reduction levels, 1000, 100’s, 10’s. For each, the issues and challenges will be discussed, potential solutions will be identified, and the verification technologies and chain of custody measures that address these solutions will be surveyed. It is important to note that many of the issues that need to be addressed have no current solution. In these cases, the paper will explore new or novel technologies that could be applied. These technologies will draw from the research and development that is ongoing throughout the national laboratory complex, and will look at technologies utilized in other areas of industry for their application to arms control verification.

Nuclear Weapons↗
DOE OSTI2014

A Runtime Verification Framework for Control System Simulation

n a standard workflow for the validation of a control system, the control system is implemented as an extension to a simulator. Such simulators are complex software systems, and engineers may unknowingly violate constraints a simulator places on extensions. As such, errors may be introduced in the implementation of either the control system or the simulator leading to invalid simulation results. This paper presents a novel runtime verification approach for verifying control system implementations within simulators. The major contribution of the approach is the two-tier specification process. In the first tier, engineers model constraints using a domain-specific language tailored to modeling a controller’s response to changes in its input. The language is high-level and effectively hides the implementation details of the simulator, allowing engineers to specify design-level constraints independent of low-level simulator interfaces. In the second tier, simulator developers provide mapping rules for mapping design-level constraints to the implementation of the simulator. Using the rules, an automated tool transforms the design-level specifications into simulator-specific runtime verification specifications and generates monitoring code which is injected into the implementation of the simulator. During simulation, these monitors observe the input and output variables of the control system and report changes to the verifier. The verifier checks whether these changes follow the constraints of the control system. We describe application of this approach to the verification of the constraints of an HVAC control system implemented with the power grid simulator GridLAB-D.

control system↗
DOE OSTI2014

NUMERICAL VERIFICATION OF THE RELAP-7 CORE CHANNEL SINGLE-PHASE MODEL

The RELAP-7 code is the next generation of nuclear reactor system safety analysis code being developed at the Idaho National Laboratory (INL). All the physics in RELAP-7 are fully coupled and the errors resulted from the traditional operator-splitting approach are eliminated. By using 2nd order methods in both time and space and eliminating operator-splitting errors, the numerical error of RELAP-7 can be minimized. Numerical verification is the process to verify the orders of numerical methods. It is an important part of modern verification and validation process. The core channel component in RELAP-7 is designed to simulate coolant flow as well as the conjugated heat transfer between coolant flow and the fuel rod. A special treatment at fuel centerline to avoid numerical singularity for the cylindrical heat conduction in the continuous finite element mesh is discussed. One steady state test case and one fast power up transient test case are utilized for the verification of the core channel model with single-phase flow. Analytical solution for the fuel pin temperature and figures of merit such as peak clad temperature and peak fuel temperature are used to define numerical errors. These cases prove that the mass and energy are well conserved and 2nd order convergence rates for both time and space are achieved in the core channel model.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2014

AUTOMATED, HIGHLY ACCURATE VERIFICATION OF RELAP5-3D

Computer programs that analyze light water reactor safety solve complex systems of governing, closure and special process equations to model the underlying physics. In addition, these programs incorporate many other features and are quite large. RELAP5-3D[1] has over 300,000 lines of coding for physics, input, output, data management, user-interaction, and post-processing. For software quality assurance, the code must be verified and validated before being released to users. Verification ensures that a program is built right by checking that it meets its design specifications. Recently, there has been an increased importance on the development of automated verification processes that compare coding against its documented algorithms and equations and compares its calculations against analytical solutions and the method of manufactured solutions[2]. For the first time, the ability exists to ensure that the data transfer operations associated with timestep advancement/repeating and writing/reading a solution to a file have no unintended consequences. To ensure that the code performs as intended over its extensive list of applications, an automated and highly accurate verification method has been modified and applied to RELAP5-3D. Furthermore, mathematical analysis of the adequacy of the checks used in the comparisons is provided.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗