Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Development, validation, and verification of multi-pass thermo-mechanical welding simulations using the open-source MOOSE framework: NeT TG4 benchmark weldment

This study develops and validates a sequentially coupled thermo-mechanical welding simulation for the three-pass 316L stainless steel NeT TG4 benchmark weldment using the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) and the Nuclear Engineering Material model Library (NEML). A diffused ellipsoidal heat source was calibrated against thermocouple data and weld macrographs to accurately model the fusion zone geometry and transient thermal fields. Material hardening is represented using the Lemaitre-Chaboche mixed isotropic-kinematic hardening model, while four annealing models - no annealing, single-stage at 1050 °C and 1300 °C, and two-stage at 800 °C/1300 °C - were implemented to assess the impact of annealing models on the accuracy of the predicted welding-induced plasticity, distortions, and residual stresses. The predictions were validated against experimental measurements and benchmarked against results from commercial software, demonstrating that thermo-mechanical MOOSE welding simulations achieve comparable accuracy with enhanced computational efficiency. This work highlights the potential of using open-source finite element frameworks like MOOSE for advanced manufacturing simulations.

Ji, Wendy [Australian Nuclear Science and Technolo↗

ECP SOLLVE: Validation and Verification Testsuite Status Update and Compiler Insight for OpenMP

The OpenMP language continues to evolve with every new specification release, as does the need to validate and verify the new features that have been implemented by the different vendors. With the release of OpenMP 5.0 and OpenMP 5.1, new target offload and host-based features have been introduced to the programming model. While OpenMP continues to grow in maturity, there is an observable growth in the number of compiler and hardware vendors that support OpenMP. In this manuscript, the main focus is on evaluating the conformity and OpenMP implementation progress of various compiler vendors such as Cray, IBM, GNU, Clang/LLVM, NVIDIA, and Intel. More specifically, the 4.5, 5.0, and 5.1 versions of the OpenMP specification are analyzed. For our experimental setup, the Crusher and Summit computing systems hosted by Oak Ridge National Lab’s Computing Facilities are utilized. The effort of vendor agnostic analysis of these implementations is especially valuable for application developers who are using new OpenMP features to accelerate their scientific codes. Insights are presented into the current implementation status of various vendors, the progression of specific compiler’s support for OpenMP overtime, the subset of OpenMP 4.5, 5.0, and 5.1 that is supported by all compilers, and examples of how our test suite has influenced discussion regarding the correct interpretation of the OpenMP specification. By evaluating OpenMP conformity of pre-Exascale computing systems, the aim is to detail progress and status of AMD + Cray ecosystem before the system and their OpenMP implementation is used for mission critical applications when the first Exascale Computer Frontier is made available to applications.

Huber, Thomas↗

Formal verification and validation of run-to-completion style state charts using Event-B

State chart notations with ‘run to completion’ semantics are popular with engineers for designing controllers that react to environment events with a sequence of state transitions but lack formal refinement and rigorous verification methods. State chart models are typically used to design complex control systems that respond to environmental triggers with a sequential process. The model is usually constructed at a concrete level and verified and validated using animation techniques relying on human judgement. Event-B, on the other hand, is based on refinement from an initial abstraction and is designed to make formal verification by automatic theorem provers feasible. Abstraction and formal verification provide greater assurance that critical (e.g. safety or security) properties are not violated by the control system. In this paper, we introduce a notion of refinement into a ‘run to completion’ state chart modelling notation and leverage Event-B’s tool support for theorem proving. We describe the difficulties in translating ‘run to completion’ semantics into Event-B refinements and suggest a solution. We illustrate our approach and show how models can be validated at different refinement levels using our scenario checker animation tools. We show how critical invariant properties can be verified by proof despite the reactive nature of the system and how behavioural aspects of the system can be verified by testing the expected reactions using a temporal logic, model checking approach. To verify liveness, we outline a proof that the run to completion is deadlock-free and converges to complete the run.

97 MATHEMATICS AND COMPUTING↗

Europa Clipper Magnetometer Thermal Design Evolution, Validation, and Verification

Europa Clipper Spacecraft is exposed to a high radiation environment and is limited with power. This makes the thermal design of Europa Clipper magnetometer more challenging. The design is required to be highly insulated and at the same time magnetically clean. This paper describes the thermal design evolution of Europa Clipper Magnetometer. A thermal vacuum test is conducted in Europa Clipper orbit’s relevant environment to characterize the requirements and estimate the power consumption. A flight-like engineering model magnetometer from the University of California at Los Angeles (UCLA) is used for this test. Magnetically clean Cernox temperature measurement sensors with the accuracy of ±0.1ºC are used to measure the temperature value inside the sensor head and across the main frame interfaces. This level of accuracy in temperature measurement is required to capture the temperature gradient across the sensor head in a relevant environment which impacts the magnetometer measurement accuracy. Test data is used to correlate the thermal model and characterize the heat leak across the interfaces. Validated thermal model is used as a verification tool for power consumption and thermal requirements.

Caron, Ryan↗

Verification and Validation Data for Computational Unsteady Aerodynamics

Computational Unsteady Aerodynamics computer codes are being increasingly used. In order to validate their results they must be tested against valid experimental data. The present report aims at collecting reliable experimental data on unsteady aerodynamics and presenting them in a form which permits use for verification of codes. For ease of handling, the data are also presented in machine readable form (CD-ROM). Data on increasingly complex generic forms were selected and the following categories are covered: flutter, buffet, stability and control, dynamic stall, cavity flows, store separation. Computational solutions are included in order to permit evaluation of codes and analysis of solutions which differ from experimental data.

Applied Vehicle Technology Panel (AVT)Task Group↗

Verification and validation of the Alternative Nonlinear Two-phase Subchannel (ANTS) code

The Alternative Nonlinear Two-phase Subchannel solver (ANTS) code was written to provide a fast-running, steady-state, pin-resolved modeling and simulation tool for analysis of common boiling water reactor (BWR) geometry and common operating conditions. ANTS has been integrated into the Virtual Environment for Reactor Application (VERA) core simulator software, where it can be used to provide a thermal/hydraulic (T/H) subchannel solution that is then used to provide neutronic feedback as well as perform the fuel depletion and temperature solution. Herein, this paper presents the rigorous analysis performed on the ANTS code, which includes both code and solution verification testing, benchmarking with the existing two-phase subchannel capability in VERA, CTF, and validation testing using popular two-phase experiments such as PWR Sub-channel and Bundle Tests (PSBT), BWR Full-size Fine-mesh Bundle Tests (BFBT), Risø, and FRIGG. This assessment was used to qualify ANTS for its intended applications before its use for core-scale, multiphysics BWR simulations. In general, it was found that agreement with experimental data was good; errors were within the range of experimental data uncertainty. Furthermore, code and solution verification confirmed that the governing equations and the most important closure terms were correctly implemented and behaving as expected.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Verification and Validation of High Explosive Reactive Burn Models Implemented in LANL's EAP and LAP Code Base (FY2020 L2 Milestone MRT 7129) [Slides]

Advanced reactive burn models enable phenomena to be modeled that aren’t possible with programmed burn. Programmed burn propagates a burn front at a prescribed speed from a prescribed initiation point. Reactive burn uses the hydro variables (pressure, temperature, etc.) to determine when, how rapidly, and to what extent a zone detonates. This enables a completely new set of problems to be simulated. Our goal in this milestone has been to improve the usability of reactive burn modes. Thus, the closure criteria are: 1. Perform code and solution verification of the reactive burn models using appropriate solutions and simplified HE setups. 2. Perform validation assessments using small-scale experiments. 3. Evaluate sensitivities and quantify uncertainties due to model form variations and mesh dependencies.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

(Doublon) Benchmarking of Different Inverse Point Kinetics Implementations for an Autocorrected Reactimeter Algorithm

In November 2017, the Transient Reactor Test Facility returned to operation. Since that time, many transient test series have been completed, such as the Transient Heatsink Overpower Response capsule (THOR), the Transient Water Irradiation System for TREAT (TWIST), and Sirius. Each has provided valuable data for materials performance and reactor safety that can be applied in future designs. During each experimental series, detector count rates provided important information on the core behavior during transients. However, a limitation of these data is that variations in the neutron distribution during experiments can cause errors when attempting to infer reactivity evolution from detector signals. Neutron physics codes can be used to compute the flux shape variations. However, this is a poor solution when the experimental data is used for code verification, validation and uncertainty quantification. Indeed, if the output of the code is used both as a reference and to correct what the reference is compared to, the circular dependency limits the quality of the verification, validation and uncertainty quantification approach. To overcome this problem, the autocorrected reactimeter algorithm (ACRA) has been developed. This approach infers a time-dependent reactivity evolution by testing different spatial corrections and selecting the one that minimizes reactivity variations when the core is in a frozen configuration (i.e., when there is no variation in parameters affecting reactivity). However, the scope of this method was limited to transients where there were negligible thermal feedback. Indeed, the core is never in a frozen configuration when the fuel temperature varies during the whole transient. This is our motivation for developing an improved version of the ACRA that does not require frozen configurations. To develop this new algorithm, we need a precise and unbiased implementation of the inverse point kinetic equations (IPKEs) as any error in the reactivity evaluation will be propagated into the choice of the optimal spatial correction. Indeed, the previous reactimeter algorithm would use approximations, such as a negligible flux amplitude derivative, to focus on rapidity. For the numerical validation of ACRA, we aim at absolute error under for reactivity derived from signals similar to the one of this study. In this summary, we test eight different IPKE implementations. Each will process a mockup signal built for this study, similar to those that the future ACRA will process. Each reactivity output will be compared to the reference reactivity that has been used to generate the mockup signal. The implementation minimizing the difference with the reference reactivity will be used in the development of a new ACRA formulation.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Development, Verification and Validation of Parallel, Scalable Volume of Fluid CFD Program for Propulsion Applications

There are many instances involving liquid/gas interfaces and their dynamics in the design of liquid engine powered rockets such as the Space Launch System (SLS). Some examples of these applications are: Propellant tank draining and slosh, subcritical condition injector analysis for gas generators, preburners and thrust chambers, water deluge mitigation for launch induced environments and even solid rocket motor liquid slag dynamics. Commercially available CFD programs simulating gas/liquid interfaces using the Volume of Fluid approach are currently limited in their parallel scalability. In 2010 for instance, an internal NASA/MSFC review of three commercial tools revealed that parallel scalability was seriously compromised at 8 cpus and no additional speedup was possible after 32 cpus. Other non-interface CFD applications at the time were demonstrating useful parallel scalability up to 4,096 processors or more. Based on this review, NASA/MSFC initiated an effort to implement a Volume of Fluid implementation within the unstructured mesh, pressure-based algorithm CFD program, Loci-STREAM. After verification was achieved by comparing results to the commercial CFD program CFD-Ace+, and validation by direct comparison with data, Loci-STREAM-VoF is now the production CFD tool for propellant slosh force and slosh damping rate simulations at NASA/MSFC. On these applications, good parallel scalability has been demonstrated for problems sizes of tens of millions of cells and thousands of cpu cores. Ongoing efforts are focused on the application of Loci-STREAM-VoF to predict the transient flow patterns of water on the SLS Mobile Launch Platform in order to support the phasing of water for launch environment mitigation so that vehicle determinantal effects are not realized.

West, Jeff↗

Investigation of a Verification and Validation Tool with a Turbofan Aircraft Engine Application

The development of more advanced control architectures for turbofan aircraft engines can yield gains in performance and efficiency over the lifetime of an engine. However, the implementation of these increasingly complex controllers is contingent on their ability to provide safe, reliable engine operation. Therefore, having the means to verify the safety of new control algorithms is crucial. As a step towards this goal, CoCoSim, a publicly available verification tool for Simulink, is used to analyze C-MAPSS40k, a 40,000 lbf class turbo-fan engine model developed at NASA for testing new control algorithms. Due to current limitations of the verification software, several modifications are made to C-MAPSS40k to achieve compatibility with CoCoSim. Some of these modifications sacrifice fidelity to the original model. Several safety and performance requirements typical for turbofan engines are identified and constructed into a verification framework. Preliminary results using an industry standard baseline controller for these requirements are presented. While verification capabilities are demonstrated, a truly comprehensive analysis will require further development of the verification tool.

V&↗

Zero power experiment for molten salt reactor - code verification and validation request

Molten Salt Reactors (MSRs) are one of the six perspective Generation IV nuclear technologies that should fulfil sustainability, economics, safety, and proliferation resistance goals. In recent years these concepts have attracted increased interest in research as well as in industry. In the context of developing new, innovative technology, a zero-power experiment would be an ideal option for improving the understanding of MSR systems under low-risk and low-cost conditions. The given research was focused on cross-verification of the nuclear codes for MSR modelling and simulations as required for the development of the first design of a potential facility and a draft of an experimental programme. The analysis included the evaluation of lattice and Monte Carlo codes. The study showed that all considered nuclear tools were in good agreement with each other. Although this does not mean the results will match the real experimental data, which is essential for the regulatory approval of a novel technology. Currently, experimental data is almost absent for MSR technology. Hence, a zero power experiment is needed to assure we can rely on modelling and simulation studies for further development of MSR technology and deliver real-world experiments to support the licensing of future MSRs. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Validation and Verification of Operational Land Analysis Activities at the Air Force Weather Agency

The NASA developed Land Information System (LIS) is the Air Force Weather Agency's (AFWA) operational Land Data Assimilation System (LDAS) combining real time precipitation observations and analyses, global forecast model data, vegetation, terrain, and soil parameters with the community Noah land surface model, along with other hydrology module options, to generate profile analyses of global soil moisture, soil temperature, and other important land surface characteristics. (1) A range of satellite data products and surface observations used to generate the land analysis products (2) Global, 1/4 deg spatial resolution (3) Model analysis generated at 3 hours. AFWA recognizes the importance of operational benchmarking and uncertainty characterization for land surface modeling and is developing standard methods, software, and metrics to verify and/or validate LIS output products. To facilitate this and other needs for land analysis activities at AFWA, the Model Evaluation Toolkit (MET) -- a joint product of the National Center for Atmospheric Research Developmental Testbed Center (NCAR DTC), AFWA, and the user community -- and the Land surface Verification Toolkit (LVT), developed at the Goddard Space Flight Center (GSFC), have been adapted to operational benchmarking needs of AFWA's land characterization activities.

Shaw, Michael↗

A Tool for Verification and Validation of Neural Network Based Adaptive Controllers for High Assurance Systems

High reliability of mission- and safety-critical software systems has been identified by NASA as a high-priority technology challenge. We present an approach for the performance analysis of a neural network (NN) in an advanced adaptive control system. This problem is important in the context of safety-critical applications that require certification, such as flight software in aircraft. We have developed a tool to measure the performance of the NN during operation by calculating a confidence interval (error bar) around the NN's output. Our tool can be used during pre-deployment verification as well as monitoring the network performance during operation. The tool has been implemented in Simulink and simulation results on a F-15 aircraft are presented.

Gupta, Pramod↗

Recommendations for Radiological Data Assessment Implementation

Since thorough verification, validation, and data quality assessment (DQA) processes may delay incident commanders and elected and appointed officials from making key decisions, the U.S. Department of Homeland Security (DHS) National Urban Security Technology Laboratory (NUSTL) tasked Pacific Northwest National Laboratory (PNNL) to develop tools and guidance for the federal, state, local, tribal, and territorial (FSLTT) responders using research, discussions with select FSLTT responders, and the experiences of subject matter experts in incident response. This report provides NUSTL with recommendations on best practices for verification, validation, and data quality assessment for the data collected by responders during a radiological or nuclear event. The purpose of this report is to inform the development of a DQA toolkit aimed at the needs of data assessors during the response to a radiological incident.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

CTF Validation and Verification: Version 4.3

Coolant-Boiling in Rod Arrays- Two Fluids (COBRA-TF) is a thermal/hydraulic (T/H) simulation code designed for light water reactor (LWR) analysis. It uses a two-fluid, three-field (i.e., fluid film, fluid drops, and vapor) modeling approach. Both subchannel and 3D Cartesian forms of nine conservation equations are available for LWR modeling. The code was originally developed by Pacific Northwest Laboratory in 1980 and has been used and modified by several institutions over the last several decades. COBRA-TF is also used at the Pennsylvania State University (PSU) by the Reactor Dynamics and Fuel Modeling Group (RDFMG) and has been improved, updated, and subsequently became the CTF. One part of the improvement process includes validating the methods in CTF. This document seeks to provide a certain level of certainty and confidence in the predictive capabilities of the code for the scenarios it was designed to model—rod bundle geometries with operating conditions that are representative of prototypical pressurized water reactor (PWR)s and boiling water reactor (BWR)s in both normal and accident conditions. This is done by modeling a variety of experiments that simulate these scenarios and then presenting a qualitative and quantitative analysis of the results that demonstrates the accuracy to which CTF is capable of capturing specific quantities of interest.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

CTF Validation and Verification: Version 4.4

Coolant-Boiling in Rod Arrays- Two Fluids (COBRA-TF) is a thermal/hydraulic (T/H) simulation code designed for light water reactor (LWR) analysis. It uses a two-fluid, three-field (i.e., fluid film, fluid drops, and vapor) modeling approach. Both subchannel and 3D Cartesian forms of nine conservation equations are available for LWR modeling. The code was originally developed by Pacific Northwest Laboratory in 1980 and has been used and modified by several institutions over the last several decades. COBRA-TF is also used at the Pennsylvania State University (PSU) by the Reactor Dynamics and Fuel Modeling Group (RDFMG) and has been improved, updated, and subsequently became the CTF. One part of the improvement process includes validating the methods in CTF. This document seeks to provide a certain level of certainty and confidence in the predictive capabilities of the code for the scenarios it was designed to model—rod bundle geometries with operating conditions that are representative of prototypical pressurized water reactor (PWR)s and boiling water reactor (BWR)s in both normal and accident conditions. This is done by modeling a variety of experiments that simulate these scenarios and then presenting a qualitative and quantitative analysis of the results that demonstrates the accuracy to which CTF is capable of capturing specific quantities of interest.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

CTF Validation and Verification (V.4.2)

Coolant-Boiling in Rod Arrays- Two Fluids (COBRA-TF) is a thermal/hydraulic (T/H) simulation code designed for Light Water Reactor (LWR) analysis. It uses a two-fluid, three-field (i.e. fluid film, fluid drops, and vapor) modeling approach. Both sub-channel and 3D Cartesian forms of nine conservation equations are available for LWR modeling. The code was originally developed by Pacific Northwest Laboratory in 1980 and has been used and modified by several institutions over the last several decades. COBRA-TF is also used at the Pennsylvania State University (PSU) by the Reactor Dynamics and Fuel Modeling Group (RDFMG) and has been improved, updated, and subsequently became the PSU RDFMG version of COBRA-TF (CTF). One part of the improvement process includes validating the methods in CTF. This document seeks to provide a certain level of certainty and confidence in the predictive capabilities of the code for the scenarios it was designed to model-rod bundle geometries with operating conditions that are representative of prototypical Pressurized Water Reactor (PWR)s and Boiling Water Reactor (BWR)s in both normal and accident conditions. This is done by modeling a variety of experiments that simulate these scenarios and then presenting a qualitative and quantitative analysis of the results that demonstrates the accuracy to which CTF is capable of capturing specific quantities of interest.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

FreeMHD: Validation and verification of the open-source, multi-domain, multi-phase solver for electrically conductive flows

The extreme heat fluxes in the divertor region of tokamaks may require an alternative to solid plasma-facing components, for the extraction of heat and the protection of the surrounding walls. Flowing liquid metals are proposed as an alternative, but raise additional challenges that require investigation and numerical simulations. Free surface designs are desirable for plasma-facing components, but steady flow profiles and surface stability must be ensured to limit undesirable interactions with the plasma. Previous studies have mainly used steady-state, 2D, or simplified models for internal flows and have not been able to adequately model free-surface liquid metal (LM) experiments. Therefore, FreeMHD has been recently developed as an open-source magnetohydrodynamics (MHD) solver for free-surface electrically conductive flows subject to a strong external magnetic field. The FreeMHD solver computes incompressible free-surface flows with multi-region coupling for the investigation of MHD phenomena involving fluid and solid domains. The model utilizes the finite-volume OpenFOAM framework under the low magnetic Reynolds number approximation. FreeMHD is validated using analytical solutions for the velocity profiles of closed channel flows with various Hartmann numbers and wall conductance ratios. Next, experimental measurements are then used to verify FreeMHD, through a series of cases involving dam breaking, 3D magnetic fields, and free-surface LM flows. These results demonstrate that FreeMHD is a reliable tool for the design of LM systems under free surface conditions at the reactor scale. Furthermore, it is flexible, computationally inexpensive, and can be used to solve fully 3D transient MHD flows.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗