Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

An Open-Source Python Package for CFD Solution Verification

Informed decision-making using computational fluid dynamics (CFD) results requires quantifying the errors and uncertainties of a simulation. Verification, validation, and uncertainty quantification (VVUQ) methods were developed to address this need and have matured. However, these VVUQ analyses are often non-trivial and require CFD analysts and practitioners to have specific skill sets. This has led to the uneven adoption of VVUQ analyses, in part, based on the availability of software tools to aid CFD analysts and practitioners. Solution verification, a procedure to evaluate the accuracy of a simulation by estimating potential errors arising from the computational model and computing the uncertainties without comparing to results from a physical system, is one of the lagging VVUQ analyses as the absence of software has forced CFD analysts and practitioners to develop their own codes or piece together incomplete software from across the internet. This work presents an opensource Python package, CFDverify, to lower the barrier of entry and fill in the technological gap in solution verification. CFDverify also provides a streamlined framework to remove some potential errors in post-processing CFD results. The hope is that CFDverify can improve the quality and quantity of CFD solution verification in scientific and research studies and attract interest in developing a communal tool. This paper describes the design, features, and an example use of CFDverify.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗
NASA NTRS2008

Assessing Requirements Quality through Requirements Coverage

In model-based development, the development effort is centered around a formal description of the proposed software system the model. This model is derived from some high-level requirements describing the expected behavior of the software. For validation and verification purposes, this model can then be subjected to various types of analysis, for example, completeness and consistency analysis [6], model checking [3], theorem proving [1], and test-case generation [4, 7]. This development paradigm is making rapid inroads in certain industries, e.g., automotive, avionics, space applications, and medical technology. This shift towards model-based development naturally leads to changes in the verification and validation (V&V) process. The model validation problem determining that the model accurately captures the customer's high-level requirements has received little attention and the sufficiency of the validation activities has been largely determined through ad-hoc methods. Since the model serves as the central artifact, its correctness with respect to the users needs is absolutely crucial. In our investigation, we attempt to answer the following two questions with respect to validation (1) Are the requirements sufficiently defined for the system? and (2) How well does the model implement the behaviors specified by the requirements? The second question can be addressed using formal verification. Nevertheless, the size and complexity of many industrial systems make formal verification infeasible even if we have a formal model and formalized requirements. Thus, presently, there is no objective way of answering these two questions. To this end, we propose an approach based on testing that, when given a set of formal requirements, explores the relationship between requirements-based structural test-adequacy coverage and model-based structural test-adequacy coverage. The proposed technique uses requirements coverage metrics defined in [9] on formal high-level software requirements and existing model coverage metrics such as the Modified Condition and Decision Coverage (MC/DC) used when testing highly critical software in the avionics industry [8]. Our work is related to Chockler et al. [2], but we base our work on traditional testing techniques as opposed to verification techniques.

Rajan, Ajitha↗
DOE OSTI2023

Standards, dissemination, and best practices in systems biology

In this study, the reproducibility of scientific research is crucial to the success of the scientific method. Here, we review the current best practices when publishing mechanistic models in systems biology. We recommend, where possible, to use software engineering strategies such as testing, verification, validation, documentation, versioning, iterative development, and continuous integration. In addition, adhering to the Findable, Accessible, Interoperable, and Reusable modeling principles allows other scientists to collaborate and build off of each other’s work. Existing standards such as Systems Biology Markup Language, CellML, or Simulation Experiment Description Markup Language can greatly improve the likelihood that a published model is reproducible, especially if such models are deposited in well-established model repositories. Where models are published in executable programming languages, the source code and their data should be published as open-source in public code repositories together with any documentation and testing code. For complex models, we recommend container-based solutions where any software dependencies and the run-time context can be easily replicated.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Steam generator model design parameter sensitivity study for small modular reactor system

Here, this study focuses on design parameter sensitivity studies pertaining to several Once-Through Steam Generator (OTSG) model cases both with and without a riser using python and advanced risk assessment and optimization tool, i.e. Risk Analysis Virtual Environment (RAVEN) developed at Idaho National Laboratory (INL), to support a Small Modular Reactor (SMR) system. The presented Steam Generator (SG) python-based model is a mathematical representation of a steam-generating unit for a Pressurized Water Reactor (PWR)-type SMR system, including fluid flow and heat transfer equations, models, and correlations. Design studies involve changing the model’s input design parameters (e.g., temperature, pressure, mass flow rate) to observe the resulting effects on the output of the system, such as the Heat Transfer Coefficient (HTC), Reynolds number, Nusselt number, and heat transfer performance. Sensitivity studies analyze the degree to which system output and/or desired parameters (e.g., HTC or heat transfer performance) are sensitive to changes in the input parameters. By using RAVEN, detailed design parametric sensitivity studies. Six input parameters—namely, the pressure, temperature, and mass flow rate for the inlet of the primary-side (hot fluid) and secondary-side (cold fluid) of the SG—were randomly perturbed via RAVEN’s Monte Carlo Sampler module, using uniform distributions (i.e., ±1%, ±5% and ±10 % relative changes) for 600 samples. The analysis results give valuable insights into SG system performance, and provide justification for further research and development such as optimized sensor placement, design verification, validation, and optimization.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Explicit modeling of pebble temperature in the porous-media model for pebble-bed reactors

In this study, we developed a multiscale model to include an explicit pebble-temperature model nested in the porous-media model for pebble-bed reactor applications. The multiscale solid-phase energy balance model, including the pebble surface energy balance equation and an explicit modeling of pebble temperature, can predict the macroscopic (pebble bed) and microscopic (pebble) temperature distributions under both steady-state and transient conditions. The proposed multiscale model is solved in a fully coupled manner using the Newton- Krylov method, and therefore iterations between the macroscopic (pebble-bed-scale) and microscopic (pebble- scale) model are avoided. Extensive code verifications, validation, and demonstrations have been performed for this newly developed model. By explicitly modeling pebble temperatures, this new model addresses a major deficiency of the basic porous-media model, which assumes homogeneous solid-phase temperature and is not appropriate for pebble-bed reactor design and safety analyses.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

Machine learning enhanced predictions of ICRF heating: Overcoming numerical limitations via data curation

In this work, we present the development of robust surrogate models for Ion Cyclotron Range of Frequencies (ICRF) and High-Harmonic Fast Wave (HHFW) heating predictions in fusion plasmas. Building upon our previous efforts to achieve real-time capable models, we identify the cause of the outliers found using TORIC in certain HHFW heating scenarios. The outliers are observed to be spurious ion Bernstein wave (IBW)-like modes caused by a wavelength control algorithm designed to address challenging scenarios with high perpendicular wavenumbers. The effect arises from the modulation in the perpendicular susceptibility, which can induce sign reversal and IBW-like propagation for scenarios featuring normalized ion Larmor radius λ i ≫ 1. We use TORIC with this algorithm disabled to generate a novel HHFW-NSTX database that is free of outliers. Surrogate models trained on this database, including Random Forest Regressor (RFR), Multi-Layer Perceptrons, and Gaussian Process Regressors (GPR), demonstrate the ability to accurately predict HHFW heating profiles, with regression scores of R 2 ∈[0.93−0.99]. Additionally we demonstrate that it is possible to generalize predictions beyond training data by the use of both RFR and GPR models, enabling the prediction of scenarios previously limited to the original model. GPR models also provide uncertainty quantification, offering insights into model confidence. This work introduces a comprehensive Verification, Validation, and Uncertainty Quantification methodology for surrogate modeling, applicable not only to ICRF heating but also to other RF heating challenges and fusion physics problems. Beyond accelerated inference, these models show effective extrapolation capabilities, providing an alternative for addressing numerical challenges.

Artificial neural networks↗
DOE OSTI2025

Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning, the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but under-rated area is uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions, due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of UQ of ML. We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling will be discussed, demonstrated, and compared. We will also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian Processes and conformal prediction. Lastly, we will discuss the need for building a verification, validation and UQ framework to establish ML credibility.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

A critical review on additive manufacturing of refractory alloys from a data analytics perspective- beyond nickel-based superalloys

Refractory alloys (RAs) are promising materials due to their exceptional physicochemical properties, but most research remains at the laboratory scale. For broader adoption, advancements in manufacturing are essential. Because their high stability makes conventional methods like machining and casting difficult, additive manufacturing (AM) is emerging as an effective approach for fabricating refractory alloy components. However, AM's repeated non-equilibrium thermal cycles introduce undesired features (e.g. defects, anisotropic microstructures, and residual stresses), which are magnified due to RAs’ unique properties. This paper comprehensively reviews the state-of-the-art methods of AM for refractory alloys. It explores data analytics techniques to establish design rules based on multi-fidelity experimental and computational methods. Furthermore, it investigates integrated, collaborative efforts to harmonise standalone databases, information, knowledge, and predictive models at multi-physics, multi-stage, and multi-scale. Unlike the existing literature that focuses primarily on material systems or process fundamentals, this work provides an integrated perspective on AM of refractory alloys from a data analytics standpoint, highlighting the roles of integrated computational materials engineering (ICME), verification, validation, and uncertainty quantification (VV&UQ), and digital twin-driven qualification in overcoming data scarcity and accelerating rapid qualification.

Additive manufacturing↗
DOE OSTI2020

Multiple Degradation Mechanisms in Reinforced Concrete Structures, Modeling and Risk Analysis

The overarching goal of this project is to complement ongoing Department of Energy (DOE) Light Water Reactor Sustainability (LWRS)-funded Grizzly concrete modeling development efforts by providing improved multi-physics models for incorporation into the Grizzly and BlackBear codes. Objectives of this work identified at the outset of this project include: 1. Coupling between mechanical damage and transport processes. The mechanical damage will be characterized by nonlinear mechanical constitutive models appropriate for concrete used in nuclear power plants (NPPs). 2. Improved representation of the coupling among the transport process, building on the current moisture and thermal transport models in Grizzly. The parameters for coupling among heat and mass transport in the multi-physics model will be experimentally determined. 3. Coupling among different scale levels of concrete constituents, fine and coarse aggregates, cement paste, hydration products, and pore structure. 4. Probabilistic analysis of the random nature of the heterogeneous concrete structure and the environmental factors (ambient temperature and humidity) and their multiple effects on concrete deterioration. 5. Benchmark verification, validation and uncertainty quantification of the thermo-hygro- chemo-mechanical (THCM) concrete formulation. The end goal of this work was to provide a comprehensive and robust simulation capability that can be applied at the engineering scale for analysis of realistic deterioration scenarios in NPP concrete structures.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Initial Atmospheric Transport of Particles

A new version of the Initial Atmospheric Transport (IAT) model has been developed. This report fully characterizes the new model. IAT is designed to model heat release into the atmosphere that contains contaminated particles. The heat release forms a buoyant puff or plume. Buoyancy is quickly dissipated in 10's to 100's of seconds through drag and entrainment of ambient air. The final location of contaminant particles in the atmosphere after dissipation of abnormal heat is an important input to long term transportation codes such as the National Oceanic and Atmospheric Administration's (NOAA) Hybrid Single Particle Lagrangian Integrated Trajectory (HYSPLIT) code. Ultimately the settling of contaminants after long term transport provides a basis for estimating health consequences. In comparison to the old version, the new model includes explicit simulation of particles, newly formulated equations of motion for a rising puff, and a new plume model. The plume model releases many instances of a single puff along a single trajectory in a way that balances energy released over a longer period than would be appropriate for a single puff. The puff model has additions for vorticity, turbulence, virtual mass, drag, and entrainment. The particle model leads to direct interfacing between IAT and the long-term transport code HYSPLIT. Previously, particle locations were given a pattern based on IAT's puff trajectory with no particle simulation. Now, particles are explicitly simulated, and 3-D spatial distributions of particles are output based on final positions. Ballistic particles are thrown outward at high velocity from explosive events and tend to hit the ground before the end of the simulation. Non-ballistic particles tend to get pulled into the rising puff s ring of vorticity with some escaping to ambient conditions as buoyant energy dissipates. A new model verification and validation effort was made for the new plume model. The Wallops Island Solid Rocket Propellant (SRP) fire plume was extracted to a 3-D transient dataset. After calibration, the model performs better than previous single puff attempts. Even so, error levels in the process used to estimate location and ambient weather conditions approach 30%. This leads to the conclusion that unplanned photogrammetry evaluations of the IAT model are useful but not sufficiently accurate to quantify model parameters. Like previous attempts, the vertical motion entrainment parameters come out much too high in comparison to values for other studies in the literature. It is suspected that a new model of shear entrainment would alleviate this problem. Unfortunately, IAT already has more calibration parameters than the data comparisons thus far can resolve with statistical certainty. This underscores the need to verify and validate the IAT model using detailed CFD studies that allow ideal statistical comparison with the IAT model for hundreds of cases. Even so, validation by comparison to carefully planned experiments (i.e. with acceptable error levels) is also necessary because the dynamics of entrainment are at a sub-grid level for CFD. Though better validation and verification are most important, many new enhancements are still envisioned for future IAT development. These include particle rain out due to condensation and agglomeration of water, direct connection to the North American Mesoscale (NAM) weather dataset, dynamic weather conditions, including ambient particle entrainment into the puff, and detrainment at the puff to ambient mixing interface. In general, including detailed aerosol dynamics is the greatest coming challenge which will require an entire new sub-model in IAT. This is very important if contaminant particles change in such a way that they increase or decrease in their potential harm to the environment.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Data Evaluation of Actinide Cross Sections: 230 Th and 231 Th

A new evaluation of the ENDL cross section set for Thorium (Z=90) is developed using the TALYS statistical model cross section code. The primary goal of this effort is to produce an evaluation that attempts to match as closely as possible fission cross sections developed through surrogate reaction techniques on actinide targets 230 Th and 231 Th. This evaluation effort and the processing needed to render its results into data libraries is a necessary step in making the efforts of nuclear experimentalists useful to the broad community of researchers engaged in simulations of nuclear fusion for basic and applied science. Another aspect, verification and validation against various AGEX experiments, is also presented. The end-product is an updated library that includes the latest measurements of fission cross sections for comparison against those measured via traditional techniques. All the steps in the evaluation, processing, validation and verification, and library release are described in the following sections. For completeness, the appendix contains all the parameters used in the TALYS cross section evaluation for neutrons incident on 230 Th and 231 Th.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

Calculation of the First Moment of Energy Using D-T Reactivity Formalisms Under the Maxwell-Boltzmann Distribution (Pt. 1)

Nuclear fusion science is an example of a scientific field with a rich history of expert involvement and scientific publications, which together, form an expert-knowledge base. One example of this history is the utilization of published reaction rates from a variety of authors. Investigators for Deuterium-Tritium (D-T) ion fusion can choose from using frequently cited methods: the Bosch and Hale reactivity, thermonuclear reaction rates from Caughlan and Fowler, and the reactivity evaluation from Miley, Towner & Ivich which forms the basis of the Naval Research Lab (NRL) formulary. There are other choices available. Each of the reactivity formulations considered here, are based upon the Maxwell- Boltzmann velocity distribution for D-T fusion ion reactants. Numerical methods for computer codes simulating hot, energetic plasmas, include tabulations of the reactivity, and the first moment of energy. The purpose of this study is to answer the question: what is the first moment of energy, and how has it been formulated? This report is part of a series of reports by the authors on D-T fusion-reaction formalism. The present focus is on defining the mathematical relationship of the first moment of D-T fusion ion kinetic energy, $\langle$E$\rangle$, with the fusion crosssection, fusion reactivity and its derivative with respect to ion-temperature. Three variants of the first moment $\langle$E$\rangle$ are analytically developed and explored: 1) constant cross-section, 2) a normalized first moment, and 3) a particular function of the first moment. Much of the mathematical development is relegated to the Appendices, for a concise presentation of topics and results. A research component of this discussion is our comparison of some of the reactivity and its derivative results between evaluations (1) versus (2) and (3), which the authors have not found in any other publication to date. This investigation is useful for validation, verification (V&V) and uncertainty quantification (UQ) as it compares the work of several authors.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Dakota-NAERM Integration

This report presents the results of a collaborative effort under the Verification, Validation, and Uncertainty Quantification (VVUQ) thrust area of the North American Energy Resilience Model (NAERM) program. The goal of the effort described in this report was to integrate the Dakota software with the NAERM software framework to demonstrate sensitivity analysis of a co-simulation for NAERM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deterministic Calibration of MFiX-PIC, Part 1: Settling Bed

The Particle-in-cell (PIC) numerical approach for modeling granular solids in fluid flow has gained significant interest in recent years. Valued for its often shorter time-to-solution, the PIC formulation relies on modeling statistical groupings of particles called parcels in cooperation with a solids stress model to affect local solids velocity. This is in contrast to the discrete element model (DEM) where every particle in a system is modelled individually and directly coupled to local solids velocity through Newtonian mechanics. The U.S. Department of Energy (DOE), National Energy Technology Laboratory (NETL) develops and maintains Multiphase Flow with Interphase eXchanges (MFiX), a collection of open-source computational fluid dynamics (CFD) solvers. Included in the MFiX suite are traditional two-fluid model (TFM) and DEM solvers, and a recently added PIC solver (NETL, 2021). In general, PIC methodologies offer an accuracy trade-off in lieu of computational speed; and therefore, it is important to assess the credibility of MFiX-PIC simulations. For this purpose, a systematic verification, validation and uncertainty quantification (VVUQ) effort was initiated at NETL to assess the new PIC solver

42 ENGINEERING↗
DOE OSTI2020

Sensitivity and Uncertainty Analysis of Generator Failures under Extreme Temperature Scenarios in Power Systems

This report summarizes work done under the Verification, Validation, and Uncertainty Quantification (VVUQ) thrust area of the North American Energy Resilience Model (NAERM) Program. The specific task of interest described in this report is focused on sensitivity analysis of scenarios involving failures of both wind turbines and thermal generators under extreme cold-weather temperature conditions as would be observed in a Polar Vortex event.

17 WIND ENERGY↗
DOE OSTI2021

FY2021 Progress Report on BISON Metallic Fuel Model Development and V&V Using EBR-II Legacy Data

In this report, the activities and achievements made by Argonne National Laboratory for the Nuclear Energy Advanced Modeling and Simulation (NEAMS) BISON code metallic fuel validation and verification project in FY2021 are summarized. The cladding degradation model based on the FCCI/CCCI wastage calculations has been developed and implemented into BISON. A comprehensive evaluation of the cladding degradation model was performed based on FIPD data of the IFR experiment X447. BISON objects were also developed to enable direct use of time-varying cladding outer surface temperature profile as temperature boundary conditions, which proved to provide more accurate temperature predictions for the metallic fuel pins irradiated in EBR-II. Additionally, a new BISON object was implemented to enable direct comparison between BISON predicted data and FIPD-based post-irradiation examination (PIE) results, which would significantly facilitate BISON metallic fuel verification and validation (V&V) activities. These new BISON-FIPD integration features were used in the establishment of a low-burnup fuel swelling evaluation framework as demonstration. The framework was successfully used to evaluate current fuel swelling models based on the IFR experiment X423.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

Gemma V&V/UQ/Credibility Activities: FY2020 Progress

This report describes the credibility activities undertaken in support of Gemma code development in FY20, which include Verification & Validation (V&V), Uncertainty Quantification (UQ), and Credibility process application. The main goal of these activities is to establish capabilities and process frameworks that can be more broadly applied to new and more advanced problems as the Gemma code development effort matures. This will provide Gemma developers and analysts with the tools needed to generate credibility evidence in support of Gemma predictions for future use cases. The FY20 Gemma V&V/UQ/Credibility activities described in this report include experimental uncertainty analysis, the development and use of methods for optimal design of computer experiments, and the development of a framework for validation. These initial activities supported the development of broader credibility planning for Gemma that continued into FY21.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Comparative Critical Mass Calculations for NNL and ENDF/B-VIII.0 Zirconium Hydride Thermal Scattering Laws [Slides]

Zirconium Hydride (ZrH x ) is a moderator material for TRIGA reactors and SNAP-10A.The thermal scattering has the behavior of quantum oscillator favorable for moderator feedback. ZrH x occurs as a corrosion product in LWR. Previously new phase specific TSL evaluations were generated and submitted to ENDF/B- VIII.1. These include the effects of crystal structure on elastic scattering of Zr(ZrH x ), permit flexibility in stoichiometry for use in reactor physics calculations and fuel characterization, and expand the TSL material sub-library. ENDF/B-VIII.0 ZrH TSLs do not distinguish between material phase. Current work involves preliminary validation & verification testing on the effect of TSLs on critical mass and associated thermal flux.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗