Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Statistical Validation of Multiple Related Data Sets—Case Study Using Interstellar Boundary Explorer Satellite Data

Abstract Space scientists often face the question of whether data collected by different instruments are measurements of the same source population. This paper proposes a statistical validation method for evaluating the agreement between such related data sets. It offers a detailed case study focused on validating a new data set from the Interstellar Boundary Explorer (IBEX) mission, which serves as a practical how-to guide for similar analyses. Since 2008, the IBEX satellite has been gathering data on heliospheric energetic neutral atoms (ENAs) while being exposed to various sources of background noise, such as cosmic rays and solar energetic particles. The IBEX mission initially released only a qualified triple-coincidence (qABC) data product, which was designed to provide observations of ENAs free of background contamination. Further measurements revealed that the qABC data were in fact susceptible to contamination, having relatively low ENA counts and high background rates. To mitigate this issue, the mission team recently considered releasing a certain qualified double-coincidence (qBC) data product, which has roughly twice the detection rate of the qABC data product. This paper presents a simulation-based validation of the new qBC data product against the already-released qABC data product. The results show that the qBCs can plausibly be said to be measuring the same source population as the qABCs up to an average absolute deviation of 3.6%. Visual diagnostics provide additional confirmation of source rate coherence across data products. The framework introduced here is general and can be applied to other validation problems both within and outside the field of space physics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Data-driven model validation for neutrino-nucleus cross section measurements

Neutrino-nucleus cross section measurements are needed to improve interaction modeling to meet the precision needs of neutrino experiments in efforts to measure oscillation parameters and search for physics beyond the Standard Model. We review the difficulties associated with modeling neutrino-nucleus interactions that lead to a dependence on event generators in oscillation analyses and cross section measurements alike. We then describe data-driven model validation techniques intended to address this model dependence. The method relies on utilizing various goodness-of-fit tests and the correlations between different observables and channels to probe the model for defects in the phase space relevant for the desired analysis. These techniques shed light on relevant mismodeling, allowing it to be detected before it begins to bias the cross section results. We compare more commonly used model validation methods which directly validate the model against alternative ones to these data-driven techniques and show their efficacy with fake data studies. These studies demonstrate that employing data-driven model validation in cross section measurements represents a reliable strategy to produce robust results that will stimulate the desired improvements to interaction modeling.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Extension and Validation of NEMA-Style Dual-Ring Controller in SUMO

Until recently, SUMO users could not model the behavior of a ring-and-barrier traffic signal via existing signal types, which left North American SUMO users without the direct ability to capture traffic dynamics of their local networks. This work presents the methods, implementation overview, and validation of a 'dual-ring' NEMA style traffic controller which has recently been added to the main SUMO code base. A brief explanation of the 'dual-ring' implementation is also provided as context for those new to this type of traffic controller. The foundation for this work was presented at the SUMO User Conference 2021 by researchers at the US Department of Energy's National Renewable Energy Laboratory, but was not integrated into SUMO code base at the time. Following the initial inclusion of the controller to SUMO, the authors began validation of the SUMO controller against an Econolite software-in-the-loop (SIL) traffic signal controller configured with actual setup parameters from controllers in a real-world three-intersection corridor in Tuscaloosa, Alabama, USA. This paper documents the process of adding new features to the controller code as well as validating their implementation through simulation-based and automated grey-box testing is presented in this paper. Key features such as fully-actuated operation, various timing offset plans, proper next-phase fit algorithms and more, have been added and validated against this SIL system. Though not an exhaustive demonstration of features, this work is intended make more users aware of this extension of SUMO capabilities.

MATHEMATICS AND COMPUTING↗
DOE OSTI2019

VALIDATION OF KENO V.A AND KENO-VI IN SCALE 6.3 BETA 3 USING ENDF/B-VII.1 AND ENDF/B-VIII LIBRARIES

In early 2019, internal beta releases of SCALE 6.3 were identified for internal users, as well as a limited number of external users, to apply when testing key features. Recently developed AMPX-formatted libraries based on ENDF/B-VIII have also been generated for use in internal testing. In this effort, a criticality safety validation was performed using nuclear data libraries based on the ENDF/B-VII.1 and ENDF/B-VIII releases to quantify the changes caused by code development and to identify differences caused by the new nuclear data. As with past validations, the primary suite of experiments used is the Verified, Archived Library of Inputs and Data (VALID) maintained in the Reactor and Nuclear Systems Division at Oak Ridge National Laboratory. The VALID suite contains a total of 618 individual configurations, including systems with fast, thermal, mixed, and intermediate neutron spectra. Fissile species include a range of uranium enrichments, plutonium, mixtures of uranium and plutonium, and 233U. Metal, solution, and compound, mostly in pin array fissile forms, are included in the suite. The results presented here include the 252-group general purpose multigroup library based on ENDF/B-VII.1 and the continuous-energy libraries based on ENDF/B-VII.1 and ENDF/B-VIII.

Marshall, William (B.J.)↗
DOE OSTI2019

Initial Application of TSUNAMI for Validation of Advanced Fuel Systems

A number of advanced reactor concepts and accident-tolerant fuels (ATFs) are under development or in the initial stages of deployment and testing in the nuclear power industry. Many of these concepts incorporate chemical or material forms that have not traditionally been used in the nuclear industry and may present challenges for nuclear criticality safety validation. Sensitivity/uncertainty methods are ideally suited for assessing the applicability of existing critical experiments to these potentially challenging applications. This paper presents two case studies recently completed at Oak Ridge National Laboratory (ORNL) which investigated the use of the TSUNAMI tools within the SCALE code system to perform such assessments.Many advanced reactor concepts include uranium enrichments above 5 wt% 235U but below the 20 wt% limit of low-enriched uranium (LEU). This material, referred to as high assay LEU (HA-LEU), poses difficulties across the entire infrastructure of nuclear power, including enrichment, transportation, fabrication, and storage. A potentially representative transportation package was selected to perform a criticality safety validation assessment for UF6 containing HA-LEU. Sensitivity data were generated using the TSUNAMI-3D sequence, and an assessment of the applicability of a range of critical experiments was performed using the TSUNAMI-IP tool. The critical experiments were drawn from the set of available sensitivity data files distributed with the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The results indicate that a sufficient number of available experiments may be available to perform validations for these systems.Many different cladding materials and fuel dopants are included in ATFs to mitigate the consequences of a loss of coolant accident. Some of these systems have already been fabricated and introduced as lead test assemblies in commercial reactors, while others are still being developed. Several concepts were considered for pressurized-water reactors (PWRs) and boiling-water reactors (BWRs) in comparison to the current Zircaloy and UO2 fuel system. In most of these cases, the ATF materials did not significantly impact the number of applicable experiments for validation. Some fuel and cladding materials were identified, however, which may reduce the number of available experiments and which will increase the nuclear data–induced uncertainty in neutron multiplication.

Marshall, William (B.J.)↗
DOE OSTI2019

Empirical Validation of Building Energy Modeling using Flexible Research Platform

Recently, efforts have been made to provide reliable empirical data for ANSI/ASHRAE Standard 140, Standard Method of Test for the Evaluation of Building Energy Analysis Computer Programs, to enable improved accuracy of building energy model (BEM) engines and improved characterization of their accuracy. The motivation for this effort is that the use of reliable empirical validation data sets in the evaluation of building energy modeling tools will lead to more consistent and validated simulation engines across all software vendors. This would expedite the use of building energy modeling in designing new buildings and retrofitting existing buildings, which delivers more energy efficient buildings.As part of a three-year multi-lab empirical validation project sponsored by U.S. DOE, this research project generated cooling season test plans by reviewing ASHRAE Standard 140, and the tests were performed based on the test plan. Finally, the experimental data sets were compared with the EnergyPlus model to demonstrate the validation procedure.

Im, Piljae↗
DOE OSTI2023

Advancements in Validation of TSLs through Inelastic Neutron Scattering and Transmission Measurements [Abstract]

Historically, the free gas approximation has been used to treat the thermal scattering of neutrons with energies below a few electron-volts (eV) in unevaluated materials. However, this method inadequately reproduces neutron scattering at these energies. Until recently, only a limited number of materials had available thermal scattering law (TSL) files/libraries in the ENDF nuclear data libraries in this energy range. With advancements in atomistic modeling techniques, such as molecular dynamics, ab-initio molecular dynamics, and density functional theory, TSL libraries have become available for many more materials. This is particularly relevant due to the rising interest in several advanced reactor systems that require novel moderator and reflector materials. While quasi-integral and integral benchmarks have been designed to validate historically important moderator materials (such as light water and polyethylene), there is currently a lack of standard validation methods for TSLs, especially when multiple conflicting TSLs exist. To address this issue, the Oak Ridge National Lab Nuclear Data group has been working on utilizing inelastic neutron scattering (INS) measurements combined with transmission (i.e., total cross section) measurements to evaluate and validate TSLs for different materials. We plan to demonstrate how this method has worked on materials such as polyethylene, lucite, and polystyrene. In addition, we will compare the newly created libraries to ENDF libraries for these materials and explain why integral benchmarks should not be used for validation when multiple TSLs exist.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Results of an In-Field Validation Exercise in Support of Wide-Area Environmental Sampling

The National Nuclear Security Administration’s (NNSA) Office of Nonproliferation and Arms Control (NA-24) is evaluating Wide-Area Environmental Sampling (WAES) as an additional safeguards verification tool for the International Atomic Energy Agency to detect undeclared nuclear activities. The NNSA is evaluating strategies for conducting a generic WAES campaign, the cost of a WAES campaign, and the effect of technological advancements that have occurred since the last major WAES review in 1999. Until now, the NNSA effort has focused on tabletop exercises (TTXs) in which high-performance computing allows for advanced modeling and simulation efforts to be applied to the WAES question. Although the modeling and simulations used in the TTXs are extremely valuable, field campaigns are still needed to validate the assumptions that underpin the models and the modeling process itself. During a 7 week period beginning in May 2023 and ending in June 2023, which included 4 weeks of active field collections, a multilaboratory team conducted its first in-field validation exercise. Prior to the in-field exercise, abbreviated TTXs were conducted to estimate the performance of all collection systems to be used during the field test. These TTXs guided the selection of materials to be released and the placement of the collection system. Based on these determinations, materials were procured to use in the field test, and an injection/release system was designed, built, and installed at the test facility. Background samples were collected during weeks one and four, and environmental collections against active releases were conducted during weeks two and three. The goals of this validation exercise included a demonstration of (1) the ability to provide controlled releases of particulates of surrogate materials, (2) the fielding and operation of collection systems (including deposition and active air collectors), and (3) the flexibility to revise equipment and campaign plans in the field. This paper presents the results and preliminary conclusions for this initial validation test. Based on these results, subsequent field campaigns are anticipated and will include the addition of other released materials.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Validation of the DESI DR2 Ly$α$ forest full-shape analysis

We present the validation of the Dark Energy Spectroscopic Instrument (DESI) Data Release 2 (DR2) Lyman-$α$ (Ly$α$) forest full-shape analysis. This analysis combines three-dimensional Ly$α$ forest auto-correlations and cross-correlations with quasars to extract information from both the baryon acoustic oscillation (BAO) feature and the broadband clustering signal, with primary emphasis on the Alcock-Paczynski (AP) measurement. Compared to the DESI DR1 analysis, the DR2 validation uses substantially larger and more realistic mock datasets, including CoLoRe 2LPT and AbacusSummit Ly$α$ forest simulations. The modeling framework is also improved through analytic marginalization over small scales ($<10$$h^{-1}$Mpc) and the impact of ultraviolet background fluctuations. The validation program was completed prior to unblinding and defines quantitative requirements for the cosmological parameters of interest, which are evaluated using hundreds of mock realizations. We further test the analysis through independent fits to the auto- and cross-correlations, multiple catalog splits, and a broad suite of analysis and modeling variations applied to both mocks and blinded observational data. We find that the BAO and AP parameters satisfy all validation requirements and remain stable across all tests. In contrast, mock studies reveal a significant bias in the inferred growth-rate parameter $fσ_8$, leading us to exclude this measurement from the final analysis. The consistency across mocks, data splits, and robustness tests demonstrates that the DR2 Ly$α$ full-shape analysis provides a reliable and substantially improved broadband AP measurement over previous Ly$α$ forest studies.

Herbold, M. [Chicago U., KICP; Ohio State U.] (ORC↗
DOE OSTI2023

Systematic Validation of Ensemble Cloud-Process Simulations using Polarimetric Radar Observations and Simulator over the NASA Wallops Flight Facility

The BiLateral Operational Storm-Scale Observation and Modeling (BLOSSOM) project was initiated to establish a long-term supersite to improve understanding of cloud physical states and processes as well as to support satellite and climate model programs over the WFF site via a bilateral approach of storm-scale observations and process modeling. This study highlights a noble systematic validation framework of the BLOSSOM ensemble cloud-process simulations through mixed-phase, light-rain, and deep-convective precipitation cases. The framework consists of creating a domain-shifted ensemble of large-scale forcing datasets, and configuring and performing cloud-process simulations with three different bulk microphysics schemes. Validation uses NASA S-band dual-POLarimetric radar (NPOL) observations in the form of statistical composites and skill scores via a polarimetric radar simulator and newly developed CfRad Data tool (CfRAD). While the simulations capture the overall structures of the reflectivity composites, polarimetric signals are still poorly simulated, mainly due to a lack of representation of ice microphysics diversity in shapes, orientation distributions, and their complex mixtures. Despite the limitation, this new ensemble-based validation framework demonstrates that 1) no particular forcing or microphysics scheme outperforms the rest and 2) the skill scores of coarse- and fine-resolution ensemble simulations with different domain-shifted forcing and microphysics schemes are highly correlated with each other with no clear improvement. On the other hand, this suggests that coarse-resolution ensemble simulations are relevant for selecting the best meteorological forcing and microphysics scheme before conducting computationally demanding large eddy simulations (LESs) in support of aircraft and satellite instrument development as well as cloud-precipitation-convection parameterizations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

ORNL’s Continuing Efforts in Evaluating and Validating TSLs [Slides]

This presentation finds that INS measurements are not as variable based on sample composition, because phonon spectra is a properly of the bulk material. Additionally, INS and transmission measurements are and should be the most significant part of TSL validation, as they are the fundamental quantities used by Monte Carlo neutronics codes. This presentation reasons that relaying solely on theoretical atomistic calculations for TSL evaluation is not enough, they are highly dependent on inter-atomic potentials (which are not the most accurate depending on the quantity trying to be reproduced) and evaluators knowledge and understanding of the material being studied. Finally on Critical benchmarks, while extremely useful for validation of nuclear data at all energies, critical benchmarks are not the best tool to provide a definitive answer on conflicting TSLs. The presentation concludes that INS and transmission measurements need to be the basis of validation of TSLs.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

OC6 Phase IV: Validation of CFD Models for Stiesdal TetraSpar Floating Offshore Wind Platform

ABSTRACT With only a few floating offshore wind turbine (FOWT) farms deployed anywhere in the world, FOWT technology is still in its infancy, building on a modicum of real‐world experience to advance the nascent industry. To support further development, engineers rely heavily on modeling tools to accurately portray the behavior of these complex systems under realistic environmental conditions. This reliance creates a need for verification and validation of such tools to improve reliability of load and dynamic response prediction and analysis capabilities of FOWT systems. The Offshore Code Comparison Collaboration, Continued with Correlation and unCertainty (OC6) project was created under the framework of the International Energy Agency to address this need and considers a three‐sided verification and validation between engineering level models, computational fluid dynamics (CFD), and experimental results. In this paper, a novel floating offshore wind platform, the Stiesdal TetraSpar, is simulated using CFD under the load conditions defined by Phase IV of the OC6 project. The comparison of these CFD results against the experimental results demonstrated the ability to predict the platform response to waves when imposing the measured wave signals as input. Although validation versus experiment was largely successful, the damping behavior was impacted by uncertainties likely originating from the mooring system and sensor umbilical cable. This extensive comparison effort with multiple CFD practitioners offers insight into best practices to achieve reliable results.

17 WIND ENERGY↗
DOE OSTI2021

Which nuclear data can be validated with LLNL pulsed-sphere experiments?

It is shown here that 14-MeV D+T LLNL pulsed-sphere experiments bring complementary information into the process of validating nuclear data compared to experiments that are traditionally used for this purpose—such as critical assemblies. To be more specific, the 14-MeV D+T LLNL pulsed-sphere neutron-leakage spectra enable to validate scattering and fission nuclear data up to 15 MeV (compared to approximately up to 5 MeV when using criticality experiments) and employ to this end simple compound targets containing only few isotopes. In this work, sensitivity profiles of the spectra to nuclear data are calculated in order to understand in detail which isotopes, observables, and energy ranges of nuclear data contribute significantly to their simulation. These profiles are presented for a few selected spheres containing 16 O, 12 C, 56 Fe, and 239 Pu. It is shown that the neutron-leakage spectra of spheres containing light elements are mostly sensitive to elastic- and inelastic-scattering cross sections on discrete levels and corresponding angular distributions. Spheres of structural materials are sensitive to elastic- and inelastic-scattering cross sections, including scattering on discrete levels and the continuum, and double-differential cross sections. Actinide spheres are also strongly sensitive to the fission observables, in particular to the total-fission neutron spectrum. Thin spheres (in which neutrons experience on average less than one scatter) are mostly sensitive to data near the elastic peak, in the energy range from 12–15 MeV, while thicker ones can be sensitive to data at lower incident-neutron energies due to multiple-scattering effects. This information is brought together with simulations of 71 pulsed-sphere neutron-leakage spectra using the ENDF/B-VII.1 and ENDF/B-VIII.0 nuclear-data libraries. This analysis highlights ENDF/B-VIII.0 data that could be further investigated for potential shortcomings ( 6 Li, 12 C, 16 O, 24-26 Mg, 27 Al, 48 Ti, 56 Fe, and 208 Pb) or are likely reliable ( 1,2 H, 7 Li, 9 Be, 14 N, 235,238 U, and 239 Pu) as indicated by validating with LLNL pulsed-sphere experiments.

14-MeV D+T LLNL pulsed-sphere neutron-leakage spec↗
DOE OSTI2024

Validation of Phasor-Domain Transmission and Distribution Co-simulation Against Electromagnetic Transient Simulation

The rapid deployment of renewable energy resources has led to the widespread use of power electronics in modern power systems. As these systems transition from being dominated by large synchronous machines to increasingly incorporating inverter-based resources (IBRs), traditional methods are becoming inadequate. Addressing this challenge, this paper introduces a scalable phasor-domain T\&D co-simulation framework based on open-source software. It focuses on the framework's validation against the PSCAD Electromagnetic Transient (EMT) analysis tool. The validation results demonstrate the framework's high-fidelity and a computational time speed-up of 60 to 100 times, marking a pioneering validation effort in T\&D co-simulation research.

Inverter-based resources, co-simulation, Electroma↗
DOE OSTI2022

First-In-Human Validation of CT-Based Proton Range Prediction Using Prompt Gamma Imaging in Prostate Cancer Treatments

Uncertainty in computed tomography (CT)-based range prediction substantially impairs the accuracy of proton therapy. Direct determination of the stopping-power ratio (SPR) from dual-energy CT (DECT) has been proposed (DirectSPR), and initial validation studies in phantoms and biological tissues have proven a high accuracy. However, a thorough validation of range prediction in patients has not yet been achieved by any means. Here, we present the first systematic validation of CT-based proton range prediction in patients using prompt gamma imaging (PGI).

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2022

Development and validation of a quantitative proton NMR method for the analysis of pregnenolone

Pregnenolone (PREG) is an endogenous steroid frequently sold as an over-the-counter dietary supplement touted to promote neurological and immunological health. While the PREG dietary supplement is added to the diet for health benefits, there are no FDA approved PREG drugs. However, compounded PREG drug products are available to U.S. patients. The FDA works with state regulatory authorities on the oversight of compounding activities, including developing 503A and 503B lists of bulk substances that compounders are permitted to use. PREG is one of the substances publicly nominated to be included on the 503B list. Compounded hormone therapies such as those using PREG are of interest given the lack of standardization in compounded drug products which may increase the possibility of underdosing, overdosing, or contamination. However, no USP monograph currently exists to evaluate the quality of PREG drug substance or product. To address knowledge gaps and assist in quality control, a simple and rapid quantitative proton nuclear magnetic resonance spectroscopy (qNMR) method for the identification and assay of PREG in different types of PREG products was developed and validated. PREG samples were characterized using 1D 1 H and 2D 1 H– 13 C HSQC NMR spectra. The qNMR assay method (taking approximately 10 min per NMR spectrum) was validated for precision, accuracy, specificity, robustness and linearity per ICH Q2(R1) guidance. The method was validated in a range from 0.032 to 3.2 mg/mL. As a proof of concept, seven PREG bulk substance samples, three tablet and two capsule PREG dietary supplements were assessed by the qNMR analytical procedure. NMR data from all tested samples met the expected criteria for identification and assay. The results demonstrate the potential of qNMR for the quality assessment of different types of PREG samples.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Validation of electron backscattering in the MCNP6.3 code

A new electron backscattering validation suite has been developed for the MCNP® code. The calculations in this suite cover both condensed history and single-event transport methods, seven different elemental materials ranging from beryllium to uranium, and incident electron energies from 200 eV to 14.1 MeV. In general, the accuracy of the MCNP code for electron backscattering calculations is energy-dependent. For incident electron energies below 2 keV, the condensed history method is not applied while the single-event method shows poor agreement with experimental measurements due to poor accuracy of the underlying atomic data. For incident energies between 2 keV and 256 keV, the single-event method typically shows better agreement with experiments, while for incident energies above 256 keV the condensed history method gives better accuracy. Additionally, agreement with experimental data is generally worse for low- Z materials such as beryllium and carbon, which may reflect deficiencies in the data or physics underlying electron transport through light elements in the MCNP code. The new electron backscattering validation suite, along with the previously introduced electron stopping power and Lockwood energy deposition validation suites, provide MCNP users with a quantified understanding of the uncertainties in electron transport calculations. Future improvements to electron transport in the MCNP code are necessary for low energies (≤ 2 keV) and for low- Z materials.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Experimental validation of a high fidelity Monte Carlo neutron transport model of the MIT graphite exponential pile

High-fidelity modeling and simulation were performed for the MIT graphite exponential pile (MGEP) using Monte Carlo neutron transport codes OpenMC and MCNP, and the results were validated by experimental data. The MGEP is being used as the test bed for the design of an autonomous control system for the pile's neutron flux distribution. The main contribution of this work is to generate the training data sets of neutron flux distributions with different locations of control rods that perturb the neutron flux profiles. First, code -to-code cross verification between OpenMC and MCNP was performed to ensure consistency of the numerical modeling within statistical uncertainties. To validate the accuracy of this high-fidelity model, a series of neutron flux measurements were conducted using a Helium-3 (He-3) neutron detector on a mobile platform that is placed inside the pile. Second, the neutron flux profiles were measured in four vertical layers of interest, and compared to the corresponding simulation results. The comparison results shows that the root mean square error is less than 2.5% in the two upper layers, and less than 4.5% in all four measured layers. Here the results validated the accuracy of the modeling and simulation. Finally, the relative change of the neutron flux profiles from moving control rods was analyzed, which identified the layer that has the best sensitivity regarding the control rods movements. Thus, this work identified and provided training data sets of both simulated and experimental neutron flux profiles in the most sensitive layer, paving the path forward to the real-time experimental demonstration of the autonomous control system.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗