Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “simulation-based testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Simulation-based inference for neutrino interaction model parameter tuning

High-energy physics experiments studying neutrinos rely heavily on simulations of their interactions with atomic nuclei. Limitations in the theoretical understanding of these interactions typically necessitate ad hoc tuning of simulation model parameters to data. Traditional tuning methods for neutrino experiments have largely relied on simple algorithms for numerical optimization. While adequate for the modest goals of initial efforts, the complexity of future neutrino tuning campaigns is expected to increase substantially, and new approaches will be needed to make progress. In this paper, we examine the application of simulation-based inference (SBI) to the neutrino interaction model tuning for the first time. Using a previous tuning study performed by the MicroBooNE experiment as a test case, we find that our SBI algorithm can correctly infer the tuned parameter values when confronted with a mock data set generated according to the MicroBooNE procedure. This initial proof-of-principle illustrates a promising new technique for next-generation simulation tuning campaigns for the neutrino experimental community.

Tame-Narvaez, Karla [Fermilab] (ORCID:000000022249↗
DOE OSTI2026

An analytic approximation to the covariance between pre- and post-reconstruction galaxy two-point statistics

We present a simple analytic approximation for the covariance between pre-reconstruction galaxy power spectrum measurements and post-reconstruction two-point correlation functions. This cross-covariance is essential for joint analyses that combine full-shape clustering information with baryon acoustic oscillation (BAO) measurements, as commonly performed in modern spectroscopic surveys. Our model builds on the disconnected contribution to the covariance and accounts for the damping of correlations due to the BAO reconstruction process. We validate our analytic prescription against numerical simulations from the Dark Energy Spectroscopic Instrument (DESI), testing both idealized cubic geometries and realistic survey configurations including complex footprints and fiber assignment effects. Despite neglecting survey window functions in the analytic calculation, we find excellent agreement with simulation-based covariances and demonstrate that cosmological parameter constraints are virtually unchanged when using our approximation. Our results show that the pre-post cross-covariance is sufficiently small that even approximate treatments are adequate for cosmological inference, opening a pathway toward fully analytic covariance matrices for next-generation galaxy surveys.

baryon acoustic oscillations↗
NASA NTRS2004

Simulation-Based Verification of Autonomous Controllers via Livingstone PathFinder

AI software is often used as a means for providing greater autonomy to automated systems, capable of coping with harsh and unpredictable environments. Due in part to the enormous space of possible situations that they aim to addrs, autonomous systems pose a serious challenge to traditional test-based verification approaches. Efficient verification approaches need to be perfected before these systems can reliably control critical applications. This publication describes Livingstone PathFinder (LPF), a verification tool for autonomous control software. LPF applies state space exploration algorithms to an instrumented testbed, consisting of the controller embedded in a simulated operating environment. Although LPF has focused on NASA s Livingstone model-based diagnosis system applications, the architecture is modular and adaptable to other systems. This article presents different facets of LPF and experimental results from applying the software to a Livingstone model of the main propulsion feed subsystem for a prototype space vehicle.

Lindsey, A. E.↗
NASA NTRS1992

Estimation and enhancement of real-time software reliability through mutation analysis

A simulation-based technique for obtaining numerical estimates of the reliability of N-version, real-time software is presented. An extended stochastic Petri net is employed to represent the synchronization structure of N versions of the software, where dependencies among versions are modeled through correlated sampling of module execution times. Test results utilizing specifications for NASA's planetary lander control software indicate that mutation-based testing could hold greater potential for enhancing reliability than the desirable but perhaps unachievable goal of independence among N versions.

Geist, Robert↗
DOE OSTI2025

FY25 Mid-Year Report: FNCL Enhancements Implementation

During the first half of FY25 the FNCL team has made consistent progress toward the completion of our project goals. The FNCL prototype panel design has been successfully applied to a fully instrumented 3-panel system which is actively under construction. The FNCL Demonstrator System contains solid scintillators instrumented with SiPMs, which operate on an updated CAEN digitizer, requires no high-voltage, and has a smaller overall footprint. The onboard software will include the LLNL-developed GMM-PSD signal processing. Later this year the system will be experimentally tested alongside the baseline FNCL instrument at LLNLs ISSA facility. In addition to a full systems test, the performance of a DD generator for active interrogation measurements compared to the standard AmLi source will be established for both systems. The data collected at the ISSA facility will be used to experimentally validate the FNCL-Fast Isotopic Fuel Assay’s (FIFA) capability to measure U-235 loading and to predict gadolinium poison content with passive interrogation. The FNCL-FIFA modal was benchmarked with simulation-based data and a user-friendly GUI was added earlier this year. Three separate codes have been submitted to the LLNL ESW system for review prior to their transfers. These include the Predictive Modeling Response toolkit, GMM-PSD firmware beta version, and the FNCL-FIFA analysis package with GUI and user documentation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Dark energy survey year 3 results: likelihood-free, simulation-based w CDM inference with neural compression of weak-lensing map statistics

We present simulation-based cosmological wcold dark matter (wCDM) inference using dark energy survey year 3 weak-lensing maps, via neural data compression of weak-lensing map summary statistics: power spectra, peak counts, and direct map-level compression/inference with convolutional neural networks (CNN). Using simulation-based inference, also known as likelihood-free or implicit inference, we use forward-modelled mock data to estimate posterior probability distributions of unknown parameters. This approach allows all statistical assumptions and uncertainties to be propagated through the forward-modelled mock data; these include sky masks, non-Gaussian shape noise, shape measurement bias, source galaxy clustering, photometric redshift uncertainty, intrinsic galaxy alignments, non-Gaussian density fields, neutrinos, and non-linear summary statistics. We include a series of tests to validate our inference results. This paper also describes the Gower Street simulation suite: 791 full-sky pkdgrav3 dark matter simulations, with cosmological model parameters sampled with a mixed active-learning strategy, from which we construct over 3000 mock dark energy survey lensing data sets. For wCDM inference, for which we allow –1 < w < –$\frac{1}{3}$⁠, our most constraining result uses power spectra combined with map-level (CNN) inference. Using gravitational lensing data only, this map-level combination gives Ω m = 0.283$^{+0.020}_{–0.027}$⁠, S 8 = 0.804$^{+0.025}_{–0.017⁠}$, and w < –0.80 (with a 68 per cent credible interval); compared to the power spectrum inference, this is more than a factor of two improvement in dark energy parameter (Ω⁠ DE , w⁠) precision.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

A DECADE of dwarfs: first detection of weak lensing around spectroscopically confirmed low-mass galaxies

We present the first detection of weak gravitational lensing around spectroscopically confirmed dwarf galaxies, using the large overlap between DESI DR1 spectroscopic data and DECADE/DES weak lensing catalogs. A clean dwarf galaxy sample with well-defined redshift and stellar mass cuts enables excess surface mass density measurements in two stellar mass bins ($\log \rm{M}_*=[8.2, 9.2]~M_\odot$ and $\log \rm{M}_*=[9.2, 10.2]~M_\odot$), with signal-to-noise ratios of $5.6$ and $12.4$ respectively. This signal-to-noise drops to $4.5$ and $9.2$ respectively for measurements without applying individual inverse probability (IIP) weights, which mitigates fiber incompleteness from DESI's targeting. The measurements are robust against variations in stellar mass estimates, photometric shredding, and lensing calibration systematics. Using a simulation-based modeling framework with stellar mass function priors, we constrain the stellar mass-halo mass relation and find a satellite fraction of $\simeq 0.3$, which is higher than previous photometric studies but $1.5σ$ lower than $Λ$CDM predictions. We find that IIP weights have a significant impact on lensing measurements and can change the inferred $f_{\rm{sat}}$ by a factor of two, highlighting the need for accurate fiber incompleteness corrections for dwarf galaxy samples. Our results open a new observational window into the galaxy-halo connection at low masses, showing that future massively multiplexed spectroscopic observations and weak lensing data will enable stringent tests of galaxy formation models and $Λ$CDM predictions.

To, Chun-Hao [Chicago U., Astron. Astrophys. Ctr.;↗
DOE OSTI2025

A Parameter-masked Mock Data Challenge for Beyond-two-point Galaxy Clustering Statistics

The past few years have seen the emergence of a wide array of novel techniques for analyzing high-precision data from upcoming galaxy surveys, which aim to extend the statistical analysis of galaxy clustering data beyond the linear regime and the canonical two-point (2pt) statistics. We test and benchmark some of these new techniques in a community data challenge named “Beyond-2pt,” initiated during the Aspen 2022 Summer Program “Large-Scale Structure Cosmology beyond 2-Point Statistics,” whose first round of results we present here. The challenge data set consists of high-precision mock galaxy catalogs for clustering in real space, in redshift space, and on a light cone. Participants in the challenge have developed end-to-end pipelines to analyze mock catalogs and extract unknown (“masked”) cosmological parameters of the underlying ΛCDM models with their methods. The methods represented are density-split clustering, nearest neighbor statistics, BACCO power spectrum emulator, void statistics, LEFTfield field-level inference using effective field theory (EFT), and joint power spectrum and bispectrum analyses using both EFT and simulation-based inference. In this work, we review the results of the challenge, focusing on problems solved, lessons learned, and future research needed to perfect the emerging beyond-2pt approaches. The unbiased parameter recovery demonstrated in this challenge by multiple statistics and the associated modeling and inference frameworks supports the credibility of cosmology constraints from these methods. The challenge data set is publicly available, and we welcome future submissions from methods that are not yet represented.

Krause, Elisabeth [Univ. of Arizona, Tucson, AZ (U↗
DOE OSTI2025

Myna: Connecting powder bed fusion build data to simulation tools for digital twin applications

Additive manufacturing (AM), as a digital process, can generate a detailed digital thread linking a part’s design and manufacturing to its operational performance. As AM systems advance, an increasing amount of process data is stored in manufacturing databases. In principle, this data can be utilized by simulation-based digital twin approaches, such as real-time process control and asynchronous post-processing guidance. However, few tools currently exist for systematically integrating digital thread data with computational tools. Here, in this study, we propose a software package, called Myna, for connecting data from powder bed fusion processes to simulation tools. The utility of such a platform is demonstrated using build data from the Oak Ridge National Laboratory Manufacturing Demonstration Facility “Peregrine v2023-10” public dataset to automatically configure and run 54 semi-analytical 3DThesis melt pool simulations, 78 numerical Additive FOAM melt pool simulations, and 3 ExaCA microstructure simulations. The simulated, spatially registered microstructures are then compared directly with electron backscatter diffraction characterization of the corresponding as-built part locations. The resulting simulated microstructure showed variation as a function of process parameters, particularly stripe width; however, the experimental data had little variation between the microstructure texture and grain size resulting from different processing conditions. Analysis of the discrepancies suggest that it is possible a two-phase ferritic-austenitic solidification model is needed to accurately predict grain size and texture for certain stainless steel 316L feedstock compositions under powder bed fusion conditions, providing direction for future research. As illustrated here, due to the number and complexity of the simulations involved in AM process-structure–property predictions, automated methods to connect process data and simulations will remain necessary tools for testing hypotheses and implementing digital twin applications.

Knapp, Gerald L. [Oak Ridge National Laboratory (O↗
DOE OSTI2025

ACTIVE

The Automated Control Testbed for Integration, Verification, and Emulation (ACTIVE) framework is a software platform designed to support the optimized operation and management of a wide range of building types. It enables the development, testing, and validation of diverse control strategies, including AI-based, rule-based, and model-based approaches. The platform facilitates a seamless transition from simulation-based evaluation of control strategies to real-world field validation and deployment. ACTIVE supports the full building management lifecycle, encompassing data acquisition and management, system monitoring, optimized control, adaptive learning services, device dispatch and coordination, as well as advanced analytics and visualization. Together, these capabilities provide an integrated environment for improving building performance, operational efficiency, reducing energy cost, and reliability.

Smith, Robert [Oak Ridge National Laboratory (ORNL↗
DOE OSTI2024

Scattering Calorimeter FY24 Deliverable Report

A simulation-based method has been developed to prototype new detector designs for nuclear data measurements utilizing neutron scattering. This method uses representative physics inputs for signal and background generation, full detector resolution smearing benchmarked by experimental data, and a neutron beam timing simulation to produce analyzable output like a physical measurement. A test case has been studied using a hybrid time-of-flight calorimeter detector for scattering cross-section measurements on 239 Pu with 1-5 MeV incident monoenergetic neutrons. Data analysis methods have been developed to perform event-level particle reconstruction and reaction channel discrimination. This analysis has been used to estimate the capability of the test detector to perform simultaneous scattering and fission cross section measurements, as well as its ability to provide neutron spectra and particle angular information.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Feasibility of an Accelerometer-Based Structural Health Monitoring System for the LANL Blast Tube

A modeling- and simulation-based study was conducted on the feasibility of implementing an accelerometer-based SHM system on the Los Alamos National Laboratory blast tube. A blast tube experiment was modeled using the Abaqus explicit finite element solver. A custom user subroutine was written to apply test-like pressure loading to the inside surface of the blast tube. The subroutine applies analytically defined pressure loads derived from tracer output taken from a Compressible Flow Computational Fluid Dynamics Solver model of the blast tube. Five unique versions of the model were created: an undamaged reference model at 65°F was used as the baseline and compared to equivalent models at 10°F and 100°F. These three models were compared to models with small damage at the reference temperature. The two types of damage considered were a radial (circumferential) crack in the main tube body and a longitudinal crack in the supports. Acceleration outputs were extracted from accelerometer bodies included in the model and were post processed using a variety of standard SHM techniques. Different potential features signaling failure were extracted and compared using statistical methods in the time and frequency domains. A method was identified that clearly shows that differences in structural response resulting from the modeled damage can be differentiated from the structural response resulting from changing environmental conditions. However, the amount of damage applied to create observable differences in the accelerometer data was so large that simpler methods of damage detection would be more cost effective in locating damage.

42 ENGINEERING↗
DOE OSTI2025

Analyzing the Impact of Future Weather Data on Energy Consumption in Weatherization Assistant

This study supports the mission of the U.S. Department of Energy’s Weatherization Assistance Program (WAP), which aims to increase the energy efficiency of dwellings and reduce their total residential expenditures. Specifically, we examine how projected future climate conditions may affect residential building energy performance by integrating future weather data into the National Energy Audit Tool (NEAT). Since WAP evaluates the cost-effectiveness of retrofit measures over lifespans of up to 30 years, accounting for evolving climate conditions is increasingly important. To reflect future household energy demands, this study replaces historically based Typical Meteorological Year (TMY3) weather inputs with Future Typical Meteorological Year (fTMY) datasets derived from global climate model (GCM) projections. A simulation-based framework was established to enable NEAT analysis under future weather conditions. This workflow involves converting EPW-format weather files into JSON inputs compatible with NEAT and generating degree-hour metrics needed for load calculations. The fTMY dataset used in this study was developed by Oak Ridge National Laboratory through downscaling of six GCMs under different emission scenarios and covers the period from 2020 to 2100. In contrast, the TMY3 dataset is based on historical weather data from 1961 to 1990. Simulations were conducted for benchmark single-family prototype buildings across ASHRAE climate zones 1–7, which cover all regions of the U.S. except the subarctic Zone 8 in northern Alaska, evaluating both heating and cooling loads under TMY3 and fTMY conditions. Four foundation types were tested, while heating systems were standardized, as NEAT does not differentiate thermal energy load by HVAC system type in its load calculations. Results show that fTMY weather input consistently yield lower heating loads and higher cooling loads across most locations, aligning with expected climate warming trends. Notably, colder regions such as zones 6A, 6B, and 7 experience marked reductions in heating load, while warmer and transitional zones, such as 2A (Lufkin, TX) and 3C (San Francisco, CA), have substantial increases in cooling loads. Although this study does not directly assess the performance of retrofit measures under future climate conditions, it provides a critical foundation for doing so. By quantifying shifts in baseline (i.e., pre-retrofit case) energy loads between historical and future weather files, the study highlights the importance of integrating climate-responsive data into audit tools. These findings will inform future efforts to evaluate the long-term effectiveness and cost-effectiveness of weatherization measures under changing climate conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Practical Insights on Applying Simulation-Based Control Methods in Experimental Studies

Advanced nuclear reactors are crucial to the future of energy both in the United States and around the globe. In contrast to the current operating fleet, they are characterized as being deployable in remote locations and able to operate in semi-autonomous or autonomous fashion. This leap forward necessitates a new reactor control paradigm. Because advanced nuclear reactors are still under development in the United States, the creation of new control methods to achieve autonomous operations has been based on systems modeling and simulation. However, an important factor in successfully deploying these new control methods is the ability to seamlessly transition from simulation environments to real-world settings. Control methods tested in both simulation and experimental settings need to be investigated in the context of advanced reactor applications. This work developed a series of simple controllers for Idaho National Laboratory (INL)’s Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor operating in load-following scenarios. These controllers were tested in both simulation and experimental settings, and a comparative performance analysis was performed. The simulation tests leveraged the Control and Optimization Modular Modeling Application for Nuclear Deployment (COMMAND) software developed in a previous stage of the current effort, along with the MARVEL Reactor Excursion and Leak Analysis Program (RELAP5-3D) and Monte Carlo N-Particle (MCNP) models. The experimental tests leveraged the COMMAND software, MARVEL models, and the U.S. Department of Energy Microreactor Program’s Microreactor Automated Control System (MACS). MACS was developed to serve as a control method testbed. It was customized to mirror the MARVEL microreactor, and COMMAND enabled MACS to emulate the physics of MARVEL. The load-following controller was developed using the simulation platform, with efforts to emulate real systems by introducing actuator saturation and noise. These factors were incrementally accounted for in the controller design. After finalizing the controller design, it was implemented with the experimental setup. The experimental conditions tested included an initial test under conditions similar to the final simulation test, and two additional scenarios. The first scenario introduced additional actuator saturation to account for equipment aging over time, which was unknown to the controller. The second scenario introduced sensor delay, a phenomenon anticipated with the use of remote operations or wireless communication in advanced reactors. These tests revealed several notable differences. While the controller performed well in simulation, it exhibited several limitations when transitioning to hardware. The main challenges involved maintaining the steady-state target power, as evidenced by larger error values between the true reactor power and setpoint power, as well as persistent oscillations in controlled reactor power. These issues could lead to unacceptable transient conditions in real reactor testing. Introducing actuator aging and stochastic delays in the experimental setup significantly impacted controller performance, resulting in increased overshoot and undershoot, and exacerbated error and oscillations previously mentioned. These findings underscore the importance of experimental testbeds for testing and validating control methods, as controllers developed using only theory and/or simulation may perform unexpectedly when applied to actual hardware. This research emphasizes the need for an experimental testbed for achieving such validation.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
DOE OSTI2024

Simulation-based inference for parameter estimation of complex watershed simulators

High-resolution, spatially distributed process-based (PB) simulators are widely employed in the study of complex catchment processes and their responses to a changing climate. However, calibrating these PB simulators using observed data remains a significant challenge due to several persistent issues, including the following: (1) intractability stemming from the computational demands and complex responses of simulators, which renders infeasible calculation of the conditional probability of parameters and data, and (2) uncertainty stemming from the choice of simplified representations of complex natural hydrologic processes. Here, we demonstrate how simulation-based inference (SBI) can help address both of these challenges with respect to parameter estimation. SBI uses a learned mapping between the parameter space and observed data to estimate parameters for the generation of calibrated simulations. To demonstrate the potential of SBI in hydrologic modeling, we conduct a set of synthetic experiments to infer two common physical parameters – Manning's coefficient and hydraulic conductivity – using a representation of a snowmelt-dominated catchment in Colorado, USA. We introduce novel deep-learning (DL) components to the SBI approach, including an “emulator” as a surrogate for the PB simulator to rapidly explore parameter responses. We also employ a density-based neural network to represent the joint probability of parameters and data without strong assumptions about its functional form. While addressing intractability, we also show that, if the simulator does not represent the system under study well enough, SBI can yield unreliable parameter estimates. Approaches to adopting the SBI framework for cases in which multiple simulator(s) may be adequate are introduced using a performance-weighting approach. The synthetic experiments presented here test the performance of SBI, using the relationship between the surrogate and PB simulators as a proxy for the real case.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1993

The effects of a dynamic graphical model during simulation-based training of console operation skill

LOADER is a Windows-based simulation of a complex procedural task. The task requires subjects to execute long sequences of console-operation actions (e.g., button presses, switch actuations, dial rotations) to accomplish specific goals. The LOADER interface is a graphical computer-simulated console which controls railroad cars, tracks, and cranes in a fictitious railroad yard. We hypothesized that acquisition of LOADER performance skill would be supported by the representation of a dynamic graphical model linking console actions to goal and goal states in the 'railroad yard'. Twenty-nine subjects were randomly assigned to one of two treatments (i.e., dynamic model or no model). During training, both groups received identical text-based instruction in an instructional-window above the LOADER interface. One group, however, additionally saw a dynamic version of the bird's-eye view of the railroad yard. After training, both groups were tested under identical conditions. They were asked to perform the complete procedure without guidance and without access to either type of railroad yard representation. Results indicate that rather than becoming dependent on the animated rail yard model, subjects in the dynamic model condition apparently internalized the model, as evidenced by their performance after the model was removed.

Farquhar, John D.↗
NASA NTRS2003

Rapid Technology Assessment via Unified Deployment of Global Optical and Virtual Diagnostics

This paper discusses recent developments in rapid technology assessment resulting from an active collaboration between researchers at the Air Force Research Laboratory (AFRL) at Wright Patterson Air Force Base (WPAFB) and the NASA Langley Research Center (LaRC). This program targets the unified development and deployment of global measurement technologies coupled with a virtual diagnostic interface to enable the comparative evaluation of experimental and computational results. Continuing efforts focus on the development of seamless data translation methods to enable integration of data sets of disparate file format in a common platform. Results from a successful low-speed wind tunnel test at WPAFB in which global surface pressure distributions were acquired simultaneously with model deformation and geometry measurements are discussed and comparatively evaluated with numerical simulations. Intensity- and lifetime-based pressure-sensitive paint (PSP) and projection moire interferometry (PMI) results are presented within the context of rapid technology assessment to enable simulation-based R&D.

Jordan, Jeffrey D.↗
NASA NTRS2001

Adjustable Autonomy Testbed

The Adjustable Autonomy Testbed (AAT) is a simulation-based testbed located in the Intelligent Systems Laboratory in the Automation, Robotics and Simulation Division at NASA Johnson Space Center. The purpose of the testbed is to support evaluation and validation of prototypes of adjustable autonomous agent software for control and fault management for complex systems. The AA T project has developed prototype adjustable autonomous agent software and human interfaces for cooperative fault management. This software builds on current autonomous agent technology by altering the architecture, components and interfaces for effective teamwork between autonomous systems and human experts. Autonomous agents include a planner, flexible executive, low level control and deductive model-based fault isolation. Adjustable autonomy is intended to increase the flexibility and effectiveness of fault management with an autonomous system. The test domain for this work is control of advanced life support systems for habitats for planetary exploration. The CONFIG hybrid discrete event simulation environment provides flexible and dynamically reconfigurable models of the behavior of components and fluids in the life support systems. Both discrete event and continuous (discrete time) simulation are supported, and flows and pressures are computed globally. This provides fast dynamic simulations of interacting hardware systems in closed loops that can be reconfigured during operations scenarios, producing complex cascading effects of operations and failures. Current object-oriented model libraries support modeling of fluid systems, and models have been developed of physico-chemical and biological subsystems for processing advanced life support gases. In FY01, water recovery system models will be developed.

Malin, Jane T.↗