Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “prediction uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Near‐Surface Hydrology and Soil Properties Drive Heterogeneity in Permafrost Distribution, Vegetation Dynamics, and Carbon Cycling in a Sub‐Arctic Watershed

Abstract Discontinuous permafrost environments exhibit strong spatial heterogeneity at scales too small to be driven by weather forcing or captured by Earth System Models. Here we analyze effects of observed spatial heterogeneity in soil and vegetation properties, hydrology, and thermal dynamics on ecosystem carbon dynamics in a watershed on the Seward Peninsula in Alaska. We apply a Morris global sensitivity analysis to a process‐rich, successfully tested terrestrial ecosystem model (TEM), ecosys , varying soil properties, boundary conditions, and weather forcing. We show that landscape heterogeneity strongly impacts soil temperatures and vegetation composition. Snow depth, O‐horizon thickness, and near‐surface water content, which vary at scales of O(m), control the soil thermal regime more than an air temperature gradient corresponding to a 140 km north–south distance. High shrub productivity is simulated only in talik (perennially unfrozen) soils with high nitrogen availability. Through these effects on plant and permafrost dynamics, landscape heterogeneity impacts ecosystem productivity. Simulations with near‐surface taliks have higher microbial respiration (by 78.0 gC m −2 yr −1 ) and higher net primary productivity (by 104.9 gC m −2 yr −1 ) compared to runs with near‐surface permafrost, and simulations with high shrub productivity have outlying values of net carbon uptake. We explored the prediction uncertainty associated with ignoring observed landscape heterogeneity, and found that watershed net carbon uptake is 60% larger when heterogeneity is accounted for. Our results highlight the complexity inherent in discontinuous permafrost environments and demonstrate that missing representation of subgrid heterogeneity in TEMs could bias predictions of high‐latitude carbon budget.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Femtosecond pulse parameter estimation from photoelectron momenta using machine learning

Abstract Deep learning models have provided huge interpretation power for image-like data. Specifically, convolutional neural networks (CNNs) have demonstrated incredible acuity for tasks such as feature extraction or parameter estimation. Here we test CNNs on strong-field ionization photoelectron spectra, training on theoretical data sets to ‘invert’ experimental data. Pulse characterization is used as a ‘testing ground’, specifically we retrieve the laser intensity, where ‘traditional’ measurements typically lead to 20% uncertainty. We report on crucial data augmentation techniques required to successfully train on theoretical data and return consistent results from experiments, including accounting for detector saturation. The same procedure can be repeated to apply CNNs in a range of scenarios for strong-field ionization. Using a predictive uncertainty estimation, reliable laser intensity uncertainties of a few percent can be extracted, which are consistently lower than those given by traditional techniques. Using interpretability methods can reveal parts of the distribution that are most sensitive to laser intensity, which can be directly associated with holographic interferences. The CNNs employed provide an accurate and convenient ways to extract parameters, and represent a novel interpretational tool for strong-field ionization spectra.

Szołdra, Tomasz (ORCID:0000000228970506)↗
DOE OSTI2022

A Framework for Inverse Prediction Using Functional Response Data

Inverse prediction models have commonly been developed to handle scalar data from physical experiments. However, it is not uncommon for data to be collected in functional form. When data are collected in functional form, it must be aggregated to fit the form of traditional methods, which often results in a loss of information. For expensive experiments, this loss of information can be costly. In this study, we introduce the functional inverse prediction (FIP) framework, a general approach which uses the full information in functional response data to provide inverse predictions with probabilistic prediction uncertainties obtained with the bootstrap. The FIP framework is a general methodology that can be modified by practitioners to accommodate many different applications and types of data. We demonstrate the framework, highlighting points of flexibility, with a simulation example and applications to weather data and to nuclear forensics. Results show how functional models can improve the accuracy and precision of predictions.

42 ENGINEERING↗
DOE OSTI2023

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Towards this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. In conclusion, two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Mass measurements of 60–63 Ga reduce x-ray burst model uncertainties and extend the evaluated T=1 isobaric multiplet mass equation

We report precision mass measurements of neutron-deficient gallium isotopes approaching the proton drip line. The measurements of 60–63 Ga performed with the TITAN multiple-reflection time-of-flight mass spectrometer provide a more than threefold improvement over the current literature mass uncertainty of 61 Ga and mark the first direct mass measurement of 60 Ga. The improved precision of the 61 Ga mass has important implications for the astrophysical rp process, as it constrains essential reaction Q values near the 60 Zn waiting point. Based on calculations with a one-zone model, we demonstrate the impact of the improved mass data on prediction uncertainties of x-ray burst models. The first-time measurement of the 60 Ga ground-state mass establishes the proton-bound nature of this nuclide, thus constraining the location of the proton drip line along this isotopic chain. Including the measured mass of 60 Ga further enables us to extend the evaluated T = 1 isobaric multiplet mass equation up to A = 60.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Examples of Mission-driven Data Science from Jefferson Lab and ACES

This presentation details mission-driven data science initiatives at Jefferson Lab and the Joint Institute for Advanced Computing on Environmental Studies (ACES). JLab, a U.S. Department of Energy Office of Science national laboratory, operates the Continuous Electron Beam Accelerator Facility (CEBAF), and is the lead institute for the new High Performance Data Facility (HPDF) Hub. The Joint Institute for ACES brings together interdisciplinary teams in health informatics, climate modeling, computer science, and physics to address environmental challenges, including flood modeling. The Hampton Roads region, particularly Norfolk and Virginia Beach, faces increasing flood risks, motivating the need for rapid, reliable, and risk-aware decision support. ACES’s flooding work has a focus on uncertainty quantification (UQ) and machine learning (ML) for coastal flood management. The work is motivated by the increasing vulnerability of communities such as Norfolk and Virginia Beach, Virginia, to frequent coastal flooding events, and the need for rapid, reliable decision support. The research develops computationally efficient ML surrogate models to forecast water levels and flooding risk. A central theme is the quantification and calibration of predictive uncertainty, especially for out-of-distribution (OOD) scenarios, using techniques such as Monte Carlo Dropout, Deep Ensembles, Gaussian Processes, and Deep Quantile Regression (DQR). The study demonstrates that distance-aware UQ is critical for reliable scientific AI, particularly in high-dimensional, safety-critical, and real-time applications.

McSpadden, Diana [Thomas Jefferson National Accele↗
DOE OSTI2024

Uncertainty-aware particle segmentation for electron microscopy at varied length scales

Electron microscopy is indispensable for examining the morphology and composition of solid materials at the sub-micron scale. To study the powder samples that are widely used in materials development, scanning electron microscopes (SEMs) are increasingly used at the laboratory scale to generate large datasets with hundreds of images. Parsing these images to identify distinct particles and determine their morphology requires careful analysis, and automating this process remains challenging. In this work, we enhance the Mask R-CNN architecture to develop a method for automated segmentation of particles in SEM images. We address several challenges inherent to measurements, such as image blur and particle agglomeration. Moreover, our method accounts for prediction uncertainty when such issues prevent accurate segmentation of a particle. Recognizing that disparate length scales are often present in large datasets, we use this framework to create two models that are separately trained to handle images obtained at low or high magnification. By testing these models on a variety of inorganic samples, our approach to particle segmentation surpasses an established automated segmentation method and yields comparable results to the predictions of three domain experts, revealing comparable accuracy while requiring a fraction of the time. These findings highlight the potential of deep learning in advancing autonomous workflows for materials characterization.

36 MATERIALS SCIENCE↗
NASA NTRS2018

Concept of Operations for Management by Trajectory

This document describes Management by Trajectory (MBT), a concept for future air traffic management (ATM) in which every flight operates in accordance with a four-dimensional trajectory (4DT) that is negotiated between the airspace user and the Federal Aviation Administration (FAA) to respect the airspace user's goals while complying with National Airspace System (NAS) constraints. In the present-day NAS, the ATM system attempts to predict the trajectory for each flight based on the approved flight plan and scheduled or controlled departure time. However, once the aircraft starts to move, controllers tactically manage the aircraft to implement traffic management restrictions, separate otherwise conflicting aircraft, and address arising NAS constraints. Tactical controller actions are not directly communicated to the automation systems or other stakeholders. Furthermore, the initial trajectory prediction does not anticipate these disruptions or how they will impact the flight. Consequently, and compounded by gaps in required data and models, trajectory predictions are less accurate than possible, which affects Traffic Flow Management (TFM) performance. A cornerstone of the MBT concept is that all air vehicles have, at all times, an assigned 4DT from their current state to their destination. These assigned trajectories consist of trajectory constraints and descriptions. Pilots and air traffic controllers, with the aid of automation, operate the aircraft to comply with the assigned trajectory, unless first negotiating a revision. Equipped aircraft have substantial responsibility for complying with the assigned trajectory without controller intervention. To maximize the operational flexibility available to the airspace user, the assigned trajectory only imposes trajectory constraints as required to achieve the ATM goals of NAS constraint compliance and aircraft separation. Trajectory descriptions are added to the assigned trajectory to ensure sufficient predictability. To further improve trajectory prediction accuracy, airspace users supplement the assigned trajectory by broadcasting intent information and updating it as necessary. Air vehicle intent is a more detailed description of the airspace user's plan for how the flight will fly the assigned trajectory. Air vehicle intent can change freely, without negotiation, as long as it remains in compliance with the assigned trajectory. Aircraft assigned trajectories, air vehicle intent, and predicted trajectories are shared, creating a common view among stakeholders. A NAS Constraint Service gathers and publishes information about all known NAS constraints, enabling airspace users to be informed participants in trajectory negotiation. Trajectory constraints in the assigned trajectory are mapped to NAS constraints to facilitate identifying which aircraft are affected when NAS constraints change. To support efficient trajectory negotiation, all aircraft provide current information about air vehicle capabilities. Assigned trajectories are constructed to satisfy all known NAS constraints, improving trajectory stability and predictability. Uncertainty and disruptions are handled by modifying the assigned trajectory as far in advance as possible. By proactively negotiating changes to the assigned trajectory, rather than relying on controller-selected tactical actions such as vectors to resolve traffic conflicts or implement miles-in-trail restrictions, MBT keeps aircraft on closed trajectories that are fully known to all stakeholders. Since reactive air traffic control actions cannot be predicted in advance, the downstream trajectory cannot be accurately predicted until they happen. Reliable trajectory predictions allow the system to identify needed modifications to trajectories further in advance, where they can be negotiated and communicated as amendments (i.e., additional or altered trajectory constraints) to the assigned trajectory. Decision Support Tools (DSTs) aid controllers in rapidly defining and communicating closed trajectories to the aircraft and support all stakeholders in trajectory negotiation. Anticipated MBT benefit mechanisms include more accurate trajectory predictions, improved ATM performance and robustness to off-nominal conditions, increased flexibility and operational efficiency, reduced impediments to emerging classes of airspace users accessing NAS resources, reduced environmental impacts, and enhanced safety.

ConOps↗
DOE OSTI2023

BayFlux: A Bayesian method to quantify metabolic Fluxes and their uncertainty at the genome scale

Metabolic fluxes, the number of metabolites traversing each biochemical reaction in a cell per unit time, are crucial for assessing and understanding cell function. 13 C Metabolic Flux Analysis ( 13 C MFA) is considered to be the gold standard for measuring metabolic fluxes. 13 C MFA typically works by leveraging extracellular exchange fluxes as well as data from 13 C labeling experiments to calculate the flux profile which best fit the data for a small, central carbon, metabolic model. However, the nonlinear nature of the 13 C MFA fitting procedure means that several flux profiles fit the experimental data within the experimental error, and traditional optimization methods offer only a partial or skewed picture, especially in “non-gaussian” situations where multiple very distinct flux regions fit the data equally well. Here, we present a method for flux space sampling through Bayesian inference (BayFlux), that identifies the full distribution of fluxes compatible with experimental data for a comprehensive genome-scale model. This Bayesian approach allows us to accurately quantify uncertainty in calculated fluxes. We also find that, surprisingly, the genome-scale model of metabolism produces narrower flux distributions (reduced uncertainty) than the small core metabolic models traditionally used in 13 C MFA. The different results for some reactions when using genome-scale models vs core metabolic models advise caution in assuming strong inferences from 13 C MFA since the results may depend significantly on the completeness of the model used. Based on BayFlux, we developed and evaluated novel methods (P- 13 C MOMA and P- 13 C ROOM) to predict the biological results of a gene knockout, that improve on the traditional MOMA and ROOM methods by quantifying prediction uncertainty.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2007

Propagation of Computational Uncertainty Using the Modern Design of Experiments

This paper describes the use of formally designed experiments to aid in the error analysis of a computational experiment. A method is described by which the underlying code is approximated with relatively low-order polynomial graduating functions represented by truncated Taylor series approximations to the true underlying response function. A resource-minimal approach is outlined by which such graduating functions can be estimated from a minimum number of case runs of the underlying computational code. Certain practical considerations are discussed, including ways and means of coping with high-order response functions. The distributional properties of prediction residuals are presented and discussed. A practical method is presented for quantifying that component of the prediction uncertainty of a computational code that can be attributed to imperfect knowledge of independent variable levels. This method is illustrated with a recent assessment of uncertainty in computational estimates of Space Shuttle thermal and structural reentry loads attributable to ice and foam debris impact on ascent.

DeLoach, Richard↗
DOE OSTI2024

Reduced‐Order Probabilistic Emulation of Physics‐Based Ring Current Models: Application to RAM‐SCB Particle Flux

Abstract In this work, we address the computational challenge of large‐scale physics‐based simulation models for the ring current. Reduced computational cost allows for significantly faster than real‐time forecasting, enhancing our ability to predict and respond to dynamic changes in the ring current, valuable for space weather monitoring and mitigation efforts. Additionally, it can also be used for a comprehensive investigation of the system. Thus, we aim to create an emulator for the Ring current‐Atmosphere interactions Model with Self‐Consistent magnetic field (RAM‐SCB) particle flux that not only improves efficiency but also facilitates forecasting with reliable estimates of prediction uncertainties. The probabilistic emulator is built upon the methodology developed by Licata and Mehta (2023), https://doi.org/10.1029/2022sw003345 . A novel discrete sampling is used to identify 30 simulation periods over 20 years of solar and geomagnetic activity. Focusing on a subset of particle flux, we use Principal Component Analysis for dimensionality reduction and Long Short‐Term Memory (LSTM) neural networks to perform dynamic modeling. Hyperparameter space was explored extensively resulting in about 5% median symmetric accuracy across all data sets for one‐step dynamic prediction. Using a hierarchical ensemble of LSTMs, we have developed a reduced‐order probabilistic emulator (ROPE) tailored for time‐series forecasting of particle flux in the ring current. This ROPE offers accurate predictions of omnidirectional flux at a single energy with no pitch angle information, providing robust predictions on the test set with an error score below 11% and calibration scores under 8% with bias under 2% providing a significant speed up as compared to the full RAM‐SCB run.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Using PyBioNetFit to leverage qualitative and quantitative data in biological model parameterization and uncertainty quantification

Data generated in studies of cellular regulatory systems are often qualitative. For example, measurements of signaling readouts in the presence and absence of mutations may reveal a rank ordering of responses across conditions but not the precise extents of mutation-induced differences. Qualitative data are often ignored by mathematical modelers or are considered in an ad hoc manner, as in the study of Kocieniewski and Lipniacki (2013) [Phys Biol 10: 035006], which was focused on the roles of MEK isoforms in ERK activation. In this earlier study, model parameter values were tuned manually to obtain consistency with a combination of qualitative and quantitative data. This approach is not reproducible, nor does it provide insights into parametric or prediction uncertainties. Here, starting from the same data and the same ordinary differential equation (ODE) model structure, we generate formalized statements of qualitative observations, making these observations more reusable, and we improve the model parameterization procedure by applying a systematic and automated approach enabled by the software package PyBioNetFit. We also demonstrate uncertainty quantification (UQ), which was absent in the original study. Our results show that PyBioNetFit enables qualitative data to be leveraged, together with quantitative data, in parameterization of systems biology models and facilitates UQ. These capabilities are important for reliable estimation of model parameters and model analyses in studies of cellular regulatory systems and reproducibility.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1973

Range of earth structure nonuniqueness implied by body wave observations.

The Herglotz-Wiechert integral for the direct inversion of ray parameter versus distance curves can be manipulated to find the envelope of all possible models consistent with geometrical body wave observations (travel time and ray parameter versus distance). Such an extremal inversion approach has been used to find the uncertainty bounds for the velocity structure in the mantle and core. It is found, for example, that there is an uncertainty of plus or minus 40 km in the radius of the inner core boundary, plus or minus 18 km at the core-mantle boundary, and plus or minus 35 km at the 435-km transition zone. The velocity uncertainty is about plus or minus 0.08 km/sec for P and S waves in the lower mantle and about plus or minus 0.20 km/sec in the core. Experiments with various combinations of ray types in the core indicate that rather crude observations of SKKS-SKS travel times confine the range of possible models far more dramatically than do the most precise estimates of PmKP travel times. Comparisons of results from extremal inversion and linearized perturbation inversions indicate that body wave behavior is too strongly nonlinear for linearized schemes to be effective for predicting uncertainty.

Wiggins, R. A.↗
DOE OSTI2023

Comparative Analysis of Standard and Advanced USL Methodologies [Slides]

This presentation is on the comparative analysis of standard and advanced USL methodologies. Each method has a different way to address/account for experimental biases and uncertainties. This study showed why bias and uncertainty predictions by these popular methods differ significantly in some cases. In order to establish confidence to our validation efforts, especially with new reactor concepts, in designing new critical experiments and claiming experimental coverage for our existing applications, we need to understand capabilities of our methods further.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Wall-Modeled Large-Eddy Simulations of Jet Noise in Flight Conditions

A campaign of wall-modeled large-eddy simulations (WMLES) using structured curvilinear overlapping grids has been performed with the Launch Ascent and Vehicle Aerodynamics (LAVA) computational fluid dynamics (CFD) software to predict jet noise for single-stream axisymmetric round jets. The simulations address the new Prediction Uncertainty Reduction (PUR) technical challenge within the context of NASA’s Commercial Supersonic Technology (CST) project. The focus of this effort is to generate a simulation database for single-stream axisymmetric round nozzles at several operating points both for static (no ambient co-flow), and in-flight (M ͚ =0.3 co-flow) conditions. The operating conditions range in jet exit Mach number from 0.38 to 1.1 with nozzle temperature ratios (NTR) from 0.84 to 2.7. The effect of the flight-stream on far-field noise sound spectra is assessed. Comparison of LES predictions to microphone array measurements demonstrate good agreement within the resolved frequency range. A dip in the predicted low frequency noise spectra for observers between 120° and 145° is observed. This dip is smaller for lower Mach numbers and seems to be correlated to the Mach wave radiation angle. The Mach 1.1 jet shows broadband-shock associated noise. While the onset of BBSN appears to be captured correctly in WMLES, some differences in its magnitude and the prominent frequency at which it occurs persist between the experiment and the simulations. The effect of the outer nozzle boundary layer state created by the co-flow is assessed and shows to be important for accurate comparisons with experiments. A change of 2.5dB between a slip-wall condition and a artificially thickened turbulent boundary layer was observed. In addition, simulations were performed with an alternative nozzle geometry that includes a internal plug and has twice the nozzle exit diameter. These two configurations resulted in very comparable spectra which is consistent with experimental observations. A generally stronger deviation from experimental results is observed for in-flight cases compared to static conditions. The applicability and correct usage of acoustic analogies used for far-field propagation with strong turbulent co-flows needs to be investigated more systematically using canonical problems to improve comparisons between experiments and WMLES. This is especially true for coherent noise sources seen in BBSN.

CST↗
DOE OSTI2023

PANDEMIC: Occupancy driven predictive ventilation control to minimize energy consumption and infection risk

During the SARS-CoV-2 (COVID-19) pandemic, governments around the world have formulated policies requiring ventilation systems to operate at a higher outdoor fresh air flow rate for a sufficient time, which has led to a sharp increase in building energy consumption. Therefore, it is necessary to identify an energy-efficient ventilation strategy to reduce the risk of infection. In this study, we developed an occupant-number-based model predictive control (OBMPC) algorithm for building ventilation systems. First, we collected the occupancy and Heating, ventilation, and air conditioning system (HVAC) data from March to July 2021. Then, four different models (Auto regression moving average-based multilayer perceptron (ARMA_MLP), Recurrent neural networks (RNN), Long short-term memory networks (LSTM), and Nonhomogeneous Markov with change points detection (NH_Markov)) were used to predict the number of room occupants from 15 min to 24 h ahead with an interval output. We found that each model could predict the number of occupants with 85% accuracy using a one-person offset. Furthermore, the accuracy of 15 min of the ahead prediction could reach 95% with a one-person offset, but none of them could track abrupt changes. The occupancy prediction results were used to calculate the ventilation demand using the Wells-Riley equation, and the upper bound can maintain an infection risk lower than 2% for 93% of the day. This OBMPC model could reduce the coil load by 52.44% and shift the peak load by 3 h up to 5 kW compared with 24 × 7 h full outdoor air (OA) system when people wear masks in the space. The occupancy prediction uncertainty could cause a 9% to 26% difference in demand ventilation, a 0.3°C to 2.4°C difference in zone temperature, a 28.5% to 44.5% difference in outdoor airflow rate, and a 10.7% to 28.2% difference in coil load.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1984

Relationship between Allan variances and Kalman Filter parameters

A relationship was constructed between the Allan variance parameters (H sub z, H sub 1, H sub 0, H sub -1 and H sub -2) and a Kalman Filter model that would be used to estimate and predict clock phase, frequency and frequency drift. To start with the meaning of those Allan Variance parameters and how they are arrived at for a given frequency source is reviewed. Although a subset of these parameters is arrived at by measuring phase as a function of time rather than as a spectral density, they all represent phase noise spectral density coefficients, though not necessarily that of a rational spectral density. The phase noise spectral density is then transformed into a time domain covariance model which can then be used to derive the Kalman Filter model parameters. Simulation results of that covariance model are presented and compared to clock uncertainties predicted by Allan variance parameters. A two state Kalman Filter model is then derived and the significance of each state is explained.

Vandierendonck, A. J.↗